2024-04-24T23:55:44.321 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T23:55:44.321 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/sessionmap} --verbose --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231 --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231/orig.config.yaml 2024-04-24T23:55:44.367 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:55:44.456 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/sessionmap} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672231' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - client session with non-allowable root sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: syntax: v2 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ6JFaHLmbX/T4deYFAUKteDyTGelz4imH2PGSNYJRqmNTyBQqbF3IW2OFEKQJs3Kjrt8+7eiCdNuXAcTYQGVWE= smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEZht8wji+G4TbU4ZJ+WMQCY6etPdDOYxTOBqVjUPorZrNlOpUn/QnacYhcC2qB+rR8wH8st4rbUbnQhSTJL9kY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_sessionmap teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T23:55:44.456 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T23:55:44.457 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T23:55:44.457 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T23:55:44.458 INFO:teuthology.task.internal:Checking packages... 2024-04-24T23:55:44.480 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T23:55:44.480 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T23:55:44.481 INFO:teuthology.packaging:ref: None 2024-04-24T23:55:44.481 INFO:teuthology.packaging:tag: None 2024-04-24T23:55:44.481 INFO:teuthology.packaging:branch: squid 2024-04-24T23:55:44.481 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:55:44.481 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-04-24T23:55:44.672 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445-gd2c45d97-1jammy 2024-04-24T23:55:44.674 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T23:55:44.681 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T23:55:44.681 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T23:55:44.687 INFO:teuthology.task.internal:Saving configuration 2024-04-24T23:55:44.703 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T23:55:44.710 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:55:44.735 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:49:55.943433', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEb1/Ec1JrFBmvWX6F51NiwnzWjUXc/lF/MKCb7BByt9'} 2024-04-24T23:55:44.757 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:49:55.945859', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAgK45b6kU0FBxNWK+KhA3tl0j2fZTn9FQDEDirAm8G+'} 2024-04-24T23:55:44.758 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T23:55:44.764 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-24T23:55:44.764 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-24T23:55:44.764 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T23:55:44.832 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f228c5abee0>, signals=[15]) 2024-04-24T23:55:44.833 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T23:55:44.839 INFO:teuthology.task.internal:Opening connections... 2024-04-24T23:55:44.839 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-04-24T23:55:44.841 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:55:44.925 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-04-24T23:55:44.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:55:45.006 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T23:55:45.012 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-04-24T23:55:45.017 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-04-24T23:55:45.018 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-04-24T23:55:45.064 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-24T23:55:45.064 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="Ubuntu" 2024-04-24T23:55:45.064 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="22.04" 2024-04-24T23:55:45.064 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-24T23:55:45.064 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_CODENAME=jammy 2024-04-24T23:55:45.064 INFO:teuthology.orchestra.run.smithi083.stdout:ID=ubuntu 2024-04-24T23:55:45.064 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE=debian 2024-04-24T23:55:45.064 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-24T23:55:45.065 INFO:teuthology.orchestra.run.smithi083.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-24T23:55:45.065 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-24T23:55:45.065 INFO:teuthology.orchestra.run.smithi083.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-24T23:55:45.065 INFO:teuthology.orchestra.run.smithi083.stdout:UBUNTU_CODENAME=jammy 2024-04-24T23:55:45.065 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-04-24T23:55:45.087 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-04-24T23:55:45.092 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-04-24T23:55:45.092 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-04-24T23:55:45.138 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-24T23:55:45.139 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Ubuntu" 2024-04-24T23:55:45.139 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="22.04" 2024-04-24T23:55:45.139 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-24T23:55:45.139 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_CODENAME=jammy 2024-04-24T23:55:45.139 INFO:teuthology.orchestra.run.smithi086.stdout:ID=ubuntu 2024-04-24T23:55:45.139 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE=debian 2024-04-24T23:55:45.139 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-24T23:55:45.139 INFO:teuthology.orchestra.run.smithi086.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-24T23:55:45.139 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-24T23:55:45.139 INFO:teuthology.orchestra.run.smithi086.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-24T23:55:45.139 INFO:teuthology.orchestra.run.smithi086.stdout:UBUNTU_CODENAME=jammy 2024-04-24T23:55:45.139 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-04-24T23:55:45.162 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T23:55:45.172 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T23:55:45.222 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T23:55:45.222 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:55:45.224 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:55:45.229 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T23:55:45.235 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T23:55:45.235 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:55:45.270 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:55:45.276 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T23:55:45.334 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T23:55:45.349 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-24T23:55:45.349 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-24T23:55:45.350 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-24T23:55:45.350 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:55:45.350 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:55:45.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:55:45.355 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-04-24T23:55:45.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:55:45.358 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-04-24T23:55:45.526 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T23:55:45.526 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T23:55:45.527 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-24T23:55:45.532 INFO:teuthology.orchestra.run.smithi086.stdout:5.15.0-56-generic 2024-04-24T23:55:45.533 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T23:55:45.533 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T23:55:45.533 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-04-24T23:55:45.533 DEBUG:teuthology.orchestra.run.smithi086:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-04-24T23:55:45.534 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T23:55:45.535 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T23:55:45.535 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-04-24T23:55:45.540 INFO:teuthology.orchestra.run.smithi083.stdout:5.15.0-56-generic 2024-04-24T23:55:45.540 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T23:55:45.540 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T23:55:45.541 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-04-24T23:55:45.541 DEBUG:teuthology.orchestra.run.smithi083:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-04-24T23:55:45.598 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-04-24T23:55:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-04-24T23:55:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-24T23:55:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-04-24T23:55:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-04-24T23:55:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:hint: 2024-04-24T23:55:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:hint: git config --global init.defaultBranch 2024-04-24T23:55:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:hint: 2024-04-24T23:55:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-24T23:55:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-04-24T23:55:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:hint: 2024-04-24T23:55:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:hint: git branch -m 2024-04-24T23:55:45.722 INFO:teuthology.orchestra.run.smithi083.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-04-24T23:55:45.724 DEBUG:teuthology.orchestra.run.smithi083:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-04-24T23:55:45.725 INFO:teuthology.orchestra.run.smithi086.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-24T23:55:45.725 INFO:teuthology.orchestra.run.smithi086.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-04-24T23:55:45.725 INFO:teuthology.orchestra.run.smithi086.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-04-24T23:55:45.725 INFO:teuthology.orchestra.run.smithi086.stderr:hint: 2024-04-24T23:55:45.725 INFO:teuthology.orchestra.run.smithi086.stderr:hint: git config --global init.defaultBranch 2024-04-24T23:55:45.725 INFO:teuthology.orchestra.run.smithi086.stderr:hint: 2024-04-24T23:55:45.725 INFO:teuthology.orchestra.run.smithi086.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-24T23:55:45.725 INFO:teuthology.orchestra.run.smithi086.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-04-24T23:55:45.725 INFO:teuthology.orchestra.run.smithi086.stderr:hint: 2024-04-24T23:55:45.726 INFO:teuthology.orchestra.run.smithi086.stderr:hint: git branch -m 2024-04-24T23:55:45.726 INFO:teuthology.orchestra.run.smithi086.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-04-24T23:55:45.727 DEBUG:teuthology.orchestra.run.smithi086:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-04-24T23:55:45.792 DEBUG:teuthology.orchestra.run.smithi083:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-04-24T23:55:45.794 DEBUG:teuthology.orchestra.run.smithi086:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-04-24T23:55:45.850 DEBUG:teuthology.orchestra.run.smithi083:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-04-24T23:55:45.851 DEBUG:teuthology.orchestra.run.smithi086:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-04-24T23:56:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-04-24T23:56:25.938 INFO:teuthology.orchestra.run.smithi086.stderr: * [new branch] main -> origin/main 2024-04-24T23:56:25.971 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190312 -> 20190312 2024-04-24T23:56:25.971 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190416 -> 20190416 2024-04-24T23:56:25.971 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190514 -> 20190514 2024-04-24T23:56:25.971 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190618 -> 20190618 2024-04-24T23:56:25.971 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190717 -> 20190717 2024-04-24T23:56:25.971 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190815 -> 20190815 2024-04-24T23:56:25.971 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190923 -> 20190923 2024-04-24T23:56:25.971 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20191022 -> 20191022 2024-04-24T23:56:25.971 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20191215 -> 20191215 2024-04-24T23:56:25.973 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200122 -> 20200122 2024-04-24T23:56:25.974 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200316 -> 20200316 2024-04-24T23:56:25.974 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200421 -> 20200421 2024-04-24T23:56:25.974 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200519 -> 20200519 2024-04-24T23:56:25.974 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200619 -> 20200619 2024-04-24T23:56:25.974 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200721 -> 20200721 2024-04-24T23:56:25.974 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200817 -> 20200817 2024-04-24T23:56:25.974 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200918 -> 20200918 2024-04-24T23:56:25.974 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20201022 -> 20201022 2024-04-24T23:56:25.974 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20201118 -> 20201118 2024-04-24T23:56:25.974 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20201218 -> 20201218 2024-04-24T23:56:25.974 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210208 -> 20210208 2024-04-24T23:56:25.974 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210315 -> 20210315 2024-04-24T23:56:25.974 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210511 -> 20210511 2024-04-24T23:56:25.974 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210716 -> 20210716 2024-04-24T23:56:25.975 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210818 -> 20210818 2024-04-24T23:56:25.975 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210919 -> 20210919 2024-04-24T23:56:25.975 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20211027 -> 20211027 2024-04-24T23:56:25.975 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20211216 -> 20211216 2024-04-24T23:56:25.977 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220209 -> 20220209 2024-04-24T23:56:25.977 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220310 -> 20220310 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220411 -> 20220411 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220509 -> 20220509 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220610 -> 20220610 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220708 -> 20220708 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220815 -> 20220815 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220913 -> 20220913 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20221012 -> 20221012 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20221109 -> 20221109 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20221214 -> 20221214 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230117 -> 20230117 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230210 -> 20230210 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230310 -> 20230310 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230404 -> 20230404 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230515 -> 20230515 2024-04-24T23:56:25.978 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230625 -> 20230625 2024-04-24T23:56:25.979 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230804 -> 20230804 2024-04-24T23:56:25.979 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230919 -> 20230919 2024-04-24T23:56:25.979 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20231030 -> 20231030 2024-04-24T23:56:25.979 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20231111 -> 20231111 2024-04-24T23:56:25.979 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20231211 -> 20231211 2024-04-24T23:56:25.979 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20240115 -> 20240115 2024-04-24T23:56:25.979 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20240220 -> 20240220 2024-04-24T23:56:25.979 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20240312 -> 20240312 2024-04-24T23:56:25.979 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20240410 -> 20240410 2024-04-24T23:56:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-04-24T23:56:28.087 INFO:teuthology.orchestra.run.smithi083.stderr: * [new branch] main -> origin/main 2024-04-24T23:56:28.119 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20190312 -> 20190312 2024-04-24T23:56:28.120 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20190416 -> 20190416 2024-04-24T23:56:28.120 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20190514 -> 20190514 2024-04-24T23:56:28.120 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20190618 -> 20190618 2024-04-24T23:56:28.121 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20190717 -> 20190717 2024-04-24T23:56:28.121 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20190815 -> 20190815 2024-04-24T23:56:28.121 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20190923 -> 20190923 2024-04-24T23:56:28.121 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20191022 -> 20191022 2024-04-24T23:56:28.122 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20191215 -> 20191215 2024-04-24T23:56:28.122 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20200122 -> 20200122 2024-04-24T23:56:28.122 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20200316 -> 20200316 2024-04-24T23:56:28.122 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20200421 -> 20200421 2024-04-24T23:56:28.122 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20200519 -> 20200519 2024-04-24T23:56:28.122 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20200619 -> 20200619 2024-04-24T23:56:28.123 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20200721 -> 20200721 2024-04-24T23:56:28.123 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20200817 -> 20200817 2024-04-24T23:56:28.123 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20200918 -> 20200918 2024-04-24T23:56:28.123 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20201022 -> 20201022 2024-04-24T23:56:28.123 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20201118 -> 20201118 2024-04-24T23:56:28.123 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20201218 -> 20201218 2024-04-24T23:56:28.123 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20210208 -> 20210208 2024-04-24T23:56:28.125 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20210315 -> 20210315 2024-04-24T23:56:28.125 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20210511 -> 20210511 2024-04-24T23:56:28.125 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20210716 -> 20210716 2024-04-24T23:56:28.125 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20210818 -> 20210818 2024-04-24T23:56:28.125 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20210919 -> 20210919 2024-04-24T23:56:28.125 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20211027 -> 20211027 2024-04-24T23:56:28.125 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20211216 -> 20211216 2024-04-24T23:56:28.125 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20220209 -> 20220209 2024-04-24T23:56:28.125 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20220310 -> 20220310 2024-04-24T23:56:28.127 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20220411 -> 20220411 2024-04-24T23:56:28.127 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20220509 -> 20220509 2024-04-24T23:56:28.127 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20220610 -> 20220610 2024-04-24T23:56:28.127 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20220708 -> 20220708 2024-04-24T23:56:28.127 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20220815 -> 20220815 2024-04-24T23:56:28.127 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20220913 -> 20220913 2024-04-24T23:56:28.127 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20221012 -> 20221012 2024-04-24T23:56:28.127 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20221109 -> 20221109 2024-04-24T23:56:28.127 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20221214 -> 20221214 2024-04-24T23:56:28.128 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20230117 -> 20230117 2024-04-24T23:56:28.128 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20230210 -> 20230210 2024-04-24T23:56:28.129 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20230310 -> 20230310 2024-04-24T23:56:28.129 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20230404 -> 20230404 2024-04-24T23:56:28.129 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20230515 -> 20230515 2024-04-24T23:56:28.129 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20230625 -> 20230625 2024-04-24T23:56:28.129 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20230804 -> 20230804 2024-04-24T23:56:28.129 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20230919 -> 20230919 2024-04-24T23:56:28.129 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20231030 -> 20231030 2024-04-24T23:56:28.129 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20231111 -> 20231111 2024-04-24T23:56:28.129 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20231211 -> 20231211 2024-04-24T23:56:28.130 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20240115 -> 20240115 2024-04-24T23:56:28.130 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20240220 -> 20240220 2024-04-24T23:56:28.130 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20240312 -> 20240312 2024-04-24T23:56:28.130 INFO:teuthology.orchestra.run.smithi083.stderr: * [new tag] 20240410 -> 20240410 2024-04-24T23:56:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:Updating files: 44% (1511/3381) Updating files: 45% (1522/3381) Updating files: 46% (1556/3381) Updating files: 47% (1590/3381) Updating files: 48% (1623/3381) Updating files: 49% (1657/3381) Updating files: 50% (1691/3381) Updating files: 51% (1725/3381) Updating files: 52% (1759/3381) Updating files: 53% (1792/3381) Updating files: 54% (1826/3381) Updating files: 55% (1860/3381) Updating files: 56% (1894/3381) Updating files: 57% (1928/3381) Updating files: 58% (1961/3381) Updating files: 59% (1995/3381) Updating files: 59% (2003/3381) Updating files: 60% (2029/3381) Updating files: 61% (2063/3381) Updating files: 62% (2097/3381) Updating files: 63% (2131/3381) Updating files: 64% (2164/3381) Updating files: 65% (2198/3381) Updating files: 66% (2232/3381) Updating files: 67% (2266/3381) Updating files: 68% (2300/3381) Updating files: 69% (2333/3381) Updating files: 69% (2362/3381) Updating files: 70% (2367/3381) Updating files: 71% (2401/3381) Updating files: 72% (2435/3381) Updating files: 73% (2469/3381) Updating files: 74% (2502/3381) Updating files: 75% (2536/3381) Updating files: 76% (2570/3381) Updating files: 77% (2604/3381) Updating files: 78% (2638/3381) Updating files: 79% (2671/3381) Updating files: 80% (2705/3381) Updating files: 81% (2739/3381) Updating files: 82% (2773/3381) Updating files: 83% (2807/3381) Updating files: 84% (2841/3381) Updating files: 85% (2874/3381) Updating files: 86% (2908/3381) Updating files: 87% (2942/3381) Updating files: 88% (2976/3381) Updating files: 89% (3010/3381) Updating files: 90% (3043/3381) Updating files: 91% (3077/3381) Updating files: 92% (3111/3381) Updating files: 93% (3145/3381) Updating files: 94% (3179/3381) Updating files: 95% (3212/3381) Updating files: 96% (3246/3381) Updating files: 97% (3280/3381) Updating files: 98% (3314/3381) Updating files: 99% (3348/3381) Updating files: 100% (3381/3381) Updating files: 100% (3381/3381), done. 2024-04-24T23:56:30.276 INFO:teuthology.orchestra.run.smithi086.stdout:HEAD is now at 63a66b66 Merge branch 'robot/pr-0-1713990671' into 'main' 2024-04-24T23:56:30.297 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-24T23:56:30.298 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T23:56:30.627 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ 2024-04-24T23:56:30.628 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ --input-file=- 2024-04-24T23:56:30.630 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi086.front.sepia.ceph.com to complete... 2024-04-24T23:56:32.426 INFO:teuthology.orchestra.run.smithi083.stderr:Updating files: 44% (1515/3381) Updating files: 45% (1522/3381) Updating files: 46% (1556/3381) Updating files: 47% (1590/3381) Updating files: 48% (1623/3381) Updating files: 49% (1657/3381) Updating files: 50% (1691/3381) Updating files: 51% (1725/3381) Updating files: 52% (1759/3381) Updating files: 53% (1792/3381) Updating files: 54% (1826/3381) Updating files: 55% (1860/3381) Updating files: 56% (1894/3381) Updating files: 57% (1928/3381) Updating files: 58% (1961/3381) Updating files: 59% (1995/3381) Updating files: 59% (2004/3381) Updating files: 60% (2029/3381) Updating files: 61% (2063/3381) Updating files: 62% (2097/3381) Updating files: 63% (2131/3381) Updating files: 64% (2164/3381) Updating files: 65% (2198/3381) Updating files: 66% (2232/3381) Updating files: 67% (2266/3381) Updating files: 68% (2300/3381) Updating files: 69% (2333/3381) Updating files: 69% (2362/3381) Updating files: 70% (2367/3381) Updating files: 71% (2401/3381) Updating files: 72% (2435/3381) Updating files: 73% (2469/3381) Updating files: 74% (2502/3381) Updating files: 75% (2536/3381) Updating files: 76% (2570/3381) Updating files: 77% (2604/3381) Updating files: 78% (2638/3381) Updating files: 79% (2671/3381) Updating files: 80% (2705/3381) Updating files: 81% (2739/3381) Updating files: 82% (2773/3381) Updating files: 83% (2807/3381) Updating files: 83% (2808/3381) Updating files: 84% (2841/3381) Updating files: 85% (2874/3381) Updating files: 86% (2908/3381) Updating files: 87% (2942/3381) Updating files: 88% (2976/3381) Updating files: 89% (3010/3381) Updating files: 90% (3043/3381) Updating files: 91% (3077/3381) Updating files: 92% (3111/3381) Updating files: 93% (3145/3381) Updating files: 94% (3179/3381) Updating files: 95% (3212/3381) Updating files: 96% (3246/3381) Updating files: 97% (3280/3381) Updating files: 98% (3314/3381) Updating files: 99% (3348/3381) Updating files: 100% (3381/3381) Updating files: 100% (3381/3381), done. 2024-04-24T23:56:32.431 INFO:teuthology.orchestra.run.smithi083.stdout:HEAD is now at 63a66b66 Merge branch 'robot/pr-0-1713990671' into 'main' 2024-04-24T23:56:32.452 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-24T23:56:32.452 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T23:56:32.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-24 23:56:32 URL:https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb [44805284/44805284] -> "/tmp/linux-image.deb" [1] 2024-04-24T23:56:32.632 INFO:teuthology.orchestra.run.smithi086.stderr:FINISHED --2024-04-24 23:56:32-- 2024-04-24T23:56:32.632 INFO:teuthology.orchestra.run.smithi086.stderr:Total wall clock time: 2.0s 2024-04-24T23:56:32.632 INFO:teuthology.orchestra.run.smithi086.stderr:Downloaded: 1 files, 43M in 1.7s (25.5 MB/s) 2024-04-24T23:56:32.636 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-24T23:56:32.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo dpkg -i /tmp/linux-image.deb 2024-04-24T23:56:32.675 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ 2024-04-24T23:56:32.675 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ --input-file=- 2024-04-24T23:56:32.677 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi083.front.sepia.ceph.com to complete... 2024-04-24T23:56:32.882 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g806ec36b2112. 2024-04-24T23:56:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24 23:56:34 URL:https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb [44805284/44805284] -> "/tmp/linux-image.deb" [1] 2024-04-24T23:56:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:FINISHED --2024-04-24 23:56:34-- 2024-04-24T23:56:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:Total wall clock time: 1.7s 2024-04-24T23:56:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:Downloaded: 1 files, 43M in 1.6s (27.2 MB/s) 2024-04-24T23:56:34.404 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-24T23:56:34.404 DEBUG:teuthology.orchestra.run.smithi083:> sudo dpkg -i /tmp/linux-image.deb 2024-04-24T23:56:34.670 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g806ec36b2112. 2024-04-24T23:56:34.831 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-04-24T23:56:34.835 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-04-24T23:56:34.942 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-24T23:56:36.696 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-04-24T23:56:36.700 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-04-24T23:56:36.876 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-24T23:56:38.650 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-24T23:56:38.760 INFO:teuthology.orchestra.run.smithi086.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-24T23:56:40.979 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-24T23:56:41.116 INFO:teuthology.orchestra.run.smithi083.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-24T23:56:59.878 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub' 2024-04-24T23:56:59.896 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-24T23:56:59.913 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-04-24T23:57:00.191 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:00.208 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:00.479 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-24T23:57:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-24T23:57:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-24T23:57:00.565 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-24T23:57:00.714 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-24T23:57:00.714 INFO:teuthology.orchestra.run.smithi086.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-24T23:57:00.714 INFO:teuthology.orchestra.run.smithi086.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-24T23:57:00.760 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-04-24T23:57:00.924 DEBUG:teuthology.orchestra.run.smithi086:> dpkg-deb -c /tmp/linux-image.deb 2024-04-24T23:57:00.932 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./ 2024-04-24T23:57:00.932 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./boot/ 2024-04-24T23:57:00.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7002012 2024-04-19 08:55 ./boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:00.947 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 172073 2024-04-19 08:55 ./boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:00.947 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13120512 2024-04-19 08:55 ./boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:00.965 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/ 2024-04-24T23:57:00.966 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/ 2024-04-24T23:57:00.966 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postinst.d/ 2024-04-24T23:57:00.966 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postrm.d/ 2024-04-24T23:57:00.966 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/preinst.d/ 2024-04-24T23:57:00.966 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/prerm.d/ 2024-04-24T23:57:00.966 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/ 2024-04-24T23:57:00.966 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/ 2024-04-24T23:57:00.966 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/ 2024-04-24T23:57:00.966 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/ 2024-04-24T23:57:00.966 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/ 2024-04-24T23:57:00.966 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/ 2024-04-24T23:57:00.966 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ 2024-04-24T23:57:00.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 507569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/aesni-intel.ko 2024-04-24T23:57:00.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T23:57:00.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T23:57:00.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T23:57:00.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T23:57:00.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 109905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T23:57:00.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 155249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T23:57:00.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T23:57:00.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T23:57:00.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 87009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T23:57:00.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T23:57:00.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T23:57:00.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T23:57:00.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T23:57:00.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T23:57:00.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T23:57:00.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T23:57:00.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T23:57:00.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T23:57:00.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T23:57:00.971 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/ 2024-04-24T23:57:00.971 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/ 2024-04-24T23:57:00.971 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/ 2024-04-24T23:57:00.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T23:57:00.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpuid.ko 2024-04-24T23:57:00.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/msr.ko 2024-04-24T23:57:00.972 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/ 2024-04-24T23:57:00.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 444425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T23:57:00.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1035881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T23:57:00.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2849609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T23:57:00.989 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ 2024-04-24T23:57:00.989 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/af_alg.ko 2024-04-24T23:57:00.989 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_hash.ko 2024-04-24T23:57:00.989 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_skcipher.ko 2024-04-24T23:57:00.989 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T23:57:00.990 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T23:57:00.990 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T23:57:00.990 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/ 2024-04-24T23:57:00.990 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-04-24T23:57:00.990 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/ 2024-04-24T23:57:00.990 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T23:57:00.990 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T23:57:00.990 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T23:57:00.990 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T23:57:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cryptd.ko 2024-04-24T23:57:00.992 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T23:57:00.992 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_simd.ko 2024-04-24T23:57:00.992 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T23:57:00.992 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T23:57:00.992 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T23:57:00.992 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T23:57:00.992 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T23:57:00.992 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T23:57:00.992 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T23:57:00.992 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T23:57:00.992 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lrw.ko 2024-04-24T23:57:00.992 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T23:57:00.992 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T23:57:00.993 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T23:57:00.993 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T23:57:00.993 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T23:57:00.993 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T23:57:00.993 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T23:57:00.993 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T23:57:00.993 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T23:57:00.993 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 169065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T23:57:00.993 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T23:57:00.993 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T23:57:00.993 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T23:57:00.993 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T23:57:00.993 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T23:57:00.993 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T23:57:00.994 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T23:57:00.994 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xts.ko 2024-04-24T23:57:00.994 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T23:57:00.994 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ 2024-04-24T23:57:00.994 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ 2024-04-24T23:57:00.994 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_extlog.ko 2024-04-24T23:57:00.994 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T23:57:00.994 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T23:57:00.994 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/ 2024-04-24T23:57:00.994 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/einj.ko 2024-04-24T23:57:00.994 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T23:57:00.994 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 119033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 96825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci.ko 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 107737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci.ko 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T23:57:00.995 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T23:57:00.996 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T23:57:00.996 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T23:57:00.996 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T23:57:00.996 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T23:57:00.996 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T23:57:00.996 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T23:57:00.996 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T23:57:00.996 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T23:57:00.996 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T23:57:00.996 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T23:57:00.996 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T23:57:00.996 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_legacy.ko 2024-04-24T23:57:00.996 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T23:57:00.996 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_radisys.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rz1000.ko 2024-04-24T23:57:00.997 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 102121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T23:57:00.998 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/ 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 163969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/ 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/ 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 140865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/ 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1241657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 203289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T23:57:00.999 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/ 2024-04-24T23:57:01.000 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 115265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T23:57:01.000 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 143433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T23:57:01.000 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/ 2024-04-24T23:57:01.000 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 126617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T23:57:01.000 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 119873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T23:57:01.000 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 313217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T23:57:01.000 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/ 2024-04-24T23:57:01.000 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 117233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T23:57:01.000 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/ 2024-04-24T23:57:01.000 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T23:57:01.000 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ 2024-04-24T23:57:01.000 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/applicom.ko 2024-04-24T23:57:01.000 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T23:57:01.000 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/ 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 186921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 179561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/ 2024-04-24T23:57:01.001 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T23:57:01.002 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/nvram.ko 2024-04-24T23:57:01.002 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T23:57:01.002 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T23:57:01.002 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/ 2024-04-24T23:57:01.002 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T23:57:01.002 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-04-24T23:57:01.002 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-04-24T23:57:01.002 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-04-24T23:57:01.002 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T23:57:01.002 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T23:57:01.002 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-04-24T23:57:01.002 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/ 2024-04-24T23:57:01.002 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_class.ko 2024-04-24T23:57:01.003 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_core.ko 2024-04-24T23:57:01.003 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-04-24T23:57:01.003 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/ 2024-04-24T23:57:01.003 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-palmas.ko 2024-04-24T23:57:01.003 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-si5351.ko 2024-04-24T23:57:01.003 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-twl6040.ko 2024-04-24T23:57:01.003 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-wm831x.ko 2024-04-24T23:57:01.003 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/ 2024-04-24T23:57:01.003 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T23:57:01.004 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T23:57:01.004 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T23:57:01.004 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ 2024-04-24T23:57:01.004 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ 2024-04-24T23:57:01.004 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 89257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T23:57:01.004 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 333657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T23:57:01.004 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/ 2024-04-24T23:57:01.004 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/ 2024-04-24T23:57:01.004 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/ 2024-04-24T23:57:01.004 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 751713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T23:57:01.054 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-04-24T23:57:01.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T23:57:01.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T23:57:01.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T23:57:01.054 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/ 2024-04-24T23:57:01.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T23:57:01.054 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/ 2024-04-24T23:57:01.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T23:57:01.054 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ 2024-04-24T23:57:01.055 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/ 2024-04-24T23:57:01.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T23:57:01.055 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ 2024-04-24T23:57:01.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 146769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T23:57:01.055 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ 2024-04-24T23:57:01.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 96145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T23:57:01.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T23:57:01.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T23:57:01.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T23:57:01.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T23:57:01.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T23:57:01.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T23:57:01.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T23:57:01.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T23:57:01.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T23:57:01.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T23:57:01.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 68697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T23:57:01.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T23:57:01.056 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/ 2024-04-24T23:57:01.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-gpio.ko 2024-04-24T23:57:01.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max14577.ko 2024-04-24T23:57:01.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max77693.ko 2024-04-24T23:57:01.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max8997.ko 2024-04-24T23:57:01.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-palmas.ko 2024-04-24T23:57:01.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-rt8973a.ko 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-sm5502.ko 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/ 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/dmi-sysfs.ko 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/ 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/efi-pstore.ko 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/ 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-adp5520.ko 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-amd8111.ko 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-arizona.ko 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9052.ko 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9055.ko 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-dln2.ko 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-f7188x.ko 2024-04-24T23:57:01.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-generic.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-kempld.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-lp3943.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max7300.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max730x.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max732x.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pca953x.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pcf857x.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-rdc321x.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch311x.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-tps65912.ko 2024-04-24T23:57:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl4030.ko 2024-04-24T23:57:01.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl6040.ko 2024-04-24T23:57:01.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T23:57:01.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-vx855.ko 2024-04-24T23:57:01.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm831x.ko 2024-04-24T23:57:01.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8350.ko 2024-04-24T23:57:01.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8994.ko 2024-04-24T23:57:01.059 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/ 2024-04-24T23:57:01.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-a4tech.ko 2024-04-24T23:57:01.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-apple.ko 2024-04-24T23:57:01.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T23:57:01.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T23:57:01.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T23:57:01.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-belkin.ko 2024-04-24T23:57:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cherry.ko 2024-04-24T23:57:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-chicony.ko 2024-04-24T23:57:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 74393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cp2112.ko 2024-04-24T23:57:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cypress.ko 2024-04-24T23:57:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T23:57:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T23:57:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T23:57:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T23:57:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ezkey.ko 2024-04-24T23:57:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T23:57:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-generic.ko 2024-04-24T23:57:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T23:57:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T23:57:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T23:57:01.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T23:57:01.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T23:57:01.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T23:57:01.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T23:57:01.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kensington.ko 2024-04-24T23:57:01.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T23:57:01.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T23:57:01.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T23:57:01.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T23:57:01.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T23:57:01.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lg-g15.ko 2024-04-24T23:57:01.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 68081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T23:57:01.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 121369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-magicmouse.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-microsoft.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-monterey.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ntrig.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T23:57:01.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T23:57:01.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T23:57:01.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T23:57:01.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T23:57:01.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T23:57:01.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T23:57:01.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T23:57:01.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T23:57:01.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T23:57:01.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T23:57:01.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T23:57:01.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T23:57:01.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T23:57:01.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 184745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T23:57:01.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T23:57:01.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T23:57:01.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T23:57:01.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 319921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid.ko 2024-04-24T23:57:01.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T23:57:01.065 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/ 2024-04-24T23:57:01.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 153633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbhid.ko 2024-04-24T23:57:01.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbkbd.ko 2024-04-24T23:57:01.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbmouse.ko 2024-04-24T23:57:01.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 261945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T23:57:01.065 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/ 2024-04-24T23:57:01.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 75065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T23:57:01.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 110681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T23:57:01.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 382033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T23:57:01.065 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ 2024-04-24T23:57:01.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 77153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T23:57:01.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T23:57:01.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T23:57:01.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T23:57:01.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T23:57:01.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T23:57:01.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T23:57:01.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 115057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T23:57:01.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T23:57:01.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T23:57:01.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T23:57:01.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T23:57:01.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T23:57:01.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T23:57:01.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 89025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T23:57:01.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T23:57:01.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T23:57:01.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T23:57:01.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T23:57:01.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 68865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T23:57:01.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T23:57:01.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T23:57:01.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T23:57:01.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T23:57:01.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T23:57:01.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9052-hwmon.ko 2024-04-24T23:57:01.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9055-hwmon.ko 2024-04-24T23:57:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T23:57:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 113201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T23:57:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T23:57:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T23:57:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T23:57:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc2103.ko 2024-04-24T23:57:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T23:57:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T23:57:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T23:57:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T23:57:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T23:57:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T23:57:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T23:57:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T23:57:01.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T23:57:01.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T23:57:01.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hih6130.ko 2024-04-24T23:57:01.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T23:57:01.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T23:57:01.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T23:57:01.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T23:57:01.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T23:57:01.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T23:57:01.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T23:57:01.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 156465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T23:57:01.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/jc42.ko 2024-04-24T23:57:01.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T23:57:01.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T23:57:01.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T23:57:01.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T23:57:01.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T23:57:01.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T23:57:01.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T23:57:01.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T23:57:01.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T23:57:01.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T23:57:01.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 86497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T23:57:01.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T23:57:01.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T23:57:01.071 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T23:57:01.071 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 131505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T23:57:01.071 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T23:57:01.071 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T23:57:01.071 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T23:57:01.071 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T23:57:01.071 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T23:57:01.071 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T23:57:01.071 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T23:57:01.071 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T23:57:01.071 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T23:57:01.071 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T23:57:01.071 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T23:57:01.071 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T23:57:01.072 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T23:57:01.072 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T23:57:01.072 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T23:57:01.072 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T23:57:01.072 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T23:57:01.072 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mc13783-adc.ko 2024-04-24T23:57:01.072 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T23:57:01.072 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-04-24T23:57:01.072 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T23:57:01.072 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 141353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T23:57:01.072 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T23:57:01.072 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T23:57:01.072 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T23:57:01.073 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T23:57:01.073 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T23:57:01.073 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ 2024-04-24T23:57:01.073 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T23:57:01.073 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T23:57:01.073 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T23:57:01.073 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T23:57:01.073 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T23:57:01.073 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T23:57:01.073 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T23:57:01.073 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T23:57:01.073 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T23:57:01.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T23:57:01.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T23:57:01.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T23:57:01.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T23:57:01.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T23:57:01.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T23:57:01.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T23:57:01.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T23:57:01.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T23:57:01.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T23:57:01.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T23:57:01.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T23:57:01.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T23:57:01.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T23:57:01.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T23:57:01.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T23:57:01.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T23:57:01.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T23:57:01.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T23:57:01.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T23:57:01.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T23:57:01.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T23:57:01.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T23:57:01.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 71169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T23:57:01.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 96577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T23:57:01.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T23:57:01.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 75129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T23:57:01.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 83425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T23:57:01.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 116777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T23:57:01.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 134441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T23:57:01.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T23:57:01.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T23:57:01.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-04-24T23:57:01.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-04-24T23:57:01.076 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/ 2024-04-24T23:57:01.076 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/ 2024-04-24T23:57:01.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T23:57:01.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T23:57:01.076 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/ 2024-04-24T23:57:01.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-04-24T23:57:01.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-04-24T23:57:01.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-04-24T23:57:01.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T23:57:01.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T23:57:01.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T23:57:01.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-04-24T23:57:01.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T23:57:01.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-04-24T23:57:01.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T23:57:01.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T23:57:01.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T23:57:01.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T23:57:01.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-04-24T23:57:01.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-04-24T23:57:01.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 76969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T23:57:01.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T23:57:01.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T23:57:01.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-04-24T23:57:01.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T23:57:01.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T23:57:01.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-04-24T23:57:01.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T23:57:01.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T23:57:01.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T23:57:01.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-04-24T23:57:01.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T23:57:01.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T23:57:01.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-04-24T23:57:01.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-04-24T23:57:01.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T23:57:01.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-04-24T23:57:01.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T23:57:01.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T23:57:01.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T23:57:01.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T23:57:01.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-04-24T23:57:01.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T23:57:01.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T23:57:01.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T23:57:01.080 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/ 2024-04-24T23:57:01.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-04-24T23:57:01.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-04-24T23:57:01.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-04-24T23:57:01.080 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ 2024-04-24T23:57:01.080 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ 2024-04-24T23:57:01.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 329697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T23:57:01.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1119913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T23:57:01.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 83905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T23:57:01.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 408433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T23:57:01.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 120921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T23:57:01.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 347217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 84945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/ 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 590929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/ 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 514641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/ 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 911665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 375545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 197121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/ 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 212553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-04-24T23:57:01.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ 2024-04-24T23:57:01.082 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ 2024-04-24T23:57:01.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 334401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T23:57:01.082 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ 2024-04-24T23:57:01.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 120673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T23:57:01.082 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ 2024-04-24T23:57:01.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 132585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T23:57:01.082 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ 2024-04-24T23:57:01.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 157937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T23:57:01.082 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ 2024-04-24T23:57:01.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 156673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T23:57:01.082 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ 2024-04-24T23:57:01.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/evbug.ko 2024-04-24T23:57:01.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ff-memless.ko 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/ 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/ 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lkkbd.ko 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8323.ko 2024-04-24T23:57:01.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8333.ko 2024-04-24T23:57:01.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-04-24T23:57:01.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-04-24T23:57:01.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-04-24T23:57:01.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-04-24T23:57:01.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/newtonkbd.ko 2024-04-24T23:57:01.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-04-24T23:57:01.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt1070.ko 2024-04-24T23:57:01.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt2160.ko 2024-04-24T23:57:01.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-04-24T23:57:01.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/stowaway.ko 2024-04-24T23:57:01.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/sunkbd.ko 2024-04-24T23:57:01.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-04-24T23:57:01.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-04-24T23:57:01.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-04-24T23:57:01.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/xtkbd.ko 2024-04-24T23:57:01.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/matrix-keymap.ko 2024-04-24T23:57:01.085 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/ 2024-04-24T23:57:01.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T23:57:01.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T23:57:01.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 160017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T23:57:01.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 114273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T23:57:01.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/gpio_mouse.ko 2024-04-24T23:57:01.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 428185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/psmouse.ko 2024-04-24T23:57:01.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T23:57:01.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T23:57:01.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T23:57:01.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T23:57:01.086 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/ 2024-04-24T23:57:01.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 140969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T23:57:01.086 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ 2024-04-24T23:57:01.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T23:57:01.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T23:57:01.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ct82c710.ko 2024-04-24T23:57:01.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T23:57:01.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/parkbd.ko 2024-04-24T23:57:01.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/pcips2.ko 2024-04-24T23:57:01.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ps2mult.ko 2024-04-24T23:57:01.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T23:57:01.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serport.ko 2024-04-24T23:57:01.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T23:57:01.087 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/ 2024-04-24T23:57:01.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-88pm860x.ko 2024-04-24T23:57:01.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-adp5520.ko 2024-04-24T23:57:01.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 77265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-bd2802.ko 2024-04-24T23:57:01.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T23:57:01.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da903x.ko 2024-04-24T23:57:01.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da9052.ko 2024-04-24T23:57:01.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-gpio.ko 2024-04-24T23:57:01.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T23:57:01.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3533.ko 2024-04-24T23:57:01.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm355x.ko 2024-04-24T23:57:01.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3642.ko 2024-04-24T23:57:01.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T23:57:01.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp8788.ko 2024-04-24T23:57:01.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-max8997.ko 2024-04-24T23:57:01.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-mc13783.ko 2024-04-24T23:57:01.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-menf21bmc.ko 2024-04-24T23:57:01.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca9532.ko 2024-04-24T23:57:01.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca955x.ko 2024-04-24T23:57:01.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca963x.ko 2024-04-24T23:57:01.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T23:57:01.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-tca6507.ko 2024-04-24T23:57:01.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm831x-status.ko 2024-04-24T23:57:01.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm8350.ko 2024-04-24T23:57:01.088 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ 2024-04-24T23:57:01.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T23:57:01.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T23:57:01.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T23:57:01.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T23:57:01.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T23:57:01.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T23:57:01.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T23:57:01.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T23:57:01.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T23:57:01.089 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/ 2024-04-24T23:57:01.089 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/ 2024-04-24T23:57:01.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 790753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T23:57:01.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T23:57:01.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 116825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bufio.ko 2024-04-24T23:57:01.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T23:57:01.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 199041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T23:57:01.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 134945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T23:57:01.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T23:57:01.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-era.ko 2024-04-24T23:57:01.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T23:57:01.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T23:57:01.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log.ko 2024-04-24T23:57:01.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-mirror.ko 2024-04-24T23:57:01.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T23:57:01.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T23:57:01.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T23:57:01.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-region-hash.ko 2024-04-24T23:57:01.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T23:57:01.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T23:57:01.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-snapshot.ko 2024-04-24T23:57:01.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T23:57:01.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 234457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T23:57:01.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T23:57:01.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-zero.ko 2024-04-24T23:57:01.091 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/ 2024-04-24T23:57:01.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 264329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T23:57:01.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T23:57:01.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 125009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T23:57:01.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 157681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T23:57:01.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 444665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T23:57:01.091 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/ 2024-04-24T23:57:01.091 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/ 2024-04-24T23:57:01.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 214841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T23:57:01.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 77985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T23:57:01.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T23:57:01.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T23:57:01.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 150761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T23:57:01.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T23:57:01.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 63873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T23:57:01.092 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/ 2024-04-24T23:57:01.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm800.ko 2024-04-24T23:57:01.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm805.ko 2024-04-24T23:57:01.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm80x.ko 2024-04-24T23:57:01.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona-i2c.ko 2024-04-24T23:57:01.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 148633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona.ko 2024-04-24T23:57:01.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/bcm590xx.ko 2024-04-24T23:57:01.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/cros_ec_dev.ko 2024-04-24T23:57:01.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/dln2.ko 2024-04-24T23:57:01.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/janz-cmodio.ko 2024-04-24T23:57:01.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/kempld-core.ko 2024-04-24T23:57:01.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-core.ko 2024-04-24T23:57:01.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-04-24T23:57:01.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lp3943.ko 2024-04-24T23:57:01.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T23:57:01.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T23:57:01.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/max8907.ko 2024-04-24T23:57:01.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-core.ko 2024-04-24T23:57:01.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-04-24T23:57:01.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/menf21bmc.ko 2024-04-24T23:57:01.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-adc.ko 2024-04-24T23:57:01.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-gpio.ko 2024-04-24T23:57:01.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633.ko 2024-04-24T23:57:01.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-04-24T23:57:01.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/retu-mfd.ko 2024-04-24T23:57:01.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 99425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/si476x-core.ko 2024-04-24T23:57:01.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T23:57:01.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps65010.ko 2024-04-24T23:57:01.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps6507x.ko 2024-04-24T23:57:01.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T23:57:01.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T23:57:01.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T23:57:01.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wm8994.ko 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot.ko 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/ 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/ 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/core.ko 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/ 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ds1682.ko 2024-04-24T23:57:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/dummy-irq.ko 2024-04-24T23:57:01.096 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/ 2024-04-24T23:57:01.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/echo.ko 2024-04-24T23:57:01.096 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/ 2024-04-24T23:57:01.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T23:57:01.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T23:57:01.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T23:57:01.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T23:57:01.096 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/ 2024-04-24T23:57:01.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 191649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-04-24T23:57:01.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hmc6352.ko 2024-04-24T23:57:01.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T23:57:01.096 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ 2024-04-24T23:57:01.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 92345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-04-24T23:57:01.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ics932s401.ko 2024-04-24T23:57:01.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T23:57:01.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T23:57:01.097 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/ 2024-04-24T23:57:01.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T23:57:01.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T23:57:01.097 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/ 2024-04-24T23:57:01.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 154169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T23:57:01.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T23:57:01.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 389033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T23:57:01.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/phantom.ko 2024-04-24T23:57:01.097 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/ 2024-04-24T23:57:01.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/st_drv.ko 2024-04-24T23:57:01.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T23:57:01.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T23:57:01.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T23:57:01.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T23:57:01.098 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/ 2024-04-24T23:57:01.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 234905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T23:57:01.098 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ 2024-04-24T23:57:01.098 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/ 2024-04-24T23:57:01.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 620961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T23:57:01.098 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/ 2024-04-24T23:57:01.098 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/ 2024-04-24T23:57:01.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T23:57:01.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T23:57:01.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T23:57:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/ 2024-04-24T23:57:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 587065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T23:57:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T23:57:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T23:57:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ 2024-04-24T23:57:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/ 2024-04-24T23:57:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T23:57:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T23:57:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 158649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T23:57:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T23:57:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ 2024-04-24T23:57:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T23:57:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T23:57:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T23:57:01.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 109521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T23:57:01.100 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/ 2024-04-24T23:57:01.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T23:57:01.100 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/ 2024-04-24T23:57:01.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 84329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T23:57:01.100 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/ 2024-04-24T23:57:01.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T23:57:01.100 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/ 2024-04-24T23:57:01.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T23:57:01.100 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/ 2024-04-24T23:57:01.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T23:57:01.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T23:57:01.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 115881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T23:57:01.100 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/ 2024-04-24T23:57:01.101 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/ 2024-04-24T23:57:01.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 124497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T23:57:01.101 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-04-24T23:57:01.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 128409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T23:57:01.101 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-04-24T23:57:01.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 110345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T23:57:01.101 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/ 2024-04-24T23:57:01.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T23:57:01.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 88953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T23:57:01.101 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/ 2024-04-24T23:57:01.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 105633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T23:57:01.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 228985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T23:57:01.101 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-04-24T23:57:01.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1733353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T23:57:01.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 167185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T23:57:01.102 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/ 2024-04-24T23:57:01.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 161417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T23:57:01.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 440625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T23:57:01.102 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/ 2024-04-24T23:57:01.102 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/ 2024-04-24T23:57:01.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 432201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T23:57:01.102 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/ 2024-04-24T23:57:01.102 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-04-24T23:57:01.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 257201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T23:57:01.102 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-04-24T23:57:01.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 446177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T23:57:01.102 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-04-24T23:57:01.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1311433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T23:57:01.103 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-04-24T23:57:01.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 206689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T23:57:01.103 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-04-24T23:57:01.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T23:57:01.103 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/ 2024-04-24T23:57:01.103 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/ 2024-04-24T23:57:01.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 269313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T23:57:01.103 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/ 2024-04-24T23:57:01.103 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/ 2024-04-24T23:57:01.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T23:57:01.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 77457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T23:57:01.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 169089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T23:57:01.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T23:57:01.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 76209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T23:57:01.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T23:57:01.104 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/ 2024-04-24T23:57:01.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 75945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T23:57:01.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T23:57:01.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T23:57:01.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ec_bhf.ko 2024-04-24T23:57:01.104 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/ 2024-04-24T23:57:01.104 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/ 2024-04-24T23:57:01.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 356377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T23:57:01.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T23:57:01.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 63185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T23:57:01.104 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/ 2024-04-24T23:57:01.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-04-24T23:57:01.105 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ 2024-04-24T23:57:01.105 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 115569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T23:57:01.105 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/ 2024-04-24T23:57:01.105 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 366217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T23:57:01.105 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/ 2024-04-24T23:57:01.105 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 755793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T23:57:01.160 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/ 2024-04-24T23:57:01.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 324577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T23:57:01.160 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/ 2024-04-24T23:57:01.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1196297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T23:57:01.160 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/ 2024-04-24T23:57:01.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 362593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T23:57:01.160 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/ 2024-04-24T23:57:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 671801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T23:57:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/ 2024-04-24T23:57:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 121537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T23:57:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-04-24T23:57:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 966705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T23:57:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-04-24T23:57:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 193585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T23:57:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T23:57:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/ 2024-04-24T23:57:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T23:57:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 126881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T23:57:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 150777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T23:57:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/ 2024-04-24T23:57:01.161 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-04-24T23:57:01.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 905033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T23:57:01.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 344145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T23:57:01.162 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-04-24T23:57:01.162 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-04-24T23:57:01.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1412417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T23:57:01.162 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ 2024-04-24T23:57:01.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-04-24T23:57:01.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-04-24T23:57:01.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-04-24T23:57:01.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 117353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T23:57:01.162 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/ 2024-04-24T23:57:01.162 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-04-24T23:57:01.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 131425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/ 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 181081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/ 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 171417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/ 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/ 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 300817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 105689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T23:57:01.163 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-04-24T23:57:01.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 885129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T23:57:01.164 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/ 2024-04-24T23:57:01.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T23:57:01.164 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/ 2024-04-24T23:57:01.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T23:57:01.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 100689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T23:57:01.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T23:57:01.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 264161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T23:57:01.164 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/ 2024-04-24T23:57:01.164 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-04-24T23:57:01.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 142449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-04-24T23:57:01.164 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/ 2024-04-24T23:57:01.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1350385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/ 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/ 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 80065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/ 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 74289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 79457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 100785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 83441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/ 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T23:57:01.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-04-24T23:57:01.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 644537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T23:57:01.166 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/ 2024-04-24T23:57:01.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 128497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T23:57:01.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 207993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T23:57:01.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 82633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T23:57:01.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T23:57:01.166 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/ 2024-04-24T23:57:01.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T23:57:01.166 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/ 2024-04-24T23:57:01.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 110729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T23:57:01.166 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/ 2024-04-24T23:57:01.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 98689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T23:57:01.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 102513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/ 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/ 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/ 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/defxx.ko 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/ 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 275801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/skfp.ko 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/ 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 290441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/ 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T23:57:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T23:57:01.168 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ 2024-04-24T23:57:01.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T23:57:01.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T23:57:01.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T23:57:01.168 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/ 2024-04-24T23:57:01.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T23:57:01.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T23:57:01.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-gpio.ko 2024-04-24T23:57:01.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T23:57:01.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T23:57:01.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T23:57:01.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T23:57:01.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T23:57:01.168 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/ 2024-04-24T23:57:01.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T23:57:01.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T23:57:01.169 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ 2024-04-24T23:57:01.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T23:57:01.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T23:57:01.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T23:57:01.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T23:57:01.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T23:57:01.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T23:57:01.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T23:57:01.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T23:57:01.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T23:57:01.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T23:57:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T23:57:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 96025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T23:57:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T23:57:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 121105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T23:57:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T23:57:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T23:57:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T23:57:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T23:57:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T23:57:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/ 2024-04-24T23:57:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/plip.ko 2024-04-24T23:57:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ 2024-04-24T23:57:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T23:57:01.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T23:57:01.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T23:57:01.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T23:57:01.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T23:57:01.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T23:57:01.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T23:57:01.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T23:57:01.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sb1000.ko 2024-04-24T23:57:01.171 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/ 2024-04-24T23:57:01.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T23:57:01.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T23:57:01.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 80161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T23:57:01.171 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/ 2024-04-24T23:57:01.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 137321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T23:57:01.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T23:57:01.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T23:57:01.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T23:57:01.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T23:57:01.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T23:57:01.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T23:57:01.172 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/ 2024-04-24T23:57:01.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 176497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T23:57:01.172 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/ 2024-04-24T23:57:01.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 383593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T23:57:01.172 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/ 2024-04-24T23:57:01.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 175537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T23:57:01.172 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ 2024-04-24T23:57:01.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T23:57:01.173 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/ 2024-04-24T23:57:01.173 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/ 2024-04-24T23:57:01.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 335857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T23:57:01.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 126553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T23:57:01.173 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/ 2024-04-24T23:57:01.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 168145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T23:57:01.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T23:57:01.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T23:57:01.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T23:57:01.173 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/ 2024-04-24T23:57:01.173 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/ 2024-04-24T23:57:01.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T23:57:01.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-04-24T23:57:01.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-04-24T23:57:01.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pci-stub.ko 2024-04-24T23:57:01.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T23:57:01.174 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/ 2024-04-24T23:57:01.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T23:57:01.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 179961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia.ko 2024-04-24T23:57:01.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_core.ko 2024-04-24T23:57:01.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-04-24T23:57:01.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T23:57:01.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 133753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T23:57:01.174 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/ 2024-04-24T23:57:01.174 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/ 2024-04-24T23:57:01.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-04-24T23:57:01.175 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/ 2024-04-24T23:57:01.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-04-24T23:57:01.175 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/ 2024-04-24T23:57:01.175 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/ 2024-04-24T23:57:01.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T23:57:01.175 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/ 2024-04-24T23:57:01.175 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/ 2024-04-24T23:57:01.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-04-24T23:57:01.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec.ko 2024-04-24T23:57:01.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-04-24T23:57:01.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-04-24T23:57:01.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-04-24T23:57:01.176 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-04-24T23:57:01.176 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-04-24T23:57:01.176 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ 2024-04-24T23:57:01.176 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 87481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T23:57:01.176 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T23:57:01.176 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T23:57:01.176 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T23:57:01.176 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T23:57:01.176 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 164769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T23:57:01.176 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T23:57:01.176 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T23:57:01.176 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T23:57:01.176 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T23:57:01.176 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T23:57:01.177 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T23:57:01.177 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ibm_rtl.ko 2024-04-24T23:57:01.177 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/ 2024-04-24T23:57:01.177 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T23:57:01.177 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-04-24T23:57:01.177 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T23:57:01.177 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T23:57:01.177 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T23:57:01.177 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T23:57:01.177 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T23:57:01.177 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T23:57:01.177 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T23:57:01.177 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T23:57:01.177 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T23:57:01.178 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T23:57:01.178 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T23:57:01.178 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/ 2024-04-24T23:57:01.178 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ 2024-04-24T23:57:01.178 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_battery.ko 2024-04-24T23:57:01.178 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_charger.ko 2024-04-24T23:57:01.178 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 68833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq2415x_charger.ko 2024-04-24T23:57:01.178 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24190_charger.ko 2024-04-24T23:57:01.178 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24735-charger.ko 2024-04-24T23:57:01.178 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-04-24T23:57:01.178 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9030_battery.ko 2024-04-24T23:57:01.178 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9052-battery.ko 2024-04-24T23:57:01.179 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2760_battery.ko 2024-04-24T23:57:01.179 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2780_battery.ko 2024-04-24T23:57:01.179 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2781_battery.ko 2024-04-24T23:57:01.179 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2782_battery.ko 2024-04-24T23:57:01.179 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/gpio-charger.ko 2024-04-24T23:57:01.179 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/isp1704_charger.ko 2024-04-24T23:57:01.179 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/lp8727_charger.ko 2024-04-24T23:57:01.179 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max14577_charger.ko 2024-04-24T23:57:01.179 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max17042_battery.ko 2024-04-24T23:57:01.179 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8903_charger.ko 2024-04-24T23:57:01.179 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8925_power.ko 2024-04-24T23:57:01.179 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/pcf50633-charger.ko 2024-04-24T23:57:01.179 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/sbs-battery.ko 2024-04-24T23:57:01.180 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/test_power.ko 2024-04-24T23:57:01.180 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/tps65090-charger.ko 2024-04-24T23:57:01.180 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_backup.ko 2024-04-24T23:57:01.180 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_power.ko 2024-04-24T23:57:01.180 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm8350_power.ko 2024-04-24T23:57:01.180 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/ 2024-04-24T23:57:01.180 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T23:57:01.180 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T23:57:01.180 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/ 2024-04-24T23:57:01.180 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/ 2024-04-24T23:57:01.180 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T23:57:01.180 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T23:57:01.180 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T23:57:01.180 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/ 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/ 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/ 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/ 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm80x.ko 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm860x.ko 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9052.ko 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9055.ko 2024-04-24T23:57:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9063.ko 2024-04-24T23:57:01.182 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T23:57:01.182 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 82225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T23:57:01.182 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T23:57:01.182 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T23:57:01.182 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T23:57:01.182 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T23:57:01.182 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T23:57:01.182 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T23:57:01.182 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T23:57:01.182 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T23:57:01.182 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T23:57:01.182 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T23:57:01.182 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T23:57:01.182 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-lp8788.ko 2024-04-24T23:57:01.183 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T23:57:01.183 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T23:57:01.183 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T23:57:01.183 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t86.ko 2024-04-24T23:57:01.183 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T23:57:01.183 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8907.ko 2024-04-24T23:57:01.183 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8925.ko 2024-04-24T23:57:01.183 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8997.ko 2024-04-24T23:57:01.183 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8998.ko 2024-04-24T23:57:01.183 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-04-24T23:57:01.183 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T23:57:01.183 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-palmas.ko 2024-04-24T23:57:01.183 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T23:57:01.184 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf50633.ko 2024-04-24T23:57:01.184 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T23:57:01.184 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T23:57:01.184 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T23:57:01.184 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T23:57:01.184 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rc5t583.ko 2024-04-24T23:57:01.184 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T23:57:01.184 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T23:57:01.184 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T23:57:01.184 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T23:57:01.184 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T23:57:01.184 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-s35390a.ko 2024-04-24T23:57:01.184 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T23:57:01.185 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps6586x.ko 2024-04-24T23:57:01.185 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps65910.ko 2024-04-24T23:57:01.185 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm831x.ko 2024-04-24T23:57:01.185 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm8350.ko 2024-04-24T23:57:01.185 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T23:57:01.185 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ 2024-04-24T23:57:01.185 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T23:57:01.185 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T23:57:01.185 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T23:57:01.185 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 164569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T23:57:01.185 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T23:57:01.185 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/ 2024-04-24T23:57:01.185 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 340721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T23:57:01.186 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 120337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T23:57:01.186 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/ 2024-04-24T23:57:01.186 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 468081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T23:57:01.186 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 380673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T23:57:01.186 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/ 2024-04-24T23:57:01.186 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 243705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-04-24T23:57:01.186 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T23:57:01.186 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/ 2024-04-24T23:57:01.186 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 135617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T23:57:01.186 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T23:57:01.186 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/ 2024-04-24T23:57:01.186 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 304993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T23:57:01.186 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/ 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1527113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/ 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 297801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/ 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 144401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/ 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 390713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/ 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 88881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 117481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 173017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T23:57:01.187 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T23:57:01.188 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T23:57:01.188 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/ 2024-04-24T23:57:01.188 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 267577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T23:57:01.188 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T23:57:01.188 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/ 2024-04-24T23:57:01.188 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 95329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T23:57:01.188 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 194017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T23:57:01.188 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fdomain.ko 2024-04-24T23:57:01.188 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/ 2024-04-24T23:57:01.188 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 366529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T23:57:01.188 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 251505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T23:57:01.188 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T23:57:01.188 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T23:57:01.188 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/imm.ko 2024-04-24T23:57:01.189 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T23:57:01.189 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 327841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T23:57:01.189 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 129785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T23:57:01.189 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/ 2024-04-24T23:57:01.189 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 397785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T23:57:01.189 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T23:57:01.189 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 63121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T23:57:01.189 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/ 2024-04-24T23:57:01.189 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 386849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T23:57:01.189 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 178177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T23:57:01.189 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T23:57:01.189 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/ 2024-04-24T23:57:01.189 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 274009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T23:57:01.189 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/ 2024-04-24T23:57:01.190 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2736001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T23:57:01.190 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/ 2024-04-24T23:57:01.190 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 88529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T23:57:01.190 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T23:57:01.190 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 389177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T23:57:01.190 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 104769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T23:57:01.190 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/ 2024-04-24T23:57:01.190 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 836945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T23:57:01.190 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/ 2024-04-24T23:57:01.190 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 158729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T23:57:01.190 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 96537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T23:57:01.190 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/ 2024-04-24T23:57:01.190 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-04-24T23:57:01.191 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-04-24T23:57:01.191 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-04-24T23:57:01.191 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-04-24T23:57:01.191 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/ 2024-04-24T23:57:01.191 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 443745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T23:57:01.191 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 152033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T23:57:01.191 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ppa.ko 2024-04-24T23:57:01.191 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T23:57:01.191 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/ 2024-04-24T23:57:01.191 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2324809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T23:57:01.191 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T23:57:01.191 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/ 2024-04-24T23:57:01.191 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 697337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T23:57:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qlogicfas408.ko 2024-04-24T23:57:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T23:57:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 322865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T23:57:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 219857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T23:57:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 357153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T23:57:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 121185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T23:57:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 93481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T23:57:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T23:57:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T23:57:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 138121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T23:57:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T23:57:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/ 2024-04-24T23:57:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 205025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T23:57:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T23:57:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T23:57:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T23:57:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/xen-scsifront.ko 2024-04-24T23:57:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ 2024-04-24T23:57:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 174913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T23:57:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/ 2024-04-24T23:57:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/ 2024-04-24T23:57:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 920169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T23:57:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/ 2024-04-24T23:57:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T23:57:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T23:57:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T23:57:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1170033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T23:57:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T23:57:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 149721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T23:57:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/ 2024-04-24T23:57:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T23:57:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/ 2024-04-24T23:57:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/ 2024-04-24T23:57:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/ 2024-04-24T23:57:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T23:57:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T23:57:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T23:57:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T23:57:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T23:57:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T23:57:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T23:57:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T23:57:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T23:57:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T23:57:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T23:57:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T23:57:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T23:57:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T23:57:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T23:57:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-04-24T23:57:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T23:57:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T23:57:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T23:57:01.196 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ 2024-04-24T23:57:01.196 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ 2024-04-24T23:57:01.196 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 99313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T23:57:01.196 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/moxa.ko 2024-04-24T23:57:01.196 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/mxser.ko 2024-04-24T23:57:01.196 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 134401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T23:57:01.196 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T23:57:01.196 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T23:57:01.196 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/ 2024-04-24T23:57:01.196 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/ 2024-04-24T23:57:01.196 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-04-24T23:57:01.196 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T23:57:01.196 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-04-24T23:57:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_uart.ko 2024-04-24T23:57:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T23:57:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-04-24T23:57:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/ 2024-04-24T23:57:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 120577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T23:57:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/rp2.ko 2024-04-24T23:57:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/sc16is7xx.ko 2024-04-24T23:57:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/ 2024-04-24T23:57:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ 2024-04-24T23:57:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 693081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T23:57:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ 2024-04-24T23:57:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T23:57:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-04-24T23:57:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/ 2024-04-24T23:57:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T23:57:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T23:57:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T23:57:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_dmem_genirq.ko 2024-04-24T23:57:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_mf624.ko 2024-04-24T23:57:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_netx.ko 2024-04-24T23:57:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T23:57:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-04-24T23:57:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T23:57:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/ 2024-04-24T23:57:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/ 2024-04-24T23:57:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/c67x00.ko 2024-04-24T23:57:01.199 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ 2024-04-24T23:57:01.199 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 189209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-04-24T23:57:01.199 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-04-24T23:57:01.199 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-04-24T23:57:01.199 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-04-24T23:57:01.199 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-04-24T23:57:01.199 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/ 2024-04-24T23:57:01.199 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T23:57:01.199 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 63073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T23:57:01.199 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T23:57:01.199 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 107417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T23:57:01.199 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ 2024-04-24T23:57:01.199 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ulpi.ko 2024-04-24T23:57:01.199 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/ 2024-04-24T23:57:01.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-04-24T23:57:01.200 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/ 2024-04-24T23:57:01.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-04-24T23:57:01.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-04-24T23:57:01.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 574593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3.ko 2024-04-24T23:57:01.200 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/ 2024-04-24T23:57:01.200 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/ 2024-04-24T23:57:01.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_ether.ko 2024-04-24T23:57:01.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_serial.ko 2024-04-24T23:57:01.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-04-24T23:57:01.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-04-24T23:57:01.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-04-24T23:57:01.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-04-24T23:57:01.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 131345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-04-24T23:57:01.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-04-24T23:57:01.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 146617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-04-24T23:57:01.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-04-24T23:57:01.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-04-24T23:57:01.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-04-24T23:57:01.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-04-24T23:57:01.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-04-24T23:57:01.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-04-24T23:57:01.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 71257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-04-24T23:57:01.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 84937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-04-24T23:57:01.201 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/ 2024-04-24T23:57:01.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-04-24T23:57:01.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-04-24T23:57:01.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-04-24T23:57:01.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-04-24T23:57:01.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-04-24T23:57:01.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-04-24T23:57:01.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-04-24T23:57:01.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-04-24T23:57:01.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-04-24T23:57:01.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-04-24T23:57:01.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-04-24T23:57:01.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-04-24T23:57:01.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-04-24T23:57:01.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-04-24T23:57:01.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 204049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/libcomposite.ko 2024-04-24T23:57:01.203 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/ 2024-04-24T23:57:01.203 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-04-24T23:57:01.203 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/ 2024-04-24T23:57:01.203 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 182561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-04-24T23:57:01.203 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-04-24T23:57:01.203 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-04-24T23:57:01.203 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-04-24T23:57:01.203 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-04-24T23:57:01.203 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2272.ko 2024-04-24T23:57:01.203 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2280.ko 2024-04-24T23:57:01.203 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 94841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-04-24T23:57:01.203 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-04-24T23:57:01.203 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 87465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-04-24T23:57:01.203 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 95441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-04-24T23:57:01.204 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 195969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-04-24T23:57:01.204 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ 2024-04-24T23:57:01.204 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/bcma-hcd.ko 2024-04-24T23:57:01.204 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/isp116x-hcd.ko 2024-04-24T23:57:01.204 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 104377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-04-24T23:57:01.204 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 102273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-04-24T23:57:01.204 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T23:57:01.204 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811_cs.ko 2024-04-24T23:57:01.204 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ssb-hcd.ko 2024-04-24T23:57:01.204 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-04-24T23:57:01.204 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/ 2024-04-24T23:57:01.204 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T23:57:01.204 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T23:57:01.205 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ 2024-04-24T23:57:01.205 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T23:57:01.205 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T23:57:01.205 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-04-24T23:57:01.205 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cytherm.ko 2024-04-24T23:57:01.205 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ehset.ko 2024-04-24T23:57:01.205 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T23:57:01.205 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T23:57:01.205 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T23:57:01.205 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T23:57:01.205 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T23:57:01.205 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T23:57:01.205 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T23:57:01.205 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T23:57:01.206 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/lvstest.ko 2024-04-24T23:57:01.206 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/ 2024-04-24T23:57:01.206 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T23:57:01.206 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T23:57:01.206 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T23:57:01.206 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T23:57:01.206 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T23:57:01.206 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 84265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbtest.ko 2024-04-24T23:57:01.206 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T23:57:01.206 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T23:57:01.206 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/ 2024-04-24T23:57:01.206 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/usbmon.ko 2024-04-24T23:57:01.206 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/ 2024-04-24T23:57:01.207 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 396209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/musb_hdrc.ko 2024-04-24T23:57:01.207 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/ 2024-04-24T23:57:01.207 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T23:57:01.207 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-04-24T23:57:01.207 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-isp1301.ko 2024-04-24T23:57:01.207 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-tahvo.ko 2024-04-24T23:57:01.207 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ 2024-04-24T23:57:01.207 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T23:57:01.207 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T23:57:01.207 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T23:57:01.207 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T23:57:01.207 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T23:57:01.207 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T23:57:01.207 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/f81232.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 188609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 152305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 142833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 122329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T23:57:01.208 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T23:57:01.209 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T23:57:01.209 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/metro-usb.ko 2024-04-24T23:57:01.209 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T23:57:01.209 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 89969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T23:57:01.209 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mxuport.ko 2024-04-24T23:57:01.209 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T23:57:01.209 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T23:57:01.209 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T23:57:01.209 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 193857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T23:57:01.209 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T23:57:01.209 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T23:57:01.209 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T23:57:01.209 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T23:57:01.210 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T23:57:01.210 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T23:57:01.210 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T23:57:01.210 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T23:57:01.210 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T23:57:01.210 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T23:57:01.210 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T23:57:01.210 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T23:57:01.210 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T23:57:01.210 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T23:57:01.210 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 151137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usbserial.ko 2024-04-24T23:57:01.210 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T23:57:01.210 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T23:57:01.211 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/wishbone-serial.ko 2024-04-24T23:57:01.211 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T23:57:01.211 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ 2024-04-24T23:57:01.211 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T23:57:01.211 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T23:57:01.211 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T23:57:01.211 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T23:57:01.211 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T23:57:01.211 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T23:57:01.211 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T23:57:01.211 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T23:57:01.211 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T23:57:01.211 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T23:57:01.211 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T23:57:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T23:57:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T23:57:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T23:57:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 201193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T23:57:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/ 2024-04-24T23:57:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 76041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T23:57:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T23:57:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 122321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T23:57:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/ 2024-04-24T23:57:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/ 2024-04-24T23:57:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 206921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T23:57:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T23:57:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 134265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T23:57:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 93673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T23:57:01.213 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/ 2024-04-24T23:57:01.213 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 138721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T23:57:01.213 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T23:57:01.213 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 71281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T23:57:01.213 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 107905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T23:57:01.213 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/ 2024-04-24T23:57:01.213 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/ 2024-04-24T23:57:01.213 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-04-24T23:57:01.213 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/aat2870_bl.ko 2024-04-24T23:57:01.213 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp5520_bl.ko 2024-04-24T23:57:01.213 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8860_bl.ko 2024-04-24T23:57:01.213 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8870_bl.ko 2024-04-24T23:57:01.213 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T23:57:01.214 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/as3711_bl.ko 2024-04-24T23:57:01.214 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/bd6107.ko 2024-04-24T23:57:01.214 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da903x_bl.ko 2024-04-24T23:57:01.214 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da9052_bl.ko 2024-04-24T23:57:01.214 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/gpio_backlight.ko 2024-04-24T23:57:01.214 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/kb3886_bl.ko 2024-04-24T23:57:01.214 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T23:57:01.214 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3533_bl.ko 2024-04-24T23:57:01.214 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3639_bl.ko 2024-04-24T23:57:01.214 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lv5207lp.ko 2024-04-24T23:57:01.215 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/max8925_bl.ko 2024-04-24T23:57:01.215 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pandora_bl.ko 2024-04-24T23:57:01.215 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-04-24T23:57:01.215 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T23:57:01.215 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/wm831x_bl.ko 2024-04-24T23:57:01.215 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/ 2024-04-24T23:57:01.215 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ 2024-04-24T23:57:01.215 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T23:57:01.215 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T23:57:01.215 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/matrox_w1.ko 2024-04-24T23:57:01.215 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/w1-gpio.ko 2024-04-24T23:57:01.215 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/ 2024-04-24T23:57:01.215 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T23:57:01.216 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T23:57:01.216 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T23:57:01.216 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T23:57:01.216 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T23:57:01.216 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T23:57:01.216 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T23:57:01.216 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T23:57:01.216 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T23:57:01.216 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T23:57:01.216 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T23:57:01.216 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 129433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T23:57:01.216 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ 2024-04-24T23:57:01.216 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/acquirewdt.ko 2024-04-24T23:57:01.217 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/advantechwdt.ko 2024-04-24T23:57:01.217 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T23:57:01.217 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T23:57:01.217 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/cpu5wdt.ko 2024-04-24T23:57:01.217 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9052_wdt.ko 2024-04-24T23:57:01.217 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9055_wdt.ko 2024-04-24T23:57:01.217 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9063_wdt.ko 2024-04-24T23:57:01.217 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/dw_wdt.ko 2024-04-24T23:57:01.217 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/eurotechwdt.ko 2024-04-24T23:57:01.217 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T23:57:01.217 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T23:57:01.217 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T23:57:01.217 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T23:57:01.218 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T23:57:01.218 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T23:57:01.218 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T23:57:01.218 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T23:57:01.218 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T23:57:01.218 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T23:57:01.218 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/kempld_wdt.ko 2024-04-24T23:57:01.218 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T23:57:01.218 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/mena21_wdt.ko 2024-04-24T23:57:01.218 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-04-24T23:57:01.218 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T23:57:01.218 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-04-24T23:57:01.218 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pc87413_wdt.ko 2024-04-24T23:57:01.218 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T23:57:01.219 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T23:57:01.219 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/retu_wdt.ko 2024-04-24T23:57:01.219 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-04-24T23:57:01.219 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-04-24T23:57:01.219 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T23:57:01.219 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sc1200wdt.ko 2024-04-24T23:57:01.219 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T23:57:01.219 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-04-24T23:57:01.219 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T23:57:01.219 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T23:57:01.219 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/twl4030_wdt.ko 2024-04-24T23:57:01.219 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T23:57:01.220 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T23:57:01.220 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T23:57:01.220 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T23:57:01.220 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wafer5823wdt.ko 2024-04-24T23:57:01.220 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T23:57:01.220 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm831x_wdt.ko 2024-04-24T23:57:01.220 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm8350_wdt.ko 2024-04-24T23:57:01.220 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T23:57:01.220 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/ 2024-04-24T23:57:01.220 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T23:57:01.220 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T23:57:01.220 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T23:57:01.220 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/ 2024-04-24T23:57:01.220 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 213913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T23:57:01.221 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T23:57:01.221 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-scsiback.ko 2024-04-24T23:57:01.221 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/ 2024-04-24T23:57:01.221 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T23:57:01.221 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ 2024-04-24T23:57:01.221 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/ 2024-04-24T23:57:01.221 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 192617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T23:57:01.221 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/ 2024-04-24T23:57:01.221 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/adfs.ko 2024-04-24T23:57:01.221 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/ 2024-04-24T23:57:01.221 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 197849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T23:57:01.221 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/ 2024-04-24T23:57:01.221 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1261113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/kafs.ko 2024-04-24T23:57:01.222 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/ 2024-04-24T23:57:01.222 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 127377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/autofs4.ko 2024-04-24T23:57:01.222 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/ 2024-04-24T23:57:01.222 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T23:57:01.222 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/ 2024-04-24T23:57:01.222 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/bfs.ko 2024-04-24T23:57:01.222 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T23:57:01.222 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/ 2024-04-24T23:57:01.222 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 4489593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T23:57:01.222 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/ 2024-04-24T23:57:01.222 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 376857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T23:57:01.222 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ 2024-04-24T23:57:01.222 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1586065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T23:57:01.259 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/ 2024-04-24T23:57:01.259 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 130073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T23:57:01.259 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/ 2024-04-24T23:57:01.259 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 139081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/configfs.ko 2024-04-24T23:57:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/ 2024-04-24T23:57:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T23:57:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/ 2024-04-24T23:57:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 813225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T23:57:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/ 2024-04-24T23:57:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/efs.ko 2024-04-24T23:57:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/ 2024-04-24T23:57:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1906817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T23:57:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/ 2024-04-24T23:57:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T23:57:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/ 2024-04-24T23:57:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/freevxfs.ko 2024-04-24T23:57:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/ 2024-04-24T23:57:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/ 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1486521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/ 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 183305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/ 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 282097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/ 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 249009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/hpfs.ko 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/ 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 93033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/ 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 533073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/ 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 361369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T23:57:01.261 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/ 2024-04-24T23:57:01.262 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 112609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T23:57:01.262 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/ 2024-04-24T23:57:01.262 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 591585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T23:57:01.262 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/ 2024-04-24T23:57:01.262 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/ 2024-04-24T23:57:01.262 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 102561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T23:57:01.262 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/ 2024-04-24T23:57:01.262 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 74353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T23:57:01.262 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/ 2024-04-24T23:57:01.262 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 147689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T23:57:01.262 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1320249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T23:57:01.262 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 82105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv2.ko 2024-04-24T23:57:01.262 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 151641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2518505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/ 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/ 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2024049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/ 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 734025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/ 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T23:57:01.263 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ascii.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T23:57:01.264 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 95625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 144209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 167137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 119513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T23:57:01.265 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T23:57:01.266 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T23:57:01.266 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T23:57:01.266 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T23:57:01.266 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T23:57:01.266 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T23:57:01.266 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T23:57:01.266 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T23:57:01.266 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T23:57:01.266 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T23:57:01.266 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T23:57:01.266 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T23:57:01.266 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T23:57:01.266 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T23:57:01.267 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T23:57:01.267 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ 2024-04-24T23:57:01.267 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ 2024-04-24T23:57:01.267 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 302073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T23:57:01.267 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ 2024-04-24T23:57:01.267 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 548169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T23:57:01.267 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ 2024-04-24T23:57:01.267 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T23:57:01.267 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 3245617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T23:57:01.267 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T23:57:01.267 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T23:57:01.267 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T23:57:01.268 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/ 2024-04-24T23:57:01.268 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/omfs.ko 2024-04-24T23:57:01.268 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/ 2024-04-24T23:57:01.268 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 462873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T23:57:01.268 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ 2024-04-24T23:57:01.268 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T23:57:01.268 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/ 2024-04-24T23:57:01.268 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/qnx4.ko 2024-04-24T23:57:01.268 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/ 2024-04-24T23:57:01.268 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/qnx6.ko 2024-04-24T23:57:01.268 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/ 2024-04-24T23:57:01.268 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_tree.ko 2024-04-24T23:57:01.268 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v1.ko 2024-04-24T23:57:01.268 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v2.ko 2024-04-24T23:57:01.269 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/ 2024-04-24T23:57:01.269 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 684841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T23:57:01.269 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/ 2024-04-24T23:57:01.269 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T23:57:01.269 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/ 2024-04-24T23:57:01.269 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/ 2024-04-24T23:57:01.269 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 3319041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T23:57:01.269 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/ 2024-04-24T23:57:01.269 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T23:57:01.269 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T23:57:01.269 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/ 2024-04-24T23:57:01.269 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 164185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T23:57:01.269 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/ 2024-04-24T23:57:01.269 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 115977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T23:57:01.270 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/ 2024-04-24T23:57:01.270 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 310017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T23:57:01.270 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ 2024-04-24T23:57:01.270 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 198233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T23:57:01.270 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/ 2024-04-24T23:57:01.270 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 5192705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T23:57:01.270 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/ 2024-04-24T23:57:01.270 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/ 2024-04-24T23:57:01.270 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/locktorture.ko 2024-04-24T23:57:01.270 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/ 2024-04-24T23:57:01.270 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/test_udelay.ko 2024-04-24T23:57:01.270 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T23:57:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ 2024-04-24T23:57:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T23:57:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc7.ko 2024-04-24T23:57:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T23:57:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/ 2024-04-24T23:57:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T23:57:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T23:57:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/interval_tree_test.ko 2024-04-24T23:57:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T23:57:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T23:57:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/ 2024-04-24T23:57:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T23:57:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T23:57:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/ 2024-04-24T23:57:01.272 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T23:57:01.272 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/memory-notifier-error-inject.ko 2024-04-24T23:57:01.272 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/notifier-error-inject.ko 2024-04-24T23:57:01.272 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/percpu_test.ko 2024-04-24T23:57:01.272 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/pm-notifier-error-inject.ko 2024-04-24T23:57:01.272 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/ 2024-04-24T23:57:01.272 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 152025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T23:57:01.272 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/rbtree_test.ko 2024-04-24T23:57:01.272 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/ 2024-04-24T23:57:01.272 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T23:57:01.272 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test-kstrtox.ko 2024-04-24T23:57:01.272 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 4912441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_bpf.ko 2024-04-24T23:57:01.272 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_firmware.ko 2024-04-24T23:57:01.272 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_module.ko 2024-04-24T23:57:01.273 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_user_copy.ko 2024-04-24T23:57:01.273 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T23:57:01.273 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T23:57:01.273 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T23:57:01.273 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/ 2024-04-24T23:57:01.273 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/xz_dec_test.ko 2024-04-24T23:57:01.273 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/ 2024-04-24T23:57:01.273 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T23:57:01.273 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ 2024-04-24T23:57:01.273 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/ 2024-04-24T23:57:01.273 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T23:57:01.273 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T23:57:01.273 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T23:57:01.273 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T23:57:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T23:57:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T23:57:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T23:57:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T23:57:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/ 2024-04-24T23:57:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T23:57:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T23:57:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T23:57:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T23:57:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T23:57:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/ 2024-04-24T23:57:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 106793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T23:57:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/ 2024-04-24T23:57:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 236793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T23:57:01.275 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T23:57:01.275 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T23:57:01.275 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T23:57:01.275 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/ 2024-04-24T23:57:01.275 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 879697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T23:57:01.275 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/ 2024-04-24T23:57:01.275 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 84761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T23:57:01.275 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1030873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T23:57:01.275 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ 2024-04-24T23:57:01.275 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T23:57:01.275 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T23:57:01.275 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T23:57:01.275 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T23:57:01.276 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T23:57:01.276 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T23:57:01.276 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T23:57:01.276 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T23:57:01.276 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T23:57:01.276 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T23:57:01.276 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T23:57:01.276 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T23:57:01.276 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T23:57:01.276 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T23:57:01.276 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T23:57:01.276 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T23:57:01.276 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T23:57:01.276 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T23:57:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T23:57:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T23:57:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T23:57:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T23:57:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T23:57:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/ 2024-04-24T23:57:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1243505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T23:57:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/ 2024-04-24T23:57:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 153801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T23:57:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/ 2024-04-24T23:57:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 244889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T23:57:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T23:57:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 71033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T23:57:01.277 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 74993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T23:57:01.278 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/ 2024-04-24T23:57:01.278 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 507705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T23:57:01.278 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T23:57:01.278 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T23:57:01.278 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T23:57:01.278 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T23:57:01.278 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/ 2024-04-24T23:57:01.278 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 128873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/hsr.ko 2024-04-24T23:57:01.278 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ 2024-04-24T23:57:01.278 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ 2024-04-24T23:57:01.278 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T23:57:01.278 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 357425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T23:57:01.279 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T23:57:01.279 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ 2024-04-24T23:57:01.279 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T23:57:01.279 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T23:57:01.279 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T23:57:01.279 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T23:57:01.279 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T23:57:01.279 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 75489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T23:57:01.279 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 77721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T23:57:01.279 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T23:57:01.279 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T23:57:01.279 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T23:57:01.279 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ 2024-04-24T23:57:01.279 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T23:57:01.280 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T23:57:01.280 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T23:57:01.280 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ip_tables.ko 2024-04-24T23:57:01.280 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T23:57:01.280 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-04-24T23:57:01.280 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T23:57:01.280 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T23:57:01.280 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T23:57:01.280 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-04-24T23:57:01.280 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T23:57:01.280 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T23:57:01.280 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T23:57:01.280 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T23:57:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T23:57:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T23:57:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T23:57:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T23:57:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T23:57:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-04-24T23:57:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T23:57:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T23:57:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T23:57:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T23:57:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T23:57:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T23:57:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T23:57:01.281 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T23:57:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T23:57:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T23:57:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T23:57:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T23:57:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T23:57:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T23:57:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T23:57:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T23:57:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T23:57:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T23:57:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T23:57:01.282 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_gre.ko 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 118881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 71089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/mip6.ko 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T23:57:01.283 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-04-24T23:57:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T23:57:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T23:57:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T23:57:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T23:57:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T23:57:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T23:57:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T23:57:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T23:57:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T23:57:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T23:57:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T23:57:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T23:57:01.284 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T23:57:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T23:57:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T23:57:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T23:57:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T23:57:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T23:57:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T23:57:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T23:57:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T23:57:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T23:57:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T23:57:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 94089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T23:57:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T23:57:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T23:57:01.285 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/ 2024-04-24T23:57:01.286 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 106369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T23:57:01.286 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/ 2024-04-24T23:57:01.286 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 153217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T23:57:01.286 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T23:57:01.286 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T23:57:01.286 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T23:57:01.286 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T23:57:01.286 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T23:57:01.286 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T23:57:01.286 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/ 2024-04-24T23:57:01.286 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 63345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/lapb.ko 2024-04-24T23:57:01.286 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/ 2024-04-24T23:57:01.286 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T23:57:01.287 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 332889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc2.ko 2024-04-24T23:57:01.287 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/ 2024-04-24T23:57:01.287 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 388873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T23:57:01.287 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/ 2024-04-24T23:57:01.287 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T23:57:01.287 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ 2024-04-24T23:57:01.287 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ 2024-04-24T23:57:01.287 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 147745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T23:57:01.287 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T23:57:01.287 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T23:57:01.287 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T23:57:01.287 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 124041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T23:57:01.287 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 124345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T23:57:01.288 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 126681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T23:57:01.288 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 126473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T23:57:01.288 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 137833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T23:57:01.288 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T23:57:01.288 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 133681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T23:57:01.288 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 139073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T23:57:01.288 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 138265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T23:57:01.288 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 136177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T23:57:01.288 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 139897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T23:57:01.288 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T23:57:01.288 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ 2024-04-24T23:57:01.288 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 534113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T23:57:01.288 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T23:57:01.288 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T23:57:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T23:57:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T23:57:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T23:57:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T23:57:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T23:57:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T23:57:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T23:57:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T23:57:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T23:57:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T23:57:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T23:57:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T23:57:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 496817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T23:57:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T23:57:01.290 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T23:57:01.290 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T23:57:01.290 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 134377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T23:57:01.290 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T23:57:01.290 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T23:57:01.290 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 140801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T23:57:01.290 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T23:57:01.290 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T23:57:01.290 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 79873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T23:57:01.290 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T23:57:01.290 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T23:57:01.290 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T23:57:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 117201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T23:57:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T23:57:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T23:57:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 118369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T23:57:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T23:57:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T23:57:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T23:57:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T23:57:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T23:57:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T23:57:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 807985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T23:57:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T23:57:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T23:57:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-04-24T23:57:01.292 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T23:57:01.292 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T23:57:01.292 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T23:57:01.292 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T23:57:01.292 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T23:57:01.292 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T23:57:01.292 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T23:57:01.292 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T23:57:01.292 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T23:57:01.292 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T23:57:01.292 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T23:57:01.292 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T23:57:01.292 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T23:57:01.292 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T23:57:01.293 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T23:57:01.293 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_osf.ko 2024-04-24T23:57:01.293 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T23:57:01.293 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T23:57:01.293 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T23:57:01.293 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T23:57:01.293 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T23:57:01.293 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_synproxy.ko 2024-04-24T23:57:01.293 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tproxy.ko 2024-04-24T23:57:01.293 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/x_tables.ko 2024-04-24T23:57:01.293 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T23:57:01.293 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T23:57:01.293 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T23:57:01.293 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T23:57:01.294 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T23:57:01.294 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T23:57:01.294 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T23:57:01.294 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T23:57:01.294 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T23:57:01.294 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T23:57:01.294 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T23:57:01.294 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T23:57:01.294 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T23:57:01.294 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T23:57:01.294 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T23:57:01.294 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T23:57:01.294 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T23:57:01.295 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T23:57:01.296 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T23:57:01.297 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T23:57:01.297 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T23:57:01.297 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T23:57:01.297 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T23:57:01.297 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T23:57:01.297 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T23:57:01.297 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T23:57:01.297 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T23:57:01.297 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T23:57:01.297 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T23:57:01.297 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T23:57:01.297 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T23:57:01.297 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T23:57:01.297 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T23:57:01.298 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T23:57:01.298 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T23:57:01.298 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T23:57:01.298 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T23:57:01.298 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T23:57:01.298 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T23:57:01.298 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T23:57:01.298 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpudp.ko 2024-04-24T23:57:01.298 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T23:57:01.298 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T23:57:01.298 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/ 2024-04-24T23:57:01.298 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T23:57:01.298 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/ 2024-04-24T23:57:01.298 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T23:57:01.299 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/ 2024-04-24T23:57:01.299 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 488825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T23:57:01.299 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T23:57:01.299 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T23:57:01.299 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T23:57:01.299 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/ 2024-04-24T23:57:01.299 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T23:57:01.299 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/ 2024-04-24T23:57:01.299 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 142025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/phonet.ko 2024-04-24T23:57:01.299 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 68817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/pn_pep.ko 2024-04-24T23:57:01.299 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/ 2024-04-24T23:57:01.299 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 318617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T23:57:01.300 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 202305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T23:57:01.300 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T23:57:01.300 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/ 2024-04-24T23:57:01.300 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1027897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/rxrpc.ko 2024-04-24T23:57:01.300 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/ 2024-04-24T23:57:01.300 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T23:57:01.300 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T23:57:01.300 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T23:57:01.300 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T23:57:01.300 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T23:57:01.300 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T23:57:01.300 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T23:57:01.300 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T23:57:01.300 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T23:57:01.301 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T23:57:01.301 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T23:57:01.301 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_cgroup.ko 2024-04-24T23:57:01.301 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T23:57:01.301 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T23:57:01.301 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T23:57:01.301 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 79145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T23:57:01.301 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T23:57:01.301 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T23:57:01.301 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T23:57:01.301 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T23:57:01.301 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T23:57:01.301 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T23:57:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T23:57:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T23:57:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T23:57:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T23:57:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq_codel.ko 2024-04-24T23:57:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T23:57:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T23:57:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T23:57:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T23:57:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T23:57:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T23:57:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T23:57:01.302 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T23:57:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T23:57:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T23:57:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T23:57:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T23:57:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T23:57:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T23:57:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T23:57:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T23:57:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T23:57:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T23:57:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/ 2024-04-24T23:57:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1187313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T23:57:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T23:57:01.303 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/ 2024-04-24T23:57:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/ 2024-04-24T23:57:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 432409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T23:57:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T23:57:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1940889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T23:57:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/ 2024-04-24T23:57:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 890729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T23:57:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/ 2024-04-24T23:57:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1122337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T23:57:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T23:57:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/ 2024-04-24T23:57:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T23:57:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/ 2024-04-24T23:57:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 105609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T23:57:01.305 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T23:57:01.305 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 123769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T23:57:01.305 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T23:57:01.305 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T23:57:01.305 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/ 2024-04-24T23:57:01.305 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 165841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/x25.ko 2024-04-24T23:57:01.305 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/ 2024-04-24T23:57:01.305 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_algo.ko 2024-04-24T23:57:01.305 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T23:57:01.305 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 131865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_user.ko 2024-04-24T23:57:01.305 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 644769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T23:57:01.305 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 628542 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T23:57:01.305 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7706 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T23:57:01.305 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T23:57:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9941 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T23:57:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70990 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T23:57:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 146959 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T23:57:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 232835 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T23:57:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 210 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T23:57:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72602 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T23:57:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 879 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T23:57:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 258450 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T23:57:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 313019 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T23:57:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/ 2024-04-24T23:57:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/ 2024-04-24T23:57:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/ 2024-04-24T23:57:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/ 2024-04-24T23:57:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 172 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/changelog.Debian.gz 2024-04-24T23:57:01.307 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 691 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/copyright 2024-04-24T23:57:01.311 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:01.311 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:01.311 DEBUG:teuthology.orchestra.run.smithi086:> egrep '(submenu|menuentry.*6.9.0-rc3-g806ec36b2112).*{' /boot/grub/grub.cfg 2024-04-24T23:57:01.318 INFO:teuthology.orchestra.run.smithi086.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-24T23:57:01.318 INFO:teuthology.orchestra.run.smithi086.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-24T23:57:01.319 INFO:teuthology.orchestra.run.smithi086.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-24T23:57:01.319 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-04-24T23:57:01.319 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:01.319 DEBUG:teuthology.orchestra.run.smithi086:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-04-24T23:57:01.363 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi086.front.sepia.ceph.com to complete... 2024-04-24T23:57:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub' 2024-04-24T23:57:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-24T23:57:01.786 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-04-24T23:57:02.045 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:02.051 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:02.299 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-24T23:57:02.305 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-24T23:57:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-24T23:57:02.355 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-24T23:57:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-24T23:57:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-24T23:57:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-24T23:57:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-04-24T23:57:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:Sourcing file `/etc/default/grub' 2024-04-24T23:57:03.813 INFO:teuthology.orchestra.run.smithi083.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-24T23:57:03.841 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2024-04-24T23:57:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:04.185 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:04.503 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-24T23:57:04.510 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-24T23:57:04.571 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-24T23:57:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-24T23:57:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-24T23:57:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-24T23:57:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-24T23:57:04.786 INFO:teuthology.orchestra.run.smithi083.stderr:done 2024-04-24T23:57:05.007 DEBUG:teuthology.orchestra.run.smithi083:> dpkg-deb -c /tmp/linux-image.deb 2024-04-24T23:57:05.015 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./ 2024-04-24T23:57:05.015 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./boot/ 2024-04-24T23:57:05.015 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7002012 2024-04-19 08:55 ./boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:05.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 172073 2024-04-19 08:55 ./boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:05.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13120512 2024-04-19 08:55 ./boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:05.053 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/ 2024-04-24T23:57:05.053 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/ 2024-04-24T23:57:05.053 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postinst.d/ 2024-04-24T23:57:05.053 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postrm.d/ 2024-04-24T23:57:05.053 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/preinst.d/ 2024-04-24T23:57:05.054 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/prerm.d/ 2024-04-24T23:57:05.054 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/ 2024-04-24T23:57:05.054 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/ 2024-04-24T23:57:05.054 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/ 2024-04-24T23:57:05.054 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/ 2024-04-24T23:57:05.054 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/ 2024-04-24T23:57:05.054 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/ 2024-04-24T23:57:05.054 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ 2024-04-24T23:57:05.054 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 507569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/aesni-intel.ko 2024-04-24T23:57:05.056 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T23:57:05.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T23:57:05.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T23:57:05.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T23:57:05.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 109905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T23:57:05.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 155249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T23:57:05.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T23:57:05.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T23:57:05.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 87009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T23:57:05.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T23:57:05.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 50305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T23:57:05.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 50377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T23:57:05.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 54513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T23:57:05.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T23:57:05.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 48049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T23:57:05.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 62873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T23:57:05.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 53569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T23:57:05.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T23:57:05.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T23:57:05.058 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/ 2024-04-24T23:57:05.058 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/ 2024-04-24T23:57:05.058 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/ 2024-04-24T23:57:05.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T23:57:05.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpuid.ko 2024-04-24T23:57:05.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/msr.ko 2024-04-24T23:57:05.058 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/ 2024-04-24T23:57:05.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 444425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T23:57:05.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1035881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T23:57:05.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 2849609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T23:57:05.111 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ 2024-04-24T23:57:05.112 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 65025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/af_alg.ko 2024-04-24T23:57:05.112 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_hash.ko 2024-04-24T23:57:05.112 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_skcipher.ko 2024-04-24T23:57:05.112 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T23:57:05.112 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T23:57:05.112 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T23:57:05.112 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/ 2024-04-24T23:57:05.112 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-04-24T23:57:05.112 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/ 2024-04-24T23:57:05.112 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T23:57:05.112 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T23:57:05.112 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T23:57:05.112 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T23:57:05.112 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T23:57:05.113 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T23:57:05.113 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T23:57:05.113 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T23:57:05.113 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T23:57:05.113 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T23:57:05.113 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T23:57:05.113 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T23:57:05.113 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T23:57:05.113 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T23:57:05.113 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T23:57:05.113 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T23:57:05.113 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T23:57:05.113 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T23:57:05.114 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 59129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cryptd.ko 2024-04-24T23:57:05.114 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 48113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T23:57:05.114 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_simd.ko 2024-04-24T23:57:05.114 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T23:57:05.114 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T23:57:05.114 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T23:57:05.114 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T23:57:05.114 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T23:57:05.114 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T23:57:05.114 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T23:57:05.114 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T23:57:05.114 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lrw.ko 2024-04-24T23:57:05.114 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T23:57:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T23:57:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T23:57:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T23:57:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T23:57:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T23:57:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T23:57:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T23:57:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T23:57:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 169065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T23:57:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T23:57:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T23:57:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T23:57:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T23:57:05.115 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T23:57:05.116 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T23:57:05.116 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T23:57:05.116 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xts.ko 2024-04-24T23:57:05.116 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T23:57:05.116 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ 2024-04-24T23:57:05.116 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ 2024-04-24T23:57:05.116 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_extlog.ko 2024-04-24T23:57:05.116 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T23:57:05.116 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T23:57:05.116 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/ 2024-04-24T23:57:05.116 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/einj.ko 2024-04-24T23:57:05.116 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T23:57:05.116 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T23:57:05.116 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T23:57:05.117 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T23:57:05.117 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 119033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T23:57:05.117 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ 2024-04-24T23:57:05.117 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T23:57:05.117 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 96825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci.ko 2024-04-24T23:57:05.117 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T23:57:05.117 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 107737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci.ko 2024-04-24T23:57:05.117 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T23:57:05.117 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T23:57:05.117 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T23:57:05.117 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T23:57:05.117 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T23:57:05.117 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T23:57:05.117 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T23:57:05.118 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T23:57:05.118 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T23:57:05.118 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T23:57:05.118 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T23:57:05.118 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T23:57:05.118 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T23:57:05.118 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T23:57:05.118 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T23:57:05.118 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T23:57:05.118 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T23:57:05.118 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T23:57:05.118 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 60257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_legacy.ko 2024-04-24T23:57:05.118 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T23:57:05.119 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T23:57:05.119 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T23:57:05.119 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T23:57:05.119 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T23:57:05.119 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T23:57:05.119 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T23:57:05.119 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T23:57:05.119 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T23:57:05.119 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T23:57:05.119 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T23:57:05.119 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T23:57:05.119 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T23:57:05.119 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_radisys.ko 2024-04-24T23:57:05.119 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rz1000.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 102121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 85409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T23:57:05.120 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 45905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 53353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/ 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 163969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/ 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/ 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 140865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/ 2024-04-24T23:57:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1241657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T23:57:05.122 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 203289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T23:57:05.122 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/ 2024-04-24T23:57:05.122 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 115265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T23:57:05.122 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 143433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T23:57:05.122 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/ 2024-04-24T23:57:05.122 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 126617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T23:57:05.122 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 119873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T23:57:05.122 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 313217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T23:57:05.122 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/ 2024-04-24T23:57:05.122 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 117233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T23:57:05.122 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/ 2024-04-24T23:57:05.122 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 70385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T23:57:05.122 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ 2024-04-24T23:57:05.123 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/applicom.ko 2024-04-24T23:57:05.123 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T23:57:05.123 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/ 2024-04-24T23:57:05.123 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T23:57:05.123 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T23:57:05.123 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T23:57:05.123 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T23:57:05.123 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T23:57:05.123 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ 2024-04-24T23:57:05.123 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 37257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T23:57:05.123 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 186921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T23:57:05.123 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T23:57:05.123 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 179561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T23:57:05.123 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 85441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 65313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/ 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 103409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/nvram.ko 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 59241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/ 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 39193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T23:57:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/ 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_class.ko 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 62545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_core.ko 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/ 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-palmas.ko 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 57289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-si5351.ko 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-twl6040.ko 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-wm831x.ko 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/ 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ 2024-04-24T23:57:05.125 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 89257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T23:57:05.126 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 333657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T23:57:05.126 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/ 2024-04-24T23:57:05.126 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/ 2024-04-24T23:57:05.126 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/ 2024-04-24T23:57:05.126 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 751713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T23:57:05.126 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-04-24T23:57:05.126 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T23:57:05.126 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T23:57:05.126 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T23:57:05.126 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/ 2024-04-24T23:57:05.126 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 90441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T23:57:05.126 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/ 2024-04-24T23:57:05.126 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/ 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 146769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 96145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 51105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T23:57:05.127 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 64161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T23:57:05.128 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T23:57:05.128 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T23:57:05.128 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 68697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T23:57:05.128 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T23:57:05.128 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/ 2024-04-24T23:57:05.128 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-gpio.ko 2024-04-24T23:57:05.128 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max14577.ko 2024-04-24T23:57:05.128 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max77693.ko 2024-04-24T23:57:05.128 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max8997.ko 2024-04-24T23:57:05.128 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-palmas.ko 2024-04-24T23:57:05.128 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-rt8973a.ko 2024-04-24T23:57:05.128 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-sm5502.ko 2024-04-24T23:57:05.128 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/ 2024-04-24T23:57:05.128 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/dmi-sysfs.ko 2024-04-24T23:57:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/ 2024-04-24T23:57:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/efi-pstore.ko 2024-04-24T23:57:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T23:57:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/ 2024-04-24T23:57:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-adp5520.ko 2024-04-24T23:57:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-amd8111.ko 2024-04-24T23:57:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-arizona.ko 2024-04-24T23:57:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9052.ko 2024-04-24T23:57:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9055.ko 2024-04-24T23:57:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-dln2.ko 2024-04-24T23:57:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 81073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-f7188x.ko 2024-04-24T23:57:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-generic.ko 2024-04-24T23:57:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T23:57:05.129 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-04-24T23:57:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-kempld.ko 2024-04-24T23:57:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-lp3943.ko 2024-04-24T23:57:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max7300.ko 2024-04-24T23:57:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max730x.ko 2024-04-24T23:57:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max732x.ko 2024-04-24T23:57:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-04-24T23:57:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pca953x.ko 2024-04-24T23:57:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pcf857x.ko 2024-04-24T23:57:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-rdc321x.ko 2024-04-24T23:57:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch.ko 2024-04-24T23:57:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch311x.ko 2024-04-24T23:57:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-tps65912.ko 2024-04-24T23:57:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl4030.ko 2024-04-24T23:57:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl6040.ko 2024-04-24T23:57:05.131 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T23:57:05.131 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-vx855.ko 2024-04-24T23:57:05.131 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm831x.ko 2024-04-24T23:57:05.131 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8350.ko 2024-04-24T23:57:05.131 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8994.ko 2024-04-24T23:57:05.131 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/ 2024-04-24T23:57:05.131 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-a4tech.ko 2024-04-24T23:57:05.131 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-apple.ko 2024-04-24T23:57:05.131 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T23:57:05.131 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T23:57:05.131 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T23:57:05.131 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-belkin.ko 2024-04-24T23:57:05.131 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cherry.ko 2024-04-24T23:57:05.131 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-chicony.ko 2024-04-24T23:57:05.132 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 74393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cp2112.ko 2024-04-24T23:57:05.132 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cypress.ko 2024-04-24T23:57:05.132 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T23:57:05.132 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T23:57:05.132 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T23:57:05.132 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T23:57:05.132 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ezkey.ko 2024-04-24T23:57:05.132 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T23:57:05.132 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-generic.ko 2024-04-24T23:57:05.132 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T23:57:05.132 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T23:57:05.132 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T23:57:05.132 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T23:57:05.132 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kensington.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 54217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 37553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lg-g15.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 68081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 121369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-magicmouse.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-microsoft.ko 2024-04-24T23:57:05.133 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-monterey.ko 2024-04-24T23:57:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 59001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T23:57:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 39697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ntrig.ko 2024-04-24T23:57:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T23:57:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T23:57:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T23:57:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 91241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T23:57:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T23:57:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T23:57:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T23:57:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T23:57:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T23:57:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T23:57:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T23:57:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 62129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T23:57:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T23:57:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T23:57:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T23:57:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T23:57:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T23:57:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T23:57:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T23:57:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 81113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T23:57:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T23:57:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 184745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T23:57:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T23:57:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T23:57:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T23:57:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 319921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid.ko 2024-04-24T23:57:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 39529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/ 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 153633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbhid.ko 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbkbd.ko 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbmouse.ko 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 261945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/ 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 75065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 110681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 382033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 77153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 67009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T23:57:05.137 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T23:57:05.138 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T23:57:05.138 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T23:57:05.138 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 115057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T23:57:05.138 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T23:57:05.138 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 55225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T23:57:05.138 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T23:57:05.138 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T23:57:05.138 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T23:57:05.138 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T23:57:05.138 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 89025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T23:57:05.138 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 49153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T23:57:05.138 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 81473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T23:57:05.138 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T23:57:05.138 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 48177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 68865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 60193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 69009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 47977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9052-hwmon.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9055-hwmon.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 113201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc2103.ko 2024-04-24T23:57:05.139 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 78921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 108961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 50697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 65769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 59761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 56313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hih6130.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T23:57:05.140 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T23:57:05.141 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T23:57:05.141 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T23:57:05.141 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 156465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T23:57:05.141 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/jc42.ko 2024-04-24T23:57:05.141 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T23:57:05.141 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T23:57:05.141 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T23:57:05.141 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 57769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T23:57:05.141 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T23:57:05.141 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T23:57:05.141 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T23:57:05.141 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T23:57:05.141 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 37049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T23:57:05.141 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T23:57:05.142 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 86497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T23:57:05.142 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 55641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T23:57:05.142 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T23:57:05.142 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T23:57:05.142 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 131505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T23:57:05.142 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T23:57:05.142 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T23:57:05.142 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T23:57:05.142 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T23:57:05.142 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T23:57:05.142 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T23:57:05.142 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T23:57:05.142 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T23:57:05.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T23:57:05.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T23:57:05.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T23:57:05.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T23:57:05.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T23:57:05.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T23:57:05.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T23:57:05.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T23:57:05.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 39729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T23:57:05.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mc13783-adc.ko 2024-04-24T23:57:05.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T23:57:05.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-04-24T23:57:05.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 56505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T23:57:05.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 141353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T23:57:05.144 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 72737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T23:57:05.144 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 67841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T23:57:05.144 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 97777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T23:57:05.144 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 61921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T23:57:05.144 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T23:57:05.144 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ 2024-04-24T23:57:05.144 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T23:57:05.144 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T23:57:05.144 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T23:57:05.144 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T23:57:05.144 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T23:57:05.144 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T23:57:05.144 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T23:57:05.144 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 101233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T23:57:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T23:57:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T23:57:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T23:57:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T23:57:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T23:57:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T23:57:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T23:57:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T23:57:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T23:57:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T23:57:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T23:57:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 48713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T23:57:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T23:57:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 46473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T23:57:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T23:57:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T23:57:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T23:57:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T23:57:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T23:57:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T23:57:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T23:57:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 50217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T23:57:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 59665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T23:57:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 62737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T23:57:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 71169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T23:57:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 96577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T23:57:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 85913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T23:57:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 75129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T23:57:05.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 83425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T23:57:05.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 116777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T23:57:05.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 134441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T23:57:05.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T23:57:05.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T23:57:05.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-04-24T23:57:05.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-04-24T23:57:05.147 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/ 2024-04-24T23:57:05.147 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/ 2024-04-24T23:57:05.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T23:57:05.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T23:57:05.147 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/ 2024-04-24T23:57:05.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-04-24T23:57:05.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-04-24T23:57:05.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-04-24T23:57:05.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T23:57:05.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T23:57:05.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T23:57:05.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-04-24T23:57:05.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T23:57:05.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-04-24T23:57:05.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 64233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T23:57:05.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T23:57:05.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T23:57:05.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T23:57:05.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-04-24T23:57:05.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 76969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 52937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 57217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-04-24T23:57:05.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-04-24T23:57:05.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T23:57:05.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-04-24T23:57:05.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T23:57:05.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T23:57:05.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T23:57:05.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T23:57:05.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 59193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-04-24T23:57:05.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T23:57:05.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T23:57:05.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T23:57:05.150 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/ 2024-04-24T23:57:05.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-04-24T23:57:05.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-04-24T23:57:05.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-04-24T23:57:05.151 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ 2024-04-24T23:57:05.151 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ 2024-04-24T23:57:05.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 329697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T23:57:05.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1119913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T23:57:05.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 83905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T23:57:05.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 408433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T23:57:05.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 120921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T23:57:05.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 347217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T23:57:05.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 84945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T23:57:05.151 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ 2024-04-24T23:57:05.151 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/ 2024-04-24T23:57:05.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 590929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T23:57:05.152 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/ 2024-04-24T23:57:05.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 514641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T23:57:05.152 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/ 2024-04-24T23:57:05.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 911665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T23:57:05.152 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ 2024-04-24T23:57:05.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 375545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T23:57:05.152 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ 2024-04-24T23:57:05.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 197121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-04-24T23:57:05.208 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/ 2024-04-24T23:57:05.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 212553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-04-24T23:57:05.209 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ 2024-04-24T23:57:05.209 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ 2024-04-24T23:57:05.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 334401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T23:57:05.209 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ 2024-04-24T23:57:05.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 120673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T23:57:05.209 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ 2024-04-24T23:57:05.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 132585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T23:57:05.209 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ 2024-04-24T23:57:05.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 157937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T23:57:05.209 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ 2024-04-24T23:57:05.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 156673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T23:57:05.209 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ 2024-04-24T23:57:05.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/evbug.ko 2024-04-24T23:57:05.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ff-memless.ko 2024-04-24T23:57:05.210 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/ 2024-04-24T23:57:05.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T23:57:05.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T23:57:05.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 50713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T23:57:05.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 49441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T23:57:05.210 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/ 2024-04-24T23:57:05.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-04-24T23:57:05.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-04-24T23:57:05.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-04-24T23:57:05.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-04-24T23:57:05.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 45473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T23:57:05.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-04-24T23:57:05.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lkkbd.ko 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8323.ko 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8333.ko 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/newtonkbd.ko 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt1070.ko 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt2160.ko 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/stowaway.ko 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/sunkbd.ko 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/xtkbd.ko 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/matrix-keymap.ko 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/ 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 39553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 160017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 114273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/gpio_mouse.ko 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 428185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/psmouse.ko 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/ 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 140969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ct82c710.ko 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/parkbd.ko 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/pcips2.ko 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ps2mult.ko 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serport.ko 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/ 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-88pm860x.ko 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-adp5520.ko 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 77265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-bd2802.ko 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da903x.ko 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da9052.ko 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-gpio.ko 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3533.ko 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm355x.ko 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3642.ko 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp8788.ko 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-max8997.ko 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-mc13783.ko 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-menf21bmc.ko 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca9532.ko 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca955x.ko 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca963x.ko 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-tca6507.ko 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm831x-status.ko 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm8350.ko 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/ 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/ 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 790753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 116825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bufio.ko 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 58281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T23:57:05.216 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 199041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T23:57:05.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 134945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T23:57:05.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T23:57:05.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 67497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-era.ko 2024-04-24T23:57:05.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T23:57:05.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 50457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T23:57:05.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log.ko 2024-04-24T23:57:05.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 60977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-mirror.ko 2024-04-24T23:57:05.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 97209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T23:57:05.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T23:57:05.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 85689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T23:57:05.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-region-hash.ko 2024-04-24T23:57:05.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T23:57:05.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T23:57:05.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-snapshot.ko 2024-04-24T23:57:05.218 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T23:57:05.218 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 234457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T23:57:05.218 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 66425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T23:57:05.218 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-zero.ko 2024-04-24T23:57:05.218 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/ 2024-04-24T23:57:05.218 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 264329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T23:57:05.218 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T23:57:05.218 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 125009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T23:57:05.218 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 157681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T23:57:05.218 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 444665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T23:57:05.218 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/ 2024-04-24T23:57:05.218 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/ 2024-04-24T23:57:05.218 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 214841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T23:57:05.219 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 77985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T23:57:05.219 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 54785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T23:57:05.219 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T23:57:05.219 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 150761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T23:57:05.219 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 97281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T23:57:05.219 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 63873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T23:57:05.219 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/ 2024-04-24T23:57:05.219 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm800.ko 2024-04-24T23:57:05.219 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm805.ko 2024-04-24T23:57:05.219 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm80x.ko 2024-04-24T23:57:05.219 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona-i2c.ko 2024-04-24T23:57:05.219 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 148633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona.ko 2024-04-24T23:57:05.219 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/bcm590xx.ko 2024-04-24T23:57:05.219 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/cros_ec_dev.ko 2024-04-24T23:57:05.220 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/dln2.ko 2024-04-24T23:57:05.220 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/janz-cmodio.ko 2024-04-24T23:57:05.220 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 59089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/kempld-core.ko 2024-04-24T23:57:05.220 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-core.ko 2024-04-24T23:57:05.220 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-04-24T23:57:05.220 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lp3943.ko 2024-04-24T23:57:05.220 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 60497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T23:57:05.220 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T23:57:05.220 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/max8907.ko 2024-04-24T23:57:05.220 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-core.ko 2024-04-24T23:57:05.220 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-04-24T23:57:05.220 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/menf21bmc.ko 2024-04-24T23:57:05.220 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-adc.ko 2024-04-24T23:57:05.220 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-gpio.ko 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633.ko 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/retu-mfd.ko 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 99425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/si476x-core.ko 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 58057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 60681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps65010.ko 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps6507x.ko 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 52361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wm8994.ko 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 64265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot.ko 2024-04-24T23:57:05.221 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/ 2024-04-24T23:57:05.222 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 57521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T23:57:05.222 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T23:57:05.222 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T23:57:05.222 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 58633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T23:57:05.222 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/ 2024-04-24T23:57:05.222 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-04-24T23:57:05.222 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 47449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/core.ko 2024-04-24T23:57:05.222 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/ 2024-04-24T23:57:05.222 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T23:57:05.222 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ds1682.ko 2024-04-24T23:57:05.222 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/dummy-irq.ko 2024-04-24T23:57:05.222 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/ 2024-04-24T23:57:05.222 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/echo.ko 2024-04-24T23:57:05.222 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/ 2024-04-24T23:57:05.223 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T23:57:05.223 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T23:57:05.223 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T23:57:05.223 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T23:57:05.223 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/ 2024-04-24T23:57:05.223 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 191649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-04-24T23:57:05.223 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hmc6352.ko 2024-04-24T23:57:05.223 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 37761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T23:57:05.223 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ 2024-04-24T23:57:05.223 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 92345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-04-24T23:57:05.223 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ics932s401.ko 2024-04-24T23:57:05.223 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T23:57:05.223 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T23:57:05.223 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/ 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 54865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/ 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 154169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 69809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 389033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 37961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/phantom.ko 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/ 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 91145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/st_drv.ko 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 55793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/ 2024-04-24T23:57:05.224 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 234905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/ 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 620961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/ 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/ 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 49545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/ 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 587065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/ 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 53753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T23:57:05.225 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 49201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T23:57:05.226 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 158649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T23:57:05.226 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 85953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T23:57:05.226 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ 2024-04-24T23:57:05.226 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T23:57:05.226 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T23:57:05.226 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T23:57:05.226 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 109521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T23:57:05.226 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/ 2024-04-24T23:57:05.226 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 64705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T23:57:05.226 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/ 2024-04-24T23:57:05.226 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 84329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T23:57:05.226 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/ 2024-04-24T23:57:05.226 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 70737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T23:57:05.226 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/ 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 52705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/ 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 65857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 56057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 115881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/ 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/ 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 124497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 128409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 110345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/ 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 101857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T23:57:05.227 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 88953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/ 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 105633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 228985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1733353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 167185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/ 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 161417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 440625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/ 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/ 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 432201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/ 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-04-24T23:57:05.228 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 257201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 446177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1311433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 206689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/ 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/ 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 269313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/ 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/ 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 85385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 77457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T23:57:05.229 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 169089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T23:57:05.230 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T23:57:05.230 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 76209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T23:57:05.230 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T23:57:05.230 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/ 2024-04-24T23:57:05.230 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 75945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T23:57:05.230 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 78705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T23:57:05.230 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T23:57:05.230 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ec_bhf.ko 2024-04-24T23:57:05.230 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/ 2024-04-24T23:57:05.230 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/ 2024-04-24T23:57:05.230 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 356377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T23:57:05.230 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 61737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T23:57:05.230 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 63185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T23:57:05.230 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/ 2024-04-24T23:57:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 51513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-04-24T23:57:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ 2024-04-24T23:57:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 115569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T23:57:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/ 2024-04-24T23:57:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 366217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T23:57:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/ 2024-04-24T23:57:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 755793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T23:57:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/ 2024-04-24T23:57:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 324577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T23:57:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/ 2024-04-24T23:57:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1196297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T23:57:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/ 2024-04-24T23:57:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 362593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T23:57:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/ 2024-04-24T23:57:05.232 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 671801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T23:57:05.232 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/ 2024-04-24T23:57:05.232 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 121537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T23:57:05.232 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-04-24T23:57:05.232 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 966705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T23:57:05.232 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-04-24T23:57:05.232 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 193585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T23:57:05.232 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 108633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T23:57:05.232 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/ 2024-04-24T23:57:05.232 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T23:57:05.232 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 126881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T23:57:05.232 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 150777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T23:57:05.232 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/ 2024-04-24T23:57:05.232 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-04-24T23:57:05.233 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 905033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T23:57:05.233 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 344145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T23:57:05.233 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-04-24T23:57:05.233 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-04-24T23:57:05.233 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1412417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T23:57:05.233 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ 2024-04-24T23:57:05.233 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-04-24T23:57:05.233 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 54497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-04-24T23:57:05.233 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-04-24T23:57:05.233 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 117353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T23:57:05.233 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/ 2024-04-24T23:57:05.233 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-04-24T23:57:05.233 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 131425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T23:57:05.234 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ 2024-04-24T23:57:05.234 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 91465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T23:57:05.234 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 56689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T23:57:05.234 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/ 2024-04-24T23:57:05.234 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 181081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T23:57:05.234 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/ 2024-04-24T23:57:05.234 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 171417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T23:57:05.234 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/ 2024-04-24T23:57:05.234 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 58249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T23:57:05.234 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 50121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T23:57:05.234 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/ 2024-04-24T23:57:05.234 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-04-24T23:57:05.234 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 300817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T23:57:05.235 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 105689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T23:57:05.235 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-04-24T23:57:05.235 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 885129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T23:57:05.235 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/ 2024-04-24T23:57:05.235 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 52305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T23:57:05.235 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/ 2024-04-24T23:57:05.235 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 81641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T23:57:05.235 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 100689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T23:57:05.235 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T23:57:05.235 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 264161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T23:57:05.235 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/ 2024-04-24T23:57:05.235 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-04-24T23:57:05.235 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 142449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/ 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1350385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/ 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 57953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/ 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 66505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 80065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/ 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 74289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 79457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 100785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 83441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/ 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-04-24T23:57:05.236 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-04-24T23:57:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 46985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T23:57:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-04-24T23:57:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 644537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T23:57:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/ 2024-04-24T23:57:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 128497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T23:57:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 207993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T23:57:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 82633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T23:57:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 91353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T23:57:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/ 2024-04-24T23:57:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 59865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T23:57:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/ 2024-04-24T23:57:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 110729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T23:57:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/ 2024-04-24T23:57:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 98689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T23:57:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 102513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T23:57:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/ 2024-04-24T23:57:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 61497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T23:57:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T23:57:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/ 2024-04-24T23:57:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 69913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T23:57:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/ 2024-04-24T23:57:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 49449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/defxx.ko 2024-04-24T23:57:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/ 2024-04-24T23:57:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 275801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/skfp.ko 2024-04-24T23:57:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 91105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T23:57:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/ 2024-04-24T23:57:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 290441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T23:57:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/ 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 108329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 78065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/ 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-gpio.ko 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 62681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T23:57:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/ 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 48313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 61985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 52865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 53409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T23:57:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 103377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 96025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 121105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 62113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/ 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 55353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/plip.ko 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T23:57:05.241 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 53233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sb1000.ko 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/ 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 52073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 80161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/ 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 137321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T23:57:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 108089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/ 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 176497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/ 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 383593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/ 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 175537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/ 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/ 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 335857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 126553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/ 2024-04-24T23:57:05.243 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 168145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 103889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/ 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/ 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pci-stub.ko 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 55329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/ 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 179961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia.ko 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 78161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_core.ko 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-04-24T23:57:05.244 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 133753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/ 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/ 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/ 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/ 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/ 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 78193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/ 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/ 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 61041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec.ko 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-04-24T23:57:05.245 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-04-24T23:57:05.246 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-04-24T23:57:05.246 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-04-24T23:57:05.246 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-04-24T23:57:05.246 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ 2024-04-24T23:57:05.246 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 87481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T23:57:05.246 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T23:57:05.246 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 48257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T23:57:05.246 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T23:57:05.246 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T23:57:05.246 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 164769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T23:57:05.246 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 46185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T23:57:05.246 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 62073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T23:57:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T23:57:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 51881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T23:57:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T23:57:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T23:57:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ibm_rtl.ko 2024-04-24T23:57:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/ 2024-04-24T23:57:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T23:57:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-04-24T23:57:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 54001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T23:57:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T23:57:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T23:57:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T23:57:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T23:57:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 61417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/ 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_battery.ko 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_charger.ko 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 68833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq2415x_charger.ko 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 73073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24190_charger.ko 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24735-charger.ko 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9030_battery.ko 2024-04-24T23:57:05.248 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9052-battery.ko 2024-04-24T23:57:05.249 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2760_battery.ko 2024-04-24T23:57:05.249 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2780_battery.ko 2024-04-24T23:57:05.249 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2781_battery.ko 2024-04-24T23:57:05.249 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2782_battery.ko 2024-04-24T23:57:05.249 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/gpio-charger.ko 2024-04-24T23:57:05.249 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/isp1704_charger.ko 2024-04-24T23:57:05.249 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/lp8727_charger.ko 2024-04-24T23:57:05.249 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max14577_charger.ko 2024-04-24T23:57:05.249 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 37833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max17042_battery.ko 2024-04-24T23:57:05.249 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8903_charger.ko 2024-04-24T23:57:05.249 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8925_power.ko 2024-04-24T23:57:05.249 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/pcf50633-charger.ko 2024-04-24T23:57:05.249 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/sbs-battery.ko 2024-04-24T23:57:05.249 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/test_power.ko 2024-04-24T23:57:05.250 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/tps65090-charger.ko 2024-04-24T23:57:05.250 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_backup.ko 2024-04-24T23:57:05.250 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_power.ko 2024-04-24T23:57:05.250 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm8350_power.ko 2024-04-24T23:57:05.250 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/ 2024-04-24T23:57:05.250 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 73297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T23:57:05.250 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T23:57:05.250 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/ 2024-04-24T23:57:05.250 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/ 2024-04-24T23:57:05.250 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T23:57:05.250 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T23:57:05.250 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T23:57:05.250 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 49817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T23:57:05.250 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 91393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/ 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/ 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/ 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 67513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 47137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/ 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm80x.ko 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm860x.ko 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9052.ko 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9055.ko 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9063.ko 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T23:57:05.251 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 82225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-lp8788.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 46913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T23:57:05.252 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t86.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8907.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8925.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8997.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8998.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-palmas.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 56777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf50633.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T23:57:05.253 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T23:57:05.254 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rc5t583.ko 2024-04-24T23:57:05.254 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T23:57:05.254 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T23:57:05.254 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T23:57:05.254 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T23:57:05.254 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T23:57:05.254 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-s35390a.ko 2024-04-24T23:57:05.254 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T23:57:05.254 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps6586x.ko 2024-04-24T23:57:05.254 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps65910.ko 2024-04-24T23:57:05.254 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm831x.ko 2024-04-24T23:57:05.254 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm8350.ko 2024-04-24T23:57:05.254 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 90377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 69593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 65945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 164569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/ 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 340721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 120337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/ 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 468081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 380673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/ 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 243705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T23:57:05.255 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/ 2024-04-24T23:57:05.256 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 135617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T23:57:05.256 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T23:57:05.256 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/ 2024-04-24T23:57:05.256 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 304993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T23:57:05.256 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/ 2024-04-24T23:57:05.256 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1527113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T23:57:05.256 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/ 2024-04-24T23:57:05.256 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 297801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T23:57:05.256 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/ 2024-04-24T23:57:05.256 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 144401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T23:57:05.256 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 47457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T23:57:05.256 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/ 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 390713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/ 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 88881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 117481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 173017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 91569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 45401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/ 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 267577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/ 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 95329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T23:57:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 194017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T23:57:05.258 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fdomain.ko 2024-04-24T23:57:05.258 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/ 2024-04-24T23:57:05.258 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 366529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T23:57:05.258 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 251505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T23:57:05.258 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 65761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T23:57:05.258 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 57097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T23:57:05.258 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/imm.ko 2024-04-24T23:57:05.258 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 55153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T23:57:05.258 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 327841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T23:57:05.258 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 129785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T23:57:05.258 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/ 2024-04-24T23:57:05.258 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 397785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T23:57:05.258 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T23:57:05.258 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 63121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T23:57:05.259 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/ 2024-04-24T23:57:05.259 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 386849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T23:57:05.259 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 178177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T23:57:05.259 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 64809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T23:57:05.259 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/ 2024-04-24T23:57:05.259 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 274009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T23:57:05.259 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/ 2024-04-24T23:57:05.259 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 2736001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T23:57:05.303 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/ 2024-04-24T23:57:05.304 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 88529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T23:57:05.304 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T23:57:05.304 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 389177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T23:57:05.304 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 104769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T23:57:05.304 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/ 2024-04-24T23:57:05.304 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 836945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T23:57:05.304 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/ 2024-04-24T23:57:05.304 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 158729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T23:57:05.304 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 96537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T23:57:05.304 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/ 2024-04-24T23:57:05.304 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 101625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-04-24T23:57:05.304 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-04-24T23:57:05.304 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/ 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 443745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 152033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ppa.ko 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 66977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/ 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 2324809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 103601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/ 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 697337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qlogicfas408.ko 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 322865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T23:57:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 219857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T23:57:05.306 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 357153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T23:57:05.306 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 121185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T23:57:05.306 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 93481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T23:57:05.306 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 55401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T23:57:05.306 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T23:57:05.306 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 138121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T23:57:05.306 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 57401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T23:57:05.306 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/ 2024-04-24T23:57:05.306 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 205025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T23:57:05.306 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 55425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T23:57:05.306 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 59185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T23:57:05.306 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 48777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 59417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/xen-scsifront.ko 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 174913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/ 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/ 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 920169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/ 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 60321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 48129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 54233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1170033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 149721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/ 2024-04-24T23:57:05.307 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 90057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T23:57:05.308 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/ 2024-04-24T23:57:05.308 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/ 2024-04-24T23:57:05.308 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/ 2024-04-24T23:57:05.308 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T23:57:05.308 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T23:57:05.308 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T23:57:05.308 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T23:57:05.308 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T23:57:05.308 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T23:57:05.308 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T23:57:05.308 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T23:57:05.308 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T23:57:05.308 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T23:57:05.309 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T23:57:05.309 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T23:57:05.309 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 46065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T23:57:05.309 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T23:57:05.309 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T23:57:05.309 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-04-24T23:57:05.309 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T23:57:05.309 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T23:57:05.309 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T23:57:05.309 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ 2024-04-24T23:57:05.309 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ 2024-04-24T23:57:05.309 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 99313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T23:57:05.309 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 69729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/moxa.ko 2024-04-24T23:57:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 61649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/mxser.ko 2024-04-24T23:57:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 134401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T23:57:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 37905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T23:57:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T23:57:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/ 2024-04-24T23:57:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/ 2024-04-24T23:57:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-04-24T23:57:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 65121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T23:57:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-04-24T23:57:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_uart.ko 2024-04-24T23:57:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T23:57:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 91665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-04-24T23:57:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/ 2024-04-24T23:57:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 120577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T23:57:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/rp2.ko 2024-04-24T23:57:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 65641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/sc16is7xx.ko 2024-04-24T23:57:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/ 2024-04-24T23:57:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ 2024-04-24T23:57:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 693081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T23:57:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ 2024-04-24T23:57:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T23:57:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-04-24T23:57:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/ 2024-04-24T23:57:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 55617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T23:57:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T23:57:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T23:57:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_dmem_genirq.ko 2024-04-24T23:57:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_mf624.ko 2024-04-24T23:57:05.312 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_netx.ko 2024-04-24T23:57:05.312 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T23:57:05.312 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-04-24T23:57:05.312 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T23:57:05.312 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/ 2024-04-24T23:57:05.312 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/ 2024-04-24T23:57:05.312 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 90321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/c67x00.ko 2024-04-24T23:57:05.312 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ 2024-04-24T23:57:05.312 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 189209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-04-24T23:57:05.312 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-04-24T23:57:05.312 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-04-24T23:57:05.312 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-04-24T23:57:05.312 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-04-24T23:57:05.312 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/ 2024-04-24T23:57:05.313 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 101545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T23:57:05.313 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 63073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T23:57:05.313 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 56721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T23:57:05.313 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 107417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T23:57:05.313 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ 2024-04-24T23:57:05.313 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ulpi.ko 2024-04-24T23:57:05.313 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/ 2024-04-24T23:57:05.313 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-04-24T23:57:05.313 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/ 2024-04-24T23:57:05.313 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-04-24T23:57:05.313 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-04-24T23:57:05.313 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 574593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3.ko 2024-04-24T23:57:05.313 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/ 2024-04-24T23:57:05.313 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/ 2024-04-24T23:57:05.314 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 72249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_ether.ko 2024-04-24T23:57:05.314 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 61953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_serial.ko 2024-04-24T23:57:05.314 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-04-24T23:57:05.314 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 49737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-04-24T23:57:05.314 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-04-24T23:57:05.314 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-04-24T23:57:05.314 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 131345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-04-24T23:57:05.314 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 62321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-04-24T23:57:05.314 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 146617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-04-24T23:57:05.314 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 70689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-04-24T23:57:05.314 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-04-24T23:57:05.314 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 37649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-04-24T23:57:05.314 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 70017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-04-24T23:57:05.314 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 101249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-04-24T23:57:05.315 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-04-24T23:57:05.315 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 71257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-04-24T23:57:05.315 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 84937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-04-24T23:57:05.315 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/ 2024-04-24T23:57:05.315 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-04-24T23:57:05.315 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-04-24T23:57:05.315 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-04-24T23:57:05.315 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-04-24T23:57:05.315 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-04-24T23:57:05.315 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-04-24T23:57:05.315 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-04-24T23:57:05.315 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-04-24T23:57:05.315 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-04-24T23:57:05.315 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 78073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 204049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/libcomposite.ko 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/ 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/ 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 182561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 45545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 66513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 85217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 69313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 90313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2272.ko 2024-04-24T23:57:05.316 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 90361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2280.ko 2024-04-24T23:57:05.317 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 94841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-04-24T23:57:05.317 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-04-24T23:57:05.317 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 87465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-04-24T23:57:05.317 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 95441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-04-24T23:57:05.317 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 195969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-04-24T23:57:05.317 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ 2024-04-24T23:57:05.317 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/bcma-hcd.ko 2024-04-24T23:57:05.317 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 108321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/isp116x-hcd.ko 2024-04-24T23:57:05.317 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 104377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-04-24T23:57:05.317 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 102273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-04-24T23:57:05.317 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 60105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T23:57:05.317 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811_cs.ko 2024-04-24T23:57:05.317 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ssb-hcd.ko 2024-04-24T23:57:05.318 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-04-24T23:57:05.318 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/ 2024-04-24T23:57:05.318 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T23:57:05.318 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T23:57:05.318 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ 2024-04-24T23:57:05.318 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 47353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T23:57:05.318 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T23:57:05.318 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-04-24T23:57:05.318 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cytherm.ko 2024-04-24T23:57:05.318 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ehset.ko 2024-04-24T23:57:05.318 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T23:57:05.318 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T23:57:05.319 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T23:57:05.319 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T23:57:05.319 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T23:57:05.319 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T23:57:05.319 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T23:57:05.319 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T23:57:05.319 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/lvstest.ko 2024-04-24T23:57:05.319 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/ 2024-04-24T23:57:05.319 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 85313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T23:57:05.319 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T23:57:05.319 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T23:57:05.319 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T23:57:05.319 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T23:57:05.320 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 84265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbtest.ko 2024-04-24T23:57:05.320 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 52065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T23:57:05.320 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T23:57:05.320 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/ 2024-04-24T23:57:05.320 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 91529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/usbmon.ko 2024-04-24T23:57:05.320 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/ 2024-04-24T23:57:05.320 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 396209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/musb_hdrc.ko 2024-04-24T23:57:05.320 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/ 2024-04-24T23:57:05.320 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T23:57:05.320 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-04-24T23:57:05.320 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-isp1301.ko 2024-04-24T23:57:05.320 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-tahvo.ko 2024-04-24T23:57:05.320 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ 2024-04-24T23:57:05.320 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T23:57:05.321 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T23:57:05.321 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T23:57:05.321 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T23:57:05.321 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T23:57:05.321 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T23:57:05.321 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 66873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T23:57:05.321 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 58329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T23:57:05.321 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T23:57:05.321 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 53009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/f81232.ko 2024-04-24T23:57:05.321 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 188609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T23:57:05.321 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 61489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T23:57:05.321 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 152305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T23:57:05.321 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 142833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T23:57:05.321 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 69217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T23:57:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T23:57:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T23:57:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T23:57:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 122329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T23:57:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T23:57:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T23:57:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T23:57:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T23:57:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/metro-usb.ko 2024-04-24T23:57:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T23:57:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 89969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T23:57:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 65073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mxuport.ko 2024-04-24T23:57:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T23:57:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T23:57:05.323 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T23:57:05.323 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 193857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T23:57:05.323 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 45337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T23:57:05.323 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 61761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T23:57:05.323 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T23:57:05.323 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 39985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T23:57:05.323 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T23:57:05.323 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T23:57:05.323 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 59345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T23:57:05.323 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T23:57:05.323 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T23:57:05.323 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T23:57:05.323 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 70497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T23:57:05.323 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 39321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 151137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usbserial.ko 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/wishbone-serial.ko 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 64049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T23:57:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T23:57:05.325 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T23:57:05.325 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T23:57:05.325 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T23:57:05.325 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T23:57:05.325 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T23:57:05.325 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T23:57:05.325 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T23:57:05.325 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 201193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T23:57:05.325 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/ 2024-04-24T23:57:05.325 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 76041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T23:57:05.325 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 85257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T23:57:05.325 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 122321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T23:57:05.325 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/ 2024-04-24T23:57:05.325 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/ 2024-04-24T23:57:05.326 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 206921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T23:57:05.326 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T23:57:05.326 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 134265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T23:57:05.326 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 93673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T23:57:05.326 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/ 2024-04-24T23:57:05.326 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 138721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T23:57:05.326 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T23:57:05.326 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 71281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T23:57:05.326 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 107905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T23:57:05.326 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/ 2024-04-24T23:57:05.326 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/ 2024-04-24T23:57:05.326 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-04-24T23:57:05.326 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/aat2870_bl.ko 2024-04-24T23:57:05.326 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp5520_bl.ko 2024-04-24T23:57:05.327 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8860_bl.ko 2024-04-24T23:57:05.327 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 51745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8870_bl.ko 2024-04-24T23:57:05.327 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T23:57:05.327 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/as3711_bl.ko 2024-04-24T23:57:05.327 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/bd6107.ko 2024-04-24T23:57:05.327 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da903x_bl.ko 2024-04-24T23:57:05.327 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da9052_bl.ko 2024-04-24T23:57:05.327 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/gpio_backlight.ko 2024-04-24T23:57:05.327 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/kb3886_bl.ko 2024-04-24T23:57:05.327 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T23:57:05.327 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3533_bl.ko 2024-04-24T23:57:05.327 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3639_bl.ko 2024-04-24T23:57:05.327 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lv5207lp.ko 2024-04-24T23:57:05.328 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/max8925_bl.ko 2024-04-24T23:57:05.328 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pandora_bl.ko 2024-04-24T23:57:05.328 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-04-24T23:57:05.328 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T23:57:05.328 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/wm831x_bl.ko 2024-04-24T23:57:05.328 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/ 2024-04-24T23:57:05.328 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ 2024-04-24T23:57:05.328 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T23:57:05.328 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 46801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T23:57:05.328 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/matrox_w1.ko 2024-04-24T23:57:05.328 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/w1-gpio.ko 2024-04-24T23:57:05.328 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/ 2024-04-24T23:57:05.328 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T23:57:05.328 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T23:57:05.329 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T23:57:05.329 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T23:57:05.329 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T23:57:05.329 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T23:57:05.329 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T23:57:05.329 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T23:57:05.329 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T23:57:05.329 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T23:57:05.329 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 70369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T23:57:05.329 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 129433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T23:57:05.329 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ 2024-04-24T23:57:05.329 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/acquirewdt.ko 2024-04-24T23:57:05.329 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/advantechwdt.ko 2024-04-24T23:57:05.329 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T23:57:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T23:57:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/cpu5wdt.ko 2024-04-24T23:57:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9052_wdt.ko 2024-04-24T23:57:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9055_wdt.ko 2024-04-24T23:57:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9063_wdt.ko 2024-04-24T23:57:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/dw_wdt.ko 2024-04-24T23:57:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/eurotechwdt.ko 2024-04-24T23:57:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T23:57:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T23:57:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T23:57:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T23:57:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T23:57:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T23:57:05.330 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/kempld_wdt.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/mena21_wdt.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pc87413_wdt.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 46953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 37193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/retu_wdt.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-04-24T23:57:05.331 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-04-24T23:57:05.332 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T23:57:05.332 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sc1200wdt.ko 2024-04-24T23:57:05.332 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T23:57:05.332 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-04-24T23:57:05.332 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T23:57:05.332 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T23:57:05.332 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/twl4030_wdt.ko 2024-04-24T23:57:05.332 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T23:57:05.332 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T23:57:05.332 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T23:57:05.332 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T23:57:05.332 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wafer5823wdt.ko 2024-04-24T23:57:05.332 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm831x_wdt.ko 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm8350_wdt.ko 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/ 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/ 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 213913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 85713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-scsiback.ko 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/ 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ 2024-04-24T23:57:05.333 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/ 2024-04-24T23:57:05.334 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 192617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T23:57:05.334 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/ 2024-04-24T23:57:05.334 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 78153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/adfs.ko 2024-04-24T23:57:05.334 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/ 2024-04-24T23:57:05.334 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 197849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T23:57:05.334 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/ 2024-04-24T23:57:05.334 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1261113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/kafs.ko 2024-04-24T23:57:05.334 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/ 2024-04-24T23:57:05.334 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 127377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/autofs4.ko 2024-04-24T23:57:05.334 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/ 2024-04-24T23:57:05.334 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T23:57:05.334 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/ 2024-04-24T23:57:05.334 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 50473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/bfs.ko 2024-04-24T23:57:05.334 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 45561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T23:57:05.335 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/ 2024-04-24T23:57:05.335 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 4489593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T23:57:05.335 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/ 2024-04-24T23:57:05.335 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 376857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T23:57:05.335 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ 2024-04-24T23:57:05.335 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1586065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T23:57:05.335 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/ 2024-04-24T23:57:05.335 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 130073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T23:57:05.335 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/ 2024-04-24T23:57:05.335 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 139081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/configfs.ko 2024-04-24T23:57:05.335 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/ 2024-04-24T23:57:05.335 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T23:57:05.335 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/ 2024-04-24T23:57:05.335 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 813225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/ 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 46409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/efs.ko 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/ 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1906817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/ 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/ 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/freevxfs.ko 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/ 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/ 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1486521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/ 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 183305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T23:57:05.336 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/ 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 282097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/ 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 249009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/hpfs.ko 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/ 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 93033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/ 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 533073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/ 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 361369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/ 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 112609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/ 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 591585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/ 2024-04-24T23:57:05.337 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/ 2024-04-24T23:57:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 102561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T23:57:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/ 2024-04-24T23:57:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 74353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T23:57:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/ 2024-04-24T23:57:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 147689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T23:57:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1320249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T23:57:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 82105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv2.ko 2024-04-24T23:57:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 151641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T23:57:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 2518505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T23:57:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/ 2024-04-24T23:57:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T23:57:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T23:57:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/ 2024-04-24T23:57:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 2024049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T23:57:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/ 2024-04-24T23:57:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 734025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T23:57:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/ 2024-04-24T23:57:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T23:57:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T23:57:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T23:57:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T23:57:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T23:57:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T23:57:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T23:57:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T23:57:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T23:57:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T23:57:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T23:57:05.340 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ascii.ko 2024-04-24T23:57:05.340 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T23:57:05.340 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T23:57:05.340 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T23:57:05.340 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T23:57:05.340 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T23:57:05.340 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T23:57:05.340 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T23:57:05.340 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T23:57:05.340 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T23:57:05.340 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T23:57:05.340 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T23:57:05.340 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T23:57:05.340 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T23:57:05.341 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T23:57:05.341 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T23:57:05.341 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T23:57:05.341 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T23:57:05.341 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T23:57:05.341 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 95625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T23:57:05.341 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 144209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T23:57:05.341 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 167137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T23:57:05.341 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 119513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T23:57:05.341 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T23:57:05.341 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T23:57:05.341 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T23:57:05.341 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T23:57:05.341 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T23:57:05.342 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T23:57:05.342 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T23:57:05.342 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T23:57:05.342 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T23:57:05.342 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T23:57:05.342 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T23:57:05.342 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T23:57:05.342 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T23:57:05.342 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T23:57:05.342 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T23:57:05.342 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T23:57:05.342 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T23:57:05.342 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ 2024-04-24T23:57:05.343 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ 2024-04-24T23:57:05.343 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 302073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T23:57:05.343 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ 2024-04-24T23:57:05.343 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 548169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T23:57:05.343 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ 2024-04-24T23:57:05.343 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T23:57:05.343 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 3245617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T23:57:05.343 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 22113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T23:57:05.343 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 39313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T23:57:05.343 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T23:57:05.345 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/ 2024-04-24T23:57:05.345 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 67489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/omfs.ko 2024-04-24T23:57:05.345 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/ 2024-04-24T23:57:05.345 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 462873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T23:57:05.345 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ 2024-04-24T23:57:05.345 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 67977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T23:57:05.345 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/ 2024-04-24T23:57:05.345 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/qnx4.ko 2024-04-24T23:57:05.345 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/ 2024-04-24T23:57:05.345 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 56097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/qnx6.ko 2024-04-24T23:57:05.346 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/ 2024-04-24T23:57:05.346 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_tree.ko 2024-04-24T23:57:05.346 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v1.ko 2024-04-24T23:57:05.346 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v2.ko 2024-04-24T23:57:05.346 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/ 2024-04-24T23:57:05.346 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 684841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T23:57:05.386 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/ 2024-04-24T23:57:05.386 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T23:57:05.386 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/ 2024-04-24T23:57:05.386 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/ 2024-04-24T23:57:05.386 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 3319041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T23:57:05.386 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/ 2024-04-24T23:57:05.386 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T23:57:05.386 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T23:57:05.386 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/ 2024-04-24T23:57:05.386 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 164185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T23:57:05.386 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/ 2024-04-24T23:57:05.387 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 115977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T23:57:05.387 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/ 2024-04-24T23:57:05.387 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 310017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T23:57:05.387 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ 2024-04-24T23:57:05.387 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 198233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T23:57:05.387 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/ 2024-04-24T23:57:05.387 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 5192705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T23:57:05.387 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/ 2024-04-24T23:57:05.387 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/ 2024-04-24T23:57:05.387 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 97625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/locktorture.ko 2024-04-24T23:57:05.387 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/ 2024-04-24T23:57:05.387 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/test_udelay.ko 2024-04-24T23:57:05.387 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 72665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T23:57:05.388 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ 2024-04-24T23:57:05.388 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T23:57:05.388 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc7.ko 2024-04-24T23:57:05.388 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T23:57:05.388 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/ 2024-04-24T23:57:05.388 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T23:57:05.388 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T23:57:05.388 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/interval_tree_test.ko 2024-04-24T23:57:05.388 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T23:57:05.388 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T23:57:05.388 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/ 2024-04-24T23:57:05.388 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T23:57:05.388 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T23:57:05.388 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/ 2024-04-24T23:57:05.389 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T23:57:05.389 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/memory-notifier-error-inject.ko 2024-04-24T23:57:05.389 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/notifier-error-inject.ko 2024-04-24T23:57:05.389 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/percpu_test.ko 2024-04-24T23:57:05.389 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/pm-notifier-error-inject.ko 2024-04-24T23:57:05.389 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/ 2024-04-24T23:57:05.389 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 152025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T23:57:05.389 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/rbtree_test.ko 2024-04-24T23:57:05.389 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/ 2024-04-24T23:57:05.389 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T23:57:05.389 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test-kstrtox.ko 2024-04-24T23:57:05.389 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 4912441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_bpf.ko 2024-04-24T23:57:05.389 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 61169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_firmware.ko 2024-04-24T23:57:05.389 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_module.ko 2024-04-24T23:57:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_user_copy.ko 2024-04-24T23:57:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T23:57:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T23:57:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T23:57:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/ 2024-04-24T23:57:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/xz_dec_test.ko 2024-04-24T23:57:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/ 2024-04-24T23:57:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T23:57:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ 2024-04-24T23:57:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/ 2024-04-24T23:57:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 66969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T23:57:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T23:57:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T23:57:05.391 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T23:57:05.391 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T23:57:05.391 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T23:57:05.391 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T23:57:05.391 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T23:57:05.391 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/ 2024-04-24T23:57:05.391 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T23:57:05.391 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T23:57:05.391 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T23:57:05.391 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T23:57:05.391 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T23:57:05.391 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/ 2024-04-24T23:57:05.391 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 106793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T23:57:05.391 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/ 2024-04-24T23:57:05.392 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 236793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T23:57:05.392 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T23:57:05.392 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T23:57:05.392 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 39513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T23:57:05.392 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/ 2024-04-24T23:57:05.392 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 879697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T23:57:05.392 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/ 2024-04-24T23:57:05.392 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 84761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T23:57:05.392 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1030873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T23:57:05.392 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ 2024-04-24T23:57:05.392 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T23:57:05.392 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T23:57:05.392 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T23:57:05.392 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T23:57:05.393 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T23:57:05.393 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T23:57:05.393 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T23:57:05.393 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T23:57:05.393 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T23:57:05.393 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T23:57:05.393 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T23:57:05.393 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T23:57:05.393 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T23:57:05.393 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T23:57:05.393 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T23:57:05.393 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T23:57:05.393 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T23:57:05.393 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T23:57:05.394 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T23:57:05.394 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T23:57:05.394 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 70529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T23:57:05.394 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T23:57:05.394 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T23:57:05.394 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/ 2024-04-24T23:57:05.394 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1243505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T23:57:05.394 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/ 2024-04-24T23:57:05.394 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 153801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T23:57:05.394 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/ 2024-04-24T23:57:05.394 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 244889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T23:57:05.394 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T23:57:05.394 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 71033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T23:57:05.395 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 74993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T23:57:05.395 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/ 2024-04-24T23:57:05.395 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 507705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T23:57:05.395 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T23:57:05.395 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T23:57:05.395 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T23:57:05.395 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T23:57:05.395 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/ 2024-04-24T23:57:05.395 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 128873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/hsr.ko 2024-04-24T23:57:05.395 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ 2024-04-24T23:57:05.395 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ 2024-04-24T23:57:05.395 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T23:57:05.395 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 357425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T23:57:05.395 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 59393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T23:57:05.396 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ 2024-04-24T23:57:05.396 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T23:57:05.396 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 46417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T23:57:05.396 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 60233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T23:57:05.396 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T23:57:05.396 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 65665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T23:57:05.396 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 75489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T23:57:05.396 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 77721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T23:57:05.396 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T23:57:05.396 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T23:57:05.396 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T23:57:05.396 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ 2024-04-24T23:57:05.396 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 45521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T23:57:05.396 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T23:57:05.397 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T23:57:05.397 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 50169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ip_tables.ko 2024-04-24T23:57:05.397 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T23:57:05.397 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-04-24T23:57:05.397 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T23:57:05.397 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T23:57:05.397 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T23:57:05.397 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-04-24T23:57:05.397 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T23:57:05.397 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T23:57:05.397 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T23:57:05.397 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T23:57:05.397 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T23:57:05.398 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T23:57:05.398 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T23:57:05.398 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T23:57:05.398 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T23:57:05.398 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-04-24T23:57:05.398 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T23:57:05.398 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T23:57:05.398 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T23:57:05.398 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T23:57:05.398 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T23:57:05.398 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T23:57:05.398 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T23:57:05.398 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 54441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T23:57:05.399 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 47433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 103177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_gre.ko 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 118881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 71089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/mip6.ko 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 50673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T23:57:05.400 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T23:57:05.401 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T23:57:05.401 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T23:57:05.401 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T23:57:05.401 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T23:57:05.401 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T23:57:05.401 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T23:57:05.401 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T23:57:05.401 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T23:57:05.401 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T23:57:05.401 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T23:57:05.401 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T23:57:05.401 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T23:57:05.401 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 46313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T23:57:05.401 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T23:57:05.402 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 29817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T23:57:05.402 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T23:57:05.402 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T23:57:05.402 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T23:57:05.402 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T23:57:05.402 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T23:57:05.402 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 94089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T23:57:05.402 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 30297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T23:57:05.402 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T23:57:05.402 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/ 2024-04-24T23:57:05.402 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 106369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T23:57:05.402 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/ 2024-04-24T23:57:05.402 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 153217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T23:57:05.403 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T23:57:05.403 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T23:57:05.403 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T23:57:05.403 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T23:57:05.403 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T23:57:05.403 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 66241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T23:57:05.403 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/ 2024-04-24T23:57:05.403 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 63345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/lapb.ko 2024-04-24T23:57:05.403 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/ 2024-04-24T23:57:05.403 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T23:57:05.403 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 332889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc2.ko 2024-04-24T23:57:05.403 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/ 2024-04-24T23:57:05.403 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 388873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T23:57:05.403 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/ 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 147745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 124041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 124345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 126681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 126473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 137833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 72353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 133681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T23:57:05.404 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 139073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T23:57:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 138265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T23:57:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 136177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T23:57:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 139897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T23:57:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 37129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T23:57:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ 2024-04-24T23:57:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 534113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T23:57:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T23:57:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T23:57:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T23:57:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T23:57:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T23:57:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T23:57:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T23:57:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T23:57:05.406 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T23:57:05.406 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T23:57:05.406 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T23:57:05.406 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T23:57:05.406 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T23:57:05.406 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 55217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T23:57:05.406 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 496817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T23:57:05.406 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T23:57:05.406 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T23:57:05.406 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 37865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T23:57:05.406 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 134377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T23:57:05.406 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T23:57:05.406 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T23:57:05.406 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 140801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 79873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 117201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 46105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 118369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T23:57:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 53393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 807985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 53049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 37209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 49297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 65145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 50657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T23:57:05.408 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T23:57:05.409 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T23:57:05.409 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T23:57:05.409 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T23:57:05.409 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T23:57:05.409 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T23:57:05.409 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_osf.ko 2024-04-24T23:57:05.409 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T23:57:05.409 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T23:57:05.409 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T23:57:05.409 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T23:57:05.409 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T23:57:05.409 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 24993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_synproxy.ko 2024-04-24T23:57:05.409 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tproxy.ko 2024-04-24T23:57:05.409 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 108393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/x_tables.ko 2024-04-24T23:57:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T23:57:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T23:57:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T23:57:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T23:57:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 21865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T23:57:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T23:57:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T23:57:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T23:57:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 45217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T23:57:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T23:57:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T23:57:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T23:57:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T23:57:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T23:57:05.411 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T23:57:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T23:57:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T23:57:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T23:57:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T23:57:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T23:57:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T23:57:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T23:57:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T23:57:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T23:57:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T23:57:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 52817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T23:57:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T23:57:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T23:57:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 16161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T23:57:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T23:57:05.414 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T23:57:05.414 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 11881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T23:57:05.414 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T23:57:05.414 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T23:57:05.414 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T23:57:05.414 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T23:57:05.414 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T23:57:05.414 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T23:57:05.414 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T23:57:05.414 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T23:57:05.414 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T23:57:05.414 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpudp.ko 2024-04-24T23:57:05.414 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/ 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 17233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/ 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/ 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 488825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/ 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/ 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 142025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/phonet.ko 2024-04-24T23:57:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 68817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/pn_pep.ko 2024-04-24T23:57:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/ 2024-04-24T23:57:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 318617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T23:57:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 202305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T23:57:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 85105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T23:57:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/ 2024-04-24T23:57:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1027897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/rxrpc.ko 2024-04-24T23:57:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/ 2024-04-24T23:57:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 39001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T23:57:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 28721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T23:57:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 45409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T23:57:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T23:57:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 53465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T23:57:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T23:57:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 45673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 41457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_cgroup.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 37305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 39601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 49321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 79145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 48785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T23:57:05.417 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 8377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T23:57:05.418 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T23:57:05.418 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 25097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T23:57:05.418 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T23:57:05.418 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 43153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T23:57:05.418 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 39177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq_codel.ko 2024-04-24T23:57:05.418 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T23:57:05.418 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 62817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T23:57:05.418 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T23:57:05.418 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 81833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T23:57:05.418 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T23:57:05.418 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 36625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T23:57:05.418 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T23:57:05.418 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T23:57:05.419 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 44793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T23:57:05.419 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 32833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T23:57:05.419 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 13505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T23:57:05.419 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T23:57:05.419 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 60249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T23:57:05.419 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T23:57:05.419 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 33433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T23:57:05.419 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T23:57:05.419 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 35777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T23:57:05.419 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 34161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T23:57:05.419 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/ 2024-04-24T23:57:05.419 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1187313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T23:57:05.419 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T23:57:05.419 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/ 2024-04-24T23:57:05.420 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/ 2024-04-24T23:57:05.420 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 432409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T23:57:05.420 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 70113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T23:57:05.420 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1940889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T23:57:05.420 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/ 2024-04-24T23:57:05.420 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 890729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T23:57:05.420 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/ 2024-04-24T23:57:05.420 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 1122337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T23:57:05.420 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T23:57:05.420 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/ 2024-04-24T23:57:05.420 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 18801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T23:57:05.420 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/ 2024-04-24T23:57:05.420 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 105609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T23:57:05.420 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 90265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 123769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/ 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 165841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/x25.ko 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/ 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_algo.ko 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 131865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_user.ko 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 644769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 628542 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 7706 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 23609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 9941 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T23:57:05.421 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 70990 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T23:57:05.422 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 146959 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T23:57:05.422 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 232835 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T23:57:05.422 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 210 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T23:57:05.422 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 72602 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T23:57:05.422 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 879 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T23:57:05.422 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 258450 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T23:57:05.422 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 313019 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T23:57:05.422 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/ 2024-04-24T23:57:05.422 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/ 2024-04-24T23:57:05.422 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/ 2024-04-24T23:57:05.422 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/ 2024-04-24T23:57:05.422 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 172 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/changelog.Debian.gz 2024-04-24T23:57:05.422 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r-- root/root 691 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/copyright 2024-04-24T23:57:05.427 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:05.427 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:05.427 DEBUG:teuthology.orchestra.run.smithi083:> egrep '(submenu|menuentry.*6.9.0-rc3-g806ec36b2112).*{' /boot/grub/grub.cfg 2024-04-24T23:57:05.434 INFO:teuthology.orchestra.run.smithi083.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-24T23:57:05.434 INFO:teuthology.orchestra.run.smithi083.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-24T23:57:05.434 INFO:teuthology.orchestra.run.smithi083.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-24T23:57:05.434 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-04-24T23:57:05.434 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:05.434 DEBUG:teuthology.orchestra.run.smithi083:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-04-24T23:57:05.477 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi083.front.sepia.ceph.com to complete... 2024-04-24T23:57:05.884 INFO:teuthology.orchestra.run.smithi083.stderr:Sourcing file `/etc/default/grub' 2024-04-24T23:57:05.884 INFO:teuthology.orchestra.run.smithi083.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-24T23:57:05.914 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2024-04-24T23:57:06.145 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:06.151 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-24T23:57:06.383 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-24T23:57:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-24T23:57:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-24T23:57:06.434 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-24T23:57:06.562 INFO:teuthology.orchestra.run.smithi083.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-24T23:57:06.562 INFO:teuthology.orchestra.run.smithi083.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-24T23:57:06.562 INFO:teuthology.orchestra.run.smithi083.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-24T23:57:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:done 2024-04-24T23:57:33.624 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-24T23:57:33.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-24T23:57:33.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:57:37.701 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-24T23:57:37.701 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-24T23:57:37.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:57:52.031 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-24T23:57:56.126 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-24T23:58:01.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-24T23:58:01.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:58:04.098 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-24T23:58:05.128 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-24T23:58:05.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:58:08.194 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-24T23:58:16.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-24T23:58:16.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:58:20.198 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-24T23:58:20.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:58:34.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-24T23:58:49.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-24T23:58:49.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:58:49.984 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-24T23:58:51.386 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-24T23:58:51.386 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T23:58:51.386 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-24T23:58:51.435 INFO:teuthology.orchestra.run.smithi086.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T23:58:51.435 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T23:58:51.435 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T23:58:51.436 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-24T23:58:51.683 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-24T23:58:52.437 DEBUG:teuthology.parallel:result is None 2024-04-24T23:59:06.684 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-24T23:59:06.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:59:06.812 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-24T23:59:07.400 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-24T23:59:07.400 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T23:59:07.400 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-04-24T23:59:07.451 INFO:teuthology.orchestra.run.smithi083.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T23:59:07.451 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T23:59:07.451 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T23:59:07.451 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-24T23:59:08.452 DEBUG:teuthology.parallel:result is None 2024-04-24T23:59:08.453 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T23:59:08.465 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T23:59:08.465 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:59:08.468 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:59:08.477 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T23:59:08.484 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T23:59:08.492 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T23:59:08.493 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:59:08.519 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:59:08.544 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T23:59:08.552 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T23:59:08.552 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:59:08.583 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:59:08.670 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:59:08.693 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:59:08.699 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:59:08.723 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:59:08.726 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T23:59:08.733 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T23:59:08.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:59:08.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:59:08.758 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T23:59:08.780 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T23:59:08.780 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:59:08.799 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:59:08.807 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:59:08.856 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:59:08.908 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T23:59:08.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:59:08.983 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:59:08.994 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:59:09.043 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-24T23:59:09.043 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:59:09.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-04-24T23:59:09.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-04-24T23:59:09.253 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T23:59:09.261 INFO:teuthology.task.internal:Starting timer... 2024-04-24T23:59:09.261 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T23:59:09.271 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T23:59:09.279 DEBUG:teuthology.task.selinux:Excluding smithi083: OS 'ubuntu' does not support SELinux 2024-04-24T23:59:09.279 DEBUG:teuthology.task.selinux:Excluding smithi086: OS 'ubuntu' does not support SELinux 2024-04-24T23:59:09.279 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T23:59:09.280 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-24T23:59:09.280 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T23:59:09.280 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T23:59:09.288 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T23:59:09.305 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T23:59:09.307 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi083.front.sepia.ceph.com,smithi086.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T00:06:50.873 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi086.front.sepia.ceph.com')] 2024-04-25T00:06:50.875 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-25T00:06:50.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:06:50.963 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-25T00:06:51.090 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-25T00:06:51.090 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T00:06:51.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:06:51.187 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-25T00:06:51.317 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T00:06:51.318 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T00:06:51.392 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T00:06:51.393 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:06:51.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:06:51.397 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:06:51.397 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:06:51.434 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T00:06:51.435 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: Command line: ntpd -gq 2024-04-25T00:06:51.435 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: ---------------------------------------------------- 2024-04-25T00:06:51.435 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T00:06:51.435 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T00:06:51.435 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: corporation. Support and training for ntp-4 are 2024-04-25T00:06:51.435 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: available at https://www.nwtime.org/support 2024-04-25T00:06:51.435 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: ---------------------------------------------------- 2024-04-25T00:06:51.436 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: proto: precision = 0.042 usec (-24) 2024-04-25T00:06:51.436 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: basedate set to 2022-02-04 2024-04-25T00:06:51.436 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: gps base set to 2022-02-06 (week 2196) 2024-04-25T00:06:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:06:51.437 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:06:51.437 INFO:teuthology.orchestra.run.smithi083.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:06:51.438 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:06:51.438 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T00:06:51.438 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T00:06:51.438 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T00:06:51.438 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: Listen normally on 3 enp3s0f1 172.21.15.83:123 2024-04-25T00:06:51.438 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: Listen normally on 4 lo [::1]:123 2024-04-25T00:06:51.438 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce91%5]:123 2024-04-25T00:06:51.438 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:51 ntpd[17329]: Listening on routing socket on fd #22 for interface updates 2024-04-25T00:06:51.439 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T00:06:51.439 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: Command line: ntpd -gq 2024-04-25T00:06:51.440 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: ---------------------------------------------------- 2024-04-25T00:06:51.440 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T00:06:51.440 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T00:06:51.440 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: corporation. Support and training for ntp-4 are 2024-04-25T00:06:51.440 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: available at https://www.nwtime.org/support 2024-04-25T00:06:51.440 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: ---------------------------------------------------- 2024-04-25T00:06:51.440 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: proto: precision = 0.044 usec (-24) 2024-04-25T00:06:51.440 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: basedate set to 2022-02-04 2024-04-25T00:06:51.440 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: gps base set to 2022-02-06 (week 2196) 2024-04-25T00:06:51.440 INFO:teuthology.orchestra.run.smithi086.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:06:51.441 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:06:51.441 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:06:51.441 INFO:teuthology.orchestra.run.smithi086.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:06:51.442 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T00:06:51.442 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T00:06:51.442 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T00:06:51.442 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: Listen normally on 3 enp3s0f1 172.21.15.86:123 2024-04-25T00:06:51.442 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: Listen normally on 4 lo [::1]:123 2024-04-25T00:06:51.442 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d809%5]:123 2024-04-25T00:06:51.442 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:51 ntpd[17342]: Listening on routing socket on fd #22 for interface updates 2024-04-25T00:06:52.435 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:52 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:52.435 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:52 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:52.435 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:52 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:52.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:52 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:52.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:52 ntpd[17329]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:52.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:52 ntpd[17329]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:52.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:52 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:52.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:52 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:52.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:52 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:52.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:52 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:52.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:52 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:52.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:52 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:52.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:52 ntpd[17342]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:52.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:52 ntpd[17342]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:53.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:53 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:53.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:53 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:53.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:53 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:53.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:53 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:53.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:53 ntpd[17342]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:53.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:53 ntpd[17342]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:54 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:54 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:54 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:54 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:54 ntpd[17329]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:54 ntpd[17329]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:54.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:54 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:54.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:54 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:54.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:54 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:54.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:54 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:54.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:54 ntpd[17342]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:54.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:54 ntpd[17342]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:55.435 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:55 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:55.435 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:55 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:55.435 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:55 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:55 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:55 ntpd[17329]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:55 ntpd[17329]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:55.437 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:55 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:55.437 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:55 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:55.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:55 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:55.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:55 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:55.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:55 ntpd[17342]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:55.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:55 ntpd[17342]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:56.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:56 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:56.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:56 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:56.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:56 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:56.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:56 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:56.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:56 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:56.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:56 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:56.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:56 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:56.437 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:56 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:56.437 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:56 ntpd[17329]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:56.437 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:56 ntpd[17329]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:56.437 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:56 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:56.437 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:56 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:56.437 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:56 ntpd[17329]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:56.437 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:56 ntpd[17329]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:56 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:56 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:56 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:56 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:56 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:56 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:56 ntpd[17342]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:56 ntpd[17342]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:56 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:56 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:56 ntpd[17342]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:56 ntpd[17342]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:57.435 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:57 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:57.435 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:57 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:57.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:57 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:57.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:57 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:57.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:57 ntpd[17329]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:57.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:57 ntpd[17329]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:57.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:57 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:57.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:57 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:57.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:57 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:57.438 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:57 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:57.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:57 ntpd[17342]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:57.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:57 ntpd[17342]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:58 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:58 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:58 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:58 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:58 ntpd[17329]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:58 ntpd[17329]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:58 ntpd[17329]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:58 ntpd[17329]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:58.436 INFO:teuthology.orchestra.run.smithi083.stdout:25 Apr 00:06:58 ntpd[17329]: ntpd: time slew -0.001156 s 2024-04-25T00:06:58.437 INFO:teuthology.orchestra.run.smithi083.stdout:ntpd: time slew -0.001156s 2024-04-25T00:06:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:58 ntpd[17329]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T00:06:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:25 Apr 00:06:58 ntpd[17329]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T00:06:58.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:58 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:58.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:58 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:58.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:58 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:58.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:58 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:58.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:58 ntpd[17342]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:06:58.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:58 ntpd[17342]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:06:58.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:58 ntpd[17342]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:06:58.439 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:58 ntpd[17342]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:06:58.440 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 00:06:58 ntpd[17342]: ntpd: time slew -0.000898 s 2024-04-25T00:06:58.440 INFO:teuthology.orchestra.run.smithi086.stdout:ntpd: time slew -0.000898s 2024-04-25T00:06:58.440 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:58 ntpd[17342]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T00:06:58.440 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 00:06:58 ntpd[17342]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T00:06:58.533 INFO:teuthology.orchestra.run.smithi086.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:06:58.533 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================== 2024-04-25T00:06:58.533 INFO:teuthology.orchestra.run.smithi086.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:06:58.533 INFO:teuthology.orchestra.run.smithi086.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:06:58.533 INFO:teuthology.orchestra.run.smithi086.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:06:58.533 INFO:teuthology.orchestra.run.smithi086.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:06:58.545 INFO:teuthology.orchestra.run.smithi083.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:06:58.545 INFO:teuthology.orchestra.run.smithi083.stdout:============================================================================== 2024-04-25T00:06:58.545 INFO:teuthology.orchestra.run.smithi083.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:06:58.545 INFO:teuthology.orchestra.run.smithi083.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:06:58.545 INFO:teuthology.orchestra.run.smithi083.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:06:58.545 INFO:teuthology.orchestra.run.smithi083.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:06:58.547 INFO:teuthology.run_tasks:Running task install... 2024-04-25T00:06:58.676 DEBUG:teuthology.task.install:project ceph 2024-04-25T00:06:58.676 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T00:06:58.676 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T00:06:58.676 INFO:teuthology.task.install:Using flavor: default 2024-04-25T00:06:58.684 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T00:06:58.684 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T00:06:58.684 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-key list | grep Ceph 2024-04-25T00:06:58.685 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-key list | grep Ceph 2024-04-25T00:06:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T00:06:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T00:06:58.838 INFO:teuthology.orchestra.run.smithi083.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T00:06:58.839 INFO:teuthology.orchestra.run.smithi083.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T00:06:58.840 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T00:06:58.840 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T00:06:58.840 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:06:58.847 INFO:teuthology.orchestra.run.smithi086.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T00:06:58.847 INFO:teuthology.orchestra.run.smithi086.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T00:06:58.847 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T00:06:58.848 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T00:06:58.848 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:06:58.994 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default/ 2024-04-25T00:06:58.995 INFO:teuthology.task.install.deb:Package version is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:06:59.062 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default/ 2024-04-25T00:06:59.062 INFO:teuthology.task.install.deb:Package version is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:06:59.101 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:06:59.101 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T00:06:59.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get update 2024-04-25T00:06:59.173 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:06:59.173 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T00:06:59.193 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get update 2024-04-25T00:06:59.360 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:06:59.360 INFO:teuthology.orchestra.run.smithi083.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T00:06:59.383 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:06:59.390 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T00:06:59.412 INFO:teuthology.orchestra.run.smithi083.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T00:06:59.420 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:06:59.434 INFO:teuthology.orchestra.run.smithi086.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T00:06:59.435 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T00:06:59.455 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T00:06:59.469 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-04-25T00:06:59.478 INFO:teuthology.orchestra.run.smithi086.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T00:06:59.495 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:06:59.501 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-04-25T00:06:59.503 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:06:59.533 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T00:06:59.577 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:06:59.585 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:06:59.658 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:06:59.946 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 28.9 kB in 1s (47.8 kB/s) 2024-04-25T00:06:59.970 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 28.9 kB in 1s (52.1 kB/s) 2024-04-25T00:07:00.928 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-25T00:07:00.943 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2445-gd2c45d97-1jammy cephadm=19.0.0-2445-gd2c45d97-1jammy ceph-mds=19.0.0-2445-gd2c45d97-1jammy ceph-mgr=19.0.0-2445-gd2c45d97-1jammy ceph-common=19.0.0-2445-gd2c45d97-1jammy ceph-fuse=19.0.0-2445-gd2c45d97-1jammy ceph-test=19.0.0-2445-gd2c45d97-1jammy ceph-volume=19.0.0-2445-gd2c45d97-1jammy radosgw=19.0.0-2445-gd2c45d97-1jammy python3-rados=19.0.0-2445-gd2c45d97-1jammy python3-rgw=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy python3-rbd=19.0.0-2445-gd2c45d97-1jammy libcephfs2=19.0.0-2445-gd2c45d97-1jammy libcephfs-dev=19.0.0-2445-gd2c45d97-1jammy librados2=19.0.0-2445-gd2c45d97-1jammy librbd1=19.0.0-2445-gd2c45d97-1jammy rbd-fuse=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy cephfs-shell=19.0.0-2445-gd2c45d97-1jammy cephfs-top=19.0.0-2445-gd2c45d97-1jammy cephfs-mirror=19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:07:00.951 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-04-25T00:07:00.966 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2445-gd2c45d97-1jammy cephadm=19.0.0-2445-gd2c45d97-1jammy ceph-mds=19.0.0-2445-gd2c45d97-1jammy ceph-mgr=19.0.0-2445-gd2c45d97-1jammy ceph-common=19.0.0-2445-gd2c45d97-1jammy ceph-fuse=19.0.0-2445-gd2c45d97-1jammy ceph-test=19.0.0-2445-gd2c45d97-1jammy ceph-volume=19.0.0-2445-gd2c45d97-1jammy radosgw=19.0.0-2445-gd2c45d97-1jammy python3-rados=19.0.0-2445-gd2c45d97-1jammy python3-rgw=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy python3-rbd=19.0.0-2445-gd2c45d97-1jammy libcephfs2=19.0.0-2445-gd2c45d97-1jammy libcephfs-dev=19.0.0-2445-gd2c45d97-1jammy librados2=19.0.0-2445-gd2c45d97-1jammy librbd1=19.0.0-2445-gd2c45d97-1jammy rbd-fuse=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy cephfs-shell=19.0.0-2445-gd2c45d97-1jammy cephfs-top=19.0.0-2445-gd2c45d97-1jammy cephfs-mirror=19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:07:01.017 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-25T00:07:01.039 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-04-25T00:07:01.190 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-04-25T00:07:01.190 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-04-25T00:07:01.214 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2024-04-25T00:07:01.214 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2024-04-25T00:07:01.365 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:07:01.365 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:07:01.365 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:07:01.365 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:07:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:07:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:07:01.366 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:07:01.367 INFO:teuthology.orchestra.run.smithi083.stdout:The following additional packages will be installed: 2024-04-25T00:07:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T00:07:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T00:07:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:07:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T00:07:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T00:07:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T00:07:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T00:07:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T00:07:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T00:07:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T00:07:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T00:07:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T00:07:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T00:07:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T00:07:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T00:07:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T00:07:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T00:07:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T00:07:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T00:07:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T00:07:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T00:07:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T00:07:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T00:07:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T00:07:01.370 INFO:teuthology.orchestra.run.smithi083.stdout:Suggested packages: 2024-04-25T00:07:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T00:07:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T00:07:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T00:07:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T00:07:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T00:07:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T00:07:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T00:07:01.371 INFO:teuthology.orchestra.run.smithi083.stdout:Recommended packages: 2024-04-25T00:07:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: btrfs-tools 2024-04-25T00:07:01.388 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:07:01.388 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:07:01.388 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:07:01.388 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:07:01.389 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:07:01.389 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:07:01.389 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:07:01.390 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2024-04-25T00:07:01.390 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T00:07:01.390 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T00:07:01.391 INFO:teuthology.orchestra.run.smithi086.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:07:01.391 INFO:teuthology.orchestra.run.smithi086.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T00:07:01.391 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T00:07:01.392 INFO:teuthology.orchestra.run.smithi086.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T00:07:01.392 INFO:teuthology.orchestra.run.smithi086.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T00:07:01.392 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T00:07:01.392 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T00:07:01.392 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T00:07:01.392 INFO:teuthology.orchestra.run.smithi086.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T00:07:01.392 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T00:07:01.392 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T00:07:01.392 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T00:07:01.392 INFO:teuthology.orchestra.run.smithi086.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T00:07:01.392 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T00:07:01.392 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T00:07:01.392 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T00:07:01.392 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T00:07:01.393 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T00:07:01.393 INFO:teuthology.orchestra.run.smithi086.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T00:07:01.393 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T00:07:01.393 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T00:07:01.393 INFO:teuthology.orchestra.run.smithi086.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T00:07:01.393 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2024-04-25T00:07:01.393 INFO:teuthology.orchestra.run.smithi086.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T00:07:01.393 INFO:teuthology.orchestra.run.smithi086.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T00:07:01.393 INFO:teuthology.orchestra.run.smithi086.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T00:07:01.393 INFO:teuthology.orchestra.run.smithi086.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T00:07:01.394 INFO:teuthology.orchestra.run.smithi086.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T00:07:01.394 INFO:teuthology.orchestra.run.smithi086.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T00:07:01.394 INFO:teuthology.orchestra.run.smithi086.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T00:07:01.394 INFO:teuthology.orchestra.run.smithi086.stdout:Recommended packages: 2024-04-25T00:07:01.394 INFO:teuthology.orchestra.run.smithi086.stdout: btrfs-tools 2024-04-25T00:07:01.521 INFO:teuthology.orchestra.run.smithi083.stdout:The following NEW packages will be installed: 2024-04-25T00:07:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T00:07:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T00:07:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T00:07:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T00:07:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:07:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T00:07:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T00:07:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T00:07:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T00:07:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T00:07:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T00:07:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T00:07:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T00:07:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T00:07:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T00:07:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T00:07:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T00:07:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T00:07:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T00:07:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T00:07:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T00:07:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T00:07:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T00:07:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T00:07:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T00:07:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T00:07:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T00:07:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: socat unzip xmlstarlet zip 2024-04-25T00:07:01.544 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2024-04-25T00:07:01.544 INFO:teuthology.orchestra.run.smithi086.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T00:07:01.544 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T00:07:01.544 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T00:07:01.544 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T00:07:01.544 INFO:teuthology.orchestra.run.smithi086.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:07:01.544 INFO:teuthology.orchestra.run.smithi086.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T00:07:01.544 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T00:07:01.545 INFO:teuthology.orchestra.run.smithi086.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T00:07:01.545 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T00:07:01.545 INFO:teuthology.orchestra.run.smithi086.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T00:07:01.545 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T00:07:01.545 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T00:07:01.545 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T00:07:01.545 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T00:07:01.545 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T00:07:01.545 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T00:07:01.545 INFO:teuthology.orchestra.run.smithi086.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T00:07:01.545 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T00:07:01.545 INFO:teuthology.orchestra.run.smithi086.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T00:07:01.545 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T00:07:01.545 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T00:07:01.546 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T00:07:01.546 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T00:07:01.546 INFO:teuthology.orchestra.run.smithi086.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T00:07:01.546 INFO:teuthology.orchestra.run.smithi086.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T00:07:01.546 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T00:07:01.546 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T00:07:01.546 INFO:teuthology.orchestra.run.smithi086.stdout: socat unzip xmlstarlet zip 2024-04-25T00:07:01.693 INFO:teuthology.orchestra.run.smithi086.stdout:0 upgraded, 124 newly installed, 0 to remove and 305 not upgraded. 2024-04-25T00:07:01.694 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 180 MB of archives. 2024-04-25T00:07:01.694 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 823 MB of additional disk space will be used. 2024-04-25T00:07:01.694 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2445-gd2c45d97-1jammy [3,540 kB] 2024-04-25T00:07:01.718 INFO:teuthology.orchestra.run.smithi083.stdout:0 upgraded, 124 newly installed, 0 to remove and 305 not upgraded. 2024-04-25T00:07:01.718 INFO:teuthology.orchestra.run.smithi083.stdout:Need to get 180 MB of archives. 2024-04-25T00:07:01.718 INFO:teuthology.orchestra.run.smithi083.stdout:After this operation, 823 MB of additional disk space will be used. 2024-04-25T00:07:01.718 INFO:teuthology.orchestra.run.smithi083.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T00:07:01.740 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T00:07:01.801 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T00:07:01.824 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T00:07:01.848 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T00:07:01.868 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T00:07:02.049 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T00:07:02.076 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T00:07:02.144 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T00:07:02.158 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T00:07:02.176 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T00:07:02.189 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T00:07:02.209 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T00:07:02.242 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T00:07:02.385 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T00:07:02.395 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T00:07:02.417 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T00:07:02.424 INFO:teuthology.orchestra.run.smithi083.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T00:07:02.426 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T00:07:02.432 INFO:teuthology.orchestra.run.smithi083.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T00:07:02.454 INFO:teuthology.orchestra.run.smithi086.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T00:07:02.462 INFO:teuthology.orchestra.run.smithi086.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T00:07:02.632 INFO:teuthology.orchestra.run.smithi083.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T00:07:02.648 INFO:teuthology.orchestra.run.smithi086.stdout:Get:13 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2445-gd2c45d97-1jammy [3,218 kB] 2024-04-25T00:07:02.659 INFO:teuthology.orchestra.run.smithi083.stdout:Get:13 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2445-gd2c45d97-1jammy [3,540 kB] 2024-04-25T00:07:02.665 INFO:teuthology.orchestra.run.smithi086.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T00:07:02.881 INFO:teuthology.orchestra.run.smithi086.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2445-gd2c45d97-1jammy [711 kB] 2024-04-25T00:07:02.923 INFO:teuthology.orchestra.run.smithi086.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2445-gd2c45d97-1jammy [356 kB] 2024-04-25T00:07:02.955 INFO:teuthology.orchestra.run.smithi086.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2445-gd2c45d97-1jammy [32.8 kB] 2024-04-25T00:07:02.960 INFO:teuthology.orchestra.run.smithi086.stdout:Get:18 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2445-gd2c45d97-1jammy [180 kB] 2024-04-25T00:07:02.985 INFO:teuthology.orchestra.run.smithi086.stdout:Get:19 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2445-gd2c45d97-1jammy [65.7 kB] 2024-04-25T00:07:02.988 INFO:teuthology.orchestra.run.smithi086.stdout:Get:20 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2445-gd2c45d97-1jammy [330 kB] 2024-04-25T00:07:03.068 INFO:teuthology.orchestra.run.smithi086.stdout:Get:21 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2445-gd2c45d97-1jammy [6,775 kB] 2024-04-25T00:07:03.220 INFO:teuthology.orchestra.run.smithi083.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T00:07:03.235 INFO:teuthology.orchestra.run.smithi083.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T00:07:03.251 INFO:teuthology.orchestra.run.smithi083.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T00:07:03.252 INFO:teuthology.orchestra.run.smithi086.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T00:07:03.260 INFO:teuthology.orchestra.run.smithi083.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T00:07:03.261 INFO:teuthology.orchestra.run.smithi083.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T00:07:03.266 INFO:teuthology.orchestra.run.smithi086.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T00:07:03.277 INFO:teuthology.orchestra.run.smithi083.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T00:07:03.281 INFO:teuthology.orchestra.run.smithi086.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T00:07:03.290 INFO:teuthology.orchestra.run.smithi086.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T00:07:03.291 INFO:teuthology.orchestra.run.smithi086.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T00:07:03.307 INFO:teuthology.orchestra.run.smithi086.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T00:07:03.314 INFO:teuthology.orchestra.run.smithi083.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T00:07:03.330 INFO:teuthology.orchestra.run.smithi083.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T00:07:03.331 INFO:teuthology.orchestra.run.smithi083.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T00:07:03.332 INFO:teuthology.orchestra.run.smithi083.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T00:07:03.341 INFO:teuthology.orchestra.run.smithi086.stdout:Get:28 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2445-gd2c45d97-1jammy [112 kB] 2024-04-25T00:07:03.343 INFO:teuthology.orchestra.run.smithi086.stdout:Get:29 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2445-gd2c45d97-1jammy [466 kB] 2024-04-25T00:07:03.345 INFO:teuthology.orchestra.run.smithi086.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T00:07:03.359 INFO:teuthology.orchestra.run.smithi086.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T00:07:03.363 INFO:teuthology.orchestra.run.smithi086.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T00:07:03.363 INFO:teuthology.orchestra.run.smithi086.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T00:07:03.369 INFO:teuthology.orchestra.run.smithi086.stdout:Get:34 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2445-gd2c45d97-1jammy [25.6 MB] 2024-04-25T00:07:03.504 INFO:teuthology.orchestra.run.smithi083.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T00:07:03.534 INFO:teuthology.orchestra.run.smithi086.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T00:07:03.589 INFO:teuthology.orchestra.run.smithi083.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T00:07:03.595 INFO:teuthology.orchestra.run.smithi083.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T00:07:03.616 INFO:teuthology.orchestra.run.smithi086.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T00:07:03.636 INFO:teuthology.orchestra.run.smithi086.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T00:07:03.671 INFO:teuthology.orchestra.run.smithi083.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T00:07:03.698 INFO:teuthology.orchestra.run.smithi086.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T00:07:03.699 INFO:teuthology.orchestra.run.smithi083.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T00:07:03.713 INFO:teuthology.orchestra.run.smithi083.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T00:07:03.740 INFO:teuthology.orchestra.run.smithi086.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T00:07:03.752 INFO:teuthology.orchestra.run.smithi083.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T00:07:03.760 INFO:teuthology.orchestra.run.smithi086.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T00:07:03.789 INFO:teuthology.orchestra.run.smithi086.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T00:07:03.936 INFO:teuthology.orchestra.run.smithi083.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T00:07:03.952 INFO:teuthology.orchestra.run.smithi083.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T00:07:03.959 INFO:teuthology.orchestra.run.smithi083.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T00:07:03.964 INFO:teuthology.orchestra.run.smithi083.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T00:07:03.975 INFO:teuthology.orchestra.run.smithi086.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T00:07:03.990 INFO:teuthology.orchestra.run.smithi086.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T00:07:03.997 INFO:teuthology.orchestra.run.smithi086.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T00:07:04.002 INFO:teuthology.orchestra.run.smithi086.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T00:07:04.156 INFO:teuthology.orchestra.run.smithi083.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T00:07:04.187 INFO:teuthology.orchestra.run.smithi086.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T00:07:04.247 INFO:teuthology.orchestra.run.smithi083.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T00:07:04.262 INFO:teuthology.orchestra.run.smithi083.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T00:07:04.274 INFO:teuthology.orchestra.run.smithi086.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T00:07:04.283 INFO:teuthology.orchestra.run.smithi086.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2445-gd2c45d97-1jammy [5,084 kB] 2024-04-25T00:07:04.292 INFO:teuthology.orchestra.run.smithi086.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T00:07:04.368 INFO:teuthology.orchestra.run.smithi083.stdout:Get:38 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2445-gd2c45d97-1jammy [3,218 kB] 2024-04-25T00:07:04.448 INFO:teuthology.orchestra.run.smithi083.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T00:07:04.456 INFO:teuthology.orchestra.run.smithi086.stdout:Get:50 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2445-gd2c45d97-1jammy [244 kB] 2024-04-25T00:07:04.462 INFO:teuthology.orchestra.run.smithi086.stdout:Get:51 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2445-gd2c45d97-1jammy [124 kB] 2024-04-25T00:07:04.467 INFO:teuthology.orchestra.run.smithi086.stdout:Get:52 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2445-gd2c45d97-1jammy [1,525 kB] 2024-04-25T00:07:04.475 INFO:teuthology.orchestra.run.smithi086.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T00:07:04.498 INFO:teuthology.orchestra.run.smithi083.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T00:07:04.511 INFO:teuthology.orchestra.run.smithi083.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T00:07:04.527 INFO:teuthology.orchestra.run.smithi086.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T00:07:04.540 INFO:teuthology.orchestra.run.smithi086.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T00:07:04.597 INFO:teuthology.orchestra.run.smithi086.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2445-gd2c45d97-1jammy [6,245 kB] 2024-04-25T00:07:04.669 INFO:teuthology.orchestra.run.smithi083.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T00:07:04.669 INFO:teuthology.orchestra.run.smithi083.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T00:07:04.674 INFO:teuthology.orchestra.run.smithi083.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T00:07:04.680 INFO:teuthology.orchestra.run.smithi083.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T00:07:04.694 INFO:teuthology.orchestra.run.smithi083.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T00:07:04.696 INFO:teuthology.orchestra.run.smithi086.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T00:07:04.697 INFO:teuthology.orchestra.run.smithi086.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T00:07:04.701 INFO:teuthology.orchestra.run.smithi086.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T00:07:04.706 INFO:teuthology.orchestra.run.smithi086.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T00:07:04.720 INFO:teuthology.orchestra.run.smithi086.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T00:07:04.760 INFO:teuthology.orchestra.run.smithi086.stdout:Get:62 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2445-gd2c45d97-1jammy [22.7 MB] 2024-04-25T00:07:04.881 INFO:teuthology.orchestra.run.smithi083.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T00:07:04.905 INFO:teuthology.orchestra.run.smithi086.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T00:07:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T00:07:05.168 INFO:teuthology.orchestra.run.smithi086.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T00:07:05.281 INFO:teuthology.orchestra.run.smithi083.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T00:07:05.295 INFO:teuthology.orchestra.run.smithi086.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T00:07:05.322 INFO:teuthology.orchestra.run.smithi083.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T00:07:05.339 INFO:teuthology.orchestra.run.smithi086.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T00:07:05.412 INFO:teuthology.orchestra.run.smithi083.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T00:07:05.422 INFO:teuthology.orchestra.run.smithi083.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T00:07:05.427 INFO:teuthology.orchestra.run.smithi086.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T00:07:05.435 INFO:teuthology.orchestra.run.smithi083.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2445-gd2c45d97-1jammy [711 kB] 2024-04-25T00:07:05.435 INFO:teuthology.orchestra.run.smithi083.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T00:07:05.437 INFO:teuthology.orchestra.run.smithi086.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T00:07:05.450 INFO:teuthology.orchestra.run.smithi086.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T00:07:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T00:07:05.491 INFO:teuthology.orchestra.run.smithi086.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T00:07:05.594 INFO:teuthology.orchestra.run.smithi086.stdout:Get:71 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2445-gd2c45d97-1jammy [14.0 kB] 2024-04-25T00:07:05.594 INFO:teuthology.orchestra.run.smithi086.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [869 kB] 2024-04-25T00:07:05.619 INFO:teuthology.orchestra.run.smithi086.stdout:Get:73 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2445-gd2c45d97-1jammy [2,437 kB] 2024-04-25T00:07:05.671 INFO:teuthology.orchestra.run.smithi083.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2445-gd2c45d97-1jammy [356 kB] 2024-04-25T00:07:05.707 INFO:teuthology.orchestra.run.smithi086.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2445-gd2c45d97-1jammy [790 kB] 2024-04-25T00:07:05.735 INFO:teuthology.orchestra.run.smithi086.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2445-gd2c45d97-1jammy [147 kB] 2024-04-25T00:07:05.740 INFO:teuthology.orchestra.run.smithi086.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2445-gd2c45d97-1jammy [2,293 kB] 2024-04-25T00:07:05.748 INFO:teuthology.orchestra.run.smithi083.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T00:07:05.749 INFO:teuthology.orchestra.run.smithi083.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T00:07:05.750 INFO:teuthology.orchestra.run.smithi083.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T00:07:05.753 INFO:teuthology.orchestra.run.smithi086.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T00:07:05.754 INFO:teuthology.orchestra.run.smithi086.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T00:07:05.755 INFO:teuthology.orchestra.run.smithi086.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T00:07:05.782 INFO:teuthology.orchestra.run.smithi083.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2445-gd2c45d97-1jammy [32.8 kB] 2024-04-25T00:07:05.794 INFO:teuthology.orchestra.run.smithi083.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2445-gd2c45d97-1jammy [180 kB] 2024-04-25T00:07:05.826 INFO:teuthology.orchestra.run.smithi086.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2445-gd2c45d97-1jammy [8,625 kB] 2024-04-25T00:07:05.853 INFO:teuthology.orchestra.run.smithi083.stdout:Get:62 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2445-gd2c45d97-1jammy [65.7 kB] 2024-04-25T00:07:05.882 INFO:teuthology.orchestra.run.smithi083.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2445-gd2c45d97-1jammy [330 kB] 2024-04-25T00:07:05.915 INFO:teuthology.orchestra.run.smithi083.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T00:07:05.920 INFO:teuthology.orchestra.run.smithi086.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T00:07:05.989 INFO:teuthology.orchestra.run.smithi083.stdout:Get:65 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2445-gd2c45d97-1jammy [6,775 kB] 2024-04-25T00:07:06.143 INFO:teuthology.orchestra.run.smithi086.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2445-gd2c45d97-1jammy [14.2 kB] 2024-04-25T00:07:06.143 INFO:teuthology.orchestra.run.smithi086.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2445-gd2c45d97-1jammy [50.7 MB] 2024-04-25T00:07:06.261 INFO:teuthology.orchestra.run.smithi083.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T00:07:06.267 INFO:teuthology.orchestra.run.smithi086.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T00:07:06.665 INFO:teuthology.orchestra.run.smithi083.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T00:07:06.670 INFO:teuthology.orchestra.run.smithi086.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T00:07:06.675 INFO:teuthology.orchestra.run.smithi083.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T00:07:06.676 INFO:teuthology.orchestra.run.smithi083.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T00:07:06.680 INFO:teuthology.orchestra.run.smithi086.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T00:07:06.681 INFO:teuthology.orchestra.run.smithi086.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T00:07:06.746 INFO:teuthology.orchestra.run.smithi083.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T00:07:06.746 INFO:teuthology.orchestra.run.smithi083.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T00:07:06.747 INFO:teuthology.orchestra.run.smithi083.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T00:07:06.747 INFO:teuthology.orchestra.run.smithi083.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T00:07:06.748 INFO:teuthology.orchestra.run.smithi083.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T00:07:06.749 INFO:teuthology.orchestra.run.smithi083.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T00:07:06.750 INFO:teuthology.orchestra.run.smithi086.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T00:07:06.751 INFO:teuthology.orchestra.run.smithi086.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T00:07:06.752 INFO:teuthology.orchestra.run.smithi086.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T00:07:06.753 INFO:teuthology.orchestra.run.smithi086.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T00:07:06.754 INFO:teuthology.orchestra.run.smithi086.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T00:07:06.754 INFO:teuthology.orchestra.run.smithi086.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T00:07:06.914 INFO:teuthology.orchestra.run.smithi083.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T00:07:06.919 INFO:teuthology.orchestra.run.smithi086.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T00:07:07.123 INFO:teuthology.orchestra.run.smithi083.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T00:07:07.131 INFO:teuthology.orchestra.run.smithi086.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T00:07:07.451 INFO:teuthology.orchestra.run.smithi083.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T00:07:07.462 INFO:teuthology.orchestra.run.smithi086.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T00:07:07.498 INFO:teuthology.orchestra.run.smithi083.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T00:07:07.510 INFO:teuthology.orchestra.run.smithi086.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T00:07:07.525 INFO:teuthology.orchestra.run.smithi083.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T00:07:07.533 INFO:teuthology.orchestra.run.smithi083.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T00:07:07.536 INFO:teuthology.orchestra.run.smithi086.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T00:07:07.547 INFO:teuthology.orchestra.run.smithi086.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T00:07:07.578 INFO:teuthology.orchestra.run.smithi083.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T00:07:07.590 INFO:teuthology.orchestra.run.smithi086.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T00:07:07.604 INFO:teuthology.orchestra.run.smithi083.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T00:07:07.607 INFO:teuthology.orchestra.run.smithi083.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T00:07:07.613 INFO:teuthology.orchestra.run.smithi083.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T00:07:07.616 INFO:teuthology.orchestra.run.smithi086.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T00:07:07.617 INFO:teuthology.orchestra.run.smithi083.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T00:07:07.618 INFO:teuthology.orchestra.run.smithi086.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T00:07:07.625 INFO:teuthology.orchestra.run.smithi086.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T00:07:07.628 INFO:teuthology.orchestra.run.smithi086.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T00:07:07.815 INFO:teuthology.orchestra.run.smithi083.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T00:07:07.826 INFO:teuthology.orchestra.run.smithi086.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T00:07:08.029 INFO:teuthology.orchestra.run.smithi086.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2445-gd2c45d97-1jammy [121 kB] 2024-04-25T00:07:08.030 INFO:teuthology.orchestra.run.smithi086.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2445-gd2c45d97-1jammy [229 kB] 2024-04-25T00:07:08.032 INFO:teuthology.orchestra.run.smithi086.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2445-gd2c45d97-1jammy [34.0 kB] 2024-04-25T00:07:08.033 INFO:teuthology.orchestra.run.smithi086.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2445-gd2c45d97-1jammy [23.4 kB] 2024-04-25T00:07:08.033 INFO:teuthology.orchestra.run.smithi086.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2445-gd2c45d97-1jammy [38.9 kB] 2024-04-25T00:07:08.034 INFO:teuthology.orchestra.run.smithi086.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2445-gd2c45d97-1jammy [13.4 MB] 2024-04-25T00:07:08.074 INFO:teuthology.orchestra.run.smithi083.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T00:07:08.086 INFO:teuthology.orchestra.run.smithi086.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T00:07:08.167 INFO:teuthology.orchestra.run.smithi083.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2445-gd2c45d97-1jammy [112 kB] 2024-04-25T00:07:08.216 INFO:teuthology.orchestra.run.smithi083.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2445-gd2c45d97-1jammy [466 kB] 2024-04-25T00:07:08.292 INFO:teuthology.orchestra.run.smithi083.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2445-gd2c45d97-1jammy [25.6 MB] 2024-04-25T00:07:08.349 INFO:teuthology.orchestra.run.smithi083.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T00:07:08.350 INFO:teuthology.orchestra.run.smithi083.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T00:07:08.362 INFO:teuthology.orchestra.run.smithi086.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T00:07:08.366 INFO:teuthology.orchestra.run.smithi086.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T00:07:08.368 INFO:teuthology.orchestra.run.smithi083.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T00:07:08.375 INFO:teuthology.orchestra.run.smithi083.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T00:07:08.386 INFO:teuthology.orchestra.run.smithi086.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T00:07:08.396 INFO:teuthology.orchestra.run.smithi086.stdout:Get:116 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T00:07:08.424 INFO:teuthology.orchestra.run.smithi083.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T00:07:08.442 INFO:teuthology.orchestra.run.smithi083.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T00:07:08.444 INFO:teuthology.orchestra.run.smithi086.stdout:Get:117 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T00:07:08.457 INFO:teuthology.orchestra.run.smithi086.stdout:Get:118 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T00:07:08.537 INFO:teuthology.orchestra.run.smithi086.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [91.2 kB] 2024-04-25T00:07:08.558 INFO:teuthology.orchestra.run.smithi083.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T00:07:08.559 INFO:teuthology.orchestra.run.smithi083.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T00:07:08.561 INFO:teuthology.orchestra.run.smithi083.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T00:07:08.567 INFO:teuthology.orchestra.run.smithi086.stdout:Get:120 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T00:07:08.569 INFO:teuthology.orchestra.run.smithi086.stdout:Get:121 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T00:07:08.570 INFO:teuthology.orchestra.run.smithi086.stdout:Get:122 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T00:07:08.745 INFO:teuthology.orchestra.run.smithi083.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T00:07:08.751 INFO:teuthology.orchestra.run.smithi086.stdout:Get:123 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T00:07:09.019 INFO:teuthology.orchestra.run.smithi083.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T00:07:09.026 INFO:teuthology.orchestra.run.smithi086.stdout:Get:124 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T00:07:12.110 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 180 MB in 10s (18.2 MB/s) 2024-04-25T00:07:12.133 INFO:teuthology.orchestra.run.smithi083.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2445-gd2c45d97-1jammy [5,084 kB] 2024-04-25T00:07:12.231 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T00:07:12.298 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-04-25T00:07:12.301 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T00:07:12.472 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T00:07:12.741 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T00:07:12.746 INFO:teuthology.orchestra.run.smithi083.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2445-gd2c45d97-1jammy [244 kB] 2024-04-25T00:07:12.754 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T00:07:12.779 INFO:teuthology.orchestra.run.smithi083.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2445-gd2c45d97-1jammy [124 kB] 2024-04-25T00:07:12.812 INFO:teuthology.orchestra.run.smithi083.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2445-gd2c45d97-1jammy [1,525 kB] 2024-04-25T00:07:12.824 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:07:12.991 INFO:teuthology.orchestra.run.smithi083.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2445-gd2c45d97-1jammy [6,245 kB] 2024-04-25T00:07:13.135 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T00:07:13.150 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T00:07:13.194 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T00:07:13.547 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T00:07:13.561 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T00:07:13.655 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:07:13.708 INFO:teuthology.orchestra.run.smithi083.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2445-gd2c45d97-1jammy [22.7 MB] 2024-04-25T00:07:13.984 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T00:07:13.998 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T00:07:14.042 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:07:14.387 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T00:07:14.445 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:07:14.873 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T00:07:14.887 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:07:14.931 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:07:15.318 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T00:07:15.332 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:07:15.401 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:07:15.738 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T00:07:15.751 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:07:15.804 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:07:16.103 INFO:teuthology.orchestra.run.smithi083.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2445-gd2c45d97-1jammy [14.0 kB] 2024-04-25T00:07:16.104 INFO:teuthology.orchestra.run.smithi083.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [869 kB] 2024-04-25T00:07:16.132 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T00:07:16.146 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T00:07:16.176 INFO:teuthology.orchestra.run.smithi083.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2445-gd2c45d97-1jammy [2,437 kB] 2024-04-25T00:07:16.182 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:07:16.393 INFO:teuthology.orchestra.run.smithi083.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2445-gd2c45d97-1jammy [790 kB] 2024-04-25T00:07:16.476 INFO:teuthology.orchestra.run.smithi083.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2445-gd2c45d97-1jammy [147 kB] 2024-04-25T00:07:16.502 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librados2. 2024-04-25T00:07:16.515 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../010-librados2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:16.549 INFO:teuthology.orchestra.run.smithi083.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2445-gd2c45d97-1jammy [2,293 kB] 2024-04-25T00:07:16.560 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:16.981 INFO:teuthology.orchestra.run.smithi083.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2445-gd2c45d97-1jammy [8,625 kB] 2024-04-25T00:07:17.030 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librbd1. 2024-04-25T00:07:17.043 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../011-librbd1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:17.516 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T00:07:17.530 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:17.574 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:17.810 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rados. 2024-04-25T00:07:17.823 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:17.860 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:18.196 INFO:teuthology.orchestra.run.smithi083.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2445-gd2c45d97-1jammy [14.2 kB] 2024-04-25T00:07:18.196 INFO:teuthology.orchestra.run.smithi083.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2445-gd2c45d97-1jammy [50.7 MB] 2024-04-25T00:07:18.204 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T00:07:18.219 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:07:18.262 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:18.517 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T00:07:18.531 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:18.575 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:18.911 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T00:07:18.925 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:07:18.969 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:19.314 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T00:07:19.329 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T00:07:19.372 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T00:07:19.641 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T00:07:19.656 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:19.699 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:20.052 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T00:07:20.067 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T00:07:20.110 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:07:20.380 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T00:07:20.394 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T00:07:20.438 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:07:20.766 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T00:07:20.781 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T00:07:20.824 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:07:21.135 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua5.1. 2024-04-25T00:07:21.150 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T00:07:21.227 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:07:21.530 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua-any. 2024-04-25T00:07:21.544 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T00:07:21.588 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T00:07:21.865 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package zip. 2024-04-25T00:07:21.881 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T00:07:21.924 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T00:07:22.243 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package unzip. 2024-04-25T00:07:22.257 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T00:07:22.302 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:07:22.822 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package luarocks. 2024-04-25T00:07:22.837 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T00:07:22.872 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:07:22.970 INFO:teuthology.orchestra.run.smithi083.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2445-gd2c45d97-1jammy [121 kB] 2024-04-25T00:07:22.971 INFO:teuthology.orchestra.run.smithi083.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2445-gd2c45d97-1jammy [229 kB] 2024-04-25T00:07:22.973 INFO:teuthology.orchestra.run.smithi083.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2445-gd2c45d97-1jammy [34.0 kB] 2024-04-25T00:07:22.973 INFO:teuthology.orchestra.run.smithi083.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2445-gd2c45d97-1jammy [23.4 kB] 2024-04-25T00:07:22.974 INFO:teuthology.orchestra.run.smithi083.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2445-gd2c45d97-1jammy [38.9 kB] 2024-04-25T00:07:22.974 INFO:teuthology.orchestra.run.smithi083.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2445-gd2c45d97-1jammy [13.4 MB] 2024-04-25T00:07:23.287 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librgw2. 2024-04-25T00:07:23.301 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../027-librgw2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:23.345 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:23.584 INFO:teuthology.orchestra.run.smithi083.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [91.2 kB] 2024-04-25T00:07:23.839 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T00:07:23.853 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:23.897 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:24.219 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 180 MB in 22s (8,160 kB/s) 2024-04-25T00:07:24.234 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T00:07:24.248 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T00:07:24.292 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:07:24.391 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T00:07:24.458 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-04-25T00:07:24.462 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T00:07:24.531 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T00:07:24.570 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T00:07:24.584 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:24.628 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:24.817 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T00:07:24.830 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T00:07:24.868 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:07:25.064 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-common. 2024-04-25T00:07:25.079 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:25.156 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:25.157 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T00:07:25.170 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T00:07:25.212 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T00:07:25.498 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T00:07:25.511 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T00:07:25.590 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:07:25.935 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T00:07:25.948 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T00:07:26.037 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:07:26.353 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-base. 2024-04-25T00:07:26.368 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:26.380 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T00:07:26.393 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T00:07:26.438 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:07:26.458 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:26.858 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T00:07:26.872 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:07:26.916 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:07:26.998 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T00:07:27.012 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T00:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:07:27.345 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T00:07:27.358 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:07:27.384 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T00:07:27.398 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T00:07:27.403 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:07:27.442 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:07:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T00:07:27.804 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:07:27.848 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:07:27.887 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T00:07:27.900 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T00:07:27.937 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:07:28.142 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T00:07:28.156 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T00:07:28.192 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:07:28.239 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T00:07:28.253 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T00:07:28.297 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:07:28.537 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librados2. 2024-04-25T00:07:28.551 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../010-librados2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:28.584 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T00:07:28.595 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:28.598 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T00:07:28.642 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:07:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T00:07:28.943 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T00:07:28.986 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T00:07:29.031 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librbd1. 2024-04-25T00:07:29.046 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../011-librbd1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:29.098 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:29.306 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-portend. 2024-04-25T00:07:29.320 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T00:07:29.364 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T00:07:29.692 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T00:07:29.706 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T00:07:29.726 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T00:07:29.740 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:29.750 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T00:07:29.776 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:30.028 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T00:07:30.042 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T00:07:30.045 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rados. 2024-04-25T00:07:30.059 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:30.086 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:07:30.103 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:30.448 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T00:07:30.448 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T00:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T00:07:30.462 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:07:30.506 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:30.506 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T00:07:30.744 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T00:07:30.758 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:30.784 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T00:07:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:30.798 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T00:07:30.834 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T00:07:31.138 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T00:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T00:07:31.153 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:07:31.160 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T00:07:31.196 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:31.203 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:07:31.505 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-mako. 2024-04-25T00:07:31.520 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T00:07:31.533 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T00:07:31.548 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T00:07:31.600 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T00:07:31.606 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:07:31.869 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T00:07:31.884 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:31.944 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T00:07:31.965 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T00:07:32.008 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:07:32.247 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T00:07:32.262 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T00:07:32.297 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:07:32.328 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T00:07:32.342 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T00:07:32.386 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:07:32.591 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T00:07:32.606 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T00:07:32.649 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:07:32.655 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-webob. 2024-04-25T00:07:32.669 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T00:07:32.730 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:07:32.995 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T00:07:33.009 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T00:07:33.053 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:07:33.100 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T00:07:33.114 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T00:07:33.169 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:07:33.364 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua5.1. 2024-04-25T00:07:33.379 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T00:07:33.414 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:07:33.477 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T00:07:33.492 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T00:07:33.552 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:07:33.760 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-any. 2024-04-25T00:07:33.774 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T00:07:33.818 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T00:07:33.855 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-paste. 2024-04-25T00:07:33.870 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T00:07:33.913 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:07:34.054 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package zip. 2024-04-25T00:07:34.069 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T00:07:34.112 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T00:07:34.250 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T00:07:34.264 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T00:07:34.316 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:07:35.251 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package unzip. 2024-04-25T00:07:35.266 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T00:07:35.286 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T00:07:35.300 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T00:07:35.307 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:07:35.344 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:07:35.672 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T00:07:35.686 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T00:07:35.703 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package luarocks. 2024-04-25T00:07:35.717 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T00:07:35.730 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T00:07:35.761 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:07:36.041 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T00:07:36.055 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T00:07:36.099 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:07:36.169 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librgw2. 2024-04-25T00:07:36.183 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../027-librgw2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:36.227 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:36.454 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T00:07:36.467 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T00:07:36.512 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T00:07:36.722 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T00:07:36.736 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:36.780 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:36.864 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-idna. 2024-04-25T00:07:36.878 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T00:07:36.923 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T00:07:37.124 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T00:07:37.139 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T00:07:37.183 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:07:37.200 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T00:07:37.214 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T00:07:37.250 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:07:37.494 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T00:07:37.509 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:37.553 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:37.561 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-requests. 2024-04-25T00:07:37.575 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T00:07:37.612 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:07:37.881 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T00:07:37.894 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T00:07:37.931 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:07:38.014 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-common. 2024-04-25T00:07:38.029 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:38.072 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:38.375 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T00:07:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:07:38.426 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:38.830 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T00:07:38.843 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:38.888 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:39.199 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T00:07:39.212 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-base. 2024-04-25T00:07:39.212 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T00:07:39.227 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:39.257 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:07:39.279 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:39.602 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T00:07:39.615 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T00:07:39.660 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T00:07:39.866 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T00:07:39.879 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T00:07:39.916 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:07:40.013 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T00:07:40.027 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:40.071 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:40.203 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T00:07:40.216 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T00:07:40.261 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:07:40.449 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T00:07:40.463 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:40.507 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:40.589 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T00:07:40.604 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T00:07:40.647 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:07:40.950 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T00:07:40.965 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T00:07:41.008 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:07:41.035 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T00:07:41.049 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:41.093 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:41.336 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T00:07:41.352 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T00:07:41.395 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:07:41.664 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T00:07:41.679 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T00:07:41.714 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T00:07:41.838 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph. 2024-04-25T00:07:41.852 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../068-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:41.896 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:42.009 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-portend. 2024-04-25T00:07:42.024 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T00:07:42.086 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T00:07:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T00:07:42.255 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:42.298 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:42.387 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T00:07:42.402 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T00:07:42.445 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T00:07:42.685 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T00:07:42.700 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:42.715 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T00:07:42.730 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T00:07:42.743 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:42.765 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:07:43.104 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephadm. 2024-04-25T00:07:43.117 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../071-cephadm_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:43.143 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T00:07:43.157 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T00:07:43.162 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:43.202 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T00:07:43.498 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T00:07:43.512 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T00:07:43.522 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T00:07:43.537 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T00:07:43.573 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T00:07:43.580 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T00:07:43.883 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T00:07:43.884 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T00:07:43.898 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:07:43.898 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T00:07:43.941 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:07:43.967 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:07:44.228 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-mako. 2024-04-25T00:07:44.242 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T00:07:44.286 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:07:44.865 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T00:07:44.879 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T00:07:44.922 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:07:44.946 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-babel. 2024-04-25T00:07:44.959 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:07:44.996 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:07:45.201 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T00:07:45.215 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T00:07:45.250 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:07:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T00:07:45.347 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T00:07:45.391 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:07:45.553 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-webob. 2024-04-25T00:07:45.568 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T00:07:45.612 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:07:45.744 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T00:07:45.757 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:07:45.802 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:45.948 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T00:07:45.963 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T00:07:46.009 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:07:46.172 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T00:07:46.186 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T00:07:46.230 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T00:07:46.343 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T00:07:46.357 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T00:07:46.410 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:07:46.532 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-routes. 2024-04-25T00:07:46.546 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T00:07:46.616 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:07:46.696 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-paste. 2024-04-25T00:07:46.711 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T00:07:46.754 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:07:46.944 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T00:07:46.958 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:07:47.002 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:47.099 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T00:07:47.114 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T00:07:47.157 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:07:47.427 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T00:07:47.442 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T00:07:47.477 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:07:47.738 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T00:07:47.753 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T00:07:47.797 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T00:07:48.141 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T00:07:48.156 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T00:07:48.190 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T00:07:48.199 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:07:48.204 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T00:07:48.240 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:07:48.563 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T00:07:48.578 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T00:07:48.621 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T00:07:48.935 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T00:07:48.952 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T00:07:48.983 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-idna. 2024-04-25T00:07:48.984 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:07:48.997 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T00:07:49.041 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T00:07:49.311 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T00:07:49.325 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T00:07:49.362 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:07:49.372 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T00:07:49.386 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T00:07:49.507 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:07:49.681 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-requests. 2024-04-25T00:07:49.696 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T00:07:49.730 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:07:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T00:07:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T00:07:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:07:50.017 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T00:07:50.032 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T00:07:50.075 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:07:50.370 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T00:07:50.384 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:07:50.453 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:50.585 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T00:07:50.599 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:07:50.643 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:50.808 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T00:07:50.822 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:50.867 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:51.203 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T00:07:51.217 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T00:07:51.278 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:07:51.571 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T00:07:51.580 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T00:07:51.586 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T00:07:51.596 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T00:07:51.629 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T00:07:51.630 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T00:07:51.931 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T00:07:51.947 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T00:07:51.967 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T00:07:51.982 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:52.006 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:07:52.025 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:52.326 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T00:07:52.342 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T00:07:52.376 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:07:52.429 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T00:07:52.443 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:52.487 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:52.679 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T00:07:52.694 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T00:07:52.737 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T00:07:53.066 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T00:07:53.081 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T00:07:53.090 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T00:07:53.104 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:53.124 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T00:07:53.149 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:53.502 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T00:07:53.517 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T00:07:53.560 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:07:53.896 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T00:07:53.912 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T00:07:53.954 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T00:07:53.969 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph. 2024-04-25T00:07:53.983 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../068-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:54.027 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:54.266 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T00:07:54.281 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T00:07:54.324 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:07:54.330 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T00:07:54.344 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:54.388 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:54.725 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T00:07:54.738 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:54.783 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:55.021 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T00:07:55.035 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:07:55.079 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:55.170 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package cephadm. 2024-04-25T00:07:55.183 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../071-cephadm_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:55.227 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:55.398 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T00:07:55.407 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T00:07:55.448 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:07:55.581 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T00:07:55.594 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T00:07:55.755 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T00:07:55.910 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T00:07:55.918 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:07:55.968 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:07:56.092 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T00:07:56.105 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:07:56.149 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:07:56.237 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package jq. 2024-04-25T00:07:56.246 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:07:56.287 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T00:07:56.565 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package socat. 2024-04-25T00:07:56.577 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T00:07:56.614 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:07:56.917 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T00:07:56.926 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T00:07:56.967 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:07:57.172 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-babel. 2024-04-25T00:07:57.184 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:07:57.222 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:07:57.356 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-test. 2024-04-25T00:07:57.365 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:07:57.406 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:57.524 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T00:07:57.537 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T00:07:57.583 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:07:57.945 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T00:07:57.958 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:07:58.003 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:58.331 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T00:07:58.346 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T00:07:58.389 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T00:07:58.676 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-routes. 2024-04-25T00:07:58.690 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T00:07:58.752 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:07:59.087 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T00:07:59.101 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:07:59.145 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:07:59.585 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T00:07:59.596 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:07:59.635 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:00.014 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T00:08:00.024 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:08:00.064 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:00.316 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T00:08:00.326 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T00:08:00.366 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T00:08:00.618 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T00:08:00.628 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:08:00.668 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:00.701 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T00:08:00.714 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T00:08:00.750 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:08:00.929 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T00:08:00.939 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:08:00.979 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:01.240 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T00:08:01.246 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T00:08:01.248 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:08:01.268 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T00:08:01.289 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:01.346 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:08:01.592 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T00:08:01.601 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T00:08:01.642 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:08:01.708 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T00:08:01.720 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T00:08:01.843 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:08:01.996 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T00:08:02.006 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T00:08:02.046 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:08:02.197 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T00:08:02.208 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T00:08:02.246 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:08:02.416 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T00:08:02.426 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T00:08:02.466 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:08:02.889 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T00:08:02.896 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package pkg-config. 2024-04-25T00:08:02.898 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:08:02.908 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T00:08:02.945 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:08:02.964 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:03.174 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T00:08:03.184 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T00:08:03.241 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:08:03.651 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T00:08:03.662 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T00:08:03.692 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T00:08:03.925 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T00:08:03.936 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T00:08:03.967 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T00:08:03.995 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T00:08:04.006 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T00:08:04.045 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T00:08:04.270 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T00:08:04.280 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T00:08:04.320 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:08:04.398 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T00:08:04.409 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T00:08:04.448 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:08:04.665 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T00:08:04.675 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T00:08:04.715 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:08:04.734 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T00:08:04.745 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T00:08:04.776 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T00:08:05.035 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T00:08:05.045 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T00:08:05.085 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T00:08:05.137 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-py. 2024-04-25T00:08:05.147 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T00:08:05.186 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T00:08:05.356 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T00:08:05.365 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T00:08:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T00:08:05.490 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T00:08:05.497 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T00:08:05.531 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:08:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T00:08:05.752 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T00:08:05.792 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:08:05.985 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T00:08:05.992 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T00:08:06.035 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:08:06.112 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T00:08:06.125 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T00:08:06.170 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T00:08:06.337 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-toml. 2024-04-25T00:08:06.347 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T00:08:06.412 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T00:08:06.532 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T00:08:06.543 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T00:08:06.582 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:08:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T00:08:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T00:08:06.781 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:08:07.143 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T00:08:07.154 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T00:08:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:08:07.214 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T00:08:07.224 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:08:07.264 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:07.470 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T00:08:07.480 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T00:08:07.520 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:08:07.592 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T00:08:07.602 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T00:08:07.642 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:08:07.995 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T00:08:08.005 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:08:08.045 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:08:08.182 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package radosgw. 2024-04-25T00:08:08.191 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../122-radosgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:08:08.232 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:08.281 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package jq. 2024-04-25T00:08:08.292 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:08:08.322 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T00:08:08.542 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package socat. 2024-04-25T00:08:08.555 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T00:08:08.600 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:08:08.810 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T00:08:08.820 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:08:08.860 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:08.937 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T00:08:08.948 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T00:08:08.987 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:08:09.184 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T00:08:09.333 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-test. 2024-04-25T00:08:09.344 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:08:09.383 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:09.657 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:08:09.774 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:08:10.008 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T00:08:10.426 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T00:08:11.138 INFO:teuthology.orchestra.run.smithi086.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T00:08:11.158 INFO:teuthology.orchestra.run.smithi086.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-04-25T00:08:11.201 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:11.604 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T00:08:11.616 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:08:11.654 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:11.812 INFO:teuthology.orchestra.run.smithi086.stdout:Adding system user cephadm....done 2024-04-25T00:08:12.017 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T00:08:12.028 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:08:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:08:12.066 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:12.412 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T00:08:12.423 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T00:08:12.461 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T00:08:12.471 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:08:12.680 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T00:08:12.692 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:08:12.730 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:12.824 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:08:13.000 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T00:08:13.012 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:08:13.017 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:08:13.050 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:13.286 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T00:08:13.297 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:08:13.327 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:13.378 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T00:08:13.647 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T00:08:13.647 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:08:13.658 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T00:08:13.696 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:08:13.772 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T00:08:14.074 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:08:14.093 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T00:08:14.105 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T00:08:14.134 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:08:14.484 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T00:08:14.513 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T00:08:14.525 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T00:08:14.563 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:08:14.752 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T00:08:14.985 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package pkg-config. 2024-04-25T00:08:14.997 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T00:08:15.027 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:08:15.061 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T00:08:15.263 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T00:08:15.274 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T00:08:15.313 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:08:15.488 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:08:15.742 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T00:08:15.755 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T00:08:15.765 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:08:15.800 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T00:08:16.015 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:08:16.103 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T00:08:16.115 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T00:08:16.152 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T00:08:16.266 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:08:16.506 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T00:08:16.518 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T00:08:16.526 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:16.556 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:08:16.810 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:08:16.851 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T00:08:16.863 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T00:08:16.901 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T00:08:16.936 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:08:17.062 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T00:08:17.270 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-py. 2024-04-25T00:08:17.282 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T00:08:17.320 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T00:08:17.338 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:08:17.472 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:08:17.657 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T00:08:17.669 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T00:08:17.707 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:08:17.715 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:08:17.898 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T00:08:17.927 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T00:08:17.992 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:08:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T00:08:18.152 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T00:08:18.164 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T00:08:18.194 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:08:18.489 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-toml. 2024-04-25T00:08:18.500 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T00:08:18.535 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:08:18.538 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T00:08:18.682 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T00:08:18.867 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T00:08:18.878 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T00:08:18.916 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:08:18.971 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:08:19.237 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T00:08:19.248 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T00:08:19.255 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:08:19.286 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:08:19.556 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T00:08:19.567 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T00:08:19.590 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T00:08:19.614 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:08:19.715 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:08:20.401 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:08:20.418 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package radosgw. 2024-04-25T00:08:20.430 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../122-radosgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:08:20.468 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:20.677 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:08:20.802 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:08:20.928 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:08:21.096 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T00:08:21.108 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:08:21.146 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:21.272 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T00:08:21.523 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T00:08:21.673 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T00:08:21.811 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:08:21.936 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:08:21.949 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:08:22.195 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T00:08:22.325 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:08:22.607 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T00:08:22.626 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T00:08:22.944 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:08:23.212 INFO:teuthology.orchestra.run.smithi083.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T00:08:23.262 INFO:teuthology.orchestra.run.smithi083.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-04-25T00:08:23.304 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:08:23.305 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:23.563 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:08:23.689 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:23.782 INFO:teuthology.orchestra.run.smithi083.stdout:Adding system user cephadm....done 2024-04-25T00:08:23.961 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:08:23.982 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T00:08:24.238 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:08:24.324 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:08:24.481 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:08:24.607 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:08:24.892 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T00:08:25.151 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:08:25.269 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T00:08:25.555 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:08:25.587 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:08:25.807 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:08:25.916 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T00:08:25.956 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T00:08:26.184 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:08:26.215 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T00:08:26.309 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:08:26.460 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:08:26.475 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T00:08:26.743 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:08:26.761 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:08:27.011 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:08:27.021 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T00:08:27.271 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:08:27.306 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:08:27.415 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:08:27.547 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:08:27.756 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:08:27.807 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:28.017 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T00:08:28.074 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:08:28.184 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:08:28.285 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T00:08:28.285 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T00:08:28.544 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:08:28.569 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:08:28.670 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:08:28.837 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T00:08:28.930 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:08:29.095 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T00:08:29.113 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:08:29.134 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T00:08:29.190 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:08:29.230 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T00:08:29.317 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T00:08:29.498 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:08:29.599 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:08:29.685 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:08:29.810 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T00:08:29.917 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:08:30.087 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:08:30.234 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:08:30.354 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:08:30.527 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:08:30.647 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T00:08:30.661 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:08:30.774 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:08:30.787 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:08:30.867 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T00:08:31.106 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:08:31.226 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:08:31.451 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:08:31.616 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:08:31.718 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:08:31.844 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:08:31.884 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T00:08:31.970 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:08:32.010 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T00:08:32.278 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:08:32.288 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T00:08:32.404 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:08:32.673 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T00:08:32.738 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:08:32.933 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:08:33.097 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:08:33.351 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:08:33.357 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T00:08:33.585 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T00:08:33.652 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T00:08:33.861 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T00:08:33.978 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:08:34.162 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:08:34.321 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:08:34.438 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:08:34.581 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:08:34.749 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:34.823 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T00:08:35.050 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T00:08:35.116 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:08:35.376 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:08:35.469 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:08:35.595 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:35.729 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:35.855 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:08:36.183 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:36.309 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:36.435 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T00:08:36.599 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:08:36.753 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:36.809 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:08:36.879 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:36.927 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T00:08:37.014 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:37.212 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:08:37.213 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:08:37.228 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:08:37.346 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:08:37.447 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:08:37.748 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:08:37.856 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:38.016 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:38.016 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T00:08:38.141 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:38.267 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:38.301 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:08:38.393 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:08:38.410 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:08:38.720 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:08:38.980 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T00:08:39.239 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T00:08:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:08:39.680 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:39.766 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T00:08:39.806 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:39.965 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:40.026 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:08:40.091 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:40.152 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T00:08:40.447 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:08:40.550 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:40.595 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:08:40.676 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:40.880 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:08:41.173 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:08:41.262 INFO:teuthology.orchestra.run.smithi086.stdout:Adding group ceph....done 2024-04-25T00:08:41.449 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:08:41.575 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:08:41.621 INFO:teuthology.orchestra.run.smithi086.stdout:Adding system user ceph....done 2024-04-25T00:08:41.701 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:08:41.789 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T00:08:41.790 INFO:teuthology.orchestra.run.smithi086.stdout:Setting system user ceph properties....done 2024-04-25T00:08:41.809 INFO:teuthology.orchestra.run.smithi086.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T00:08:41.949 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T00:08:42.036 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:08:42.154 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:08:42.347 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T00:08:42.564 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:08:42.840 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T00:08:42.966 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T00:08:43.010 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:43.242 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:08:43.244 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:08:43.244 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:08:43.351 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:08:43.678 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:08:43.886 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:43.995 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:44.012 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:08:44.280 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T00:08:44.508 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T00:08:44.522 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:08:44.522 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:08:44.768 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T00:08:45.044 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:08:45.164 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:45.312 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:08:45.498 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T00:08:45.689 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T00:08:45.991 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:08:46.174 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:46.447 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:08:46.447 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:08:46.702 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:08:46.828 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:46.946 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:47.055 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:08:47.101 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:47.312 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:08:47.312 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:08:47.400 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:47.526 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:47.652 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T00:08:47.954 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:48.018 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:48.080 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:48.231 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:48.349 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:08:48.349 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:08:48.447 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:08:48.447 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:08:49.004 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:49.091 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:49.138 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:49.217 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:49.264 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:49.343 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:49.456 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:08:49.456 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:08:49.468 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:49.595 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:08:50.074 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:50.241 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:50.367 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:50.509 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:50.862 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:08:50.874 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:50.964 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:08:51.000 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:51.549 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:08:51.550 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:51.550 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:51.550 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:51.550 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:52.040 INFO:teuthology.orchestra.run.smithi083.stdout:Adding group ceph....done 2024-04-25T00:08:52.242 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:08:52.442 INFO:teuthology.orchestra.run.smithi083.stdout:Adding system user ceph....done 2024-04-25T00:08:52.644 INFO:teuthology.orchestra.run.smithi083.stdout:Setting system user ceph properties....done 2024-04-25T00:08:52.664 INFO:teuthology.orchestra.run.smithi083.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T00:08:52.810 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T00:08:52.940 INFO:teuthology.orchestra.run.smithi086.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:08:52.945 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T00:08:53.019 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-04-25T00:08:53.200 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2024-04-25T00:08:53.201 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2024-04-25T00:08:53.212 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T00:08:53.370 INFO:teuthology.orchestra.run.smithi086.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T00:08:53.370 INFO:teuthology.orchestra.run.smithi086.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T00:08:53.370 INFO:teuthology.orchestra.run.smithi086.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T00:08:53.370 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:08:53.370 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:08:53.370 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:08:53.370 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:08:53.370 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:08:53.371 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:08:53.371 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:08:53.371 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2024-04-25T00:08:53.372 INFO:teuthology.orchestra.run.smithi086.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T00:08:53.372 INFO:teuthology.orchestra.run.smithi086.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T00:08:53.372 INFO:teuthology.orchestra.run.smithi086.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T00:08:53.372 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T00:08:53.372 INFO:teuthology.orchestra.run.smithi086.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:08:53.373 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2024-04-25T00:08:53.373 INFO:teuthology.orchestra.run.smithi086.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T00:08:53.373 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T00:08:53.373 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T00:08:53.427 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2024-04-25T00:08:53.428 INFO:teuthology.orchestra.run.smithi086.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T00:08:53.428 INFO:teuthology.orchestra.run.smithi086.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T00:08:53.429 INFO:teuthology.orchestra.run.smithi086.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T00:08:53.429 INFO:teuthology.orchestra.run.smithi086.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T00:08:53.429 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T00:08:53.429 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T00:08:53.429 INFO:teuthology.orchestra.run.smithi086.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:08:53.430 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages will be upgraded: 2024-04-25T00:08:53.430 INFO:teuthology.orchestra.run.smithi086.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T00:08:53.491 INFO:teuthology.orchestra.run.smithi086.stdout:4 upgraded, 39 newly installed, 0 to remove and 301 not upgraded. 2024-04-25T00:08:53.492 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 55.7 MB of archives. 2024-04-25T00:08:53.492 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T00:08:53.492 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T00:08:53.696 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T00:08:53.700 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T00:08:53.736 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T00:08:53.736 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T00:08:53.737 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T00:08:53.747 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T00:08:53.747 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T00:08:53.749 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T00:08:53.750 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T00:08:53.751 INFO:teuthology.orchestra.run.smithi086.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T00:08:53.752 INFO:teuthology.orchestra.run.smithi086.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T00:08:53.762 INFO:teuthology.orchestra.run.smithi086.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T00:08:53.763 INFO:teuthology.orchestra.run.smithi086.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T00:08:53.763 INFO:teuthology.orchestra.run.smithi086.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T00:08:53.783 INFO:teuthology.orchestra.run.smithi086.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T00:08:53.914 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:54.139 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:08:54.139 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:08:54.149 INFO:teuthology.orchestra.run.smithi086.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T00:08:54.151 INFO:teuthology.orchestra.run.smithi086.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T00:08:54.151 INFO:teuthology.orchestra.run.smithi086.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T00:08:54.155 INFO:teuthology.orchestra.run.smithi086.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T00:08:54.155 INFO:teuthology.orchestra.run.smithi086.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T00:08:54.160 INFO:teuthology.orchestra.run.smithi086.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T00:08:54.162 INFO:teuthology.orchestra.run.smithi086.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T00:08:54.169 INFO:teuthology.orchestra.run.smithi086.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T00:08:54.170 INFO:teuthology.orchestra.run.smithi086.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T00:08:54.192 INFO:teuthology.orchestra.run.smithi086.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T00:08:54.197 INFO:teuthology.orchestra.run.smithi086.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T00:08:54.208 INFO:teuthology.orchestra.run.smithi086.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T00:08:54.233 INFO:teuthology.orchestra.run.smithi086.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T00:08:54.237 INFO:teuthology.orchestra.run.smithi086.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T00:08:54.238 INFO:teuthology.orchestra.run.smithi086.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T00:08:54.238 INFO:teuthology.orchestra.run.smithi086.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T00:08:54.239 INFO:teuthology.orchestra.run.smithi086.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T00:08:54.253 INFO:teuthology.orchestra.run.smithi086.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T00:08:54.257 INFO:teuthology.orchestra.run.smithi086.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T00:08:54.512 INFO:teuthology.orchestra.run.smithi086.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T00:08:54.512 INFO:teuthology.orchestra.run.smithi086.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T00:08:54.512 INFO:teuthology.orchestra.run.smithi086.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T00:08:54.512 INFO:teuthology.orchestra.run.smithi086.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T00:08:54.513 INFO:teuthology.orchestra.run.smithi086.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T00:08:54.513 INFO:teuthology.orchestra.run.smithi086.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T00:08:54.526 INFO:teuthology.orchestra.run.smithi086.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T00:08:54.527 INFO:teuthology.orchestra.run.smithi086.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T00:08:54.799 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:54.903 INFO:teuthology.orchestra.run.smithi086.stdout:Preconfiguring packages ... 2024-04-25T00:08:54.991 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:55.049 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 55.7 MB in 1s (51.2 MB/s) 2024-04-25T00:08:55.342 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140911 files and directories currently installed.) 2024-04-25T00:08:55.345 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:08:55.490 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:08:55.540 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:08:55.540 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:08:56.141 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T00:08:56.203 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:56.564 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T00:08:57.237 INFO:teuthology.orchestra.run.smithi086.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T00:08:57.272 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:57.496 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:08:57.496 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:08:57.621 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package flex. 2024-04-25T00:08:57.692 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140911 files and directories currently installed.) 2024-04-25T00:08:57.696 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T00:08:57.745 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T00:08:58.151 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:58.273 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:08:58.362 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:08:58.362 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:08:58.466 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T00:08:58.850 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:08:58.969 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:59.271 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:08:59.271 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:08:59.419 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140984 files and directories currently installed.) 2024-04-25T00:08:59.423 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:08:59.550 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:08:59.904 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:08:59.937 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:09:00.057 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:09:00.063 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:09:00.206 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:09:00.371 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:09:00.423 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:09:00.423 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:09:00.740 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package bison. 2024-04-25T00:09:00.811 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140984 files and directories currently installed.) 2024-04-25T00:09:00.814 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T00:09:00.851 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:09:01.057 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:09:01.183 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:09:01.260 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T00:09:01.274 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T00:09:01.309 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:09:01.335 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:09:01.436 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:09:01.655 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package dump. 2024-04-25T00:09:01.669 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T00:09:01.746 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T00:09:01.772 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:09:01.888 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:09:02.042 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package indent. 2024-04-25T00:09:02.056 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:09:02.075 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:09:02.100 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:09:02.344 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T00:09:02.358 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T00:09:02.445 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:09:02.782 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T00:09:02.798 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T00:09:02.841 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:09:03.118 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T00:09:03.134 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T00:09:03.168 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:09:03.318 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:09:03.463 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T00:09:03.479 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T00:09:03.522 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:09:03.808 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T00:09:03.824 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T00:09:03.866 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:09:04.047 INFO:teuthology.orchestra.run.smithi083.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:09:04.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T00:09:04.118 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T00:09:04.126 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-25T00:09:04.134 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T00:09:04.177 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T00:09:04.300 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-04-25T00:09:04.301 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-04-25T00:09:04.471 INFO:teuthology.orchestra.run.smithi083.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T00:09:04.471 INFO:teuthology.orchestra.run.smithi083.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T00:09:04.471 INFO:teuthology.orchestra.run.smithi083.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T00:09:04.471 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:09:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:09:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:09:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:09:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:09:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:09:04.472 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:09:04.473 INFO:teuthology.orchestra.run.smithi083.stdout:The following additional packages will be installed: 2024-04-25T00:09:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T00:09:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T00:09:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T00:09:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T00:09:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:09:04.474 INFO:teuthology.orchestra.run.smithi083.stdout:Suggested packages: 2024-04-25T00:09:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T00:09:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T00:09:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T00:09:04.488 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T00:09:04.504 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T00:09:04.528 INFO:teuthology.orchestra.run.smithi083.stdout:The following NEW packages will be installed: 2024-04-25T00:09:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T00:09:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T00:09:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T00:09:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T00:09:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T00:09:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T00:09:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:09:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages will be upgraded: 2024-04-25T00:09:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T00:09:04.538 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:09:04.717 INFO:teuthology.orchestra.run.smithi083.stdout:4 upgraded, 39 newly installed, 0 to remove and 301 not upgraded. 2024-04-25T00:09:04.717 INFO:teuthology.orchestra.run.smithi083.stdout:Need to get 55.7 MB of archives. 2024-04-25T00:09:04.717 INFO:teuthology.orchestra.run.smithi083.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T00:09:04.718 INFO:teuthology.orchestra.run.smithi083.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T00:09:05.398 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T00:09:05.426 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T00:09:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T00:09:05.568 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T00:09:05.568 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T00:09:05.587 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T00:09:05.589 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T00:09:05.594 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T00:09:05.597 INFO:teuthology.orchestra.run.smithi083.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T00:09:05.597 INFO:teuthology.orchestra.run.smithi083.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T00:09:05.764 INFO:teuthology.orchestra.run.smithi083.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T00:09:05.900 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T00:09:05.915 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:09:05.958 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:09:06.057 INFO:teuthology.orchestra.run.smithi083.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T00:09:06.067 INFO:teuthology.orchestra.run.smithi083.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T00:09:06.072 INFO:teuthology.orchestra.run.smithi083.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T00:09:06.156 INFO:teuthology.orchestra.run.smithi083.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T00:09:06.244 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T00:09:06.259 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:09:06.302 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:09:06.613 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T00:09:06.629 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:09:06.663 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:09:07.016 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T00:09:07.032 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T00:09:07.150 INFO:teuthology.orchestra.run.smithi083.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T00:09:07.152 INFO:teuthology.orchestra.run.smithi083.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T00:09:07.152 INFO:teuthology.orchestra.run.smithi083.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T00:09:07.155 INFO:teuthology.orchestra.run.smithi083.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T00:09:07.155 INFO:teuthology.orchestra.run.smithi083.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T00:09:07.159 INFO:teuthology.orchestra.run.smithi083.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T00:09:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:09:07.326 INFO:teuthology.orchestra.run.smithi083.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T00:09:07.620 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T00:09:07.636 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:09:07.678 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:09:07.935 INFO:teuthology.orchestra.run.smithi083.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T00:09:07.936 INFO:teuthology.orchestra.run.smithi083.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T00:09:07.956 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T00:09:07.972 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:09:08.006 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:09:08.120 INFO:teuthology.orchestra.run.smithi083.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T00:09:08.121 INFO:teuthology.orchestra.run.smithi083.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T00:09:08.159 INFO:teuthology.orchestra.run.smithi083.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T00:09:08.221 INFO:teuthology.orchestra.run.smithi083.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T00:09:08.226 INFO:teuthology.orchestra.run.smithi083.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T00:09:08.227 INFO:teuthology.orchestra.run.smithi083.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T00:09:08.228 INFO:teuthology.orchestra.run.smithi083.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T00:09:08.229 INFO:teuthology.orchestra.run.smithi083.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T00:09:08.359 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T00:09:08.374 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:09:08.417 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:09:08.437 INFO:teuthology.orchestra.run.smithi083.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T00:09:08.704 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T00:09:08.719 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:09:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:09:08.855 INFO:teuthology.orchestra.run.smithi083.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T00:09:09.031 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T00:09:09.046 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:09:09.089 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:09:09.618 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T00:09:09.632 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T00:09:09.676 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:09:09.988 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package modemmanager. 2024-04-25T00:09:10.003 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:09:10.079 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:09:10.761 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package network-manager. 2024-04-25T00:09:10.777 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:09:10.828 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:09:11.469 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ppp. 2024-04-25T00:09:11.484 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T00:09:11.874 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:09:12.223 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T00:09:12.237 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T00:09:12.281 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T00:09:12.534 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T00:09:12.549 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T00:09:12.593 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T00:09:13.505 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T00:09:13.521 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T00:09:13.555 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T00:09:13.867 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T00:09:13.883 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:09:13.925 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:09:14.513 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T00:09:14.529 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T00:09:14.590 INFO:teuthology.orchestra.run.smithi086.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T00:09:14.620 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-common (238) ... 2024-04-25T00:09:15.098 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T00:09:15.113 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:09:15.166 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:09:15.986 INFO:teuthology.orchestra.run.smithi083.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T00:09:15.987 INFO:teuthology.orchestra.run.smithi083.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T00:09:15.987 INFO:teuthology.orchestra.run.smithi083.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T00:09:15.988 INFO:teuthology.orchestra.run.smithi083.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T00:09:15.988 INFO:teuthology.orchestra.run.smithi083.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T00:09:15.988 INFO:teuthology.orchestra.run.smithi083.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T00:09:16.083 INFO:teuthology.orchestra.run.smithi083.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T00:09:16.084 INFO:teuthology.orchestra.run.smithi083.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T00:09:16.248 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql. 2024-04-25T00:09:16.263 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T00:09:16.307 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql (14+238) ... 2024-04-25T00:09:16.471 INFO:teuthology.orchestra.run.smithi083.stdout:Preconfiguring packages ... 2024-04-25T00:09:16.534 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T00:09:16.548 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T00:09:16.592 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T00:09:16.663 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 55.7 MB in 12s (4,826 kB/s) 2024-04-25T00:09:16.836 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T00:09:16.852 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T00:09:16.895 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T00:09:16.964 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140911 files and directories currently installed.) 2024-04-25T00:09:16.969 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:09:17.100 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:09:17.230 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T00:09:17.245 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T00:09:17.288 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:09:17.558 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package tcl. 2024-04-25T00:09:17.573 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T00:09:17.620 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T00:09:17.985 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T00:09:18.001 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T00:09:18.043 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:09:18.118 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T00:09:18.398 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T00:09:18.414 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T00:09:18.481 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T00:09:18.968 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T00:09:18.984 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T00:09:19.043 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:09:19.133 INFO:teuthology.orchestra.run.smithi083.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T00:09:19.344 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T00:09:19.431 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package flex. 2024-04-25T00:09:19.501 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140911 files and directories currently installed.) 2024-04-25T00:09:19.504 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T00:09:19.547 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T00:09:19.556 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T00:09:19.683 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:09:19.826 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:09:19.952 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:09:20.077 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:09:20.108 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:09:20.212 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:09:20.269 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T00:09:20.660 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:09:21.154 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T00:09:21.213 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140984 files and directories currently installed.) 2024-04-25T00:09:21.217 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:09:21.251 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:09:21.336 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:09:21.611 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:09:21.611 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:09:21.764 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:09:21.901 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:09:22.222 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:09:22.345 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:09:22.446 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:09:22.564 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:09:22.609 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package bison. 2024-04-25T00:09:22.690 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T00:09:22.691 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140984 files and directories currently installed.) 2024-04-25T00:09:22.695 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T00:09:22.816 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:09:22.871 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:09:22.975 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:09:23.101 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:09:23.235 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:09:23.272 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T00:09:23.286 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T00:09:23.356 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:09:23.659 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package dump. 2024-04-25T00:09:23.673 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T00:09:23.718 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T00:09:24.004 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package indent. 2024-04-25T00:09:24.018 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:09:24.062 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:09:24.365 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T00:09:24.379 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T00:09:24.423 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:09:24.424 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:09:24.551 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T00:09:24.646 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T00:09:24.711 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:09:24.711 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T00:09:24.725 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T00:09:24.769 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:09:24.820 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:09:24.958 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T00:09:25.039 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T00:09:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T00:09:25.088 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T00:09:25.097 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:09:25.214 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:09:25.304 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T00:09:25.366 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:09:25.418 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T00:09:25.432 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T00:09:25.467 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:09:25.500 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:09:25.678 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T00:09:25.693 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:09:25.694 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T00:09:25.736 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:09:25.820 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:09:25.920 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T00:09:26.006 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T00:09:26.021 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T00:09:26.064 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T00:09:26.080 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T00:09:26.206 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:09:26.384 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T00:09:26.399 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T00:09:26.442 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:09:26.834 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:09:26.834 INFO:teuthology.orchestra.run.smithi086.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T00:09:26.834 INFO:teuthology.orchestra.run.smithi086.stdout:which means they are currently configured by ifupdown: 2024-04-25T00:09:26.834 INFO:teuthology.orchestra.run.smithi086.stdout:- enp3s0f1 2024-04-25T00:09:26.834 INFO:teuthology.orchestra.run.smithi086.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T00:09:26.834 INFO:teuthology.orchestra.run.smithi086.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T00:09:26.834 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:09:26.965 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T00:09:27.397 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T00:09:27.824 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T00:09:27.878 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T00:09:27.894 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:09:27.937 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:09:28.181 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T00:09:28.196 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:09:28.239 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:09:28.526 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:09:28.601 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T00:09:28.616 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:09:28.651 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:09:28.668 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:09:28.820 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T00:09:28.946 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-common (238) ... 2024-04-25T00:09:28.987 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T00:09:29.003 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T00:09:29.045 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:09:29.456 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T00:09:29.472 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:09:29.515 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:09:29.885 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T00:09:29.900 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:09:29.935 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:09:29.936 INFO:teuthology.orchestra.run.smithi086.stdout:Adding user postgres to group ssl-cert 2024-04-25T00:09:30.190 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:09:30.190 INFO:teuthology.orchestra.run.smithi086.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T00:09:30.254 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T00:09:30.270 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:09:30.312 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:09:30.429 INFO:teuthology.orchestra.run.smithi086.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T00:09:30.429 INFO:teuthology.orchestra.run.smithi086.stdout:Removing obsolete dictionary files: 2024-04-25T00:09:30.607 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:09:30.665 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:09:30.951 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T00:09:30.966 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:09:31.009 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:09:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T00:09:31.538 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T00:09:31.553 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T00:09:31.588 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:09:31.638 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:09:31.924 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package modemmanager. 2024-04-25T00:09:31.940 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:09:31.982 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:09:32.054 INFO:teuthology.orchestra.run.smithi086.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T00:09:32.069 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T00:09:32.119 INFO:teuthology.orchestra.run.smithi086.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T00:09:32.119 INFO:teuthology.orchestra.run.smithi086.stdout:This user must also own the server process. 2024-04-25T00:09:32.119 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:09:32.119 INFO:teuthology.orchestra.run.smithi086.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T00:09:32.119 INFO:teuthology.orchestra.run.smithi086.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T00:09:32.120 INFO:teuthology.orchestra.run.smithi086.stdout:The default text search configuration will be set to "english". 2024-04-25T00:09:32.120 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:09:32.120 INFO:teuthology.orchestra.run.smithi086.stdout:Data page checksums are disabled. 2024-04-25T00:09:32.120 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:09:32.120 INFO:teuthology.orchestra.run.smithi086.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T00:09:32.121 INFO:teuthology.orchestra.run.smithi086.stdout:creating subdirectories ... ok 2024-04-25T00:09:32.121 INFO:teuthology.orchestra.run.smithi086.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T00:09:32.144 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default max_connections ... 100 2024-04-25T00:09:32.177 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default shared_buffers ... 128MB 2024-04-25T00:09:32.191 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default time zone ... Etc/UTC 2024-04-25T00:09:32.191 INFO:teuthology.orchestra.run.smithi086.stdout:creating configuration files ... ok 2024-04-25T00:09:32.355 INFO:teuthology.orchestra.run.smithi086.stdout:running bootstrap script ... ok 2024-04-25T00:09:32.648 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package network-manager. 2024-04-25T00:09:32.663 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:09:32.706 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:09:32.961 INFO:teuthology.orchestra.run.smithi086.stdout:performing post-bootstrap initialization ... ok 2024-04-25T00:09:33.380 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ppp. 2024-04-25T00:09:33.395 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T00:09:33.527 INFO:teuthology.orchestra.run.smithi086.stdout:syncing data to disk ... ok 2024-04-25T00:09:33.751 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:09:34.053 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T00:09:34.085 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T00:09:34.099 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T00:09:34.143 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T00:09:34.421 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T00:09:34.435 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T00:09:34.555 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T00:09:34.857 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T00:09:34.872 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T00:09:34.907 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T00:09:35.220 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T00:09:35.235 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:09:35.278 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:09:35.874 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T00:09:35.889 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T00:09:35.951 INFO:teuthology.orchestra.run.smithi083.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T00:09:35.981 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-common (238) ... 2024-04-25T00:09:36.448 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T00:09:36.464 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:09:36.501 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:09:37.098 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:09:37.249 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T00:09:37.375 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql (14+238) ... 2024-04-25T00:09:37.500 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:09:37.629 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:09:37.653 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql. 2024-04-25T00:09:37.666 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T00:09:37.703 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql (14+238) ... 2024-04-25T00:09:37.895 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:09:37.895 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:09:37.928 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T00:09:37.943 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T00:09:37.987 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T00:09:38.639 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T00:09:38.655 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T00:09:38.679 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:09:38.697 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T00:09:38.763 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T00:09:39.016 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T00:09:39.032 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T00:09:39.073 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:09:39.091 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:09:39.258 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:09:39.336 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package tcl. 2024-04-25T00:09:39.351 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T00:09:39.390 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T00:09:39.739 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T00:09:39.754 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T00:09:39.797 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:09:40.176 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T00:09:40.191 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T00:09:40.243 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T00:09:40.579 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T00:09:40.595 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T00:09:40.618 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T00:09:40.637 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:09:40.719 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:09:40.906 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T00:09:41.118 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T00:09:41.244 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:09:41.260 INFO:teuthology.orchestra.run.smithi086.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:09:41.267 DEBUG:teuthology.parallel:result is None 2024-04-25T00:09:41.371 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:09:41.497 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:09:41.622 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:09:41.774 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:09:42.703 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T00:09:42.796 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:09:43.173 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:09:43.173 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:09:43.840 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:09:43.941 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:09:44.042 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:09:44.169 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:09:44.445 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:09:44.571 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:09:44.697 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:09:45.353 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:09:45.479 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T00:09:45.574 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T00:09:45.639 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:09:45.765 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:09:45.916 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T00:09:46.025 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T00:09:46.126 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:09:46.208 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T00:09:46.278 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:09:46.404 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:09:46.580 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:09:46.741 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:09:46.857 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T00:09:46.992 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T00:09:47.109 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:09:47.716 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:09:47.716 INFO:teuthology.orchestra.run.smithi083.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T00:09:47.716 INFO:teuthology.orchestra.run.smithi083.stdout:which means they are currently configured by ifupdown: 2024-04-25T00:09:47.716 INFO:teuthology.orchestra.run.smithi083.stdout:- enp3s0f1 2024-04-25T00:09:47.716 INFO:teuthology.orchestra.run.smithi083.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T00:09:47.716 INFO:teuthology.orchestra.run.smithi083.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T00:09:47.716 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:09:47.849 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T00:09:48.262 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T00:09:48.703 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T00:09:49.489 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:09:49.717 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:09:49.926 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T00:09:50.154 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-common (238) ... 2024-04-25T00:09:51.504 INFO:teuthology.orchestra.run.smithi083.stdout:Adding user postgres to group ssl-cert 2024-04-25T00:09:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:09:51.747 INFO:teuthology.orchestra.run.smithi083.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T00:09:51.998 INFO:teuthology.orchestra.run.smithi083.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T00:09:51.998 INFO:teuthology.orchestra.run.smithi083.stdout:Removing obsolete dictionary files: 2024-04-25T00:09:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T00:09:53.187 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:09:53.532 INFO:teuthology.orchestra.run.smithi083.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T00:09:53.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T00:09:53.594 INFO:teuthology.orchestra.run.smithi083.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T00:09:53.594 INFO:teuthology.orchestra.run.smithi083.stdout:This user must also own the server process. 2024-04-25T00:09:53.594 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:09:53.594 INFO:teuthology.orchestra.run.smithi083.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T00:09:53.594 INFO:teuthology.orchestra.run.smithi083.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T00:09:53.595 INFO:teuthology.orchestra.run.smithi083.stdout:The default text search configuration will be set to "english". 2024-04-25T00:09:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:09:53.595 INFO:teuthology.orchestra.run.smithi083.stdout:Data page checksums are disabled. 2024-04-25T00:09:53.595 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:09:53.595 INFO:teuthology.orchestra.run.smithi083.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T00:09:53.595 INFO:teuthology.orchestra.run.smithi083.stdout:creating subdirectories ... ok 2024-04-25T00:09:53.595 INFO:teuthology.orchestra.run.smithi083.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T00:09:53.618 INFO:teuthology.orchestra.run.smithi083.stdout:selecting default max_connections ... 100 2024-04-25T00:09:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:selecting default shared_buffers ... 128MB 2024-04-25T00:09:53.665 INFO:teuthology.orchestra.run.smithi083.stdout:selecting default time zone ... Etc/UTC 2024-04-25T00:09:53.666 INFO:teuthology.orchestra.run.smithi083.stdout:creating configuration files ... ok 2024-04-25T00:09:53.831 INFO:teuthology.orchestra.run.smithi083.stdout:running bootstrap script ... ok 2024-04-25T00:09:54.460 INFO:teuthology.orchestra.run.smithi083.stdout:performing post-bootstrap initialization ... ok 2024-04-25T00:09:55.003 INFO:teuthology.orchestra.run.smithi083.stdout:syncing data to disk ... ok 2024-04-25T00:09:55.526 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T00:09:58.520 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:09:58.621 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T00:09:58.739 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql (14+238) ... 2024-04-25T00:09:58.865 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:09:58.990 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:09:59.267 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:09:59.267 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:10:00.060 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:10:00.177 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T00:10:00.505 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:10:00.640 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:10:01.741 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T00:10:01.874 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:10:02.416 INFO:teuthology.orchestra.run.smithi083.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:10:02.422 DEBUG:teuthology.parallel:result is None 2024-04-25T00:10:02.423 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:10:02.573 DEBUG:teuthology.orchestra.run.smithi083:> dpkg-query -W -f '${Version}' ceph 2024-04-25T00:10:02.595 INFO:teuthology.orchestra.run.smithi083.stdout:19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:10:02.595 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:10:02.595 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445-gd2c45d97-1jammy is installed. 2024-04-25T00:10:02.597 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:10:02.739 DEBUG:teuthology.orchestra.run.smithi086:> dpkg-query -W -f '${Version}' ceph 2024-04-25T00:10:02.763 INFO:teuthology.orchestra.run.smithi086.stdout:19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:10:02.763 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:10:02.763 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445-gd2c45d97-1jammy is installed. 2024-04-25T00:10:02.765 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T00:10:02.766 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:02.766 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:10:02.785 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:02.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:10:02.827 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T00:10:02.828 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:02.828 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:10:02.847 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:10:02.909 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:02.909 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:10:02.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:10:02.991 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T00:10:02.991 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:02.991 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:10:03.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:10:03.073 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:03.073 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:10:03.093 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:10:03.154 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T00:10:03.155 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:03.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:10:03.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:10:03.232 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:03.233 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:10:03.253 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:10:03.315 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T00:10:03.450 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T00:10:03.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /var/log/ceph 2024-04-25T00:10:03.453 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /var/log/ceph 2024-04-25T00:10:03.473 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T00:10:03.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T00:10:03.513 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T00:10:03.535 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T00:10:03.535 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T00:10:03.580 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T00:10:03.607 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T00:10:03.607 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T00:10:03.608 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T00:10:03.608 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:03.608 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T00:10:03.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T00:10:03.735 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:03.735 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T00:10:03.743 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T00:10:03.841 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T00:10:03.841 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'client session with non-allowable root'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T00:10:03.841 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/sessionmap}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7672231', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'client session with non-allowable root'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8528, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'sleep_before_teardown': 0, 'subset': '122/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi083.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ6JFaHLmbX/T4deYFAUKteDyTGelz4imH2PGSNYJRqmNTyBQqbF3IW2OFEKQJs3Kjrt8+7eiCdNuXAcTYQGVWE=', 'smithi086.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEZht8wji+G4TbU4ZJ+WMQCY6etPdDOYxTOBqVjUPorZrNlOpUn/QnacYhcC2qB+rR8wH8st4rbUbnQhSTJL9kY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'client session with non-allowable root'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_sessionmap']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T00:10:03.842 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T00:10:03.845 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T00:10:03.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T00:10:03.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T00:10:03.915 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:03.915 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:10:03.958 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:10:03.958 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-04-25T00:10:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:10:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:10:04.011 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 891 Links: 1 2024-04-25T00:10:04.011 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:10:04.011 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-25 00:03:21.619796926 +0000 2024-04-25T00:10:04.011 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-25 00:03:21.355799285 +0000 2024-04-25T00:10:04.011 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-25 00:03:21.355799285 +0000 2024-04-25T00:10:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-25 00:03:21.355799285 +0000 2024-04-25T00:10:04.012 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:10:04.072 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-25T00:10:04.072 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-25T00:10:04.072 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000379642 s, 1.3 MB/s 2024-04-25T00:10:04.074 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:10:04.124 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-04-25T00:10:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:10:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:10:04.179 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 909 Links: 1 2024-04-25T00:10:04.179 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:10:04.179 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-25 00:03:22.159792103 +0000 2024-04-25T00:10:04.179 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-25 00:03:21.899794427 +0000 2024-04-25T00:10:04.179 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-25 00:03:21.899794427 +0000 2024-04-25T00:10:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-25 00:03:21.899794427 +0000 2024-04-25T00:10:04.179 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:10:04.239 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-25T00:10:04.239 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-25T00:10:04.239 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000333267 s, 1.5 MB/s 2024-04-25T00:10:04.241 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:10:04.292 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-04-25T00:10:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:10:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:10:04.343 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 926 Links: 1 2024-04-25T00:10:04.343 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:10:04.343 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-25 00:03:22.719787102 +0000 2024-04-25T00:10:04.343 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-25 00:03:22.439789603 +0000 2024-04-25T00:10:04.343 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-25 00:03:22.439789603 +0000 2024-04-25T00:10:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-25 00:03:22.439789603 +0000 2024-04-25T00:10:04.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:10:04.399 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-25T00:10:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-25T00:10:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000366308 s, 1.4 MB/s 2024-04-25T00:10:04.401 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:10:04.452 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-04-25T00:10:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:10:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:10:04.503 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 944 Links: 1 2024-04-25T00:10:04.503 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:10:04.503 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-25 00:03:23.299781921 +0000 2024-04-25T00:10:04.503 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-25 00:03:23.015784457 +0000 2024-04-25T00:10:04.503 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-25 00:03:23.015784457 +0000 2024-04-25T00:10:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-25 00:03:23.015784457 +0000 2024-04-25T00:10:04.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:10:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-25T00:10:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-25T00:10:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.00036839 s, 1.4 MB/s 2024-04-25T00:10:04.566 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:10:04.616 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:10:04.616 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:04.616 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:10:04.625 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:10:04.625 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-04-25T00:10:04.675 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:10:04.676 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:10:04.676 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 891 Links: 1 2024-04-25T00:10:04.676 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:10:04.676 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-25 00:03:21.657855144 +0000 2024-04-25T00:10:04.676 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-25 00:03:21.385857573 +0000 2024-04-25T00:10:04.676 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-25 00:03:21.385857573 +0000 2024-04-25T00:10:04.676 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-25 00:03:21.385857573 +0000 2024-04-25T00:10:04.676 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:10:04.737 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T00:10:04.737 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T00:10:04.737 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000420107 s, 1.2 MB/s 2024-04-25T00:10:04.739 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:10:04.793 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-04-25T00:10:04.843 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:10:04.844 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:10:04.844 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 909 Links: 1 2024-04-25T00:10:04.844 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:10:04.844 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-25 00:03:22.249849856 +0000 2024-04-25T00:10:04.844 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-25 00:03:21.969852357 +0000 2024-04-25T00:10:04.844 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-25 00:03:21.969852357 +0000 2024-04-25T00:10:04.844 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-25 00:03:21.969852357 +0000 2024-04-25T00:10:04.844 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:10:04.905 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T00:10:04.906 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T00:10:04.906 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000381439 s, 1.3 MB/s 2024-04-25T00:10:04.908 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:10:04.961 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-04-25T00:10:05.015 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:10:05.015 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:10:05.015 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 925 Links: 1 2024-04-25T00:10:05.015 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:10:05.016 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-25 00:03:22.833844639 +0000 2024-04-25T00:10:05.016 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-25 00:03:22.561847069 +0000 2024-04-25T00:10:05.016 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-25 00:03:22.561847069 +0000 2024-04-25T00:10:05.016 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-25 00:03:22.561847069 +0000 2024-04-25T00:10:05.017 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:10:05.078 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T00:10:05.078 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T00:10:05.078 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000379825 s, 1.3 MB/s 2024-04-25T00:10:05.080 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:10:05.134 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-04-25T00:10:05.188 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:10:05.188 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:10:05.189 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 942 Links: 1 2024-04-25T00:10:05.189 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:10:05.189 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-25 00:03:23.461839030 +0000 2024-04-25T00:10:05.189 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-25 00:03:23.177841566 +0000 2024-04-25T00:10:05.189 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-25 00:03:23.177841566 +0000 2024-04-25T00:10:05.189 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-25 00:03:23.177841566 +0000 2024-04-25T00:10:05.190 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:10:05.249 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T00:10:05.250 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T00:10:05.250 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000395775 s, 1.3 MB/s 2024-04-25T00:10:05.251 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:10:05.305 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:10:05.306 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T00:10:05.306 INFO:tasks.ceph:Generating config... 2024-04-25T00:10:05.307 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T00:10:05.307 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T00:10:05.307 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T00:10:05.307 INFO:tasks.ceph:[client] pid file = 2024-04-25T00:10:05.307 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T00:10:05.308 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T00:10:05.308 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T00:10:05.308 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T00:10:05.308 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T00:10:05.308 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T00:10:05.308 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T00:10:05.308 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T00:10:05.308 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T00:10:05.308 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T00:10:05.308 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T00:10:05.308 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T00:10:05.308 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T00:10:05.308 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T00:10:05.308 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T00:10:05.309 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T00:10:05.309 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T00:10:05.309 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T00:10:05.309 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T00:10:05.309 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T00:10:05.309 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T00:10:05.309 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T00:10:05.309 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T00:10:05.309 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T00:10:05.309 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T00:10:05.309 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T00:10:05.309 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T00:10:05.309 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T00:10:05.309 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T00:10:05.309 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T00:10:05.310 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T00:10:05.310 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T00:10:05.310 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T00:10:05.310 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T00:10:05.310 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T00:10:05.310 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T00:10:05.355 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T00:10:05.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T00:10:05.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T00:10:05.486 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.83'), ('mon.b', '172.21.15.86'), ('mon.c', '[v2:172.21.15.86:3301,v1:172.21.15.86:6790]')] 2024-04-25T00:10:05.487 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.83,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T00:10:05.487 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:05.488 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T00:10:05.538 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.83 --add b 172.21.15.86 --addv c '[v2:172.21.15.86:3301,v1:172.21.15.86:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:10:05.630 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:10:05.630 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: generated fsid d58ae83a-ad72-4e18-824a-86e779979c35 2024-04-25T00:10:05.630 INFO:teuthology.orchestra.run.smithi083.stdout:setting min_mon_release = quincy 2024-04-25T00:10:05.630 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 0 2024-04-25T00:10:05.630 INFO:teuthology.orchestra.run.smithi083.stdout:fsid d58ae83a-ad72-4e18-824a-86e779979c35 2024-04-25T00:10:05.630 INFO:teuthology.orchestra.run.smithi083.stdout:last_changed 2024-04-25T00:10:05.632416+0000 2024-04-25T00:10:05.630 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-04-25T00:10:05.632416+0000 2024-04-25T00:10:05.630 INFO:teuthology.orchestra.run.smithi083.stdout:min_mon_release 17 (quincy) 2024-04-25T00:10:05.630 INFO:teuthology.orchestra.run.smithi083.stdout:election_strategy: 1 2024-04-25T00:10:05.631 INFO:teuthology.orchestra.run.smithi083.stdout:0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-25T00:10:05.631 INFO:teuthology.orchestra.run.smithi083.stdout:1: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-04-25T00:10:05.631 INFO:teuthology.orchestra.run.smithi083.stdout:2: [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] mon.c 2024-04-25T00:10:05.631 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T00:10:05.632 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T00:10:05.687 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d58ae83a-ad72-4e18-824a-86e779979c35... 2024-04-25T00:10:05.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:10:05.734 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:10:05.773 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-04-25T00:10:05.773 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2024-04-25T00:10:05.773 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:10:05.773 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2024-04-25T00:10:05.773 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.773 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2024-04-25T00:10:05.773 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.774 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2024-04-25T00:10:05.774 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.774 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2024-04-25T00:10:05.774 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2024-04-25T00:10:05.774 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.774 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2024-04-25T00:10:05.775 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2024-04-25T00:10:05.775 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.775 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:10:05.775 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2024-04-25T00:10:05.775 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.775 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:10:05.775 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:10:05.776 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.776 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2024-04-25T00:10:05.776 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.776 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2024-04-25T00:10:05.776 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2024-04-25T00:10:05.776 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:10:05.776 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:10:05.777 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2024-04-25T00:10:05.777 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:10:05.777 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2024-04-25T00:10:05.777 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:10:05.777 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:10:05.777 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2024-04-25T00:10:05.777 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.777 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:10:05.777 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.777 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2024-04-25T00:10:05.777 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.778 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2024-04-25T00:10:05.778 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.778 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2024-04-25T00:10:05.778 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2024-04-25T00:10:05.778 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2024-04-25T00:10:05.778 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.83,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]" 2024-04-25T00:10:05.778 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on skipped message = False 2024-04-25T00:10:05.778 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = d58ae83a-ad72-4e18-824a-86e779979c35 2024-04-25T00:10:05.778 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.778 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2024-04-25T00:10:05.778 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2024-04-25T00:10:05.779 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.779 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2024-04-25T00:10:05.779 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2024-04-25T00:10:05.779 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:10:05.779 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.779 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2024-04-25T00:10:05.779 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2024-04-25T00:10:05.779 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.779 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2024-04-25T00:10:05.779 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2024-04-25T00:10:05.779 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2024-04-25T00:10:05.779 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2024-04-25T00:10:05.779 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.779 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2024-04-25T00:10:05.780 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 20 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 20 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 10 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:10:05.781 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2024-04-25T00:10:05.782 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-04-25T00:10:05.782 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-04-25T00:10:05.782 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:10:05.782 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:10:05.782 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2024-04-25T00:10:05.782 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-04-25T00:10:05.782 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.782 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2024-04-25T00:10:05.782 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:10:05.782 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-04-25T00:10:05.782 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2024-04-25T00:10:05.782 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-04-25T00:10:05.782 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2024-04-25T00:10:05.782 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2024-04-25T00:10:05.783 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: mon down mkfs grace = 300 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: mon op complaint time = 120 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: client mount timeout = 600 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:10:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = "" 2024-04-25T00:10:05.785 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-04-25T00:10:05.785 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-04-25T00:10:05.785 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2024-04-25T00:10:05.785 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2024-04-25T00:10:05.785 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2024-04-25T00:10:05.785 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2024-04-25T00:10:05.785 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2024-04-25T00:10:05.785 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2024-04-25T00:10:05.785 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2024-04-25T00:10:05.785 INFO:teuthology.orchestra.run.smithi083.stdout:[mds] 2024-04-25T00:10:05.785 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds = 20 2024-04-25T00:10:05.785 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds balancer = 20 2024-04-25T00:10:05.785 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:10:05.785 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug frag = True 2024-04-25T00:10:05.786 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug scatterstat = True 2024-04-25T00:10:05.786 INFO:teuthology.orchestra.run.smithi083.stdout: mds op complaint time = 180 2024-04-25T00:10:05.786 INFO:teuthology.orchestra.run.smithi083.stdout: mds verify scatter = True 2024-04-25T00:10:05.786 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-04-25T00:10:05.786 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-04-25T00:10:05.786 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-04-25T00:10:05.786 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-25T00:10:05.787 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-04-25T00:10:05.787 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:10:05.787 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-04-25T00:10:05.787 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.787 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-04-25T00:10:05.787 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.787 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-04-25T00:10:05.787 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.787 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-04-25T00:10:05.787 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-04-25T00:10:05.787 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.787 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-04-25T00:10:05.787 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-04-25T00:10:05.788 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.788 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:10:05.788 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-04-25T00:10:05.788 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.788 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:10:05.788 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:10:05.788 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.788 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-04-25T00:10:05.788 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.788 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-04-25T00:10:05.789 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-04-25T00:10:05.789 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:10:05.789 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:10:05.789 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-04-25T00:10:05.789 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:10:05.789 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-04-25T00:10:05.789 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:10:05.789 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:10:05.789 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-04-25T00:10:05.789 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.789 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:10:05.789 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.789 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-04-25T00:10:05.789 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.790 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-04-25T00:10:05.790 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.790 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-04-25T00:10:05.790 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-04-25T00:10:05.790 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-04-25T00:10:05.790 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.83,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]" 2024-04-25T00:10:05.790 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on skipped message = False 2024-04-25T00:10:05.790 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = d58ae83a-ad72-4e18-824a-86e779979c35 2024-04-25T00:10:05.790 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.790 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-04-25T00:10:05.790 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-04-25T00:10:05.790 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.790 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-04-25T00:10:05.791 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-04-25T00:10:05.791 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:10:05.791 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.791 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-04-25T00:10:05.791 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-04-25T00:10:05.791 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.791 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-04-25T00:10:05.791 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-04-25T00:10:05.791 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-04-25T00:10:05.791 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-04-25T00:10:05.791 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.791 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-04-25T00:10:05.791 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-04-25T00:10:05.792 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.792 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:10:05.792 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.792 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-04-25T00:10:05.792 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.792 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-04-25T00:10:05.792 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.792 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:10:05.792 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-04-25T00:10:05.792 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.792 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-04-25T00:10:05.792 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-04-25T00:10:05.792 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 20 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 20 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 10 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-04-25T00:10:05.793 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:10:05.794 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-04-25T00:10:05.794 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-25T00:10:05.794 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-25T00:10:05.794 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:10:05.794 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:10:05.794 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-04-25T00:10:05.794 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-25T00:10:05.794 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.794 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-04-25T00:10:05.794 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:10:05.794 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-25T00:10:05.794 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-04-25T00:10:05.794 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-25T00:10:05.794 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.795 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-25T00:10:05.796 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:10:05.797 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-04-25T00:10:05.797 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-25T00:10:05.797 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-25T00:10:05.797 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-04-25T00:10:05.797 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-04-25T00:10:05.797 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-04-25T00:10:05.797 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-25T00:10:05.797 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-04-25T00:10:05.797 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-04-25T00:10:05.797 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-04-25T00:10:05.797 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-04-25T00:10:05.797 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-04-25T00:10:05.797 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-04-25T00:10:05.797 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:10:05.798 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-04-25T00:10:05.798 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = True 2024-04-25T00:10:05.798 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-04-25T00:10:05.798 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = True 2024-04-25T00:10:05.798 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-25T00:10:05.798 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-25T00:10:05.798 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-25T00:10:05.800 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T00:10:05.800 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T00:10:05.899 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T00:10:05.899 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:05.900 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T00:10:05.908 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:05.908 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T00:10:05.958 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:10:05.958 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:05.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T00:10:05.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T00:10:06.029 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:06.030 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:10:06.079 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:10:06.079 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:06.079 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T00:10:06.079 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T00:10:06.114 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:06.114 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:10:06.167 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T00:10:06.167 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T00:10:06.167 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T00:10:06.221 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T00:10:06.226 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T00:10:06.284 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T00:10:06.300 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T00:10:06.300 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T00:10:06.354 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T00:10:06.358 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T00:10:06.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T00:10:06.472 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T00:10:06.476 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T00:10:06.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T00:10:06.551 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T00:10:06.556 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T00:10:06.575 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T00:10:06.672 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T00:10:06.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T00:10:06.696 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T00:10:06.696 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T00:10:06.737 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T00:10:06.754 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T00:10:06.835 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T00:10:06.853 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T00:10:06.894 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T00:10:06.910 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T00:10:06.997 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T00:10:07.013 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T00:10:07.014 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T00:10:07.014 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T00:10:07.033 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:10:07.033 INFO:tasks.ceph:role: osd.0 2024-04-25T00:10:07.033 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:10:07.033 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T00:10:07.101 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:10:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:10:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:10:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:10:07.101 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:10:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:10:07.101 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:10:07.101 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:10:07.101 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:10:07.101 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:10:07.501 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-25T00:10:07.505 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-25T00:10:07.505 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T00:10:07.601 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T00:10:07.617 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:10:07.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:10:07.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T00:10:07.682 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:10:07.682 INFO:tasks.ceph:role: osd.1 2024-04-25T00:10:07.682 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:10:07.682 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T00:10:07.753 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:10:07.753 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:10:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:10:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:10:07.754 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:10:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:10:07.754 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:10:07.754 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:10:07.754 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:10:07.754 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:10:08.166 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-25T00:10:08.169 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-25T00:10:08.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T00:10:08.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T00:10:08.311 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:10:08.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:10:08.312 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T00:10:08.374 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:10:08.375 INFO:tasks.ceph:role: osd.2 2024-04-25T00:10:08.375 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:10:08.375 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T00:10:08.446 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:10:08.447 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:10:08.447 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:10:08.447 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:10:08.447 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:10:08.447 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:10:08.447 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:10:08.447 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:10:08.447 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:10:08.447 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:10:08.865 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-25T00:10:08.870 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-25T00:10:08.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T00:10:08.994 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T00:10:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:10:09.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:10:09.014 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T00:10:09.078 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:10:09.078 INFO:tasks.ceph:role: osd.3 2024-04-25T00:10:09.078 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:10:09.078 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T00:10:09.152 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:10:09.153 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:10:09.153 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:10:09.153 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:10:09.153 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:10:09.153 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:10:09.154 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:10:09.154 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:10:09.154 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:10:09.154 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:10:09.518 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-25T00:10:09.520 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-25T00:10:09.521 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T00:10:09.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T00:10:09.631 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:10:09.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:10:09.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:10:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:09.759+0000 7f2f938937c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T00:10:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:09.763+0000 7f2f938937c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T00:10:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:09.763+0000 7f2f938937c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T00:10:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:09.763+0000 7f2f938937c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T00:10:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:09.763+0000 7f2f938937c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T00:10:09.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:09.767+0000 7f2f938937c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T00:10:12.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T00:10:12.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:10:12.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:12.935+0000 7faa8c4c17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T00:10:12.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:12.935+0000 7faa8c4c17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T00:10:12.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:12.935+0000 7faa8c4c17c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T00:10:12.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:12.935+0000 7faa8c4c17c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T00:10:12.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:12.935+0000 7faa8c4c17c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T00:10:12.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:12.939+0000 7faa8c4c17c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T00:10:15.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T00:10:16.000 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:10:16.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:16.095+0000 7fdf4b7ab7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T00:10:16.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:16.095+0000 7fdf4b7ab7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T00:10:16.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:16.095+0000 7fdf4b7ab7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T00:10:16.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:16.095+0000 7fdf4b7ab7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T00:10:16.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:16.095+0000 7fdf4b7ab7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T00:10:16.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:16.099+0000 7fdf4b7ab7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T00:10:19.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T00:10:19.211 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:10:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:19.307+0000 7fa6b3da97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T00:10:19.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:19.311+0000 7fa6b3da97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T00:10:19.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:19.311+0000 7fa6b3da97c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T00:10:19.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:19.311+0000 7fa6b3da97c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T00:10:19.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:19.311+0000 7fa6b3da97c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T00:10:19.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:19.311+0000 7fa6b3da97c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T00:10:22.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T00:10:22.393 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T00:10:22.415 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:10:22.415 INFO:tasks.ceph:role: osd.4 2024-04-25T00:10:22.415 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:10:22.415 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T00:10:22.487 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:10:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:10:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:10:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:10:22.487 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:10:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:10:22.487 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:10:22.487 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:10:22.488 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:10:22.488 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:10:22.888 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-25T00:10:22.891 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-25T00:10:22.891 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T00:10:22.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T00:10:23.007 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:10:23.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:10:23.008 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T00:10:23.075 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:10:23.075 INFO:tasks.ceph:role: osd.5 2024-04-25T00:10:23.075 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:10:23.075 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T00:10:23.147 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:10:23.147 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:10:23.147 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:10:23.147 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:10:23.147 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:10:23.147 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:10:23.147 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:10:23.147 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:10:23.147 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:10:23.147 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:10:23.553 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-25T00:10:23.555 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-25T00:10:23.556 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T00:10:23.691 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T00:10:23.708 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:10:23.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:10:23.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T00:10:23.772 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:10:23.772 INFO:tasks.ceph:role: osd.6 2024-04-25T00:10:23.772 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:10:23.772 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T00:10:23.843 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:10:23.843 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:10:23.843 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:10:23.843 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:10:23.843 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:10:23.843 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:10:23.843 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:10:23.843 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:10:23.843 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:10:23.843 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:10:24.259 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-25T00:10:24.824 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-25T00:10:24.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T00:10:24.900 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T00:10:24.960 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:10:24.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:10:24.962 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T00:10:25.024 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:10:25.024 INFO:tasks.ceph:role: osd.7 2024-04-25T00:10:25.024 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:10:25.024 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T00:10:25.095 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:10:25.095 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:10:25.095 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:10:25.095 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:10:25.095 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:10:25.095 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:10:25.095 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:10:25.096 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:10:25.096 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:10:25.096 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:10:25.493 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-25T00:10:25.495 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-25T00:10:25.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T00:10:25.592 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T00:10:25.609 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:10:25.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:10:25.610 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:10:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:25.729+0000 7f6c03c2d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T00:10:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:25.737+0000 7f6c03c2d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T00:10:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:25.737+0000 7f6c03c2d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T00:10:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:25.737+0000 7f6c03c2d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T00:10:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:25.737+0000 7f6c03c2d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T00:10:25.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:25.745+0000 7f6c03c2d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T00:10:28.786 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T00:10:28.807 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:10:28.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:28.897+0000 7fa5372f57c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T00:10:28.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:28.897+0000 7fa5372f57c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T00:10:28.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:28.897+0000 7fa5372f57c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T00:10:28.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:28.897+0000 7fa5372f57c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T00:10:28.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:28.897+0000 7fa5372f57c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T00:10:28.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:28.901+0000 7fa5372f57c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T00:10:31.911 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T00:10:31.931 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:10:32.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:32.021+0000 7f0610a307c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T00:10:32.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:32.021+0000 7f0610a307c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T00:10:32.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:32.021+0000 7f0610a307c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T00:10:32.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:32.021+0000 7f0610a307c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T00:10:32.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:32.021+0000 7f0610a307c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T00:10:32.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:32.025+0000 7f0610a307c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T00:10:33.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:10:33.847 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:10:35.077 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T00:10:35.097 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:10:35.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:35.189+0000 7f54a1f207c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T00:10:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:35.189+0000 7f54a1f207c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T00:10:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:35.189+0000 7f54a1f207c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T00:10:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:35.189+0000 7f54a1f207c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T00:10:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:35.189+0000 7f54a1f207c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T00:10:35.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:35.193+0000 7f54a1f207c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T00:10:38.233 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T00:10:38.253 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T00:10:38.253 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:38.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T00:10:38.275 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:38.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T00:10:38.336 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:38.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T00:10:38.399 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:38.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T00:10:38.463 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:38.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T00:10:38.523 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:38.524 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T00:10:38.587 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:38.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T00:10:38.652 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:38.652 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T00:10:38.673 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:38.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T00:10:38.736 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:38.736 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T00:10:38.799 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:38.800 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T00:10:38.863 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:38.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T00:10:38.927 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:38.927 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T00:10:38.991 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:38.991 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T00:10:39.051 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:39.051 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T00:10:39.059 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:39.059 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T00:10:39.111 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:39.112 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T00:10:39.120 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:39.120 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T00:10:39.171 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T00:10:39.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T00:10:39.174 DEBUG:teuthology.orchestra.run.smithi086:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T00:10:39.219 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.y] 2024-04-25T00:10:39.219 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylm9VRpDRAA5XUtYwjiWBpF1ObJ/Y5g0w== 2024-04-25T00:10:39.219 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2024-04-25T00:10:39.219 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylmytlRFRAA3vHLqK3vHSB+8+TZHAuj7Q== 2024-04-25T00:10:39.219 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2024-04-25T00:10:39.219 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylmCqZXHBAAPy5JGbEU11i4gc3gFtbGIQ== 2024-04-25T00:10:39.219 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.0] 2024-04-25T00:10:39.219 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBhnylmMOiqLRAAdqrCTz4KfghoJ7cZc+b4qg== 2024-04-25T00:10:39.220 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.1] 2024-04-25T00:10:39.220 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBknylmlp8OOBAAl+y4vBrCzDyV/ne9qICQ+w== 2024-04-25T00:10:39.220 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.2] 2024-04-25T00:10:39.220 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBonylm85MOBhAA3ciY1UT/Qg4Pe72OA0IvnA== 2024-04-25T00:10:39.220 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.3] 2024-04-25T00:10:39.220 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBrnylmnAbDEhAAkCQXjbiKINiftX71HBKvQA== 2024-04-25T00:10:39.220 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.x] 2024-04-25T00:10:39.220 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylm6TMqERAAiPL81HRFrlz4ku20HemS4w== 2024-04-25T00:10:39.220 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2024-04-25T00:10:39.220 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylm088MIRAApJvvePXyFoO8JnVErd57zQ== 2024-04-25T00:10:39.220 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2024-04-25T00:10:39.220 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylmiJZGKBAA5CBwH/cA+LOAj+inS/Hx2g== 2024-04-25T00:10:39.220 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.4] 2024-04-25T00:10:39.220 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBxnylmxhy2KxAAs6qjxZA+Fj5umHU/4FVqGw== 2024-04-25T00:10:39.221 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.5] 2024-04-25T00:10:39.221 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQB0nylmGfSlNRAA45tuuh+5ISOMlhaFdCE5rQ== 2024-04-25T00:10:39.221 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.6] 2024-04-25T00:10:39.221 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQB4nylmouyBARAAz4s4S1BSiDU9YxQ/Vj0d0A== 2024-04-25T00:10:39.221 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.7] 2024-04-25T00:10:39.221 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQB7nylmtIh0CxAApWJLvC8Zlt298VJ55gPrqw== 2024-04-25T00:10:39.221 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-25T00:10:39.221 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylmOqooLBAApte+X6YkRmrpe12YdjYekg== 2024-04-25T00:10:39.221 INFO:teuthology.orchestra.run.smithi083.stdout:[client.2] 2024-04-25T00:10:39.221 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylmHw3+MRAAciXRmexPxmrf7U2jKwpZQQ== 2024-04-25T00:10:39.221 INFO:teuthology.orchestra.run.smithi083.stdout:[client.1] 2024-04-25T00:10:39.221 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylmDH93NRAASQzJNC3nI0NeU8cv951eBA== 2024-04-25T00:10:39.221 INFO:teuthology.orchestra.run.smithi083.stdout:[client.3] 2024-04-25T00:10:39.221 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBfnylmIsMCABAAu+yzPRupFFqA/yH4GSZ+8w== 2024-04-25T00:10:39.233 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.y] 2024-04-25T00:10:39.233 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBenylm9VRpDRAA5XUtYwjiWBpF1ObJ/Y5g0w== 2024-04-25T00:10:39.233 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-25T00:10:39.233 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBenylmytlRFRAA3vHLqK3vHSB+8+TZHAuj7Q== 2024-04-25T00:10:39.233 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBenylmCqZXHBAAPy5JGbEU11i4gc3gFtbGIQ== 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.0] 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBhnylmMOiqLRAAdqrCTz4KfghoJ7cZc+b4qg== 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.1] 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBknylmlp8OOBAAl+y4vBrCzDyV/ne9qICQ+w== 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.2] 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBonylm85MOBhAA3ciY1UT/Qg4Pe72OA0IvnA== 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.3] 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBrnylmnAbDEhAAkCQXjbiKINiftX71HBKvQA== 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.x] 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBenylm6TMqERAAiPL81HRFrlz4ku20HemS4w== 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBenylm088MIRAApJvvePXyFoO8JnVErd57zQ== 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-04-25T00:10:39.234 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBenylmiJZGKBAA5CBwH/cA+LOAj+inS/Hx2g== 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.4] 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBxnylmxhy2KxAAs6qjxZA+Fj5umHU/4FVqGw== 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.5] 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB0nylmGfSlNRAA45tuuh+5ISOMlhaFdCE5rQ== 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.6] 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB4nylmouyBARAAz4s4S1BSiDU9YxQ/Vj0d0A== 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.7] 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB7nylmtIh0CxAApWJLvC8Zlt298VJ55gPrqw== 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBenylmOqooLBAApte+X6YkRmrpe12YdjYekg== 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout:[client.2] 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBenylmHw3+MRAAciXRmexPxmrf7U2jKwpZQQ== 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout:[client.1] 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBenylmDH93NRAASQzJNC3nI0NeU8cv951eBA== 2024-04-25T00:10:39.235 INFO:teuthology.orchestra.run.smithi086.stdout:[client.3] 2024-04-25T00:10:39.236 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBfnylmIsMCABAAu+yzPRupFFqA/yH4GSZ+8w== 2024-04-25T00:10:39.236 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:10:39.272 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:10:39.335 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:10:39.338 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:10:39.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:10:39.398 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:10:39.454 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:39.457 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:39.510 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:39.513 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:39.564 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:39.567 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:39.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:39.666 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:39.719 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:10:39.722 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:10:39.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:10:39.778 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:10:39.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:10:39.846 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:10:39.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:39.912 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:39.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:39.983 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:40.070 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:40.073 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:40.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:40.130 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:10:40.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:10:40.235 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:10:40.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:10:40.301 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:10:40.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:10:40.379 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:10:40.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:10:40.443 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:10:40.508 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T00:10:40.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T00:10:40.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T00:10:40.647 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T00:10:40.670 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T00:10:40.691 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T00:10:40.804 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T00:10:40.825 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T00:10:40.887 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T00:10:40.999 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T00:10:41.019 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:10:41.022 DEBUG:teuthology.orchestra.run.smithi086:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:10:41.067 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T00:10:41.067 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T00:10:41.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T00:10:41.070 INFO:tasks.ceph.mon.a:Started 2024-04-25T00:10:41.070 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T00:10:41.071 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T00:10:41.114 INFO:tasks.ceph.mon.b:Started 2024-04-25T00:10:41.114 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T00:10:41.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T00:10:41.117 INFO:tasks.ceph.mon.c:Started 2024-04-25T00:10:41.117 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T00:10:41.118 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T00:10:41.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T00:10:41.121 INFO:tasks.ceph.mgr.y:Started 2024-04-25T00:10:41.121 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T00:10:41.121 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T00:10:41.125 INFO:tasks.ceph.mgr.x:Started 2024-04-25T00:10:41.125 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T00:10:41.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T00:10:41.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.262+0000 7f336be00640 1 Processor -- start 2024-04-25T00:10:41.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.262+0000 7f336be00640 1 -- start start 2024-04-25T00:10:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.262+0000 7f336be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f336c144650 0x7f336c15e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.262+0000 7f336be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f336c15eeb0 0x7f336c071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.262+0000 7f336be00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f336c06d970 con 0x7f336c072130 2024-04-25T00:10:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.262+0000 7f336be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f336c15f430 con 0x7f336c15eeb0 2024-04-25T00:10:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.262+0000 7f336be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f336c15f710 con 0x7f336c144650 2024-04-25T00:10:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.262+0000 7f336b400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f336c072130 0x7f336c15b370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54110/0 (socket says 172.21.15.83:54110) 2024-04-25T00:10:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.262+0000 7f336b400640 1 -- 172.21.15.83:0/3053087983 learned_addr learned my addr 172.21.15.83:0/3053087983 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:10:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.262+0000 7f336a000640 1 --2- 172.21.15.83:0/3053087983 >> v2:172.21.15.86:3300/0 conn(0x7f336c15eeb0 0x7f336c071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.262+0000 7f336a000640 1 -- 172.21.15.83:0/3053087983 >> v2:172.21.15.86:3300/0 conn(0x7f336c15eeb0 msgr2=0x7f336c071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T00:10:41.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.262+0000 7f336a000640 1 -- 172.21.15.83:0/3053087983 >> v2:172.21.15.86:3300/0 conn(0x7f336c15eeb0 msgr2=0x7f336c071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T00:10:41.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.262+0000 7f336a000640 1 --2- 172.21.15.83:0/3053087983 >> v2:172.21.15.86:3300/0 conn(0x7f336c15eeb0 0x7f336c071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:10:41.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.262+0000 7f336aa00640 1 --2- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f336c144650 0x7f336c15e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.266+0000 7f336a000640 1 --2- 172.21.15.83:0/3053087983 >> v2:172.21.15.86:3300/0 conn(0x7f336c15eeb0 0x7f336c071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:10:41.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.266+0000 7f336aa00640 1 -- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f336c144650 msgr2=0x7f336c15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T00:10:41.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.266+0000 7f336aa00640 1 -- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f336c144650 msgr2=0x7f336c15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T00:10:41.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.266+0000 7f336aa00640 1 --2- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f336c144650 0x7f336c15e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:10:41.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.266+0000 7f336aa00640 1 --2- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f336c144650 0x7f336c15e970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:10:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.430+0000 7f3369600640 1 -- 172.21.15.83:0/3053087983 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 179021922 0 0) 0x7f336c06d970 con 0x7f336c072130 2024-04-25T00:10:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.430+0000 7f3369600640 1 -- 172.21.15.83:0/3053087983 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3350003530 con 0x7f336c072130 2024-04-25T00:10:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f3369600640 1 -- 172.21.15.83:0/3053087983 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3747454516 0 0) 0x7f3350003530 con 0x7f336c072130 2024-04-25T00:10:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f3369600640 1 -- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f336c144650 msgr2=0x7f336c15e970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:10:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f3369600640 1 --2- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f336c144650 0x7f336c15e970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f3369600640 1 -- 172.21.15.83:0/3053087983 >> v2:172.21.15.86:3300/0 conn(0x7f336c15eeb0 msgr2=0x7f336c071bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:10:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f3369600640 1 --2- 172.21.15.83:0/3053087983 >> v2:172.21.15.86:3300/0 conn(0x7f336c15eeb0 0x7f336c071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f3369600640 1 -- 172.21.15.83:0/3053087983 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c0cb9d0 con 0x7f336c072130 2024-04-25T00:10:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f3369600640 1 -- 172.21.15.83:0/3053087983 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f3364001680 con 0x7f336c072130 2024-04-25T00:10:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f3369600640 1 -- 172.21.15.83:0/3053087983 >> v1:172.21.15.83:6789/0 conn(0x7f336c072130 legacy=0x7f336c15b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f3369600640 1 --2- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33500039c0 0x7f3350003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f3369600640 1 --2- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f336c144650 0x7f3350004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f3369600640 1 --2- 172.21.15.83:0/3053087983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3350004d50 0x7f33500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f3369600640 1 -- 172.21.15.83:0/3053087983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f336c15f430 con 0x7f3350004d50 2024-04-25T00:10:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f3369600640 1 -- 172.21.15.83:0/3053087983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f336c15f710 con 0x7f33500039c0 2024-04-25T00:10:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f3369600640 1 -- 172.21.15.83:0/3053087983 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f335000bb00 con 0x7f336c144650 2024-04-25T00:10:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f336b400640 1 --2- 172.21.15.83:0/3053087983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3350004d50 0x7f33500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f336aa00640 1 --2- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33500039c0 0x7f3350003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f336a000640 1 --2- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f336c144650 0x7f3350004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f336b400640 1 -- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f336c144650 msgr2=0x7f3350004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f336b400640 1 --2- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f336c144650 0x7f3350004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f336b400640 1 -- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33500039c0 msgr2=0x7f3350003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f336b400640 1 --2- 172.21.15.83:0/3053087983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33500039c0 0x7f3350003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.434+0000 7f336b400640 1 -- 172.21.15.83:0/3053087983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3350003530 con 0x7f3350004d50 2024-04-25T00:10:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.438+0000 7f336b400640 1 --2- 172.21.15.83:0/3053087983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3350004d50 0x7f33500051d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3364001060 tx=0x7f336401f2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504c7a8858d09dec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.438+0000 7f3369600640 1 -- 172.21.15.83:0/3053087983 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364007d40 con 0x7f3350004d50 2024-04-25T00:10:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.438+0000 7f3369600640 1 -- 172.21.15.83:0/3053087983 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3364005ae0 con 0x7f3350004d50 2024-04-25T00:10:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.438+0000 7f3369600640 1 -- 172.21.15.83:0/3053087983 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33640070d0 con 0x7f3350004d50 2024-04-25T00:10:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.438+0000 7f336be00640 1 -- 172.21.15.83:0/3053087983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3350004d50 msgr2=0x7f33500051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.438+0000 7f336be00640 1 --2- 172.21.15.83:0/3053087983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3350004d50 0x7f33500051d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3364001060 tx=0x7f336401f2b0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.438+0000 7f336b400640 1 -- 172.21.15.83:0/3053087983 reap_dead start 2024-04-25T00:10:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.438+0000 7f336be00640 1 -- 172.21.15.83:0/3053087983 shutdown_connections 2024-04-25T00:10:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.438+0000 7f336be00640 1 -- 172.21.15.83:0/3053087983 >> 172.21.15.83:0/3053087983 conn(0x7f336c0cd480 msgr2=0x7f336c0cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.438+0000 7f336be00640 1 -- 172.21.15.83:0/3053087983 shutdown_connections 2024-04-25T00:10:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.438+0000 7f336be00640 1 -- 172.21.15.83:0/3053087983 wait complete. 2024-04-25T00:10:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.438+0000 7f336be00640 1 Processor -- start 2024-04-25T00:10:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.438+0000 7f336be00640 1 -- start start 2024-04-25T00:10:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3350004d50 0x7f336c0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f336c0c5e80 0x7f336c1f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f336c1f8e30 0x7f336c1ee830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f336c144a30 con 0x7f336c1f8e30 2024-04-25T00:10:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f336c0cb6c0 con 0x7f336c0c5e80 2024-04-25T00:10:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f336c0729f0 con 0x7f3350004d50 2024-04-25T00:10:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f336c1f8e30 0x7f336c1ee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f336c1f8e30 0x7f336c1ee830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56890/0 (socket says 172.21.15.83:56890) 2024-04-25T00:10:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336b400640 1 -- 172.21.15.83:0/2184496092 learned_addr learned my addr 172.21.15.83:0/2184496092 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:10:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336aa00640 1 --2- 172.21.15.83:0/2184496092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3350004d50 0x7f336c0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336a000640 1 --2- 172.21.15.83:0/2184496092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f336c0c5e80 0x7f336c1f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336b400640 1 -- 172.21.15.83:0/2184496092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3350004d50 msgr2=0x7f336c0c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336b400640 1 --2- 172.21.15.83:0/2184496092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3350004d50 0x7f336c0c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336b400640 1 -- 172.21.15.83:0/2184496092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f336c0c5e80 msgr2=0x7f336c1f88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336b400640 1 --2- 172.21.15.83:0/2184496092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f336c0c5e80 0x7f336c1f88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336b400640 1 -- 172.21.15.83:0/2184496092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c1f92e0 con 0x7f336c1f8e30 2024-04-25T00:10:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336b400640 1 --2- 172.21.15.83:0/2184496092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f336c1f8e30 0x7f336c1ee830 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3364001060 tx=0x7f336401c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f3362a00640 1 -- 172.21.15.83:0/2184496092 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364001c00 con 0x7f336c1f8e30 2024-04-25T00:10:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f3362a00640 1 -- 172.21.15.83:0/2184496092 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3364007d20 con 0x7f336c1f8e30 2024-04-25T00:10:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f3362a00640 1 -- 172.21.15.83:0/2184496092 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33640074e0 con 0x7f336c1f8e30 2024-04-25T00:10:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336be00640 1 -- 172.21.15.83:0/2184496092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c0c8820 con 0x7f336c1f8e30 2024-04-25T00:10:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336be00640 1 -- 172.21.15.83:0/2184496092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c0c8de0 con 0x7f336c1f8e30 2024-04-25T00:10:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f3362a00640 1 -- 172.21.15.83:0/2184496092 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f33640b0810 con 0x7f336c1f8e30 2024-04-25T00:10:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f3362a00640 1 -- 172.21.15.83:0/2184496092 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f33640b9b00 con 0x7f336c1f8e30 2024-04-25T00:10:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.442+0000 7f336be00640 1 -- 172.21.15.83:0/2184496092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f336c04f510 con 0x7f336c1f8e30 2024-04-25T00:10:41.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.470+0000 7f3362a00640 1 -- 172.21.15.83:0/2184496092 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f33640c0030 con 0x7f336c1f8e30 2024-04-25T00:10:41.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.530+0000 7f336be00640 1 -- 172.21.15.83:0/2184496092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f336c0c6260 con 0x7f336c1f8e30 2024-04-25T00:10:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.534+0000 7f3362a00640 1 -- 172.21.15.83:0/2184496092 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f3364018fc0 con 0x7f336c1f8e30 2024-04-25T00:10:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:adjusted tunables profile to default 2024-04-25T00:10:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.534+0000 7f336be00640 1 -- 172.21.15.83:0/2184496092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f336c1f8e30 msgr2=0x7f336c1ee830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.534+0000 7f336be00640 1 --2- 172.21.15.83:0/2184496092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f336c1f8e30 0x7f336c1ee830 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3364001060 tx=0x7f336401c050 comp rx=0 tx=0).stop 2024-04-25T00:10:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.534+0000 7f336be00640 1 -- 172.21.15.83:0/2184496092 shutdown_connections 2024-04-25T00:10:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.534+0000 7f336be00640 1 --2- 172.21.15.83:0/2184496092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f336c1f8e30 0x7f336c1ee830 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.534+0000 7f336be00640 1 --2- 172.21.15.83:0/2184496092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f336c0c5e80 0x7f336c1f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.534+0000 7f336be00640 1 --2- 172.21.15.83:0/2184496092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3350004d50 0x7f336c0c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.534+0000 7f336be00640 1 -- 172.21.15.83:0/2184496092 >> 172.21.15.83:0/2184496092 conn(0x7f336c0cd480 msgr2=0x7f336c1425c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.534+0000 7f336be00640 1 -- 172.21.15.83:0/2184496092 shutdown_connections 2024-04-25T00:10:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:41.534+0000 7f336be00640 1 -- 172.21.15.83:0/2184496092 wait complete. 2024-04-25T00:10:41.552 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T00:10:41.553 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T00:10:41.553 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:41.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T00:10:41.572 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:41.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T00:10:41.634 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:41.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T00:10:41.640 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:41.640+0000 7f0220643180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:10:41.651 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:41.650+0000 7fecae24b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:10:41.670 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:10:41.670 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T00:10:41.731 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:41.731 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T00:10:41.750 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:41.750 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T00:10:41.815 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:41.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T00:10:41.879 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:10:41.879 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T00:10:41.943 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new fa38ca70-c391-4e90-a2fd-7700d7fa60e0 0 2024-04-25T00:10:42.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.120+0000 7f3dc5e00640 1 Processor -- start 2024-04-25T00:10:42.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.120+0000 7f3dc5e00640 1 -- start start 2024-04-25T00:10:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.120+0000 7f3dc5e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3dc0162920 0x7f3dc0072c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.120+0000 7f3dc5e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dc0071ed0 con 0x7f3dc015e110 2024-04-25T00:10:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.120+0000 7f3dc5e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dc00721b0 con 0x7f3dc006b1e0 2024-04-25T00:10:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.120+0000 7f3dc5e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc0072490 con 0x7f3dc0162920 2024-04-25T00:10:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.120+0000 7f3dbe000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3dc0162920 0x7f3dc0072c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.120+0000 7f3dbf400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3dc006b1e0 0x7f3dc006e820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50358/0 (socket says 172.21.15.86:50358) 2024-04-25T00:10:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.120+0000 7f3dbf400640 1 -- 172.21.15.86:0/1871664691 learned_addr learned my addr 172.21.15.86:0/1871664691 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T00:10:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.120+0000 7f3dbe000640 1 -- 172.21.15.86:0/1871664691 >> v1:172.21.15.86:6789/0 conn(0x7f3dc006b1e0 legacy=0x7f3dc006e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.120+0000 7f3dbe000640 1 -- 172.21.15.86:0/1871664691 >> v1:172.21.15.83:6789/0 conn(0x7f3dc015e110 legacy=0x7f3dc00afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.120+0000 7f3dbe000640 1 -- 172.21.15.86:0/1871664691 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc0072770 con 0x7f3dc0162920 2024-04-25T00:10:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.120+0000 7f3dbd600640 1 -- 172.21.15.86:0/1871664691 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3331134844 0 0) 0x7f3dc00721b0 con 0x7f3dc006b1e0 2024-04-25T00:10:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.124+0000 7f3dbe000640 1 --2- 172.21.15.86:0/1871664691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3dc0162920 0x7f3dc0072c90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0097f0 tx=0x7f3dac01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e2331ad98997b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.124+0000 7f3dbd600640 1 -- 172.21.15.86:0/1871664691 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0b0070 con 0x7f3dc0162920 2024-04-25T00:10:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.124+0000 7f3dbd600640 1 -- 172.21.15.86:0/1871664691 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dac0ab600 con 0x7f3dc0162920 2024-04-25T00:10:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.124+0000 7f3dbd600640 1 -- 172.21.15.86:0/1871664691 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0ab9d0 con 0x7f3dc0162920 2024-04-25T00:10:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.124+0000 7f3dc5e00640 1 -- 172.21.15.86:0/1871664691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3dc0162920 msgr2=0x7f3dc0072c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.124+0000 7f3dc5e00640 1 --2- 172.21.15.86:0/1871664691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3dc0162920 0x7f3dc0072c90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0097f0 tx=0x7f3dac01c450 comp rx=0 tx=0).stop 2024-04-25T00:10:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.124+0000 7f3dc5e00640 1 -- 172.21.15.86:0/1871664691 shutdown_connections 2024-04-25T00:10:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.124+0000 7f3dc5e00640 1 --2- 172.21.15.86:0/1871664691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3dc0162920 0x7f3dc0072c90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.124+0000 7f3dc5e00640 1 -- 172.21.15.86:0/1871664691 >> 172.21.15.86:0/1871664691 conn(0x7f3dc006c2f0 msgr2=0x7f3dc006c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.124+0000 7f3dc5e00640 1 -- 172.21.15.86:0/1871664691 shutdown_connections 2024-04-25T00:10:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.124+0000 7f3dc5e00640 1 -- 172.21.15.86:0/1871664691 wait complete. 2024-04-25T00:10:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.124+0000 7f3dc5e00640 1 Processor -- start 2024-04-25T00:10:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.124+0000 7f3dc5e00640 1 -- start start 2024-04-25T00:10:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dc5e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3dc006b1e0 0x7f3dc0145bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dc5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dc015e110 0x7f3dc00ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dc5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc0162920 0x7f3dc0149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dc5e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc016c420 con 0x7f3dc0162920 2024-04-25T00:10:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dc5e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc016c2a0 con 0x7f3dc015e110 2024-04-25T00:10:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dc5e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc016c5a0 con 0x7f3dc006b1e0 2024-04-25T00:10:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dbe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dc015e110 0x7f3dc00ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dbea00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3dc006b1e0 0x7f3dc0145bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dbea00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3dc006b1e0 0x7f3dc0145bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:43464/0 (socket says 172.21.15.86:43464) 2024-04-25T00:10:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dbea00640 1 -- 172.21.15.86:0/752672355 learned_addr learned my addr 172.21.15.86:0/752672355 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T00:10:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dbe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dc015e110 0x7f3dc00ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45298/0 (socket says 172.21.15.86:45298) 2024-04-25T00:10:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dbf400640 1 --2- 172.21.15.86:0/752672355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc0162920 0x7f3dc0149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dbe000640 1 -- 172.21.15.86:0/752672355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3dc006b1e0 msgr2=0x7f3dc0145bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dbe000640 1 --2- 172.21.15.86:0/752672355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3dc006b1e0 0x7f3dc0145bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dbe000640 1 -- 172.21.15.86:0/752672355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc0162920 msgr2=0x7f3dc0149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dbe000640 1 --2- 172.21.15.86:0/752672355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc0162920 0x7f3dc0149900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dbe000640 1 -- 172.21.15.86:0/752672355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc0132ce0 con 0x7f3dc015e110 2024-04-25T00:10:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dbea00640 1 --2- 172.21.15.86:0/752672355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3dc006b1e0 0x7f3dc0145bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:42.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dbe000640 1 --2- 172.21.15.86:0/752672355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dc015e110 0x7f3dc00ad860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0ab000 tx=0x7f3dac01ac90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:42.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3db7400640 1 -- 172.21.15.86:0/752672355 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0b0050 con 0x7f3dc015e110 2024-04-25T00:10:42.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3db7400640 1 -- 172.21.15.86:0/752672355 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dac0bc070 con 0x7f3dc015e110 2024-04-25T00:10:42.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dc5e00640 1 -- 172.21.15.86:0/752672355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc0131f20 con 0x7f3dc015e110 2024-04-25T00:10:42.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3db7400640 1 -- 172.21.15.86:0/752672355 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0b5a50 con 0x7f3dc015e110 2024-04-25T00:10:42.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dc5e00640 1 -- 172.21.15.86:0/752672355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc01324e0 con 0x7f3dc015e110 2024-04-25T00:10:42.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3db7400640 1 -- 172.21.15.86:0/752672355 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3dac0b5bf0 con 0x7f3dc015e110 2024-04-25T00:10:42.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3db7400640 1 -- 172.21.15.86:0/752672355 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f3dac0c1aa0 con 0x7f3dc015e110 2024-04-25T00:10:42.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.128+0000 7f3dc5e00640 1 -- 172.21.15.86:0/752672355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc004f510 con 0x7f3dc015e110 2024-04-25T00:10:42.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.132+0000 7f3db7400640 1 -- 172.21.15.86:0/752672355 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3dac0160d0 con 0x7f3dc015e110 2024-04-25T00:10:42.160 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:42.160+0000 7f0220643180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:10:42.162 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:42.162+0000 7fecae24b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:10:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.200+0000 7f3dc5e00640 1 -- 172.21.15.86:0/752672355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fa38ca70-c391-4e90-a2fd-7700d7fa60e0", "id": 0} v 0) -- 0x7f3dc006f8e0 con 0x7f3dc015e110 2024-04-25T00:10:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.204+0000 7f3db7400640 1 -- 172.21.15.86:0/752672355 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fa38ca70-c391-4e90-a2fd-7700d7fa60e0", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f3dac0ab540 con 0x7f3dc015e110 2024-04-25T00:10:42.205 INFO:teuthology.orchestra.run.smithi086.stdout:0 2024-04-25T00:10:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.204+0000 7f3dc5e00640 1 -- 172.21.15.86:0/752672355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dc015e110 msgr2=0x7f3dc00ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.204+0000 7f3dc5e00640 1 --2- 172.21.15.86:0/752672355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dc015e110 0x7f3dc00ad860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0ab000 tx=0x7f3dac01ac90 comp rx=0 tx=0).stop 2024-04-25T00:10:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.204+0000 7f3dc5e00640 1 -- 172.21.15.86:0/752672355 shutdown_connections 2024-04-25T00:10:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.204+0000 7f3dc5e00640 1 --2- 172.21.15.86:0/752672355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc0162920 0x7f3dc0149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.204+0000 7f3dc5e00640 1 --2- 172.21.15.86:0/752672355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dc015e110 0x7f3dc00ad860 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.204+0000 7f3dc5e00640 1 --2- 172.21.15.86:0/752672355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3dc006b1e0 0x7f3dc0145bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.204+0000 7f3dc5e00640 1 -- 172.21.15.86:0/752672355 >> 172.21.15.86:0/752672355 conn(0x7f3dc006c2f0 msgr2=0x7f3dc00d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.204+0000 7f3dc5e00640 1 -- 172.21.15.86:0/752672355 shutdown_connections 2024-04-25T00:10:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.204+0000 7f3dc5e00640 1 -- 172.21.15.86:0/752672355 wait complete. 2024-04-25T00:10:42.224 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new a944117d-465b-4cfa-9120-d9d9a3516af4 1 2024-04-25T00:10:42.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.356+0000 7fc800600640 1 Processor -- start 2024-04-25T00:10:42.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.356+0000 7fc800600640 1 -- start start 2024-04-25T00:10:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.360+0000 7fc800600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7f815e0e0 0x7fc7f80d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.360+0000 7fc800600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7f80cec50 0x7fc7f80cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.360+0000 7fc800600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7f80b4b10 con 0x7fc7f815e4c0 2024-04-25T00:10:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.360+0000 7fc800600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc7f80d0f50 con 0x7fc7f815e0e0 2024-04-25T00:10:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.360+0000 7fc800600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f80cc310 con 0x7fc7f80cec50 2024-04-25T00:10:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.360+0000 7fc7fdc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7f815e0e0 0x7fc7f80d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.360+0000 7fc7fe600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7f80cec50 0x7fc7f80cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.360+0000 7fc7fdc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7f815e0e0 0x7fc7f80d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45310/0 (socket says 172.21.15.86:45310) 2024-04-25T00:10:42.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.360+0000 7fc7fdc00640 1 -- 172.21.15.86:0/3235113764 learned_addr learned my addr 172.21.15.86:0/3235113764 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T00:10:42.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.360+0000 7fc7fdc00640 1 -- 172.21.15.86:0/3235113764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7f80cec50 msgr2=0x7fc7f80cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:42.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.360+0000 7fc7fdc00640 1 --2- 172.21.15.86:0/3235113764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7f80cec50 0x7fc7f80cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.360+0000 7fc7fdc00640 1 -- 172.21.15.86:0/3235113764 >> v1:172.21.15.86:6789/0 conn(0x7fc7f815e4c0 legacy=0x7fc7f80ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.360+0000 7fc7fdc00640 1 -- 172.21.15.86:0/3235113764 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f80d1180 con 0x7fc7f815e0e0 2024-04-25T00:10:42.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.360+0000 7fc7f7e00640 1 -- 172.21.15.86:0/3235113764 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3574035445 0 0) 0x7fc7e4001120 con 0x7fc7f815e4c0 2024-04-25T00:10:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.364+0000 7fc7fdc00640 1 --2- 172.21.15.86:0/3235113764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f815e0e0 0x7fc7f80d0a10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009870 tx=0x7fc7ec01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6966c22b8d12176f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:42.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.364+0000 7fc7f7e00640 1 -- 172.21.15.86:0/3235113764 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0ab650 con 0x7fc7f815e0e0 2024-04-25T00:10:42.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.364+0000 7fc7f7e00640 1 -- 172.21.15.86:0/3235113764 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7ec0b0070 con 0x7fc7f815e0e0 2024-04-25T00:10:42.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.364+0000 7fc7f7e00640 1 -- 172.21.15.86:0/3235113764 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0aba10 con 0x7fc7f815e0e0 2024-04-25T00:10:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.364+0000 7fc800600640 1 -- 172.21.15.86:0/3235113764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f815e0e0 msgr2=0x7fc7f80d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.364+0000 7fc800600640 1 --2- 172.21.15.86:0/3235113764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f815e0e0 0x7fc7f80d0a10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009870 tx=0x7fc7ec01c560 comp rx=0 tx=0).stop 2024-04-25T00:10:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.364+0000 7fc800600640 1 -- 172.21.15.86:0/3235113764 shutdown_connections 2024-04-25T00:10:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.364+0000 7fc800600640 1 --2- 172.21.15.86:0/3235113764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7f80cec50 0x7fc7f80cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.364+0000 7fc800600640 1 --2- 172.21.15.86:0/3235113764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f815e0e0 0x7fc7f80d0a10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.364+0000 7fc800600640 1 -- 172.21.15.86:0/3235113764 >> 172.21.15.86:0/3235113764 conn(0x7fc7f80d1cc0 msgr2=0x7fc7f80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.364+0000 7fc800600640 1 -- 172.21.15.86:0/3235113764 shutdown_connections 2024-04-25T00:10:42.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.364+0000 7fc800600640 1 -- 172.21.15.86:0/3235113764 wait complete. 2024-04-25T00:10:42.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.368+0000 7fc800600640 1 Processor -- start 2024-04-25T00:10:42.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.368+0000 7fc800600640 1 -- start start 2024-04-25T00:10:42.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.368+0000 7fc800600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7f80cec50 0x7fc7f80c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.368+0000 7fc800600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7f815e0e0 0x7fc7f80c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.368+0000 7fc800600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f815e4c0 0x7fc7f80c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.368+0000 7fc800600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f80ce020 con 0x7fc7f80cec50 2024-04-25T00:10:42.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.368+0000 7fc800600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f816aed0 con 0x7fc7f815e4c0 2024-04-25T00:10:42.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.368+0000 7fc800600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f8152d30 con 0x7fc7f815e0e0 2024-04-25T00:10:42.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.368+0000 7fc7fd200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7f815e0e0 0x7fc7f80c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.368+0000 7fc7fe600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f815e4c0 0x7fc7f80c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.368+0000 7fc7fd200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7f815e0e0 0x7fc7f80c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:43484/0 (socket says 172.21.15.86:43484) 2024-04-25T00:10:42.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.368+0000 7fc7fd200640 1 -- 172.21.15.86:0/2935948462 learned_addr learned my addr 172.21.15.86:0/2935948462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T00:10:42.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc7fdc00640 1 --2- 172.21.15.86:0/2935948462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7f80cec50 0x7fc7f80c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc7fd200640 1 -- 172.21.15.86:0/2935948462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f815e4c0 msgr2=0x7fc7f80c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc7fd200640 1 --2- 172.21.15.86:0/2935948462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f815e4c0 0x7fc7f80c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc7fd200640 1 -- 172.21.15.86:0/2935948462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7f80cec50 msgr2=0x7fc7f80c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc7fd200640 1 --2- 172.21.15.86:0/2935948462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7f80cec50 0x7fc7f80c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc7fd200640 1 -- 172.21.15.86:0/2935948462 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f8152f80 con 0x7fc7f815e0e0 2024-04-25T00:10:42.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc7fd200640 1 --2- 172.21.15.86:0/2935948462 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7f815e0e0 0x7fc7f80c5d30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400c750 tx=0x7fc7e400cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:42.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc7f5600640 1 -- 172.21.15.86:0/2935948462 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4017070 con 0x7fc7f815e0e0 2024-04-25T00:10:42.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc7f5600640 1 -- 172.21.15.86:0/2935948462 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e4005290 con 0x7fc7f815e0e0 2024-04-25T00:10:42.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc800600640 1 -- 172.21.15.86:0/2935948462 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f8153260 con 0x7fc7f815e0e0 2024-04-25T00:10:42.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc7f5600640 1 -- 172.21.15.86:0/2935948462 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e400a640 con 0x7fc7f815e0e0 2024-04-25T00:10:42.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc800600640 1 -- 172.21.15.86:0/2935948462 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f8153820 con 0x7fc7f815e0e0 2024-04-25T00:10:42.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc7f5600640 1 -- 172.21.15.86:0/2935948462 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc7e401a070 con 0x7fc7f815e0e0 2024-04-25T00:10:42.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc7f5600640 1 -- 172.21.15.86:0/2935948462 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fc7e401c440 con 0x7fc7f815e0e0 2024-04-25T00:10:42.376 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:42.374+0000 7fecae24b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:10:42.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.372+0000 7fc800600640 1 -- 172.21.15.86:0/2935948462 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f804f510 con 0x7fc7f815e0e0 2024-04-25T00:10:42.380 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:42.376+0000 7f0220643180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:10:42.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.384+0000 7fc7f5600640 1 -- 172.21.15.86:0/2935948462 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc7e4021070 con 0x7fc7f815e0e0 2024-04-25T00:10:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.452+0000 7fc800600640 1 -- 172.21.15.86:0/2935948462 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a944117d-465b-4cfa-9120-d9d9a3516af4", "id": 1} v 0) -- 0x7fc7f806d630 con 0x7fc7f815e0e0 2024-04-25T00:10:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.460+0000 7fc7f5600640 1 -- 172.21.15.86:0/2935948462 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a944117d-465b-4cfa-9120-d9d9a3516af4", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fc7f80b4b10 con 0x7fc7f815e0e0 2024-04-25T00:10:42.462 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T00:10:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.460+0000 7fc800600640 1 -- 172.21.15.86:0/2935948462 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7f815e0e0 msgr2=0x7fc7f80c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.460+0000 7fc800600640 1 --2- 172.21.15.86:0/2935948462 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7f815e0e0 0x7fc7f80c5d30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400c750 tx=0x7fc7e400cc10 comp rx=0 tx=0).stop 2024-04-25T00:10:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.460+0000 7fc800600640 1 -- 172.21.15.86:0/2935948462 shutdown_connections 2024-04-25T00:10:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.460+0000 7fc800600640 1 --2- 172.21.15.86:0/2935948462 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7f815e0e0 0x7fc7f80c5d30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.460+0000 7fc800600640 1 --2- 172.21.15.86:0/2935948462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f815e4c0 0x7fc7f80c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.460+0000 7fc800600640 1 --2- 172.21.15.86:0/2935948462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7f80cec50 0x7fc7f80c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.460+0000 7fc800600640 1 -- 172.21.15.86:0/2935948462 >> 172.21.15.86:0/2935948462 conn(0x7fc7f80d1cc0 msgr2=0x7fc7f8070510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.460+0000 7fc800600640 1 -- 172.21.15.86:0/2935948462 shutdown_connections 2024-04-25T00:10:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.460+0000 7fc800600640 1 -- 172.21.15.86:0/2935948462 wait complete. 2024-04-25T00:10:42.481 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 60f71d92-2018-4399-a106-2bd7d9c5f087 2 2024-04-25T00:10:42.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.612+0000 7f3e26600640 1 Processor -- start 2024-04-25T00:10:42.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.612+0000 7f3e26600640 1 -- start start 2024-04-25T00:10:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e26600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e2015e0e0 0x7f3e200d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e26600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e2015e4c0 0x7f3e200ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e26600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e200b4b10 con 0x7f3e200cea80 2024-04-25T00:10:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e26600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3e200d0f50 con 0x7f3e2015e0e0 2024-04-25T00:10:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e26600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3e200cc310 con 0x7f3e2015e4c0 2024-04-25T00:10:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e1f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e2015e0e0 0x7f3e200d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e1ea00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e2015e4c0 0x7f3e200ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e1f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e2015e0e0 0x7f3e200d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45320/0 (socket says 172.21.15.86:45320) 2024-04-25T00:10:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e1f400640 1 -- 172.21.15.86:0/1607349370 learned_addr learned my addr 172.21.15.86:0/1607349370 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T00:10:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e1f400640 1 -- 172.21.15.86:0/1607349370 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e2015e4c0 msgr2=0x7f3e200ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e1f400640 1 --2- 172.21.15.86:0/1607349370 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e2015e4c0 0x7f3e200ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e1f400640 1 -- 172.21.15.86:0/1607349370 >> v1:172.21.15.86:6789/0 conn(0x7f3e200cea80 legacy=0x7f3e2006c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e1f400640 1 -- 172.21.15.86:0/1607349370 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e200d1180 con 0x7f3e2015e0e0 2024-04-25T00:10:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e1e000640 1 -- 172.21.15.86:0/1607349370 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1788263540 0 0) 0x7f3e180014a0 con 0x7f3e200cea80 2024-04-25T00:10:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.616+0000 7f3e1ea00640 1 --2- 172.21.15.86:0/1607349370 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e2015e4c0 0x7f3e200ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:42.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.620+0000 7f3e1f400640 1 --2- 172.21.15.86:0/1607349370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2015e0e0 0x7f3e200d0a10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009870 tx=0x7f3e1001c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ce0b19913d6ab36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:42.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.620+0000 7f3e1e000640 1 -- 172.21.15.86:0/1607349370 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100ab650 con 0x7f3e2015e0e0 2024-04-25T00:10:42.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.620+0000 7f3e1e000640 1 -- 172.21.15.86:0/1607349370 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e100b0070 con 0x7f3e2015e0e0 2024-04-25T00:10:42.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.620+0000 7f3e1e000640 1 -- 172.21.15.86:0/1607349370 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100aba10 con 0x7f3e2015e0e0 2024-04-25T00:10:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.620+0000 7f3e26600640 1 -- 172.21.15.86:0/1607349370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2015e0e0 msgr2=0x7f3e200d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.620+0000 7f3e26600640 1 --2- 172.21.15.86:0/1607349370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2015e0e0 0x7f3e200d0a10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009870 tx=0x7f3e1001c560 comp rx=0 tx=0).stop 2024-04-25T00:10:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.620+0000 7f3e26600640 1 -- 172.21.15.86:0/1607349370 shutdown_connections 2024-04-25T00:10:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.620+0000 7f3e26600640 1 --2- 172.21.15.86:0/1607349370 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e2015e4c0 0x7f3e200ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.620+0000 7f3e26600640 1 --2- 172.21.15.86:0/1607349370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2015e0e0 0x7f3e200d0a10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.620+0000 7f3e26600640 1 -- 172.21.15.86:0/1607349370 >> 172.21.15.86:0/1607349370 conn(0x7f3e200d1cc0 msgr2=0x7f3e200d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.620+0000 7f3e26600640 1 -- 172.21.15.86:0/1607349370 shutdown_connections 2024-04-25T00:10:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.620+0000 7f3e26600640 1 -- 172.21.15.86:0/1607349370 wait complete. 2024-04-25T00:10:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.620+0000 7f3e26600640 1 Processor -- start 2024-04-25T00:10:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.620+0000 7f3e26600640 1 -- start start 2024-04-25T00:10:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e26600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e200cea80 0x7f3e200caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e26600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2015e0e0 0x7f3e201522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e26600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2015e4c0 0x7f3e20150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e26600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2016aed0 con 0x7f3e2015e4c0 2024-04-25T00:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e26600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e200ce050 con 0x7f3e2015e0e0 2024-04-25T00:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e26600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3e2016b050 con 0x7f3e200cea80 2024-04-25T00:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e1f400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e200cea80 0x7f3e200caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e1ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2015e0e0 0x7f3e201522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e1f400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e200cea80 0x7f3e200caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:43506/0 (socket says 172.21.15.86:43506) 2024-04-25T00:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e1f400640 1 -- 172.21.15.86:0/3862004050 learned_addr learned my addr 172.21.15.86:0/3862004050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T00:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e1fe00640 1 --2- 172.21.15.86:0/3862004050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2015e4c0 0x7f3e20150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e1f400640 1 -- 172.21.15.86:0/3862004050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2015e0e0 msgr2=0x7f3e201522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e1f400640 1 --2- 172.21.15.86:0/3862004050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2015e0e0 0x7f3e201522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e1f400640 1 -- 172.21.15.86:0/3862004050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2015e4c0 msgr2=0x7f3e20150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e1f400640 1 --2- 172.21.15.86:0/3862004050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2015e4c0 0x7f3e20150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e1f400640 1 -- 172.21.15.86:0/3862004050 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e20151530 con 0x7f3e200cea80 2024-04-25T00:10:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e1ea00640 1 --2- 172.21.15.86:0/3862004050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2015e0e0 0x7f3e201522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e1fe00640 1 --2- 172.21.15.86:0/3862004050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2015e4c0 0x7f3e20150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e1f400640 1 --2- 172.21.15.86:0/3862004050 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e200cea80 0x7f3e200caa20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3e1001ac10 tx=0x7f3e1001ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e17400640 1 -- 172.21.15.86:0/3862004050 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100be5f0 con 0x7f3e200cea80 2024-04-25T00:10:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e17400640 1 -- 172.21.15.86:0/3862004050 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e100be790 con 0x7f3e200cea80 2024-04-25T00:10:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e26600640 1 -- 172.21.15.86:0/3862004050 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e20151810 con 0x7f3e200cea80 2024-04-25T00:10:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e17400640 1 -- 172.21.15.86:0/3862004050 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100beb80 con 0x7f3e200cea80 2024-04-25T00:10:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e26600640 1 -- 172.21.15.86:0/3862004050 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e20151dd0 con 0x7f3e200cea80 2024-04-25T00:10:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e17400640 1 -- 172.21.15.86:0/3862004050 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3e100b4780 con 0x7f3e200cea80 2024-04-25T00:10:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e17400640 1 -- 172.21.15.86:0/3862004050 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f3e100c8da0 con 0x7f3e200cea80 2024-04-25T00:10:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.624+0000 7f3e26600640 1 -- 172.21.15.86:0/3862004050 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e2004f510 con 0x7f3e200cea80 2024-04-25T00:10:42.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.628+0000 7f3e17400640 1 -- 172.21.15.86:0/3862004050 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3e100c4060 con 0x7f3e200cea80 2024-04-25T00:10:42.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.696+0000 7f3e26600640 1 -- 172.21.15.86:0/3862004050 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "60f71d92-2018-4399-a106-2bd7d9c5f087", "id": 2} v 0) -- 0x7f3e2006d630 con 0x7f3e200cea80 2024-04-25T00:10:42.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.700+0000 7f3e17400640 1 -- 172.21.15.86:0/3862004050 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "60f71d92-2018-4399-a106-2bd7d9c5f087", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f3e10016150 con 0x7f3e200cea80 2024-04-25T00:10:42.703 INFO:teuthology.orchestra.run.smithi086.stdout:2 2024-04-25T00:10:42.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.700+0000 7f3e26600640 1 -- 172.21.15.86:0/3862004050 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e200cea80 msgr2=0x7f3e200caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.700+0000 7f3e26600640 1 --2- 172.21.15.86:0/3862004050 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e200cea80 0x7f3e200caa20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3e1001ac10 tx=0x7f3e1001ada0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.700+0000 7f3e26600640 1 -- 172.21.15.86:0/3862004050 shutdown_connections 2024-04-25T00:10:42.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.700+0000 7f3e26600640 1 --2- 172.21.15.86:0/3862004050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2015e4c0 0x7f3e20150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.700+0000 7f3e26600640 1 --2- 172.21.15.86:0/3862004050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e2015e0e0 0x7f3e201522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.700+0000 7f3e26600640 1 --2- 172.21.15.86:0/3862004050 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e200cea80 0x7f3e200caa20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.700+0000 7f3e26600640 1 -- 172.21.15.86:0/3862004050 >> 172.21.15.86:0/3862004050 conn(0x7f3e200d1cc0 msgr2=0x7f3e200b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:42.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.700+0000 7f3e26600640 1 -- 172.21.15.86:0/3862004050 shutdown_connections 2024-04-25T00:10:42.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.700+0000 7f3e26600640 1 -- 172.21.15.86:0/3862004050 wait complete. 2024-04-25T00:10:42.722 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9 3 2024-04-25T00:10:42.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.856+0000 7fbcb7200640 1 Processor -- start 2024-04-25T00:10:42.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.856+0000 7fbcb7200640 1 -- start start 2024-04-25T00:10:42.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.856+0000 7fbcb7200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbcb8144580 0x7fbcb806e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.856+0000 7fbcb7200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcb8147310 0x7fbcb806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.856+0000 7fbcb7200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcb8162a50 con 0x7fbcb80ceb70 2024-04-25T00:10:42.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.856+0000 7fbcb7200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbcb8072f30 con 0x7fbcb8144580 2024-04-25T00:10:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.856+0000 7fbcb7200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb8144960 con 0x7fbcb8147310 2024-04-25T00:10:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.856+0000 7fbcafe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcb8147310 0x7fbcb806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.856+0000 7fbcb6800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbcb80ceb70 0x7fbcb80c9d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50378/0 (socket says 172.21.15.86:50378) 2024-04-25T00:10:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.856+0000 7fbcb6800640 1 -- 172.21.15.86:0/3443560546 learned_addr learned my addr 172.21.15.86:0/3443560546 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T00:10:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.856+0000 7fbcb5e00640 1 --2- 172.21.15.86:0/3443560546 >> v2:172.21.15.83:3300/0 conn(0x7fbcb8144580 0x7fbcb806e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.860+0000 7fbcafe00640 1 -- 172.21.15.86:0/3443560546 >> v2:172.21.15.83:3300/0 conn(0x7fbcb8144580 msgr2=0x7fbcb806e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.860+0000 7fbcafe00640 1 --2- 172.21.15.86:0/3443560546 >> v2:172.21.15.83:3300/0 conn(0x7fbcb8144580 0x7fbcb806e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.860+0000 7fbcafe00640 1 -- 172.21.15.86:0/3443560546 >> v1:172.21.15.86:6789/0 conn(0x7fbcb80ceb70 legacy=0x7fbcb80c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.860+0000 7fbcafe00640 1 -- 172.21.15.86:0/3443560546 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb8161ed0 con 0x7fbcb8147310 2024-04-25T00:10:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.860+0000 7fbcb5400640 1 -- 172.21.15.86:0/3443560546 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3160940365 0 0) 0x7fbcb8162a50 con 0x7fbcb80ceb70 2024-04-25T00:10:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.860+0000 7fbcafe00640 1 --2- 172.21.15.86:0/3443560546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcb8147310 0x7fbcb806e8c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbca00097c0 tx=0x7fbca001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d2d14a6209aa237 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.860+0000 7fbcb5400640 1 -- 172.21.15.86:0/3443560546 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca00b2070 con 0x7fbcb8147310 2024-04-25T00:10:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.860+0000 7fbcb5400640 1 -- 172.21.15.86:0/3443560546 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca00ad680 con 0x7fbcb8147310 2024-04-25T00:10:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.860+0000 7fbcb5400640 1 -- 172.21.15.86:0/3443560546 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca00ada50 con 0x7fbcb8147310 2024-04-25T00:10:42.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.860+0000 7fbcb7200640 1 -- 172.21.15.86:0/3443560546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcb8147310 msgr2=0x7fbcb806e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.860+0000 7fbcb7200640 1 --2- 172.21.15.86:0/3443560546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcb8147310 0x7fbcb806e8c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbca00097c0 tx=0x7fbca001c470 comp rx=0 tx=0).stop 2024-04-25T00:10:42.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb7200640 1 -- 172.21.15.86:0/3443560546 shutdown_connections 2024-04-25T00:10:42.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb7200640 1 --2- 172.21.15.86:0/3443560546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcb8147310 0x7fbcb806e8c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb7200640 1 --2- 172.21.15.86:0/3443560546 >> v2:172.21.15.83:3300/0 conn(0x7fbcb8144580 0x7fbcb806e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb7200640 1 -- 172.21.15.86:0/3443560546 >> 172.21.15.86:0/3443560546 conn(0x7fbcb80cd480 msgr2=0x7fbcb80cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:42.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb7200640 1 -- 172.21.15.86:0/3443560546 shutdown_connections 2024-04-25T00:10:42.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb7200640 1 -- 172.21.15.86:0/3443560546 wait complete. 2024-04-25T00:10:42.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb7200640 1 Processor -- start 2024-04-25T00:10:42.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb7200640 1 -- start start 2024-04-25T00:10:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb7200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcb80ceb70 0x7fbcb815c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8144580 0x7fbcb8140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb8147310 0x7fbcb813d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8161c70 con 0x7fbcb8147310 2024-04-25T00:10:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8161af0 con 0x7fbcb8144580 2024-04-25T00:10:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb7200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb816ae90 con 0x7fbcb80ceb70 2024-04-25T00:10:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb5e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcb80ceb70 0x7fbcb815c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcafe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8144580 0x7fbcb8140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb5e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcb80ceb70 0x7fbcb815c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:43520/0 (socket says 172.21.15.86:43520) 2024-04-25T00:10:42.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb5e00640 1 -- 172.21.15.86:0/601209591 learned_addr learned my addr 172.21.15.86:0/601209591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T00:10:42.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb6800640 1 --2- 172.21.15.86:0/601209591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb8147310 0x7fbcb813d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb5e00640 1 -- 172.21.15.86:0/601209591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8144580 msgr2=0x7fbcb8140bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb5e00640 1 --2- 172.21.15.86:0/601209591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8144580 0x7fbcb8140bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb5e00640 1 -- 172.21.15.86:0/601209591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb8147310 msgr2=0x7fbcb813d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb5e00640 1 --2- 172.21.15.86:0/601209591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb8147310 0x7fbcb813d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb5e00640 1 -- 172.21.15.86:0/601209591 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb815bab0 con 0x7fbcb80ceb70 2024-04-25T00:10:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcb5e00640 1 --2- 172.21.15.86:0/601209591 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcb80ceb70 0x7fbcb815c870 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbca800e9e0 tx=0x7fbca800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.864+0000 7fbcae000640 1 -- 172.21.15.86:0/601209591 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca8017070 con 0x7fbcb80ceb70 2024-04-25T00:10:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.868+0000 7fbcae000640 1 -- 172.21.15.86:0/601209591 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca800c9d0 con 0x7fbcb80ceb70 2024-04-25T00:10:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.868+0000 7fbcae000640 1 -- 172.21.15.86:0/601209591 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca800cda0 con 0x7fbcb80ceb70 2024-04-25T00:10:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.868+0000 7fbcb7200640 1 -- 172.21.15.86:0/601209591 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb815bd90 con 0x7fbcb80ceb70 2024-04-25T00:10:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.868+0000 7fbcb7200640 1 -- 172.21.15.86:0/601209591 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb8141130 con 0x7fbcb80ceb70 2024-04-25T00:10:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.868+0000 7fbcae000640 1 -- 172.21.15.86:0/601209591 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbca801a070 con 0x7fbcb80ceb70 2024-04-25T00:10:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.868+0000 7fbcae000640 1 -- 172.21.15.86:0/601209591 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fbca8013900 con 0x7fbcb80ceb70 2024-04-25T00:10:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.868+0000 7fbcb7200640 1 -- 172.21.15.86:0/601209591 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb804f510 con 0x7fbcb80ceb70 2024-04-25T00:10:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.868+0000 7fbcae000640 1 -- 172.21.15.86:0/601209591 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbca8013c20 con 0x7fbcb80ceb70 2024-04-25T00:10:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.936+0000 7fbcb7200640 1 -- 172.21.15.86:0/601209591 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9", "id": 3} v 0) -- 0x7fbcb806c6f0 con 0x7fbcb80ceb70 2024-04-25T00:10:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.940+0000 7fbcae000640 1 -- 172.21.15.86:0/601209591 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fbca8013470 con 0x7fbcb80ceb70 2024-04-25T00:10:42.944 INFO:teuthology.orchestra.run.smithi086.stdout:3 2024-04-25T00:10:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.944+0000 7fbcb7200640 1 -- 172.21.15.86:0/601209591 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcb80ceb70 msgr2=0x7fbcb815c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.944+0000 7fbcb7200640 1 --2- 172.21.15.86:0/601209591 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcb80ceb70 0x7fbcb815c870 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbca800e9e0 tx=0x7fbca800eea0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.944+0000 7fbcb7200640 1 -- 172.21.15.86:0/601209591 shutdown_connections 2024-04-25T00:10:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.944+0000 7fbcb7200640 1 --2- 172.21.15.86:0/601209591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb8147310 0x7fbcb813d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.944+0000 7fbcb7200640 1 --2- 172.21.15.86:0/601209591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8144580 0x7fbcb8140bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.944+0000 7fbcb7200640 1 --2- 172.21.15.86:0/601209591 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcb80ceb70 0x7fbcb815c870 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.944+0000 7fbcb7200640 1 -- 172.21.15.86:0/601209591 >> 172.21.15.86:0/601209591 conn(0x7fbcb80cd480 msgr2=0x7fbcb80b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.944+0000 7fbcb7200640 1 -- 172.21.15.86:0/601209591 shutdown_connections 2024-04-25T00:10:42.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:42.944+0000 7fbcb7200640 1 -- 172.21.15.86:0/601209591 wait complete. 2024-04-25T00:10:42.963 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new dce8a21c-99a4-403f-83ff-ae3a6477b6e3 4 2024-04-25T00:10:42.982 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:42.982+0000 7fecae24b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:10:43.018 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:43.016+0000 7f0220643180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:10:43.043 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:43.042+0000 7fecae24b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:10:43.082 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:43.080+0000 7f0220643180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:10:43.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f6841800640 1 Processor -- start 2024-04-25T00:10:43.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f6841800640 1 -- start start 2024-04-25T00:10:43.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f6841800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f683c15e0e0 0x7f683c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f6841800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f683c15e4c0 0x7f683c0ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f6841800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c0cea80 0x7f683c0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f6841800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f683c0d0f50 con 0x7f683c15e0e0 2024-04-25T00:10:43.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f6841800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f683c0cc310 con 0x7f683c15e4c0 2024-04-25T00:10:43.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f6841800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f683c0ccbb0 con 0x7f683c0cea80 2024-04-25T00:10:43.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f683b400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c0cea80 0x7f683c0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f683a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f683c15e4c0 0x7f683c0ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f683b400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c0cea80 0x7f683c0cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:43536/0 (socket says 172.21.15.86:43536) 2024-04-25T00:10:43.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f683a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f683c15e4c0 0x7f683c0ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45354/0 (socket says 172.21.15.86:45354) 2024-04-25T00:10:43.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f683b400640 1 -- 172.21.15.86:0/3004067604 learned_addr learned my addr 172.21.15.86:0/3004067604 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T00:10:43.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f683aa00640 1 --2- 172.21.15.86:0/3004067604 >> v2:172.21.15.83:3300/0 conn(0x7f683c15e0e0 0x7f683c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f683b400640 1 -- 172.21.15.86:0/3004067604 >> v2:172.21.15.86:3300/0 conn(0x7f683c15e4c0 msgr2=0x7f683c0ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:43.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f683b400640 1 --2- 172.21.15.86:0/3004067604 >> v2:172.21.15.86:3300/0 conn(0x7f683c15e4c0 0x7f683c0ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f683b400640 1 -- 172.21.15.86:0/3004067604 >> v2:172.21.15.83:3300/0 conn(0x7f683c15e0e0 msgr2=0x7f683c0d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:43.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f683b400640 1 --2- 172.21.15.86:0/3004067604 >> v2:172.21.15.83:3300/0 conn(0x7f683c15e0e0 0x7f683c0d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f683b400640 1 -- 172.21.15.86:0/3004067604 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c0b4b10 con 0x7f683c0cea80 2024-04-25T00:10:43.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.096+0000 7f683a000640 1 --2- 172.21.15.86:0/3004067604 >> v2:172.21.15.86:3300/0 conn(0x7f683c15e4c0 0x7f683c0ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:43.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.100+0000 7f683b400640 1 --2- 172.21.15.86:0/3004067604 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c0cea80 0x7f683c0cef10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f683c06ee00 tx=0x7f683401c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fea1d5ce6b5730c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.100+0000 7f6839600640 1 -- 172.21.15.86:0/3004067604 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68340a5840 con 0x7f683c0cea80 2024-04-25T00:10:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.100+0000 7f6839600640 1 -- 172.21.15.86:0/3004067604 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68340a59e0 con 0x7f683c0cea80 2024-04-25T00:10:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.100+0000 7f6839600640 1 -- 172.21.15.86:0/3004067604 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68340a5db0 con 0x7f683c0cea80 2024-04-25T00:10:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.100+0000 7f6841800640 1 -- 172.21.15.86:0/3004067604 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c0cea80 msgr2=0x7f683c0cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.100+0000 7f6841800640 1 --2- 172.21.15.86:0/3004067604 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c0cea80 0x7f683c0cef10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f683c06ee00 tx=0x7f683401c9c0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.100+0000 7f6841800640 1 -- 172.21.15.86:0/3004067604 shutdown_connections 2024-04-25T00:10:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.100+0000 7f6841800640 1 --2- 172.21.15.86:0/3004067604 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c0cea80 0x7f683c0cef10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.100+0000 7f6841800640 1 --2- 172.21.15.86:0/3004067604 >> v2:172.21.15.86:3300/0 conn(0x7f683c15e4c0 0x7f683c0ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.100+0000 7f6841800640 1 --2- 172.21.15.86:0/3004067604 >> v2:172.21.15.83:3300/0 conn(0x7f683c15e0e0 0x7f683c0d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.100+0000 7f6841800640 1 -- 172.21.15.86:0/3004067604 >> 172.21.15.86:0/3004067604 conn(0x7f683c0d1cc0 msgr2=0x7f683c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.100+0000 7f6841800640 1 -- 172.21.15.86:0/3004067604 shutdown_connections 2024-04-25T00:10:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.100+0000 7f6841800640 1 -- 172.21.15.86:0/3004067604 wait complete. 2024-04-25T00:10:43.104 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:43.101+0000 7fecae24b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:10:43.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.100+0000 7f6841800640 1 Processor -- start 2024-04-25T00:10:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6841800640 1 -- start start 2024-04-25T00:10:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6841800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f683c0cea80 0x7f683c0cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6841800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f683c15e0e0 0x7f683c0d0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6841800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c15e4c0 0x7f683c147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6841800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f683c06ce90 con 0x7f683c15e0e0 2024-04-25T00:10:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6841800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f683c06cd10 con 0x7f683c0cea80 2024-04-25T00:10:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6841800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f683c06d010 con 0x7f683c15e4c0 2024-04-25T00:10:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f683aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f683c0cea80 0x7f683c0cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f683b400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c15e4c0 0x7f683c147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f683aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f683c0cea80 0x7f683c0cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45358/0 (socket says 172.21.15.86:45358) 2024-04-25T00:10:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f683aa00640 1 -- 172.21.15.86:0/1083240028 learned_addr learned my addr 172.21.15.86:0/1083240028 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T00:10:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f683a000640 1 --2- 172.21.15.86:0/1083240028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f683c15e0e0 0x7f683c0d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f683aa00640 1 -- 172.21.15.86:0/1083240028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c15e4c0 msgr2=0x7f683c147410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f683aa00640 1 --2- 172.21.15.86:0/1083240028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c15e4c0 0x7f683c147410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f683aa00640 1 -- 172.21.15.86:0/1083240028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f683c15e0e0 msgr2=0x7f683c0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f683aa00640 1 --2- 172.21.15.86:0/1083240028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f683c15e0e0 0x7f683c0d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f683aa00640 1 -- 172.21.15.86:0/1083240028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c159650 con 0x7f683c0cea80 2024-04-25T00:10:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f683aa00640 1 --2- 172.21.15.86:0/1083240028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f683c0cea80 0x7f683c0cfec0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f682c00e960 tx=0x7f682c00ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6832a00640 1 -- 172.21.15.86:0/1083240028 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c017070 con 0x7f683c0cea80 2024-04-25T00:10:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6832a00640 1 -- 172.21.15.86:0/1083240028 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f682c00ca00 con 0x7f683c0cea80 2024-04-25T00:10:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6832a00640 1 -- 172.21.15.86:0/1083240028 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c00cdb0 con 0x7f683c0cea80 2024-04-25T00:10:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6841800640 1 -- 172.21.15.86:0/1083240028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683c159930 con 0x7f683c0cea80 2024-04-25T00:10:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6841800640 1 -- 172.21.15.86:0/1083240028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f683c159ef0 con 0x7f683c0cea80 2024-04-25T00:10:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6832a00640 1 -- 172.21.15.86:0/1083240028 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f682c01a070 con 0x7f683c0cea80 2024-04-25T00:10:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6832a00640 1 -- 172.21.15.86:0/1083240028 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f682c013980 con 0x7f683c0cea80 2024-04-25T00:10:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.104+0000 7f6841800640 1 -- 172.21.15.86:0/1083240028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f683c04f510 con 0x7f683c0cea80 2024-04-25T00:10:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.108+0000 7f6832a00640 1 -- 172.21.15.86:0/1083240028 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f682c013470 con 0x7f683c0cea80 2024-04-25T00:10:43.147 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:43.144+0000 7f0220643180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:10:43.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.176+0000 7f6841800640 1 -- 172.21.15.86:0/1083240028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dce8a21c-99a4-403f-83ff-ae3a6477b6e3", "id": 4} v 0) -- 0x7f683c0b46d0 con 0x7f683c0cea80 2024-04-25T00:10:43.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.180+0000 7f6832a00640 1 -- 172.21.15.86:0/1083240028 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dce8a21c-99a4-403f-83ff-ae3a6477b6e3", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f682c013650 con 0x7f683c0cea80 2024-04-25T00:10:43.181 INFO:teuthology.orchestra.run.smithi086.stdout:4 2024-04-25T00:10:43.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.180+0000 7f6841800640 1 -- 172.21.15.86:0/1083240028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f683c0cea80 msgr2=0x7f683c0cfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.180+0000 7f6841800640 1 --2- 172.21.15.86:0/1083240028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f683c0cea80 0x7f683c0cfec0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f682c00e960 tx=0x7f682c00ee20 comp rx=0 tx=0).stop 2024-04-25T00:10:43.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.180+0000 7f6841800640 1 -- 172.21.15.86:0/1083240028 shutdown_connections 2024-04-25T00:10:43.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.180+0000 7f6841800640 1 --2- 172.21.15.86:0/1083240028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c15e4c0 0x7f683c147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.180+0000 7f6841800640 1 --2- 172.21.15.86:0/1083240028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f683c15e0e0 0x7f683c0d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.180+0000 7f6841800640 1 --2- 172.21.15.86:0/1083240028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f683c0cea80 0x7f683c0cfec0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.180+0000 7f6841800640 1 -- 172.21.15.86:0/1083240028 >> 172.21.15.86:0/1083240028 conn(0x7f683c0d1cc0 msgr2=0x7f683c0706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:43.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.180+0000 7f6841800640 1 -- 172.21.15.86:0/1083240028 shutdown_connections 2024-04-25T00:10:43.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.180+0000 7f6841800640 1 -- 172.21.15.86:0/1083240028 wait complete. 2024-04-25T00:10:43.191 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:43.189+0000 7fecae24b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:10:43.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 31352365-f5e7-41ee-9423-d44ec6db1f57 5 2024-04-25T00:10:43.241 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:43.240+0000 7f0220643180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:10:43.251 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:43.249+0000 7fecae24b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:10:43.303 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:43.300+0000 7f0220643180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:10:43.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.332+0000 7f9150000640 1 Processor -- start 2024-04-25T00:10:43.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.332+0000 7f9150000640 1 -- start start 2024-04-25T00:10:43.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.336+0000 7f9150000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f914815e4c0 0x7f91480d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.336+0000 7f9150000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f91480cec50 0x7f91480cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.336+0000 7f9150000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91480b4b10 con 0x7f914815e0e0 2024-04-25T00:10:43.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.336+0000 7f9150000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f91480d0f50 con 0x7f914815e4c0 2024-04-25T00:10:43.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.336+0000 7f9150000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f91480cc310 con 0x7f91480cec50 2024-04-25T00:10:43.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.336+0000 7f914cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f914815e4c0 0x7f91480d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.336+0000 7f914e000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f91480cec50 0x7f91480cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.336+0000 7f914d600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f914815e0e0 0x7f91480ce540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50382/0 (socket says 172.21.15.86:50382) 2024-04-25T00:10:43.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.336+0000 7f914d600640 1 -- 172.21.15.86:0/3192978607 learned_addr learned my addr 172.21.15.86:0/3192978607 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T00:10:43.338 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:43.337+0000 7fecae24b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:10:43.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.336+0000 7f914e000640 1 -- 172.21.15.86:0/3192978607 >> v2:172.21.15.86:3300/0 conn(0x7f914815e4c0 msgr2=0x7f91480d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:43.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.336+0000 7f914e000640 1 --2- 172.21.15.86:0/3192978607 >> v2:172.21.15.86:3300/0 conn(0x7f914815e4c0 0x7f91480d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.336+0000 7f914e000640 1 -- 172.21.15.86:0/3192978607 >> v1:172.21.15.86:6789/0 conn(0x7f914815e0e0 legacy=0x7f91480ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.336+0000 7f914e000640 1 -- 172.21.15.86:0/3192978607 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91480d1180 con 0x7f91480cec50 2024-04-25T00:10:43.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.336+0000 7f914cc00640 1 --2- 172.21.15.86:0/3192978607 >> v2:172.21.15.86:3300/0 conn(0x7f914815e4c0 0x7f91480d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:43.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.340+0000 7f914e000640 1 --2- 172.21.15.86:0/3192978607 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f91480cec50 0x7f91480cd040 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9140009d90 tx=0x7f914001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c96978aab13f7fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:43.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.340+0000 7f9147e00640 1 -- 172.21.15.86:0/3192978607 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91400b2070 con 0x7f91480cec50 2024-04-25T00:10:43.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.340+0000 7f9147e00640 1 -- 172.21.15.86:0/3192978607 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91400a5920 con 0x7f91480cec50 2024-04-25T00:10:43.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.340+0000 7f9147e00640 1 -- 172.21.15.86:0/3192978607 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91400a5cf0 con 0x7f91480cec50 2024-04-25T00:10:43.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.340+0000 7f9150000640 1 -- 172.21.15.86:0/3192978607 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f91480cec50 msgr2=0x7f91480cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.340+0000 7f9150000640 1 --2- 172.21.15.86:0/3192978607 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f91480cec50 0x7f91480cd040 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9140009d90 tx=0x7f914001c970 comp rx=0 tx=0).stop 2024-04-25T00:10:43.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.340+0000 7f9150000640 1 -- 172.21.15.86:0/3192978607 shutdown_connections 2024-04-25T00:10:43.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.340+0000 7f9150000640 1 --2- 172.21.15.86:0/3192978607 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f91480cec50 0x7f91480cd040 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.340+0000 7f9150000640 1 --2- 172.21.15.86:0/3192978607 >> v2:172.21.15.86:3300/0 conn(0x7f914815e4c0 0x7f91480d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.340+0000 7f9150000640 1 -- 172.21.15.86:0/3192978607 >> 172.21.15.86:0/3192978607 conn(0x7f91480d1cc0 msgr2=0x7f91480d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:43.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.340+0000 7f9150000640 1 -- 172.21.15.86:0/3192978607 shutdown_connections 2024-04-25T00:10:43.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.340+0000 7f9150000640 1 -- 172.21.15.86:0/3192978607 wait complete. 2024-04-25T00:10:43.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.340+0000 7f9150000640 1 Processor -- start 2024-04-25T00:10:43.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.340+0000 7f9150000640 1 -- start start 2024-04-25T00:10:43.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f9150000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f91480cec50 0x7f9148157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f9150000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914815e0e0 0x7f91481520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f9150000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f914815e4c0 0x7f91480c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f9150000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f91480ce160 con 0x7f914815e0e0 2024-04-25T00:10:43.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f9150000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91480cdfe0 con 0x7f914815e4c0 2024-04-25T00:10:43.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f9150000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f914816aed0 con 0x7f91480cec50 2024-04-25T00:10:43.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f914e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f914815e4c0 0x7f91480c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f914d600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f91480cec50 0x7f9148157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f914d600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f91480cec50 0x7f9148157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:43566/0 (socket says 172.21.15.86:43566) 2024-04-25T00:10:43.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f914e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f914815e4c0 0x7f91480c57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45378/0 (socket says 172.21.15.86:45378) 2024-04-25T00:10:43.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f914d600640 1 -- 172.21.15.86:0/2226552953 learned_addr learned my addr 172.21.15.86:0/2226552953 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T00:10:43.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f914cc00640 1 --2- 172.21.15.86:0/2226552953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914815e0e0 0x7f91481520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f914d600640 1 -- 172.21.15.86:0/2226552953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f914815e4c0 msgr2=0x7f91480c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f914d600640 1 --2- 172.21.15.86:0/2226552953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f914815e4c0 0x7f91480c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f914d600640 1 -- 172.21.15.86:0/2226552953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914815e0e0 msgr2=0x7f91481520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f914d600640 1 --2- 172.21.15.86:0/2226552953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914815e0e0 0x7f91481520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f914d600640 1 -- 172.21.15.86:0/2226552953 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91480c5d30 con 0x7f91480cec50 2024-04-25T00:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f914d600640 1 --2- 172.21.15.86:0/2226552953 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f91480cec50 0x7f9148157df0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f913c00c3a0 tx=0x7f913c00c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f9145600640 1 -- 172.21.15.86:0/2226552953 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c017070 con 0x7f91480cec50 2024-04-25T00:10:43.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f9145600640 1 -- 172.21.15.86:0/2226552953 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f913c005cf0 con 0x7f91480cec50 2024-04-25T00:10:43.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f9150000640 1 -- 172.21.15.86:0/2226552953 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91480c5fd0 con 0x7f91480cec50 2024-04-25T00:10:43.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f9145600640 1 -- 172.21.15.86:0/2226552953 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c0072f0 con 0x7f91480cec50 2024-04-25T00:10:43.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f9150000640 1 -- 172.21.15.86:0/2226552953 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f91480c6590 con 0x7f91480cec50 2024-04-25T00:10:43.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f9145600640 1 -- 172.21.15.86:0/2226552953 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f913c01a070 con 0x7f91480cec50 2024-04-25T00:10:43.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f9145600640 1 -- 172.21.15.86:0/2226552953 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f913c012d90 con 0x7f91480cec50 2024-04-25T00:10:43.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.344+0000 7f9150000640 1 -- 172.21.15.86:0/2226552953 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f914804f510 con 0x7f91480cec50 2024-04-25T00:10:43.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.348+0000 7f9145600640 1 -- 172.21.15.86:0/2226552953 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f91480b4b10 con 0x7f91480cec50 2024-04-25T00:10:43.396 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:43.392+0000 7f0220643180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:10:43.400 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:43.401+0000 7fecae24b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:10:43.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.412+0000 7f9150000640 1 -- 172.21.15.86:0/2226552953 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "31352365-f5e7-41ee-9423-d44ec6db1f57", "id": 5} v 0) -- 0x7f91481618e0 con 0x7f91480cec50 2024-04-25T00:10:43.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.420+0000 7f9145600640 1 -- 172.21.15.86:0/2226552953 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "31352365-f5e7-41ee-9423-d44ec6db1f57", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f913c007490 con 0x7f91480cec50 2024-04-25T00:10:43.420 INFO:teuthology.orchestra.run.smithi086.stdout:5 2024-04-25T00:10:43.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.420+0000 7f9150000640 1 -- 172.21.15.86:0/2226552953 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f91480cec50 msgr2=0x7f9148157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.420+0000 7f9150000640 1 --2- 172.21.15.86:0/2226552953 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f91480cec50 0x7f9148157df0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f913c00c3a0 tx=0x7f913c00c860 comp rx=0 tx=0).stop 2024-04-25T00:10:43.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.420+0000 7f9150000640 1 -- 172.21.15.86:0/2226552953 shutdown_connections 2024-04-25T00:10:43.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.420+0000 7f9150000640 1 --2- 172.21.15.86:0/2226552953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f914815e4c0 0x7f91480c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.420+0000 7f9150000640 1 --2- 172.21.15.86:0/2226552953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914815e0e0 0x7f91481520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.420+0000 7f9150000640 1 --2- 172.21.15.86:0/2226552953 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f91480cec50 0x7f9148157df0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.420+0000 7f9150000640 1 -- 172.21.15.86:0/2226552953 >> 172.21.15.86:0/2226552953 conn(0x7f91480d1cc0 msgr2=0x7f91480b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:43.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.420+0000 7f9150000640 1 -- 172.21.15.86:0/2226552953 shutdown_connections 2024-04-25T00:10:43.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.420+0000 7f9150000640 1 -- 172.21.15.86:0/2226552953 wait complete. 2024-04-25T00:10:43.442 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 16fb48a3-c013-4db3-bdd3-34041bfec53d 6 2024-04-25T00:10:43.464 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:43.460+0000 7f0220643180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:10:43.565 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:43.565+0000 7fecae24b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:10:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.596+0000 7fe8d8e00640 1 Processor -- start 2024-04-25T00:10:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.596+0000 7fe8d8e00640 1 -- start start 2024-04-25T00:10:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.596+0000 7fe8d8e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8d415c160 0x7fe8d40d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.596+0000 7fe8d8e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe8d40cd7e0 0x7fe8d40d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.596+0000 7fe8d8e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d415c540 con 0x7fe8d40d0280 2024-04-25T00:10:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.596+0000 7fe8d8e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe8d40cf660 con 0x7fe8d40cd7e0 2024-04-25T00:10:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.596+0000 7fe8d8e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d40d0c20 con 0x7fe8d415c160 2024-04-25T00:10:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.596+0000 7fe8d1e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8d415c160 0x7fe8d40d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.600+0000 7fe8d1e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8d415c160 0x7fe8d40d27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:43576/0 (socket says 172.21.15.86:43576) 2024-04-25T00:10:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.600+0000 7fe8d1e00640 1 -- 172.21.15.86:0/1447809021 learned_addr learned my addr 172.21.15.86:0/1447809021 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T00:10:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.600+0000 7fe8d2800640 1 --2- 172.21.15.86:0/1447809021 >> v2:172.21.15.83:3300/0 conn(0x7fe8d40cd7e0 0x7fe8d40d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.600+0000 7fe8d1e00640 1 -- 172.21.15.86:0/1447809021 >> v2:172.21.15.83:3300/0 conn(0x7fe8d40cd7e0 msgr2=0x7fe8d40d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.600+0000 7fe8d1e00640 1 --2- 172.21.15.86:0/1447809021 >> v2:172.21.15.83:3300/0 conn(0x7fe8d40cd7e0 0x7fe8d40d06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.600+0000 7fe8d1e00640 1 -- 172.21.15.86:0/1447809021 >> v1:172.21.15.83:6789/0 conn(0x7fe8d40d0280 legacy=0x7fe8d40b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.600+0000 7fe8d1e00640 1 -- 172.21.15.86:0/1447809021 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d40cfa30 con 0x7fe8d415c160 2024-04-25T00:10:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.600+0000 7fe8d1e00640 1 --2- 172.21.15.86:0/1447809021 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8d415c160 0x7fe8d40d27e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009870 tx=0x7fe8c401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d63983815aeeb42a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.600+0000 7fe8d0a00640 1 -- 172.21.15.86:0/1447809021 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40ab650 con 0x7fe8d415c160 2024-04-25T00:10:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.600+0000 7fe8d0a00640 1 -- 172.21.15.86:0/1447809021 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c40b0070 con 0x7fe8d415c160 2024-04-25T00:10:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.600+0000 7fe8d0a00640 1 -- 172.21.15.86:0/1447809021 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40aba10 con 0x7fe8d415c160 2024-04-25T00:10:43.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.604+0000 7fe8d8e00640 1 -- 172.21.15.86:0/1447809021 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8d415c160 msgr2=0x7fe8d40d27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.604+0000 7fe8d8e00640 1 --2- 172.21.15.86:0/1447809021 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8d415c160 0x7fe8d40d27e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009870 tx=0x7fe8c401c560 comp rx=0 tx=0).stop 2024-04-25T00:10:43.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.604+0000 7fe8d8e00640 1 -- 172.21.15.86:0/1447809021 shutdown_connections 2024-04-25T00:10:43.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.604+0000 7fe8d8e00640 1 --2- 172.21.15.86:0/1447809021 >> v2:172.21.15.83:3300/0 conn(0x7fe8d40cd7e0 0x7fe8d40d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.604+0000 7fe8d8e00640 1 --2- 172.21.15.86:0/1447809021 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8d415c160 0x7fe8d40d27e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.604+0000 7fe8d8e00640 1 -- 172.21.15.86:0/1447809021 >> 172.21.15.86:0/1447809021 conn(0x7fe8d40d1cc0 msgr2=0x7fe8d40d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:43.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.604+0000 7fe8d8e00640 1 -- 172.21.15.86:0/1447809021 shutdown_connections 2024-04-25T00:10:43.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.604+0000 7fe8d8e00640 1 -- 172.21.15.86:0/1447809021 wait complete. 2024-04-25T00:10:43.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.604+0000 7fe8d8e00640 1 Processor -- start 2024-04-25T00:10:43.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.604+0000 7fe8d8e00640 1 -- start start 2024-04-25T00:10:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d40cd7e0 0x7fe8d4164e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8d40d0280 0x7fe8d40c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d8e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8d415c160 0x7fe8d4070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d8e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d40b2de0 con 0x7fe8d40d0280 2024-04-25T00:10:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d8e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d40b19a0 con 0x7fe8d40cd7e0 2024-04-25T00:10:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d8e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d40b2f60 con 0x7fe8d415c160 2024-04-25T00:10:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d2800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8d415c160 0x7fe8d4070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d40cd7e0 0x7fe8d4164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8d40d0280 0x7fe8d40c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d40cd7e0 0x7fe8d4164e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45390/0 (socket says 172.21.15.86:45390) 2024-04-25T00:10:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d1e00640 1 -- 172.21.15.86:0/2648979947 learned_addr learned my addr 172.21.15.86:0/2648979947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T00:10:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d1e00640 1 -- 172.21.15.86:0/2648979947 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8d415c160 msgr2=0x7fe8d4070280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d1e00640 1 --2- 172.21.15.86:0/2648979947 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8d415c160 0x7fe8d4070280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d1e00640 1 -- 172.21.15.86:0/2648979947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8d40d0280 msgr2=0x7fe8d40c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d1e00640 1 --2- 172.21.15.86:0/2648979947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8d40d0280 0x7fe8d40c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d1e00640 1 -- 172.21.15.86:0/2648979947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d4159800 con 0x7fe8d40cd7e0 2024-04-25T00:10:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d1e00640 1 --2- 172.21.15.86:0/2648979947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d40cd7e0 0x7fe8d4164e60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40ad040 tx=0x7fe8c4019300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8ca000640 1 -- 172.21.15.86:0/2648979947 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40bf610 con 0x7fe8d40cd7e0 2024-04-25T00:10:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8ca000640 1 -- 172.21.15.86:0/2648979947 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c40bf7d0 con 0x7fe8d40cd7e0 2024-04-25T00:10:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d8e00640 1 -- 172.21.15.86:0/2648979947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d4158220 con 0x7fe8d40cd7e0 2024-04-25T00:10:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8ca000640 1 -- 172.21.15.86:0/2648979947 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40bfbc0 con 0x7fe8d40cd7e0 2024-04-25T00:10:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d8e00640 1 -- 172.21.15.86:0/2648979947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d41587e0 con 0x7fe8d40cd7e0 2024-04-25T00:10:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8ca000640 1 -- 172.21.15.86:0/2648979947 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe8c40bd020 con 0x7fe8d40cd7e0 2024-04-25T00:10:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8ca000640 1 -- 172.21.15.86:0/2648979947 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fe8c40bfdb0 con 0x7fe8d40cd7e0 2024-04-25T00:10:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.608+0000 7fe8d8e00640 1 -- 172.21.15.86:0/2648979947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d404f510 con 0x7fe8d40cd7e0 2024-04-25T00:10:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.612+0000 7fe8ca000640 1 -- 172.21.15.86:0/2648979947 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe8c4016150 con 0x7fe8d40cd7e0 2024-04-25T00:10:43.637 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:43.637+0000 7fecae24b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:10:43.639 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:43.636+0000 7f0220643180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:10:43.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.680+0000 7fe8d8e00640 1 -- 172.21.15.86:0/2648979947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "16fb48a3-c013-4db3-bdd3-34041bfec53d", "id": 6} v 0) -- 0x7fe8d40748c0 con 0x7fe8d40cd7e0 2024-04-25T00:10:43.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.688+0000 7fe8ca000640 1 -- 172.21.15.86:0/2648979947 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "16fb48a3-c013-4db3-bdd3-34041bfec53d", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fe8c40b5790 con 0x7fe8d40cd7e0 2024-04-25T00:10:43.689 INFO:teuthology.orchestra.run.smithi086.stdout:6 2024-04-25T00:10:43.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.688+0000 7fe8d8e00640 1 -- 172.21.15.86:0/2648979947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d40cd7e0 msgr2=0x7fe8d4164e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.688+0000 7fe8d8e00640 1 --2- 172.21.15.86:0/2648979947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d40cd7e0 0x7fe8d4164e60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40ad040 tx=0x7fe8c4019300 comp rx=0 tx=0).stop 2024-04-25T00:10:43.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.688+0000 7fe8d8e00640 1 -- 172.21.15.86:0/2648979947 shutdown_connections 2024-04-25T00:10:43.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.688+0000 7fe8d8e00640 1 --2- 172.21.15.86:0/2648979947 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8d415c160 0x7fe8d4070280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.688+0000 7fe8d8e00640 1 --2- 172.21.15.86:0/2648979947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8d40d0280 0x7fe8d40c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.688+0000 7fe8d8e00640 1 --2- 172.21.15.86:0/2648979947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d40cd7e0 0x7fe8d4164e60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.688+0000 7fe8d8e00640 1 -- 172.21.15.86:0/2648979947 >> 172.21.15.86:0/2648979947 conn(0x7fe8d40d1cc0 msgr2=0x7fe8d4157450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:43.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.688+0000 7fe8d8e00640 1 -- 172.21.15.86:0/2648979947 shutdown_connections 2024-04-25T00:10:43.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.688+0000 7fe8d8e00640 1 -- 172.21.15.86:0/2648979947 wait complete. 2024-04-25T00:10:43.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29 7 2024-04-25T00:10:43.714 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:43.712+0000 7f0220643180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:10:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.840+0000 7f3e78a00640 1 Processor -- start 2024-04-25T00:10:43.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.840+0000 7f3e78a00640 1 -- start start 2024-04-25T00:10:43.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.840+0000 7f3e78a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e7415e0e0 0x7f3e740d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.840+0000 7f3e78a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e7415e4c0 0x7f3e740ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.840+0000 7f3e78a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e740cea80 0x7f3e740cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.840+0000 7f3e78a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3e740d0f50 con 0x7f3e7415e0e0 2024-04-25T00:10:43.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.840+0000 7f3e78a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3e740cc310 con 0x7f3e740cea80 2024-04-25T00:10:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.840+0000 7f3e78a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3e740ccbb0 con 0x7f3e7415e4c0 2024-04-25T00:10:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.840+0000 7f3e6be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e7415e4c0 0x7f3e740ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.840+0000 7f3e6be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e7415e4c0 0x7f3e740ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:43598/0 (socket says 172.21.15.86:43598) 2024-04-25T00:10:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.840+0000 7f3e6be00640 1 -- 172.21.15.86:0/2141291680 learned_addr learned my addr 172.21.15.86:0/2141291680 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T00:10:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.840+0000 7f3e72800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e740cea80 0x7f3e740cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e71e00640 1 --2- 172.21.15.86:0/2141291680 >> v2:172.21.15.83:3300/0 conn(0x7f3e7415e0e0 0x7f3e740d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e6be00640 1 -- 172.21.15.86:0/2141291680 >> v2:172.21.15.86:3300/0 conn(0x7f3e740cea80 msgr2=0x7f3e740cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e6be00640 1 --2- 172.21.15.86:0/2141291680 >> v2:172.21.15.86:3300/0 conn(0x7f3e740cea80 0x7f3e740cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e6be00640 1 -- 172.21.15.86:0/2141291680 >> v2:172.21.15.83:3300/0 conn(0x7f3e7415e0e0 msgr2=0x7f3e740d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e6be00640 1 --2- 172.21.15.86:0/2141291680 >> v2:172.21.15.83:3300/0 conn(0x7f3e7415e0e0 0x7f3e740d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e6be00640 1 -- 172.21.15.86:0/2141291680 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e740b4b10 con 0x7f3e7415e4c0 2024-04-25T00:10:43.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e71e00640 1 --2- 172.21.15.86:0/2141291680 >> v2:172.21.15.83:3300/0 conn(0x7f3e7415e0e0 0x7f3e740d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:43.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e6be00640 1 --2- 172.21.15.86:0/2141291680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e7415e4c0 0x7f3e740ce540 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3e7415e920 tx=0x7f3e5c01c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e0f2b5b139b00a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:43.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e71400640 1 -- 172.21.15.86:0/2141291680 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0aa610 con 0x7f3e7415e4c0 2024-04-25T00:10:43.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e71400640 1 -- 172.21.15.86:0/2141291680 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e5c0af070 con 0x7f3e7415e4c0 2024-04-25T00:10:43.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e71400640 1 -- 172.21.15.86:0/2141291680 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0aa9d0 con 0x7f3e7415e4c0 2024-04-25T00:10:43.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e78a00640 1 -- 172.21.15.86:0/2141291680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e7415e4c0 msgr2=0x7f3e740ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e78a00640 1 --2- 172.21.15.86:0/2141291680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e7415e4c0 0x7f3e740ce540 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3e7415e920 tx=0x7f3e5c01c4f0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e78a00640 1 -- 172.21.15.86:0/2141291680 shutdown_connections 2024-04-25T00:10:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e78a00640 1 --2- 172.21.15.86:0/2141291680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e7415e4c0 0x7f3e740ce540 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e78a00640 1 --2- 172.21.15.86:0/2141291680 >> v2:172.21.15.86:3300/0 conn(0x7f3e740cea80 0x7f3e740cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e78a00640 1 --2- 172.21.15.86:0/2141291680 >> v2:172.21.15.83:3300/0 conn(0x7f3e7415e0e0 0x7f3e740d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.844+0000 7f3e78a00640 1 -- 172.21.15.86:0/2141291680 >> 172.21.15.86:0/2141291680 conn(0x7f3e740d1cc0 msgr2=0x7f3e740d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e78a00640 1 -- 172.21.15.86:0/2141291680 shutdown_connections 2024-04-25T00:10:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e78a00640 1 -- 172.21.15.86:0/2141291680 wait complete. 2024-04-25T00:10:43.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e78a00640 1 Processor -- start 2024-04-25T00:10:43.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e78a00640 1 -- start start 2024-04-25T00:10:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e78a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e740cea80 0x7f3e74158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e78a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e7415e0e0 0x7f3e7406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e78a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e7415e4c0 0x7f3e74158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e78a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7406ce90 con 0x7f3e740cea80 2024-04-25T00:10:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e78a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7406cd10 con 0x7f3e7415e4c0 2024-04-25T00:10:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e78a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7406d010 con 0x7f3e7415e0e0 2024-04-25T00:10:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e6be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e7415e0e0 0x7f3e7406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e6be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e7415e0e0 0x7f3e7406fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:43606/0 (socket says 172.21.15.86:43606) 2024-04-25T00:10:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e6be00640 1 -- 172.21.15.86:0/2676434768 learned_addr learned my addr 172.21.15.86:0/2676434768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T00:10:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e72800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e7415e4c0 0x7f3e74158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e71e00640 1 --2- 172.21.15.86:0/2676434768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e740cea80 0x7f3e74158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:43.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e6be00640 1 -- 172.21.15.86:0/2676434768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e7415e4c0 msgr2=0x7f3e74158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e6be00640 1 --2- 172.21.15.86:0/2676434768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e7415e4c0 0x7f3e74158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e6be00640 1 -- 172.21.15.86:0/2676434768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e740cea80 msgr2=0x7f3e74158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e6be00640 1 --2- 172.21.15.86:0/2676434768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e740cea80 0x7f3e74158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e6be00640 1 -- 172.21.15.86:0/2676434768 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e74159920 con 0x7f3e7415e0e0 2024-04-25T00:10:43.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.848+0000 7f3e6be00640 1 --2- 172.21.15.86:0/2676434768 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e7415e0e0 0x7f3e7406fb60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c01aba0 tx=0x7f3e5c01ad30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:43.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.852+0000 7f3e6a000640 1 -- 172.21.15.86:0/2676434768 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0aae60 con 0x7f3e7415e0e0 2024-04-25T00:10:43.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.852+0000 7f3e6a000640 1 -- 172.21.15.86:0/2676434768 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e5c0bd610 con 0x7f3e7415e0e0 2024-04-25T00:10:43.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.852+0000 7f3e78a00640 1 -- 172.21.15.86:0/2676434768 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e74159c00 con 0x7f3e7415e0e0 2024-04-25T00:10:43.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.852+0000 7f3e6a000640 1 -- 172.21.15.86:0/2676434768 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0bda00 con 0x7f3e7415e0e0 2024-04-25T00:10:43.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.852+0000 7f3e78a00640 1 -- 172.21.15.86:0/2676434768 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e7415a1c0 con 0x7f3e7415e0e0 2024-04-25T00:10:43.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.852+0000 7f3e6a000640 1 -- 172.21.15.86:0/2676434768 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3e5c0b33f0 con 0x7f3e7415e0e0 2024-04-25T00:10:43.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.852+0000 7f3e6a000640 1 -- 172.21.15.86:0/2676434768 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f3e5c0b3d40 con 0x7f3e7415e0e0 2024-04-25T00:10:43.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.852+0000 7f3e78a00640 1 -- 172.21.15.86:0/2676434768 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7404f510 con 0x7f3e7415e0e0 2024-04-25T00:10:43.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.852+0000 7f3e6a000640 1 -- 172.21.15.86:0/2676434768 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3e5c0c3060 con 0x7f3e7415e0e0 2024-04-25T00:10:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.920+0000 7f3e78a00640 1 -- 172.21.15.86:0/2676434768 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29", "id": 7} v 0) -- 0x7f3e7406d630 con 0x7f3e7415e0e0 2024-04-25T00:10:43.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.924+0000 7f3e6a000640 1 -- 172.21.15.86:0/2676434768 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f3e740b4b10 con 0x7f3e7415e0e0 2024-04-25T00:10:43.928 INFO:teuthology.orchestra.run.smithi086.stdout:7 2024-04-25T00:10:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.928+0000 7f3e78a00640 1 -- 172.21.15.86:0/2676434768 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e7415e0e0 msgr2=0x7f3e7406fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.928+0000 7f3e78a00640 1 --2- 172.21.15.86:0/2676434768 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e7415e0e0 0x7f3e7406fb60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c01aba0 tx=0x7f3e5c01ad30 comp rx=0 tx=0).stop 2024-04-25T00:10:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.928+0000 7f3e78a00640 1 -- 172.21.15.86:0/2676434768 shutdown_connections 2024-04-25T00:10:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.928+0000 7f3e78a00640 1 --2- 172.21.15.86:0/2676434768 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e7415e0e0 0x7f3e7406fb60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.928+0000 7f3e78a00640 1 --2- 172.21.15.86:0/2676434768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e7415e4c0 0x7f3e74158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.928+0000 7f3e78a00640 1 --2- 172.21.15.86:0/2676434768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e740cea80 0x7f3e74158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.928+0000 7f3e78a00640 1 -- 172.21.15.86:0/2676434768 >> 172.21.15.86:0/2676434768 conn(0x7f3e740d1cc0 msgr2=0x7f3e740707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.928+0000 7f3e78a00640 1 -- 172.21.15.86:0/2676434768 shutdown_connections 2024-04-25T00:10:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:10:43.928+0000 7f3e78a00640 1 -- 172.21.15.86:0/2676434768 wait complete. 2024-04-25T00:10:43.946 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T00:10:43.946 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T00:10:43.951 INFO:tasks.ceph.osd.0:Started 2024-04-25T00:10:43.952 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T00:10:43.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T00:10:43.960 INFO:tasks.ceph.osd.1:Started 2024-04-25T00:10:43.961 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T00:10:43.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T00:10:43.965 INFO:tasks.ceph.osd.2:Started 2024-04-25T00:10:43.965 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T00:10:43.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T00:10:43.975 INFO:tasks.ceph.osd.3:Started 2024-04-25T00:10:43.975 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T00:10:43.975 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T00:10:43.991 INFO:tasks.ceph.osd.4:Started 2024-04-25T00:10:43.991 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T00:10:43.991 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T00:10:43.994 INFO:tasks.ceph.osd.5:Started 2024-04-25T00:10:43.994 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T00:10:43.994 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T00:10:43.997 INFO:tasks.ceph.osd.6:Started 2024-04-25T00:10:43.997 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T00:10:43.998 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T00:10:44.000 INFO:tasks.ceph.osd.7:Started 2024-04-25T00:10:44.001 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:10:44.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7ab2a00640 1 Processor -- start 2024-04-25T00:10:44.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7ab2a00640 1 -- start start 2024-04-25T00:10:44.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7ab2a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aac1599f0 0x7f7aac0cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:44.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7ab2a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7aac15ce90 0x7f7aac0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:44.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7ab2a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aac0ccc60 con 0x7f7aac15c340 2024-04-25T00:10:44.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7ab2a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7aac159dd0 con 0x7f7aac15ce90 2024-04-25T00:10:44.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7ab2a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7aac06dba0 con 0x7f7aac1599f0 2024-04-25T00:10:44.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7ab0a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7aac15ce90 0x7f7aac0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:44.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7aa3e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7aac15c340 0x7f7aac15c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54154/0 (socket says 172.21.15.83:54154) 2024-04-25T00:10:44.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7aa3e00640 1 -- 172.21.15.83:0/4074977967 learned_addr learned my addr 172.21.15.83:0/4074977967 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:10:44.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7aabe00640 1 --2- 172.21.15.83:0/4074977967 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aac1599f0 0x7f7aac0cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:44.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7ab0a00640 1 -- 172.21.15.83:0/4074977967 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aac1599f0 msgr2=0x7f7aac0cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:44.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7ab0a00640 1 --2- 172.21.15.83:0/4074977967 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aac1599f0 0x7f7aac0cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:44.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7ab0a00640 1 -- 172.21.15.83:0/4074977967 >> v1:172.21.15.83:6789/0 conn(0x7f7aac15c340 legacy=0x7f7aac15c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:44.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7ab0a00640 1 -- 172.21.15.83:0/4074977967 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac06de80 con 0x7f7aac15ce90 2024-04-25T00:10:44.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.177+0000 7f7aab400640 1 -- 172.21.15.83:0/4074977967 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 291517059 0 0) 0x7f7a94001120 con 0x7f7aac15c340 2024-04-25T00:10:44.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.181+0000 7f7ab0a00640 1 --2- 172.21.15.83:0/4074977967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15ce90 0x7f7aac0ce6e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009dc0 tx=0x7f7aa401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a395f70486c369ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:44.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.181+0000 7f7aab400640 1 -- 172.21.15.83:0/4074977967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40b2070 con 0x7f7aac15ce90 2024-04-25T00:10:44.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.181+0000 7f7aab400640 1 -- 172.21.15.83:0/4074977967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa40a5780 con 0x7f7aac15ce90 2024-04-25T00:10:44.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.181+0000 7f7aab400640 1 -- 172.21.15.83:0/4074977967 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40a5b50 con 0x7f7aac15ce90 2024-04-25T00:10:44.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 -- 172.21.15.83:0/4074977967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15ce90 msgr2=0x7f7aac0ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:44.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 --2- 172.21.15.83:0/4074977967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15ce90 0x7f7aac0ce6e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009dc0 tx=0x7f7aa401c8c0 comp rx=0 tx=0).stop 2024-04-25T00:10:44.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 -- 172.21.15.83:0/4074977967 shutdown_connections 2024-04-25T00:10:44.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 --2- 172.21.15.83:0/4074977967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15ce90 0x7f7aac0ce6e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:44.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 --2- 172.21.15.83:0/4074977967 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aac1599f0 0x7f7aac0cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:44.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 -- 172.21.15.83:0/4074977967 >> 172.21.15.83:0/4074977967 conn(0x7f7aac0b1830 msgr2=0x7f7aac0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:44.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 -- 172.21.15.83:0/4074977967 shutdown_connections 2024-04-25T00:10:44.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 -- 172.21.15.83:0/4074977967 wait complete. 2024-04-25T00:10:44.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 Processor -- start 2024-04-25T00:10:44.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 -- start start 2024-04-25T00:10:44.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aac1599f0 0x7f7aac1662c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:44.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aac15c340 0x7f7aac159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:44.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15ce90 0x7f7aac1554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:44.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac0cf7c0 con 0x7f7aac15ce90 2024-04-25T00:10:44.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac0cf640 con 0x7f7aac15c340 2024-04-25T00:10:44.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab2a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7aac168cd0 con 0x7f7aac1599f0 2024-04-25T00:10:44.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15ce90 0x7f7aac1554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:44.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7aabe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aac1599f0 0x7f7aac1662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:44.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15ce90 0x7f7aac1554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56942/0 (socket says 172.21.15.83:56942) 2024-04-25T00:10:44.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab0a00640 1 -- 172.21.15.83:0/3815723012 learned_addr learned my addr 172.21.15.83:0/3815723012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:10:44.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7aa3e00640 1 --2- 172.21.15.83:0/3815723012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aac15c340 0x7f7aac159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:44.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab0a00640 1 -- 172.21.15.83:0/3815723012 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aac1599f0 msgr2=0x7f7aac1662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:44.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab0a00640 1 --2- 172.21.15.83:0/3815723012 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aac1599f0 0x7f7aac1662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:44.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab0a00640 1 -- 172.21.15.83:0/3815723012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aac15c340 msgr2=0x7f7aac159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:44.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab0a00640 1 --2- 172.21.15.83:0/3815723012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aac15c340 0x7f7aac159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:44.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7ab0a00640 1 -- 172.21.15.83:0/3815723012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac15a4a0 con 0x7f7aac15ce90 2024-04-25T00:10:44.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.185+0000 7f7aabe00640 1 --2- 172.21.15.83:0/3815723012 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aac1599f0 0x7f7aac1662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.189+0000 7f7ab0a00640 1 --2- 172.21.15.83:0/3815723012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15ce90 0x7f7aac1554e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7aa401cda0 tx=0x7f7aa4019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.189+0000 7f7aa8c00640 1 -- 172.21.15.83:0/3815723012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40b2050 con 0x7f7aac15ce90 2024-04-25T00:10:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.189+0000 7f7aa8c00640 1 -- 172.21.15.83:0/3815723012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa40be070 con 0x7f7aac15ce90 2024-04-25T00:10:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.189+0000 7f7aa8c00640 1 -- 172.21.15.83:0/3815723012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40ad870 con 0x7f7aac15ce90 2024-04-25T00:10:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.189+0000 7f7ab2a00640 1 -- 172.21.15.83:0/3815723012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aac15a740 con 0x7f7aac15ce90 2024-04-25T00:10:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.189+0000 7f7ab2a00640 1 -- 172.21.15.83:0/3815723012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aac15ad00 con 0x7f7aac15ce90 2024-04-25T00:10:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.189+0000 7f7aa8c00640 1 -- 172.21.15.83:0/3815723012 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7aa40ada10 con 0x7f7aac15ce90 2024-04-25T00:10:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.189+0000 7f7aa8c00640 1 -- 172.21.15.83:0/3815723012 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7aa40add00 con 0x7f7aac15ce90 2024-04-25T00:10:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.189+0000 7f7ab2a00640 1 -- 172.21.15.83:0/3815723012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aac0737f0 con 0x7f7aac15ce90 2024-04-25T00:10:44.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.189+0000 7f7aa8c00640 1 -- 172.21.15.83:0/3815723012 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7aa40c7020 con 0x7f7aac15ce90 2024-04-25T00:10:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.249+0000 7f7ab2a00640 1 -- 172.21.15.83:0/3815723012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7aac0b4b70 con 0x7f7aac15ce90 2024-04-25T00:10:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.249+0000 7f7aa8c00640 1 -- 172.21.15.83:0/3815723012 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7aa4016540 con 0x7f7aac15ce90 2024-04-25T00:10:44.250 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:10:44.250 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":10,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:10:43.925460+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:10:44.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.249+0000 7f7ab2a00640 1 -- 172.21.15.83:0/3815723012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15ce90 msgr2=0x7f7aac1554e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:44.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.249+0000 7f7ab2a00640 1 --2- 172.21.15.83:0/3815723012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15ce90 0x7f7aac1554e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7aa401cda0 tx=0x7f7aa4019660 comp rx=0 tx=0).stop 2024-04-25T00:10:44.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.249+0000 7f7ab2a00640 1 -- 172.21.15.83:0/3815723012 shutdown_connections 2024-04-25T00:10:44.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.249+0000 7f7ab2a00640 1 --2- 172.21.15.83:0/3815723012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15ce90 0x7f7aac1554e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:44.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.249+0000 7f7ab2a00640 1 --2- 172.21.15.83:0/3815723012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aac15c340 0x7f7aac159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:44.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.249+0000 7f7ab2a00640 1 --2- 172.21.15.83:0/3815723012 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aac1599f0 0x7f7aac1662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.249+0000 7f7ab2a00640 1 -- 172.21.15.83:0/3815723012 >> 172.21.15.83:0/3815723012 conn(0x7f7aac0b1830 msgr2=0x7f7aac155e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.249+0000 7f7ab2a00640 1 -- 172.21.15.83:0/3815723012 shutdown_connections 2024-04-25T00:10:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:44.249+0000 7f7ab2a00640 1 -- 172.21.15.83:0/3815723012 wait complete. 2024-04-25T00:10:44.269 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T00:10:44.269 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T00:10:44.347 INFO:tasks.ceph.mgr.y.smithi083.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T00:10:44.347 INFO:tasks.ceph.mgr.y.smithi083.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T00:10:44.347 INFO:tasks.ceph.mgr.y.smithi083.stderr: from numpy import show_config as show_numpy_config 2024-04-25T00:10:44.397 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:44.397+0000 7fecae24b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:10:44.416 INFO:tasks.ceph.mgr.x.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T00:10:44.416 INFO:tasks.ceph.mgr.x.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T00:10:44.416 INFO:tasks.ceph.mgr.x.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T00:10:44.455 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:44.452+0000 7f0220643180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:10:44.544 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:44.545+0000 7fecae24b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:10:44.602 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:44.600+0000 7f0220643180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:10:44.629 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-04-25T00:10:44.629+0000 7feb0e8f57c0 -1 Falling back to public interface 2024-04-25T00:10:44.650 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-04-25T00:10:44.649+0000 7facb47887c0 -1 Falling back to public interface 2024-04-25T00:10:44.650 INFO:tasks.ceph.osd.3.smithi083.stderr:2024-04-25T00:10:44.649+0000 7f2f4282b7c0 -1 Falling back to public interface 2024-04-25T00:10:44.653 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-04-25T00:10:44.652+0000 7fda06e3d7c0 -1 Falling back to public interface 2024-04-25T00:10:44.654 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-04-25T00:10:44.652+0000 7f3457e2b7c0 -1 Falling back to public interface 2024-04-25T00:10:44.665 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-04-25T00:10:44.664+0000 7fb3b4c2b7c0 -1 Falling back to public interface 2024-04-25T00:10:44.669 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-04-25T00:10:44.668+0000 7f24a6c2b7c0 -1 Falling back to public interface 2024-04-25T00:10:44.673 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-04-25T00:10:44.673+0000 7fcb774f37c0 -1 Falling back to public interface 2024-04-25T00:10:44.725 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:44.725+0000 7fecae24b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:10:44.792 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:44.788+0000 7f0220643180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:10:44.809 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:44.809+0000 7fecae24b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:10:44.864 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:44.860+0000 7f0220643180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:10:44.957 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:44.957+0000 7fecae24b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:10:45.015 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:45.012+0000 7f0220643180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:10:45.074 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T00:10:45.201 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd122400640 1 Processor -- start 2024-04-25T00:10:45.201 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd122400640 1 -- start start 2024-04-25T00:10:45.204 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd122400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd11c0ced00 0x7fd11c0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:45.204 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd122400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd11c071060 0x7fd11c0714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:45.204 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd122400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd11c06f690 con 0x7fd11c070c80 2024-04-25T00:10:45.204 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd122400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd11c0cdcf0 con 0x7fd11c071060 2024-04-25T00:10:45.205 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd122400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd11c0cdf40 con 0x7fd11c0ced00 2024-04-25T00:10:45.205 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd11be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd11c071060 0x7fd11c0714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:45.205 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd11b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd11c070c80 0x7fd11c0b29d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:57820/0 (socket says 172.21.15.83:57820) 2024-04-25T00:10:45.205 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd11b400640 1 -- 172.21.15.83:0/1362101656 learned_addr learned my addr 172.21.15.83:0/1362101656 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:10:45.205 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd113e00640 1 --2- 172.21.15.83:0/1362101656 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd11c0ced00 0x7fd11c0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:45.205 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd11aa00640 1 -- 172.21.15.83:0/1362101656 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2127354845 0 0) 0x7fd11c06f690 con 0x7fd11c070c80 2024-04-25T00:10:45.205 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd11aa00640 1 -- 172.21.15.83:0/1362101656 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd100005740 con 0x7fd11c070c80 2024-04-25T00:10:45.205 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd113e00640 1 -- 172.21.15.83:0/1362101656 >> v2:172.21.15.86:3300/0 conn(0x7fd11c071060 msgr2=0x7fd11c0714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:45.206 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd113e00640 1 --2- 172.21.15.83:0/1362101656 >> v2:172.21.15.86:3300/0 conn(0x7fd11c071060 0x7fd11c0714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:45.206 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd113e00640 1 -- 172.21.15.83:0/1362101656 >> v1:172.21.15.86:6789/0 conn(0x7fd11c070c80 legacy=0x7fd11c0b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:45.206 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd113e00640 1 -- 172.21.15.83:0/1362101656 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11c0ce220 con 0x7fd11c0ced00 2024-04-25T00:10:45.206 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.201+0000 7fd11be00640 1 --2- 172.21.15.83:0/1362101656 >> v2:172.21.15.86:3300/0 conn(0x7fd11c071060 0x7fd11c0714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:45.208 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.205+0000 7fd113e00640 1 --2- 172.21.15.83:0/1362101656 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd11c0ced00 0x7fd11c0cf160 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd1040097c0 tx=0x7fd10401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6dd005273101ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:45.208 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.205+0000 7fd11aa00640 1 -- 172.21.15.83:0/1362101656 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1040b0070 con 0x7fd11c0ced00 2024-04-25T00:10:45.208 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.205+0000 7fd11aa00640 1 -- 172.21.15.83:0/1362101656 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1040ab600 con 0x7fd11c0ced00 2024-04-25T00:10:45.208 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.205+0000 7fd11aa00640 1 -- 172.21.15.83:0/1362101656 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1040ab9b0 con 0x7fd11c0ced00 2024-04-25T00:10:45.208 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.205+0000 7fd122400640 1 -- 172.21.15.83:0/1362101656 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd11c0ced00 msgr2=0x7fd11c0cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:45.208 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.205+0000 7fd122400640 1 --2- 172.21.15.83:0/1362101656 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd11c0ced00 0x7fd11c0cf160 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd1040097c0 tx=0x7fd10401c410 comp rx=0 tx=0).stop 2024-04-25T00:10:45.208 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.205+0000 7fd122400640 1 -- 172.21.15.83:0/1362101656 shutdown_connections 2024-04-25T00:10:45.208 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.205+0000 7fd122400640 1 --2- 172.21.15.83:0/1362101656 >> v2:172.21.15.86:3300/0 conn(0x7fd11c071060 0x7fd11c0714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:45.209 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.205+0000 7fd122400640 1 --2- 172.21.15.83:0/1362101656 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd11c0ced00 0x7fd11c0cf160 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:45.209 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.205+0000 7fd122400640 1 -- 172.21.15.83:0/1362101656 >> 172.21.15.83:0/1362101656 conn(0x7fd11c0b16d0 msgr2=0x7fd11c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:45.209 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.205+0000 7fd122400640 1 -- 172.21.15.83:0/1362101656 shutdown_connections 2024-04-25T00:10:45.209 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.205+0000 7fd122400640 1 -- 172.21.15.83:0/1362101656 wait complete. 2024-04-25T00:10:45.212 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd122400640 1 Processor -- start 2024-04-25T00:10:45.213 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd122400640 1 -- start start 2024-04-25T00:10:45.213 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd122400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd11c070c80 0x7fd11c13fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:45.213 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd122400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd11c071060 0x7fd11c1412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:45.213 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd122400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd11c0ced00 0x7fd11c154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:45.213 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd122400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c072690 con 0x7fd11c070c80 2024-04-25T00:10:45.213 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd122400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c072510 con 0x7fd11c0ced00 2024-04-25T00:10:45.214 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd122400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd11c072810 con 0x7fd11c071060 2024-04-25T00:10:45.214 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd11b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd11c070c80 0x7fd11c13fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:45.214 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd11b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd11c070c80 0x7fd11c13fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40134/0 (socket says 172.21.15.83:40134) 2024-04-25T00:10:45.214 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd11b400640 1 -- 172.21.15.83:0/1908389248 learned_addr learned my addr 172.21.15.83:0/1908389248 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:10:45.214 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd113e00640 1 --2- 172.21.15.83:0/1908389248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd11c071060 0x7fd11c1412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:45.214 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd11be00640 1 --2- 172.21.15.83:0/1908389248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd11c0ced00 0x7fd11c154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:45.214 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd113e00640 1 -- 172.21.15.83:0/1908389248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd11c0ced00 msgr2=0x7fd11c154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:45.214 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd113e00640 1 --2- 172.21.15.83:0/1908389248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd11c0ced00 0x7fd11c154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:45.215 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd113e00640 1 -- 172.21.15.83:0/1908389248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd11c070c80 msgr2=0x7fd11c13fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:45.215 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd113e00640 1 --2- 172.21.15.83:0/1908389248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd11c070c80 0x7fd11c13fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:45.215 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd113e00640 1 -- 172.21.15.83:0/1908389248 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11c140200 con 0x7fd11c071060 2024-04-25T00:10:45.215 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd11be00640 1 --2- 172.21.15.83:0/1908389248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd11c0ced00 0x7fd11c154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:45.215 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd11b400640 1 --2- 172.21.15.83:0/1908389248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd11c070c80 0x7fd11c13fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:45.215 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd113e00640 1 --2- 172.21.15.83:0/1908389248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd11c071060 0x7fd11c1412a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd1040ab000 tx=0x7fd1040191b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:45.215 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd113400640 1 -- 172.21.15.83:0/1908389248 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1040b0040 con 0x7fd11c071060 2024-04-25T00:10:45.216 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd113400640 1 -- 172.21.15.83:0/1908389248 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1040ab400 con 0x7fd11c071060 2024-04-25T00:10:45.216 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd113400640 1 -- 172.21.15.83:0/1908389248 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104006e20 con 0x7fd11c071060 2024-04-25T00:10:45.216 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd122400640 1 -- 172.21.15.83:0/1908389248 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11c1404e0 con 0x7fd11c071060 2024-04-25T00:10:45.216 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd122400640 1 -- 172.21.15.83:0/1908389248 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd11c140aa0 con 0x7fd11c071060 2024-04-25T00:10:45.216 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd113400640 1 -- 172.21.15.83:0/1908389248 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd1040bb020 con 0x7fd11c071060 2024-04-25T00:10:45.216 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.209+0000 7fd113400640 1 -- 172.21.15.83:0/1908389248 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd1040ab5b0 con 0x7fd11c071060 2024-04-25T00:10:45.216 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.213+0000 7fd122400640 1 -- 172.21.15.83:0/1908389248 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd11c075360 con 0x7fd11c071060 2024-04-25T00:10:45.217 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.213+0000 7fd113400640 1 -- 172.21.15.83:0/1908389248 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd1040c2020 con 0x7fd11c071060 2024-04-25T00:10:45.272 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.269+0000 7fd122400640 1 -- 172.21.15.83:0/1908389248 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd11c0d0540 con 0x7fd11c071060 2024-04-25T00:10:45.273 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.273+0000 7fd113400640 1 -- 172.21.15.83:0/1908389248 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd104016090 con 0x7fd11c071060 2024-04-25T00:10:45.274 INFO:teuthology.misc.health.smithi083.stdout: 2024-04-25T00:10:45.274 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":10,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:10:43.925460+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:10:45.275 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.273+0000 7fd122400640 1 -- 172.21.15.83:0/1908389248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd11c071060 msgr2=0x7fd11c1412a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:45.275 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.273+0000 7fd122400640 1 --2- 172.21.15.83:0/1908389248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd11c071060 0x7fd11c1412a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd1040ab000 tx=0x7fd1040191b0 comp rx=0 tx=0).stop 2024-04-25T00:10:45.276 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.273+0000 7fd122400640 1 -- 172.21.15.83:0/1908389248 shutdown_connections 2024-04-25T00:10:45.276 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.273+0000 7fd122400640 1 --2- 172.21.15.83:0/1908389248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd11c071060 0x7fd11c1412a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:45.276 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.273+0000 7fd122400640 1 --2- 172.21.15.83:0/1908389248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd11c0ced00 0x7fd11c154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:45.276 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.273+0000 7fd122400640 1 --2- 172.21.15.83:0/1908389248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd11c070c80 0x7fd11c13fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:45.276 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.273+0000 7fd122400640 1 -- 172.21.15.83:0/1908389248 >> 172.21.15.83:0/1908389248 conn(0x7fd11c0b16d0 msgr2=0x7fd11c0adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:45.276 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.273+0000 7fd122400640 1 -- 172.21.15.83:0/1908389248 shutdown_connections 2024-04-25T00:10:45.276 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:45.273+0000 7fd122400640 1 -- 172.21.15.83:0/1908389248 wait complete. 2024-04-25T00:10:45.290 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T00:10:45.662 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:45.657+0000 7fecae24b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:10:45.707 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:45.704+0000 7f0220643180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:10:45.744 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:45.741+0000 7fecae24b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:10:45.793 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:45.792+0000 7f0220643180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:10:45.946 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:45.945+0000 7fecae24b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:10:45.995 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:45.992+0000 7f0220643180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:10:46.030 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:46.029+0000 7fecae24b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:10:46.079 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:46.076+0000 7f0220643180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:10:46.356 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:46.353+0000 7fecae24b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:10:46.408 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:46.404+0000 7f0220643180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:10:46.426 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:46.425+0000 7fecae24b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:10:46.484 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-25T00:10:46.480+0000 7f0220643180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:10:47.453 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:47.453+0000 7fec53800640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T00:10:47.456 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-25T00:10:47.453+0000 7fec53800640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T00:10:48.559 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-04-25T00:10:48.557+0000 7feb0e8f57c0 -1 osd.0 0 log_to_monitors true 2024-04-25T00:10:48.600 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-04-25T00:10:48.596+0000 7fda06e3d7c0 -1 osd.6 0 log_to_monitors true 2024-04-25T00:10:48.617 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-04-25T00:10:48.616+0000 7f3457e2b7c0 -1 osd.5 0 log_to_monitors true 2024-04-25T00:10:48.642 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-04-25T00:10:48.640+0000 7f24a6c2b7c0 -1 osd.7 0 log_to_monitors true 2024-04-25T00:10:48.676 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-04-25T00:10:48.676+0000 7fb3b4c2b7c0 -1 osd.4 0 log_to_monitors true 2024-04-25T00:10:48.702 INFO:tasks.ceph.osd.3.smithi083.stderr:2024-04-25T00:10:48.701+0000 7f2f4282b7c0 -1 osd.3 0 log_to_monitors true 2024-04-25T00:10:48.721 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-04-25T00:10:48.721+0000 7fcb774f37c0 -1 osd.2 0 log_to_monitors true 2024-04-25T00:10:48.755 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-04-25T00:10:48.753+0000 7facb47887c0 -1 osd.1 0 log_to_monitors true 2024-04-25T00:10:52.097 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T00:10:52.283 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.281+0000 7f825d800640 1 Processor -- start 2024-04-25T00:10:52.283 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.281+0000 7f825d800640 1 -- start start 2024-04-25T00:10:52.285 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f825d800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f825815dfb0 0x7f825815e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:52.285 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f825d800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f825806bb10 0x7f825806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:52.285 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f825d800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f82580cfe20 0x7f82580d0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:52.285 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f825d800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f825806c4b0 con 0x7f825815dfb0 2024-04-25T00:10:52.285 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f825d800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f82580d0c50 con 0x7f82580cfe20 2024-04-25T00:10:52.285 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f825d800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f82580d0f30 con 0x7f825806bb10 2024-04-25T00:10:52.286 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f8257400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f82580cfe20 0x7f82580d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:52.286 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f8257400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f82580cfe20 0x7f82580d0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:56904/0 (socket says 172.21.15.83:56904) 2024-04-25T00:10:52.286 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f8257400640 1 -- 172.21.15.83:0/4265731260 learned_addr learned my addr 172.21.15.83:0/4265731260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:10:52.287 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f8256000640 1 --2- 172.21.15.83:0/4265731260 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f825806bb10 0x7f825806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:52.287 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f8257400640 1 -- 172.21.15.83:0/4265731260 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f825806bb10 msgr2=0x7f825806bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:52.287 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f8257400640 1 --2- 172.21.15.83:0/4265731260 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f825806bb10 0x7f825806bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:52.287 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f8257400640 1 -- 172.21.15.83:0/4265731260 >> v2:172.21.15.83:3300/0 conn(0x7f825815dfb0 msgr2=0x7f825815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:52.287 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f8256a00640 1 --2- 172.21.15.83:0/4265731260 >> v2:172.21.15.83:3300/0 conn(0x7f825815dfb0 0x7f825815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:52.287 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f8257400640 1 --2- 172.21.15.83:0/4265731260 >> v2:172.21.15.83:3300/0 conn(0x7f825815dfb0 0x7f825815e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:52.287 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f8257400640 1 -- 172.21.15.83:0/4265731260 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82580d1210 con 0x7f82580cfe20 2024-04-25T00:10:52.287 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f8256000640 1 --2- 172.21.15.83:0/4265731260 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f825806bb10 0x7f825806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:52.287 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.285+0000 7f8256a00640 1 --2- 172.21.15.83:0/4265731260 >> v2:172.21.15.83:3300/0 conn(0x7f825815dfb0 0x7f825815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:52.291 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.289+0000 7f8257400640 1 --2- 172.21.15.83:0/4265731260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82580cfe20 0x7f82580d0260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f82580b7d10 tx=0x7f825001c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=423d4a3102afcfb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:52.292 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.289+0000 7f8255600640 1 -- 172.21.15.83:0/4265731260 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82500a57b0 con 0x7f82580cfe20 2024-04-25T00:10:52.292 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.289+0000 7f8255600640 1 -- 172.21.15.83:0/4265731260 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82500a5950 con 0x7f82580cfe20 2024-04-25T00:10:52.292 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.289+0000 7f8255600640 1 -- 172.21.15.83:0/4265731260 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82500a5d20 con 0x7f82580cfe20 2024-04-25T00:10:52.294 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 -- 172.21.15.83:0/4265731260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82580cfe20 msgr2=0x7f82580d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:52.294 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 --2- 172.21.15.83:0/4265731260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82580cfe20 0x7f82580d0260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f82580b7d10 tx=0x7f825001c930 comp rx=0 tx=0).stop 2024-04-25T00:10:52.294 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 -- 172.21.15.83:0/4265731260 shutdown_connections 2024-04-25T00:10:52.294 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 --2- 172.21.15.83:0/4265731260 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f825806bb10 0x7f825806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:52.294 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 --2- 172.21.15.83:0/4265731260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82580cfe20 0x7f82580d0260 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:52.295 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 --2- 172.21.15.83:0/4265731260 >> v2:172.21.15.83:3300/0 conn(0x7f825815dfb0 0x7f825815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:52.295 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 -- 172.21.15.83:0/4265731260 >> 172.21.15.83:0/4265731260 conn(0x7f825809aa70 msgr2=0x7f825809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:52.295 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 -- 172.21.15.83:0/4265731260 shutdown_connections 2024-04-25T00:10:52.295 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 -- 172.21.15.83:0/4265731260 wait complete. 2024-04-25T00:10:52.298 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 Processor -- start 2024-04-25T00:10:52.298 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 -- start start 2024-04-25T00:10:52.298 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f825806bb10 0x7f82580c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:52.299 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82580cfe20 0x7f825814aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:52.299 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f825815dfb0 0x7f825814af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:52.299 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82580cf0b0 con 0x7f825815dfb0 2024-04-25T00:10:52.299 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82580cef30 con 0x7f825806bb10 2024-04-25T00:10:52.299 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f825d800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f82580cf230 con 0x7f82580cfe20 2024-04-25T00:10:52.299 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f8256000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82580cfe20 0x7f825814aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:52.300 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f8256a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f825806bb10 0x7f82580c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:52.300 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f8256000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82580cfe20 0x7f825814aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:41130/0 (socket says 172.21.15.83:41130) 2024-04-25T00:10:52.300 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.293+0000 7f8256000640 1 -- 172.21.15.83:0/91184707 learned_addr learned my addr 172.21.15.83:0/91184707 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:10:52.300 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f8257400640 1 --2- 172.21.15.83:0/91184707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f825815dfb0 0x7f825814af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:52.300 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f8256a00640 1 -- 172.21.15.83:0/91184707 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82580cfe20 msgr2=0x7f825814aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:52.300 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f8256a00640 1 --2- 172.21.15.83:0/91184707 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82580cfe20 0x7f825814aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:52.301 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f8256a00640 1 -- 172.21.15.83:0/91184707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f825815dfb0 msgr2=0x7f825814af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:52.301 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f8256a00640 1 --2- 172.21.15.83:0/91184707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f825815dfb0 0x7f825814af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:52.301 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f8256a00640 1 -- 172.21.15.83:0/91184707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825815b430 con 0x7f825806bb10 2024-04-25T00:10:52.301 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f8256000640 1 --2- 172.21.15.83:0/91184707 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82580cfe20 0x7f825814aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:52.301 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f8256a00640 1 --2- 172.21.15.83:0/91184707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f825806bb10 0x7f82580c57a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f824800c820 tx=0x7f824800cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:52.302 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f824ea00640 1 -- 172.21.15.83:0/91184707 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8248015070 con 0x7f825806bb10 2024-04-25T00:10:52.302 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f824ea00640 1 -- 172.21.15.83:0/91184707 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8248004220 con 0x7f825806bb10 2024-04-25T00:10:52.302 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f824ea00640 1 -- 172.21.15.83:0/91184707 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824800ce10 con 0x7f825806bb10 2024-04-25T00:10:52.302 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f825d800640 1 -- 172.21.15.83:0/91184707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825815b710 con 0x7f825806bb10 2024-04-25T00:10:52.304 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f825d800640 1 -- 172.21.15.83:0/91184707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825815bcd0 con 0x7f825806bb10 2024-04-25T00:10:52.304 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f824ea00640 1 -- 172.21.15.83:0/91184707 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f8248007830 con 0x7f825806bb10 2024-04-25T00:10:52.304 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.297+0000 7f825d800640 1 -- 172.21.15.83:0/91184707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8258075360 con 0x7f825806bb10 2024-04-25T00:10:52.305 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.301+0000 7f824ea00640 1 --2- 172.21.15.83:0/91184707 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8238067830 0x7f8238069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:52.305 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.301+0000 7f824ea00640 1 -- 172.21.15.83:0/91184707 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f8248084af0 con 0x7f825806bb10 2024-04-25T00:10:52.305 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.301+0000 7f8256000640 1 --2- 172.21.15.83:0/91184707 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8238067830 0x7f8238069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:52.305 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.301+0000 7f8256000640 1 --2- 172.21.15.83:0/91184707 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8238067830 0x7f8238069cf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f82580701c0 tx=0x7f8244008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:52.318 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.317+0000 7f824ea00640 1 -- 172.21.15.83:0/91184707 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8248055e50 con 0x7f825806bb10 2024-04-25T00:10:52.627 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.625+0000 7f825d800640 1 -- 172.21.15.83:0/91184707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f825806ff10 con 0x7f825806bb10 2024-04-25T00:10:52.630 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.629+0000 7f824ea00640 1 -- 172.21.15.83:0/91184707 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f824805a150 con 0x7f825806bb10 2024-04-25T00:10:52.637 INFO:teuthology.misc.health.smithi083.stdout: 2024-04-25T00:10:52.637 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":12,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:10:50.455967+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:10:52.638 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.637+0000 7f825d800640 1 -- 172.21.15.83:0/91184707 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8238067830 msgr2=0x7f8238069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:52.638 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.637+0000 7f825d800640 1 --2- 172.21.15.83:0/91184707 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8238067830 0x7f8238069cf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f82580701c0 tx=0x7f8244008040 comp rx=0 tx=0).stop 2024-04-25T00:10:52.639 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.637+0000 7f825d800640 1 -- 172.21.15.83:0/91184707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f825806bb10 msgr2=0x7f82580c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:52.639 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.637+0000 7f825d800640 1 --2- 172.21.15.83:0/91184707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f825806bb10 0x7f82580c57a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f824800c820 tx=0x7f824800cce0 comp rx=0 tx=0).stop 2024-04-25T00:10:52.639 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.637+0000 7f825d800640 1 -- 172.21.15.83:0/91184707 shutdown_connections 2024-04-25T00:10:52.639 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.637+0000 7f825d800640 1 --2- 172.21.15.83:0/91184707 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8238067830 0x7f8238069cf0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f82580701c0 tx=0x7f8244008040 comp rx=0 tx=0).stop 2024-04-25T00:10:52.639 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.637+0000 7f825d800640 1 --2- 172.21.15.83:0/91184707 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82580cfe20 0x7f825814aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:52.639 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.637+0000 7f825d800640 1 --2- 172.21.15.83:0/91184707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f825815dfb0 0x7f825814af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:52.639 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.637+0000 7f825d800640 1 --2- 172.21.15.83:0/91184707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f825806bb10 0x7f82580c57a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:52.639 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.637+0000 7f825d800640 1 -- 172.21.15.83:0/91184707 >> 172.21.15.83:0/91184707 conn(0x7f825809aa70 msgr2=0x7f8258070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:52.644 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.641+0000 7f825d800640 1 -- 172.21.15.83:0/91184707 shutdown_connections 2024-04-25T00:10:52.644 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:52.645+0000 7f825d800640 1 -- 172.21.15.83:0/91184707 wait complete. 2024-04-25T00:10:52.672 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T00:10:52.730 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-04-25T00:10:52.724+0000 7f3452200640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T00:10:52.763 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-04-25T00:10:52.760+0000 7f3448200640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:10:52.797 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-04-25T00:10:52.792+0000 7fda01e00640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T00:10:52.814 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-04-25T00:10:52.808+0000 7f24a1000640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T00:10:52.819 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-04-25T00:10:52.816+0000 7fb3af000640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T00:10:52.821 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-04-25T00:10:52.820+0000 7fd9f7e00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:10:52.837 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-04-25T00:10:52.836+0000 7f2497000640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:10:52.857 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-04-25T00:10:52.856+0000 7fb3a5000640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:10:52.888 INFO:tasks.ceph.osd.3.smithi083.stderr:2024-04-25T00:10:52.889+0000 7f2f3d600640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T00:10:52.922 INFO:tasks.ceph.osd.3.smithi083.stderr:2024-04-25T00:10:52.921+0000 7f2f32c00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:10:52.944 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-04-25T00:10:52.941+0000 7facaf600640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T00:10:52.952 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-04-25T00:10:52.949+0000 7fcb72400640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T00:10:52.957 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-04-25T00:10:52.957+0000 7feb09800640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T00:10:52.974 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-04-25T00:10:52.973+0000 7faca5600640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:10:52.975 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-04-25T00:10:52.973+0000 7fcb68400640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:10:52.985 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-04-25T00:10:52.985+0000 7feaff800640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:10:59.479 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T00:10:59.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.609+0000 7f2868800640 1 Processor -- start 2024-04-25T00:10:59.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.609+0000 7f2868800640 1 -- start start 2024-04-25T00:10:59.613 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.609+0000 7f2868800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f286013ee30 0x7f28600ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.613 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.613+0000 7f2868800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f286006f690 con 0x7f2860160210 2024-04-25T00:10:59.613 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.613+0000 7f2868800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28600ceca0 con 0x7f28600ce760 2024-04-25T00:10:59.613 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.613+0000 7f2868800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f286006cda0 con 0x7f286013ee30 2024-04-25T00:10:59.613 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.613+0000 7f2865e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f286013ee30 0x7f28600ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.613 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.613+0000 7f2865e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f286013ee30 0x7f28600ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55030/0 (socket says 172.21.15.83:55030) 2024-04-25T00:10:59.613 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.613+0000 7f2865e00640 1 -- 172.21.15.83:0/697250591 learned_addr learned my addr 172.21.15.83:0/697250591 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:10:59.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.613+0000 7f2865e00640 1 -- 172.21.15.83:0/697250591 >> v1:172.21.15.86:6789/0 conn(0x7f28600ce760 legacy=0x7f286015fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.613+0000 7f2865e00640 1 -- 172.21.15.83:0/697250591 >> v1:172.21.15.83:6789/0 conn(0x7f2860160210 legacy=0x7f286006d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.613+0000 7f2865e00640 1 -- 172.21.15.83:0/697250591 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28600cef80 con 0x7f286013ee30 2024-04-25T00:10:59.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.613+0000 7f2864a00640 1 -- 172.21.15.83:0/697250591 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 617102707 0 0) 0x7f286006f690 con 0x7f2860160210 2024-04-25T00:10:59.616 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.613+0000 7f2865e00640 1 --2- 172.21.15.83:0/697250591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f286013ee30 0x7f28600ce220 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f286006cf20 tx=0x7f285401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466d065ba5c5f7c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:59.617 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.613+0000 7f2864a00640 1 -- 172.21.15.83:0/697250591 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28540a95f0 con 0x7f286013ee30 2024-04-25T00:10:59.617 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.613+0000 7f2864a00640 1 -- 172.21.15.83:0/697250591 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28540a9790 con 0x7f286013ee30 2024-04-25T00:10:59.617 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.613+0000 7f2864a00640 1 -- 172.21.15.83:0/697250591 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28540a9b60 con 0x7f286013ee30 2024-04-25T00:10:59.617 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 -- 172.21.15.83:0/697250591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f286013ee30 msgr2=0x7f28600ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.617 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 --2- 172.21.15.83:0/697250591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f286013ee30 0x7f28600ce220 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f286006cf20 tx=0x7f285401c510 comp rx=0 tx=0).stop 2024-04-25T00:10:59.617 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 -- 172.21.15.83:0/697250591 shutdown_connections 2024-04-25T00:10:59.617 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 --2- 172.21.15.83:0/697250591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f286013ee30 0x7f28600ce220 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.618 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 -- 172.21.15.83:0/697250591 >> 172.21.15.83:0/697250591 conn(0x7f28600b16d0 msgr2=0x7f28600b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:59.618 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 -- 172.21.15.83:0/697250591 shutdown_connections 2024-04-25T00:10:59.618 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 -- 172.21.15.83:0/697250591 wait complete. 2024-04-25T00:10:59.618 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 Processor -- start 2024-04-25T00:10:59.619 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 -- start start 2024-04-25T00:10:59.621 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28600ce760 0x7f286013d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.621 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f286013ee30 0x7f2860155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.621 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860160210 0x7f28601564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.621 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f286006e2d0 con 0x7f286013ee30 2024-04-25T00:10:59.622 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f286006e150 con 0x7f2860160210 2024-04-25T00:10:59.622 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2868800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f286006e450 con 0x7f28600ce760 2024-04-25T00:10:59.622 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2865400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f286013ee30 0x7f2860155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.622 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2865400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f286013ee30 0x7f2860155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55044/0 (socket says 172.21.15.83:55044) 2024-04-25T00:10:59.622 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2865400640 1 -- 172.21.15.83:0/3667101089 learned_addr learned my addr 172.21.15.83:0/3667101089 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:10:59.622 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2865e00640 1 --2- 172.21.15.83:0/3667101089 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28600ce760 0x7f286013d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.622 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2865400640 1 -- 172.21.15.83:0/3667101089 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28600ce760 msgr2=0x7f286013d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.622 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2865400640 1 --2- 172.21.15.83:0/3667101089 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28600ce760 0x7f286013d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.622 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2865400640 1 -- 172.21.15.83:0/3667101089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860160210 msgr2=0x7f28601564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.622 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2865400640 1 --2- 172.21.15.83:0/3667101089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860160210 0x7f28601564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.622 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.617+0000 7f2865400640 1 -- 172.21.15.83:0/3667101089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28601391d0 con 0x7f286013ee30 2024-04-25T00:10:59.623 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.621+0000 7f2865400640 1 --2- 172.21.15.83:0/3667101089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f286013ee30 0x7f2860155f90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f285000c4d0 tx=0x7f285000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:59.623 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.621+0000 7f285a000640 1 -- 172.21.15.83:0/3667101089 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28500052d0 con 0x7f286013ee30 2024-04-25T00:10:59.623 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.621+0000 7f285a000640 1 -- 172.21.15.83:0/3667101089 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2850005470 con 0x7f286013ee30 2024-04-25T00:10:59.623 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.621+0000 7f2868800640 1 -- 172.21.15.83:0/3667101089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2860138650 con 0x7f286013ee30 2024-04-25T00:10:59.623 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.621+0000 7f285a000640 1 -- 172.21.15.83:0/3667101089 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850012830 con 0x7f286013ee30 2024-04-25T00:10:59.623 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.621+0000 7f2868800640 1 -- 172.21.15.83:0/3667101089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2860138c10 con 0x7f286013ee30 2024-04-25T00:10:59.624 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.621+0000 7f2868800640 1 -- 172.21.15.83:0/3667101089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2860075360 con 0x7f286013ee30 2024-04-25T00:10:59.624 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.621+0000 7f285a000640 1 -- 172.21.15.83:0/3667101089 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f28500129d0 con 0x7f286013ee30 2024-04-25T00:10:59.624 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.621+0000 7f285a000640 1 --2- 172.21.15.83:0/3667101089 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28440678c0 0x7f2844069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.624 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.621+0000 7f285a000640 1 -- 172.21.15.83:0/3667101089 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2850085490 con 0x7f286013ee30 2024-04-25T00:10:59.624 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.621+0000 7f2865e00640 1 --2- 172.21.15.83:0/3667101089 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28440678c0 0x7f2844069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.624 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.625+0000 7f2865e00640 1 --2- 172.21.15.83:0/3667101089 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28440678c0 0x7f2844069d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f28540ae000 tx=0x7f28540b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:59.628 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.629+0000 7f285a000640 1 -- 172.21.15.83:0/3667101089 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f28500182b0 con 0x7f286013ee30 2024-04-25T00:10:59.784 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.781+0000 7f2868800640 1 -- 172.21.15.83:0/3667101089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2860074980 con 0x7f286013ee30 2024-04-25T00:10:59.785 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.785+0000 7f285a000640 1 -- 172.21.15.83:0/3667101089 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12800 (secure 0 0 0) 0x7f28500563b0 con 0x7f286013ee30 2024-04-25T00:10:59.785 INFO:teuthology.misc.health.smithi083.stdout: 2024-04-25T00:10:59.785 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":16,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:10:56.767409+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:10:59.788 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.785+0000 7f2868800640 1 -- 172.21.15.83:0/3667101089 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28440678c0 msgr2=0x7f2844069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.788 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.785+0000 7f2868800640 1 --2- 172.21.15.83:0/3667101089 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28440678c0 0x7f2844069d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f28540ae000 tx=0x7f28540b3040 comp rx=0 tx=0).stop 2024-04-25T00:10:59.788 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.785+0000 7f2868800640 1 -- 172.21.15.83:0/3667101089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f286013ee30 msgr2=0x7f2860155f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.788 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.785+0000 7f2868800640 1 --2- 172.21.15.83:0/3667101089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f286013ee30 0x7f2860155f90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f285000c4d0 tx=0x7f285000c990 comp rx=0 tx=0).stop 2024-04-25T00:10:59.788 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.785+0000 7f2868800640 1 -- 172.21.15.83:0/3667101089 shutdown_connections 2024-04-25T00:10:59.788 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.785+0000 7f2868800640 1 --2- 172.21.15.83:0/3667101089 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28440678c0 0x7f2844069d80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.788 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.785+0000 7f2868800640 1 --2- 172.21.15.83:0/3667101089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860160210 0x7f28601564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.788 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.785+0000 7f2868800640 1 --2- 172.21.15.83:0/3667101089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f286013ee30 0x7f2860155f90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.788 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.785+0000 7f2868800640 1 --2- 172.21.15.83:0/3667101089 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28600ce760 0x7f286013d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.788 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.785+0000 7f2868800640 1 -- 172.21.15.83:0/3667101089 >> 172.21.15.83:0/3667101089 conn(0x7f28600b16d0 msgr2=0x7f286013f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:59.788 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.785+0000 7f2868800640 1 -- 172.21.15.83:0/3667101089 shutdown_connections 2024-04-25T00:10:59.788 INFO:teuthology.misc.health.smithi083.stderr:2024-04-25T00:10:59.785+0000 7f2868800640 1 -- 172.21.15.83:0/3667101089 wait complete. 2024-04-25T00:10:59.804 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T00:10:59.804 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T00:10:59.805 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:10:59.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:10:59.808 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:10:59.808 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:10:59.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:10:59.811 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:10:59.811 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:10:59.811 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:10:59.814 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:10:59.815 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:10:59.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:10:59.817 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:10:59.818 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T00:10:59.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:10:59.910 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-25T00:10:59.912 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-25T00:10:59.917 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2024-04-25T00:10:59.917 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2024-04-25T00:10:59.919 INFO:tasks.ceph.mds.b.smithi086.stderr:2024-04-25T00:10:59.916+0000 7f7771c00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T00:10:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.961+0000 7f7ad2600640 1 Processor -- start 2024-04-25T00:10:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.961+0000 7f7ad2600640 1 -- start start 2024-04-25T00:10:59.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.961+0000 7f7ad2600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7acc15e110 0x7f7acc0ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.961+0000 7f7ad2600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7acc15e4f0 0x7f7acc0ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.961+0000 7f7ad2600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7acc0ceec0 0x7f7acc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.961+0000 7f7ad2600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7acc0d0ad0 con 0x7f7acc15e110 2024-04-25T00:10:59.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.961+0000 7f7ad2600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7acc0d0db0 con 0x7f7acc0ceec0 2024-04-25T00:10:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.961+0000 7f7ad2600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7acc0cc3b0 con 0x7f7acc15e4f0 2024-04-25T00:10:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.965+0000 7f7ad1200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7acc15e110 0x7f7acc0ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.965+0000 7f7ad1200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7acc15e110 0x7f7acc0ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55086/0 (socket says 172.21.15.83:55086) 2024-04-25T00:10:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.965+0000 7f7ad1200640 1 -- 172.21.15.83:0/1828825425 learned_addr learned my addr 172.21.15.83:0/1828825425 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:10:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.965+0000 7f7acbe00640 1 --2- 172.21.15.83:0/1828825425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7acc15e4f0 0x7f7acc0ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.965+0000 7f7ad1c00640 1 --2- 172.21.15.83:0/1828825425 >> v2:172.21.15.86:3300/0 conn(0x7f7acc0ceec0 0x7f7acc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.965+0000 7f7ad1200640 1 -- 172.21.15.83:0/1828825425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7acc15e4f0 msgr2=0x7f7acc0ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.965+0000 7f7ad1200640 1 --2- 172.21.15.83:0/1828825425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7acc15e4f0 0x7f7acc0ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.965+0000 7f7ad1200640 1 -- 172.21.15.83:0/1828825425 >> v2:172.21.15.86:3300/0 conn(0x7f7acc0ceec0 msgr2=0x7f7acc06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.965+0000 7f7ad1200640 1 --2- 172.21.15.83:0/1828825425 >> v2:172.21.15.86:3300/0 conn(0x7f7acc0ceec0 0x7f7acc06edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.965+0000 7f7ad1200640 1 -- 172.21.15.83:0/1828825425 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7acc0d0760 con 0x7f7acc15e110 2024-04-25T00:10:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.965+0000 7f7ad1200640 1 --2- 172.21.15.83:0/1828825425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7acc15e110 0x7f7acc0ce440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7acc072ad0 tx=0x7f7ac001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6f88ba5f376378 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7acb400640 1 -- 172.21.15.83:0/1828825425 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac00b2070 con 0x7f7acc15e110 2024-04-25T00:10:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7acb400640 1 -- 172.21.15.83:0/1828825425 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac00a9070 con 0x7f7acc15e110 2024-04-25T00:10:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7acb400640 1 -- 172.21.15.83:0/1828825425 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac00ad880 con 0x7f7acc15e110 2024-04-25T00:10:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 -- 172.21.15.83:0/1828825425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7acc15e110 msgr2=0x7f7acc0ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 --2- 172.21.15.83:0/1828825425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7acc15e110 0x7f7acc0ce440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7acc072ad0 tx=0x7f7ac001c490 comp rx=0 tx=0).stop 2024-04-25T00:10:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 -- 172.21.15.83:0/1828825425 shutdown_connections 2024-04-25T00:10:59.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 --2- 172.21.15.83:0/1828825425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7acc15e4f0 0x7f7acc0ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 --2- 172.21.15.83:0/1828825425 >> v2:172.21.15.86:3300/0 conn(0x7f7acc0ceec0 0x7f7acc06edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 --2- 172.21.15.83:0/1828825425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7acc15e110 0x7f7acc0ce440 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 -- 172.21.15.83:0/1828825425 >> 172.21.15.83:0/1828825425 conn(0x7f7acc0d1bc0 msgr2=0x7f7acc0d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:59.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 -- 172.21.15.83:0/1828825425 shutdown_connections 2024-04-25T00:10:59.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 -- 172.21.15.83:0/1828825425 wait complete. 2024-04-25T00:10:59.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 Processor -- start 2024-04-25T00:10:59.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 -- start start 2024-04-25T00:10:59.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7acc0ceec0 0x7f7acc159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7acc15e110 0x7f7acc158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7acc15e4f0 0x7f7acc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc06fe90 con 0x7f7acc0ceec0 2024-04-25T00:10:59.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc06fd10 con 0x7f7acc15e110 2024-04-25T00:10:59.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad2600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7acc070010 con 0x7f7acc15e4f0 2024-04-25T00:10:59.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7acc0ceec0 0x7f7acc159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7acc0ceec0 0x7f7acc159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55090/0 (socket says 172.21.15.83:55090) 2024-04-25T00:10:59.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad1200640 1 -- 172.21.15.83:0/491296173 learned_addr learned my addr 172.21.15.83:0/491296173 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:10:59.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7acbe00640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7acc15e110 0x7f7acc158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad1c00640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7acc15e4f0 0x7f7acc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad1200640 1 -- 172.21.15.83:0/491296173 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7acc15e4f0 msgr2=0x7f7acc06d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad1200640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7acc15e4f0 0x7f7acc06d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad1200640 1 -- 172.21.15.83:0/491296173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7acc15e110 msgr2=0x7f7acc158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad1200640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7acc15e110 0x7f7acc158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad1200640 1 -- 172.21.15.83:0/491296173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7acc0ce5d0 con 0x7f7acc0ceec0 2024-04-25T00:10:59.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.969+0000 7f7ad1c00640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7acc15e4f0 0x7f7acc06d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:59.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.973+0000 7f7ad1200640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7acc0ceec0 0x7f7acc159470 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ac001ab40 tx=0x7f7ac0019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:59.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.973+0000 7f7ac8c00640 1 -- 172.21.15.83:0/491296173 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac00b2070 con 0x7f7acc0ceec0 2024-04-25T00:10:59.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.973+0000 7f7ac8c00640 1 -- 172.21.15.83:0/491296173 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac00ade50 con 0x7f7acc0ceec0 2024-04-25T00:10:59.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.973+0000 7f7ad2600640 1 -- 172.21.15.83:0/491296173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7acc1f7010 con 0x7f7acc0ceec0 2024-04-25T00:10:59.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.973+0000 7f7ac8c00640 1 -- 172.21.15.83:0/491296173 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac00ab990 con 0x7f7acc0ceec0 2024-04-25T00:10:59.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.973+0000 7f7ad2600640 1 -- 172.21.15.83:0/491296173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7acc1f75d0 con 0x7f7acc0ceec0 2024-04-25T00:10:59.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.973+0000 7f7ad2600640 1 -- 172.21.15.83:0/491296173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7acc04f510 con 0x7f7acc0ceec0 2024-04-25T00:10:59.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.973+0000 7f7ac8c00640 1 -- 172.21.15.83:0/491296173 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7ac00ab070 con 0x7f7acc0ceec0 2024-04-25T00:10:59.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.973+0000 7f7ac8c00640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7aac0678f0 0x7f7aac069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.973+0000 7f7ac8c00640 1 -- 172.21.15.83:0/491296173 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f7ac011f730 con 0x7f7acc0ceec0 2024-04-25T00:10:59.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.973+0000 7f7acbe00640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7aac0678f0 0x7f7aac069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.977+0000 7f7acbe00640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7aac0678f0 0x7f7aac069db0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0c8620 tx=0x7f7ab8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:59.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:10:59.977+0000 7f7ac8c00640 1 -- 172.21.15.83:0/491296173 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7ac00c5020 con 0x7f7acc0ceec0 2024-04-25T00:11:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.141+0000 7f7ad2600640 1 -- 172.21.15.83:0/491296173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7acc0d1550 con 0x7f7acc0ceec0 2024-04-25T00:11:00.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.141+0000 7f7ac8c00640 1 -- 172.21.15.83:0/491296173 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12800 (secure 0 0 0) 0x7f7ac0016110 con 0x7f7acc0ceec0 2024-04-25T00:11:00.142 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:00.143 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:10:56.767409+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:00.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.141+0000 7f7ad2600640 1 -- 172.21.15.83:0/491296173 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7aac0678f0 msgr2=0x7f7aac069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.141+0000 7f7ad2600640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7aac0678f0 0x7f7aac069db0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0c8620 tx=0x7f7ab8009190 comp rx=0 tx=0).stop 2024-04-25T00:11:00.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.141+0000 7f7ad2600640 1 -- 172.21.15.83:0/491296173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7acc0ceec0 msgr2=0x7f7acc159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.141+0000 7f7ad2600640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7acc0ceec0 0x7f7acc159470 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ac001ab40 tx=0x7f7ac0019230 comp rx=0 tx=0).stop 2024-04-25T00:11:00.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.141+0000 7f7ad2600640 1 -- 172.21.15.83:0/491296173 shutdown_connections 2024-04-25T00:11:00.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.145+0000 7f7ad2600640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7aac0678f0 0x7f7aac069db0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.145+0000 7f7ad2600640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7acc15e4f0 0x7f7acc06d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.145+0000 7f7ad2600640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7acc15e110 0x7f7acc158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.145+0000 7f7ad2600640 1 --2- 172.21.15.83:0/491296173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7acc0ceec0 0x7f7acc159470 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.145+0000 7f7ad2600640 1 -- 172.21.15.83:0/491296173 >> 172.21.15.83:0/491296173 conn(0x7f7acc0d1bc0 msgr2=0x7f7acc1571d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.145+0000 7f7ad2600640 1 -- 172.21.15.83:0/491296173 shutdown_connections 2024-04-25T00:11:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.145+0000 7f7ad2600640 1 -- 172.21.15.83:0/491296173 wait complete. 2024-04-25T00:11:00.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:11:00.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:11:00.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.297+0000 7f3ce8800640 1 Processor -- start 2024-04-25T00:11:00.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.297+0000 7f3ce8800640 1 -- start start 2024-04-25T00:11:00.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.301+0000 7f3ce8800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3ce006e950 0x7f3ce015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.301+0000 7f3ce8800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ce00ce000 con 0x7f3ce0159810 2024-04-25T00:11:00.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.301+0000 7f3ce8800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ce006eec0 con 0x7f3ce015fe50 2024-04-25T00:11:00.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.301+0000 7f3ce8800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3ce006e210 con 0x7f3ce006e950 2024-04-25T00:11:00.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.301+0000 7f3ce5e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3ce0159810 0x7f3ce00cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50490/0 (socket says 172.21.15.83:50490) 2024-04-25T00:11:00.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.301+0000 7f3ce5e00640 1 -- 172.21.15.83:0/1339868725 learned_addr learned my addr 172.21.15.83:0/1339868725 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:00.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.301+0000 7f3ce5400640 1 --2- 172.21.15.83:0/1339868725 >> v2:172.21.15.83:3300/0 conn(0x7f3ce006e950 0x7f3ce015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.301+0000 7f3ce4a00640 1 -- 172.21.15.83:0/1339868725 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3845338140 0 0) 0x7f3ce00ce000 con 0x7f3ce0159810 2024-04-25T00:11:00.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.301+0000 7f3ce4a00640 1 -- 172.21.15.83:0/1339868725 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cc8003530 con 0x7f3ce0159810 2024-04-25T00:11:00.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.301+0000 7f3ce5400640 1 -- 172.21.15.83:0/1339868725 >> v1:172.21.15.86:6789/0 conn(0x7f3ce015fe50 legacy=0x7f3ce00ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.301+0000 7f3ce5400640 1 -- 172.21.15.83:0/1339868725 >> v1:172.21.15.83:6789/0 conn(0x7f3ce0159810 legacy=0x7f3ce00cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.301+0000 7f3ce5400640 1 -- 172.21.15.83:0/1339868725 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce006dc50 con 0x7f3ce006e950 2024-04-25T00:11:00.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.305+0000 7f3ce5400640 1 --2- 172.21.15.83:0/1339868725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce006e950 0x7f3ce015f910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00097c0 tx=0x7f3cd001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fd0ce9c2c2e9e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:00.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.305+0000 7f3ce4a00640 1 -- 172.21.15.83:0/1339868725 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00b0070 con 0x7f3ce006e950 2024-04-25T00:11:00.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.305+0000 7f3ce4a00640 1 -- 172.21.15.83:0/1339868725 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd00ab600 con 0x7f3ce006e950 2024-04-25T00:11:00.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.305+0000 7f3ce4a00640 1 -- 172.21.15.83:0/1339868725 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00ab9b0 con 0x7f3ce006e950 2024-04-25T00:11:00.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.305+0000 7f3ce8800640 1 -- 172.21.15.83:0/1339868725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce006e950 msgr2=0x7f3ce015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.305+0000 7f3ce8800640 1 --2- 172.21.15.83:0/1339868725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce006e950 0x7f3ce015f910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00097c0 tx=0x7f3cd001c410 comp rx=0 tx=0).stop 2024-04-25T00:11:00.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.305+0000 7f3ce8800640 1 -- 172.21.15.83:0/1339868725 shutdown_connections 2024-04-25T00:11:00.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.305+0000 7f3ce8800640 1 --2- 172.21.15.83:0/1339868725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce006e950 0x7f3ce015f910 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.305+0000 7f3ce8800640 1 -- 172.21.15.83:0/1339868725 >> 172.21.15.83:0/1339868725 conn(0x7f3ce00b1830 msgr2=0x7f3ce00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:00.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.305+0000 7f3ce8800640 1 -- 172.21.15.83:0/1339868725 shutdown_connections 2024-04-25T00:11:00.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.305+0000 7f3ce8800640 1 -- 172.21.15.83:0/1339868725 wait complete. 2024-04-25T00:11:00.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.305+0000 7f3ce8800640 1 Processor -- start 2024-04-25T00:11:00.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.305+0000 7f3ce8800640 1 -- start start 2024-04-25T00:11:00.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce8800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce006e950 0x7f3ce014f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce8800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ce0159810 0x7f3ce0150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce8800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce015fe50 0x7f3ce0071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce8800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0169e40 con 0x7f3ce006e950 2024-04-25T00:11:00.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce8800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0169cc0 con 0x7f3ce015fe50 2024-04-25T00:11:00.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce8800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3ce0169fc0 con 0x7f3ce0159810 2024-04-25T00:11:00.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce006e950 0x7f3ce014f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce006e950 0x7f3ce014f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55112/0 (socket says 172.21.15.83:55112) 2024-04-25T00:11:00.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce5e00640 1 -- 172.21.15.83:0/2927956769 learned_addr learned my addr 172.21.15.83:0/2927956769 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:00.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce5400640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ce0159810 0x7f3ce0150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce6800640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce015fe50 0x7f3ce0071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce5e00640 1 -- 172.21.15.83:0/2927956769 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ce0159810 msgr2=0x7f3ce0150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce5e00640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ce0159810 0x7f3ce0150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce5e00640 1 -- 172.21.15.83:0/2927956769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce015fe50 msgr2=0x7f3ce0071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce5e00640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce015fe50 0x7f3ce0071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce5e00640 1 -- 172.21.15.83:0/2927956769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce00b2aa0 con 0x7f3ce006e950 2024-04-25T00:11:00.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce6800640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce015fe50 0x7f3ce0071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:00.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce5e00640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce006e950 0x7f3ce014f860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400c420 tx=0x7f3cd400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:00.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3cda000640 1 -- 172.21.15.83:0/2927956769 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4017070 con 0x7f3ce006e950 2024-04-25T00:11:00.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3cda000640 1 -- 172.21.15.83:0/2927956769 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd4005cf0 con 0x7f3ce006e950 2024-04-25T00:11:00.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3cda000640 1 -- 172.21.15.83:0/2927956769 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd40072d0 con 0x7f3ce006e950 2024-04-25T00:11:00.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce8800640 1 -- 172.21.15.83:0/2927956769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce00b0700 con 0x7f3ce006e950 2024-04-25T00:11:00.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce8800640 1 -- 172.21.15.83:0/2927956769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce00b0c40 con 0x7f3ce006e950 2024-04-25T00:11:00.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3ce8800640 1 -- 172.21.15.83:0/2927956769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce004f510 con 0x7f3ce006e950 2024-04-25T00:11:00.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.309+0000 7f3cda000640 1 -- 172.21.15.83:0/2927956769 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3cd401a070 con 0x7f3ce006e950 2024-04-25T00:11:00.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.313+0000 7f3cda000640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cc40678c0 0x7f3cc4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.313+0000 7f3cda000640 1 -- 172.21.15.83:0/2927956769 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f3cd4084bc0 con 0x7f3ce006e950 2024-04-25T00:11:00.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.313+0000 7f3ce5400640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cc40678c0 0x7f3cc4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.313+0000 7f3ce5400640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cc40678c0 0x7f3cc4069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00ad000 tx=0x7f3cd00a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:00.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.317+0000 7f3cda000640 1 -- 172.21.15.83:0/2927956769 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3cd40554b0 con 0x7f3ce006e950 2024-04-25T00:11:00.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.473+0000 7f3ce8800640 1 -- 172.21.15.83:0/2927956769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3ce0072c90 con 0x7f3ce006e950 2024-04-25T00:11:00.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.473+0000 7f3cda000640 1 -- 172.21.15.83:0/2927956769 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f3cd40597b0 con 0x7f3ce006e950 2024-04-25T00:11:00.474 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:11:00.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.473+0000 7f3ce8800640 1 -- 172.21.15.83:0/2927956769 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cc40678c0 msgr2=0x7f3cc4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.473+0000 7f3ce8800640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cc40678c0 0x7f3cc4069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00ad000 tx=0x7f3cd00a6000 comp rx=0 tx=0).stop 2024-04-25T00:11:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.473+0000 7f3ce8800640 1 -- 172.21.15.83:0/2927956769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce006e950 msgr2=0x7f3ce014f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.473+0000 7f3ce8800640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce006e950 0x7f3ce014f860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400c420 tx=0x7f3cd400c8e0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.473+0000 7f3ce8800640 1 -- 172.21.15.83:0/2927956769 shutdown_connections 2024-04-25T00:11:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.473+0000 7f3ce8800640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cc40678c0 0x7f3cc4069d80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.473+0000 7f3ce8800640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ce0159810 0x7f3ce0150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.473+0000 7f3ce8800640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce015fe50 0x7f3ce0071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.473+0000 7f3ce8800640 1 --2- 172.21.15.83:0/2927956769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ce006e950 0x7f3ce014f860 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.473+0000 7f3ce8800640 1 -- 172.21.15.83:0/2927956769 >> 172.21.15.83:0/2927956769 conn(0x7f3ce00b1830 msgr2=0x7f3ce015f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.473+0000 7f3ce8800640 1 -- 172.21.15.83:0/2927956769 shutdown_connections 2024-04-25T00:11:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.473+0000 7f3ce8800640 1 -- 172.21.15.83:0/2927956769 wait complete. 2024-04-25T00:11:00.495 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:11:00.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1ee2c00640 1 Processor -- start 2024-04-25T00:11:00.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1ee2c00640 1 -- start start 2024-04-25T00:11:00.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1ee2c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1edc15bf70 0x7f1edc0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1ee2c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1edc0cfd80 0x7f1edc0d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1ee2c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1edc0cc9b0 con 0x7f1edc0d0770 2024-04-25T00:11:00.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1ee2c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1edc0d0bc0 con 0x7f1edc0cfd80 2024-04-25T00:11:00.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1ee2c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1edc0ccc90 con 0x7f1edc15bf70 2024-04-25T00:11:00.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1ed3400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1edc0cfd80 0x7f1edc0d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1ed3400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1edc0cfd80 0x7f1edc0d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55120/0 (socket says 172.21.15.83:55120) 2024-04-25T00:11:00.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1ed3400640 1 -- 172.21.15.83:0/1722055340 learned_addr learned my addr 172.21.15.83:0/1722055340 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:00.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1edbe00640 1 --2- 172.21.15.83:0/1722055340 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1edc15bf70 0x7f1edc0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1ed3400640 1 -- 172.21.15.83:0/1722055340 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1edc15bf70 msgr2=0x7f1edc0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:00.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1ed3400640 1 --2- 172.21.15.83:0/1722055340 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1edc15bf70 0x7f1edc0cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1ed3400640 1 -- 172.21.15.83:0/1722055340 >> v1:172.21.15.86:6789/0 conn(0x7f1edc0d0770 legacy=0x7f1edc0b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1ed3400640 1 -- 172.21.15.83:0/1722055340 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1edc0ccf70 con 0x7f1edc0cfd80 2024-04-25T00:11:00.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.629+0000 7f1edbe00640 1 --2- 172.21.15.83:0/1722055340 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1edc15bf70 0x7f1edc0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:00.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.633+0000 7f1ed3400640 1 --2- 172.21.15.83:0/1722055340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1edc0cfd80 0x7f1edc0d0230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009e50 tx=0x7f1ec401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09aca5da25eaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:00.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.633+0000 7f1edb400640 1 -- 172.21.15.83:0/1722055340 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec40a58d0 con 0x7f1edc0cfd80 2024-04-25T00:11:00.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.633+0000 7f1edb400640 1 -- 172.21.15.83:0/1722055340 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec40a5a70 con 0x7f1edc0cfd80 2024-04-25T00:11:00.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.633+0000 7f1edb400640 1 -- 172.21.15.83:0/1722055340 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec40a5e40 con 0x7f1edc0cfd80 2024-04-25T00:11:00.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.633+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1722055340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1edc0cfd80 msgr2=0x7f1edc0d0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.633+0000 7f1ee2c00640 1 --2- 172.21.15.83:0/1722055340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1edc0cfd80 0x7f1edc0d0230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009e50 tx=0x7f1ec401ca00 comp rx=0 tx=0).stop 2024-04-25T00:11:00.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.633+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1722055340 shutdown_connections 2024-04-25T00:11:00.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.633+0000 7f1ee2c00640 1 --2- 172.21.15.83:0/1722055340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1edc0cfd80 0x7f1edc0d0230 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.633+0000 7f1ee2c00640 1 --2- 172.21.15.83:0/1722055340 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1edc15bf70 0x7f1edc0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.633+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1722055340 >> 172.21.15.83:0/1722055340 conn(0x7f1edc0d1c70 msgr2=0x7f1edc0d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:00.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.633+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1722055340 shutdown_connections 2024-04-25T00:11:00.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.633+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1722055340 wait complete. 2024-04-25T00:11:00.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ee2c00640 1 Processor -- start 2024-04-25T00:11:00.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ee2c00640 1 -- start start 2024-04-25T00:11:00.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ee2c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1edc0cfd80 0x7f1edc153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ee2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1edc0d0770 0x7f1edc14ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ee2c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1edc15bf70 0x7f1edc14b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ee2c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc0cda50 con 0x7f1edc0cfd80 2024-04-25T00:11:00.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ee2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc15c350 con 0x7f1edc0d0770 2024-04-25T00:11:00.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ee2c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1edc158b50 con 0x7f1edc15bf70 2024-04-25T00:11:00.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1edbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1edc0cfd80 0x7f1edc153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1edbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1edc0cfd80 0x7f1edc153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55122/0 (socket says 172.21.15.83:55122) 2024-04-25T00:11:00.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1edbe00640 1 -- 172.21.15.83:0/1040983163 learned_addr learned my addr 172.21.15.83:0/1040983163 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:00.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1edbe00640 1 -- 172.21.15.83:0/1040983163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1edc15bf70 msgr2=0x7f1edc14b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1edbe00640 1 --2- 172.21.15.83:0/1040983163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1edc15bf70 0x7f1edc14b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ed3400640 1 --2- 172.21.15.83:0/1040983163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1edc0d0770 0x7f1edc14ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1edbe00640 1 -- 172.21.15.83:0/1040983163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1edc0d0770 msgr2=0x7f1edc14ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1edbe00640 1 --2- 172.21.15.83:0/1040983163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1edc0d0770 0x7f1edc14ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1edbe00640 1 -- 172.21.15.83:0/1040983163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1edc158e30 con 0x7f1edc0cfd80 2024-04-25T00:11:00.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ed3400640 1 --2- 172.21.15.83:0/1040983163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1edc0d0770 0x7f1edc14ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:00.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1edbe00640 1 --2- 172.21.15.83:0/1040983163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1edc0cfd80 0x7f1edc153d50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00c7c0 tx=0x7f1ecc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:00.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ed8c00640 1 -- 172.21.15.83:0/1040983163 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc007a30 con 0x7f1edc0cfd80 2024-04-25T00:11:00.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ed8c00640 1 -- 172.21.15.83:0/1040983163 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ecc007bd0 con 0x7f1edc0cfd80 2024-04-25T00:11:00.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ed8c00640 1 -- 172.21.15.83:0/1040983163 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc0113f0 con 0x7f1edc0cfd80 2024-04-25T00:11:00.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1040983163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1edc159110 con 0x7f1edc0cfd80 2024-04-25T00:11:00.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1040983163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1edc154290 con 0x7f1edc0cfd80 2024-04-25T00:11:00.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.637+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1040983163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1edc09fd60 con 0x7f1edc0cfd80 2024-04-25T00:11:00.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.641+0000 7f1ed8c00640 1 -- 172.21.15.83:0/1040983163 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1ecc011590 con 0x7f1edc0cfd80 2024-04-25T00:11:00.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.641+0000 7f1ed8c00640 1 --2- 172.21.15.83:0/1040983163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1eb80678c0 0x7f1eb8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.641+0000 7f1ed8c00640 1 -- 172.21.15.83:0/1040983163 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f1ecc085b60 con 0x7f1edc0cfd80 2024-04-25T00:11:00.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.641+0000 7f1ed3400640 1 --2- 172.21.15.83:0/1040983163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1eb80678c0 0x7f1eb8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.641+0000 7f1ed3400640 1 --2- 172.21.15.83:0/1040983163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1eb80678c0 0x7f1eb8069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40ae000 tx=0x7f1ec40a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:00.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.645+0000 7f1ed8c00640 1 -- 172.21.15.83:0/1040983163 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1ecc015020 con 0x7f1edc0cfd80 2024-04-25T00:11:00.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.833+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1040983163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1edc1592e0 con 0x7f1edc0cfd80 2024-04-25T00:11:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.833+0000 7f1ed8c00640 1 -- 172.21.15.83:0/1040983163 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7f1ecc0569e0 con 0x7f1edc0cfd80 2024-04-25T00:11:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:00.835 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/911805587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":911805587},{"type":"v1","addr":"172.21.15.83:6836","nonce":911805587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:11:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 2 2024-04-25T00:11:00.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.837+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1040983163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1eb80678c0 msgr2=0x7f1eb8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.837+0000 7f1ee2c00640 1 --2- 172.21.15.83:0/1040983163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1eb80678c0 0x7f1eb8069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40ae000 tx=0x7f1ec40a7070 comp rx=0 tx=0).stop 2024-04-25T00:11:00.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.837+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1040983163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1edc0cfd80 msgr2=0x7f1edc153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.837+0000 7f1ee2c00640 1 --2- 172.21.15.83:0/1040983163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1edc0cfd80 0x7f1edc153d50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00c7c0 tx=0x7f1ecc00cc80 comp rx=0 tx=0).stop 2024-04-25T00:11:00.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.837+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1040983163 shutdown_connections 2024-04-25T00:11:00.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.837+0000 7f1ee2c00640 1 --2- 172.21.15.83:0/1040983163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1eb80678c0 0x7f1eb8069d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.837+0000 7f1ee2c00640 1 --2- 172.21.15.83:0/1040983163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1edc15bf70 0x7f1edc14b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.837+0000 7f1ee2c00640 1 --2- 172.21.15.83:0/1040983163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1edc0d0770 0x7f1edc14ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.837+0000 7f1ee2c00640 1 --2- 172.21.15.83:0/1040983163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1edc0cfd80 0x7f1edc153d50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.837+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1040983163 >> 172.21.15.83:0/1040983163 conn(0x7f1edc0d1c70 msgr2=0x7f1edc14b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:00.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.837+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1040983163 shutdown_connections 2024-04-25T00:11:00.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:00.837+0000 7f1ee2c00640 1 -- 172.21.15.83:0/1040983163 wait complete. 2024-04-25T00:11:02.856 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:11:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d0b200640 1 Processor -- start 2024-04-25T00:11:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d0b200640 1 -- start start 2024-04-25T00:11:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d0b200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8d0c06b1e0 0x7f8d0c0b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d0b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d0c0ce090 0x7f8d0c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d0b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8d0c15f6e0 0x7f8d0c071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d0b200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8d0c0b0520 con 0x7f8d0c06b1e0 2024-04-25T00:11:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d0b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8d0c06bb00 con 0x7f8d0c15f6e0 2024-04-25T00:11:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d0b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0c15e6f0 con 0x7f8d0c0ce090 2024-04-25T00:11:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d09e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8d0c06b1e0 0x7f8d0c0b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d09e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8d0c06b1e0 0x7f8d0c0b59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55130/0 (socket says 172.21.15.83:55130) 2024-04-25T00:11:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d09e00640 1 -- 172.21.15.83:0/489745518 learned_addr learned my addr 172.21.15.83:0/489745518 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d09400640 1 --2- 172.21.15.83:0/489745518 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d0c0ce090 0x7f8d0c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d0a800640 1 --2- 172.21.15.83:0/489745518 >> v2:172.21.15.86:3300/0 conn(0x7f8d0c15f6e0 0x7f8d0c071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d09e00640 1 -- 172.21.15.83:0/489745518 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d0c0ce090 msgr2=0x7f8d0c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:03.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d09e00640 1 --2- 172.21.15.83:0/489745518 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d0c0ce090 0x7f8d0c06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d09e00640 1 -- 172.21.15.83:0/489745518 >> v2:172.21.15.86:3300/0 conn(0x7f8d0c15f6e0 msgr2=0x7f8d0c071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:03.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d09e00640 1 --2- 172.21.15.83:0/489745518 >> v2:172.21.15.86:3300/0 conn(0x7f8d0c15f6e0 0x7f8d0c071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d09e00640 1 -- 172.21.15.83:0/489745518 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c15e9d0 con 0x7f8d0c06b1e0 2024-04-25T00:11:03.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.001+0000 7f8d0a800640 1 --2- 172.21.15.83:0/489745518 >> v2:172.21.15.86:3300/0 conn(0x7f8d0c15f6e0 0x7f8d0c071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.005+0000 7f8d09e00640 1 --2- 172.21.15.83:0/489745518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d0c06b1e0 0x7f8d0c0b59c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c070820 tx=0x7f8cfc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce262f16a12583f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.005+0000 7f8d08a00640 1 -- 172.21.15.83:0/489745518 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0b0070 con 0x7f8d0c06b1e0 2024-04-25T00:11:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.005+0000 7f8d08a00640 1 -- 172.21.15.83:0/489745518 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cfc0ab600 con 0x7f8d0c06b1e0 2024-04-25T00:11:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.005+0000 7f8d08a00640 1 -- 172.21.15.83:0/489745518 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0ab9d0 con 0x7f8d0c06b1e0 2024-04-25T00:11:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.005+0000 7f8d0b200640 1 -- 172.21.15.83:0/489745518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d0c06b1e0 msgr2=0x7f8d0c0b59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.005+0000 7f8d0b200640 1 --2- 172.21.15.83:0/489745518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d0c06b1e0 0x7f8d0c0b59c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c070820 tx=0x7f8cfc01c470 comp rx=0 tx=0).stop 2024-04-25T00:11:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.005+0000 7f8d0b200640 1 -- 172.21.15.83:0/489745518 shutdown_connections 2024-04-25T00:11:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.005+0000 7f8d0b200640 1 --2- 172.21.15.83:0/489745518 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d0c0ce090 0x7f8d0c06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.005+0000 7f8d0b200640 1 --2- 172.21.15.83:0/489745518 >> v2:172.21.15.86:3300/0 conn(0x7f8d0c15f6e0 0x7f8d0c071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.005+0000 7f8d0b200640 1 --2- 172.21.15.83:0/489745518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d0c06b1e0 0x7f8d0c0b59c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.005+0000 7f8d0b200640 1 -- 172.21.15.83:0/489745518 >> 172.21.15.83:0/489745518 conn(0x7f8d0c0cd500 msgr2=0x7f8d0c0cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.005+0000 7f8d0b200640 1 -- 172.21.15.83:0/489745518 shutdown_connections 2024-04-25T00:11:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.005+0000 7f8d0b200640 1 -- 172.21.15.83:0/489745518 wait complete. 2024-04-25T00:11:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d0b200640 1 Processor -- start 2024-04-25T00:11:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d0b200640 1 -- start start 2024-04-25T00:11:03.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d0b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0c06b1e0 0x7f8d0c1508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d0b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d0c0ce090 0x7f8d0c06f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d0b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d0c15f6e0 0x7f8d0c0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d0b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c15fc40 con 0x7f8d0c0ce090 2024-04-25T00:11:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d0b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c15fac0 con 0x7f8d0c06b1e0 2024-04-25T00:11:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d0b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0c161900 con 0x7f8d0c15f6e0 2024-04-25T00:11:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d09400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d0c0ce090 0x7f8d0c06f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d09400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d0c0ce090 0x7f8d0c06f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55146/0 (socket says 172.21.15.83:55146) 2024-04-25T00:11:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d09400640 1 -- 172.21.15.83:0/2933269881 learned_addr learned my addr 172.21.15.83:0/2933269881 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d09e00640 1 --2- 172.21.15.83:0/2933269881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0c06b1e0 0x7f8d0c1508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d0a800640 1 --2- 172.21.15.83:0/2933269881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d0c15f6e0 0x7f8d0c0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d09400640 1 -- 172.21.15.83:0/2933269881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d0c15f6e0 msgr2=0x7f8d0c0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d09400640 1 --2- 172.21.15.83:0/2933269881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d0c15f6e0 0x7f8d0c0b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d09400640 1 -- 172.21.15.83:0/2933269881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0c06b1e0 msgr2=0x7f8d0c1508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d09400640 1 --2- 172.21.15.83:0/2933269881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0c06b1e0 0x7f8d0c1508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d09400640 1 -- 172.21.15.83:0/2933269881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c1fec80 con 0x7f8d0c0ce090 2024-04-25T00:11:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d09400640 1 --2- 172.21.15.83:0/2933269881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d0c0ce090 0x7f8d0c06f490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800d830 tx=0x7f8cf800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d02000640 1 -- 172.21.15.83:0/2933269881 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf80096a0 con 0x7f8d0c0ce090 2024-04-25T00:11:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d02000640 1 -- 172.21.15.83:0/2933269881 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf8019070 con 0x7f8d0c0ce090 2024-04-25T00:11:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d0b200640 1 -- 172.21.15.83:0/2933269881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0c1feef0 con 0x7f8d0c0ce090 2024-04-25T00:11:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d02000640 1 -- 172.21.15.83:0/2933269881 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8009a80 con 0x7f8d0c0ce090 2024-04-25T00:11:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d0b200640 1 -- 172.21.15.83:0/2933269881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0c06fe00 con 0x7f8d0c0ce090 2024-04-25T00:11:03.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.009+0000 7f8d0b200640 1 -- 172.21.15.83:0/2933269881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0c04f510 con 0x7f8d0c0ce090 2024-04-25T00:11:03.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.013+0000 7f8d02000640 1 -- 172.21.15.83:0/2933269881 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8cf8009c20 con 0x7f8d0c0ce090 2024-04-25T00:11:03.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.013+0000 7f8d02000640 1 --2- 172.21.15.83:0/2933269881 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8cec0678f0 0x7f8cec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.013+0000 7f8d02000640 1 -- 172.21.15.83:0/2933269881 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f8cf8085c70 con 0x7f8d0c0ce090 2024-04-25T00:11:03.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.013+0000 7f8d09e00640 1 --2- 172.21.15.83:0/2933269881 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8cec0678f0 0x7f8cec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.013+0000 7f8d09e00640 1 --2- 172.21.15.83:0/2933269881 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8cec0678f0 0x7f8cec069db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0ad000 tx=0x7f8cfc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.017+0000 7f8d02000640 1 -- 172.21.15.83:0/2933269881 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8cf80564a0 con 0x7f8d0c0ce090 2024-04-25T00:11:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.205+0000 7f8d0b200640 1 -- 172.21.15.83:0/2933269881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d0c072d50 con 0x7f8d0c0ce090 2024-04-25T00:11:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.205+0000 7f8d02000640 1 -- 172.21.15.83:0/2933269881 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3861 (secure 0 0 0) 0x7f8cf805a7a0 con 0x7f8d0c0ce090 2024-04-25T00:11:03.206 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:03.206 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/1335208265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1335208265},{"type":"v1","addr":"172.21.15.86:6834","nonce":1335208265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/911805587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":911805587},{"type":"v1","addr":"172.21.15.83:6836","nonce":911805587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2309157687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2309157687},{"type":"v1","addr":"172.21.15.83:6837","nonce":2309157687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1603954817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":1603954817},{"type":"v1","addr":"172.21.15.86:6835","nonce":1603954817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T00:11:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 3 2024-04-25T00:11:03.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.205+0000 7f8d0b200640 1 -- 172.21.15.83:0/2933269881 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8cec0678f0 msgr2=0x7f8cec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.205+0000 7f8d0b200640 1 --2- 172.21.15.83:0/2933269881 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8cec0678f0 0x7f8cec069db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0ad000 tx=0x7f8cfc0a6000 comp rx=0 tx=0).stop 2024-04-25T00:11:03.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.205+0000 7f8d0b200640 1 -- 172.21.15.83:0/2933269881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d0c0ce090 msgr2=0x7f8d0c06f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.205+0000 7f8d0b200640 1 --2- 172.21.15.83:0/2933269881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d0c0ce090 0x7f8d0c06f490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800d830 tx=0x7f8cf800dcf0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.205+0000 7f8d0b200640 1 -- 172.21.15.83:0/2933269881 shutdown_connections 2024-04-25T00:11:03.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.205+0000 7f8d0b200640 1 --2- 172.21.15.83:0/2933269881 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8cec0678f0 0x7f8cec069db0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.205+0000 7f8d0b200640 1 --2- 172.21.15.83:0/2933269881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d0c15f6e0 0x7f8d0c0b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.205+0000 7f8d0b200640 1 --2- 172.21.15.83:0/2933269881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d0c0ce090 0x7f8d0c06f490 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.205+0000 7f8d0b200640 1 --2- 172.21.15.83:0/2933269881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d0c06b1e0 0x7f8d0c1508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.205+0000 7f8d0b200640 1 -- 172.21.15.83:0/2933269881 >> 172.21.15.83:0/2933269881 conn(0x7f8d0c0cd500 msgr2=0x7f8d0c161050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:03.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.205+0000 7f8d0b200640 1 -- 172.21.15.83:0/2933269881 shutdown_connections 2024-04-25T00:11:03.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.205+0000 7f8d0b200640 1 -- 172.21.15.83:0/2933269881 wait complete. 2024-04-25T00:11:03.226 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:03.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad58200640 1 Processor -- start 2024-04-25T00:11:03.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad58200640 1 -- start start 2024-04-25T00:11:03.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad58200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad501599f0 0x7fad500cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad58200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad5015cec0 0x7fad500ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad58200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad500ccc60 con 0x7fad5015c340 2024-04-25T00:11:03.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad58200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fad50159dd0 con 0x7fad5015cec0 2024-04-25T00:11:03.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad58200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fad5006dba0 con 0x7fad501599f0 2024-04-25T00:11:03.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad56200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad5015cec0 0x7fad500ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad56200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad5015cec0 0x7fad500ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55148/0 (socket says 172.21.15.83:55148) 2024-04-25T00:11:03.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad56200640 1 -- 172.21.15.83:0/1420230478 learned_addr learned my addr 172.21.15.83:0/1420230478 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:03.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad55800640 1 --2- 172.21.15.83:0/1420230478 >> v2:172.21.15.86:3300/0 conn(0x7fad501599f0 0x7fad500cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad56200640 1 -- 172.21.15.83:0/1420230478 >> v2:172.21.15.86:3300/0 conn(0x7fad501599f0 msgr2=0x7fad500cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:03.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad56200640 1 --2- 172.21.15.83:0/1420230478 >> v2:172.21.15.86:3300/0 conn(0x7fad501599f0 0x7fad500cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad56200640 1 -- 172.21.15.83:0/1420230478 >> v1:172.21.15.86:6789/0 conn(0x7fad5015c340 legacy=0x7fad5015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad56200640 1 -- 172.21.15.83:0/1420230478 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5006de80 con 0x7fad5015cec0 2024-04-25T00:11:03.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.361+0000 7fad55800640 1 --2- 172.21.15.83:0/1420230478 >> v2:172.21.15.86:3300/0 conn(0x7fad501599f0 0x7fad500cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:03.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.365+0000 7fad56200640 1 --2- 172.21.15.83:0/1420230478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad5015cec0 0x7fad500ce6e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fad48009e50 tx=0x7fad4801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7573c6f53248e430 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.365+0000 7fad4fe00640 1 -- 172.21.15.83:0/1420230478 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad480a5620 con 0x7fad5015cec0 2024-04-25T00:11:03.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.365+0000 7fad4fe00640 1 -- 172.21.15.83:0/1420230478 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad480a57c0 con 0x7fad5015cec0 2024-04-25T00:11:03.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.365+0000 7fad4fe00640 1 -- 172.21.15.83:0/1420230478 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad480a5b90 con 0x7fad5015cec0 2024-04-25T00:11:03.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.365+0000 7fad58200640 1 -- 172.21.15.83:0/1420230478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad5015cec0 msgr2=0x7fad500ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.365+0000 7fad58200640 1 --2- 172.21.15.83:0/1420230478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad5015cec0 0x7fad500ce6e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fad48009e50 tx=0x7fad4801c960 comp rx=0 tx=0).stop 2024-04-25T00:11:03.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.365+0000 7fad58200640 1 -- 172.21.15.83:0/1420230478 shutdown_connections 2024-04-25T00:11:03.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.365+0000 7fad58200640 1 --2- 172.21.15.83:0/1420230478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad5015cec0 0x7fad500ce6e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.365+0000 7fad58200640 1 --2- 172.21.15.83:0/1420230478 >> v2:172.21.15.86:3300/0 conn(0x7fad501599f0 0x7fad500cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.365+0000 7fad58200640 1 -- 172.21.15.83:0/1420230478 >> 172.21.15.83:0/1420230478 conn(0x7fad500b1830 msgr2=0x7fad500b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:03.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.365+0000 7fad58200640 1 -- 172.21.15.83:0/1420230478 shutdown_connections 2024-04-25T00:11:03.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.365+0000 7fad58200640 1 -- 172.21.15.83:0/1420230478 wait complete. 2024-04-25T00:11:03.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.365+0000 7fad58200640 1 Processor -- start 2024-04-25T00:11:03.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.365+0000 7fad58200640 1 -- start start 2024-04-25T00:11:03.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad58200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad501599f0 0x7fad5014d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad58200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad5015c340 0x7fad500b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad58200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad5015cec0 0x7fad5014e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad58200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad500cf800 con 0x7fad501599f0 2024-04-25T00:11:03.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad58200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad500cf680 con 0x7fad5015cec0 2024-04-25T00:11:03.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad58200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fad50168cd0 con 0x7fad5015c340 2024-04-25T00:11:03.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad55800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad501599f0 0x7fad5014d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad55800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad501599f0 0x7fad5014d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55158/0 (socket says 172.21.15.83:55158) 2024-04-25T00:11:03.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad55800640 1 -- 172.21.15.83:0/1259104498 learned_addr learned my addr 172.21.15.83:0/1259104498 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:03.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad56200640 1 --2- 172.21.15.83:0/1259104498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad5015cec0 0x7fad5014e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad54e00640 1 --2- 172.21.15.83:0/1259104498 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad5015c340 0x7fad500b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad55800640 1 -- 172.21.15.83:0/1259104498 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad5015c340 msgr2=0x7fad500b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad55800640 1 --2- 172.21.15.83:0/1259104498 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad5015c340 0x7fad500b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad55800640 1 -- 172.21.15.83:0/1259104498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad5015cec0 msgr2=0x7fad5014e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad55800640 1 --2- 172.21.15.83:0/1259104498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad5015cec0 0x7fad5014e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad55800640 1 -- 172.21.15.83:0/1259104498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5015b0e0 con 0x7fad501599f0 2024-04-25T00:11:03.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad55800640 1 --2- 172.21.15.83:0/1259104498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad501599f0 0x7fad5014d890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad4400d800 tx=0x7fad4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad4d600640 1 -- 172.21.15.83:0/1259104498 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44018020 con 0x7fad501599f0 2024-04-25T00:11:03.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad4d600640 1 -- 172.21.15.83:0/1259104498 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad440096c0 con 0x7fad501599f0 2024-04-25T00:11:03.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad4d600640 1 -- 172.21.15.83:0/1259104498 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44009ab0 con 0x7fad501599f0 2024-04-25T00:11:03.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad58200640 1 -- 172.21.15.83:0/1259104498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad500b38f0 con 0x7fad501599f0 2024-04-25T00:11:03.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad58200640 1 -- 172.21.15.83:0/1259104498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5015b680 con 0x7fad501599f0 2024-04-25T00:11:03.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad58200640 1 -- 172.21.15.83:0/1259104498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad500737f0 con 0x7fad501599f0 2024-04-25T00:11:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.369+0000 7fad4d600640 1 -- 172.21.15.83:0/1259104498 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fad4401b020 con 0x7fad501599f0 2024-04-25T00:11:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.373+0000 7fad4d600640 1 --2- 172.21.15.83:0/1259104498 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fad300678f0 0x7fad30069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.373+0000 7fad4d600640 1 -- 172.21.15.83:0/1259104498 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fad44085f40 con 0x7fad501599f0 2024-04-25T00:11:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.373+0000 7fad54e00640 1 --2- 172.21.15.83:0/1259104498 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fad300678f0 0x7fad30069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.373+0000 7fad54e00640 1 --2- 172.21.15.83:0/1259104498 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fad300678f0 0x7fad30069db0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fad500b2bb0 tx=0x7fad3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.377+0000 7fad4d600640 1 -- 172.21.15.83:0/1259104498 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fad44022020 con 0x7fad501599f0 2024-04-25T00:11:03.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.533+0000 7fad58200640 1 -- 172.21.15.83:0/1259104498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad5004f5f0 con 0x7fad501599f0 2024-04-25T00:11:03.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.533+0000 7fad4d600640 1 -- 172.21.15.83:0/1259104498 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12800 (secure 0 0 0) 0x7fad44056ca0 con 0x7fad501599f0 2024-04-25T00:11:03.535 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:03.535 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:10:56.767409+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.537+0000 7fad58200640 1 -- 172.21.15.83:0/1259104498 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fad300678f0 msgr2=0x7fad30069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.537+0000 7fad58200640 1 --2- 172.21.15.83:0/1259104498 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fad300678f0 0x7fad30069db0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fad500b2bb0 tx=0x7fad3c009040 comp rx=0 tx=0).stop 2024-04-25T00:11:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.537+0000 7fad58200640 1 -- 172.21.15.83:0/1259104498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad501599f0 msgr2=0x7fad5014d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.537+0000 7fad58200640 1 --2- 172.21.15.83:0/1259104498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad501599f0 0x7fad5014d890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad4400d800 tx=0x7fad4400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.537+0000 7fad58200640 1 -- 172.21.15.83:0/1259104498 shutdown_connections 2024-04-25T00:11:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.537+0000 7fad58200640 1 --2- 172.21.15.83:0/1259104498 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fad300678f0 0x7fad30069db0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.537+0000 7fad58200640 1 --2- 172.21.15.83:0/1259104498 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad5015c340 0x7fad500b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.537+0000 7fad58200640 1 --2- 172.21.15.83:0/1259104498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad5015cec0 0x7fad5014e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.537+0000 7fad58200640 1 --2- 172.21.15.83:0/1259104498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad501599f0 0x7fad5014d890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.537+0000 7fad58200640 1 -- 172.21.15.83:0/1259104498 >> 172.21.15.83:0/1259104498 conn(0x7fad500b1830 msgr2=0x7fad501550d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.537+0000 7fad58200640 1 -- 172.21.15.83:0/1259104498 shutdown_connections 2024-04-25T00:11:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.537+0000 7fad58200640 1 -- 172.21.15.83:0/1259104498 wait complete. 2024-04-25T00:11:03.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:11:03.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:11:03.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff433a00640 1 Processor -- start 2024-04-25T00:11:03.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff433a00640 1 -- start start 2024-04-25T00:11:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff433a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff42c15e0e0 0x7ff42c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff433a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff42c15e4c0 0x7ff42c0ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff433a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff42c0b4b10 con 0x7ff42c0cea80 2024-04-25T00:11:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff433a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff42c0d0f50 con 0x7ff42c15e0e0 2024-04-25T00:11:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff433a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff42c0cc310 con 0x7ff42c15e4c0 2024-04-25T00:11:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff431000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff42c15e0e0 0x7ff42c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff431a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff42c0cea80 0x7ff42c06c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50494/0 (socket says 172.21.15.83:50494) 2024-04-25T00:11:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff431a00640 1 -- 172.21.15.83:0/205438425 learned_addr learned my addr 172.21.15.83:0/205438425 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff42be00640 1 --2- 172.21.15.83:0/205438425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff42c15e4c0 0x7ff42c0ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff431000640 1 -- 172.21.15.83:0/205438425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff42c15e4c0 msgr2=0x7ff42c0ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff431000640 1 --2- 172.21.15.83:0/205438425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff42c15e4c0 0x7ff42c0ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff431000640 1 -- 172.21.15.83:0/205438425 >> v1:172.21.15.83:6789/0 conn(0x7ff42c0cea80 legacy=0x7ff42c06c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff431000640 1 -- 172.21.15.83:0/205438425 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c0d1180 con 0x7ff42c15e0e0 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff42be00640 1 --2- 172.21.15.83:0/205438425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff42c15e4c0 0x7ff42c0ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.693+0000 7ff42b400640 1 -- 172.21.15.83:0/205438425 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800986895 0 0) 0x7ff424001120 con 0x7ff42c0cea80 2024-04-25T00:11:03.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.697+0000 7ff431000640 1 --2- 172.21.15.83:0/205438425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff42c15e0e0 0x7ff42c0d0a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009d90 tx=0x7ff41c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd7023a464c0bad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.697+0000 7ff42b400640 1 -- 172.21.15.83:0/205438425 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0b2070 con 0x7ff42c15e0e0 2024-04-25T00:11:03.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.697+0000 7ff42b400640 1 -- 172.21.15.83:0/205438425 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff41c0a5920 con 0x7ff42c15e0e0 2024-04-25T00:11:03.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.697+0000 7ff42b400640 1 -- 172.21.15.83:0/205438425 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0a5cf0 con 0x7ff42c15e0e0 2024-04-25T00:11:03.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.697+0000 7ff433a00640 1 -- 172.21.15.83:0/205438425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff42c15e0e0 msgr2=0x7ff42c0d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.697+0000 7ff433a00640 1 --2- 172.21.15.83:0/205438425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff42c15e0e0 0x7ff42c0d0a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009d90 tx=0x7ff41c01c970 comp rx=0 tx=0).stop 2024-04-25T00:11:03.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.697+0000 7ff433a00640 1 -- 172.21.15.83:0/205438425 shutdown_connections 2024-04-25T00:11:03.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.697+0000 7ff433a00640 1 --2- 172.21.15.83:0/205438425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff42c15e4c0 0x7ff42c0ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.697+0000 7ff433a00640 1 --2- 172.21.15.83:0/205438425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff42c15e0e0 0x7ff42c0d0a10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.697+0000 7ff433a00640 1 -- 172.21.15.83:0/205438425 >> 172.21.15.83:0/205438425 conn(0x7ff42c0d1cc0 msgr2=0x7ff42c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:03.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.697+0000 7ff433a00640 1 -- 172.21.15.83:0/205438425 shutdown_connections 2024-04-25T00:11:03.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.697+0000 7ff433a00640 1 -- 172.21.15.83:0/205438425 wait complete. 2024-04-25T00:11:03.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff433a00640 1 Processor -- start 2024-04-25T00:11:03.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff433a00640 1 -- start start 2024-04-25T00:11:03.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff433a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c0cea80 0x7ff42c151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff433a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff42c15e0e0 0x7ff42c1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff433a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff42c15e4c0 0x7ff42c15e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff433a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c16aed0 con 0x7ff42c15e4c0 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff433a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c0ce050 con 0x7ff42c0cea80 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff433a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff42c16b050 con 0x7ff42c15e0e0 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff431a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff42c15e4c0 0x7ff42c15e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff431a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff42c15e4c0 0x7ff42c15e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55176/0 (socket says 172.21.15.83:55176) 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff431a00640 1 -- 172.21.15.83:0/3773798671 learned_addr learned my addr 172.21.15.83:0/3773798671 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff42be00640 1 --2- 172.21.15.83:0/3773798671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff42c15e0e0 0x7ff42c1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff431000640 1 --2- 172.21.15.83:0/3773798671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c0cea80 0x7ff42c151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff431a00640 1 -- 172.21.15.83:0/3773798671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff42c15e0e0 msgr2=0x7ff42c1579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff431a00640 1 --2- 172.21.15.83:0/3773798671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff42c15e0e0 0x7ff42c1579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff431a00640 1 -- 172.21.15.83:0/3773798671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c0cea80 msgr2=0x7ff42c151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff431a00640 1 --2- 172.21.15.83:0/3773798671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c0cea80 0x7ff42c151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff431a00640 1 -- 172.21.15.83:0/3773798671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c0c8ba0 con 0x7ff42c15e4c0 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff431a00640 1 --2- 172.21.15.83:0/3773798671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff42c15e4c0 0x7ff42c15e950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff42400c3e0 tx=0x7ff42400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff428c00640 1 -- 172.21.15.83:0/3773798671 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4240074e0 con 0x7ff42c15e4c0 2024-04-25T00:11:03.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff428c00640 1 -- 172.21.15.83:0/3773798671 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4240047f0 con 0x7ff42c15e4c0 2024-04-25T00:11:03.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff433a00640 1 -- 172.21.15.83:0/3773798671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c15f990 con 0x7ff42c15e4c0 2024-04-25T00:11:03.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff428c00640 1 -- 172.21.15.83:0/3773798671 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424004be0 con 0x7ff42c15e4c0 2024-04-25T00:11:03.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff433a00640 1 -- 172.21.15.83:0/3773798671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c15f6b0 con 0x7ff42c15e4c0 2024-04-25T00:11:03.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.701+0000 7ff433a00640 1 -- 172.21.15.83:0/3773798671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42c04f510 con 0x7ff42c15e4c0 2024-04-25T00:11:03.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.705+0000 7ff428c00640 1 -- 172.21.15.83:0/3773798671 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff424011070 con 0x7ff42c15e4c0 2024-04-25T00:11:03.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.705+0000 7ff428c00640 1 --2- 172.21.15.83:0/3773798671 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff40c0678f0 0x7ff40c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.705+0000 7ff428c00640 1 -- 172.21.15.83:0/3773798671 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff424085340 con 0x7ff42c15e4c0 2024-04-25T00:11:03.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.705+0000 7ff431000640 1 --2- 172.21.15.83:0/3773798671 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff40c0678f0 0x7ff40c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.705+0000 7ff431000640 1 --2- 172.21.15.83:0/3773798671 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff40c0678f0 0x7ff40c069db0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0af000 tx=0x7ff41c0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.709+0000 7ff428c00640 1 -- 172.21.15.83:0/3773798671 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff42c0b4b10 con 0x7ff42c15e4c0 2024-04-25T00:11:03.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.869+0000 7ff433a00640 1 -- 172.21.15.83:0/3773798671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff42c161980 con 0x7ff42c15e4c0 2024-04-25T00:11:03.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.869+0000 7ff428c00640 1 -- 172.21.15.83:0/3773798671 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7ff424056260 con 0x7ff42c15e4c0 2024-04-25T00:11:03.869 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:11:03.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.869+0000 7ff433a00640 1 -- 172.21.15.83:0/3773798671 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff40c0678f0 msgr2=0x7ff40c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.869+0000 7ff433a00640 1 --2- 172.21.15.83:0/3773798671 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff40c0678f0 0x7ff40c069db0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0af000 tx=0x7ff41c0a7070 comp rx=0 tx=0).stop 2024-04-25T00:11:03.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.869+0000 7ff433a00640 1 -- 172.21.15.83:0/3773798671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff42c15e4c0 msgr2=0x7ff42c15e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.869+0000 7ff433a00640 1 --2- 172.21.15.83:0/3773798671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff42c15e4c0 0x7ff42c15e950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff42400c3e0 tx=0x7ff42400c8a0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.869+0000 7ff433a00640 1 -- 172.21.15.83:0/3773798671 shutdown_connections 2024-04-25T00:11:03.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.869+0000 7ff433a00640 1 --2- 172.21.15.83:0/3773798671 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff40c0678f0 0x7ff40c069db0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.869+0000 7ff433a00640 1 --2- 172.21.15.83:0/3773798671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff42c15e0e0 0x7ff42c1579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.869+0000 7ff433a00640 1 --2- 172.21.15.83:0/3773798671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff42c15e4c0 0x7ff42c15e950 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.869+0000 7ff433a00640 1 --2- 172.21.15.83:0/3773798671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c0cea80 0x7ff42c151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.869+0000 7ff433a00640 1 -- 172.21.15.83:0/3773798671 >> 172.21.15.83:0/3773798671 conn(0x7ff42c0d1cc0 msgr2=0x7ff42c0cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:03.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.869+0000 7ff433a00640 1 -- 172.21.15.83:0/3773798671 shutdown_connections 2024-04-25T00:11:03.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:03.869+0000 7ff433a00640 1 -- 172.21.15.83:0/3773798671 wait complete. 2024-04-25T00:11:03.890 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:11:03.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:11:03.939 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:11:04.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.025+0000 7f4272e00640 1 Processor -- start 2024-04-25T00:11:04.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.025+0000 7f4272e00640 1 -- start start 2024-04-25T00:11:04.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.025+0000 7f4272e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426c0b4360 0x7f426c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.025+0000 7f4272e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f426c0b2b30 0x7f426c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.025+0000 7f4272e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f426c06ef80 con 0x7f426c1553e0 2024-04-25T00:11:04.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.025+0000 7f4272e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f426c15b510 con 0x7f426c0b2b30 2024-04-25T00:11:04.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.025+0000 7f4272e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f426c15b7f0 con 0x7f426c0b4360 2024-04-25T00:11:04.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.025+0000 7f426be00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f426c1553e0 0x7f426c06e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50506/0 (socket says 172.21.15.83:50506) 2024-04-25T00:11:04.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.025+0000 7f426be00640 1 -- 172.21.15.83:0/2198889555 learned_addr learned my addr 172.21.15.83:0/2198889555 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:04.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.025+0000 7f4270e00640 1 --2- 172.21.15.83:0/2198889555 >> v2:172.21.15.86:3300/0 conn(0x7f426c0b2b30 0x7f426c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.025+0000 7f4263400640 1 --2- 172.21.15.83:0/2198889555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426c0b4360 0x7f426c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 -- 172.21.15.83:0/2198889555 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4009838990 0 0) 0x7f426c06ef80 con 0x7f426c1553e0 2024-04-25T00:11:04.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 -- 172.21.15.83:0/2198889555 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4250005740 con 0x7f426c1553e0 2024-04-25T00:11:04.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 -- 172.21.15.83:0/2198889555 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3907275543 0 0) 0x7f4250005740 con 0x7f426c1553e0 2024-04-25T00:11:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 -- 172.21.15.83:0/2198889555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426c0b4360 msgr2=0x7f426c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 --2- 172.21.15.83:0/2198889555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426c0b4360 0x7f426c0b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 -- 172.21.15.83:0/2198889555 >> v2:172.21.15.86:3300/0 conn(0x7f426c0b2b30 msgr2=0x7f426c0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 --2- 172.21.15.83:0/2198889555 >> v2:172.21.15.86:3300/0 conn(0x7f426c0b2b30 0x7f426c0b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 -- 172.21.15.83:0/2198889555 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426c15bad0 con 0x7f426c1553e0 2024-04-25T00:11:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 -- 172.21.15.83:0/2198889555 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f425c001a00 con 0x7f426c1553e0 2024-04-25T00:11:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 -- 172.21.15.83:0/2198889555 >> v1:172.21.15.83:6789/0 conn(0x7f426c1553e0 legacy=0x7f426c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 --2- 172.21.15.83:0/2198889555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4250005bd0 0x7f4250005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 --2- 172.21.15.83:0/2198889555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426c0b4360 0x7f4250006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 --2- 172.21.15.83:0/2198889555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4250006f60 0x7f42500073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 -- 172.21.15.83:0/2198889555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f426c15b510 con 0x7f4250005bd0 2024-04-25T00:11:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 -- 172.21.15.83:0/2198889555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f426c15b7f0 con 0x7f4250006f60 2024-04-25T00:11:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426b400640 1 -- 172.21.15.83:0/2198889555 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f425000dbe0 con 0x7f426c0b4360 2024-04-25T00:11:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f4270e00640 1 --2- 172.21.15.83:0/2198889555 >> v2:172.21.15.86:3300/0 conn(0x7f426c0b2b30 0x7f426c0b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426be00640 1 --2- 172.21.15.83:0/2198889555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4250005bd0 0x7f4250005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426be00640 1 -- 172.21.15.83:0/2198889555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426c0b4360 msgr2=0x7f4250006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426be00640 1 --2- 172.21.15.83:0/2198889555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426c0b4360 0x7f4250006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426be00640 1 -- 172.21.15.83:0/2198889555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4250006f60 msgr2=0x7f42500073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426be00640 1 --2- 172.21.15.83:0/2198889555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4250006f60 0x7f42500073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426be00640 1 -- 172.21.15.83:0/2198889555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4250005740 con 0x7f4250005bd0 2024-04-25T00:11:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.029+0000 7f426be00640 1 --2- 172.21.15.83:0/2198889555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4250005bd0 0x7f4250005ff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f425c00a920 tx=0x7f425c01d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578887f0ab08ff0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f426b400640 1 -- 172.21.15.83:0/2198889555 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c00a4c0 con 0x7f4250005bd0 2024-04-25T00:11:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f426b400640 1 -- 172.21.15.83:0/2198889555 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f425c0ad650 con 0x7f4250005bd0 2024-04-25T00:11:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f426b400640 1 -- 172.21.15.83:0/2198889555 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c0ada40 con 0x7f4250005bd0 2024-04-25T00:11:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4272e00640 1 -- 172.21.15.83:0/2198889555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4250005bd0 msgr2=0x7f4250005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4272e00640 1 --2- 172.21.15.83:0/2198889555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4250005bd0 0x7f4250005ff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f425c00a920 tx=0x7f425c01d440 comp rx=0 tx=0).stop 2024-04-25T00:11:04.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4270e00640 1 -- 172.21.15.83:0/2198889555 reap_dead start 2024-04-25T00:11:04.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4272e00640 1 -- 172.21.15.83:0/2198889555 shutdown_connections 2024-04-25T00:11:04.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4272e00640 1 -- 172.21.15.83:0/2198889555 >> 172.21.15.83:0/2198889555 conn(0x7f426c0b1830 msgr2=0x7f426c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:04.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4272e00640 1 -- 172.21.15.83:0/2198889555 shutdown_connections 2024-04-25T00:11:04.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4272e00640 1 -- 172.21.15.83:0/2198889555 wait complete. 2024-04-25T00:11:04.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4272e00640 1 Processor -- start 2024-04-25T00:11:04.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4272e00640 1 -- start start 2024-04-25T00:11:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4272e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4250005bd0 0x7f426c1608d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4272e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c160e10 0x7f426c0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4272e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c0ce2a0 0x7f426c0ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4272e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f426c06ebd0 con 0x7f426c0ce2a0 2024-04-25T00:11:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4272e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f426c0b4b50 con 0x7f426c160e10 2024-04-25T00:11:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4272e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f426c143d40 con 0x7f4250005bd0 2024-04-25T00:11:04.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4270e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c0ce2a0 0x7f426c0ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4270e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c0ce2a0 0x7f426c0ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55188/0 (socket says 172.21.15.83:55188) 2024-04-25T00:11:04.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4270e00640 1 -- 172.21.15.83:0/2455510776 learned_addr learned my addr 172.21.15.83:0/2455510776 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:04.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f4263400640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c160e10 0x7f426c0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.033+0000 7f426be00640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4250005bd0 0x7f426c1608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4270e00640 1 -- 172.21.15.83:0/2455510776 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4250005bd0 msgr2=0x7f426c1608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4270e00640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4250005bd0 0x7f426c1608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4270e00640 1 -- 172.21.15.83:0/2455510776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c160e10 msgr2=0x7f426c0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4270e00640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c160e10 0x7f426c0cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4270e00640 1 -- 172.21.15.83:0/2455510776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426c161600 con 0x7f426c0ce2a0 2024-04-25T00:11:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f426be00640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4250005bd0 0x7f426c1608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4270e00640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c0ce2a0 0x7f426c0ce720 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f426c068b90 tx=0x7f426400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4268c00640 1 -- 172.21.15.83:0/2455510776 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426401a070 con 0x7f426c0ce2a0 2024-04-25T00:11:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4268c00640 1 -- 172.21.15.83:0/2455510776 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4264004030 con 0x7f426c0ce2a0 2024-04-25T00:11:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4268c00640 1 -- 172.21.15.83:0/2455510776 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264004420 con 0x7f426c0ce2a0 2024-04-25T00:11:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4272e00640 1 -- 172.21.15.83:0/2455510776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426c14c620 con 0x7f426c0ce2a0 2024-04-25T00:11:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4272e00640 1 -- 172.21.15.83:0/2455510776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f426c14cbe0 con 0x7f426c0ce2a0 2024-04-25T00:11:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4272e00640 1 -- 172.21.15.83:0/2455510776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f426c14cde0 con 0x7f426c0ce2a0 2024-04-25T00:11:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4268c00640 1 -- 172.21.15.83:0/2455510776 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f426401d070 con 0x7f426c0ce2a0 2024-04-25T00:11:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4268c00640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4248067940 0x7f4248069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f4268c00640 1 -- 172.21.15.83:0/2455510776 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f4264084de0 con 0x7f426c0ce2a0 2024-04-25T00:11:04.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.037+0000 7f426be00640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4248067940 0x7f4248069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.041+0000 7f426be00640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4248067940 0x7f4248069e00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f425c0ad000 tx=0x7f425c001dd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:04.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.041+0000 7f4268c00640 1 -- 172.21.15.83:0/2455510776 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f426c14cde0 con 0x7f426c0ce2a0 2024-04-25T00:11:04.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.197+0000 7f4272e00640 1 -- 172.21.15.83:0/2455510776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f426c0737f0 con 0x7f426c0ce2a0 2024-04-25T00:11:04.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.197+0000 7f4268c00640 1 -- 172.21.15.83:0/2455510776 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f4264055c50 con 0x7f426c0ce2a0 2024-04-25T00:11:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:04.200 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-25T00:11:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:11:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-25T00:11:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-25T00:11:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:11:04.201 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-25T00:11:04.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.201+0000 7f4272e00640 1 -- 172.21.15.83:0/2455510776 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4248067940 msgr2=0x7f4248069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.201+0000 7f4272e00640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4248067940 0x7f4248069e00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f425c0ad000 tx=0x7f425c001dd0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.201+0000 7f4272e00640 1 -- 172.21.15.83:0/2455510776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c0ce2a0 msgr2=0x7f426c0ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.201+0000 7f4272e00640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c0ce2a0 0x7f426c0ce720 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f426c068b90 tx=0x7f426400ec70 comp rx=0 tx=0).stop 2024-04-25T00:11:04.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.201+0000 7f4272e00640 1 -- 172.21.15.83:0/2455510776 shutdown_connections 2024-04-25T00:11:04.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.201+0000 7f4272e00640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4248067940 0x7f4248069e00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.201+0000 7f4272e00640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c0ce2a0 0x7f426c0ce720 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.201+0000 7f4272e00640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c160e10 0x7f426c0cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.201+0000 7f4272e00640 1 --2- 172.21.15.83:0/2455510776 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4250005bd0 0x7f426c1608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.201+0000 7f4272e00640 1 -- 172.21.15.83:0/2455510776 >> 172.21.15.83:0/2455510776 conn(0x7f426c0b1830 msgr2=0x7f426c15c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:04.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.201+0000 7f4272e00640 1 -- 172.21.15.83:0/2455510776 shutdown_connections 2024-04-25T00:11:04.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.201+0000 7f4272e00640 1 -- 172.21.15.83:0/2455510776 wait complete. 2024-04-25T00:11:04.220 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:11:04.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:11:04.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07dc200640 1 Processor -- start 2024-04-25T00:11:04.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07dc200640 1 -- start start 2024-04-25T00:11:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07dc200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07d415f5d0 0x7f07d40b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07dc200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f07d40b1410 0x7f07d40b1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07dc200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f07d40b1dd0 0x7f07d40cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07dc200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f07d40aea70 con 0x7f07d40b1410 2024-04-25T00:11:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07dc200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f07d40b2240 con 0x7f07d40b1dd0 2024-04-25T00:11:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07dc200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f07d40d2870 con 0x7f07d415f5d0 2024-04-25T00:11:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07d8e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f07d40b1410 0x7f07d40b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07d8e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f07d40b1410 0x7f07d40b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45192/0 (socket says 172.21.15.83:45192) 2024-04-25T00:11:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07d8e00640 1 -- 172.21.15.83:0/1214298189 learned_addr learned my addr 172.21.15.83:0/1214298189 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07da200640 1 --2- 172.21.15.83:0/1214298189 >> v2:172.21.15.86:3300/0 conn(0x7f07d40b1dd0 0x7f07d40cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07d9800640 1 --2- 172.21.15.83:0/1214298189 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07d415f5d0 0x7f07d40b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07d8e00640 1 -- 172.21.15.83:0/1214298189 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07d415f5d0 msgr2=0x7f07d40b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:04.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07d8e00640 1 --2- 172.21.15.83:0/1214298189 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07d415f5d0 0x7f07d40b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07d8e00640 1 -- 172.21.15.83:0/1214298189 >> v2:172.21.15.86:3300/0 conn(0x7f07d40b1dd0 msgr2=0x7f07d40cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:04.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07d8e00640 1 --2- 172.21.15.83:0/1214298189 >> v2:172.21.15.86:3300/0 conn(0x7f07d40b1dd0 0x7f07d40cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.401+0000 7f07d8e00640 1 -- 172.21.15.83:0/1214298189 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d40d2b10 con 0x7f07d40b1410 2024-04-25T00:11:04.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.405+0000 7f07d8e00640 1 --2- 172.21.15.83:0/1214298189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07d40b1410 0x7f07d40b1890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f07d4071fc0 tx=0x7f07c001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c282394c5cf9cbf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:04.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.405+0000 7f07d3e00640 1 -- 172.21.15.83:0/1214298189 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c00b0070 con 0x7f07d40b1410 2024-04-25T00:11:04.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.405+0000 7f07d3e00640 1 -- 172.21.15.83:0/1214298189 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c00ab600 con 0x7f07d40b1410 2024-04-25T00:11:04.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.405+0000 7f07d3e00640 1 -- 172.21.15.83:0/1214298189 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c00ab9d0 con 0x7f07d40b1410 2024-04-25T00:11:04.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.405+0000 7f07dc200640 1 -- 172.21.15.83:0/1214298189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07d40b1410 msgr2=0x7f07d40b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.405+0000 7f07dc200640 1 --2- 172.21.15.83:0/1214298189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07d40b1410 0x7f07d40b1890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f07d4071fc0 tx=0x7f07c001c490 comp rx=0 tx=0).stop 2024-04-25T00:11:04.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.405+0000 7f07dc200640 1 -- 172.21.15.83:0/1214298189 shutdown_connections 2024-04-25T00:11:04.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.405+0000 7f07dc200640 1 --2- 172.21.15.83:0/1214298189 >> v2:172.21.15.86:3300/0 conn(0x7f07d40b1dd0 0x7f07d40cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.405+0000 7f07dc200640 1 --2- 172.21.15.83:0/1214298189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07d40b1410 0x7f07d40b1890 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.405+0000 7f07dc200640 1 --2- 172.21.15.83:0/1214298189 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07d415f5d0 0x7f07d40b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.405+0000 7f07dc200640 1 -- 172.21.15.83:0/1214298189 >> 172.21.15.83:0/1214298189 conn(0x7f07d40d1d50 msgr2=0x7f07d40d2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:04.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.405+0000 7f07dc200640 1 -- 172.21.15.83:0/1214298189 shutdown_connections 2024-04-25T00:11:04.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.405+0000 7f07dc200640 1 -- 172.21.15.83:0/1214298189 wait complete. 2024-04-25T00:11:04.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07dc200640 1 Processor -- start 2024-04-25T00:11:04.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07dc200640 1 -- start start 2024-04-25T00:11:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07dc200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07d40b1410 0x7f07d41658d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07dc200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07d40b1dd0 0x7f07d409f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07dc200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07d415f5d0 0x7f07d415b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07dc200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07d40af640 con 0x7f07d40b1410 2024-04-25T00:11:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07dc200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07d40af4c0 con 0x7f07d415f5d0 2024-04-25T00:11:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07dc200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f07d40af7c0 con 0x7f07d40b1dd0 2024-04-25T00:11:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07d9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07d40b1410 0x7f07d41658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07d9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07d40b1410 0x7f07d41658d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45200/0 (socket says 172.21.15.83:45200) 2024-04-25T00:11:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07d9800640 1 -- 172.21.15.83:0/3831405328 learned_addr learned my addr 172.21.15.83:0/3831405328 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07da200640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07d415f5d0 0x7f07d415b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07d8e00640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07d40b1dd0 0x7f07d409f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07d9800640 1 -- 172.21.15.83:0/3831405328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07d40b1dd0 msgr2=0x7f07d409f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07d9800640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07d40b1dd0 0x7f07d409f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07d9800640 1 -- 172.21.15.83:0/3831405328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07d415f5d0 msgr2=0x7f07d415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07d9800640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07d415f5d0 0x7f07d415b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07d9800640 1 -- 172.21.15.83:0/3831405328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d40c9840 con 0x7f07d40b1410 2024-04-25T00:11:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07da200640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07d415f5d0 0x7f07d415b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07d8e00640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07d40b1dd0 0x7f07d409f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07d9800640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07d40b1410 0x7f07d41658d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f07c800d620 tx=0x7f07c800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07d1600640 1 -- 172.21.15.83:0/3831405328 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c800b620 con 0x7f07d40b1410 2024-04-25T00:11:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07d1600640 1 -- 172.21.15.83:0/3831405328 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c8019070 con 0x7f07d40b1410 2024-04-25T00:11:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07d1600640 1 -- 172.21.15.83:0/3831405328 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c800b9e0 con 0x7f07d40b1410 2024-04-25T00:11:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07dc200640 1 -- 172.21.15.83:0/3831405328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d40c9ab0 con 0x7f07d40b1410 2024-04-25T00:11:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.409+0000 7f07dc200640 1 -- 172.21.15.83:0/3831405328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d40c5880 con 0x7f07d40b1410 2024-04-25T00:11:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.413+0000 7f07dc200640 1 -- 172.21.15.83:0/3831405328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d409fdf0 con 0x7f07d40b1410 2024-04-25T00:11:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.413+0000 7f07d1600640 1 -- 172.21.15.83:0/3831405328 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f07c800bb80 con 0x7f07d40b1410 2024-04-25T00:11:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.413+0000 7f07d1600640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f07b40678c0 0x7f07b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.413+0000 7f07d1600640 1 -- 172.21.15.83:0/3831405328 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f07c8085d70 con 0x7f07d40b1410 2024-04-25T00:11:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.413+0000 7f07d8e00640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f07b40678c0 0x7f07b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.413+0000 7f07d8e00640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f07b40678c0 0x7f07b4069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f07d406e770 tx=0x7f07c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:04.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.417+0000 7f07d1600640 1 -- 172.21.15.83:0/3831405328 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f07c8056bf0 con 0x7f07d40b1410 2024-04-25T00:11:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.573+0000 7f07dc200640 1 -- 172.21.15.83:0/3831405328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f07d404f5f0 con 0x7f07d40b1410 2024-04-25T00:11:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.801+0000 7f07d1600640 1 -- 172.21.15.83:0/3831405328 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f07c80564f0 con 0x7f07d40b1410 2024-04-25T00:11:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-25T00:11:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.805+0000 7f07dc200640 1 -- 172.21.15.83:0/3831405328 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f07b40678c0 msgr2=0x7f07b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.805+0000 7f07dc200640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f07b40678c0 0x7f07b4069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f07d406e770 tx=0x7f07c0008040 comp rx=0 tx=0).stop 2024-04-25T00:11:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.805+0000 7f07dc200640 1 -- 172.21.15.83:0/3831405328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07d40b1410 msgr2=0x7f07d41658d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.805+0000 7f07dc200640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07d40b1410 0x7f07d41658d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f07c800d620 tx=0x7f07c800dae0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.805+0000 7f07dc200640 1 -- 172.21.15.83:0/3831405328 shutdown_connections 2024-04-25T00:11:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.805+0000 7f07dc200640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f07b40678c0 0x7f07b4069d80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.805+0000 7f07dc200640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07d40b1dd0 0x7f07d409f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.805+0000 7f07dc200640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07d415f5d0 0x7f07d415b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.805+0000 7f07dc200640 1 --2- 172.21.15.83:0/3831405328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07d40b1410 0x7f07d41658d0 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f07c800d620 tx=0x7f07c800dae0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.805+0000 7f07dc200640 1 -- 172.21.15.83:0/3831405328 >> 172.21.15.83:0/3831405328 conn(0x7f07d40d1d50 msgr2=0x7f07d406f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.805+0000 7f07dc200640 1 -- 172.21.15.83:0/3831405328 shutdown_connections 2024-04-25T00:11:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.805+0000 7f07dc200640 1 -- 172.21.15.83:0/3831405328 wait complete. 2024-04-25T00:11:04.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T00:11:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.973+0000 7f470f200640 1 Processor -- start 2024-04-25T00:11:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.973+0000 7f470f200640 1 -- start start 2024-04-25T00:11:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.973+0000 7f470f200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f471015ff80 0x7f4710070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.973+0000 7f470f200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47100cee10 con 0x7f4710145500 2024-04-25T00:11:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.973+0000 7f470f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47100cf0f0 con 0x7f471006e1c0 2024-04-25T00:11:04.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.973+0000 7f470f200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f471006cd20 con 0x7f471015ff80 2024-04-25T00:11:04.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.973+0000 7f470d400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f471015ff80 0x7f4710070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.973+0000 7f470de00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4710145500 0x7f471015f870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42548/0 (socket says 172.21.15.83:42548) 2024-04-25T00:11:04.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.973+0000 7f470de00640 1 -- 172.21.15.83:0/2533873754 learned_addr learned my addr 172.21.15.83:0/2533873754 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:04.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.973+0000 7f470d400640 1 -- 172.21.15.83:0/2533873754 >> v1:172.21.15.86:6789/0 conn(0x7f471006e1c0 legacy=0x7f4710071180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.973+0000 7f470d400640 1 -- 172.21.15.83:0/2533873754 >> v1:172.21.15.83:6789/0 conn(0x7f4710145500 legacy=0x7f471015f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.973+0000 7f470d400640 1 -- 172.21.15.83:0/2533873754 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47100cf3d0 con 0x7f471015ff80 2024-04-25T00:11:04.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.977+0000 7f470d400640 1 --2- 172.21.15.83:0/2533873754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471015ff80 0x7f4710070c40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0097c0 tx=0x7f46fc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbefa278937eba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:04.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.977+0000 7f470ca00640 1 -- 172.21.15.83:0/2533873754 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0b1070 con 0x7f471015ff80 2024-04-25T00:11:04.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.977+0000 7f470ca00640 1 -- 172.21.15.83:0/2533873754 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46fc0ac410 con 0x7f471015ff80 2024-04-25T00:11:04.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.977+0000 7f470ca00640 1 -- 172.21.15.83:0/2533873754 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0ac7e0 con 0x7f471015ff80 2024-04-25T00:11:04.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.977+0000 7f470f200640 1 -- 172.21.15.83:0/2533873754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471015ff80 msgr2=0x7f4710070c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.977+0000 7f470f200640 1 --2- 172.21.15.83:0/2533873754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471015ff80 0x7f4710070c40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0097c0 tx=0x7f46fc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.977+0000 7f470f200640 1 -- 172.21.15.83:0/2533873754 shutdown_connections 2024-04-25T00:11:04.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.977+0000 7f470f200640 1 --2- 172.21.15.83:0/2533873754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471015ff80 0x7f4710070c40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.977+0000 7f470f200640 1 -- 172.21.15.83:0/2533873754 >> 172.21.15.83:0/2533873754 conn(0x7f471006b1e0 msgr2=0x7f471006b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:04.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.977+0000 7f470f200640 1 -- 172.21.15.83:0/2533873754 shutdown_connections 2024-04-25T00:11:04.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.977+0000 7f470f200640 1 -- 172.21.15.83:0/2533873754 wait complete. 2024-04-25T00:11:04.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.977+0000 7f470f200640 1 Processor -- start 2024-04-25T00:11:04.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.977+0000 7f470f200640 1 -- start start 2024-04-25T00:11:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f471006e1c0 0x7f4710130eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4710145500 0x7f4710135a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471015ff80 0x7f471015ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4710169d40 con 0x7f471015ff80 2024-04-25T00:11:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4710169bc0 con 0x7f4710145500 2024-04-25T00:11:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4710169ec0 con 0x7f471006e1c0 2024-04-25T00:11:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471015ff80 0x7f471015ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471015ff80 0x7f471015ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45216/0 (socket says 172.21.15.83:45216) 2024-04-25T00:11:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470e800640 1 -- 172.21.15.83:0/7454162 learned_addr learned my addr 172.21.15.83:0/7454162 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470d400640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4710145500 0x7f4710135a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470de00640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f471006e1c0 0x7f4710130eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470e800640 1 -- 172.21.15.83:0/7454162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f471006e1c0 msgr2=0x7f4710130eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470e800640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f471006e1c0 0x7f4710130eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470e800640 1 -- 172.21.15.83:0/7454162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4710145500 msgr2=0x7f4710135a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470e800640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4710145500 0x7f4710135a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470e800640 1 -- 172.21.15.83:0/7454162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47101313f0 con 0x7f471015ff80 2024-04-25T00:11:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470de00640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f471006e1c0 0x7f4710130eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470e800640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471015ff80 0x7f471015ad60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f470800c730 tx=0x7f470800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f4706000640 1 -- 172.21.15.83:0/7454162 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4708003b80 con 0x7f471015ff80 2024-04-25T00:11:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f4706000640 1 -- 172.21.15.83:0/7454162 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4708003d20 con 0x7f471015ff80 2024-04-25T00:11:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f4706000640 1 -- 172.21.15.83:0/7454162 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4708005310 con 0x7f471015ff80 2024-04-25T00:11:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470f200640 1 -- 172.21.15.83:0/7454162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47100b2b80 con 0x7f471015ff80 2024-04-25T00:11:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470f200640 1 -- 172.21.15.83:0/7454162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47100b3140 con 0x7f471015ff80 2024-04-25T00:11:04.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.981+0000 7f470f200640 1 -- 172.21.15.83:0/7454162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f471004f510 con 0x7f471015ff80 2024-04-25T00:11:04.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.985+0000 7f4706000640 1 -- 172.21.15.83:0/7454162 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f47080056c0 con 0x7f471015ff80 2024-04-25T00:11:04.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.985+0000 7f4706000640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f46f00678c0 0x7f46f0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.985+0000 7f4706000640 1 -- 172.21.15.83:0/7454162 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f4708085a40 con 0x7f471015ff80 2024-04-25T00:11:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.985+0000 7f470de00640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f46f00678c0 0x7f46f0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.985+0000 7f470de00640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f46f00678c0 0x7f46f0069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4700005890 tx=0x7f4700009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:04.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:04.989+0000 7f4706000640 1 -- 172.21.15.83:0/7454162 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4708056750 con 0x7f471015ff80 2024-04-25T00:11:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.145+0000 7f470f200640 1 -- 172.21.15.83:0/7454162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f47100cd460 con 0x7f471015ff80 2024-04-25T00:11:05.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.805+0000 7f4706000640 1 -- 172.21.15.83:0/7454162 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f4708056050 con 0x7f471015ff80 2024-04-25T00:11:05.807 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-25T00:11:05.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.809+0000 7f470f200640 1 -- 172.21.15.83:0/7454162 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f46f00678c0 msgr2=0x7f46f0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.809+0000 7f470f200640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f46f00678c0 0x7f46f0069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4700005890 tx=0x7f4700009040 comp rx=0 tx=0).stop 2024-04-25T00:11:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.809+0000 7f470f200640 1 -- 172.21.15.83:0/7454162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471015ff80 msgr2=0x7f471015ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.809+0000 7f470f200640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471015ff80 0x7f471015ad60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f470800c730 tx=0x7f470800cbf0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.809+0000 7f470f200640 1 -- 172.21.15.83:0/7454162 shutdown_connections 2024-04-25T00:11:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.809+0000 7f470f200640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f46f00678c0 0x7f46f0069d80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.809+0000 7f470f200640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471015ff80 0x7f471015ad60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.809+0000 7f470f200640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4710145500 0x7f4710135a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.809+0000 7f470f200640 1 --2- 172.21.15.83:0/7454162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f471006e1c0 0x7f4710130eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.809+0000 7f470f200640 1 -- 172.21.15.83:0/7454162 >> 172.21.15.83:0/7454162 conn(0x7f471006b1e0 msgr2=0x7f471015c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.809+0000 7f470f200640 1 -- 172.21.15.83:0/7454162 shutdown_connections 2024-04-25T00:11:05.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.809+0000 7f470f200640 1 -- 172.21.15.83:0/7454162 wait complete. 2024-04-25T00:11:05.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:11:05.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f6576000640 1 Processor -- start 2024-04-25T00:11:05.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f6576000640 1 -- start start 2024-04-25T00:11:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f6576000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65701551e0 0x7f657015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f6576000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f65701555c0 0x7f6570071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f6576000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f657015c560 con 0x7f657006e4c0 2024-04-25T00:11:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f6576000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6570072460 con 0x7f65701555c0 2024-04-25T00:11:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f6576000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f657015c7d0 con 0x7f65701551e0 2024-04-25T00:11:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f656fe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f657006e4c0 0x7f657015e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42554/0 (socket says 172.21.15.83:42554) 2024-04-25T00:11:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f656fe00640 1 -- 172.21.15.83:0/2024587101 learned_addr learned my addr 172.21.15.83:0/2024587101 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f656f400640 1 --2- 172.21.15.83:0/2024587101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65701551e0 0x7f657015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f6567e00640 1 --2- 172.21.15.83:0/2024587101 >> v2:172.21.15.86:3300/0 conn(0x7f65701555c0 0x7f6570071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f656ea00640 1 -- 172.21.15.83:0/2024587101 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 146458899 0 0) 0x7f657015c560 con 0x7f657006e4c0 2024-04-25T00:11:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f656ea00640 1 -- 172.21.15.83:0/2024587101 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6554003530 con 0x7f657006e4c0 2024-04-25T00:11:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f656f400640 1 -- 172.21.15.83:0/2024587101 >> v2:172.21.15.86:3300/0 conn(0x7f65701555c0 msgr2=0x7f6570071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f656f400640 1 --2- 172.21.15.83:0/2024587101 >> v2:172.21.15.86:3300/0 conn(0x7f65701555c0 0x7f6570071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f656f400640 1 -- 172.21.15.83:0/2024587101 >> v1:172.21.15.83:6789/0 conn(0x7f657006e4c0 legacy=0x7f657015e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f656f400640 1 -- 172.21.15.83:0/2024587101 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f657015cab0 con 0x7f65701551e0 2024-04-25T00:11:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.993+0000 7f6567e00640 1 --2- 172.21.15.83:0/2024587101 >> v2:172.21.15.86:3300/0 conn(0x7f65701555c0 0x7f6570071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:05.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.997+0000 7f656f400640 1 --2- 172.21.15.83:0/2024587101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65701551e0 0x7f657015bea0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65600097c0 tx=0x7f656001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f01cd8962b534e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.997+0000 7f656ea00640 1 -- 172.21.15.83:0/2024587101 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65600b1070 con 0x7f65701551e0 2024-04-25T00:11:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.997+0000 7f656ea00640 1 -- 172.21.15.83:0/2024587101 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65600ac620 con 0x7f65701551e0 2024-04-25T00:11:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.997+0000 7f656ea00640 1 -- 172.21.15.83:0/2024587101 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65600ac9d0 con 0x7f65701551e0 2024-04-25T00:11:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.997+0000 7f6576000640 1 -- 172.21.15.83:0/2024587101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65701551e0 msgr2=0x7f657015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.997+0000 7f6576000640 1 --2- 172.21.15.83:0/2024587101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65701551e0 0x7f657015bea0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65600097c0 tx=0x7f656001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.997+0000 7f6576000640 1 -- 172.21.15.83:0/2024587101 shutdown_connections 2024-04-25T00:11:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.997+0000 7f6576000640 1 --2- 172.21.15.83:0/2024587101 >> v2:172.21.15.86:3300/0 conn(0x7f65701555c0 0x7f6570071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.997+0000 7f6576000640 1 --2- 172.21.15.83:0/2024587101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65701551e0 0x7f657015bea0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.997+0000 7f6576000640 1 -- 172.21.15.83:0/2024587101 >> 172.21.15.83:0/2024587101 conn(0x7f65700b1830 msgr2=0x7f65700b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:06.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.997+0000 7f6576000640 1 -- 172.21.15.83:0/2024587101 shutdown_connections 2024-04-25T00:11:06.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.997+0000 7f6576000640 1 -- 172.21.15.83:0/2024587101 wait complete. 2024-04-25T00:11:06.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:05.997+0000 7f6576000640 1 Processor -- start 2024-04-25T00:11:06.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6576000640 1 -- start start 2024-04-25T00:11:06.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6576000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657006e4c0 0x7f65700b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6576000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65701551e0 0x7f65700ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6576000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65701555c0 0x7f6570166c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6576000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6570165e00 con 0x7f657006e4c0 2024-04-25T00:11:06.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6576000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6570165c80 con 0x7f65701555c0 2024-04-25T00:11:06.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6576000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6570165f80 con 0x7f65701551e0 2024-04-25T00:11:06.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f656f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657006e4c0 0x7f65700b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f656f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657006e4c0 0x7f65700b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45232/0 (socket says 172.21.15.83:45232) 2024-04-25T00:11:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f656f400640 1 -- 172.21.15.83:0/396293987 learned_addr learned my addr 172.21.15.83:0/396293987 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f656fe00640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65701555c0 0x7f6570166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6567e00640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65701551e0 0x7f65700ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f656f400640 1 -- 172.21.15.83:0/396293987 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65701551e0 msgr2=0x7f65700ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f656f400640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65701551e0 0x7f65700ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f656f400640 1 -- 172.21.15.83:0/396293987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65701555c0 msgr2=0x7f6570166c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f656f400640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65701555c0 0x7f6570166c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f656f400640 1 -- 172.21.15.83:0/396293987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65701476e0 con 0x7f657006e4c0 2024-04-25T00:11:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6567e00640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65701551e0 0x7f65700ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f656fe00640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65701555c0 0x7f6570166c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f656f400640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657006e4c0 0x7f65700b3060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f656001a0d0 tx=0x7f6560019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6567400640 1 -- 172.21.15.83:0/396293987 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65600b1040 con 0x7f657006e4c0 2024-04-25T00:11:06.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6567400640 1 -- 172.21.15.83:0/396293987 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65600ac420 con 0x7f657006e4c0 2024-04-25T00:11:06.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6567400640 1 -- 172.21.15.83:0/396293987 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560006e20 con 0x7f657006e4c0 2024-04-25T00:11:06.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6576000640 1 -- 172.21.15.83:0/396293987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65701479c0 con 0x7f657006e4c0 2024-04-25T00:11:06.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6576000640 1 -- 172.21.15.83:0/396293987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6570147f80 con 0x7f657006e4c0 2024-04-25T00:11:06.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.001+0000 7f6576000640 1 -- 172.21.15.83:0/396293987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f657004f510 con 0x7f657006e4c0 2024-04-25T00:11:06.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.005+0000 7f6567400640 1 -- 172.21.15.83:0/396293987 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f65600be030 con 0x7f657006e4c0 2024-04-25T00:11:06.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.005+0000 7f6567400640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f654c0678c0 0x7f654c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.005+0000 7f6567400640 1 -- 172.21.15.83:0/396293987 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f656011f0e0 con 0x7f657006e4c0 2024-04-25T00:11:06.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.005+0000 7f6567e00640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f654c0678c0 0x7f654c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.005+0000 7f6567e00640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f654c0678c0 0x7f654c069d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f65701fa4f0 tx=0x7f6558008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.009+0000 7f6567400640 1 -- 172.21.15.83:0/396293987 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f65600c5060 con 0x7f657006e4c0 2024-04-25T00:11:06.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.197+0000 7f6576000640 1 -- 172.21.15.83:0/396293987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f657006ce30 con 0x7f657006e4c0 2024-04-25T00:11:06.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.837+0000 7f6567400640 1 -- 172.21.15.83:0/396293987 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f6560016090 con 0x7f657006e4c0 2024-04-25T00:11:06.843 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T00:11:06.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.841+0000 7f6576000640 1 -- 172.21.15.83:0/396293987 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f654c0678c0 msgr2=0x7f654c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.841+0000 7f6576000640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f654c0678c0 0x7f654c069d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f65701fa4f0 tx=0x7f6558008040 comp rx=0 tx=0).stop 2024-04-25T00:11:06.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.841+0000 7f6576000640 1 -- 172.21.15.83:0/396293987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657006e4c0 msgr2=0x7f65700b3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.841+0000 7f6576000640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657006e4c0 0x7f65700b3060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f656001a0d0 tx=0x7f6560019240 comp rx=0 tx=0).stop 2024-04-25T00:11:06.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.841+0000 7f6576000640 1 -- 172.21.15.83:0/396293987 shutdown_connections 2024-04-25T00:11:06.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.841+0000 7f6576000640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f654c0678c0 0x7f654c069d80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.841+0000 7f6576000640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65701551e0 0x7f65700ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.841+0000 7f6576000640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65701555c0 0x7f6570166c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.841+0000 7f6576000640 1 --2- 172.21.15.83:0/396293987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f657006e4c0 0x7f65700b3060 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.841+0000 7f6576000640 1 -- 172.21.15.83:0/396293987 >> 172.21.15.83:0/396293987 conn(0x7f65700b1830 msgr2=0x7f657015e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:06.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.845+0000 7f6576000640 1 -- 172.21.15.83:0/396293987 shutdown_connections 2024-04-25T00:11:06.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:06.845+0000 7f6576000640 1 -- 172.21.15.83:0/396293987 wait complete. 2024-04-25T00:11:06.880 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T00:11:06.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T00:11:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.016+0000 7f0c45c00640 1 Processor -- start 2024-04-25T00:11:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.016+0000 7f0c45c00640 1 -- start start 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.016+0000 7f0c45c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0c401455d0 0x7f0c4006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.016+0000 7f0c45c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0c400d0450 0x7f0c400b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.016+0000 7f0c45c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c400ce610 con 0x7f0c400ce080 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.016+0000 7f0c45c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0c400ce8f0 con 0x7f0c401455d0 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.016+0000 7f0c45c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0c400cebd0 con 0x7f0c400d0450 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.020+0000 7f0c3ea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0c401455d0 0x7f0c4006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.020+0000 7f0c3ea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0c401455d0 0x7f0c4006d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45236/0 (socket says 172.21.15.83:45236) 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.020+0000 7f0c3ea00640 1 -- 172.21.15.83:0/529646092 learned_addr learned my addr 172.21.15.83:0/529646092 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.020+0000 7f0c3f400640 1 --2- 172.21.15.83:0/529646092 >> v2:172.21.15.86:3300/0 conn(0x7f0c400d0450 0x7f0c400b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.020+0000 7f0c3ea00640 1 -- 172.21.15.83:0/529646092 >> v2:172.21.15.86:3300/0 conn(0x7f0c400d0450 msgr2=0x7f0c400b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.020+0000 7f0c3ea00640 1 --2- 172.21.15.83:0/529646092 >> v2:172.21.15.86:3300/0 conn(0x7f0c400d0450 0x7f0c400b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.020+0000 7f0c3ea00640 1 -- 172.21.15.83:0/529646092 >> v1:172.21.15.86:6789/0 conn(0x7f0c400ce080 legacy=0x7f0c400cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.020+0000 7f0c3ea00640 1 -- 172.21.15.83:0/529646092 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c400ceeb0 con 0x7f0c401455d0 2024-04-25T00:11:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.020+0000 7f0c3ea00640 1 --2- 172.21.15.83:0/529646092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c401455d0 0x7f0c4006d7e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c401609d0 tx=0x7f0c3001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d3fca30b901dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.020+0000 7f0c3e000640 1 -- 172.21.15.83:0/529646092 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c300ab640 con 0x7f0c401455d0 2024-04-25T00:11:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.020+0000 7f0c3e000640 1 -- 172.21.15.83:0/529646092 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c300b0070 con 0x7f0c401455d0 2024-04-25T00:11:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.020+0000 7f0c3e000640 1 -- 172.21.15.83:0/529646092 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c300aba00 con 0x7f0c401455d0 2024-04-25T00:11:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 -- 172.21.15.83:0/529646092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c401455d0 msgr2=0x7f0c4006d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 --2- 172.21.15.83:0/529646092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c401455d0 0x7f0c4006d7e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c401609d0 tx=0x7f0c3001c4e0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 -- 172.21.15.83:0/529646092 shutdown_connections 2024-04-25T00:11:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 --2- 172.21.15.83:0/529646092 >> v2:172.21.15.86:3300/0 conn(0x7f0c400d0450 0x7f0c400b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 --2- 172.21.15.83:0/529646092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c401455d0 0x7f0c4006d7e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 -- 172.21.15.83:0/529646092 >> 172.21.15.83:0/529646092 conn(0x7f0c4006b3b0 msgr2=0x7f0c4006b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 -- 172.21.15.83:0/529646092 shutdown_connections 2024-04-25T00:11:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 -- 172.21.15.83:0/529646092 wait complete. 2024-04-25T00:11:07.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 Processor -- start 2024-04-25T00:11:07.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 -- start start 2024-04-25T00:11:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c400ce080 0x7f0c40140740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c400d0450 0x7f0c4015f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c401455d0 0x7f0c40070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c400b3a00 con 0x7f0c400ce080 2024-04-25T00:11:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c400b3880 con 0x7f0c401455d0 2024-04-25T00:11:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c45c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0c400b3b80 con 0x7f0c400d0450 2024-04-25T00:11:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c400ce080 0x7f0c40140740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c400ce080 0x7f0c40140740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45238/0 (socket says 172.21.15.83:45238) 2024-04-25T00:11:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c3ea00640 1 -- 172.21.15.83:0/3186413832 learned_addr learned my addr 172.21.15.83:0/3186413832 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c3f400640 1 --2- 172.21.15.83:0/3186413832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c401455d0 0x7f0c40070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c36000640 1 --2- 172.21.15.83:0/3186413832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c400d0450 0x7f0c4015f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c3ea00640 1 -- 172.21.15.83:0/3186413832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c400d0450 msgr2=0x7f0c4015f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c3ea00640 1 --2- 172.21.15.83:0/3186413832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c400d0450 0x7f0c4015f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c3ea00640 1 -- 172.21.15.83:0/3186413832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c401455d0 msgr2=0x7f0c40070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c3ea00640 1 --2- 172.21.15.83:0/3186413832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c401455d0 0x7f0c40070dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.024+0000 7f0c3ea00640 1 -- 172.21.15.83:0/3186413832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c400b2e40 con 0x7f0c400ce080 2024-04-25T00:11:07.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.028+0000 7f0c3ea00640 1 --2- 172.21.15.83:0/3186413832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c400ce080 0x7f0c40140740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c300ad040 tx=0x7f0c3001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.028+0000 7f0c37400640 1 -- 172.21.15.83:0/3186413832 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c300be3f0 con 0x7f0c400ce080 2024-04-25T00:11:07.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.028+0000 7f0c37400640 1 -- 172.21.15.83:0/3186413832 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c300be5b0 con 0x7f0c400ce080 2024-04-25T00:11:07.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.028+0000 7f0c37400640 1 -- 172.21.15.83:0/3186413832 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c300be9a0 con 0x7f0c400ce080 2024-04-25T00:11:07.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.028+0000 7f0c45c00640 1 -- 172.21.15.83:0/3186413832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c40072c30 con 0x7f0c400ce080 2024-04-25T00:11:07.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.028+0000 7f0c45c00640 1 -- 172.21.15.83:0/3186413832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4015eed0 con 0x7f0c400ce080 2024-04-25T00:11:07.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.028+0000 7f0c45c00640 1 -- 172.21.15.83:0/3186413832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c400736f0 con 0x7f0c400ce080 2024-04-25T00:11:07.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.028+0000 7f0c37400640 1 -- 172.21.15.83:0/3186413832 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0c300c7440 con 0x7f0c400ce080 2024-04-25T00:11:07.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.028+0000 7f0c37400640 1 --2- 172.21.15.83:0/3186413832 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0c1c0678c0 0x7f0c1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.028+0000 7f0c37400640 1 -- 172.21.15.83:0/3186413832 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f0c30120270 con 0x7f0c400ce080 2024-04-25T00:11:07.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.028+0000 7f0c36000640 1 --2- 172.21.15.83:0/3186413832 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0c1c0678c0 0x7f0c1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.028+0000 7f0c36000640 1 --2- 172.21.15.83:0/3186413832 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0c1c0678c0 0x7f0c1c069d80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c401fa870 tx=0x7f0c28007e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.032+0000 7f0c37400640 1 -- 172.21.15.83:0/3186413832 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0c300160d0 con 0x7f0c400ce080 2024-04-25T00:11:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.196+0000 7f0c45c00640 1 -- 172.21.15.83:0/3186413832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f0c4006eb90 con 0x7f0c400ce080 2024-04-25T00:11:07.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.844+0000 7f0c37400640 1 -- 172.21.15.83:0/3186413832 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f0c300f0870 con 0x7f0c400ce080 2024-04-25T00:11:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.844+0000 7f0c45c00640 1 -- 172.21.15.83:0/3186413832 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0c1c0678c0 msgr2=0x7f0c1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.844+0000 7f0c45c00640 1 --2- 172.21.15.83:0/3186413832 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0c1c0678c0 0x7f0c1c069d80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c401fa870 tx=0x7f0c28007e40 comp rx=0 tx=0).stop 2024-04-25T00:11:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.844+0000 7f0c45c00640 1 -- 172.21.15.83:0/3186413832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c400ce080 msgr2=0x7f0c40140740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.844+0000 7f0c45c00640 1 --2- 172.21.15.83:0/3186413832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c400ce080 0x7f0c40140740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c300ad040 tx=0x7f0c3001ad20 comp rx=0 tx=0).stop 2024-04-25T00:11:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.844+0000 7f0c45c00640 1 -- 172.21.15.83:0/3186413832 shutdown_connections 2024-04-25T00:11:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.844+0000 7f0c45c00640 1 --2- 172.21.15.83:0/3186413832 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0c1c0678c0 0x7f0c1c069d80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.844+0000 7f0c45c00640 1 --2- 172.21.15.83:0/3186413832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c400d0450 0x7f0c4015f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.844+0000 7f0c45c00640 1 --2- 172.21.15.83:0/3186413832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c401455d0 0x7f0c40070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.844+0000 7f0c45c00640 1 --2- 172.21.15.83:0/3186413832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c400ce080 0x7f0c40140740 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c300ad040 tx=0x7f0c3001ad20 comp rx=0 tx=0).stop 2024-04-25T00:11:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.844+0000 7f0c45c00640 1 -- 172.21.15.83:0/3186413832 >> 172.21.15.83:0/3186413832 conn(0x7f0c4006b3b0 msgr2=0x7f0c40142150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:07.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.844+0000 7f0c45c00640 1 -- 172.21.15.83:0/3186413832 shutdown_connections 2024-04-25T00:11:07.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:07.844+0000 7f0c45c00640 1 -- 172.21.15.83:0/3186413832 wait complete. 2024-04-25T00:11:07.871 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:11:08.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f685d000640 1 Processor -- start 2024-04-25T00:11:08.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f685d000640 1 -- start start 2024-04-25T00:11:08.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f685d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6858159810 0x7f68580cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f685d000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f685806e950 0x7f685815f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f685d000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f685815fe50 0x7f685815c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f685d000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f685806e210 con 0x7f685815fe50 2024-04-25T00:11:08.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f685d000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f685806da00 con 0x7f6858159810 2024-04-25T00:11:08.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f685d000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f68580ce000 con 0x7f685806e950 2024-04-25T00:11:08.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f6856800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f685815fe50 0x7f685815c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f6856800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f685815fe50 0x7f685815c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45250/0 (socket says 172.21.15.83:45250) 2024-04-25T00:11:08.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f6856800640 1 -- 172.21.15.83:0/4282569696 learned_addr learned my addr 172.21.15.83:0/4282569696 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:08.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f684d400640 1 --2- 172.21.15.83:0/4282569696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f685806e950 0x7f685815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f6855e00640 1 --2- 172.21.15.83:0/4282569696 >> v2:172.21.15.86:3300/0 conn(0x7f6858159810 0x7f68580cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f6856800640 1 -- 172.21.15.83:0/4282569696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f685806e950 msgr2=0x7f685815f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f6856800640 1 --2- 172.21.15.83:0/4282569696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f685806e950 0x7f685815f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f6856800640 1 -- 172.21.15.83:0/4282569696 >> v2:172.21.15.86:3300/0 conn(0x7f6858159810 msgr2=0x7f68580cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f6856800640 1 --2- 172.21.15.83:0/4282569696 >> v2:172.21.15.86:3300/0 conn(0x7f6858159810 0x7f68580cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.012+0000 7f6856800640 1 -- 172.21.15.83:0/4282569696 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685806eec0 con 0x7f685815fe50 2024-04-25T00:11:08.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.016+0000 7f6856800640 1 --2- 172.21.15.83:0/4282569696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685815fe50 0x7f685815c320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f68580b4f50 tx=0x7f685001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dd8d8a5b7be45d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.016+0000 7f6855400640 1 -- 172.21.15.83:0/4282569696 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68500a56d0 con 0x7f685815fe50 2024-04-25T00:11:08.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.016+0000 7f6855400640 1 -- 172.21.15.83:0/4282569696 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68500a5870 con 0x7f685815fe50 2024-04-25T00:11:08.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.016+0000 7f6855400640 1 -- 172.21.15.83:0/4282569696 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68500a5c40 con 0x7f685815fe50 2024-04-25T00:11:08.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.016+0000 7f685d000640 1 -- 172.21.15.83:0/4282569696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685815fe50 msgr2=0x7f685815c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.016+0000 7f685d000640 1 --2- 172.21.15.83:0/4282569696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685815fe50 0x7f685815c320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f68580b4f50 tx=0x7f685001c960 comp rx=0 tx=0).stop 2024-04-25T00:11:08.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.016+0000 7f685d000640 1 -- 172.21.15.83:0/4282569696 shutdown_connections 2024-04-25T00:11:08.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.016+0000 7f685d000640 1 --2- 172.21.15.83:0/4282569696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f685806e950 0x7f685815f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.016+0000 7f685d000640 1 --2- 172.21.15.83:0/4282569696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685815fe50 0x7f685815c320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.016+0000 7f685d000640 1 --2- 172.21.15.83:0/4282569696 >> v2:172.21.15.86:3300/0 conn(0x7f6858159810 0x7f68580cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.016+0000 7f685d000640 1 -- 172.21.15.83:0/4282569696 >> 172.21.15.83:0/4282569696 conn(0x7f68580b1830 msgr2=0x7f68580b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.016+0000 7f685d000640 1 -- 172.21.15.83:0/4282569696 shutdown_connections 2024-04-25T00:11:08.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.016+0000 7f685d000640 1 -- 172.21.15.83:0/4282569696 wait complete. 2024-04-25T00:11:08.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f685d000640 1 Processor -- start 2024-04-25T00:11:08.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f685d000640 1 -- start start 2024-04-25T00:11:08.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f685d000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f685806e950 0x7f68580ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f685d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6858159810 0x7f685814eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f685d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685815fe50 0x7f68580afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f685d000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6858159d70 con 0x7f685815fe50 2024-04-25T00:11:08.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f685d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6858159bf0 con 0x7f6858159810 2024-04-25T00:11:08.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f685d000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6858070da0 con 0x7f685806e950 2024-04-25T00:11:08.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f6856800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685815fe50 0x7f68580afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f6856800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685815fe50 0x7f68580afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45254/0 (socket says 172.21.15.83:45254) 2024-04-25T00:11:08.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f6856800640 1 -- 172.21.15.83:0/1887810676 learned_addr learned my addr 172.21.15.83:0/1887810676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:08.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f6855e00640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f685806e950 0x7f68580ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f684d400640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6858159810 0x7f685814eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f6856800640 1 -- 172.21.15.83:0/1887810676 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f685806e950 msgr2=0x7f68580ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f6856800640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f685806e950 0x7f68580ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f6856800640 1 -- 172.21.15.83:0/1887810676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6858159810 msgr2=0x7f685814eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f6856800640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6858159810 0x7f685814eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f6856800640 1 -- 172.21.15.83:0/1887810676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68580b3a50 con 0x7f685815fe50 2024-04-25T00:11:08.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f6855e00640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f685806e950 0x7f68580ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:08.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f684d400640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6858159810 0x7f685814eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:08.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f6856800640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685815fe50 0x7f68580afb70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f685001b010 tx=0x7f6850019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f684ea00640 1 -- 172.21.15.83:0/1887810676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68500bc020 con 0x7f685815fe50 2024-04-25T00:11:08.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f684ea00640 1 -- 172.21.15.83:0/1887810676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68500b8670 con 0x7f685815fe50 2024-04-25T00:11:08.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f685d000640 1 -- 172.21.15.83:0/1887810676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6858159ef0 con 0x7f685815fe50 2024-04-25T00:11:08.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f684ea00640 1 -- 172.21.15.83:0/1887810676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68500b8a60 con 0x7f685815fe50 2024-04-25T00:11:08.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.020+0000 7f685d000640 1 -- 172.21.15.83:0/1887810676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685815a4b0 con 0x7f685815fe50 2024-04-25T00:11:08.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.024+0000 7f685d000640 1 -- 172.21.15.83:0/1887810676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f685804f510 con 0x7f685815fe50 2024-04-25T00:11:08.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.024+0000 7f684ea00640 1 -- 172.21.15.83:0/1887810676 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f68500bf020 con 0x7f685815fe50 2024-04-25T00:11:08.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.024+0000 7f684ea00640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f68340678c0 0x7f6834069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.024+0000 7f684ea00640 1 -- 172.21.15.83:0/1887810676 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f6850120eb0 con 0x7f685815fe50 2024-04-25T00:11:08.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.024+0000 7f6855e00640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f68340678c0 0x7f6834069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.024+0000 7f6855e00640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f68340678c0 0x7f6834069d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f68480079f0 tx=0x7f6848008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.028+0000 7f684ea00640 1 -- 172.21.15.83:0/1887810676 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f68500c6030 con 0x7f685815fe50 2024-04-25T00:11:08.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:08.188+0000 7f685d000640 1 -- 172.21.15.83:0/1887810676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f68580cdc00 con 0x7f685815fe50 2024-04-25T00:11:09.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:09.856+0000 7f684ea00640 1 -- 172.21.15.83:0/1887810676 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f68500165e0 con 0x7f685815fe50 2024-04-25T00:11:09.861 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data_ec' created 2024-04-25T00:11:09.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:09.860+0000 7f685d000640 1 -- 172.21.15.83:0/1887810676 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f68340678c0 msgr2=0x7f6834069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:09.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:09.860+0000 7f685d000640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f68340678c0 0x7f6834069d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f68480079f0 tx=0x7f6848008040 comp rx=0 tx=0).stop 2024-04-25T00:11:09.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:09.860+0000 7f685d000640 1 -- 172.21.15.83:0/1887810676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685815fe50 msgr2=0x7f68580afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:09.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:09.860+0000 7f685d000640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685815fe50 0x7f68580afb70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f685001b010 tx=0x7f6850019700 comp rx=0 tx=0).stop 2024-04-25T00:11:09.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:09.864+0000 7f685d000640 1 -- 172.21.15.83:0/1887810676 shutdown_connections 2024-04-25T00:11:09.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:09.864+0000 7f685d000640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f68340678c0 0x7f6834069d80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:09.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:09.864+0000 7f685d000640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685815fe50 0x7f68580afb70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:09.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:09.864+0000 7f685d000640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6858159810 0x7f685814eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:09.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:09.864+0000 7f685d000640 1 --2- 172.21.15.83:0/1887810676 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f685806e950 0x7f68580ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:09.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:09.864+0000 7f685d000640 1 -- 172.21.15.83:0/1887810676 >> 172.21.15.83:0/1887810676 conn(0x7f68580b1830 msgr2=0x7f685815d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:09.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:09.864+0000 7f685d000640 1 -- 172.21.15.83:0/1887810676 shutdown_connections 2024-04-25T00:11:09.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:09.864+0000 7f685d000640 1 -- 172.21.15.83:0/1887810676 wait complete. 2024-04-25T00:11:09.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T00:11:10.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.036+0000 7fab36600640 1 Processor -- start 2024-04-25T00:11:10.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.036+0000 7fab36600640 1 -- start start 2024-04-25T00:11:10.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab36600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab300cfd40 0x7fab300d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:10.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab36600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3015b480 con 0x7fab301445e0 2024-04-25T00:11:10.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab36600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3015c370 con 0x7fab300b4330 2024-04-25T00:11:10.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab36600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fab300d0710 con 0x7fab300cfd40 2024-04-25T00:11:10.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2f400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fab301445e0 0x7fab30072120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42556/0 (socket says 172.21.15.83:42556) 2024-04-25T00:11:10.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2f400640 1 -- 172.21.15.83:0/3815646110 learned_addr learned my addr 172.21.15.83:0/3815646110 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 -- 172.21.15.83:0/3815646110 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3120438099 0 0) 0x7fab3015b480 con 0x7fab301445e0 2024-04-25T00:11:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 -- 172.21.15.83:0/3815646110 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab14003530 con 0x7fab301445e0 2024-04-25T00:11:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2fe00640 1 --2- 172.21.15.83:0/3815646110 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab300cfd40 0x7fab300d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 -- 172.21.15.83:0/3815646110 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1329016150 0 0) 0x7fab14003530 con 0x7fab301445e0 2024-04-25T00:11:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 -- 172.21.15.83:0/3815646110 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab300cfd40 msgr2=0x7fab300d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 --2- 172.21.15.83:0/3815646110 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab300cfd40 0x7fab300d01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 -- 172.21.15.83:0/3815646110 >> v1:172.21.15.86:6789/0 conn(0x7fab300b4330 legacy=0x7fab3015bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 -- 172.21.15.83:0/3815646110 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab301449c0 con 0x7fab301445e0 2024-04-25T00:11:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 -- 172.21.15.83:0/3815646110 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fab20001a00 con 0x7fab301445e0 2024-04-25T00:11:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2fe00640 1 --2- 172.21.15.83:0/3815646110 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab300cfd40 0x7fab300d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 -- 172.21.15.83:0/3815646110 >> v1:172.21.15.83:6789/0 conn(0x7fab301445e0 legacy=0x7fab30072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 --2- 172.21.15.83:0/3815646110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab140039f0 0x7fab14003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 --2- 172.21.15.83:0/3815646110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab14004810 0x7fab14004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 --2- 172.21.15.83:0/3815646110 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab300cfd40 0x7fab140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 -- 172.21.15.83:0/3815646110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab1400bad0 con 0x7fab140039f0 2024-04-25T00:11:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 -- 172.21.15.83:0/3815646110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab1400bc80 con 0x7fab14004810 2024-04-25T00:11:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2ea00640 1 -- 172.21.15.83:0/3815646110 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fab1400be80 con 0x7fab300cfd40 2024-04-25T00:11:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2f400640 1 --2- 172.21.15.83:0/3815646110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab140039f0 0x7fab14003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab26a00640 1 --2- 172.21.15.83:0/3815646110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab14004810 0x7fab14004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2fe00640 1 --2- 172.21.15.83:0/3815646110 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab300cfd40 0x7fab140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2f400640 1 -- 172.21.15.83:0/3815646110 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab300cfd40 msgr2=0x7fab140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2f400640 1 --2- 172.21.15.83:0/3815646110 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab300cfd40 0x7fab140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2f400640 1 -- 172.21.15.83:0/3815646110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab14004810 msgr2=0x7fab14004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2f400640 1 --2- 172.21.15.83:0/3815646110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab14004810 0x7fab14004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.040+0000 7fab2f400640 1 -- 172.21.15.83:0/3815646110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab14003530 con 0x7fab140039f0 2024-04-25T00:11:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.044+0000 7fab2f400640 1 --2- 172.21.15.83:0/3815646110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab140039f0 0x7fab14003e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab2000a850 tx=0x7fab2001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1209247e64519c38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.044+0000 7fab2ea00640 1 -- 172.21.15.83:0/3815646110 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2000a4a0 con 0x7fab140039f0 2024-04-25T00:11:10.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.044+0000 7fab2ea00640 1 -- 172.21.15.83:0/3815646110 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab200ad650 con 0x7fab140039f0 2024-04-25T00:11:10.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.044+0000 7fab2ea00640 1 -- 172.21.15.83:0/3815646110 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab200ada40 con 0x7fab140039f0 2024-04-25T00:11:10.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.044+0000 7fab36600640 1 -- 172.21.15.83:0/3815646110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab140039f0 msgr2=0x7fab14003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:10.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.044+0000 7fab36600640 1 --2- 172.21.15.83:0/3815646110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab140039f0 0x7fab14003e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab2000a850 tx=0x7fab2001d3a0 comp rx=0 tx=0).stop 2024-04-25T00:11:10.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.044+0000 7fab2fe00640 1 -- 172.21.15.83:0/3815646110 reap_dead start 2024-04-25T00:11:10.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.044+0000 7fab36600640 1 -- 172.21.15.83:0/3815646110 shutdown_connections 2024-04-25T00:11:10.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.044+0000 7fab36600640 1 -- 172.21.15.83:0/3815646110 >> 172.21.15.83:0/3815646110 conn(0x7fab3006b3b0 msgr2=0x7fab3006b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:10.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.044+0000 7fab36600640 1 -- 172.21.15.83:0/3815646110 shutdown_connections 2024-04-25T00:11:10.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.044+0000 7fab36600640 1 -- 172.21.15.83:0/3815646110 wait complete. 2024-04-25T00:11:10.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.044+0000 7fab36600640 1 Processor -- start 2024-04-25T00:11:10.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.044+0000 7fab36600640 1 -- start start 2024-04-25T00:11:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab36600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab140039f0 0x7fab30157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab36600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab30137cf0 0x7fab3013c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab36600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab30138170 0x7fab3013c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab36600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab300b4740 con 0x7fab30138170 2024-04-25T00:11:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab36600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab300d0560 con 0x7fab30137cf0 2024-04-25T00:11:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab36600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fab30166e10 con 0x7fab140039f0 2024-04-25T00:11:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab2fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab30138170 0x7fab3013c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab2fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab30138170 0x7fab3013c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45264/0 (socket says 172.21.15.83:45264) 2024-04-25T00:11:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab2fe00640 1 -- 172.21.15.83:0/4283575673 learned_addr learned my addr 172.21.15.83:0/4283575673 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab2f400640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab140039f0 0x7fab30157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab26a00640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab30137cf0 0x7fab3013c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab2fe00640 1 -- 172.21.15.83:0/4283575673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab140039f0 msgr2=0x7fab30157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab2fe00640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab140039f0 0x7fab30157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:10.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab2fe00640 1 -- 172.21.15.83:0/4283575673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab30137cf0 msgr2=0x7fab3013c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:10.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab2fe00640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab30137cf0 0x7fab3013c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:10.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab2fe00640 1 -- 172.21.15.83:0/4283575673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab301398f0 con 0x7fab30138170 2024-04-25T00:11:10.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab26a00640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab30137cf0 0x7fab3013c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:10.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab2fe00640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab30138170 0x7fab3013c800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab300d0980 tx=0x7fab28010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:10.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab27e00640 1 -- 172.21.15.83:0/4283575673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2800f040 con 0x7fab30138170 2024-04-25T00:11:10.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab27e00640 1 -- 172.21.15.83:0/4283575673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab2801d070 con 0x7fab30138170 2024-04-25T00:11:10.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab36600640 1 -- 172.21.15.83:0/4283575673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab301f8f20 con 0x7fab30138170 2024-04-25T00:11:10.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab27e00640 1 -- 172.21.15.83:0/4283575673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28015ac0 con 0x7fab30138170 2024-04-25T00:11:10.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab36600640 1 -- 172.21.15.83:0/4283575673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab301f94e0 con 0x7fab30138170 2024-04-25T00:11:10.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.048+0000 7fab36600640 1 -- 172.21.15.83:0/4283575673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab300736f0 con 0x7fab30138170 2024-04-25T00:11:10.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.052+0000 7fab27e00640 1 -- 172.21.15.83:0/4283575673 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fab28015c60 con 0x7fab30138170 2024-04-25T00:11:10.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.052+0000 7fab27e00640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab0c0678c0 0x7fab0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:10.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.052+0000 7fab27e00640 1 -- 172.21.15.83:0/4283575673 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7fab28086b90 con 0x7fab30138170 2024-04-25T00:11:10.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.052+0000 7fab2f400640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab0c0678c0 0x7fab0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:10.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.052+0000 7fab2f400640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab0c0678c0 0x7fab0c069d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fab200ad000 tx=0x7fab200a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:10.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.056+0000 7fab27e00640 1 -- 172.21.15.83:0/4283575673 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fab28057390 con 0x7fab30138170 2024-04-25T00:11:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.212+0000 7fab36600640 1 -- 172.21.15.83:0/4283575673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fab3004f5f0 con 0x7fab30138170 2024-04-25T00:11:10.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.864+0000 7fab27e00640 1 -- 172.21.15.83:0/4283575673 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7fab28056c90 con 0x7fab30138170 2024-04-25T00:11:10.866 INFO:teuthology.orchestra.run.smithi083.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T00:11:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.868+0000 7fab36600640 1 -- 172.21.15.83:0/4283575673 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab0c0678c0 msgr2=0x7fab0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.868+0000 7fab36600640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab0c0678c0 0x7fab0c069d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fab200ad000 tx=0x7fab200a7000 comp rx=0 tx=0).stop 2024-04-25T00:11:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.868+0000 7fab36600640 1 -- 172.21.15.83:0/4283575673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab30138170 msgr2=0x7fab3013c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.868+0000 7fab36600640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab30138170 0x7fab3013c800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab300d0980 tx=0x7fab28010910 comp rx=0 tx=0).stop 2024-04-25T00:11:10.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.868+0000 7fab36600640 1 -- 172.21.15.83:0/4283575673 shutdown_connections 2024-04-25T00:11:10.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.868+0000 7fab36600640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab0c0678c0 0x7fab0c069d80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:10.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.868+0000 7fab36600640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab30138170 0x7fab3013c800 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:10.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.868+0000 7fab36600640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab30137cf0 0x7fab3013c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:10.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.868+0000 7fab36600640 1 --2- 172.21.15.83:0/4283575673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab140039f0 0x7fab30157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:10.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.868+0000 7fab36600640 1 -- 172.21.15.83:0/4283575673 >> 172.21.15.83:0/4283575673 conn(0x7fab3006b3b0 msgr2=0x7fab3006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:10.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.868+0000 7fab36600640 1 -- 172.21.15.83:0/4283575673 shutdown_connections 2024-04-25T00:11:10.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:10.872+0000 7fab36600640 1 -- 172.21.15.83:0/4283575673 wait complete. 2024-04-25T00:11:10.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T00:11:10.932 INFO:tasks.ceph.mds.b.smithi086.stderr:2024-04-25T00:11:10.928+0000 7f776a400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:11:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.052+0000 7f7af5c00640 1 Processor -- start 2024-04-25T00:11:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.052+0000 7f7af5c00640 1 -- start start 2024-04-25T00:11:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.056+0000 7f7af5c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7af0159810 0x7f7af00cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.056+0000 7f7af5c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7af006e950 0x7f7af015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.056+0000 7f7af5c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af00ce000 con 0x7f7af015fe50 2024-04-25T00:11:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.056+0000 7f7af5c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7af006e210 con 0x7f7af0159810 2024-04-25T00:11:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.056+0000 7f7af5c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7af01602a0 con 0x7f7af006e950 2024-04-25T00:11:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.056+0000 7f7aeea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7af0159810 0x7f7af00cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.056+0000 7f7aeea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7af0159810 0x7f7af00cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45266/0 (socket says 172.21.15.83:45266) 2024-04-25T00:11:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.056+0000 7f7aeea00640 1 -- 172.21.15.83:0/2030632080 learned_addr learned my addr 172.21.15.83:0/2030632080 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.056+0000 7f7aee000640 1 --2- 172.21.15.83:0/2030632080 >> v2:172.21.15.86:3300/0 conn(0x7f7af006e950 0x7f7af015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.056+0000 7f7aeea00640 1 -- 172.21.15.83:0/2030632080 >> v2:172.21.15.86:3300/0 conn(0x7f7af006e950 msgr2=0x7f7af015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.056+0000 7f7aeea00640 1 --2- 172.21.15.83:0/2030632080 >> v2:172.21.15.86:3300/0 conn(0x7f7af006e950 0x7f7af015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.056+0000 7f7aeea00640 1 -- 172.21.15.83:0/2030632080 >> v1:172.21.15.86:6789/0 conn(0x7f7af015fe50 legacy=0x7f7af015c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.056+0000 7f7aeea00640 1 -- 172.21.15.83:0/2030632080 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af006eec0 con 0x7f7af0159810 2024-04-25T00:11:11.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7aeea00640 1 --2- 172.21.15.83:0/2030632080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7af0159810 0x7f7af00cc4a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7af00709a0 tx=0x7f7ae001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b7946862057a62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:11.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7aed600640 1 -- 172.21.15.83:0/2030632080 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae00a5730 con 0x7f7af0159810 2024-04-25T00:11:11.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7aed600640 1 -- 172.21.15.83:0/2030632080 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae00a58d0 con 0x7f7af0159810 2024-04-25T00:11:11.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7aed600640 1 -- 172.21.15.83:0/2030632080 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae00a5ca0 con 0x7f7af0159810 2024-04-25T00:11:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 -- 172.21.15.83:0/2030632080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7af0159810 msgr2=0x7f7af00cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 --2- 172.21.15.83:0/2030632080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7af0159810 0x7f7af00cc4a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7af00709a0 tx=0x7f7ae001c960 comp rx=0 tx=0).stop 2024-04-25T00:11:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 -- 172.21.15.83:0/2030632080 shutdown_connections 2024-04-25T00:11:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 --2- 172.21.15.83:0/2030632080 >> v2:172.21.15.86:3300/0 conn(0x7f7af006e950 0x7f7af015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 --2- 172.21.15.83:0/2030632080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7af0159810 0x7f7af00cc4a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 -- 172.21.15.83:0/2030632080 >> 172.21.15.83:0/2030632080 conn(0x7f7af00b1830 msgr2=0x7f7af00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 -- 172.21.15.83:0/2030632080 shutdown_connections 2024-04-25T00:11:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 -- 172.21.15.83:0/2030632080 wait complete. 2024-04-25T00:11:11.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 Processor -- start 2024-04-25T00:11:11.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 -- start start 2024-04-25T00:11:11.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7af006e950 0x7f7af01fb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:11.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7af0159810 0x7f7af0071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:11.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7af015fe50 0x7f7af0159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7af0159d70 con 0x7f7af0159810 2024-04-25T00:11:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7af0159bf0 con 0x7f7af006e950 2024-04-25T00:11:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7af5c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7af0070da0 con 0x7f7af015fe50 2024-04-25T00:11:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7aee000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7af0159810 0x7f7af0071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7aee000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7af0159810 0x7f7af0071360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45280/0 (socket says 172.21.15.83:45280) 2024-04-25T00:11:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.060+0000 7f7aee000640 1 -- 172.21.15.83:0/3771651190 learned_addr learned my addr 172.21.15.83:0/3771651190 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7aeea00640 1 --2- 172.21.15.83:0/3771651190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7af006e950 0x7f7af01fb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7aef400640 1 --2- 172.21.15.83:0/3771651190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7af015fe50 0x7f7af0159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7aee000640 1 -- 172.21.15.83:0/3771651190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7af015fe50 msgr2=0x7f7af0159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7aee000640 1 --2- 172.21.15.83:0/3771651190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7af015fe50 0x7f7af0159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7aee000640 1 -- 172.21.15.83:0/3771651190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7af006e950 msgr2=0x7f7af01fb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7aee000640 1 --2- 172.21.15.83:0/3771651190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7af006e950 0x7f7af01fb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7aee000640 1 -- 172.21.15.83:0/3771651190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af015bdb0 con 0x7f7af0159810 2024-04-25T00:11:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7aee000640 1 --2- 172.21.15.83:0/3771651190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7af0159810 0x7f7af0071360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00e9b0 tx=0x7f7adc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7ae6a00640 1 -- 172.21.15.83:0/3771651190 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc017070 con 0x7f7af0159810 2024-04-25T00:11:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7ae6a00640 1 -- 172.21.15.83:0/3771651190 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7adc00c9a0 con 0x7f7af0159810 2024-04-25T00:11:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7af5c00640 1 -- 172.21.15.83:0/3771651190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af01fbb70 con 0x7f7af0159810 2024-04-25T00:11:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7ae6a00640 1 -- 172.21.15.83:0/3771651190 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00cd70 con 0x7f7af0159810 2024-04-25T00:11:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7af5c00640 1 -- 172.21.15.83:0/3771651190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af00b4530 con 0x7f7af0159810 2024-04-25T00:11:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7af5c00640 1 -- 172.21.15.83:0/3771651190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af004f510 con 0x7f7af0159810 2024-04-25T00:11:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7ae6a00640 1 -- 172.21.15.83:0/3771651190 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7adc01a070 con 0x7f7af0159810 2024-04-25T00:11:11.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7ae6a00640 1 --2- 172.21.15.83:0/3771651190 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ad00678c0 0x7f7ad0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:11.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7ae6a00640 1 -- 172.21.15.83:0/3771651190 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f7adc0860a0 con 0x7f7af0159810 2024-04-25T00:11:11.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.064+0000 7f7aeea00640 1 --2- 172.21.15.83:0/3771651190 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ad00678c0 0x7f7ad0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:11.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.068+0000 7f7aeea00640 1 --2- 172.21.15.83:0/3771651190 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ad00678c0 0x7f7ad0069d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00ae000 tx=0x7f7ae00a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:11.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.072+0000 7f7ae6a00640 1 -- 172.21.15.83:0/3771651190 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7af01fbb70 con 0x7f7af0159810 2024-04-25T00:11:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.256+0000 7f7af5c00640 1 -- 172.21.15.83:0/3771651190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f7af00cc7c0 con 0x7f7af0159810 2024-04-25T00:11:11.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.864+0000 7f7ae6a00640 1 -- 172.21.15.83:0/3771651190 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f7adc0568b0 con 0x7f7af0159810 2024-04-25T00:11:11.868 INFO:teuthology.orchestra.run.smithi083.stderr:added data pool 4 to fsmap 2024-04-25T00:11:11.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.868+0000 7f7af5c00640 1 -- 172.21.15.83:0/3771651190 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ad00678c0 msgr2=0x7f7ad0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:11.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.868+0000 7f7af5c00640 1 --2- 172.21.15.83:0/3771651190 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ad00678c0 0x7f7ad0069d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00ae000 tx=0x7f7ae00a7070 comp rx=0 tx=0).stop 2024-04-25T00:11:11.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.868+0000 7f7af5c00640 1 -- 172.21.15.83:0/3771651190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7af0159810 msgr2=0x7f7af0071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:11.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.868+0000 7f7af5c00640 1 --2- 172.21.15.83:0/3771651190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7af0159810 0x7f7af0071360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00e9b0 tx=0x7f7adc00ee70 comp rx=0 tx=0).stop 2024-04-25T00:11:11.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.868+0000 7f7af5c00640 1 -- 172.21.15.83:0/3771651190 shutdown_connections 2024-04-25T00:11:11.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.868+0000 7f7af5c00640 1 --2- 172.21.15.83:0/3771651190 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ad00678c0 0x7f7ad0069d80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:11.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.868+0000 7f7af5c00640 1 --2- 172.21.15.83:0/3771651190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7af015fe50 0x7f7af0159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:11.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.868+0000 7f7af5c00640 1 --2- 172.21.15.83:0/3771651190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7af0159810 0x7f7af0071360 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.868+0000 7f7af5c00640 1 --2- 172.21.15.83:0/3771651190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7af006e950 0x7f7af01fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.868+0000 7f7af5c00640 1 -- 172.21.15.83:0/3771651190 >> 172.21.15.83:0/3771651190 conn(0x7f7af00b1830 msgr2=0x7f7af0154ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.872+0000 7f7af5c00640 1 -- 172.21.15.83:0/3771651190 shutdown_connections 2024-04-25T00:11:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:11.872+0000 7f7af5c00640 1 -- 172.21.15.83:0/3771651190 wait complete. 2024-04-25T00:11:11.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:11:12.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.032+0000 7f4e33200640 1 Processor -- start 2024-04-25T00:11:12.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e33200640 1 -- start start 2024-04-25T00:11:12.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e33200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4e2c161470 0x7f4e2c06c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e33200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e2c0cd8b0 0x7f4e2c06ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e33200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e2c0d09c0 con 0x7f4e2c0b1080 2024-04-25T00:11:12.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e33200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4e2c0d1260 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e33200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4e2c0d0ca0 con 0x7f4e2c161470 2024-04-25T00:11:12.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e32800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e2c0cd8b0 0x7f4e2c06ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e31400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4e2c0b1080 0x7f4e2c0ce450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42570/0 (socket says 172.21.15.83:42570) 2024-04-25T00:11:12.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e31400640 1 -- 172.21.15.83:0/1580126607 learned_addr learned my addr 172.21.15.83:0/1580126607 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:12.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e31e00640 1 --2- 172.21.15.83:0/1580126607 >> v2:172.21.15.86:3300/0 conn(0x7f4e2c161470 0x7f4e2c06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e32800640 1 -- 172.21.15.83:0/1580126607 >> v2:172.21.15.86:3300/0 conn(0x7f4e2c161470 msgr2=0x7f4e2c06c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:12.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e32800640 1 --2- 172.21.15.83:0/1580126607 >> v2:172.21.15.86:3300/0 conn(0x7f4e2c161470 0x7f4e2c06c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e32800640 1 -- 172.21.15.83:0/1580126607 >> v1:172.21.15.83:6789/0 conn(0x7f4e2c0b1080 legacy=0x7f4e2c0ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e32800640 1 -- 172.21.15.83:0/1580126607 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c0d0f80 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.036+0000 7f4e30a00640 1 -- 172.21.15.83:0/1580126607 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 154522314 0 0) 0x7f4e2c0d09c0 con 0x7f4e2c0b1080 2024-04-25T00:11:12.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.040+0000 7f4e32800640 1 --2- 172.21.15.83:0/1580126607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c0cd8b0 0x7f4e2c06ce40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c072720 tx=0x7f4e2801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d299bd6d8c6c2c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:12.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.040+0000 7f4e30a00640 1 -- 172.21.15.83:0/1580126607 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e280b2070 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.040+0000 7f4e30a00640 1 -- 172.21.15.83:0/1580126607 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e2c06d740 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.040+0000 7f4e30a00640 1 -- 172.21.15.83:0/1580126607 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e280a5a90 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.040+0000 7f4e33200640 1 -- 172.21.15.83:0/1580126607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c0cd8b0 msgr2=0x7f4e2c06ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.040+0000 7f4e33200640 1 --2- 172.21.15.83:0/1580126607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c0cd8b0 0x7f4e2c06ce40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c072720 tx=0x7f4e2801c8a0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.040+0000 7f4e33200640 1 -- 172.21.15.83:0/1580126607 shutdown_connections 2024-04-25T00:11:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.040+0000 7f4e33200640 1 --2- 172.21.15.83:0/1580126607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c0cd8b0 0x7f4e2c06ce40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.040+0000 7f4e33200640 1 --2- 172.21.15.83:0/1580126607 >> v2:172.21.15.86:3300/0 conn(0x7f4e2c161470 0x7f4e2c06c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.040+0000 7f4e33200640 1 -- 172.21.15.83:0/1580126607 >> 172.21.15.83:0/1580126607 conn(0x7f4e2c0d1bd0 msgr2=0x7f4e2c0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.040+0000 7f4e33200640 1 -- 172.21.15.83:0/1580126607 shutdown_connections 2024-04-25T00:11:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.040+0000 7f4e33200640 1 -- 172.21.15.83:0/1580126607 wait complete. 2024-04-25T00:11:12.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.040+0000 7f4e33200640 1 Processor -- start 2024-04-25T00:11:12.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.040+0000 7f4e33200640 1 -- start start 2024-04-25T00:11:12.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e33200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e2c0b1080 0x7f4e2c155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e33200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c0cd8b0 0x7f4e2c0cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e33200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e2c161470 0x7f4e2c06fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e33200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c16af20 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e33200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c16ada0 con 0x7f4e2c0b1080 2024-04-25T00:11:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e33200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2c16b0a0 con 0x7f4e2c161470 2024-04-25T00:11:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e31400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c0cd8b0 0x7f4e2c0cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e31400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c0cd8b0 0x7f4e2c0cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45296/0 (socket says 172.21.15.83:45296) 2024-04-25T00:11:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e31400640 1 -- 172.21.15.83:0/3033761177 learned_addr learned my addr 172.21.15.83:0/3033761177 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e32800640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e2c161470 0x7f4e2c06fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e31400640 1 -- 172.21.15.83:0/3033761177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e2c161470 msgr2=0x7f4e2c06fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e31e00640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e2c0b1080 0x7f4e2c155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e31400640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e2c161470 0x7f4e2c06fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e31400640 1 -- 172.21.15.83:0/3033761177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e2c0b1080 msgr2=0x7f4e2c155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e31400640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e2c0b1080 0x7f4e2c155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e31400640 1 -- 172.21.15.83:0/3033761177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c1f6ce0 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e32800640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e2c161470 0x7f4e2c06fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e31e00640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e2c0b1080 0x7f4e2c155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e31400640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c0cd8b0 0x7f4e2c0cb310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00c4b0 tx=0x7f4e1c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e26000640 1 -- 172.21.15.83:0/3033761177 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c018070 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e26000640 1 -- 172.21.15.83:0/3033761177 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e1c007500 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e33200640 1 -- 172.21.15.83:0/3033761177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2c1f6fc0 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e26000640 1 -- 172.21.15.83:0/3033761177 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c004a30 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e33200640 1 -- 172.21.15.83:0/3033761177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2c1f7580 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e33200640 1 -- 172.21.15.83:0/3033761177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e2c04f510 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.044+0000 7f4e26000640 1 -- 172.21.15.83:0/3033761177 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4e1c01b070 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.048+0000 7f4e26000640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4e100678c0 0x7f4e10069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.048+0000 7f4e26000640 1 -- 172.21.15.83:0/3033761177 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4e1c085300 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.048+0000 7f4e31e00640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4e100678c0 0x7f4e10069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.048+0000 7f4e31e00640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4e100678c0 0x7f4e10069d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e200097a0 tx=0x7f4e20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:12.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.052+0000 7f4e26000640 1 -- 172.21.15.83:0/3033761177 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4e1c055bb0 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.240+0000 7f4e33200640 1 -- 172.21.15.83:0/3033761177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e2c0d00a0 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.240+0000 7f4e26000640 1 -- 172.21.15.83:0/3033761177 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5174 (secure 0 0 0) 0x7f4e1c0554b0 con 0x7f4e2c0cd8b0 2024-04-25T00:11:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:12.242 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/1335208265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1335208265},{"type":"v1","addr":"172.21.15.86:6834","nonce":1335208265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/911805587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":911805587},{"type":"v1","addr":"172.21.15.83:6836","nonce":911805587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2309157687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2309157687},{"type":"v1","addr":"172.21.15.83:6837","nonce":2309157687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:11:06.200989+0000","modified":"2024-04-25T00:11:11.262751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1603954817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":1603954817},{"type":"v1","addr":"172.21.15.86:6835","nonce":1603954817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T00:11:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-04-25T00:11:12.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.240+0000 7f4e33200640 1 -- 172.21.15.83:0/3033761177 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4e100678c0 msgr2=0x7f4e10069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.240+0000 7f4e33200640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4e100678c0 0x7f4e10069d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e200097a0 tx=0x7f4e20008040 comp rx=0 tx=0).stop 2024-04-25T00:11:12.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.240+0000 7f4e33200640 1 -- 172.21.15.83:0/3033761177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c0cd8b0 msgr2=0x7f4e2c0cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.240+0000 7f4e33200640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c0cd8b0 0x7f4e2c0cb310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00c4b0 tx=0x7f4e1c00c970 comp rx=0 tx=0).stop 2024-04-25T00:11:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.240+0000 7f4e33200640 1 -- 172.21.15.83:0/3033761177 shutdown_connections 2024-04-25T00:11:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.240+0000 7f4e33200640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4e100678c0 0x7f4e10069d80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.240+0000 7f4e33200640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e2c161470 0x7f4e2c06fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.240+0000 7f4e33200640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c0cd8b0 0x7f4e2c0cb310 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.240+0000 7f4e33200640 1 --2- 172.21.15.83:0/3033761177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e2c0b1080 0x7f4e2c155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.240+0000 7f4e33200640 1 -- 172.21.15.83:0/3033761177 >> 172.21.15.83:0/3033761177 conn(0x7f4e2c0d1bd0 msgr2=0x7f4e2c0af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.240+0000 7f4e33200640 1 -- 172.21.15.83:0/3033761177 shutdown_connections 2024-04-25T00:11:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.240+0000 7f4e33200640 1 -- 172.21.15.83:0/3033761177 wait complete. 2024-04-25T00:11:12.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:12.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f53a00640 1 Processor -- start 2024-04-25T00:11:12.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f53a00640 1 -- start start 2024-04-25T00:11:12.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f53a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f4c0cf810 0x7f0f4c144130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f53a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f4c15e5b0 0x7f0f4c06e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f53a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0f4c06eb90 0x7f0f4c162040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f53a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0f4c141400 con 0x7f0f4c0cf810 2024-04-25T00:11:12.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f53a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0f4c144670 con 0x7f0f4c06eb90 2024-04-25T00:11:12.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f53a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4c15eb70 con 0x7f0f4c15e5b0 2024-04-25T00:11:12.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f51000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f4c0cf810 0x7f0f4c144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f51000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f4c0cf810 0x7f0f4c144130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45312/0 (socket says 172.21.15.83:45312) 2024-04-25T00:11:12.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f51000640 1 -- 172.21.15.83:0/2068931960 learned_addr learned my addr 172.21.15.83:0/2068931960 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:12.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f51000640 1 -- 172.21.15.83:0/2068931960 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f4c15e5b0 msgr2=0x7f0f4c06e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:12.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f51000640 1 --2- 172.21.15.83:0/2068931960 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f4c15e5b0 0x7f0f4c06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f51000640 1 -- 172.21.15.83:0/2068931960 >> v2:172.21.15.86:3300/0 conn(0x7f0f4c06eb90 msgr2=0x7f0f4c162040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:12.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f51a00640 1 --2- 172.21.15.83:0/2068931960 >> v2:172.21.15.86:3300/0 conn(0x7f0f4c06eb90 0x7f0f4c162040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f51000640 1 --2- 172.21.15.83:0/2068931960 >> v2:172.21.15.86:3300/0 conn(0x7f0f4c06eb90 0x7f0f4c162040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f51000640 1 -- 172.21.15.83:0/2068931960 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c15ee50 con 0x7f0f4c0cf810 2024-04-25T00:11:12.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.396+0000 7f0f51a00640 1 --2- 172.21.15.83:0/2068931960 >> v2:172.21.15.86:3300/0 conn(0x7f0f4c06eb90 0x7f0f4c162040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:12.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.400+0000 7f0f51000640 1 --2- 172.21.15.83:0/2068931960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4c0cf810 0x7f0f4c144130 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c158c40 tx=0x7f0f4001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ce81578fe37d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:12.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.400+0000 7f0f4b400640 1 -- 172.21.15.83:0/2068931960 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f400b0070 con 0x7f0f4c0cf810 2024-04-25T00:11:12.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.400+0000 7f0f4b400640 1 -- 172.21.15.83:0/2068931960 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f400ab600 con 0x7f0f4c0cf810 2024-04-25T00:11:12.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.400+0000 7f0f4b400640 1 -- 172.21.15.83:0/2068931960 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f400ab9d0 con 0x7f0f4c0cf810 2024-04-25T00:11:12.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.400+0000 7f0f53a00640 1 -- 172.21.15.83:0/2068931960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4c0cf810 msgr2=0x7f0f4c144130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.400+0000 7f0f53a00640 1 --2- 172.21.15.83:0/2068931960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4c0cf810 0x7f0f4c144130 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c158c40 tx=0x7f0f4001c520 comp rx=0 tx=0).stop 2024-04-25T00:11:12.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.400+0000 7f0f53a00640 1 -- 172.21.15.83:0/2068931960 shutdown_connections 2024-04-25T00:11:12.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.400+0000 7f0f53a00640 1 --2- 172.21.15.83:0/2068931960 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f4c15e5b0 0x7f0f4c06e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.400+0000 7f0f53a00640 1 --2- 172.21.15.83:0/2068931960 >> v2:172.21.15.86:3300/0 conn(0x7f0f4c06eb90 0x7f0f4c162040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.400+0000 7f0f53a00640 1 --2- 172.21.15.83:0/2068931960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4c0cf810 0x7f0f4c144130 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.400+0000 7f0f53a00640 1 -- 172.21.15.83:0/2068931960 >> 172.21.15.83:0/2068931960 conn(0x7f0f4c0b1d50 msgr2=0x7f0f4c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:12.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.400+0000 7f0f53a00640 1 -- 172.21.15.83:0/2068931960 shutdown_connections 2024-04-25T00:11:12.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.400+0000 7f0f53a00640 1 -- 172.21.15.83:0/2068931960 wait complete. 2024-04-25T00:11:12.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f53a00640 1 Processor -- start 2024-04-25T00:11:12.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f53a00640 1 -- start start 2024-04-25T00:11:12.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f53a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f4c06eb90 0x7f0f4c143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f53a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4c0cf810 0x7f0f4c140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f53a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f4c15e5b0 0x7f0f4c140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f53a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c162990 con 0x7f0f4c0cf810 2024-04-25T00:11:12.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f53a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c0cfe00 con 0x7f0f4c06eb90 2024-04-25T00:11:12.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f53a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4c162b10 con 0x7f0f4c15e5b0 2024-04-25T00:11:12.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f4be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4c0cf810 0x7f0f4c140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f4be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4c0cf810 0x7f0f4c140210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45318/0 (socket says 172.21.15.83:45318) 2024-04-25T00:11:12.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f4be00640 1 -- 172.21.15.83:0/1130309059 learned_addr learned my addr 172.21.15.83:0/1130309059 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:12.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f51000640 1 --2- 172.21.15.83:0/1130309059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f4c06eb90 0x7f0f4c143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f51a00640 1 --2- 172.21.15.83:0/1130309059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f4c15e5b0 0x7f0f4c140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f4be00640 1 -- 172.21.15.83:0/1130309059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f4c15e5b0 msgr2=0x7f0f4c140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f4be00640 1 --2- 172.21.15.83:0/1130309059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f4c15e5b0 0x7f0f4c140750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f4be00640 1 -- 172.21.15.83:0/1130309059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f4c06eb90 msgr2=0x7f0f4c143250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f4be00640 1 --2- 172.21.15.83:0/1130309059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f4c06eb90 0x7f0f4c143250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f4be00640 1 -- 172.21.15.83:0/1130309059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c15afa0 con 0x7f0f4c0cf810 2024-04-25T00:11:12.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f4be00640 1 --2- 172.21.15.83:0/1130309059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4c0cf810 0x7f0f4c140210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800b410 tx=0x7f0f3800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:12.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f48c00640 1 -- 172.21.15.83:0/1130309059 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38015020 con 0x7f0f4c0cf810 2024-04-25T00:11:12.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f48c00640 1 -- 172.21.15.83:0/1130309059 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f38011830 con 0x7f0f4c0cf810 2024-04-25T00:11:12.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f48c00640 1 -- 172.21.15.83:0/1130309059 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38011c20 con 0x7f0f4c0cf810 2024-04-25T00:11:12.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f53a00640 1 -- 172.21.15.83:0/1130309059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4c15b2c0 con 0x7f0f4c0cf810 2024-04-25T00:11:12.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f53a00640 1 -- 172.21.15.83:0/1130309059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f4c15bb00 con 0x7f0f4c0cf810 2024-04-25T00:11:12.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.404+0000 7f0f53a00640 1 -- 172.21.15.83:0/1130309059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f4c06e440 con 0x7f0f4c0cf810 2024-04-25T00:11:12.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.408+0000 7f0f48c00640 1 -- 172.21.15.83:0/1130309059 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0f38018020 con 0x7f0f4c0cf810 2024-04-25T00:11:12.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.408+0000 7f0f48c00640 1 --2- 172.21.15.83:0/1130309059 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0f2c0678c0 0x7f0f2c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.408+0000 7f0f48c00640 1 -- 172.21.15.83:0/1130309059 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0f38085d80 con 0x7f0f4c0cf810 2024-04-25T00:11:12.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.408+0000 7f0f51000640 1 --2- 172.21.15.83:0/1130309059 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0f2c0678c0 0x7f0f2c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.408+0000 7f0f51000640 1 --2- 172.21.15.83:0/1130309059 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0f2c0678c0 0x7f0f2c069d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f400ad000 tx=0x7f0f400a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:12.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.412+0000 7f0f48c00640 1 -- 172.21.15.83:0/1130309059 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0f3801f020 con 0x7f0f4c0cf810 2024-04-25T00:11:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.572+0000 7f0f53a00640 1 -- 172.21.15.83:0/1130309059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f4c0b29f0 con 0x7f0f4c0cf810 2024-04-25T00:11:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.576+0000 7f0f48c00640 1 -- 172.21.15.83:0/1130309059 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7f0f4c15b2c0 con 0x7f0f4c0cf810 2024-04-25T00:11:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:12.577 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:12.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.576+0000 7f0f53a00640 1 -- 172.21.15.83:0/1130309059 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0f2c0678c0 msgr2=0x7f0f2c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.576+0000 7f0f53a00640 1 --2- 172.21.15.83:0/1130309059 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0f2c0678c0 0x7f0f2c069d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f400ad000 tx=0x7f0f400a6000 comp rx=0 tx=0).stop 2024-04-25T00:11:12.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.576+0000 7f0f53a00640 1 -- 172.21.15.83:0/1130309059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4c0cf810 msgr2=0x7f0f4c140210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.576+0000 7f0f53a00640 1 --2- 172.21.15.83:0/1130309059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4c0cf810 0x7f0f4c140210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800b410 tx=0x7f0f3800b8d0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.576+0000 7f0f53a00640 1 -- 172.21.15.83:0/1130309059 shutdown_connections 2024-04-25T00:11:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.576+0000 7f0f53a00640 1 --2- 172.21.15.83:0/1130309059 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0f2c0678c0 0x7f0f2c069d80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.576+0000 7f0f53a00640 1 --2- 172.21.15.83:0/1130309059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f4c15e5b0 0x7f0f4c140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.576+0000 7f0f53a00640 1 --2- 172.21.15.83:0/1130309059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4c0cf810 0x7f0f4c140210 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.576+0000 7f0f53a00640 1 --2- 172.21.15.83:0/1130309059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f4c06eb90 0x7f0f4c143250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.576+0000 7f0f53a00640 1 -- 172.21.15.83:0/1130309059 >> 172.21.15.83:0/1130309059 conn(0x7f0f4c0b1d50 msgr2=0x7f0f4c0b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.576+0000 7f0f53a00640 1 -- 172.21.15.83:0/1130309059 shutdown_connections 2024-04-25T00:11:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.576+0000 7f0f53a00640 1 -- 172.21.15.83:0/1130309059 wait complete. 2024-04-25T00:11:12.599 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:12.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa870400640 1 Processor -- start 2024-04-25T00:11:12.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa870400640 1 -- start start 2024-04-25T00:11:12.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa870400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa8681599f0 0x7fa8680cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa870400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8680ccc60 con 0x7fa86815cec0 2024-04-25T00:11:12.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa870400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa86806dba0 con 0x7fa86815c340 2024-04-25T00:11:12.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa870400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa868159dd0 con 0x7fa8681599f0 2024-04-25T00:11:12.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa86e400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa86815cec0 0x7fa8680ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42586/0 (socket says 172.21.15.83:42586) 2024-04-25T00:11:12.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa86e400640 1 -- 172.21.15.83:0/2382217263 learned_addr learned my addr 172.21.15.83:0/2382217263 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:12.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 -- 172.21.15.83:0/2382217263 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4074477155 0 0) 0x7fa8600014a0 con 0x7fa86815cec0 2024-04-25T00:11:12.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 -- 172.21.15.83:0/2382217263 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa850003530 con 0x7fa86815cec0 2024-04-25T00:11:12.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa86da00640 1 --2- 172.21.15.83:0/2382217263 >> v2:172.21.15.86:3300/0 conn(0x7fa8681599f0 0x7fa8680cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 -- 172.21.15.83:0/2382217263 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1088782131 0 0) 0x7fa850003530 con 0x7fa86815cec0 2024-04-25T00:11:12.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 -- 172.21.15.83:0/2382217263 >> v2:172.21.15.86:3300/0 conn(0x7fa8681599f0 msgr2=0x7fa8680cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:12.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 --2- 172.21.15.83:0/2382217263 >> v2:172.21.15.86:3300/0 conn(0x7fa8681599f0 0x7fa8680cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 -- 172.21.15.83:0/2382217263 >> v1:172.21.15.86:6789/0 conn(0x7fa86815c340 legacy=0x7fa86815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 -- 172.21.15.83:0/2382217263 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86806de80 con 0x7fa86815cec0 2024-04-25T00:11:12.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 -- 172.21.15.83:0/2382217263 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fa860001bd0 con 0x7fa86815cec0 2024-04-25T00:11:12.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 -- 172.21.15.83:0/2382217263 >> v1:172.21.15.83:6789/0 conn(0x7fa86815cec0 legacy=0x7fa8680ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 --2- 172.21.15.83:0/2382217263 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8500039c0 0x7fa850003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 --2- 172.21.15.83:0/2382217263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa850004810 0x7fa850004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 --2- 172.21.15.83:0/2382217263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8500052f0 0x7fa85000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 -- 172.21.15.83:0/2382217263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa85000c190 con 0x7fa850004810 2024-04-25T00:11:12.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 -- 172.21.15.83:0/2382217263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa85000c340 con 0x7fa8500052f0 2024-04-25T00:11:12.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.732+0000 7fa867e00640 1 -- 172.21.15.83:0/2382217263 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa85000c540 con 0x7fa8500039c0 2024-04-25T00:11:12.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa86d000640 1 --2- 172.21.15.83:0/2382217263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa850004810 0x7fa850004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa86da00640 1 --2- 172.21.15.83:0/2382217263 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8500039c0 0x7fa850003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa86e400640 1 --2- 172.21.15.83:0/2382217263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8500052f0 0x7fa85000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa86d000640 1 -- 172.21.15.83:0/2382217263 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8500039c0 msgr2=0x7fa850003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:12.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa86d000640 1 --2- 172.21.15.83:0/2382217263 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8500039c0 0x7fa850003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa86d000640 1 -- 172.21.15.83:0/2382217263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8500052f0 msgr2=0x7fa85000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:12.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa86d000640 1 --2- 172.21.15.83:0/2382217263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8500052f0 0x7fa85000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa86d000640 1 -- 172.21.15.83:0/2382217263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa850003530 con 0x7fa850004810 2024-04-25T00:11:12.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa86e400640 1 -- 172.21.15.83:0/2382217263 reap_dead start 2024-04-25T00:11:12.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa86d000640 1 --2- 172.21.15.83:0/2382217263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa850004810 0x7fa850004c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa854001750 tx=0x7fa85401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329238e6f1635ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:12.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa867e00640 1 -- 172.21.15.83:0/2382217263 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8540013a0 con 0x7fa850004810 2024-04-25T00:11:12.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa867e00640 1 -- 172.21.15.83:0/2382217263 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8540052f0 con 0x7fa850004810 2024-04-25T00:11:12.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa867e00640 1 -- 172.21.15.83:0/2382217263 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa854003840 con 0x7fa850004810 2024-04-25T00:11:12.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa870400640 1 -- 172.21.15.83:0/2382217263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa850004810 msgr2=0x7fa850004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.736+0000 7fa870400640 1 --2- 172.21.15.83:0/2382217263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa850004810 0x7fa850004c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa854001750 tx=0x7fa85401c390 comp rx=0 tx=0).stop 2024-04-25T00:11:12.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa870400640 1 -- 172.21.15.83:0/2382217263 shutdown_connections 2024-04-25T00:11:12.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa870400640 1 --2- 172.21.15.83:0/2382217263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa850004810 0x7fa850004c90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa870400640 1 -- 172.21.15.83:0/2382217263 >> 172.21.15.83:0/2382217263 conn(0x7fa8680b1830 msgr2=0x7fa8680b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:12.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa870400640 1 -- 172.21.15.83:0/2382217263 shutdown_connections 2024-04-25T00:11:12.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa870400640 1 -- 172.21.15.83:0/2382217263 wait complete. 2024-04-25T00:11:12.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa870400640 1 Processor -- start 2024-04-25T00:11:12.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa870400640 1 -- start start 2024-04-25T00:11:12.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa870400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa850004810 0x7fa868154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa870400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8681554a0 0x7fa868156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa870400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b3b20 0x7fa868156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa870400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa86815cce0 con 0x7fa8680b3b20 2024-04-25T00:11:12.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa870400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa86815cb60 con 0x7fa8681554a0 2024-04-25T00:11:12.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa870400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa868160140 con 0x7fa850004810 2024-04-25T00:11:12.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa86e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b3b20 0x7fa868156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa86e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b3b20 0x7fa868156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45330/0 (socket says 172.21.15.83:45330) 2024-04-25T00:11:12.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa86e400640 1 -- 172.21.15.83:0/506863400 learned_addr learned my addr 172.21.15.83:0/506863400 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:12.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa86da00640 1 --2- 172.21.15.83:0/506863400 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa850004810 0x7fa868154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa86d000640 1 --2- 172.21.15.83:0/506863400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8681554a0 0x7fa868156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa86e400640 1 -- 172.21.15.83:0/506863400 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa850004810 msgr2=0x7fa868154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa86e400640 1 --2- 172.21.15.83:0/506863400 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa850004810 0x7fa868154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa86e400640 1 -- 172.21.15.83:0/506863400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8681554a0 msgr2=0x7fa868156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa86e400640 1 --2- 172.21.15.83:0/506863400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8681554a0 0x7fa868156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.740+0000 7fa86e400640 1 -- 172.21.15.83:0/506863400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8680b2a80 con 0x7fa8680b3b20 2024-04-25T00:11:12.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.744+0000 7fa86e400640 1 --2- 172.21.15.83:0/506863400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b3b20 0x7fa868156840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa86815d2a0 tx=0x7fa86000ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:12.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.744+0000 7fa865600640 1 -- 172.21.15.83:0/506863400 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860007840 con 0x7fa8680b3b20 2024-04-25T00:11:12.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.744+0000 7fa865600640 1 -- 172.21.15.83:0/506863400 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa860019070 con 0x7fa8680b3b20 2024-04-25T00:11:12.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.744+0000 7fa870400640 1 -- 172.21.15.83:0/506863400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8680b2d60 con 0x7fa8680b3b20 2024-04-25T00:11:12.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.744+0000 7fa865600640 1 -- 172.21.15.83:0/506863400 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8600133f0 con 0x7fa8680b3b20 2024-04-25T00:11:12.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.744+0000 7fa870400640 1 -- 172.21.15.83:0/506863400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8680b3320 con 0x7fa8680b3b20 2024-04-25T00:11:12.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.744+0000 7fa870400640 1 -- 172.21.15.83:0/506863400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8680737f0 con 0x7fa8680b3b20 2024-04-25T00:11:12.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.744+0000 7fa865600640 1 -- 172.21.15.83:0/506863400 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa860013590 con 0x7fa8680b3b20 2024-04-25T00:11:12.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.744+0000 7fa865600640 1 --2- 172.21.15.83:0/506863400 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8480678c0 0x7fa848069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:12.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.744+0000 7fa865600640 1 -- 172.21.15.83:0/506863400 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa8600863f0 con 0x7fa8680b3b20 2024-04-25T00:11:12.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.744+0000 7fa86da00640 1 --2- 172.21.15.83:0/506863400 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8480678c0 0x7fa848069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:12.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.744+0000 7fa86da00640 1 --2- 172.21.15.83:0/506863400 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8480678c0 0x7fa848069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa85c003730 tx=0x7fa85c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:12.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.748+0000 7fa865600640 1 -- 172.21.15.83:0/506863400 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa8680ccc60 con 0x7fa8680b3b20 2024-04-25T00:11:12.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.908+0000 7fa870400640 1 -- 172.21.15.83:0/506863400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa86804f5f0 con 0x7fa8680b3b20 2024-04-25T00:11:12.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.912+0000 7fa865600640 1 -- 172.21.15.83:0/506863400 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7fa860056590 con 0x7fa8680b3b20 2024-04-25T00:11:12.914 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:12.914 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:12.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.912+0000 7fa870400640 1 -- 172.21.15.83:0/506863400 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8480678c0 msgr2=0x7fa848069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.912+0000 7fa870400640 1 --2- 172.21.15.83:0/506863400 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8480678c0 0x7fa848069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa85c003730 tx=0x7fa85c008040 comp rx=0 tx=0).stop 2024-04-25T00:11:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.912+0000 7fa870400640 1 -- 172.21.15.83:0/506863400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b3b20 msgr2=0x7fa868156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.912+0000 7fa870400640 1 --2- 172.21.15.83:0/506863400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b3b20 0x7fa868156840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa86815d2a0 tx=0x7fa86000ae80 comp rx=0 tx=0).stop 2024-04-25T00:11:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.912+0000 7fa870400640 1 -- 172.21.15.83:0/506863400 shutdown_connections 2024-04-25T00:11:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.912+0000 7fa870400640 1 --2- 172.21.15.83:0/506863400 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8480678c0 0x7fa848069d80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.912+0000 7fa870400640 1 --2- 172.21.15.83:0/506863400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8680b3b20 0x7fa868156840 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.912+0000 7fa870400640 1 --2- 172.21.15.83:0/506863400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8681554a0 0x7fa868156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.912+0000 7fa870400640 1 --2- 172.21.15.83:0/506863400 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa850004810 0x7fa868154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.912+0000 7fa870400640 1 -- 172.21.15.83:0/506863400 >> 172.21.15.83:0/506863400 conn(0x7fa8680b1830 msgr2=0x7fa8680b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.912+0000 7fa870400640 1 -- 172.21.15.83:0/506863400 shutdown_connections 2024-04-25T00:11:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:12.912+0000 7fa870400640 1 -- 172.21.15.83:0/506863400 wait complete. 2024-04-25T00:11:12.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:11:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.084+0000 7fecb0200640 1 Processor -- start 2024-04-25T00:11:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.084+0000 7fecb0200640 1 -- start start 2024-04-25T00:11:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.084+0000 7fecb0200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feca815df40 0x7feca80b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.084+0000 7fecb0200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feca806fa50 0x7feca80cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.084+0000 7fecb0200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feca806fee0 con 0x7feca80cc820 2024-04-25T00:11:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.084+0000 7fecb0200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7feca80704a0 con 0x7feca806fa50 2024-04-25T00:11:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.084+0000 7fecb0200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7feca80701c0 con 0x7feca815df40 2024-04-25T00:11:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.084+0000 7fecace00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feca806fa50 0x7feca80cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.084+0000 7fecace00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feca806fa50 0x7feca80cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45334/0 (socket says 172.21.15.83:45334) 2024-04-25T00:11:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.084+0000 7fecace00640 1 -- 172.21.15.83:0/90992275 learned_addr learned my addr 172.21.15.83:0/90992275 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecace00640 1 -- 172.21.15.83:0/90992275 >> v2:172.21.15.86:3300/0 conn(0x7feca815df40 msgr2=0x7feca80b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecad800640 1 --2- 172.21.15.83:0/90992275 >> v2:172.21.15.86:3300/0 conn(0x7feca815df40 0x7feca80b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecace00640 1 --2- 172.21.15.83:0/90992275 >> v2:172.21.15.86:3300/0 conn(0x7feca815df40 0x7feca80b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecace00640 1 -- 172.21.15.83:0/90992275 >> v1:172.21.15.86:6789/0 conn(0x7feca80cc820 legacy=0x7feca80ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecace00640 1 -- 172.21.15.83:0/90992275 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca80d00a0 con 0x7feca806fa50 2024-04-25T00:11:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecad800640 1 --2- 172.21.15.83:0/90992275 >> v2:172.21.15.86:3300/0 conn(0x7feca815df40 0x7feca80b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecace00640 1 --2- 172.21.15.83:0/90992275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca806fa50 0x7feca80cc2e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feca8071f00 tx=0x7fec9401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967e493f1ffecf1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:13.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7feca7e00640 1 -- 172.21.15.83:0/90992275 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca80b15b0 con 0x7feca806fa50 2024-04-25T00:11:13.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7feca7e00640 1 -- 172.21.15.83:0/90992275 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec940af070 con 0x7feca806fa50 2024-04-25T00:11:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7feca7e00640 1 -- 172.21.15.83:0/90992275 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec940aa880 con 0x7feca806fa50 2024-04-25T00:11:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecb0200640 1 -- 172.21.15.83:0/90992275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca806fa50 msgr2=0x7feca80cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecb0200640 1 --2- 172.21.15.83:0/90992275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca806fa50 0x7feca80cc2e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feca8071f00 tx=0x7fec9401c4e0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecb0200640 1 -- 172.21.15.83:0/90992275 shutdown_connections 2024-04-25T00:11:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecb0200640 1 --2- 172.21.15.83:0/90992275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca806fa50 0x7feca80cc2e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecb0200640 1 --2- 172.21.15.83:0/90992275 >> v2:172.21.15.86:3300/0 conn(0x7feca815df40 0x7feca80b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecb0200640 1 -- 172.21.15.83:0/90992275 >> 172.21.15.83:0/90992275 conn(0x7feca80b37e0 msgr2=0x7feca80b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecb0200640 1 -- 172.21.15.83:0/90992275 shutdown_connections 2024-04-25T00:11:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.088+0000 7fecb0200640 1 -- 172.21.15.83:0/90992275 wait complete. 2024-04-25T00:11:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecb0200640 1 Processor -- start 2024-04-25T00:11:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecb0200640 1 -- start start 2024-04-25T00:11:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecb0200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feca80cc820 0x7feca80c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecb0200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feca815df40 0x7feca80c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecb0200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca80c9e30 0x7feca80b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecb0200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feca80cec20 con 0x7feca80c9e30 2024-04-25T00:11:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecb0200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feca80d12c0 con 0x7feca815df40 2024-04-25T00:11:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecb0200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7feca80ceda0 con 0x7feca80cc820 2024-04-25T00:11:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecae200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca80c9e30 0x7feca80b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecae200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca80c9e30 0x7feca80b6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45350/0 (socket says 172.21.15.83:45350) 2024-04-25T00:11:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecae200640 1 -- 172.21.15.83:0/3629590020 learned_addr learned my addr 172.21.15.83:0/3629590020 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecace00640 1 --2- 172.21.15.83:0/3629590020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feca815df40 0x7feca80c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecad800640 1 --2- 172.21.15.83:0/3629590020 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feca80cc820 0x7feca80c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecae200640 1 -- 172.21.15.83:0/3629590020 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feca80cc820 msgr2=0x7feca80c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecae200640 1 --2- 172.21.15.83:0/3629590020 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feca80cc820 0x7feca80c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecae200640 1 -- 172.21.15.83:0/3629590020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feca815df40 msgr2=0x7feca80c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecae200640 1 --2- 172.21.15.83:0/3629590020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feca815df40 0x7feca80c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecae200640 1 -- 172.21.15.83:0/3629590020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca80cbc80 con 0x7feca80c9e30 2024-04-25T00:11:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecae200640 1 --2- 172.21.15.83:0/3629590020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca80c9e30 0x7feca80b6230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feca000c7b0 tx=0x7feca000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7feca5600640 1 -- 172.21.15.83:0/3629590020 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0017070 con 0x7feca80c9e30 2024-04-25T00:11:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7feca5600640 1 -- 172.21.15.83:0/3629590020 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca0003db0 con 0x7feca80c9e30 2024-04-25T00:11:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7feca5600640 1 -- 172.21.15.83:0/3629590020 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca00052b0 con 0x7feca80c9e30 2024-04-25T00:11:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.092+0000 7fecb0200640 1 -- 172.21.15.83:0/3629590020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca80cb3e0 con 0x7feca80c9e30 2024-04-25T00:11:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.096+0000 7fecb0200640 1 -- 172.21.15.83:0/3629590020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca80cb9a0 con 0x7feca80c9e30 2024-04-25T00:11:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.096+0000 7fecb0200640 1 -- 172.21.15.83:0/3629590020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca809fe40 con 0x7feca80c9e30 2024-04-25T00:11:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.096+0000 7feca5600640 1 -- 172.21.15.83:0/3629590020 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feca001a070 con 0x7feca80c9e30 2024-04-25T00:11:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.096+0000 7feca5600640 1 --2- 172.21.15.83:0/3629590020 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec880678c0 0x7fec88069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.096+0000 7feca5600640 1 -- 172.21.15.83:0/3629590020 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7feca0085570 con 0x7feca80c9e30 2024-04-25T00:11:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.096+0000 7fecad800640 1 --2- 172.21.15.83:0/3629590020 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec880678c0 0x7fec88069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.096+0000 7fecad800640 1 --2- 172.21.15.83:0/3629590020 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec880678c0 0x7fec88069d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fec9c006fd0 tx=0x7fec9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:13.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.100+0000 7feca5600640 1 -- 172.21.15.83:0/3629590020 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7feca0055710 con 0x7feca80c9e30 2024-04-25T00:11:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.288+0000 7fecb0200640 1 -- 172.21.15.83:0/3629590020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feca8073280 con 0x7feca80c9e30 2024-04-25T00:11:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.288+0000 7feca5600640 1 -- 172.21.15.83:0/3629590020 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5174 (secure 0 0 0) 0x7feca0059a10 con 0x7feca80c9e30 2024-04-25T00:11:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:13.293 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/1335208265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1335208265},{"type":"v1","addr":"172.21.15.86:6834","nonce":1335208265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/911805587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":911805587},{"type":"v1","addr":"172.21.15.83:6836","nonce":911805587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2309157687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2309157687},{"type":"v1","addr":"172.21.15.83:6837","nonce":2309157687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:11:06.200989+0000","modified":"2024-04-25T00:11:11.262751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1603954817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":1603954817},{"type":"v1","addr":"172.21.15.86:6835","nonce":1603954817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T00:11:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-04-25T00:11:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.292+0000 7fecb0200640 1 -- 172.21.15.83:0/3629590020 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec880678c0 msgr2=0x7fec88069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.292+0000 7fecb0200640 1 --2- 172.21.15.83:0/3629590020 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec880678c0 0x7fec88069d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fec9c006fd0 tx=0x7fec9c008040 comp rx=0 tx=0).stop 2024-04-25T00:11:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.292+0000 7fecb0200640 1 -- 172.21.15.83:0/3629590020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca80c9e30 msgr2=0x7feca80b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.292+0000 7fecb0200640 1 --2- 172.21.15.83:0/3629590020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca80c9e30 0x7feca80b6230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feca000c7b0 tx=0x7feca000cc70 comp rx=0 tx=0).stop 2024-04-25T00:11:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.292+0000 7fecb0200640 1 -- 172.21.15.83:0/3629590020 shutdown_connections 2024-04-25T00:11:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.292+0000 7fecb0200640 1 --2- 172.21.15.83:0/3629590020 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec880678c0 0x7fec88069d80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.292+0000 7fecb0200640 1 --2- 172.21.15.83:0/3629590020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca80c9e30 0x7feca80b6230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.292+0000 7fecb0200640 1 --2- 172.21.15.83:0/3629590020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feca815df40 0x7feca80c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.292+0000 7fecb0200640 1 --2- 172.21.15.83:0/3629590020 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feca80cc820 0x7feca80c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.292+0000 7fecb0200640 1 -- 172.21.15.83:0/3629590020 >> 172.21.15.83:0/3629590020 conn(0x7feca80b37e0 msgr2=0x7feca80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:13.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.292+0000 7fecb0200640 1 -- 172.21.15.83:0/3629590020 shutdown_connections 2024-04-25T00:11:13.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.292+0000 7fecb0200640 1 -- 172.21.15.83:0/3629590020 wait complete. 2024-04-25T00:11:13.313 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.444+0000 7f37f1e00640 1 Processor -- start 2024-04-25T00:11:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.444+0000 7f37f1e00640 1 -- start start 2024-04-25T00:11:13.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.444+0000 7f37f1e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f37ec1599f0 0x7f37ec0cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:13.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.444+0000 7f37f1e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec15cec0 0x7f37ec0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:13.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.444+0000 7f37f1e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37ec0ccc60 con 0x7f37ec15c340 2024-04-25T00:11:13.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.444+0000 7f37f1e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f37ec159dd0 con 0x7f37ec1599f0 2024-04-25T00:11:13.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.444+0000 7f37f1e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f37ec06dba0 con 0x7f37ec15cec0 2024-04-25T00:11:13.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.444+0000 7f37eaa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f37ec1599f0 0x7f37ec0cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:13.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.444+0000 7f37eaa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f37ec1599f0 0x7f37ec0cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45364/0 (socket says 172.21.15.83:45364) 2024-04-25T00:11:13.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.444+0000 7f37eaa00640 1 -- 172.21.15.83:0/3295037060 learned_addr learned my addr 172.21.15.83:0/3295037060 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:13.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.448+0000 7f37eb400640 1 --2- 172.21.15.83:0/3295037060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec15cec0 0x7f37ec0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:13.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.448+0000 7f37eaa00640 1 -- 172.21.15.83:0/3295037060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec15cec0 msgr2=0x7f37ec0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:13.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.448+0000 7f37eaa00640 1 --2- 172.21.15.83:0/3295037060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec15cec0 0x7f37ec0ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.448+0000 7f37eaa00640 1 -- 172.21.15.83:0/3295037060 >> v1:172.21.15.86:6789/0 conn(0x7f37ec15c340 legacy=0x7f37ec15c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:13.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.448+0000 7f37eaa00640 1 -- 172.21.15.83:0/3295037060 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec06de80 con 0x7f37ec1599f0 2024-04-25T00:11:13.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.448+0000 7f37eaa00640 1 --2- 172.21.15.83:0/3295037060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec1599f0 0x7f37ec0cc510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0098a0 tx=0x7f37dc01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52940d2fa18c7988 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:13.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.448+0000 7f37e9600640 1 -- 172.21.15.83:0/3295037060 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0b0070 con 0x7f37ec1599f0 2024-04-25T00:11:13.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.448+0000 7f37e9600640 1 -- 172.21.15.83:0/3295037060 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37dc0ab600 con 0x7f37ec1599f0 2024-04-25T00:11:13.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.448+0000 7f37e9600640 1 -- 172.21.15.83:0/3295037060 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0ab9d0 con 0x7f37ec1599f0 2024-04-25T00:11:13.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.448+0000 7f37f1e00640 1 -- 172.21.15.83:0/3295037060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec1599f0 msgr2=0x7f37ec0cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:13.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.448+0000 7f37f1e00640 1 --2- 172.21.15.83:0/3295037060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec1599f0 0x7f37ec0cc510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0098a0 tx=0x7f37dc01c4b0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 -- 172.21.15.83:0/3295037060 shutdown_connections 2024-04-25T00:11:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 --2- 172.21.15.83:0/3295037060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec15cec0 0x7f37ec0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 --2- 172.21.15.83:0/3295037060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec1599f0 0x7f37ec0cc510 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 -- 172.21.15.83:0/3295037060 >> 172.21.15.83:0/3295037060 conn(0x7f37ec0b1830 msgr2=0x7f37ec0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 -- 172.21.15.83:0/3295037060 shutdown_connections 2024-04-25T00:11:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 -- 172.21.15.83:0/3295037060 wait complete. 2024-04-25T00:11:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 Processor -- start 2024-04-25T00:11:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 -- start start 2024-04-25T00:11:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec1599f0 0x7f37ec0b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec15c340 0x7f37ec15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec15cec0 0x7f37ec15e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec0cf7c0 con 0x7f37ec15c340 2024-04-25T00:11:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec0cf640 con 0x7f37ec1599f0 2024-04-25T00:11:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f37ec168cd0 con 0x7f37ec15cec0 2024-04-25T00:11:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37ea000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec15c340 0x7f37ec15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37ea000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec15c340 0x7f37ec15ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45372/0 (socket says 172.21.15.83:45372) 2024-04-25T00:11:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37ea000640 1 -- 172.21.15.83:0/2360211372 learned_addr learned my addr 172.21.15.83:0/2360211372 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37eaa00640 1 --2- 172.21.15.83:0/2360211372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec1599f0 0x7f37ec0b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37eb400640 1 --2- 172.21.15.83:0/2360211372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec15cec0 0x7f37ec15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37ea000640 1 -- 172.21.15.83:0/2360211372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec15cec0 msgr2=0x7f37ec15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37ea000640 1 --2- 172.21.15.83:0/2360211372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec15cec0 0x7f37ec15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37ea000640 1 -- 172.21.15.83:0/2360211372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec1599f0 msgr2=0x7f37ec0b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37ea000640 1 --2- 172.21.15.83:0/2360211372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec1599f0 0x7f37ec0b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37ea000640 1 -- 172.21.15.83:0/2360211372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec152ac0 con 0x7f37ec15c340 2024-04-25T00:11:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37ea000640 1 --2- 172.21.15.83:0/2360211372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec15c340 0x7f37ec15ae90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f37d800c370 tx=0x7f37d800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37e3400640 1 -- 172.21.15.83:0/2360211372 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d80052d0 con 0x7f37ec15c340 2024-04-25T00:11:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37e3400640 1 -- 172.21.15.83:0/2360211372 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d8005470 con 0x7f37ec15c340 2024-04-25T00:11:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37e3400640 1 -- 172.21.15.83:0/2360211372 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8007260 con 0x7f37ec15c340 2024-04-25T00:11:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.452+0000 7f37f1e00640 1 -- 172.21.15.83:0/2360211372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec152da0 con 0x7f37ec15c340 2024-04-25T00:11:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.456+0000 7f37f1e00640 1 -- 172.21.15.83:0/2360211372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec0b4940 con 0x7f37ec15c340 2024-04-25T00:11:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.456+0000 7f37f1e00640 1 -- 172.21.15.83:0/2360211372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ec0737f0 con 0x7f37ec15c340 2024-04-25T00:11:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.456+0000 7f37e3400640 1 -- 172.21.15.83:0/2360211372 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f37d8007400 con 0x7f37ec15c340 2024-04-25T00:11:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.456+0000 7f37e3400640 1 --2- 172.21.15.83:0/2360211372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f37cc0678c0 0x7f37cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.456+0000 7f37e3400640 1 -- 172.21.15.83:0/2360211372 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f37d8085310 con 0x7f37ec15c340 2024-04-25T00:11:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.456+0000 7f37eaa00640 1 --2- 172.21.15.83:0/2360211372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f37cc0678c0 0x7f37cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.456+0000 7f37eaa00640 1 --2- 172.21.15.83:0/2360211372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f37cc0678c0 0x7f37cc069d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0ad000 tx=0x7f37dc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.460+0000 7f37e3400640 1 -- 172.21.15.83:0/2360211372 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f37ec0ccc60 con 0x7f37ec15c340 2024-04-25T00:11:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.616+0000 7f37f1e00640 1 -- 172.21.15.83:0/2360211372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37ec04f5f0 con 0x7f37ec15c340 2024-04-25T00:11:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.616+0000 7f37e3400640 1 -- 172.21.15.83:0/2360211372 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7f37d8055550 con 0x7f37ec15c340 2024-04-25T00:11:13.621 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:13.621 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:13.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.620+0000 7f37f1e00640 1 -- 172.21.15.83:0/2360211372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f37cc0678c0 msgr2=0x7f37cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:13.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.620+0000 7f37f1e00640 1 --2- 172.21.15.83:0/2360211372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f37cc0678c0 0x7f37cc069d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0ad000 tx=0x7f37dc0a6000 comp rx=0 tx=0).stop 2024-04-25T00:11:13.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.620+0000 7f37f1e00640 1 -- 172.21.15.83:0/2360211372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec15c340 msgr2=0x7f37ec15ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:13.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.620+0000 7f37f1e00640 1 --2- 172.21.15.83:0/2360211372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec15c340 0x7f37ec15ae90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f37d800c370 tx=0x7f37d800c830 comp rx=0 tx=0).stop 2024-04-25T00:11:13.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.620+0000 7f37f1e00640 1 -- 172.21.15.83:0/2360211372 shutdown_connections 2024-04-25T00:11:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.620+0000 7f37f1e00640 1 --2- 172.21.15.83:0/2360211372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f37cc0678c0 0x7f37cc069d80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.620+0000 7f37f1e00640 1 --2- 172.21.15.83:0/2360211372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec15cec0 0x7f37ec15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.620+0000 7f37f1e00640 1 --2- 172.21.15.83:0/2360211372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec15c340 0x7f37ec15ae90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.620+0000 7f37f1e00640 1 --2- 172.21.15.83:0/2360211372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec1599f0 0x7f37ec0b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.620+0000 7f37f1e00640 1 -- 172.21.15.83:0/2360211372 >> 172.21.15.83:0/2360211372 conn(0x7f37ec0b1830 msgr2=0x7f37ec165c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.620+0000 7f37f1e00640 1 -- 172.21.15.83:0/2360211372 shutdown_connections 2024-04-25T00:11:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:13.620+0000 7f37f1e00640 1 -- 172.21.15.83:0/2360211372 wait complete. 2024-04-25T00:11:14.045 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:11:06.200989+0000', 'modified': '2024-04-25T00:11:11.262751+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6835/1603954817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6833', 'nonce': 1603954817}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 1603954817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4285, 'qdb_cluster': [4285]} 2024-04-25T00:11:14.045 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:11:14.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:11:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.184+0000 7fa42b000640 1 Processor -- start 2024-04-25T00:11:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.184+0000 7fa42b000640 1 -- start start 2024-04-25T00:11:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.184+0000 7fa42b000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa4240cb640 0x7fa42406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.184+0000 7fa42b000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa424071180 con 0x7fa42406cfd0 2024-04-25T00:11:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.184+0000 7fa42b000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa42406e160 con 0x7fa424070d10 2024-04-25T00:11:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.184+0000 7fa42b000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa42406ea10 con 0x7fa4240cb640 2024-04-25T00:11:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.184+0000 7fa41b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa4240cb640 0x7fa42406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.184+0000 7fa423e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa42406cfd0 0x7fa42406d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42590/0 (socket says 172.21.15.83:42590) 2024-04-25T00:11:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.184+0000 7fa423e00640 1 -- 172.21.15.83:0/3420704654 learned_addr learned my addr 172.21.15.83:0/3420704654 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.184+0000 7fa41b400640 1 -- 172.21.15.83:0/3420704654 >> v1:172.21.15.86:6789/0 conn(0x7fa424070d10 legacy=0x7fa424165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.184+0000 7fa41b400640 1 -- 172.21.15.83:0/3420704654 >> v1:172.21.15.83:6789/0 conn(0x7fa42406cfd0 legacy=0x7fa42406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.184+0000 7fa41b400640 1 -- 172.21.15.83:0/3420704654 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42406ecf0 con 0x7fa4240cb640 2024-04-25T00:11:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.184+0000 7fa423400640 1 -- 172.21.15.83:0/3420704654 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1266735747 0 0) 0x7fa424071180 con 0x7fa42406cfd0 2024-04-25T00:11:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.188+0000 7fa41b400640 1 --2- 172.21.15.83:0/3420704654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4240cb640 0x7fa42406dac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0097c0 tx=0x7fa40c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa292f8a437ba34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.188+0000 7fa423400640 1 -- 172.21.15.83:0/3420704654 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0b0070 con 0x7fa4240cb640 2024-04-25T00:11:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.188+0000 7fa423400640 1 -- 172.21.15.83:0/3420704654 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c0ab600 con 0x7fa4240cb640 2024-04-25T00:11:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.188+0000 7fa423400640 1 -- 172.21.15.83:0/3420704654 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0ab9d0 con 0x7fa4240cb640 2024-04-25T00:11:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.188+0000 7fa42b000640 1 -- 172.21.15.83:0/3420704654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4240cb640 msgr2=0x7fa42406dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.188+0000 7fa42b000640 1 --2- 172.21.15.83:0/3420704654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4240cb640 0x7fa42406dac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0097c0 tx=0x7fa40c01c460 comp rx=0 tx=0).stop 2024-04-25T00:11:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.188+0000 7fa42b000640 1 -- 172.21.15.83:0/3420704654 shutdown_connections 2024-04-25T00:11:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.188+0000 7fa42b000640 1 --2- 172.21.15.83:0/3420704654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4240cb640 0x7fa42406dac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.188+0000 7fa42b000640 1 -- 172.21.15.83:0/3420704654 >> 172.21.15.83:0/3420704654 conn(0x7fa4240cfa60 msgr2=0x7fa4240cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.188+0000 7fa42b000640 1 -- 172.21.15.83:0/3420704654 shutdown_connections 2024-04-25T00:11:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.188+0000 7fa42b000640 1 -- 172.21.15.83:0/3420704654 wait complete. 2024-04-25T00:11:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.188+0000 7fa42b000640 1 Processor -- start 2024-04-25T00:11:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.188+0000 7fa42b000640 1 -- start start 2024-04-25T00:11:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa42b000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa42406cfd0 0x7fa4240cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa42b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa424070d10 0x7fa42413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa42b000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4240cb640 0x7fa42413b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa42b000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa424169c00 con 0x7fa4240cb640 2024-04-25T00:11:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa42b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa424169a80 con 0x7fa424070d10 2024-04-25T00:11:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa42b000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa424169d80 con 0x7fa42406cfd0 2024-04-25T00:11:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa429000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4240cb640 0x7fa42413b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa429000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4240cb640 0x7fa42413b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45384/0 (socket says 172.21.15.83:45384) 2024-04-25T00:11:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa429000640 1 -- 172.21.15.83:0/1473362172 learned_addr learned my addr 172.21.15.83:0/1473362172 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa41b400640 1 --2- 172.21.15.83:0/1473362172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa424070d10 0x7fa42413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa423e00640 1 --2- 172.21.15.83:0/1473362172 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa42406cfd0 0x7fa4240cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa429000640 1 -- 172.21.15.83:0/1473362172 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa42406cfd0 msgr2=0x7fa4240cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa429000640 1 --2- 172.21.15.83:0/1473362172 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa42406cfd0 0x7fa4240cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa429000640 1 -- 172.21.15.83:0/1473362172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa424070d10 msgr2=0x7fa42413a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa429000640 1 --2- 172.21.15.83:0/1473362172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa424070d10 0x7fa42413a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa429000640 1 -- 172.21.15.83:0/1473362172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4241f7290 con 0x7fa4240cb640 2024-04-25T00:11:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa429000640 1 --2- 172.21.15.83:0/1473362172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4240cb640 0x7fa42413b4b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00c7b0 tx=0x7fa41c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa420c00640 1 -- 172.21.15.83:0/1473362172 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c004b70 con 0x7fa4240cb640 2024-04-25T00:11:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa420c00640 1 -- 172.21.15.83:0/1473362172 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41c004d10 con 0x7fa4240cb640 2024-04-25T00:11:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa42b000640 1 -- 172.21.15.83:0/1473362172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4241f7570 con 0x7fa4240cb640 2024-04-25T00:11:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa420c00640 1 -- 172.21.15.83:0/1473362172 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c00aa90 con 0x7fa4240cb640 2024-04-25T00:11:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa42b000640 1 -- 172.21.15.83:0/1473362172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4241f7b30 con 0x7fa4240cb640 2024-04-25T00:11:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.192+0000 7fa420c00640 1 -- 172.21.15.83:0/1473362172 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa41c00ac30 con 0x7fa4240cb640 2024-04-25T00:11:14.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa420c00640 1 --2- 172.21.15.83:0/1473362172 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4000678c0 0x7fa400069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa420c00640 1 -- 172.21.15.83:0/1473362172 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa41c086290 con 0x7fa4240cb640 2024-04-25T00:11:14.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa423e00640 1 --2- 172.21.15.83:0/1473362172 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4000678c0 0x7fa400069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa42e442000 1 Processor -- start 2024-04-25T00:11:14.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa42e442000 1 -- start start 2024-04-25T00:11:14.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa423e00640 1 --2- 172.21.15.83:0/1473362172 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4000678c0 0x7fa400069d80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa414000f70 tx=0x7fa414007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:14.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa42e442000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7d17ef0 0x5596e7d182d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa42e442000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d18810 0x5596e7cce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa42e442000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7ccebc0 0x5596e7ccefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa42e442000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5596e7b1c5e0 con 0x5596e7d18810 2024-04-25T00:11:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa42e442000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5596e7aa3c50 con 0x5596e7d17ef0 2024-04-25T00:11:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa42e442000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x5596e7b185a0 con 0x5596e7ccebc0 2024-04-25T00:11:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa41b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d18810 0x5596e7cce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa41b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d18810 0x5596e7cce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45396/0 (socket says 172.21.15.83:45396) 2024-04-25T00:11:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa41b400640 1 -- 172.21.15.83:0/2388739746 learned_addr learned my addr 172.21.15.83:0/2388739746 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa423e00640 1 --2- 172.21.15.83:0/2388739746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7d17ef0 0x5596e7d182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa429000640 1 --2- 172.21.15.83:0/2388739746 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7ccebc0 0x5596e7ccefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa41b400640 1 -- 172.21.15.83:0/2388739746 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7ccebc0 msgr2=0x5596e7ccefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa41b400640 1 --2- 172.21.15.83:0/2388739746 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7ccebc0 0x5596e7ccefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa41b400640 1 -- 172.21.15.83:0/2388739746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7d17ef0 msgr2=0x5596e7d182d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa41b400640 1 --2- 172.21.15.83:0/2388739746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7d17ef0 0x5596e7d182d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa41b400640 1 -- 172.21.15.83:0/2388739746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e7aa1d10 con 0x5596e7d18810 2024-04-25T00:11:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa41b400640 1 --2- 172.21.15.83:0/2388739746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d18810 0x5596e7cce680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0097c0 tx=0x7fa40c0a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47003297a03351 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa418c00640 1 -- 172.21.15.83:0/2388739746 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0bc070 con 0x5596e7d18810 2024-04-25T00:11:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa418c00640 1 -- 172.21.15.83:0/2388739746 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c0ab950 con 0x5596e7d18810 2024-04-25T00:11:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.196+0000 7fa418c00640 1 -- 172.21.15.83:0/2388739746 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0b0040 con 0x5596e7d18810 2024-04-25T00:11:14.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 -- 172.21.15.83:0/2388739746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d18810 msgr2=0x5596e7cce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 --2- 172.21.15.83:0/2388739746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d18810 0x5596e7cce680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0097c0 tx=0x7fa40c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:11:14.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 -- 172.21.15.83:0/2388739746 shutdown_connections 2024-04-25T00:11:14.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 --2- 172.21.15.83:0/2388739746 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7ccebc0 0x5596e7ccefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 --2- 172.21.15.83:0/2388739746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d18810 0x5596e7cce680 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 --2- 172.21.15.83:0/2388739746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7d17ef0 0x5596e7d182d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 -- 172.21.15.83:0/2388739746 >> 172.21.15.83:0/2388739746 conn(0x5596e7ccab70 msgr2=0x5596e7cccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:14.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 -- 172.21.15.83:0/2388739746 shutdown_connections 2024-04-25T00:11:14.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 -- 172.21.15.83:0/2388739746 wait complete. 2024-04-25T00:11:14.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 -- start start 2024-04-25T00:11:14.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 Processor -- start 2024-04-25T00:11:14.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7ccebc0 0x5596e7cfcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7d17ef0 0x5596e7cfd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 0x5596e7cfd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5596e7ad6520 con 0x5596e7ccebc0 2024-04-25T00:11:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5596e7cca9f0 con 0x5596e7d17ef0 2024-04-25T00:11:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x5596e7ad66a0 con 0x5596e7d18810 2024-04-25T00:11:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 10 client.0 _ll_register_callbacks cb 0x5596e7cb95c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:11:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa423e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7ccebc0 0x5596e7cfcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa423e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7ccebc0 0x5596e7cfcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45398/0 (socket says 172.21.15.83:45398) 2024-04-25T00:11:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa423e00640 1 -- 172.21.15.83:0/3995089699 learned_addr learned my addr 172.21.15.83:0/3995089699 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa429000640 1 --2- 172.21.15.83:0/3995089699 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 0x5596e7cfd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa41b400640 1 --2- 172.21.15.83:0/3995089699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7d17ef0 0x5596e7cfd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa423e00640 1 -- 172.21.15.83:0/3995089699 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 msgr2=0x5596e7cfd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa423e00640 1 --2- 172.21.15.83:0/3995089699 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 0x5596e7cfd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa423e00640 1 -- 172.21.15.83:0/3995089699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7d17ef0 msgr2=0x5596e7cfd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa423e00640 1 --2- 172.21.15.83:0/3995089699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7d17ef0 0x5596e7cfd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa423e00640 1 -- 172.21.15.83:0/3995089699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e7ab47b0 con 0x5596e7ccebc0 2024-04-25T00:11:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa423e00640 1 --2- 172.21.15.83:0/3995089699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7ccebc0 0x5596e7cfcf20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa414001d00 tx=0x7fa414000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69584edea3e45c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa411600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:11:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa411600640 1 -- 172.21.15.83:0/3995089699 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4140213f0 con 0x5596e7ccebc0 2024-04-25T00:11:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa411600640 1 -- 172.21.15.83:0/3995089699 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa414010040 con 0x5596e7ccebc0 2024-04-25T00:11:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 -- 172.21.15.83:0/3995089699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa40c0ab950 con 0x5596e7ccebc0 2024-04-25T00:11:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa411600640 1 -- 172.21.15.83:0/3995089699 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fa41402e050 con 0x5596e7ccebc0 2024-04-25T00:11:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 10 client.4428 fetch_fsmap learned FSMap version 7 2024-04-25T00:11:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.200+0000 7fa42e442000 1 -- 172.21.15.83:0/3995089699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5596e7aa3ea0 con 0x5596e7ccebc0 2024-04-25T00:11:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa411600640 1 -- 172.21.15.83:0/3995089699 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c0ab950 con 0x5596e7ccebc0 2024-04-25T00:11:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa411600640 1 -- 172.21.15.83:0/3995089699 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fa414031020 con 0x5596e7ccebc0 2024-04-25T00:11:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T00:11:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 10 client.4428 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:11:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 20 client.4428 populate_metadata read hostname 'smithi083' 2024-04-25T00:11:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 1 --2- 172.21.15.83:0/3995089699 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x5596e7d27f90 0x5596e7d2a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 4 client.4428 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:11:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 1 -- 172.21.15.83:0/3995089699 --> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596e7ab7ab0 con 0x5596e7d27f90 2024-04-25T00:11:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa41b400640 1 --2- 172.21.15.83:0/3995089699 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x5596e7d27f90 0x5596e7d2a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa41b400640 1 --2- 172.21.15.83:0/3995089699 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x5596e7d27f90 0x5596e7d2a380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=504bde1b88979f69 server_cookie=e25cc3fae71d903c in_seq=0 out_seq=0 2024-04-25T00:11:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa411600640 10 client.4428 ms_handle_connect on v2:172.21.15.86:6833/1603954817 2024-04-25T00:11:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa411600640 1 -- 172.21.15.83:0/3995089699 <== mds.0 v2:172.21.15.86:6833/1603954817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596e7ab7ab0 con 0x5596e7d27f90 2024-04-25T00:11:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa411600640 10 client.4428 handle_command_reply: tid=0 2024-04-25T00:11:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 1 client.4428 shutdown 2024-04-25T00:11:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 1 -- 172.21.15.83:0/3995089699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7ccebc0 msgr2=0x5596e7cfcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 1 --2- 172.21.15.83:0/3995089699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7ccebc0 0x5596e7cfcf20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa414001d00 tx=0x7fa414000ee0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 1 -- 172.21.15.83:0/3995089699 shutdown_connections 2024-04-25T00:11:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 1 --2- 172.21.15.83:0/3995089699 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x5596e7d27f90 0x5596e7d2a380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 1 --2- 172.21.15.83:0/3995089699 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 0x5596e7cfd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa411600640 0 client.4428 ms_handle_reset on v2:172.21.15.86:6833/1603954817 2024-04-25T00:11:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 1 --2- 172.21.15.83:0/3995089699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7d17ef0 0x5596e7cfd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 1 --2- 172.21.15.83:0/3995089699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7ccebc0 0x5596e7cfcf20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 1 -- 172.21.15.83:0/3995089699 >> 172.21.15.83:0/3995089699 conn(0x5596e7ccab70 msgr2=0x5596e7ccd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 1 -- 172.21.15.83:0/3995089699 shutdown_connections 2024-04-25T00:11:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 1 -- 172.21.15.83:0/3995089699 wait complete. 2024-04-25T00:11:14.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.204+0000 7fa42e442000 20 client.4428 trim_cache size 0 max 16384 2024-04-25T00:11:14.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 Processor -- start 2024-04-25T00:11:14.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 -- start start 2024-04-25T00:11:14.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7ccebc0 0x5596e7d83500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d17ef0 0x5596e7d83a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 0x5596e7d83f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4242038e0 con 0x5596e7d17ef0 2024-04-25T00:11:14.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4240cdf90 con 0x5596e7ccebc0 2024-04-25T00:11:14.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa424203340 con 0x5596e7d18810 2024-04-25T00:11:14.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa41b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d17ef0 0x5596e7d83a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa41b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d17ef0 0x5596e7d83a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45400/0 (socket says 172.21.15.83:45400) 2024-04-25T00:11:14.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa41b400640 1 -- 172.21.15.83:0/4175003178 learned_addr learned my addr 172.21.15.83:0/4175003178 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:14.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa423e00640 1 --2- 172.21.15.83:0/4175003178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7ccebc0 0x5596e7d83500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa429000640 1 --2- 172.21.15.83:0/4175003178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 0x5596e7d83f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa41b400640 1 -- 172.21.15.83:0/4175003178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 msgr2=0x5596e7d83f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:14.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa41b400640 1 --2- 172.21.15.83:0/4175003178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 0x5596e7d83f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa41b400640 1 -- 172.21.15.83:0/4175003178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7ccebc0 msgr2=0x5596e7d83500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:14.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa41b400640 1 --2- 172.21.15.83:0/4175003178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7ccebc0 0x5596e7d83500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa41b400640 1 -- 172.21.15.83:0/4175003178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e7aa4e80 con 0x5596e7d17ef0 2024-04-25T00:11:14.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa41b400640 1 --2- 172.21.15.83:0/4175003178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d17ef0 0x5596e7d83a40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa40c01ab10 tx=0x7fa40c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b08b6ee6627403 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:14.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa413e00640 1 -- 172.21.15.83:0/4175003178 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0cd070 con 0x5596e7d17ef0 2024-04-25T00:11:14.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa413e00640 1 -- 172.21.15.83:0/4175003178 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c0c6030 con 0x5596e7d17ef0 2024-04-25T00:11:14.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa413e00640 1 -- 172.21.15.83:0/4175003178 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0bc040 con 0x5596e7d17ef0 2024-04-25T00:11:14.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 -- 172.21.15.83:0/4175003178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d17ef0 msgr2=0x5596e7d83a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 --2- 172.21.15.83:0/4175003178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d17ef0 0x5596e7d83a40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa40c01ab10 tx=0x7fa40c004740 comp rx=0 tx=0).stop 2024-04-25T00:11:14.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 -- 172.21.15.83:0/4175003178 shutdown_connections 2024-04-25T00:11:14.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 --2- 172.21.15.83:0/4175003178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 0x5596e7d83f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 --2- 172.21.15.83:0/4175003178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d17ef0 0x5596e7d83a40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 --2- 172.21.15.83:0/4175003178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7ccebc0 0x5596e7d83500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 -- 172.21.15.83:0/4175003178 >> 172.21.15.83:0/4175003178 conn(0x5596e7ccab70 msgr2=0x5596e7d254e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:14.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 -- 172.21.15.83:0/4175003178 shutdown_connections 2024-04-25T00:11:14.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 -- 172.21.15.83:0/4175003178 wait complete. 2024-04-25T00:11:14.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 -- start start 2024-04-25T00:11:14.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.224+0000 7fa42e442000 1 Processor -- start 2024-04-25T00:11:14.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7ccebc0 0x5596e7d87dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d17ef0 0x5596e7d88300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 0x5596e7d2a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5596e7ad0c40 con 0x5596e7d17ef0 2024-04-25T00:11:14.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5596e7bcee80 con 0x5596e7ccebc0 2024-04-25T00:11:14.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x5596e7ae5810 con 0x5596e7d18810 2024-04-25T00:11:14.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 10 client.0 _ll_register_callbacks cb 0x7fa4140019b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:11:14.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa41b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d17ef0 0x5596e7d88300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa41b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d17ef0 0x5596e7d88300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45412/0 (socket says 172.21.15.83:45412) 2024-04-25T00:11:14.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa41b400640 1 -- 172.21.15.83:0/2456501887 learned_addr learned my addr 172.21.15.83:0/2456501887 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:14.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa429000640 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 0x5596e7d2a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa423e00640 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7ccebc0 0x5596e7d87dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa41b400640 1 -- 172.21.15.83:0/2456501887 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 msgr2=0x5596e7d2a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:14.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa41b400640 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 0x5596e7d2a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa41b400640 1 -- 172.21.15.83:0/2456501887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7ccebc0 msgr2=0x5596e7d87dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:14.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa41b400640 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7ccebc0 0x5596e7d87dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa41b400640 1 -- 172.21.15.83:0/2456501887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e7ae9760 con 0x5596e7d17ef0 2024-04-25T00:11:14.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa423e00640 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7ccebc0 0x5596e7d87dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:14.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa429000640 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 0x5596e7d2a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:14.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa41b400640 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d17ef0 0x5596e7d88300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa40c019070 tx=0x7fa40c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3eb7a8a21da503 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:14.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa411600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:11:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa411600640 1 -- 172.21.15.83:0/2456501887 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0cd070 con 0x5596e7d17ef0 2024-04-25T00:11:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa411600640 1 -- 172.21.15.83:0/2456501887 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa40c004240 con 0x5596e7d17ef0 2024-04-25T00:11:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 -- 172.21.15.83:0/2456501887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa40c0c6030 con 0x5596e7d17ef0 2024-04-25T00:11:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa411600640 1 -- 172.21.15.83:0/2456501887 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fa40c0d9050 con 0x5596e7d17ef0 2024-04-25T00:11:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 10 client.4434 fetch_fsmap learned FSMap version 7 2024-04-25T00:11:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 -- 172.21.15.83:0/2456501887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5596e7b759e0 con 0x5596e7d17ef0 2024-04-25T00:11:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa411600640 1 -- 172.21.15.83:0/2456501887 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c0c6030 con 0x5596e7d17ef0 2024-04-25T00:11:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa411600640 1 -- 172.21.15.83:0/2456501887 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fa40c0dc020 con 0x5596e7d17ef0 2024-04-25T00:11:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 10 client.4434 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T00:11:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 10 client.4434 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:11:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 20 client.4434 populate_metadata read hostname 'smithi083' 2024-04-25T00:11:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x5596e7d2a810 0x5596e7d2abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 4 client.4434 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:11:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 -- 172.21.15.83:0/2456501887 --> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] -- command(tid 0: {"prefix": "status"}) -- 0x5596e7b29b20 con 0x5596e7d2a810 2024-04-25T00:11:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa423e00640 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x5596e7d2a810 0x5596e7d2abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa423e00640 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x5596e7d2a810 0x5596e7d2abf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126653b1ae685a55 server_cookie=7b5c894a830295d1 in_seq=0 out_seq=0 2024-04-25T00:11:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa411600640 10 client.4434 ms_handle_connect on v2:172.21.15.86:6833/1603954817 2024-04-25T00:11:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa411600640 1 -- 172.21.15.83:0/2456501887 <== mds.0 v2:172.21.15.86:6833/1603954817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5596e7b29b20 con 0x5596e7d2a810 2024-04-25T00:11:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa411600640 10 client.4434 handle_command_reply: tid=0 2024-04-25T00:11:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 client.4434 shutdown 2024-04-25T00:11:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 -- 172.21.15.83:0/2456501887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d17ef0 msgr2=0x5596e7d88300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d17ef0 0x5596e7d88300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa40c019070 tx=0x7fa40c01aca0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 -- 172.21.15.83:0/2456501887 shutdown_connections 2024-04-25T00:11:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x5596e7d2a810 0x5596e7d2abf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5596e7d18810 0x5596e7d2a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5596e7d17ef0 0x5596e7d88300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa411600640 0 client.4434 ms_handle_reset on v2:172.21.15.86:6833/1603954817 2024-04-25T00:11:14.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 --2- 172.21.15.83:0/2456501887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5596e7ccebc0 0x5596e7d87dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.228+0000 7fa42e442000 1 -- 172.21.15.83:0/2456501887 >> 172.21.15.83:0/2456501887 conn(0x5596e7ccab70 msgr2=0x5596e7d167d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:14.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42e442000 1 -- 172.21.15.83:0/2456501887 shutdown_connections 2024-04-25T00:11:14.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42e442000 1 -- 172.21.15.83:0/2456501887 wait complete. 2024-04-25T00:11:14.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42e442000 20 client.4434 trim_cache size 0 max 16384 2024-04-25T00:11:14.238 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:11:14.238 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "d58ae83a-ad72-4e18-824a-86e779979c35", 2024-04-25T00:11:14.238 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-25T00:11:14.238 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4285, 2024-04-25T00:11:14.238 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T00:11:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T00:11:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T00:11:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 7.3889010070000003, 2024-04-25T00:11:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 7, 2024-04-25T00:11:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 24, 2024-04-25T00:11:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 23, 2024-04-25T00:11:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 14.310493835999999 2024-04-25T00:11:14.239 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:11:14.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42b000640 1 -- 172.21.15.83:0/1473362172 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4000678c0 msgr2=0x7fa400069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42b000640 1 --2- 172.21.15.83:0/1473362172 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4000678c0 0x7fa400069d80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa414000f70 tx=0x7fa414007c10 comp rx=0 tx=0).stop 2024-04-25T00:11:14.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42b000640 1 -- 172.21.15.83:0/1473362172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4240cb640 msgr2=0x7fa42413b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42b000640 1 --2- 172.21.15.83:0/1473362172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4240cb640 0x7fa42413b4b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00c7b0 tx=0x7fa41c00cc70 comp rx=0 tx=0).stop 2024-04-25T00:11:14.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42b000640 1 -- 172.21.15.83:0/1473362172 shutdown_connections 2024-04-25T00:11:14.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42b000640 1 --2- 172.21.15.83:0/1473362172 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4000678c0 0x7fa400069d80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42b000640 1 --2- 172.21.15.83:0/1473362172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4240cb640 0x7fa42413b4b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42b000640 1 --2- 172.21.15.83:0/1473362172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa424070d10 0x7fa42413a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42b000640 1 --2- 172.21.15.83:0/1473362172 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa42406cfd0 0x7fa4240cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42b000640 1 -- 172.21.15.83:0/1473362172 >> 172.21.15.83:0/1473362172 conn(0x7fa4240cfa60 msgr2=0x7fa42413d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:14.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42b000640 1 -- 172.21.15.83:0/1473362172 shutdown_connections 2024-04-25T00:11:14.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.232+0000 7fa42b000640 1 -- 172.21.15.83:0/1473362172 wait complete. 2024-04-25T00:11:14.252 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:11:14.252 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:11:14.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:11:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.384+0000 7f8893800640 1 Processor -- start 2024-04-25T00:11:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.384+0000 7f8893800640 1 -- start start 2024-04-25T00:11:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.384+0000 7f8893800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f888c1553e0 0x7f888c06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.384+0000 7f8893800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f888c06ea00 0x7f888c0b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.384+0000 7f8893800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f888c0b48a0 0x7f888c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.384+0000 7f8893800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f888c06ef80 con 0x7f888c0b48a0 2024-04-25T00:11:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.384+0000 7f8893800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f888c15b510 con 0x7f888c1553e0 2024-04-25T00:11:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.384+0000 7f8893800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f888c15b7f0 con 0x7f888c06ea00 2024-04-25T00:11:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.384+0000 7f8891800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f888c0b48a0 0x7f888c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.384+0000 7f8891800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f888c0b48a0 0x7f888c15e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57780/0 (socket says 172.21.15.83:57780) 2024-04-25T00:11:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.384+0000 7f8891800640 1 -- 172.21.15.83:0/1102114078 learned_addr learned my addr 172.21.15.83:0/1102114078 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.388+0000 7f8890e00640 1 --2- 172.21.15.83:0/1102114078 >> v2:172.21.15.86:3300/0 conn(0x7f888c1553e0 0x7f888c06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.388+0000 7f8883e00640 1 --2- 172.21.15.83:0/1102114078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f888c06ea00 0x7f888c0b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.388+0000 7f8891800640 1 -- 172.21.15.83:0/1102114078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f888c06ea00 msgr2=0x7f888c0b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.388+0000 7f8891800640 1 --2- 172.21.15.83:0/1102114078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f888c06ea00 0x7f888c0b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.388+0000 7f8891800640 1 -- 172.21.15.83:0/1102114078 >> v2:172.21.15.86:3300/0 conn(0x7f888c1553e0 msgr2=0x7f888c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.388+0000 7f8891800640 1 --2- 172.21.15.83:0/1102114078 >> v2:172.21.15.86:3300/0 conn(0x7f888c1553e0 0x7f888c06e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.388+0000 7f8891800640 1 -- 172.21.15.83:0/1102114078 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888c15bad0 con 0x7f888c0b48a0 2024-04-25T00:11:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.388+0000 7f8891800640 1 --2- 172.21.15.83:0/1102114078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f888c0b48a0 0x7f888c15e9f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f888c068b30 tx=0x7f888401c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ece10dfba359d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.388+0000 7f888be00640 1 -- 172.21.15.83:0/1102114078 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88840a9430 con 0x7f888c0b48a0 2024-04-25T00:11:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.388+0000 7f888be00640 1 -- 172.21.15.83:0/1102114078 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88840ae070 con 0x7f888c0b48a0 2024-04-25T00:11:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.388+0000 7f888be00640 1 -- 172.21.15.83:0/1102114078 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88840a97f0 con 0x7f888c0b48a0 2024-04-25T00:11:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.388+0000 7f8893800640 1 -- 172.21.15.83:0/1102114078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f888c0b48a0 msgr2=0x7f888c15e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.388+0000 7f8893800640 1 --2- 172.21.15.83:0/1102114078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f888c0b48a0 0x7f888c15e9f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f888c068b30 tx=0x7f888401c6a0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 -- 172.21.15.83:0/1102114078 shutdown_connections 2024-04-25T00:11:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 --2- 172.21.15.83:0/1102114078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f888c06ea00 0x7f888c0b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 --2- 172.21.15.83:0/1102114078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f888c0b48a0 0x7f888c15e9f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 --2- 172.21.15.83:0/1102114078 >> v2:172.21.15.86:3300/0 conn(0x7f888c1553e0 0x7f888c06e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 -- 172.21.15.83:0/1102114078 >> 172.21.15.83:0/1102114078 conn(0x7f888c0b1830 msgr2=0x7f888c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:14.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 -- 172.21.15.83:0/1102114078 shutdown_connections 2024-04-25T00:11:14.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 -- 172.21.15.83:0/1102114078 wait complete. 2024-04-25T00:11:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 Processor -- start 2024-04-25T00:11:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 -- start start 2024-04-25T00:11:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f888c06ea00 0x7f888c1414d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f888c0b48a0 0x7f888c1fa640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f888c1553e0 0x7f888c1f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f888c06d530 con 0x7f888c06ea00 2024-04-25T00:11:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f888c06d3b0 con 0x7f888c0b48a0 2024-04-25T00:11:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f888c06d6b0 con 0x7f888c1553e0 2024-04-25T00:11:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8890e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f888c06ea00 0x7f888c1414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8890e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f888c06ea00 0x7f888c1414d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57788/0 (socket says 172.21.15.83:57788) 2024-04-25T00:11:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8890e00640 1 -- 172.21.15.83:0/959535990 learned_addr learned my addr 172.21.15.83:0/959535990 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8883e00640 1 --2- 172.21.15.83:0/959535990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f888c0b48a0 0x7f888c1fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8891800640 1 --2- 172.21.15.83:0/959535990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f888c1553e0 0x7f888c1f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8890e00640 1 -- 172.21.15.83:0/959535990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f888c1553e0 msgr2=0x7f888c1f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8890e00640 1 --2- 172.21.15.83:0/959535990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f888c1553e0 0x7f888c1f7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8890e00640 1 -- 172.21.15.83:0/959535990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f888c0b48a0 msgr2=0x7f888c1fa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8890e00640 1 --2- 172.21.15.83:0/959535990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f888c0b48a0 0x7f888c1fa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8890e00640 1 -- 172.21.15.83:0/959535990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888c141a10 con 0x7f888c06ea00 2024-04-25T00:11:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8890e00640 1 --2- 172.21.15.83:0/959535990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f888c06ea00 0x7f888c1414d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f887c00d620 tx=0x7f887c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8889600640 1 -- 172.21.15.83:0/959535990 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c00b620 con 0x7f888c06ea00 2024-04-25T00:11:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8889600640 1 -- 172.21.15.83:0/959535990 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f887c019070 con 0x7f888c06ea00 2024-04-25T00:11:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 -- 172.21.15.83:0/959535990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888c1fae90 con 0x7f888c06ea00 2024-04-25T00:11:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8889600640 1 -- 172.21.15.83:0/959535990 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c00b9e0 con 0x7f888c06ea00 2024-04-25T00:11:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.392+0000 7f8893800640 1 -- 172.21.15.83:0/959535990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f888c1f7360 con 0x7f888c06ea00 2024-04-25T00:11:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.396+0000 7f8893800640 1 -- 172.21.15.83:0/959535990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f888c1fb280 con 0x7f888c06ea00 2024-04-25T00:11:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.396+0000 7f8889600640 1 -- 172.21.15.83:0/959535990 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f887c00bb80 con 0x7f888c06ea00 2024-04-25T00:11:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.396+0000 7f8889600640 1 --2- 172.21.15.83:0/959535990 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88680678c0 0x7f8868069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.396+0000 7f8889600640 1 -- 172.21.15.83:0/959535990 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f887c086360 con 0x7f888c06ea00 2024-04-25T00:11:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.396+0000 7f8883e00640 1 --2- 172.21.15.83:0/959535990 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88680678c0 0x7f8868069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.396+0000 7f8883e00640 1 --2- 172.21.15.83:0/959535990 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88680678c0 0x7f8868069d80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f888c14ed10 tx=0x7f8874008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.400+0000 7f8889600640 1 -- 172.21.15.83:0/959535990 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f887c056b50 con 0x7f888c06ea00 2024-04-25T00:11:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.592+0000 7f8893800640 1 -- 172.21.15.83:0/959535990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f888c0737f0 con 0x7f888c06ea00 2024-04-25T00:11:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.592+0000 7f8889600640 1 -- 172.21.15.83:0/959535990 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f887c056450 con 0x7f888c06ea00 2024-04-25T00:11:14.596 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T00:11:14.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.596+0000 7f8893800640 1 -- 172.21.15.83:0/959535990 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88680678c0 msgr2=0x7f8868069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.596+0000 7f8893800640 1 --2- 172.21.15.83:0/959535990 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88680678c0 0x7f8868069d80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f888c14ed10 tx=0x7f8874008040 comp rx=0 tx=0).stop 2024-04-25T00:11:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.596+0000 7f8893800640 1 -- 172.21.15.83:0/959535990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f888c06ea00 msgr2=0x7f888c1414d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.596+0000 7f8893800640 1 --2- 172.21.15.83:0/959535990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f888c06ea00 0x7f888c1414d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f887c00d620 tx=0x7f887c00dae0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.596+0000 7f8893800640 1 -- 172.21.15.83:0/959535990 shutdown_connections 2024-04-25T00:11:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.596+0000 7f8893800640 1 --2- 172.21.15.83:0/959535990 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88680678c0 0x7f8868069d80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.596+0000 7f8893800640 1 --2- 172.21.15.83:0/959535990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f888c1553e0 0x7f888c1f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.596+0000 7f8893800640 1 --2- 172.21.15.83:0/959535990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f888c0b48a0 0x7f888c1fa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.596+0000 7f8893800640 1 --2- 172.21.15.83:0/959535990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f888c06ea00 0x7f888c1414d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.596+0000 7f8893800640 1 -- 172.21.15.83:0/959535990 >> 172.21.15.83:0/959535990 conn(0x7f888c0b1830 msgr2=0x7f888c071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:14.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.596+0000 7f8893800640 1 -- 172.21.15.83:0/959535990 shutdown_connections 2024-04-25T00:11:14.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14.596+0000 7f8893800640 1 -- 172.21.15.83:0/959535990 wait complete. 2024-04-25T00:11:14.617 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:11:14.617 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:11:14.617 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-25T00:11:14.617 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T00:11:14.617 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:11:14.617 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:11:14.618 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:11:14.618 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:11:14.618 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T00:11:14.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:14.618 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-25T00:11:14.627 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:11:14.627 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:11:14.628 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:11:14.628 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:14.628 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-25T00:11:14.628 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:14.628 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:14.628 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:14.628 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-25T00:11:14.628 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:14.628 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:14.628 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-25T00:11:14.628 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:14.628 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:14.628 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:11:14.629 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:14.629 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:11:14.629 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42460sec preferred_lft 42460sec 2024-04-25T00:11:14.629 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-25T00:11:14.629 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:14.629 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:11:14.629 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:8d:d4:7d:b9 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:14.629 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:11:14.629 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:14.631 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:11:14.631 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:14.631 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:11:14.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:11:14.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2024-04-25T00:11:14.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2024-04-25T00:11:14.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:11:14.632 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:11:14.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:14.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:14.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:14.787 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:11:14.805 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-25T00:11:14.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:14.808 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-25T00:11:14.885 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-25T00:11:14.885 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:11:14.885 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:11:14.885 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:11:14.885 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:11:14.885 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:11:14.885 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:11:14.885 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:14.885 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:14.885 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:14.886 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:11:14.886 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:14.886 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:11:14.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:11:14.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:11:14.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:11:14.886 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:11:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:15.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:15.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:15.025 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:11:15.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:15.035 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-04-25T00:11:15.089 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:15.089 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:11:15.089 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:11:15.089 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T00:11:15.089 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:11:15.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:15.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:15.220 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T00:11:15.221 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:15.221 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:11:15.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T00:11:15.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:11:15.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T00:11:15.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T00:11:15.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T00:11:15.221 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:11:15.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:15.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:15.386 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:15.386 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:11:15.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-25T00:11:15.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:11:15.386 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:11:15.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:15.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:15.512 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T00:11:15.512 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:11:15.512 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:11:15.521 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T00:11:15.521 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:11:15.521 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:11:15.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-25T00:11:15.632 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:11:15.681 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-25T00:11:15.681 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-25T00:11:15.681 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-25T00:11:15.682 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-25T00:11:15.682 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-25T00:11:15.682 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-25T00:11:15.682 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-25T00:11:15.682 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-25T00:11:15.682 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-25T00:11:15.682 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-25T00:11:15.682 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-25T00:11:15.682 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-25T00:11:15.682 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:11:15.683 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-25T00:11:15.683 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-25T00:11:15.683 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-25T00:11:15.683 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-25T00:11:15.683 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-25T00:11:15.683 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-25T00:11:15.683 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-25T00:11:15.683 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:11:15.683 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-25T00:11:15.683 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-25T00:11:15.683 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-25T00:11:15.683 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:11:15.684 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:11:15.684 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:11:15.684 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:11:15.684 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-25T00:11:15.684 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:11:15.686 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:11:15.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:15.687 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-25T00:11:15.740 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:11:15.740 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T00:11:15.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-25T00:11:15.803 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:11:15.857 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-25T00:11:15.858 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-25T00:11:15.858 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-25T00:11:15.858 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-25T00:11:15.858 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-25T00:11:15.858 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-25T00:11:15.858 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-25T00:11:15.858 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-25T00:11:15.858 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-25T00:11:15.858 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-25T00:11:15.858 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-25T00:11:15.858 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-25T00:11:15.859 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:11:15.859 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-25T00:11:15.859 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-25T00:11:15.859 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-25T00:11:15.859 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-25T00:11:15.859 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-25T00:11:15.859 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-25T00:11:15.859 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-25T00:11:15.859 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:11:15.859 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-25T00:11:15.859 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-25T00:11:15.859 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-25T00:11:15.859 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:11:15.860 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:11:15.860 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:11:15.860 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:11:15.860 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-25T00:11:15.860 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:11:15.860 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:11:15.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:15.861 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-25T00:11:15.879 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi083.stderr:2024-04-25T00:11:15.876+0000 7ffacab4a580 -1 init, newargv = 0x55a5a5dd2d30 newargc=15 2024-04-25T00:11:15.879 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi083.stderr:ceph-fuse[34291]: starting ceph client 2024-04-25T00:11:15.889 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi083.stderr:ceph-fuse[34291]: starting fuse 2024-04-25T00:11:16.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-25T00:11:17.012 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:11:17.066 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-25T00:11:17.066 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-25T00:11:17.066 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-25T00:11:17.067 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-25T00:11:17.067 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-25T00:11:17.067 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-25T00:11:17.067 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-25T00:11:17.067 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-25T00:11:17.067 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-25T00:11:17.067 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-25T00:11:17.067 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-25T00:11:17.067 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-25T00:11:17.067 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:11:17.067 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-25T00:11:17.067 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-25T00:11:17.067 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-25T00:11:17.067 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-25T00:11:17.068 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-25T00:11:17.068 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-25T00:11:17.068 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-25T00:11:17.068 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:11:17.068 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-25T00:11:17.068 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-25T00:11:17.068 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-25T00:11:17.068 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:11:17.068 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:11:17.068 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:11:17.069 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:11:17.069 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-25T00:11:17.069 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:11:17.069 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:11:17.069 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-25T00:11:17.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:17.070 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-25T00:11:17.121 INFO:teuthology.orchestra.run.smithi083.stdout:44 2024-04-25T00:11:17.121 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-25T00:11:17.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:11:17.121 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-25T00:11:17.121 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-25T00:11:17.121 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-25T00:11:17.121 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-25T00:11:17.121 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:11:17.122 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-25T00:11:17.123 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-25T00:11:17.123 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-25T00:11:17.123 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:11:17.123 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-25T00:11:17.123 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.admin")) 2024-04-25T00:11:17.123 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-25T00:11:17.253 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.admin.34291.asok 2024-04-25T00:11:17.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:17.265 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.34291.asok 2024-04-25T00:11:17.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:17.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.34291.asok status 2024-04-25T00:11:17.438 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:11:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-25T00:11:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:11:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:11:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "admin", 2024-04-25T00:11:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-25T00:11:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T00:11:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "34291", 2024-04-25T00:11:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-25T00:11:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4446, 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 4446 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4105817294 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:11:17.439 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-25T00:11:17.440 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-25T00:11:17.440 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:11:17.440 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4105817294 2024-04-25T00:11:17.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:11:17.440 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.4446 192.168.0.1:0/4105817294", 2024-04-25T00:11:17.440 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/4105817294", 2024-04-25T00:11:17.440 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-04-25T00:11:17.440 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 7, 2024-04-25T00:11:17.440 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 24, 2024-04-25T00:11:17.440 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:11:17.440 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-25T00:11:17.440 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-25T00:11:17.440 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:11:17.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:17.453 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-25T00:11:17.453 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:11:17.462 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-25T00:11:17.462 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:11:17.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:17.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:11:17.529 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T00:11:17.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:11:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:# file: . 2024-04-25T00:11:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T00:11:17.655 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:17.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:17.662 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:11:17.662 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:17.670 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.844+0000 7f580ee00640 1 Processor -- start 2024-04-25T00:11:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.844+0000 7f580ee00640 1 -- start start 2024-04-25T00:11:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.844+0000 7f580ee00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f580815bf70 0x7f58080cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.844+0000 7f580ee00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58080cc9b0 con 0x7f58080d0940 2024-04-25T00:11:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.844+0000 7f580ee00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58080ccc90 con 0x7f58080cfd80 2024-04-25T00:11:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.844+0000 7f580ee00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f58080b5ad0 con 0x7f580815bf70 2024-04-25T00:11:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.844+0000 7f5807e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f580815bf70 0x7f58080cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.844+0000 7f5807e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f580815bf70 0x7f58080cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57804/0 (socket says 172.21.15.83:57804) 2024-04-25T00:11:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.844+0000 7f5807e00640 1 -- 172.21.15.83:0/3584613935 learned_addr learned my addr 172.21.15.83:0/3584613935 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.844+0000 7f5807e00640 1 -- 172.21.15.83:0/3584613935 >> v1:172.21.15.86:6789/0 conn(0x7f58080cfd80 legacy=0x7f58080d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.844+0000 7f5807e00640 1 -- 172.21.15.83:0/3584613935 >> v1:172.21.15.83:6789/0 conn(0x7f58080d0940 legacy=0x7f5808165b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.844+0000 7f5807e00640 1 -- 172.21.15.83:0/3584613935 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58080ccf70 con 0x7f580815bf70 2024-04-25T00:11:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.848+0000 7f5807e00640 1 --2- 172.21.15.83:0/3584613935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580815bf70 0x7f58080cf840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009e50 tx=0x7f57f801c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08c4ce4d3eb3213 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.848+0000 7f5807400640 1 -- 172.21.15.83:0/3584613935 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f80a5720 con 0x7f580815bf70 2024-04-25T00:11:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.848+0000 7f5807400640 1 -- 172.21.15.83:0/3584613935 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f80a58c0 con 0x7f580815bf70 2024-04-25T00:11:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.848+0000 7f5807400640 1 -- 172.21.15.83:0/3584613935 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f80a5c90 con 0x7f580815bf70 2024-04-25T00:11:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.848+0000 7f580ee00640 1 -- 172.21.15.83:0/3584613935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580815bf70 msgr2=0x7f58080cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.848+0000 7f580ee00640 1 --2- 172.21.15.83:0/3584613935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580815bf70 0x7f58080cf840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009e50 tx=0x7f57f801c880 comp rx=0 tx=0).stop 2024-04-25T00:11:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.848+0000 7f580ee00640 1 -- 172.21.15.83:0/3584613935 shutdown_connections 2024-04-25T00:11:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.848+0000 7f580ee00640 1 --2- 172.21.15.83:0/3584613935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580815bf70 0x7f58080cf840 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.848+0000 7f580ee00640 1 -- 172.21.15.83:0/3584613935 >> 172.21.15.83:0/3584613935 conn(0x7f58080d1c70 msgr2=0x7f58080d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.848+0000 7f580ee00640 1 -- 172.21.15.83:0/3584613935 shutdown_connections 2024-04-25T00:11:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.848+0000 7f580ee00640 1 -- 172.21.15.83:0/3584613935 wait complete. 2024-04-25T00:11:17.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f580ee00640 1 Processor -- start 2024-04-25T00:11:17.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f580ee00640 1 -- start start 2024-04-25T00:11:17.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f580ee00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58080cfd80 0x7f5808157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:17.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f580ee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58080d0940 0x7f58081e56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:17.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f580ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580815bf70 0x7f58080c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:17.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f580ee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f58080d29d0 con 0x7f58080d0940 2024-04-25T00:11:17.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f580ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58080cd530 con 0x7f580815bf70 2024-04-25T00:11:17.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f580ee00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f580815c350 con 0x7f58080cfd80 2024-04-25T00:11:17.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f57ff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58080d0940 0x7f58081e56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:17.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f57ff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58080d0940 0x7f58081e56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57812/0 (socket says 172.21.15.83:57812) 2024-04-25T00:11:17.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f57ff400640 1 -- 172.21.15.83:0/3842428604 learned_addr learned my addr 172.21.15.83:0/3842428604 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:17.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f5807e00640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58080cfd80 0x7f5808157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:17.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f580ce00640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580815bf70 0x7f58080c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:17.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f57ff400640 1 -- 172.21.15.83:0/3842428604 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58080cfd80 msgr2=0x7f5808157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:17.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f57ff400640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58080cfd80 0x7f5808157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:17.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f57ff400640 1 -- 172.21.15.83:0/3842428604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580815bf70 msgr2=0x7f58080c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:17.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f57ff400640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580815bf70 0x7f58080c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:17.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f57ff400640 1 -- 172.21.15.83:0/3842428604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58081e6170 con 0x7f58080d0940 2024-04-25T00:11:17.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f580ce00640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580815bf70 0x7f58080c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:17.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f5807e00640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58080cfd80 0x7f5808157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:17.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f57ff400640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58080d0940 0x7f58081e56a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f57f000b410 tx=0x7f57f000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:17.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f5804c00640 1 -- 172.21.15.83:0/3842428604 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0015030 con 0x7f58080d0940 2024-04-25T00:11:17.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f5804c00640 1 -- 172.21.15.83:0/3842428604 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f0004020 con 0x7f58080d0940 2024-04-25T00:11:17.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f5804c00640 1 -- 172.21.15.83:0/3842428604 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0004410 con 0x7f58080d0940 2024-04-25T00:11:17.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f580ee00640 1 -- 172.21.15.83:0/3842428604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58081e6450 con 0x7f58080d0940 2024-04-25T00:11:17.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f580ee00640 1 -- 172.21.15.83:0/3842428604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58081e6a10 con 0x7f58080d0940 2024-04-25T00:11:17.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.852+0000 7f580ee00640 1 -- 172.21.15.83:0/3842428604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f580809fd60 con 0x7f58080d0940 2024-04-25T00:11:17.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.856+0000 7f5804c00640 1 -- 172.21.15.83:0/3842428604 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f57f0018070 con 0x7f58080d0940 2024-04-25T00:11:17.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.856+0000 7f5804c00640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f57e40678c0 0x7f57e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:17.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.856+0000 7f5804c00640 1 -- 172.21.15.83:0/3842428604 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f57f0085bc0 con 0x7f58080d0940 2024-04-25T00:11:17.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.856+0000 7f5807e00640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f57e40678c0 0x7f57e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:17.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.856+0000 7f5807e00640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f57e40678c0 0x7f57e4069d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f57f80ae000 tx=0x7f57f80a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:17.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:17.860+0000 7f5804c00640 1 -- 172.21.15.83:0/3842428604 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f57f0056320 con 0x7f58080d0940 2024-04-25T00:11:18.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.020+0000 7f580ee00640 1 -- 172.21.15.83:0/3842428604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f580804f5f0 con 0x7f58080d0940 2024-04-25T00:11:18.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.020+0000 7f5804c00640 1 -- 172.21.15.83:0/3842428604 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7f57f0055c20 con 0x7f58080d0940 2024-04-25T00:11:18.023 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:18.023 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:18.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.020+0000 7f580ee00640 1 -- 172.21.15.83:0/3842428604 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f57e40678c0 msgr2=0x7f57e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.020+0000 7f580ee00640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f57e40678c0 0x7f57e4069d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f57f80ae000 tx=0x7f57f80a7070 comp rx=0 tx=0).stop 2024-04-25T00:11:18.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.020+0000 7f580ee00640 1 -- 172.21.15.83:0/3842428604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58080d0940 msgr2=0x7f58081e56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.020+0000 7f580ee00640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58080d0940 0x7f58081e56a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f57f000b410 tx=0x7f57f000b8d0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.024+0000 7f580ee00640 1 -- 172.21.15.83:0/3842428604 shutdown_connections 2024-04-25T00:11:18.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.024+0000 7f580ee00640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f57e40678c0 0x7f57e4069d80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.024+0000 7f580ee00640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580815bf70 0x7f58080c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.024+0000 7f580ee00640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58080d0940 0x7f58081e56a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.024+0000 7f580ee00640 1 --2- 172.21.15.83:0/3842428604 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58080cfd80 0x7f5808157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.024+0000 7f580ee00640 1 -- 172.21.15.83:0/3842428604 >> 172.21.15.83:0/3842428604 conn(0x7f58080d1c70 msgr2=0x7f58080b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:18.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.024+0000 7f580ee00640 1 -- 172.21.15.83:0/3842428604 shutdown_connections 2024-04-25T00:11:18.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.024+0000 7f580ee00640 1 -- 172.21.15.83:0/3842428604 wait complete. 2024-04-25T00:11:18.044 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:11:04.621429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:11:05.189716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:11:08.902215+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:11:18.044 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:11:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.180+0000 7fb993800640 1 Processor -- start 2024-04-25T00:11:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.180+0000 7fb993800640 1 -- start start 2024-04-25T00:11:18.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.180+0000 7fb993800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb98c06e950 0x7fb98c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.180+0000 7fb993800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb98c0ce000 con 0x7fb98c15fe50 2024-04-25T00:11:18.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.180+0000 7fb993800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb98c06eec0 con 0x7fb98c159810 2024-04-25T00:11:18.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.180+0000 7fb993800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb98c06e210 con 0x7fb98c06e950 2024-04-25T00:11:18.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.180+0000 7fb983e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb98c06e950 0x7fb98c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.180+0000 7fb983e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb98c06e950 0x7fb98c15f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57826/0 (socket says 172.21.15.83:57826) 2024-04-25T00:11:18.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.180+0000 7fb983e00640 1 -- 172.21.15.83:0/3270325639 learned_addr learned my addr 172.21.15.83:0/3270325639 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:18.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.180+0000 7fb983e00640 1 -- 172.21.15.83:0/3270325639 >> v1:172.21.15.86:6789/0 conn(0x7fb98c159810 legacy=0x7fb98c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.180+0000 7fb983e00640 1 -- 172.21.15.83:0/3270325639 >> v1:172.21.15.83:6789/0 conn(0x7fb98c15fe50 legacy=0x7fb98c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.180+0000 7fb983e00640 1 -- 172.21.15.83:0/3270325639 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c06dc50 con 0x7fb98c06e950 2024-04-25T00:11:18.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.184+0000 7fb983e00640 1 --2- 172.21.15.83:0/3270325639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c06e950 0x7fb98c15f910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb974009870 tx=0x7fb97401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b360d03ae48d273e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:18.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.184+0000 7fb98be00640 1 -- 172.21.15.83:0/3270325639 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9740b0070 con 0x7fb98c06e950 2024-04-25T00:11:18.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.184+0000 7fb98be00640 1 -- 172.21.15.83:0/3270325639 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9740ab600 con 0x7fb98c06e950 2024-04-25T00:11:18.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.184+0000 7fb98be00640 1 -- 172.21.15.83:0/3270325639 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9740ab9d0 con 0x7fb98c06e950 2024-04-25T00:11:18.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.184+0000 7fb993800640 1 -- 172.21.15.83:0/3270325639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c06e950 msgr2=0x7fb98c15f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.184+0000 7fb993800640 1 --2- 172.21.15.83:0/3270325639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c06e950 0x7fb98c15f910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb974009870 tx=0x7fb97401c4d0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.184+0000 7fb993800640 1 -- 172.21.15.83:0/3270325639 shutdown_connections 2024-04-25T00:11:18.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.184+0000 7fb993800640 1 --2- 172.21.15.83:0/3270325639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c06e950 0x7fb98c15f910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.184+0000 7fb993800640 1 -- 172.21.15.83:0/3270325639 >> 172.21.15.83:0/3270325639 conn(0x7fb98c0b1830 msgr2=0x7fb98c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:18.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.184+0000 7fb993800640 1 -- 172.21.15.83:0/3270325639 shutdown_connections 2024-04-25T00:11:18.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.184+0000 7fb993800640 1 -- 172.21.15.83:0/3270325639 wait complete. 2024-04-25T00:11:18.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb993800640 1 Processor -- start 2024-04-25T00:11:18.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb993800640 1 -- start start 2024-04-25T00:11:18.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb993800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb98c06e950 0x7fb98c071830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb993800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb98c159810 0x7fb98c0b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb993800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c15fe50 0x7fb98c0b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb993800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c169e40 con 0x7fb98c15fe50 2024-04-25T00:11:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb993800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c169cc0 con 0x7fb98c06e950 2024-04-25T00:11:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb993800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb98c169fc0 con 0x7fb98c159810 2024-04-25T00:11:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb991800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c15fe50 0x7fb98c0b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb991800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c15fe50 0x7fb98c0b3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57828/0 (socket says 172.21.15.83:57828) 2024-04-25T00:11:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb991800640 1 -- 172.21.15.83:0/1729139004 learned_addr learned my addr 172.21.15.83:0/1729139004 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb983e00640 1 --2- 172.21.15.83:0/1729139004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb98c159810 0x7fb98c0b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb990e00640 1 --2- 172.21.15.83:0/1729139004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb98c06e950 0x7fb98c071830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb991800640 1 -- 172.21.15.83:0/1729139004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb98c159810 msgr2=0x7fb98c0b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb991800640 1 --2- 172.21.15.83:0/1729139004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb98c159810 0x7fb98c0b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb991800640 1 -- 172.21.15.83:0/1729139004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb98c06e950 msgr2=0x7fb98c071830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb991800640 1 --2- 172.21.15.83:0/1729139004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb98c06e950 0x7fb98c071830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb991800640 1 -- 172.21.15.83:0/1729139004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c15e840 con 0x7fb98c15fe50 2024-04-25T00:11:18.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb991800640 1 --2- 172.21.15.83:0/1729139004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c15fe50 0x7fb98c0b3170 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb98400c7a0 tx=0x7fb98400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb989600640 1 -- 172.21.15.83:0/1729139004 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984003ba0 con 0x7fb98c15fe50 2024-04-25T00:11:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb989600640 1 -- 172.21.15.83:0/1729139004 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb984003d40 con 0x7fb98c15fe50 2024-04-25T00:11:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb993800640 1 -- 172.21.15.83:0/1729139004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c0b3760 con 0x7fb98c15fe50 2024-04-25T00:11:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb989600640 1 -- 172.21.15.83:0/1729139004 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98400aa30 con 0x7fb98c15fe50 2024-04-25T00:11:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.188+0000 7fb993800640 1 -- 172.21.15.83:0/1729139004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c0b4000 con 0x7fb98c15fe50 2024-04-25T00:11:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.192+0000 7fb993800640 1 -- 172.21.15.83:0/1729139004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98c0b3b50 con 0x7fb98c15fe50 2024-04-25T00:11:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.192+0000 7fb989600640 1 -- 172.21.15.83:0/1729139004 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb98400abd0 con 0x7fb98c15fe50 2024-04-25T00:11:18.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.192+0000 7fb989600640 1 --2- 172.21.15.83:0/1729139004 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb9680678c0 0x7fb968069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.192+0000 7fb989600640 1 -- 172.21.15.83:0/1729139004 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb984085fa0 con 0x7fb98c15fe50 2024-04-25T00:11:18.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.192+0000 7fb990e00640 1 --2- 172.21.15.83:0/1729139004 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb9680678c0 0x7fb968069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.192+0000 7fb990e00640 1 --2- 172.21.15.83:0/1729139004 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb9680678c0 0x7fb968069d80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb97c000f70 tx=0x7fb97c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.196+0000 7fb989600640 1 -- 172.21.15.83:0/1729139004 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb984056700 con 0x7fb98c15fe50 2024-04-25T00:11:18.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.352+0000 7fb993800640 1 -- 172.21.15.83:0/1729139004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb98c072c90 con 0x7fb98c15fe50 2024-04-25T00:11:18.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.352+0000 7fb989600640 1 -- 172.21.15.83:0/1729139004 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fb984056000 con 0x7fb98c15fe50 2024-04-25T00:11:18.355 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:11:18.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.352+0000 7fb993800640 1 -- 172.21.15.83:0/1729139004 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb9680678c0 msgr2=0x7fb968069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.352+0000 7fb993800640 1 --2- 172.21.15.83:0/1729139004 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb9680678c0 0x7fb968069d80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb97c000f70 tx=0x7fb97c009040 comp rx=0 tx=0).stop 2024-04-25T00:11:18.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.352+0000 7fb993800640 1 -- 172.21.15.83:0/1729139004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c15fe50 msgr2=0x7fb98c0b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.352+0000 7fb993800640 1 --2- 172.21.15.83:0/1729139004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c15fe50 0x7fb98c0b3170 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb98400c7a0 tx=0x7fb98400cc60 comp rx=0 tx=0).stop 2024-04-25T00:11:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.352+0000 7fb993800640 1 -- 172.21.15.83:0/1729139004 shutdown_connections 2024-04-25T00:11:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.352+0000 7fb993800640 1 --2- 172.21.15.83:0/1729139004 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb9680678c0 0x7fb968069d80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.352+0000 7fb993800640 1 --2- 172.21.15.83:0/1729139004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb98c159810 0x7fb98c0b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.352+0000 7fb993800640 1 --2- 172.21.15.83:0/1729139004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c15fe50 0x7fb98c0b3170 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.352+0000 7fb993800640 1 --2- 172.21.15.83:0/1729139004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb98c06e950 0x7fb98c071830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.352+0000 7fb993800640 1 -- 172.21.15.83:0/1729139004 >> 172.21.15.83:0/1729139004 conn(0x7fb98c0b1830 msgr2=0x7fb98c0cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.352+0000 7fb993800640 1 -- 172.21.15.83:0/1729139004 shutdown_connections 2024-04-25T00:11:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.352+0000 7fb993800640 1 -- 172.21.15.83:0/1729139004 wait complete. 2024-04-25T00:11:18.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:11:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.508+0000 7f6191200640 1 Processor -- start 2024-04-25T00:11:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6191200640 1 -- start start 2024-04-25T00:11:18.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6191200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f618c06e950 0x7f618c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6191200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f618c15fe50 0x7f618c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6191200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f618c0ce000 con 0x7f618c159810 2024-04-25T00:11:18.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6191200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f618c06e210 con 0x7f618c06e950 2024-04-25T00:11:18.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6191200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f618c06da00 con 0x7f618c15fe50 2024-04-25T00:11:18.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6189e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f618c159810 0x7f618c0cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57958/0 (socket says 172.21.15.83:57958) 2024-04-25T00:11:18.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6189e00640 1 -- 172.21.15.83:0/3617099610 learned_addr learned my addr 172.21.15.83:0/3617099610 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6189400640 1 --2- 172.21.15.83:0/3617099610 >> v2:172.21.15.86:3300/0 conn(0x7f618c06e950 0x7f618c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f618a800640 1 --2- 172.21.15.83:0/3617099610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f618c15fe50 0x7f618c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 -- 172.21.15.83:0/3617099610 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 256837708 0 0) 0x7f618c0ce000 con 0x7f618c159810 2024-04-25T00:11:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 -- 172.21.15.83:0/3617099610 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6170003530 con 0x7f618c159810 2024-04-25T00:11:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 -- 172.21.15.83:0/3617099610 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2448043281 0 0) 0x7f6170003530 con 0x7f618c159810 2024-04-25T00:11:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 -- 172.21.15.83:0/3617099610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f618c15fe50 msgr2=0x7f618c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 --2- 172.21.15.83:0/3617099610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f618c15fe50 0x7f618c0ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 -- 172.21.15.83:0/3617099610 >> v2:172.21.15.86:3300/0 conn(0x7f618c06e950 msgr2=0x7f618c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 --2- 172.21.15.83:0/3617099610 >> v2:172.21.15.86:3300/0 conn(0x7f618c06e950 0x7f618c15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 -- 172.21.15.83:0/3617099610 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c06eec0 con 0x7f618c159810 2024-04-25T00:11:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 -- 172.21.15.83:0/3617099610 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f617c001680 con 0x7f618c159810 2024-04-25T00:11:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 -- 172.21.15.83:0/3617099610 >> v1:172.21.15.83:6789/0 conn(0x7f618c159810 legacy=0x7f618c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 --2- 172.21.15.83:0/3617099610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f618c15fe50 0x7f6170003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 --2- 172.21.15.83:0/3617099610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6170004020 0x7f6170004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 --2- 172.21.15.83:0/3617099610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6170004d50 0x7f61700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 -- 172.21.15.83:0/3617099610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f618c06e210 con 0x7f6170004d50 2024-04-25T00:11:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 -- 172.21.15.83:0/3617099610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f618c06da00 con 0x7f6170004020 2024-04-25T00:11:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6188a00640 1 -- 172.21.15.83:0/3617099610 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f617000bb00 con 0x7f618c15fe50 2024-04-25T00:11:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f618a800640 1 --2- 172.21.15.83:0/3617099610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6170004d50 0x7f61700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6189e00640 1 --2- 172.21.15.83:0/3617099610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f618c15fe50 0x7f6170003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f6189400640 1 --2- 172.21.15.83:0/3617099610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6170004020 0x7f6170004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f618a800640 1 -- 172.21.15.83:0/3617099610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f618c15fe50 msgr2=0x7f6170003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:18.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f618a800640 1 --2- 172.21.15.83:0/3617099610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f618c15fe50 0x7f6170003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f618a800640 1 -- 172.21.15.83:0/3617099610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6170004020 msgr2=0x7f6170004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:18.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f618a800640 1 --2- 172.21.15.83:0/3617099610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6170004020 0x7f6170004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.512+0000 7f618a800640 1 -- 172.21.15.83:0/3617099610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6170003530 con 0x7f6170004d50 2024-04-25T00:11:18.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.516+0000 7f618a800640 1 --2- 172.21.15.83:0/3617099610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6170004d50 0x7f61700051d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6184009fa0 tx=0x7f618401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b78325232927db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:18.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.516+0000 7f6188a00640 1 -- 172.21.15.83:0/3617099610 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61840b2070 con 0x7f6170004d50 2024-04-25T00:11:18.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.516+0000 7f6188a00640 1 -- 172.21.15.83:0/3617099610 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61840a5d10 con 0x7f6170004d50 2024-04-25T00:11:18.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.516+0000 7f6188a00640 1 -- 172.21.15.83:0/3617099610 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184009bf0 con 0x7f6170004d50 2024-04-25T00:11:18.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.516+0000 7f6191200640 1 -- 172.21.15.83:0/3617099610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6170004d50 msgr2=0x7f61700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.516+0000 7f6191200640 1 --2- 172.21.15.83:0/3617099610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6170004d50 0x7f61700051d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6184009fa0 tx=0x7f618401cc40 comp rx=0 tx=0).stop 2024-04-25T00:11:18.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.516+0000 7f618a800640 1 -- 172.21.15.83:0/3617099610 reap_dead start 2024-04-25T00:11:18.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.516+0000 7f6191200640 1 -- 172.21.15.83:0/3617099610 shutdown_connections 2024-04-25T00:11:18.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.516+0000 7f6191200640 1 -- 172.21.15.83:0/3617099610 >> 172.21.15.83:0/3617099610 conn(0x7f618c0b1830 msgr2=0x7f618c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:18.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.516+0000 7f6191200640 1 -- 172.21.15.83:0/3617099610 shutdown_connections 2024-04-25T00:11:18.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.516+0000 7f6191200640 1 -- 172.21.15.83:0/3617099610 wait complete. 2024-04-25T00:11:18.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6191200640 1 Processor -- start 2024-04-25T00:11:18.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6191200640 1 -- start start 2024-04-25T00:11:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6191200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6170004d50 0x7f618c14f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6191200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f618c1f7d70 0x7f618c1444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6191200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f618c15e540 0x7f618c15e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6191200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f618c159bf0 con 0x7f6170004d50 2024-04-25T00:11:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6191200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f618c06df30 con 0x7f618c15e540 2024-04-25T00:11:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6191200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f618c159d70 con 0x7f618c1f7d70 2024-04-25T00:11:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6189e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6170004d50 0x7f618c14f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6189e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6170004d50 0x7f618c14f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57852/0 (socket says 172.21.15.83:57852) 2024-04-25T00:11:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6189e00640 1 -- 172.21.15.83:0/2058928014 learned_addr learned my addr 172.21.15.83:0/2058928014 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f618a800640 1 --2- 172.21.15.83:0/2058928014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f618c15e540 0x7f618c15e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6189400640 1 --2- 172.21.15.83:0/2058928014 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f618c1f7d70 0x7f618c1444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6189e00640 1 -- 172.21.15.83:0/2058928014 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f618c1f7d70 msgr2=0x7f618c1444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6189e00640 1 --2- 172.21.15.83:0/2058928014 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f618c1f7d70 0x7f618c1444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6189e00640 1 -- 172.21.15.83:0/2058928014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f618c15e540 msgr2=0x7f618c15e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6189e00640 1 --2- 172.21.15.83:0/2058928014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f618c15e540 0x7f618c15e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6189e00640 1 -- 172.21.15.83:0/2058928014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c1507d0 con 0x7f6170004d50 2024-04-25T00:11:18.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6189e00640 1 --2- 172.21.15.83:0/2058928014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6170004d50 0x7f618c14f910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f617c012260 tx=0x7f617c012720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:18.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6182000640 1 -- 172.21.15.83:0/2058928014 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c01c070 con 0x7f6170004d50 2024-04-25T00:11:18.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6182000640 1 -- 172.21.15.83:0/2058928014 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f617c014770 con 0x7f6170004d50 2024-04-25T00:11:18.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6191200640 1 -- 172.21.15.83:0/2058928014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618c150ab0 con 0x7f6170004d50 2024-04-25T00:11:18.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6182000640 1 -- 172.21.15.83:0/2058928014 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c014b60 con 0x7f6170004d50 2024-04-25T00:11:18.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6191200640 1 -- 172.21.15.83:0/2058928014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618c1f81f0 con 0x7f6170004d50 2024-04-25T00:11:18.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.520+0000 7f6191200640 1 -- 172.21.15.83:0/2058928014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f618c04f510 con 0x7f6170004d50 2024-04-25T00:11:18.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.524+0000 7f6182000640 1 -- 172.21.15.83:0/2058928014 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f617c01f070 con 0x7f6170004d50 2024-04-25T00:11:18.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.524+0000 7f6182000640 1 --2- 172.21.15.83:0/2058928014 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f616c0678c0 0x7f616c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.524+0000 7f6182000640 1 -- 172.21.15.83:0/2058928014 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f617c0889a0 con 0x7f6170004d50 2024-04-25T00:11:18.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.524+0000 7f6189400640 1 --2- 172.21.15.83:0/2058928014 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f616c0678c0 0x7f616c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.524+0000 7f6189400640 1 --2- 172.21.15.83:0/2058928014 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f616c0678c0 0x7f616c069d80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f618c0cfcc0 tx=0x7f6178008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:18.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.528+0000 7f6182000640 1 -- 172.21.15.83:0/2058928014 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f617c059190 con 0x7f6170004d50 2024-04-25T00:11:18.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.684+0000 7f6191200640 1 -- 172.21.15.83:0/2058928014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f618c0b10e0 con 0x7f6170004d50 2024-04-25T00:11:18.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.684+0000 7f6182000640 1 -- 172.21.15.83:0/2058928014 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f617c058a90 con 0x7f6170004d50 2024-04-25T00:11:18.687 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:18.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.688+0000 7f6191200640 1 -- 172.21.15.83:0/2058928014 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f616c0678c0 msgr2=0x7f616c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.688+0000 7f6191200640 1 --2- 172.21.15.83:0/2058928014 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f616c0678c0 0x7f616c069d80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f618c0cfcc0 tx=0x7f6178008040 comp rx=0 tx=0).stop 2024-04-25T00:11:18.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.688+0000 7f6191200640 1 -- 172.21.15.83:0/2058928014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6170004d50 msgr2=0x7f618c14f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.688+0000 7f6191200640 1 --2- 172.21.15.83:0/2058928014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6170004d50 0x7f618c14f910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f617c012260 tx=0x7f617c012720 comp rx=0 tx=0).stop 2024-04-25T00:11:18.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.688+0000 7f6191200640 1 -- 172.21.15.83:0/2058928014 shutdown_connections 2024-04-25T00:11:18.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.688+0000 7f6191200640 1 --2- 172.21.15.83:0/2058928014 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f616c0678c0 0x7f616c069d80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.688+0000 7f6191200640 1 --2- 172.21.15.83:0/2058928014 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f618c1f7d70 0x7f618c1444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.688+0000 7f6191200640 1 --2- 172.21.15.83:0/2058928014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f618c15e540 0x7f618c15e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.688+0000 7f6191200640 1 --2- 172.21.15.83:0/2058928014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6170004d50 0x7f618c14f910 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.688+0000 7f6191200640 1 -- 172.21.15.83:0/2058928014 >> 172.21.15.83:0/2058928014 conn(0x7f618c0b1830 msgr2=0x7f618c0ad9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:18.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.688+0000 7f6191200640 1 -- 172.21.15.83:0/2058928014 shutdown_connections 2024-04-25T00:11:18.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.688+0000 7f6191200640 1 -- 172.21.15.83:0/2058928014 wait complete. 2024-04-25T00:11:18.709 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:11:18.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bbdc00640 1 Processor -- start 2024-04-25T00:11:18.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bbdc00640 1 -- start start 2024-04-25T00:11:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bbdc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5bb81551e0 0x7f5bb815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bbdc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5bb81555c0 0x7f5bb8071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bbdc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bb806e4c0 0x7f5bb806e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bbdc00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5bb815c560 con 0x7f5bb81551e0 2024-04-25T00:11:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bbdc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5bb8072460 con 0x7f5bb81555c0 2024-04-25T00:11:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bbdc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb815c7d0 con 0x7f5bb806e4c0 2024-04-25T00:11:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bb6a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5bb81551e0 0x7f5bb815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bb6a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5bb81551e0 0x7f5bb815bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57866/0 (socket says 172.21.15.83:57866) 2024-04-25T00:11:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bb6a00640 1 -- 172.21.15.83:0/2532708791 learned_addr learned my addr 172.21.15.83:0/2532708791 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:18.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bb6a00640 1 -- 172.21.15.83:0/2532708791 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bb806e4c0 msgr2=0x7f5bb806e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:18.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bb6a00640 1 --2- 172.21.15.83:0/2532708791 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bb806e4c0 0x7f5bb806e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bb6a00640 1 -- 172.21.15.83:0/2532708791 >> v2:172.21.15.86:3300/0 conn(0x7f5bb81555c0 msgr2=0x7f5bb8071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:18.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bb6a00640 1 --2- 172.21.15.83:0/2532708791 >> v2:172.21.15.86:3300/0 conn(0x7f5bb81555c0 0x7f5bb8071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.844+0000 7f5bb6a00640 1 -- 172.21.15.83:0/2532708791 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb815cab0 con 0x7f5bb81551e0 2024-04-25T00:11:18.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.848+0000 7f5bb6a00640 1 --2- 172.21.15.83:0/2532708791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bb81551e0 0x7f5bb815bea0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009870 tx=0x7f5ba801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc901dad80dea19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:18.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.848+0000 7f5bb6000640 1 -- 172.21.15.83:0/2532708791 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80ab5f0 con 0x7f5bb81551e0 2024-04-25T00:11:18.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.848+0000 7f5bb6000640 1 -- 172.21.15.83:0/2532708791 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba80b0070 con 0x7f5bb81551e0 2024-04-25T00:11:18.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.848+0000 7f5bb6000640 1 -- 172.21.15.83:0/2532708791 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80ab9b0 con 0x7f5bb81551e0 2024-04-25T00:11:18.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.848+0000 7f5bbdc00640 1 -- 172.21.15.83:0/2532708791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bb81551e0 msgr2=0x7f5bb815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.848+0000 7f5bbdc00640 1 --2- 172.21.15.83:0/2532708791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bb81551e0 0x7f5bb815bea0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009870 tx=0x7f5ba801c520 comp rx=0 tx=0).stop 2024-04-25T00:11:18.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.848+0000 7f5bbdc00640 1 -- 172.21.15.83:0/2532708791 shutdown_connections 2024-04-25T00:11:18.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.848+0000 7f5bbdc00640 1 --2- 172.21.15.83:0/2532708791 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bb806e4c0 0x7f5bb806e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.848+0000 7f5bbdc00640 1 --2- 172.21.15.83:0/2532708791 >> v2:172.21.15.86:3300/0 conn(0x7f5bb81555c0 0x7f5bb8071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.848+0000 7f5bbdc00640 1 --2- 172.21.15.83:0/2532708791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bb81551e0 0x7f5bb815bea0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.848+0000 7f5bbdc00640 1 -- 172.21.15.83:0/2532708791 >> 172.21.15.83:0/2532708791 conn(0x7f5bb80b1830 msgr2=0x7f5bb80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:18.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.848+0000 7f5bbdc00640 1 -- 172.21.15.83:0/2532708791 shutdown_connections 2024-04-25T00:11:18.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.848+0000 7f5bbdc00640 1 -- 172.21.15.83:0/2532708791 wait complete. 2024-04-25T00:11:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bbdc00640 1 Processor -- start 2024-04-25T00:11:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bbdc00640 1 -- start start 2024-04-25T00:11:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bbdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bb806e4c0 0x7f5bb80cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bbdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb81551e0 0x7f5bb80b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bbdc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bb81555c0 0x7f5bb815a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bbdc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb815f490 con 0x7f5bb806e4c0 2024-04-25T00:11:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bbdc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb815f310 con 0x7f5bb81551e0 2024-04-25T00:11:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bbdc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb815f610 con 0x7f5bb81555c0 2024-04-25T00:11:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bb6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bb806e4c0 0x7f5bb80cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bb6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bb806e4c0 0x7f5bb80cdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57882/0 (socket says 172.21.15.83:57882) 2024-04-25T00:11:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bb6a00640 1 -- 172.21.15.83:0/3828817550 learned_addr learned my addr 172.21.15.83:0/3828817550 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bae000640 1 --2- 172.21.15.83:0/3828817550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb81551e0 0x7f5bb80b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bb7400640 1 --2- 172.21.15.83:0/3828817550 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bb81555c0 0x7f5bb815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bb6a00640 1 -- 172.21.15.83:0/3828817550 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bb81555c0 msgr2=0x7f5bb815a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bb6a00640 1 --2- 172.21.15.83:0/3828817550 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bb81555c0 0x7f5bb815a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bb6a00640 1 -- 172.21.15.83:0/3828817550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb81551e0 msgr2=0x7f5bb80b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bb6a00640 1 --2- 172.21.15.83:0/3828817550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb81551e0 0x7f5bb80b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bb6a00640 1 -- 172.21.15.83:0/3828817550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb80ce320 con 0x7f5bb806e4c0 2024-04-25T00:11:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bb6a00640 1 --2- 172.21.15.83:0/3828817550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bb806e4c0 0x7f5bb80cdc90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80ad040 tx=0x7f5ba801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:18.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5baf400640 1 -- 172.21.15.83:0/3828817550 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80be5f0 con 0x7f5bb806e4c0 2024-04-25T00:11:18.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5baf400640 1 -- 172.21.15.83:0/3828817550 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba80be7b0 con 0x7f5bb806e4c0 2024-04-25T00:11:18.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5baf400640 1 -- 172.21.15.83:0/3828817550 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80beba0 con 0x7f5bb806e4c0 2024-04-25T00:11:18.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bbdc00640 1 -- 172.21.15.83:0/3828817550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb80ce600 con 0x7f5bb806e4c0 2024-04-25T00:11:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.852+0000 7f5bbdc00640 1 -- 172.21.15.83:0/3828817550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb80cebc0 con 0x7f5bb806e4c0 2024-04-25T00:11:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.856+0000 7f5bbdc00640 1 -- 172.21.15.83:0/3828817550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb804f510 con 0x7f5bb806e4c0 2024-04-25T00:11:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.856+0000 7f5baf400640 1 -- 172.21.15.83:0/3828817550 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5ba80b43f0 con 0x7f5bb806e4c0 2024-04-25T00:11:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.856+0000 7f5baf400640 1 --2- 172.21.15.83:0/3828817550 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5b940678c0 0x7f5b94069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.856+0000 7f5baf400640 1 -- 172.21.15.83:0/3828817550 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5ba8120bf0 con 0x7f5bb806e4c0 2024-04-25T00:11:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.856+0000 7f5bae000640 1 --2- 172.21.15.83:0/3828817550 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5b940678c0 0x7f5b94069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.856+0000 7f5bae000640 1 --2- 172.21.15.83:0/3828817550 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5b940678c0 0x7f5b94069d80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80cd210 tx=0x7f5ba0009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:18.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:18.860+0000 7f5baf400640 1 -- 172.21.15.83:0/3828817550 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5ba80c4060 con 0x7f5bb806e4c0 2024-04-25T00:11:19.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.020+0000 7f5bbdc00640 1 -- 172.21.15.83:0/3828817550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5bb806ce30 con 0x7f5bb806e4c0 2024-04-25T00:11:19.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.020+0000 7f5baf400640 1 -- 172.21.15.83:0/3828817550 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f5ba8016110 con 0x7f5bb806e4c0 2024-04-25T00:11:19.023 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:11:19.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.024+0000 7f5bbdc00640 1 -- 172.21.15.83:0/3828817550 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5b940678c0 msgr2=0x7f5b94069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.024+0000 7f5bbdc00640 1 --2- 172.21.15.83:0/3828817550 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5b940678c0 0x7f5b94069d80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80cd210 tx=0x7f5ba0009110 comp rx=0 tx=0).stop 2024-04-25T00:11:19.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.024+0000 7f5bbdc00640 1 -- 172.21.15.83:0/3828817550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bb806e4c0 msgr2=0x7f5bb80cdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.024+0000 7f5bbdc00640 1 --2- 172.21.15.83:0/3828817550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bb806e4c0 0x7f5bb80cdc90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80ad040 tx=0x7f5ba801ad60 comp rx=0 tx=0).stop 2024-04-25T00:11:19.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.024+0000 7f5bbdc00640 1 -- 172.21.15.83:0/3828817550 shutdown_connections 2024-04-25T00:11:19.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.024+0000 7f5bbdc00640 1 --2- 172.21.15.83:0/3828817550 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5b940678c0 0x7f5b94069d80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.024+0000 7f5bbdc00640 1 --2- 172.21.15.83:0/3828817550 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bb81555c0 0x7f5bb815a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.024+0000 7f5bbdc00640 1 --2- 172.21.15.83:0/3828817550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb81551e0 0x7f5bb80b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.024+0000 7f5bbdc00640 1 --2- 172.21.15.83:0/3828817550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bb806e4c0 0x7f5bb80cdc90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.024+0000 7f5bbdc00640 1 -- 172.21.15.83:0/3828817550 >> 172.21.15.83:0/3828817550 conn(0x7f5bb80b1830 msgr2=0x7f5bb80b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:19.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.024+0000 7f5bbdc00640 1 -- 172.21.15.83:0/3828817550 shutdown_connections 2024-04-25T00:11:19.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.024+0000 7f5bbdc00640 1 -- 172.21.15.83:0/3828817550 wait complete. 2024-04-25T00:11:19.044 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:11:19.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.180+0000 7fe264c00640 1 Processor -- start 2024-04-25T00:11:19.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.180+0000 7fe264c00640 1 -- start start 2024-04-25T00:11:19.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.180+0000 7fe264c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe26015de90 0x7fe260071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.184+0000 7fe264c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe260162a10 con 0x7fe26006e840 2024-04-25T00:11:19.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.184+0000 7fe264c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26006dcf0 con 0x7fe26015e270 2024-04-25T00:11:19.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.184+0000 7fe264c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe2600b4460 con 0x7fe26015de90 2024-04-25T00:11:19.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.184+0000 7fe25de00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe26015de90 0x7fe260071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.184+0000 7fe25de00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe26015de90 0x7fe260071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57896/0 (socket says 172.21.15.83:57896) 2024-04-25T00:11:19.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.184+0000 7fe25de00640 1 -- 172.21.15.83:0/1426282734 learned_addr learned my addr 172.21.15.83:0/1426282734 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:19.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.184+0000 7fe25de00640 1 -- 172.21.15.83:0/1426282734 >> v1:172.21.15.86:6789/0 conn(0x7fe26015e270 legacy=0x7fe260161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.184+0000 7fe25de00640 1 -- 172.21.15.83:0/1426282734 >> v1:172.21.15.83:6789/0 conn(0x7fe26006e840 legacy=0x7fe260162300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.184+0000 7fe25de00640 1 -- 172.21.15.83:0/1426282734 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26006dfd0 con 0x7fe26015de90 2024-04-25T00:11:19.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.184+0000 7fe25de00640 1 --2- 172.21.15.83:0/1426282734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe26015de90 0x7fe260071740 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe250009e50 tx=0x7fe25001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78638413b9a1ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:19.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.184+0000 7fe25ca00640 1 -- 172.21.15.83:0/1426282734 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2500a5660 con 0x7fe26015de90 2024-04-25T00:11:19.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.184+0000 7fe25ca00640 1 -- 172.21.15.83:0/1426282734 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2500a5800 con 0x7fe26015de90 2024-04-25T00:11:19.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.184+0000 7fe25ca00640 1 -- 172.21.15.83:0/1426282734 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2500a5bd0 con 0x7fe26015de90 2024-04-25T00:11:19.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 -- 172.21.15.83:0/1426282734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe26015de90 msgr2=0x7fe260071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 --2- 172.21.15.83:0/1426282734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe26015de90 0x7fe260071740 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe250009e50 tx=0x7fe25001c9a0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 -- 172.21.15.83:0/1426282734 shutdown_connections 2024-04-25T00:11:19.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 --2- 172.21.15.83:0/1426282734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe26015de90 0x7fe260071740 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 -- 172.21.15.83:0/1426282734 >> 172.21.15.83:0/1426282734 conn(0x7fe2600b1830 msgr2=0x7fe2600b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:19.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 -- 172.21.15.83:0/1426282734 shutdown_connections 2024-04-25T00:11:19.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 -- 172.21.15.83:0/1426282734 wait complete. 2024-04-25T00:11:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 Processor -- start 2024-04-25T00:11:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 -- start start 2024-04-25T00:11:19.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26006e840 0x7fe26014d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe26015de90 0x7fe26015cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe26015e270 0x7fe26014ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2600ad860 con 0x7fe26015de90 2024-04-25T00:11:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe260072090 con 0x7fe26006e840 2024-04-25T00:11:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe264c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe26016c580 con 0x7fe26015e270 2024-04-25T00:11:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe25d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe26015de90 0x7fe26015cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe25d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe26015de90 0x7fe26015cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57902/0 (socket says 172.21.15.83:57902) 2024-04-25T00:11:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe25d400640 1 -- 172.21.15.83:0/217789609 learned_addr learned my addr 172.21.15.83:0/217789609 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe25de00640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26006e840 0x7fe26014d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe25e800640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe26015e270 0x7fe26014ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe25d400640 1 -- 172.21.15.83:0/217789609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe26015e270 msgr2=0x7fe26014ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe25d400640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe26015e270 0x7fe26014ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe25d400640 1 -- 172.21.15.83:0/217789609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26006e840 msgr2=0x7fe26014d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe25d400640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26006e840 0x7fe26014d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe25d400640 1 -- 172.21.15.83:0/217789609 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26015ea30 con 0x7fe26015de90 2024-04-25T00:11:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.188+0000 7fe25de00640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26006e840 0x7fe26014d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:19.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.192+0000 7fe25d400640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe26015de90 0x7fe26015cf00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00c450 tx=0x7fe24c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:19.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.192+0000 7fe256000640 1 -- 172.21.15.83:0/217789609 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c016020 con 0x7fe26015de90 2024-04-25T00:11:19.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.192+0000 7fe256000640 1 -- 172.21.15.83:0/217789609 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe24c0052d0 con 0x7fe26015de90 2024-04-25T00:11:19.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.192+0000 7fe264c00640 1 -- 172.21.15.83:0/217789609 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26015ed10 con 0x7fe26015de90 2024-04-25T00:11:19.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.192+0000 7fe256000640 1 -- 172.21.15.83:0/217789609 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c007260 con 0x7fe26015de90 2024-04-25T00:11:19.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.192+0000 7fe264c00640 1 -- 172.21.15.83:0/217789609 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26015f5b0 con 0x7fe26015de90 2024-04-25T00:11:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.192+0000 7fe264c00640 1 -- 172.21.15.83:0/217789609 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26015eee0 con 0x7fe26015de90 2024-04-25T00:11:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.192+0000 7fe256000640 1 -- 172.21.15.83:0/217789609 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe24c007400 con 0x7fe26015de90 2024-04-25T00:11:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.192+0000 7fe256000640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2400678c0 0x7fe240069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.192+0000 7fe256000640 1 -- 172.21.15.83:0/217789609 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe24c0859e0 con 0x7fe26015de90 2024-04-25T00:11:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.192+0000 7fe25de00640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2400678c0 0x7fe240069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.192+0000 7fe25de00640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2400678c0 0x7fe240069d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe2500ae000 tx=0x7fe2500a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.196+0000 7fe256000640 1 -- 172.21.15.83:0/217789609 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe24c020020 con 0x7fe26015de90 2024-04-25T00:11:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.348+0000 7fe264c00640 1 -- 172.21.15.83:0/217789609 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe26015f7b0 con 0x7fe26015de90 2024-04-25T00:11:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.348+0000 7fe256000640 1 -- 172.21.15.83:0/217789609 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fe26015f7b0 con 0x7fe26015de90 2024-04-25T00:11:19.352 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.352+0000 7fe264c00640 1 -- 172.21.15.83:0/217789609 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2400678c0 msgr2=0x7fe240069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.352+0000 7fe264c00640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2400678c0 0x7fe240069d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe2500ae000 tx=0x7fe2500a7070 comp rx=0 tx=0).stop 2024-04-25T00:11:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.352+0000 7fe264c00640 1 -- 172.21.15.83:0/217789609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe26015de90 msgr2=0x7fe26015cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.352+0000 7fe264c00640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe26015de90 0x7fe26015cf00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00c450 tx=0x7fe24c00c910 comp rx=0 tx=0).stop 2024-04-25T00:11:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.352+0000 7fe264c00640 1 -- 172.21.15.83:0/217789609 shutdown_connections 2024-04-25T00:11:19.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.352+0000 7fe264c00640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2400678c0 0x7fe240069d80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.352+0000 7fe264c00640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe26015e270 0x7fe26014ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.352+0000 7fe264c00640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe26015de90 0x7fe26015cf00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.352+0000 7fe264c00640 1 --2- 172.21.15.83:0/217789609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26006e840 0x7fe26014d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.352+0000 7fe264c00640 1 -- 172.21.15.83:0/217789609 >> 172.21.15.83:0/217789609 conn(0x7fe2600b1830 msgr2=0x7fe2600afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:19.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.352+0000 7fe264c00640 1 -- 172.21.15.83:0/217789609 shutdown_connections 2024-04-25T00:11:19.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.352+0000 7fe264c00640 1 -- 172.21.15.83:0/217789609 wait complete. 2024-04-25T00:11:19.374 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:11:19.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.508+0000 7f9301000640 1 Processor -- start 2024-04-25T00:11:19.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.508+0000 7f9301000640 1 -- start start 2024-04-25T00:11:19.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f9301000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f92fc06cfd0 0x7f92fc070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f9301000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f92fc06d3b0 0x7f92fc06d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f9301000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92fc0ca630 0x7f92fc0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f9301000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f92fc06e160 con 0x7f92fc06cfd0 2024-04-25T00:11:19.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f9301000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f92fc06ea10 con 0x7f92fc06d3b0 2024-04-25T00:11:19.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f9301000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f92fc06ecf0 con 0x7f92fc0ca630 2024-04-25T00:11:19.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f92f9e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f92fc06cfd0 0x7f92fc070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f92f9e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f92fc06cfd0 0x7f92fc070d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57918/0 (socket says 172.21.15.83:57918) 2024-04-25T00:11:19.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f92f9e00640 1 -- 172.21.15.83:0/1740413844 learned_addr learned my addr 172.21.15.83:0/1740413844 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:19.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f92f1400640 1 --2- 172.21.15.83:0/1740413844 >> v2:172.21.15.86:3300/0 conn(0x7f92fc06d3b0 0x7f92fc06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f92fa800640 1 --2- 172.21.15.83:0/1740413844 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92fc0ca630 0x7f92fc0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f92f9e00640 1 -- 172.21.15.83:0/1740413844 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92fc0ca630 msgr2=0x7f92fc0caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:19.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f92f9e00640 1 --2- 172.21.15.83:0/1740413844 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92fc0ca630 0x7f92fc0caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f92f9e00640 1 -- 172.21.15.83:0/1740413844 >> v2:172.21.15.86:3300/0 conn(0x7f92fc06d3b0 msgr2=0x7f92fc06d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:19.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f92f9e00640 1 --2- 172.21.15.83:0/1740413844 >> v2:172.21.15.86:3300/0 conn(0x7f92fc06d3b0 0x7f92fc06d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.512+0000 7f92f9e00640 1 -- 172.21.15.83:0/1740413844 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc06efd0 con 0x7f92fc06cfd0 2024-04-25T00:11:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f92f9e00640 1 --2- 172.21.15.83:0/1740413844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc06cfd0 0x7f92fc070d10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f92fc073d40 tx=0x7f92ec01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5570c3a0e6b54dc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f92f9400640 1 -- 172.21.15.83:0/1740413844 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec0a5670 con 0x7f92fc06cfd0 2024-04-25T00:11:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f92f9400640 1 -- 172.21.15.83:0/1740413844 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ec0a5810 con 0x7f92fc06cfd0 2024-04-25T00:11:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f92f9400640 1 -- 172.21.15.83:0/1740413844 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec0a5be0 con 0x7f92fc06cfd0 2024-04-25T00:11:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 -- 172.21.15.83:0/1740413844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc06cfd0 msgr2=0x7f92fc070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 --2- 172.21.15.83:0/1740413844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc06cfd0 0x7f92fc070d10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f92fc073d40 tx=0x7f92ec01c9f0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 -- 172.21.15.83:0/1740413844 shutdown_connections 2024-04-25T00:11:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 --2- 172.21.15.83:0/1740413844 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92fc0ca630 0x7f92fc0caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 --2- 172.21.15.83:0/1740413844 >> v2:172.21.15.86:3300/0 conn(0x7f92fc06d3b0 0x7f92fc06d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 --2- 172.21.15.83:0/1740413844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc06cfd0 0x7f92fc070d10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 -- 172.21.15.83:0/1740413844 >> 172.21.15.83:0/1740413844 conn(0x7f92fc0cfa60 msgr2=0x7f92fc0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:19.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 -- 172.21.15.83:0/1740413844 shutdown_connections 2024-04-25T00:11:19.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 -- 172.21.15.83:0/1740413844 wait complete. 2024-04-25T00:11:19.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 Processor -- start 2024-04-25T00:11:19.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 -- start start 2024-04-25T00:11:19.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92fc06cfd0 0x7f92fc1f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92fc06d3b0 0x7f92fc131270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc0ca630 0x7f92fc0c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc167b50 con 0x7f92fc0ca630 2024-04-25T00:11:19.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc145970 con 0x7f92fc06d3b0 2024-04-25T00:11:19.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f9301000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f92fc167cd0 con 0x7f92fc06cfd0 2024-04-25T00:11:19.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.516+0000 7f92fa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc0ca630 0x7f92fc0c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92fa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc0ca630 0x7f92fc0c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57924/0 (socket says 172.21.15.83:57924) 2024-04-25T00:11:19.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92fa800640 1 -- 172.21.15.83:0/2322504275 learned_addr learned my addr 172.21.15.83:0/2322504275 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:19.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92f9e00640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92fc06cfd0 0x7f92fc1f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92f1400640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92fc06d3b0 0x7f92fc131270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92fa800640 1 -- 172.21.15.83:0/2322504275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92fc06cfd0 msgr2=0x7f92fc1f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92fa800640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92fc06cfd0 0x7f92fc1f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92fa800640 1 -- 172.21.15.83:0/2322504275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92fc06d3b0 msgr2=0x7f92fc131270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92fa800640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92fc06d3b0 0x7f92fc131270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92fa800640 1 -- 172.21.15.83:0/2322504275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc1317b0 con 0x7f92fc0ca630 2024-04-25T00:11:19.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92f1400640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92fc06d3b0 0x7f92fc131270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:19.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92fa800640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc0ca630 0x7f92fc0c9300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f92f400d560 tx=0x7f92f400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:19.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92f2a00640 1 -- 172.21.15.83:0/2322504275 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f40125f0 con 0x7f92fc0ca630 2024-04-25T00:11:19.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92f2a00640 1 -- 172.21.15.83:0/2322504275 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92f4019070 con 0x7f92fc0ca630 2024-04-25T00:11:19.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92f2a00640 1 -- 172.21.15.83:0/2322504275 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f40129f0 con 0x7f92fc0ca630 2024-04-25T00:11:19.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f9301000640 1 -- 172.21.15.83:0/2322504275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92fc0c4d10 con 0x7f92fc0ca630 2024-04-25T00:11:19.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f9301000640 1 -- 172.21.15.83:0/2322504275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc0c52d0 con 0x7f92fc0ca630 2024-04-25T00:11:19.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f9301000640 1 -- 172.21.15.83:0/2322504275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92fc068230 con 0x7f92fc0ca630 2024-04-25T00:11:19.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92f2a00640 1 -- 172.21.15.83:0/2322504275 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f92f4012b90 con 0x7f92fc0ca630 2024-04-25T00:11:19.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92f2a00640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92d80678c0 0x7f92d8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.520+0000 7f92f2a00640 1 -- 172.21.15.83:0/2322504275 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f92f4086370 con 0x7f92fc0ca630 2024-04-25T00:11:19.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.524+0000 7f92f9e00640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92d80678c0 0x7f92d8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.524+0000 7f92f9e00640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92d80678c0 0x7f92d8069d80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0ae000 tx=0x7f92ec0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:19.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.524+0000 7f92f2a00640 1 -- 172.21.15.83:0/2322504275 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f92f4056b70 con 0x7f92fc0ca630 2024-04-25T00:11:19.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.696+0000 7f9301000640 1 -- 172.21.15.83:0/2322504275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f92fc0cacd0 con 0x7f92fc0ca630 2024-04-25T00:11:19.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.696+0000 7f92f2a00640 1 -- 172.21.15.83:0/2322504275 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f92f4056470 con 0x7f92fc0ca630 2024-04-25T00:11:19.699 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:11:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.700+0000 7f9301000640 1 -- 172.21.15.83:0/2322504275 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92d80678c0 msgr2=0x7f92d8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.700+0000 7f9301000640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92d80678c0 0x7f92d8069d80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0ae000 tx=0x7f92ec0a7070 comp rx=0 tx=0).stop 2024-04-25T00:11:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.700+0000 7f9301000640 1 -- 172.21.15.83:0/2322504275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc0ca630 msgr2=0x7f92fc0c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.700+0000 7f9301000640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc0ca630 0x7f92fc0c9300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f92f400d560 tx=0x7f92f400da20 comp rx=0 tx=0).stop 2024-04-25T00:11:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.700+0000 7f9301000640 1 -- 172.21.15.83:0/2322504275 shutdown_connections 2024-04-25T00:11:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.700+0000 7f9301000640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92d80678c0 0x7f92d8069d80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.700+0000 7f9301000640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc0ca630 0x7f92fc0c9300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.700+0000 7f9301000640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92fc06d3b0 0x7f92fc131270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.700+0000 7f9301000640 1 --2- 172.21.15.83:0/2322504275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92fc06cfd0 0x7f92fc1f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.700+0000 7f9301000640 1 -- 172.21.15.83:0/2322504275 >> 172.21.15.83:0/2322504275 conn(0x7f92fc0cfa60 msgr2=0x7f92fc15d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.700+0000 7f9301000640 1 -- 172.21.15.83:0/2322504275 shutdown_connections 2024-04-25T00:11:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.700+0000 7f9301000640 1 -- 172.21.15.83:0/2322504275 wait complete. 2024-04-25T00:11:19.721 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:11:19.721 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:19.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:19.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b9000640 1 Processor -- start 2024-04-25T00:11:19.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b9000640 1 -- start start 2024-04-25T00:11:19.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b9000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2b40afac0 0x7fe2b40afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b9000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2b406bbf0 0x7fe2b40ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b9000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2b406df20 con 0x7fe2b40d16b0 2024-04-25T00:11:19.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b9000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe2b40b03e0 con 0x7fe2b406bbf0 2024-04-25T00:11:19.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b9000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b415e430 con 0x7fe2b40afac0 2024-04-25T00:11:19.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2b406bbf0 0x7fe2b40ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b1e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2b40afac0 0x7fe2b40afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2a9400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe2b40d16b0 0x7fe2b40ada10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:56932/0 (socket says 172.21.15.83:56932) 2024-04-25T00:11:19.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2a9400640 1 -- 172.21.15.83:0/2081008329 learned_addr learned my addr 172.21.15.83:0/2081008329 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:19.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b1400640 1 -- 172.21.15.83:0/2081008329 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 65431259 0 0) 0x7fe2b406df20 con 0x7fe2b40d16b0 2024-04-25T00:11:19.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b1400640 1 -- 172.21.15.83:0/2081008329 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe298003530 con 0x7fe2b40d16b0 2024-04-25T00:11:19.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b2800640 1 -- 172.21.15.83:0/2081008329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2b40afac0 msgr2=0x7fe2b40afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:19.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b2800640 1 --2- 172.21.15.83:0/2081008329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2b40afac0 0x7fe2b40afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b2800640 1 -- 172.21.15.83:0/2081008329 >> v1:172.21.15.86:6789/0 conn(0x7fe2b40d16b0 legacy=0x7fe2b40ada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b2800640 1 -- 172.21.15.83:0/2081008329 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b406ec00 con 0x7fe2b406bbf0 2024-04-25T00:11:19.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.908+0000 7fe2b1e00640 1 --2- 172.21.15.83:0/2081008329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2b40afac0 0x7fe2b40afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:19.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.912+0000 7fe2b2800640 1 --2- 172.21.15.83:0/2081008329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b406bbf0 0x7fe2b40ae120 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0097c0 tx=0x7fe2ac01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=191e24562f4f1fec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:19.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.912+0000 7fe2b1400640 1 -- 172.21.15.83:0/2081008329 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0a7070 con 0x7fe2b406bbf0 2024-04-25T00:11:19.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.912+0000 7fe2b1400640 1 -- 172.21.15.83:0/2081008329 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2ac0b0070 con 0x7fe2b406bbf0 2024-04-25T00:11:19.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.912+0000 7fe2b1400640 1 -- 172.21.15.83:0/2081008329 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0ab890 con 0x7fe2b406bbf0 2024-04-25T00:11:19.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.912+0000 7fe2b9000640 1 -- 172.21.15.83:0/2081008329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b406bbf0 msgr2=0x7fe2b40ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.912+0000 7fe2b9000640 1 --2- 172.21.15.83:0/2081008329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b406bbf0 0x7fe2b40ae120 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0097c0 tx=0x7fe2ac01c470 comp rx=0 tx=0).stop 2024-04-25T00:11:19.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.912+0000 7fe2b9000640 1 -- 172.21.15.83:0/2081008329 shutdown_connections 2024-04-25T00:11:19.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.912+0000 7fe2b9000640 1 --2- 172.21.15.83:0/2081008329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b406bbf0 0x7fe2b40ae120 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.912+0000 7fe2b9000640 1 --2- 172.21.15.83:0/2081008329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2b40afac0 0x7fe2b40afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.912+0000 7fe2b9000640 1 -- 172.21.15.83:0/2081008329 >> 172.21.15.83:0/2081008329 conn(0x7fe2b40d0b90 msgr2=0x7fe2b40d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:19.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.912+0000 7fe2b9000640 1 -- 172.21.15.83:0/2081008329 shutdown_connections 2024-04-25T00:11:19.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.912+0000 7fe2b9000640 1 -- 172.21.15.83:0/2081008329 wait complete. 2024-04-25T00:11:19.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b9000640 1 Processor -- start 2024-04-25T00:11:19.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b9000640 1 -- start start 2024-04-25T00:11:19.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b9000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2b406bbf0 0x7fe2b4142f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b40afac0 0x7fe2b4152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b40d16b0 0x7fe2b4155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b9000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b406f060 con 0x7fe2b40d16b0 2024-04-25T00:11:19.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b9000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b406eee0 con 0x7fe2b40afac0 2024-04-25T00:11:19.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b9000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b416ac40 con 0x7fe2b406bbf0 2024-04-25T00:11:19.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b40d16b0 0x7fe2b4155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b40d16b0 0x7fe2b4155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57926/0 (socket says 172.21.15.83:57926) 2024-04-25T00:11:19.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b2800640 1 -- 172.21.15.83:0/791432529 learned_addr learned my addr 172.21.15.83:0/791432529 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:19.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2a9400640 1 --2- 172.21.15.83:0/791432529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b40afac0 0x7fe2b4152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b1e00640 1 --2- 172.21.15.83:0/791432529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2b406bbf0 0x7fe2b4142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b2800640 1 -- 172.21.15.83:0/791432529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2b406bbf0 msgr2=0x7fe2b4142f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b2800640 1 --2- 172.21.15.83:0/791432529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2b406bbf0 0x7fe2b4142f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b2800640 1 -- 172.21.15.83:0/791432529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b40afac0 msgr2=0x7fe2b4152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:19.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b2800640 1 --2- 172.21.15.83:0/791432529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b40afac0 0x7fe2b4152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:19.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b2800640 1 -- 172.21.15.83:0/791432529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b4152ee0 con 0x7fe2b40d16b0 2024-04-25T00:11:19.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b2800640 1 --2- 172.21.15.83:0/791432529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b40d16b0 0x7fe2b4155910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac01a0a0 tx=0x7fe2ac019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:19.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2aaa00640 1 -- 172.21.15.83:0/791432529 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0bf410 con 0x7fe2b40d16b0 2024-04-25T00:11:19.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2aaa00640 1 -- 172.21.15.83:0/791432529 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2ac0bf5b0 con 0x7fe2b40d16b0 2024-04-25T00:11:19.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b9000640 1 -- 172.21.15.83:0/791432529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b41531c0 con 0x7fe2b40d16b0 2024-04-25T00:11:19.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2aaa00640 1 -- 172.21.15.83:0/791432529 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0bf9a0 con 0x7fe2b40d16b0 2024-04-25T00:11:19.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.916+0000 7fe2b9000640 1 -- 172.21.15.83:0/791432529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b4143490 con 0x7fe2b40d16b0 2024-04-25T00:11:19.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.920+0000 7fe2b9000640 1 -- 172.21.15.83:0/791432529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b4154f30 con 0x7fe2b40d16b0 2024-04-25T00:11:19.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.920+0000 7fe2aaa00640 1 -- 172.21.15.83:0/791432529 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe2ac0bd020 con 0x7fe2b40d16b0 2024-04-25T00:11:19.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.920+0000 7fe2aaa00640 1 --2- 172.21.15.83:0/791432529 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2900678c0 0x7fe290069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:19.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.920+0000 7fe2aaa00640 1 -- 172.21.15.83:0/791432529 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe2ac120580 con 0x7fe2b40d16b0 2024-04-25T00:11:19.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.920+0000 7fe2b1e00640 1 --2- 172.21.15.83:0/791432529 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2900678c0 0x7fe290069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:19.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.920+0000 7fe2b1e00640 1 --2- 172.21.15.83:0/791432529 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2900678c0 0x7fe290069d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009bd0 tx=0x7fe2a400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:19.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:19.924+0000 7fe2aaa00640 1 -- 172.21.15.83:0/791432529 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe2b4154f30 con 0x7fe2b40d16b0 2024-04-25T00:11:20.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.080+0000 7fe2b9000640 1 -- 172.21.15.83:0/791432529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2b40737d0 con 0x7fe2b40d16b0 2024-04-25T00:11:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.084+0000 7fe2aaa00640 1 -- 172.21.15.83:0/791432529 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7fe2ac016090 con 0x7fe2b40d16b0 2024-04-25T00:11:20.087 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:20.088 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.088+0000 7fe2b9000640 1 -- 172.21.15.83:0/791432529 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2900678c0 msgr2=0x7fe290069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.088+0000 7fe2b9000640 1 --2- 172.21.15.83:0/791432529 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2900678c0 0x7fe290069d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009bd0 tx=0x7fe2a400b040 comp rx=0 tx=0).stop 2024-04-25T00:11:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.088+0000 7fe2b9000640 1 -- 172.21.15.83:0/791432529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b40d16b0 msgr2=0x7fe2b4155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.088+0000 7fe2b9000640 1 --2- 172.21.15.83:0/791432529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b40d16b0 0x7fe2b4155910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac01a0a0 tx=0x7fe2ac019210 comp rx=0 tx=0).stop 2024-04-25T00:11:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.088+0000 7fe2b9000640 1 -- 172.21.15.83:0/791432529 shutdown_connections 2024-04-25T00:11:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.088+0000 7fe2b9000640 1 --2- 172.21.15.83:0/791432529 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2900678c0 0x7fe290069d80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.088+0000 7fe2b9000640 1 --2- 172.21.15.83:0/791432529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b40d16b0 0x7fe2b4155910 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.088+0000 7fe2b9000640 1 --2- 172.21.15.83:0/791432529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b40afac0 0x7fe2b4152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.088+0000 7fe2b9000640 1 --2- 172.21.15.83:0/791432529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2b406bbf0 0x7fe2b4142f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.088+0000 7fe2b9000640 1 -- 172.21.15.83:0/791432529 >> 172.21.15.83:0/791432529 conn(0x7fe2b40d0b90 msgr2=0x7fe2b415be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:20.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.088+0000 7fe2b9000640 1 -- 172.21.15.83:0/791432529 shutdown_connections 2024-04-25T00:11:20.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.088+0000 7fe2b9000640 1 -- 172.21.15.83:0/791432529 wait complete. 2024-04-25T00:11:20.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:11:04.621429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:11:05.189716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:11:08.902215+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:11:20.109 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:11:20.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.244+0000 7f328fe00640 1 Processor -- start 2024-04-25T00:11:20.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.244+0000 7f328fe00640 1 -- start start 2024-04-25T00:11:20.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.244+0000 7f328fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f32881551e0 0x7f328815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.244+0000 7f328fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32881555c0 0x7f3288071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.244+0000 7f328fe00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f328815c560 con 0x7f328806e4c0 2024-04-25T00:11:20.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.244+0000 7f328fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3288072460 con 0x7f32881551e0 2024-04-25T00:11:20.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.244+0000 7f328fe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f328815c7d0 con 0x7f32881555c0 2024-04-25T00:11:20.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.244+0000 7f328de00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f328806e4c0 0x7f328815e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57980/0 (socket says 172.21.15.83:57980) 2024-04-25T00:11:20.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.244+0000 7f328de00640 1 -- 172.21.15.83:0/883502352 learned_addr learned my addr 172.21.15.83:0/883502352 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:20.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.244+0000 7f328ca00640 1 --2- 172.21.15.83:0/883502352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32881555c0 0x7f3288071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.244+0000 7f328d400640 1 --2- 172.21.15.83:0/883502352 >> v2:172.21.15.86:3300/0 conn(0x7f32881551e0 0x7f328815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.244+0000 7f3287e00640 1 -- 172.21.15.83:0/883502352 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 783736804 0 0) 0x7f328815c560 con 0x7f328806e4c0 2024-04-25T00:11:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 -- 172.21.15.83:0/883502352 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3270005740 con 0x7f328806e4c0 2024-04-25T00:11:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 -- 172.21.15.83:0/883502352 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3260669712 0 0) 0x7f3270005740 con 0x7f328806e4c0 2024-04-25T00:11:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 -- 172.21.15.83:0/883502352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32881555c0 msgr2=0x7f3288071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 --2- 172.21.15.83:0/883502352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32881555c0 0x7f3288071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 -- 172.21.15.83:0/883502352 >> v2:172.21.15.86:3300/0 conn(0x7f32881551e0 msgr2=0x7f328815bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 --2- 172.21.15.83:0/883502352 >> v2:172.21.15.86:3300/0 conn(0x7f32881551e0 0x7f328815bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 -- 172.21.15.83:0/883502352 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328815cab0 con 0x7f328806e4c0 2024-04-25T00:11:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 -- 172.21.15.83:0/883502352 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f3280001a00 con 0x7f328806e4c0 2024-04-25T00:11:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 -- 172.21.15.83:0/883502352 >> v1:172.21.15.83:6789/0 conn(0x7f328806e4c0 legacy=0x7f328815e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 --2- 172.21.15.83:0/883502352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3270005c40 0x7f3270006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 --2- 172.21.15.83:0/883502352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3270006a20 0x7f3270006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 --2- 172.21.15.83:0/883502352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32700073e0 0x7f327000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 -- 172.21.15.83:0/883502352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3288072460 con 0x7f32700073e0 2024-04-25T00:11:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 -- 172.21.15.83:0/883502352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f328815c7d0 con 0x7f3270006a20 2024-04-25T00:11:20.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f3287e00640 1 -- 172.21.15.83:0/883502352 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f327000e120 con 0x7f3270005c40 2024-04-25T00:11:20.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f328de00640 1 --2- 172.21.15.83:0/883502352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32700073e0 0x7f327000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f328d400640 1 --2- 172.21.15.83:0/883502352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3270005c40 0x7f3270006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f328ca00640 1 --2- 172.21.15.83:0/883502352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3270006a20 0x7f3270006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f328de00640 1 -- 172.21.15.83:0/883502352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3270005c40 msgr2=0x7f3270006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:20.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f328de00640 1 --2- 172.21.15.83:0/883502352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3270005c40 0x7f3270006060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f328de00640 1 -- 172.21.15.83:0/883502352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3270006a20 msgr2=0x7f3270006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:20.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f328de00640 1 --2- 172.21.15.83:0/883502352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3270006a20 0x7f3270006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f328de00640 1 -- 172.21.15.83:0/883502352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3270005740 con 0x7f32700073e0 2024-04-25T00:11:20.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.248+0000 7f328de00640 1 --2- 172.21.15.83:0/883502352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32700073e0 0x7f327000dbe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f328000a950 tx=0x7f328001d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db2959fb830e2d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f3287e00640 1 -- 172.21.15.83:0/883502352 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328000a4c0 con 0x7f32700073e0 2024-04-25T00:11:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f3287e00640 1 -- 172.21.15.83:0/883502352 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32800072f0 con 0x7f32700073e0 2024-04-25T00:11:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f3287e00640 1 -- 172.21.15.83:0/883502352 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280007720 con 0x7f32700073e0 2024-04-25T00:11:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328fe00640 1 -- 172.21.15.83:0/883502352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32700073e0 msgr2=0x7f327000dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328fe00640 1 --2- 172.21.15.83:0/883502352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32700073e0 0x7f327000dbe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f328000a950 tx=0x7f328001d3e0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328de00640 1 -- 172.21.15.83:0/883502352 reap_dead start 2024-04-25T00:11:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328fe00640 1 -- 172.21.15.83:0/883502352 shutdown_connections 2024-04-25T00:11:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328fe00640 1 -- 172.21.15.83:0/883502352 >> 172.21.15.83:0/883502352 conn(0x7f32880b1830 msgr2=0x7f32880b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328fe00640 1 -- 172.21.15.83:0/883502352 shutdown_connections 2024-04-25T00:11:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328fe00640 1 -- 172.21.15.83:0/883502352 wait complete. 2024-04-25T00:11:20.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328fe00640 1 Processor -- start 2024-04-25T00:11:20.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328fe00640 1 -- start start 2024-04-25T00:11:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700073e0 0x7f3288143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32881433e0 0x7f32880b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3288143840 0x7f32880b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f328815e160 con 0x7f3288143840 2024-04-25T00:11:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f328815dfe0 con 0x7f32881433e0 2024-04-25T00:11:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328fe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f328815e2e0 con 0x7f32700073e0 2024-04-25T00:11:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3288143840 0x7f32880b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3288143840 0x7f32880b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57938/0 (socket says 172.21.15.83:57938) 2024-04-25T00:11:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328de00640 1 -- 172.21.15.83:0/3312669112 learned_addr learned my addr 172.21.15.83:0/3312669112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.252+0000 7f328d400640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700073e0 0x7f3288143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f328ca00640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32881433e0 0x7f32880b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f328de00640 1 -- 172.21.15.83:0/3312669112 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700073e0 msgr2=0x7f3288143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f328de00640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700073e0 0x7f3288143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f328de00640 1 -- 172.21.15.83:0/3312669112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32881433e0 msgr2=0x7f32880b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f328de00640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32881433e0 0x7f32880b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f328de00640 1 -- 172.21.15.83:0/3312669112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3288151a30 con 0x7f3288143840 2024-04-25T00:11:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f328ca00640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32881433e0 0x7f32880b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f328de00640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3288143840 0x7f32880b0e40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f328000a950 tx=0x7f3270007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f3285600640 1 -- 172.21.15.83:0/3312669112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328000a4c0 con 0x7f3288143840 2024-04-25T00:11:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f3285600640 1 -- 172.21.15.83:0/3312669112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32800011d0 con 0x7f3288143840 2024-04-25T00:11:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f3285600640 1 -- 172.21.15.83:0/3312669112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280002010 con 0x7f3288143840 2024-04-25T00:11:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f328fe00640 1 -- 172.21.15.83:0/3312669112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3288151d10 con 0x7f3288143840 2024-04-25T00:11:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f328fe00640 1 -- 172.21.15.83:0/3312669112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32881522d0 con 0x7f3288143840 2024-04-25T00:11:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f328fe00640 1 -- 172.21.15.83:0/3312669112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f328804f510 con 0x7f3288143840 2024-04-25T00:11:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f3285600640 1 -- 172.21.15.83:0/3312669112 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3280004b70 con 0x7f3288143840 2024-04-25T00:11:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f3285600640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f32680678f0 0x7f3268069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f3285600640 1 -- 172.21.15.83:0/3312669112 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f328011fbe0 con 0x7f3288143840 2024-04-25T00:11:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.256+0000 7f328d400640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f32680678f0 0x7f3268069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.260+0000 7f328d400640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f32680678f0 0x7f3268069db0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f327c009be0 tx=0x7f327c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.260+0000 7f3285600640 1 -- 172.21.15.83:0/3312669112 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3280017150 con 0x7f3288143840 2024-04-25T00:11:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.416+0000 7f328fe00640 1 -- 172.21.15.83:0/3312669112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f328815a000 con 0x7f3288143840 2024-04-25T00:11:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.416+0000 7f3285600640 1 -- 172.21.15.83:0/3312669112 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f32800f02b0 con 0x7f3288143840 2024-04-25T00:11:20.420 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:11:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.420+0000 7f328fe00640 1 -- 172.21.15.83:0/3312669112 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f32680678f0 msgr2=0x7f3268069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.420+0000 7f328fe00640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f32680678f0 0x7f3268069db0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f327c009be0 tx=0x7f327c009190 comp rx=0 tx=0).stop 2024-04-25T00:11:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.420+0000 7f328fe00640 1 -- 172.21.15.83:0/3312669112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3288143840 msgr2=0x7f32880b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.420+0000 7f328fe00640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3288143840 0x7f32880b0e40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f328000a950 tx=0x7f3270007a50 comp rx=0 tx=0).stop 2024-04-25T00:11:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.420+0000 7f328fe00640 1 -- 172.21.15.83:0/3312669112 shutdown_connections 2024-04-25T00:11:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.420+0000 7f328fe00640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f32680678f0 0x7f3268069db0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.420+0000 7f328fe00640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3288143840 0x7f32880b0e40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.420+0000 7f328fe00640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32881433e0 0x7f32880b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.420+0000 7f328fe00640 1 --2- 172.21.15.83:0/3312669112 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700073e0 0x7f3288143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.420+0000 7f328fe00640 1 -- 172.21.15.83:0/3312669112 >> 172.21.15.83:0/3312669112 conn(0x7f32880b1830 msgr2=0x7f328815a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.420+0000 7f328fe00640 1 -- 172.21.15.83:0/3312669112 shutdown_connections 2024-04-25T00:11:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.420+0000 7f328fe00640 1 -- 172.21.15.83:0/3312669112 wait complete. 2024-04-25T00:11:20.441 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:11:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.576+0000 7f2881e00640 1 Processor -- start 2024-04-25T00:11:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.576+0000 7f2881e00640 1 -- start start 2024-04-25T00:11:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f2881e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f287c1555c0 0x7f287c15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f2881e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f287c06e4c0 0x7f287c06e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f2881e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f287c15c560 con 0x7f287c1551e0 2024-04-25T00:11:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f2881e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f287c06ee70 con 0x7f287c06e4c0 2024-04-25T00:11:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f2881e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f287c072460 con 0x7f287c1555c0 2024-04-25T00:11:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f287b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f287c06e4c0 0x7f287c06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f287aa00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f287c1551e0 0x7f287c071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57984/0 (socket says 172.21.15.83:57984) 2024-04-25T00:11:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f287b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f287c06e4c0 0x7f287c06e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57946/0 (socket says 172.21.15.83:57946) 2024-04-25T00:11:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f287aa00640 1 -- 172.21.15.83:0/2347478855 learned_addr learned my addr 172.21.15.83:0/2347478855 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f2872000640 1 --2- 172.21.15.83:0/2347478855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f287c1555c0 0x7f287c15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f287b400640 1 -- 172.21.15.83:0/2347478855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f287c1555c0 msgr2=0x7f287c15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f287b400640 1 --2- 172.21.15.83:0/2347478855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f287c1555c0 0x7f287c15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f287b400640 1 -- 172.21.15.83:0/2347478855 >> v1:172.21.15.83:6789/0 conn(0x7f287c1551e0 legacy=0x7f287c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f287b400640 1 -- 172.21.15.83:0/2347478855 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c15c7d0 con 0x7f287c06e4c0 2024-04-25T00:11:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.580+0000 7f287a000640 1 -- 172.21.15.83:0/2347478855 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 484908933 0 0) 0x7f287c15c560 con 0x7f287c1551e0 2024-04-25T00:11:20.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f287b400640 1 --2- 172.21.15.83:0/2347478855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c06e4c0 0x7f287c06e930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2874009d90 tx=0x7f287401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d582c30ea1a03930 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:20.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f287a000640 1 -- 172.21.15.83:0/2347478855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28740b2070 con 0x7f287c06e4c0 2024-04-25T00:11:20.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f287a000640 1 -- 172.21.15.83:0/2347478855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28740a5940 con 0x7f287c06e4c0 2024-04-25T00:11:20.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f287a000640 1 -- 172.21.15.83:0/2347478855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28740a5d10 con 0x7f287c06e4c0 2024-04-25T00:11:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 -- 172.21.15.83:0/2347478855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c06e4c0 msgr2=0x7f287c06e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 --2- 172.21.15.83:0/2347478855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c06e4c0 0x7f287c06e930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2874009d90 tx=0x7f287401c970 comp rx=0 tx=0).stop 2024-04-25T00:11:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 -- 172.21.15.83:0/2347478855 shutdown_connections 2024-04-25T00:11:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 --2- 172.21.15.83:0/2347478855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c06e4c0 0x7f287c06e930 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 --2- 172.21.15.83:0/2347478855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f287c1555c0 0x7f287c15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 -- 172.21.15.83:0/2347478855 >> 172.21.15.83:0/2347478855 conn(0x7f287c0b1830 msgr2=0x7f287c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 -- 172.21.15.83:0/2347478855 shutdown_connections 2024-04-25T00:11:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 -- 172.21.15.83:0/2347478855 wait complete. 2024-04-25T00:11:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 Processor -- start 2024-04-25T00:11:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 -- start start 2024-04-25T00:11:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f287c06e4c0 0x7f287c140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c1551e0 0x7f287c157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c1555c0 0x7f287c0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f287c15db70 con 0x7f287c1551e0 2024-04-25T00:11:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f287c15d9f0 con 0x7f287c1555c0 2024-04-25T00:11:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2881e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f287c15dcf0 con 0x7f287c06e4c0 2024-04-25T00:11:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2872000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c1551e0 0x7f287c157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2872000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c1551e0 0x7f287c157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57954/0 (socket says 172.21.15.83:57954) 2024-04-25T00:11:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f2872000640 1 -- 172.21.15.83:0/47780372 learned_addr learned my addr 172.21.15.83:0/47780372 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.584+0000 7f287b400640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c1555c0 0x7f287c0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f287aa00640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f287c06e4c0 0x7f287c140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2872000640 1 -- 172.21.15.83:0/47780372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f287c06e4c0 msgr2=0x7f287c140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2872000640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f287c06e4c0 0x7f287c140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2872000640 1 -- 172.21.15.83:0/47780372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c1555c0 msgr2=0x7f287c0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2872000640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c1555c0 0x7f287c0cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2872000640 1 -- 172.21.15.83:0/47780372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c158520 con 0x7f287c1551e0 2024-04-25T00:11:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f287b400640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c1555c0 0x7f287c0cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2872000640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c1551e0 0x7f287c157bb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f286400d5f0 tx=0x7f286400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2873e00640 1 -- 172.21.15.83:0/47780372 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28640125f0 con 0x7f287c1551e0 2024-04-25T00:11:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2873e00640 1 -- 172.21.15.83:0/47780372 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2864019070 con 0x7f287c1551e0 2024-04-25T00:11:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2881e00640 1 -- 172.21.15.83:0/47780372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287c158800 con 0x7f287c1551e0 2024-04-25T00:11:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2873e00640 1 -- 172.21.15.83:0/47780372 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28640129b0 con 0x7f287c1551e0 2024-04-25T00:11:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2881e00640 1 -- 172.21.15.83:0/47780372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f287c155ef0 con 0x7f287c1551e0 2024-04-25T00:11:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2881e00640 1 -- 172.21.15.83:0/47780372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f287c04f510 con 0x7f287c1551e0 2024-04-25T00:11:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2873e00640 1 -- 172.21.15.83:0/47780372 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2864012b50 con 0x7f287c1551e0 2024-04-25T00:11:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2873e00640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28580678c0 0x7f2858069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f2873e00640 1 -- 172.21.15.83:0/47780372 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2864086380 con 0x7f287c1551e0 2024-04-25T00:11:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.588+0000 7f287aa00640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28580678c0 0x7f2858069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.592+0000 7f287aa00640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28580678c0 0x7f2858069d80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f286c000f70 tx=0x7f286c007b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.592+0000 7f2873e00640 1 -- 172.21.15.83:0/47780372 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2864056bf0 con 0x7f287c1551e0 2024-04-25T00:11:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.752+0000 7f2881e00640 1 -- 172.21.15.83:0/47780372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f287c0cc670 con 0x7f287c1551e0 2024-04-25T00:11:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.752+0000 7f2873e00640 1 -- 172.21.15.83:0/47780372 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f28640564f0 con 0x7f287c1551e0 2024-04-25T00:11:20.754 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.752+0000 7f2881e00640 1 -- 172.21.15.83:0/47780372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28580678c0 msgr2=0x7f2858069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.752+0000 7f2881e00640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28580678c0 0x7f2858069d80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f286c000f70 tx=0x7f286c007b60 comp rx=0 tx=0).stop 2024-04-25T00:11:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.752+0000 7f2881e00640 1 -- 172.21.15.83:0/47780372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c1551e0 msgr2=0x7f287c157bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.752+0000 7f2881e00640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c1551e0 0x7f287c157bb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f286400d5f0 tx=0x7f286400dab0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.752+0000 7f2881e00640 1 -- 172.21.15.83:0/47780372 shutdown_connections 2024-04-25T00:11:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.752+0000 7f2881e00640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28580678c0 0x7f2858069d80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.752+0000 7f2881e00640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c1555c0 0x7f287c0cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.752+0000 7f2881e00640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c1551e0 0x7f287c157bb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.752+0000 7f2881e00640 1 --2- 172.21.15.83:0/47780372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f287c06e4c0 0x7f287c140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.752+0000 7f2881e00640 1 -- 172.21.15.83:0/47780372 >> 172.21.15.83:0/47780372 conn(0x7f287c0b1830 msgr2=0x7f287c0aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.752+0000 7f2881e00640 1 -- 172.21.15.83:0/47780372 shutdown_connections 2024-04-25T00:11:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.752+0000 7f2881e00640 1 -- 172.21.15.83:0/47780372 wait complete. 2024-04-25T00:11:20.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:11:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.908+0000 7fb986000640 1 Processor -- start 2024-04-25T00:11:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb986000640 1 -- start start 2024-04-25T00:11:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb986000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb980159810 0x7fb9800cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb986000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb98006e950 0x7fb98015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb986000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9800ce000 con 0x7fb98015fe50 2024-04-25T00:11:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb986000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb98006e210 con 0x7fb98006e950 2024-04-25T00:11:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb986000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb98006da00 con 0x7fb980159810 2024-04-25T00:11:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97fe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb98015fe50 0x7fb98015c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57988/0 (socket says 172.21.15.83:57988) 2024-04-25T00:11:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97fe00640 1 -- 172.21.15.83:0/1514457009 learned_addr learned my addr 172.21.15.83:0/1514457009 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:20.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb976a00640 1 --2- 172.21.15.83:0/1514457009 >> v2:172.21.15.86:3300/0 conn(0x7fb98006e950 0x7fb98015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97f400640 1 --2- 172.21.15.83:0/1514457009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb980159810 0x7fb9800cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 -- 172.21.15.83:0/1514457009 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3217007088 0 0) 0x7fb9800ce000 con 0x7fb98015fe50 2024-04-25T00:11:20.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 -- 172.21.15.83:0/1514457009 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb964003530 con 0x7fb98015fe50 2024-04-25T00:11:20.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 -- 172.21.15.83:0/1514457009 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3169362683 0 0) 0x7fb964003530 con 0x7fb98015fe50 2024-04-25T00:11:20.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 -- 172.21.15.83:0/1514457009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb980159810 msgr2=0x7fb9800cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:20.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 --2- 172.21.15.83:0/1514457009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb980159810 0x7fb9800cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 -- 172.21.15.83:0/1514457009 >> v2:172.21.15.86:3300/0 conn(0x7fb98006e950 msgr2=0x7fb98015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:20.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 --2- 172.21.15.83:0/1514457009 >> v2:172.21.15.86:3300/0 conn(0x7fb98006e950 0x7fb98015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 -- 172.21.15.83:0/1514457009 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98006eec0 con 0x7fb98015fe50 2024-04-25T00:11:20.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 -- 172.21.15.83:0/1514457009 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fb978001680 con 0x7fb98015fe50 2024-04-25T00:11:20.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 -- 172.21.15.83:0/1514457009 >> v1:172.21.15.83:6789/0 conn(0x7fb98015fe50 legacy=0x7fb98015c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 --2- 172.21.15.83:0/1514457009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb980159810 0x7fb964003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 --2- 172.21.15.83:0/1514457009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb964004000 0x7fb964004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 --2- 172.21.15.83:0/1514457009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb964004d50 0x7fb9640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 -- 172.21.15.83:0/1514457009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb98006e210 con 0x7fb964004000 2024-04-25T00:11:20.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 -- 172.21.15.83:0/1514457009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb98006da00 con 0x7fb964004d50 2024-04-25T00:11:20.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97ea00640 1 -- 172.21.15.83:0/1514457009 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb96400bad0 con 0x7fb980159810 2024-04-25T00:11:20.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb976a00640 1 --2- 172.21.15.83:0/1514457009 >> v2:172.21.15.86:3300/0 conn(0x7fb98006e950 0x7fb98015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:20.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb976a00640 1 --2- 172.21.15.83:0/1514457009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb964004000 0x7fb964004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97f400640 1 --2- 172.21.15.83:0/1514457009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb980159810 0x7fb964003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb97fe00640 1 --2- 172.21.15.83:0/1514457009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb964004d50 0x7fb9640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb976a00640 1 -- 172.21.15.83:0/1514457009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb980159810 msgr2=0x7fb964003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:20.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb976a00640 1 --2- 172.21.15.83:0/1514457009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb980159810 0x7fb964003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb976a00640 1 -- 172.21.15.83:0/1514457009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb964004d50 msgr2=0x7fb9640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:20.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb976a00640 1 --2- 172.21.15.83:0/1514457009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb964004d50 0x7fb9640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.912+0000 7fb976a00640 1 -- 172.21.15.83:0/1514457009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb964003530 con 0x7fb964004000 2024-04-25T00:11:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.916+0000 7fb976a00640 1 --2- 172.21.15.83:0/1514457009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb964004000 0x7fb964004810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb968009c90 tx=0x7fb96801cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82486b6cb821482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.916+0000 7fb97ea00640 1 -- 172.21.15.83:0/1514457009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968009860 con 0x7fb964004000 2024-04-25T00:11:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.916+0000 7fb97ea00640 1 -- 172.21.15.83:0/1514457009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb968004020 con 0x7fb964004000 2024-04-25T00:11:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.916+0000 7fb97ea00640 1 -- 172.21.15.83:0/1514457009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968004450 con 0x7fb964004000 2024-04-25T00:11:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.916+0000 7fb986000640 1 -- 172.21.15.83:0/1514457009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb964004000 msgr2=0x7fb964004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.916+0000 7fb986000640 1 --2- 172.21.15.83:0/1514457009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb964004000 0x7fb964004810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb968009c90 tx=0x7fb96801cc70 comp rx=0 tx=0).stop 2024-04-25T00:11:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.916+0000 7fb97fe00640 1 -- 172.21.15.83:0/1514457009 reap_dead start 2024-04-25T00:11:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.916+0000 7fb986000640 1 -- 172.21.15.83:0/1514457009 shutdown_connections 2024-04-25T00:11:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.916+0000 7fb986000640 1 -- 172.21.15.83:0/1514457009 >> 172.21.15.83:0/1514457009 conn(0x7fb9800b1830 msgr2=0x7fb9800b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.916+0000 7fb986000640 1 -- 172.21.15.83:0/1514457009 shutdown_connections 2024-04-25T00:11:20.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.916+0000 7fb986000640 1 -- 172.21.15.83:0/1514457009 wait complete. 2024-04-25T00:11:20.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb986000640 1 Processor -- start 2024-04-25T00:11:20.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb986000640 1 -- start start 2024-04-25T00:11:20.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb986000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb964004000 0x7fb9800b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb986000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb980148c20 0x7fb9800ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb986000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9801478b0 0x7fb980147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb986000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb980070f20 con 0x7fb980148c20 2024-04-25T00:11:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb986000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb980070da0 con 0x7fb9801478b0 2024-04-25T00:11:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb986000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb9800cc8b0 con 0x7fb964004000 2024-04-25T00:11:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb976a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb980148c20 0x7fb9800ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb976a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb980148c20 0x7fb9800ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57964/0 (socket says 172.21.15.83:57964) 2024-04-25T00:11:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb976a00640 1 -- 172.21.15.83:0/2773040062 learned_addr learned my addr 172.21.15.83:0/2773040062 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb97f400640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb964004000 0x7fb9800b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb97fe00640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9801478b0 0x7fb980147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb976a00640 1 -- 172.21.15.83:0/2773040062 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb964004000 msgr2=0x7fb9800b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb976a00640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb964004000 0x7fb9800b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb976a00640 1 -- 172.21.15.83:0/2773040062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9801478b0 msgr2=0x7fb980147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb976a00640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9801478b0 0x7fb980147d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb976a00640 1 -- 172.21.15.83:0/2773040062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9801e7990 con 0x7fb980148c20 2024-04-25T00:11:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb97fe00640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9801478b0 0x7fb980147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb97f400640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb964004000 0x7fb9800b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb976a00640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb980148c20 0x7fb9800ce700 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb96801caa0 tx=0x7fb964005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb977e00640 1 -- 172.21.15.83:0/2773040062 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9680ba020 con 0x7fb980148c20 2024-04-25T00:11:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb977e00640 1 -- 172.21.15.83:0/2773040062 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb968005ba0 con 0x7fb980148c20 2024-04-25T00:11:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb977e00640 1 -- 172.21.15.83:0/2773040062 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9680a5820 con 0x7fb980148c20 2024-04-25T00:11:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb986000640 1 -- 172.21.15.83:0/2773040062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9801e7c70 con 0x7fb980148c20 2024-04-25T00:11:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb986000640 1 -- 172.21.15.83:0/2773040062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9801e8230 con 0x7fb980148c20 2024-04-25T00:11:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.920+0000 7fb986000640 1 -- 172.21.15.83:0/2773040062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98004f510 con 0x7fb980148c20 2024-04-25T00:11:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.924+0000 7fb977e00640 1 -- 172.21.15.83:0/2773040062 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb9680bd020 con 0x7fb980148c20 2024-04-25T00:11:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.924+0000 7fb977e00640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb95c0678c0 0x7fb95c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.924+0000 7fb977e00640 1 -- 172.21.15.83:0/2773040062 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb968120330 con 0x7fb980148c20 2024-04-25T00:11:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.924+0000 7fb97f400640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb95c0678c0 0x7fb95c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.924+0000 7fb97f400640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb95c0678c0 0x7fb95c069d80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb9800cf100 tx=0x7fb97000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:20.928+0000 7fb977e00640 1 -- 172.21.15.83:0/2773040062 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb9680c4030 con 0x7fb980148c20 2024-04-25T00:11:21.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.080+0000 7fb986000640 1 -- 172.21.15.83:0/2773040062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb980072c90 con 0x7fb980148c20 2024-04-25T00:11:21.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.084+0000 7fb977e00640 1 -- 172.21.15.83:0/2773040062 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fb968016980 con 0x7fb980148c20 2024-04-25T00:11:21.086 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:11:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.088+0000 7fb986000640 1 -- 172.21.15.83:0/2773040062 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb95c0678c0 msgr2=0x7fb95c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.088+0000 7fb986000640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb95c0678c0 0x7fb95c069d80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb9800cf100 tx=0x7fb97000b040 comp rx=0 tx=0).stop 2024-04-25T00:11:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.088+0000 7fb986000640 1 -- 172.21.15.83:0/2773040062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb980148c20 msgr2=0x7fb9800ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.088+0000 7fb986000640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb980148c20 0x7fb9800ce700 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb96801caa0 tx=0x7fb964005840 comp rx=0 tx=0).stop 2024-04-25T00:11:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.088+0000 7fb986000640 1 -- 172.21.15.83:0/2773040062 shutdown_connections 2024-04-25T00:11:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.088+0000 7fb986000640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb95c0678c0 0x7fb95c069d80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.088+0000 7fb986000640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9801478b0 0x7fb980147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.088+0000 7fb986000640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb980148c20 0x7fb9800ce700 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.088+0000 7fb986000640 1 --2- 172.21.15.83:0/2773040062 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb964004000 0x7fb9800b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.088+0000 7fb986000640 1 -- 172.21.15.83:0/2773040062 >> 172.21.15.83:0/2773040062 conn(0x7fb9800b1830 msgr2=0x7fb9800ade20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.088+0000 7fb986000640 1 -- 172.21.15.83:0/2773040062 shutdown_connections 2024-04-25T00:11:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.088+0000 7fb986000640 1 -- 172.21.15.83:0/2773040062 wait complete. 2024-04-25T00:11:21.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:11:21.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.240+0000 7fa12b200640 1 Processor -- start 2024-04-25T00:11:21.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.240+0000 7fa12b200640 1 -- start start 2024-04-25T00:11:21.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.244+0000 7fa12b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa12c155310 0x7fa12c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:21.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.244+0000 7fa12b200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa12c0b3db0 0x7fa12c15ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:21.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.244+0000 7fa12b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa12c0719f0 con 0x7fa12c15e790 2024-04-25T00:11:21.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.244+0000 7fa12b200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa12c15f610 con 0x7fa12c0b3db0 2024-04-25T00:11:21.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.244+0000 7fa12b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa12c071cd0 con 0x7fa12c155310 2024-04-25T00:11:21.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.244+0000 7fa12a800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa12c0b3db0 0x7fa12c15ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:21.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.244+0000 7fa12a800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa12c0b3db0 0x7fa12c15ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57972/0 (socket says 172.21.15.83:57972) 2024-04-25T00:11:21.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.244+0000 7fa12a800640 1 -- 172.21.15.83:0/1315907673 learned_addr learned my addr 172.21.15.83:0/1315907673 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:21.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.244+0000 7fa129e00640 1 --2- 172.21.15.83:0/1315907673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa12c155310 0x7fa12c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:21.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.244+0000 7fa12a800640 1 -- 172.21.15.83:0/1315907673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa12c155310 msgr2=0x7fa12c06cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:21.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.244+0000 7fa12a800640 1 --2- 172.21.15.83:0/1315907673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa12c155310 0x7fa12c06cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.244+0000 7fa12a800640 1 -- 172.21.15.83:0/1315907673 >> v1:172.21.15.86:6789/0 conn(0x7fa12c15e790 legacy=0x7fa12c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:21.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.244+0000 7fa12a800640 1 -- 172.21.15.83:0/1315907673 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c071fb0 con 0x7fa12c0b3db0 2024-04-25T00:11:21.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12a800640 1 --2- 172.21.15.83:0/1315907673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa12c0b3db0 0x7fa12c15ec10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa124009e50 tx=0x7fa12401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb79be081d7b1f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:21.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa128a00640 1 -- 172.21.15.83:0/1315907673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1240a5660 con 0x7fa12c0b3db0 2024-04-25T00:11:21.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa128a00640 1 -- 172.21.15.83:0/1315907673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1240a5800 con 0x7fa12c0b3db0 2024-04-25T00:11:21.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa128a00640 1 -- 172.21.15.83:0/1315907673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1240a5bd0 con 0x7fa12c0b3db0 2024-04-25T00:11:21.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 -- 172.21.15.83:0/1315907673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa12c0b3db0 msgr2=0x7fa12c15ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:21.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 --2- 172.21.15.83:0/1315907673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa12c0b3db0 0x7fa12c15ec10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa124009e50 tx=0x7fa12401c9a0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 -- 172.21.15.83:0/1315907673 shutdown_connections 2024-04-25T00:11:21.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 --2- 172.21.15.83:0/1315907673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa12c0b3db0 0x7fa12c15ec10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 --2- 172.21.15.83:0/1315907673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa12c155310 0x7fa12c06cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 -- 172.21.15.83:0/1315907673 >> 172.21.15.83:0/1315907673 conn(0x7fa12c0cf980 msgr2=0x7fa12c0cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:21.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 -- 172.21.15.83:0/1315907673 shutdown_connections 2024-04-25T00:11:21.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 -- 172.21.15.83:0/1315907673 wait complete. 2024-04-25T00:11:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 Processor -- start 2024-04-25T00:11:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 -- start start 2024-04-25T00:11:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa12c0b3db0 0x7fa12c06b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa12c155310 0x7fa12c144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa12c15e790 0x7fa12c162dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c071740 con 0x7fa12c155310 2024-04-25T00:11:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c0715c0 con 0x7fa12c0b3db0 2024-04-25T00:11:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa12b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa12c167b30 con 0x7fa12c15e790 2024-04-25T00:11:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa129400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa12c155310 0x7fa12c144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa129400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa12c155310 0x7fa12c144160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57980/0 (socket says 172.21.15.83:57980) 2024-04-25T00:11:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.248+0000 7fa129400640 1 -- 172.21.15.83:0/375691148 learned_addr learned my addr 172.21.15.83:0/375691148 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa129e00640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa12c0b3db0 0x7fa12c06b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa129400640 1 -- 172.21.15.83:0/375691148 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa12c15e790 msgr2=0x7fa12c162dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa12a800640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa12c15e790 0x7fa12c162dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa129400640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa12c15e790 0x7fa12c162dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa129400640 1 -- 172.21.15.83:0/375691148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa12c0b3db0 msgr2=0x7fa12c06b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa129400640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa12c0b3db0 0x7fa12c06b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa129400640 1 -- 172.21.15.83:0/375691148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c161d20 con 0x7fa12c155310 2024-04-25T00:11:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa129e00640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa12c0b3db0 0x7fa12c06b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:21.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa129400640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa12c155310 0x7fa12c144160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa11800c420 tx=0x7fa11800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:21.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa122000640 1 -- 172.21.15.83:0/375691148 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1180052d0 con 0x7fa12c155310 2024-04-25T00:11:21.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa122000640 1 -- 172.21.15.83:0/375691148 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa118005470 con 0x7fa12c155310 2024-04-25T00:11:21.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa122000640 1 -- 172.21.15.83:0/375691148 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118007260 con 0x7fa12c155310 2024-04-25T00:11:21.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa12b200640 1 -- 172.21.15.83:0/375691148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12c162000 con 0x7fa12c155310 2024-04-25T00:11:21.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa12b200640 1 -- 172.21.15.83:0/375691148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12c1625c0 con 0x7fa12c155310 2024-04-25T00:11:21.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa12b200640 1 -- 172.21.15.83:0/375691148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa12c04f510 con 0x7fa12c155310 2024-04-25T00:11:21.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa122000640 1 -- 172.21.15.83:0/375691148 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa118007400 con 0x7fa12c155310 2024-04-25T00:11:21.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa122000640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa10c0678c0 0x7fa10c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:21.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.252+0000 7fa122000640 1 -- 172.21.15.83:0/375691148 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa118085520 con 0x7fa12c155310 2024-04-25T00:11:21.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.256+0000 7fa129e00640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa10c0678c0 0x7fa10c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:21.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.256+0000 7fa129e00640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa10c0678c0 0x7fa10c069d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0097a0 tx=0x7fa11c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.256+0000 7fa122000640 1 -- 172.21.15.83:0/375691148 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa118055d10 con 0x7fa12c155310 2024-04-25T00:11:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.412+0000 7fa12b200640 1 -- 172.21.15.83:0/375691148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa12c0748c0 con 0x7fa12c155310 2024-04-25T00:11:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.412+0000 7fa122000640 1 -- 172.21.15.83:0/375691148 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fa118055610 con 0x7fa12c155310 2024-04-25T00:11:21.416 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:21.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.412+0000 7fa12b200640 1 -- 172.21.15.83:0/375691148 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa10c0678c0 msgr2=0x7fa10c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:21.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.412+0000 7fa12b200640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa10c0678c0 0x7fa10c069d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0097a0 tx=0x7fa11c008040 comp rx=0 tx=0).stop 2024-04-25T00:11:21.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.412+0000 7fa12b200640 1 -- 172.21.15.83:0/375691148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa12c155310 msgr2=0x7fa12c144160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:21.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.412+0000 7fa12b200640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa12c155310 0x7fa12c144160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa11800c420 tx=0x7fa11800c8e0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.412+0000 7fa12b200640 1 -- 172.21.15.83:0/375691148 shutdown_connections 2024-04-25T00:11:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.412+0000 7fa12b200640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa10c0678c0 0x7fa10c069d80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.412+0000 7fa12b200640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa12c15e790 0x7fa12c162dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.412+0000 7fa12b200640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa12c155310 0x7fa12c144160 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.412+0000 7fa12b200640 1 --2- 172.21.15.83:0/375691148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa12c0b3db0 0x7fa12c06b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.412+0000 7fa12b200640 1 -- 172.21.15.83:0/375691148 >> 172.21.15.83:0/375691148 conn(0x7fa12c0cf980 msgr2=0x7fa12c0ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.416+0000 7fa12b200640 1 -- 172.21.15.83:0/375691148 shutdown_connections 2024-04-25T00:11:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.416+0000 7fa12b200640 1 -- 172.21.15.83:0/375691148 wait complete. 2024-04-25T00:11:21.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:11:21.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.588+0000 7f8fe1c00640 1 Processor -- start 2024-04-25T00:11:21.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.588+0000 7f8fe1c00640 1 -- start start 2024-04-25T00:11:21.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.588+0000 7f8fe1c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fdc1446e0 0x7f8fdc0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:21.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.592+0000 7f8fe1c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fdc0cb210 con 0x7f8fdc15f350 2024-04-25T00:11:21.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.592+0000 7f8fe1c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fdc0cb4f0 con 0x7f8fdc06ee20 2024-04-25T00:11:21.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.592+0000 7f8fe1c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8fdc0715b0 con 0x7f8fdc1446e0 2024-04-25T00:11:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.592+0000 7f8fd3e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8fdc15f350 0x7f8fdc06e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58004/0 (socket says 172.21.15.83:58004) 2024-04-25T00:11:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.592+0000 7f8fd3e00640 1 -- 172.21.15.83:0/3420239926 learned_addr learned my addr 172.21.15.83:0/3420239926 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.592+0000 7f8fdaa00640 1 --2- 172.21.15.83:0/3420239926 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fdc1446e0 0x7f8fdc0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.592+0000 7f8fda000640 1 -- 172.21.15.83:0/3420239926 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2489013260 0 0) 0x7f8fdc0cb210 con 0x7f8fdc15f350 2024-04-25T00:11:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.592+0000 7f8fda000640 1 -- 172.21.15.83:0/3420239926 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fc0003530 con 0x7f8fdc15f350 2024-04-25T00:11:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.592+0000 7f8fdaa00640 1 -- 172.21.15.83:0/3420239926 >> v1:172.21.15.86:6789/0 conn(0x7f8fdc06ee20 legacy=0x7f8fdc071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.592+0000 7f8fda000640 1 -- 172.21.15.83:0/3420239926 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1987690506 0 0) 0x7f8fc0003530 con 0x7f8fdc15f350 2024-04-25T00:11:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.592+0000 7f8fdaa00640 1 -- 172.21.15.83:0/3420239926 >> v1:172.21.15.83:6789/0 conn(0x7f8fdc15f350 legacy=0x7f8fdc06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.592+0000 7f8fdaa00640 1 -- 172.21.15.83:0/3420239926 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fdc0717e0 con 0x7f8fdc1446e0 2024-04-25T00:11:21.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fdaa00640 1 --2- 172.21.15.83:0/3420239926 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fdc1446e0 0x7f8fdc0cab50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009770 tx=0x7f8fcc01c230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44632e798574e85e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fda000640 1 -- 172.21.15.83:0/3420239926 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc0b0070 con 0x7f8fdc1446e0 2024-04-25T00:11:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fda000640 1 -- 172.21.15.83:0/3420239926 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fcc0ab400 con 0x7f8fdc1446e0 2024-04-25T00:11:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fda000640 1 -- 172.21.15.83:0/3420239926 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc0a4e60 con 0x7f8fdc1446e0 2024-04-25T00:11:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 -- 172.21.15.83:0/3420239926 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fdc1446e0 msgr2=0x7f8fdc0cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 --2- 172.21.15.83:0/3420239926 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fdc1446e0 0x7f8fdc0cab50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009770 tx=0x7f8fcc01c230 comp rx=0 tx=0).stop 2024-04-25T00:11:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 -- 172.21.15.83:0/3420239926 shutdown_connections 2024-04-25T00:11:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 --2- 172.21.15.83:0/3420239926 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fdc1446e0 0x7f8fdc0cab50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 -- 172.21.15.83:0/3420239926 >> 172.21.15.83:0/3420239926 conn(0x7f8fdc0cfa60 msgr2=0x7f8fdc0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 -- 172.21.15.83:0/3420239926 shutdown_connections 2024-04-25T00:11:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 -- 172.21.15.83:0/3420239926 wait complete. 2024-04-25T00:11:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 Processor -- start 2024-04-25T00:11:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 -- start start 2024-04-25T00:11:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fdc06ee20 0x7f8fdc15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fdc1446e0 0x7f8fdc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fdc15f350 0x7f8fdc1e8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc071b50 con 0x7f8fdc06ee20 2024-04-25T00:11:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc144ac0 con 0x7f8fdc1446e0 2024-04-25T00:11:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fe1c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8fdc0cd630 con 0x7f8fdc15f350 2024-04-25T00:11:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fdaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fdc06ee20 0x7f8fdc15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fdaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fdc06ee20 0x7f8fdc15a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57992/0 (socket says 172.21.15.83:57992) 2024-04-25T00:11:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.596+0000 7f8fdaa00640 1 -- 172.21.15.83:0/4156383252 learned_addr learned my addr 172.21.15.83:0/4156383252 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fdaa00640 1 -- 172.21.15.83:0/4156383252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fdc15f350 msgr2=0x7f8fdc1e8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fdb400640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fdc15f350 0x7f8fdc1e8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fd3e00640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fdc1446e0 0x7f8fdc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fdaa00640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fdc15f350 0x7f8fdc1e8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fdaa00640 1 -- 172.21.15.83:0/4156383252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fdc1446e0 msgr2=0x7f8fdc0b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fdaa00640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fdc1446e0 0x7f8fdc0b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fdaa00640 1 -- 172.21.15.83:0/4156383252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fdc0b5440 con 0x7f8fdc06ee20 2024-04-25T00:11:21.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fdb400640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fdc15f350 0x7f8fdc1e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:21.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fd3e00640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fdc1446e0 0x7f8fdc0b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:21.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fdaa00640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fdc06ee20 0x7f8fdc15a5e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0ae040 tx=0x7f8fcc01aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:21.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fd2a00640 1 -- 172.21.15.83:0/4156383252 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc0b0050 con 0x7f8fdc06ee20 2024-04-25T00:11:21.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fd2a00640 1 -- 172.21.15.83:0/4156383252 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fcc0b1040 con 0x7f8fdc06ee20 2024-04-25T00:11:21.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fe1c00640 1 -- 172.21.15.83:0/4156383252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fdc15c570 con 0x7f8fdc06ee20 2024-04-25T00:11:21.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fd2a00640 1 -- 172.21.15.83:0/4156383252 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4001680 con 0x7f8fdc06ee20 2024-04-25T00:11:21.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fe1c00640 1 -- 172.21.15.83:0/4156383252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fdc15cb30 con 0x7f8fdc06ee20 2024-04-25T00:11:21.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fe1c00640 1 -- 172.21.15.83:0/4156383252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fdc068230 con 0x7f8fdc06ee20 2024-04-25T00:11:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fd2a00640 1 -- 172.21.15.83:0/4156383252 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8fcc0be030 con 0x7f8fdc06ee20 2024-04-25T00:11:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fd2a00640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8fb80678c0 0x7f8fb8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fd2a00640 1 -- 172.21.15.83:0/4156383252 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8fcc11fd60 con 0x7f8fdc06ee20 2024-04-25T00:11:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.600+0000 7f8fd3e00640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8fb80678c0 0x7f8fb8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.604+0000 7f8fd3e00640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8fb80678c0 0x7f8fb8069d80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc15bce0 tx=0x7f8fc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:21.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.604+0000 7f8fd2a00640 1 -- 172.21.15.83:0/4156383252 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8fcc0c5060 con 0x7f8fdc06ee20 2024-04-25T00:11:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.772+0000 7f8fe1c00640 1 -- 172.21.15.83:0/4156383252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8fdc0b4ce0 con 0x7f8fdc06ee20 2024-04-25T00:11:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.772+0000 7f8fd2a00640 1 -- 172.21.15.83:0/4156383252 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f8fcc015f70 con 0x7f8fdc06ee20 2024-04-25T00:11:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:11:21.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.776+0000 7f8fe1c00640 1 -- 172.21.15.83:0/4156383252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8fb80678c0 msgr2=0x7f8fb8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:21.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.776+0000 7f8fe1c00640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8fb80678c0 0x7f8fb8069d80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc15bce0 tx=0x7f8fc4009040 comp rx=0 tx=0).stop 2024-04-25T00:11:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.776+0000 7f8fe1c00640 1 -- 172.21.15.83:0/4156383252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fdc06ee20 msgr2=0x7f8fdc15a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.776+0000 7f8fe1c00640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fdc06ee20 0x7f8fdc15a5e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0ae040 tx=0x7f8fcc01aa70 comp rx=0 tx=0).stop 2024-04-25T00:11:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.776+0000 7f8fe1c00640 1 -- 172.21.15.83:0/4156383252 shutdown_connections 2024-04-25T00:11:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.776+0000 7f8fe1c00640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8fb80678c0 0x7f8fb8069d80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.776+0000 7f8fe1c00640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fdc15f350 0x7f8fdc1e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.776+0000 7f8fe1c00640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fdc1446e0 0x7f8fdc0b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.776+0000 7f8fe1c00640 1 --2- 172.21.15.83:0/4156383252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fdc06ee20 0x7f8fdc15a5e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.776+0000 7f8fe1c00640 1 -- 172.21.15.83:0/4156383252 >> 172.21.15.83:0/4156383252 conn(0x7f8fdc0cfa60 msgr2=0x7f8fdc143970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.776+0000 7f8fe1c00640 1 -- 172.21.15.83:0/4156383252 shutdown_connections 2024-04-25T00:11:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:21.776+0000 7f8fe1c00640 1 -- 172.21.15.83:0/4156383252 wait complete. 2024-04-25T00:11:21.798 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-25T00:11:21.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:21.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:11:21.826 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi083.stderr:ceph-fuse[34291]: fuse finished with error 0 and tester_r 0 2024-04-25T00:11:21.829 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:21.865 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:11:21.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:21.865 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T00:11:21.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.104+0000 7f66c1400640 1 Processor -- start 2024-04-25T00:11:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.104+0000 7f66c1400640 1 -- start start 2024-04-25T00:11:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.104+0000 7f66c1400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66bc1581d0 0x7f66bc0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.104+0000 7f66c1400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66bc0b4c30 con 0x7f66bc0cf3c0 2024-04-25T00:11:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.104+0000 7f66c1400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66bc0b3440 con 0x7f66bc15e6b0 2024-04-25T00:11:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.104+0000 7f66c1400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f66bc0b3720 con 0x7f66bc1581d0 2024-04-25T00:11:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.104+0000 7f66b1e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f66bc0cf3c0 0x7f66bc15dfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58014/0 (socket says 172.21.15.83:58014) 2024-04-25T00:11:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.104+0000 7f66b1e00640 1 -- 172.21.15.83:0/1896958279 learned_addr learned my addr 172.21.15.83:0/1896958279 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66ba800640 1 --2- 172.21.15.83:0/1896958279 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66bc1581d0 0x7f66bc0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 -- 172.21.15.83:0/1896958279 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1942631953 0 0) 0x7f66bc0b4c30 con 0x7f66bc0cf3c0 2024-04-25T00:11:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 -- 172.21.15.83:0/1896958279 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66a0003530 con 0x7f66bc0cf3c0 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 -- 172.21.15.83:0/1896958279 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1609475548 0 0) 0x7f66a0003530 con 0x7f66bc0cf3c0 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 -- 172.21.15.83:0/1896958279 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66bc1581d0 msgr2=0x7f66bc0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 --2- 172.21.15.83:0/1896958279 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66bc1581d0 0x7f66bc0b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 -- 172.21.15.83:0/1896958279 >> v1:172.21.15.86:6789/0 conn(0x7f66bc15e6b0 legacy=0x7f66bc06d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 -- 172.21.15.83:0/1896958279 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc0b3a00 con 0x7f66bc0cf3c0 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 -- 172.21.15.83:0/1896958279 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f66a4001680 con 0x7f66bc0cf3c0 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 -- 172.21.15.83:0/1896958279 >> v1:172.21.15.83:6789/0 conn(0x7f66bc0cf3c0 legacy=0x7f66bc15dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 --2- 172.21.15.83:0/1896958279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a00039f0 0x7f66a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 --2- 172.21.15.83:0/1896958279 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66bc1581d0 0x7f66a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 --2- 172.21.15.83:0/1896958279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66a0004d50 0x7f66a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 -- 172.21.15.83:0/1896958279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc0b3720 con 0x7f66a00039f0 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 -- 172.21.15.83:0/1896958279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66a000ba80 con 0x7f66a0004d50 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b9e00640 1 -- 172.21.15.83:0/1896958279 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f66a000de40 con 0x7f66bc1581d0 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66ba800640 1 --2- 172.21.15.83:0/1896958279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a00039f0 0x7f66a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66bb200640 1 --2- 172.21.15.83:0/1896958279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66a0004d50 0x7f66a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66b1e00640 1 --2- 172.21.15.83:0/1896958279 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66bc1581d0 0x7f66a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66ba800640 1 -- 172.21.15.83:0/1896958279 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66bc1581d0 msgr2=0x7f66a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66ba800640 1 --2- 172.21.15.83:0/1896958279 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66bc1581d0 0x7f66a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66ba800640 1 -- 172.21.15.83:0/1896958279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66a0004d50 msgr2=0x7f66a00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66ba800640 1 --2- 172.21.15.83:0/1896958279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66a0004d50 0x7f66a00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66ba800640 1 -- 172.21.15.83:0/1896958279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a0003530 con 0x7f66a00039f0 2024-04-25T00:11:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.108+0000 7f66bb200640 1 --2- 172.21.15.83:0/1896958279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66a0004d50 0x7f66a00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:22.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66ba800640 1 --2- 172.21.15.83:0/1896958279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a00039f0 0x7f66a0003dd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009fc0 tx=0x7f66ac01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c85b733cb67005a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66b9e00640 1 -- 172.21.15.83:0/1896958279 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0b2070 con 0x7f66a00039f0 2024-04-25T00:11:22.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66b9e00640 1 -- 172.21.15.83:0/1896958279 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac0a5c90 con 0x7f66a00039f0 2024-04-25T00:11:22.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66b9e00640 1 -- 172.21.15.83:0/1896958279 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac009c10 con 0x7f66a00039f0 2024-04-25T00:11:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66c1400640 1 -- 172.21.15.83:0/1896958279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a00039f0 msgr2=0x7f66a0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66c1400640 1 --2- 172.21.15.83:0/1896958279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a00039f0 0x7f66a0003dd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009fc0 tx=0x7f66ac01cc80 comp rx=0 tx=0).stop 2024-04-25T00:11:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66bb200640 1 -- 172.21.15.83:0/1896958279 reap_dead start 2024-04-25T00:11:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66c1400640 1 -- 172.21.15.83:0/1896958279 shutdown_connections 2024-04-25T00:11:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66c1400640 1 -- 172.21.15.83:0/1896958279 >> 172.21.15.83:0/1896958279 conn(0x7f66bc0b1830 msgr2=0x7f66bc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66c1400640 1 -- 172.21.15.83:0/1896958279 shutdown_connections 2024-04-25T00:11:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66c1400640 1 -- 172.21.15.83:0/1896958279 wait complete. 2024-04-25T00:11:22.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66c1400640 1 Processor -- start 2024-04-25T00:11:22.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66c1400640 1 -- start start 2024-04-25T00:11:22.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66c1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a00039f0 0x7f66bc15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66c1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc14bf20 0x7f66bc1423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66c1400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66bc151560 0x7f66bc1519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66c1400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc0b4800 con 0x7f66a00039f0 2024-04-25T00:11:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66c1400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc15ed00 con 0x7f66bc14bf20 2024-04-25T00:11:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66c1400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f66bc0b4980 con 0x7f66bc151560 2024-04-25T00:11:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66ba800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a00039f0 0x7f66bc15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66ba800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a00039f0 0x7f66bc15a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58000/0 (socket says 172.21.15.83:58000) 2024-04-25T00:11:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66ba800640 1 -- 172.21.15.83:0/2277833342 learned_addr learned my addr 172.21.15.83:0/2277833342 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66b1e00640 1 --2- 172.21.15.83:0/2277833342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc14bf20 0x7f66bc1423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66bb200640 1 --2- 172.21.15.83:0/2277833342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66bc151560 0x7f66bc1519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66ba800640 1 -- 172.21.15.83:0/2277833342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66bc151560 msgr2=0x7f66bc1519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66ba800640 1 --2- 172.21.15.83:0/2277833342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66bc151560 0x7f66bc1519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.112+0000 7f66ba800640 1 -- 172.21.15.83:0/2277833342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc14bf20 msgr2=0x7f66bc1423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.116+0000 7f66ba800640 1 --2- 172.21.15.83:0/2277833342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc14bf20 0x7f66bc1423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.116+0000 7f66ba800640 1 -- 172.21.15.83:0/2277833342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc158770 con 0x7f66a00039f0 2024-04-25T00:11:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.116+0000 7f66ba800640 1 --2- 172.21.15.83:0/2277833342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a00039f0 0x7f66bc15a0b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f66ac01c6f0 tx=0x7f66ac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.116+0000 7f66b3400640 1 -- 172.21.15.83:0/2277833342 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0ba070 con 0x7f66a00039f0 2024-04-25T00:11:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.116+0000 7f66b3400640 1 -- 172.21.15.83:0/2277833342 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac0ada20 con 0x7f66a00039f0 2024-04-25T00:11:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.116+0000 7f66b3400640 1 -- 172.21.15.83:0/2277833342 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0b2040 con 0x7f66a00039f0 2024-04-25T00:11:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.116+0000 7f66c1400640 1 -- 172.21.15.83:0/2277833342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66bc158a50 con 0x7f66a00039f0 2024-04-25T00:11:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.116+0000 7f66c1400640 1 -- 172.21.15.83:0/2277833342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66bc159010 con 0x7f66a00039f0 2024-04-25T00:11:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.116+0000 7f66c1400640 1 -- 172.21.15.83:0/2277833342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66bc0737f0 con 0x7f66a00039f0 2024-04-25T00:11:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.116+0000 7f66b3400640 1 -- 172.21.15.83:0/2277833342 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f66ac0b6410 con 0x7f66a00039f0 2024-04-25T00:11:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.116+0000 7f66b3400640 1 --2- 172.21.15.83:0/2277833342 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66980678c0 0x7f6698069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.120+0000 7f66b3400640 1 -- 172.21.15.83:0/2277833342 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f66ac1209c0 con 0x7f66a00039f0 2024-04-25T00:11:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.120+0000 7f66b1e00640 1 --2- 172.21.15.83:0/2277833342 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66980678c0 0x7f6698069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.120+0000 7f66b1e00640 1 --2- 172.21.15.83:0/2277833342 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66980678c0 0x7f6698069d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66a4001060 tx=0x7f66a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.124+0000 7f66b3400640 1 -- 172.21.15.83:0/2277833342 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f66ac016990 con 0x7f66a00039f0 2024-04-25T00:11:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.276+0000 7f66c1400640 1 -- 172.21.15.83:0/2277833342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66bc04f5f0 con 0x7f66a00039f0 2024-04-25T00:11:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.280+0000 7f66b3400640 1 -- 172.21.15.83:0/2277833342 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7f66ac0f0b60 con 0x7f66a00039f0 2024-04-25T00:11:22.281 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:22.281 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.280+0000 7f66c1400640 1 -- 172.21.15.83:0/2277833342 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66980678c0 msgr2=0x7f6698069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.280+0000 7f66c1400640 1 --2- 172.21.15.83:0/2277833342 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66980678c0 0x7f6698069d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66a4001060 tx=0x7f66a4009040 comp rx=0 tx=0).stop 2024-04-25T00:11:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.280+0000 7f66c1400640 1 -- 172.21.15.83:0/2277833342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a00039f0 msgr2=0x7f66bc15a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.280+0000 7f66c1400640 1 --2- 172.21.15.83:0/2277833342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a00039f0 0x7f66bc15a0b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f66ac01c6f0 tx=0x7f66ac009750 comp rx=0 tx=0).stop 2024-04-25T00:11:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.280+0000 7f66c1400640 1 -- 172.21.15.83:0/2277833342 shutdown_connections 2024-04-25T00:11:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.280+0000 7f66c1400640 1 --2- 172.21.15.83:0/2277833342 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66980678c0 0x7f6698069d80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.280+0000 7f66c1400640 1 --2- 172.21.15.83:0/2277833342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66bc151560 0x7f66bc1519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.280+0000 7f66c1400640 1 --2- 172.21.15.83:0/2277833342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc14bf20 0x7f66bc1423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.280+0000 7f66c1400640 1 --2- 172.21.15.83:0/2277833342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a00039f0 0x7f66bc15a0b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.280+0000 7f66c1400640 1 -- 172.21.15.83:0/2277833342 >> 172.21.15.83:0/2277833342 conn(0x7f66bc0b1830 msgr2=0x7f66bc156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.280+0000 7f66c1400640 1 -- 172.21.15.83:0/2277833342 shutdown_connections 2024-04-25T00:11:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.280+0000 7f66c1400640 1 -- 172.21.15.83:0/2277833342 wait complete. 2024-04-25T00:11:22.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:22.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67b9000640 1 Processor -- start 2024-04-25T00:11:22.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67b9000640 1 -- start start 2024-04-25T00:11:22.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67b9000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f67b40d0230 0x7f67b40d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67b9000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67b4163b00 0x7f67b4165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67b9000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67b40cc9b0 con 0x7f67b415bf70 2024-04-25T00:11:22.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67b9000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f67b40d0be0 con 0x7f67b40d0230 2024-04-25T00:11:22.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67b9000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f67b40b5ad0 con 0x7f67b4163b00 2024-04-25T00:11:22.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67a9400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f67b40d0230 0x7f67b40d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67a9400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f67b40d0230 0x7f67b40d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58016/0 (socket says 172.21.15.83:58016) 2024-04-25T00:11:22.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67a9400640 1 -- 172.21.15.83:0/219859834 learned_addr learned my addr 172.21.15.83:0/219859834 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:22.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67b1e00640 1 --1- 172.21.15.83:0/219859834 >> v1:172.21.15.83:6789/0 conn(0x7f67b415bf70 0x7f67b40cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58016/0 (socket says 172.21.15.83:58016) 2024-04-25T00:11:22.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67b2800640 1 --2- 172.21.15.83:0/219859834 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67b4163b00 0x7f67b4165ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67a9400640 1 -- 172.21.15.83:0/219859834 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67b4163b00 msgr2=0x7f67b4165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:22.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67a9400640 1 --2- 172.21.15.83:0/219859834 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67b4163b00 0x7f67b4165ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67a9400640 1 -- 172.21.15.83:0/219859834 >> v1:172.21.15.83:6789/0 conn(0x7f67b415bf70 legacy=0x7f67b40cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67a9400640 1 -- 172.21.15.83:0/219859834 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b40ccc90 con 0x7f67b40d0230 2024-04-25T00:11:22.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.436+0000 7f67b1400640 1 -- 172.21.15.83:0/219859834 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 888726304 0 0) 0x7f67b40cc9b0 con 0x7f67b415bf70 2024-04-25T00:11:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.440+0000 7f67a9400640 1 --2- 172.21.15.83:0/219859834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67b40d0230 0x7f67b40d06a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f679c009870 tx=0x7f679c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd068742fb1d3173 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.440+0000 7f67b1400640 1 -- 172.21.15.83:0/219859834 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c0a95f0 con 0x7f67b40d0230 2024-04-25T00:11:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.440+0000 7f67b1400640 1 -- 172.21.15.83:0/219859834 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f679c0a9790 con 0x7f67b40d0230 2024-04-25T00:11:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.440+0000 7f67b1400640 1 -- 172.21.15.83:0/219859834 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c0a9b60 con 0x7f67b40d0230 2024-04-25T00:11:22.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.440+0000 7f67b9000640 1 -- 172.21.15.83:0/219859834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67b40d0230 msgr2=0x7f67b40d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.440+0000 7f67b9000640 1 --2- 172.21.15.83:0/219859834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67b40d0230 0x7f67b40d06a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f679c009870 tx=0x7f679c01c530 comp rx=0 tx=0).stop 2024-04-25T00:11:22.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.440+0000 7f67b9000640 1 -- 172.21.15.83:0/219859834 shutdown_connections 2024-04-25T00:11:22.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.440+0000 7f67b9000640 1 --2- 172.21.15.83:0/219859834 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67b4163b00 0x7f67b4165ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.440+0000 7f67b9000640 1 --2- 172.21.15.83:0/219859834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67b40d0230 0x7f67b40d06a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.440+0000 7f67b9000640 1 -- 172.21.15.83:0/219859834 >> 172.21.15.83:0/219859834 conn(0x7f67b40d1c70 msgr2=0x7f67b40d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:22.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.440+0000 7f67b9000640 1 -- 172.21.15.83:0/219859834 shutdown_connections 2024-04-25T00:11:22.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.440+0000 7f67b9000640 1 -- 172.21.15.83:0/219859834 wait complete. 2024-04-25T00:11:22.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.440+0000 7f67b9000640 1 Processor -- start 2024-04-25T00:11:22.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b9000640 1 -- start start 2024-04-25T00:11:22.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67b40d0230 0x7f67b4149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b9000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67b415bf70 0x7f67b406fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b4163b00 0x7f67b4070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b9000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67b415c350 con 0x7f67b40d0230 2024-04-25T00:11:22.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b9000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67b40d29d0 con 0x7f67b4163b00 2024-04-25T00:11:22.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b9000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f67b40cff50 con 0x7f67b415bf70 2024-04-25T00:11:22.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67b40d0230 0x7f67b4149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67b40d0230 0x7f67b4149a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58032/0 (socket says 172.21.15.83:58032) 2024-04-25T00:11:22.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b1e00640 1 -- 172.21.15.83:0/3822040730 learned_addr learned my addr 172.21.15.83:0/3822040730 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:22.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67a9400640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67b415bf70 0x7f67b406fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b2800640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b4163b00 0x7f67b4070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b1e00640 1 -- 172.21.15.83:0/3822040730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67b415bf70 msgr2=0x7f67b406fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b1e00640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67b415bf70 0x7f67b406fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b1e00640 1 -- 172.21.15.83:0/3822040730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b4163b00 msgr2=0x7f67b4070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b1e00640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b4163b00 0x7f67b4070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b1e00640 1 -- 172.21.15.83:0/3822040730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b41f6620 con 0x7f67b40d0230 2024-04-25T00:11:22.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67a9400640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67b415bf70 0x7f67b406fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:22.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b1e00640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67b40d0230 0x7f67b4149a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f67a400c480 tx=0x7f67a400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67aaa00640 1 -- 172.21.15.83:0/3822040730 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4018070 con 0x7f67b40d0230 2024-04-25T00:11:22.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67aaa00640 1 -- 172.21.15.83:0/3822040730 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a4007500 con 0x7f67b40d0230 2024-04-25T00:11:22.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b9000640 1 -- 172.21.15.83:0/3822040730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b41656e0 con 0x7f67b40d0230 2024-04-25T00:11:22.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67aaa00640 1 -- 172.21.15.83:0/3822040730 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4004a30 con 0x7f67b40d0230 2024-04-25T00:11:22.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b9000640 1 -- 172.21.15.83:0/3822040730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b4165ca0 con 0x7f67b40d0230 2024-04-25T00:11:22.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.444+0000 7f67b9000640 1 -- 172.21.15.83:0/3822040730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b409fd60 con 0x7f67b40d0230 2024-04-25T00:11:22.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.448+0000 7f67aaa00640 1 -- 172.21.15.83:0/3822040730 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f67a401b070 con 0x7f67b40d0230 2024-04-25T00:11:22.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.448+0000 7f67aaa00640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f67900678c0 0x7f6790069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.448+0000 7f67aaa00640 1 -- 172.21.15.83:0/3822040730 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f67a40852c0 con 0x7f67b40d0230 2024-04-25T00:11:22.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.448+0000 7f67a9400640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f67900678c0 0x7f6790069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.448+0000 7f67a9400640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f67900678c0 0x7f6790069d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f679c0ae000 tx=0x7f679c0b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.452+0000 7f67aaa00640 1 -- 172.21.15.83:0/3822040730 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f67a4055ab0 con 0x7f67b40d0230 2024-04-25T00:11:22.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.608+0000 7f67b9000640 1 -- 172.21.15.83:0/3822040730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67b404f5f0 con 0x7f67b40d0230 2024-04-25T00:11:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.608+0000 7f67aaa00640 1 -- 172.21.15.83:0/3822040730 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7f67a40553b0 con 0x7f67b40d0230 2024-04-25T00:11:22.612 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:22.612 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:22.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.612+0000 7f67b9000640 1 -- 172.21.15.83:0/3822040730 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f67900678c0 msgr2=0x7f6790069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.612+0000 7f67b9000640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f67900678c0 0x7f6790069d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f679c0ae000 tx=0x7f679c0b3040 comp rx=0 tx=0).stop 2024-04-25T00:11:22.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.612+0000 7f67b9000640 1 -- 172.21.15.83:0/3822040730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67b40d0230 msgr2=0x7f67b4149a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.612+0000 7f67b9000640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67b40d0230 0x7f67b4149a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f67a400c480 tx=0x7f67a400c940 comp rx=0 tx=0).stop 2024-04-25T00:11:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.612+0000 7f67b9000640 1 -- 172.21.15.83:0/3822040730 shutdown_connections 2024-04-25T00:11:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.612+0000 7f67b9000640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f67900678c0 0x7f6790069d80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.612+0000 7f67b9000640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67b415bf70 0x7f67b406fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.612+0000 7f67b9000640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b4163b00 0x7f67b4070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.612+0000 7f67b9000640 1 --2- 172.21.15.83:0/3822040730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67b40d0230 0x7f67b4149a00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.612+0000 7f67b9000640 1 -- 172.21.15.83:0/3822040730 >> 172.21.15.83:0/3822040730 conn(0x7f67b40d1c70 msgr2=0x7f67b40cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.612+0000 7f67b9000640 1 -- 172.21.15.83:0/3822040730 shutdown_connections 2024-04-25T00:11:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.612+0000 7f67b9000640 1 -- 172.21.15.83:0/3822040730 wait complete. 2024-04-25T00:11:22.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:11:22.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a3ea00640 1 Processor -- start 2024-04-25T00:11:22.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a3ea00640 1 -- start start 2024-04-25T00:11:22.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a3ea00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a380d0990 0x7f7a380d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a3ea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a380b2ed0 0x7f7a380cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a3ea00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a3815c540 con 0x7f7a3815c160 2024-04-25T00:11:22.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a3ea00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7a380cf660 con 0x7f7a380b2ed0 2024-04-25T00:11:22.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a3ea00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7a380cfa30 con 0x7f7a380d0990 2024-04-25T00:11:22.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a3ca00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a380b2ed0 0x7f7a380cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a3ca00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a380b2ed0 0x7f7a380cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58048/0 (socket says 172.21.15.83:58048) 2024-04-25T00:11:22.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a3ca00640 1 -- 172.21.15.83:0/575767256 learned_addr learned my addr 172.21.15.83:0/575767256 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:22.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a37400640 1 --2- 172.21.15.83:0/575767256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a380d0990 0x7f7a380d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a3ca00640 1 -- 172.21.15.83:0/575767256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a380d0990 msgr2=0x7f7a380d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:22.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a3ca00640 1 --2- 172.21.15.83:0/575767256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a380d0990 0x7f7a380d27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a3ca00640 1 -- 172.21.15.83:0/575767256 >> v1:172.21.15.86:6789/0 conn(0x7f7a3815c160 legacy=0x7f7a380d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.772+0000 7f7a3ca00640 1 -- 172.21.15.83:0/575767256 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a380cfd10 con 0x7f7a380b2ed0 2024-04-25T00:11:22.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.776+0000 7f7a3ca00640 1 --2- 172.21.15.83:0/575767256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a380b2ed0 0x7f7a380cc7d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009870 tx=0x7f7a3001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b773f29f5f906 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.776+0000 7f7a36a00640 1 -- 172.21.15.83:0/575767256 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a300a95f0 con 0x7f7a380b2ed0 2024-04-25T00:11:22.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.776+0000 7f7a36a00640 1 -- 172.21.15.83:0/575767256 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a300a9790 con 0x7f7a380b2ed0 2024-04-25T00:11:22.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.776+0000 7f7a36a00640 1 -- 172.21.15.83:0/575767256 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a300a9b60 con 0x7f7a380b2ed0 2024-04-25T00:11:22.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.776+0000 7f7a3ea00640 1 -- 172.21.15.83:0/575767256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a380b2ed0 msgr2=0x7f7a380cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.776+0000 7f7a3ea00640 1 --2- 172.21.15.83:0/575767256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a380b2ed0 0x7f7a380cc7d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009870 tx=0x7f7a3001c530 comp rx=0 tx=0).stop 2024-04-25T00:11:22.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.776+0000 7f7a3ea00640 1 -- 172.21.15.83:0/575767256 shutdown_connections 2024-04-25T00:11:22.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.776+0000 7f7a3ea00640 1 --2- 172.21.15.83:0/575767256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a380b2ed0 0x7f7a380cc7d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.776+0000 7f7a3ea00640 1 --2- 172.21.15.83:0/575767256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a380d0990 0x7f7a380d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.776+0000 7f7a3ea00640 1 -- 172.21.15.83:0/575767256 >> 172.21.15.83:0/575767256 conn(0x7f7a380d1cc0 msgr2=0x7f7a380d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:22.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.776+0000 7f7a3ea00640 1 -- 172.21.15.83:0/575767256 shutdown_connections 2024-04-25T00:11:22.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.776+0000 7f7a3ea00640 1 -- 172.21.15.83:0/575767256 wait complete. 2024-04-25T00:11:22.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a3ea00640 1 Processor -- start 2024-04-25T00:11:22.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a3ea00640 1 -- start start 2024-04-25T00:11:22.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a380b2ed0 0x7f7a3814b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a3ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a380d0990 0x7f7a3806c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a3ea00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a3815c160 0x7f7a3806db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a3ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a380cd6a0 con 0x7f7a380b2ed0 2024-04-25T00:11:22.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a3ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a380cd520 con 0x7f7a380d0990 2024-04-25T00:11:22.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a3ea00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7a380cd820 con 0x7f7a3815c160 2024-04-25T00:11:22.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a37e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a380b2ed0 0x7f7a3814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a37e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a380b2ed0 0x7f7a3814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58056/0 (socket says 172.21.15.83:58056) 2024-04-25T00:11:22.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a37e00640 1 -- 172.21.15.83:0/383932033 learned_addr learned my addr 172.21.15.83:0/383932033 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:22.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a37e00640 1 -- 172.21.15.83:0/383932033 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a3815c160 msgr2=0x7f7a3806db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a37e00640 1 --2- 172.21.15.83:0/383932033 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a3815c160 0x7f7a3806db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a37e00640 1 -- 172.21.15.83:0/383932033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a380d0990 msgr2=0x7f7a3806c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a37e00640 1 --2- 172.21.15.83:0/383932033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a380d0990 0x7f7a3806c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a37e00640 1 -- 172.21.15.83:0/383932033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3814b690 con 0x7f7a380b2ed0 2024-04-25T00:11:22.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a37e00640 1 --2- 172.21.15.83:0/383932033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a380b2ed0 0x7f7a3814b150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c7d0 tx=0x7f7a2800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a2fe00640 1 -- 172.21.15.83:0/383932033 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28003b80 con 0x7f7a380b2ed0 2024-04-25T00:11:22.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a2fe00640 1 -- 172.21.15.83:0/383932033 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a28003d20 con 0x7f7a380b2ed0 2024-04-25T00:11:22.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a3ea00640 1 -- 172.21.15.83:0/383932033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3806c830 con 0x7f7a380b2ed0 2024-04-25T00:11:22.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a2fe00640 1 -- 172.21.15.83:0/383932033 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2800aa90 con 0x7f7a380b2ed0 2024-04-25T00:11:22.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a3ea00640 1 -- 172.21.15.83:0/383932033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a3806cd90 con 0x7f7a380b2ed0 2024-04-25T00:11:22.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.780+0000 7f7a3ea00640 1 -- 172.21.15.83:0/383932033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3804f510 con 0x7f7a380b2ed0 2024-04-25T00:11:22.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.784+0000 7f7a2fe00640 1 -- 172.21.15.83:0/383932033 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7a2800ac30 con 0x7f7a380b2ed0 2024-04-25T00:11:22.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.784+0000 7f7a2fe00640 1 --2- 172.21.15.83:0/383932033 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7a180678c0 0x7f7a18069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.784+0000 7f7a2fe00640 1 -- 172.21.15.83:0/383932033 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7a28085ff0 con 0x7f7a380b2ed0 2024-04-25T00:11:22.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.784+0000 7f7a37400640 1 --2- 172.21.15.83:0/383932033 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7a180678c0 0x7f7a18069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.784+0000 7f7a37400640 1 --2- 172.21.15.83:0/383932033 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7a180678c0 0x7f7a18069d80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a381522e0 tx=0x7f7a24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.788+0000 7f7a2fe00640 1 -- 172.21.15.83:0/383932033 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7a3806c830 con 0x7f7a380b2ed0 2024-04-25T00:11:22.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:22.980+0000 7f7a3ea00640 1 -- 172.21.15.83:0/383932033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7a380d2760 con 0x7f7a380b2ed0 2024-04-25T00:11:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.088+0000 7f7a2fe00640 1 -- 172.21.15.83:0/383932033 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f7a28056750 con 0x7f7a380b2ed0 2024-04-25T00:11:23.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.092+0000 7f7a3ea00640 1 -- 172.21.15.83:0/383932033 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7a180678c0 msgr2=0x7f7a18069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:23.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.092+0000 7f7a3ea00640 1 --2- 172.21.15.83:0/383932033 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7a180678c0 0x7f7a18069d80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a381522e0 tx=0x7f7a24008040 comp rx=0 tx=0).stop 2024-04-25T00:11:23.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.092+0000 7f7a3ea00640 1 -- 172.21.15.83:0/383932033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a380b2ed0 msgr2=0x7f7a3814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:23.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.092+0000 7f7a3ea00640 1 --2- 172.21.15.83:0/383932033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a380b2ed0 0x7f7a3814b150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c7d0 tx=0x7f7a2800cc90 comp rx=0 tx=0).stop 2024-04-25T00:11:23.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.092+0000 7f7a3ea00640 1 -- 172.21.15.83:0/383932033 shutdown_connections 2024-04-25T00:11:23.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.092+0000 7f7a3ea00640 1 --2- 172.21.15.83:0/383932033 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7a180678c0 0x7f7a18069d80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:23.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.092+0000 7f7a3ea00640 1 --2- 172.21.15.83:0/383932033 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a3815c160 0x7f7a3806db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:23.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.092+0000 7f7a3ea00640 1 --2- 172.21.15.83:0/383932033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a380d0990 0x7f7a3806c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:23.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.092+0000 7f7a3ea00640 1 --2- 172.21.15.83:0/383932033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a380b2ed0 0x7f7a3814b150 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:23.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.092+0000 7f7a3ea00640 1 -- 172.21.15.83:0/383932033 >> 172.21.15.83:0/383932033 conn(0x7f7a380d1cc0 msgr2=0x7f7a38158710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:23.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.092+0000 7f7a3ea00640 1 -- 172.21.15.83:0/383932033 shutdown_connections 2024-04-25T00:11:23.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.092+0000 7f7a3ea00640 1 -- 172.21.15.83:0/383932033 wait complete. 2024-04-25T00:11:23.115 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T00:11:23.115 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:11:23.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.248+0000 7f76c8c00640 1 Processor -- start 2024-04-25T00:11:23.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.248+0000 7f76c8c00640 1 -- start start 2024-04-25T00:11:23.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.252+0000 7f76c8c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f76c406e950 0x7f76c415f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:23.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.252+0000 7f76c8c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76c40ce000 con 0x7f76c4159810 2024-04-25T00:11:23.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.252+0000 7f76c8c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76c406eec0 con 0x7f76c415fe50 2024-04-25T00:11:23.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.252+0000 7f76c8c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f76c406e210 con 0x7f76c406e950 2024-04-25T00:11:23.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.252+0000 7f76c1e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f76c4159810 0x7f76c40cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58022/0 (socket says 172.21.15.83:58022) 2024-04-25T00:11:23.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.252+0000 7f76c1e00640 1 -- 172.21.15.83:0/919137512 learned_addr learned my addr 172.21.15.83:0/919137512 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:23.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.252+0000 7f76b9400640 1 --2- 172.21.15.83:0/919137512 >> v2:172.21.15.86:3300/0 conn(0x7f76c406e950 0x7f76c415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:23.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.252+0000 7f76c1400640 1 -- 172.21.15.83:0/919137512 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2672316079 0 0) 0x7f76c40ce000 con 0x7f76c4159810 2024-04-25T00:11:23.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.252+0000 7f76c1400640 1 -- 172.21.15.83:0/919137512 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76a8003530 con 0x7f76c4159810 2024-04-25T00:11:23.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.252+0000 7f76b9400640 1 -- 172.21.15.83:0/919137512 >> v1:172.21.15.86:6789/0 conn(0x7f76c415fe50 legacy=0x7f76c40ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:23.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.252+0000 7f76b9400640 1 -- 172.21.15.83:0/919137512 >> v1:172.21.15.83:6789/0 conn(0x7f76c4159810 legacy=0x7f76c40cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:23.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.252+0000 7f76c1400640 1 -- 172.21.15.83:0/919137512 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1711300908 0 0) 0x7f76a8003530 con 0x7f76c4159810 2024-04-25T00:11:23.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.252+0000 7f76b9400640 1 -- 172.21.15.83:0/919137512 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c406dc50 con 0x7f76c406e950 2024-04-25T00:11:23.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76b9400640 1 --2- 172.21.15.83:0/919137512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c406e950 0x7f76c415f910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0097c0 tx=0x7f76ac01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2eaf93335748aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:23.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c1400640 1 -- 172.21.15.83:0/919137512 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac0b0070 con 0x7f76c406e950 2024-04-25T00:11:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c1400640 1 -- 172.21.15.83:0/919137512 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76ac0ab400 con 0x7f76c406e950 2024-04-25T00:11:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c1400640 1 -- 172.21.15.83:0/919137512 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b40016a0 con 0x7f76c406e950 2024-04-25T00:11:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 -- 172.21.15.83:0/919137512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c406e950 msgr2=0x7f76c415f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 --2- 172.21.15.83:0/919137512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c406e950 0x7f76c415f910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0097c0 tx=0x7f76ac01c330 comp rx=0 tx=0).stop 2024-04-25T00:11:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 -- 172.21.15.83:0/919137512 shutdown_connections 2024-04-25T00:11:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 --2- 172.21.15.83:0/919137512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c406e950 0x7f76c415f910 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 -- 172.21.15.83:0/919137512 >> 172.21.15.83:0/919137512 conn(0x7f76c40b1830 msgr2=0x7f76c40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 -- 172.21.15.83:0/919137512 shutdown_connections 2024-04-25T00:11:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 -- 172.21.15.83:0/919137512 wait complete. 2024-04-25T00:11:23.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 Processor -- start 2024-04-25T00:11:23.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 -- start start 2024-04-25T00:11:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c406e950 0x7f76c4150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c4159810 0x7f76c41557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76c415fe50 0x7f76c4155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4169e40 con 0x7f76c4159810 2024-04-25T00:11:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4169cc0 con 0x7f76c406e950 2024-04-25T00:11:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76c8c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f76c4169fc0 con 0x7f76c415fe50 2024-04-25T00:11:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76b9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c4159810 0x7f76c41557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76b9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c4159810 0x7f76c41557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58060/0 (socket says 172.21.15.83:58060) 2024-04-25T00:11:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.256+0000 7f76b9400640 1 -- 172.21.15.83:0/2830011322 learned_addr learned my addr 172.21.15.83:0/2830011322 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76b9400640 1 -- 172.21.15.83:0/2830011322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76c415fe50 msgr2=0x7f76c4155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76b9400640 1 --2- 172.21.15.83:0/2830011322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76c415fe50 0x7f76c4155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76b9400640 1 -- 172.21.15.83:0/2830011322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c406e950 msgr2=0x7f76c4150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76b9400640 1 --2- 172.21.15.83:0/2830011322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c406e950 0x7f76c4150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76b9400640 1 -- 172.21.15.83:0/2830011322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c415da40 con 0x7f76c4159810 2024-04-25T00:11:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76b9400640 1 --2- 172.21.15.83:0/2830011322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c4159810 0x7f76c41557d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0097c0 tx=0x7f76ac01ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76baa00640 1 -- 172.21.15.83:0/2830011322 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac0bb020 con 0x7f76c4159810 2024-04-25T00:11:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76baa00640 1 -- 172.21.15.83:0/2830011322 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76ac0b1040 con 0x7f76c4159810 2024-04-25T00:11:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76baa00640 1 -- 172.21.15.83:0/2830011322 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac0b0050 con 0x7f76c4159810 2024-04-25T00:11:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76c8c00640 1 -- 172.21.15.83:0/2830011322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c4139a80 con 0x7f76c4159810 2024-04-25T00:11:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76c8c00640 1 -- 172.21.15.83:0/2830011322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c4153890 con 0x7f76c4159810 2024-04-25T00:11:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76c8c00640 1 -- 172.21.15.83:0/2830011322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c404f510 con 0x7f76c4159810 2024-04-25T00:11:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76baa00640 1 -- 172.21.15.83:0/2830011322 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f76ac0be070 con 0x7f76c4159810 2024-04-25T00:11:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76baa00640 1 --2- 172.21.15.83:0/2830011322 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76a00678c0 0x7f76a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.260+0000 7f76baa00640 1 -- 172.21.15.83:0/2830011322 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f76ac11f440 con 0x7f76c4159810 2024-04-25T00:11:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.264+0000 7f76c1e00640 1 --2- 172.21.15.83:0/2830011322 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76a00678c0 0x7f76a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.264+0000 7f76c1e00640 1 --2- 172.21.15.83:0/2830011322 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76a00678c0 0x7f76a0069d80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f76b4001ec0 tx=0x7f76b4001a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.264+0000 7f76baa00640 1 -- 172.21.15.83:0/2830011322 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f76ac015fb0 con 0x7f76c4159810 2024-04-25T00:11:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:23.456+0000 7f76c8c00640 1 -- 172.21.15.83:0/2830011322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f76c4072c90 con 0x7f76c4159810 2024-04-25T00:11:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.096+0000 7f76baa00640 1 -- 172.21.15.83:0/2830011322 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f76ac0efc40 con 0x7f76c4159810 2024-04-25T00:11:24.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.100+0000 7f76c8c00640 1 -- 172.21.15.83:0/2830011322 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76a00678c0 msgr2=0x7f76a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.100+0000 7f76c8c00640 1 --2- 172.21.15.83:0/2830011322 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76a00678c0 0x7f76a0069d80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f76b4001ec0 tx=0x7f76b4001a30 comp rx=0 tx=0).stop 2024-04-25T00:11:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.100+0000 7f76c8c00640 1 -- 172.21.15.83:0/2830011322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c4159810 msgr2=0x7f76c41557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.100+0000 7f76c8c00640 1 --2- 172.21.15.83:0/2830011322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c4159810 0x7f76c41557d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0097c0 tx=0x7f76ac01ab70 comp rx=0 tx=0).stop 2024-04-25T00:11:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.100+0000 7f76c8c00640 1 -- 172.21.15.83:0/2830011322 shutdown_connections 2024-04-25T00:11:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.100+0000 7f76c8c00640 1 --2- 172.21.15.83:0/2830011322 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76a00678c0 0x7f76a0069d80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.100+0000 7f76c8c00640 1 --2- 172.21.15.83:0/2830011322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76c415fe50 0x7f76c4155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.100+0000 7f76c8c00640 1 --2- 172.21.15.83:0/2830011322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c4159810 0x7f76c41557d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.100+0000 7f76c8c00640 1 --2- 172.21.15.83:0/2830011322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c406e950 0x7f76c4150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.100+0000 7f76c8c00640 1 -- 172.21.15.83:0/2830011322 >> 172.21.15.83:0/2830011322 conn(0x7f76c40b1830 msgr2=0x7f76c4071900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.100+0000 7f76c8c00640 1 -- 172.21.15.83:0/2830011322 shutdown_connections 2024-04-25T00:11:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.100+0000 7f76c8c00640 1 -- 172.21.15.83:0/2830011322 wait complete. 2024-04-25T00:11:24.126 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:11:24.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:11:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.260+0000 7f6e14400640 1 Processor -- start 2024-04-25T00:11:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.260+0000 7f6e14400640 1 -- start start 2024-04-25T00:11:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e14400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6e0c164cb0 0x7f6e0c1670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e14400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e0c0ccc60 con 0x7f6e0c1599f0 2024-04-25T00:11:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e14400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e0c06dba0 con 0x7f6e0c0cc510 2024-04-25T00:11:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e14400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6e0c159dd0 con 0x7f6e0c164cb0 2024-04-25T00:11:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e11a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6e0c1599f0 0x7f6e0c15c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58026/0 (socket says 172.21.15.83:58026) 2024-04-25T00:11:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e11a00640 1 -- 172.21.15.83:0/192192416 learned_addr learned my addr 172.21.15.83:0/192192416 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e12400640 1 --2- 172.21.15.83:0/192192416 >> v2:172.21.15.86:3300/0 conn(0x7f6e0c164cb0 0x7f6e0c1670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 -- 172.21.15.83:0/192192416 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3586281156 0 0) 0x7f6e000014a0 con 0x7f6e0c1599f0 2024-04-25T00:11:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 -- 172.21.15.83:0/192192416 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6df4003530 con 0x7f6e0c1599f0 2024-04-25T00:11:24.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 -- 172.21.15.83:0/192192416 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3170585376 0 0) 0x7f6df4003530 con 0x7f6e0c1599f0 2024-04-25T00:11:24.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 -- 172.21.15.83:0/192192416 >> v2:172.21.15.86:3300/0 conn(0x7f6e0c164cb0 msgr2=0x7f6e0c1670a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:24.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 --2- 172.21.15.83:0/192192416 >> v2:172.21.15.86:3300/0 conn(0x7f6e0c164cb0 0x7f6e0c1670a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 -- 172.21.15.83:0/192192416 >> v1:172.21.15.86:6789/0 conn(0x7f6e0c0cc510 legacy=0x7f6e0c15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 -- 172.21.15.83:0/192192416 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c06de80 con 0x7f6e0c1599f0 2024-04-25T00:11:24.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 -- 172.21.15.83:0/192192416 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f6e00001bd0 con 0x7f6e0c1599f0 2024-04-25T00:11:24.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 -- 172.21.15.83:0/192192416 >> v1:172.21.15.83:6789/0 conn(0x7f6e0c1599f0 legacy=0x7f6e0c15c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 --2- 172.21.15.83:0/192192416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40039c0 0x7f6df4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 --2- 172.21.15.83:0/192192416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40043c0 0x7f6df4004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 --2- 172.21.15.83:0/192192416 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6df4005130 0x7f6df400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 -- 172.21.15.83:0/192192416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6df400c040 con 0x7f6df40039c0 2024-04-25T00:11:24.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 -- 172.21.15.83:0/192192416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6df400c1f0 con 0x7f6df40043c0 2024-04-25T00:11:24.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e0be00640 1 -- 172.21.15.83:0/192192416 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6df400c3f0 con 0x7f6df4005130 2024-04-25T00:11:24.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e11a00640 1 --2- 172.21.15.83:0/192192416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40039c0 0x7f6df4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e12400640 1 --2- 172.21.15.83:0/192192416 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6df4005130 0x7f6df400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e11000640 1 --2- 172.21.15.83:0/192192416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40043c0 0x7f6df4004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e11a00640 1 -- 172.21.15.83:0/192192416 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6df4005130 msgr2=0x7f6df400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:24.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e11a00640 1 --2- 172.21.15.83:0/192192416 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6df4005130 0x7f6df400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e11a00640 1 -- 172.21.15.83:0/192192416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40043c0 msgr2=0x7f6df4004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:24.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e11a00640 1 --2- 172.21.15.83:0/192192416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40043c0 0x7f6df4004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e11a00640 1 -- 172.21.15.83:0/192192416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df4003530 con 0x7f6df40039c0 2024-04-25T00:11:24.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e11000640 1 --2- 172.21.15.83:0/192192416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40043c0 0x7f6df4004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:24.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.264+0000 7f6e12400640 1 -- 172.21.15.83:0/192192416 reap_dead start 2024-04-25T00:11:24.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.268+0000 7f6e11a00640 1 --2- 172.21.15.83:0/192192416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40039c0 0x7f6df4003da0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6e00010a30 tx=0x7f6e00021580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab43d414b724884a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:24.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.268+0000 7f6e0be00640 1 -- 172.21.15.83:0/192192416 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e000b4070 con 0x7f6df40039c0 2024-04-25T00:11:24.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.268+0000 7f6e0be00640 1 -- 172.21.15.83:0/192192416 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e000af600 con 0x7f6df40039c0 2024-04-25T00:11:24.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.268+0000 7f6e0be00640 1 -- 172.21.15.83:0/192192416 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00010650 con 0x7f6df40039c0 2024-04-25T00:11:24.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.268+0000 7f6e14400640 1 -- 172.21.15.83:0/192192416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40039c0 msgr2=0x7f6df4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.268+0000 7f6e14400640 1 --2- 172.21.15.83:0/192192416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40039c0 0x7f6df4003da0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6e00010a30 tx=0x7f6e00021580 comp rx=0 tx=0).stop 2024-04-25T00:11:24.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.268+0000 7f6e14400640 1 -- 172.21.15.83:0/192192416 shutdown_connections 2024-04-25T00:11:24.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.268+0000 7f6e14400640 1 --2- 172.21.15.83:0/192192416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40039c0 0x7f6df4003da0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.268+0000 7f6e14400640 1 -- 172.21.15.83:0/192192416 >> 172.21.15.83:0/192192416 conn(0x7f6e0c0b1830 msgr2=0x7f6e0c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:24.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.268+0000 7f6e14400640 1 -- 172.21.15.83:0/192192416 shutdown_connections 2024-04-25T00:11:24.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.268+0000 7f6e14400640 1 -- 172.21.15.83:0/192192416 wait complete. 2024-04-25T00:11:24.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e14400640 1 Processor -- start 2024-04-25T00:11:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e14400640 1 -- start start 2024-04-25T00:11:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e14400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40039c0 0x7f6e0c152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e14400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e0c1f2f50 0x7f6e0c1f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e14400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e0c0cf640 0x7f6e0c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e14400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c15ccd0 con 0x7f6e0c1f2f50 2024-04-25T00:11:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e14400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c15cb50 con 0x7f6df40039c0 2024-04-25T00:11:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e14400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0c15ce50 con 0x7f6e0c0cf640 2024-04-25T00:11:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e11000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e0c1f2f50 0x7f6e0c1f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e11000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e0c1f2f50 0x7f6e0c1f33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58092/0 (socket says 172.21.15.83:58092) 2024-04-25T00:11:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e11000640 1 -- 172.21.15.83:0/3515635277 learned_addr learned my addr 172.21.15.83:0/3515635277 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e11a00640 1 --2- 172.21.15.83:0/3515635277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40039c0 0x7f6e0c152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e12400640 1 --2- 172.21.15.83:0/3515635277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e0c0cf640 0x7f6e0c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e11000640 1 -- 172.21.15.83:0/3515635277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e0c0cf640 msgr2=0x7f6e0c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e11000640 1 --2- 172.21.15.83:0/3515635277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e0c0cf640 0x7f6e0c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e11000640 1 -- 172.21.15.83:0/3515635277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40039c0 msgr2=0x7f6e0c152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e11000640 1 --2- 172.21.15.83:0/3515635277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40039c0 0x7f6e0c152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e11000640 1 -- 172.21.15.83:0/3515635277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c0ce5e0 con 0x7f6e0c1f2f50 2024-04-25T00:11:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e11000640 1 --2- 172.21.15.83:0/3515635277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e0c1f2f50 0x7f6e0c1f33d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6df8007fd0 tx=0x7f6df800e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e09600640 1 -- 172.21.15.83:0/3515635277 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df80052e0 con 0x7f6e0c1f2f50 2024-04-25T00:11:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e09600640 1 -- 172.21.15.83:0/3515635277 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df8005480 con 0x7f6e0c1f2f50 2024-04-25T00:11:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e14400640 1 -- 172.21.15.83:0/3515635277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0c0ce880 con 0x7f6e0c1f2f50 2024-04-25T00:11:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e09600640 1 -- 172.21.15.83:0/3515635277 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8007020 con 0x7f6e0c1f2f50 2024-04-25T00:11:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e14400640 1 -- 172.21.15.83:0/3515635277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e0c0cee40 con 0x7f6e0c1f2f50 2024-04-25T00:11:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.272+0000 7f6e14400640 1 -- 172.21.15.83:0/3515635277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e0c153490 con 0x7f6e0c1f2f50 2024-04-25T00:11:24.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.276+0000 7f6e09600640 1 -- 172.21.15.83:0/3515635277 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6df801a020 con 0x7f6e0c1f2f50 2024-04-25T00:11:24.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.276+0000 7f6e09600640 1 --2- 172.21.15.83:0/3515635277 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dec0678f0 0x7f6dec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.276+0000 7f6e09600640 1 -- 172.21.15.83:0/3515635277 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6df8085620 con 0x7f6e0c1f2f50 2024-04-25T00:11:24.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.276+0000 7f6e11a00640 1 --2- 172.21.15.83:0/3515635277 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dec0678f0 0x7f6dec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.276+0000 7f6e11a00640 1 --2- 172.21.15.83:0/3515635277 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dec0678f0 0x7f6dec069db0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6e000b1000 tx=0x7f6e0c06dd70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.280+0000 7f6e09600640 1 -- 172.21.15.83:0/3515635277 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6df801a310 con 0x7f6e0c1f2f50 2024-04-25T00:11:24.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.464+0000 7f6e14400640 1 -- 172.21.15.83:0/3515635277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e0c0737f0 con 0x7f6e0c1f2f50 2024-04-25T00:11:24.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.464+0000 7f6e09600640 1 -- 172.21.15.83:0/3515635277 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7f6df8055d80 con 0x7f6e0c1f2f50 2024-04-25T00:11:24.468 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:24.468 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/1335208265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1335208265},{"type":"v1","addr":"172.21.15.86:6834","nonce":1335208265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/911805587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":911805587},{"type":"v1","addr":"172.21.15.83:6836","nonce":911805587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2309157687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2309157687},{"type":"v1","addr":"172.21.15.83:6837","nonce":2309157687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:11:06.200989+0000","modified":"2024-04-25T00:11:23.462521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1603954817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":1603954817},{"type":"v1","addr":"172.21.15.86:6835","nonce":1603954817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T00:11:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2024-04-25T00:11:24.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.468+0000 7f6e14400640 1 -- 172.21.15.83:0/3515635277 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dec0678f0 msgr2=0x7f6dec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.468+0000 7f6e14400640 1 --2- 172.21.15.83:0/3515635277 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dec0678f0 0x7f6dec069db0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6e000b1000 tx=0x7f6e0c06dd70 comp rx=0 tx=0).stop 2024-04-25T00:11:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.468+0000 7f6e14400640 1 -- 172.21.15.83:0/3515635277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e0c1f2f50 msgr2=0x7f6e0c1f33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.468+0000 7f6e14400640 1 --2- 172.21.15.83:0/3515635277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e0c1f2f50 0x7f6e0c1f33d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6df8007fd0 tx=0x7f6df800e9e0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.468+0000 7f6e14400640 1 -- 172.21.15.83:0/3515635277 shutdown_connections 2024-04-25T00:11:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.468+0000 7f6e14400640 1 --2- 172.21.15.83:0/3515635277 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dec0678f0 0x7f6dec069db0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.468+0000 7f6e14400640 1 --2- 172.21.15.83:0/3515635277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e0c0cf640 0x7f6e0c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.468+0000 7f6e14400640 1 --2- 172.21.15.83:0/3515635277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e0c1f2f50 0x7f6e0c1f33d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.468+0000 7f6e14400640 1 --2- 172.21.15.83:0/3515635277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40039c0 0x7f6e0c152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.468+0000 7f6e14400640 1 -- 172.21.15.83:0/3515635277 >> 172.21.15.83:0/3515635277 conn(0x7f6e0c0b1830 msgr2=0x7f6e0c0b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.468+0000 7f6e14400640 1 -- 172.21.15.83:0/3515635277 shutdown_connections 2024-04-25T00:11:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.468+0000 7f6e14400640 1 -- 172.21.15.83:0/3515635277 wait complete. 2024-04-25T00:11:24.490 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.620+0000 7f743e400640 1 Processor -- start 2024-04-25T00:11:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.620+0000 7f743e400640 1 -- start start 2024-04-25T00:11:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f743e400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7438164cb0 0x7f74381670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f743e400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74380ccc60 con 0x7f74381599f0 2024-04-25T00:11:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f743e400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f743806dba0 con 0x7f74380cc510 2024-04-25T00:11:24.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f743e400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7438159dd0 con 0x7f7438164cb0 2024-04-25T00:11:24.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7437400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f74381599f0 0x7f743815c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52102/0 (socket says 172.21.15.83:52102) 2024-04-25T00:11:24.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7437400640 1 -- 172.21.15.83:0/2877423669 learned_addr learned my addr 172.21.15.83:0/2877423669 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:24.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7437e00640 1 --2- 172.21.15.83:0/2877423669 >> v2:172.21.15.86:3300/0 conn(0x7f7438164cb0 0x7f74381670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 -- 172.21.15.83:0/2877423669 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 899298027 0 0) 0x7f74280014a0 con 0x7f74381599f0 2024-04-25T00:11:24.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 -- 172.21.15.83:0/2877423669 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f741c003530 con 0x7f74381599f0 2024-04-25T00:11:24.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 -- 172.21.15.83:0/2877423669 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 303647996 0 0) 0x7f741c003530 con 0x7f74381599f0 2024-04-25T00:11:24.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 -- 172.21.15.83:0/2877423669 >> v2:172.21.15.86:3300/0 conn(0x7f7438164cb0 msgr2=0x7f74381670a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:24.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 --2- 172.21.15.83:0/2877423669 >> v2:172.21.15.86:3300/0 conn(0x7f7438164cb0 0x7f74381670a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 -- 172.21.15.83:0/2877423669 >> v1:172.21.15.86:6789/0 conn(0x7f74380cc510 legacy=0x7f743815ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 -- 172.21.15.83:0/2877423669 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743806de80 con 0x7f74381599f0 2024-04-25T00:11:24.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 -- 172.21.15.83:0/2877423669 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f7428001bd0 con 0x7f74381599f0 2024-04-25T00:11:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 -- 172.21.15.83:0/2877423669 >> v1:172.21.15.83:6789/0 conn(0x7f74381599f0 legacy=0x7f743815c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 --2- 172.21.15.83:0/2877423669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c0039c0 0x7f741c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 --2- 172.21.15.83:0/2877423669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f741c0043c0 0x7f741c004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 --2- 172.21.15.83:0/2877423669 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f741c005130 0x7f741c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 -- 172.21.15.83:0/2877423669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f741c00c040 con 0x7f741c0043c0 2024-04-25T00:11:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 -- 172.21.15.83:0/2877423669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f741c00c1f0 con 0x7f741c0039c0 2024-04-25T00:11:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436000640 1 -- 172.21.15.83:0/2877423669 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f741c00c3f0 con 0x7f741c005130 2024-04-25T00:11:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436a00640 1 --2- 172.21.15.83:0/2877423669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f741c0043c0 0x7f741c004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7437400640 1 --2- 172.21.15.83:0/2877423669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c0039c0 0x7f741c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7437e00640 1 --2- 172.21.15.83:0/2877423669 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f741c005130 0x7f741c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436a00640 1 -- 172.21.15.83:0/2877423669 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f741c005130 msgr2=0x7f741c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436a00640 1 --2- 172.21.15.83:0/2877423669 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f741c005130 0x7f741c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436a00640 1 -- 172.21.15.83:0/2877423669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c0039c0 msgr2=0x7f741c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436a00640 1 --2- 172.21.15.83:0/2877423669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c0039c0 0x7f741c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7436a00640 1 -- 172.21.15.83:0/2877423669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741c003530 con 0x7f741c0043c0 2024-04-25T00:11:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.624+0000 7f7437e00640 1 -- 172.21.15.83:0/2877423669 reap_dead start 2024-04-25T00:11:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.628+0000 7f7436a00640 1 --2- 172.21.15.83:0/2877423669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f741c0043c0 0x7f741c004890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7424001860 tx=0x7f742401cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6108291e3967e872 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.628+0000 7f7436000640 1 -- 172.21.15.83:0/2877423669 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424001460 con 0x7f741c0043c0 2024-04-25T00:11:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.628+0000 7f7436000640 1 -- 172.21.15.83:0/2877423669 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74240b0070 con 0x7f741c0043c0 2024-04-25T00:11:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.628+0000 7f7436000640 1 -- 172.21.15.83:0/2877423669 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74240ab870 con 0x7f741c0043c0 2024-04-25T00:11:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.628+0000 7f743e400640 1 -- 172.21.15.83:0/2877423669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f741c0043c0 msgr2=0x7f741c004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.628+0000 7f743e400640 1 --2- 172.21.15.83:0/2877423669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f741c0043c0 0x7f741c004890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7424001860 tx=0x7f742401cf10 comp rx=0 tx=0).stop 2024-04-25T00:11:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.628+0000 7f743e400640 1 -- 172.21.15.83:0/2877423669 shutdown_connections 2024-04-25T00:11:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.628+0000 7f743e400640 1 --2- 172.21.15.83:0/2877423669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f741c0043c0 0x7f741c004890 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.628+0000 7f743e400640 1 -- 172.21.15.83:0/2877423669 >> 172.21.15.83:0/2877423669 conn(0x7f74380b1830 msgr2=0x7f74380b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.628+0000 7f743e400640 1 -- 172.21.15.83:0/2877423669 shutdown_connections 2024-04-25T00:11:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.628+0000 7f743e400640 1 -- 172.21.15.83:0/2877423669 wait complete. 2024-04-25T00:11:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.628+0000 7f743e400640 1 Processor -- start 2024-04-25T00:11:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.628+0000 7f743e400640 1 -- start start 2024-04-25T00:11:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f743e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c0043c0 0x7f7438153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f743e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f743814e610 0x7f7438163510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f743e400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7438163a50 0x7f7438152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f743e400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f743815ccd0 con 0x7f743814e610 2024-04-25T00:11:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f743e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f743815cb50 con 0x7f741c0043c0 2024-04-25T00:11:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f743e400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f743815ce50 con 0x7f7438163a50 2024-04-25T00:11:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f7436a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f743814e610 0x7f7438163510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f7436a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f743814e610 0x7f7438163510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36196/0 (socket says 172.21.15.83:36196) 2024-04-25T00:11:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f7436a00640 1 -- 172.21.15.83:0/466266781 learned_addr learned my addr 172.21.15.83:0/466266781 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f7437400640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c0043c0 0x7f7438153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f7437e00640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7438163a50 0x7f7438152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f7436a00640 1 -- 172.21.15.83:0/466266781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7438163a50 msgr2=0x7f7438152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f7436a00640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7438163a50 0x7f7438152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f7436a00640 1 -- 172.21.15.83:0/466266781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c0043c0 msgr2=0x7f7438153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f7436a00640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c0043c0 0x7f7438153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f7436a00640 1 -- 172.21.15.83:0/466266781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7438152f50 con 0x7f743814e610 2024-04-25T00:11:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f7437400640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c0043c0 0x7f7438153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f7436a00640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f743814e610 0x7f7438163510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f742401c2e0 tx=0x7f741c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f742f400640 1 -- 172.21.15.83:0/466266781 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424006b20 con 0x7f743814e610 2024-04-25T00:11:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f742f400640 1 -- 172.21.15.83:0/466266781 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7424006cc0 con 0x7f743814e610 2024-04-25T00:11:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f742f400640 1 -- 172.21.15.83:0/466266781 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424007280 con 0x7f743814e610 2024-04-25T00:11:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f743e400640 1 -- 172.21.15.83:0/466266781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7438153230 con 0x7f743814e610 2024-04-25T00:11:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f743e400640 1 -- 172.21.15.83:0/466266781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74381537f0 con 0x7f743814e610 2024-04-25T00:11:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f743e400640 1 -- 172.21.15.83:0/466266781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7438163e60 con 0x7f743814e610 2024-04-25T00:11:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.632+0000 7f742f400640 1 -- 172.21.15.83:0/466266781 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f74240c3060 con 0x7f743814e610 2024-04-25T00:11:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.636+0000 7f742f400640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f74180678f0 0x7f7418069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.636+0000 7f742f400640 1 -- 172.21.15.83:0/466266781 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f742411fa20 con 0x7f743814e610 2024-04-25T00:11:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.636+0000 7f7437400640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f74180678f0 0x7f7418069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.636+0000 7f7437400640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f74180678f0 0x7f7418069db0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7428002240 tx=0x7f7428009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:24.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.640+0000 7f742f400640 1 -- 172.21.15.83:0/466266781 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7438163e60 con 0x7f743814e610 2024-04-25T00:11:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.792+0000 7f743e400640 1 -- 172.21.15.83:0/466266781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74380737f0 con 0x7f743814e610 2024-04-25T00:11:24.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.796+0000 7f742f400640 1 -- 172.21.15.83:0/466266781 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7f74240161f0 con 0x7f743814e610 2024-04-25T00:11:24.797 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:24.797 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:24.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.796+0000 7f743e400640 1 -- 172.21.15.83:0/466266781 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f74180678f0 msgr2=0x7f7418069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.796+0000 7f743e400640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f74180678f0 0x7f7418069db0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7428002240 tx=0x7f7428009040 comp rx=0 tx=0).stop 2024-04-25T00:11:24.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.796+0000 7f743e400640 1 -- 172.21.15.83:0/466266781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f743814e610 msgr2=0x7f7438163510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.796+0000 7f743e400640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f743814e610 0x7f7438163510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f742401c2e0 tx=0x7f741c005840 comp rx=0 tx=0).stop 2024-04-25T00:11:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.796+0000 7f743e400640 1 -- 172.21.15.83:0/466266781 shutdown_connections 2024-04-25T00:11:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.796+0000 7f743e400640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f74180678f0 0x7f7418069db0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.796+0000 7f743e400640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7438163a50 0x7f7438152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.796+0000 7f743e400640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f743814e610 0x7f7438163510 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.796+0000 7f743e400640 1 --2- 172.21.15.83:0/466266781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c0043c0 0x7f7438153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.796+0000 7f743e400640 1 -- 172.21.15.83:0/466266781 >> 172.21.15.83:0/466266781 conn(0x7f74380b1830 msgr2=0x7f7438158800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.796+0000 7f743e400640 1 -- 172.21.15.83:0/466266781 shutdown_connections 2024-04-25T00:11:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.796+0000 7f743e400640 1 -- 172.21.15.83:0/466266781 wait complete. 2024-04-25T00:11:24.819 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:11:24.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:11:24.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c4a000640 1 Processor -- start 2024-04-25T00:11:24.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c4a000640 1 -- start start 2024-04-25T00:11:24.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c4a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c44164cb0 0x7f7c441670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c4a000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c440ccc60 con 0x7f7c441599f0 2024-04-25T00:11:24.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c4a000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c4406dba0 con 0x7f7c440cc510 2024-04-25T00:11:24.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c4a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7c44159dd0 con 0x7f7c44164cb0 2024-04-25T00:11:24.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c43400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7c441599f0 0x7f7c4415c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52112/0 (socket says 172.21.15.83:52112) 2024-04-25T00:11:24.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c43400640 1 -- 172.21.15.83:0/2696312325 learned_addr learned my addr 172.21.15.83:0/2696312325 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:24.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c43e00640 1 --2- 172.21.15.83:0/2696312325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c44164cb0 0x7f7c441670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3121655760 0 0) 0x7f7c34001120 con 0x7f7c441599f0 2024-04-25T00:11:24.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c28003530 con 0x7f7c441599f0 2024-04-25T00:11:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 731802833 0 0) 0x7f7c4406dba0 con 0x7f7c440cc510 2024-04-25T00:11:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c34001120 con 0x7f7c440cc510 2024-04-25T00:11:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 54657894 0 0) 0x7f7c28003530 con 0x7f7c441599f0 2024-04-25T00:11:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c44164cb0 msgr2=0x7f7c441670a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 --2- 172.21.15.83:0/2696312325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c44164cb0 0x7f7c441670a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 >> v1:172.21.15.86:6789/0 conn(0x7f7c440cc510 legacy=0x7f7c4415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4406de80 con 0x7f7c441599f0 2024-04-25T00:11:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f7c34001850 con 0x7f7c441599f0 2024-04-25T00:11:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c43e00640 1 --2- 172.21.15.83:0/2696312325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c44164cb0 0x7f7c441670a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 >> v1:172.21.15.83:6789/0 conn(0x7f7c441599f0 legacy=0x7f7c4415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 --2- 172.21.15.83:0/2696312325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c28003b80 0x7f7c28003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 --2- 172.21.15.83:0/2696312325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c28004810 0x7f7c28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 --2- 172.21.15.83:0/2696312325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c44164cb0 0x7f7c280051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2800bab0 con 0x7f7c28003b80 2024-04-25T00:11:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2800bc60 con 0x7f7c28004810 2024-04-25T00:11:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7c2800be60 con 0x7f7c44164cb0 2024-04-25T00:11:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c43400640 1 --2- 172.21.15.83:0/2696312325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c28003b80 0x7f7c28003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c3aa00640 1 --2- 172.21.15.83:0/2696312325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c28004810 0x7f7c28004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c43e00640 1 --2- 172.21.15.83:0/2696312325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c44164cb0 0x7f7c280051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c43400640 1 -- 172.21.15.83:0/2696312325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c44164cb0 msgr2=0x7f7c280051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c43400640 1 --2- 172.21.15.83:0/2696312325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c44164cb0 0x7f7c280051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c43400640 1 -- 172.21.15.83:0/2696312325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c28004810 msgr2=0x7f7c28004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c43400640 1 --2- 172.21.15.83:0/2696312325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c28004810 0x7f7c28004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.952+0000 7f7c43400640 1 -- 172.21.15.83:0/2696312325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c28003530 con 0x7f7c28003b80 2024-04-25T00:11:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.956+0000 7f7c43400640 1 --2- 172.21.15.83:0/2696312325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c28003b80 0x7f7c28003f80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400c320 tx=0x7f7c3401cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cecaf5e1a2f277 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.956+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34007dc0 con 0x7f7c28003b80 2024-04-25T00:11:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.956+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c340a9070 con 0x7f7c28003b80 2024-04-25T00:11:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.956+0000 7f7c42a00640 1 -- 172.21.15.83:0/2696312325 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34005d20 con 0x7f7c28003b80 2024-04-25T00:11:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.956+0000 7f7c4a000640 1 -- 172.21.15.83:0/2696312325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c28003b80 msgr2=0x7f7c28003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.956+0000 7f7c4a000640 1 --2- 172.21.15.83:0/2696312325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c28003b80 0x7f7c28003f80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400c320 tx=0x7f7c3401cb40 comp rx=0 tx=0).stop 2024-04-25T00:11:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.956+0000 7f7c43e00640 1 -- 172.21.15.83:0/2696312325 reap_dead start 2024-04-25T00:11:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.956+0000 7f7c4a000640 1 -- 172.21.15.83:0/2696312325 shutdown_connections 2024-04-25T00:11:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.956+0000 7f7c4a000640 1 -- 172.21.15.83:0/2696312325 >> 172.21.15.83:0/2696312325 conn(0x7f7c440b1830 msgr2=0x7f7c440b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.956+0000 7f7c4a000640 1 -- 172.21.15.83:0/2696312325 shutdown_connections 2024-04-25T00:11:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.956+0000 7f7c4a000640 1 -- 172.21.15.83:0/2696312325 wait complete. 2024-04-25T00:11:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c4a000640 1 Processor -- start 2024-04-25T00:11:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c4a000640 1 -- start start 2024-04-25T00:11:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c4a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c28003b80 0x7f7c44160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c4a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c4415bbb0 0x7f7c4415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c4a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c4415f0a0 0x7f7c440cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c4a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c440ce240 con 0x7f7c28003b80 2024-04-25T00:11:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c4a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c440ce0c0 con 0x7f7c4415f0a0 2024-04-25T00:11:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c4a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7c440cf6f0 con 0x7f7c4415bbb0 2024-04-25T00:11:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c43400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c28003b80 0x7f7c44160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c43400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c28003b80 0x7f7c44160a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36218/0 (socket says 172.21.15.83:36218) 2024-04-25T00:11:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c43400640 1 -- 172.21.15.83:0/4252831505 learned_addr learned my addr 172.21.15.83:0/4252831505 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c43e00640 1 --2- 172.21.15.83:0/4252831505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c4415f0a0 0x7f7c440cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c3aa00640 1 --2- 172.21.15.83:0/4252831505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c4415bbb0 0x7f7c4415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c43400640 1 -- 172.21.15.83:0/4252831505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c4415bbb0 msgr2=0x7f7c4415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c43400640 1 --2- 172.21.15.83:0/4252831505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c4415bbb0 0x7f7c4415eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c43400640 1 -- 172.21.15.83:0/4252831505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c4415f0a0 msgr2=0x7f7c440cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:24.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c43400640 1 --2- 172.21.15.83:0/4252831505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c4415f0a0 0x7f7c440cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:24.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c43400640 1 -- 172.21.15.83:0/4252831505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4415d680 con 0x7f7c28003b80 2024-04-25T00:11:24.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c43400640 1 --2- 172.21.15.83:0/4252831505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c28003b80 0x7f7c44160a80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c3401c970 tx=0x7f7c28005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:24.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c3be00640 1 -- 172.21.15.83:0/4252831505 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34007dc0 con 0x7f7c28003b80 2024-04-25T00:11:24.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c3be00640 1 -- 172.21.15.83:0/4252831505 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c340047f0 con 0x7f7c28003b80 2024-04-25T00:11:24.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c3be00640 1 -- 172.21.15.83:0/4252831505 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34004ba0 con 0x7f7c28003b80 2024-04-25T00:11:24.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c4a000640 1 -- 172.21.15.83:0/4252831505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4415d960 con 0x7f7c28003b80 2024-04-25T00:11:24.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.960+0000 7f7c4a000640 1 -- 172.21.15.83:0/4252831505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4415df20 con 0x7f7c28003b80 2024-04-25T00:11:24.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.964+0000 7f7c4a000640 1 -- 172.21.15.83:0/4252831505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c440737f0 con 0x7f7c28003b80 2024-04-25T00:11:24.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.964+0000 7f7c3be00640 1 -- 172.21.15.83:0/4252831505 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7c340a5730 con 0x7f7c28003b80 2024-04-25T00:11:24.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.964+0000 7f7c3be00640 1 --2- 172.21.15.83:0/4252831505 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c200678f0 0x7f7c20069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:24.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.964+0000 7f7c3be00640 1 -- 172.21.15.83:0/4252831505 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7c3411f230 con 0x7f7c28003b80 2024-04-25T00:11:24.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.964+0000 7f7c3aa00640 1 --2- 172.21.15.83:0/4252831505 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c200678f0 0x7f7c20069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:24.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.964+0000 7f7c3aa00640 1 --2- 172.21.15.83:0/4252831505 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c200678f0 0x7f7c20069db0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c440d05e0 tx=0x7f7c2c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:24.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:24.968+0000 7f7c3be00640 1 -- 172.21.15.83:0/4252831505 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7c340c0030 con 0x7f7c28003b80 2024-04-25T00:11:25.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.120+0000 7f7c4a000640 1 -- 172.21.15.83:0/4252831505 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c44072c40 con 0x7f7c200678f0 2024-04-25T00:11:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.132+0000 7f7c3be00640 1 -- 172.21.15.83:0/4252831505 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332558 (secure 0 0 0) 0x7f7c44072c40 con 0x7f7c200678f0 2024-04-25T00:11:25.135 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:11:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.136+0000 7f7c4a000640 1 -- 172.21.15.83:0/4252831505 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c200678f0 msgr2=0x7f7c20069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.136+0000 7f7c4a000640 1 --2- 172.21.15.83:0/4252831505 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c200678f0 0x7f7c20069db0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c440d05e0 tx=0x7f7c2c009040 comp rx=0 tx=0).stop 2024-04-25T00:11:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.136+0000 7f7c4a000640 1 -- 172.21.15.83:0/4252831505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c28003b80 msgr2=0x7f7c44160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.136+0000 7f7c4a000640 1 --2- 172.21.15.83:0/4252831505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c28003b80 0x7f7c44160a80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c3401c970 tx=0x7f7c28005840 comp rx=0 tx=0).stop 2024-04-25T00:11:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.136+0000 7f7c4a000640 1 -- 172.21.15.83:0/4252831505 shutdown_connections 2024-04-25T00:11:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.136+0000 7f7c4a000640 1 --2- 172.21.15.83:0/4252831505 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c200678f0 0x7f7c20069db0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.136+0000 7f7c4a000640 1 --2- 172.21.15.83:0/4252831505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c4415bbb0 0x7f7c4415eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.136+0000 7f7c4a000640 1 --2- 172.21.15.83:0/4252831505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c4415f0a0 0x7f7c440cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.136+0000 7f7c4a000640 1 --2- 172.21.15.83:0/4252831505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c28003b80 0x7f7c44160a80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.136+0000 7f7c4a000640 1 -- 172.21.15.83:0/4252831505 >> 172.21.15.83:0/4252831505 conn(0x7f7c440b1830 msgr2=0x7f7c440b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.136+0000 7f7c4a000640 1 -- 172.21.15.83:0/4252831505 shutdown_connections 2024-04-25T00:11:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.136+0000 7f7c4a000640 1 -- 172.21.15.83:0/4252831505 wait complete. 2024-04-25T00:11:25.157 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-04-25T00:11:24.446699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219404,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755316,"statfs":{"total":773094113280,"available":772869443584,"internally_reserved":0,"allocated":3284992,"data_stored":1991782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002397"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515374+0000","last_change":"2024-04-25T00:11:05.833868+0000","last_active":"2024-04-25T00:11:12.515374+0000","last_peered":"2024-04-25T00:11:12.515374+0000","last_clean":"2024-04-25T00:11:12.515374+0000","last_became_active":"2024-04-25T00:11:05.833603+0000","last_became_peered":"2024-04-25T00:11:05.833603+0000","last_unstale":"2024-04-25T00:11:12.515374+0000","last_undegraded":"2024-04-25T00:11:12.515374+0000","last_fullsized":"2024-04-25T00:11:12.515374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:32.084989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874560+0000","last_change":"2024-04-25T00:11:06.856161+0000","last_active":"2024-04-25T00:11:11.874560+0000","last_peered":"2024-04-25T00:11:11.874560+0000","last_clean":"2024-04-25T00:11:11.874560+0000","last_became_active":"2024-04-25T00:11:06.855897+0000","last_became_peered":"2024-04-25T00:11:06.855897+0000","last_unstale":"2024-04-25T00:11:11.874560+0000","last_undegraded":"2024-04-25T00:11:11.874560+0000","last_fullsized":"2024-04-25T00:11:11.874560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.219642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873904+0000","last_change":"2024-04-25T00:11:10.976976+0000","last_active":"2024-04-25T00:11:11.873904+0000","last_peered":"2024-04-25T00:11:11.873904+0000","last_clean":"2024-04-25T00:11:11.873904+0000","last_became_active":"2024-04-25T00:11:10.975609+0000","last_became_peered":"2024-04-25T00:11:10.975609+0000","last_unstale":"2024-04-25T00:11:11.873904+0000","last_undegraded":"2024-04-25T00:11:11.873904+0000","last_fullsized":"2024-04-25T00:11:11.873904+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:07.644687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874723+0000","last_change":"2024-04-25T00:11:05.835352+0000","last_active":"2024-04-25T00:11:11.874723+0000","last_peered":"2024-04-25T00:11:11.874723+0000","last_clean":"2024-04-25T00:11:11.874723+0000","last_became_active":"2024-04-25T00:11:05.835056+0000","last_became_peered":"2024-04-25T00:11:05.835056+0000","last_unstale":"2024-04-25T00:11:11.874723+0000","last_undegraded":"2024-04-25T00:11:11.874723+0000","last_fullsized":"2024-04-25T00:11:11.874723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515231+0000","last_change":"2024-04-25T00:11:06.855492+0000","last_active":"2024-04-25T00:11:12.515231+0000","last_peered":"2024-04-25T00:11:12.515231+0000","last_clean":"2024-04-25T00:11:12.515231+0000","last_became_active":"2024-04-25T00:11:06.854970+0000","last_became_peered":"2024-04-25T00:11:06.854970+0000","last_unstale":"2024-04-25T00:11:12.515231+0000","last_undegraded":"2024-04-25T00:11:12.515231+0000","last_fullsized":"2024-04-25T00:11:12.515231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515424+0000","last_change":"2024-04-25T00:11:10.899597+0000","last_active":"2024-04-25T00:11:12.515424+0000","last_peered":"2024-04-25T00:11:12.515424+0000","last_clean":"2024-04-25T00:11:12.515424+0000","last_became_active":"2024-04-25T00:11:10.899136+0000","last_became_peered":"2024-04-25T00:11:10.899136+0000","last_unstale":"2024-04-25T00:11:12.515424+0000","last_undegraded":"2024-04-25T00:11:12.515424+0000","last_fullsized":"2024-04-25T00:11:12.515424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:40.036652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514540+0000","last_change":"2024-04-25T00:11:05.848984+0000","last_active":"2024-04-25T00:11:12.514540+0000","last_peered":"2024-04-25T00:11:12.514540+0000","last_clean":"2024-04-25T00:11:12.514540+0000","last_became_active":"2024-04-25T00:11:05.848618+0000","last_became_peered":"2024-04-25T00:11:05.848618+0000","last_unstale":"2024-04-25T00:11:12.514540+0000","last_undegraded":"2024-04-25T00:11:12.514540+0000","last_fullsized":"2024-04-25T00:11:12.514540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:50.369863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514723+0000","last_change":"2024-04-25T00:11:06.875779+0000","last_active":"2024-04-25T00:11:12.514723+0000","last_peered":"2024-04-25T00:11:12.514723+0000","last_clean":"2024-04-25T00:11:12.514723+0000","last_became_active":"2024-04-25T00:11:06.875440+0000","last_became_peered":"2024-04-25T00:11:06.875440+0000","last_unstale":"2024-04-25T00:11:12.514723+0000","last_undegraded":"2024-04-25T00:11:12.514723+0000","last_fullsized":"2024-04-25T00:11:12.514723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:43.565714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516147+0000","last_change":"2024-04-25T00:11:10.899794+0000","last_active":"2024-04-25T00:11:12.516147+0000","last_peered":"2024-04-25T00:11:12.516147+0000","last_clean":"2024-04-25T00:11:12.516147+0000","last_became_active":"2024-04-25T00:11:10.899313+0000","last_became_peered":"2024-04-25T00:11:10.899313+0000","last_unstale":"2024-04-25T00:11:12.516147+0000","last_undegraded":"2024-04-25T00:11:12.516147+0000","last_fullsized":"2024-04-25T00:11:12.516147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:51.269552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514246+0000","last_change":"2024-04-25T00:11:05.847672+0000","last_active":"2024-04-25T00:11:12.514246+0000","last_peered":"2024-04-25T00:11:12.514246+0000","last_clean":"2024-04-25T00:11:12.514246+0000","last_became_active":"2024-04-25T00:11:05.847316+0000","last_became_peered":"2024-04-25T00:11:05.847316+0000","last_unstale":"2024-04-25T00:11:12.514246+0000","last_undegraded":"2024-04-25T00:11:12.514246+0000","last_fullsized":"2024-04-25T00:11:12.514246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515600+0000","last_change":"2024-04-25T00:11:06.860047+0000","last_active":"2024-04-25T00:11:12.515600+0000","last_peered":"2024-04-25T00:11:12.515600+0000","last_clean":"2024-04-25T00:11:12.515600+0000","last_became_active":"2024-04-25T00:11:06.859782+0000","last_became_peered":"2024-04-25T00:11:06.859782+0000","last_unstale":"2024-04-25T00:11:12.515600+0000","last_undegraded":"2024-04-25T00:11:12.515600+0000","last_fullsized":"2024-04-25T00:11:12.515600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:43.612611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516469+0000","last_change":"2024-04-25T00:11:10.895749+0000","last_active":"2024-04-25T00:11:12.516469+0000","last_peered":"2024-04-25T00:11:12.516469+0000","last_clean":"2024-04-25T00:11:12.516469+0000","last_became_active":"2024-04-25T00:11:10.895359+0000","last_became_peered":"2024-04-25T00:11:10.895359+0000","last_unstale":"2024-04-25T00:11:12.516469+0000","last_undegraded":"2024-04-25T00:11:12.516469+0000","last_fullsized":"2024-04-25T00:11:12.516469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873961+0000","last_change":"2024-04-25T00:11:06.865465+0000","last_active":"2024-04-25T00:11:11.873961+0000","last_peered":"2024-04-25T00:11:11.873961+0000","last_clean":"2024-04-25T00:11:11.873961+0000","last_became_active":"2024-04-25T00:11:06.865020+0000","last_became_peered":"2024-04-25T00:11:06.865020+0000","last_unstale":"2024-04-25T00:11:11.873961+0000","last_undegraded":"2024-04-25T00:11:11.873961+0000","last_fullsized":"2024-04-25T00:11:11.873961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514541+0000","last_change":"2024-04-25T00:11:05.844750+0000","last_active":"2024-04-25T00:11:12.514541+0000","last_peered":"2024-04-25T00:11:12.514541+0000","last_clean":"2024-04-25T00:11:12.514541+0000","last_became_active":"2024-04-25T00:11:05.844407+0000","last_became_peered":"2024-04-25T00:11:05.844407+0000","last_unstale":"2024-04-25T00:11:12.514541+0000","last_undegraded":"2024-04-25T00:11:12.514541+0000","last_fullsized":"2024-04-25T00:11:12.514541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:34.248155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516163+0000","last_change":"2024-04-25T00:11:10.893480+0000","last_active":"2024-04-25T00:11:12.516163+0000","last_peered":"2024-04-25T00:11:12.516163+0000","last_clean":"2024-04-25T00:11:12.516163+0000","last_became_active":"2024-04-25T00:11:10.893051+0000","last_became_peered":"2024-04-25T00:11:10.893051+0000","last_unstale":"2024-04-25T00:11:12.516163+0000","last_undegraded":"2024-04-25T00:11:12.516163+0000","last_fullsized":"2024-04-25T00:11:12.516163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:40.531377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874153+0000","last_change":"2024-04-25T00:11:06.845105+0000","last_active":"2024-04-25T00:11:11.874153+0000","last_peered":"2024-04-25T00:11:11.874153+0000","last_clean":"2024-04-25T00:11:11.874153+0000","last_became_active":"2024-04-25T00:11:06.844686+0000","last_became_peered":"2024-04-25T00:11:06.844686+0000","last_unstale":"2024-04-25T00:11:11.874153+0000","last_undegraded":"2024-04-25T00:11:11.874153+0000","last_fullsized":"2024-04-25T00:11:11.874153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:25.890860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514977+0000","last_change":"2024-04-25T00:11:05.836866+0000","last_active":"2024-04-25T00:11:12.514977+0000","last_peered":"2024-04-25T00:11:12.514977+0000","last_clean":"2024-04-25T00:11:12.514977+0000","last_became_active":"2024-04-25T00:11:05.836578+0000","last_became_peered":"2024-04-25T00:11:05.836578+0000","last_unstale":"2024-04-25T00:11:12.514977+0000","last_undegraded":"2024-04-25T00:11:12.514977+0000","last_fullsized":"2024-04-25T00:11:12.514977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515742+0000","last_change":"2024-04-25T00:11:10.903356+0000","last_active":"2024-04-25T00:11:12.515742+0000","last_peered":"2024-04-25T00:11:12.515742+0000","last_clean":"2024-04-25T00:11:12.515742+0000","last_became_active":"2024-04-25T00:11:10.902845+0000","last_became_peered":"2024-04-25T00:11:10.902845+0000","last_unstale":"2024-04-25T00:11:12.515742+0000","last_undegraded":"2024-04-25T00:11:12.515742+0000","last_fullsized":"2024-04-25T00:11:12.515742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:47.662521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874413+0000","last_change":"2024-04-25T00:11:06.849565+0000","last_active":"2024-04-25T00:11:11.874413+0000","last_peered":"2024-04-25T00:11:11.874413+0000","last_clean":"2024-04-25T00:11:11.874413+0000","last_became_active":"2024-04-25T00:11:06.849275+0000","last_became_peered":"2024-04-25T00:11:06.849275+0000","last_unstale":"2024-04-25T00:11:11.874413+0000","last_undegraded":"2024-04-25T00:11:11.874413+0000","last_fullsized":"2024-04-25T00:11:11.874413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514312+0000","last_change":"2024-04-25T00:11:05.850573+0000","last_active":"2024-04-25T00:11:12.514312+0000","last_peered":"2024-04-25T00:11:12.514312+0000","last_clean":"2024-04-25T00:11:12.514312+0000","last_became_active":"2024-04-25T00:11:05.850239+0000","last_became_peered":"2024-04-25T00:11:05.850239+0000","last_unstale":"2024-04-25T00:11:12.514312+0000","last_undegraded":"2024-04-25T00:11:12.514312+0000","last_fullsized":"2024-04-25T00:11:12.514312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:41.285583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515409+0000","last_change":"2024-04-25T00:11:10.974826+0000","last_active":"2024-04-25T00:11:12.515409+0000","last_peered":"2024-04-25T00:11:12.515409+0000","last_clean":"2024-04-25T00:11:12.515409+0000","last_became_active":"2024-04-25T00:11:10.974483+0000","last_became_peered":"2024-04-25T00:11:10.974483+0000","last_unstale":"2024-04-25T00:11:12.515409+0000","last_undegraded":"2024-04-25T00:11:12.515409+0000","last_fullsized":"2024-04-25T00:11:12.515409+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:13.028091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516075+0000","last_change":"2024-04-25T00:11:06.857094+0000","last_active":"2024-04-25T00:11:12.516075+0000","last_peered":"2024-04-25T00:11:12.516075+0000","last_clean":"2024-04-25T00:11:12.516075+0000","last_became_active":"2024-04-25T00:11:06.856782+0000","last_became_peered":"2024-04-25T00:11:06.856782+0000","last_unstale":"2024-04-25T00:11:12.516075+0000","last_undegraded":"2024-04-25T00:11:12.516075+0000","last_fullsized":"2024-04-25T00:11:12.516075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:10.105280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513838+0000","last_change":"2024-04-25T00:11:05.849504+0000","last_active":"2024-04-25T00:11:12.513838+0000","last_peered":"2024-04-25T00:11:12.513838+0000","last_clean":"2024-04-25T00:11:12.513838+0000","last_became_active":"2024-04-25T00:11:05.847651+0000","last_became_peered":"2024-04-25T00:11:05.847651+0000","last_unstale":"2024-04-25T00:11:12.513838+0000","last_undegraded":"2024-04-25T00:11:12.513838+0000","last_fullsized":"2024-04-25T00:11:12.513838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515751+0000","last_change":"2024-04-25T00:11:10.893589+0000","last_active":"2024-04-25T00:11:12.515751+0000","last_peered":"2024-04-25T00:11:12.515751+0000","last_clean":"2024-04-25T00:11:12.515751+0000","last_became_active":"2024-04-25T00:11:10.893172+0000","last_became_peered":"2024-04-25T00:11:10.893172+0000","last_unstale":"2024-04-25T00:11:12.515751+0000","last_undegraded":"2024-04-25T00:11:12.515751+0000","last_fullsized":"2024-04-25T00:11:12.515751+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:53.464531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514257+0000","last_change":"2024-04-25T00:11:06.850048+0000","last_active":"2024-04-25T00:11:12.514257+0000","last_peered":"2024-04-25T00:11:12.514257+0000","last_clean":"2024-04-25T00:11:12.514257+0000","last_became_active":"2024-04-25T00:11:06.849785+0000","last_became_peered":"2024-04-25T00:11:06.849785+0000","last_unstale":"2024-04-25T00:11:12.514257+0000","last_undegraded":"2024-04-25T00:11:12.514257+0000","last_fullsized":"2024-04-25T00:11:12.514257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874790+0000","last_change":"2024-04-25T00:11:05.836100+0000","last_active":"2024-04-25T00:11:11.874790+0000","last_peered":"2024-04-25T00:11:11.874790+0000","last_clean":"2024-04-25T00:11:11.874790+0000","last_became_active":"2024-04-25T00:11:05.835841+0000","last_became_peered":"2024-04-25T00:11:05.835841+0000","last_unstale":"2024-04-25T00:11:11.874790+0000","last_undegraded":"2024-04-25T00:11:11.874790+0000","last_fullsized":"2024-04-25T00:11:11.874790+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:53.690870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515708+0000","last_change":"2024-04-25T00:11:10.975785+0000","last_active":"2024-04-25T00:11:12.515708+0000","last_peered":"2024-04-25T00:11:12.515708+0000","last_clean":"2024-04-25T00:11:12.515708+0000","last_became_active":"2024-04-25T00:11:10.974244+0000","last_became_peered":"2024-04-25T00:11:10.974244+0000","last_unstale":"2024-04-25T00:11:12.515708+0000","last_undegraded":"2024-04-25T00:11:12.515708+0000","last_fullsized":"2024-04-25T00:11:12.515708+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:47.615624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874580+0000","last_change":"2024-04-25T00:11:06.854507+0000","last_active":"2024-04-25T00:11:11.874580+0000","last_peered":"2024-04-25T00:11:11.874580+0000","last_clean":"2024-04-25T00:11:11.874580+0000","last_became_active":"2024-04-25T00:11:06.853694+0000","last_became_peered":"2024-04-25T00:11:06.853694+0000","last_unstale":"2024-04-25T00:11:11.874580+0000","last_undegraded":"2024-04-25T00:11:11.874580+0000","last_fullsized":"2024-04-25T00:11:11.874580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:36.481467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515093+0000","last_change":"2024-04-25T00:11:05.842386+0000","last_active":"2024-04-25T00:11:12.515093+0000","last_peered":"2024-04-25T00:11:12.515093+0000","last_clean":"2024-04-25T00:11:12.515093+0000","last_became_active":"2024-04-25T00:11:05.842064+0000","last_became_peered":"2024-04-25T00:11:05.842064+0000","last_unstale":"2024-04-25T00:11:12.515093+0000","last_undegraded":"2024-04-25T00:11:12.515093+0000","last_fullsized":"2024-04-25T00:11:12.515093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873860+0000","last_change":"2024-04-25T00:11:10.969122+0000","last_active":"2024-04-25T00:11:11.873860+0000","last_peered":"2024-04-25T00:11:11.873860+0000","last_clean":"2024-04-25T00:11:11.873860+0000","last_became_active":"2024-04-25T00:11:10.968532+0000","last_became_peered":"2024-04-25T00:11:10.968532+0000","last_unstale":"2024-04-25T00:11:11.873860+0000","last_undegraded":"2024-04-25T00:11:11.873860+0000","last_fullsized":"2024-04-25T00:11:11.873860+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:23:46.222075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514772+0000","last_change":"2024-04-25T00:11:06.847752+0000","last_active":"2024-04-25T00:11:12.514772+0000","last_peered":"2024-04-25T00:11:12.514772+0000","last_clean":"2024-04-25T00:11:12.514772+0000","last_became_active":"2024-04-25T00:11:06.847378+0000","last_became_peered":"2024-04-25T00:11:06.847378+0000","last_unstale":"2024-04-25T00:11:12.514772+0000","last_undegraded":"2024-04-25T00:11:12.514772+0000","last_fullsized":"2024-04-25T00:11:12.514772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514712+0000","last_change":"2024-04-25T00:11:05.849451+0000","last_active":"2024-04-25T00:11:12.514712+0000","last_peered":"2024-04-25T00:11:12.514712+0000","last_clean":"2024-04-25T00:11:12.514712+0000","last_became_active":"2024-04-25T00:11:05.849028+0000","last_became_peered":"2024-04-25T00:11:05.849028+0000","last_unstale":"2024-04-25T00:11:12.514712+0000","last_undegraded":"2024-04-25T00:11:12.514712+0000","last_fullsized":"2024-04-25T00:11:12.514712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:41.285583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513859+0000","last_change":"2024-04-25T00:11:10.973749+0000","last_active":"2024-04-25T00:11:12.513859+0000","last_peered":"2024-04-25T00:11:12.513859+0000","last_clean":"2024-04-25T00:11:12.513859+0000","last_became_active":"2024-04-25T00:11:10.973369+0000","last_became_peered":"2024-04-25T00:11:10.973369+0000","last_unstale":"2024-04-25T00:11:12.513859+0000","last_undegraded":"2024-04-25T00:11:12.513859+0000","last_fullsized":"2024-04-25T00:11:12.513859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:40.531377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515701+0000","last_change":"2024-04-25T00:11:06.859428+0000","last_active":"2024-04-25T00:11:12.515701+0000","last_peered":"2024-04-25T00:11:12.515701+0000","last_clean":"2024-04-25T00:11:12.515701+0000","last_became_active":"2024-04-25T00:11:06.859101+0000","last_became_peered":"2024-04-25T00:11:06.859101+0000","last_unstale":"2024-04-25T00:11:12.515701+0000","last_undegraded":"2024-04-25T00:11:12.515701+0000","last_fullsized":"2024-04-25T00:11:12.515701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:59.645042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514881+0000","last_change":"2024-04-25T00:11:05.840437+0000","last_active":"2024-04-25T00:11:12.514881+0000","last_peered":"2024-04-25T00:11:12.514881+0000","last_clean":"2024-04-25T00:11:12.514881+0000","last_became_active":"2024-04-25T00:11:05.840120+0000","last_became_peered":"2024-04-25T00:11:05.840120+0000","last_unstale":"2024-04-25T00:11:12.514881+0000","last_undegraded":"2024-04-25T00:11:12.514881+0000","last_fullsized":"2024-04-25T00:11:12.514881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514595+0000","last_change":"2024-04-25T00:11:10.896951+0000","last_active":"2024-04-25T00:11:12.514595+0000","last_peered":"2024-04-25T00:11:12.514595+0000","last_clean":"2024-04-25T00:11:12.514595+0000","last_became_active":"2024-04-25T00:11:10.896547+0000","last_became_peered":"2024-04-25T00:11:10.896547+0000","last_unstale":"2024-04-25T00:11:12.514595+0000","last_undegraded":"2024-04-25T00:11:12.514595+0000","last_fullsized":"2024-04-25T00:11:12.514595+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516507+0000","last_change":"2024-04-25T00:11:06.846626+0000","last_active":"2024-04-25T00:11:12.516507+0000","last_peered":"2024-04-25T00:11:12.516507+0000","last_clean":"2024-04-25T00:11:12.516507+0000","last_became_active":"2024-04-25T00:11:06.846333+0000","last_became_peered":"2024-04-25T00:11:06.846333+0000","last_unstale":"2024-04-25T00:11:12.516507+0000","last_undegraded":"2024-04-25T00:11:12.516507+0000","last_fullsized":"2024-04-25T00:11:12.516507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:42.291050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515107+0000","last_change":"2024-04-25T00:11:05.839781+0000","last_active":"2024-04-25T00:11:12.515107+0000","last_peered":"2024-04-25T00:11:12.515107+0000","last_clean":"2024-04-25T00:11:12.515107+0000","last_became_active":"2024-04-25T00:11:05.839400+0000","last_became_peered":"2024-04-25T00:11:05.839400+0000","last_unstale":"2024-04-25T00:11:12.515107+0000","last_undegraded":"2024-04-25T00:11:12.515107+0000","last_fullsized":"2024-04-25T00:11:12.515107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514811+0000","last_change":"2024-04-25T00:11:10.892411+0000","last_active":"2024-04-25T00:11:12.514811+0000","last_peered":"2024-04-25T00:11:12.514811+0000","last_clean":"2024-04-25T00:11:12.514811+0000","last_became_active":"2024-04-25T00:11:10.892036+0000","last_became_peered":"2024-04-25T00:11:10.892036+0000","last_unstale":"2024-04-25T00:11:12.514811+0000","last_undegraded":"2024-04-25T00:11:12.514811+0000","last_fullsized":"2024-04-25T00:11:12.514811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:07.816085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513913+0000","last_change":"2024-04-25T00:11:06.842835+0000","last_active":"2024-04-25T00:11:12.513913+0000","last_peered":"2024-04-25T00:11:12.513913+0000","last_clean":"2024-04-25T00:11:12.513913+0000","last_became_active":"2024-04-25T00:11:06.842540+0000","last_became_peered":"2024-04-25T00:11:06.842540+0000","last_unstale":"2024-04-25T00:11:12.513913+0000","last_undegraded":"2024-04-25T00:11:12.513913+0000","last_fullsized":"2024-04-25T00:11:12.513913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:00.950133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875360+0000","last_change":"2024-04-25T00:11:05.829300+0000","last_active":"2024-04-25T00:11:11.875360+0000","last_peered":"2024-04-25T00:11:11.875360+0000","last_clean":"2024-04-25T00:11:11.875360+0000","last_became_active":"2024-04-25T00:11:05.828940+0000","last_became_peered":"2024-04-25T00:11:05.828940+0000","last_unstale":"2024-04-25T00:11:11.875360+0000","last_undegraded":"2024-04-25T00:11:11.875360+0000","last_fullsized":"2024-04-25T00:11:11.875360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:15.738255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515526+0000","last_change":"2024-04-25T00:11:10.895520+0000","last_active":"2024-04-25T00:11:12.515526+0000","last_peered":"2024-04-25T00:11:12.515526+0000","last_clean":"2024-04-25T00:11:12.515526+0000","last_became_active":"2024-04-25T00:11:10.895096+0000","last_became_peered":"2024-04-25T00:11:10.895096+0000","last_unstale":"2024-04-25T00:11:12.515526+0000","last_undegraded":"2024-04-25T00:11:12.515526+0000","last_fullsized":"2024-04-25T00:11:12.515526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:51.996256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513553+0000","last_change":"2024-04-25T00:11:06.862814+0000","last_active":"2024-04-25T00:11:12.513553+0000","last_peered":"2024-04-25T00:11:12.513553+0000","last_clean":"2024-04-25T00:11:12.513553+0000","last_became_active":"2024-04-25T00:11:06.862534+0000","last_became_peered":"2024-04-25T00:11:06.862534+0000","last_unstale":"2024-04-25T00:11:12.513553+0000","last_undegraded":"2024-04-25T00:11:12.513553+0000","last_fullsized":"2024-04-25T00:11:12.513553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:39.726185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513959+0000","last_change":"2024-04-25T00:11:05.849841+0000","last_active":"2024-04-25T00:11:12.513959+0000","last_peered":"2024-04-25T00:11:12.513959+0000","last_clean":"2024-04-25T00:11:12.513959+0000","last_became_active":"2024-04-25T00:11:05.847646+0000","last_became_peered":"2024-04-25T00:11:05.847646+0000","last_unstale":"2024-04-25T00:11:12.513959+0000","last_undegraded":"2024-04-25T00:11:12.513959+0000","last_fullsized":"2024-04-25T00:11:12.513959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:36.649759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514155+0000","last_change":"2024-04-25T00:11:10.974744+0000","last_active":"2024-04-25T00:11:12.514155+0000","last_peered":"2024-04-25T00:11:12.514155+0000","last_clean":"2024-04-25T00:11:12.514155+0000","last_became_active":"2024-04-25T00:11:10.973737+0000","last_became_peered":"2024-04-25T00:11:10.973737+0000","last_unstale":"2024-04-25T00:11:12.514155+0000","last_undegraded":"2024-04-25T00:11:12.514155+0000","last_fullsized":"2024-04-25T00:11:12.514155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:13.028091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875061+0000","last_change":"2024-04-25T00:11:06.850937+0000","last_active":"2024-04-25T00:11:11.875061+0000","last_peered":"2024-04-25T00:11:11.875061+0000","last_clean":"2024-04-25T00:11:11.875061+0000","last_became_active":"2024-04-25T00:11:06.850620+0000","last_became_peered":"2024-04-25T00:11:06.850620+0000","last_unstale":"2024-04-25T00:11:11.875061+0000","last_undegraded":"2024-04-25T00:11:11.875061+0000","last_fullsized":"2024-04-25T00:11:11.875061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514342+0000","last_change":"2024-04-25T00:11:05.841594+0000","last_active":"2024-04-25T00:11:12.514342+0000","last_peered":"2024-04-25T00:11:12.514342+0000","last_clean":"2024-04-25T00:11:12.514342+0000","last_became_active":"2024-04-25T00:11:05.838964+0000","last_became_peered":"2024-04-25T00:11:05.838964+0000","last_unstale":"2024-04-25T00:11:12.514342+0000","last_undegraded":"2024-04-25T00:11:12.514342+0000","last_fullsized":"2024-04-25T00:11:12.514342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514070+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:11:12.514070+0000","last_peered":"2024-04-25T00:11:12.514070+0000","last_clean":"2024-04-25T00:11:12.514070+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:11:12.514070+0000","last_undegraded":"2024-04-25T00:11:12.514070+0000","last_fullsized":"2024-04-25T00:11:12.514070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874137+0000","last_change":"2024-04-25T00:11:10.976384+0000","last_active":"2024-04-25T00:11:11.874137+0000","last_peered":"2024-04-25T00:11:11.874137+0000","last_clean":"2024-04-25T00:11:11.874137+0000","last_became_active":"2024-04-25T00:11:10.975924+0000","last_became_peered":"2024-04-25T00:11:10.975924+0000","last_unstale":"2024-04-25T00:11:11.874137+0000","last_undegraded":"2024-04-25T00:11:11.874137+0000","last_fullsized":"2024-04-25T00:11:11.874137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514383+0000","last_change":"2024-04-25T00:11:06.855965+0000","last_active":"2024-04-25T00:11:12.514383+0000","last_peered":"2024-04-25T00:11:12.514383+0000","last_clean":"2024-04-25T00:11:12.514383+0000","last_became_active":"2024-04-25T00:11:06.853447+0000","last_became_peered":"2024-04-25T00:11:06.853447+0000","last_unstale":"2024-04-25T00:11:12.514383+0000","last_undegraded":"2024-04-25T00:11:12.514383+0000","last_fullsized":"2024-04-25T00:11:12.514383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:14.704188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875959+0000","last_change":"2024-04-25T00:11:05.842420+0000","last_active":"2024-04-25T00:11:11.875959+0000","last_peered":"2024-04-25T00:11:11.875959+0000","last_clean":"2024-04-25T00:11:11.875959+0000","last_became_active":"2024-04-25T00:11:05.842102+0000","last_became_peered":"2024-04-25T00:11:05.842102+0000","last_unstale":"2024-04-25T00:11:11.875959+0000","last_undegraded":"2024-04-25T00:11:11.875959+0000","last_fullsized":"2024-04-25T00:11:11.875959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874194+0000","last_change":"2024-04-25T00:11:10.969423+0000","last_active":"2024-04-25T00:11:11.874194+0000","last_peered":"2024-04-25T00:11:11.874194+0000","last_clean":"2024-04-25T00:11:11.874194+0000","last_became_active":"2024-04-25T00:11:10.968799+0000","last_became_peered":"2024-04-25T00:11:10.968799+0000","last_unstale":"2024-04-25T00:11:11.874194+0000","last_undegraded":"2024-04-25T00:11:11.874194+0000","last_fullsized":"2024-04-25T00:11:11.874194+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514836+0000","last_change":"2024-04-25T00:11:06.859625+0000","last_active":"2024-04-25T00:11:12.514836+0000","last_peered":"2024-04-25T00:11:12.514836+0000","last_clean":"2024-04-25T00:11:12.514836+0000","last_became_active":"2024-04-25T00:11:06.859161+0000","last_became_peered":"2024-04-25T00:11:06.859161+0000","last_unstale":"2024-04-25T00:11:12.514836+0000","last_undegraded":"2024-04-25T00:11:12.514836+0000","last_fullsized":"2024-04-25T00:11:12.514836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:36.481467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515075+0000","last_change":"2024-04-25T00:11:05.839603+0000","last_active":"2024-04-25T00:11:12.515075+0000","last_peered":"2024-04-25T00:11:12.515075+0000","last_clean":"2024-04-25T00:11:12.515075+0000","last_became_active":"2024-04-25T00:11:05.838608+0000","last_became_peered":"2024-04-25T00:11:05.838608+0000","last_unstale":"2024-04-25T00:11:12.515075+0000","last_undegraded":"2024-04-25T00:11:12.515075+0000","last_fullsized":"2024-04-25T00:11:12.515075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:53.690870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514117+0000","last_change":"2024-04-25T00:11:10.900742+0000","last_active":"2024-04-25T00:11:12.514117+0000","last_peered":"2024-04-25T00:11:12.514117+0000","last_clean":"2024-04-25T00:11:12.514117+0000","last_became_active":"2024-04-25T00:11:10.900246+0000","last_became_peered":"2024-04-25T00:11:10.900246+0000","last_unstale":"2024-04-25T00:11:12.514117+0000","last_undegraded":"2024-04-25T00:11:12.514117+0000","last_fullsized":"2024-04-25T00:11:12.514117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:07.816085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514836+0000","last_change":"2024-04-25T00:11:06.870304+0000","last_active":"2024-04-25T00:11:12.514836+0000","last_peered":"2024-04-25T00:11:12.514836+0000","last_clean":"2024-04-25T00:11:12.514836+0000","last_became_active":"2024-04-25T00:11:06.870010+0000","last_became_peered":"2024-04-25T00:11:06.870010+0000","last_unstale":"2024-04-25T00:11:12.514836+0000","last_undegraded":"2024-04-25T00:11:12.514836+0000","last_fullsized":"2024-04-25T00:11:12.514836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:08.978181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.876556+0000","last_change":"2024-04-25T00:11:05.839098+0000","last_active":"2024-04-25T00:11:11.876556+0000","last_peered":"2024-04-25T00:11:11.876556+0000","last_clean":"2024-04-25T00:11:11.876556+0000","last_became_active":"2024-04-25T00:11:05.838694+0000","last_became_peered":"2024-04-25T00:11:05.838694+0000","last_unstale":"2024-04-25T00:11:11.876556+0000","last_undegraded":"2024-04-25T00:11:11.876556+0000","last_fullsized":"2024-04-25T00:11:11.876556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:35.709574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513908+0000","last_change":"2024-04-25T00:11:10.973661+0000","last_active":"2024-04-25T00:11:12.513908+0000","last_peered":"2024-04-25T00:11:12.513908+0000","last_clean":"2024-04-25T00:11:12.513908+0000","last_became_active":"2024-04-25T00:11:10.973198+0000","last_became_peered":"2024-04-25T00:11:10.973198+0000","last_unstale":"2024-04-25T00:11:12.513908+0000","last_undegraded":"2024-04-25T00:11:12.513908+0000","last_fullsized":"2024-04-25T00:11:12.513908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:23:46.222075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514875+0000","last_change":"2024-04-25T00:11:06.878377+0000","last_active":"2024-04-25T00:11:12.514875+0000","last_peered":"2024-04-25T00:11:12.514875+0000","last_clean":"2024-04-25T00:11:12.514875+0000","last_became_active":"2024-04-25T00:11:06.878046+0000","last_became_peered":"2024-04-25T00:11:06.878046+0000","last_unstale":"2024-04-25T00:11:12.514875+0000","last_undegraded":"2024-04-25T00:11:12.514875+0000","last_fullsized":"2024-04-25T00:11:12.514875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:59.645042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515717+0000","last_change":"2024-04-25T00:11:05.839981+0000","last_active":"2024-04-25T00:11:12.515717+0000","last_peered":"2024-04-25T00:11:12.515717+0000","last_clean":"2024-04-25T00:11:12.515717+0000","last_became_active":"2024-04-25T00:11:05.839527+0000","last_became_peered":"2024-04-25T00:11:05.839527+0000","last_unstale":"2024-04-25T00:11:12.515717+0000","last_undegraded":"2024-04-25T00:11:12.515717+0000","last_fullsized":"2024-04-25T00:11:12.515717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:14.133421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515439+0000","last_change":"2024-04-25T00:11:10.900219+0000","last_active":"2024-04-25T00:11:12.515439+0000","last_peered":"2024-04-25T00:11:12.515439+0000","last_clean":"2024-04-25T00:11:12.515439+0000","last_became_active":"2024-04-25T00:11:10.899780+0000","last_became_peered":"2024-04-25T00:11:10.899780+0000","last_unstale":"2024-04-25T00:11:12.515439+0000","last_undegraded":"2024-04-25T00:11:12.515439+0000","last_fullsized":"2024-04-25T00:11:12.515439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:29.940770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515975+0000","last_change":"2024-04-25T00:11:06.863060+0000","last_active":"2024-04-25T00:11:12.515975+0000","last_peered":"2024-04-25T00:11:12.515975+0000","last_clean":"2024-04-25T00:11:12.515975+0000","last_became_active":"2024-04-25T00:11:06.862320+0000","last_became_peered":"2024-04-25T00:11:06.862320+0000","last_unstale":"2024-04-25T00:11:12.515975+0000","last_undegraded":"2024-04-25T00:11:12.515975+0000","last_fullsized":"2024-04-25T00:11:12.515975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:49.414621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875244+0000","last_change":"2024-04-25T00:11:05.826923+0000","last_active":"2024-04-25T00:11:11.875244+0000","last_peered":"2024-04-25T00:11:11.875244+0000","last_clean":"2024-04-25T00:11:11.875244+0000","last_became_active":"2024-04-25T00:11:05.826543+0000","last_became_peered":"2024-04-25T00:11:05.826543+0000","last_unstale":"2024-04-25T00:11:11.875244+0000","last_undegraded":"2024-04-25T00:11:11.875244+0000","last_fullsized":"2024-04-25T00:11:11.875244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513942+0000","last_change":"2024-04-25T00:11:10.893190+0000","last_active":"2024-04-25T00:11:12.513942+0000","last_peered":"2024-04-25T00:11:12.513942+0000","last_clean":"2024-04-25T00:11:12.513942+0000","last_became_active":"2024-04-25T00:11:10.892676+0000","last_became_peered":"2024-04-25T00:11:10.892676+0000","last_unstale":"2024-04-25T00:11:12.513942+0000","last_undegraded":"2024-04-25T00:11:12.513942+0000","last_fullsized":"2024-04-25T00:11:12.513942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874062+0000","last_change":"2024-04-25T00:11:06.854380+0000","last_active":"2024-04-25T00:11:11.874062+0000","last_peered":"2024-04-25T00:11:11.874062+0000","last_clean":"2024-04-25T00:11:11.874062+0000","last_became_active":"2024-04-25T00:11:06.854089+0000","last_became_peered":"2024-04-25T00:11:06.854089+0000","last_unstale":"2024-04-25T00:11:11.874062+0000","last_undegraded":"2024-04-25T00:11:11.874062+0000","last_fullsized":"2024-04-25T00:11:11.874062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874367+0000","last_change":"2024-04-25T00:11:05.848317+0000","last_active":"2024-04-25T00:11:11.874367+0000","last_peered":"2024-04-25T00:11:11.874367+0000","last_clean":"2024-04-25T00:11:11.874367+0000","last_became_active":"2024-04-25T00:11:05.847969+0000","last_became_peered":"2024-04-25T00:11:05.847969+0000","last_unstale":"2024-04-25T00:11:11.874367+0000","last_undegraded":"2024-04-25T00:11:11.874367+0000","last_fullsized":"2024-04-25T00:11:11.874367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874567+0000","last_change":"2024-04-25T00:11:10.974665+0000","last_active":"2024-04-25T00:11:11.874567+0000","last_peered":"2024-04-25T00:11:11.874567+0000","last_clean":"2024-04-25T00:11:11.874567+0000","last_became_active":"2024-04-25T00:11:10.974306+0000","last_became_peered":"2024-04-25T00:11:10.974306+0000","last_unstale":"2024-04-25T00:11:11.874567+0000","last_undegraded":"2024-04-25T00:11:11.874567+0000","last_fullsized":"2024-04-25T00:11:11.874567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:09.712979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514422+0000","last_change":"2024-04-25T00:11:06.870996+0000","last_active":"2024-04-25T00:11:12.514422+0000","last_peered":"2024-04-25T00:11:12.514422+0000","last_clean":"2024-04-25T00:11:12.514422+0000","last_became_active":"2024-04-25T00:11:06.870692+0000","last_became_peered":"2024-04-25T00:11:06.870692+0000","last_unstale":"2024-04-25T00:11:12.514422+0000","last_undegraded":"2024-04-25T00:11:12.514422+0000","last_fullsized":"2024-04-25T00:11:12.514422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:36.481467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515027+0000","last_change":"2024-04-25T00:11:05.838027+0000","last_active":"2024-04-25T00:11:12.515027+0000","last_peered":"2024-04-25T00:11:12.515027+0000","last_clean":"2024-04-25T00:11:12.515027+0000","last_became_active":"2024-04-25T00:11:05.837712+0000","last_became_peered":"2024-04-25T00:11:05.837712+0000","last_unstale":"2024-04-25T00:11:12.515027+0000","last_undegraded":"2024-04-25T00:11:12.515027+0000","last_fullsized":"2024-04-25T00:11:12.515027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:53.690870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875060+0000","last_change":"2024-04-25T00:11:10.973710+0000","last_active":"2024-04-25T00:11:11.875060+0000","last_peered":"2024-04-25T00:11:11.875060+0000","last_clean":"2024-04-25T00:11:11.875060+0000","last_became_active":"2024-04-25T00:11:10.973387+0000","last_became_peered":"2024-04-25T00:11:10.973387+0000","last_unstale":"2024-04-25T00:11:11.875060+0000","last_undegraded":"2024-04-25T00:11:11.875060+0000","last_fullsized":"2024-04-25T00:11:11.875060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:39.160853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514207+0000","last_change":"2024-04-25T00:11:06.847434+0000","last_active":"2024-04-25T00:11:12.514207+0000","last_peered":"2024-04-25T00:11:12.514207+0000","last_clean":"2024-04-25T00:11:12.514207+0000","last_became_active":"2024-04-25T00:11:06.847112+0000","last_became_peered":"2024-04-25T00:11:06.847112+0000","last_unstale":"2024-04-25T00:11:12.514207+0000","last_undegraded":"2024-04-25T00:11:12.514207+0000","last_fullsized":"2024-04-25T00:11:12.514207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:27.928927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514359+0000","last_change":"2024-04-25T00:11:05.837912+0000","last_active":"2024-04-25T00:11:12.514359+0000","last_peered":"2024-04-25T00:11:12.514359+0000","last_clean":"2024-04-25T00:11:12.514359+0000","last_became_active":"2024-04-25T00:11:05.837544+0000","last_became_peered":"2024-04-25T00:11:05.837544+0000","last_unstale":"2024-04-25T00:11:12.514359+0000","last_undegraded":"2024-04-25T00:11:12.514359+0000","last_fullsized":"2024-04-25T00:11:12.514359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:14.133421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515539+0000","last_change":"2024-04-25T00:11:10.968996+0000","last_active":"2024-04-25T00:11:12.515539+0000","last_peered":"2024-04-25T00:11:12.515539+0000","last_clean":"2024-04-25T00:11:12.515539+0000","last_became_active":"2024-04-25T00:11:10.968625+0000","last_became_peered":"2024-04-25T00:11:10.968625+0000","last_unstale":"2024-04-25T00:11:12.515539+0000","last_undegraded":"2024-04-25T00:11:12.515539+0000","last_fullsized":"2024-04-25T00:11:12.515539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:07.644687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514303+0000","last_change":"2024-04-25T00:11:06.857856+0000","last_active":"2024-04-25T00:11:12.514303+0000","last_peered":"2024-04-25T00:11:12.514303+0000","last_clean":"2024-04-25T00:11:12.514303+0000","last_became_active":"2024-04-25T00:11:06.857469+0000","last_became_peered":"2024-04-25T00:11:06.857469+0000","last_unstale":"2024-04-25T00:11:12.514303+0000","last_undegraded":"2024-04-25T00:11:12.514303+0000","last_fullsized":"2024-04-25T00:11:12.514303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:47.946346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:15.888628+0000","last_change":"2024-04-25T00:11:05.836491+0000","last_active":"2024-04-25T00:11:15.888628+0000","last_peered":"2024-04-25T00:11:15.888628+0000","last_clean":"2024-04-25T00:11:15.888628+0000","last_became_active":"2024-04-25T00:11:05.835903+0000","last_became_peered":"2024-04-25T00:11:05.835903+0000","last_unstale":"2024-04-25T00:11:15.888628+0000","last_undegraded":"2024-04-25T00:11:15.888628+0000","last_fullsized":"2024-04-25T00:11:15.888628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514752+0000","last_change":"2024-04-25T00:11:10.902870+0000","last_active":"2024-04-25T00:11:12.514752+0000","last_peered":"2024-04-25T00:11:12.514752+0000","last_clean":"2024-04-25T00:11:12.514752+0000","last_became_active":"2024-04-25T00:11:10.896980+0000","last_became_peered":"2024-04-25T00:11:10.896980+0000","last_unstale":"2024-04-25T00:11:12.514752+0000","last_undegraded":"2024-04-25T00:11:12.514752+0000","last_fullsized":"2024-04-25T00:11:12.514752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516066+0000","last_change":"2024-04-25T00:11:06.864856+0000","last_active":"2024-04-25T00:11:12.516066+0000","last_peered":"2024-04-25T00:11:12.516066+0000","last_clean":"2024-04-25T00:11:12.516066+0000","last_became_active":"2024-04-25T00:11:06.863698+0000","last_became_peered":"2024-04-25T00:11:06.863698+0000","last_unstale":"2024-04-25T00:11:12.516066+0000","last_undegraded":"2024-04-25T00:11:12.516066+0000","last_fullsized":"2024-04-25T00:11:12.516066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:35.253622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514963+0000","last_change":"2024-04-25T00:11:05.843072+0000","last_active":"2024-04-25T00:11:12.514963+0000","last_peered":"2024-04-25T00:11:12.514963+0000","last_clean":"2024-04-25T00:11:12.514963+0000","last_became_active":"2024-04-25T00:11:05.842548+0000","last_became_peered":"2024-04-25T00:11:05.842548+0000","last_unstale":"2024-04-25T00:11:12.514963+0000","last_undegraded":"2024-04-25T00:11:12.514963+0000","last_fullsized":"2024-04-25T00:11:12.514963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516339+0000","last_change":"2024-04-25T00:11:10.900180+0000","last_active":"2024-04-25T00:11:12.516339+0000","last_peered":"2024-04-25T00:11:12.516339+0000","last_clean":"2024-04-25T00:11:12.516339+0000","last_became_active":"2024-04-25T00:11:10.899321+0000","last_became_peered":"2024-04-25T00:11:10.899321+0000","last_unstale":"2024-04-25T00:11:12.516339+0000","last_undegraded":"2024-04-25T00:11:12.516339+0000","last_fullsized":"2024-04-25T00:11:12.516339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515303+0000","last_change":"2024-04-25T00:11:06.855642+0000","last_active":"2024-04-25T00:11:12.515303+0000","last_peered":"2024-04-25T00:11:12.515303+0000","last_clean":"2024-04-25T00:11:12.515303+0000","last_became_active":"2024-04-25T00:11:06.855374+0000","last_became_peered":"2024-04-25T00:11:06.855374+0000","last_unstale":"2024-04-25T00:11:12.515303+0000","last_undegraded":"2024-04-25T00:11:12.515303+0000","last_fullsized":"2024-04-25T00:11:12.515303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516104+0000","last_change":"2024-04-25T00:11:05.846074+0000","last_active":"2024-04-25T00:11:12.516104+0000","last_peered":"2024-04-25T00:11:12.516104+0000","last_clean":"2024-04-25T00:11:12.516104+0000","last_became_active":"2024-04-25T00:11:05.845371+0000","last_became_peered":"2024-04-25T00:11:05.845371+0000","last_unstale":"2024-04-25T00:11:12.516104+0000","last_undegraded":"2024-04-25T00:11:12.516104+0000","last_fullsized":"2024-04-25T00:11:12.516104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874963+0000","last_change":"2024-04-25T00:11:10.898262+0000","last_active":"2024-04-25T00:11:11.874963+0000","last_peered":"2024-04-25T00:11:11.874963+0000","last_clean":"2024-04-25T00:11:11.874963+0000","last_became_active":"2024-04-25T00:11:10.897653+0000","last_became_peered":"2024-04-25T00:11:10.897653+0000","last_unstale":"2024-04-25T00:11:11.874963+0000","last_undegraded":"2024-04-25T00:11:11.874963+0000","last_fullsized":"2024-04-25T00:11:11.874963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:12.584307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516002+0000","last_change":"2024-04-25T00:11:06.865617+0000","last_active":"2024-04-25T00:11:12.516002+0000","last_peered":"2024-04-25T00:11:12.516002+0000","last_clean":"2024-04-25T00:11:12.516002+0000","last_became_active":"2024-04-25T00:11:06.865286+0000","last_became_peered":"2024-04-25T00:11:06.865286+0000","last_unstale":"2024-04-25T00:11:12.516002+0000","last_undegraded":"2024-04-25T00:11:12.516002+0000","last_fullsized":"2024-04-25T00:11:12.516002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:03.766175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874678+0000","last_change":"2024-04-25T00:11:05.845733+0000","last_active":"2024-04-25T00:11:11.874678+0000","last_peered":"2024-04-25T00:11:11.874678+0000","last_clean":"2024-04-25T00:11:11.874678+0000","last_became_active":"2024-04-25T00:11:05.845410+0000","last_became_peered":"2024-04-25T00:11:05.845410+0000","last_unstale":"2024-04-25T00:11:11.874678+0000","last_undegraded":"2024-04-25T00:11:11.874678+0000","last_fullsized":"2024-04-25T00:11:11.874678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874418+0000","last_change":"2024-04-25T00:11:10.977787+0000","last_active":"2024-04-25T00:11:11.874418+0000","last_peered":"2024-04-25T00:11:11.874418+0000","last_clean":"2024-04-25T00:11:11.874418+0000","last_became_active":"2024-04-25T00:11:10.977434+0000","last_became_peered":"2024-04-25T00:11:10.977434+0000","last_unstale":"2024-04-25T00:11:11.874418+0000","last_undegraded":"2024-04-25T00:11:11.874418+0000","last_fullsized":"2024-04-25T00:11:11.874418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:27.014726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513916+0000","last_change":"2024-04-25T00:11:10.977497+0000","last_active":"2024-04-25T00:11:12.513916+0000","last_peered":"2024-04-25T00:11:12.513916+0000","last_clean":"2024-04-25T00:11:12.513916+0000","last_became_active":"2024-04-25T00:11:10.973670+0000","last_became_peered":"2024-04-25T00:11:10.973670+0000","last_unstale":"2024-04-25T00:11:12.513916+0000","last_undegraded":"2024-04-25T00:11:12.513916+0000","last_fullsized":"2024-04-25T00:11:12.513916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514138+0000","last_change":"2024-04-25T00:11:06.851862+0000","last_active":"2024-04-25T00:11:12.514138+0000","last_peered":"2024-04-25T00:11:12.514138+0000","last_clean":"2024-04-25T00:11:12.514138+0000","last_became_active":"2024-04-25T00:11:06.851683+0000","last_became_peered":"2024-04-25T00:11:06.851683+0000","last_unstale":"2024-04-25T00:11:12.514138+0000","last_undegraded":"2024-04-25T00:11:12.514138+0000","last_fullsized":"2024-04-25T00:11:12.514138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515242+0000","last_change":"2024-04-25T00:11:05.842503+0000","last_active":"2024-04-25T00:11:12.515242+0000","last_peered":"2024-04-25T00:11:12.515242+0000","last_clean":"2024-04-25T00:11:12.515242+0000","last_became_active":"2024-04-25T00:11:05.842214+0000","last_became_peered":"2024-04-25T00:11:05.842214+0000","last_unstale":"2024-04-25T00:11:12.515242+0000","last_undegraded":"2024-04-25T00:11:12.515242+0000","last_fullsized":"2024-04-25T00:11:12.515242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874565+0000","last_change":"2024-04-25T00:11:10.901930+0000","last_active":"2024-04-25T00:11:11.874565+0000","last_peered":"2024-04-25T00:11:11.874565+0000","last_clean":"2024-04-25T00:11:11.874565+0000","last_became_active":"2024-04-25T00:11:10.901405+0000","last_became_peered":"2024-04-25T00:11:10.901405+0000","last_unstale":"2024-04-25T00:11:11.874565+0000","last_undegraded":"2024-04-25T00:11:11.874565+0000","last_fullsized":"2024-04-25T00:11:11.874565+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:07.816085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874298+0000","last_change":"2024-04-25T00:11:06.855999+0000","last_active":"2024-04-25T00:11:11.874298+0000","last_peered":"2024-04-25T00:11:11.874298+0000","last_clean":"2024-04-25T00:11:11.874298+0000","last_became_active":"2024-04-25T00:11:06.855758+0000","last_became_peered":"2024-04-25T00:11:06.855758+0000","last_unstale":"2024-04-25T00:11:11.874298+0000","last_undegraded":"2024-04-25T00:11:11.874298+0000","last_fullsized":"2024-04-25T00:11:11.874298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874267+0000","last_change":"2024-04-25T00:11:05.834141+0000","last_active":"2024-04-25T00:11:11.874267+0000","last_peered":"2024-04-25T00:11:11.874267+0000","last_clean":"2024-04-25T00:11:11.874267+0000","last_became_active":"2024-04-25T00:11:05.833022+0000","last_became_peered":"2024-04-25T00:11:05.833022+0000","last_unstale":"2024-04-25T00:11:11.874267+0000","last_undegraded":"2024-04-25T00:11:11.874267+0000","last_fullsized":"2024-04-25T00:11:11.874267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:15.738255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874362+0000","last_change":"2024-04-25T00:11:05.844727+0000","last_active":"2024-04-25T00:11:11.874362+0000","last_peered":"2024-04-25T00:11:11.874362+0000","last_clean":"2024-04-25T00:11:11.874362+0000","last_became_active":"2024-04-25T00:11:05.844303+0000","last_became_peered":"2024-04-25T00:11:05.844303+0000","last_unstale":"2024-04-25T00:11:11.874362+0000","last_undegraded":"2024-04-25T00:11:11.874362+0000","last_fullsized":"2024-04-25T00:11:11.874362+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:41.285583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514255+0000","last_change":"2024-04-25T00:11:06.862764+0000","last_active":"2024-04-25T00:11:12.514255+0000","last_peered":"2024-04-25T00:11:12.514255+0000","last_clean":"2024-04-25T00:11:12.514255+0000","last_became_active":"2024-04-25T00:11:06.862372+0000","last_became_peered":"2024-04-25T00:11:06.862372+0000","last_unstale":"2024-04-25T00:11:12.514255+0000","last_undegraded":"2024-04-25T00:11:12.514255+0000","last_fullsized":"2024-04-25T00:11:12.514255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875902+0000","last_change":"2024-04-25T00:11:10.902130+0000","last_active":"2024-04-25T00:11:11.875902+0000","last_peered":"2024-04-25T00:11:11.875902+0000","last_clean":"2024-04-25T00:11:11.875902+0000","last_became_active":"2024-04-25T00:11:10.901717+0000","last_became_peered":"2024-04-25T00:11:10.901717+0000","last_unstale":"2024-04-25T00:11:11.875902+0000","last_undegraded":"2024-04-25T00:11:11.875902+0000","last_fullsized":"2024-04-25T00:11:11.875902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:43.776095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875180+0000","last_change":"2024-04-25T00:11:05.833116+0000","last_active":"2024-04-25T00:11:11.875180+0000","last_peered":"2024-04-25T00:11:11.875180+0000","last_clean":"2024-04-25T00:11:11.875180+0000","last_became_active":"2024-04-25T00:11:05.832202+0000","last_became_peered":"2024-04-25T00:11:05.832202+0000","last_unstale":"2024-04-25T00:11:11.875180+0000","last_undegraded":"2024-04-25T00:11:11.875180+0000","last_fullsized":"2024-04-25T00:11:11.875180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516469+0000","last_change":"2024-04-25T00:11:06.840973+0000","last_active":"2024-04-25T00:11:12.516469+0000","last_peered":"2024-04-25T00:11:12.516469+0000","last_clean":"2024-04-25T00:11:12.516469+0000","last_became_active":"2024-04-25T00:11:06.840670+0000","last_became_peered":"2024-04-25T00:11:06.840670+0000","last_unstale":"2024-04-25T00:11:12.516469+0000","last_undegraded":"2024-04-25T00:11:12.516469+0000","last_fullsized":"2024-04-25T00:11:12.516469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:29.296632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874813+0000","last_change":"2024-04-25T00:11:10.898183+0000","last_active":"2024-04-25T00:11:11.874813+0000","last_peered":"2024-04-25T00:11:11.874813+0000","last_clean":"2024-04-25T00:11:11.874813+0000","last_became_active":"2024-04-25T00:11:10.897738+0000","last_became_peered":"2024-04-25T00:11:10.897738+0000","last_unstale":"2024-04-25T00:11:11.874813+0000","last_undegraded":"2024-04-25T00:11:11.874813+0000","last_fullsized":"2024-04-25T00:11:11.874813+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:44.129988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873858+0000","last_change":"2024-04-25T00:11:06.854098+0000","last_active":"2024-04-25T00:11:11.873858+0000","last_peered":"2024-04-25T00:11:11.873858+0000","last_clean":"2024-04-25T00:11:11.873858+0000","last_became_active":"2024-04-25T00:11:06.853839+0000","last_became_peered":"2024-04-25T00:11:06.853839+0000","last_unstale":"2024-04-25T00:11:11.873858+0000","last_undegraded":"2024-04-25T00:11:11.873858+0000","last_fullsized":"2024-04-25T00:11:11.873858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:00.950133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875315+0000","last_change":"2024-04-25T00:11:06.853055+0000","last_active":"2024-04-25T00:11:11.875315+0000","last_peered":"2024-04-25T00:11:11.875315+0000","last_clean":"2024-04-25T00:11:11.875315+0000","last_became_active":"2024-04-25T00:11:06.852814+0000","last_became_peered":"2024-04-25T00:11:06.852814+0000","last_unstale":"2024-04-25T00:11:11.875315+0000","last_undegraded":"2024-04-25T00:11:11.875315+0000","last_fullsized":"2024-04-25T00:11:11.875315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:42.291050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515002+0000","last_change":"2024-04-25T00:11:06.858759+0000","last_active":"2024-04-25T00:11:12.515002+0000","last_peered":"2024-04-25T00:11:12.515002+0000","last_clean":"2024-04-25T00:11:12.515002+0000","last_became_active":"2024-04-25T00:11:06.858472+0000","last_became_peered":"2024-04-25T00:11:06.858472+0000","last_unstale":"2024-04-25T00:11:12.515002+0000","last_undegraded":"2024-04-25T00:11:12.515002+0000","last_fullsized":"2024-04-25T00:11:12.515002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514169+0000","last_change":"2024-04-25T00:11:06.851389+0000","last_active":"2024-04-25T00:11:12.514169+0000","last_peered":"2024-04-25T00:11:12.514169+0000","last_clean":"2024-04-25T00:11:12.514169+0000","last_became_active":"2024-04-25T00:11:06.845828+0000","last_became_peered":"2024-04-25T00:11:06.845828+0000","last_unstale":"2024-04-25T00:11:12.514169+0000","last_undegraded":"2024-04-25T00:11:12.514169+0000","last_fullsized":"2024-04-25T00:11:12.514169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:00.950133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875364+0000","last_change":"2024-04-25T00:11:06.851807+0000","last_active":"2024-04-25T00:11:11.875364+0000","last_peered":"2024-04-25T00:11:11.875364+0000","last_clean":"2024-04-25T00:11:11.875364+0000","last_became_active":"2024-04-25T00:11:06.851495+0000","last_became_peered":"2024-04-25T00:11:06.851495+0000","last_unstale":"2024-04-25T00:11:11.875364+0000","last_undegraded":"2024-04-25T00:11:11.875364+0000","last_fullsized":"2024-04-25T00:11:11.875364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:49.414621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514751+0000","last_change":"2024-04-25T00:11:06.848495+0000","last_active":"2024-04-25T00:11:12.514751+0000","last_peered":"2024-04-25T00:11:12.514751+0000","last_clean":"2024-04-25T00:11:12.514751+0000","last_became_active":"2024-04-25T00:11:06.848242+0000","last_became_peered":"2024-04-25T00:11:06.848242+0000","last_unstale":"2024-04-25T00:11:12.514751+0000","last_undegraded":"2024-04-25T00:11:12.514751+0000","last_fullsized":"2024-04-25T00:11:12.514751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874704+0000","last_change":"2024-04-25T00:11:06.850991+0000","last_active":"2024-04-25T00:11:11.874704+0000","last_peered":"2024-04-25T00:11:11.874704+0000","last_clean":"2024-04-25T00:11:11.874704+0000","last_became_active":"2024-04-25T00:11:06.850666+0000","last_became_peered":"2024-04-25T00:11:06.850666+0000","last_unstale":"2024-04-25T00:11:11.874704+0000","last_undegraded":"2024-04-25T00:11:11.874704+0000","last_fullsized":"2024-04-25T00:11:11.874704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:25.890860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874498+0000","last_change":"2024-04-25T00:11:06.857585+0000","last_active":"2024-04-25T00:11:11.874498+0000","last_peered":"2024-04-25T00:11:11.874498+0000","last_clean":"2024-04-25T00:11:11.874498+0000","last_became_active":"2024-04-25T00:11:06.857315+0000","last_became_peered":"2024-04-25T00:11:06.857315+0000","last_unstale":"2024-04-25T00:11:11.874498+0000","last_undegraded":"2024-04-25T00:11:11.874498+0000","last_fullsized":"2024-04-25T00:11:11.874498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:43.612611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513979+0000","last_change":"2024-04-25T00:11:06.858752+0000","last_active":"2024-04-25T00:11:12.513979+0000","last_peered":"2024-04-25T00:11:12.513979+0000","last_clean":"2024-04-25T00:11:12.513979+0000","last_became_active":"2024-04-25T00:11:06.858428+0000","last_became_peered":"2024-04-25T00:11:06.858428+0000","last_unstale":"2024-04-25T00:11:12.513979+0000","last_undegraded":"2024-04-25T00:11:12.513979+0000","last_fullsized":"2024-04-25T00:11:12.513979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.219642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874132+0000","last_change":"2024-04-25T00:11:06.870006+0000","last_active":"2024-04-25T00:11:11.874132+0000","last_peered":"2024-04-25T00:11:11.874132+0000","last_clean":"2024-04-25T00:11:11.874132+0000","last_became_active":"2024-04-25T00:11:06.869687+0000","last_became_peered":"2024-04-25T00:11:06.869687+0000","last_unstale":"2024-04-25T00:11:11.874132+0000","last_undegraded":"2024-04-25T00:11:11.874132+0000","last_fullsized":"2024-04-25T00:11:11.874132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:43.565714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514922+0000","last_change":"2024-04-25T00:11:06.863927+0000","last_active":"2024-04-25T00:11:12.514922+0000","last_peered":"2024-04-25T00:11:12.514922+0000","last_clean":"2024-04-25T00:11:12.514922+0000","last_became_active":"2024-04-25T00:11:06.863758+0000","last_became_peered":"2024-04-25T00:11:06.863758+0000","last_unstale":"2024-04-25T00:11:12.514922+0000","last_undegraded":"2024-04-25T00:11:12.514922+0000","last_fullsized":"2024-04-25T00:11:12.514922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:25.890860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513912+0000","last_change":"2024-04-25T00:11:06.852070+0000","last_active":"2024-04-25T00:11:12.513912+0000","last_peered":"2024-04-25T00:11:12.513912+0000","last_clean":"2024-04-25T00:11:12.513912+0000","last_became_active":"2024-04-25T00:11:06.849951+0000","last_became_peered":"2024-04-25T00:11:06.849951+0000","last_unstale":"2024-04-25T00:11:12.513912+0000","last_undegraded":"2024-04-25T00:11:12.513912+0000","last_fullsized":"2024-04-25T00:11:12.513912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:54.204077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874289+0000","last_change":"2024-04-25T00:11:06.867310+0000","last_active":"2024-04-25T00:11:11.874289+0000","last_peered":"2024-04-25T00:11:11.874289+0000","last_clean":"2024-04-25T00:11:11.874289+0000","last_became_active":"2024-04-25T00:11:06.867010+0000","last_became_peered":"2024-04-25T00:11:06.867010+0000","last_unstale":"2024-04-25T00:11:11.874289+0000","last_undegraded":"2024-04-25T00:11:11.874289+0000","last_fullsized":"2024-04-25T00:11:11.874289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515126+0000","last_change":"2024-04-25T00:11:06.855941+0000","last_active":"2024-04-25T00:11:12.515126+0000","last_peered":"2024-04-25T00:11:12.515126+0000","last_clean":"2024-04-25T00:11:12.515126+0000","last_became_active":"2024-04-25T00:11:06.855737+0000","last_became_peered":"2024-04-25T00:11:06.855737+0000","last_unstale":"2024-04-25T00:11:12.515126+0000","last_undegraded":"2024-04-25T00:11:12.515126+0000","last_fullsized":"2024-04-25T00:11:12.515126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:18.665263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.876458+0000","last_change":"2024-04-25T00:11:06.865551+0000","last_active":"2024-04-25T00:11:11.876458+0000","last_peered":"2024-04-25T00:11:11.876458+0000","last_clean":"2024-04-25T00:11:11.876458+0000","last_became_active":"2024-04-25T00:11:06.865289+0000","last_became_peered":"2024-04-25T00:11:06.865289+0000","last_unstale":"2024-04-25T00:11:11.876458+0000","last_undegraded":"2024-04-25T00:11:11.876458+0000","last_fullsized":"2024-04-25T00:11:11.876458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:51.375329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516337+0000","last_change":"2024-04-25T00:11:06.863407+0000","last_active":"2024-04-25T00:11:12.516337+0000","last_peered":"2024-04-25T00:11:12.516337+0000","last_clean":"2024-04-25T00:11:12.516337+0000","last_became_active":"2024-04-25T00:11:06.863119+0000","last_became_peered":"2024-04-25T00:11:06.863119+0000","last_unstale":"2024-04-25T00:11:12.516337+0000","last_undegraded":"2024-04-25T00:11:12.516337+0000","last_fullsized":"2024-04-25T00:11:12.516337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:08.978181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513899+0000","last_change":"2024-04-25T00:11:06.854703+0000","last_active":"2024-04-25T00:11:12.513899+0000","last_peered":"2024-04-25T00:11:12.513899+0000","last_clean":"2024-04-25T00:11:12.513899+0000","last_became_active":"2024-04-25T00:11:06.854412+0000","last_became_peered":"2024-04-25T00:11:06.854412+0000","last_unstale":"2024-04-25T00:11:12.513899+0000","last_undegraded":"2024-04-25T00:11:12.513899+0000","last_fullsized":"2024-04-25T00:11:12.513899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:17.037033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514230+0000","last_change":"2024-04-25T00:11:06.850481+0000","last_active":"2024-04-25T00:11:12.514230+0000","last_peered":"2024-04-25T00:11:12.514230+0000","last_clean":"2024-04-25T00:11:12.514230+0000","last_became_active":"2024-04-25T00:11:06.850219+0000","last_became_peered":"2024-04-25T00:11:06.850219+0000","last_unstale":"2024-04-25T00:11:12.514230+0000","last_undegraded":"2024-04-25T00:11:12.514230+0000","last_fullsized":"2024-04-25T00:11:12.514230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514123+0000","last_change":"2024-04-25T00:11:06.841945+0000","last_active":"2024-04-25T00:11:12.514123+0000","last_peered":"2024-04-25T00:11:12.514123+0000","last_clean":"2024-04-25T00:11:12.514123+0000","last_became_active":"2024-04-25T00:11:06.841645+0000","last_became_peered":"2024-04-25T00:11:06.841645+0000","last_unstale":"2024-04-25T00:11:12.514123+0000","last_undegraded":"2024-04-25T00:11:12.514123+0000","last_fullsized":"2024-04-25T00:11:12.514123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:40.495315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874037+0000","last_change":"2024-04-25T00:11:06.856378+0000","last_active":"2024-04-25T00:11:11.874037+0000","last_peered":"2024-04-25T00:11:11.874037+0000","last_clean":"2024-04-25T00:11:11.874037+0000","last_became_active":"2024-04-25T00:11:06.856108+0000","last_became_peered":"2024-04-25T00:11:06.856108+0000","last_unstale":"2024-04-25T00:11:11.874037+0000","last_undegraded":"2024-04-25T00:11:11.874037+0000","last_fullsized":"2024-04-25T00:11:11.874037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515971+0000","last_change":"2024-04-25T00:11:06.868107+0000","last_active":"2024-04-25T00:11:12.515971+0000","last_peered":"2024-04-25T00:11:12.515971+0000","last_clean":"2024-04-25T00:11:12.515971+0000","last_became_active":"2024-04-25T00:11:06.867822+0000","last_became_peered":"2024-04-25T00:11:06.867822+0000","last_unstale":"2024-04-25T00:11:12.515971+0000","last_undegraded":"2024-04-25T00:11:12.515971+0000","last_fullsized":"2024-04-25T00:11:12.515971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:51.375329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874207+0000","last_change":"2024-04-25T00:11:06.848480+0000","last_active":"2024-04-25T00:11:11.874207+0000","last_peered":"2024-04-25T00:11:11.874207+0000","last_clean":"2024-04-25T00:11:11.874207+0000","last_became_active":"2024-04-25T00:11:06.848228+0000","last_became_peered":"2024-04-25T00:11:06.848228+0000","last_unstale":"2024-04-25T00:11:11.874207+0000","last_undegraded":"2024-04-25T00:11:11.874207+0000","last_fullsized":"2024-04-25T00:11:11.874207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874653+0000","last_change":"2024-04-25T00:11:06.852645+0000","last_active":"2024-04-25T00:11:11.874653+0000","last_peered":"2024-04-25T00:11:11.874653+0000","last_clean":"2024-04-25T00:11:11.874653+0000","last_became_active":"2024-04-25T00:11:06.852364+0000","last_became_peered":"2024-04-25T00:11:06.852364+0000","last_unstale":"2024-04-25T00:11:11.874653+0000","last_undegraded":"2024-04-25T00:11:11.874653+0000","last_fullsized":"2024-04-25T00:11:11.874653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514727+0000","last_change":"2024-04-25T00:11:06.858052+0000","last_active":"2024-04-25T00:11:12.514727+0000","last_peered":"2024-04-25T00:11:12.514727+0000","last_clean":"2024-04-25T00:11:12.514727+0000","last_became_active":"2024-04-25T00:11:06.857402+0000","last_became_peered":"2024-04-25T00:11:06.857402+0000","last_unstale":"2024-04-25T00:11:12.514727+0000","last_undegraded":"2024-04-25T00:11:12.514727+0000","last_fullsized":"2024-04-25T00:11:12.514727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:03.766175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874165+0000","last_change":"2024-04-25T00:11:06.852942+0000","last_active":"2024-04-25T00:11:11.874165+0000","last_peered":"2024-04-25T00:11:11.874165+0000","last_clean":"2024-04-25T00:11:11.874165+0000","last_became_active":"2024-04-25T00:11:06.852671+0000","last_became_peered":"2024-04-25T00:11:06.852671+0000","last_unstale":"2024-04-25T00:11:11.874165+0000","last_undegraded":"2024-04-25T00:11:11.874165+0000","last_fullsized":"2024-04-25T00:11:11.874165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515899+0000","last_change":"2024-04-25T00:11:06.853657+0000","last_active":"2024-04-25T00:11:12.515899+0000","last_peered":"2024-04-25T00:11:12.515899+0000","last_clean":"2024-04-25T00:11:12.515899+0000","last_became_active":"2024-04-25T00:11:06.853342+0000","last_became_peered":"2024-04-25T00:11:06.853342+0000","last_unstale":"2024-04-25T00:11:12.515899+0000","last_undegraded":"2024-04-25T00:11:12.515899+0000","last_fullsized":"2024-04-25T00:11:12.515899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:43.612611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514817+0000","last_change":"2024-04-25T00:11:06.854058+0000","last_active":"2024-04-25T00:11:12.514817+0000","last_peered":"2024-04-25T00:11:12.514817+0000","last_clean":"2024-04-25T00:11:12.514817+0000","last_became_active":"2024-04-25T00:11:06.853803+0000","last_became_peered":"2024-04-25T00:11:06.853803+0000","last_unstale":"2024-04-25T00:11:12.514817+0000","last_undegraded":"2024-04-25T00:11:12.514817+0000","last_fullsized":"2024-04-25T00:11:12.514817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514199+0000","last_change":"2024-04-25T00:11:06.851312+0000","last_active":"2024-04-25T00:11:12.514199+0000","last_peered":"2024-04-25T00:11:12.514199+0000","last_clean":"2024-04-25T00:11:12.514199+0000","last_became_active":"2024-04-25T00:11:06.851038+0000","last_became_peered":"2024-04-25T00:11:06.851038+0000","last_unstale":"2024-04-25T00:11:12.514199+0000","last_undegraded":"2024-04-25T00:11:12.514199+0000","last_fullsized":"2024-04-25T00:11:12.514199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.219642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514597+0000","last_change":"2024-04-25T00:11:06.860160+0000","last_active":"2024-04-25T00:11:12.514597+0000","last_peered":"2024-04-25T00:11:12.514597+0000","last_clean":"2024-04-25T00:11:12.514597+0000","last_became_active":"2024-04-25T00:11:06.859259+0000","last_became_peered":"2024-04-25T00:11:06.859259+0000","last_unstale":"2024-04-25T00:11:12.514597+0000","last_undegraded":"2024-04-25T00:11:12.514597+0000","last_fullsized":"2024-04-25T00:11:12.514597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:49.414621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513796+0000","last_change":"2024-04-25T00:11:06.848844+0000","last_active":"2024-04-25T00:11:12.513796+0000","last_peered":"2024-04-25T00:11:12.513796+0000","last_clean":"2024-04-25T00:11:12.513796+0000","last_became_active":"2024-04-25T00:11:06.848597+0000","last_became_peered":"2024-04-25T00:11:06.848597+0000","last_unstale":"2024-04-25T00:11:12.513796+0000","last_undegraded":"2024-04-25T00:11:12.513796+0000","last_fullsized":"2024-04-25T00:11:12.513796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515039+0000","last_change":"2024-04-25T00:11:06.844764+0000","last_active":"2024-04-25T00:11:12.515039+0000","last_peered":"2024-04-25T00:11:12.515039+0000","last_clean":"2024-04-25T00:11:12.515039+0000","last_became_active":"2024-04-25T00:11:06.844484+0000","last_became_peered":"2024-04-25T00:11:06.844484+0000","last_unstale":"2024-04-25T00:11:12.515039+0000","last_undegraded":"2024-04-25T00:11:12.515039+0000","last_fullsized":"2024-04-25T00:11:12.515039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514486+0000","last_change":"2024-04-25T00:11:06.860556+0000","last_active":"2024-04-25T00:11:12.514486+0000","last_peered":"2024-04-25T00:11:12.514486+0000","last_clean":"2024-04-25T00:11:12.514486+0000","last_became_active":"2024-04-25T00:11:06.860210+0000","last_became_peered":"2024-04-25T00:11:06.860210+0000","last_unstale":"2024-04-25T00:11:12.514486+0000","last_undegraded":"2024-04-25T00:11:12.514486+0000","last_fullsized":"2024-04-25T00:11:12.514486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:17.037033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:11:25.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:11:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.336+0000 7f6152400640 1 Processor -- start 2024-04-25T00:11:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.336+0000 7f6152400640 1 -- start start 2024-04-25T00:11:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.336+0000 7f6152400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f614c15e250 0x7f614c0d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:25.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.336+0000 7f6152400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f614c0d2c60 con 0x7f614c0d1ed0 2024-04-25T00:11:25.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.336+0000 7f6152400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f614c0b7850 con 0x7f614c0cf9e0 2024-04-25T00:11:25.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.336+0000 7f6152400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f614c0b7b30 con 0x7f614c15e250 2024-04-25T00:11:25.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.336+0000 7f614aa00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f614c0d1ed0 0x7f614c0b88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52120/0 (socket says 172.21.15.83:52120) 2024-04-25T00:11:25.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.336+0000 7f614aa00640 1 -- 172.21.15.83:0/3532683237 learned_addr learned my addr 172.21.15.83:0/3532683237 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.336+0000 7f614b400640 1 --2- 172.21.15.83:0/3532683237 >> v2:172.21.15.83:3300/0 conn(0x7f614c15e250 0x7f614c0d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.336+0000 7f614b400640 1 -- 172.21.15.83:0/3532683237 >> v1:172.21.15.86:6789/0 conn(0x7f614c0cf9e0 legacy=0x7f614c0b9000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:11:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.336+0000 7f614a000640 1 -- 172.21.15.83:0/3532683237 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1589117972 0 0) 0x7f614c0d2c60 con 0x7f614c0d1ed0 2024-04-25T00:11:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.336+0000 7f614b400640 1 -- 172.21.15.83:0/3532683237 >> v1:172.21.15.83:6789/0 conn(0x7f614c0d1ed0 legacy=0x7f614c0b88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.336+0000 7f614b400640 1 -- 172.21.15.83:0/3532683237 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c0b7e10 con 0x7f614c15e250 2024-04-25T00:11:25.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.340+0000 7f614b400640 1 --2- 172.21.15.83:0/3532683237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f614c15e250 0x7f614c0d1990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f613c0097c0 tx=0x7f613c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be63db51221aea8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:25.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.340+0000 7f614a000640 1 -- 172.21.15.83:0/3532683237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c0b1070 con 0x7f614c15e250 2024-04-25T00:11:25.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.340+0000 7f614a000640 1 -- 172.21.15.83:0/3532683237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613c0ac600 con 0x7f614c15e250 2024-04-25T00:11:25.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.340+0000 7f614a000640 1 -- 172.21.15.83:0/3532683237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c0ac9d0 con 0x7f614c15e250 2024-04-25T00:11:25.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.340+0000 7f6152400640 1 -- 172.21.15.83:0/3532683237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f614c15e250 msgr2=0x7f614c0d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.340+0000 7f6152400640 1 --2- 172.21.15.83:0/3532683237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f614c15e250 0x7f614c0d1990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f613c0097c0 tx=0x7f613c01c3a0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.340+0000 7f6152400640 1 -- 172.21.15.83:0/3532683237 shutdown_connections 2024-04-25T00:11:25.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.340+0000 7f6152400640 1 --2- 172.21.15.83:0/3532683237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f614c15e250 0x7f614c0d1990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.340+0000 7f6152400640 1 -- 172.21.15.83:0/3532683237 >> 172.21.15.83:0/3532683237 conn(0x7f614c0d3e50 msgr2=0x7f614c0d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:25.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.340+0000 7f6152400640 1 -- 172.21.15.83:0/3532683237 shutdown_connections 2024-04-25T00:11:25.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.340+0000 7f6152400640 1 -- 172.21.15.83:0/3532683237 wait complete. 2024-04-25T00:11:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f6152400640 1 Processor -- start 2024-04-25T00:11:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f6152400640 1 -- start start 2024-04-25T00:11:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f6152400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f614c0cf9e0 0x7f614c0c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f6152400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f614c0d1ed0 0x7f614c0ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f6152400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f614c15e250 0x7f614c0c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f6152400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f614c16c430 con 0x7f614c15e250 2024-04-25T00:11:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f6152400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f614c0b9b20 con 0x7f614c0cf9e0 2024-04-25T00:11:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f6152400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f614c16c5b0 con 0x7f614c0d1ed0 2024-04-25T00:11:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f614be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f614c15e250 0x7f614c0c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f614be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f614c15e250 0x7f614c0c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36246/0 (socket says 172.21.15.83:36246) 2024-04-25T00:11:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f614be00640 1 -- 172.21.15.83:0/466306046 learned_addr learned my addr 172.21.15.83:0/466306046 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f614b400640 1 --2- 172.21.15.83:0/466306046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f614c0cf9e0 0x7f614c0c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f614aa00640 1 --2- 172.21.15.83:0/466306046 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f614c0d1ed0 0x7f614c0ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f614be00640 1 -- 172.21.15.83:0/466306046 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f614c0d1ed0 msgr2=0x7f614c0ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f614be00640 1 --2- 172.21.15.83:0/466306046 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f614c0d1ed0 0x7f614c0ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f614be00640 1 -- 172.21.15.83:0/466306046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f614c0cf9e0 msgr2=0x7f614c0c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f614be00640 1 --2- 172.21.15.83:0/466306046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f614c0cf9e0 0x7f614c0c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f614be00640 1 -- 172.21.15.83:0/466306046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c0c88d0 con 0x7f614c15e250 2024-04-25T00:11:25.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f614be00640 1 --2- 172.21.15.83:0/466306046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f614c15e250 0x7f614c0c8390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f614400b990 tx=0x7f614400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:25.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f6143400640 1 -- 172.21.15.83:0/466306046 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144019070 con 0x7f614c15e250 2024-04-25T00:11:25.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f6143400640 1 -- 172.21.15.83:0/466306046 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f614400cba0 con 0x7f614c15e250 2024-04-25T00:11:25.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f6143400640 1 -- 172.21.15.83:0/466306046 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144014440 con 0x7f614c15e250 2024-04-25T00:11:25.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f6152400640 1 -- 172.21.15.83:0/466306046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614c0c9360 con 0x7f614c15e250 2024-04-25T00:11:25.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.344+0000 7f6152400640 1 -- 172.21.15.83:0/466306046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614c0c9920 con 0x7f614c15e250 2024-04-25T00:11:25.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.348+0000 7f6152400640 1 -- 172.21.15.83:0/466306046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614c0a1dc0 con 0x7f614c15e250 2024-04-25T00:11:25.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.348+0000 7f6143400640 1 -- 172.21.15.83:0/466306046 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f614401c070 con 0x7f614c15e250 2024-04-25T00:11:25.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.348+0000 7f6143400640 1 --2- 172.21.15.83:0/466306046 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f612c0678f0 0x7f612c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:25.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.348+0000 7f6143400640 1 -- 172.21.15.83:0/466306046 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6144085d90 con 0x7f614c15e250 2024-04-25T00:11:25.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.348+0000 7f614b400640 1 --2- 172.21.15.83:0/466306046 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f612c0678f0 0x7f612c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:25.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.348+0000 7f614b400640 1 --2- 172.21.15.83:0/466306046 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f612c0678f0 0x7f612c069db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f613c0ae000 tx=0x7f613c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:25.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.352+0000 7f6143400640 1 -- 172.21.15.83:0/466306046 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f61440564f0 con 0x7f614c15e250 2024-04-25T00:11:25.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.504+0000 7f6152400640 1 -- 172.21.15.83:0/466306046 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f614c0d02d0 con 0x7f612c0678f0 2024-04-25T00:11:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.512+0000 7f6143400640 1 -- 172.21.15.83:0/466306046 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332558 (secure 0 0 0) 0x7f614c0d02d0 con 0x7f612c0678f0 2024-04-25T00:11:25.519 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:11:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.516+0000 7f6152400640 1 -- 172.21.15.83:0/466306046 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f612c0678f0 msgr2=0x7f612c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.516+0000 7f6152400640 1 --2- 172.21.15.83:0/466306046 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f612c0678f0 0x7f612c069db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f613c0ae000 tx=0x7f613c0a7000 comp rx=0 tx=0).stop 2024-04-25T00:11:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.516+0000 7f6152400640 1 -- 172.21.15.83:0/466306046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f614c15e250 msgr2=0x7f614c0c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.516+0000 7f6152400640 1 --2- 172.21.15.83:0/466306046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f614c15e250 0x7f614c0c8390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f614400b990 tx=0x7f614400be50 comp rx=0 tx=0).stop 2024-04-25T00:11:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.516+0000 7f6152400640 1 -- 172.21.15.83:0/466306046 shutdown_connections 2024-04-25T00:11:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.516+0000 7f6152400640 1 --2- 172.21.15.83:0/466306046 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f612c0678f0 0x7f612c069db0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.516+0000 7f6152400640 1 --2- 172.21.15.83:0/466306046 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f614c0d1ed0 0x7f614c0ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.516+0000 7f6152400640 1 --2- 172.21.15.83:0/466306046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f614c15e250 0x7f614c0c8390 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.516+0000 7f6152400640 1 --2- 172.21.15.83:0/466306046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f614c0cf9e0 0x7f614c0c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.516+0000 7f6152400640 1 -- 172.21.15.83:0/466306046 >> 172.21.15.83:0/466306046 conn(0x7f614c0d3e50 msgr2=0x7f614c06cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.516+0000 7f6152400640 1 -- 172.21.15.83:0/466306046 shutdown_connections 2024-04-25T00:11:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.516+0000 7f6152400640 1 -- 172.21.15.83:0/466306046 wait complete. 2024-04-25T00:11:25.540 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-04-25T00:11:24.446699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219404,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755316,"statfs":{"total":773094113280,"available":772869443584,"internally_reserved":0,"allocated":3284992,"data_stored":1991782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002397"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515374+0000","last_change":"2024-04-25T00:11:05.833868+0000","last_active":"2024-04-25T00:11:12.515374+0000","last_peered":"2024-04-25T00:11:12.515374+0000","last_clean":"2024-04-25T00:11:12.515374+0000","last_became_active":"2024-04-25T00:11:05.833603+0000","last_became_peered":"2024-04-25T00:11:05.833603+0000","last_unstale":"2024-04-25T00:11:12.515374+0000","last_undegraded":"2024-04-25T00:11:12.515374+0000","last_fullsized":"2024-04-25T00:11:12.515374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:32.084989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874560+0000","last_change":"2024-04-25T00:11:06.856161+0000","last_active":"2024-04-25T00:11:11.874560+0000","last_peered":"2024-04-25T00:11:11.874560+0000","last_clean":"2024-04-25T00:11:11.874560+0000","last_became_active":"2024-04-25T00:11:06.855897+0000","last_became_peered":"2024-04-25T00:11:06.855897+0000","last_unstale":"2024-04-25T00:11:11.874560+0000","last_undegraded":"2024-04-25T00:11:11.874560+0000","last_fullsized":"2024-04-25T00:11:11.874560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.219642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873904+0000","last_change":"2024-04-25T00:11:10.976976+0000","last_active":"2024-04-25T00:11:11.873904+0000","last_peered":"2024-04-25T00:11:11.873904+0000","last_clean":"2024-04-25T00:11:11.873904+0000","last_became_active":"2024-04-25T00:11:10.975609+0000","last_became_peered":"2024-04-25T00:11:10.975609+0000","last_unstale":"2024-04-25T00:11:11.873904+0000","last_undegraded":"2024-04-25T00:11:11.873904+0000","last_fullsized":"2024-04-25T00:11:11.873904+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:07.644687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874723+0000","last_change":"2024-04-25T00:11:05.835352+0000","last_active":"2024-04-25T00:11:11.874723+0000","last_peered":"2024-04-25T00:11:11.874723+0000","last_clean":"2024-04-25T00:11:11.874723+0000","last_became_active":"2024-04-25T00:11:05.835056+0000","last_became_peered":"2024-04-25T00:11:05.835056+0000","last_unstale":"2024-04-25T00:11:11.874723+0000","last_undegraded":"2024-04-25T00:11:11.874723+0000","last_fullsized":"2024-04-25T00:11:11.874723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515231+0000","last_change":"2024-04-25T00:11:06.855492+0000","last_active":"2024-04-25T00:11:12.515231+0000","last_peered":"2024-04-25T00:11:12.515231+0000","last_clean":"2024-04-25T00:11:12.515231+0000","last_became_active":"2024-04-25T00:11:06.854970+0000","last_became_peered":"2024-04-25T00:11:06.854970+0000","last_unstale":"2024-04-25T00:11:12.515231+0000","last_undegraded":"2024-04-25T00:11:12.515231+0000","last_fullsized":"2024-04-25T00:11:12.515231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515424+0000","last_change":"2024-04-25T00:11:10.899597+0000","last_active":"2024-04-25T00:11:12.515424+0000","last_peered":"2024-04-25T00:11:12.515424+0000","last_clean":"2024-04-25T00:11:12.515424+0000","last_became_active":"2024-04-25T00:11:10.899136+0000","last_became_peered":"2024-04-25T00:11:10.899136+0000","last_unstale":"2024-04-25T00:11:12.515424+0000","last_undegraded":"2024-04-25T00:11:12.515424+0000","last_fullsized":"2024-04-25T00:11:12.515424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:40.036652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514540+0000","last_change":"2024-04-25T00:11:05.848984+0000","last_active":"2024-04-25T00:11:12.514540+0000","last_peered":"2024-04-25T00:11:12.514540+0000","last_clean":"2024-04-25T00:11:12.514540+0000","last_became_active":"2024-04-25T00:11:05.848618+0000","last_became_peered":"2024-04-25T00:11:05.848618+0000","last_unstale":"2024-04-25T00:11:12.514540+0000","last_undegraded":"2024-04-25T00:11:12.514540+0000","last_fullsized":"2024-04-25T00:11:12.514540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:50.369863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514723+0000","last_change":"2024-04-25T00:11:06.875779+0000","last_active":"2024-04-25T00:11:12.514723+0000","last_peered":"2024-04-25T00:11:12.514723+0000","last_clean":"2024-04-25T00:11:12.514723+0000","last_became_active":"2024-04-25T00:11:06.875440+0000","last_became_peered":"2024-04-25T00:11:06.875440+0000","last_unstale":"2024-04-25T00:11:12.514723+0000","last_undegraded":"2024-04-25T00:11:12.514723+0000","last_fullsized":"2024-04-25T00:11:12.514723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:43.565714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516147+0000","last_change":"2024-04-25T00:11:10.899794+0000","last_active":"2024-04-25T00:11:12.516147+0000","last_peered":"2024-04-25T00:11:12.516147+0000","last_clean":"2024-04-25T00:11:12.516147+0000","last_became_active":"2024-04-25T00:11:10.899313+0000","last_became_peered":"2024-04-25T00:11:10.899313+0000","last_unstale":"2024-04-25T00:11:12.516147+0000","last_undegraded":"2024-04-25T00:11:12.516147+0000","last_fullsized":"2024-04-25T00:11:12.516147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:51.269552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514246+0000","last_change":"2024-04-25T00:11:05.847672+0000","last_active":"2024-04-25T00:11:12.514246+0000","last_peered":"2024-04-25T00:11:12.514246+0000","last_clean":"2024-04-25T00:11:12.514246+0000","last_became_active":"2024-04-25T00:11:05.847316+0000","last_became_peered":"2024-04-25T00:11:05.847316+0000","last_unstale":"2024-04-25T00:11:12.514246+0000","last_undegraded":"2024-04-25T00:11:12.514246+0000","last_fullsized":"2024-04-25T00:11:12.514246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515600+0000","last_change":"2024-04-25T00:11:06.860047+0000","last_active":"2024-04-25T00:11:12.515600+0000","last_peered":"2024-04-25T00:11:12.515600+0000","last_clean":"2024-04-25T00:11:12.515600+0000","last_became_active":"2024-04-25T00:11:06.859782+0000","last_became_peered":"2024-04-25T00:11:06.859782+0000","last_unstale":"2024-04-25T00:11:12.515600+0000","last_undegraded":"2024-04-25T00:11:12.515600+0000","last_fullsized":"2024-04-25T00:11:12.515600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:43.612611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516469+0000","last_change":"2024-04-25T00:11:10.895749+0000","last_active":"2024-04-25T00:11:12.516469+0000","last_peered":"2024-04-25T00:11:12.516469+0000","last_clean":"2024-04-25T00:11:12.516469+0000","last_became_active":"2024-04-25T00:11:10.895359+0000","last_became_peered":"2024-04-25T00:11:10.895359+0000","last_unstale":"2024-04-25T00:11:12.516469+0000","last_undegraded":"2024-04-25T00:11:12.516469+0000","last_fullsized":"2024-04-25T00:11:12.516469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873961+0000","last_change":"2024-04-25T00:11:06.865465+0000","last_active":"2024-04-25T00:11:11.873961+0000","last_peered":"2024-04-25T00:11:11.873961+0000","last_clean":"2024-04-25T00:11:11.873961+0000","last_became_active":"2024-04-25T00:11:06.865020+0000","last_became_peered":"2024-04-25T00:11:06.865020+0000","last_unstale":"2024-04-25T00:11:11.873961+0000","last_undegraded":"2024-04-25T00:11:11.873961+0000","last_fullsized":"2024-04-25T00:11:11.873961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514541+0000","last_change":"2024-04-25T00:11:05.844750+0000","last_active":"2024-04-25T00:11:12.514541+0000","last_peered":"2024-04-25T00:11:12.514541+0000","last_clean":"2024-04-25T00:11:12.514541+0000","last_became_active":"2024-04-25T00:11:05.844407+0000","last_became_peered":"2024-04-25T00:11:05.844407+0000","last_unstale":"2024-04-25T00:11:12.514541+0000","last_undegraded":"2024-04-25T00:11:12.514541+0000","last_fullsized":"2024-04-25T00:11:12.514541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:34.248155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516163+0000","last_change":"2024-04-25T00:11:10.893480+0000","last_active":"2024-04-25T00:11:12.516163+0000","last_peered":"2024-04-25T00:11:12.516163+0000","last_clean":"2024-04-25T00:11:12.516163+0000","last_became_active":"2024-04-25T00:11:10.893051+0000","last_became_peered":"2024-04-25T00:11:10.893051+0000","last_unstale":"2024-04-25T00:11:12.516163+0000","last_undegraded":"2024-04-25T00:11:12.516163+0000","last_fullsized":"2024-04-25T00:11:12.516163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:40.531377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874153+0000","last_change":"2024-04-25T00:11:06.845105+0000","last_active":"2024-04-25T00:11:11.874153+0000","last_peered":"2024-04-25T00:11:11.874153+0000","last_clean":"2024-04-25T00:11:11.874153+0000","last_became_active":"2024-04-25T00:11:06.844686+0000","last_became_peered":"2024-04-25T00:11:06.844686+0000","last_unstale":"2024-04-25T00:11:11.874153+0000","last_undegraded":"2024-04-25T00:11:11.874153+0000","last_fullsized":"2024-04-25T00:11:11.874153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:25.890860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514977+0000","last_change":"2024-04-25T00:11:05.836866+0000","last_active":"2024-04-25T00:11:12.514977+0000","last_peered":"2024-04-25T00:11:12.514977+0000","last_clean":"2024-04-25T00:11:12.514977+0000","last_became_active":"2024-04-25T00:11:05.836578+0000","last_became_peered":"2024-04-25T00:11:05.836578+0000","last_unstale":"2024-04-25T00:11:12.514977+0000","last_undegraded":"2024-04-25T00:11:12.514977+0000","last_fullsized":"2024-04-25T00:11:12.514977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515742+0000","last_change":"2024-04-25T00:11:10.903356+0000","last_active":"2024-04-25T00:11:12.515742+0000","last_peered":"2024-04-25T00:11:12.515742+0000","last_clean":"2024-04-25T00:11:12.515742+0000","last_became_active":"2024-04-25T00:11:10.902845+0000","last_became_peered":"2024-04-25T00:11:10.902845+0000","last_unstale":"2024-04-25T00:11:12.515742+0000","last_undegraded":"2024-04-25T00:11:12.515742+0000","last_fullsized":"2024-04-25T00:11:12.515742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:47.662521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874413+0000","last_change":"2024-04-25T00:11:06.849565+0000","last_active":"2024-04-25T00:11:11.874413+0000","last_peered":"2024-04-25T00:11:11.874413+0000","last_clean":"2024-04-25T00:11:11.874413+0000","last_became_active":"2024-04-25T00:11:06.849275+0000","last_became_peered":"2024-04-25T00:11:06.849275+0000","last_unstale":"2024-04-25T00:11:11.874413+0000","last_undegraded":"2024-04-25T00:11:11.874413+0000","last_fullsized":"2024-04-25T00:11:11.874413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514312+0000","last_change":"2024-04-25T00:11:05.850573+0000","last_active":"2024-04-25T00:11:12.514312+0000","last_peered":"2024-04-25T00:11:12.514312+0000","last_clean":"2024-04-25T00:11:12.514312+0000","last_became_active":"2024-04-25T00:11:05.850239+0000","last_became_peered":"2024-04-25T00:11:05.850239+0000","last_unstale":"2024-04-25T00:11:12.514312+0000","last_undegraded":"2024-04-25T00:11:12.514312+0000","last_fullsized":"2024-04-25T00:11:12.514312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:41.285583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515409+0000","last_change":"2024-04-25T00:11:10.974826+0000","last_active":"2024-04-25T00:11:12.515409+0000","last_peered":"2024-04-25T00:11:12.515409+0000","last_clean":"2024-04-25T00:11:12.515409+0000","last_became_active":"2024-04-25T00:11:10.974483+0000","last_became_peered":"2024-04-25T00:11:10.974483+0000","last_unstale":"2024-04-25T00:11:12.515409+0000","last_undegraded":"2024-04-25T00:11:12.515409+0000","last_fullsized":"2024-04-25T00:11:12.515409+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:13.028091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516075+0000","last_change":"2024-04-25T00:11:06.857094+0000","last_active":"2024-04-25T00:11:12.516075+0000","last_peered":"2024-04-25T00:11:12.516075+0000","last_clean":"2024-04-25T00:11:12.516075+0000","last_became_active":"2024-04-25T00:11:06.856782+0000","last_became_peered":"2024-04-25T00:11:06.856782+0000","last_unstale":"2024-04-25T00:11:12.516075+0000","last_undegraded":"2024-04-25T00:11:12.516075+0000","last_fullsized":"2024-04-25T00:11:12.516075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:10.105280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513838+0000","last_change":"2024-04-25T00:11:05.849504+0000","last_active":"2024-04-25T00:11:12.513838+0000","last_peered":"2024-04-25T00:11:12.513838+0000","last_clean":"2024-04-25T00:11:12.513838+0000","last_became_active":"2024-04-25T00:11:05.847651+0000","last_became_peered":"2024-04-25T00:11:05.847651+0000","last_unstale":"2024-04-25T00:11:12.513838+0000","last_undegraded":"2024-04-25T00:11:12.513838+0000","last_fullsized":"2024-04-25T00:11:12.513838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515751+0000","last_change":"2024-04-25T00:11:10.893589+0000","last_active":"2024-04-25T00:11:12.515751+0000","last_peered":"2024-04-25T00:11:12.515751+0000","last_clean":"2024-04-25T00:11:12.515751+0000","last_became_active":"2024-04-25T00:11:10.893172+0000","last_became_peered":"2024-04-25T00:11:10.893172+0000","last_unstale":"2024-04-25T00:11:12.515751+0000","last_undegraded":"2024-04-25T00:11:12.515751+0000","last_fullsized":"2024-04-25T00:11:12.515751+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:53.464531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514257+0000","last_change":"2024-04-25T00:11:06.850048+0000","last_active":"2024-04-25T00:11:12.514257+0000","last_peered":"2024-04-25T00:11:12.514257+0000","last_clean":"2024-04-25T00:11:12.514257+0000","last_became_active":"2024-04-25T00:11:06.849785+0000","last_became_peered":"2024-04-25T00:11:06.849785+0000","last_unstale":"2024-04-25T00:11:12.514257+0000","last_undegraded":"2024-04-25T00:11:12.514257+0000","last_fullsized":"2024-04-25T00:11:12.514257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874790+0000","last_change":"2024-04-25T00:11:05.836100+0000","last_active":"2024-04-25T00:11:11.874790+0000","last_peered":"2024-04-25T00:11:11.874790+0000","last_clean":"2024-04-25T00:11:11.874790+0000","last_became_active":"2024-04-25T00:11:05.835841+0000","last_became_peered":"2024-04-25T00:11:05.835841+0000","last_unstale":"2024-04-25T00:11:11.874790+0000","last_undegraded":"2024-04-25T00:11:11.874790+0000","last_fullsized":"2024-04-25T00:11:11.874790+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:53.690870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515708+0000","last_change":"2024-04-25T00:11:10.975785+0000","last_active":"2024-04-25T00:11:12.515708+0000","last_peered":"2024-04-25T00:11:12.515708+0000","last_clean":"2024-04-25T00:11:12.515708+0000","last_became_active":"2024-04-25T00:11:10.974244+0000","last_became_peered":"2024-04-25T00:11:10.974244+0000","last_unstale":"2024-04-25T00:11:12.515708+0000","last_undegraded":"2024-04-25T00:11:12.515708+0000","last_fullsized":"2024-04-25T00:11:12.515708+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:47.615624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874580+0000","last_change":"2024-04-25T00:11:06.854507+0000","last_active":"2024-04-25T00:11:11.874580+0000","last_peered":"2024-04-25T00:11:11.874580+0000","last_clean":"2024-04-25T00:11:11.874580+0000","last_became_active":"2024-04-25T00:11:06.853694+0000","last_became_peered":"2024-04-25T00:11:06.853694+0000","last_unstale":"2024-04-25T00:11:11.874580+0000","last_undegraded":"2024-04-25T00:11:11.874580+0000","last_fullsized":"2024-04-25T00:11:11.874580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:36.481467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515093+0000","last_change":"2024-04-25T00:11:05.842386+0000","last_active":"2024-04-25T00:11:12.515093+0000","last_peered":"2024-04-25T00:11:12.515093+0000","last_clean":"2024-04-25T00:11:12.515093+0000","last_became_active":"2024-04-25T00:11:05.842064+0000","last_became_peered":"2024-04-25T00:11:05.842064+0000","last_unstale":"2024-04-25T00:11:12.515093+0000","last_undegraded":"2024-04-25T00:11:12.515093+0000","last_fullsized":"2024-04-25T00:11:12.515093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873860+0000","last_change":"2024-04-25T00:11:10.969122+0000","last_active":"2024-04-25T00:11:11.873860+0000","last_peered":"2024-04-25T00:11:11.873860+0000","last_clean":"2024-04-25T00:11:11.873860+0000","last_became_active":"2024-04-25T00:11:10.968532+0000","last_became_peered":"2024-04-25T00:11:10.968532+0000","last_unstale":"2024-04-25T00:11:11.873860+0000","last_undegraded":"2024-04-25T00:11:11.873860+0000","last_fullsized":"2024-04-25T00:11:11.873860+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:23:46.222075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514772+0000","last_change":"2024-04-25T00:11:06.847752+0000","last_active":"2024-04-25T00:11:12.514772+0000","last_peered":"2024-04-25T00:11:12.514772+0000","last_clean":"2024-04-25T00:11:12.514772+0000","last_became_active":"2024-04-25T00:11:06.847378+0000","last_became_peered":"2024-04-25T00:11:06.847378+0000","last_unstale":"2024-04-25T00:11:12.514772+0000","last_undegraded":"2024-04-25T00:11:12.514772+0000","last_fullsized":"2024-04-25T00:11:12.514772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514712+0000","last_change":"2024-04-25T00:11:05.849451+0000","last_active":"2024-04-25T00:11:12.514712+0000","last_peered":"2024-04-25T00:11:12.514712+0000","last_clean":"2024-04-25T00:11:12.514712+0000","last_became_active":"2024-04-25T00:11:05.849028+0000","last_became_peered":"2024-04-25T00:11:05.849028+0000","last_unstale":"2024-04-25T00:11:12.514712+0000","last_undegraded":"2024-04-25T00:11:12.514712+0000","last_fullsized":"2024-04-25T00:11:12.514712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:41.285583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513859+0000","last_change":"2024-04-25T00:11:10.973749+0000","last_active":"2024-04-25T00:11:12.513859+0000","last_peered":"2024-04-25T00:11:12.513859+0000","last_clean":"2024-04-25T00:11:12.513859+0000","last_became_active":"2024-04-25T00:11:10.973369+0000","last_became_peered":"2024-04-25T00:11:10.973369+0000","last_unstale":"2024-04-25T00:11:12.513859+0000","last_undegraded":"2024-04-25T00:11:12.513859+0000","last_fullsized":"2024-04-25T00:11:12.513859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:40.531377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515701+0000","last_change":"2024-04-25T00:11:06.859428+0000","last_active":"2024-04-25T00:11:12.515701+0000","last_peered":"2024-04-25T00:11:12.515701+0000","last_clean":"2024-04-25T00:11:12.515701+0000","last_became_active":"2024-04-25T00:11:06.859101+0000","last_became_peered":"2024-04-25T00:11:06.859101+0000","last_unstale":"2024-04-25T00:11:12.515701+0000","last_undegraded":"2024-04-25T00:11:12.515701+0000","last_fullsized":"2024-04-25T00:11:12.515701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:59.645042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514881+0000","last_change":"2024-04-25T00:11:05.840437+0000","last_active":"2024-04-25T00:11:12.514881+0000","last_peered":"2024-04-25T00:11:12.514881+0000","last_clean":"2024-04-25T00:11:12.514881+0000","last_became_active":"2024-04-25T00:11:05.840120+0000","last_became_peered":"2024-04-25T00:11:05.840120+0000","last_unstale":"2024-04-25T00:11:12.514881+0000","last_undegraded":"2024-04-25T00:11:12.514881+0000","last_fullsized":"2024-04-25T00:11:12.514881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514595+0000","last_change":"2024-04-25T00:11:10.896951+0000","last_active":"2024-04-25T00:11:12.514595+0000","last_peered":"2024-04-25T00:11:12.514595+0000","last_clean":"2024-04-25T00:11:12.514595+0000","last_became_active":"2024-04-25T00:11:10.896547+0000","last_became_peered":"2024-04-25T00:11:10.896547+0000","last_unstale":"2024-04-25T00:11:12.514595+0000","last_undegraded":"2024-04-25T00:11:12.514595+0000","last_fullsized":"2024-04-25T00:11:12.514595+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516507+0000","last_change":"2024-04-25T00:11:06.846626+0000","last_active":"2024-04-25T00:11:12.516507+0000","last_peered":"2024-04-25T00:11:12.516507+0000","last_clean":"2024-04-25T00:11:12.516507+0000","last_became_active":"2024-04-25T00:11:06.846333+0000","last_became_peered":"2024-04-25T00:11:06.846333+0000","last_unstale":"2024-04-25T00:11:12.516507+0000","last_undegraded":"2024-04-25T00:11:12.516507+0000","last_fullsized":"2024-04-25T00:11:12.516507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:42.291050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515107+0000","last_change":"2024-04-25T00:11:05.839781+0000","last_active":"2024-04-25T00:11:12.515107+0000","last_peered":"2024-04-25T00:11:12.515107+0000","last_clean":"2024-04-25T00:11:12.515107+0000","last_became_active":"2024-04-25T00:11:05.839400+0000","last_became_peered":"2024-04-25T00:11:05.839400+0000","last_unstale":"2024-04-25T00:11:12.515107+0000","last_undegraded":"2024-04-25T00:11:12.515107+0000","last_fullsized":"2024-04-25T00:11:12.515107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514811+0000","last_change":"2024-04-25T00:11:10.892411+0000","last_active":"2024-04-25T00:11:12.514811+0000","last_peered":"2024-04-25T00:11:12.514811+0000","last_clean":"2024-04-25T00:11:12.514811+0000","last_became_active":"2024-04-25T00:11:10.892036+0000","last_became_peered":"2024-04-25T00:11:10.892036+0000","last_unstale":"2024-04-25T00:11:12.514811+0000","last_undegraded":"2024-04-25T00:11:12.514811+0000","last_fullsized":"2024-04-25T00:11:12.514811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:07.816085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513913+0000","last_change":"2024-04-25T00:11:06.842835+0000","last_active":"2024-04-25T00:11:12.513913+0000","last_peered":"2024-04-25T00:11:12.513913+0000","last_clean":"2024-04-25T00:11:12.513913+0000","last_became_active":"2024-04-25T00:11:06.842540+0000","last_became_peered":"2024-04-25T00:11:06.842540+0000","last_unstale":"2024-04-25T00:11:12.513913+0000","last_undegraded":"2024-04-25T00:11:12.513913+0000","last_fullsized":"2024-04-25T00:11:12.513913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:00.950133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875360+0000","last_change":"2024-04-25T00:11:05.829300+0000","last_active":"2024-04-25T00:11:11.875360+0000","last_peered":"2024-04-25T00:11:11.875360+0000","last_clean":"2024-04-25T00:11:11.875360+0000","last_became_active":"2024-04-25T00:11:05.828940+0000","last_became_peered":"2024-04-25T00:11:05.828940+0000","last_unstale":"2024-04-25T00:11:11.875360+0000","last_undegraded":"2024-04-25T00:11:11.875360+0000","last_fullsized":"2024-04-25T00:11:11.875360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:15.738255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515526+0000","last_change":"2024-04-25T00:11:10.895520+0000","last_active":"2024-04-25T00:11:12.515526+0000","last_peered":"2024-04-25T00:11:12.515526+0000","last_clean":"2024-04-25T00:11:12.515526+0000","last_became_active":"2024-04-25T00:11:10.895096+0000","last_became_peered":"2024-04-25T00:11:10.895096+0000","last_unstale":"2024-04-25T00:11:12.515526+0000","last_undegraded":"2024-04-25T00:11:12.515526+0000","last_fullsized":"2024-04-25T00:11:12.515526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:51.996256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513553+0000","last_change":"2024-04-25T00:11:06.862814+0000","last_active":"2024-04-25T00:11:12.513553+0000","last_peered":"2024-04-25T00:11:12.513553+0000","last_clean":"2024-04-25T00:11:12.513553+0000","last_became_active":"2024-04-25T00:11:06.862534+0000","last_became_peered":"2024-04-25T00:11:06.862534+0000","last_unstale":"2024-04-25T00:11:12.513553+0000","last_undegraded":"2024-04-25T00:11:12.513553+0000","last_fullsized":"2024-04-25T00:11:12.513553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:39.726185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513959+0000","last_change":"2024-04-25T00:11:05.849841+0000","last_active":"2024-04-25T00:11:12.513959+0000","last_peered":"2024-04-25T00:11:12.513959+0000","last_clean":"2024-04-25T00:11:12.513959+0000","last_became_active":"2024-04-25T00:11:05.847646+0000","last_became_peered":"2024-04-25T00:11:05.847646+0000","last_unstale":"2024-04-25T00:11:12.513959+0000","last_undegraded":"2024-04-25T00:11:12.513959+0000","last_fullsized":"2024-04-25T00:11:12.513959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:36.649759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514155+0000","last_change":"2024-04-25T00:11:10.974744+0000","last_active":"2024-04-25T00:11:12.514155+0000","last_peered":"2024-04-25T00:11:12.514155+0000","last_clean":"2024-04-25T00:11:12.514155+0000","last_became_active":"2024-04-25T00:11:10.973737+0000","last_became_peered":"2024-04-25T00:11:10.973737+0000","last_unstale":"2024-04-25T00:11:12.514155+0000","last_undegraded":"2024-04-25T00:11:12.514155+0000","last_fullsized":"2024-04-25T00:11:12.514155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:13.028091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875061+0000","last_change":"2024-04-25T00:11:06.850937+0000","last_active":"2024-04-25T00:11:11.875061+0000","last_peered":"2024-04-25T00:11:11.875061+0000","last_clean":"2024-04-25T00:11:11.875061+0000","last_became_active":"2024-04-25T00:11:06.850620+0000","last_became_peered":"2024-04-25T00:11:06.850620+0000","last_unstale":"2024-04-25T00:11:11.875061+0000","last_undegraded":"2024-04-25T00:11:11.875061+0000","last_fullsized":"2024-04-25T00:11:11.875061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514342+0000","last_change":"2024-04-25T00:11:05.841594+0000","last_active":"2024-04-25T00:11:12.514342+0000","last_peered":"2024-04-25T00:11:12.514342+0000","last_clean":"2024-04-25T00:11:12.514342+0000","last_became_active":"2024-04-25T00:11:05.838964+0000","last_became_peered":"2024-04-25T00:11:05.838964+0000","last_unstale":"2024-04-25T00:11:12.514342+0000","last_undegraded":"2024-04-25T00:11:12.514342+0000","last_fullsized":"2024-04-25T00:11:12.514342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514070+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:11:12.514070+0000","last_peered":"2024-04-25T00:11:12.514070+0000","last_clean":"2024-04-25T00:11:12.514070+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:11:12.514070+0000","last_undegraded":"2024-04-25T00:11:12.514070+0000","last_fullsized":"2024-04-25T00:11:12.514070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874137+0000","last_change":"2024-04-25T00:11:10.976384+0000","last_active":"2024-04-25T00:11:11.874137+0000","last_peered":"2024-04-25T00:11:11.874137+0000","last_clean":"2024-04-25T00:11:11.874137+0000","last_became_active":"2024-04-25T00:11:10.975924+0000","last_became_peered":"2024-04-25T00:11:10.975924+0000","last_unstale":"2024-04-25T00:11:11.874137+0000","last_undegraded":"2024-04-25T00:11:11.874137+0000","last_fullsized":"2024-04-25T00:11:11.874137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514383+0000","last_change":"2024-04-25T00:11:06.855965+0000","last_active":"2024-04-25T00:11:12.514383+0000","last_peered":"2024-04-25T00:11:12.514383+0000","last_clean":"2024-04-25T00:11:12.514383+0000","last_became_active":"2024-04-25T00:11:06.853447+0000","last_became_peered":"2024-04-25T00:11:06.853447+0000","last_unstale":"2024-04-25T00:11:12.514383+0000","last_undegraded":"2024-04-25T00:11:12.514383+0000","last_fullsized":"2024-04-25T00:11:12.514383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:14.704188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875959+0000","last_change":"2024-04-25T00:11:05.842420+0000","last_active":"2024-04-25T00:11:11.875959+0000","last_peered":"2024-04-25T00:11:11.875959+0000","last_clean":"2024-04-25T00:11:11.875959+0000","last_became_active":"2024-04-25T00:11:05.842102+0000","last_became_peered":"2024-04-25T00:11:05.842102+0000","last_unstale":"2024-04-25T00:11:11.875959+0000","last_undegraded":"2024-04-25T00:11:11.875959+0000","last_fullsized":"2024-04-25T00:11:11.875959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874194+0000","last_change":"2024-04-25T00:11:10.969423+0000","last_active":"2024-04-25T00:11:11.874194+0000","last_peered":"2024-04-25T00:11:11.874194+0000","last_clean":"2024-04-25T00:11:11.874194+0000","last_became_active":"2024-04-25T00:11:10.968799+0000","last_became_peered":"2024-04-25T00:11:10.968799+0000","last_unstale":"2024-04-25T00:11:11.874194+0000","last_undegraded":"2024-04-25T00:11:11.874194+0000","last_fullsized":"2024-04-25T00:11:11.874194+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514836+0000","last_change":"2024-04-25T00:11:06.859625+0000","last_active":"2024-04-25T00:11:12.514836+0000","last_peered":"2024-04-25T00:11:12.514836+0000","last_clean":"2024-04-25T00:11:12.514836+0000","last_became_active":"2024-04-25T00:11:06.859161+0000","last_became_peered":"2024-04-25T00:11:06.859161+0000","last_unstale":"2024-04-25T00:11:12.514836+0000","last_undegraded":"2024-04-25T00:11:12.514836+0000","last_fullsized":"2024-04-25T00:11:12.514836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:36.481467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515075+0000","last_change":"2024-04-25T00:11:05.839603+0000","last_active":"2024-04-25T00:11:12.515075+0000","last_peered":"2024-04-25T00:11:12.515075+0000","last_clean":"2024-04-25T00:11:12.515075+0000","last_became_active":"2024-04-25T00:11:05.838608+0000","last_became_peered":"2024-04-25T00:11:05.838608+0000","last_unstale":"2024-04-25T00:11:12.515075+0000","last_undegraded":"2024-04-25T00:11:12.515075+0000","last_fullsized":"2024-04-25T00:11:12.515075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:53.690870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514117+0000","last_change":"2024-04-25T00:11:10.900742+0000","last_active":"2024-04-25T00:11:12.514117+0000","last_peered":"2024-04-25T00:11:12.514117+0000","last_clean":"2024-04-25T00:11:12.514117+0000","last_became_active":"2024-04-25T00:11:10.900246+0000","last_became_peered":"2024-04-25T00:11:10.900246+0000","last_unstale":"2024-04-25T00:11:12.514117+0000","last_undegraded":"2024-04-25T00:11:12.514117+0000","last_fullsized":"2024-04-25T00:11:12.514117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:07.816085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514836+0000","last_change":"2024-04-25T00:11:06.870304+0000","last_active":"2024-04-25T00:11:12.514836+0000","last_peered":"2024-04-25T00:11:12.514836+0000","last_clean":"2024-04-25T00:11:12.514836+0000","last_became_active":"2024-04-25T00:11:06.870010+0000","last_became_peered":"2024-04-25T00:11:06.870010+0000","last_unstale":"2024-04-25T00:11:12.514836+0000","last_undegraded":"2024-04-25T00:11:12.514836+0000","last_fullsized":"2024-04-25T00:11:12.514836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:08.978181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.876556+0000","last_change":"2024-04-25T00:11:05.839098+0000","last_active":"2024-04-25T00:11:11.876556+0000","last_peered":"2024-04-25T00:11:11.876556+0000","last_clean":"2024-04-25T00:11:11.876556+0000","last_became_active":"2024-04-25T00:11:05.838694+0000","last_became_peered":"2024-04-25T00:11:05.838694+0000","last_unstale":"2024-04-25T00:11:11.876556+0000","last_undegraded":"2024-04-25T00:11:11.876556+0000","last_fullsized":"2024-04-25T00:11:11.876556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:35.709574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513908+0000","last_change":"2024-04-25T00:11:10.973661+0000","last_active":"2024-04-25T00:11:12.513908+0000","last_peered":"2024-04-25T00:11:12.513908+0000","last_clean":"2024-04-25T00:11:12.513908+0000","last_became_active":"2024-04-25T00:11:10.973198+0000","last_became_peered":"2024-04-25T00:11:10.973198+0000","last_unstale":"2024-04-25T00:11:12.513908+0000","last_undegraded":"2024-04-25T00:11:12.513908+0000","last_fullsized":"2024-04-25T00:11:12.513908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:23:46.222075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514875+0000","last_change":"2024-04-25T00:11:06.878377+0000","last_active":"2024-04-25T00:11:12.514875+0000","last_peered":"2024-04-25T00:11:12.514875+0000","last_clean":"2024-04-25T00:11:12.514875+0000","last_became_active":"2024-04-25T00:11:06.878046+0000","last_became_peered":"2024-04-25T00:11:06.878046+0000","last_unstale":"2024-04-25T00:11:12.514875+0000","last_undegraded":"2024-04-25T00:11:12.514875+0000","last_fullsized":"2024-04-25T00:11:12.514875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:59.645042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515717+0000","last_change":"2024-04-25T00:11:05.839981+0000","last_active":"2024-04-25T00:11:12.515717+0000","last_peered":"2024-04-25T00:11:12.515717+0000","last_clean":"2024-04-25T00:11:12.515717+0000","last_became_active":"2024-04-25T00:11:05.839527+0000","last_became_peered":"2024-04-25T00:11:05.839527+0000","last_unstale":"2024-04-25T00:11:12.515717+0000","last_undegraded":"2024-04-25T00:11:12.515717+0000","last_fullsized":"2024-04-25T00:11:12.515717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:14.133421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515439+0000","last_change":"2024-04-25T00:11:10.900219+0000","last_active":"2024-04-25T00:11:12.515439+0000","last_peered":"2024-04-25T00:11:12.515439+0000","last_clean":"2024-04-25T00:11:12.515439+0000","last_became_active":"2024-04-25T00:11:10.899780+0000","last_became_peered":"2024-04-25T00:11:10.899780+0000","last_unstale":"2024-04-25T00:11:12.515439+0000","last_undegraded":"2024-04-25T00:11:12.515439+0000","last_fullsized":"2024-04-25T00:11:12.515439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:29.940770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515975+0000","last_change":"2024-04-25T00:11:06.863060+0000","last_active":"2024-04-25T00:11:12.515975+0000","last_peered":"2024-04-25T00:11:12.515975+0000","last_clean":"2024-04-25T00:11:12.515975+0000","last_became_active":"2024-04-25T00:11:06.862320+0000","last_became_peered":"2024-04-25T00:11:06.862320+0000","last_unstale":"2024-04-25T00:11:12.515975+0000","last_undegraded":"2024-04-25T00:11:12.515975+0000","last_fullsized":"2024-04-25T00:11:12.515975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:49.414621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875244+0000","last_change":"2024-04-25T00:11:05.826923+0000","last_active":"2024-04-25T00:11:11.875244+0000","last_peered":"2024-04-25T00:11:11.875244+0000","last_clean":"2024-04-25T00:11:11.875244+0000","last_became_active":"2024-04-25T00:11:05.826543+0000","last_became_peered":"2024-04-25T00:11:05.826543+0000","last_unstale":"2024-04-25T00:11:11.875244+0000","last_undegraded":"2024-04-25T00:11:11.875244+0000","last_fullsized":"2024-04-25T00:11:11.875244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513942+0000","last_change":"2024-04-25T00:11:10.893190+0000","last_active":"2024-04-25T00:11:12.513942+0000","last_peered":"2024-04-25T00:11:12.513942+0000","last_clean":"2024-04-25T00:11:12.513942+0000","last_became_active":"2024-04-25T00:11:10.892676+0000","last_became_peered":"2024-04-25T00:11:10.892676+0000","last_unstale":"2024-04-25T00:11:12.513942+0000","last_undegraded":"2024-04-25T00:11:12.513942+0000","last_fullsized":"2024-04-25T00:11:12.513942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874062+0000","last_change":"2024-04-25T00:11:06.854380+0000","last_active":"2024-04-25T00:11:11.874062+0000","last_peered":"2024-04-25T00:11:11.874062+0000","last_clean":"2024-04-25T00:11:11.874062+0000","last_became_active":"2024-04-25T00:11:06.854089+0000","last_became_peered":"2024-04-25T00:11:06.854089+0000","last_unstale":"2024-04-25T00:11:11.874062+0000","last_undegraded":"2024-04-25T00:11:11.874062+0000","last_fullsized":"2024-04-25T00:11:11.874062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874367+0000","last_change":"2024-04-25T00:11:05.848317+0000","last_active":"2024-04-25T00:11:11.874367+0000","last_peered":"2024-04-25T00:11:11.874367+0000","last_clean":"2024-04-25T00:11:11.874367+0000","last_became_active":"2024-04-25T00:11:05.847969+0000","last_became_peered":"2024-04-25T00:11:05.847969+0000","last_unstale":"2024-04-25T00:11:11.874367+0000","last_undegraded":"2024-04-25T00:11:11.874367+0000","last_fullsized":"2024-04-25T00:11:11.874367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874567+0000","last_change":"2024-04-25T00:11:10.974665+0000","last_active":"2024-04-25T00:11:11.874567+0000","last_peered":"2024-04-25T00:11:11.874567+0000","last_clean":"2024-04-25T00:11:11.874567+0000","last_became_active":"2024-04-25T00:11:10.974306+0000","last_became_peered":"2024-04-25T00:11:10.974306+0000","last_unstale":"2024-04-25T00:11:11.874567+0000","last_undegraded":"2024-04-25T00:11:11.874567+0000","last_fullsized":"2024-04-25T00:11:11.874567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:09.712979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514422+0000","last_change":"2024-04-25T00:11:06.870996+0000","last_active":"2024-04-25T00:11:12.514422+0000","last_peered":"2024-04-25T00:11:12.514422+0000","last_clean":"2024-04-25T00:11:12.514422+0000","last_became_active":"2024-04-25T00:11:06.870692+0000","last_became_peered":"2024-04-25T00:11:06.870692+0000","last_unstale":"2024-04-25T00:11:12.514422+0000","last_undegraded":"2024-04-25T00:11:12.514422+0000","last_fullsized":"2024-04-25T00:11:12.514422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:36.481467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515027+0000","last_change":"2024-04-25T00:11:05.838027+0000","last_active":"2024-04-25T00:11:12.515027+0000","last_peered":"2024-04-25T00:11:12.515027+0000","last_clean":"2024-04-25T00:11:12.515027+0000","last_became_active":"2024-04-25T00:11:05.837712+0000","last_became_peered":"2024-04-25T00:11:05.837712+0000","last_unstale":"2024-04-25T00:11:12.515027+0000","last_undegraded":"2024-04-25T00:11:12.515027+0000","last_fullsized":"2024-04-25T00:11:12.515027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:53.690870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875060+0000","last_change":"2024-04-25T00:11:10.973710+0000","last_active":"2024-04-25T00:11:11.875060+0000","last_peered":"2024-04-25T00:11:11.875060+0000","last_clean":"2024-04-25T00:11:11.875060+0000","last_became_active":"2024-04-25T00:11:10.973387+0000","last_became_peered":"2024-04-25T00:11:10.973387+0000","last_unstale":"2024-04-25T00:11:11.875060+0000","last_undegraded":"2024-04-25T00:11:11.875060+0000","last_fullsized":"2024-04-25T00:11:11.875060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:39.160853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514207+0000","last_change":"2024-04-25T00:11:06.847434+0000","last_active":"2024-04-25T00:11:12.514207+0000","last_peered":"2024-04-25T00:11:12.514207+0000","last_clean":"2024-04-25T00:11:12.514207+0000","last_became_active":"2024-04-25T00:11:06.847112+0000","last_became_peered":"2024-04-25T00:11:06.847112+0000","last_unstale":"2024-04-25T00:11:12.514207+0000","last_undegraded":"2024-04-25T00:11:12.514207+0000","last_fullsized":"2024-04-25T00:11:12.514207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:27.928927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514359+0000","last_change":"2024-04-25T00:11:05.837912+0000","last_active":"2024-04-25T00:11:12.514359+0000","last_peered":"2024-04-25T00:11:12.514359+0000","last_clean":"2024-04-25T00:11:12.514359+0000","last_became_active":"2024-04-25T00:11:05.837544+0000","last_became_peered":"2024-04-25T00:11:05.837544+0000","last_unstale":"2024-04-25T00:11:12.514359+0000","last_undegraded":"2024-04-25T00:11:12.514359+0000","last_fullsized":"2024-04-25T00:11:12.514359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:14.133421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515539+0000","last_change":"2024-04-25T00:11:10.968996+0000","last_active":"2024-04-25T00:11:12.515539+0000","last_peered":"2024-04-25T00:11:12.515539+0000","last_clean":"2024-04-25T00:11:12.515539+0000","last_became_active":"2024-04-25T00:11:10.968625+0000","last_became_peered":"2024-04-25T00:11:10.968625+0000","last_unstale":"2024-04-25T00:11:12.515539+0000","last_undegraded":"2024-04-25T00:11:12.515539+0000","last_fullsized":"2024-04-25T00:11:12.515539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:07.644687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514303+0000","last_change":"2024-04-25T00:11:06.857856+0000","last_active":"2024-04-25T00:11:12.514303+0000","last_peered":"2024-04-25T00:11:12.514303+0000","last_clean":"2024-04-25T00:11:12.514303+0000","last_became_active":"2024-04-25T00:11:06.857469+0000","last_became_peered":"2024-04-25T00:11:06.857469+0000","last_unstale":"2024-04-25T00:11:12.514303+0000","last_undegraded":"2024-04-25T00:11:12.514303+0000","last_fullsized":"2024-04-25T00:11:12.514303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:47.946346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:15.888628+0000","last_change":"2024-04-25T00:11:05.836491+0000","last_active":"2024-04-25T00:11:15.888628+0000","last_peered":"2024-04-25T00:11:15.888628+0000","last_clean":"2024-04-25T00:11:15.888628+0000","last_became_active":"2024-04-25T00:11:05.835903+0000","last_became_peered":"2024-04-25T00:11:05.835903+0000","last_unstale":"2024-04-25T00:11:15.888628+0000","last_undegraded":"2024-04-25T00:11:15.888628+0000","last_fullsized":"2024-04-25T00:11:15.888628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514752+0000","last_change":"2024-04-25T00:11:10.902870+0000","last_active":"2024-04-25T00:11:12.514752+0000","last_peered":"2024-04-25T00:11:12.514752+0000","last_clean":"2024-04-25T00:11:12.514752+0000","last_became_active":"2024-04-25T00:11:10.896980+0000","last_became_peered":"2024-04-25T00:11:10.896980+0000","last_unstale":"2024-04-25T00:11:12.514752+0000","last_undegraded":"2024-04-25T00:11:12.514752+0000","last_fullsized":"2024-04-25T00:11:12.514752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516066+0000","last_change":"2024-04-25T00:11:06.864856+0000","last_active":"2024-04-25T00:11:12.516066+0000","last_peered":"2024-04-25T00:11:12.516066+0000","last_clean":"2024-04-25T00:11:12.516066+0000","last_became_active":"2024-04-25T00:11:06.863698+0000","last_became_peered":"2024-04-25T00:11:06.863698+0000","last_unstale":"2024-04-25T00:11:12.516066+0000","last_undegraded":"2024-04-25T00:11:12.516066+0000","last_fullsized":"2024-04-25T00:11:12.516066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:35.253622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514963+0000","last_change":"2024-04-25T00:11:05.843072+0000","last_active":"2024-04-25T00:11:12.514963+0000","last_peered":"2024-04-25T00:11:12.514963+0000","last_clean":"2024-04-25T00:11:12.514963+0000","last_became_active":"2024-04-25T00:11:05.842548+0000","last_became_peered":"2024-04-25T00:11:05.842548+0000","last_unstale":"2024-04-25T00:11:12.514963+0000","last_undegraded":"2024-04-25T00:11:12.514963+0000","last_fullsized":"2024-04-25T00:11:12.514963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516339+0000","last_change":"2024-04-25T00:11:10.900180+0000","last_active":"2024-04-25T00:11:12.516339+0000","last_peered":"2024-04-25T00:11:12.516339+0000","last_clean":"2024-04-25T00:11:12.516339+0000","last_became_active":"2024-04-25T00:11:10.899321+0000","last_became_peered":"2024-04-25T00:11:10.899321+0000","last_unstale":"2024-04-25T00:11:12.516339+0000","last_undegraded":"2024-04-25T00:11:12.516339+0000","last_fullsized":"2024-04-25T00:11:12.516339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515303+0000","last_change":"2024-04-25T00:11:06.855642+0000","last_active":"2024-04-25T00:11:12.515303+0000","last_peered":"2024-04-25T00:11:12.515303+0000","last_clean":"2024-04-25T00:11:12.515303+0000","last_became_active":"2024-04-25T00:11:06.855374+0000","last_became_peered":"2024-04-25T00:11:06.855374+0000","last_unstale":"2024-04-25T00:11:12.515303+0000","last_undegraded":"2024-04-25T00:11:12.515303+0000","last_fullsized":"2024-04-25T00:11:12.515303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516104+0000","last_change":"2024-04-25T00:11:05.846074+0000","last_active":"2024-04-25T00:11:12.516104+0000","last_peered":"2024-04-25T00:11:12.516104+0000","last_clean":"2024-04-25T00:11:12.516104+0000","last_became_active":"2024-04-25T00:11:05.845371+0000","last_became_peered":"2024-04-25T00:11:05.845371+0000","last_unstale":"2024-04-25T00:11:12.516104+0000","last_undegraded":"2024-04-25T00:11:12.516104+0000","last_fullsized":"2024-04-25T00:11:12.516104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874963+0000","last_change":"2024-04-25T00:11:10.898262+0000","last_active":"2024-04-25T00:11:11.874963+0000","last_peered":"2024-04-25T00:11:11.874963+0000","last_clean":"2024-04-25T00:11:11.874963+0000","last_became_active":"2024-04-25T00:11:10.897653+0000","last_became_peered":"2024-04-25T00:11:10.897653+0000","last_unstale":"2024-04-25T00:11:11.874963+0000","last_undegraded":"2024-04-25T00:11:11.874963+0000","last_fullsized":"2024-04-25T00:11:11.874963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:12.584307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516002+0000","last_change":"2024-04-25T00:11:06.865617+0000","last_active":"2024-04-25T00:11:12.516002+0000","last_peered":"2024-04-25T00:11:12.516002+0000","last_clean":"2024-04-25T00:11:12.516002+0000","last_became_active":"2024-04-25T00:11:06.865286+0000","last_became_peered":"2024-04-25T00:11:06.865286+0000","last_unstale":"2024-04-25T00:11:12.516002+0000","last_undegraded":"2024-04-25T00:11:12.516002+0000","last_fullsized":"2024-04-25T00:11:12.516002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:03.766175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874678+0000","last_change":"2024-04-25T00:11:05.845733+0000","last_active":"2024-04-25T00:11:11.874678+0000","last_peered":"2024-04-25T00:11:11.874678+0000","last_clean":"2024-04-25T00:11:11.874678+0000","last_became_active":"2024-04-25T00:11:05.845410+0000","last_became_peered":"2024-04-25T00:11:05.845410+0000","last_unstale":"2024-04-25T00:11:11.874678+0000","last_undegraded":"2024-04-25T00:11:11.874678+0000","last_fullsized":"2024-04-25T00:11:11.874678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874418+0000","last_change":"2024-04-25T00:11:10.977787+0000","last_active":"2024-04-25T00:11:11.874418+0000","last_peered":"2024-04-25T00:11:11.874418+0000","last_clean":"2024-04-25T00:11:11.874418+0000","last_became_active":"2024-04-25T00:11:10.977434+0000","last_became_peered":"2024-04-25T00:11:10.977434+0000","last_unstale":"2024-04-25T00:11:11.874418+0000","last_undegraded":"2024-04-25T00:11:11.874418+0000","last_fullsized":"2024-04-25T00:11:11.874418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:27.014726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513916+0000","last_change":"2024-04-25T00:11:10.977497+0000","last_active":"2024-04-25T00:11:12.513916+0000","last_peered":"2024-04-25T00:11:12.513916+0000","last_clean":"2024-04-25T00:11:12.513916+0000","last_became_active":"2024-04-25T00:11:10.973670+0000","last_became_peered":"2024-04-25T00:11:10.973670+0000","last_unstale":"2024-04-25T00:11:12.513916+0000","last_undegraded":"2024-04-25T00:11:12.513916+0000","last_fullsized":"2024-04-25T00:11:12.513916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514138+0000","last_change":"2024-04-25T00:11:06.851862+0000","last_active":"2024-04-25T00:11:12.514138+0000","last_peered":"2024-04-25T00:11:12.514138+0000","last_clean":"2024-04-25T00:11:12.514138+0000","last_became_active":"2024-04-25T00:11:06.851683+0000","last_became_peered":"2024-04-25T00:11:06.851683+0000","last_unstale":"2024-04-25T00:11:12.514138+0000","last_undegraded":"2024-04-25T00:11:12.514138+0000","last_fullsized":"2024-04-25T00:11:12.514138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515242+0000","last_change":"2024-04-25T00:11:05.842503+0000","last_active":"2024-04-25T00:11:12.515242+0000","last_peered":"2024-04-25T00:11:12.515242+0000","last_clean":"2024-04-25T00:11:12.515242+0000","last_became_active":"2024-04-25T00:11:05.842214+0000","last_became_peered":"2024-04-25T00:11:05.842214+0000","last_unstale":"2024-04-25T00:11:12.515242+0000","last_undegraded":"2024-04-25T00:11:12.515242+0000","last_fullsized":"2024-04-25T00:11:12.515242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874565+0000","last_change":"2024-04-25T00:11:10.901930+0000","last_active":"2024-04-25T00:11:11.874565+0000","last_peered":"2024-04-25T00:11:11.874565+0000","last_clean":"2024-04-25T00:11:11.874565+0000","last_became_active":"2024-04-25T00:11:10.901405+0000","last_became_peered":"2024-04-25T00:11:10.901405+0000","last_unstale":"2024-04-25T00:11:11.874565+0000","last_undegraded":"2024-04-25T00:11:11.874565+0000","last_fullsized":"2024-04-25T00:11:11.874565+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:07.816085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874298+0000","last_change":"2024-04-25T00:11:06.855999+0000","last_active":"2024-04-25T00:11:11.874298+0000","last_peered":"2024-04-25T00:11:11.874298+0000","last_clean":"2024-04-25T00:11:11.874298+0000","last_became_active":"2024-04-25T00:11:06.855758+0000","last_became_peered":"2024-04-25T00:11:06.855758+0000","last_unstale":"2024-04-25T00:11:11.874298+0000","last_undegraded":"2024-04-25T00:11:11.874298+0000","last_fullsized":"2024-04-25T00:11:11.874298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874267+0000","last_change":"2024-04-25T00:11:05.834141+0000","last_active":"2024-04-25T00:11:11.874267+0000","last_peered":"2024-04-25T00:11:11.874267+0000","last_clean":"2024-04-25T00:11:11.874267+0000","last_became_active":"2024-04-25T00:11:05.833022+0000","last_became_peered":"2024-04-25T00:11:05.833022+0000","last_unstale":"2024-04-25T00:11:11.874267+0000","last_undegraded":"2024-04-25T00:11:11.874267+0000","last_fullsized":"2024-04-25T00:11:11.874267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:15.738255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874362+0000","last_change":"2024-04-25T00:11:05.844727+0000","last_active":"2024-04-25T00:11:11.874362+0000","last_peered":"2024-04-25T00:11:11.874362+0000","last_clean":"2024-04-25T00:11:11.874362+0000","last_became_active":"2024-04-25T00:11:05.844303+0000","last_became_peered":"2024-04-25T00:11:05.844303+0000","last_unstale":"2024-04-25T00:11:11.874362+0000","last_undegraded":"2024-04-25T00:11:11.874362+0000","last_fullsized":"2024-04-25T00:11:11.874362+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:41.285583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514255+0000","last_change":"2024-04-25T00:11:06.862764+0000","last_active":"2024-04-25T00:11:12.514255+0000","last_peered":"2024-04-25T00:11:12.514255+0000","last_clean":"2024-04-25T00:11:12.514255+0000","last_became_active":"2024-04-25T00:11:06.862372+0000","last_became_peered":"2024-04-25T00:11:06.862372+0000","last_unstale":"2024-04-25T00:11:12.514255+0000","last_undegraded":"2024-04-25T00:11:12.514255+0000","last_fullsized":"2024-04-25T00:11:12.514255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875902+0000","last_change":"2024-04-25T00:11:10.902130+0000","last_active":"2024-04-25T00:11:11.875902+0000","last_peered":"2024-04-25T00:11:11.875902+0000","last_clean":"2024-04-25T00:11:11.875902+0000","last_became_active":"2024-04-25T00:11:10.901717+0000","last_became_peered":"2024-04-25T00:11:10.901717+0000","last_unstale":"2024-04-25T00:11:11.875902+0000","last_undegraded":"2024-04-25T00:11:11.875902+0000","last_fullsized":"2024-04-25T00:11:11.875902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:43.776095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875180+0000","last_change":"2024-04-25T00:11:05.833116+0000","last_active":"2024-04-25T00:11:11.875180+0000","last_peered":"2024-04-25T00:11:11.875180+0000","last_clean":"2024-04-25T00:11:11.875180+0000","last_became_active":"2024-04-25T00:11:05.832202+0000","last_became_peered":"2024-04-25T00:11:05.832202+0000","last_unstale":"2024-04-25T00:11:11.875180+0000","last_undegraded":"2024-04-25T00:11:11.875180+0000","last_fullsized":"2024-04-25T00:11:11.875180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516469+0000","last_change":"2024-04-25T00:11:06.840973+0000","last_active":"2024-04-25T00:11:12.516469+0000","last_peered":"2024-04-25T00:11:12.516469+0000","last_clean":"2024-04-25T00:11:12.516469+0000","last_became_active":"2024-04-25T00:11:06.840670+0000","last_became_peered":"2024-04-25T00:11:06.840670+0000","last_unstale":"2024-04-25T00:11:12.516469+0000","last_undegraded":"2024-04-25T00:11:12.516469+0000","last_fullsized":"2024-04-25T00:11:12.516469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:29.296632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874813+0000","last_change":"2024-04-25T00:11:10.898183+0000","last_active":"2024-04-25T00:11:11.874813+0000","last_peered":"2024-04-25T00:11:11.874813+0000","last_clean":"2024-04-25T00:11:11.874813+0000","last_became_active":"2024-04-25T00:11:10.897738+0000","last_became_peered":"2024-04-25T00:11:10.897738+0000","last_unstale":"2024-04-25T00:11:11.874813+0000","last_undegraded":"2024-04-25T00:11:11.874813+0000","last_fullsized":"2024-04-25T00:11:11.874813+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:44.129988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873858+0000","last_change":"2024-04-25T00:11:06.854098+0000","last_active":"2024-04-25T00:11:11.873858+0000","last_peered":"2024-04-25T00:11:11.873858+0000","last_clean":"2024-04-25T00:11:11.873858+0000","last_became_active":"2024-04-25T00:11:06.853839+0000","last_became_peered":"2024-04-25T00:11:06.853839+0000","last_unstale":"2024-04-25T00:11:11.873858+0000","last_undegraded":"2024-04-25T00:11:11.873858+0000","last_fullsized":"2024-04-25T00:11:11.873858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:00.950133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875315+0000","last_change":"2024-04-25T00:11:06.853055+0000","last_active":"2024-04-25T00:11:11.875315+0000","last_peered":"2024-04-25T00:11:11.875315+0000","last_clean":"2024-04-25T00:11:11.875315+0000","last_became_active":"2024-04-25T00:11:06.852814+0000","last_became_peered":"2024-04-25T00:11:06.852814+0000","last_unstale":"2024-04-25T00:11:11.875315+0000","last_undegraded":"2024-04-25T00:11:11.875315+0000","last_fullsized":"2024-04-25T00:11:11.875315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:42.291050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515002+0000","last_change":"2024-04-25T00:11:06.858759+0000","last_active":"2024-04-25T00:11:12.515002+0000","last_peered":"2024-04-25T00:11:12.515002+0000","last_clean":"2024-04-25T00:11:12.515002+0000","last_became_active":"2024-04-25T00:11:06.858472+0000","last_became_peered":"2024-04-25T00:11:06.858472+0000","last_unstale":"2024-04-25T00:11:12.515002+0000","last_undegraded":"2024-04-25T00:11:12.515002+0000","last_fullsized":"2024-04-25T00:11:12.515002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514169+0000","last_change":"2024-04-25T00:11:06.851389+0000","last_active":"2024-04-25T00:11:12.514169+0000","last_peered":"2024-04-25T00:11:12.514169+0000","last_clean":"2024-04-25T00:11:12.514169+0000","last_became_active":"2024-04-25T00:11:06.845828+0000","last_became_peered":"2024-04-25T00:11:06.845828+0000","last_unstale":"2024-04-25T00:11:12.514169+0000","last_undegraded":"2024-04-25T00:11:12.514169+0000","last_fullsized":"2024-04-25T00:11:12.514169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:00.950133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875364+0000","last_change":"2024-04-25T00:11:06.851807+0000","last_active":"2024-04-25T00:11:11.875364+0000","last_peered":"2024-04-25T00:11:11.875364+0000","last_clean":"2024-04-25T00:11:11.875364+0000","last_became_active":"2024-04-25T00:11:06.851495+0000","last_became_peered":"2024-04-25T00:11:06.851495+0000","last_unstale":"2024-04-25T00:11:11.875364+0000","last_undegraded":"2024-04-25T00:11:11.875364+0000","last_fullsized":"2024-04-25T00:11:11.875364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:49.414621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514751+0000","last_change":"2024-04-25T00:11:06.848495+0000","last_active":"2024-04-25T00:11:12.514751+0000","last_peered":"2024-04-25T00:11:12.514751+0000","last_clean":"2024-04-25T00:11:12.514751+0000","last_became_active":"2024-04-25T00:11:06.848242+0000","last_became_peered":"2024-04-25T00:11:06.848242+0000","last_unstale":"2024-04-25T00:11:12.514751+0000","last_undegraded":"2024-04-25T00:11:12.514751+0000","last_fullsized":"2024-04-25T00:11:12.514751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874704+0000","last_change":"2024-04-25T00:11:06.850991+0000","last_active":"2024-04-25T00:11:11.874704+0000","last_peered":"2024-04-25T00:11:11.874704+0000","last_clean":"2024-04-25T00:11:11.874704+0000","last_became_active":"2024-04-25T00:11:06.850666+0000","last_became_peered":"2024-04-25T00:11:06.850666+0000","last_unstale":"2024-04-25T00:11:11.874704+0000","last_undegraded":"2024-04-25T00:11:11.874704+0000","last_fullsized":"2024-04-25T00:11:11.874704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:25.890860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874498+0000","last_change":"2024-04-25T00:11:06.857585+0000","last_active":"2024-04-25T00:11:11.874498+0000","last_peered":"2024-04-25T00:11:11.874498+0000","last_clean":"2024-04-25T00:11:11.874498+0000","last_became_active":"2024-04-25T00:11:06.857315+0000","last_became_peered":"2024-04-25T00:11:06.857315+0000","last_unstale":"2024-04-25T00:11:11.874498+0000","last_undegraded":"2024-04-25T00:11:11.874498+0000","last_fullsized":"2024-04-25T00:11:11.874498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:43.612611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513979+0000","last_change":"2024-04-25T00:11:06.858752+0000","last_active":"2024-04-25T00:11:12.513979+0000","last_peered":"2024-04-25T00:11:12.513979+0000","last_clean":"2024-04-25T00:11:12.513979+0000","last_became_active":"2024-04-25T00:11:06.858428+0000","last_became_peered":"2024-04-25T00:11:06.858428+0000","last_unstale":"2024-04-25T00:11:12.513979+0000","last_undegraded":"2024-04-25T00:11:12.513979+0000","last_fullsized":"2024-04-25T00:11:12.513979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.219642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874132+0000","last_change":"2024-04-25T00:11:06.870006+0000","last_active":"2024-04-25T00:11:11.874132+0000","last_peered":"2024-04-25T00:11:11.874132+0000","last_clean":"2024-04-25T00:11:11.874132+0000","last_became_active":"2024-04-25T00:11:06.869687+0000","last_became_peered":"2024-04-25T00:11:06.869687+0000","last_unstale":"2024-04-25T00:11:11.874132+0000","last_undegraded":"2024-04-25T00:11:11.874132+0000","last_fullsized":"2024-04-25T00:11:11.874132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:43.565714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514922+0000","last_change":"2024-04-25T00:11:06.863927+0000","last_active":"2024-04-25T00:11:12.514922+0000","last_peered":"2024-04-25T00:11:12.514922+0000","last_clean":"2024-04-25T00:11:12.514922+0000","last_became_active":"2024-04-25T00:11:06.863758+0000","last_became_peered":"2024-04-25T00:11:06.863758+0000","last_unstale":"2024-04-25T00:11:12.514922+0000","last_undegraded":"2024-04-25T00:11:12.514922+0000","last_fullsized":"2024-04-25T00:11:12.514922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:25.890860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513912+0000","last_change":"2024-04-25T00:11:06.852070+0000","last_active":"2024-04-25T00:11:12.513912+0000","last_peered":"2024-04-25T00:11:12.513912+0000","last_clean":"2024-04-25T00:11:12.513912+0000","last_became_active":"2024-04-25T00:11:06.849951+0000","last_became_peered":"2024-04-25T00:11:06.849951+0000","last_unstale":"2024-04-25T00:11:12.513912+0000","last_undegraded":"2024-04-25T00:11:12.513912+0000","last_fullsized":"2024-04-25T00:11:12.513912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:54.204077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874289+0000","last_change":"2024-04-25T00:11:06.867310+0000","last_active":"2024-04-25T00:11:11.874289+0000","last_peered":"2024-04-25T00:11:11.874289+0000","last_clean":"2024-04-25T00:11:11.874289+0000","last_became_active":"2024-04-25T00:11:06.867010+0000","last_became_peered":"2024-04-25T00:11:06.867010+0000","last_unstale":"2024-04-25T00:11:11.874289+0000","last_undegraded":"2024-04-25T00:11:11.874289+0000","last_fullsized":"2024-04-25T00:11:11.874289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515126+0000","last_change":"2024-04-25T00:11:06.855941+0000","last_active":"2024-04-25T00:11:12.515126+0000","last_peered":"2024-04-25T00:11:12.515126+0000","last_clean":"2024-04-25T00:11:12.515126+0000","last_became_active":"2024-04-25T00:11:06.855737+0000","last_became_peered":"2024-04-25T00:11:06.855737+0000","last_unstale":"2024-04-25T00:11:12.515126+0000","last_undegraded":"2024-04-25T00:11:12.515126+0000","last_fullsized":"2024-04-25T00:11:12.515126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:18.665263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.876458+0000","last_change":"2024-04-25T00:11:06.865551+0000","last_active":"2024-04-25T00:11:11.876458+0000","last_peered":"2024-04-25T00:11:11.876458+0000","last_clean":"2024-04-25T00:11:11.876458+0000","last_became_active":"2024-04-25T00:11:06.865289+0000","last_became_peered":"2024-04-25T00:11:06.865289+0000","last_unstale":"2024-04-25T00:11:11.876458+0000","last_undegraded":"2024-04-25T00:11:11.876458+0000","last_fullsized":"2024-04-25T00:11:11.876458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:51.375329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516337+0000","last_change":"2024-04-25T00:11:06.863407+0000","last_active":"2024-04-25T00:11:12.516337+0000","last_peered":"2024-04-25T00:11:12.516337+0000","last_clean":"2024-04-25T00:11:12.516337+0000","last_became_active":"2024-04-25T00:11:06.863119+0000","last_became_peered":"2024-04-25T00:11:06.863119+0000","last_unstale":"2024-04-25T00:11:12.516337+0000","last_undegraded":"2024-04-25T00:11:12.516337+0000","last_fullsized":"2024-04-25T00:11:12.516337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:08.978181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513899+0000","last_change":"2024-04-25T00:11:06.854703+0000","last_active":"2024-04-25T00:11:12.513899+0000","last_peered":"2024-04-25T00:11:12.513899+0000","last_clean":"2024-04-25T00:11:12.513899+0000","last_became_active":"2024-04-25T00:11:06.854412+0000","last_became_peered":"2024-04-25T00:11:06.854412+0000","last_unstale":"2024-04-25T00:11:12.513899+0000","last_undegraded":"2024-04-25T00:11:12.513899+0000","last_fullsized":"2024-04-25T00:11:12.513899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:17.037033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514230+0000","last_change":"2024-04-25T00:11:06.850481+0000","last_active":"2024-04-25T00:11:12.514230+0000","last_peered":"2024-04-25T00:11:12.514230+0000","last_clean":"2024-04-25T00:11:12.514230+0000","last_became_active":"2024-04-25T00:11:06.850219+0000","last_became_peered":"2024-04-25T00:11:06.850219+0000","last_unstale":"2024-04-25T00:11:12.514230+0000","last_undegraded":"2024-04-25T00:11:12.514230+0000","last_fullsized":"2024-04-25T00:11:12.514230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514123+0000","last_change":"2024-04-25T00:11:06.841945+0000","last_active":"2024-04-25T00:11:12.514123+0000","last_peered":"2024-04-25T00:11:12.514123+0000","last_clean":"2024-04-25T00:11:12.514123+0000","last_became_active":"2024-04-25T00:11:06.841645+0000","last_became_peered":"2024-04-25T00:11:06.841645+0000","last_unstale":"2024-04-25T00:11:12.514123+0000","last_undegraded":"2024-04-25T00:11:12.514123+0000","last_fullsized":"2024-04-25T00:11:12.514123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:40.495315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874037+0000","last_change":"2024-04-25T00:11:06.856378+0000","last_active":"2024-04-25T00:11:11.874037+0000","last_peered":"2024-04-25T00:11:11.874037+0000","last_clean":"2024-04-25T00:11:11.874037+0000","last_became_active":"2024-04-25T00:11:06.856108+0000","last_became_peered":"2024-04-25T00:11:06.856108+0000","last_unstale":"2024-04-25T00:11:11.874037+0000","last_undegraded":"2024-04-25T00:11:11.874037+0000","last_fullsized":"2024-04-25T00:11:11.874037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515971+0000","last_change":"2024-04-25T00:11:06.868107+0000","last_active":"2024-04-25T00:11:12.515971+0000","last_peered":"2024-04-25T00:11:12.515971+0000","last_clean":"2024-04-25T00:11:12.515971+0000","last_became_active":"2024-04-25T00:11:06.867822+0000","last_became_peered":"2024-04-25T00:11:06.867822+0000","last_unstale":"2024-04-25T00:11:12.515971+0000","last_undegraded":"2024-04-25T00:11:12.515971+0000","last_fullsized":"2024-04-25T00:11:12.515971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:51.375329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874207+0000","last_change":"2024-04-25T00:11:06.848480+0000","last_active":"2024-04-25T00:11:11.874207+0000","last_peered":"2024-04-25T00:11:11.874207+0000","last_clean":"2024-04-25T00:11:11.874207+0000","last_became_active":"2024-04-25T00:11:06.848228+0000","last_became_peered":"2024-04-25T00:11:06.848228+0000","last_unstale":"2024-04-25T00:11:11.874207+0000","last_undegraded":"2024-04-25T00:11:11.874207+0000","last_fullsized":"2024-04-25T00:11:11.874207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874653+0000","last_change":"2024-04-25T00:11:06.852645+0000","last_active":"2024-04-25T00:11:11.874653+0000","last_peered":"2024-04-25T00:11:11.874653+0000","last_clean":"2024-04-25T00:11:11.874653+0000","last_became_active":"2024-04-25T00:11:06.852364+0000","last_became_peered":"2024-04-25T00:11:06.852364+0000","last_unstale":"2024-04-25T00:11:11.874653+0000","last_undegraded":"2024-04-25T00:11:11.874653+0000","last_fullsized":"2024-04-25T00:11:11.874653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514727+0000","last_change":"2024-04-25T00:11:06.858052+0000","last_active":"2024-04-25T00:11:12.514727+0000","last_peered":"2024-04-25T00:11:12.514727+0000","last_clean":"2024-04-25T00:11:12.514727+0000","last_became_active":"2024-04-25T00:11:06.857402+0000","last_became_peered":"2024-04-25T00:11:06.857402+0000","last_unstale":"2024-04-25T00:11:12.514727+0000","last_undegraded":"2024-04-25T00:11:12.514727+0000","last_fullsized":"2024-04-25T00:11:12.514727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:03.766175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874165+0000","last_change":"2024-04-25T00:11:06.852942+0000","last_active":"2024-04-25T00:11:11.874165+0000","last_peered":"2024-04-25T00:11:11.874165+0000","last_clean":"2024-04-25T00:11:11.874165+0000","last_became_active":"2024-04-25T00:11:06.852671+0000","last_became_peered":"2024-04-25T00:11:06.852671+0000","last_unstale":"2024-04-25T00:11:11.874165+0000","last_undegraded":"2024-04-25T00:11:11.874165+0000","last_fullsized":"2024-04-25T00:11:11.874165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515899+0000","last_change":"2024-04-25T00:11:06.853657+0000","last_active":"2024-04-25T00:11:12.515899+0000","last_peered":"2024-04-25T00:11:12.515899+0000","last_clean":"2024-04-25T00:11:12.515899+0000","last_became_active":"2024-04-25T00:11:06.853342+0000","last_became_peered":"2024-04-25T00:11:06.853342+0000","last_unstale":"2024-04-25T00:11:12.515899+0000","last_undegraded":"2024-04-25T00:11:12.515899+0000","last_fullsized":"2024-04-25T00:11:12.515899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:43.612611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514817+0000","last_change":"2024-04-25T00:11:06.854058+0000","last_active":"2024-04-25T00:11:12.514817+0000","last_peered":"2024-04-25T00:11:12.514817+0000","last_clean":"2024-04-25T00:11:12.514817+0000","last_became_active":"2024-04-25T00:11:06.853803+0000","last_became_peered":"2024-04-25T00:11:06.853803+0000","last_unstale":"2024-04-25T00:11:12.514817+0000","last_undegraded":"2024-04-25T00:11:12.514817+0000","last_fullsized":"2024-04-25T00:11:12.514817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514199+0000","last_change":"2024-04-25T00:11:06.851312+0000","last_active":"2024-04-25T00:11:12.514199+0000","last_peered":"2024-04-25T00:11:12.514199+0000","last_clean":"2024-04-25T00:11:12.514199+0000","last_became_active":"2024-04-25T00:11:06.851038+0000","last_became_peered":"2024-04-25T00:11:06.851038+0000","last_unstale":"2024-04-25T00:11:12.514199+0000","last_undegraded":"2024-04-25T00:11:12.514199+0000","last_fullsized":"2024-04-25T00:11:12.514199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.219642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514597+0000","last_change":"2024-04-25T00:11:06.860160+0000","last_active":"2024-04-25T00:11:12.514597+0000","last_peered":"2024-04-25T00:11:12.514597+0000","last_clean":"2024-04-25T00:11:12.514597+0000","last_became_active":"2024-04-25T00:11:06.859259+0000","last_became_peered":"2024-04-25T00:11:06.859259+0000","last_unstale":"2024-04-25T00:11:12.514597+0000","last_undegraded":"2024-04-25T00:11:12.514597+0000","last_fullsized":"2024-04-25T00:11:12.514597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:49.414621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513796+0000","last_change":"2024-04-25T00:11:06.848844+0000","last_active":"2024-04-25T00:11:12.513796+0000","last_peered":"2024-04-25T00:11:12.513796+0000","last_clean":"2024-04-25T00:11:12.513796+0000","last_became_active":"2024-04-25T00:11:06.848597+0000","last_became_peered":"2024-04-25T00:11:06.848597+0000","last_unstale":"2024-04-25T00:11:12.513796+0000","last_undegraded":"2024-04-25T00:11:12.513796+0000","last_fullsized":"2024-04-25T00:11:12.513796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515039+0000","last_change":"2024-04-25T00:11:06.844764+0000","last_active":"2024-04-25T00:11:12.515039+0000","last_peered":"2024-04-25T00:11:12.515039+0000","last_clean":"2024-04-25T00:11:12.515039+0000","last_became_active":"2024-04-25T00:11:06.844484+0000","last_became_peered":"2024-04-25T00:11:06.844484+0000","last_unstale":"2024-04-25T00:11:12.515039+0000","last_undegraded":"2024-04-25T00:11:12.515039+0000","last_fullsized":"2024-04-25T00:11:12.515039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514486+0000","last_change":"2024-04-25T00:11:06.860556+0000","last_active":"2024-04-25T00:11:12.514486+0000","last_peered":"2024-04-25T00:11:12.514486+0000","last_clean":"2024-04-25T00:11:12.514486+0000","last_became_active":"2024-04-25T00:11:06.860210+0000","last_became_peered":"2024-04-25T00:11:06.860210+0000","last_unstale":"2024-04-25T00:11:12.514486+0000","last_undegraded":"2024-04-25T00:11:12.514486+0000","last_fullsized":"2024-04-25T00:11:12.514486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:17.037033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:11:25.545 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:11:25.545 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T00:11:25.546 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T00:11:25.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T00:11:25.546 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T00:11:25.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.716+0000 7fa9f0800640 1 Processor -- start 2024-04-25T00:11:25.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.716+0000 7fa9f0800640 1 -- start start 2024-04-25T00:11:25.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.716+0000 7fa9f0800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9e81599f0 0x7fa9e80cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:25.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.716+0000 7fa9f0800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa9e815ce90 0x7fa9e80ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:25.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.716+0000 7fa9f0800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e80ccc60 con 0x7fa9e815c340 2024-04-25T00:11:25.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.716+0000 7fa9f0800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e8159dd0 con 0x7fa9e815ce90 2024-04-25T00:11:25.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.716+0000 7fa9f0800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e806dba0 con 0x7fa9e81599f0 2024-04-25T00:11:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.716+0000 7fa9ee800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa9e815ce90 0x7fa9e80ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.716+0000 7fa9ede00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9e81599f0 0x7fa9e80cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.716+0000 7fa9ee800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa9e815ce90 0x7fa9e80ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:38838/0 (socket says 172.21.15.83:38838) 2024-04-25T00:11:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.716+0000 7fa9ee800640 1 -- 172.21.15.83:0/99456754 learned_addr learned my addr 172.21.15.83:0/99456754 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.720+0000 7fa9ee800640 1 -- 172.21.15.83:0/99456754 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9e81599f0 msgr2=0x7fa9e80cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.720+0000 7fa9ee800640 1 --2- 172.21.15.83:0/99456754 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9e81599f0 0x7fa9e80cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.720+0000 7fa9ee800640 1 -- 172.21.15.83:0/99456754 >> v1:172.21.15.86:6789/0 conn(0x7fa9e815c340 legacy=0x7fa9e815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.720+0000 7fa9ee800640 1 -- 172.21.15.83:0/99456754 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e806de80 con 0x7fa9e815ce90 2024-04-25T00:11:25.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.720+0000 7fa9eca00640 1 -- 172.21.15.83:0/99456754 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3502511877 0 0) 0x7fa9d8001120 con 0x7fa9e815c340 2024-04-25T00:11:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.720+0000 7fa9ee800640 1 --2- 172.21.15.83:0/99456754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e815ce90 0x7fa9e80ce6e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009e80 tx=0x7fa9e401c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de623fc18f0a28ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.720+0000 7fa9eca00640 1 -- 172.21.15.83:0/99456754 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40a5640 con 0x7fa9e815ce90 2024-04-25T00:11:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.720+0000 7fa9eca00640 1 -- 172.21.15.83:0/99456754 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e40a57e0 con 0x7fa9e815ce90 2024-04-25T00:11:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.720+0000 7fa9eca00640 1 -- 172.21.15.83:0/99456754 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40a5bb0 con 0x7fa9e815ce90 2024-04-25T00:11:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 -- 172.21.15.83:0/99456754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e815ce90 msgr2=0x7fa9e80ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 --2- 172.21.15.83:0/99456754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e815ce90 0x7fa9e80ce6e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009e80 tx=0x7fa9e401c980 comp rx=0 tx=0).stop 2024-04-25T00:11:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 -- 172.21.15.83:0/99456754 shutdown_connections 2024-04-25T00:11:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 --2- 172.21.15.83:0/99456754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e815ce90 0x7fa9e80ce6e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 --2- 172.21.15.83:0/99456754 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9e81599f0 0x7fa9e80cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 -- 172.21.15.83:0/99456754 >> 172.21.15.83:0/99456754 conn(0x7fa9e80b1830 msgr2=0x7fa9e80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 -- 172.21.15.83:0/99456754 shutdown_connections 2024-04-25T00:11:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 -- 172.21.15.83:0/99456754 wait complete. 2024-04-25T00:11:25.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 Processor -- start 2024-04-25T00:11:25.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 -- start start 2024-04-25T00:11:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81599f0 0x7fa9e81580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9e815c340 0x7fa9e80b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e815ce90 0x7fa9e8146fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e80cf7c0 con 0x7fa9e81599f0 2024-04-25T00:11:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e80cf640 con 0x7fa9e815ce90 2024-04-25T00:11:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9f0800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e8168cd0 con 0x7fa9e815c340 2024-04-25T00:11:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9ede00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81599f0 0x7fa9e81580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9ede00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81599f0 0x7fa9e81580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36250/0 (socket says 172.21.15.83:36250) 2024-04-25T00:11:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9ede00640 1 -- 172.21.15.83:0/2350611859 learned_addr learned my addr 172.21.15.83:0/2350611859 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9ee800640 1 --2- 172.21.15.83:0/2350611859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e815ce90 0x7fa9e8146fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9ed400640 1 --2- 172.21.15.83:0/2350611859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9e815c340 0x7fa9e80b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9ede00640 1 -- 172.21.15.83:0/2350611859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9e815c340 msgr2=0x7fa9e80b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9ede00640 1 --2- 172.21.15.83:0/2350611859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9e815c340 0x7fa9e80b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9ede00640 1 -- 172.21.15.83:0/2350611859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e815ce90 msgr2=0x7fa9e8146fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9ede00640 1 --2- 172.21.15.83:0/2350611859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e815ce90 0x7fa9e8146fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.724+0000 7fa9ede00640 1 -- 172.21.15.83:0/2350611859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e8156f00 con 0x7fa9e81599f0 2024-04-25T00:11:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.728+0000 7fa9ede00640 1 --2- 172.21.15.83:0/2350611859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81599f0 0x7fa9e81580e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00e870 tx=0x7fa9dc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.728+0000 7fa9e2000640 1 -- 172.21.15.83:0/2350611859 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc017070 con 0x7fa9e81599f0 2024-04-25T00:11:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.728+0000 7fa9e2000640 1 -- 172.21.15.83:0/2350611859 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9dc00c940 con 0x7fa9e81599f0 2024-04-25T00:11:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.728+0000 7fa9e2000640 1 -- 172.21.15.83:0/2350611859 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc00ccf0 con 0x7fa9e81599f0 2024-04-25T00:11:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.728+0000 7fa9f0800640 1 -- 172.21.15.83:0/2350611859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e8155e60 con 0x7fa9e81599f0 2024-04-25T00:11:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.728+0000 7fa9f0800640 1 -- 172.21.15.83:0/2350611859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e8156420 con 0x7fa9e81599f0 2024-04-25T00:11:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.728+0000 7fa9f0800640 1 -- 172.21.15.83:0/2350611859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e80737f0 con 0x7fa9e81599f0 2024-04-25T00:11:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.728+0000 7fa9e2000640 1 -- 172.21.15.83:0/2350611859 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa9dc01a070 con 0x7fa9e81599f0 2024-04-25T00:11:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.728+0000 7fa9e2000640 1 --2- 172.21.15.83:0/2350611859 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa9cc0678c0 0x7fa9cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.728+0000 7fa9e2000640 1 -- 172.21.15.83:0/2350611859 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa9dc085ea0 con 0x7fa9e81599f0 2024-04-25T00:11:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.728+0000 7fa9ed400640 1 --2- 172.21.15.83:0/2350611859 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa9cc0678c0 0x7fa9cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:25.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.732+0000 7fa9ed400640 1 --2- 172.21.15.83:0/2350611859 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa9cc0678c0 0x7fa9cc069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8157330 tx=0x7fa9d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:25.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.732+0000 7fa9e2000640 1 -- 172.21.15.83:0/2350611859 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa9dc055fc0 con 0x7fa9e81599f0 2024-04-25T00:11:25.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.928+0000 7fa9f0800640 1 -- 172.21.15.83:0/2350611859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa9e804f5f0 con 0x7fa9e81599f0 2024-04-25T00:11:25.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.932+0000 7fa9e2000640 1 -- 172.21.15.83:0/2350611859 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+166883 (secure 0 0 0) 0x7fa9dc05a2c0 con 0x7fa9e81599f0 2024-04-25T00:11:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:25.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.936+0000 7fa9f0800640 1 -- 172.21.15.83:0/2350611859 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa9cc0678c0 msgr2=0x7fa9cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.936+0000 7fa9f0800640 1 --2- 172.21.15.83:0/2350611859 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa9cc0678c0 0x7fa9cc069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8157330 tx=0x7fa9d8009040 comp rx=0 tx=0).stop 2024-04-25T00:11:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.936+0000 7fa9f0800640 1 -- 172.21.15.83:0/2350611859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81599f0 msgr2=0x7fa9e81580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.936+0000 7fa9f0800640 1 --2- 172.21.15.83:0/2350611859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81599f0 0x7fa9e81580e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00e870 tx=0x7fa9dc00ec70 comp rx=0 tx=0).stop 2024-04-25T00:11:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.936+0000 7fa9f0800640 1 -- 172.21.15.83:0/2350611859 shutdown_connections 2024-04-25T00:11:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.936+0000 7fa9f0800640 1 --2- 172.21.15.83:0/2350611859 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa9cc0678c0 0x7fa9cc069d80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.936+0000 7fa9f0800640 1 --2- 172.21.15.83:0/2350611859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9e815c340 0x7fa9e80b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.936+0000 7fa9f0800640 1 --2- 172.21.15.83:0/2350611859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e815ce90 0x7fa9e8146fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.936+0000 7fa9f0800640 1 --2- 172.21.15.83:0/2350611859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81599f0 0x7fa9e81580e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.936+0000 7fa9f0800640 1 -- 172.21.15.83:0/2350611859 >> 172.21.15.83:0/2350611859 conn(0x7fa9e80b1830 msgr2=0x7fa9e8164e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:25.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.936+0000 7fa9f0800640 1 -- 172.21.15.83:0/2350611859 shutdown_connections 2024-04-25T00:11:25.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:25.936+0000 7fa9f0800640 1 -- 172.21.15.83:0/2350611859 wait complete. 2024-04-25T00:11:25.956 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":565012784},{"type":"v1","addr":"172.21.15.83:6833","nonce":565012784}]},"active_addr":"172.21.15.83:6833/565012784","active_change":"2024-04-25T00:10:46.430335+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":2819734865}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":285689663}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":4152948097}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3808261995}]}]} 2024-04-25T00:11:25.961 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T00:11:25.961 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T00:11:25.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:26.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.096+0000 7f9df3a00640 1 Processor -- start 2024-04-25T00:11:26.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.096+0000 7f9df3a00640 1 -- start start 2024-04-25T00:11:26.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.096+0000 7f9df3a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9dec15bf70 0x7f9dec0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.096+0000 7f9df3a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dec0cc9b0 con 0x7f9dec0cfd80 2024-04-25T00:11:26.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.096+0000 7f9df3a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dec0ccc90 con 0x7f9dec0d0940 2024-04-25T00:11:26.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.096+0000 7f9df3a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9dec0b5ad0 con 0x7f9dec15bf70 2024-04-25T00:11:26.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.096+0000 7f9debe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9dec0cfd80 0x7f9dec0d0230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52122/0 (socket says 172.21.15.83:52122) 2024-04-25T00:11:26.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.096+0000 7f9df1000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9dec15bf70 0x7f9dec0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.096+0000 7f9debe00640 1 -- 172.21.15.83:0/2565807369 learned_addr learned my addr 172.21.15.83:0/2565807369 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:26.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.096+0000 7f9df1000640 1 -- 172.21.15.83:0/2565807369 >> v1:172.21.15.86:6789/0 conn(0x7f9dec0d0940 legacy=0x7f9dec165b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:26.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.096+0000 7f9df1000640 1 -- 172.21.15.83:0/2565807369 >> v1:172.21.15.83:6789/0 conn(0x7f9dec0cfd80 legacy=0x7f9dec0d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.096+0000 7f9df1000640 1 -- 172.21.15.83:0/2565807369 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec0ccf70 con 0x7f9dec15bf70 2024-04-25T00:11:26.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.100+0000 7f9df1000640 1 --2- 172.21.15.83:0/2565807369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dec15bf70 0x7f9dec0cf840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0097c0 tx=0x7f9ddc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9677fa6263d12ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.100+0000 7f9deb400640 1 -- 172.21.15.83:0/2565807369 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc0ab430 con 0x7f9dec15bf70 2024-04-25T00:11:26.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.100+0000 7f9deb400640 1 -- 172.21.15.83:0/2565807369 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ddc0b0070 con 0x7f9dec15bf70 2024-04-25T00:11:26.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.100+0000 7f9deb400640 1 -- 172.21.15.83:0/2565807369 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc0ab7f0 con 0x7f9dec15bf70 2024-04-25T00:11:26.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.100+0000 7f9df3a00640 1 -- 172.21.15.83:0/2565807369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dec15bf70 msgr2=0x7f9dec0cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.100+0000 7f9df3a00640 1 --2- 172.21.15.83:0/2565807369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dec15bf70 0x7f9dec0cf840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0097c0 tx=0x7f9ddc01c430 comp rx=0 tx=0).stop 2024-04-25T00:11:26.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.100+0000 7f9df3a00640 1 -- 172.21.15.83:0/2565807369 shutdown_connections 2024-04-25T00:11:26.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.100+0000 7f9df3a00640 1 --2- 172.21.15.83:0/2565807369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dec15bf70 0x7f9dec0cf840 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.100+0000 7f9df3a00640 1 -- 172.21.15.83:0/2565807369 >> 172.21.15.83:0/2565807369 conn(0x7f9dec0d1c70 msgr2=0x7f9dec0d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.100+0000 7f9df3a00640 1 -- 172.21.15.83:0/2565807369 shutdown_connections 2024-04-25T00:11:26.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.100+0000 7f9df3a00640 1 -- 172.21.15.83:0/2565807369 wait complete. 2024-04-25T00:11:26.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.100+0000 7f9df3a00640 1 Processor -- start 2024-04-25T00:11:26.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.100+0000 7f9df3a00640 1 -- start start 2024-04-25T00:11:26.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9df3a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9dec0cfd80 0x7f9dec070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9df3a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dec0d0940 0x7f9dec0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9df3a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dec15bf70 0x7f9dec1530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9df3a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec0cda50 con 0x7f9dec0d0940 2024-04-25T00:11:26.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9df3a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec15c350 con 0x7f9dec15bf70 2024-04-25T00:11:26.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9df3a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9dec158ea0 con 0x7f9dec0cfd80 2024-04-25T00:11:26.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9debe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dec0d0940 0x7f9dec0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9debe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dec0d0940 0x7f9dec0b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36268/0 (socket says 172.21.15.83:36268) 2024-04-25T00:11:26.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9debe00640 1 -- 172.21.15.83:0/879514124 learned_addr learned my addr 172.21.15.83:0/879514124 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9df1a00640 1 --2- 172.21.15.83:0/879514124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dec15bf70 0x7f9dec1530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9df1000640 1 --2- 172.21.15.83:0/879514124 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9dec0cfd80 0x7f9dec070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9debe00640 1 -- 172.21.15.83:0/879514124 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9dec0cfd80 msgr2=0x7f9dec070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9debe00640 1 --2- 172.21.15.83:0/879514124 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9dec0cfd80 0x7f9dec070540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9debe00640 1 -- 172.21.15.83:0/879514124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dec15bf70 msgr2=0x7f9dec1530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9debe00640 1 --2- 172.21.15.83:0/879514124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dec15bf70 0x7f9dec1530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9debe00640 1 -- 172.21.15.83:0/879514124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec0c5cf0 con 0x7f9dec0d0940 2024-04-25T00:11:26.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9debe00640 1 --2- 172.21.15.83:0/879514124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dec0d0940 0x7f9dec0b3e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800c370 tx=0x7f9dd800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9de8c00640 1 -- 172.21.15.83:0/879514124 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8017070 con 0x7f9dec0d0940 2024-04-25T00:11:26.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9df3a00640 1 -- 172.21.15.83:0/879514124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dec0c5fd0 con 0x7f9dec0d0940 2024-04-25T00:11:26.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9df3a00640 1 -- 172.21.15.83:0/879514124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dec0c6590 con 0x7f9dec0d0940 2024-04-25T00:11:26.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9de8c00640 1 -- 172.21.15.83:0/879514124 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd8005cf0 con 0x7f9dec0d0940 2024-04-25T00:11:26.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9de8c00640 1 -- 172.21.15.83:0/879514124 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8021060 con 0x7f9dec0d0940 2024-04-25T00:11:26.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.104+0000 7f9df3a00640 1 -- 172.21.15.83:0/879514124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc40052a0 con 0x7f9dec0d0940 2024-04-25T00:11:26.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.108+0000 7f9de8c00640 1 -- 172.21.15.83:0/879514124 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9dd801a070 con 0x7f9dec0d0940 2024-04-25T00:11:26.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.108+0000 7f9de8c00640 1 --2- 172.21.15.83:0/879514124 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9dcc0678c0 0x7f9dcc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.108+0000 7f9de8c00640 1 -- 172.21.15.83:0/879514124 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9dd80852d0 con 0x7f9dec0d0940 2024-04-25T00:11:26.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.108+0000 7f9df1000640 1 --2- 172.21.15.83:0/879514124 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9dcc0678c0 0x7f9dcc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.108+0000 7f9df1000640 1 --2- 172.21.15.83:0/879514124 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9dcc0678c0 0x7f9dcc069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0af000 tx=0x7f9ddc0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.112+0000 7f9de8c00640 1 -- 172.21.15.83:0/879514124 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9dd80553e0 con 0x7f9dec0d0940 2024-04-25T00:11:26.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.268+0000 7f9df3a00640 1 -- 172.21.15.83:0/879514124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9dc4005590 con 0x7f9dec0d0940 2024-04-25T00:11:26.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.272+0000 7f9de8c00640 1 -- 172.21.15.83:0/879514124 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7f9dd80596e0 con 0x7f9dec0d0940 2024-04-25T00:11:26.274 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:26.274 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:26.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.272+0000 7f9df3a00640 1 -- 172.21.15.83:0/879514124 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9dcc0678c0 msgr2=0x7f9dcc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.272+0000 7f9df3a00640 1 --2- 172.21.15.83:0/879514124 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9dcc0678c0 0x7f9dcc069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0af000 tx=0x7f9ddc0a7000 comp rx=0 tx=0).stop 2024-04-25T00:11:26.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.272+0000 7f9df3a00640 1 -- 172.21.15.83:0/879514124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dec0d0940 msgr2=0x7f9dec0b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.272+0000 7f9df3a00640 1 --2- 172.21.15.83:0/879514124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dec0d0940 0x7f9dec0b3e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800c370 tx=0x7f9dd800c830 comp rx=0 tx=0).stop 2024-04-25T00:11:26.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.272+0000 7f9df3a00640 1 -- 172.21.15.83:0/879514124 shutdown_connections 2024-04-25T00:11:26.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.272+0000 7f9df3a00640 1 --2- 172.21.15.83:0/879514124 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9dcc0678c0 0x7f9dcc069d80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.272+0000 7f9df3a00640 1 --2- 172.21.15.83:0/879514124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dec15bf70 0x7f9dec1530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.272+0000 7f9df3a00640 1 --2- 172.21.15.83:0/879514124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dec0d0940 0x7f9dec0b3e40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.272+0000 7f9df3a00640 1 --2- 172.21.15.83:0/879514124 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9dec0cfd80 0x7f9dec070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.272+0000 7f9df3a00640 1 -- 172.21.15.83:0/879514124 >> 172.21.15.83:0/879514124 conn(0x7f9dec0d1c70 msgr2=0x7f9dec156ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.272+0000 7f9df3a00640 1 -- 172.21.15.83:0/879514124 shutdown_connections 2024-04-25T00:11:26.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.272+0000 7f9df3a00640 1 -- 172.21.15.83:0/879514124 wait complete. 2024-04-25T00:11:26.294 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T00:11:26.295 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:26.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.428+0000 7fbdef800640 1 Processor -- start 2024-04-25T00:11:26.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.428+0000 7fbdef800640 1 -- start start 2024-04-25T00:11:26.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.428+0000 7fbdef800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbde815e2c0 0x7fbde80715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.428+0000 7fbdef800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbde80aee90 con 0x7fbde80b4bc0 2024-04-25T00:11:26.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.428+0000 7fbdef800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbde80af170 con 0x7fbde815dee0 2024-04-25T00:11:26.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.428+0000 7fbdef800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbde80af7c0 con 0x7fbde815e2c0 2024-04-25T00:11:26.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.428+0000 7fbddfe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbde815e2c0 0x7fbde80715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.428+0000 7fbded800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbde80b4bc0 0x7fbde806d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52124/0 (socket says 172.21.15.83:52124) 2024-04-25T00:11:26.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.428+0000 7fbded800640 1 -- 172.21.15.83:0/1418406741 learned_addr learned my addr 172.21.15.83:0/1418406741 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:26.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.432+0000 7fbddfe00640 1 -- 172.21.15.83:0/1418406741 >> v1:172.21.15.86:6789/0 conn(0x7fbde815dee0 legacy=0x7fbde80b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.432+0000 7fbddfe00640 1 -- 172.21.15.83:0/1418406741 >> v1:172.21.15.83:6789/0 conn(0x7fbde80b4bc0 legacy=0x7fbde806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.432+0000 7fbddfe00640 1 -- 172.21.15.83:0/1418406741 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde80af450 con 0x7fbde815e2c0 2024-04-25T00:11:26.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.432+0000 7fbddfe00640 1 --2- 172.21.15.83:0/1418406741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde815e2c0 0x7fbde80715c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbde80af940 tx=0x7fbdd001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950de21f30bbe00b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.432+0000 7fbde7e00640 1 -- 172.21.15.83:0/1418406741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd00b0070 con 0x7fbde815e2c0 2024-04-25T00:11:26.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.432+0000 7fbde7e00640 1 -- 172.21.15.83:0/1418406741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdd00ab600 con 0x7fbde815e2c0 2024-04-25T00:11:26.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.432+0000 7fbde7e00640 1 -- 172.21.15.83:0/1418406741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd00ab9d0 con 0x7fbde815e2c0 2024-04-25T00:11:26.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.432+0000 7fbdef800640 1 -- 172.21.15.83:0/1418406741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde815e2c0 msgr2=0x7fbde80715c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.432+0000 7fbdef800640 1 --2- 172.21.15.83:0/1418406741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde815e2c0 0x7fbde80715c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbde80af940 tx=0x7fbdd001c380 comp rx=0 tx=0).stop 2024-04-25T00:11:26.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.432+0000 7fbdef800640 1 -- 172.21.15.83:0/1418406741 shutdown_connections 2024-04-25T00:11:26.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.432+0000 7fbdef800640 1 --2- 172.21.15.83:0/1418406741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde815e2c0 0x7fbde80715c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.432+0000 7fbdef800640 1 -- 172.21.15.83:0/1418406741 >> 172.21.15.83:0/1418406741 conn(0x7fbde806b1e0 msgr2=0x7fbde806b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.432+0000 7fbdef800640 1 -- 172.21.15.83:0/1418406741 shutdown_connections 2024-04-25T00:11:26.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.432+0000 7fbdef800640 1 -- 172.21.15.83:0/1418406741 wait complete. 2024-04-25T00:11:26.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdef800640 1 Processor -- start 2024-04-25T00:11:26.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdef800640 1 -- start start 2024-04-25T00:11:26.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdef800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80b4bc0 0x7fbde814d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdef800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbde815dee0 0x7fbde815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdef800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde815d480 0x7fbde81495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdef800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbde816c140 con 0x7fbde80b4bc0 2024-04-25T00:11:26.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdef800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbde816bfc0 con 0x7fbde815d480 2024-04-25T00:11:26.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdef800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbde816c2c0 con 0x7fbde815dee0 2024-04-25T00:11:26.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdece00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80b4bc0 0x7fbde814d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdece00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80b4bc0 0x7fbde814d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36282/0 (socket says 172.21.15.83:36282) 2024-04-25T00:11:26.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdece00640 1 -- 172.21.15.83:0/55193346 learned_addr learned my addr 172.21.15.83:0/55193346 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbded800640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde815d480 0x7fbde81495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbddfe00640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbde815dee0 0x7fbde815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdece00640 1 -- 172.21.15.83:0/55193346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbde815dee0 msgr2=0x7fbde815cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdece00640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbde815dee0 0x7fbde815cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdece00640 1 -- 172.21.15.83:0/55193346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde815d480 msgr2=0x7fbde81495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdece00640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde815d480 0x7fbde81495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdece00640 1 -- 172.21.15.83:0/55193346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde814fa80 con 0x7fbde80b4bc0 2024-04-25T00:11:26.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbddfe00640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbde815dee0 0x7fbde815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:26.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbded800640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde815d480 0x7fbde81495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:26.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdece00640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80b4bc0 0x7fbde814d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800c530 tx=0x7fbdd800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbde5600640 1 -- 172.21.15.83:0/55193346 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8016020 con 0x7fbde80b4bc0 2024-04-25T00:11:26.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbde5600640 1 -- 172.21.15.83:0/55193346 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdd8005220 con 0x7fbde80b4bc0 2024-04-25T00:11:26.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbde5600640 1 -- 172.21.15.83:0/55193346 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8007030 con 0x7fbde80b4bc0 2024-04-25T00:11:26.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdef800640 1 -- 172.21.15.83:0/55193346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde814fd60 con 0x7fbde80b4bc0 2024-04-25T00:11:26.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.436+0000 7fbdef800640 1 -- 172.21.15.83:0/55193346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde80b10d0 con 0x7fbde80b4bc0 2024-04-25T00:11:26.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.440+0000 7fbdef800640 1 -- 172.21.15.83:0/55193346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde8073000 con 0x7fbde80b4bc0 2024-04-25T00:11:26.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.440+0000 7fbde5600640 1 -- 172.21.15.83:0/55193346 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbdd80071d0 con 0x7fbde80b4bc0 2024-04-25T00:11:26.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.440+0000 7fbde5600640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbdc40678f0 0x7fbdc4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.440+0000 7fbde5600640 1 -- 172.21.15.83:0/55193346 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbdd8085a80 con 0x7fbde80b4bc0 2024-04-25T00:11:26.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.440+0000 7fbddfe00640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbdc40678f0 0x7fbdc4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.440+0000 7fbddfe00640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbdc40678f0 0x7fbdc4069db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00ad000 tx=0x7fbdd0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.444+0000 7fbde5600640 1 -- 172.21.15.83:0/55193346 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbdd8087e20 con 0x7fbde80b4bc0 2024-04-25T00:11:26.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.604+0000 7fbdef800640 1 -- 172.21.15.83:0/55193346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbde804f5f0 con 0x7fbde80b4bc0 2024-04-25T00:11:26.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.604+0000 7fbde5600640 1 -- 172.21.15.83:0/55193346 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7fbdd8020020 con 0x7fbde80b4bc0 2024-04-25T00:11:26.607 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:26.607 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.604+0000 7fbdef800640 1 -- 172.21.15.83:0/55193346 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbdc40678f0 msgr2=0x7fbdc4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.604+0000 7fbdef800640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbdc40678f0 0x7fbdc4069db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00ad000 tx=0x7fbdd0008040 comp rx=0 tx=0).stop 2024-04-25T00:11:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.604+0000 7fbdef800640 1 -- 172.21.15.83:0/55193346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80b4bc0 msgr2=0x7fbde814d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.604+0000 7fbdef800640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80b4bc0 0x7fbde814d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800c530 tx=0x7fbdd800c9f0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.604+0000 7fbdef800640 1 -- 172.21.15.83:0/55193346 shutdown_connections 2024-04-25T00:11:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.604+0000 7fbdef800640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbdc40678f0 0x7fbdc4069db0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.604+0000 7fbdef800640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbde815dee0 0x7fbde815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.604+0000 7fbdef800640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde815d480 0x7fbde81495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.604+0000 7fbdef800640 1 --2- 172.21.15.83:0/55193346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80b4bc0 0x7fbde814d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.604+0000 7fbdef800640 1 -- 172.21.15.83:0/55193346 >> 172.21.15.83:0/55193346 conn(0x7fbde806b1e0 msgr2=0x7fbde81609d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.604+0000 7fbdef800640 1 -- 172.21.15.83:0/55193346 shutdown_connections 2024-04-25T00:11:26.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.608+0000 7fbdef800640 1 -- 172.21.15.83:0/55193346 wait complete. 2024-04-25T00:11:26.629 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T00:11:26.629 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T00:11:26.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T00:11:26.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T00:11:26.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T00:11:26.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T00:11:26.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T00:11:26.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T00:11:26.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.868+0000 7f15d5200640 1 Processor -- start 2024-04-25T00:11:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.868+0000 7f15d5200640 1 -- start start 2024-04-25T00:11:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.872+0000 7f15d5200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f15c80389c0 0x7f15c8039f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.872+0000 7f15d5200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15c8054250 con 0x7f15c80362d0 2024-04-25T00:11:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.872+0000 7f15d5200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15c80359a0 con 0x7f15c8035c80 2024-04-25T00:11:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.872+0000 7f15d5200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f15c80552c0 con 0x7f15c80389c0 2024-04-25T00:11:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.872+0000 7f15cf400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f15c80389c0 0x7f15c8039f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.872+0000 7f15cf400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f15c80389c0 0x7f15c8039f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36298/0 (socket says 172.21.15.83:36298) 2024-04-25T00:11:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.872+0000 7f15cf400640 1 -- 172.21.15.83:0/3838622842 learned_addr learned my addr 172.21.15.83:0/3838622842 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.872+0000 7f3cfde00640 1 Processor -- start 2024-04-25T00:11:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.872+0000 7f3cfde00640 1 -- start start 2024-04-25T00:11:26.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.872+0000 7f95a3000640 1 Processor -- start 2024-04-25T00:11:26.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.872+0000 7f95a3000640 1 -- start start 2024-04-25T00:11:26.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.872+0000 7f15cf400640 1 -- 172.21.15.83:0/3838622842 >> v1:172.21.15.86:6789/0 conn(0x7f15c8035c80 legacy=0x7f15c80e0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.872+0000 7f15cf400640 1 -- 172.21.15.83:0/3838622842 >> v1:172.21.15.83:6789/0 conn(0x7f15c80362d0 legacy=0x7f15c8055750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.872+0000 7f15cf400640 1 -- 172.21.15.83:0/3838622842 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c80544c0 con 0x7f15c80389c0 2024-04-25T00:11:26.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f3cfde00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cf806ee20 0x7f3cf80cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f3cfde00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cf815f350 0x7f3cf8071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f3cfde00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cf80cb210 con 0x7f3cf81446e0 2024-04-25T00:11:26.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f3cfde00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3cf80715b0 con 0x7f3cf806ee20 2024-04-25T00:11:26.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f3cfde00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf8144ac0 con 0x7f3cf815f350 2024-04-25T00:11:26.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f95a3000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9594047c20 0x7f95940eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f95a3000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9594005ef0 0x7f9594006300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f95a3000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9594034220 con 0x7f9594033e40 2024-04-25T00:11:26.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f95a3000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9594050210 con 0x7f9594005ef0 2024-04-25T00:11:26.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f95a3000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9594037bc0 con 0x7f9594047c20 2024-04-25T00:11:26.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f3cf6000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cf806ee20 0x7f3cf80cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f95a2600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9594005ef0 0x7f9594006300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f95a2600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9594005ef0 0x7f9594006300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36316/0 (socket says 172.21.15.83:36316) 2024-04-25T00:11:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f95a2600640 1 -- 172.21.15.83:0/3881830428 learned_addr learned my addr 172.21.15.83:0/3881830428 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f3cf6000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cf806ee20 0x7f3cf80cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36304/0 (socket says 172.21.15.83:36304) 2024-04-25T00:11:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f3cf7400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cf815f350 0x7f3cf8071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f3cf6000640 1 -- 172.21.15.83:0/1468957847 learned_addr learned my addr 172.21.15.83:0/1468957847 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f95a1200640 1 --2- 172.21.15.83:0/3881830428 >> v2:172.21.15.86:3300/0 conn(0x7f9594047c20 0x7f95940eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f95a2600640 1 -- 172.21.15.83:0/3881830428 >> v2:172.21.15.86:3300/0 conn(0x7f9594047c20 msgr2=0x7f95940eb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:26.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f95a2600640 1 --2- 172.21.15.83:0/3881830428 >> v2:172.21.15.86:3300/0 conn(0x7f9594047c20 0x7f95940eb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f95a2600640 1 -- 172.21.15.83:0/3881830428 >> v1:172.21.15.86:6789/0 conn(0x7f9594033e40 legacy=0x7f95940eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f95a2600640 1 -- 172.21.15.83:0/3881830428 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959404fa30 con 0x7f9594005ef0 2024-04-25T00:11:26.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f3cf5600640 1 -- 172.21.15.83:0/1468957847 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2780650121 0 0) 0x7f3cf80cb210 con 0x7f3cf81446e0 2024-04-25T00:11:26.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f3cf5600640 1 -- 172.21.15.83:0/1468957847 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cdc003530 con 0x7f3cf81446e0 2024-04-25T00:11:26.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f95a1200640 1 --2- 172.21.15.83:0/3881830428 >> v2:172.21.15.86:3300/0 conn(0x7f9594047c20 0x7f95940eb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:26.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f3cf7400640 1 -- 172.21.15.83:0/1468957847 >> v2:172.21.15.83:3300/0 conn(0x7f3cf806ee20 msgr2=0x7f3cf80cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:26.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.876+0000 7f3cf5600640 1 -- 172.21.15.83:0/1468957847 <== mon.1 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2291828718 0 0) 0x7f3cdc003530 con 0x7f3cf81446e0 2024-04-25T00:11:26.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f3cf7400640 1 --2- 172.21.15.83:0/1468957847 >> v2:172.21.15.83:3300/0 conn(0x7f3cf806ee20 0x7f3cf80cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f3cf7400640 1 -- 172.21.15.83:0/1468957847 >> v1:172.21.15.86:6789/0 conn(0x7f3cf81446e0 legacy=0x7f3cf806e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f3cf7400640 1 -- 172.21.15.83:0/1468957847 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf80cb4f0 con 0x7f3cf815f350 2024-04-25T00:11:26.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f3cf6000640 1 --2- 172.21.15.83:0/1468957847 >> v2:172.21.15.83:3300/0 conn(0x7f3cf806ee20 0x7f3cf80cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:26.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f15cf400640 1 --2- 172.21.15.83:0/3838622842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c80389c0 0x7f15c8039f10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f15c40097f0 tx=0x7f15c401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e2e14cb8e091d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f384a000640 1 Processor -- start 2024-04-25T00:11:26.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f384a000640 1 -- start start 2024-04-25T00:11:26.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f15ce000640 1 -- 172.21.15.83:0/3838622842 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c40b0070 con 0x7f15c80389c0 2024-04-25T00:11:26.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f15ce000640 1 -- 172.21.15.83:0/3838622842 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c40ab600 con 0x7f15c80389c0 2024-04-25T00:11:26.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f15ce000640 1 -- 172.21.15.83:0/3838622842 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c40ab9d0 con 0x7f15c80389c0 2024-04-25T00:11:26.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f0c2d000640 1 Processor -- start 2024-04-25T00:11:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f95a2600640 1 --2- 172.21.15.83:0/3881830428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9594005ef0 0x7f9594006300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f959c074f60 tx=0x7f959c09a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7d7f283e58db1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f15d5200640 1 -- 172.21.15.83:0/3838622842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c80389c0 msgr2=0x7f15c8039f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f15d5200640 1 --2- 172.21.15.83:0/3838622842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c80389c0 0x7f15c8039f10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f15c40097f0 tx=0x7f15c401c300 comp rx=0 tx=0).stop 2024-04-25T00:11:26.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f0c2d000640 1 -- start start 2024-04-25T00:11:26.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f15d5200640 1 -- 172.21.15.83:0/3838622842 shutdown_connections 2024-04-25T00:11:26.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f15d5200640 1 --2- 172.21.15.83:0/3838622842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c80389c0 0x7f15c8039f10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f15d5200640 1 -- 172.21.15.83:0/3838622842 >> 172.21.15.83:0/3838622842 conn(0x7f15c8053760 msgr2=0x7f15c8053b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f959be00640 1 -- 172.21.15.83:0/3881830428 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c12f070 con 0x7f9594005ef0 2024-04-25T00:11:26.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f384a000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f383c052a50 0x7f383c053c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f384a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f383c052e30 0x7f383c0532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f384a000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f383c04f090 con 0x7f383c104870 2024-04-25T00:11:26.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f384a000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f383c04fa30 con 0x7f383c052a50 2024-04-25T00:11:26.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f384a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f383c04fd10 con 0x7f383c052e30 2024-04-25T00:11:26.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f959be00640 1 -- 172.21.15.83:0/3881830428 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f959c06e6e0 con 0x7f9594005ef0 2024-04-25T00:11:26.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f15d5200640 1 -- 172.21.15.83:0/3838622842 shutdown_connections 2024-04-25T00:11:26.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.880+0000 7f959be00640 1 -- 172.21.15.83:0/3881830428 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c06eb90 con 0x7f9594005ef0 2024-04-25T00:11:26.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f15d5200640 1 -- 172.21.15.83:0/3838622842 wait complete. 2024-04-25T00:11:26.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e8000640 1 Processor -- start 2024-04-25T00:11:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f0c2d000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c2806e0d0 0x7f0c2815e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3849600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f383c104870 0x7f383c104d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:46822/0 (socket says 172.21.15.83:46822) 2024-04-25T00:11:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3849600640 1 -- 172.21.15.83:0/1321591863 learned_addr learned my addr 172.21.15.83:0/1321591863 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3843e00640 1 --2- 172.21.15.83:0/1321591863 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f383c052e30 0x7f383c0532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3848c00640 1 --2- 172.21.15.83:0/1321591863 >> v2:172.21.15.83:3300/0 conn(0x7f383c052a50 0x7f383c053c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e8000640 1 -- start start 2024-04-25T00:11:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f0c2d000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c2815ec10 con 0x7f0c280b3ec0 2024-04-25T00:11:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f0c2d000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c2815eef0 con 0x7f0c2806e4b0 2024-04-25T00:11:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f0c2d000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2815f1d0 con 0x7f0c2806e0d0 2024-04-25T00:11:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3cf7400640 1 --2- 172.21.15.83:0/1468957847 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cf815f350 0x7f3cf8071fe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400dbe0 tx=0x7f3ce40206d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df171e935157c676 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3843e00640 1 -- 172.21.15.83:0/1321591863 >> v2:172.21.15.83:3300/0 conn(0x7f383c052a50 msgr2=0x7f383c053c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3843e00640 1 --2- 172.21.15.83:0/1321591863 >> v2:172.21.15.83:3300/0 conn(0x7f383c052a50 0x7f383c053c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3843e00640 1 -- 172.21.15.83:0/1321591863 >> v1:172.21.15.86:6789/0 conn(0x7f383c104870 legacy=0x7f383c104d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3843e00640 1 -- 172.21.15.83:0/1321591863 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383c04f370 con 0x7f383c052e30 2024-04-25T00:11:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f0c27400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c2806e0d0 0x7f0c2815e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f0c27400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c2806e0d0 0x7f0c2815e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:49770/0 (socket says 172.21.15.83:49770) 2024-04-25T00:11:26.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f0c27400640 1 -- 172.21.15.83:0/4101709347 learned_addr learned my addr 172.21.15.83:0/4101709347 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3cf5600640 1 -- 172.21.15.83:0/1468957847 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce40b0070 con 0x7f3cf815f350 2024-04-25T00:11:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3cf5600640 1 -- 172.21.15.83:0/1468957847 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce400d0d0 con 0x7f3cf815f350 2024-04-25T00:11:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3cf5600640 1 -- 172.21.15.83:0/1468957847 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0001a00 con 0x7f3cf815f350 2024-04-25T00:11:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f0c27400640 1 -- 172.21.15.83:0/4101709347 >> v1:172.21.15.86:6789/0 conn(0x7f0c2806e4b0 legacy=0x7f0c2806d2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:11:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f0c27400640 1 -- 172.21.15.83:0/4101709347 >> v1:172.21.15.83:6789/0 conn(0x7f0c280b3ec0 legacy=0x7f0c2806eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f0c27400640 1 -- 172.21.15.83:0/4101709347 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c280717a0 con 0x7f0c2806e0d0 2024-04-25T00:11:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e8000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e014bdc0 0x7f98e00cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e8000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98e0162a20 0x7f98e0162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e8000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98e014c220 con 0x7f98e014b870 2024-04-25T00:11:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e8000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f98e006f060 con 0x7f98e0162a20 2024-04-25T00:11:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e8000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f98e006cea0 con 0x7f98e014bdc0 2024-04-25T00:11:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98e0162a20 0x7f98e0162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e4c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e014bdc0 0x7f98e00cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e4c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e014bdc0 0x7f98e00cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:49780/0 (socket says 172.21.15.83:49780) 2024-04-25T00:11:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e4c00640 1 -- 172.21.15.83:0/3240549452 learned_addr learned my addr 172.21.15.83:0/3240549452 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e4c00640 1 -- 172.21.15.83:0/3240549452 >> v2:172.21.15.86:3300/0 conn(0x7f98e0162a20 msgr2=0x7f98e0162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e4c00640 1 --2- 172.21.15.83:0/3240549452 >> v2:172.21.15.86:3300/0 conn(0x7f98e0162a20 0x7f98e0162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e4c00640 1 -- 172.21.15.83:0/3240549452 >> v1:172.21.15.83:6789/0 conn(0x7f98e014b870 legacy=0x7f98e00cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f98e4c00640 1 -- 172.21.15.83:0/3240549452 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e006eaa0 con 0x7f98e014bdc0 2024-04-25T00:11:26.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f95a3000640 1 -- 172.21.15.83:0/3881830428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9594005ef0 msgr2=0x7f9594006300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f95a3000640 1 --2- 172.21.15.83:0/3881830428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9594005ef0 0x7f9594006300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f959c074f60 tx=0x7f959c09a340 comp rx=0 tx=0).stop 2024-04-25T00:11:26.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3cfde00640 1 -- 172.21.15.83:0/1468957847 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cf815f350 msgr2=0x7f3cf8071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3cfde00640 1 --2- 172.21.15.83:0/1468957847 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cf815f350 0x7f3cf8071fe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400dbe0 tx=0x7f3ce40206d0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f95a3000640 1 -- 172.21.15.83:0/3881830428 shutdown_connections 2024-04-25T00:11:26.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f95a3000640 1 --2- 172.21.15.83:0/3881830428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9594005ef0 0x7f9594006300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f95a3000640 1 --2- 172.21.15.83:0/3881830428 >> v2:172.21.15.86:3300/0 conn(0x7f9594047c20 0x7f95940eb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f95a3000640 1 -- 172.21.15.83:0/3881830428 >> 172.21.15.83:0/3881830428 conn(0x7f9594009fc0 msgr2=0x7f959400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f95a3000640 1 -- 172.21.15.83:0/3881830428 shutdown_connections 2024-04-25T00:11:26.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f95a3000640 1 -- 172.21.15.83:0/3881830428 wait complete. 2024-04-25T00:11:26.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3cfde00640 1 -- 172.21.15.83:0/1468957847 shutdown_connections 2024-04-25T00:11:26.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3cfde00640 1 --2- 172.21.15.83:0/1468957847 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cf815f350 0x7f3cf8071fe0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3cfde00640 1 --2- 172.21.15.83:0/1468957847 >> v2:172.21.15.83:3300/0 conn(0x7f3cf806ee20 0x7f3cf80cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3cfde00640 1 -- 172.21.15.83:0/1468957847 >> 172.21.15.83:0/1468957847 conn(0x7f3cf80cfa60 msgr2=0x7f3cf80cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3cdb600640 1 Processor -- start 2024-04-25T00:11:26.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.884+0000 7f3cdb600640 1 -- start start 2024-04-25T00:11:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f3cfde00640 1 -- 172.21.15.83:0/1468957847 shutdown_connections 2024-04-25T00:11:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f15d5200640 1 Processor -- start 2024-04-25T00:11:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f3cfde00640 1 -- 172.21.15.83:0/1468957847 wait complete. 2024-04-25T00:11:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f15d5200640 1 -- start start 2024-04-25T00:11:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f3cdb600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3cd41446e0 0x7f3cd40cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f23b3c00640 1 Processor -- start 2024-04-25T00:11:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f0c27400640 1 --2- 172.21.15.83:0/4101709347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c2806e0d0 0x7f0c2815e350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0c180098a0 tx=0x7f0c1801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29faf5f88e69f1b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f3cdb600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cd40cb210 con 0x7f3cd415f350 2024-04-25T00:11:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f3cdb600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cd40cb4f0 con 0x7f3cd406ee20 2024-04-25T00:11:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f3cd9600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3cd406ee20 0x7f3cd4071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:46836/0 (socket says 172.21.15.83:46836) 2024-04-25T00:11:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f3cd9600640 1 -- 172.21.15.83:0/3420328548 learned_addr learned my addr 172.21.15.83:0/3420328548 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cdb600640 1 -- 172.21.15.83:0/3420328548 --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3cd40715b0 con 0x7f3cd41446e0 2024-04-25T00:11:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15d5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15c80362d0 0x7f15c80cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f0c26000640 1 -- 172.21.15.83:0/4101709347 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c180aa610 con 0x7f0c2806e0d0 2024-04-25T00:11:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f0c26000640 1 -- 172.21.15.83:0/4101709347 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c180af070 con 0x7f0c2806e0d0 2024-04-25T00:11:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f0c26000640 1 -- 172.21.15.83:0/4101709347 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c180aa9d0 con 0x7f0c2806e0d0 2024-04-25T00:11:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f23b3c00640 1 -- start start 2024-04-25T00:11:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f3cfde00640 1 Processor -- start 2024-04-25T00:11:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.888+0000 7f3cfde00640 1 -- start start 2024-04-25T00:11:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15d5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c80389c0 0x7f15c804ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15d5200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15c804b540 0x7f15c80d0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15d5200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f15c80eef10 con 0x7f15c80389c0 2024-04-25T00:11:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15d5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15c8038fb0 con 0x7f15c80362d0 2024-04-25T00:11:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15d5200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f15c80ef090 con 0x7f15c804b540 2024-04-25T00:11:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15c80362d0 0x7f15c80cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15c80362d0 0x7f15c80cb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:38914/0 (socket says 172.21.15.83:38914) 2024-04-25T00:11:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15cf400640 1 -- 172.21.15.83:0/1981083757 learned_addr learned my addr 172.21.15.83:0/1981083757 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15cfe00640 1 --2- 172.21.15.83:0/1981083757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15c804b540 0x7f15c80d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cfde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf806ee20 0x7f3cf8140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cfde00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cf81446e0 0x7f3cf813b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cfde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cf815f350 0x7f3cf8159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cfde00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf8072eb0 con 0x7f3cf806ee20 2024-04-25T00:11:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cfde00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf8072d30 con 0x7f3cf815f350 2024-04-25T00:11:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cfde00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf8073030 con 0x7f3cf81446e0 2024-04-25T00:11:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cf6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf806ee20 0x7f3cf8140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cf7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cf815f350 0x7f3cf8159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cf7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cf815f350 0x7f3cf8159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:38928/0 (socket says 172.21.15.83:38928) 2024-04-25T00:11:26.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cf7400640 1 -- 172.21.15.83:0/1540398834 learned_addr learned my addr 172.21.15.83:0/1540398834 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cf6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf806ee20 0x7f3cf8140e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36352/0 (socket says 172.21.15.83:36352) 2024-04-25T00:11:26.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f0c2d000640 1 -- 172.21.15.83:0/4101709347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c2806e0d0 msgr2=0x7f0c2815e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f0c2d000640 1 --2- 172.21.15.83:0/4101709347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c2806e0d0 0x7f0c2815e350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0c180098a0 tx=0x7f0c1801c4d0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3843e00640 1 --2- 172.21.15.83:0/1321591863 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f383c052e30 0x7f383c0532b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f38300097c0 tx=0x7f383001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a398fc409350a125 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f98e4c00640 1 --2- 172.21.15.83:0/3240549452 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e014bdc0 0x7f98e00cbe50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009870 tx=0x7f98cc01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8199487ae68ff49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f98dfe00640 1 -- 172.21.15.83:0/3240549452 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0b0070 con 0x7f98e014bdc0 2024-04-25T00:11:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f98dfe00640 1 -- 172.21.15.83:0/3240549452 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98cc0a4e50 con 0x7f98e014bdc0 2024-04-25T00:11:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f98dfe00640 1 -- 172.21.15.83:0/3240549452 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0ab820 con 0x7f98e014bdc0 2024-04-25T00:11:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 -- 172.21.15.83:0/3420328548 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3504770450 0 0) 0x7f3cd40cb4f0 con 0x7f3cd406ee20 2024-04-25T00:11:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 -- 172.21.15.83:0/3420328548 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cb8003530 con 0x7f3cd406ee20 2024-04-25T00:11:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd8c00640 1 --2- 172.21.15.83:0/3420328548 >> v2:172.21.15.86:3300/0 conn(0x7f3cd41446e0 0x7f3cd40cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 -- 172.21.15.83:0/3420328548 <== mon.1 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 197312769 0 0) 0x7f3cb8003530 con 0x7f3cd406ee20 2024-04-25T00:11:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 -- 172.21.15.83:0/3420328548 >> v2:172.21.15.86:3300/0 conn(0x7f3cd41446e0 msgr2=0x7f3cd40cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 --2- 172.21.15.83:0/3420328548 >> v2:172.21.15.86:3300/0 conn(0x7f3cd41446e0 0x7f3cd40cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 -- 172.21.15.83:0/3420328548 >> v1:172.21.15.83:6789/0 conn(0x7f3cd415f350 legacy=0x7f3cd406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 -- 172.21.15.83:0/3420328548 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd40717e0 con 0x7f3cd406ee20 2024-04-25T00:11:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd8c00640 1 --2- 172.21.15.83:0/3420328548 >> v2:172.21.15.86:3300/0 conn(0x7f3cd41446e0 0x7f3cd40cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 -- 172.21.15.83:0/3420328548 <== mon.1 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f3cc0001680 con 0x7f3cd406ee20 2024-04-25T00:11:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 -- 172.21.15.83:0/3420328548 >> v1:172.21.15.86:6789/0 conn(0x7f3cd406ee20 legacy=0x7f3cd4071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 --2- 172.21.15.83:0/3420328548 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cb80039c0 0x7f3cb8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 --2- 172.21.15.83:0/3420328548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb8004810 0x7f3cb8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 --2- 172.21.15.83:0/3420328548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb80051d0 0x7f3cb800ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f23b3c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f23ac0c4c50 0x7f23ac0d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f23b3c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23ac0cf040 con 0x7f23ac0b1060 2024-04-25T00:11:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f23b3c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23ac0cf2e0 con 0x7f23ac162760 2024-04-25T00:11:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f23b3c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f23ac0cf5c0 con 0x7f23ac0c4c50 2024-04-25T00:11:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15cf400640 1 -- 172.21.15.83:0/1981083757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15c804b540 msgr2=0x7f15c80d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15cf400640 1 --2- 172.21.15.83:0/1981083757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15c804b540 0x7f15c80d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 -- 172.21.15.83:0/3420328548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8010520 con 0x7f3cb80051d0 2024-04-25T00:11:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 -- 172.21.15.83:0/3420328548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb80106d0 con 0x7f3cb8004810 2024-04-25T00:11:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3400640 1 -- 172.21.15.83:0/3420328548 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb80108d0 con 0x7f3cb80039c0 2024-04-25T00:11:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15cf400640 1 -- 172.21.15.83:0/1981083757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c80389c0 msgr2=0x7f15c804ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15cf400640 1 --2- 172.21.15.83:0/1981083757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c80389c0 0x7f15c804ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15cf400640 1 -- 172.21.15.83:0/1981083757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c80cbc40 con 0x7f15c80362d0 2024-04-25T00:11:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3e00640 1 --2- 172.21.15.83:0/3420328548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb8004810 0x7f3cb8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f23abe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f23ac162760 0x7f23ac0c5030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:46846/0 (socket says 172.21.15.83:46846) 2024-04-25T00:11:26.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f23abe00640 1 -- 172.21.15.83:0/2875660332 learned_addr learned my addr 172.21.15.83:0/2875660332 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:26.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cf6a00640 1 -- 172.21.15.83:0/1540398834 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cf81446e0 msgr2=0x7f3cf813b2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:11:26.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cf6a00640 1 --2- 172.21.15.83:0/1540398834 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cf81446e0 0x7f3cf813b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cf6a00640 1 -- 172.21.15.83:0/1540398834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cf815f350 msgr2=0x7f3cf8159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cf6a00640 1 --2- 172.21.15.83:0/1540398834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cf815f350 0x7f3cf8159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cf6a00640 1 -- 172.21.15.83:0/1540398834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf815d2e0 con 0x7f3cf806ee20 2024-04-25T00:11:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cf6a00640 1 --2- 172.21.15.83:0/1540398834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf806ee20 0x7f3cf8140e10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000cac0 tx=0x7f3cf000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3e00640 1 -- 172.21.15.83:0/3420328548 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cb80039c0 msgr2=0x7f3cb8003de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:11:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3e00640 1 --2- 172.21.15.83:0/3420328548 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cb80039c0 0x7f3cb8003de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3e00640 1 -- 172.21.15.83:0/3420328548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb80051d0 msgr2=0x7f3cb800ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3e00640 1 --2- 172.21.15.83:0/3420328548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb80051d0 0x7f3cb800ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd3e00640 1 -- 172.21.15.83:0/3420328548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb8003530 con 0x7f3cb8004810 2024-04-25T00:11:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3cd9600640 1 -- 172.21.15.83:0/3420328548 reap_dead start 2024-04-25T00:11:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f23ab400640 1 -- 172.21.15.83:0/2875660332 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3959435070 0 0) 0x7f23ac0cf2e0 con 0x7f23ac162760 2024-04-25T00:11:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f23ab400640 1 -- 172.21.15.83:0/2875660332 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2390003530 con 0x7f23ac162760 2024-04-25T00:11:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f15cfe00640 1 --2- 172.21.15.83:0/1981083757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15c804b540 0x7f15c80d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f0c2d000640 1 -- 172.21.15.83:0/4101709347 shutdown_connections 2024-04-25T00:11:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f0c2d000640 1 --2- 172.21.15.83:0/4101709347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c2806e0d0 0x7f0c2815e350 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f0c2d000640 1 -- 172.21.15.83:0/4101709347 >> 172.21.15.83:0/4101709347 conn(0x7f0c280b1650 msgr2=0x7f0c280b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f0c2d000640 1 -- 172.21.15.83:0/4101709347 shutdown_connections 2024-04-25T00:11:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f3843400640 1 -- 172.21.15.83:0/1321591863 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38300b0070 con 0x7f383c052e30 2024-04-25T00:11:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f3843400640 1 -- 172.21.15.83:0/1321591863 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38300ab600 con 0x7f383c052e30 2024-04-25T00:11:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f3843400640 1 -- 172.21.15.83:0/1321591863 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38300ab9d0 con 0x7f383c052e30 2024-04-25T00:11:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f98e8000640 1 -- 172.21.15.83:0/3240549452 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e014bdc0 msgr2=0x7f98e00cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f98e8000640 1 --2- 172.21.15.83:0/3240549452 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e014bdc0 0x7f98e00cbe50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009870 tx=0x7f98cc01c400 comp rx=0 tx=0).stop 2024-04-25T00:11:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f98e8000640 1 -- 172.21.15.83:0/3240549452 shutdown_connections 2024-04-25T00:11:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f98e8000640 1 --2- 172.21.15.83:0/3240549452 >> v2:172.21.15.86:3300/0 conn(0x7f98e0162a20 0x7f98e0162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f98e8000640 1 --2- 172.21.15.83:0/3240549452 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e014bdc0 0x7f98e00cbe50 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009870 tx=0x7f98cc01c400 comp rx=0 tx=0).stop 2024-04-25T00:11:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f98e8000640 1 -- 172.21.15.83:0/3240549452 >> 172.21.15.83:0/3240549452 conn(0x7f98e00cfa60 msgr2=0x7f98e00cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23ab400640 1 -- 172.21.15.83:0/2875660332 <== mon.1 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2498567644 0 0) 0x7f2390003530 con 0x7f23ac162760 2024-04-25T00:11:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23ab400640 1 -- 172.21.15.83:0/2875660332 >> v2:172.21.15.86:3300/0 conn(0x7f23ac0c4c50 msgr2=0x7f23ac0d27a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:11:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f0c2d000640 1 -- 172.21.15.83:0/4101709347 wait complete. 2024-04-25T00:11:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f3cef400640 1 -- 172.21.15.83:0/1540398834 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0016020 con 0x7f3cf806ee20 2024-04-25T00:11:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f15cf400640 1 --2- 172.21.15.83:0/1981083757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15c80362d0 0x7f15c80cb700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f15c4018f10 tx=0x7f15c40190a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f15c3400640 1 -- 172.21.15.83:0/1981083757 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c40c5070 con 0x7f15c80362d0 2024-04-25T00:11:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f15c3400640 1 -- 172.21.15.83:0/1981083757 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c40c0630 con 0x7f15c80362d0 2024-04-25T00:11:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f15c3400640 1 -- 172.21.15.83:0/1981083757 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c40c0a20 con 0x7f15c80362d0 2024-04-25T00:11:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f15d5200640 1 -- 172.21.15.83:0/1981083757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c804b9d0 con 0x7f15c80362d0 2024-04-25T00:11:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f15d5200640 1 -- 172.21.15.83:0/1981083757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c804bf90 con 0x7f15c80362d0 2024-04-25T00:11:26.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f15d5200640 1 -- 172.21.15.83:0/1981083757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f15ac001040 con 0x7f15c80362d0 2024-04-25T00:11:26.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.892+0000 7f95a3000640 1 Processor -- start 2024-04-25T00:11:26.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f95a3000640 1 -- start start 2024-04-25T00:11:26.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f3cef400640 1 -- 172.21.15.83:0/1540398834 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf000aab0 con 0x7f3cf806ee20 2024-04-25T00:11:26.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f3cef400640 1 -- 172.21.15.83:0/1540398834 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0012420 con 0x7f3cf806ee20 2024-04-25T00:11:26.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f3cfde00640 1 -- 172.21.15.83:0/1540398834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf815d5c0 con 0x7f3cf806ee20 2024-04-25T00:11:26.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f3cfde00640 1 -- 172.21.15.83:0/1540398834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf815db80 con 0x7f3cf806ee20 2024-04-25T00:11:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f0c2d000640 1 Processor -- start 2024-04-25T00:11:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f384a000640 1 -- 172.21.15.83:0/1321591863 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f383c052e30 msgr2=0x7f383c0532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f384a000640 1 --2- 172.21.15.83:0/1321591863 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f383c052e30 0x7f383c0532b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f38300097c0 tx=0x7f383001c440 comp rx=0 tx=0).stop 2024-04-25T00:11:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f384a000640 1 -- 172.21.15.83:0/1321591863 shutdown_connections 2024-04-25T00:11:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f384a000640 1 --2- 172.21.15.83:0/1321591863 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f383c052e30 0x7f383c0532b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f384a000640 1 --2- 172.21.15.83:0/1321591863 >> v2:172.21.15.83:3300/0 conn(0x7f383c052a50 0x7f383c053c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f384a000640 1 -- 172.21.15.83:0/1321591863 >> 172.21.15.83:0/1321591863 conn(0x7f383c009fc0 msgr2=0x7f383c00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f384a000640 1 -- 172.21.15.83:0/1321591863 shutdown_connections 2024-04-25T00:11:26.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f384a000640 1 -- 172.21.15.83:0/1321591863 wait complete. 2024-04-25T00:11:26.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f98e8000640 1 -- 172.21.15.83:0/3240549452 shutdown_connections 2024-04-25T00:11:26.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3cd3e00640 1 --2- 172.21.15.83:0/3420328548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb8004810 0x7f3cb8004c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4001780 tx=0x7f3cc401c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d106d562eb2ebe84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23b1200640 1 --2- 172.21.15.83:0/2875660332 >> v2:172.21.15.86:3300/0 conn(0x7f23ac0c4c50 0x7f23ac0d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23ab400640 1 --2- 172.21.15.83:0/2875660332 >> v2:172.21.15.86:3300/0 conn(0x7f23ac0c4c50 0x7f23ac0d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23ab400640 1 -- 172.21.15.83:0/2875660332 >> v1:172.21.15.83:6789/0 conn(0x7f23ac0b1060 legacy=0x7f23ac06c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:26.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23ab400640 1 -- 172.21.15.83:0/2875660332 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac06f100 con 0x7f23ac162760 2024-04-25T00:11:26.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23ab400640 1 -- 172.21.15.83:0/2875660332 <== mon.1 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f2398001680 con 0x7f23ac162760 2024-04-25T00:11:26.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23ab400640 1 -- 172.21.15.83:0/2875660332 >> v1:172.21.15.86:6789/0 conn(0x7f23ac162760 legacy=0x7f23ac0c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23ab400640 1 --2- 172.21.15.83:0/2875660332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23900039c0 0x7f2390003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23ab400640 1 --2- 172.21.15.83:0/2875660332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2390004810 0x7f2390004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23ab400640 1 --2- 172.21.15.83:0/2875660332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23900053d0 0x7f239000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23ab400640 1 -- 172.21.15.83:0/2875660332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac0cf5c0 con 0x7f23900039c0 2024-04-25T00:11:26.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23ab400640 1 -- 172.21.15.83:0/2875660332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f239000c270 con 0x7f2390004810 2024-04-25T00:11:26.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23ab400640 1 -- 172.21.15.83:0/2875660332 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f239000c470 con 0x7f23900053d0 2024-04-25T00:11:26.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23b1200640 1 --2- 172.21.15.83:0/2875660332 >> v2:172.21.15.86:3300/0 conn(0x7f23ac0c4c50 0x7f23ac0d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:26.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23abe00640 1 --2- 172.21.15.83:0/2875660332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2390004810 0x7f2390004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23abe00640 1 -- 172.21.15.83:0/2875660332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23900053d0 msgr2=0x7f239000b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:11:26.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23abe00640 1 --2- 172.21.15.83:0/2875660332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23900053d0 0x7f239000b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23abe00640 1 -- 172.21.15.83:0/2875660332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23900039c0 msgr2=0x7f2390003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:26.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23abe00640 1 --2- 172.21.15.83:0/2875660332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23900039c0 0x7f2390003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23abe00640 1 -- 172.21.15.83:0/2875660332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac06f100 con 0x7f2390004810 2024-04-25T00:11:26.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f23b1c00640 1 -- 172.21.15.83:0/2875660332 reap_dead start 2024-04-25T00:11:26.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.896+0000 7f3cfde00640 1 -- 172.21.15.83:0/1540398834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3cf8068230 con 0x7f3cf806ee20 2024-04-25T00:11:26.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3cd3400640 1 -- 172.21.15.83:0/3420328548 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40013a0 con 0x7f3cb8004810 2024-04-25T00:11:26.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3cd3400640 1 -- 172.21.15.83:0/3420328548 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc40052d0 con 0x7f3cb8004810 2024-04-25T00:11:26.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3cd3400640 1 -- 172.21.15.83:0/3420328548 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40013a0 con 0x7f3cb8004810 2024-04-25T00:11:26.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f384a000640 1 Processor -- start 2024-04-25T00:11:26.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f98e8000640 1 -- 172.21.15.83:0/3240549452 wait complete. 2024-04-25T00:11:26.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f384a000640 1 -- start start 2024-04-25T00:11:26.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a3000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9594005ef0 0x7f9594048e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a3000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9594033e40 0x7f9594049820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a3000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9594047c20 0x7f95940e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a3000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f959404fd80 con 0x7f9594005ef0 2024-04-25T00:11:26.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a3000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f959404f090 con 0x7f9594047c20 2024-04-25T00:11:26.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a3000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f959404f5c0 con 0x7f9594033e40 2024-04-25T00:11:26.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f0c2d000640 1 -- start start 2024-04-25T00:11:26.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f0c2d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2806e0d0 0x7f0c2814ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f0c2d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c2806e4b0 0x7f0c280adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3cdb600640 1 -- 172.21.15.83:0/3420328548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb8004810 msgr2=0x7f3cb8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3cdb600640 1 --2- 172.21.15.83:0/3420328548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb8004810 0x7f3cb8004c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4001780 tx=0x7f3cc401c360 comp rx=0 tx=0).stop 2024-04-25T00:11:26.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f0c27400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2806e0d0 0x7f0c2814ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f0c27400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2806e0d0 0x7f0c2814ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:38954/0 (socket says 172.21.15.83:38954) 2024-04-25T00:11:26.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9594047c20 0x7f95940e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f0c2d000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c280b3ec0 0x7f0c280a9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3cdb600640 1 -- 172.21.15.83:0/3420328548 shutdown_connections 2024-04-25T00:11:26.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3cdb600640 1 --2- 172.21.15.83:0/3420328548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb8004810 0x7f3cb8004c90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3cdb600640 1 -- 172.21.15.83:0/3420328548 >> 172.21.15.83:0/3420328548 conn(0x7f3cd40cfa60 msgr2=0x7f3cd40cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9594047c20 0x7f95940e49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:38956/0 (socket says 172.21.15.83:38956) 2024-04-25T00:11:26.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a2600640 1 -- 172.21.15.83:0/2503404545 learned_addr learned my addr 172.21.15.83:0/2503404545 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f0c27400640 1 -- 172.21.15.83:0/1386140072 learned_addr learned my addr 172.21.15.83:0/1386140072 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f0c26a00640 1 --2- 172.21.15.83:0/1386140072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c2806e4b0 0x7f0c280adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f0c27e00640 1 --2- 172.21.15.83:0/1386140072 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c280b3ec0 0x7f0c280a9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f384a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f383c052a50 0x7f383c103840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f384a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f383c052e30 0x7f383c0f1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f384a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f383c104870 0x7f383c0efd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f384a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f383c037bc0 con 0x7f383c052a50 2024-04-25T00:11:26.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f384a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f383c101080 con 0x7f383c104870 2024-04-25T00:11:26.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f384a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f383c050180 con 0x7f383c052e30 2024-04-25T00:11:26.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a2600640 1 -- 172.21.15.83:0/2503404545 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9594033e40 msgr2=0x7f9594049820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:11:26.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f0c2d000640 1 -- 172.21.15.83:0/1386140072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2806f3d0 con 0x7f0c2806e4b0 2024-04-25T00:11:26.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3843e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f383c052e30 0x7f383c0f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3843e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f383c052e30 0x7f383c0f1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:49830/0 (socket says 172.21.15.83:49830) 2024-04-25T00:11:26.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3843e00640 1 -- 172.21.15.83:0/3962973764 learned_addr learned my addr 172.21.15.83:0/3962973764 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a2600640 1 --2- 172.21.15.83:0/2503404545 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9594033e40 0x7f9594049820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a2600640 1 -- 172.21.15.83:0/2503404545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9594005ef0 msgr2=0x7f9594048e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a2600640 1 --2- 172.21.15.83:0/2503404545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9594005ef0 0x7f9594048e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a2600640 1 -- 172.21.15.83:0/2503404545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959404a050 con 0x7f9594047c20 2024-04-25T00:11:26.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f0c2d000640 1 -- 172.21.15.83:0/1386140072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2806f250 con 0x7f0c2806e0d0 2024-04-25T00:11:26.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f0c2d000640 1 -- 172.21.15.83:0/1386140072 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0c280727b0 con 0x7f0c280b3ec0 2024-04-25T00:11:26.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f98e8000640 1 Processor -- start 2024-04-25T00:11:26.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3cef400640 1 -- 172.21.15.83:0/1540398834 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3cf0019020 con 0x7f3cf806ee20 2024-04-25T00:11:26.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f15c3400640 1 -- 172.21.15.83:0/1981083757 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f15c40c0bc0 con 0x7f15c80362d0 2024-04-25T00:11:26.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f95a2600640 1 --2- 172.21.15.83:0/2503404545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9594047c20 0x7f95940e49c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f959c064d10 tx=0x7f959c097130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3843e00640 1 -- 172.21.15.83:0/3962973764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f383c104870 msgr2=0x7f383c0efd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3843e00640 1 --2- 172.21.15.83:0/3962973764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f383c104870 0x7f383c0efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3843e00640 1 -- 172.21.15.83:0/3962973764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f383c052a50 msgr2=0x7f383c103840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3843e00640 1 --2- 172.21.15.83:0/3962973764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f383c052a50 0x7f383c103840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f3843e00640 1 -- 172.21.15.83:0/3962973764 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383c1024f0 con 0x7f383c052e30 2024-04-25T00:11:26.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3843e00640 1 --2- 172.21.15.83:0/3962973764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f383c052e30 0x7f383c0f1740 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f383001cce0 tx=0x7f38300191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.900+0000 7f23abe00640 1 --2- 172.21.15.83:0/2875660332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2390004810 0x7f2390004c90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f239800c260 tx=0x7f239801ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=467eae6bb3e0bc7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3cdb600640 1 -- 172.21.15.83:0/3420328548 shutdown_connections 2024-04-25T00:11:26.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3cdb600640 1 -- 172.21.15.83:0/3420328548 wait complete. 2024-04-25T00:11:26.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3cef400640 1 --2- 172.21.15.83:0/1540398834 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cd8067910 0x7f3cd8069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f23ab400640 1 -- 172.21.15.83:0/2875660332 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398007d80 con 0x7f2390004810 2024-04-25T00:11:26.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f98e8000640 1 -- start start 2024-04-25T00:11:26.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f23ab400640 1 -- 172.21.15.83:0/2875660332 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23980a9070 con 0x7f2390004810 2024-04-25T00:11:26.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3cef400640 1 -- 172.21.15.83:0/1540398834 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3cf0086920 con 0x7f3cf806ee20 2024-04-25T00:11:26.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f9599600640 1 -- 172.21.15.83:0/2503404545 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c12f040 con 0x7f9594047c20 2024-04-25T00:11:26.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f9599600640 1 -- 172.21.15.83:0/2503404545 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f959c134480 con 0x7f9594047c20 2024-04-25T00:11:26.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f9599600640 1 -- 172.21.15.83:0/2503404545 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c12f040 con 0x7f9594047c20 2024-04-25T00:11:26.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f95a3000640 1 -- 172.21.15.83:0/2503404545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9594048370 con 0x7f9594047c20 2024-04-25T00:11:26.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f95a3000640 1 -- 172.21.15.83:0/2503404545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9594048930 con 0x7f9594047c20 2024-04-25T00:11:26.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3cef400640 1 --2- 172.21.15.83:0/1540398834 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x7f3cd8071090 0x7f3cd80734d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3cef400640 1 -- 172.21.15.83:0/1540398834 --> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3cf0026ec0 con 0x7f3cd8071090 2024-04-25T00:11:26.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3cef400640 1 -- 172.21.15.83:0/1540398834 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3cf0086e00 con 0x7f3cf806ee20 2024-04-25T00:11:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3840c00640 1 -- 172.21.15.83:0/3962973764 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38300b0040 con 0x7f383c052e30 2024-04-25T00:11:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3840c00640 1 -- 172.21.15.83:0/3962973764 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38300bb070 con 0x7f383c052e30 2024-04-25T00:11:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3840c00640 1 -- 172.21.15.83:0/3962973764 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38300b58e0 con 0x7f383c052e30 2024-04-25T00:11:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f384a000640 1 -- 172.21.15.83:0/3962973764 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f383c04bd40 con 0x7f383c052e30 2024-04-25T00:11:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f384a000640 1 -- 172.21.15.83:0/3962973764 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f383c04c300 con 0x7f383c052e30 2024-04-25T00:11:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f23b3c00640 1 -- 172.21.15.83:0/2875660332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2390004810 msgr2=0x7f2390004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f23b3c00640 1 --2- 172.21.15.83:0/2875660332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2390004810 0x7f2390004c90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f239800c260 tx=0x7f239801ccb0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3cf7400640 1 --2- 172.21.15.83:0/1540398834 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x7f3cd8071090 0x7f3cd80734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f0c27e00640 1 -- 172.21.15.83:0/1386140072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2806e0d0 msgr2=0x7f0c2814ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f0c27e00640 1 --2- 172.21.15.83:0/1386140072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2806e0d0 0x7f0c2814ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f0c27e00640 1 -- 172.21.15.83:0/1386140072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c2806e4b0 msgr2=0x7f0c280adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f0c27e00640 1 --2- 172.21.15.83:0/1386140072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c2806e4b0 0x7f0c280adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f0c27e00640 1 -- 172.21.15.83:0/1386140072 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c280ae990 con 0x7f0c280b3ec0 2024-04-25T00:11:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f0c27e00640 1 --2- 172.21.15.83:0/1386140072 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c280b3ec0 0x7f0c280a9dd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400c410 tx=0x7f0c1400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f0c1f400640 1 -- 172.21.15.83:0/1386140072 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14017070 con 0x7f0c280b3ec0 2024-04-25T00:11:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f0c1f400640 1 -- 172.21.15.83:0/1386140072 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c14005230 con 0x7f0c280b3ec0 2024-04-25T00:11:26.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f0c1f400640 1 -- 172.21.15.83:0/1386140072 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14004590 con 0x7f0c280b3ec0 2024-04-25T00:11:26.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f0c2d000640 1 -- 172.21.15.83:0/1386140072 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c28203cb0 con 0x7f0c280b3ec0 2024-04-25T00:11:26.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f0c2d000640 1 -- 172.21.15.83:0/1386140072 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c28204270 con 0x7f0c280b3ec0 2024-04-25T00:11:26.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f384a000640 1 -- 172.21.15.83:0/3962973764 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f383c0f1c80 con 0x7f383c052e30 2024-04-25T00:11:26.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f98e8000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e014b870 0x7f98e0146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f98e8000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e0162a20 0x7f98e0161b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f98e8000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98e0160f60 0x7f98e01613d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f98e8000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98e016f3e0 con 0x7f98e014b870 2024-04-25T00:11:26.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f98e8000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98e00ce050 con 0x7f98e0160f60 2024-04-25T00:11:26.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f98e8000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f98e016f560 con 0x7f98e0162a20 2024-04-25T00:11:26.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3cf6000640 1 --2- 172.21.15.83:0/1540398834 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cd8067910 0x7f3cd8069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f98e4c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e0162a20 0x7f98e0161b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f15c3400640 1 --2- 172.21.15.83:0/1981083757 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f15b0067990 0x7f15b0069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f15c3400640 1 -- 172.21.15.83:0/1981083757 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f15c41283f0 con 0x7f15c80362d0 2024-04-25T00:11:26.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f15c3400640 1 --2- 172.21.15.83:0/1981083757 >> [v2:172.21.15.83:6800/397603850,v1:172.21.15.83:6801/397603850] conn(0x7f15b0071110 0x7f15b0073550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f15c3400640 1 -- 172.21.15.83:0/1981083757 --> [v2:172.21.15.83:6800/397603850,v1:172.21.15.83:6801/397603850] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f15c40b87e0 con 0x7f15b0071110 2024-04-25T00:11:26.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f15c3400640 1 -- 172.21.15.83:0/1981083757 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f15c412f050 con 0x7f15c80362d0 2024-04-25T00:11:26.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f98e4c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e0162a20 0x7f98e0161b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:49842/0 (socket says 172.21.15.83:49842) 2024-04-25T00:11:26.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f98e4c00640 1 -- 172.21.15.83:0/3222476169 learned_addr learned my addr 172.21.15.83:0/3222476169 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3cf7400640 1 --2- 172.21.15.83:0/1540398834 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x7f3cd8071090 0x7f3cd80734d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3cdb600640 1 Processor -- start 2024-04-25T00:11:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cf6000640 1 --2- 172.21.15.83:0/1540398834 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cd8067910 0x7f3cd8069dd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009630 tx=0x7f3ce8009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f95a3000640 1 -- 172.21.15.83:0/2503404545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9578001040 con 0x7f9594047c20 2024-04-25T00:11:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.904+0000 7f3cdb600640 1 -- start start 2024-04-25T00:11:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f98e6000640 1 --2- 172.21.15.83:0/3222476169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98e0160f60 0x7f98e01613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f98e4c00640 1 -- 172.21.15.83:0/3222476169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98e0160f60 msgr2=0x7f98e01613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cef400640 1 -- 172.21.15.83:0/1540398834 <== osd.2 v2:172.21.15.83:6824/1702952090 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3cf0026ec0 con 0x7f3cd8071090 2024-04-25T00:11:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f98e4c00640 1 --2- 172.21.15.83:0/3222476169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98e0160f60 0x7f98e01613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f98e4c00640 1 -- 172.21.15.83:0/3222476169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e014b870 msgr2=0x7f98e0146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f98e4c00640 1 --2- 172.21.15.83:0/3222476169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e014b870 0x7f98e0146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f98e4c00640 1 -- 172.21.15.83:0/3222476169 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e0144600 con 0x7f98e0162a20 2024-04-25T00:11:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f23b3c00640 1 -- 172.21.15.83:0/2875660332 shutdown_connections 2024-04-25T00:11:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f23b3c00640 1 --2- 172.21.15.83:0/2875660332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2390004810 0x7f2390004c90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f23b3c00640 1 -- 172.21.15.83:0/2875660332 >> 172.21.15.83:0/2875660332 conn(0x7f23ac0d1c80 msgr2=0x7f23ac0d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f15cea00640 1 --2- 172.21.15.83:0/1981083757 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f15b0067990 0x7f15b0069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f0c1f400640 1 -- 172.21.15.83:0/1386140072 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0c1401a070 con 0x7f0c280b3ec0 2024-04-25T00:11:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3840c00640 1 -- 172.21.15.83:0/3962973764 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f38300b3070 con 0x7f383c052e30 2024-04-25T00:11:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f98e4c00640 1 --2- 172.21.15.83:0/3222476169 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e0162a20 0x7f98e0161b90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0ae040 tx=0x7f98cc01ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cdb600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb8004810 0x7f3cd40b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cdb600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cd415a570 0x7f3cd40afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cdb600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd40b0240 0x7f3cd40b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cdb600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd4071b50 con 0x7f3cb8004810 2024-04-25T00:11:26.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cdb600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd4144ac0 con 0x7f3cd40b0240 2024-04-25T00:11:26.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cdb600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd40cd630 con 0x7f3cd415a570 2024-04-25T00:11:26.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f0c2d000640 1 -- 172.21.15.83:0/1386140072 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c2804f510 con 0x7f0c280b3ec0 2024-04-25T00:11:26.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb8004810 0x7f3cd40b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb8004810 0x7f3cd40b46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36406/0 (socket says 172.21.15.83:36406) 2024-04-25T00:11:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd8c00640 1 -- 172.21.15.83:0/505029996 learned_addr learned my addr 172.21.15.83:0/505029996 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f15cea00640 1 --2- 172.21.15.83:0/1981083757 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f15b0067990 0x7f15b0069e50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f15c80cffa0 tx=0x7f15bc007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f98dd600640 1 -- 172.21.15.83:0/3222476169 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0ba020 con 0x7f98e0162a20 2024-04-25T00:11:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f0c1f400640 1 --2- 172.21.15.83:0/1386140072 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0c08067940 0x7f0c08069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f98e8000640 1 -- 172.21.15.83:0/3222476169 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e0143d60 con 0x7f98e0162a20 2024-04-25T00:11:26.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f98dd600640 1 -- 172.21.15.83:0/3222476169 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98cc0abdf0 con 0x7f98e0162a20 2024-04-25T00:11:26.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f98dd600640 1 -- 172.21.15.83:0/3222476169 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0b0040 con 0x7f98e0162a20 2024-04-25T00:11:26.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f98e8000640 1 -- 172.21.15.83:0/3222476169 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e0144320 con 0x7f98e0162a20 2024-04-25T00:11:26.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd9600640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd40b0240 0x7f3cd40b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd3e00640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cd415a570 0x7f3cd40afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd8c00640 1 -- 172.21.15.83:0/505029996 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cd415a570 msgr2=0x7f3cd40afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd8c00640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cd415a570 0x7f3cd40afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd8c00640 1 -- 172.21.15.83:0/505029996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd40b0240 msgr2=0x7f3cd40b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd8c00640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd40b0240 0x7f3cd40b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd8c00640 1 -- 172.21.15.83:0/505029996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd40b3610 con 0x7f3cb8004810 2024-04-25T00:11:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd3e00640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cd415a570 0x7f3cd40afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f23b3c00640 1 -- 172.21.15.83:0/2875660332 shutdown_connections 2024-04-25T00:11:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f23b3c00640 1 -- 172.21.15.83:0/2875660332 wait complete. 2024-04-25T00:11:26.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f15cfe00640 1 --2- 172.21.15.83:0/1981083757 >> [v2:172.21.15.83:6800/397603850,v1:172.21.15.83:6801/397603850] conn(0x7f15b0071110 0x7f15b0073550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f0c1f400640 1 -- 172.21.15.83:0/1386140072 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0c140850c0 con 0x7f0c280b3ec0 2024-04-25T00:11:26.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f0c1f400640 1 --2- 172.21.15.83:0/1386140072 >> [v2:172.21.15.83:6809/1635741185,v1:172.21.15.83:6811/1635741185] conn(0x7f0c080710c0 0x7f0c08073500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f0c1f400640 1 -- 172.21.15.83:0/1386140072 --> [v2:172.21.15.83:6809/1635741185,v1:172.21.15.83:6811/1635741185] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c08073bd0 con 0x7f0c080710c0 2024-04-25T00:11:26.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f0c1f400640 1 -- 172.21.15.83:0/1386140072 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0c1408c050 con 0x7f0c280b3ec0 2024-04-25T00:11:26.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3840c00640 1 --2- 172.21.15.83:0/3962973764 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3824067910 0x7f3824069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3840c00640 1 -- 172.21.15.83:0/3962973764 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f383011fa80 con 0x7f383c052e30 2024-04-25T00:11:26.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3840c00640 1 --2- 172.21.15.83:0/3962973764 >> [v2:172.21.15.86:6816/1560646044,v1:172.21.15.86:6817/1560646044] conn(0x7f3824071080 0x7f38240734c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3840c00640 1 -- 172.21.15.83:0/3962973764 --> [v2:172.21.15.86:6816/1560646044,v1:172.21.15.86:6817/1560646044] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f38300adec0 con 0x7f3824071080 2024-04-25T00:11:26.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3840c00640 1 -- 172.21.15.83:0/3962973764 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f38301242c0 con 0x7f383c052e30 2024-04-25T00:11:26.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f0c27400640 1 --2- 172.21.15.83:0/1386140072 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0c08067940 0x7f0c08069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f0c26a00640 1 --2- 172.21.15.83:0/1386140072 >> [v2:172.21.15.83:6809/1635741185,v1:172.21.15.83:6811/1635741185] conn(0x7f0c080710c0 0x7f0c08073500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f9599600640 1 -- 172.21.15.83:0/2503404545 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f959c13d030 con 0x7f9594047c20 2024-04-25T00:11:26.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f98d2a00640 1 -- 172.21.15.83:0/3222476169 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98e0068230 con 0x7f98e0162a20 2024-04-25T00:11:26.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd8c00640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb8004810 0x7f3cd40b46b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc007ed0 tx=0x7f3ccc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd0c00640 1 -- 172.21.15.83:0/505029996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc01a070 con 0x7f3cb8004810 2024-04-25T00:11:26.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd0c00640 1 -- 172.21.15.83:0/505029996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ccc003ba0 con 0x7f3cb8004810 2024-04-25T00:11:26.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cd0c00640 1 -- 172.21.15.83:0/505029996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc004030 con 0x7f3cb8004810 2024-04-25T00:11:26.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.908+0000 7f3cdb600640 1 -- 172.21.15.83:0/505029996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd40b38f0 con 0x7f3cb8004810 2024-04-25T00:11:26.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f0c27400640 1 --2- 172.21.15.83:0/1386140072 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0c08067940 0x7f0c08069e00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0c1801cd70 tx=0x7f0c180a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f3cdb600640 1 -- 172.21.15.83:0/505029996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd40b3eb0 con 0x7f3cb8004810 2024-04-25T00:11:26.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f9599600640 1 --2- 172.21.15.83:0/2503404545 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f957c067af0 0x7f957c069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f9599600640 1 -- 172.21.15.83:0/2503404545 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f959c19e610 con 0x7f9594047c20 2024-04-25T00:11:26.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f9599600640 1 --2- 172.21.15.83:0/2503404545 >> [v2:172.21.15.83:6808/228430303,v1:172.21.15.83:6810/228430303] conn(0x7f957c0711f0 0x7f957c073630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f9599600640 1 -- 172.21.15.83:0/2503404545 --> [v2:172.21.15.83:6808/228430303,v1:172.21.15.83:6810/228430303] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f959c129e80 con 0x7f957c0711f0 2024-04-25T00:11:26.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f9599600640 1 -- 172.21.15.83:0/2503404545 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f959c1a5050 con 0x7f9594047c20 2024-04-25T00:11:26.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f3cdb600640 1 -- 172.21.15.83:0/505029996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3cd40b4290 con 0x7f3cb8004810 2024-04-25T00:11:26.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f95a1200640 1 --2- 172.21.15.83:0/2503404545 >> [v2:172.21.15.83:6808/228430303,v1:172.21.15.83:6810/228430303] conn(0x7f957c0711f0 0x7f957c073630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f98dd600640 1 -- 172.21.15.83:0/3222476169 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f98cc0bd020 con 0x7f98e0162a20 2024-04-25T00:11:26.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f0c26a00640 1 --2- 172.21.15.83:0/1386140072 >> [v2:172.21.15.83:6809/1635741185,v1:172.21.15.83:6811/1635741185] conn(0x7f0c080710c0 0x7f0c08073500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f3849600640 1 --2- 172.21.15.83:0/3962973764 >> [v2:172.21.15.86:6816/1560646044,v1:172.21.15.86:6817/1560646044] conn(0x7f3824071080 0x7f38240734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f23b3c00640 1 Processor -- start 2024-04-25T00:11:26.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f23b3c00640 1 -- start start 2024-04-25T00:11:26.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f95a1200640 1 --2- 172.21.15.83:0/2503404545 >> [v2:172.21.15.83:6808/228430303,v1:172.21.15.83:6810/228430303] conn(0x7f957c0711f0 0x7f957c073630 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f23b3c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2390004810 0x7f23ac14ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f23b3c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ac165b60 0x7f23ac0b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f23b3c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23ac162b40 0x7f23ac162fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f23b3c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac1669f0 con 0x7f23ac165b60 2024-04-25T00:11:26.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f23b3c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac166870 con 0x7f23ac162b40 2024-04-25T00:11:26.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f23b3c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f23ac166b70 con 0x7f2390004810 2024-04-25T00:11:26.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f98dd600640 1 --2- 172.21.15.83:0/3222476169 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98c00678f0 0x7f98c0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f98dd600640 1 -- 172.21.15.83:0/3222476169 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f98cc11fff0 con 0x7f98e0162a20 2024-04-25T00:11:26.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f98dd600640 1 --2- 172.21.15.83:0/3222476169 >> [v2:172.21.15.86:6805/340393403,v1:172.21.15.86:6807/340393403] conn(0x7f98c0070ff0 0x7f98c0073430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f98dd600640 1 -- 172.21.15.83:0/3222476169 --> [v2:172.21.15.86:6805/340393403,v1:172.21.15.86:6807/340393403] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98c0073b00 con 0x7f98c0070ff0 2024-04-25T00:11:26.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f98dd600640 1 -- 172.21.15.83:0/3222476169 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f98cc122610 con 0x7f98e0162a20 2024-04-25T00:11:26.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f3cd0c00640 1 -- 172.21.15.83:0/505029996 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3ccc01d070 con 0x7f3cb8004810 2024-04-25T00:11:26.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f98e5600640 1 --2- 172.21.15.83:0/3222476169 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98c00678f0 0x7f98c0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f3cd0c00640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cb4067990 0x7f3cb4069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f3848c00640 1 --2- 172.21.15.83:0/3962973764 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3824067910 0x7f3824069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f3cd0c00640 1 -- 172.21.15.83:0/505029996 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3ccc086340 con 0x7f3cb8004810 2024-04-25T00:11:26.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f0c1f400640 1 -- 172.21.15.83:0/1386140072 <== osd.3 v2:172.21.15.83:6809/1635741185 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c08073bd0 con 0x7f0c080710c0 2024-04-25T00:11:26.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f23b1200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2390004810 0x7f23ac14ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f23b1200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2390004810 0x7f23ac14ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:49864/0 (socket says 172.21.15.83:49864) 2024-04-25T00:11:26.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f23b1200640 1 -- 172.21.15.83:0/896617916 learned_addr learned my addr 172.21.15.83:0/896617916 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:26.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.912+0000 7f23b1c00640 1 --2- 172.21.15.83:0/896617916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23ac162b40 0x7f23ac162fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f23b1200640 1 -- 172.21.15.83:0/896617916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23ac162b40 msgr2=0x7f23ac162fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f23b1200640 1 --2- 172.21.15.83:0/896617916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23ac162b40 0x7f23ac162fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f23b1200640 1 -- 172.21.15.83:0/896617916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ac165b60 msgr2=0x7f23ac0b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f23b1200640 1 --2- 172.21.15.83:0/896617916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23ac165b60 0x7f23ac0b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f23b1200640 1 -- 172.21.15.83:0/896617916 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac0b2f80 con 0x7f2390004810 2024-04-25T00:11:26.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f23b1200640 1 --2- 172.21.15.83:0/896617916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2390004810 0x7f23ac14ca40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f23a400ced0 tx=0x7f23a400a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f23a8c00640 1 -- 172.21.15.83:0/896617916 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a401a070 con 0x7f2390004810 2024-04-25T00:11:26.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f95a1c00640 1 --2- 172.21.15.83:0/2503404545 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f957c067af0 0x7f957c069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f9599600640 1 -- 172.21.15.83:0/2503404545 <== osd.1 v2:172.21.15.83:6808/228430303 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f959c129e80 con 0x7f957c0711f0 2024-04-25T00:11:26.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f3849600640 1 --2- 172.21.15.83:0/3962973764 >> [v2:172.21.15.86:6816/1560646044,v1:172.21.15.86:6817/1560646044] conn(0x7f3824071080 0x7f38240734c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f98e6000640 1 --2- 172.21.15.83:0/3222476169 >> [v2:172.21.15.86:6805/340393403,v1:172.21.15.86:6807/340393403] conn(0x7f98c0070ff0 0x7f98c0073430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f98e5600640 1 --2- 172.21.15.83:0/3222476169 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98c00678f0 0x7f98c0069db0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f98d4003cf0 tx=0x7f98d40079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f23a8c00640 1 -- 172.21.15.83:0/896617916 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23a400d840 con 0x7f2390004810 2024-04-25T00:11:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f23a8c00640 1 -- 172.21.15.83:0/896617916 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a400dc30 con 0x7f2390004810 2024-04-25T00:11:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f23b3c00640 1 -- 172.21.15.83:0/896617916 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ac0b3260 con 0x7f2390004810 2024-04-25T00:11:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f23b3c00640 1 -- 172.21.15.83:0/896617916 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ac0b3820 con 0x7f2390004810 2024-04-25T00:11:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f23b3c00640 1 -- 172.21.15.83:0/896617916 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2384001040 con 0x7f2390004810 2024-04-25T00:11:26.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f3848c00640 1 --2- 172.21.15.83:0/3962973764 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3824067910 0x7f3824069dd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f383800b850 tx=0x7f3838008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f3840c00640 1 -- 172.21.15.83:0/3962973764 <== osd.4 v2:172.21.15.86:6816/1560646044 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f38300adec0 con 0x7f3824071080 2024-04-25T00:11:26.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f98e6000640 1 --2- 172.21.15.83:0/3222476169 >> [v2:172.21.15.86:6805/340393403,v1:172.21.15.86:6807/340393403] conn(0x7f98c0070ff0 0x7f98c0073430 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f15cfe00640 1 --2- 172.21.15.83:0/1981083757 >> [v2:172.21.15.83:6800/397603850,v1:172.21.15.83:6801/397603850] conn(0x7f15b0071110 0x7f15b0073550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.920+0000 7f95a1c00640 1 --2- 172.21.15.83:0/2503404545 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f957c067af0 0x7f957c069fb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9588000f70 tx=0x7f95880079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f3cd0c00640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.86:6800/3882786818,v1:172.21.15.86:6801/3882786818] conn(0x7f3cb4071090 0x7f3cb40734d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f3cd0c00640 1 -- 172.21.15.83:0/505029996 --> [v2:172.21.15.86:6800/3882786818,v1:172.21.15.86:6801/3882786818] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3cb4073ba0 con 0x7f3cb4071090 2024-04-25T00:11:26.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f3cd9600640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.86:6800/3882786818,v1:172.21.15.86:6801/3882786818] conn(0x7f3cb4071090 0x7f3cb40734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f3cd3e00640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cb4067990 0x7f3cb4069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f3cd9600640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.86:6800/3882786818,v1:172.21.15.86:6801/3882786818] conn(0x7f3cb4071090 0x7f3cb40734d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.916+0000 7f3cd3e00640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cb4067990 0x7f3cb4069e50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4001750 tx=0x7f3cc40011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.920+0000 7f3cd0c00640 1 -- 172.21.15.83:0/505029996 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3ccc08d050 con 0x7f3cb8004810 2024-04-25T00:11:26.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.920+0000 7f98dd600640 1 -- 172.21.15.83:0/3222476169 <== osd.5 v2:172.21.15.86:6805/340393403 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98c0073b00 con 0x7f98c0070ff0 2024-04-25T00:11:26.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.920+0000 7f3cd0c00640 1 -- 172.21.15.83:0/505029996 <== osd.6 v2:172.21.15.86:6800/3882786818 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3cb4073ba0 con 0x7f3cb4071090 2024-04-25T00:11:26.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.920+0000 7f15c3400640 1 -- 172.21.15.83:0/1981083757 <== osd.0 v2:172.21.15.83:6800/397603850 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f15c40b87e0 con 0x7f15b0071110 2024-04-25T00:11:26.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.920+0000 7f23a8c00640 1 -- 172.21.15.83:0/896617916 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f23a401d070 con 0x7f2390004810 2024-04-25T00:11:26.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.920+0000 7f23a8c00640 1 --2- 172.21.15.83:0/896617916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f238c0788d0 0x7f238c07ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.920+0000 7f23a8c00640 1 -- 172.21.15.83:0/896617916 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f23a4086700 con 0x7f2390004810 2024-04-25T00:11:26.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.924+0000 7f23a8c00640 1 --2- 172.21.15.83:0/896617916 >> [v2:172.21.15.86:6820/957843207,v1:172.21.15.86:6822/957843207] conn(0x7f238c082050 0x7f238c084490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:26.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.924+0000 7f23a8c00640 1 -- 172.21.15.83:0/896617916 --> [v2:172.21.15.86:6820/957843207,v1:172.21.15.86:6822/957843207] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f238c084b60 con 0x7f238c082050 2024-04-25T00:11:26.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.924+0000 7f23a8c00640 1 -- 172.21.15.83:0/896617916 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f23a408d050 con 0x7f2390004810 2024-04-25T00:11:26.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.924+0000 7f23abe00640 1 --2- 172.21.15.83:0/896617916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f238c0788d0 0x7f238c07ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.924+0000 7f23b1c00640 1 --2- 172.21.15.83:0/896617916 >> [v2:172.21.15.86:6820/957843207,v1:172.21.15.86:6822/957843207] conn(0x7f238c082050 0x7f238c084490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:26.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.924+0000 7f23abe00640 1 --2- 172.21.15.83:0/896617916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f238c0788d0 0x7f238c07ad90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f23980ab000 tx=0x7f2398001980 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.924+0000 7f23b1c00640 1 --2- 172.21.15.83:0/896617916 >> [v2:172.21.15.86:6820/957843207,v1:172.21.15.86:6822/957843207] conn(0x7f238c082050 0x7f238c084490 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:26.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.924+0000 7f23a8c00640 1 -- 172.21.15.83:0/896617916 <== osd.7 v2:172.21.15.86:6820/957843207 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f238c084b60 con 0x7f238c082050 2024-04-25T00:11:26.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.936+0000 7f3cfde00640 1 -- 172.21.15.83:0/1540398834 --> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3cf815dfd0 con 0x7f3cd8071090 2024-04-25T00:11:26.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.940+0000 7f3cef400640 1 -- 172.21.15.83:0/1540398834 <== osd.2 v2:172.21.15.83:6824/1702952090 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3cf815dfd0 con 0x7f3cd8071090 2024-04-25T00:11:26.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.940+0000 7f3cfde00640 1 -- 172.21.15.83:0/1540398834 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x7f3cd8071090 msgr2=0x7f3cd80734d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.940+0000 7f3cfde00640 1 --2- 172.21.15.83:0/1540398834 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x7f3cd8071090 0x7f3cd80734d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.940+0000 7f0c2d000640 1 -- 172.21.15.83:0/1386140072 --> [v2:172.21.15.83:6809/1635741185,v1:172.21.15.83:6811/1635741185] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c282046a0 con 0x7f0c080710c0 2024-04-25T00:11:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.940+0000 7f3cfde00640 1 -- 172.21.15.83:0/1540398834 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cd8067910 msgr2=0x7f3cd8069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.940+0000 7f3cfde00640 1 --2- 172.21.15.83:0/1540398834 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cd8067910 0x7f3cd8069dd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009630 tx=0x7f3ce8009240 comp rx=0 tx=0).stop 2024-04-25T00:11:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.940+0000 7f3cfde00640 1 -- 172.21.15.83:0/1540398834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf806ee20 msgr2=0x7f3cf8140e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.940+0000 7f3cfde00640 1 --2- 172.21.15.83:0/1540398834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf806ee20 0x7f3cf8140e10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000cac0 tx=0x7f3cf000cf80 comp rx=0 tx=0).stop 2024-04-25T00:11:26.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.940+0000 7f3cf7400640 1 -- 172.21.15.83:0/1540398834 reap_dead start 2024-04-25T00:11:26.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.944+0000 7f3cfde00640 1 -- 172.21.15.83:0/1540398834 shutdown_connections 2024-04-25T00:11:26.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.944+0000 7f0c1f400640 1 -- 172.21.15.83:0/1386140072 <== osd.3 v2:172.21.15.83:6809/1635741185 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0c282046a0 con 0x7f0c080710c0 2024-04-25T00:11:26.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.944+0000 7f3cfde00640 1 -- 172.21.15.83:0/1540398834 >> 172.21.15.83:0/1540398834 conn(0x7f3cf80cfa60 msgr2=0x7f3cf81652f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.944+0000 7f3cfde00640 1 -- 172.21.15.83:0/1540398834 shutdown_connections 2024-04-25T00:11:26.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.944+0000 7f3cfde00640 1 -- 172.21.15.83:0/1540398834 wait complete. 2024-04-25T00:11:26.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f0c2d000640 1 -- 172.21.15.83:0/1386140072 >> [v2:172.21.15.83:6809/1635741185,v1:172.21.15.83:6811/1635741185] conn(0x7f0c080710c0 msgr2=0x7f0c08073500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f0c2d000640 1 --2- 172.21.15.83:0/1386140072 >> [v2:172.21.15.83:6809/1635741185,v1:172.21.15.83:6811/1635741185] conn(0x7f0c080710c0 0x7f0c08073500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f0c2d000640 1 -- 172.21.15.83:0/1386140072 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0c08067940 msgr2=0x7f0c08069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f0c2d000640 1 --2- 172.21.15.83:0/1386140072 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0c08067940 0x7f0c08069e00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0c1801cd70 tx=0x7f0c180a7000 comp rx=0 tx=0).stop 2024-04-25T00:11:26.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f0c2d000640 1 -- 172.21.15.83:0/1386140072 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c280b3ec0 msgr2=0x7f0c280a9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f0c2d000640 1 --2- 172.21.15.83:0/1386140072 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c280b3ec0 0x7f0c280a9dd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400c410 tx=0x7f0c1400c8d0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f0c27e00640 1 -- 172.21.15.83:0/1386140072 reap_dead start 2024-04-25T00:11:26.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f0c2d000640 1 -- 172.21.15.83:0/1386140072 shutdown_connections 2024-04-25T00:11:26.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f0c2d000640 1 -- 172.21.15.83:0/1386140072 >> 172.21.15.83:0/1386140072 conn(0x7f0c280b1650 msgr2=0x7f0c28160770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f95a3000640 1 -- 172.21.15.83:0/2503404545 --> [v2:172.21.15.83:6808/228430303,v1:172.21.15.83:6810/228430303] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9578002de0 con 0x7f957c0711f0 2024-04-25T00:11:26.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f384a000640 1 -- 172.21.15.83:0/3962973764 --> [v2:172.21.15.86:6816/1560646044,v1:172.21.15.86:6817/1560646044] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f383c0052d0 con 0x7f3824071080 2024-04-25T00:11:26.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f0c2d000640 1 -- 172.21.15.83:0/1386140072 shutdown_connections 2024-04-25T00:11:26.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f15d5200640 1 -- 172.21.15.83:0/1981083757 --> [v2:172.21.15.83:6800/397603850,v1:172.21.15.83:6801/397603850] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f15ac002de0 con 0x7f15b0071110 2024-04-25T00:11:26.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f0c2d000640 1 -- 172.21.15.83:0/1386140072 wait complete. 2024-04-25T00:11:26.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f3840c00640 1 -- 172.21.15.83:0/3962973764 <== osd.4 v2:172.21.15.86:6816/1560646044 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f383c0052d0 con 0x7f3824071080 2024-04-25T00:11:26.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f9599600640 1 -- 172.21.15.83:0/2503404545 <== osd.1 v2:172.21.15.83:6808/228430303 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9578002de0 con 0x7f957c0711f0 2024-04-25T00:11:26.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f3836000640 1 -- 172.21.15.83:0/3962973764 >> [v2:172.21.15.86:6816/1560646044,v1:172.21.15.86:6817/1560646044] conn(0x7f3824071080 msgr2=0x7f38240734c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f3836000640 1 --2- 172.21.15.83:0/3962973764 >> [v2:172.21.15.86:6816/1560646044,v1:172.21.15.86:6817/1560646044] conn(0x7f3824071080 0x7f38240734c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f3836000640 1 -- 172.21.15.83:0/3962973764 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3824067910 msgr2=0x7f3824069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f3cdb600640 1 -- 172.21.15.83:0/505029996 --> [v2:172.21.15.86:6800/3882786818,v1:172.21.15.86:6801/3882786818] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3cd415abd0 con 0x7f3cb4071090 2024-04-25T00:11:26.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f95a3000640 1 -- 172.21.15.83:0/2503404545 >> [v2:172.21.15.83:6808/228430303,v1:172.21.15.83:6810/228430303] conn(0x7f957c0711f0 msgr2=0x7f957c073630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f95a3000640 1 --2- 172.21.15.83:0/2503404545 >> [v2:172.21.15.83:6808/228430303,v1:172.21.15.83:6810/228430303] conn(0x7f957c0711f0 0x7f957c073630 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f95a3000640 1 -- 172.21.15.83:0/2503404545 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f957c067af0 msgr2=0x7f957c069fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f95a3000640 1 --2- 172.21.15.83:0/2503404545 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f957c067af0 0x7f957c069fb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9588000f70 tx=0x7f95880079c0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f98d2a00640 1 -- 172.21.15.83:0/3222476169 --> [v2:172.21.15.86:6805/340393403,v1:172.21.15.86:6807/340393403] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f98e00cf290 con 0x7f98c0070ff0 2024-04-25T00:11:26.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f95a3000640 1 -- 172.21.15.83:0/2503404545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9594047c20 msgr2=0x7f95940e49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f3836000640 1 --2- 172.21.15.83:0/3962973764 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3824067910 0x7f3824069dd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f383800b850 tx=0x7f3838008040 comp rx=0 tx=0).stop 2024-04-25T00:11:26.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f3836000640 1 -- 172.21.15.83:0/3962973764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f383c052e30 msgr2=0x7f383c0f1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f3836000640 1 --2- 172.21.15.83:0/3962973764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f383c052e30 0x7f383c0f1740 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f383001cce0 tx=0x7f38300191e0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.948+0000 7f3849600640 1 -- 172.21.15.83:0/3962973764 reap_dead start 2024-04-25T00:11:26.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f23b3c00640 1 -- 172.21.15.83:0/896617916 --> [v2:172.21.15.86:6820/957843207,v1:172.21.15.86:6822/957843207] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2384002de0 con 0x7f238c082050 2024-04-25T00:11:26.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3836000640 1 -- 172.21.15.83:0/3962973764 shutdown_connections 2024-04-25T00:11:26.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3836000640 1 -- 172.21.15.83:0/3962973764 >> 172.21.15.83:0/3962973764 conn(0x7f383c009fc0 msgr2=0x7f383c0f6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3cd0c00640 1 -- 172.21.15.83:0/505029996 <== osd.6 v2:172.21.15.86:6800/3882786818 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3cd415abd0 con 0x7f3cb4071090 2024-04-25T00:11:26.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f95a3000640 1 --2- 172.21.15.83:0/2503404545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9594047c20 0x7f95940e49c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f959c064d10 tx=0x7f959c097130 comp rx=0 tx=0).stop 2024-04-25T00:11:26.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f95a2600640 1 -- 172.21.15.83:0/2503404545 reap_dead start 2024-04-25T00:11:26.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f98dd600640 1 -- 172.21.15.83:0/3222476169 <== osd.5 v2:172.21.15.86:6805/340393403 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f98e00cf290 con 0x7f98c0070ff0 2024-04-25T00:11:26.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f95a3000640 1 -- 172.21.15.83:0/2503404545 shutdown_connections 2024-04-25T00:11:26.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f95a3000640 1 -- 172.21.15.83:0/2503404545 >> 172.21.15.83:0/2503404545 conn(0x7f9594009fc0 msgr2=0x7f9594039e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f23a8c00640 1 -- 172.21.15.83:0/896617916 <== osd.7 v2:172.21.15.86:6820/957843207 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2384002de0 con 0x7f238c082050 2024-04-25T00:11:26.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3cdb600640 1 -- 172.21.15.83:0/505029996 >> [v2:172.21.15.86:6800/3882786818,v1:172.21.15.86:6801/3882786818] conn(0x7f3cb4071090 msgr2=0x7f3cb40734d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3cdb600640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.86:6800/3882786818,v1:172.21.15.86:6801/3882786818] conn(0x7f3cb4071090 0x7f3cb40734d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3836000640 1 -- 172.21.15.83:0/3962973764 shutdown_connections 2024-04-25T00:11:26.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3cdb600640 1 -- 172.21.15.83:0/505029996 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cb4067990 msgr2=0x7f3cb4069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3cdb600640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3cb4067990 0x7f3cb4069e50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4001750 tx=0x7f3cc40011a0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3836000640 1 -- 172.21.15.83:0/3962973764 wait complete. 2024-04-25T00:11:26.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3cdb600640 1 -- 172.21.15.83:0/505029996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb8004810 msgr2=0x7f3cd40b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3cdb600640 1 --2- 172.21.15.83:0/505029996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb8004810 0x7f3cd40b46b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc007ed0 tx=0x7f3ccc00d4a0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f95a3000640 1 -- 172.21.15.83:0/2503404545 shutdown_connections 2024-04-25T00:11:26.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f95a3000640 1 -- 172.21.15.83:0/2503404545 wait complete. 2024-04-25T00:11:26.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3cd9600640 1 -- 172.21.15.83:0/505029996 reap_dead start 2024-04-25T00:11:26.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3cdb600640 1 -- 172.21.15.83:0/505029996 shutdown_connections 2024-04-25T00:11:26.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3cdb600640 1 -- 172.21.15.83:0/505029996 >> 172.21.15.83:0/505029996 conn(0x7f3cd40cfa60 msgr2=0x7f3cd40cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f23b3c00640 1 -- 172.21.15.83:0/896617916 >> [v2:172.21.15.86:6820/957843207,v1:172.21.15.86:6822/957843207] conn(0x7f238c082050 msgr2=0x7f238c084490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f23b3c00640 1 --2- 172.21.15.83:0/896617916 >> [v2:172.21.15.86:6820/957843207,v1:172.21.15.86:6822/957843207] conn(0x7f238c082050 0x7f238c084490 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f15c3400640 1 -- 172.21.15.83:0/1981083757 <== osd.0 v2:172.21.15.83:6800/397603850 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f15ac002de0 con 0x7f15b0071110 2024-04-25T00:11:26.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f98d2a00640 1 -- 172.21.15.83:0/3222476169 >> [v2:172.21.15.86:6805/340393403,v1:172.21.15.86:6807/340393403] conn(0x7f98c0070ff0 msgr2=0x7f98c0073430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f98d2a00640 1 --2- 172.21.15.83:0/3222476169 >> [v2:172.21.15.86:6805/340393403,v1:172.21.15.86:6807/340393403] conn(0x7f98c0070ff0 0x7f98c0073430 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3cdb600640 1 -- 172.21.15.83:0/505029996 shutdown_connections 2024-04-25T00:11:26.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f3cdb600640 1 -- 172.21.15.83:0/505029996 wait complete. 2024-04-25T00:11:26.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f23b3c00640 1 -- 172.21.15.83:0/896617916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f238c0788d0 msgr2=0x7f238c07ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f23b3c00640 1 --2- 172.21.15.83:0/896617916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f238c0788d0 0x7f238c07ad90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f23980ab000 tx=0x7f2398001980 comp rx=0 tx=0).stop 2024-04-25T00:11:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f23b3c00640 1 -- 172.21.15.83:0/896617916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2390004810 msgr2=0x7f23ac14ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f23b3c00640 1 --2- 172.21.15.83:0/896617916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2390004810 0x7f23ac14ca40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f23a400ced0 tx=0x7f23a400a710 comp rx=0 tx=0).stop 2024-04-25T00:11:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f98d2a00640 1 -- 172.21.15.83:0/3222476169 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98c00678f0 msgr2=0x7f98c0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f98d2a00640 1 --2- 172.21.15.83:0/3222476169 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98c00678f0 0x7f98c0069db0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f98d4003cf0 tx=0x7f98d40079c0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f98d2a00640 1 -- 172.21.15.83:0/3222476169 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e0162a20 msgr2=0x7f98e0161b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f98d2a00640 1 --2- 172.21.15.83:0/3222476169 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e0162a20 0x7f98e0161b90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0ae040 tx=0x7f98cc01ac40 comp rx=0 tx=0).stop 2024-04-25T00:11:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f98e6000640 1 -- 172.21.15.83:0/3222476169 reap_dead start 2024-04-25T00:11:26.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f23b1c00640 1 -- 172.21.15.83:0/896617916 reap_dead start 2024-04-25T00:11:26.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f23b3c00640 1 -- 172.21.15.83:0/896617916 shutdown_connections 2024-04-25T00:11:26.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f23b3c00640 1 -- 172.21.15.83:0/896617916 >> 172.21.15.83:0/896617916 conn(0x7f23ac0d1c80 msgr2=0x7f23ac0b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f98d2a00640 1 -- 172.21.15.83:0/3222476169 shutdown_connections 2024-04-25T00:11:26.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f98d2a00640 1 -- 172.21.15.83:0/3222476169 >> 172.21.15.83:0/3222476169 conn(0x7f98e00cfa60 msgr2=0x7f98e0070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f23b3c00640 1 -- 172.21.15.83:0/896617916 shutdown_connections 2024-04-25T00:11:26.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f23b3c00640 1 -- 172.21.15.83:0/896617916 wait complete. 2024-04-25T00:11:26.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f15d5200640 1 -- 172.21.15.83:0/1981083757 >> [v2:172.21.15.83:6800/397603850,v1:172.21.15.83:6801/397603850] conn(0x7f15b0071110 msgr2=0x7f15b0073550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f15d5200640 1 --2- 172.21.15.83:0/1981083757 >> [v2:172.21.15.83:6800/397603850,v1:172.21.15.83:6801/397603850] conn(0x7f15b0071110 0x7f15b0073550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f98d2a00640 1 -- 172.21.15.83:0/3222476169 shutdown_connections 2024-04-25T00:11:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f15d5200640 1 -- 172.21.15.83:0/1981083757 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f15b0067990 msgr2=0x7f15b0069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f15d5200640 1 --2- 172.21.15.83:0/1981083757 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f15b0067990 0x7f15b0069e50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f15c80cffa0 tx=0x7f15bc007960 comp rx=0 tx=0).stop 2024-04-25T00:11:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f15d5200640 1 -- 172.21.15.83:0/1981083757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15c80362d0 msgr2=0x7f15c80cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f15d5200640 1 --2- 172.21.15.83:0/1981083757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15c80362d0 0x7f15c80cb700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f15c4018f10 tx=0x7f15c40190a0 comp rx=0 tx=0).stop 2024-04-25T00:11:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f15cfe00640 1 -- 172.21.15.83:0/1981083757 reap_dead start 2024-04-25T00:11:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f98d2a00640 1 -- 172.21.15.83:0/3222476169 wait complete. 2024-04-25T00:11:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f15d5200640 1 -- 172.21.15.83:0/1981083757 shutdown_connections 2024-04-25T00:11:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f15d5200640 1 -- 172.21.15.83:0/1981083757 >> 172.21.15.83:0/1981083757 conn(0x7f15c8053760 msgr2=0x7f15c80d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f15d5200640 1 -- 172.21.15.83:0/1981083757 shutdown_connections 2024-04-25T00:11:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:26.952+0000 7f15d5200640 1 -- 172.21.15.83:0/1981083757 wait complete. 2024-04-25T00:11:26.996 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2024-04-25T00:11:26.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T00:11:26.998 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2024-04-25T00:11:26.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T00:11:27.002 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2024-04-25T00:11:27.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T00:11:27.004 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2024-04-25T00:11:27.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T00:11:27.009 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2024-04-25T00:11:27.009 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2024-04-25T00:11:27.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T00:11:27.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T00:11:27.011 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2024-04-25T00:11:27.011 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T00:11:27.012 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2024-04-25T00:11:27.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T00:11:27.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.236+0000 7f6a01000640 1 Processor -- start 2024-04-25T00:11:27.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.236+0000 7f6a01000640 1 -- start start 2024-04-25T00:11:27.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.236+0000 7f140b200640 1 Processor -- start 2024-04-25T00:11:27.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7fc83b000640 1 Processor -- start 2024-04-25T00:11:27.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.236+0000 7f140b200640 1 -- start start 2024-04-25T00:11:27.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f6a01000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69fc0d1350 0x7f69fc06f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f6a01000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f69fc0d1730 0x7f69fc06fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f6a01000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69fc0ccc00 con 0x7f69fc0ce5b0 2024-04-25T00:11:27.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f6a01000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f69fc0cdca0 con 0x7f69fc0d1350 2024-04-25T00:11:27.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f6a01000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f69fc0ccee0 con 0x7f69fc0d1730 2024-04-25T00:11:27.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7fc83b000640 1 -- start start 2024-04-25T00:11:27.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f140b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f13fc0df450 0x7f13fc054120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f140b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13fc0073a0 0x7f13fc04fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f69fb400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69fc0d1350 0x7f69fc06f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f140b200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13fc006300 con 0x7f13fc0500a0 2024-04-25T00:11:27.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f140b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f13fc006f10 con 0x7f13fc0df450 2024-04-25T00:11:27.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f140b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f13fc0065e0 con 0x7f13fc0073a0 2024-04-25T00:11:27.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f69faa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f69fc0d1730 0x7f69fc06fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f69faa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f69fc0d1730 0x7f69fc06fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:39024/0 (socket says 172.21.15.83:39024) 2024-04-25T00:11:27.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f69faa00640 1 -- 172.21.15.83:0/1318595032 learned_addr learned my addr 172.21.15.83:0/1318595032 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:27.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f1409200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f13fc0500a0 0x7f13fc0380c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52180/0 (socket says 172.21.15.83:52180) 2024-04-25T00:11:27.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f1409200640 1 -- 172.21.15.83:0/446921551 learned_addr learned my addr 172.21.15.83:0/446921551 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f69fb400640 1 -- 172.21.15.83:0/1318595032 >> v2:172.21.15.86:3300/0 conn(0x7f69fc0d1730 msgr2=0x7f69fc06fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f69fb400640 1 --2- 172.21.15.83:0/1318595032 >> v2:172.21.15.86:3300/0 conn(0x7f69fc0d1730 0x7f69fc06fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f69fb400640 1 -- 172.21.15.83:0/1318595032 >> v1:172.21.15.86:6789/0 conn(0x7f69fc0ce5b0 legacy=0x7f69fc0ce990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:11:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f69fb400640 1 -- 172.21.15.83:0/1318595032 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc0cd1c0 con 0x7f69fc0d1350 2024-04-25T00:11:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7fb89a000640 1 Processor -- start 2024-04-25T00:11:27.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f1403e00640 1 --2- 172.21.15.83:0/446921551 >> v2:172.21.15.86:3300/0 conn(0x7f13fc0df450 0x7f13fc054120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f1403400640 1 --2- 172.21.15.83:0/446921551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13fc0073a0 0x7f13fc04fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7fc83b000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc82c052050 0x7fc82c052430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7fc839c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc82c052050 0x7fc82c052430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7fc839c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc82c052050 0x7fc82c052430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:39040/0 (socket says 172.21.15.83:39040) 2024-04-25T00:11:27.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7fb89a000640 1 -- start start 2024-04-25T00:11:27.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f1402a00640 1 -- 172.21.15.83:0/446921551 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678757812 0 0) 0x7f13fc006300 con 0x7f13fc0500a0 2024-04-25T00:11:27.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7f1402a00640 1 -- 172.21.15.83:0/446921551 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13ec003530 con 0x7f13fc0500a0 2024-04-25T00:11:27.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7fc83b000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc82c051a80 0x7fc82c054fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7fc83b000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc82c052970 con 0x7fc82c057800 2024-04-25T00:11:27.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7fc83b000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc82c0535e0 con 0x7fc82c052050 2024-04-25T00:11:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7fc83b000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc82c051810 con 0x7fc82c051a80 2024-04-25T00:11:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7fc83a600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc82c051a80 0x7fc82c054fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.240+0000 7fc839200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc82c057800 0x7fc82c0548b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:46868/0 (socket says 172.21.15.83:46868) 2024-04-25T00:11:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7fc839200640 1 -- 172.21.15.83:0/351758555 learned_addr learned my addr 172.21.15.83:0/351758555 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7fc83a600640 1 -- 172.21.15.83:0/351758555 >> v2:172.21.15.86:3300/0 conn(0x7fc82c052050 msgr2=0x7fc82c052430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7f1403400640 1 -- 172.21.15.83:0/446921551 >> v2:172.21.15.86:3300/0 conn(0x7f13fc0df450 msgr2=0x7f13fc054120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7f1402a00640 1 -- 172.21.15.83:0/446921551 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2239412317 0 0) 0x7f13ec003530 con 0x7f13fc0500a0 2024-04-25T00:11:27.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7f1403400640 1 --2- 172.21.15.83:0/446921551 >> v2:172.21.15.86:3300/0 conn(0x7f13fc0df450 0x7f13fc054120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7f1403400640 1 -- 172.21.15.83:0/446921551 >> v1:172.21.15.83:6789/0 conn(0x7f13fc0500a0 legacy=0x7f13fc0380c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7f1403400640 1 -- 172.21.15.83:0/446921551 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc0068c0 con 0x7f13fc0073a0 2024-04-25T00:11:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7fb89a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb88c035fd0 0x7fb88c0557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7fb892a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb88c0548e0 0x7fb88c038790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52192/0 (socket says 172.21.15.83:52192) 2024-04-25T00:11:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7fb893400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb88c035fd0 0x7fb88c0557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7fb893400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb88c035fd0 0x7fb88c0557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:39046/0 (socket says 172.21.15.83:39046) 2024-04-25T00:11:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7fb892a00640 1 -- 172.21.15.83:0/1403851144 learned_addr learned my addr 172.21.15.83:0/1403851144 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7fb89a000640 1 --2- 172.21.15.83:0/1403851144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb88c038ea0 0x7fb88c0e0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7f69fb400640 1 --2- 172.21.15.83:0/1318595032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69fc0d1350 0x7f69fc06f8a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0d2cc0 tx=0x7f69ec01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518350d8c255c3f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7fc83a600640 1 --2- 172.21.15.83:0/351758555 >> v2:172.21.15.86:3300/0 conn(0x7fc82c052050 0x7fc82c052430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7fc83a600640 1 -- 172.21.15.83:0/351758555 >> v1:172.21.15.86:6789/0 conn(0x7fc82c057800 legacy=0x7fc82c0548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7fc83a600640 1 -- 172.21.15.83:0/351758555 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c051530 con 0x7fc82c051a80 2024-04-25T00:11:27.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7fb89a000640 1 -- 172.21.15.83:0/1403851144 --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb88c00ad50 con 0x7fb88c0548e0 2024-04-25T00:11:27.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7fb89a000640 1 -- 172.21.15.83:0/1403851144 --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb88c03a100 con 0x7fb88c035fd0 2024-04-25T00:11:27.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7fb89a000640 1 -- 172.21.15.83:0/1403851144 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb88c056050 con 0x7fb88c038ea0 2024-04-25T00:11:27.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.244+0000 7f1403e00640 1 --2- 172.21.15.83:0/446921551 >> v2:172.21.15.86:3300/0 conn(0x7f13fc0df450 0x7f13fc054120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:27.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f69fa000640 1 -- 172.21.15.83:0/1318595032 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec0ab610 con 0x7f69fc0d1350 2024-04-25T00:11:27.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7fb893e00640 1 --2- 172.21.15.83:0/1403851144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb88c038ea0 0x7fb88c0e0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7fb892000640 1 -- 172.21.15.83:0/1403851144 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1183329922 0 0) 0x7fb88c00ad50 con 0x7fb88c0548e0 2024-04-25T00:11:27.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7fb892000640 1 -- 172.21.15.83:0/1403851144 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb87c003530 con 0x7fb88c0548e0 2024-04-25T00:11:27.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7fb893400640 1 -- 172.21.15.83:0/1403851144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb88c038ea0 msgr2=0x7fb88c0e0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:27.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7fb893400640 1 --2- 172.21.15.83:0/1403851144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb88c038ea0 0x7fb88c0e0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f69fa000640 1 -- 172.21.15.83:0/1318595032 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69ec0b0070 con 0x7f69fc0d1350 2024-04-25T00:11:27.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f69fa000640 1 -- 172.21.15.83:0/1318595032 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec0ab610 con 0x7f69fc0d1350 2024-04-25T00:11:27.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7fb893400640 1 -- 172.21.15.83:0/1403851144 >> v1:172.21.15.83:6789/0 conn(0x7fb88c0548e0 legacy=0x7fb88c038790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7fb893400640 1 -- 172.21.15.83:0/1403851144 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88c056330 con 0x7fb88c035fd0 2024-04-25T00:11:27.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f1403400640 1 --2- 172.21.15.83:0/446921551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13fc0073a0 0x7f13fc04fb60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f13f8008260 tx=0x7f13f8020750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2889449ff5cc6321 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7fc83a600640 1 --2- 172.21.15.83:0/351758555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc82c051a80 0x7fc82c054fc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc834064d10 tx=0x7fc834097790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8896f432ba9ac5bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f1402a00640 1 -- 172.21.15.83:0/446921551 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0001680 con 0x7f13fc0073a0 2024-04-25T00:11:27.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f1402a00640 1 -- 172.21.15.83:0/446921551 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f80b3070 con 0x7f13fc0073a0 2024-04-25T00:11:27.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7fc833e00640 1 -- 172.21.15.83:0/351758555 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834123720 con 0x7fc82c051a80 2024-04-25T00:11:27.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7fc833e00640 1 -- 172.21.15.83:0/351758555 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8341238c0 con 0x7fc82c051a80 2024-04-25T00:11:27.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7fc833e00640 1 -- 172.21.15.83:0/351758555 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834123cf0 con 0x7fc82c051a80 2024-04-25T00:11:27.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f1402a00640 1 -- 172.21.15.83:0/446921551 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0001680 con 0x7f13fc0073a0 2024-04-25T00:11:27.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f6a01000640 1 -- 172.21.15.83:0/1318595032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69fc0d1350 msgr2=0x7f69fc06f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f6a01000640 1 --2- 172.21.15.83:0/1318595032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69fc0d1350 0x7f69fc06f8a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0d2cc0 tx=0x7f69ec01c410 comp rx=0 tx=0).stop 2024-04-25T00:11:27.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f140b200640 1 -- 172.21.15.83:0/446921551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13fc0073a0 msgr2=0x7f13fc04fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f140b200640 1 --2- 172.21.15.83:0/446921551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13fc0073a0 0x7f13fc04fb60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f13f8008260 tx=0x7f13f8020750 comp rx=0 tx=0).stop 2024-04-25T00:11:27.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f6a01000640 1 -- 172.21.15.83:0/1318595032 shutdown_connections 2024-04-25T00:11:27.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f6a01000640 1 --2- 172.21.15.83:0/1318595032 >> v2:172.21.15.86:3300/0 conn(0x7f69fc0d1730 0x7f69fc06fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f6a01000640 1 --2- 172.21.15.83:0/1318595032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69fc0d1350 0x7f69fc06f8a0 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0d2cc0 tx=0x7f69ec01c410 comp rx=0 tx=0).stop 2024-04-25T00:11:27.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f6a01000640 1 -- 172.21.15.83:0/1318595032 >> 172.21.15.83:0/1318595032 conn(0x7f69fc0d1be0 msgr2=0x7f69fc0d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f140b200640 1 -- 172.21.15.83:0/446921551 shutdown_connections 2024-04-25T00:11:27.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f140b200640 1 --2- 172.21.15.83:0/446921551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13fc0073a0 0x7f13fc04fb60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f140b200640 1 --2- 172.21.15.83:0/446921551 >> v2:172.21.15.86:3300/0 conn(0x7f13fc0df450 0x7f13fc054120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.248+0000 7f140b200640 1 -- 172.21.15.83:0/446921551 >> 172.21.15.83:0/446921551 conn(0x7f13fc054b00 msgr2=0x7f13fc054ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7fc83b000640 1 -- 172.21.15.83:0/351758555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc82c051a80 msgr2=0x7fc82c054fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7fc83b000640 1 --2- 172.21.15.83:0/351758555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc82c051a80 0x7fc82c054fc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc834064d10 tx=0x7fc834097790 comp rx=0 tx=0).stop 2024-04-25T00:11:27.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7f140b200640 1 -- 172.21.15.83:0/446921551 shutdown_connections 2024-04-25T00:11:27.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7f140b200640 1 -- 172.21.15.83:0/446921551 wait complete. 2024-04-25T00:11:27.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7fc83b000640 1 -- 172.21.15.83:0/351758555 shutdown_connections 2024-04-25T00:11:27.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7fc83b000640 1 --2- 172.21.15.83:0/351758555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc82c051a80 0x7fc82c054fc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7fc83b000640 1 --2- 172.21.15.83:0/351758555 >> v2:172.21.15.86:3300/0 conn(0x7fc82c052050 0x7fc82c052430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7fc83b000640 1 -- 172.21.15.83:0/351758555 >> 172.21.15.83:0/351758555 conn(0x7fc82c056ce0 msgr2=0x7fc82c0570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7f80ce800640 1 Processor -- start 2024-04-25T00:11:27.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7f80ce800640 1 -- start start 2024-04-25T00:11:27.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7fb893400640 1 --2- 172.21.15.83:0/1403851144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c035fd0 0x7fb88c0557d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb88800dbe0 tx=0x7fb888020720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43dd7f9476512aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7fc83b000640 1 -- 172.21.15.83:0/351758555 shutdown_connections 2024-04-25T00:11:27.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7fc83b000640 1 -- 172.21.15.83:0/351758555 wait complete. 2024-04-25T00:11:27.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7f6a01000640 1 -- 172.21.15.83:0/1318595032 shutdown_connections 2024-04-25T00:11:27.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7f6a01000640 1 -- 172.21.15.83:0/1318595032 wait complete. 2024-04-25T00:11:27.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7fb892000640 1 -- 172.21.15.83:0/1403851144 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8880b0070 con 0x7fb88c035fd0 2024-04-25T00:11:27.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7fb892000640 1 -- 172.21.15.83:0/1403851144 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb88800d0d0 con 0x7fb88c035fd0 2024-04-25T00:11:27.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7fb892000640 1 -- 172.21.15.83:0/1403851144 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8880ac680 con 0x7fb88c035fd0 2024-04-25T00:11:27.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7f80ce800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f80c00398f0 0x7f80c003a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7f140b200640 1 Processor -- start 2024-04-25T00:11:27.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7f80ce800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80c0039cd0 con 0x7f80c00e00b0 2024-04-25T00:11:27.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7f80ce800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80c0039f60 con 0x7f80c0050dd0 2024-04-25T00:11:27.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7f80ce800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f80c003ac50 con 0x7f80c00398f0 2024-04-25T00:11:27.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7fb89a000640 1 -- 172.21.15.83:0/1403851144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c035fd0 msgr2=0x7fb88c0557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.252+0000 7fb89a000640 1 --2- 172.21.15.83:0/1403851144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c035fd0 0x7fb88c0557d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb88800dbe0 tx=0x7fb888020720 comp rx=0 tx=0).stop 2024-04-25T00:11:27.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fb89a000640 1 -- 172.21.15.83:0/1403851144 shutdown_connections 2024-04-25T00:11:27.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fb89a000640 1 --2- 172.21.15.83:0/1403851144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb88c038ea0 0x7fb88c0e0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fb89a000640 1 --2- 172.21.15.83:0/1403851144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c035fd0 0x7fb88c0557d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fb89a000640 1 -- 172.21.15.83:0/1403851144 >> 172.21.15.83:0/1403851144 conn(0x7fb88c053790 msgr2=0x7fb88c053ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f80cfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f80c0050dd0 0x7f80c0006950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:46874/0 (socket says 172.21.15.83:46874) 2024-04-25T00:11:27.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f80cfe00640 1 -- 172.21.15.83:0/1053245497 learned_addr learned my addr 172.21.15.83:0/1053245497 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:27.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fb89a000640 1 -- 172.21.15.83:0/1403851144 shutdown_connections 2024-04-25T00:11:27.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fb89a000640 1 -- 172.21.15.83:0/1403851144 wait complete. 2024-04-25T00:11:27.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f6a01000640 1 Processor -- start 2024-04-25T00:11:27.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f80cd400640 1 --2- 172.21.15.83:0/1053245497 >> v2:172.21.15.86:3300/0 conn(0x7f80c00398f0 0x7f80c003a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fc83b000640 1 Processor -- start 2024-04-25T00:11:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f80c7e00640 1 -- 172.21.15.83:0/1053245497 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1157512440 0 0) 0x7f80c0039f60 con 0x7f80c0050dd0 2024-04-25T00:11:27.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f140b200640 1 -- start start 2024-04-25T00:11:27.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f6a01000640 1 -- start start 2024-04-25T00:11:27.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f80c7e00640 1 -- 172.21.15.83:0/1053245497 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80b0003530 con 0x7f80c0050dd0 2024-04-25T00:11:27.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fc83b000640 1 -- start start 2024-04-25T00:11:27.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f80cd400640 1 -- 172.21.15.83:0/1053245497 >> v1:172.21.15.86:6789/0 conn(0x7f80c0050dd0 legacy=0x7f80c0006950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f80cd400640 1 -- 172.21.15.83:0/1053245497 >> v1:172.21.15.83:6789/0 conn(0x7f80c00e00b0 legacy=0x7f80c00506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f80cd400640 1 -- 172.21.15.83:0/1053245497 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c0052a10 con 0x7f80c00398f0 2024-04-25T00:11:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fc83b000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82c051a80 0x7fc82c0e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fc83b000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc82c052050 0x7fc82c0e1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fc83b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc82c057800 0x7fc82c04fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f6a01000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69fc0ce5b0 0x7f69fc1ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fc83b000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c0eaf50 con 0x7fc82c051a80 2024-04-25T00:11:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fc83b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c0eadd0 con 0x7fc82c057800 2024-04-25T00:11:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fc83b000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc82c04a490 con 0x7fc82c052050 2024-04-25T00:11:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f6a01000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69fc0d1730 0x7f69fc06f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fb89a000640 1 Processor -- start 2024-04-25T00:11:27.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fc839200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc82c052050 0x7fc82c0e1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f69faa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69fc0d1730 0x7f69fc06f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f69faa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69fc0d1730 0x7f69fc06f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36432/0 (socket says 172.21.15.83:36432) 2024-04-25T00:11:27.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7f69faa00640 1 -- 172.21.15.83:0/3098984112 learned_addr learned my addr 172.21.15.83:0/3098984112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:27.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fb89a000640 1 -- start start 2024-04-25T00:11:27.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fc839200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc82c052050 0x7fc82c0e1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:49894/0 (socket says 172.21.15.83:49894) 2024-04-25T00:11:27.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f6a01000640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69fc150bd0 0x7f69fc14ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fc839200640 1 -- 172.21.15.83:0/3285712255 learned_addr learned my addr 172.21.15.83:0/3285712255 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:27.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f6a01000640 1 -- 172.21.15.83:0/3098984112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc15d1d0 con 0x7f69fc0d1730 2024-04-25T00:11:27.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f6a01000640 1 -- 172.21.15.83:0/3098984112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc15d050 con 0x7f69fc150bd0 2024-04-25T00:11:27.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f6a01000640 1 -- 172.21.15.83:0/3098984112 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f69fc15d350 con 0x7f69fc0ce5b0 2024-04-25T00:11:27.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.256+0000 7fc83a600640 1 --2- 172.21.15.83:0/3285712255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc82c057800 0x7fc82c04fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fc839200640 1 -- 172.21.15.83:0/3285712255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc82c057800 msgr2=0x7fc82c04fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fc839200640 1 --2- 172.21.15.83:0/3285712255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc82c057800 0x7fc82c04fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fc839200640 1 -- 172.21.15.83:0/3285712255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82c051a80 msgr2=0x7fc82c0e4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fc839200640 1 --2- 172.21.15.83:0/3285712255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82c051a80 0x7fc82c0e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fc839200640 1 -- 172.21.15.83:0/3285712255 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c04eea0 con 0x7fc82c052050 2024-04-25T00:11:27.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f69fb400640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69fc0ce5b0 0x7f69fc1ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f69fbe00640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69fc150bd0 0x7f69fc14ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f140b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13fc0073a0 0x7f13fc0cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f140b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13fc0500a0 0x7f13fc04bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f140b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13fc0df450 0x7f13fc048a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f140b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc053500 con 0x7f13fc0073a0 2024-04-25T00:11:27.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f140b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc053380 con 0x7f13fc0df450 2024-04-25T00:11:27.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f140b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f13fc038be0 con 0x7f13fc0500a0 2024-04-25T00:11:27.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f69faa00640 1 -- 172.21.15.83:0/3098984112 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69fc0ce5b0 msgr2=0x7f69fc1ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fc839200640 1 --2- 172.21.15.83:0/3285712255 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc82c052050 0x7fc82c0e1190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c800 tx=0x7fc82000ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f1409200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13fc0df450 0x7f13fc048a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f1409200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13fc0df450 0x7f13fc048a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:39098/0 (socket says 172.21.15.83:39098) 2024-04-25T00:11:27.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f1409200640 1 -- 172.21.15.83:0/4258901930 learned_addr learned my addr 172.21.15.83:0/4258901930 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:27.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f1403400640 1 --2- 172.21.15.83:0/4258901930 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13fc0500a0 0x7f13fc04bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb89a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb88c035fd0 0x7fb88c1808b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb89a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb88c038ea0 0x7fb88c1811e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb89a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c0548e0 0x7fb88c04ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb89a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c03a5e0 con 0x7fb88c035fd0 2024-04-25T00:11:27.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb89a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c055d10 con 0x7fb88c0548e0 2024-04-25T00:11:27.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb89a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb88c03a760 con 0x7fb88c038ea0 2024-04-25T00:11:27.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f69faa00640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69fc0ce5b0 0x7f69fc1ff270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f69faa00640 1 -- 172.21.15.83:0/3098984112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69fc150bd0 msgr2=0x7f69fc14ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f69faa00640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69fc150bd0 0x7f69fc14ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f69faa00640 1 -- 172.21.15.83:0/3098984112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc0c5650 con 0x7f69fc0d1730 2024-04-25T00:11:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f69fb400640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69fc0ce5b0 0x7f69fc1ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f69fbe00640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69fc150bd0 0x7f69fc14ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb893e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c0548e0 0x7fb88c04ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb893e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c0548e0 0x7fb88c04ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:39102/0 (socket says 172.21.15.83:39102) 2024-04-25T00:11:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb893e00640 1 -- 172.21.15.83:0/3670414679 learned_addr learned my addr 172.21.15.83:0/3670414679 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:27.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fc831600640 1 -- 172.21.15.83:0/3285712255 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820018070 con 0x7fc82c052050 2024-04-25T00:11:27.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fc831600640 1 -- 172.21.15.83:0/3285712255 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8200072d0 con 0x7fc82c052050 2024-04-25T00:11:27.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fc831600640 1 -- 172.21.15.83:0/3285712255 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200047f0 con 0x7fc82c052050 2024-04-25T00:11:27.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f1409200640 1 -- 172.21.15.83:0/4258901930 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13fc0500a0 msgr2=0x7f13fc04bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f1409200640 1 --2- 172.21.15.83:0/4258901930 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13fc0500a0 0x7f13fc04bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f1409200640 1 -- 172.21.15.83:0/4258901930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13fc0073a0 msgr2=0x7f13fc0cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f1409200640 1 --2- 172.21.15.83:0/4258901930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13fc0073a0 0x7f13fc0cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f1409200640 1 -- 172.21.15.83:0/4258901930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc0cdd60 con 0x7f13fc0df450 2024-04-25T00:11:27.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f69faa00640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69fc0d1730 0x7f69fc06f060 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f69e800e810 tx=0x7f69e800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f1409200640 1 --2- 172.21.15.83:0/4258901930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13fc0df450 0x7f13fc048a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f13f000c750 tx=0x7f13f000cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f80cd400640 1 --2- 172.21.15.83:0/1053245497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c00398f0 0x7f80c003a710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0097c0 tx=0x7f80bc01c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d98a68e0b5d2bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb893e00640 1 -- 172.21.15.83:0/3670414679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb88c038ea0 msgr2=0x7fb88c1811e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:11:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb893e00640 1 --2- 172.21.15.83:0/3670414679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb88c038ea0 0x7fb88c1811e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb893e00640 1 -- 172.21.15.83:0/3670414679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb88c035fd0 msgr2=0x7fb88c1808b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb893e00640 1 --2- 172.21.15.83:0/3670414679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb88c035fd0 0x7fb88c1808b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb893e00640 1 -- 172.21.15.83:0/3670414679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88c180df0 con 0x7fb88c0548e0 2024-04-25T00:11:27.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f69f3400640 1 -- 172.21.15.83:0/3098984112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8017070 con 0x7f69fc0d1730 2024-04-25T00:11:27.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f6a01000640 1 -- 172.21.15.83:0/3098984112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69fc0c5930 con 0x7f69fc0d1730 2024-04-25T00:11:27.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f6a01000640 1 -- 172.21.15.83:0/3098984112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69fc06f5a0 con 0x7f69fc0d1730 2024-04-25T00:11:27.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f69f3400640 1 -- 172.21.15.83:0/3098984112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e800c8d0 con 0x7f69fc0d1730 2024-04-25T00:11:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f13f7e00640 1 -- 172.21.15.83:0/4258901930 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0005ae0 con 0x7f13fc0df450 2024-04-25T00:11:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fb893e00640 1 --2- 172.21.15.83:0/3670414679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c0548e0 0x7fb88c04ffb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb894074f60 tx=0x7fb894074200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f140b200640 1 -- 172.21.15.83:0/4258901930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13fc0ce040 con 0x7f13fc0df450 2024-04-25T00:11:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7f80c7e00640 1 -- 172.21.15.83:0/1053245497 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc0b0070 con 0x7f80c00398f0 2024-04-25T00:11:27.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f140b200640 1 -- 172.21.15.83:0/4258901930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13fc04d760 con 0x7f13fc0df450 2024-04-25T00:11:27.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f13f7e00640 1 -- 172.21.15.83:0/4258901930 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f0005c80 con 0x7f13fc0df450 2024-04-25T00:11:27.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f13f7e00640 1 -- 172.21.15.83:0/4258901930 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0005ae0 con 0x7f13fc0df450 2024-04-25T00:11:27.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7fb887400640 1 -- 172.21.15.83:0/3670414679 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8940695f0 con 0x7fb88c0548e0 2024-04-25T00:11:27.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7fb887400640 1 -- 172.21.15.83:0/3670414679 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8940730c0 con 0x7fb88c0548e0 2024-04-25T00:11:27.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7fb887400640 1 -- 172.21.15.83:0/3670414679 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8940734f0 con 0x7fb88c0548e0 2024-04-25T00:11:27.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7fb89a000640 1 -- 172.21.15.83:0/3670414679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88c033560 con 0x7fb88c0548e0 2024-04-25T00:11:27.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fc83b000640 1 -- 172.21.15.83:0/3285712255 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82c04f180 con 0x7fc82c052050 2024-04-25T00:11:27.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.260+0000 7fc83b000640 1 -- 172.21.15.83:0/3285712255 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82c04f740 con 0x7fc82c052050 2024-04-25T00:11:27.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7fb89a000640 1 -- 172.21.15.83:0/3670414679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb88c033ac0 con 0x7fb88c0548e0 2024-04-25T00:11:27.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f69f3400640 1 -- 172.21.15.83:0/3098984112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8017070 con 0x7f69fc0d1730 2024-04-25T00:11:27.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f80c7e00640 1 -- 172.21.15.83:0/1053245497 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80bc0ab600 con 0x7f80c00398f0 2024-04-25T00:11:27.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f80c7e00640 1 -- 172.21.15.83:0/1053245497 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc0ab9d0 con 0x7f80c00398f0 2024-04-25T00:11:27.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7fc83b000640 1 -- 172.21.15.83:0/3285712255 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc82c04f940 con 0x7fc82c052050 2024-04-25T00:11:27.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f6a01000640 1 -- 172.21.15.83:0/3098984112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69d40052a0 con 0x7f69fc0d1730 2024-04-25T00:11:27.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7fb89a000640 1 -- 172.21.15.83:0/3670414679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb88c005750 con 0x7fb88c0548e0 2024-04-25T00:11:27.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f80ce800640 1 -- 172.21.15.83:0/1053245497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c00398f0 msgr2=0x7f80c003a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f80ce800640 1 --2- 172.21.15.83:0/1053245497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c00398f0 0x7f80c003a710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0097c0 tx=0x7f80bc01c430 comp rx=0 tx=0).stop 2024-04-25T00:11:27.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7fdb1b200640 1 Processor -- start 2024-04-25T00:11:27.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7fdb1b200640 1 -- start start 2024-04-25T00:11:27.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f140b200640 1 -- 172.21.15.83:0/4258901930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13fc005300 con 0x7f13fc0df450 2024-04-25T00:11:27.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f80ce800640 1 -- 172.21.15.83:0/1053245497 shutdown_connections 2024-04-25T00:11:27.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f80ce800640 1 --2- 172.21.15.83:0/1053245497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c00398f0 0x7f80c003a710 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f80ce800640 1 -- 172.21.15.83:0/1053245497 >> 172.21.15.83:0/1053245497 conn(0x7f80c0054ca0 msgr2=0x7f80c00550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f49b8800640 1 Processor -- start 2024-04-25T00:11:27.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f80ce800640 1 -- 172.21.15.83:0/1053245497 shutdown_connections 2024-04-25T00:11:27.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f49b8800640 1 -- start start 2024-04-25T00:11:27.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7fb887400640 1 -- 172.21.15.83:0/3670414679 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb894073690 con 0x7fb88c0548e0 2024-04-25T00:11:27.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.264+0000 7f80ce800640 1 -- 172.21.15.83:0/1053245497 wait complete. 2024-04-25T00:11:27.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fb887400640 1 --2- 172.21.15.83:0/3670414679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb874067910 0x7fb874069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fb887400640 1 -- 172.21.15.83:0/3670414679 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb894100ea0 con 0x7fb88c0548e0 2024-04-25T00:11:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fdb1b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdb100e3070 0x7fdb10057290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fdb1b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb100577d0 0x7fdb10053760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f49b8800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49b0140610 0x7f49b00ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f80ce800640 1 Processor -- start 2024-04-25T00:11:27.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f69f3400640 1 -- 172.21.15.83:0/3098984112 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f69e801a070 con 0x7f69fc0d1730 2024-04-25T00:11:27.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f69f3400640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69dc067cc0 0x7f69dc06a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fdb1b200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb10058d10 con 0x7fdb100e3450 2024-04-25T00:11:27.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f69f3400640 1 -- 172.21.15.83:0/3098984112 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f69e800c6d0 con 0x7f69fc0d1730 2024-04-25T00:11:27.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fdb1b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdb10057c70 con 0x7fdb100e3070 2024-04-25T00:11:27.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f49b5e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49b0140610 0x7f49b00ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fdb19400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb100577d0 0x7fdb10053760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fdb19400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb100577d0 0x7fdb10053760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:49928/0 (socket says 172.21.15.83:49928) 2024-04-25T00:11:27.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fdb1b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdb10057f50 con 0x7fdb100577d0 2024-04-25T00:11:27.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fdb19400640 1 -- 172.21.15.83:0/3194968905 learned_addr learned my addr 172.21.15.83:0/3194968905 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:27.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f49b5e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49b0140610 0x7f49b00ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:49930/0 (socket says 172.21.15.83:49930) 2024-04-25T00:11:27.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f13f7e00640 1 -- 172.21.15.83:0/4258901930 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f13f0007070 con 0x7f13fc0df450 2024-04-25T00:11:27.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f13f7e00640 1 --2- 172.21.15.83:0/4258901930 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f13e4067af0 0x7f13e4069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f13f7e00640 1 -- 172.21.15.83:0/4258901930 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f13f0085920 con 0x7f13fc0df450 2024-04-25T00:11:27.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f49b8800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f49b00cb9d0 0x7f49b00cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f49b5e00640 1 -- 172.21.15.83:0/1957087781 learned_addr learned my addr 172.21.15.83:0/1957087781 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:27.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f49b8800640 1 --2- 172.21.15.83:0/1957087781 >> v2:172.21.15.86:3300/0 conn(0x7f49b0071520 0x7f49b00719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f49b5400640 1 --2- 172.21.15.83:0/1957087781 >> v2:172.21.15.83:3300/0 conn(0x7f49b00cb9d0 0x7f49b00cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fb893400640 1 --2- 172.21.15.83:0/3670414679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb874067910 0x7fb874069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fdb19e00640 1 --2- 172.21.15.83:0/3194968905 >> v2:172.21.15.86:3300/0 conn(0x7fdb100e3070 0x7fdb10057290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f49b8800640 1 -- 172.21.15.83:0/1957087781 --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f49b00cd630 con 0x7f49b00cb9d0 2024-04-25T00:11:27.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f49b6800640 1 --2- 172.21.15.83:0/1957087781 >> v2:172.21.15.86:3300/0 conn(0x7f49b0071520 0x7f49b00719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f49b8800640 1 -- 172.21.15.83:0/1957087781 --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f49b01409f0 con 0x7f49b0071520 2024-04-25T00:11:27.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fdb19400640 1 -- 172.21.15.83:0/3194968905 >> v2:172.21.15.86:3300/0 conn(0x7fdb100e3070 msgr2=0x7fdb10057290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:27.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fdb19400640 1 --2- 172.21.15.83:0/3194968905 >> v2:172.21.15.86:3300/0 conn(0x7fdb100e3070 0x7fdb10057290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fdb19400640 1 -- 172.21.15.83:0/3194968905 >> v1:172.21.15.83:6789/0 conn(0x7fdb100e3450 legacy=0x7fdb10006200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fdb19400640 1 -- 172.21.15.83:0/3194968905 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb10058230 con 0x7fdb100577d0 2024-04-25T00:11:27.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f49b8800640 1 -- 172.21.15.83:0/1957087781 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f49b00cd880 con 0x7f49b0140610 2024-04-25T00:11:27.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7f1403e00640 1 --2- 172.21.15.83:0/4258901930 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f13e4067af0 0x7f13e4069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fc831600640 1 -- 172.21.15.83:0/3285712255 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc820004990 con 0x7fc82c052050 2024-04-25T00:11:27.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fc831600640 1 --2- 172.21.15.83:0/3285712255 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8140678c0 0x7fc814069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.268+0000 7fc831600640 1 -- 172.21.15.83:0/3285712255 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc820085540 con 0x7fc82c052050 2024-04-25T00:11:27.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7fb893400640 1 --2- 172.21.15.83:0/3670414679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb874067910 0x7fb874069dd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb8880b2000 tx=0x7fb888004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f49b5400640 1 -- 172.21.15.83:0/1957087781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49b0140610 msgr2=0x7f49b00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:27.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f49b5400640 1 --2- 172.21.15.83:0/1957087781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49b0140610 0x7f49b00ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f49b5400640 1 -- 172.21.15.83:0/1957087781 >> v2:172.21.15.86:3300/0 conn(0x7f49b0071520 msgr2=0x7f49b00719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:27.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f49b5400640 1 --2- 172.21.15.83:0/1957087781 >> v2:172.21.15.86:3300/0 conn(0x7f49b0071520 0x7f49b00719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f49b5400640 1 -- 172.21.15.83:0/1957087781 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b00cdb60 con 0x7f49b00cb9d0 2024-04-25T00:11:27.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f1403e00640 1 --2- 172.21.15.83:0/4258901930 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f13e4067af0 0x7f13e4069fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1404050e50 tx=0x7f140408e040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f69fb400640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69dc067cc0 0x7f69dc06a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7fc839c00640 1 --2- 172.21.15.83:0/3285712255 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8140678c0 0x7fc814069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f69fb400640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69dc067cc0 0x7f69dc06a180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0af000 tx=0x7f69ec0b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80ce800640 1 -- start start 2024-04-25T00:11:27.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80ce800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c00398f0 0x7f80c0036010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80ce800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80c0050dd0 0x7f80c00c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80ce800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80c00e00b0 0x7f80c00c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80ce800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80c00eef90 con 0x7f80c0050dd0 2024-04-25T00:11:27.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80ce800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80c0053060 con 0x7f80c00398f0 2024-04-25T00:11:27.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80ce800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f80c00ef110 con 0x7f80c00e00b0 2024-04-25T00:11:27.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7fc839c00640 1 --2- 172.21.15.83:0/3285712255 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8140678c0 0x7fc814069d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc82c052f90 tx=0x7fc828008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80cfe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80c00e00b0 0x7f80c00c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80cfe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80c00e00b0 0x7f80c00c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:49938/0 (socket says 172.21.15.83:49938) 2024-04-25T00:11:27.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80cfe00640 1 -- 172.21.15.83:0/3912534066 learned_addr learned my addr 172.21.15.83:0/3912534066 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:27.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80cca00640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80c0050dd0 0x7f80c00c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7fb887400640 1 -- 172.21.15.83:0/3670414679 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb88c033560 con 0x7fb88c0548e0 2024-04-25T00:11:27.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80cd400640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c00398f0 0x7f80c0036010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80cfe00640 1 -- 172.21.15.83:0/3912534066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c00398f0 msgr2=0x7f80c0036010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80cfe00640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c00398f0 0x7f80c0036010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80cfe00640 1 -- 172.21.15.83:0/3912534066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80c0050dd0 msgr2=0x7f80c00c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80cfe00640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80c0050dd0 0x7f80c00c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80cfe00640 1 -- 172.21.15.83:0/3912534066 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c0174040 con 0x7f80c00e00b0 2024-04-25T00:11:27.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.272+0000 7f80cca00640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80c0050dd0 0x7f80c00c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:27.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7f80cfe00640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80c00e00b0 0x7f80c00c8080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f80c80680c0 tx=0x7f80c8068d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7fdb19400640 1 --2- 172.21.15.83:0/3194968905 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb100577d0 0x7fdb10053760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009870 tx=0x7fdb0401ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67b3e176066040ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7f80c5600640 1 -- 172.21.15.83:0/3912534066 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c80741a0 con 0x7f80c00e00b0 2024-04-25T00:11:27.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7f80c5600640 1 -- 172.21.15.83:0/3912534066 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c8069060 con 0x7f80c00e00b0 2024-04-25T00:11:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7fc831600640 1 -- 172.21.15.83:0/3285712255 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc82c04f940 con 0x7fc82c052050 2024-04-25T00:11:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7fdb18a00640 1 -- 172.21.15.83:0/3194968905 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb040b5070 con 0x7fdb100577d0 2024-04-25T00:11:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7fdb18a00640 1 -- 172.21.15.83:0/3194968905 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb040a8a50 con 0x7fdb100577d0 2024-04-25T00:11:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7fdb18a00640 1 -- 172.21.15.83:0/3194968905 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb040a8e20 con 0x7fdb100577d0 2024-04-25T00:11:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7fdb1b200640 1 -- 172.21.15.83:0/3194968905 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb100577d0 msgr2=0x7fdb10053760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7fdb1b200640 1 --2- 172.21.15.83:0/3194968905 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb100577d0 0x7fdb10053760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009870 tx=0x7fdb0401ed50 comp rx=0 tx=0).stop 2024-04-25T00:11:27.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7f80c5600640 1 -- 172.21.15.83:0/3912534066 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c80694d0 con 0x7f80c00e00b0 2024-04-25T00:11:27.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7fdb1b200640 1 -- 172.21.15.83:0/3194968905 shutdown_connections 2024-04-25T00:11:27.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7fdb1b200640 1 --2- 172.21.15.83:0/3194968905 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb100577d0 0x7fdb10053760 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7fdb1b200640 1 --2- 172.21.15.83:0/3194968905 >> v2:172.21.15.86:3300/0 conn(0x7fdb100e3070 0x7fdb10057290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7fdb1b200640 1 -- 172.21.15.83:0/3194968905 >> 172.21.15.83:0/3194968905 conn(0x7fdb10058f00 msgr2=0x7fdb10059310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7fdb1b200640 1 -- 172.21.15.83:0/3194968905 shutdown_connections 2024-04-25T00:11:27.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7fdb1b200640 1 -- 172.21.15.83:0/3194968905 wait complete. 2024-04-25T00:11:27.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7f80ce800640 1 -- 172.21.15.83:0/3912534066 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c01742b0 con 0x7f80c00e00b0 2024-04-25T00:11:27.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7f80ce800640 1 -- 172.21.15.83:0/3912534066 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c0036550 con 0x7f80c00e00b0 2024-04-25T00:11:27.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7f49b5400640 1 --2- 172.21.15.83:0/1957087781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b00cb9d0 0x7f49b00cbe30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f49b00cd560 tx=0x7f49a401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a729eb3ac5a13f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7f69f3400640 1 -- 172.21.15.83:0/3098984112 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f69e8055640 con 0x7f69fc0d1730 2024-04-25T00:11:27.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7f49b4a00640 1 -- 172.21.15.83:0/1957087781 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a40b0070 con 0x7f49b00cb9d0 2024-04-25T00:11:27.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.276+0000 7f80baa00640 1 -- 172.21.15.83:0/3912534066 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c00054a0 con 0x7f80c00e00b0 2024-04-25T00:11:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f49b4a00640 1 -- 172.21.15.83:0/1957087781 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a40ab600 con 0x7f49b00cb9d0 2024-04-25T00:11:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f49b4a00640 1 -- 172.21.15.83:0/1957087781 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a40ab9d0 con 0x7f49b00cb9d0 2024-04-25T00:11:27.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1b200640 1 Processor -- start 2024-04-25T00:11:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f49b8800640 1 -- 172.21.15.83:0/1957087781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b00cb9d0 msgr2=0x7f49b00cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f49b8800640 1 --2- 172.21.15.83:0/1957087781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b00cb9d0 0x7f49b00cbe30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f49b00cd560 tx=0x7f49a401c3f0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f49b8800640 1 -- 172.21.15.83:0/1957087781 shutdown_connections 2024-04-25T00:11:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f49b8800640 1 --2- 172.21.15.83:0/1957087781 >> v2:172.21.15.86:3300/0 conn(0x7f49b0071520 0x7f49b00719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f49b8800640 1 --2- 172.21.15.83:0/1957087781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b00cb9d0 0x7f49b00cbe30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f49b8800640 1 --2- 172.21.15.83:0/1957087781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49b0140610 0x7f49b00ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f49b8800640 1 -- 172.21.15.83:0/1957087781 >> 172.21.15.83:0/1957087781 conn(0x7f49b00cfa60 msgr2=0x7f49b00cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f13f7e00640 1 -- 172.21.15.83:0/4258901930 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f13f0056130 con 0x7f13fc0df450 2024-04-25T00:11:27.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1b200640 1 -- start start 2024-04-25T00:11:27.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f49b8800640 1 -- 172.21.15.83:0/1957087781 shutdown_connections 2024-04-25T00:11:27.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb100577d0 0x7fdb1004ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb100e3070 0x7fdb100db240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb100e3450 0x7fdb100dcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1003d0b0 con 0x7fdb100577d0 2024-04-25T00:11:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb100587f0 con 0x7fdb100e3450 2024-04-25T00:11:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1003d230 con 0x7fdb100e3070 2024-04-25T00:11:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb19e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb100577d0 0x7fdb1004ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb100e3450 0x7fdb100dcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb19e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb100577d0 0x7fdb1004ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36468/0 (socket says 172.21.15.83:36468) 2024-04-25T00:11:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb100e3450 0x7fdb100dcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:39132/0 (socket says 172.21.15.83:39132) 2024-04-25T00:11:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb19e00640 1 -- 172.21.15.83:0/1810793041 learned_addr learned my addr 172.21.15.83:0/1810793041 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb19400640 1 --2- 172.21.15.83:0/1810793041 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb100e3070 0x7fdb100db240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1a800640 1 -- 172.21.15.83:0/1810793041 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb100e3070 msgr2=0x7fdb100db240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1a800640 1 --2- 172.21.15.83:0/1810793041 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb100e3070 0x7fdb100db240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1a800640 1 -- 172.21.15.83:0/1810793041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb100577d0 msgr2=0x7fdb1004ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1a800640 1 --2- 172.21.15.83:0/1810793041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb100577d0 0x7fdb1004ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1a800640 1 -- 172.21.15.83:0/1810793041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1004f210 con 0x7fdb100e3450 2024-04-25T00:11:27.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f49b8800640 1 -- 172.21.15.83:0/1957087781 wait complete. 2024-04-25T00:11:27.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7fdb1a800640 1 --2- 172.21.15.83:0/1810793041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb100e3450 0x7fdb100dcc10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdb14050360 tx=0x7fdb14068d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f80c5600640 1 -- 172.21.15.83:0/3912534066 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f80c8074530 con 0x7f80c00e00b0 2024-04-25T00:11:27.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f80c5600640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80a8067990 0x7f80a8069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.280+0000 7f80c5600640 1 -- 172.21.15.83:0/3912534066 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f80c8101740 con 0x7f80c00e00b0 2024-04-25T00:11:27.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7fdb0e000640 1 -- 172.21.15.83:0/1810793041 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14074060 con 0x7fdb100e3450 2024-04-25T00:11:27.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7fdb0e000640 1 -- 172.21.15.83:0/1810793041 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb1408f030 con 0x7fdb100e3450 2024-04-25T00:11:27.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7fdb0e000640 1 -- 172.21.15.83:0/1810793041 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14074770 con 0x7fdb100e3450 2024-04-25T00:11:27.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7fdb1b200640 1 -- 172.21.15.83:0/1810793041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb100db780 con 0x7fdb100e3450 2024-04-25T00:11:27.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7fdb1b200640 1 -- 172.21.15.83:0/1810793041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb100ddb00 con 0x7fdb100e3450 2024-04-25T00:11:27.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b8800640 1 Processor -- start 2024-04-25T00:11:27.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f80cd400640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80a8067990 0x7f80a8069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7fdb1b200640 1 -- 172.21.15.83:0/1810793041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf00052a0 con 0x7fdb100e3450 2024-04-25T00:11:27.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f80cd400640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80a8067990 0x7f80a8069e50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0ad000 tx=0x7f80bc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b8800640 1 -- start start 2024-04-25T00:11:27.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b8800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49b0071520 0x7f49b013f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b8800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b00cb9d0 0x7f49b01568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b8800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b0140610 0x7f49b0158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b8800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49b00cefe0 con 0x7f49b00cb9d0 2024-04-25T00:11:27.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b8800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49b00cee60 con 0x7f49b0140610 2024-04-25T00:11:27.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b8800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f49b006d860 con 0x7f49b0071520 2024-04-25T00:11:27.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7fdb0e000640 1 -- 172.21.15.83:0/1810793041 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdb14074910 con 0x7fdb100e3450 2024-04-25T00:11:27.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b00cb9d0 0x7f49b01568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b00cb9d0 0x7f49b01568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36472/0 (socket says 172.21.15.83:36472) 2024-04-25T00:11:27.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b5400640 1 -- 172.21.15.83:0/128100977 learned_addr learned my addr 172.21.15.83:0/128100977 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:27.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b6800640 1 --2- 172.21.15.83:0/128100977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b0140610 0x7f49b0158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b6800640 1 -- 172.21.15.83:0/128100977 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49b0071520 msgr2=0x7f49b013f600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:11:27.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b6800640 1 --2- 172.21.15.83:0/128100977 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49b0071520 0x7f49b013f600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b6800640 1 -- 172.21.15.83:0/128100977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b00cb9d0 msgr2=0x7f49b01568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b6800640 1 --2- 172.21.15.83:0/128100977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b00cb9d0 0x7f49b01568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.284+0000 7f49b6800640 1 -- 172.21.15.83:0/128100977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b0154370 con 0x7f49b0140610 2024-04-25T00:11:27.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.288+0000 7fdb0e000640 1 --2- 172.21.15.83:0/1810793041 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdaf8067910 0x7fdaf8069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.288+0000 7fdb0e000640 1 -- 172.21.15.83:0/1810793041 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdb14100cf0 con 0x7fdb100e3450 2024-04-25T00:11:27.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.288+0000 7f49b6800640 1 --2- 172.21.15.83:0/128100977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b0140610 0x7f49b0158f60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00ab10 tx=0x7f49ac00d490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.288+0000 7fdb19e00640 1 --2- 172.21.15.83:0/1810793041 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdaf8067910 0x7fdaf8069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.288+0000 7f80c5600640 1 -- 172.21.15.83:0/3912534066 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f80c80d1960 con 0x7f80c00e00b0 2024-04-25T00:11:27.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.288+0000 7fdb19e00640 1 --2- 172.21.15.83:0/1810793041 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdaf8067910 0x7fdaf8069dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800ab90 tx=0x7fdb08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.288+0000 7f49aa000640 1 -- 172.21.15.83:0/128100977 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac017030 con 0x7f49b0140610 2024-04-25T00:11:27.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.292+0000 7f49aa000640 1 -- 172.21.15.83:0/128100977 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ac0039a0 con 0x7f49b0140610 2024-04-25T00:11:27.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.292+0000 7f49aa000640 1 -- 172.21.15.83:0/128100977 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac003d70 con 0x7f49b0140610 2024-04-25T00:11:27.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.292+0000 7f49b8800640 1 -- 172.21.15.83:0/128100977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b015c8a0 con 0x7f49b0140610 2024-04-25T00:11:27.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.292+0000 7f49b8800640 1 -- 172.21.15.83:0/128100977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b015ce60 con 0x7f49b0140610 2024-04-25T00:11:27.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.292+0000 7fdb0e000640 1 -- 172.21.15.83:0/1810793041 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdb140d0f80 con 0x7fdb100e3450 2024-04-25T00:11:27.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.292+0000 7f49b8800640 1 -- 172.21.15.83:0/128100977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b0068230 con 0x7f49b0140610 2024-04-25T00:11:27.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.292+0000 7f49aa000640 1 -- 172.21.15.83:0/128100977 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f49ac01a070 con 0x7f49b0140610 2024-04-25T00:11:27.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.296+0000 7f49aa000640 1 --2- 172.21.15.83:0/128100977 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f49940678c0 0x7f4994069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.296+0000 7f49aa000640 1 -- 172.21.15.83:0/128100977 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f49ac01de00 con 0x7f49b0140610 2024-04-25T00:11:27.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.296+0000 7f49b5e00640 1 --2- 172.21.15.83:0/128100977 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f49940678c0 0x7f4994069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.296+0000 7f49b5e00640 1 --2- 172.21.15.83:0/128100977 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f49940678c0 0x7f4994069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f49a00098c0 tx=0x7f49a0009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.300+0000 7f49aa000640 1 -- 172.21.15.83:0/128100977 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f49ac056c90 con 0x7f49b0140610 2024-04-25T00:11:27.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.316+0000 7fd5f7a00640 1 Processor -- start 2024-04-25T00:11:27.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.316+0000 7fd5f7a00640 1 -- start start 2024-04-25T00:11:27.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.316+0000 7fd5f7a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd5f00c4c50 0x7fd5f00d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.316+0000 7fd5f7a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd5f00b1060 0x7fd5f006c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.320+0000 7fd5f7a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5f00cf040 con 0x7fd5f0162760 2024-04-25T00:11:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.320+0000 7fd5f7a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd5f00cf5c0 con 0x7fd5f00c4c50 2024-04-25T00:11:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.320+0000 7fd5efe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd5f0162760 0x7fd5f00c5030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:46884/0 (socket says 172.21.15.83:46884) 2024-04-25T00:11:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.320+0000 7fd5efe00640 1 -- 172.21.15.83:0/231588930 learned_addr learned my addr 172.21.15.83:0/231588930 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.320+0000 7fd5f5a00640 1 --2- 172.21.15.83:0/231588930 >> v2:172.21.15.86:3300/0 conn(0x7fd5f00b1060 0x7fd5f006c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.320+0000 7fd5f5000640 1 --2- 172.21.15.83:0/231588930 >> v2:172.21.15.83:3300/0 conn(0x7fd5f00c4c50 0x7fd5f00d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.320+0000 7fd5f7a00640 1 -- 172.21.15.83:0/231588930 --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd5f00cf2e0 con 0x7fd5f00b1060 2024-04-25T00:11:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.320+0000 7fd5f5a00640 1 -- 172.21.15.83:0/231588930 >> v2:172.21.15.83:3300/0 conn(0x7fd5f00c4c50 msgr2=0x7fd5f00d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.320+0000 7fd5f5a00640 1 --2- 172.21.15.83:0/231588930 >> v2:172.21.15.83:3300/0 conn(0x7fd5f00c4c50 0x7fd5f00d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.320+0000 7fd5f5a00640 1 -- 172.21.15.83:0/231588930 >> v1:172.21.15.86:6789/0 conn(0x7fd5f0162760 legacy=0x7fd5f00c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.320+0000 7fd5f5a00640 1 -- 172.21.15.83:0/231588930 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f006f100 con 0x7fd5f00b1060 2024-04-25T00:11:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.324+0000 7fd5f5a00640 1 --2- 172.21.15.83:0/231588930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f00b1060 0x7fd5f006c380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0072900 tx=0x7fd5e801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ccb07a2d79a55f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.328+0000 7fd5ef400640 1 -- 172.21.15.83:0/231588930 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e80b2070 con 0x7fd5f00b1060 2024-04-25T00:11:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.328+0000 7fd5ef400640 1 -- 172.21.15.83:0/231588930 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e80a5910 con 0x7fd5f00b1060 2024-04-25T00:11:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.328+0000 7fd5f7a00640 1 -- 172.21.15.83:0/231588930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f00b1060 msgr2=0x7fd5f006c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.328+0000 7fd5ef400640 1 -- 172.21.15.83:0/231588930 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e80b2070 con 0x7fd5f00b1060 2024-04-25T00:11:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.328+0000 7fd5f7a00640 1 --2- 172.21.15.83:0/231588930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f00b1060 0x7fd5f006c380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0072900 tx=0x7fd5e801c8b0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.328+0000 7fd5f7a00640 1 -- 172.21.15.83:0/231588930 shutdown_connections 2024-04-25T00:11:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.328+0000 7fd5f7a00640 1 --2- 172.21.15.83:0/231588930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f00b1060 0x7fd5f006c380 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.328+0000 7fd5f7a00640 1 --2- 172.21.15.83:0/231588930 >> v2:172.21.15.83:3300/0 conn(0x7fd5f00c4c50 0x7fd5f00d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.328+0000 7fd5f7a00640 1 -- 172.21.15.83:0/231588930 >> 172.21.15.83:0/231588930 conn(0x7fd5f00d1c80 msgr2=0x7fd5f00d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.328+0000 7fd5f7a00640 1 -- 172.21.15.83:0/231588930 shutdown_connections 2024-04-25T00:11:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.328+0000 7fd5f7a00640 1 -- 172.21.15.83:0/231588930 wait complete. 2024-04-25T00:11:27.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.332+0000 7fd5f7a00640 1 Processor -- start 2024-04-25T00:11:27.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.332+0000 7fd5f7a00640 1 -- start start 2024-04-25T00:11:27.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.332+0000 7fd5f7a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f00b1060 0x7fd5f00c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f7a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5f00c4c50 0x7fd5f0157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f7a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5f0162760 0x7fd5f00cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f7a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f006d460 con 0x7fd5f0162760 2024-04-25T00:11:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f7a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f006d2e0 con 0x7fd5f00b1060 2024-04-25T00:11:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f7a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f006d5e0 con 0x7fd5f00c4c50 2024-04-25T00:11:27.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5f0162760 0x7fd5f00cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5f0162760 0x7fd5f00cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36500/0 (socket says 172.21.15.83:36500) 2024-04-25T00:11:27.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f5a00640 1 -- 172.21.15.83:0/3250001275 learned_addr learned my addr 172.21.15.83:0/3250001275 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:27.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f5000640 1 --2- 172.21.15.83:0/3250001275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f00b1060 0x7fd5f00c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f5a00640 1 -- 172.21.15.83:0/3250001275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5f00c4c50 msgr2=0x7fd5f0157e30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:11:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f5a00640 1 --2- 172.21.15.83:0/3250001275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5f00c4c50 0x7fd5f0157e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f5a00640 1 -- 172.21.15.83:0/3250001275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f00b1060 msgr2=0x7fd5f00c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f5a00640 1 --2- 172.21.15.83:0/3250001275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f00b1060 0x7fd5f00c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f5a00640 1 -- 172.21.15.83:0/3250001275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f00b5bf0 con 0x7fd5f0162760 2024-04-25T00:11:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f5000640 1 --2- 172.21.15.83:0/3250001275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f00b1060 0x7fd5f00c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f5a00640 1 --2- 172.21.15.83:0/3250001275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5f0162760 0x7fd5f00cf740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80194c0 tx=0x7fd5e8019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5ecc00640 1 -- 172.21.15.83:0/3250001275 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e80b2050 con 0x7fd5f0162760 2024-04-25T00:11:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f7a00640 1 -- 172.21.15.83:0/3250001275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f00b5910 con 0x7fd5f0162760 2024-04-25T00:11:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.336+0000 7fd5f7a00640 1 -- 172.21.15.83:0/3250001275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f0157630 con 0x7fd5f0162760 2024-04-25T00:11:27.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.340+0000 7fd5ecc00640 1 -- 172.21.15.83:0/3250001275 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e80be070 con 0x7fd5f0162760 2024-04-25T00:11:27.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.340+0000 7fd5ecc00640 1 -- 172.21.15.83:0/3250001275 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8009a10 con 0x7fd5f0162760 2024-04-25T00:11:27.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.340+0000 7fd5f7a00640 1 -- 172.21.15.83:0/3250001275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c80052a0 con 0x7fd5f0162760 2024-04-25T00:11:27.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.340+0000 7fd5ecc00640 1 -- 172.21.15.83:0/3250001275 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd5e8004040 con 0x7fd5f0162760 2024-04-25T00:11:27.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.344+0000 7fd5ecc00640 1 --2- 172.21.15.83:0/3250001275 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd5d0067ac0 0x7fd5d0069f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:27.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.344+0000 7fd5ecc00640 1 -- 172.21.15.83:0/3250001275 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd5e8003db0 con 0x7fd5f0162760 2024-04-25T00:11:27.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.352+0000 7fd5f5000640 1 --2- 172.21.15.83:0/3250001275 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd5d0067ac0 0x7fd5d0069f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:27.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.352+0000 7fd5f5000640 1 --2- 172.21.15.83:0/3250001275 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd5d0067ac0 0x7fd5d0069f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009a40 tx=0x7fd5dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:27.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.352+0000 7fd5ecc00640 1 -- 172.21.15.83:0/3250001275 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd5e8016560 con 0x7fd5f0162760 2024-04-25T00:11:27.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.612+0000 7fb89a000640 1 -- 172.21.15.83:0/3670414679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb88c03b7e0 con 0x7fb88c0548e0 2024-04-25T00:11:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.612+0000 7fb887400640 1 -- 172.21.15.83:0/3670414679 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb8940d1810 con 0x7fb88c0548e0 2024-04-25T00:11:27.614 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2024-04-25T00:11:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.616+0000 7fb89a000640 1 -- 172.21.15.83:0/3670414679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb874067910 msgr2=0x7fb874069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.616+0000 7fb89a000640 1 --2- 172.21.15.83:0/3670414679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb874067910 0x7fb874069dd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb8880b2000 tx=0x7fb888004740 comp rx=0 tx=0).stop 2024-04-25T00:11:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.616+0000 7fb89a000640 1 -- 172.21.15.83:0/3670414679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c0548e0 msgr2=0x7fb88c04ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.616+0000 7fb89a000640 1 --2- 172.21.15.83:0/3670414679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c0548e0 0x7fb88c04ffb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb894074f60 tx=0x7fb894074200 comp rx=0 tx=0).stop 2024-04-25T00:11:27.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.616+0000 7fb89a000640 1 -- 172.21.15.83:0/3670414679 shutdown_connections 2024-04-25T00:11:27.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.616+0000 7fb89a000640 1 --2- 172.21.15.83:0/3670414679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb874067910 0x7fb874069dd0 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb8880b2000 tx=0x7fb888004740 comp rx=0 tx=0).stop 2024-04-25T00:11:27.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.616+0000 7fb89a000640 1 --2- 172.21.15.83:0/3670414679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb88c038ea0 0x7fb88c1811e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.616+0000 7fb89a000640 1 --2- 172.21.15.83:0/3670414679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c0548e0 0x7fb88c04ffb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.616+0000 7fb89a000640 1 --2- 172.21.15.83:0/3670414679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb88c035fd0 0x7fb88c1808b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.616+0000 7fb89a000640 1 -- 172.21.15.83:0/3670414679 >> 172.21.15.83:0/3670414679 conn(0x7fb88c053790 msgr2=0x7fb88c037a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.616+0000 7fb89a000640 1 -- 172.21.15.83:0/3670414679 shutdown_connections 2024-04-25T00:11:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.616+0000 7fb89a000640 1 -- 172.21.15.83:0/3670414679 wait complete. 2024-04-25T00:11:27.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.616+0000 7f6a01000640 1 -- 172.21.15.83:0/3098984112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f69d4005860 con 0x7f69fc0d1730 2024-04-25T00:11:27.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.620+0000 7f140b200640 1 -- 172.21.15.83:0/4258901930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f13fc0379e0 con 0x7f13fc0df450 2024-04-25T00:11:27.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.620+0000 7f69f3400640 1 -- 172.21.15.83:0/3098984112 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f69e8059940 con 0x7f69fc0d1730 2024-04-25T00:11:27.622 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2024-04-25T00:11:27.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.620+0000 7f13f7e00640 1 -- 172.21.15.83:0/4258901930 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f13f0055a30 con 0x7f13fc0df450 2024-04-25T00:11:27.623 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2024-04-25T00:11:27.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.620+0000 7fc83b000640 1 -- 172.21.15.83:0/3285712255 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc82c005720 con 0x7fc82c052050 2024-04-25T00:11:27.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.620+0000 7fc831600640 1 -- 172.21.15.83:0/3285712255 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc820055ca0 con 0x7fc82c052050 2024-04-25T00:11:27.624 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2024-04-25T00:11:27.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f6a01000640 1 -- 172.21.15.83:0/3098984112 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69dc067cc0 msgr2=0x7f69dc06a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f6a01000640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69dc067cc0 0x7f69dc06a180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0af000 tx=0x7f69ec0b6040 comp rx=0 tx=0).stop 2024-04-25T00:11:27.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f6a01000640 1 -- 172.21.15.83:0/3098984112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69fc0d1730 msgr2=0x7f69fc06f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f6a01000640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69fc0d1730 0x7f69fc06f060 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f69e800e810 tx=0x7f69e800ecd0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f140b200640 1 -- 172.21.15.83:0/4258901930 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f13e4067af0 msgr2=0x7f13e4069fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f140b200640 1 --2- 172.21.15.83:0/4258901930 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f13e4067af0 0x7f13e4069fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1404050e50 tx=0x7f140408e040 comp rx=0 tx=0).stop 2024-04-25T00:11:27.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f140b200640 1 -- 172.21.15.83:0/4258901930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13fc0df450 msgr2=0x7f13fc048a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f140b200640 1 --2- 172.21.15.83:0/4258901930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13fc0df450 0x7f13fc048a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f13f000c750 tx=0x7f13f000cc10 comp rx=0 tx=0).stop 2024-04-25T00:11:27.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f6a01000640 1 -- 172.21.15.83:0/3098984112 shutdown_connections 2024-04-25T00:11:27.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f6a01000640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69dc067cc0 0x7f69dc06a180 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f6a01000640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69fc150bd0 0x7f69fc14ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f6a01000640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69fc0d1730 0x7f69fc06f060 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f6a01000640 1 --2- 172.21.15.83:0/3098984112 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69fc0ce5b0 0x7f69fc1ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f6a01000640 1 -- 172.21.15.83:0/3098984112 >> 172.21.15.83:0/3098984112 conn(0x7f69fc0d1be0 msgr2=0x7f69fc151940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f140b200640 1 -- 172.21.15.83:0/4258901930 shutdown_connections 2024-04-25T00:11:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f140b200640 1 --2- 172.21.15.83:0/4258901930 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f13e4067af0 0x7f13e4069fb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f140b200640 1 --2- 172.21.15.83:0/4258901930 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13fc0500a0 0x7f13fc04bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f140b200640 1 --2- 172.21.15.83:0/4258901930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13fc0df450 0x7f13fc048a00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f140b200640 1 --2- 172.21.15.83:0/4258901930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13fc0073a0 0x7f13fc0cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f140b200640 1 -- 172.21.15.83:0/4258901930 >> 172.21.15.83:0/4258901930 conn(0x7f13fc054b00 msgr2=0x7f13fc0397b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f140b200640 1 -- 172.21.15.83:0/4258901930 shutdown_connections 2024-04-25T00:11:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f6a01000640 1 -- 172.21.15.83:0/3098984112 shutdown_connections 2024-04-25T00:11:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f6a01000640 1 -- 172.21.15.83:0/3098984112 wait complete. 2024-04-25T00:11:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7f140b200640 1 -- 172.21.15.83:0/4258901930 wait complete. 2024-04-25T00:11:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7fc83b000640 1 -- 172.21.15.83:0/3285712255 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8140678c0 msgr2=0x7fc814069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7fc83b000640 1 --2- 172.21.15.83:0/3285712255 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8140678c0 0x7fc814069d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc82c052f90 tx=0x7fc828008040 comp rx=0 tx=0).stop 2024-04-25T00:11:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7fc83b000640 1 -- 172.21.15.83:0/3285712255 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc82c052050 msgr2=0x7fc82c0e1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7fc83b000640 1 --2- 172.21.15.83:0/3285712255 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc82c052050 0x7fc82c0e1190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c800 tx=0x7fc82000ccc0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7fc83b000640 1 -- 172.21.15.83:0/3285712255 shutdown_connections 2024-04-25T00:11:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7fc83b000640 1 --2- 172.21.15.83:0/3285712255 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8140678c0 0x7fc814069d80 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc82c052f90 tx=0x7fc828008040 comp rx=0 tx=0).stop 2024-04-25T00:11:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7fc83b000640 1 --2- 172.21.15.83:0/3285712255 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc82c052050 0x7fc82c0e1190 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7fc83b000640 1 --2- 172.21.15.83:0/3285712255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc82c057800 0x7fc82c04fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7fc83b000640 1 --2- 172.21.15.83:0/3285712255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82c051a80 0x7fc82c0e4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7fc83b000640 1 -- 172.21.15.83:0/3285712255 >> 172.21.15.83:0/3285712255 conn(0x7fc82c056ce0 msgr2=0x7fc82c0550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.624+0000 7fc83b000640 1 -- 172.21.15.83:0/3285712255 shutdown_connections 2024-04-25T00:11:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.628+0000 7fc83b000640 1 -- 172.21.15.83:0/3285712255 wait complete. 2024-04-25T00:11:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.628+0000 7f80baa00640 1 -- 172.21.15.83:0/3912534066 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f80c0037b80 con 0x7f80c00e00b0 2024-04-25T00:11:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.632+0000 7f80c5600640 1 -- 172.21.15.83:0/3912534066 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f80c80d5c60 con 0x7f80c00e00b0 2024-04-25T00:11:27.636 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-04-25T00:11:27.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7f49b8800640 1 -- 172.21.15.83:0/128100977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f49b00ca2d0 con 0x7f49b0140610 2024-04-25T00:11:27.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7f80baa00640 1 -- 172.21.15.83:0/3912534066 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80a8067990 msgr2=0x7f80a8069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7f80baa00640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80a8067990 0x7f80a8069e50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0ad000 tx=0x7f80bc0a6000 comp rx=0 tx=0).stop 2024-04-25T00:11:27.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7f80baa00640 1 -- 172.21.15.83:0/3912534066 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80c00e00b0 msgr2=0x7f80c00c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7f80baa00640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80c00e00b0 0x7f80c00c8080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f80c80680c0 tx=0x7f80c8068d10 comp rx=0 tx=0).stop 2024-04-25T00:11:27.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7f49aa000640 1 -- 172.21.15.83:0/128100977 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f49ac056590 con 0x7f49b0140610 2024-04-25T00:11:27.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7f80baa00640 1 -- 172.21.15.83:0/3912534066 shutdown_connections 2024-04-25T00:11:27.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7f80baa00640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80a8067990 0x7f80a8069e50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.643 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2024-04-25T00:11:27.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7f80baa00640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80c00e00b0 0x7f80c00c8080 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7f80baa00640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80c0050dd0 0x7f80c00c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7f80baa00640 1 --2- 172.21.15.83:0/3912534066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c00398f0 0x7f80c0036010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7f80baa00640 1 -- 172.21.15.83:0/3912534066 >> 172.21.15.83:0/3912534066 conn(0x7f80c0054ca0 msgr2=0x7f80c004d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7f80baa00640 1 -- 172.21.15.83:0/3912534066 shutdown_connections 2024-04-25T00:11:27.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7f80baa00640 1 -- 172.21.15.83:0/3912534066 wait complete. 2024-04-25T00:11:27.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7fdb1b200640 1 -- 172.21.15.83:0/1810793041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fdaf0005860 con 0x7fdb100e3450 2024-04-25T00:11:27.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.640+0000 7fdb0e000640 1 -- 172.21.15.83:0/1810793041 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdb140d5280 con 0x7fdb100e3450 2024-04-25T00:11:27.645 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2024-04-25T00:11:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7f49b8800640 1 -- 172.21.15.83:0/128100977 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f49940678c0 msgr2=0x7f4994069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7f49b8800640 1 --2- 172.21.15.83:0/128100977 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f49940678c0 0x7f4994069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f49a00098c0 tx=0x7f49a0009240 comp rx=0 tx=0).stop 2024-04-25T00:11:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7f49b8800640 1 -- 172.21.15.83:0/128100977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b0140610 msgr2=0x7f49b0158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7f49b8800640 1 --2- 172.21.15.83:0/128100977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b0140610 0x7f49b0158f60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00ab10 tx=0x7f49ac00d490 comp rx=0 tx=0).stop 2024-04-25T00:11:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7f49b8800640 1 -- 172.21.15.83:0/128100977 shutdown_connections 2024-04-25T00:11:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7f49b8800640 1 --2- 172.21.15.83:0/128100977 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f49940678c0 0x7f4994069d80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7f49b8800640 1 --2- 172.21.15.83:0/128100977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b0140610 0x7f49b0158f60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7f49b8800640 1 --2- 172.21.15.83:0/128100977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b00cb9d0 0x7f49b01568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7f49b8800640 1 --2- 172.21.15.83:0/128100977 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49b0071520 0x7f49b013f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7f49b8800640 1 -- 172.21.15.83:0/128100977 >> 172.21.15.83:0/128100977 conn(0x7f49b00cfa60 msgr2=0x7f49b006dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7f49b8800640 1 -- 172.21.15.83:0/128100977 shutdown_connections 2024-04-25T00:11:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7f49b8800640 1 -- 172.21.15.83:0/128100977 wait complete. 2024-04-25T00:11:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7fdb1b200640 1 -- 172.21.15.83:0/1810793041 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdaf8067910 msgr2=0x7fdaf8069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7fdb1b200640 1 --2- 172.21.15.83:0/1810793041 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdaf8067910 0x7fdaf8069dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800ab90 tx=0x7fdb08008040 comp rx=0 tx=0).stop 2024-04-25T00:11:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7fdb1b200640 1 -- 172.21.15.83:0/1810793041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb100e3450 msgr2=0x7fdb100dcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.644+0000 7fdb1b200640 1 --2- 172.21.15.83:0/1810793041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb100e3450 0x7fdb100dcc10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdb14050360 tx=0x7fdb14068d10 comp rx=0 tx=0).stop 2024-04-25T00:11:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.648+0000 7fdb1b200640 1 -- 172.21.15.83:0/1810793041 shutdown_connections 2024-04-25T00:11:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.648+0000 7fdb1b200640 1 --2- 172.21.15.83:0/1810793041 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdaf8067910 0x7fdaf8069dd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.648+0000 7fdb1b200640 1 --2- 172.21.15.83:0/1810793041 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb100e3070 0x7fdb100db240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.648+0000 7fdb1b200640 1 --2- 172.21.15.83:0/1810793041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb100e3450 0x7fdb100dcc10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.648+0000 7fdb1b200640 1 --2- 172.21.15.83:0/1810793041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb100577d0 0x7fdb1004ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.648+0000 7fdb1b200640 1 -- 172.21.15.83:0/1810793041 >> 172.21.15.83:0/1810793041 conn(0x7fdb10058f00 msgr2=0x7fdb100e4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.648+0000 7fdb1b200640 1 -- 172.21.15.83:0/1810793041 shutdown_connections 2024-04-25T00:11:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.648+0000 7fdb1b200640 1 -- 172.21.15.83:0/1810793041 wait complete. 2024-04-25T00:11:27.661 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2024-04-25T00:11:27.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2024-04-25T00:11:27.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2024-04-25T00:11:27.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-04-25T00:11:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.676+0000 7fd5f7a00640 1 -- 172.21.15.83:0/3250001275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd5c8005590 con 0x7fd5f0162760 2024-04-25T00:11:27.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-04-25T00:11:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.676+0000 7fd5ecc00640 1 -- 172.21.15.83:0/3250001275 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd5e8016560 con 0x7fd5f0162760 2024-04-25T00:11:27.679 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2024-04-25T00:11:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.676+0000 7fd5f7a00640 1 -- 172.21.15.83:0/3250001275 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd5d0067ac0 msgr2=0x7fd5d0069f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.676+0000 7fd5f7a00640 1 --2- 172.21.15.83:0/3250001275 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd5d0067ac0 0x7fd5d0069f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009a40 tx=0x7fd5dc008040 comp rx=0 tx=0).stop 2024-04-25T00:11:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.676+0000 7fd5f7a00640 1 -- 172.21.15.83:0/3250001275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5f0162760 msgr2=0x7fd5f00cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.676+0000 7fd5f7a00640 1 --2- 172.21.15.83:0/3250001275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5f0162760 0x7fd5f00cf740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80194c0 tx=0x7fd5e8019650 comp rx=0 tx=0).stop 2024-04-25T00:11:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.680+0000 7fd5f7a00640 1 -- 172.21.15.83:0/3250001275 shutdown_connections 2024-04-25T00:11:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.680+0000 7fd5f7a00640 1 --2- 172.21.15.83:0/3250001275 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd5d0067ac0 0x7fd5d0069f80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.680+0000 7fd5f7a00640 1 --2- 172.21.15.83:0/3250001275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5f00c4c50 0x7fd5f0157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.680+0000 7fd5f7a00640 1 --2- 172.21.15.83:0/3250001275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5f0162760 0x7fd5f00cf740 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.680+0000 7fd5f7a00640 1 --2- 172.21.15.83:0/3250001275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f00b1060 0x7fd5f00c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.680+0000 7fd5f7a00640 1 -- 172.21.15.83:0/3250001275 >> 172.21.15.83:0/3250001275 conn(0x7fd5f00d1c80 msgr2=0x7fd5f00b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.680+0000 7fd5f7a00640 1 -- 172.21.15.83:0/3250001275 shutdown_connections 2024-04-25T00:11:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:27.680+0000 7fd5f7a00640 1 -- 172.21.15.83:0/3250001275 wait complete. 2024-04-25T00:11:27.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2024-04-25T00:11:27.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-04-25T00:11:27.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-04-25T00:11:28.662 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T00:11:28.665 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T00:11:28.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T00:11:28.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T00:11:28.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T00:11:28.686 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T00:11:28.688 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T00:11:28.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T00:11:28.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.880+0000 7f7f2da00640 1 Processor -- start 2024-04-25T00:11:28.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.880+0000 7f7f2da00640 1 -- start start 2024-04-25T00:11:28.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.880+0000 7f4270200640 1 Processor -- start 2024-04-25T00:11:28.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.880+0000 7f4270200640 1 -- start start 2024-04-25T00:11:28.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f2da00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f200e2d80 0x7f7f200530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f2da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7f200e01b0 0x7f7f20006630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f26a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7f2000c0e0 0x7f7f20005f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:46888/0 (socket says 172.21.15.83:46888) 2024-04-25T00:11:28.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f26a00640 1 -- 172.21.15.83:0/1896747524 learned_addr learned my addr 172.21.15.83:0/1896747524 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:28.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f4270200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f426000aae0 0x7f4260053f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f4270200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4260053090 0x7f4260007230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f27400640 1 --2- 172.21.15.83:0/1896747524 >> v2:172.21.15.86:3300/0 conn(0x7f7f200e01b0 0x7f7f20006630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f2da00640 1 -- 172.21.15.83:0/1896747524 --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f200537e0 con 0x7f7f2000c0e0 2024-04-25T00:11:28.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f2da00640 1 -- 172.21.15.83:0/1896747524 --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7f20054880 con 0x7f7f200e01b0 2024-04-25T00:11:28.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f2da00640 1 -- 172.21.15.83:0/1896747524 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2004f8b0 con 0x7f7f200e2d80 2024-04-25T00:11:28.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f25600640 1 -- 172.21.15.83:0/1896747524 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1876475405 0 0) 0x7f7f200537e0 con 0x7f7f2000c0e0 2024-04-25T00:11:28.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f426e200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f42600df290 0x7f4260037fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:46898/0 (socket says 172.21.15.83:46898) 2024-04-25T00:11:28.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f426e200640 1 -- 172.21.15.83:0/2015601954 learned_addr learned my addr 172.21.15.83:0/2015601954 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:28.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f25600640 1 -- 172.21.15.83:0/1896747524 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f0c003530 con 0x7f7f2000c0e0 2024-04-25T00:11:28.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f4270200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4260005ef0 con 0x7f42600df290 2024-04-25T00:11:28.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f426ce00640 1 --2- 172.21.15.83:0/2015601954 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4260053090 0x7f4260007230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f426d800640 1 --2- 172.21.15.83:0/2015601954 >> v2:172.21.15.83:3300/0 conn(0x7f426000aae0 0x7f4260053f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f27400640 1 -- 172.21.15.83:0/1896747524 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f200e2d80 msgr2=0x7f7f200530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:28.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f25600640 1 -- 172.21.15.83:0/1896747524 <== mon.1 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3268128779 0 0) 0x7f7f0c003530 con 0x7f7f2000c0e0 2024-04-25T00:11:28.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f2814e00640 1 Processor -- start 2024-04-25T00:11:28.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f4270200640 1 -- 172.21.15.83:0/2015601954 --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4260006190 con 0x7f426000aae0 2024-04-25T00:11:28.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f26000640 1 --2- 172.21.15.83:0/1896747524 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f200e2d80 0x7f7f200530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f4270200640 1 -- 172.21.15.83:0/2015601954 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4260006470 con 0x7f4260053090 2024-04-25T00:11:28.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f27400640 1 --2- 172.21.15.83:0/1896747524 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f200e2d80 0x7f7f200530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f27400640 1 -- 172.21.15.83:0/1896747524 >> v1:172.21.15.86:6789/0 conn(0x7f7f2000c0e0 legacy=0x7f7f20005f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f2814e00640 1 -- start start 2024-04-25T00:11:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.884+0000 7f7f27400640 1 -- 172.21.15.83:0/1896747524 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f20053ac0 con 0x7f7f200e01b0 2024-04-25T00:11:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.888+0000 7f426ce00640 1 -- 172.21.15.83:0/2015601954 >> v2:172.21.15.83:3300/0 conn(0x7f426000aae0 msgr2=0x7f4260053f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.888+0000 7f426ce00640 1 --2- 172.21.15.83:0/2015601954 >> v2:172.21.15.83:3300/0 conn(0x7f426000aae0 0x7f4260053f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.888+0000 7f426ce00640 1 -- 172.21.15.83:0/2015601954 >> v1:172.21.15.86:6789/0 conn(0x7f42600df290 legacy=0x7f4260037fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.888+0000 7f426ce00640 1 -- 172.21.15.83:0/2015601954 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4260006750 con 0x7f4260053090 2024-04-25T00:11:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.888+0000 7f2814e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2808033e40 0x7f28080eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.888+0000 7f2814e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2808047c20 0x7f28080eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.888+0000 7f2814e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28080ebc00 0x7f2808052f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.888+0000 7f2814e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2808050210 con 0x7f2808047c20 2024-04-25T00:11:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.888+0000 7f2814e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2808037bc0 con 0x7f2808033e40 2024-04-25T00:11:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.888+0000 7f2814e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f280804f090 con 0x7f28080ebc00 2024-04-25T00:11:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.888+0000 7f280e800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28080ebc00 0x7f2808052f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.888+0000 7f280e800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28080ebc00 0x7f2808052f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:49986/0 (socket says 172.21.15.83:49986) 2024-04-25T00:11:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.888+0000 7f280e800640 1 -- 172.21.15.83:0/4138063740 learned_addr learned my addr 172.21.15.83:0/4138063740 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.888+0000 7f280d400640 1 --2- 172.21.15.83:0/4138063740 >> v2:172.21.15.83:3300/0 conn(0x7f2808047c20 0x7f28080eb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f7f26000640 1 --2- 172.21.15.83:0/1896747524 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f200e2d80 0x7f7f200530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f426ce00640 1 --2- 172.21.15.83:0/2015601954 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4260053090 0x7f4260007230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f425c0097c0 tx=0x7f425c01c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2713eb8ffd4d7302 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f4267e00640 1 -- 172.21.15.83:0/2015601954 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c0b0070 con 0x7f4260053090 2024-04-25T00:11:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f4267e00640 1 -- 172.21.15.83:0/2015601954 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f425c0ab600 con 0x7f4260053090 2024-04-25T00:11:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f4267e00640 1 -- 172.21.15.83:0/2015601954 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c0ab9d0 con 0x7f4260053090 2024-04-25T00:11:28.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f280d400640 1 -- 172.21.15.83:0/4138063740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28080ebc00 msgr2=0x7f2808052f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:28.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f280d400640 1 --2- 172.21.15.83:0/4138063740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28080ebc00 0x7f2808052f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f280d400640 1 -- 172.21.15.83:0/4138063740 >> v2:172.21.15.86:3300/0 conn(0x7f2808033e40 msgr2=0x7f28080eb180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:11:28.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f280d400640 1 --2- 172.21.15.83:0/4138063740 >> v2:172.21.15.86:3300/0 conn(0x7f2808033e40 0x7f28080eb180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f280d400640 1 -- 172.21.15.83:0/4138063740 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2808034220 con 0x7f2808047c20 2024-04-25T00:11:28.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f7f27400640 1 --2- 172.21.15.83:0/1896747524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f200e01b0 0x7f7f20006630 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400a960 tx=0x7f7f1401d720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c389fb36218cc0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f7f25600640 1 -- 172.21.15.83:0/1896747524 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18001680 con 0x7f7f200e01b0 2024-04-25T00:11:28.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f7f25600640 1 -- 172.21.15.83:0/1896747524 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f140a7500 con 0x7f7f200e01b0 2024-04-25T00:11:28.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f4270200640 1 -- 172.21.15.83:0/2015601954 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4260053090 msgr2=0x7f4260007230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f4270200640 1 --2- 172.21.15.83:0/2015601954 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4260053090 0x7f4260007230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f425c0097c0 tx=0x7f425c01c330 comp rx=0 tx=0).stop 2024-04-25T00:11:28.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f4270200640 1 -- 172.21.15.83:0/2015601954 shutdown_connections 2024-04-25T00:11:28.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f4270200640 1 --2- 172.21.15.83:0/2015601954 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4260053090 0x7f4260007230 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f4270200640 1 --2- 172.21.15.83:0/2015601954 >> v2:172.21.15.83:3300/0 conn(0x7f426000aae0 0x7f4260053f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f4270200640 1 -- 172.21.15.83:0/2015601954 >> 172.21.15.83:0/2015601954 conn(0x7f4260009fc0 msgr2=0x7f426000a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:28.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f7f25600640 1 -- 172.21.15.83:0/1896747524 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400b040 con 0x7f7f200e01b0 2024-04-25T00:11:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f7f2da00640 1 -- 172.21.15.83:0/1896747524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f200e01b0 msgr2=0x7f7f20006630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.892+0000 7f7f2da00640 1 --2- 172.21.15.83:0/1896747524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f200e01b0 0x7f7f20006630 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400a960 tx=0x7f7f1401d720 comp rx=0 tx=0).stop 2024-04-25T00:11:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.896+0000 7f280e800640 1 --2- 172.21.15.83:0/4138063740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28080ebc00 0x7f2808052f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.896+0000 7f7f2da00640 1 -- 172.21.15.83:0/1896747524 shutdown_connections 2024-04-25T00:11:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.896+0000 7f7f2da00640 1 --2- 172.21.15.83:0/1896747524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f200e01b0 0x7f7f20006630 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.896+0000 7f7f2da00640 1 --2- 172.21.15.83:0/1896747524 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f200e2d80 0x7f7f200530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.896+0000 7f7f2da00640 1 -- 172.21.15.83:0/1896747524 >> 172.21.15.83:0/1896747524 conn(0x7f7f20054b00 msgr2=0x7f7f20054f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.896+0000 7f4270200640 1 -- 172.21.15.83:0/2015601954 shutdown_connections 2024-04-25T00:11:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.896+0000 7f4270200640 1 -- 172.21.15.83:0/2015601954 wait complete. 2024-04-25T00:11:28.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.896+0000 7f7f2da00640 1 -- 172.21.15.83:0/1896747524 shutdown_connections 2024-04-25T00:11:28.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.896+0000 7f7f2da00640 1 -- 172.21.15.83:0/1896747524 wait complete. 2024-04-25T00:11:28.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.896+0000 7f280d400640 1 --2- 172.21.15.83:0/4138063740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2808047c20 0x7f28080eb6c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2808054a30 tx=0x7f280001ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7734a3f4651f44b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.896+0000 7f280ca00640 1 -- 172.21.15.83:0/4138063740 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28000b5070 con 0x7f2808047c20 2024-04-25T00:11:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.896+0000 7f4270200640 1 Processor -- start 2024-04-25T00:11:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.896+0000 7f4270200640 1 -- start start 2024-04-25T00:11:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f7f2da00640 1 Processor -- start 2024-04-25T00:11:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f280ca00640 1 -- 172.21.15.83:0/4138063740 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28000a8930 con 0x7f2808047c20 2024-04-25T00:11:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f280ca00640 1 -- 172.21.15.83:0/4138063740 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28000a8d00 con 0x7f2808047c20 2024-04-25T00:11:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f7f2da00640 1 -- start start 2024-04-25T00:11:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f2814e00640 1 -- 172.21.15.83:0/4138063740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2808047c20 msgr2=0x7f28080eb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f2814e00640 1 --2- 172.21.15.83:0/4138063740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2808047c20 0x7f28080eb6c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2808054a30 tx=0x7f280001ecb0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f2814e00640 1 -- 172.21.15.83:0/4138063740 shutdown_connections 2024-04-25T00:11:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f2814e00640 1 --2- 172.21.15.83:0/4138063740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28080ebc00 0x7f2808052f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f2814e00640 1 --2- 172.21.15.83:0/4138063740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2808047c20 0x7f28080eb6c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f2814e00640 1 --2- 172.21.15.83:0/4138063740 >> v2:172.21.15.86:3300/0 conn(0x7f2808033e40 0x7f28080eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f2814e00640 1 -- 172.21.15.83:0/4138063740 >> 172.21.15.83:0/4138063740 conn(0x7f2808009fc0 msgr2=0x7f280800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f2814e00640 1 -- 172.21.15.83:0/4138063740 shutdown_connections 2024-04-25T00:11:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f2814e00640 1 -- 172.21.15.83:0/4138063740 wait complete. 2024-04-25T00:11:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f4270200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426000aae0 0x7f42600d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f4270200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4260053090 0x7f426004cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f7f2da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f2000c0e0 0x7f7f2004ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f426d800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426000aae0 0x7f42600d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f426d800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426000aae0 0x7f42600d38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:49992/0 (socket says 172.21.15.83:49992) 2024-04-25T00:11:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f426ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4260053090 0x7f426004cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f426ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4260053090 0x7f426004cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36538/0 (socket says 172.21.15.83:36538) 2024-04-25T00:11:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f426d800640 1 -- 172.21.15.83:0/636533191 learned_addr learned my addr 172.21.15.83:0/636533191 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f4270200640 1 --2- 172.21.15.83:0/636533191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42600df290 0x7f42600e1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f7f2da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f200e01b0 0x7f7f200d83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f7f2da00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f200e2d80 0x7f7f200e3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f7f26000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f200e01b0 0x7f7f200d83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f7f26000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f200e01b0 0x7f7f200d83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:39182/0 (socket says 172.21.15.83:39182) 2024-04-25T00:11:28.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f26000640 1 -- 172.21.15.83:0/1222804093 learned_addr learned my addr 172.21.15.83:0/1222804093 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:28.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f2da00640 1 -- 172.21.15.83:0/1222804093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f20006f80 con 0x7f7f2000c0e0 2024-04-25T00:11:28.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f2da00640 1 -- 172.21.15.83:0/1222804093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f200e0600 con 0x7f7f200e01b0 2024-04-25T00:11:28.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f2da00640 1 -- 172.21.15.83:0/1222804093 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7f200ec120 con 0x7f7f200e2d80 2024-04-25T00:11:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f4270200640 1 -- 172.21.15.83:0/636533191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4260007770 con 0x7f4260053090 2024-04-25T00:11:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f4270200640 1 -- 172.21.15.83:0/636533191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4260006d10 con 0x7f42600df290 2024-04-25T00:11:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.900+0000 7f4270200640 1 -- 172.21.15.83:0/636533191 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f42600386f0 con 0x7f426000aae0 2024-04-25T00:11:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f2814e00640 1 Processor -- start 2024-04-25T00:11:28.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f27400640 1 --2- 172.21.15.83:0/1222804093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f200e2d80 0x7f7f200e3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f26000640 1 -- 172.21.15.83:0/1222804093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f200e2d80 msgr2=0x7f7f200e3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f26000640 1 --2- 172.21.15.83:0/1222804093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f200e2d80 0x7f7f200e3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f26000640 1 -- 172.21.15.83:0/1222804093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f2000c0e0 msgr2=0x7f7f2004ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f26000640 1 --2- 172.21.15.83:0/1222804093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f2000c0e0 0x7f7f2004ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f26000640 1 -- 172.21.15.83:0/1222804093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f200dba80 con 0x7f7f200e01b0 2024-04-25T00:11:28.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f426ce00640 1 -- 172.21.15.83:0/636533191 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426000aae0 msgr2=0x7f42600d38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f426ce00640 1 --2- 172.21.15.83:0/636533191 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426000aae0 0x7f42600d38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f426ce00640 1 -- 172.21.15.83:0/636533191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42600df290 msgr2=0x7f42600e1af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:11:28.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f26000640 1 --2- 172.21.15.83:0/1222804093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f200e01b0 0x7f7f200d83e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7f28000b70 tx=0x7f7f28074200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f426ce00640 1 --2- 172.21.15.83:0/636533191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42600df290 0x7f42600e1af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f426ce00640 1 -- 172.21.15.83:0/636533191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426004d670 con 0x7f4260053090 2024-04-25T00:11:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f1ea00640 1 -- 172.21.15.83:0/1222804093 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f280695f0 con 0x7f7f200e01b0 2024-04-25T00:11:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f1ea00640 1 -- 172.21.15.83:0/1222804093 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f280730c0 con 0x7f7f200e01b0 2024-04-25T00:11:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f1ea00640 1 -- 172.21.15.83:0/1222804093 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f28073510 con 0x7f7f200e01b0 2024-04-25T00:11:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f2da00640 1 -- 172.21.15.83:0/1222804093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2004c370 con 0x7f7f200e01b0 2024-04-25T00:11:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f2da00640 1 -- 172.21.15.83:0/1222804093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f2004c930 con 0x7f7f200e01b0 2024-04-25T00:11:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f2814e00640 1 -- start start 2024-04-25T00:11:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f2814e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2808033e40 0x7f28080dc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f2814e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28080ebc00 0x7f2808034f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f426ce00640 1 --2- 172.21.15.83:0/636533191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4260053090 0x7f426004cbf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f425c0ab000 tx=0x7f425c0190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f2814e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2808049de0 0x7f28080e9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f2814e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f280804f2e0 con 0x7f2808049de0 2024-04-25T00:11:28.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f2814e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f280804fd80 con 0x7f2808033e40 2024-04-25T00:11:28.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f280de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2808033e40 0x7f28080dc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f7f2da00640 1 -- 172.21.15.83:0/1222804093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f20005300 con 0x7f7f200e01b0 2024-04-25T00:11:28.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f4265600640 1 -- 172.21.15.83:0/636533191 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c0b8020 con 0x7f4260053090 2024-04-25T00:11:28.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f4265600640 1 -- 172.21.15.83:0/636533191 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f425c0aa950 con 0x7f4260053090 2024-04-25T00:11:28.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f4265600640 1 -- 172.21.15.83:0/636533191 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c0b0040 con 0x7f4260053090 2024-04-25T00:11:28.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.908+0000 7f4270200640 1 -- 172.21.15.83:0/636533191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426004d950 con 0x7f4260053090 2024-04-25T00:11:28.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f2814e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f280804f5c0 con 0x7f28080ebc00 2024-04-25T00:11:28.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f280de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2808033e40 0x7f28080dc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:39192/0 (socket says 172.21.15.83:39192) 2024-04-25T00:11:28.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f280de00640 1 -- 172.21.15.83:0/3229521031 learned_addr learned my addr 172.21.15.83:0/3229521031 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:28.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f280d400640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28080ebc00 0x7f2808034f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f280de00640 1 -- 172.21.15.83:0/3229521031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28080ebc00 msgr2=0x7f2808034f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f280de00640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28080ebc00 0x7f2808034f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f280de00640 1 -- 172.21.15.83:0/3229521031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2808049de0 msgr2=0x7f28080e9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.904+0000 7f280e800640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2808049de0 0x7f28080e9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.908+0000 7f280de00640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2808049de0 0x7f28080e9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.908+0000 7f280de00640 1 -- 172.21.15.83:0/3229521031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2808035f70 con 0x7f2808033e40 2024-04-25T00:11:28.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.908+0000 7f280e800640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2808049de0 0x7f28080e9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:28.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.908+0000 7f4270200640 1 -- 172.21.15.83:0/636533191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42600d45d0 con 0x7f4260053090 2024-04-25T00:11:28.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.908+0000 7f7f1ea00640 1 -- 172.21.15.83:0/1222804093 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7f280736b0 con 0x7f7f200e01b0 2024-04-25T00:11:28.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.908+0000 7f4270200640 1 -- 172.21.15.83:0/636533191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42600052d0 con 0x7f4260053090 2024-04-25T00:11:28.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.908+0000 7f280de00640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2808033e40 0x7f28080dc1f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f281002d710 tx=0x7f2810068d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.908+0000 7f7f1ea00640 1 --2- 172.21.15.83:0/1222804093 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7f080678c0 0x7f7f08069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.908+0000 7f7f26a00640 1 --2- 172.21.15.83:0/1222804093 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7f080678c0 0x7f7f08069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.908+0000 7f7f1ea00640 1 -- 172.21.15.83:0/1222804093 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7f28101040 con 0x7f7f200e01b0 2024-04-25T00:11:28.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.912+0000 7f4265600640 1 -- 172.21.15.83:0/636533191 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f425c0bb020 con 0x7f4260053090 2024-04-25T00:11:28.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.912+0000 7f7f26a00640 1 --2- 172.21.15.83:0/1222804093 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7f080678c0 0x7f7f08069d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7f18001df0 tx=0x7f7f180010b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.912+0000 7f4265600640 1 --2- 172.21.15.83:0/636533191 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4248067990 0x7f4248069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.912+0000 7f4265600640 1 -- 172.21.15.83:0/636533191 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f425c1203c0 con 0x7f4260053090 2024-04-25T00:11:28.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.912+0000 7f426d800640 1 --2- 172.21.15.83:0/636533191 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4248067990 0x7f4248069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.912+0000 7f2806000640 1 -- 172.21.15.83:0/3229521031 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28100740e0 con 0x7f2808033e40 2024-04-25T00:11:28.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.912+0000 7f7f1ea00640 1 -- 172.21.15.83:0/1222804093 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7f280d10c0 con 0x7f7f200e01b0 2024-04-25T00:11:28.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.912+0000 7f2814e00640 1 -- 172.21.15.83:0/3229521031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2808036250 con 0x7f2808033e40 2024-04-25T00:11:28.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.912+0000 7f2814e00640 1 -- 172.21.15.83:0/3229521031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2808036810 con 0x7f2808033e40 2024-04-25T00:11:28.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.916+0000 7f426d800640 1 --2- 172.21.15.83:0/636533191 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4248067990 0x7f4248069e50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f42680680c0 tx=0x7f426808d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.912+0000 7f2806000640 1 -- 172.21.15.83:0/3229521031 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f281008f020 con 0x7f2808033e40 2024-04-25T00:11:28.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.912+0000 7f2806000640 1 -- 172.21.15.83:0/3229521031 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28100740e0 con 0x7f2808033e40 2024-04-25T00:11:28.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.916+0000 7f2814e00640 1 -- 172.21.15.83:0/3229521031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e80052a0 con 0x7f2808033e40 2024-04-25T00:11:28.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.920+0000 7f4265600640 1 -- 172.21.15.83:0/636533191 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f425c0c2030 con 0x7f4260053090 2024-04-25T00:11:28.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.920+0000 7f2806000640 1 -- 172.21.15.83:0/3229521031 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2810074530 con 0x7f2808033e40 2024-04-25T00:11:28.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.920+0000 7f2806000640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27f0067ac0 0x7f27f0069f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.920+0000 7f2806000640 1 -- 172.21.15.83:0/3229521031 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2810100bf0 con 0x7f2808033e40 2024-04-25T00:11:28.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.924+0000 7f280d400640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27f0067ac0 0x7f27f0069f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.924+0000 7f280d400640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27f0067ac0 0x7f27f0069f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f28000b2000 tx=0x7f28000aa070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.924+0000 7f2806000640 1 -- 172.21.15.83:0/3229521031 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2810099030 con 0x7f2808033e40 2024-04-25T00:11:28.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7f12be600640 1 Processor -- start 2024-04-25T00:11:28.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7f12be600640 1 -- start start 2024-04-25T00:11:28.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7fd3ecc00640 1 Processor -- start 2024-04-25T00:11:28.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7fd3ecc00640 1 -- start start 2024-04-25T00:11:28.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7f12be600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12b80d0580 0x7f12b81403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7f12be600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f12b80ce920 0x7f12b80cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7f12be600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f12b80ced80 0x7f12b80cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7f12b7400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12b80d0580 0x7f12b81403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7f12b7400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12b80d0580 0x7f12b81403b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:50028/0 (socket says 172.21.15.83:50028) 2024-04-25T00:11:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7f12b7400640 1 -- 172.21.15.83:0/612245432 learned_addr learned my addr 172.21.15.83:0/612245432 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7f12be600640 1 -- 172.21.15.83:0/612245432 --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f12b80cd630 con 0x7f12b80ce920 2024-04-25T00:11:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7f12be600640 1 -- 172.21.15.83:0/612245432 --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f12b80cdb60 con 0x7f12b80ced80 2024-04-25T00:11:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7f12be600640 1 -- 172.21.15.83:0/612245432 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f12b80cde40 con 0x7f12b80d0580 2024-04-25T00:11:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7fd3ecc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd3e806ecc0 0x7fd3e8140230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7fd3ecc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd3e8072450 0x7fd3e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7fd3ecc00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3e815de70 con 0x7fd3e8143190 2024-04-25T00:11:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7fd3ecc00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd3e815ea80 con 0x7fd3e806ecc0 2024-04-25T00:11:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.964+0000 7fd3ecc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd3e815e150 con 0x7fd3e8072450 2024-04-25T00:11:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7f12b7e00640 1 --2- 172.21.15.83:0/612245432 >> v2:172.21.15.86:3300/0 conn(0x7f12b80ced80 0x7f12b80cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7f12b6a00640 1 --2- 172.21.15.83:0/612245432 >> v2:172.21.15.83:3300/0 conn(0x7f12b80ce920 0x7f12b80cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7fd3e6a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd3e8143190 0x7fd3e806cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52228/0 (socket says 172.21.15.83:52228) 2024-04-25T00:11:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7fd3e6a00640 1 -- 172.21.15.83:0/4269229224 learned_addr learned my addr 172.21.15.83:0/4269229224 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7fd3e7e00640 1 --2- 172.21.15.83:0/4269229224 >> v2:172.21.15.86:3300/0 conn(0x7fd3e8072450 0x7fd3e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7f12b7e00640 1 -- 172.21.15.83:0/612245432 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12b80d0580 msgr2=0x7f12b81403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7f12b7e00640 1 --2- 172.21.15.83:0/612245432 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12b80d0580 0x7f12b81403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7f12b7e00640 1 -- 172.21.15.83:0/612245432 >> v2:172.21.15.83:3300/0 conn(0x7f12b80ce920 msgr2=0x7f12b80cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7f12b7e00640 1 --2- 172.21.15.83:0/612245432 >> v2:172.21.15.83:3300/0 conn(0x7f12b80ce920 0x7f12b80cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7f12b7e00640 1 -- 172.21.15.83:0/612245432 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b80ce120 con 0x7f12b80ced80 2024-04-25T00:11:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7fd3e7400640 1 --2- 172.21.15.83:0/4269229224 >> v2:172.21.15.83:3300/0 conn(0x7fd3e806ecc0 0x7fd3e8140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7fd3e6000640 1 -- 172.21.15.83:0/4269229224 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2731275043 0 0) 0x7fd3e815de70 con 0x7fd3e8143190 2024-04-25T00:11:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7fd3e6000640 1 -- 172.21.15.83:0/4269229224 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3cc003530 con 0x7fd3e8143190 2024-04-25T00:11:28.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7fd3e7400640 1 -- 172.21.15.83:0/4269229224 >> v2:172.21.15.86:3300/0 conn(0x7fd3e8072450 msgr2=0x7fd3e806d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:28.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7fd3e7400640 1 --2- 172.21.15.83:0/4269229224 >> v2:172.21.15.86:3300/0 conn(0x7fd3e8072450 0x7fd3e806d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7fd3e7400640 1 -- 172.21.15.83:0/4269229224 >> v1:172.21.15.83:6789/0 conn(0x7fd3e8143190 legacy=0x7fd3e806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7fd3e7400640 1 -- 172.21.15.83:0/4269229224 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e815e430 con 0x7fd3e806ecc0 2024-04-25T00:11:28.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.968+0000 7f12b6a00640 1 --2- 172.21.15.83:0/612245432 >> v2:172.21.15.83:3300/0 conn(0x7f12b80ce920 0x7f12b80cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:28.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f16dfa00640 1 Processor -- start 2024-04-25T00:11:28.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f6fdca00640 1 Processor -- start 2024-04-25T00:11:28.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f16dfa00640 1 -- start start 2024-04-25T00:11:28.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f6fdca00640 1 -- start start 2024-04-25T00:11:28.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f16dfa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f16d806cfd0 0x7f16d8070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f12b7e00640 1 --2- 172.21.15.83:0/612245432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b80ced80 0x7f12b80cbf10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f12b80cd560 tx=0x7f12b001c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b936f0c6a72e3a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f6fdca00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6fd80b5fe0 0x7f6fd8072e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f6fdca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6fd8073410 0x7f6fd80b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f6fdca00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd809c9f0 con 0x7f6fd8162700 2024-04-25T00:11:28.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f6fdca00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f6fd81662d0 con 0x7f6fd80b5fe0 2024-04-25T00:11:28.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f6fdca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6fd81665b0 con 0x7f6fd8073410 2024-04-25T00:11:28.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f16dfa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f16d80cb640 0x7f16d8165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f12b6000640 1 -- 172.21.15.83:0/612245432 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b00b2070 con 0x7f12b80ced80 2024-04-25T00:11:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f12b6000640 1 -- 172.21.15.83:0/612245432 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b00a5910 con 0x7f12b80ced80 2024-04-25T00:11:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f12b6000640 1 -- 172.21.15.83:0/612245432 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b00a5ce0 con 0x7f12b80ced80 2024-04-25T00:11:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f16dfa00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d806e160 con 0x7f16d806d3b0 2024-04-25T00:11:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f16dfa00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f16d806df20 con 0x7f16d806cfd0 2024-04-25T00:11:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f16dfa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f16d806ea10 con 0x7f16d80cb640 2024-04-25T00:11:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f16dd000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f16d806cfd0 0x7f16d8070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f16dd000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f16d806cfd0 0x7f16d8070d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36584/0 (socket says 172.21.15.83:36584) 2024-04-25T00:11:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f16dd000640 1 -- 172.21.15.83:0/3379989140 learned_addr learned my addr 172.21.15.83:0/3379989140 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:28.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7fd3e7400640 1 --2- 172.21.15.83:0/4269229224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e806ecc0 0x7fd3e8140230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8143620 tx=0x7fd3e001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19aa507112656fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f6fd7400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6fd8162700 0x7f6fd80b58d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52238/0 (socket says 172.21.15.83:52238) 2024-04-25T00:11:28.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.972+0000 7f6fd7400640 1 -- 172.21.15.83:0/4005050913 learned_addr learned my addr 172.21.15.83:0/4005050913 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f6fd6a00640 1 --2- 172.21.15.83:0/4005050913 >> v2:172.21.15.83:3300/0 conn(0x7f6fd80b5fe0 0x7f6fd8072e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f6fd7e00640 1 --2- 172.21.15.83:0/4005050913 >> v2:172.21.15.86:3300/0 conn(0x7f6fd8073410 0x7f6fd80b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f6fd6a00640 1 -- 172.21.15.83:0/4005050913 >> v2:172.21.15.86:3300/0 conn(0x7f6fd8073410 msgr2=0x7f6fd80b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f6fd6a00640 1 --2- 172.21.15.83:0/4005050913 >> v2:172.21.15.86:3300/0 conn(0x7f6fd8073410 0x7f6fd80b7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f6fd6a00640 1 -- 172.21.15.83:0/4005050913 >> v1:172.21.15.83:6789/0 conn(0x7f6fd8162700 legacy=0x7f6fd80b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f6fd6a00640 1 -- 172.21.15.83:0/4005050913 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd8166890 con 0x7f6fd80b5fe0 2024-04-25T00:11:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f12be600640 1 -- 172.21.15.83:0/612245432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b80ced80 msgr2=0x7f12b80cbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f12be600640 1 --2- 172.21.15.83:0/612245432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b80ced80 0x7f12b80cbf10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f12b80cd560 tx=0x7f12b001c8b0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f12be600640 1 -- 172.21.15.83:0/612245432 shutdown_connections 2024-04-25T00:11:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f12be600640 1 --2- 172.21.15.83:0/612245432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b80ced80 0x7f12b80cbf10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f12be600640 1 --2- 172.21.15.83:0/612245432 >> v2:172.21.15.83:3300/0 conn(0x7f12b80ce920 0x7f12b80cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f12be600640 1 --2- 172.21.15.83:0/612245432 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12b80d0580 0x7f12b81403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f12be600640 1 -- 172.21.15.83:0/612245432 >> 172.21.15.83:0/612245432 conn(0x7f12b80cfa60 msgr2=0x7f12b80cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f12be600640 1 -- 172.21.15.83:0/612245432 shutdown_connections 2024-04-25T00:11:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f12be600640 1 -- 172.21.15.83:0/612245432 wait complete. 2024-04-25T00:11:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f16dd000640 1 -- 172.21.15.83:0/3379989140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f16d80cb640 msgr2=0x7f16d8165930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:11:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f16dd000640 1 --2- 172.21.15.83:0/3379989140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f16d80cb640 0x7f16d8165930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f16dd000640 1 -- 172.21.15.83:0/3379989140 >> v1:172.21.15.83:6789/0 conn(0x7f16d806d3b0 legacy=0x7f16d806d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f16dd000640 1 -- 172.21.15.83:0/3379989140 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d806ecf0 con 0x7f16d806cfd0 2024-04-25T00:11:28.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7fd3e6000640 1 -- 172.21.15.83:0/4269229224 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e00b2070 con 0x7fd3e806ecc0 2024-04-25T00:11:28.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7fd3e6000640 1 -- 172.21.15.83:0/4269229224 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e00a59e0 con 0x7fd3e806ecc0 2024-04-25T00:11:28.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7fd3e6000640 1 -- 172.21.15.83:0/4269229224 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e00a5db0 con 0x7fd3e806ecc0 2024-04-25T00:11:28.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7fd3ecc00640 1 -- 172.21.15.83:0/4269229224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e806ecc0 msgr2=0x7fd3e8140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7fd3ecc00640 1 --2- 172.21.15.83:0/4269229224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e806ecc0 0x7fd3e8140230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8143620 tx=0x7fd3e001c900 comp rx=0 tx=0).stop 2024-04-25T00:11:28.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7fd3ecc00640 1 -- 172.21.15.83:0/4269229224 shutdown_connections 2024-04-25T00:11:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7fd3ecc00640 1 --2- 172.21.15.83:0/4269229224 >> v2:172.21.15.86:3300/0 conn(0x7fd3e8072450 0x7fd3e806d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7fd3ecc00640 1 --2- 172.21.15.83:0/4269229224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e806ecc0 0x7fd3e8140230 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7fd3ecc00640 1 -- 172.21.15.83:0/4269229224 >> 172.21.15.83:0/4269229224 conn(0x7fd3e806b3b0 msgr2=0x7fd3e806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7fd3ecc00640 1 -- 172.21.15.83:0/4269229224 shutdown_connections 2024-04-25T00:11:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f12be600640 1 Processor -- start 2024-04-25T00:11:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7fd3ecc00640 1 -- 172.21.15.83:0/4269229224 wait complete. 2024-04-25T00:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.976+0000 7f12be600640 1 -- start start 2024-04-25T00:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f12be600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b80ce920 0x7f12b813cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f12be600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12b80ced80 0x7f12b80b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f16dd000640 1 --2- 172.21.15.83:0/3379989140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d806cfd0 0x7f16d8070d10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16c800dc90 tx=0x7f16c8020850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b8788278fee949 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7fb23a800640 1 Processor -- start 2024-04-25T00:11:28.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7fb23a800640 1 -- start start 2024-04-25T00:11:28.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f6fd6a00640 1 --2- 172.21.15.83:0/4005050913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd80b5fe0 0x7f6fd8072e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80b2490 tx=0x7f6fc401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac20c8ca8843545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f12b7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b80ce920 0x7f12b813cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f12b7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b80ce920 0x7f12b813cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36606/0 (socket says 172.21.15.83:36606) 2024-04-25T00:11:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f12b7400640 1 -- 172.21.15.83:0/328977282 learned_addr learned my addr 172.21.15.83:0/328977282 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f16d7400640 1 -- 172.21.15.83:0/3379989140 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c800d0d0 con 0x7f16d806cfd0 2024-04-25T00:11:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f16d7400640 1 -- 172.21.15.83:0/3379989140 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16c800d270 con 0x7f16d806cfd0 2024-04-25T00:11:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f16d7400640 1 -- 172.21.15.83:0/3379989140 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c80aa710 con 0x7f16d806cfd0 2024-04-25T00:11:28.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f6fd6000640 1 -- 172.21.15.83:0/4005050913 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc40b1070 con 0x7f6fd80b5fe0 2024-04-25T00:11:28.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f12be600640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b80d0580 0x7f12b815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f12be600640 1 -- 172.21.15.83:0/328977282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12b806d3e0 con 0x7f12b80ce920 2024-04-25T00:11:28.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f12b6a00640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12b80ced80 0x7f12b80b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f12be600640 1 -- 172.21.15.83:0/328977282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12b806d260 con 0x7f12b80d0580 2024-04-25T00:11:28.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f12be600640 1 -- 172.21.15.83:0/328977282 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f12b806d560 con 0x7f12b80ced80 2024-04-25T00:11:28.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f12b7e00640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b80d0580 0x7f12b815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f16dfa00640 1 -- 172.21.15.83:0/3379989140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d806cfd0 msgr2=0x7f16d8070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f16dfa00640 1 --2- 172.21.15.83:0/3379989140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d806cfd0 0x7f16d8070d10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16c800dc90 tx=0x7f16c8020850 comp rx=0 tx=0).stop 2024-04-25T00:11:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f16dfa00640 1 -- 172.21.15.83:0/3379989140 shutdown_connections 2024-04-25T00:11:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f16dfa00640 1 --2- 172.21.15.83:0/3379989140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f16d80cb640 0x7f16d8165930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f16dfa00640 1 --2- 172.21.15.83:0/3379989140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d806cfd0 0x7f16d8070d10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f16dfa00640 1 -- 172.21.15.83:0/3379989140 >> 172.21.15.83:0/3379989140 conn(0x7f16d80cfa60 msgr2=0x7f16d80cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7fd3ecc00640 1 Processor -- start 2024-04-25T00:11:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f6fd6000640 1 -- 172.21.15.83:0/4005050913 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc40ac600 con 0x7f6fd80b5fe0 2024-04-25T00:11:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f6fd6000640 1 -- 172.21.15.83:0/4005050913 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc40b1070 con 0x7f6fd80b5fe0 2024-04-25T00:11:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f6fdca00640 1 -- 172.21.15.83:0/4005050913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd80b5fe0 msgr2=0x7f6fd8072e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f6fdca00640 1 --2- 172.21.15.83:0/4005050913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd80b5fe0 0x7f6fd8072e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80b2490 tx=0x7f6fc401c380 comp rx=0 tx=0).stop 2024-04-25T00:11:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7fb23a800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb22c0e7700 0x7fb22c0072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7fb23a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb22c047d70 0x7fb22c037ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fd3ecc00640 1 -- start start 2024-04-25T00:11:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f6fdca00640 1 -- 172.21.15.83:0/4005050913 shutdown_connections 2024-04-25T00:11:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f6fdca00640 1 --2- 172.21.15.83:0/4005050913 >> v2:172.21.15.86:3300/0 conn(0x7f6fd8073410 0x7f6fd80b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f6fdca00640 1 --2- 172.21.15.83:0/4005050913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd80b5fe0 0x7f6fd8072e60 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80b2490 tx=0x7f6fc401c380 comp rx=0 tx=0).stop 2024-04-25T00:11:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f6fdca00640 1 -- 172.21.15.83:0/4005050913 >> 172.21.15.83:0/4005050913 conn(0x7f6fd80d08d0 msgr2=0x7f6fd80d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:28.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f12b6a00640 1 -- 172.21.15.83:0/328977282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b80d0580 msgr2=0x7f12b815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f12b6a00640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b80d0580 0x7f12b815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7f12b6a00640 1 -- 172.21.15.83:0/328977282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b80ce920 msgr2=0x7f12b813cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.980+0000 7fb23a800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb22c0543d0 con 0x7fb22c0381e0 2024-04-25T00:11:28.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fb23a800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb22c006e40 con 0x7fb22c0e7700 2024-04-25T00:11:28.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fb23a800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb22c0546b0 con 0x7fb22c047d70 2024-04-25T00:11:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f12b6a00640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b80ce920 0x7f12b813cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f12b6a00640 1 -- 172.21.15.83:0/328977282 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b8152d10 con 0x7f12b80ced80 2024-04-25T00:11:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f12b7400640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b80ce920 0x7f12b813cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f16dfa00640 1 -- 172.21.15.83:0/3379989140 shutdown_connections 2024-04-25T00:11:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f16dfa00640 1 -- 172.21.15.83:0/3379989140 wait complete. 2024-04-25T00:11:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f12b6a00640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12b80ced80 0x7f12b80b45f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f12a800e810 tx=0x7f12a800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fd3ecc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd3e806ecc0 0x7fd3e8159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fd3ecc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e8072450 0x7fd3e8140a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fd3ecc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3e8143190 0x7fd3e8140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fb232a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb22c047d70 0x7fb22c037ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fb232a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb22c047d70 0x7fb22c037ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:39240/0 (socket says 172.21.15.83:39240) 2024-04-25T00:11:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fb232a00640 1 -- 172.21.15.83:0/1698638165 learned_addr learned my addr 172.21.15.83:0/1698638165 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fb233400640 1 --2- 172.21.15.83:0/1698638165 >> v2:172.21.15.83:3300/0 conn(0x7fb22c0e7700 0x7fb22c0072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fb233400640 1 -- 172.21.15.83:0/1698638165 >> v2:172.21.15.86:3300/0 conn(0x7fb22c047d70 msgr2=0x7fb22c037ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fb233400640 1 --2- 172.21.15.83:0/1698638165 >> v2:172.21.15.86:3300/0 conn(0x7fb22c047d70 0x7fb22c037ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fb233400640 1 -- 172.21.15.83:0/1698638165 >> v1:172.21.15.83:6789/0 conn(0x7fb22c0381e0 legacy=0x7fb22c0526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fb233400640 1 -- 172.21.15.83:0/1698638165 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22c006ad0 con 0x7fb22c0e7700 2024-04-25T00:11:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fd3e7400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd3e806ecc0 0x7fd3e8159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fd3e6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e8072450 0x7fd3e8140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fd3e6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e8072450 0x7fd3e8140a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36614/0 (socket says 172.21.15.83:36614) 2024-04-25T00:11:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fd3e7400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd3e806ecc0 0x7fd3e8159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:50060/0 (socket says 172.21.15.83:50060) 2024-04-25T00:11:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f12af400640 1 -- 172.21.15.83:0/328977282 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8017070 con 0x7f12b80ced80 2024-04-25T00:11:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f12af400640 1 -- 172.21.15.83:0/328977282 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12a800c6d0 con 0x7f12b80ced80 2024-04-25T00:11:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f12af400640 1 -- 172.21.15.83:0/328977282 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a800f040 con 0x7f12b80ced80 2024-04-25T00:11:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f12be600640 1 -- 172.21.15.83:0/328977282 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b8152ff0 con 0x7f12b80ced80 2024-04-25T00:11:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f12be600640 1 -- 172.21.15.83:0/328977282 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b81535b0 con 0x7f12b80ced80 2024-04-25T00:11:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f16dfa00640 1 Processor -- start 2024-04-25T00:11:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fd3e6a00640 1 -- 172.21.15.83:0/2765188149 learned_addr learned my addr 172.21.15.83:0/2765188149 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fd3ecc00640 1 -- 172.21.15.83:0/2765188149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e806d9c0 con 0x7fd3e8072450 2024-04-25T00:11:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7fd3e7e00640 1 --2- 172.21.15.83:0/2765188149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3e8143190 0x7fd3e8140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f16dfa00640 1 -- start start 2024-04-25T00:11:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.984+0000 7f12be600640 1 -- 172.21.15.83:0/328977282 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12900052a0 con 0x7f12b80ced80 2024-04-25T00:11:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f6fdca00640 1 -- 172.21.15.83:0/4005050913 shutdown_connections 2024-04-25T00:11:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f6fdca00640 1 -- 172.21.15.83:0/4005050913 wait complete. 2024-04-25T00:11:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7fd3ecc00640 1 -- 172.21.15.83:0/2765188149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e806d840 con 0x7fd3e8143190 2024-04-25T00:11:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7fd3ecc00640 1 -- 172.21.15.83:0/2765188149 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd3e8164c80 con 0x7fd3e806ecc0 2024-04-25T00:11:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7fd3e7e00640 1 -- 172.21.15.83:0/2765188149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd3e806ecc0 msgr2=0x7fd3e8159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7fd3e7e00640 1 --2- 172.21.15.83:0/2765188149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd3e806ecc0 0x7fd3e8159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7fd3e7e00640 1 -- 172.21.15.83:0/2765188149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e8072450 msgr2=0x7fd3e8140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7fd3e7e00640 1 --2- 172.21.15.83:0/2765188149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e8072450 0x7fd3e8140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7fd3e7e00640 1 -- 172.21.15.83:0/2765188149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e8166250 con 0x7fd3e8143190 2024-04-25T00:11:28.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f12af400640 1 -- 172.21.15.83:0/328977282 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f12a801a070 con 0x7f12b80ced80 2024-04-25T00:11:28.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7fd3e7e00640 1 --2- 172.21.15.83:0/2765188149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3e8143190 0x7fd3e8140fc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800e7d0 tx=0x7fd3d800ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:28.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16dfa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f16d806cfd0 0x7f16d81fb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16dfa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d806d3b0 0x7f16d815e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16dfa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d80cb640 0x7f16d815e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:28.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16dfa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8166890 con 0x7f16d80cb640 2024-04-25T00:11:28.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16dfa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8145970 con 0x7f16d806d3b0 2024-04-25T00:11:28.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16dfa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f16d8166a10 con 0x7f16d806cfd0 2024-04-25T00:11:28.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f6fdca00640 1 Processor -- start 2024-04-25T00:11:28.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f6fdca00640 1 -- start start 2024-04-25T00:11:28.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d806d3b0 0x7f16d815e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:28.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d806d3b0 0x7f16d815e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:39250/0 (socket says 172.21.15.83:39250) 2024-04-25T00:11:29.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16d7e00640 1 -- 172.21.15.83:0/1217491724 learned_addr learned my addr 172.21.15.83:0/1217491724 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:29.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7fd3df400640 1 -- 172.21.15.83:0/2765188149 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8017070 con 0x7fd3e8143190 2024-04-25T00:11:29.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7fd3df400640 1 -- 172.21.15.83:0/2765188149 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d800c940 con 0x7fd3e8143190 2024-04-25T00:11:29.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7fd3df400640 1 -- 172.21.15.83:0/2765188149 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d800cd30 con 0x7fd3e8143190 2024-04-25T00:11:29.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7fb233400640 1 --2- 172.21.15.83:0/1698638165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22c0e7700 0x7fb22c0072d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0eb1f0 tx=0x7fb22001ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64a845453454cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f12af400640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1298067910 0x7f1298069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16dd000640 1 --2- 172.21.15.83:0/1217491724 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f16d806cfd0 0x7f16d81fb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16d7e00640 1 -- 172.21.15.83:0/1217491724 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f16d806cfd0 msgr2=0x7f16d81fb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16d7e00640 1 --2- 172.21.15.83:0/1217491724 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f16d806cfd0 0x7f16d81fb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16d7e00640 1 -- 172.21.15.83:0/1217491724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d80cb640 msgr2=0x7f16d815e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16d7e00640 1 --2- 172.21.15.83:0/1217491724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d80cb640 0x7f16d815e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.988+0000 7f16d7e00640 1 -- 172.21.15.83:0/1217491724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d80b2bf0 con 0x7f16d806d3b0 2024-04-25T00:11:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7fd3ecc00640 1 -- 172.21.15.83:0/2765188149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e8152b30 con 0x7fd3e8143190 2024-04-25T00:11:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7fd3ecc00640 1 -- 172.21.15.83:0/2765188149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e8165f70 con 0x7fd3e8143190 2024-04-25T00:11:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f12af400640 1 -- 172.21.15.83:0/328977282 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f12a8085950 con 0x7f12b80ced80 2024-04-25T00:11:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f12b7400640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1298067910 0x7f1298069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f6fdca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd8073410 0x7f6fd8158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f16d7e00640 1 --2- 172.21.15.83:0/1217491724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d806d3b0 0x7f16d815e340 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00c480 tx=0x7f16cc00c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f16d4c00640 1 -- 172.21.15.83:0/1217491724 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc012830 con 0x7f16d806d3b0 2024-04-25T00:11:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f16d4c00640 1 -- 172.21.15.83:0/1217491724 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16cc017070 con 0x7f16d806d3b0 2024-04-25T00:11:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f16d4c00640 1 -- 172.21.15.83:0/1217491724 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc012c30 con 0x7f16d806d3b0 2024-04-25T00:11:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f16dfa00640 1 -- 172.21.15.83:0/1217491724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d80b2e90 con 0x7f16d806d3b0 2024-04-25T00:11:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7fb232000640 1 -- 172.21.15.83:0/1698638165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2200b5070 con 0x7fb22c0e7700 2024-04-25T00:11:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f6fdca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd8162700 0x7f6fd80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f6fdca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd80c9950 0x7f6fd815d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f6fdca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd809c700 con 0x7f6fd8073410 2024-04-25T00:11:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f16dfa00640 1 -- 172.21.15.83:0/1217491724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d80b3450 con 0x7f16d806d3b0 2024-04-25T00:11:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7fb232000640 1 -- 172.21.15.83:0/1698638165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2200a8830 con 0x7fb22c0e7700 2024-04-25T00:11:29.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7fb232000640 1 -- 172.21.15.83:0/1698638165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2200a8c00 con 0x7fb22c0e7700 2024-04-25T00:11:29.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f6fdca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd809c580 con 0x7f6fd8162700 2024-04-25T00:11:29.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f6fd7e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd80c9950 0x7f6fd815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f16dfa00640 1 -- 172.21.15.83:0/1217491724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b40052a0 con 0x7f16d806d3b0 2024-04-25T00:11:29.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f6fd7e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd80c9950 0x7f6fd815d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:50084/0 (socket says 172.21.15.83:50084) 2024-04-25T00:11:29.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f6fd7e00640 1 -- 172.21.15.83:0/934567357 learned_addr learned my addr 172.21.15.83:0/934567357 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:29.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f6fdca00640 1 -- 172.21.15.83:0/934567357 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd80b7dd0 con 0x7f6fd80c9950 2024-04-25T00:11:29.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f6fd6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd8162700 0x7f6fd80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7fd3ecc00640 1 -- 172.21.15.83:0/2765188149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e804f510 con 0x7fd3e8143190 2024-04-25T00:11:29.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f6fd7e00640 1 -- 172.21.15.83:0/934567357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd8162700 msgr2=0x7f6fd80b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f6fd7e00640 1 --2- 172.21.15.83:0/934567357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd8162700 0x7f6fd80b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f6fd7e00640 1 -- 172.21.15.83:0/934567357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd8073410 msgr2=0x7f6fd8158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.992+0000 7f6fd7e00640 1 --2- 172.21.15.83:0/934567357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd8073410 0x7f6fd8158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7f6fd7e00640 1 -- 172.21.15.83:0/934567357 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd80b53e0 con 0x7f6fd80c9950 2024-04-25T00:11:29.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7f6fd7e00640 1 --2- 172.21.15.83:0/934567357 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd80c9950 0x7f6fd815d190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0007dc0 tx=0x7f6fd000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7f6fcf400640 1 -- 172.21.15.83:0/934567357 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0014420 con 0x7f6fd80c9950 2024-04-25T00:11:29.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7f6fcf400640 1 -- 172.21.15.83:0/934567357 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fd0019070 con 0x7f6fd80c9950 2024-04-25T00:11:29.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7f6fcf400640 1 -- 172.21.15.83:0/934567357 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0014820 con 0x7f6fd80c9950 2024-04-25T00:11:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7f6fdca00640 1 -- 172.21.15.83:0/934567357 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd8159270 con 0x7f6fd80c9950 2024-04-25T00:11:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7f12b7400640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1298067910 0x7f1298069dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f12a4006f50 tx=0x7f12a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7f12af400640 1 -- 172.21.15.83:0/328977282 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f12a8055be0 con 0x7f12b80ced80 2024-04-25T00:11:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7fb23a800640 1 -- 172.21.15.83:0/1698638165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22c0e7700 msgr2=0x7fb22c0072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7fb23a800640 1 --2- 172.21.15.83:0/1698638165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22c0e7700 0x7fb22c0072d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0eb1f0 tx=0x7fb22001ec30 comp rx=0 tx=0).stop 2024-04-25T00:11:29.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7f6fdca00640 1 -- 172.21.15.83:0/934567357 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd8159830 con 0x7f6fd80c9950 2024-04-25T00:11:29.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7f16d4c00640 1 -- 172.21.15.83:0/1217491724 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f16cc01c460 con 0x7f16d806d3b0 2024-04-25T00:11:29.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7fb23a800640 1 -- 172.21.15.83:0/1698638165 shutdown_connections 2024-04-25T00:11:29.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7fb23a800640 1 --2- 172.21.15.83:0/1698638165 >> v2:172.21.15.86:3300/0 conn(0x7fb22c047d70 0x7fb22c037ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7fb23a800640 1 --2- 172.21.15.83:0/1698638165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22c0e7700 0x7fb22c0072d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7fb23a800640 1 -- 172.21.15.83:0/1698638165 >> 172.21.15.83:0/1698638165 conn(0x7fb22c054b00 msgr2=0x7fb22c054f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7fb23a800640 1 -- 172.21.15.83:0/1698638165 shutdown_connections 2024-04-25T00:11:29.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7fb23a800640 1 -- 172.21.15.83:0/1698638165 wait complete. 2024-04-25T00:11:29.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7fd3df400640 1 -- 172.21.15.83:0/2765188149 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd3d801a070 con 0x7fd3e8143190 2024-04-25T00:11:29.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7f16d4c00640 1 --2- 172.21.15.83:0/1217491724 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16bc0678c0 0x7f16bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:28.996+0000 7f16d4c00640 1 -- 172.21.15.83:0/1217491724 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f16cc085d10 con 0x7f16d806d3b0 2024-04-25T00:11:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.000+0000 7fd3df400640 1 --2- 172.21.15.83:0/2765188149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd3c80678c0 0x7fd3c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.000+0000 7fd3df400640 1 -- 172.21.15.83:0/2765188149 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd3d8085ea0 con 0x7fd3e8143190 2024-04-25T00:11:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.000+0000 7fb23a800640 1 Processor -- start 2024-04-25T00:11:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.000+0000 7f6fcf400640 1 -- 172.21.15.83:0/934567357 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6fd00149c0 con 0x7f6fd80c9950 2024-04-25T00:11:29.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.000+0000 7fd3e7400640 1 --2- 172.21.15.83:0/2765188149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd3c80678c0 0x7fd3c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.000+0000 7fb23a800640 1 -- start start 2024-04-25T00:11:29.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.000+0000 7fd3e7400640 1 --2- 172.21.15.83:0/2765188149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd3c80678c0 0x7fd3c8069d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd3e00af000 tx=0x7fd3e00a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.000+0000 7f6fcf400640 1 --2- 172.21.15.83:0/934567357 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fb8067940 0x7f6fb8069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.000+0000 7f6fd7400640 1 --2- 172.21.15.83:0/934567357 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fb8067940 0x7f6fb8069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.000+0000 7f16d4c00640 1 -- 172.21.15.83:0/1217491724 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f16cc08b020 con 0x7f16d806d3b0 2024-04-25T00:11:29.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb23a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22c0381e0 0x7fb22c0db4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7f6fdca00640 1 -- 172.21.15.83:0/934567357 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb00052a0 con 0x7f6fd80c9950 2024-04-25T00:11:29.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb23a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22c047d70 0x7fb22c181750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7f6fd7400640 1 --2- 172.21.15.83:0/934567357 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fb8067940 0x7f6fb8069e00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80096b0 tx=0x7f6fc8006be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb23a800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb22c0e7700 0x7fb22c0ea020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb23a800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c050160 con 0x7fb22c047d70 2024-04-25T00:11:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb23a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c0e7ae0 con 0x7fb22c0381e0 2024-04-25T00:11:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb23a800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb22c038610 con 0x7fb22c0e7700 2024-04-25T00:11:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb233e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb22c0e7700 0x7fb22c0ea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb233e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb22c0e7700 0x7fb22c0ea020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:50098/0 (socket says 172.21.15.83:50098) 2024-04-25T00:11:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb233e00640 1 -- 172.21.15.83:0/433791047 learned_addr learned my addr 172.21.15.83:0/433791047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:29.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7f16dd000640 1 --2- 172.21.15.83:0/1217491724 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16bc0678c0 0x7f16bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb232a00640 1 --2- 172.21.15.83:0/433791047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22c047d70 0x7fb22c181750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7f6fcf400640 1 -- 172.21.15.83:0/934567357 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6fd00866a0 con 0x7f6fd80c9950 2024-04-25T00:11:29.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb233e00640 1 -- 172.21.15.83:0/433791047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22c0381e0 msgr2=0x7fb22c0db4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb233e00640 1 --2- 172.21.15.83:0/433791047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22c0381e0 0x7fb22c0db4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb233e00640 1 -- 172.21.15.83:0/433791047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22c047d70 msgr2=0x7fb22c181750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb233e00640 1 --2- 172.21.15.83:0/433791047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22c047d70 0x7fb22c181750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb233e00640 1 -- 172.21.15.83:0/433791047 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22c0dbb90 con 0x7fb22c0e7700 2024-04-25T00:11:29.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fb233e00640 1 --2- 172.21.15.83:0/433791047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb22c0e7700 0x7fb22c0ea020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb22800c3c0 tx=0x7fb22800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7f16dd000640 1 --2- 172.21.15.83:0/1217491724 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16bc0678c0 0x7f16bc069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f16c80b2000 tx=0x7f16c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.004+0000 7fd3df400640 1 -- 172.21.15.83:0/2765188149 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd3d8056040 con 0x7fd3e8143190 2024-04-25T00:11:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.008+0000 7f6fcf400640 1 -- 172.21.15.83:0/934567357 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6fd80b53e0 con 0x7f6fd80c9950 2024-04-25T00:11:29.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.008+0000 7fb227400640 1 -- 172.21.15.83:0/433791047 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228011790 con 0x7fb22c0e7700 2024-04-25T00:11:29.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.008+0000 7fb227400640 1 -- 172.21.15.83:0/433791047 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb228017070 con 0x7fb22c0e7700 2024-04-25T00:11:29.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.008+0000 7fb227400640 1 -- 172.21.15.83:0/433791047 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228011b90 con 0x7fb22c0e7700 2024-04-25T00:11:29.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.008+0000 7fb23a800640 1 -- 172.21.15.83:0/433791047 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22c0dbe70 con 0x7fb22c0e7700 2024-04-25T00:11:29.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.008+0000 7fb23a800640 1 -- 172.21.15.83:0/433791047 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22c0dc430 con 0x7fb22c0e7700 2024-04-25T00:11:29.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.012+0000 7fb23a800640 1 -- 172.21.15.83:0/433791047 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2100052a0 con 0x7fb22c0e7700 2024-04-25T00:11:29.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.012+0000 7fb227400640 1 -- 172.21.15.83:0/433791047 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb228011d30 con 0x7fb22c0e7700 2024-04-25T00:11:29.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.012+0000 7fb227400640 1 --2- 172.21.15.83:0/433791047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb214067990 0x7fb214069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.016+0000 7fb227400640 1 -- 172.21.15.83:0/433791047 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb228085d40 con 0x7fb22c0e7700 2024-04-25T00:11:29.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.016+0000 7fb233400640 1 --2- 172.21.15.83:0/433791047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb214067990 0x7fb214069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.016+0000 7fb233400640 1 --2- 172.21.15.83:0/433791047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb214067990 0x7fb214069e50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb2200b2000 tx=0x7fb220004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.016+0000 7fb227400640 1 -- 172.21.15.83:0/433791047 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb2280565b0 con 0x7fb22c0e7700 2024-04-25T00:11:29.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.244+0000 7f7f2da00640 1 -- 172.21.15.83:0/1222804093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7f20052c90 con 0x7f7f200e01b0 2024-04-25T00:11:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.248+0000 7f7f1ea00640 1 -- 172.21.15.83:0/1222804093 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7f280d53c0 con 0x7f7f200e01b0 2024-04-25T00:11:29.252 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2024-04-25T00:11:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.252+0000 7f7f2da00640 1 -- 172.21.15.83:0/1222804093 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7f080678c0 msgr2=0x7f7f08069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.252+0000 7f7f2da00640 1 --2- 172.21.15.83:0/1222804093 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7f080678c0 0x7f7f08069d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7f18001df0 tx=0x7f7f180010b0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.252+0000 7f7f2da00640 1 -- 172.21.15.83:0/1222804093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f200e01b0 msgr2=0x7f7f200d83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.252+0000 7f7f2da00640 1 --2- 172.21.15.83:0/1222804093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f200e01b0 0x7f7f200d83e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7f28000b70 tx=0x7f7f28074200 comp rx=0 tx=0).stop 2024-04-25T00:11:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.252+0000 7f7f2da00640 1 -- 172.21.15.83:0/1222804093 shutdown_connections 2024-04-25T00:11:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.252+0000 7f7f2da00640 1 --2- 172.21.15.83:0/1222804093 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7f080678c0 0x7f7f08069d80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.252+0000 7f7f2da00640 1 --2- 172.21.15.83:0/1222804093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f200e2d80 0x7f7f200e3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.252+0000 7f7f2da00640 1 --2- 172.21.15.83:0/1222804093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f200e01b0 0x7f7f200d83e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.252+0000 7f7f2da00640 1 --2- 172.21.15.83:0/1222804093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f2000c0e0 0x7f7f2004ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.252+0000 7f7f2da00640 1 -- 172.21.15.83:0/1222804093 >> 172.21.15.83:0/1222804093 conn(0x7f7f20054b00 msgr2=0x7f7f200e08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.252+0000 7f7f2da00640 1 -- 172.21.15.83:0/1222804093 shutdown_connections 2024-04-25T00:11:29.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.252+0000 7f7f2da00640 1 -- 172.21.15.83:0/1222804093 wait complete. 2024-04-25T00:11:29.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.256+0000 7f2814e00640 1 -- 172.21.15.83:0/3229521031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f27e8005590 con 0x7f2808033e40 2024-04-25T00:11:29.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.256+0000 7f2806000640 1 -- 172.21.15.83:0/3229521031 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f28100d1350 con 0x7f2808033e40 2024-04-25T00:11:29.260 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2024-04-25T00:11:29.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.260+0000 7f2814e00640 1 -- 172.21.15.83:0/3229521031 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27f0067ac0 msgr2=0x7f27f0069f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.260+0000 7f2814e00640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27f0067ac0 0x7f27f0069f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f28000b2000 tx=0x7f28000aa070 comp rx=0 tx=0).stop 2024-04-25T00:11:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.260+0000 7f2814e00640 1 -- 172.21.15.83:0/3229521031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2808033e40 msgr2=0x7f28080dc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.260+0000 7f2814e00640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2808033e40 0x7f28080dc1f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f281002d710 tx=0x7f2810068d10 comp rx=0 tx=0).stop 2024-04-25T00:11:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.264+0000 7f2814e00640 1 -- 172.21.15.83:0/3229521031 shutdown_connections 2024-04-25T00:11:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.264+0000 7f2814e00640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27f0067ac0 0x7f27f0069f80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.264+0000 7f2814e00640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28080ebc00 0x7f2808034f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.264+0000 7f2814e00640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2808049de0 0x7f28080e9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.264+0000 7f2814e00640 1 --2- 172.21.15.83:0/3229521031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2808033e40 0x7f28080dc1f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.264+0000 7f2814e00640 1 -- 172.21.15.83:0/3229521031 >> 172.21.15.83:0/3229521031 conn(0x7f2808009fc0 msgr2=0x7f28080056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.264+0000 7f4270200640 1 -- 172.21.15.83:0/636533191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4260052bd0 con 0x7f4260053090 2024-04-25T00:11:29.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.264+0000 7f4265600640 1 -- 172.21.15.83:0/636533191 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f425c015fb0 con 0x7f4260053090 2024-04-25T00:11:29.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.264+0000 7f2814e00640 1 -- 172.21.15.83:0/3229521031 shutdown_connections 2024-04-25T00:11:29.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.264+0000 7f2814e00640 1 -- 172.21.15.83:0/3229521031 wait complete. 2024-04-25T00:11:29.271 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2024-04-25T00:11:29.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.268+0000 7f4270200640 1 -- 172.21.15.83:0/636533191 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4248067990 msgr2=0x7f4248069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.268+0000 7f4270200640 1 --2- 172.21.15.83:0/636533191 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4248067990 0x7f4248069e50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f42680680c0 tx=0x7f426808d040 comp rx=0 tx=0).stop 2024-04-25T00:11:29.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.268+0000 7f4270200640 1 -- 172.21.15.83:0/636533191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4260053090 msgr2=0x7f426004cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.268+0000 7f4270200640 1 --2- 172.21.15.83:0/636533191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4260053090 0x7f426004cbf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f425c0ab000 tx=0x7f425c0190d0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.272+0000 7f4270200640 1 -- 172.21.15.83:0/636533191 shutdown_connections 2024-04-25T00:11:29.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.272+0000 7f4270200640 1 --2- 172.21.15.83:0/636533191 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4248067990 0x7f4248069e50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.272+0000 7f4270200640 1 --2- 172.21.15.83:0/636533191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42600df290 0x7f42600e1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.272+0000 7f4270200640 1 --2- 172.21.15.83:0/636533191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4260053090 0x7f426004cbf0 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f425c0ab000 tx=0x7f425c0190d0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.272+0000 7f4270200640 1 --2- 172.21.15.83:0/636533191 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426000aae0 0x7f42600d38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.272+0000 7f4270200640 1 -- 172.21.15.83:0/636533191 >> 172.21.15.83:0/636533191 conn(0x7f4260009fc0 msgr2=0x7f426004fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.272+0000 7f4270200640 1 -- 172.21.15.83:0/636533191 shutdown_connections 2024-04-25T00:11:29.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.272+0000 7f4270200640 1 -- 172.21.15.83:0/636533191 wait complete. 2024-04-25T00:11:29.293 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2024-04-25T00:11:29.294 DEBUG:teuthology.parallel:result is None 2024-04-25T00:11:29.303 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2024-04-25T00:11:29.304 DEBUG:teuthology.parallel:result is None 2024-04-25T00:11:29.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-04-25T00:11:29.308 DEBUG:teuthology.parallel:result is None 2024-04-25T00:11:29.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.312+0000 7f16dfa00640 1 -- 172.21.15.83:0/1217491724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f16b4005590 con 0x7f16d806d3b0 2024-04-25T00:11:29.317 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2024-04-25T00:11:29.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.316+0000 7f16d4c00640 1 -- 172.21.15.83:0/1217491724 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f16cc056500 con 0x7f16d806d3b0 2024-04-25T00:11:29.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.316+0000 7f16dfa00640 1 -- 172.21.15.83:0/1217491724 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16bc0678c0 msgr2=0x7f16bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.316+0000 7f16dfa00640 1 --2- 172.21.15.83:0/1217491724 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16bc0678c0 0x7f16bc069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f16c80b2000 tx=0x7f16c8008040 comp rx=0 tx=0).stop 2024-04-25T00:11:29.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.316+0000 7f16dfa00640 1 -- 172.21.15.83:0/1217491724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d806d3b0 msgr2=0x7f16d815e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.316+0000 7f16dfa00640 1 --2- 172.21.15.83:0/1217491724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d806d3b0 0x7f16d815e340 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00c480 tx=0x7f16cc00c940 comp rx=0 tx=0).stop 2024-04-25T00:11:29.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.316+0000 7f16dfa00640 1 -- 172.21.15.83:0/1217491724 shutdown_connections 2024-04-25T00:11:29.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.316+0000 7f16dfa00640 1 --2- 172.21.15.83:0/1217491724 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16bc0678c0 0x7f16bc069d80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.316+0000 7f16dfa00640 1 --2- 172.21.15.83:0/1217491724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d80cb640 0x7f16d815e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.316+0000 7f16dfa00640 1 --2- 172.21.15.83:0/1217491724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d806d3b0 0x7f16d815e340 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.316+0000 7f16dfa00640 1 --2- 172.21.15.83:0/1217491724 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f16d806cfd0 0x7f16d81fb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.316+0000 7f16dfa00640 1 -- 172.21.15.83:0/1217491724 >> 172.21.15.83:0/1217491724 conn(0x7f16d80cfa60 msgr2=0x7f16d80cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.316+0000 7f16dfa00640 1 -- 172.21.15.83:0/1217491724 shutdown_connections 2024-04-25T00:11:29.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.316+0000 7f16dfa00640 1 -- 172.21.15.83:0/1217491724 wait complete. 2024-04-25T00:11:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.324+0000 7fd3ecc00640 1 -- 172.21.15.83:0/2765188149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd3e80b3c90 con 0x7fd3e8143190 2024-04-25T00:11:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.324+0000 7fd3df400640 1 -- 172.21.15.83:0/2765188149 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd3d805a340 con 0x7fd3e8143190 2024-04-25T00:11:29.327 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2024-04-25T00:11:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.328+0000 7fd3ecc00640 1 -- 172.21.15.83:0/2765188149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd3c80678c0 msgr2=0x7fd3c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.328+0000 7fd3ecc00640 1 --2- 172.21.15.83:0/2765188149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd3c80678c0 0x7fd3c8069d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd3e00af000 tx=0x7fd3e00a7070 comp rx=0 tx=0).stop 2024-04-25T00:11:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.328+0000 7fd3ecc00640 1 -- 172.21.15.83:0/2765188149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3e8143190 msgr2=0x7fd3e8140fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.328+0000 7fd3ecc00640 1 --2- 172.21.15.83:0/2765188149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3e8143190 0x7fd3e8140fc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800e7d0 tx=0x7fd3d800ec90 comp rx=0 tx=0).stop 2024-04-25T00:11:29.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.328+0000 7fb23a800640 1 -- 172.21.15.83:0/433791047 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb210005590 con 0x7fb22c0e7700 2024-04-25T00:11:29.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.328+0000 7fd3ecc00640 1 -- 172.21.15.83:0/2765188149 shutdown_connections 2024-04-25T00:11:29.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.328+0000 7fd3ecc00640 1 --2- 172.21.15.83:0/2765188149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd3c80678c0 0x7fd3c8069d80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.328+0000 7fd3ecc00640 1 --2- 172.21.15.83:0/2765188149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3e8143190 0x7fd3e8140fc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.328+0000 7fd3ecc00640 1 --2- 172.21.15.83:0/2765188149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e8072450 0x7fd3e8140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.328+0000 7fd3ecc00640 1 --2- 172.21.15.83:0/2765188149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd3e806ecc0 0x7fd3e8159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.328+0000 7fd3ecc00640 1 -- 172.21.15.83:0/2765188149 >> 172.21.15.83:0/2765188149 conn(0x7fd3e806b3b0 msgr2=0x7fd3e806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.328+0000 7fd3ecc00640 1 -- 172.21.15.83:0/2765188149 shutdown_connections 2024-04-25T00:11:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.328+0000 7fd3ecc00640 1 -- 172.21.15.83:0/2765188149 wait complete. 2024-04-25T00:11:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.328+0000 7fb227400640 1 -- 172.21.15.83:0/433791047 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb228055eb0 con 0x7fb22c0e7700 2024-04-25T00:11:29.333 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2024-04-25T00:11:29.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.332+0000 7fb23a800640 1 -- 172.21.15.83:0/433791047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb214067990 msgr2=0x7fb214069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.332+0000 7fb23a800640 1 --2- 172.21.15.83:0/433791047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb214067990 0x7fb214069e50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb2200b2000 tx=0x7fb220004740 comp rx=0 tx=0).stop 2024-04-25T00:11:29.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.332+0000 7fb23a800640 1 -- 172.21.15.83:0/433791047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb22c0e7700 msgr2=0x7fb22c0ea020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.332+0000 7fb23a800640 1 --2- 172.21.15.83:0/433791047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb22c0e7700 0x7fb22c0ea020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb22800c3c0 tx=0x7fb22800c880 comp rx=0 tx=0).stop 2024-04-25T00:11:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.332+0000 7fb23a800640 1 -- 172.21.15.83:0/433791047 shutdown_connections 2024-04-25T00:11:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.332+0000 7fb23a800640 1 --2- 172.21.15.83:0/433791047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb214067990 0x7fb214069e50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.332+0000 7fb23a800640 1 --2- 172.21.15.83:0/433791047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb22c0e7700 0x7fb22c0ea020 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.332+0000 7fb23a800640 1 --2- 172.21.15.83:0/433791047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22c047d70 0x7fb22c181750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.332+0000 7fb23a800640 1 --2- 172.21.15.83:0/433791047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22c0381e0 0x7fb22c0db4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.332+0000 7fb23a800640 1 -- 172.21.15.83:0/433791047 >> 172.21.15.83:0/433791047 conn(0x7fb22c054b00 msgr2=0x7fb22c038a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.332+0000 7fb23a800640 1 -- 172.21.15.83:0/433791047 shutdown_connections 2024-04-25T00:11:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.332+0000 7fb23a800640 1 -- 172.21.15.83:0/433791047 wait complete. 2024-04-25T00:11:29.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.340+0000 7f6fdca00640 1 -- 172.21.15.83:0/934567357 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6fb0005860 con 0x7f6fd80c9950 2024-04-25T00:11:29.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.340+0000 7f12be600640 1 -- 172.21.15.83:0/328977282 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1290005860 con 0x7f12b80ced80 2024-04-25T00:11:29.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.340+0000 7f6fcf400640 1 -- 172.21.15.83:0/934567357 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6fd0056ef0 con 0x7f6fd80c9950 2024-04-25T00:11:29.344 INFO:teuthology.orchestra.run.smithi083.stdout:55834574858 2024-04-25T00:11:29.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.340+0000 7f12af400640 1 -- 172.21.15.83:0/328977282 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f12a8059ee0 con 0x7f12b80ced80 2024-04-25T00:11:29.345 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2024-04-25T00:11:29.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-04-25T00:11:29.346 DEBUG:teuthology.parallel:result is None 2024-04-25T00:11:29.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f12be600640 1 -- 172.21.15.83:0/328977282 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1298067910 msgr2=0x7f1298069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f12be600640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1298067910 0x7f1298069dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f12a4006f50 tx=0x7f12a4008040 comp rx=0 tx=0).stop 2024-04-25T00:11:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f12be600640 1 -- 172.21.15.83:0/328977282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12b80ced80 msgr2=0x7f12b80b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f12be600640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12b80ced80 0x7f12b80b45f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f12a800e810 tx=0x7f12a800ecd0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f12be600640 1 -- 172.21.15.83:0/328977282 shutdown_connections 2024-04-25T00:11:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f12be600640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1298067910 0x7f1298069dd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f12be600640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12b80ced80 0x7f12b80b45f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f12be600640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b80d0580 0x7f12b815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f12be600640 1 --2- 172.21.15.83:0/328977282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b80ce920 0x7f12b813cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f12be600640 1 -- 172.21.15.83:0/328977282 >> 172.21.15.83:0/328977282 conn(0x7f12b80cfa60 msgr2=0x7f12b815cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f12be600640 1 -- 172.21.15.83:0/328977282 shutdown_connections 2024-04-25T00:11:29.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f12be600640 1 -- 172.21.15.83:0/328977282 wait complete. 2024-04-25T00:11:29.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f6fdca00640 1 -- 172.21.15.83:0/934567357 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fb8067940 msgr2=0x7f6fb8069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f6fdca00640 1 --2- 172.21.15.83:0/934567357 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fb8067940 0x7f6fb8069e00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80096b0 tx=0x7f6fc8006be0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f6fdca00640 1 -- 172.21.15.83:0/934567357 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd80c9950 msgr2=0x7f6fd815d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f6fdca00640 1 --2- 172.21.15.83:0/934567357 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd80c9950 0x7f6fd815d190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0007dc0 tx=0x7f6fd000e4b0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f6fdca00640 1 -- 172.21.15.83:0/934567357 shutdown_connections 2024-04-25T00:11:29.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f6fdca00640 1 --2- 172.21.15.83:0/934567357 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fb8067940 0x7f6fb8069e00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f6fdca00640 1 --2- 172.21.15.83:0/934567357 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd80c9950 0x7f6fd815d190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f6fdca00640 1 --2- 172.21.15.83:0/934567357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd8162700 0x7f6fd80b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f6fdca00640 1 --2- 172.21.15.83:0/934567357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd8073410 0x7f6fd8158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f6fdca00640 1 -- 172.21.15.83:0/934567357 >> 172.21.15.83:0/934567357 conn(0x7f6fd80d08d0 msgr2=0x7f6fd80d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f6fdca00640 1 -- 172.21.15.83:0/934567357 shutdown_connections 2024-04-25T00:11:29.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.344+0000 7f6fdca00640 1 -- 172.21.15.83:0/934567357 wait complete. 2024-04-25T00:11:29.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-04-25T00:11:29.359 DEBUG:teuthology.parallel:result is None 2024-04-25T00:11:29.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2024-04-25T00:11:29.360 DEBUG:teuthology.parallel:result is None 2024-04-25T00:11:29.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-04-25T00:11:29.377 DEBUG:teuthology.parallel:result is None 2024-04-25T00:11:29.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2024-04-25T00:11:29.378 DEBUG:teuthology.parallel:result is None 2024-04-25T00:11:29.378 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T00:11:29.379 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:11:29.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.556+0000 7f0599400640 1 Processor -- start 2024-04-25T00:11:29.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.556+0000 7f0599400640 1 -- start start 2024-04-25T00:11:29.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.556+0000 7f0599400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05941599f0 0x7f05940cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0599400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f059415cec0 0x7f05940ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0599400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05940ccc60 con 0x7f059415c340 2024-04-25T00:11:29.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0599400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0594159dd0 con 0x7f059415cec0 2024-04-25T00:11:29.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0599400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f059406dba0 con 0x7f05941599f0 2024-04-25T00:11:29.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0593200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f059415cec0 0x7f05940ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0593200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f059415cec0 0x7f05940ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36646/0 (socket says 172.21.15.83:36646) 2024-04-25T00:11:29.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0593200640 1 -- 172.21.15.83:0/4230301838 learned_addr learned my addr 172.21.15.83:0/4230301838 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:29.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0592800640 1 --2- 172.21.15.83:0/4230301838 >> v2:172.21.15.86:3300/0 conn(0x7f05941599f0 0x7f05940cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0593200640 1 -- 172.21.15.83:0/4230301838 >> v2:172.21.15.86:3300/0 conn(0x7f05941599f0 msgr2=0x7f05940cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:29.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0593200640 1 --2- 172.21.15.83:0/4230301838 >> v2:172.21.15.86:3300/0 conn(0x7f05941599f0 0x7f05940cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0593200640 1 -- 172.21.15.83:0/4230301838 >> v1:172.21.15.86:6789/0 conn(0x7f059415c340 legacy=0x7f059415c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:29.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0593200640 1 -- 172.21.15.83:0/4230301838 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059406de80 con 0x7f059415cec0 2024-04-25T00:11:29.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0592800640 1 --2- 172.21.15.83:0/4230301838 >> v2:172.21.15.86:3300/0 conn(0x7f05941599f0 0x7f05940cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0593200640 1 --2- 172.21.15.83:0/4230301838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f059415cec0 0x7f05940ce6e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f058c009e50 tx=0x7f058c01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512c6c00b21cd381 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0591e00640 1 -- 172.21.15.83:0/4230301838 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c0ac020 con 0x7f059415cec0 2024-04-25T00:11:29.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0591e00640 1 -- 172.21.15.83:0/4230301838 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f058c0a6410 con 0x7f059415cec0 2024-04-25T00:11:29.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.560+0000 7f0591e00640 1 -- 172.21.15.83:0/4230301838 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c0a67e0 con 0x7f059415cec0 2024-04-25T00:11:29.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 -- 172.21.15.83:0/4230301838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f059415cec0 msgr2=0x7f05940ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 --2- 172.21.15.83:0/4230301838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f059415cec0 0x7f05940ce6e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f058c009e50 tx=0x7f058c01c740 comp rx=0 tx=0).stop 2024-04-25T00:11:29.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 -- 172.21.15.83:0/4230301838 shutdown_connections 2024-04-25T00:11:29.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 --2- 172.21.15.83:0/4230301838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f059415cec0 0x7f05940ce6e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 --2- 172.21.15.83:0/4230301838 >> v2:172.21.15.86:3300/0 conn(0x7f05941599f0 0x7f05940cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 -- 172.21.15.83:0/4230301838 >> 172.21.15.83:0/4230301838 conn(0x7f05940b1830 msgr2=0x7f05940b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 -- 172.21.15.83:0/4230301838 shutdown_connections 2024-04-25T00:11:29.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 -- 172.21.15.83:0/4230301838 wait complete. 2024-04-25T00:11:29.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 Processor -- start 2024-04-25T00:11:29.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 -- start start 2024-04-25T00:11:29.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05941599f0 0x7f059414d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f059415c340 0x7f059415b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f059415cec0 0x7f059414e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05940cf800 con 0x7f059415c340 2024-04-25T00:11:29.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05940cf680 con 0x7f05941599f0 2024-04-25T00:11:29.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0599400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0594168cd0 con 0x7f059415cec0 2024-04-25T00:11:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f058be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f059415c340 0x7f059415b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f058be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f059415c340 0x7f059415b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36654/0 (socket says 172.21.15.83:36654) 2024-04-25T00:11:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f058be00640 1 -- 172.21.15.83:0/4236067644 learned_addr learned my addr 172.21.15.83:0/4236067644 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0593200640 1 --2- 172.21.15.83:0/4236067644 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f059415cec0 0x7f059414e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f0592800640 1 --2- 172.21.15.83:0/4236067644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05941599f0 0x7f059414d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f058be00640 1 -- 172.21.15.83:0/4236067644 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f059415cec0 msgr2=0x7f059414e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f058be00640 1 --2- 172.21.15.83:0/4236067644 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f059415cec0 0x7f059414e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f058be00640 1 -- 172.21.15.83:0/4236067644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05941599f0 msgr2=0x7f059414d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f058be00640 1 --2- 172.21.15.83:0/4236067644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05941599f0 0x7f059414d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.564+0000 7f058be00640 1 -- 172.21.15.83:0/4236067644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059415bb30 con 0x7f059415c340 2024-04-25T00:11:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.568+0000 7f058be00640 1 --2- 172.21.15.83:0/4236067644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f059415c340 0x7f059415b0e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f057c00c450 tx=0x7f057c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.568+0000 7f058aa00640 1 -- 172.21.15.83:0/4236067644 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c012830 con 0x7f059415c340 2024-04-25T00:11:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.568+0000 7f058aa00640 1 -- 172.21.15.83:0/4236067644 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057c017070 con 0x7f059415c340 2024-04-25T00:11:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.568+0000 7f058aa00640 1 -- 172.21.15.83:0/4236067644 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c012c30 con 0x7f059415c340 2024-04-25T00:11:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.568+0000 7f0599400640 1 -- 172.21.15.83:0/4236067644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05940b3a10 con 0x7f059415c340 2024-04-25T00:11:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.568+0000 7f0599400640 1 -- 172.21.15.83:0/4236067644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059415be10 con 0x7f059415c340 2024-04-25T00:11:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.568+0000 7f0599400640 1 -- 172.21.15.83:0/4236067644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05940737f0 con 0x7f059415c340 2024-04-25T00:11:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.568+0000 7f058aa00640 1 -- 172.21.15.83:0/4236067644 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f057c01c460 con 0x7f059415c340 2024-04-25T00:11:29.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.568+0000 7f058aa00640 1 --2- 172.21.15.83:0/4236067644 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f05700678f0 0x7f0570069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.568+0000 7f058aa00640 1 -- 172.21.15.83:0/4236067644 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f057c085c10 con 0x7f059415c340 2024-04-25T00:11:29.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.568+0000 7f0592800640 1 --2- 172.21.15.83:0/4236067644 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f05700678f0 0x7f0570069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.572+0000 7f0592800640 1 --2- 172.21.15.83:0/4236067644 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f05700678f0 0x7f0570069db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f05940cc120 tx=0x7f0584008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.572+0000 7f058aa00640 1 -- 172.21.15.83:0/4236067644 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f057c055dc0 con 0x7f059415c340 2024-04-25T00:11:29.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.728+0000 7f0599400640 1 -- 172.21.15.83:0/4236067644 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f05940b1db0 con 0x7f05700678f0 2024-04-25T00:11:29.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.736+0000 7f058aa00640 1 -- 172.21.15.83:0/4236067644 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f05940b1db0 con 0x7f05700678f0 2024-04-25T00:11:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:29.743 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:11:29.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.740+0000 7f0599400640 1 -- 172.21.15.83:0/4236067644 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f05700678f0 msgr2=0x7f0570069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.740+0000 7f0599400640 1 --2- 172.21.15.83:0/4236067644 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f05700678f0 0x7f0570069db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f05940cc120 tx=0x7f0584008040 comp rx=0 tx=0).stop 2024-04-25T00:11:29.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.740+0000 7f0599400640 1 -- 172.21.15.83:0/4236067644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f059415c340 msgr2=0x7f059415b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.740+0000 7f0599400640 1 --2- 172.21.15.83:0/4236067644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f059415c340 0x7f059415b0e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f057c00c450 tx=0x7f057c00c910 comp rx=0 tx=0).stop 2024-04-25T00:11:29.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.740+0000 7f0599400640 1 -- 172.21.15.83:0/4236067644 shutdown_connections 2024-04-25T00:11:29.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.740+0000 7f0599400640 1 --2- 172.21.15.83:0/4236067644 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f05700678f0 0x7f0570069db0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.740+0000 7f0599400640 1 --2- 172.21.15.83:0/4236067644 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f059415cec0 0x7f059414e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.740+0000 7f0599400640 1 --2- 172.21.15.83:0/4236067644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f059415c340 0x7f059415b0e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.740+0000 7f0599400640 1 --2- 172.21.15.83:0/4236067644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05941599f0 0x7f059414d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.740+0000 7f0599400640 1 -- 172.21.15.83:0/4236067644 >> 172.21.15.83:0/4236067644 conn(0x7f05940b1830 msgr2=0x7f059406e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.740+0000 7f0599400640 1 -- 172.21.15.83:0/4236067644 shutdown_connections 2024-04-25T00:11:29.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.740+0000 7f0599400640 1 -- 172.21.15.83:0/4236067644 wait complete. 2024-04-25T00:11:29.761 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-25T00:11:28.447534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219404,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755316,"statfs":{"total":773094113280,"available":772869443584,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002456"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515374+0000","last_change":"2024-04-25T00:11:05.833868+0000","last_active":"2024-04-25T00:11:12.515374+0000","last_peered":"2024-04-25T00:11:12.515374+0000","last_clean":"2024-04-25T00:11:12.515374+0000","last_became_active":"2024-04-25T00:11:05.833603+0000","last_became_peered":"2024-04-25T00:11:05.833603+0000","last_unstale":"2024-04-25T00:11:12.515374+0000","last_undegraded":"2024-04-25T00:11:12.515374+0000","last_fullsized":"2024-04-25T00:11:12.515374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:32.084989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874560+0000","last_change":"2024-04-25T00:11:06.856161+0000","last_active":"2024-04-25T00:11:11.874560+0000","last_peered":"2024-04-25T00:11:11.874560+0000","last_clean":"2024-04-25T00:11:11.874560+0000","last_became_active":"2024-04-25T00:11:06.855897+0000","last_became_peered":"2024-04-25T00:11:06.855897+0000","last_unstale":"2024-04-25T00:11:11.874560+0000","last_undegraded":"2024-04-25T00:11:11.874560+0000","last_fullsized":"2024-04-25T00:11:11.874560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.219642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873904+0000","last_change":"2024-04-25T00:11:10.976976+0000","last_active":"2024-04-25T00:11:11.873904+0000","last_peered":"2024-04-25T00:11:11.873904+0000","last_clean":"2024-04-25T00:11:11.873904+0000","last_became_active":"2024-04-25T00:11:10.975609+0000","last_became_peered":"2024-04-25T00:11:10.975609+0000","last_unstale":"2024-04-25T00:11:11.873904+0000","last_undegraded":"2024-04-25T00:11:11.873904+0000","last_fullsized":"2024-04-25T00:11:11.873904+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:07.644687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874723+0000","last_change":"2024-04-25T00:11:05.835352+0000","last_active":"2024-04-25T00:11:11.874723+0000","last_peered":"2024-04-25T00:11:11.874723+0000","last_clean":"2024-04-25T00:11:11.874723+0000","last_became_active":"2024-04-25T00:11:05.835056+0000","last_became_peered":"2024-04-25T00:11:05.835056+0000","last_unstale":"2024-04-25T00:11:11.874723+0000","last_undegraded":"2024-04-25T00:11:11.874723+0000","last_fullsized":"2024-04-25T00:11:11.874723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515231+0000","last_change":"2024-04-25T00:11:06.855492+0000","last_active":"2024-04-25T00:11:12.515231+0000","last_peered":"2024-04-25T00:11:12.515231+0000","last_clean":"2024-04-25T00:11:12.515231+0000","last_became_active":"2024-04-25T00:11:06.854970+0000","last_became_peered":"2024-04-25T00:11:06.854970+0000","last_unstale":"2024-04-25T00:11:12.515231+0000","last_undegraded":"2024-04-25T00:11:12.515231+0000","last_fullsized":"2024-04-25T00:11:12.515231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515424+0000","last_change":"2024-04-25T00:11:10.899597+0000","last_active":"2024-04-25T00:11:12.515424+0000","last_peered":"2024-04-25T00:11:12.515424+0000","last_clean":"2024-04-25T00:11:12.515424+0000","last_became_active":"2024-04-25T00:11:10.899136+0000","last_became_peered":"2024-04-25T00:11:10.899136+0000","last_unstale":"2024-04-25T00:11:12.515424+0000","last_undegraded":"2024-04-25T00:11:12.515424+0000","last_fullsized":"2024-04-25T00:11:12.515424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:40.036652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514540+0000","last_change":"2024-04-25T00:11:05.848984+0000","last_active":"2024-04-25T00:11:12.514540+0000","last_peered":"2024-04-25T00:11:12.514540+0000","last_clean":"2024-04-25T00:11:12.514540+0000","last_became_active":"2024-04-25T00:11:05.848618+0000","last_became_peered":"2024-04-25T00:11:05.848618+0000","last_unstale":"2024-04-25T00:11:12.514540+0000","last_undegraded":"2024-04-25T00:11:12.514540+0000","last_fullsized":"2024-04-25T00:11:12.514540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:50.369863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514723+0000","last_change":"2024-04-25T00:11:06.875779+0000","last_active":"2024-04-25T00:11:12.514723+0000","last_peered":"2024-04-25T00:11:12.514723+0000","last_clean":"2024-04-25T00:11:12.514723+0000","last_became_active":"2024-04-25T00:11:06.875440+0000","last_became_peered":"2024-04-25T00:11:06.875440+0000","last_unstale":"2024-04-25T00:11:12.514723+0000","last_undegraded":"2024-04-25T00:11:12.514723+0000","last_fullsized":"2024-04-25T00:11:12.514723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:43.565714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516147+0000","last_change":"2024-04-25T00:11:10.899794+0000","last_active":"2024-04-25T00:11:12.516147+0000","last_peered":"2024-04-25T00:11:12.516147+0000","last_clean":"2024-04-25T00:11:12.516147+0000","last_became_active":"2024-04-25T00:11:10.899313+0000","last_became_peered":"2024-04-25T00:11:10.899313+0000","last_unstale":"2024-04-25T00:11:12.516147+0000","last_undegraded":"2024-04-25T00:11:12.516147+0000","last_fullsized":"2024-04-25T00:11:12.516147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:51.269552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514246+0000","last_change":"2024-04-25T00:11:05.847672+0000","last_active":"2024-04-25T00:11:12.514246+0000","last_peered":"2024-04-25T00:11:12.514246+0000","last_clean":"2024-04-25T00:11:12.514246+0000","last_became_active":"2024-04-25T00:11:05.847316+0000","last_became_peered":"2024-04-25T00:11:05.847316+0000","last_unstale":"2024-04-25T00:11:12.514246+0000","last_undegraded":"2024-04-25T00:11:12.514246+0000","last_fullsized":"2024-04-25T00:11:12.514246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515600+0000","last_change":"2024-04-25T00:11:06.860047+0000","last_active":"2024-04-25T00:11:12.515600+0000","last_peered":"2024-04-25T00:11:12.515600+0000","last_clean":"2024-04-25T00:11:12.515600+0000","last_became_active":"2024-04-25T00:11:06.859782+0000","last_became_peered":"2024-04-25T00:11:06.859782+0000","last_unstale":"2024-04-25T00:11:12.515600+0000","last_undegraded":"2024-04-25T00:11:12.515600+0000","last_fullsized":"2024-04-25T00:11:12.515600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:43.612611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516469+0000","last_change":"2024-04-25T00:11:10.895749+0000","last_active":"2024-04-25T00:11:12.516469+0000","last_peered":"2024-04-25T00:11:12.516469+0000","last_clean":"2024-04-25T00:11:12.516469+0000","last_became_active":"2024-04-25T00:11:10.895359+0000","last_became_peered":"2024-04-25T00:11:10.895359+0000","last_unstale":"2024-04-25T00:11:12.516469+0000","last_undegraded":"2024-04-25T00:11:12.516469+0000","last_fullsized":"2024-04-25T00:11:12.516469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873961+0000","last_change":"2024-04-25T00:11:06.865465+0000","last_active":"2024-04-25T00:11:11.873961+0000","last_peered":"2024-04-25T00:11:11.873961+0000","last_clean":"2024-04-25T00:11:11.873961+0000","last_became_active":"2024-04-25T00:11:06.865020+0000","last_became_peered":"2024-04-25T00:11:06.865020+0000","last_unstale":"2024-04-25T00:11:11.873961+0000","last_undegraded":"2024-04-25T00:11:11.873961+0000","last_fullsized":"2024-04-25T00:11:11.873961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514541+0000","last_change":"2024-04-25T00:11:05.844750+0000","last_active":"2024-04-25T00:11:12.514541+0000","last_peered":"2024-04-25T00:11:12.514541+0000","last_clean":"2024-04-25T00:11:12.514541+0000","last_became_active":"2024-04-25T00:11:05.844407+0000","last_became_peered":"2024-04-25T00:11:05.844407+0000","last_unstale":"2024-04-25T00:11:12.514541+0000","last_undegraded":"2024-04-25T00:11:12.514541+0000","last_fullsized":"2024-04-25T00:11:12.514541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:34.248155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516163+0000","last_change":"2024-04-25T00:11:10.893480+0000","last_active":"2024-04-25T00:11:12.516163+0000","last_peered":"2024-04-25T00:11:12.516163+0000","last_clean":"2024-04-25T00:11:12.516163+0000","last_became_active":"2024-04-25T00:11:10.893051+0000","last_became_peered":"2024-04-25T00:11:10.893051+0000","last_unstale":"2024-04-25T00:11:12.516163+0000","last_undegraded":"2024-04-25T00:11:12.516163+0000","last_fullsized":"2024-04-25T00:11:12.516163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:40.531377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874153+0000","last_change":"2024-04-25T00:11:06.845105+0000","last_active":"2024-04-25T00:11:11.874153+0000","last_peered":"2024-04-25T00:11:11.874153+0000","last_clean":"2024-04-25T00:11:11.874153+0000","last_became_active":"2024-04-25T00:11:06.844686+0000","last_became_peered":"2024-04-25T00:11:06.844686+0000","last_unstale":"2024-04-25T00:11:11.874153+0000","last_undegraded":"2024-04-25T00:11:11.874153+0000","last_fullsized":"2024-04-25T00:11:11.874153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:25.890860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514977+0000","last_change":"2024-04-25T00:11:05.836866+0000","last_active":"2024-04-25T00:11:12.514977+0000","last_peered":"2024-04-25T00:11:12.514977+0000","last_clean":"2024-04-25T00:11:12.514977+0000","last_became_active":"2024-04-25T00:11:05.836578+0000","last_became_peered":"2024-04-25T00:11:05.836578+0000","last_unstale":"2024-04-25T00:11:12.514977+0000","last_undegraded":"2024-04-25T00:11:12.514977+0000","last_fullsized":"2024-04-25T00:11:12.514977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515742+0000","last_change":"2024-04-25T00:11:10.903356+0000","last_active":"2024-04-25T00:11:12.515742+0000","last_peered":"2024-04-25T00:11:12.515742+0000","last_clean":"2024-04-25T00:11:12.515742+0000","last_became_active":"2024-04-25T00:11:10.902845+0000","last_became_peered":"2024-04-25T00:11:10.902845+0000","last_unstale":"2024-04-25T00:11:12.515742+0000","last_undegraded":"2024-04-25T00:11:12.515742+0000","last_fullsized":"2024-04-25T00:11:12.515742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:47.662521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874413+0000","last_change":"2024-04-25T00:11:06.849565+0000","last_active":"2024-04-25T00:11:11.874413+0000","last_peered":"2024-04-25T00:11:11.874413+0000","last_clean":"2024-04-25T00:11:11.874413+0000","last_became_active":"2024-04-25T00:11:06.849275+0000","last_became_peered":"2024-04-25T00:11:06.849275+0000","last_unstale":"2024-04-25T00:11:11.874413+0000","last_undegraded":"2024-04-25T00:11:11.874413+0000","last_fullsized":"2024-04-25T00:11:11.874413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514312+0000","last_change":"2024-04-25T00:11:05.850573+0000","last_active":"2024-04-25T00:11:12.514312+0000","last_peered":"2024-04-25T00:11:12.514312+0000","last_clean":"2024-04-25T00:11:12.514312+0000","last_became_active":"2024-04-25T00:11:05.850239+0000","last_became_peered":"2024-04-25T00:11:05.850239+0000","last_unstale":"2024-04-25T00:11:12.514312+0000","last_undegraded":"2024-04-25T00:11:12.514312+0000","last_fullsized":"2024-04-25T00:11:12.514312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:41.285583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515409+0000","last_change":"2024-04-25T00:11:10.974826+0000","last_active":"2024-04-25T00:11:12.515409+0000","last_peered":"2024-04-25T00:11:12.515409+0000","last_clean":"2024-04-25T00:11:12.515409+0000","last_became_active":"2024-04-25T00:11:10.974483+0000","last_became_peered":"2024-04-25T00:11:10.974483+0000","last_unstale":"2024-04-25T00:11:12.515409+0000","last_undegraded":"2024-04-25T00:11:12.515409+0000","last_fullsized":"2024-04-25T00:11:12.515409+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:13.028091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516075+0000","last_change":"2024-04-25T00:11:06.857094+0000","last_active":"2024-04-25T00:11:12.516075+0000","last_peered":"2024-04-25T00:11:12.516075+0000","last_clean":"2024-04-25T00:11:12.516075+0000","last_became_active":"2024-04-25T00:11:06.856782+0000","last_became_peered":"2024-04-25T00:11:06.856782+0000","last_unstale":"2024-04-25T00:11:12.516075+0000","last_undegraded":"2024-04-25T00:11:12.516075+0000","last_fullsized":"2024-04-25T00:11:12.516075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:10.105280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513838+0000","last_change":"2024-04-25T00:11:05.849504+0000","last_active":"2024-04-25T00:11:12.513838+0000","last_peered":"2024-04-25T00:11:12.513838+0000","last_clean":"2024-04-25T00:11:12.513838+0000","last_became_active":"2024-04-25T00:11:05.847651+0000","last_became_peered":"2024-04-25T00:11:05.847651+0000","last_unstale":"2024-04-25T00:11:12.513838+0000","last_undegraded":"2024-04-25T00:11:12.513838+0000","last_fullsized":"2024-04-25T00:11:12.513838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515751+0000","last_change":"2024-04-25T00:11:10.893589+0000","last_active":"2024-04-25T00:11:12.515751+0000","last_peered":"2024-04-25T00:11:12.515751+0000","last_clean":"2024-04-25T00:11:12.515751+0000","last_became_active":"2024-04-25T00:11:10.893172+0000","last_became_peered":"2024-04-25T00:11:10.893172+0000","last_unstale":"2024-04-25T00:11:12.515751+0000","last_undegraded":"2024-04-25T00:11:12.515751+0000","last_fullsized":"2024-04-25T00:11:12.515751+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:53.464531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514257+0000","last_change":"2024-04-25T00:11:06.850048+0000","last_active":"2024-04-25T00:11:12.514257+0000","last_peered":"2024-04-25T00:11:12.514257+0000","last_clean":"2024-04-25T00:11:12.514257+0000","last_became_active":"2024-04-25T00:11:06.849785+0000","last_became_peered":"2024-04-25T00:11:06.849785+0000","last_unstale":"2024-04-25T00:11:12.514257+0000","last_undegraded":"2024-04-25T00:11:12.514257+0000","last_fullsized":"2024-04-25T00:11:12.514257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874790+0000","last_change":"2024-04-25T00:11:05.836100+0000","last_active":"2024-04-25T00:11:11.874790+0000","last_peered":"2024-04-25T00:11:11.874790+0000","last_clean":"2024-04-25T00:11:11.874790+0000","last_became_active":"2024-04-25T00:11:05.835841+0000","last_became_peered":"2024-04-25T00:11:05.835841+0000","last_unstale":"2024-04-25T00:11:11.874790+0000","last_undegraded":"2024-04-25T00:11:11.874790+0000","last_fullsized":"2024-04-25T00:11:11.874790+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:53.690870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515708+0000","last_change":"2024-04-25T00:11:10.975785+0000","last_active":"2024-04-25T00:11:12.515708+0000","last_peered":"2024-04-25T00:11:12.515708+0000","last_clean":"2024-04-25T00:11:12.515708+0000","last_became_active":"2024-04-25T00:11:10.974244+0000","last_became_peered":"2024-04-25T00:11:10.974244+0000","last_unstale":"2024-04-25T00:11:12.515708+0000","last_undegraded":"2024-04-25T00:11:12.515708+0000","last_fullsized":"2024-04-25T00:11:12.515708+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:47.615624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874580+0000","last_change":"2024-04-25T00:11:06.854507+0000","last_active":"2024-04-25T00:11:11.874580+0000","last_peered":"2024-04-25T00:11:11.874580+0000","last_clean":"2024-04-25T00:11:11.874580+0000","last_became_active":"2024-04-25T00:11:06.853694+0000","last_became_peered":"2024-04-25T00:11:06.853694+0000","last_unstale":"2024-04-25T00:11:11.874580+0000","last_undegraded":"2024-04-25T00:11:11.874580+0000","last_fullsized":"2024-04-25T00:11:11.874580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:36.481467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515093+0000","last_change":"2024-04-25T00:11:05.842386+0000","last_active":"2024-04-25T00:11:12.515093+0000","last_peered":"2024-04-25T00:11:12.515093+0000","last_clean":"2024-04-25T00:11:12.515093+0000","last_became_active":"2024-04-25T00:11:05.842064+0000","last_became_peered":"2024-04-25T00:11:05.842064+0000","last_unstale":"2024-04-25T00:11:12.515093+0000","last_undegraded":"2024-04-25T00:11:12.515093+0000","last_fullsized":"2024-04-25T00:11:12.515093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873860+0000","last_change":"2024-04-25T00:11:10.969122+0000","last_active":"2024-04-25T00:11:11.873860+0000","last_peered":"2024-04-25T00:11:11.873860+0000","last_clean":"2024-04-25T00:11:11.873860+0000","last_became_active":"2024-04-25T00:11:10.968532+0000","last_became_peered":"2024-04-25T00:11:10.968532+0000","last_unstale":"2024-04-25T00:11:11.873860+0000","last_undegraded":"2024-04-25T00:11:11.873860+0000","last_fullsized":"2024-04-25T00:11:11.873860+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:23:46.222075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514772+0000","last_change":"2024-04-25T00:11:06.847752+0000","last_active":"2024-04-25T00:11:12.514772+0000","last_peered":"2024-04-25T00:11:12.514772+0000","last_clean":"2024-04-25T00:11:12.514772+0000","last_became_active":"2024-04-25T00:11:06.847378+0000","last_became_peered":"2024-04-25T00:11:06.847378+0000","last_unstale":"2024-04-25T00:11:12.514772+0000","last_undegraded":"2024-04-25T00:11:12.514772+0000","last_fullsized":"2024-04-25T00:11:12.514772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514712+0000","last_change":"2024-04-25T00:11:05.849451+0000","last_active":"2024-04-25T00:11:12.514712+0000","last_peered":"2024-04-25T00:11:12.514712+0000","last_clean":"2024-04-25T00:11:12.514712+0000","last_became_active":"2024-04-25T00:11:05.849028+0000","last_became_peered":"2024-04-25T00:11:05.849028+0000","last_unstale":"2024-04-25T00:11:12.514712+0000","last_undegraded":"2024-04-25T00:11:12.514712+0000","last_fullsized":"2024-04-25T00:11:12.514712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:41.285583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513859+0000","last_change":"2024-04-25T00:11:10.973749+0000","last_active":"2024-04-25T00:11:12.513859+0000","last_peered":"2024-04-25T00:11:12.513859+0000","last_clean":"2024-04-25T00:11:12.513859+0000","last_became_active":"2024-04-25T00:11:10.973369+0000","last_became_peered":"2024-04-25T00:11:10.973369+0000","last_unstale":"2024-04-25T00:11:12.513859+0000","last_undegraded":"2024-04-25T00:11:12.513859+0000","last_fullsized":"2024-04-25T00:11:12.513859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:40.531377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515701+0000","last_change":"2024-04-25T00:11:06.859428+0000","last_active":"2024-04-25T00:11:12.515701+0000","last_peered":"2024-04-25T00:11:12.515701+0000","last_clean":"2024-04-25T00:11:12.515701+0000","last_became_active":"2024-04-25T00:11:06.859101+0000","last_became_peered":"2024-04-25T00:11:06.859101+0000","last_unstale":"2024-04-25T00:11:12.515701+0000","last_undegraded":"2024-04-25T00:11:12.515701+0000","last_fullsized":"2024-04-25T00:11:12.515701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:59.645042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514881+0000","last_change":"2024-04-25T00:11:05.840437+0000","last_active":"2024-04-25T00:11:12.514881+0000","last_peered":"2024-04-25T00:11:12.514881+0000","last_clean":"2024-04-25T00:11:12.514881+0000","last_became_active":"2024-04-25T00:11:05.840120+0000","last_became_peered":"2024-04-25T00:11:05.840120+0000","last_unstale":"2024-04-25T00:11:12.514881+0000","last_undegraded":"2024-04-25T00:11:12.514881+0000","last_fullsized":"2024-04-25T00:11:12.514881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514595+0000","last_change":"2024-04-25T00:11:10.896951+0000","last_active":"2024-04-25T00:11:12.514595+0000","last_peered":"2024-04-25T00:11:12.514595+0000","last_clean":"2024-04-25T00:11:12.514595+0000","last_became_active":"2024-04-25T00:11:10.896547+0000","last_became_peered":"2024-04-25T00:11:10.896547+0000","last_unstale":"2024-04-25T00:11:12.514595+0000","last_undegraded":"2024-04-25T00:11:12.514595+0000","last_fullsized":"2024-04-25T00:11:12.514595+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516507+0000","last_change":"2024-04-25T00:11:06.846626+0000","last_active":"2024-04-25T00:11:12.516507+0000","last_peered":"2024-04-25T00:11:12.516507+0000","last_clean":"2024-04-25T00:11:12.516507+0000","last_became_active":"2024-04-25T00:11:06.846333+0000","last_became_peered":"2024-04-25T00:11:06.846333+0000","last_unstale":"2024-04-25T00:11:12.516507+0000","last_undegraded":"2024-04-25T00:11:12.516507+0000","last_fullsized":"2024-04-25T00:11:12.516507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:42.291050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515107+0000","last_change":"2024-04-25T00:11:05.839781+0000","last_active":"2024-04-25T00:11:12.515107+0000","last_peered":"2024-04-25T00:11:12.515107+0000","last_clean":"2024-04-25T00:11:12.515107+0000","last_became_active":"2024-04-25T00:11:05.839400+0000","last_became_peered":"2024-04-25T00:11:05.839400+0000","last_unstale":"2024-04-25T00:11:12.515107+0000","last_undegraded":"2024-04-25T00:11:12.515107+0000","last_fullsized":"2024-04-25T00:11:12.515107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514811+0000","last_change":"2024-04-25T00:11:10.892411+0000","last_active":"2024-04-25T00:11:12.514811+0000","last_peered":"2024-04-25T00:11:12.514811+0000","last_clean":"2024-04-25T00:11:12.514811+0000","last_became_active":"2024-04-25T00:11:10.892036+0000","last_became_peered":"2024-04-25T00:11:10.892036+0000","last_unstale":"2024-04-25T00:11:12.514811+0000","last_undegraded":"2024-04-25T00:11:12.514811+0000","last_fullsized":"2024-04-25T00:11:12.514811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:07.816085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513913+0000","last_change":"2024-04-25T00:11:06.842835+0000","last_active":"2024-04-25T00:11:12.513913+0000","last_peered":"2024-04-25T00:11:12.513913+0000","last_clean":"2024-04-25T00:11:12.513913+0000","last_became_active":"2024-04-25T00:11:06.842540+0000","last_became_peered":"2024-04-25T00:11:06.842540+0000","last_unstale":"2024-04-25T00:11:12.513913+0000","last_undegraded":"2024-04-25T00:11:12.513913+0000","last_fullsized":"2024-04-25T00:11:12.513913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:00.950133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875360+0000","last_change":"2024-04-25T00:11:05.829300+0000","last_active":"2024-04-25T00:11:11.875360+0000","last_peered":"2024-04-25T00:11:11.875360+0000","last_clean":"2024-04-25T00:11:11.875360+0000","last_became_active":"2024-04-25T00:11:05.828940+0000","last_became_peered":"2024-04-25T00:11:05.828940+0000","last_unstale":"2024-04-25T00:11:11.875360+0000","last_undegraded":"2024-04-25T00:11:11.875360+0000","last_fullsized":"2024-04-25T00:11:11.875360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:15.738255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515526+0000","last_change":"2024-04-25T00:11:10.895520+0000","last_active":"2024-04-25T00:11:12.515526+0000","last_peered":"2024-04-25T00:11:12.515526+0000","last_clean":"2024-04-25T00:11:12.515526+0000","last_became_active":"2024-04-25T00:11:10.895096+0000","last_became_peered":"2024-04-25T00:11:10.895096+0000","last_unstale":"2024-04-25T00:11:12.515526+0000","last_undegraded":"2024-04-25T00:11:12.515526+0000","last_fullsized":"2024-04-25T00:11:12.515526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:51.996256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513553+0000","last_change":"2024-04-25T00:11:06.862814+0000","last_active":"2024-04-25T00:11:12.513553+0000","last_peered":"2024-04-25T00:11:12.513553+0000","last_clean":"2024-04-25T00:11:12.513553+0000","last_became_active":"2024-04-25T00:11:06.862534+0000","last_became_peered":"2024-04-25T00:11:06.862534+0000","last_unstale":"2024-04-25T00:11:12.513553+0000","last_undegraded":"2024-04-25T00:11:12.513553+0000","last_fullsized":"2024-04-25T00:11:12.513553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:39.726185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513959+0000","last_change":"2024-04-25T00:11:05.849841+0000","last_active":"2024-04-25T00:11:12.513959+0000","last_peered":"2024-04-25T00:11:12.513959+0000","last_clean":"2024-04-25T00:11:12.513959+0000","last_became_active":"2024-04-25T00:11:05.847646+0000","last_became_peered":"2024-04-25T00:11:05.847646+0000","last_unstale":"2024-04-25T00:11:12.513959+0000","last_undegraded":"2024-04-25T00:11:12.513959+0000","last_fullsized":"2024-04-25T00:11:12.513959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:36.649759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514155+0000","last_change":"2024-04-25T00:11:10.974744+0000","last_active":"2024-04-25T00:11:12.514155+0000","last_peered":"2024-04-25T00:11:12.514155+0000","last_clean":"2024-04-25T00:11:12.514155+0000","last_became_active":"2024-04-25T00:11:10.973737+0000","last_became_peered":"2024-04-25T00:11:10.973737+0000","last_unstale":"2024-04-25T00:11:12.514155+0000","last_undegraded":"2024-04-25T00:11:12.514155+0000","last_fullsized":"2024-04-25T00:11:12.514155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:13.028091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875061+0000","last_change":"2024-04-25T00:11:06.850937+0000","last_active":"2024-04-25T00:11:11.875061+0000","last_peered":"2024-04-25T00:11:11.875061+0000","last_clean":"2024-04-25T00:11:11.875061+0000","last_became_active":"2024-04-25T00:11:06.850620+0000","last_became_peered":"2024-04-25T00:11:06.850620+0000","last_unstale":"2024-04-25T00:11:11.875061+0000","last_undegraded":"2024-04-25T00:11:11.875061+0000","last_fullsized":"2024-04-25T00:11:11.875061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514342+0000","last_change":"2024-04-25T00:11:05.841594+0000","last_active":"2024-04-25T00:11:12.514342+0000","last_peered":"2024-04-25T00:11:12.514342+0000","last_clean":"2024-04-25T00:11:12.514342+0000","last_became_active":"2024-04-25T00:11:05.838964+0000","last_became_peered":"2024-04-25T00:11:05.838964+0000","last_unstale":"2024-04-25T00:11:12.514342+0000","last_undegraded":"2024-04-25T00:11:12.514342+0000","last_fullsized":"2024-04-25T00:11:12.514342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514070+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:11:12.514070+0000","last_peered":"2024-04-25T00:11:12.514070+0000","last_clean":"2024-04-25T00:11:12.514070+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:11:12.514070+0000","last_undegraded":"2024-04-25T00:11:12.514070+0000","last_fullsized":"2024-04-25T00:11:12.514070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874137+0000","last_change":"2024-04-25T00:11:10.976384+0000","last_active":"2024-04-25T00:11:11.874137+0000","last_peered":"2024-04-25T00:11:11.874137+0000","last_clean":"2024-04-25T00:11:11.874137+0000","last_became_active":"2024-04-25T00:11:10.975924+0000","last_became_peered":"2024-04-25T00:11:10.975924+0000","last_unstale":"2024-04-25T00:11:11.874137+0000","last_undegraded":"2024-04-25T00:11:11.874137+0000","last_fullsized":"2024-04-25T00:11:11.874137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514383+0000","last_change":"2024-04-25T00:11:06.855965+0000","last_active":"2024-04-25T00:11:12.514383+0000","last_peered":"2024-04-25T00:11:12.514383+0000","last_clean":"2024-04-25T00:11:12.514383+0000","last_became_active":"2024-04-25T00:11:06.853447+0000","last_became_peered":"2024-04-25T00:11:06.853447+0000","last_unstale":"2024-04-25T00:11:12.514383+0000","last_undegraded":"2024-04-25T00:11:12.514383+0000","last_fullsized":"2024-04-25T00:11:12.514383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:14.704188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875959+0000","last_change":"2024-04-25T00:11:05.842420+0000","last_active":"2024-04-25T00:11:11.875959+0000","last_peered":"2024-04-25T00:11:11.875959+0000","last_clean":"2024-04-25T00:11:11.875959+0000","last_became_active":"2024-04-25T00:11:05.842102+0000","last_became_peered":"2024-04-25T00:11:05.842102+0000","last_unstale":"2024-04-25T00:11:11.875959+0000","last_undegraded":"2024-04-25T00:11:11.875959+0000","last_fullsized":"2024-04-25T00:11:11.875959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874194+0000","last_change":"2024-04-25T00:11:10.969423+0000","last_active":"2024-04-25T00:11:11.874194+0000","last_peered":"2024-04-25T00:11:11.874194+0000","last_clean":"2024-04-25T00:11:11.874194+0000","last_became_active":"2024-04-25T00:11:10.968799+0000","last_became_peered":"2024-04-25T00:11:10.968799+0000","last_unstale":"2024-04-25T00:11:11.874194+0000","last_undegraded":"2024-04-25T00:11:11.874194+0000","last_fullsized":"2024-04-25T00:11:11.874194+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514836+0000","last_change":"2024-04-25T00:11:06.859625+0000","last_active":"2024-04-25T00:11:12.514836+0000","last_peered":"2024-04-25T00:11:12.514836+0000","last_clean":"2024-04-25T00:11:12.514836+0000","last_became_active":"2024-04-25T00:11:06.859161+0000","last_became_peered":"2024-04-25T00:11:06.859161+0000","last_unstale":"2024-04-25T00:11:12.514836+0000","last_undegraded":"2024-04-25T00:11:12.514836+0000","last_fullsized":"2024-04-25T00:11:12.514836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:36.481467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515075+0000","last_change":"2024-04-25T00:11:05.839603+0000","last_active":"2024-04-25T00:11:12.515075+0000","last_peered":"2024-04-25T00:11:12.515075+0000","last_clean":"2024-04-25T00:11:12.515075+0000","last_became_active":"2024-04-25T00:11:05.838608+0000","last_became_peered":"2024-04-25T00:11:05.838608+0000","last_unstale":"2024-04-25T00:11:12.515075+0000","last_undegraded":"2024-04-25T00:11:12.515075+0000","last_fullsized":"2024-04-25T00:11:12.515075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:53.690870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514117+0000","last_change":"2024-04-25T00:11:10.900742+0000","last_active":"2024-04-25T00:11:12.514117+0000","last_peered":"2024-04-25T00:11:12.514117+0000","last_clean":"2024-04-25T00:11:12.514117+0000","last_became_active":"2024-04-25T00:11:10.900246+0000","last_became_peered":"2024-04-25T00:11:10.900246+0000","last_unstale":"2024-04-25T00:11:12.514117+0000","last_undegraded":"2024-04-25T00:11:12.514117+0000","last_fullsized":"2024-04-25T00:11:12.514117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:07.816085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514836+0000","last_change":"2024-04-25T00:11:06.870304+0000","last_active":"2024-04-25T00:11:12.514836+0000","last_peered":"2024-04-25T00:11:12.514836+0000","last_clean":"2024-04-25T00:11:12.514836+0000","last_became_active":"2024-04-25T00:11:06.870010+0000","last_became_peered":"2024-04-25T00:11:06.870010+0000","last_unstale":"2024-04-25T00:11:12.514836+0000","last_undegraded":"2024-04-25T00:11:12.514836+0000","last_fullsized":"2024-04-25T00:11:12.514836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:08.978181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.876556+0000","last_change":"2024-04-25T00:11:05.839098+0000","last_active":"2024-04-25T00:11:11.876556+0000","last_peered":"2024-04-25T00:11:11.876556+0000","last_clean":"2024-04-25T00:11:11.876556+0000","last_became_active":"2024-04-25T00:11:05.838694+0000","last_became_peered":"2024-04-25T00:11:05.838694+0000","last_unstale":"2024-04-25T00:11:11.876556+0000","last_undegraded":"2024-04-25T00:11:11.876556+0000","last_fullsized":"2024-04-25T00:11:11.876556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:35.709574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513908+0000","last_change":"2024-04-25T00:11:10.973661+0000","last_active":"2024-04-25T00:11:12.513908+0000","last_peered":"2024-04-25T00:11:12.513908+0000","last_clean":"2024-04-25T00:11:12.513908+0000","last_became_active":"2024-04-25T00:11:10.973198+0000","last_became_peered":"2024-04-25T00:11:10.973198+0000","last_unstale":"2024-04-25T00:11:12.513908+0000","last_undegraded":"2024-04-25T00:11:12.513908+0000","last_fullsized":"2024-04-25T00:11:12.513908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:23:46.222075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514875+0000","last_change":"2024-04-25T00:11:06.878377+0000","last_active":"2024-04-25T00:11:12.514875+0000","last_peered":"2024-04-25T00:11:12.514875+0000","last_clean":"2024-04-25T00:11:12.514875+0000","last_became_active":"2024-04-25T00:11:06.878046+0000","last_became_peered":"2024-04-25T00:11:06.878046+0000","last_unstale":"2024-04-25T00:11:12.514875+0000","last_undegraded":"2024-04-25T00:11:12.514875+0000","last_fullsized":"2024-04-25T00:11:12.514875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:59.645042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515717+0000","last_change":"2024-04-25T00:11:05.839981+0000","last_active":"2024-04-25T00:11:12.515717+0000","last_peered":"2024-04-25T00:11:12.515717+0000","last_clean":"2024-04-25T00:11:12.515717+0000","last_became_active":"2024-04-25T00:11:05.839527+0000","last_became_peered":"2024-04-25T00:11:05.839527+0000","last_unstale":"2024-04-25T00:11:12.515717+0000","last_undegraded":"2024-04-25T00:11:12.515717+0000","last_fullsized":"2024-04-25T00:11:12.515717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:14.133421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515439+0000","last_change":"2024-04-25T00:11:10.900219+0000","last_active":"2024-04-25T00:11:12.515439+0000","last_peered":"2024-04-25T00:11:12.515439+0000","last_clean":"2024-04-25T00:11:12.515439+0000","last_became_active":"2024-04-25T00:11:10.899780+0000","last_became_peered":"2024-04-25T00:11:10.899780+0000","last_unstale":"2024-04-25T00:11:12.515439+0000","last_undegraded":"2024-04-25T00:11:12.515439+0000","last_fullsized":"2024-04-25T00:11:12.515439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:29.940770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515975+0000","last_change":"2024-04-25T00:11:06.863060+0000","last_active":"2024-04-25T00:11:12.515975+0000","last_peered":"2024-04-25T00:11:12.515975+0000","last_clean":"2024-04-25T00:11:12.515975+0000","last_became_active":"2024-04-25T00:11:06.862320+0000","last_became_peered":"2024-04-25T00:11:06.862320+0000","last_unstale":"2024-04-25T00:11:12.515975+0000","last_undegraded":"2024-04-25T00:11:12.515975+0000","last_fullsized":"2024-04-25T00:11:12.515975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:49.414621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875244+0000","last_change":"2024-04-25T00:11:05.826923+0000","last_active":"2024-04-25T00:11:11.875244+0000","last_peered":"2024-04-25T00:11:11.875244+0000","last_clean":"2024-04-25T00:11:11.875244+0000","last_became_active":"2024-04-25T00:11:05.826543+0000","last_became_peered":"2024-04-25T00:11:05.826543+0000","last_unstale":"2024-04-25T00:11:11.875244+0000","last_undegraded":"2024-04-25T00:11:11.875244+0000","last_fullsized":"2024-04-25T00:11:11.875244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513942+0000","last_change":"2024-04-25T00:11:10.893190+0000","last_active":"2024-04-25T00:11:12.513942+0000","last_peered":"2024-04-25T00:11:12.513942+0000","last_clean":"2024-04-25T00:11:12.513942+0000","last_became_active":"2024-04-25T00:11:10.892676+0000","last_became_peered":"2024-04-25T00:11:10.892676+0000","last_unstale":"2024-04-25T00:11:12.513942+0000","last_undegraded":"2024-04-25T00:11:12.513942+0000","last_fullsized":"2024-04-25T00:11:12.513942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874062+0000","last_change":"2024-04-25T00:11:06.854380+0000","last_active":"2024-04-25T00:11:11.874062+0000","last_peered":"2024-04-25T00:11:11.874062+0000","last_clean":"2024-04-25T00:11:11.874062+0000","last_became_active":"2024-04-25T00:11:06.854089+0000","last_became_peered":"2024-04-25T00:11:06.854089+0000","last_unstale":"2024-04-25T00:11:11.874062+0000","last_undegraded":"2024-04-25T00:11:11.874062+0000","last_fullsized":"2024-04-25T00:11:11.874062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874367+0000","last_change":"2024-04-25T00:11:05.848317+0000","last_active":"2024-04-25T00:11:11.874367+0000","last_peered":"2024-04-25T00:11:11.874367+0000","last_clean":"2024-04-25T00:11:11.874367+0000","last_became_active":"2024-04-25T00:11:05.847969+0000","last_became_peered":"2024-04-25T00:11:05.847969+0000","last_unstale":"2024-04-25T00:11:11.874367+0000","last_undegraded":"2024-04-25T00:11:11.874367+0000","last_fullsized":"2024-04-25T00:11:11.874367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874567+0000","last_change":"2024-04-25T00:11:10.974665+0000","last_active":"2024-04-25T00:11:11.874567+0000","last_peered":"2024-04-25T00:11:11.874567+0000","last_clean":"2024-04-25T00:11:11.874567+0000","last_became_active":"2024-04-25T00:11:10.974306+0000","last_became_peered":"2024-04-25T00:11:10.974306+0000","last_unstale":"2024-04-25T00:11:11.874567+0000","last_undegraded":"2024-04-25T00:11:11.874567+0000","last_fullsized":"2024-04-25T00:11:11.874567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:09.712979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514422+0000","last_change":"2024-04-25T00:11:06.870996+0000","last_active":"2024-04-25T00:11:12.514422+0000","last_peered":"2024-04-25T00:11:12.514422+0000","last_clean":"2024-04-25T00:11:12.514422+0000","last_became_active":"2024-04-25T00:11:06.870692+0000","last_became_peered":"2024-04-25T00:11:06.870692+0000","last_unstale":"2024-04-25T00:11:12.514422+0000","last_undegraded":"2024-04-25T00:11:12.514422+0000","last_fullsized":"2024-04-25T00:11:12.514422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:36.481467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515027+0000","last_change":"2024-04-25T00:11:05.838027+0000","last_active":"2024-04-25T00:11:12.515027+0000","last_peered":"2024-04-25T00:11:12.515027+0000","last_clean":"2024-04-25T00:11:12.515027+0000","last_became_active":"2024-04-25T00:11:05.837712+0000","last_became_peered":"2024-04-25T00:11:05.837712+0000","last_unstale":"2024-04-25T00:11:12.515027+0000","last_undegraded":"2024-04-25T00:11:12.515027+0000","last_fullsized":"2024-04-25T00:11:12.515027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:53.690870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875060+0000","last_change":"2024-04-25T00:11:10.973710+0000","last_active":"2024-04-25T00:11:11.875060+0000","last_peered":"2024-04-25T00:11:11.875060+0000","last_clean":"2024-04-25T00:11:11.875060+0000","last_became_active":"2024-04-25T00:11:10.973387+0000","last_became_peered":"2024-04-25T00:11:10.973387+0000","last_unstale":"2024-04-25T00:11:11.875060+0000","last_undegraded":"2024-04-25T00:11:11.875060+0000","last_fullsized":"2024-04-25T00:11:11.875060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:39.160853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514207+0000","last_change":"2024-04-25T00:11:06.847434+0000","last_active":"2024-04-25T00:11:12.514207+0000","last_peered":"2024-04-25T00:11:12.514207+0000","last_clean":"2024-04-25T00:11:12.514207+0000","last_became_active":"2024-04-25T00:11:06.847112+0000","last_became_peered":"2024-04-25T00:11:06.847112+0000","last_unstale":"2024-04-25T00:11:12.514207+0000","last_undegraded":"2024-04-25T00:11:12.514207+0000","last_fullsized":"2024-04-25T00:11:12.514207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:27.928927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514359+0000","last_change":"2024-04-25T00:11:05.837912+0000","last_active":"2024-04-25T00:11:12.514359+0000","last_peered":"2024-04-25T00:11:12.514359+0000","last_clean":"2024-04-25T00:11:12.514359+0000","last_became_active":"2024-04-25T00:11:05.837544+0000","last_became_peered":"2024-04-25T00:11:05.837544+0000","last_unstale":"2024-04-25T00:11:12.514359+0000","last_undegraded":"2024-04-25T00:11:12.514359+0000","last_fullsized":"2024-04-25T00:11:12.514359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:14.133421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515539+0000","last_change":"2024-04-25T00:11:10.968996+0000","last_active":"2024-04-25T00:11:12.515539+0000","last_peered":"2024-04-25T00:11:12.515539+0000","last_clean":"2024-04-25T00:11:12.515539+0000","last_became_active":"2024-04-25T00:11:10.968625+0000","last_became_peered":"2024-04-25T00:11:10.968625+0000","last_unstale":"2024-04-25T00:11:12.515539+0000","last_undegraded":"2024-04-25T00:11:12.515539+0000","last_fullsized":"2024-04-25T00:11:12.515539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:07.644687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514303+0000","last_change":"2024-04-25T00:11:06.857856+0000","last_active":"2024-04-25T00:11:12.514303+0000","last_peered":"2024-04-25T00:11:12.514303+0000","last_clean":"2024-04-25T00:11:12.514303+0000","last_became_active":"2024-04-25T00:11:06.857469+0000","last_became_peered":"2024-04-25T00:11:06.857469+0000","last_unstale":"2024-04-25T00:11:12.514303+0000","last_undegraded":"2024-04-25T00:11:12.514303+0000","last_fullsized":"2024-04-25T00:11:12.514303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:47.946346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:17.643019+0000","last_change":"2024-04-25T00:11:05.836491+0000","last_active":"2024-04-25T00:11:17.643019+0000","last_peered":"2024-04-25T00:11:17.643019+0000","last_clean":"2024-04-25T00:11:17.643019+0000","last_became_active":"2024-04-25T00:11:05.835903+0000","last_became_peered":"2024-04-25T00:11:05.835903+0000","last_unstale":"2024-04-25T00:11:17.643019+0000","last_undegraded":"2024-04-25T00:11:17.643019+0000","last_fullsized":"2024-04-25T00:11:17.643019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514752+0000","last_change":"2024-04-25T00:11:10.902870+0000","last_active":"2024-04-25T00:11:12.514752+0000","last_peered":"2024-04-25T00:11:12.514752+0000","last_clean":"2024-04-25T00:11:12.514752+0000","last_became_active":"2024-04-25T00:11:10.896980+0000","last_became_peered":"2024-04-25T00:11:10.896980+0000","last_unstale":"2024-04-25T00:11:12.514752+0000","last_undegraded":"2024-04-25T00:11:12.514752+0000","last_fullsized":"2024-04-25T00:11:12.514752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516066+0000","last_change":"2024-04-25T00:11:06.864856+0000","last_active":"2024-04-25T00:11:12.516066+0000","last_peered":"2024-04-25T00:11:12.516066+0000","last_clean":"2024-04-25T00:11:12.516066+0000","last_became_active":"2024-04-25T00:11:06.863698+0000","last_became_peered":"2024-04-25T00:11:06.863698+0000","last_unstale":"2024-04-25T00:11:12.516066+0000","last_undegraded":"2024-04-25T00:11:12.516066+0000","last_fullsized":"2024-04-25T00:11:12.516066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:35.253622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514963+0000","last_change":"2024-04-25T00:11:05.843072+0000","last_active":"2024-04-25T00:11:12.514963+0000","last_peered":"2024-04-25T00:11:12.514963+0000","last_clean":"2024-04-25T00:11:12.514963+0000","last_became_active":"2024-04-25T00:11:05.842548+0000","last_became_peered":"2024-04-25T00:11:05.842548+0000","last_unstale":"2024-04-25T00:11:12.514963+0000","last_undegraded":"2024-04-25T00:11:12.514963+0000","last_fullsized":"2024-04-25T00:11:12.514963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516339+0000","last_change":"2024-04-25T00:11:10.900180+0000","last_active":"2024-04-25T00:11:12.516339+0000","last_peered":"2024-04-25T00:11:12.516339+0000","last_clean":"2024-04-25T00:11:12.516339+0000","last_became_active":"2024-04-25T00:11:10.899321+0000","last_became_peered":"2024-04-25T00:11:10.899321+0000","last_unstale":"2024-04-25T00:11:12.516339+0000","last_undegraded":"2024-04-25T00:11:12.516339+0000","last_fullsized":"2024-04-25T00:11:12.516339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515303+0000","last_change":"2024-04-25T00:11:06.855642+0000","last_active":"2024-04-25T00:11:12.515303+0000","last_peered":"2024-04-25T00:11:12.515303+0000","last_clean":"2024-04-25T00:11:12.515303+0000","last_became_active":"2024-04-25T00:11:06.855374+0000","last_became_peered":"2024-04-25T00:11:06.855374+0000","last_unstale":"2024-04-25T00:11:12.515303+0000","last_undegraded":"2024-04-25T00:11:12.515303+0000","last_fullsized":"2024-04-25T00:11:12.515303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516104+0000","last_change":"2024-04-25T00:11:05.846074+0000","last_active":"2024-04-25T00:11:12.516104+0000","last_peered":"2024-04-25T00:11:12.516104+0000","last_clean":"2024-04-25T00:11:12.516104+0000","last_became_active":"2024-04-25T00:11:05.845371+0000","last_became_peered":"2024-04-25T00:11:05.845371+0000","last_unstale":"2024-04-25T00:11:12.516104+0000","last_undegraded":"2024-04-25T00:11:12.516104+0000","last_fullsized":"2024-04-25T00:11:12.516104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874963+0000","last_change":"2024-04-25T00:11:10.898262+0000","last_active":"2024-04-25T00:11:11.874963+0000","last_peered":"2024-04-25T00:11:11.874963+0000","last_clean":"2024-04-25T00:11:11.874963+0000","last_became_active":"2024-04-25T00:11:10.897653+0000","last_became_peered":"2024-04-25T00:11:10.897653+0000","last_unstale":"2024-04-25T00:11:11.874963+0000","last_undegraded":"2024-04-25T00:11:11.874963+0000","last_fullsized":"2024-04-25T00:11:11.874963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:12.584307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516002+0000","last_change":"2024-04-25T00:11:06.865617+0000","last_active":"2024-04-25T00:11:12.516002+0000","last_peered":"2024-04-25T00:11:12.516002+0000","last_clean":"2024-04-25T00:11:12.516002+0000","last_became_active":"2024-04-25T00:11:06.865286+0000","last_became_peered":"2024-04-25T00:11:06.865286+0000","last_unstale":"2024-04-25T00:11:12.516002+0000","last_undegraded":"2024-04-25T00:11:12.516002+0000","last_fullsized":"2024-04-25T00:11:12.516002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:03.766175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874678+0000","last_change":"2024-04-25T00:11:05.845733+0000","last_active":"2024-04-25T00:11:11.874678+0000","last_peered":"2024-04-25T00:11:11.874678+0000","last_clean":"2024-04-25T00:11:11.874678+0000","last_became_active":"2024-04-25T00:11:05.845410+0000","last_became_peered":"2024-04-25T00:11:05.845410+0000","last_unstale":"2024-04-25T00:11:11.874678+0000","last_undegraded":"2024-04-25T00:11:11.874678+0000","last_fullsized":"2024-04-25T00:11:11.874678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874418+0000","last_change":"2024-04-25T00:11:10.977787+0000","last_active":"2024-04-25T00:11:11.874418+0000","last_peered":"2024-04-25T00:11:11.874418+0000","last_clean":"2024-04-25T00:11:11.874418+0000","last_became_active":"2024-04-25T00:11:10.977434+0000","last_became_peered":"2024-04-25T00:11:10.977434+0000","last_unstale":"2024-04-25T00:11:11.874418+0000","last_undegraded":"2024-04-25T00:11:11.874418+0000","last_fullsized":"2024-04-25T00:11:11.874418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:27.014726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513916+0000","last_change":"2024-04-25T00:11:10.977497+0000","last_active":"2024-04-25T00:11:12.513916+0000","last_peered":"2024-04-25T00:11:12.513916+0000","last_clean":"2024-04-25T00:11:12.513916+0000","last_became_active":"2024-04-25T00:11:10.973670+0000","last_became_peered":"2024-04-25T00:11:10.973670+0000","last_unstale":"2024-04-25T00:11:12.513916+0000","last_undegraded":"2024-04-25T00:11:12.513916+0000","last_fullsized":"2024-04-25T00:11:12.513916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514138+0000","last_change":"2024-04-25T00:11:06.851862+0000","last_active":"2024-04-25T00:11:12.514138+0000","last_peered":"2024-04-25T00:11:12.514138+0000","last_clean":"2024-04-25T00:11:12.514138+0000","last_became_active":"2024-04-25T00:11:06.851683+0000","last_became_peered":"2024-04-25T00:11:06.851683+0000","last_unstale":"2024-04-25T00:11:12.514138+0000","last_undegraded":"2024-04-25T00:11:12.514138+0000","last_fullsized":"2024-04-25T00:11:12.514138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515242+0000","last_change":"2024-04-25T00:11:05.842503+0000","last_active":"2024-04-25T00:11:12.515242+0000","last_peered":"2024-04-25T00:11:12.515242+0000","last_clean":"2024-04-25T00:11:12.515242+0000","last_became_active":"2024-04-25T00:11:05.842214+0000","last_became_peered":"2024-04-25T00:11:05.842214+0000","last_unstale":"2024-04-25T00:11:12.515242+0000","last_undegraded":"2024-04-25T00:11:12.515242+0000","last_fullsized":"2024-04-25T00:11:12.515242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874565+0000","last_change":"2024-04-25T00:11:10.901930+0000","last_active":"2024-04-25T00:11:11.874565+0000","last_peered":"2024-04-25T00:11:11.874565+0000","last_clean":"2024-04-25T00:11:11.874565+0000","last_became_active":"2024-04-25T00:11:10.901405+0000","last_became_peered":"2024-04-25T00:11:10.901405+0000","last_unstale":"2024-04-25T00:11:11.874565+0000","last_undegraded":"2024-04-25T00:11:11.874565+0000","last_fullsized":"2024-04-25T00:11:11.874565+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:07.816085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874298+0000","last_change":"2024-04-25T00:11:06.855999+0000","last_active":"2024-04-25T00:11:11.874298+0000","last_peered":"2024-04-25T00:11:11.874298+0000","last_clean":"2024-04-25T00:11:11.874298+0000","last_became_active":"2024-04-25T00:11:06.855758+0000","last_became_peered":"2024-04-25T00:11:06.855758+0000","last_unstale":"2024-04-25T00:11:11.874298+0000","last_undegraded":"2024-04-25T00:11:11.874298+0000","last_fullsized":"2024-04-25T00:11:11.874298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874267+0000","last_change":"2024-04-25T00:11:05.834141+0000","last_active":"2024-04-25T00:11:11.874267+0000","last_peered":"2024-04-25T00:11:11.874267+0000","last_clean":"2024-04-25T00:11:11.874267+0000","last_became_active":"2024-04-25T00:11:05.833022+0000","last_became_peered":"2024-04-25T00:11:05.833022+0000","last_unstale":"2024-04-25T00:11:11.874267+0000","last_undegraded":"2024-04-25T00:11:11.874267+0000","last_fullsized":"2024-04-25T00:11:11.874267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:15.738255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874362+0000","last_change":"2024-04-25T00:11:05.844727+0000","last_active":"2024-04-25T00:11:11.874362+0000","last_peered":"2024-04-25T00:11:11.874362+0000","last_clean":"2024-04-25T00:11:11.874362+0000","last_became_active":"2024-04-25T00:11:05.844303+0000","last_became_peered":"2024-04-25T00:11:05.844303+0000","last_unstale":"2024-04-25T00:11:11.874362+0000","last_undegraded":"2024-04-25T00:11:11.874362+0000","last_fullsized":"2024-04-25T00:11:11.874362+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:41.285583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514255+0000","last_change":"2024-04-25T00:11:06.862764+0000","last_active":"2024-04-25T00:11:12.514255+0000","last_peered":"2024-04-25T00:11:12.514255+0000","last_clean":"2024-04-25T00:11:12.514255+0000","last_became_active":"2024-04-25T00:11:06.862372+0000","last_became_peered":"2024-04-25T00:11:06.862372+0000","last_unstale":"2024-04-25T00:11:12.514255+0000","last_undegraded":"2024-04-25T00:11:12.514255+0000","last_fullsized":"2024-04-25T00:11:12.514255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875902+0000","last_change":"2024-04-25T00:11:10.902130+0000","last_active":"2024-04-25T00:11:11.875902+0000","last_peered":"2024-04-25T00:11:11.875902+0000","last_clean":"2024-04-25T00:11:11.875902+0000","last_became_active":"2024-04-25T00:11:10.901717+0000","last_became_peered":"2024-04-25T00:11:10.901717+0000","last_unstale":"2024-04-25T00:11:11.875902+0000","last_undegraded":"2024-04-25T00:11:11.875902+0000","last_fullsized":"2024-04-25T00:11:11.875902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:43.776095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875180+0000","last_change":"2024-04-25T00:11:05.833116+0000","last_active":"2024-04-25T00:11:11.875180+0000","last_peered":"2024-04-25T00:11:11.875180+0000","last_clean":"2024-04-25T00:11:11.875180+0000","last_became_active":"2024-04-25T00:11:05.832202+0000","last_became_peered":"2024-04-25T00:11:05.832202+0000","last_unstale":"2024-04-25T00:11:11.875180+0000","last_undegraded":"2024-04-25T00:11:11.875180+0000","last_fullsized":"2024-04-25T00:11:11.875180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516469+0000","last_change":"2024-04-25T00:11:06.840973+0000","last_active":"2024-04-25T00:11:12.516469+0000","last_peered":"2024-04-25T00:11:12.516469+0000","last_clean":"2024-04-25T00:11:12.516469+0000","last_became_active":"2024-04-25T00:11:06.840670+0000","last_became_peered":"2024-04-25T00:11:06.840670+0000","last_unstale":"2024-04-25T00:11:12.516469+0000","last_undegraded":"2024-04-25T00:11:12.516469+0000","last_fullsized":"2024-04-25T00:11:12.516469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:29.296632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874813+0000","last_change":"2024-04-25T00:11:10.898183+0000","last_active":"2024-04-25T00:11:11.874813+0000","last_peered":"2024-04-25T00:11:11.874813+0000","last_clean":"2024-04-25T00:11:11.874813+0000","last_became_active":"2024-04-25T00:11:10.897738+0000","last_became_peered":"2024-04-25T00:11:10.897738+0000","last_unstale":"2024-04-25T00:11:11.874813+0000","last_undegraded":"2024-04-25T00:11:11.874813+0000","last_fullsized":"2024-04-25T00:11:11.874813+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:44.129988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873858+0000","last_change":"2024-04-25T00:11:06.854098+0000","last_active":"2024-04-25T00:11:11.873858+0000","last_peered":"2024-04-25T00:11:11.873858+0000","last_clean":"2024-04-25T00:11:11.873858+0000","last_became_active":"2024-04-25T00:11:06.853839+0000","last_became_peered":"2024-04-25T00:11:06.853839+0000","last_unstale":"2024-04-25T00:11:11.873858+0000","last_undegraded":"2024-04-25T00:11:11.873858+0000","last_fullsized":"2024-04-25T00:11:11.873858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:00.950133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875315+0000","last_change":"2024-04-25T00:11:06.853055+0000","last_active":"2024-04-25T00:11:11.875315+0000","last_peered":"2024-04-25T00:11:11.875315+0000","last_clean":"2024-04-25T00:11:11.875315+0000","last_became_active":"2024-04-25T00:11:06.852814+0000","last_became_peered":"2024-04-25T00:11:06.852814+0000","last_unstale":"2024-04-25T00:11:11.875315+0000","last_undegraded":"2024-04-25T00:11:11.875315+0000","last_fullsized":"2024-04-25T00:11:11.875315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:42.291050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515002+0000","last_change":"2024-04-25T00:11:06.858759+0000","last_active":"2024-04-25T00:11:12.515002+0000","last_peered":"2024-04-25T00:11:12.515002+0000","last_clean":"2024-04-25T00:11:12.515002+0000","last_became_active":"2024-04-25T00:11:06.858472+0000","last_became_peered":"2024-04-25T00:11:06.858472+0000","last_unstale":"2024-04-25T00:11:12.515002+0000","last_undegraded":"2024-04-25T00:11:12.515002+0000","last_fullsized":"2024-04-25T00:11:12.515002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514169+0000","last_change":"2024-04-25T00:11:06.851389+0000","last_active":"2024-04-25T00:11:12.514169+0000","last_peered":"2024-04-25T00:11:12.514169+0000","last_clean":"2024-04-25T00:11:12.514169+0000","last_became_active":"2024-04-25T00:11:06.845828+0000","last_became_peered":"2024-04-25T00:11:06.845828+0000","last_unstale":"2024-04-25T00:11:12.514169+0000","last_undegraded":"2024-04-25T00:11:12.514169+0000","last_fullsized":"2024-04-25T00:11:12.514169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:00.950133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875364+0000","last_change":"2024-04-25T00:11:06.851807+0000","last_active":"2024-04-25T00:11:11.875364+0000","last_peered":"2024-04-25T00:11:11.875364+0000","last_clean":"2024-04-25T00:11:11.875364+0000","last_became_active":"2024-04-25T00:11:06.851495+0000","last_became_peered":"2024-04-25T00:11:06.851495+0000","last_unstale":"2024-04-25T00:11:11.875364+0000","last_undegraded":"2024-04-25T00:11:11.875364+0000","last_fullsized":"2024-04-25T00:11:11.875364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:49.414621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514751+0000","last_change":"2024-04-25T00:11:06.848495+0000","last_active":"2024-04-25T00:11:12.514751+0000","last_peered":"2024-04-25T00:11:12.514751+0000","last_clean":"2024-04-25T00:11:12.514751+0000","last_became_active":"2024-04-25T00:11:06.848242+0000","last_became_peered":"2024-04-25T00:11:06.848242+0000","last_unstale":"2024-04-25T00:11:12.514751+0000","last_undegraded":"2024-04-25T00:11:12.514751+0000","last_fullsized":"2024-04-25T00:11:12.514751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874704+0000","last_change":"2024-04-25T00:11:06.850991+0000","last_active":"2024-04-25T00:11:11.874704+0000","last_peered":"2024-04-25T00:11:11.874704+0000","last_clean":"2024-04-25T00:11:11.874704+0000","last_became_active":"2024-04-25T00:11:06.850666+0000","last_became_peered":"2024-04-25T00:11:06.850666+0000","last_unstale":"2024-04-25T00:11:11.874704+0000","last_undegraded":"2024-04-25T00:11:11.874704+0000","last_fullsized":"2024-04-25T00:11:11.874704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:25.890860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874498+0000","last_change":"2024-04-25T00:11:06.857585+0000","last_active":"2024-04-25T00:11:11.874498+0000","last_peered":"2024-04-25T00:11:11.874498+0000","last_clean":"2024-04-25T00:11:11.874498+0000","last_became_active":"2024-04-25T00:11:06.857315+0000","last_became_peered":"2024-04-25T00:11:06.857315+0000","last_unstale":"2024-04-25T00:11:11.874498+0000","last_undegraded":"2024-04-25T00:11:11.874498+0000","last_fullsized":"2024-04-25T00:11:11.874498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:43.612611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513979+0000","last_change":"2024-04-25T00:11:06.858752+0000","last_active":"2024-04-25T00:11:12.513979+0000","last_peered":"2024-04-25T00:11:12.513979+0000","last_clean":"2024-04-25T00:11:12.513979+0000","last_became_active":"2024-04-25T00:11:06.858428+0000","last_became_peered":"2024-04-25T00:11:06.858428+0000","last_unstale":"2024-04-25T00:11:12.513979+0000","last_undegraded":"2024-04-25T00:11:12.513979+0000","last_fullsized":"2024-04-25T00:11:12.513979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.219642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874132+0000","last_change":"2024-04-25T00:11:06.870006+0000","last_active":"2024-04-25T00:11:11.874132+0000","last_peered":"2024-04-25T00:11:11.874132+0000","last_clean":"2024-04-25T00:11:11.874132+0000","last_became_active":"2024-04-25T00:11:06.869687+0000","last_became_peered":"2024-04-25T00:11:06.869687+0000","last_unstale":"2024-04-25T00:11:11.874132+0000","last_undegraded":"2024-04-25T00:11:11.874132+0000","last_fullsized":"2024-04-25T00:11:11.874132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:43.565714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514922+0000","last_change":"2024-04-25T00:11:06.863927+0000","last_active":"2024-04-25T00:11:12.514922+0000","last_peered":"2024-04-25T00:11:12.514922+0000","last_clean":"2024-04-25T00:11:12.514922+0000","last_became_active":"2024-04-25T00:11:06.863758+0000","last_became_peered":"2024-04-25T00:11:06.863758+0000","last_unstale":"2024-04-25T00:11:12.514922+0000","last_undegraded":"2024-04-25T00:11:12.514922+0000","last_fullsized":"2024-04-25T00:11:12.514922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:25.890860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513912+0000","last_change":"2024-04-25T00:11:06.852070+0000","last_active":"2024-04-25T00:11:12.513912+0000","last_peered":"2024-04-25T00:11:12.513912+0000","last_clean":"2024-04-25T00:11:12.513912+0000","last_became_active":"2024-04-25T00:11:06.849951+0000","last_became_peered":"2024-04-25T00:11:06.849951+0000","last_unstale":"2024-04-25T00:11:12.513912+0000","last_undegraded":"2024-04-25T00:11:12.513912+0000","last_fullsized":"2024-04-25T00:11:12.513912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:54.204077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874289+0000","last_change":"2024-04-25T00:11:06.867310+0000","last_active":"2024-04-25T00:11:11.874289+0000","last_peered":"2024-04-25T00:11:11.874289+0000","last_clean":"2024-04-25T00:11:11.874289+0000","last_became_active":"2024-04-25T00:11:06.867010+0000","last_became_peered":"2024-04-25T00:11:06.867010+0000","last_unstale":"2024-04-25T00:11:11.874289+0000","last_undegraded":"2024-04-25T00:11:11.874289+0000","last_fullsized":"2024-04-25T00:11:11.874289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515126+0000","last_change":"2024-04-25T00:11:06.855941+0000","last_active":"2024-04-25T00:11:12.515126+0000","last_peered":"2024-04-25T00:11:12.515126+0000","last_clean":"2024-04-25T00:11:12.515126+0000","last_became_active":"2024-04-25T00:11:06.855737+0000","last_became_peered":"2024-04-25T00:11:06.855737+0000","last_unstale":"2024-04-25T00:11:12.515126+0000","last_undegraded":"2024-04-25T00:11:12.515126+0000","last_fullsized":"2024-04-25T00:11:12.515126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:18.665263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.876458+0000","last_change":"2024-04-25T00:11:06.865551+0000","last_active":"2024-04-25T00:11:11.876458+0000","last_peered":"2024-04-25T00:11:11.876458+0000","last_clean":"2024-04-25T00:11:11.876458+0000","last_became_active":"2024-04-25T00:11:06.865289+0000","last_became_peered":"2024-04-25T00:11:06.865289+0000","last_unstale":"2024-04-25T00:11:11.876458+0000","last_undegraded":"2024-04-25T00:11:11.876458+0000","last_fullsized":"2024-04-25T00:11:11.876458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:51.375329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516337+0000","last_change":"2024-04-25T00:11:06.863407+0000","last_active":"2024-04-25T00:11:12.516337+0000","last_peered":"2024-04-25T00:11:12.516337+0000","last_clean":"2024-04-25T00:11:12.516337+0000","last_became_active":"2024-04-25T00:11:06.863119+0000","last_became_peered":"2024-04-25T00:11:06.863119+0000","last_unstale":"2024-04-25T00:11:12.516337+0000","last_undegraded":"2024-04-25T00:11:12.516337+0000","last_fullsized":"2024-04-25T00:11:12.516337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:08.978181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513899+0000","last_change":"2024-04-25T00:11:06.854703+0000","last_active":"2024-04-25T00:11:12.513899+0000","last_peered":"2024-04-25T00:11:12.513899+0000","last_clean":"2024-04-25T00:11:12.513899+0000","last_became_active":"2024-04-25T00:11:06.854412+0000","last_became_peered":"2024-04-25T00:11:06.854412+0000","last_unstale":"2024-04-25T00:11:12.513899+0000","last_undegraded":"2024-04-25T00:11:12.513899+0000","last_fullsized":"2024-04-25T00:11:12.513899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:17.037033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514230+0000","last_change":"2024-04-25T00:11:06.850481+0000","last_active":"2024-04-25T00:11:12.514230+0000","last_peered":"2024-04-25T00:11:12.514230+0000","last_clean":"2024-04-25T00:11:12.514230+0000","last_became_active":"2024-04-25T00:11:06.850219+0000","last_became_peered":"2024-04-25T00:11:06.850219+0000","last_unstale":"2024-04-25T00:11:12.514230+0000","last_undegraded":"2024-04-25T00:11:12.514230+0000","last_fullsized":"2024-04-25T00:11:12.514230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514123+0000","last_change":"2024-04-25T00:11:06.841945+0000","last_active":"2024-04-25T00:11:12.514123+0000","last_peered":"2024-04-25T00:11:12.514123+0000","last_clean":"2024-04-25T00:11:12.514123+0000","last_became_active":"2024-04-25T00:11:06.841645+0000","last_became_peered":"2024-04-25T00:11:06.841645+0000","last_unstale":"2024-04-25T00:11:12.514123+0000","last_undegraded":"2024-04-25T00:11:12.514123+0000","last_fullsized":"2024-04-25T00:11:12.514123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:40.495315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874037+0000","last_change":"2024-04-25T00:11:06.856378+0000","last_active":"2024-04-25T00:11:11.874037+0000","last_peered":"2024-04-25T00:11:11.874037+0000","last_clean":"2024-04-25T00:11:11.874037+0000","last_became_active":"2024-04-25T00:11:06.856108+0000","last_became_peered":"2024-04-25T00:11:06.856108+0000","last_unstale":"2024-04-25T00:11:11.874037+0000","last_undegraded":"2024-04-25T00:11:11.874037+0000","last_fullsized":"2024-04-25T00:11:11.874037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515971+0000","last_change":"2024-04-25T00:11:06.868107+0000","last_active":"2024-04-25T00:11:12.515971+0000","last_peered":"2024-04-25T00:11:12.515971+0000","last_clean":"2024-04-25T00:11:12.515971+0000","last_became_active":"2024-04-25T00:11:06.867822+0000","last_became_peered":"2024-04-25T00:11:06.867822+0000","last_unstale":"2024-04-25T00:11:12.515971+0000","last_undegraded":"2024-04-25T00:11:12.515971+0000","last_fullsized":"2024-04-25T00:11:12.515971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:51.375329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874207+0000","last_change":"2024-04-25T00:11:06.848480+0000","last_active":"2024-04-25T00:11:11.874207+0000","last_peered":"2024-04-25T00:11:11.874207+0000","last_clean":"2024-04-25T00:11:11.874207+0000","last_became_active":"2024-04-25T00:11:06.848228+0000","last_became_peered":"2024-04-25T00:11:06.848228+0000","last_unstale":"2024-04-25T00:11:11.874207+0000","last_undegraded":"2024-04-25T00:11:11.874207+0000","last_fullsized":"2024-04-25T00:11:11.874207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874653+0000","last_change":"2024-04-25T00:11:06.852645+0000","last_active":"2024-04-25T00:11:11.874653+0000","last_peered":"2024-04-25T00:11:11.874653+0000","last_clean":"2024-04-25T00:11:11.874653+0000","last_became_active":"2024-04-25T00:11:06.852364+0000","last_became_peered":"2024-04-25T00:11:06.852364+0000","last_unstale":"2024-04-25T00:11:11.874653+0000","last_undegraded":"2024-04-25T00:11:11.874653+0000","last_fullsized":"2024-04-25T00:11:11.874653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514727+0000","last_change":"2024-04-25T00:11:06.858052+0000","last_active":"2024-04-25T00:11:12.514727+0000","last_peered":"2024-04-25T00:11:12.514727+0000","last_clean":"2024-04-25T00:11:12.514727+0000","last_became_active":"2024-04-25T00:11:06.857402+0000","last_became_peered":"2024-04-25T00:11:06.857402+0000","last_unstale":"2024-04-25T00:11:12.514727+0000","last_undegraded":"2024-04-25T00:11:12.514727+0000","last_fullsized":"2024-04-25T00:11:12.514727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:03.766175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874165+0000","last_change":"2024-04-25T00:11:06.852942+0000","last_active":"2024-04-25T00:11:11.874165+0000","last_peered":"2024-04-25T00:11:11.874165+0000","last_clean":"2024-04-25T00:11:11.874165+0000","last_became_active":"2024-04-25T00:11:06.852671+0000","last_became_peered":"2024-04-25T00:11:06.852671+0000","last_unstale":"2024-04-25T00:11:11.874165+0000","last_undegraded":"2024-04-25T00:11:11.874165+0000","last_fullsized":"2024-04-25T00:11:11.874165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515899+0000","last_change":"2024-04-25T00:11:06.853657+0000","last_active":"2024-04-25T00:11:12.515899+0000","last_peered":"2024-04-25T00:11:12.515899+0000","last_clean":"2024-04-25T00:11:12.515899+0000","last_became_active":"2024-04-25T00:11:06.853342+0000","last_became_peered":"2024-04-25T00:11:06.853342+0000","last_unstale":"2024-04-25T00:11:12.515899+0000","last_undegraded":"2024-04-25T00:11:12.515899+0000","last_fullsized":"2024-04-25T00:11:12.515899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:43.612611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514817+0000","last_change":"2024-04-25T00:11:06.854058+0000","last_active":"2024-04-25T00:11:12.514817+0000","last_peered":"2024-04-25T00:11:12.514817+0000","last_clean":"2024-04-25T00:11:12.514817+0000","last_became_active":"2024-04-25T00:11:06.853803+0000","last_became_peered":"2024-04-25T00:11:06.853803+0000","last_unstale":"2024-04-25T00:11:12.514817+0000","last_undegraded":"2024-04-25T00:11:12.514817+0000","last_fullsized":"2024-04-25T00:11:12.514817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514199+0000","last_change":"2024-04-25T00:11:06.851312+0000","last_active":"2024-04-25T00:11:12.514199+0000","last_peered":"2024-04-25T00:11:12.514199+0000","last_clean":"2024-04-25T00:11:12.514199+0000","last_became_active":"2024-04-25T00:11:06.851038+0000","last_became_peered":"2024-04-25T00:11:06.851038+0000","last_unstale":"2024-04-25T00:11:12.514199+0000","last_undegraded":"2024-04-25T00:11:12.514199+0000","last_fullsized":"2024-04-25T00:11:12.514199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.219642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514597+0000","last_change":"2024-04-25T00:11:06.860160+0000","last_active":"2024-04-25T00:11:12.514597+0000","last_peered":"2024-04-25T00:11:12.514597+0000","last_clean":"2024-04-25T00:11:12.514597+0000","last_became_active":"2024-04-25T00:11:06.859259+0000","last_became_peered":"2024-04-25T00:11:06.859259+0000","last_unstale":"2024-04-25T00:11:12.514597+0000","last_undegraded":"2024-04-25T00:11:12.514597+0000","last_fullsized":"2024-04-25T00:11:12.514597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:49.414621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513796+0000","last_change":"2024-04-25T00:11:06.848844+0000","last_active":"2024-04-25T00:11:12.513796+0000","last_peered":"2024-04-25T00:11:12.513796+0000","last_clean":"2024-04-25T00:11:12.513796+0000","last_became_active":"2024-04-25T00:11:06.848597+0000","last_became_peered":"2024-04-25T00:11:06.848597+0000","last_unstale":"2024-04-25T00:11:12.513796+0000","last_undegraded":"2024-04-25T00:11:12.513796+0000","last_fullsized":"2024-04-25T00:11:12.513796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515039+0000","last_change":"2024-04-25T00:11:06.844764+0000","last_active":"2024-04-25T00:11:12.515039+0000","last_peered":"2024-04-25T00:11:12.515039+0000","last_clean":"2024-04-25T00:11:12.515039+0000","last_became_active":"2024-04-25T00:11:06.844484+0000","last_became_peered":"2024-04-25T00:11:06.844484+0000","last_unstale":"2024-04-25T00:11:12.515039+0000","last_undegraded":"2024-04-25T00:11:12.515039+0000","last_fullsized":"2024-04-25T00:11:12.515039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514486+0000","last_change":"2024-04-25T00:11:06.860556+0000","last_active":"2024-04-25T00:11:12.514486+0000","last_peered":"2024-04-25T00:11:12.514486+0000","last_clean":"2024-04-25T00:11:12.514486+0000","last_became_active":"2024-04-25T00:11:06.860210+0000","last_became_peered":"2024-04-25T00:11:06.860210+0000","last_unstale":"2024-04-25T00:11:12.514486+0000","last_undegraded":"2024-04-25T00:11:12.514486+0000","last_fullsized":"2024-04-25T00:11:12.514486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:17.037033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:11:29.766 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:11:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb761600640 1 Processor -- start 2024-04-25T00:11:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb761600640 1 -- start start 2024-04-25T00:11:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb761600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb75c15c270 0x7fb75c15ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb761600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75c0ce0f0 0x7fb75c0cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb761600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb75c0cae20 con 0x7fb75c072740 2024-04-25T00:11:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb761600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb75c15c650 con 0x7fb75c15c270 2024-04-25T00:11:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb761600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb75c0cb100 con 0x7fb75c0ce0f0 2024-04-25T00:11:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb75be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb75c15c270 0x7fb75c15ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb760c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb75c072740 0x7fb75c0ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:46912/0 (socket says 172.21.15.83:46912) 2024-04-25T00:11:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb760c00640 1 -- 172.21.15.83:0/3488310900 learned_addr learned my addr 172.21.15.83:0/3488310900 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb75b400640 1 --2- 172.21.15.83:0/3488310900 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75c0ce0f0 0x7fb75c0cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb75be00640 1 -- 172.21.15.83:0/3488310900 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75c0ce0f0 msgr2=0x7fb75c0cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb75be00640 1 --2- 172.21.15.83:0/3488310900 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75c0ce0f0 0x7fb75c0cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb75be00640 1 -- 172.21.15.83:0/3488310900 >> v1:172.21.15.86:6789/0 conn(0x7fb75c072740 legacy=0x7fb75c0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb75be00640 1 -- 172.21.15.83:0/3488310900 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75c0cb3e0 con 0x7fb75c15c270 2024-04-25T00:11:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.940+0000 7fb75aa00640 1 -- 172.21.15.83:0/3488310900 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1393679796 0 0) 0x7fb75c0cae20 con 0x7fb75c072740 2024-04-25T00:11:29.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.944+0000 7fb75be00640 1 --2- 172.21.15.83:0/3488310900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c15c270 0x7fb75c15ca20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0097f0 tx=0x7fb74c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fcee7609237f482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.944+0000 7fb75aa00640 1 -- 172.21.15.83:0/3488310900 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0b2070 con 0x7fb75c15c270 2024-04-25T00:11:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.944+0000 7fb75aa00640 1 -- 172.21.15.83:0/3488310900 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74c0ad640 con 0x7fb75c15c270 2024-04-25T00:11:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.944+0000 7fb75aa00640 1 -- 172.21.15.83:0/3488310900 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0ada10 con 0x7fb75c15c270 2024-04-25T00:11:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.944+0000 7fb761600640 1 -- 172.21.15.83:0/3488310900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c15c270 msgr2=0x7fb75c15ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.944+0000 7fb761600640 1 --2- 172.21.15.83:0/3488310900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c15c270 0x7fb75c15ca20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0097f0 tx=0x7fb74c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.944+0000 7fb761600640 1 -- 172.21.15.83:0/3488310900 shutdown_connections 2024-04-25T00:11:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.944+0000 7fb761600640 1 --2- 172.21.15.83:0/3488310900 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75c0ce0f0 0x7fb75c0cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.944+0000 7fb761600640 1 --2- 172.21.15.83:0/3488310900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c15c270 0x7fb75c15ca20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.944+0000 7fb761600640 1 -- 172.21.15.83:0/3488310900 >> 172.21.15.83:0/3488310900 conn(0x7fb75c0d0210 msgr2=0x7fb75c0d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.944+0000 7fb761600640 1 -- 172.21.15.83:0/3488310900 shutdown_connections 2024-04-25T00:11:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.944+0000 7fb761600640 1 -- 172.21.15.83:0/3488310900 wait complete. 2024-04-25T00:11:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb761600640 1 Processor -- start 2024-04-25T00:11:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb761600640 1 -- start start 2024-04-25T00:11:29.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb761600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75c072740 0x7fb75c0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb761600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c0ce0f0 0x7fb75c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb761600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c15c270 0x7fb75c154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb761600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c0cc580 con 0x7fb75c15c270 2024-04-25T00:11:29.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb761600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c0cc400 con 0x7fb75c0ce0f0 2024-04-25T00:11:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb761600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb75c0cc700 con 0x7fb75c072740 2024-04-25T00:11:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb760c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c15c270 0x7fb75c154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb760c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c15c270 0x7fb75c154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36668/0 (socket says 172.21.15.83:36668) 2024-04-25T00:11:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb760c00640 1 -- 172.21.15.83:0/476116626 learned_addr learned my addr 172.21.15.83:0/476116626 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb75be00640 1 --2- 172.21.15.83:0/476116626 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75c072740 0x7fb75c0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb75b400640 1 --2- 172.21.15.83:0/476116626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c0ce0f0 0x7fb75c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb760c00640 1 -- 172.21.15.83:0/476116626 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75c072740 msgr2=0x7fb75c0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb760c00640 1 --2- 172.21.15.83:0/476116626 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75c072740 0x7fb75c0c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb760c00640 1 -- 172.21.15.83:0/476116626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c0ce0f0 msgr2=0x7fb75c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb760c00640 1 --2- 172.21.15.83:0/476116626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c0ce0f0 0x7fb75c0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb760c00640 1 -- 172.21.15.83:0/476116626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75c0c7fd0 con 0x7fb75c15c270 2024-04-25T00:11:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb760c00640 1 --2- 172.21.15.83:0/476116626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c15c270 0x7fb75c154c70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb75400c730 tx=0x7fb75400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb753e00640 1 -- 172.21.15.83:0/476116626 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754018070 con 0x7fb75c15c270 2024-04-25T00:11:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb753e00640 1 -- 172.21.15.83:0/476116626 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75400ce40 con 0x7fb75c15c270 2024-04-25T00:11:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb761600640 1 -- 172.21.15.83:0/476116626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75c0c82b0 con 0x7fb75c15c270 2024-04-25T00:11:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb753e00640 1 -- 172.21.15.83:0/476116626 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75400a860 con 0x7fb75c15c270 2024-04-25T00:11:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.948+0000 7fb761600640 1 -- 172.21.15.83:0/476116626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75c0c8870 con 0x7fb75c15c270 2024-04-25T00:11:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.952+0000 7fb761600640 1 -- 172.21.15.83:0/476116626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb75c0c8d50 con 0x7fb75c15c270 2024-04-25T00:11:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.952+0000 7fb753e00640 1 -- 172.21.15.83:0/476116626 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb75401b070 con 0x7fb75c15c270 2024-04-25T00:11:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.952+0000 7fb753e00640 1 --2- 172.21.15.83:0/476116626 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb73c0678c0 0x7fb73c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.952+0000 7fb753e00640 1 -- 172.21.15.83:0/476116626 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb754085590 con 0x7fb75c15c270 2024-04-25T00:11:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.952+0000 7fb75be00640 1 --2- 172.21.15.83:0/476116626 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb73c0678c0 0x7fb73c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.952+0000 7fb75be00640 1 --2- 172.21.15.83:0/476116626 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb73c0678c0 0x7fb73c069d80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0af000 tx=0x7fb74c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:29.956+0000 7fb753e00640 1 -- 172.21.15.83:0/476116626 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb75c0c8d50 con 0x7fb75c15c270 2024-04-25T00:11:30.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.108+0000 7fb761600640 1 -- 172.21.15.83:0/476116626 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb75c06d800 con 0x7fb73c0678c0 2024-04-25T00:11:30.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.116+0000 7fb753e00640 1 -- 172.21.15.83:0/476116626 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fb75c06d800 con 0x7fb73c0678c0 2024-04-25T00:11:30.119 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:30.124 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:11:30.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.120+0000 7fb761600640 1 -- 172.21.15.83:0/476116626 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb73c0678c0 msgr2=0x7fb73c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.120+0000 7fb761600640 1 --2- 172.21.15.83:0/476116626 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb73c0678c0 0x7fb73c069d80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0af000 tx=0x7fb74c008040 comp rx=0 tx=0).stop 2024-04-25T00:11:30.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.120+0000 7fb761600640 1 -- 172.21.15.83:0/476116626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c15c270 msgr2=0x7fb75c154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.120+0000 7fb761600640 1 --2- 172.21.15.83:0/476116626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c15c270 0x7fb75c154c70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb75400c730 tx=0x7fb75400cbf0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.120+0000 7fb761600640 1 -- 172.21.15.83:0/476116626 shutdown_connections 2024-04-25T00:11:30.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.120+0000 7fb761600640 1 --2- 172.21.15.83:0/476116626 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb73c0678c0 0x7fb73c069d80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.120+0000 7fb761600640 1 --2- 172.21.15.83:0/476116626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c15c270 0x7fb75c154c70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.120+0000 7fb761600640 1 --2- 172.21.15.83:0/476116626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c0ce0f0 0x7fb75c0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.120+0000 7fb761600640 1 --2- 172.21.15.83:0/476116626 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75c072740 0x7fb75c0c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.120+0000 7fb761600640 1 -- 172.21.15.83:0/476116626 >> 172.21.15.83:0/476116626 conn(0x7fb75c0d0210 msgr2=0x7fb75c15e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:30.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.120+0000 7fb761600640 1 -- 172.21.15.83:0/476116626 shutdown_connections 2024-04-25T00:11:30.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.120+0000 7fb761600640 1 -- 172.21.15.83:0/476116626 wait complete. 2024-04-25T00:11:30.142 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-25T00:11:28.447534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219404,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755316,"statfs":{"total":773094113280,"available":772869443584,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002456"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515374+0000","last_change":"2024-04-25T00:11:05.833868+0000","last_active":"2024-04-25T00:11:12.515374+0000","last_peered":"2024-04-25T00:11:12.515374+0000","last_clean":"2024-04-25T00:11:12.515374+0000","last_became_active":"2024-04-25T00:11:05.833603+0000","last_became_peered":"2024-04-25T00:11:05.833603+0000","last_unstale":"2024-04-25T00:11:12.515374+0000","last_undegraded":"2024-04-25T00:11:12.515374+0000","last_fullsized":"2024-04-25T00:11:12.515374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:32.084989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874560+0000","last_change":"2024-04-25T00:11:06.856161+0000","last_active":"2024-04-25T00:11:11.874560+0000","last_peered":"2024-04-25T00:11:11.874560+0000","last_clean":"2024-04-25T00:11:11.874560+0000","last_became_active":"2024-04-25T00:11:06.855897+0000","last_became_peered":"2024-04-25T00:11:06.855897+0000","last_unstale":"2024-04-25T00:11:11.874560+0000","last_undegraded":"2024-04-25T00:11:11.874560+0000","last_fullsized":"2024-04-25T00:11:11.874560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.219642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873904+0000","last_change":"2024-04-25T00:11:10.976976+0000","last_active":"2024-04-25T00:11:11.873904+0000","last_peered":"2024-04-25T00:11:11.873904+0000","last_clean":"2024-04-25T00:11:11.873904+0000","last_became_active":"2024-04-25T00:11:10.975609+0000","last_became_peered":"2024-04-25T00:11:10.975609+0000","last_unstale":"2024-04-25T00:11:11.873904+0000","last_undegraded":"2024-04-25T00:11:11.873904+0000","last_fullsized":"2024-04-25T00:11:11.873904+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:07.644687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874723+0000","last_change":"2024-04-25T00:11:05.835352+0000","last_active":"2024-04-25T00:11:11.874723+0000","last_peered":"2024-04-25T00:11:11.874723+0000","last_clean":"2024-04-25T00:11:11.874723+0000","last_became_active":"2024-04-25T00:11:05.835056+0000","last_became_peered":"2024-04-25T00:11:05.835056+0000","last_unstale":"2024-04-25T00:11:11.874723+0000","last_undegraded":"2024-04-25T00:11:11.874723+0000","last_fullsized":"2024-04-25T00:11:11.874723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515231+0000","last_change":"2024-04-25T00:11:06.855492+0000","last_active":"2024-04-25T00:11:12.515231+0000","last_peered":"2024-04-25T00:11:12.515231+0000","last_clean":"2024-04-25T00:11:12.515231+0000","last_became_active":"2024-04-25T00:11:06.854970+0000","last_became_peered":"2024-04-25T00:11:06.854970+0000","last_unstale":"2024-04-25T00:11:12.515231+0000","last_undegraded":"2024-04-25T00:11:12.515231+0000","last_fullsized":"2024-04-25T00:11:12.515231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515424+0000","last_change":"2024-04-25T00:11:10.899597+0000","last_active":"2024-04-25T00:11:12.515424+0000","last_peered":"2024-04-25T00:11:12.515424+0000","last_clean":"2024-04-25T00:11:12.515424+0000","last_became_active":"2024-04-25T00:11:10.899136+0000","last_became_peered":"2024-04-25T00:11:10.899136+0000","last_unstale":"2024-04-25T00:11:12.515424+0000","last_undegraded":"2024-04-25T00:11:12.515424+0000","last_fullsized":"2024-04-25T00:11:12.515424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:40.036652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514540+0000","last_change":"2024-04-25T00:11:05.848984+0000","last_active":"2024-04-25T00:11:12.514540+0000","last_peered":"2024-04-25T00:11:12.514540+0000","last_clean":"2024-04-25T00:11:12.514540+0000","last_became_active":"2024-04-25T00:11:05.848618+0000","last_became_peered":"2024-04-25T00:11:05.848618+0000","last_unstale":"2024-04-25T00:11:12.514540+0000","last_undegraded":"2024-04-25T00:11:12.514540+0000","last_fullsized":"2024-04-25T00:11:12.514540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:50.369863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514723+0000","last_change":"2024-04-25T00:11:06.875779+0000","last_active":"2024-04-25T00:11:12.514723+0000","last_peered":"2024-04-25T00:11:12.514723+0000","last_clean":"2024-04-25T00:11:12.514723+0000","last_became_active":"2024-04-25T00:11:06.875440+0000","last_became_peered":"2024-04-25T00:11:06.875440+0000","last_unstale":"2024-04-25T00:11:12.514723+0000","last_undegraded":"2024-04-25T00:11:12.514723+0000","last_fullsized":"2024-04-25T00:11:12.514723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:43.565714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516147+0000","last_change":"2024-04-25T00:11:10.899794+0000","last_active":"2024-04-25T00:11:12.516147+0000","last_peered":"2024-04-25T00:11:12.516147+0000","last_clean":"2024-04-25T00:11:12.516147+0000","last_became_active":"2024-04-25T00:11:10.899313+0000","last_became_peered":"2024-04-25T00:11:10.899313+0000","last_unstale":"2024-04-25T00:11:12.516147+0000","last_undegraded":"2024-04-25T00:11:12.516147+0000","last_fullsized":"2024-04-25T00:11:12.516147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:51.269552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514246+0000","last_change":"2024-04-25T00:11:05.847672+0000","last_active":"2024-04-25T00:11:12.514246+0000","last_peered":"2024-04-25T00:11:12.514246+0000","last_clean":"2024-04-25T00:11:12.514246+0000","last_became_active":"2024-04-25T00:11:05.847316+0000","last_became_peered":"2024-04-25T00:11:05.847316+0000","last_unstale":"2024-04-25T00:11:12.514246+0000","last_undegraded":"2024-04-25T00:11:12.514246+0000","last_fullsized":"2024-04-25T00:11:12.514246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515600+0000","last_change":"2024-04-25T00:11:06.860047+0000","last_active":"2024-04-25T00:11:12.515600+0000","last_peered":"2024-04-25T00:11:12.515600+0000","last_clean":"2024-04-25T00:11:12.515600+0000","last_became_active":"2024-04-25T00:11:06.859782+0000","last_became_peered":"2024-04-25T00:11:06.859782+0000","last_unstale":"2024-04-25T00:11:12.515600+0000","last_undegraded":"2024-04-25T00:11:12.515600+0000","last_fullsized":"2024-04-25T00:11:12.515600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:43.612611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516469+0000","last_change":"2024-04-25T00:11:10.895749+0000","last_active":"2024-04-25T00:11:12.516469+0000","last_peered":"2024-04-25T00:11:12.516469+0000","last_clean":"2024-04-25T00:11:12.516469+0000","last_became_active":"2024-04-25T00:11:10.895359+0000","last_became_peered":"2024-04-25T00:11:10.895359+0000","last_unstale":"2024-04-25T00:11:12.516469+0000","last_undegraded":"2024-04-25T00:11:12.516469+0000","last_fullsized":"2024-04-25T00:11:12.516469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873961+0000","last_change":"2024-04-25T00:11:06.865465+0000","last_active":"2024-04-25T00:11:11.873961+0000","last_peered":"2024-04-25T00:11:11.873961+0000","last_clean":"2024-04-25T00:11:11.873961+0000","last_became_active":"2024-04-25T00:11:06.865020+0000","last_became_peered":"2024-04-25T00:11:06.865020+0000","last_unstale":"2024-04-25T00:11:11.873961+0000","last_undegraded":"2024-04-25T00:11:11.873961+0000","last_fullsized":"2024-04-25T00:11:11.873961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514541+0000","last_change":"2024-04-25T00:11:05.844750+0000","last_active":"2024-04-25T00:11:12.514541+0000","last_peered":"2024-04-25T00:11:12.514541+0000","last_clean":"2024-04-25T00:11:12.514541+0000","last_became_active":"2024-04-25T00:11:05.844407+0000","last_became_peered":"2024-04-25T00:11:05.844407+0000","last_unstale":"2024-04-25T00:11:12.514541+0000","last_undegraded":"2024-04-25T00:11:12.514541+0000","last_fullsized":"2024-04-25T00:11:12.514541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:34.248155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516163+0000","last_change":"2024-04-25T00:11:10.893480+0000","last_active":"2024-04-25T00:11:12.516163+0000","last_peered":"2024-04-25T00:11:12.516163+0000","last_clean":"2024-04-25T00:11:12.516163+0000","last_became_active":"2024-04-25T00:11:10.893051+0000","last_became_peered":"2024-04-25T00:11:10.893051+0000","last_unstale":"2024-04-25T00:11:12.516163+0000","last_undegraded":"2024-04-25T00:11:12.516163+0000","last_fullsized":"2024-04-25T00:11:12.516163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:40.531377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874153+0000","last_change":"2024-04-25T00:11:06.845105+0000","last_active":"2024-04-25T00:11:11.874153+0000","last_peered":"2024-04-25T00:11:11.874153+0000","last_clean":"2024-04-25T00:11:11.874153+0000","last_became_active":"2024-04-25T00:11:06.844686+0000","last_became_peered":"2024-04-25T00:11:06.844686+0000","last_unstale":"2024-04-25T00:11:11.874153+0000","last_undegraded":"2024-04-25T00:11:11.874153+0000","last_fullsized":"2024-04-25T00:11:11.874153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:25.890860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514977+0000","last_change":"2024-04-25T00:11:05.836866+0000","last_active":"2024-04-25T00:11:12.514977+0000","last_peered":"2024-04-25T00:11:12.514977+0000","last_clean":"2024-04-25T00:11:12.514977+0000","last_became_active":"2024-04-25T00:11:05.836578+0000","last_became_peered":"2024-04-25T00:11:05.836578+0000","last_unstale":"2024-04-25T00:11:12.514977+0000","last_undegraded":"2024-04-25T00:11:12.514977+0000","last_fullsized":"2024-04-25T00:11:12.514977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515742+0000","last_change":"2024-04-25T00:11:10.903356+0000","last_active":"2024-04-25T00:11:12.515742+0000","last_peered":"2024-04-25T00:11:12.515742+0000","last_clean":"2024-04-25T00:11:12.515742+0000","last_became_active":"2024-04-25T00:11:10.902845+0000","last_became_peered":"2024-04-25T00:11:10.902845+0000","last_unstale":"2024-04-25T00:11:12.515742+0000","last_undegraded":"2024-04-25T00:11:12.515742+0000","last_fullsized":"2024-04-25T00:11:12.515742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:47.662521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874413+0000","last_change":"2024-04-25T00:11:06.849565+0000","last_active":"2024-04-25T00:11:11.874413+0000","last_peered":"2024-04-25T00:11:11.874413+0000","last_clean":"2024-04-25T00:11:11.874413+0000","last_became_active":"2024-04-25T00:11:06.849275+0000","last_became_peered":"2024-04-25T00:11:06.849275+0000","last_unstale":"2024-04-25T00:11:11.874413+0000","last_undegraded":"2024-04-25T00:11:11.874413+0000","last_fullsized":"2024-04-25T00:11:11.874413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514312+0000","last_change":"2024-04-25T00:11:05.850573+0000","last_active":"2024-04-25T00:11:12.514312+0000","last_peered":"2024-04-25T00:11:12.514312+0000","last_clean":"2024-04-25T00:11:12.514312+0000","last_became_active":"2024-04-25T00:11:05.850239+0000","last_became_peered":"2024-04-25T00:11:05.850239+0000","last_unstale":"2024-04-25T00:11:12.514312+0000","last_undegraded":"2024-04-25T00:11:12.514312+0000","last_fullsized":"2024-04-25T00:11:12.514312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:41.285583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515409+0000","last_change":"2024-04-25T00:11:10.974826+0000","last_active":"2024-04-25T00:11:12.515409+0000","last_peered":"2024-04-25T00:11:12.515409+0000","last_clean":"2024-04-25T00:11:12.515409+0000","last_became_active":"2024-04-25T00:11:10.974483+0000","last_became_peered":"2024-04-25T00:11:10.974483+0000","last_unstale":"2024-04-25T00:11:12.515409+0000","last_undegraded":"2024-04-25T00:11:12.515409+0000","last_fullsized":"2024-04-25T00:11:12.515409+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:13.028091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516075+0000","last_change":"2024-04-25T00:11:06.857094+0000","last_active":"2024-04-25T00:11:12.516075+0000","last_peered":"2024-04-25T00:11:12.516075+0000","last_clean":"2024-04-25T00:11:12.516075+0000","last_became_active":"2024-04-25T00:11:06.856782+0000","last_became_peered":"2024-04-25T00:11:06.856782+0000","last_unstale":"2024-04-25T00:11:12.516075+0000","last_undegraded":"2024-04-25T00:11:12.516075+0000","last_fullsized":"2024-04-25T00:11:12.516075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:10.105280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513838+0000","last_change":"2024-04-25T00:11:05.849504+0000","last_active":"2024-04-25T00:11:12.513838+0000","last_peered":"2024-04-25T00:11:12.513838+0000","last_clean":"2024-04-25T00:11:12.513838+0000","last_became_active":"2024-04-25T00:11:05.847651+0000","last_became_peered":"2024-04-25T00:11:05.847651+0000","last_unstale":"2024-04-25T00:11:12.513838+0000","last_undegraded":"2024-04-25T00:11:12.513838+0000","last_fullsized":"2024-04-25T00:11:12.513838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515751+0000","last_change":"2024-04-25T00:11:10.893589+0000","last_active":"2024-04-25T00:11:12.515751+0000","last_peered":"2024-04-25T00:11:12.515751+0000","last_clean":"2024-04-25T00:11:12.515751+0000","last_became_active":"2024-04-25T00:11:10.893172+0000","last_became_peered":"2024-04-25T00:11:10.893172+0000","last_unstale":"2024-04-25T00:11:12.515751+0000","last_undegraded":"2024-04-25T00:11:12.515751+0000","last_fullsized":"2024-04-25T00:11:12.515751+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:53.464531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514257+0000","last_change":"2024-04-25T00:11:06.850048+0000","last_active":"2024-04-25T00:11:12.514257+0000","last_peered":"2024-04-25T00:11:12.514257+0000","last_clean":"2024-04-25T00:11:12.514257+0000","last_became_active":"2024-04-25T00:11:06.849785+0000","last_became_peered":"2024-04-25T00:11:06.849785+0000","last_unstale":"2024-04-25T00:11:12.514257+0000","last_undegraded":"2024-04-25T00:11:12.514257+0000","last_fullsized":"2024-04-25T00:11:12.514257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874790+0000","last_change":"2024-04-25T00:11:05.836100+0000","last_active":"2024-04-25T00:11:11.874790+0000","last_peered":"2024-04-25T00:11:11.874790+0000","last_clean":"2024-04-25T00:11:11.874790+0000","last_became_active":"2024-04-25T00:11:05.835841+0000","last_became_peered":"2024-04-25T00:11:05.835841+0000","last_unstale":"2024-04-25T00:11:11.874790+0000","last_undegraded":"2024-04-25T00:11:11.874790+0000","last_fullsized":"2024-04-25T00:11:11.874790+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:53.690870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515708+0000","last_change":"2024-04-25T00:11:10.975785+0000","last_active":"2024-04-25T00:11:12.515708+0000","last_peered":"2024-04-25T00:11:12.515708+0000","last_clean":"2024-04-25T00:11:12.515708+0000","last_became_active":"2024-04-25T00:11:10.974244+0000","last_became_peered":"2024-04-25T00:11:10.974244+0000","last_unstale":"2024-04-25T00:11:12.515708+0000","last_undegraded":"2024-04-25T00:11:12.515708+0000","last_fullsized":"2024-04-25T00:11:12.515708+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:47.615624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874580+0000","last_change":"2024-04-25T00:11:06.854507+0000","last_active":"2024-04-25T00:11:11.874580+0000","last_peered":"2024-04-25T00:11:11.874580+0000","last_clean":"2024-04-25T00:11:11.874580+0000","last_became_active":"2024-04-25T00:11:06.853694+0000","last_became_peered":"2024-04-25T00:11:06.853694+0000","last_unstale":"2024-04-25T00:11:11.874580+0000","last_undegraded":"2024-04-25T00:11:11.874580+0000","last_fullsized":"2024-04-25T00:11:11.874580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:36.481467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515093+0000","last_change":"2024-04-25T00:11:05.842386+0000","last_active":"2024-04-25T00:11:12.515093+0000","last_peered":"2024-04-25T00:11:12.515093+0000","last_clean":"2024-04-25T00:11:12.515093+0000","last_became_active":"2024-04-25T00:11:05.842064+0000","last_became_peered":"2024-04-25T00:11:05.842064+0000","last_unstale":"2024-04-25T00:11:12.515093+0000","last_undegraded":"2024-04-25T00:11:12.515093+0000","last_fullsized":"2024-04-25T00:11:12.515093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873860+0000","last_change":"2024-04-25T00:11:10.969122+0000","last_active":"2024-04-25T00:11:11.873860+0000","last_peered":"2024-04-25T00:11:11.873860+0000","last_clean":"2024-04-25T00:11:11.873860+0000","last_became_active":"2024-04-25T00:11:10.968532+0000","last_became_peered":"2024-04-25T00:11:10.968532+0000","last_unstale":"2024-04-25T00:11:11.873860+0000","last_undegraded":"2024-04-25T00:11:11.873860+0000","last_fullsized":"2024-04-25T00:11:11.873860+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:23:46.222075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514772+0000","last_change":"2024-04-25T00:11:06.847752+0000","last_active":"2024-04-25T00:11:12.514772+0000","last_peered":"2024-04-25T00:11:12.514772+0000","last_clean":"2024-04-25T00:11:12.514772+0000","last_became_active":"2024-04-25T00:11:06.847378+0000","last_became_peered":"2024-04-25T00:11:06.847378+0000","last_unstale":"2024-04-25T00:11:12.514772+0000","last_undegraded":"2024-04-25T00:11:12.514772+0000","last_fullsized":"2024-04-25T00:11:12.514772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514712+0000","last_change":"2024-04-25T00:11:05.849451+0000","last_active":"2024-04-25T00:11:12.514712+0000","last_peered":"2024-04-25T00:11:12.514712+0000","last_clean":"2024-04-25T00:11:12.514712+0000","last_became_active":"2024-04-25T00:11:05.849028+0000","last_became_peered":"2024-04-25T00:11:05.849028+0000","last_unstale":"2024-04-25T00:11:12.514712+0000","last_undegraded":"2024-04-25T00:11:12.514712+0000","last_fullsized":"2024-04-25T00:11:12.514712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:41.285583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513859+0000","last_change":"2024-04-25T00:11:10.973749+0000","last_active":"2024-04-25T00:11:12.513859+0000","last_peered":"2024-04-25T00:11:12.513859+0000","last_clean":"2024-04-25T00:11:12.513859+0000","last_became_active":"2024-04-25T00:11:10.973369+0000","last_became_peered":"2024-04-25T00:11:10.973369+0000","last_unstale":"2024-04-25T00:11:12.513859+0000","last_undegraded":"2024-04-25T00:11:12.513859+0000","last_fullsized":"2024-04-25T00:11:12.513859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:40.531377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515701+0000","last_change":"2024-04-25T00:11:06.859428+0000","last_active":"2024-04-25T00:11:12.515701+0000","last_peered":"2024-04-25T00:11:12.515701+0000","last_clean":"2024-04-25T00:11:12.515701+0000","last_became_active":"2024-04-25T00:11:06.859101+0000","last_became_peered":"2024-04-25T00:11:06.859101+0000","last_unstale":"2024-04-25T00:11:12.515701+0000","last_undegraded":"2024-04-25T00:11:12.515701+0000","last_fullsized":"2024-04-25T00:11:12.515701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:59.645042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514881+0000","last_change":"2024-04-25T00:11:05.840437+0000","last_active":"2024-04-25T00:11:12.514881+0000","last_peered":"2024-04-25T00:11:12.514881+0000","last_clean":"2024-04-25T00:11:12.514881+0000","last_became_active":"2024-04-25T00:11:05.840120+0000","last_became_peered":"2024-04-25T00:11:05.840120+0000","last_unstale":"2024-04-25T00:11:12.514881+0000","last_undegraded":"2024-04-25T00:11:12.514881+0000","last_fullsized":"2024-04-25T00:11:12.514881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514595+0000","last_change":"2024-04-25T00:11:10.896951+0000","last_active":"2024-04-25T00:11:12.514595+0000","last_peered":"2024-04-25T00:11:12.514595+0000","last_clean":"2024-04-25T00:11:12.514595+0000","last_became_active":"2024-04-25T00:11:10.896547+0000","last_became_peered":"2024-04-25T00:11:10.896547+0000","last_unstale":"2024-04-25T00:11:12.514595+0000","last_undegraded":"2024-04-25T00:11:12.514595+0000","last_fullsized":"2024-04-25T00:11:12.514595+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516507+0000","last_change":"2024-04-25T00:11:06.846626+0000","last_active":"2024-04-25T00:11:12.516507+0000","last_peered":"2024-04-25T00:11:12.516507+0000","last_clean":"2024-04-25T00:11:12.516507+0000","last_became_active":"2024-04-25T00:11:06.846333+0000","last_became_peered":"2024-04-25T00:11:06.846333+0000","last_unstale":"2024-04-25T00:11:12.516507+0000","last_undegraded":"2024-04-25T00:11:12.516507+0000","last_fullsized":"2024-04-25T00:11:12.516507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:42.291050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515107+0000","last_change":"2024-04-25T00:11:05.839781+0000","last_active":"2024-04-25T00:11:12.515107+0000","last_peered":"2024-04-25T00:11:12.515107+0000","last_clean":"2024-04-25T00:11:12.515107+0000","last_became_active":"2024-04-25T00:11:05.839400+0000","last_became_peered":"2024-04-25T00:11:05.839400+0000","last_unstale":"2024-04-25T00:11:12.515107+0000","last_undegraded":"2024-04-25T00:11:12.515107+0000","last_fullsized":"2024-04-25T00:11:12.515107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514811+0000","last_change":"2024-04-25T00:11:10.892411+0000","last_active":"2024-04-25T00:11:12.514811+0000","last_peered":"2024-04-25T00:11:12.514811+0000","last_clean":"2024-04-25T00:11:12.514811+0000","last_became_active":"2024-04-25T00:11:10.892036+0000","last_became_peered":"2024-04-25T00:11:10.892036+0000","last_unstale":"2024-04-25T00:11:12.514811+0000","last_undegraded":"2024-04-25T00:11:12.514811+0000","last_fullsized":"2024-04-25T00:11:12.514811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:07.816085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513913+0000","last_change":"2024-04-25T00:11:06.842835+0000","last_active":"2024-04-25T00:11:12.513913+0000","last_peered":"2024-04-25T00:11:12.513913+0000","last_clean":"2024-04-25T00:11:12.513913+0000","last_became_active":"2024-04-25T00:11:06.842540+0000","last_became_peered":"2024-04-25T00:11:06.842540+0000","last_unstale":"2024-04-25T00:11:12.513913+0000","last_undegraded":"2024-04-25T00:11:12.513913+0000","last_fullsized":"2024-04-25T00:11:12.513913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:00.950133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875360+0000","last_change":"2024-04-25T00:11:05.829300+0000","last_active":"2024-04-25T00:11:11.875360+0000","last_peered":"2024-04-25T00:11:11.875360+0000","last_clean":"2024-04-25T00:11:11.875360+0000","last_became_active":"2024-04-25T00:11:05.828940+0000","last_became_peered":"2024-04-25T00:11:05.828940+0000","last_unstale":"2024-04-25T00:11:11.875360+0000","last_undegraded":"2024-04-25T00:11:11.875360+0000","last_fullsized":"2024-04-25T00:11:11.875360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:15.738255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515526+0000","last_change":"2024-04-25T00:11:10.895520+0000","last_active":"2024-04-25T00:11:12.515526+0000","last_peered":"2024-04-25T00:11:12.515526+0000","last_clean":"2024-04-25T00:11:12.515526+0000","last_became_active":"2024-04-25T00:11:10.895096+0000","last_became_peered":"2024-04-25T00:11:10.895096+0000","last_unstale":"2024-04-25T00:11:12.515526+0000","last_undegraded":"2024-04-25T00:11:12.515526+0000","last_fullsized":"2024-04-25T00:11:12.515526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:51.996256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513553+0000","last_change":"2024-04-25T00:11:06.862814+0000","last_active":"2024-04-25T00:11:12.513553+0000","last_peered":"2024-04-25T00:11:12.513553+0000","last_clean":"2024-04-25T00:11:12.513553+0000","last_became_active":"2024-04-25T00:11:06.862534+0000","last_became_peered":"2024-04-25T00:11:06.862534+0000","last_unstale":"2024-04-25T00:11:12.513553+0000","last_undegraded":"2024-04-25T00:11:12.513553+0000","last_fullsized":"2024-04-25T00:11:12.513553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:39.726185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513959+0000","last_change":"2024-04-25T00:11:05.849841+0000","last_active":"2024-04-25T00:11:12.513959+0000","last_peered":"2024-04-25T00:11:12.513959+0000","last_clean":"2024-04-25T00:11:12.513959+0000","last_became_active":"2024-04-25T00:11:05.847646+0000","last_became_peered":"2024-04-25T00:11:05.847646+0000","last_unstale":"2024-04-25T00:11:12.513959+0000","last_undegraded":"2024-04-25T00:11:12.513959+0000","last_fullsized":"2024-04-25T00:11:12.513959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:36.649759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514155+0000","last_change":"2024-04-25T00:11:10.974744+0000","last_active":"2024-04-25T00:11:12.514155+0000","last_peered":"2024-04-25T00:11:12.514155+0000","last_clean":"2024-04-25T00:11:12.514155+0000","last_became_active":"2024-04-25T00:11:10.973737+0000","last_became_peered":"2024-04-25T00:11:10.973737+0000","last_unstale":"2024-04-25T00:11:12.514155+0000","last_undegraded":"2024-04-25T00:11:12.514155+0000","last_fullsized":"2024-04-25T00:11:12.514155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:13.028091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875061+0000","last_change":"2024-04-25T00:11:06.850937+0000","last_active":"2024-04-25T00:11:11.875061+0000","last_peered":"2024-04-25T00:11:11.875061+0000","last_clean":"2024-04-25T00:11:11.875061+0000","last_became_active":"2024-04-25T00:11:06.850620+0000","last_became_peered":"2024-04-25T00:11:06.850620+0000","last_unstale":"2024-04-25T00:11:11.875061+0000","last_undegraded":"2024-04-25T00:11:11.875061+0000","last_fullsized":"2024-04-25T00:11:11.875061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514342+0000","last_change":"2024-04-25T00:11:05.841594+0000","last_active":"2024-04-25T00:11:12.514342+0000","last_peered":"2024-04-25T00:11:12.514342+0000","last_clean":"2024-04-25T00:11:12.514342+0000","last_became_active":"2024-04-25T00:11:05.838964+0000","last_became_peered":"2024-04-25T00:11:05.838964+0000","last_unstale":"2024-04-25T00:11:12.514342+0000","last_undegraded":"2024-04-25T00:11:12.514342+0000","last_fullsized":"2024-04-25T00:11:12.514342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514070+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:11:12.514070+0000","last_peered":"2024-04-25T00:11:12.514070+0000","last_clean":"2024-04-25T00:11:12.514070+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:11:12.514070+0000","last_undegraded":"2024-04-25T00:11:12.514070+0000","last_fullsized":"2024-04-25T00:11:12.514070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874137+0000","last_change":"2024-04-25T00:11:10.976384+0000","last_active":"2024-04-25T00:11:11.874137+0000","last_peered":"2024-04-25T00:11:11.874137+0000","last_clean":"2024-04-25T00:11:11.874137+0000","last_became_active":"2024-04-25T00:11:10.975924+0000","last_became_peered":"2024-04-25T00:11:10.975924+0000","last_unstale":"2024-04-25T00:11:11.874137+0000","last_undegraded":"2024-04-25T00:11:11.874137+0000","last_fullsized":"2024-04-25T00:11:11.874137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514383+0000","last_change":"2024-04-25T00:11:06.855965+0000","last_active":"2024-04-25T00:11:12.514383+0000","last_peered":"2024-04-25T00:11:12.514383+0000","last_clean":"2024-04-25T00:11:12.514383+0000","last_became_active":"2024-04-25T00:11:06.853447+0000","last_became_peered":"2024-04-25T00:11:06.853447+0000","last_unstale":"2024-04-25T00:11:12.514383+0000","last_undegraded":"2024-04-25T00:11:12.514383+0000","last_fullsized":"2024-04-25T00:11:12.514383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:14.704188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875959+0000","last_change":"2024-04-25T00:11:05.842420+0000","last_active":"2024-04-25T00:11:11.875959+0000","last_peered":"2024-04-25T00:11:11.875959+0000","last_clean":"2024-04-25T00:11:11.875959+0000","last_became_active":"2024-04-25T00:11:05.842102+0000","last_became_peered":"2024-04-25T00:11:05.842102+0000","last_unstale":"2024-04-25T00:11:11.875959+0000","last_undegraded":"2024-04-25T00:11:11.875959+0000","last_fullsized":"2024-04-25T00:11:11.875959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:14.487344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874194+0000","last_change":"2024-04-25T00:11:10.969423+0000","last_active":"2024-04-25T00:11:11.874194+0000","last_peered":"2024-04-25T00:11:11.874194+0000","last_clean":"2024-04-25T00:11:11.874194+0000","last_became_active":"2024-04-25T00:11:10.968799+0000","last_became_peered":"2024-04-25T00:11:10.968799+0000","last_unstale":"2024-04-25T00:11:11.874194+0000","last_undegraded":"2024-04-25T00:11:11.874194+0000","last_fullsized":"2024-04-25T00:11:11.874194+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514836+0000","last_change":"2024-04-25T00:11:06.859625+0000","last_active":"2024-04-25T00:11:12.514836+0000","last_peered":"2024-04-25T00:11:12.514836+0000","last_clean":"2024-04-25T00:11:12.514836+0000","last_became_active":"2024-04-25T00:11:06.859161+0000","last_became_peered":"2024-04-25T00:11:06.859161+0000","last_unstale":"2024-04-25T00:11:12.514836+0000","last_undegraded":"2024-04-25T00:11:12.514836+0000","last_fullsized":"2024-04-25T00:11:12.514836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:36.481467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515075+0000","last_change":"2024-04-25T00:11:05.839603+0000","last_active":"2024-04-25T00:11:12.515075+0000","last_peered":"2024-04-25T00:11:12.515075+0000","last_clean":"2024-04-25T00:11:12.515075+0000","last_became_active":"2024-04-25T00:11:05.838608+0000","last_became_peered":"2024-04-25T00:11:05.838608+0000","last_unstale":"2024-04-25T00:11:12.515075+0000","last_undegraded":"2024-04-25T00:11:12.515075+0000","last_fullsized":"2024-04-25T00:11:12.515075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:53.690870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514117+0000","last_change":"2024-04-25T00:11:10.900742+0000","last_active":"2024-04-25T00:11:12.514117+0000","last_peered":"2024-04-25T00:11:12.514117+0000","last_clean":"2024-04-25T00:11:12.514117+0000","last_became_active":"2024-04-25T00:11:10.900246+0000","last_became_peered":"2024-04-25T00:11:10.900246+0000","last_unstale":"2024-04-25T00:11:12.514117+0000","last_undegraded":"2024-04-25T00:11:12.514117+0000","last_fullsized":"2024-04-25T00:11:12.514117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:07.816085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514836+0000","last_change":"2024-04-25T00:11:06.870304+0000","last_active":"2024-04-25T00:11:12.514836+0000","last_peered":"2024-04-25T00:11:12.514836+0000","last_clean":"2024-04-25T00:11:12.514836+0000","last_became_active":"2024-04-25T00:11:06.870010+0000","last_became_peered":"2024-04-25T00:11:06.870010+0000","last_unstale":"2024-04-25T00:11:12.514836+0000","last_undegraded":"2024-04-25T00:11:12.514836+0000","last_fullsized":"2024-04-25T00:11:12.514836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:08.978181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.876556+0000","last_change":"2024-04-25T00:11:05.839098+0000","last_active":"2024-04-25T00:11:11.876556+0000","last_peered":"2024-04-25T00:11:11.876556+0000","last_clean":"2024-04-25T00:11:11.876556+0000","last_became_active":"2024-04-25T00:11:05.838694+0000","last_became_peered":"2024-04-25T00:11:05.838694+0000","last_unstale":"2024-04-25T00:11:11.876556+0000","last_undegraded":"2024-04-25T00:11:11.876556+0000","last_fullsized":"2024-04-25T00:11:11.876556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:35.709574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513908+0000","last_change":"2024-04-25T00:11:10.973661+0000","last_active":"2024-04-25T00:11:12.513908+0000","last_peered":"2024-04-25T00:11:12.513908+0000","last_clean":"2024-04-25T00:11:12.513908+0000","last_became_active":"2024-04-25T00:11:10.973198+0000","last_became_peered":"2024-04-25T00:11:10.973198+0000","last_unstale":"2024-04-25T00:11:12.513908+0000","last_undegraded":"2024-04-25T00:11:12.513908+0000","last_fullsized":"2024-04-25T00:11:12.513908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:23:46.222075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514875+0000","last_change":"2024-04-25T00:11:06.878377+0000","last_active":"2024-04-25T00:11:12.514875+0000","last_peered":"2024-04-25T00:11:12.514875+0000","last_clean":"2024-04-25T00:11:12.514875+0000","last_became_active":"2024-04-25T00:11:06.878046+0000","last_became_peered":"2024-04-25T00:11:06.878046+0000","last_unstale":"2024-04-25T00:11:12.514875+0000","last_undegraded":"2024-04-25T00:11:12.514875+0000","last_fullsized":"2024-04-25T00:11:12.514875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:59.645042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515717+0000","last_change":"2024-04-25T00:11:05.839981+0000","last_active":"2024-04-25T00:11:12.515717+0000","last_peered":"2024-04-25T00:11:12.515717+0000","last_clean":"2024-04-25T00:11:12.515717+0000","last_became_active":"2024-04-25T00:11:05.839527+0000","last_became_peered":"2024-04-25T00:11:05.839527+0000","last_unstale":"2024-04-25T00:11:12.515717+0000","last_undegraded":"2024-04-25T00:11:12.515717+0000","last_fullsized":"2024-04-25T00:11:12.515717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:14.133421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515439+0000","last_change":"2024-04-25T00:11:10.900219+0000","last_active":"2024-04-25T00:11:12.515439+0000","last_peered":"2024-04-25T00:11:12.515439+0000","last_clean":"2024-04-25T00:11:12.515439+0000","last_became_active":"2024-04-25T00:11:10.899780+0000","last_became_peered":"2024-04-25T00:11:10.899780+0000","last_unstale":"2024-04-25T00:11:12.515439+0000","last_undegraded":"2024-04-25T00:11:12.515439+0000","last_fullsized":"2024-04-25T00:11:12.515439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:29.940770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515975+0000","last_change":"2024-04-25T00:11:06.863060+0000","last_active":"2024-04-25T00:11:12.515975+0000","last_peered":"2024-04-25T00:11:12.515975+0000","last_clean":"2024-04-25T00:11:12.515975+0000","last_became_active":"2024-04-25T00:11:06.862320+0000","last_became_peered":"2024-04-25T00:11:06.862320+0000","last_unstale":"2024-04-25T00:11:12.515975+0000","last_undegraded":"2024-04-25T00:11:12.515975+0000","last_fullsized":"2024-04-25T00:11:12.515975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:49.414621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875244+0000","last_change":"2024-04-25T00:11:05.826923+0000","last_active":"2024-04-25T00:11:11.875244+0000","last_peered":"2024-04-25T00:11:11.875244+0000","last_clean":"2024-04-25T00:11:11.875244+0000","last_became_active":"2024-04-25T00:11:05.826543+0000","last_became_peered":"2024-04-25T00:11:05.826543+0000","last_unstale":"2024-04-25T00:11:11.875244+0000","last_undegraded":"2024-04-25T00:11:11.875244+0000","last_fullsized":"2024-04-25T00:11:11.875244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513942+0000","last_change":"2024-04-25T00:11:10.893190+0000","last_active":"2024-04-25T00:11:12.513942+0000","last_peered":"2024-04-25T00:11:12.513942+0000","last_clean":"2024-04-25T00:11:12.513942+0000","last_became_active":"2024-04-25T00:11:10.892676+0000","last_became_peered":"2024-04-25T00:11:10.892676+0000","last_unstale":"2024-04-25T00:11:12.513942+0000","last_undegraded":"2024-04-25T00:11:12.513942+0000","last_fullsized":"2024-04-25T00:11:12.513942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874062+0000","last_change":"2024-04-25T00:11:06.854380+0000","last_active":"2024-04-25T00:11:11.874062+0000","last_peered":"2024-04-25T00:11:11.874062+0000","last_clean":"2024-04-25T00:11:11.874062+0000","last_became_active":"2024-04-25T00:11:06.854089+0000","last_became_peered":"2024-04-25T00:11:06.854089+0000","last_unstale":"2024-04-25T00:11:11.874062+0000","last_undegraded":"2024-04-25T00:11:11.874062+0000","last_fullsized":"2024-04-25T00:11:11.874062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874367+0000","last_change":"2024-04-25T00:11:05.848317+0000","last_active":"2024-04-25T00:11:11.874367+0000","last_peered":"2024-04-25T00:11:11.874367+0000","last_clean":"2024-04-25T00:11:11.874367+0000","last_became_active":"2024-04-25T00:11:05.847969+0000","last_became_peered":"2024-04-25T00:11:05.847969+0000","last_unstale":"2024-04-25T00:11:11.874367+0000","last_undegraded":"2024-04-25T00:11:11.874367+0000","last_fullsized":"2024-04-25T00:11:11.874367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874567+0000","last_change":"2024-04-25T00:11:10.974665+0000","last_active":"2024-04-25T00:11:11.874567+0000","last_peered":"2024-04-25T00:11:11.874567+0000","last_clean":"2024-04-25T00:11:11.874567+0000","last_became_active":"2024-04-25T00:11:10.974306+0000","last_became_peered":"2024-04-25T00:11:10.974306+0000","last_unstale":"2024-04-25T00:11:11.874567+0000","last_undegraded":"2024-04-25T00:11:11.874567+0000","last_fullsized":"2024-04-25T00:11:11.874567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:09.712979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514422+0000","last_change":"2024-04-25T00:11:06.870996+0000","last_active":"2024-04-25T00:11:12.514422+0000","last_peered":"2024-04-25T00:11:12.514422+0000","last_clean":"2024-04-25T00:11:12.514422+0000","last_became_active":"2024-04-25T00:11:06.870692+0000","last_became_peered":"2024-04-25T00:11:06.870692+0000","last_unstale":"2024-04-25T00:11:12.514422+0000","last_undegraded":"2024-04-25T00:11:12.514422+0000","last_fullsized":"2024-04-25T00:11:12.514422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:36.481467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515027+0000","last_change":"2024-04-25T00:11:05.838027+0000","last_active":"2024-04-25T00:11:12.515027+0000","last_peered":"2024-04-25T00:11:12.515027+0000","last_clean":"2024-04-25T00:11:12.515027+0000","last_became_active":"2024-04-25T00:11:05.837712+0000","last_became_peered":"2024-04-25T00:11:05.837712+0000","last_unstale":"2024-04-25T00:11:12.515027+0000","last_undegraded":"2024-04-25T00:11:12.515027+0000","last_fullsized":"2024-04-25T00:11:12.515027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:53.690870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875060+0000","last_change":"2024-04-25T00:11:10.973710+0000","last_active":"2024-04-25T00:11:11.875060+0000","last_peered":"2024-04-25T00:11:11.875060+0000","last_clean":"2024-04-25T00:11:11.875060+0000","last_became_active":"2024-04-25T00:11:10.973387+0000","last_became_peered":"2024-04-25T00:11:10.973387+0000","last_unstale":"2024-04-25T00:11:11.875060+0000","last_undegraded":"2024-04-25T00:11:11.875060+0000","last_fullsized":"2024-04-25T00:11:11.875060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:39.160853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514207+0000","last_change":"2024-04-25T00:11:06.847434+0000","last_active":"2024-04-25T00:11:12.514207+0000","last_peered":"2024-04-25T00:11:12.514207+0000","last_clean":"2024-04-25T00:11:12.514207+0000","last_became_active":"2024-04-25T00:11:06.847112+0000","last_became_peered":"2024-04-25T00:11:06.847112+0000","last_unstale":"2024-04-25T00:11:12.514207+0000","last_undegraded":"2024-04-25T00:11:12.514207+0000","last_fullsized":"2024-04-25T00:11:12.514207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:27.928927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514359+0000","last_change":"2024-04-25T00:11:05.837912+0000","last_active":"2024-04-25T00:11:12.514359+0000","last_peered":"2024-04-25T00:11:12.514359+0000","last_clean":"2024-04-25T00:11:12.514359+0000","last_became_active":"2024-04-25T00:11:05.837544+0000","last_became_peered":"2024-04-25T00:11:05.837544+0000","last_unstale":"2024-04-25T00:11:12.514359+0000","last_undegraded":"2024-04-25T00:11:12.514359+0000","last_fullsized":"2024-04-25T00:11:12.514359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:14.133421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515539+0000","last_change":"2024-04-25T00:11:10.968996+0000","last_active":"2024-04-25T00:11:12.515539+0000","last_peered":"2024-04-25T00:11:12.515539+0000","last_clean":"2024-04-25T00:11:12.515539+0000","last_became_active":"2024-04-25T00:11:10.968625+0000","last_became_peered":"2024-04-25T00:11:10.968625+0000","last_unstale":"2024-04-25T00:11:12.515539+0000","last_undegraded":"2024-04-25T00:11:12.515539+0000","last_fullsized":"2024-04-25T00:11:12.515539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:07.644687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514303+0000","last_change":"2024-04-25T00:11:06.857856+0000","last_active":"2024-04-25T00:11:12.514303+0000","last_peered":"2024-04-25T00:11:12.514303+0000","last_clean":"2024-04-25T00:11:12.514303+0000","last_became_active":"2024-04-25T00:11:06.857469+0000","last_became_peered":"2024-04-25T00:11:06.857469+0000","last_unstale":"2024-04-25T00:11:12.514303+0000","last_undegraded":"2024-04-25T00:11:12.514303+0000","last_fullsized":"2024-04-25T00:11:12.514303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:47.946346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:17.643019+0000","last_change":"2024-04-25T00:11:05.836491+0000","last_active":"2024-04-25T00:11:17.643019+0000","last_peered":"2024-04-25T00:11:17.643019+0000","last_clean":"2024-04-25T00:11:17.643019+0000","last_became_active":"2024-04-25T00:11:05.835903+0000","last_became_peered":"2024-04-25T00:11:05.835903+0000","last_unstale":"2024-04-25T00:11:17.643019+0000","last_undegraded":"2024-04-25T00:11:17.643019+0000","last_fullsized":"2024-04-25T00:11:17.643019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514752+0000","last_change":"2024-04-25T00:11:10.902870+0000","last_active":"2024-04-25T00:11:12.514752+0000","last_peered":"2024-04-25T00:11:12.514752+0000","last_clean":"2024-04-25T00:11:12.514752+0000","last_became_active":"2024-04-25T00:11:10.896980+0000","last_became_peered":"2024-04-25T00:11:10.896980+0000","last_unstale":"2024-04-25T00:11:12.514752+0000","last_undegraded":"2024-04-25T00:11:12.514752+0000","last_fullsized":"2024-04-25T00:11:12.514752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:21.086943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516066+0000","last_change":"2024-04-25T00:11:06.864856+0000","last_active":"2024-04-25T00:11:12.516066+0000","last_peered":"2024-04-25T00:11:12.516066+0000","last_clean":"2024-04-25T00:11:12.516066+0000","last_became_active":"2024-04-25T00:11:06.863698+0000","last_became_peered":"2024-04-25T00:11:06.863698+0000","last_unstale":"2024-04-25T00:11:12.516066+0000","last_undegraded":"2024-04-25T00:11:12.516066+0000","last_fullsized":"2024-04-25T00:11:12.516066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:35.253622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514963+0000","last_change":"2024-04-25T00:11:05.843072+0000","last_active":"2024-04-25T00:11:12.514963+0000","last_peered":"2024-04-25T00:11:12.514963+0000","last_clean":"2024-04-25T00:11:12.514963+0000","last_became_active":"2024-04-25T00:11:05.842548+0000","last_became_peered":"2024-04-25T00:11:05.842548+0000","last_unstale":"2024-04-25T00:11:12.514963+0000","last_undegraded":"2024-04-25T00:11:12.514963+0000","last_fullsized":"2024-04-25T00:11:12.514963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516339+0000","last_change":"2024-04-25T00:11:10.900180+0000","last_active":"2024-04-25T00:11:12.516339+0000","last_peered":"2024-04-25T00:11:12.516339+0000","last_clean":"2024-04-25T00:11:12.516339+0000","last_became_active":"2024-04-25T00:11:10.899321+0000","last_became_peered":"2024-04-25T00:11:10.899321+0000","last_unstale":"2024-04-25T00:11:12.516339+0000","last_undegraded":"2024-04-25T00:11:12.516339+0000","last_fullsized":"2024-04-25T00:11:12.516339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515303+0000","last_change":"2024-04-25T00:11:06.855642+0000","last_active":"2024-04-25T00:11:12.515303+0000","last_peered":"2024-04-25T00:11:12.515303+0000","last_clean":"2024-04-25T00:11:12.515303+0000","last_became_active":"2024-04-25T00:11:06.855374+0000","last_became_peered":"2024-04-25T00:11:06.855374+0000","last_unstale":"2024-04-25T00:11:12.515303+0000","last_undegraded":"2024-04-25T00:11:12.515303+0000","last_fullsized":"2024-04-25T00:11:12.515303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516104+0000","last_change":"2024-04-25T00:11:05.846074+0000","last_active":"2024-04-25T00:11:12.516104+0000","last_peered":"2024-04-25T00:11:12.516104+0000","last_clean":"2024-04-25T00:11:12.516104+0000","last_became_active":"2024-04-25T00:11:05.845371+0000","last_became_peered":"2024-04-25T00:11:05.845371+0000","last_unstale":"2024-04-25T00:11:12.516104+0000","last_undegraded":"2024-04-25T00:11:12.516104+0000","last_fullsized":"2024-04-25T00:11:12.516104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:28.291166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874963+0000","last_change":"2024-04-25T00:11:10.898262+0000","last_active":"2024-04-25T00:11:11.874963+0000","last_peered":"2024-04-25T00:11:11.874963+0000","last_clean":"2024-04-25T00:11:11.874963+0000","last_became_active":"2024-04-25T00:11:10.897653+0000","last_became_peered":"2024-04-25T00:11:10.897653+0000","last_unstale":"2024-04-25T00:11:11.874963+0000","last_undegraded":"2024-04-25T00:11:11.874963+0000","last_fullsized":"2024-04-25T00:11:11.874963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:12.584307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516002+0000","last_change":"2024-04-25T00:11:06.865617+0000","last_active":"2024-04-25T00:11:12.516002+0000","last_peered":"2024-04-25T00:11:12.516002+0000","last_clean":"2024-04-25T00:11:12.516002+0000","last_became_active":"2024-04-25T00:11:06.865286+0000","last_became_peered":"2024-04-25T00:11:06.865286+0000","last_unstale":"2024-04-25T00:11:12.516002+0000","last_undegraded":"2024-04-25T00:11:12.516002+0000","last_fullsized":"2024-04-25T00:11:12.516002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:03.766175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874678+0000","last_change":"2024-04-25T00:11:05.845733+0000","last_active":"2024-04-25T00:11:11.874678+0000","last_peered":"2024-04-25T00:11:11.874678+0000","last_clean":"2024-04-25T00:11:11.874678+0000","last_became_active":"2024-04-25T00:11:05.845410+0000","last_became_peered":"2024-04-25T00:11:05.845410+0000","last_unstale":"2024-04-25T00:11:11.874678+0000","last_undegraded":"2024-04-25T00:11:11.874678+0000","last_fullsized":"2024-04-25T00:11:11.874678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874418+0000","last_change":"2024-04-25T00:11:10.977787+0000","last_active":"2024-04-25T00:11:11.874418+0000","last_peered":"2024-04-25T00:11:11.874418+0000","last_clean":"2024-04-25T00:11:11.874418+0000","last_became_active":"2024-04-25T00:11:10.977434+0000","last_became_peered":"2024-04-25T00:11:10.977434+0000","last_unstale":"2024-04-25T00:11:11.874418+0000","last_undegraded":"2024-04-25T00:11:11.874418+0000","last_fullsized":"2024-04-25T00:11:11.874418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:27.014726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513916+0000","last_change":"2024-04-25T00:11:10.977497+0000","last_active":"2024-04-25T00:11:12.513916+0000","last_peered":"2024-04-25T00:11:12.513916+0000","last_clean":"2024-04-25T00:11:12.513916+0000","last_became_active":"2024-04-25T00:11:10.973670+0000","last_became_peered":"2024-04-25T00:11:10.973670+0000","last_unstale":"2024-04-25T00:11:12.513916+0000","last_undegraded":"2024-04-25T00:11:12.513916+0000","last_fullsized":"2024-04-25T00:11:12.513916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:03.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514138+0000","last_change":"2024-04-25T00:11:06.851862+0000","last_active":"2024-04-25T00:11:12.514138+0000","last_peered":"2024-04-25T00:11:12.514138+0000","last_clean":"2024-04-25T00:11:12.514138+0000","last_became_active":"2024-04-25T00:11:06.851683+0000","last_became_peered":"2024-04-25T00:11:06.851683+0000","last_unstale":"2024-04-25T00:11:12.514138+0000","last_undegraded":"2024-04-25T00:11:12.514138+0000","last_fullsized":"2024-04-25T00:11:12.514138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515242+0000","last_change":"2024-04-25T00:11:05.842503+0000","last_active":"2024-04-25T00:11:12.515242+0000","last_peered":"2024-04-25T00:11:12.515242+0000","last_clean":"2024-04-25T00:11:12.515242+0000","last_became_active":"2024-04-25T00:11:05.842214+0000","last_became_peered":"2024-04-25T00:11:05.842214+0000","last_unstale":"2024-04-25T00:11:12.515242+0000","last_undegraded":"2024-04-25T00:11:12.515242+0000","last_fullsized":"2024-04-25T00:11:12.515242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:09.099814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874565+0000","last_change":"2024-04-25T00:11:10.901930+0000","last_active":"2024-04-25T00:11:11.874565+0000","last_peered":"2024-04-25T00:11:11.874565+0000","last_clean":"2024-04-25T00:11:11.874565+0000","last_became_active":"2024-04-25T00:11:10.901405+0000","last_became_peered":"2024-04-25T00:11:10.901405+0000","last_unstale":"2024-04-25T00:11:11.874565+0000","last_undegraded":"2024-04-25T00:11:11.874565+0000","last_fullsized":"2024-04-25T00:11:11.874565+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:07.816085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874298+0000","last_change":"2024-04-25T00:11:06.855999+0000","last_active":"2024-04-25T00:11:11.874298+0000","last_peered":"2024-04-25T00:11:11.874298+0000","last_clean":"2024-04-25T00:11:11.874298+0000","last_became_active":"2024-04-25T00:11:06.855758+0000","last_became_peered":"2024-04-25T00:11:06.855758+0000","last_unstale":"2024-04-25T00:11:11.874298+0000","last_undegraded":"2024-04-25T00:11:11.874298+0000","last_fullsized":"2024-04-25T00:11:11.874298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874267+0000","last_change":"2024-04-25T00:11:05.834141+0000","last_active":"2024-04-25T00:11:11.874267+0000","last_peered":"2024-04-25T00:11:11.874267+0000","last_clean":"2024-04-25T00:11:11.874267+0000","last_became_active":"2024-04-25T00:11:05.833022+0000","last_became_peered":"2024-04-25T00:11:05.833022+0000","last_unstale":"2024-04-25T00:11:11.874267+0000","last_undegraded":"2024-04-25T00:11:11.874267+0000","last_fullsized":"2024-04-25T00:11:11.874267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:15.738255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874362+0000","last_change":"2024-04-25T00:11:05.844727+0000","last_active":"2024-04-25T00:11:11.874362+0000","last_peered":"2024-04-25T00:11:11.874362+0000","last_clean":"2024-04-25T00:11:11.874362+0000","last_became_active":"2024-04-25T00:11:05.844303+0000","last_became_peered":"2024-04-25T00:11:05.844303+0000","last_unstale":"2024-04-25T00:11:11.874362+0000","last_undegraded":"2024-04-25T00:11:11.874362+0000","last_fullsized":"2024-04-25T00:11:11.874362+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:41.285583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514255+0000","last_change":"2024-04-25T00:11:06.862764+0000","last_active":"2024-04-25T00:11:12.514255+0000","last_peered":"2024-04-25T00:11:12.514255+0000","last_clean":"2024-04-25T00:11:12.514255+0000","last_became_active":"2024-04-25T00:11:06.862372+0000","last_became_peered":"2024-04-25T00:11:06.862372+0000","last_unstale":"2024-04-25T00:11:12.514255+0000","last_undegraded":"2024-04-25T00:11:12.514255+0000","last_fullsized":"2024-04-25T00:11:12.514255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875902+0000","last_change":"2024-04-25T00:11:10.902130+0000","last_active":"2024-04-25T00:11:11.875902+0000","last_peered":"2024-04-25T00:11:11.875902+0000","last_clean":"2024-04-25T00:11:11.875902+0000","last_became_active":"2024-04-25T00:11:10.901717+0000","last_became_peered":"2024-04-25T00:11:10.901717+0000","last_unstale":"2024-04-25T00:11:11.875902+0000","last_undegraded":"2024-04-25T00:11:11.875902+0000","last_fullsized":"2024-04-25T00:11:11.875902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:43.776095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875180+0000","last_change":"2024-04-25T00:11:05.833116+0000","last_active":"2024-04-25T00:11:11.875180+0000","last_peered":"2024-04-25T00:11:11.875180+0000","last_clean":"2024-04-25T00:11:11.875180+0000","last_became_active":"2024-04-25T00:11:05.832202+0000","last_became_peered":"2024-04-25T00:11:05.832202+0000","last_unstale":"2024-04-25T00:11:11.875180+0000","last_undegraded":"2024-04-25T00:11:11.875180+0000","last_fullsized":"2024-04-25T00:11:11.875180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:04.800261+0000","last_clean_scrub_stamp":"2024-04-25T00:11:04.800261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:59.026101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516469+0000","last_change":"2024-04-25T00:11:06.840973+0000","last_active":"2024-04-25T00:11:12.516469+0000","last_peered":"2024-04-25T00:11:12.516469+0000","last_clean":"2024-04-25T00:11:12.516469+0000","last_became_active":"2024-04-25T00:11:06.840670+0000","last_became_peered":"2024-04-25T00:11:06.840670+0000","last_unstale":"2024-04-25T00:11:12.516469+0000","last_undegraded":"2024-04-25T00:11:12.516469+0000","last_fullsized":"2024-04-25T00:11:12.516469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:29.296632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874813+0000","last_change":"2024-04-25T00:11:10.898183+0000","last_active":"2024-04-25T00:11:11.874813+0000","last_peered":"2024-04-25T00:11:11.874813+0000","last_clean":"2024-04-25T00:11:11.874813+0000","last_became_active":"2024-04-25T00:11:10.897738+0000","last_became_peered":"2024-04-25T00:11:10.897738+0000","last_unstale":"2024-04-25T00:11:11.874813+0000","last_undegraded":"2024-04-25T00:11:11.874813+0000","last_fullsized":"2024-04-25T00:11:11.874813+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:09.855638+0000","last_clean_scrub_stamp":"2024-04-25T00:11:09.855638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:44.129988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.873858+0000","last_change":"2024-04-25T00:11:06.854098+0000","last_active":"2024-04-25T00:11:11.873858+0000","last_peered":"2024-04-25T00:11:11.873858+0000","last_clean":"2024-04-25T00:11:11.873858+0000","last_became_active":"2024-04-25T00:11:06.853839+0000","last_became_peered":"2024-04-25T00:11:06.853839+0000","last_unstale":"2024-04-25T00:11:11.873858+0000","last_undegraded":"2024-04-25T00:11:11.873858+0000","last_fullsized":"2024-04-25T00:11:11.873858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:00.950133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875315+0000","last_change":"2024-04-25T00:11:06.853055+0000","last_active":"2024-04-25T00:11:11.875315+0000","last_peered":"2024-04-25T00:11:11.875315+0000","last_clean":"2024-04-25T00:11:11.875315+0000","last_became_active":"2024-04-25T00:11:06.852814+0000","last_became_peered":"2024-04-25T00:11:06.852814+0000","last_unstale":"2024-04-25T00:11:11.875315+0000","last_undegraded":"2024-04-25T00:11:11.875315+0000","last_fullsized":"2024-04-25T00:11:11.875315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:42.291050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515002+0000","last_change":"2024-04-25T00:11:06.858759+0000","last_active":"2024-04-25T00:11:12.515002+0000","last_peered":"2024-04-25T00:11:12.515002+0000","last_clean":"2024-04-25T00:11:12.515002+0000","last_became_active":"2024-04-25T00:11:06.858472+0000","last_became_peered":"2024-04-25T00:11:06.858472+0000","last_unstale":"2024-04-25T00:11:12.515002+0000","last_undegraded":"2024-04-25T00:11:12.515002+0000","last_fullsized":"2024-04-25T00:11:12.515002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514169+0000","last_change":"2024-04-25T00:11:06.851389+0000","last_active":"2024-04-25T00:11:12.514169+0000","last_peered":"2024-04-25T00:11:12.514169+0000","last_clean":"2024-04-25T00:11:12.514169+0000","last_became_active":"2024-04-25T00:11:06.845828+0000","last_became_peered":"2024-04-25T00:11:06.845828+0000","last_unstale":"2024-04-25T00:11:12.514169+0000","last_undegraded":"2024-04-25T00:11:12.514169+0000","last_fullsized":"2024-04-25T00:11:12.514169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:00.950133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.875364+0000","last_change":"2024-04-25T00:11:06.851807+0000","last_active":"2024-04-25T00:11:11.875364+0000","last_peered":"2024-04-25T00:11:11.875364+0000","last_clean":"2024-04-25T00:11:11.875364+0000","last_became_active":"2024-04-25T00:11:06.851495+0000","last_became_peered":"2024-04-25T00:11:06.851495+0000","last_unstale":"2024-04-25T00:11:11.875364+0000","last_undegraded":"2024-04-25T00:11:11.875364+0000","last_fullsized":"2024-04-25T00:11:11.875364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:49.414621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514751+0000","last_change":"2024-04-25T00:11:06.848495+0000","last_active":"2024-04-25T00:11:12.514751+0000","last_peered":"2024-04-25T00:11:12.514751+0000","last_clean":"2024-04-25T00:11:12.514751+0000","last_became_active":"2024-04-25T00:11:06.848242+0000","last_became_peered":"2024-04-25T00:11:06.848242+0000","last_unstale":"2024-04-25T00:11:12.514751+0000","last_undegraded":"2024-04-25T00:11:12.514751+0000","last_fullsized":"2024-04-25T00:11:12.514751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874704+0000","last_change":"2024-04-25T00:11:06.850991+0000","last_active":"2024-04-25T00:11:11.874704+0000","last_peered":"2024-04-25T00:11:11.874704+0000","last_clean":"2024-04-25T00:11:11.874704+0000","last_became_active":"2024-04-25T00:11:06.850666+0000","last_became_peered":"2024-04-25T00:11:06.850666+0000","last_unstale":"2024-04-25T00:11:11.874704+0000","last_undegraded":"2024-04-25T00:11:11.874704+0000","last_fullsized":"2024-04-25T00:11:11.874704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:25.890860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874498+0000","last_change":"2024-04-25T00:11:06.857585+0000","last_active":"2024-04-25T00:11:11.874498+0000","last_peered":"2024-04-25T00:11:11.874498+0000","last_clean":"2024-04-25T00:11:11.874498+0000","last_became_active":"2024-04-25T00:11:06.857315+0000","last_became_peered":"2024-04-25T00:11:06.857315+0000","last_unstale":"2024-04-25T00:11:11.874498+0000","last_undegraded":"2024-04-25T00:11:11.874498+0000","last_fullsized":"2024-04-25T00:11:11.874498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:43.612611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513979+0000","last_change":"2024-04-25T00:11:06.858752+0000","last_active":"2024-04-25T00:11:12.513979+0000","last_peered":"2024-04-25T00:11:12.513979+0000","last_clean":"2024-04-25T00:11:12.513979+0000","last_became_active":"2024-04-25T00:11:06.858428+0000","last_became_peered":"2024-04-25T00:11:06.858428+0000","last_unstale":"2024-04-25T00:11:12.513979+0000","last_undegraded":"2024-04-25T00:11:12.513979+0000","last_fullsized":"2024-04-25T00:11:12.513979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.219642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874132+0000","last_change":"2024-04-25T00:11:06.870006+0000","last_active":"2024-04-25T00:11:11.874132+0000","last_peered":"2024-04-25T00:11:11.874132+0000","last_clean":"2024-04-25T00:11:11.874132+0000","last_became_active":"2024-04-25T00:11:06.869687+0000","last_became_peered":"2024-04-25T00:11:06.869687+0000","last_unstale":"2024-04-25T00:11:11.874132+0000","last_undegraded":"2024-04-25T00:11:11.874132+0000","last_fullsized":"2024-04-25T00:11:11.874132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:43.565714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514922+0000","last_change":"2024-04-25T00:11:06.863927+0000","last_active":"2024-04-25T00:11:12.514922+0000","last_peered":"2024-04-25T00:11:12.514922+0000","last_clean":"2024-04-25T00:11:12.514922+0000","last_became_active":"2024-04-25T00:11:06.863758+0000","last_became_peered":"2024-04-25T00:11:06.863758+0000","last_unstale":"2024-04-25T00:11:12.514922+0000","last_undegraded":"2024-04-25T00:11:12.514922+0000","last_fullsized":"2024-04-25T00:11:12.514922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:25.890860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513912+0000","last_change":"2024-04-25T00:11:06.852070+0000","last_active":"2024-04-25T00:11:12.513912+0000","last_peered":"2024-04-25T00:11:12.513912+0000","last_clean":"2024-04-25T00:11:12.513912+0000","last_became_active":"2024-04-25T00:11:06.849951+0000","last_became_peered":"2024-04-25T00:11:06.849951+0000","last_unstale":"2024-04-25T00:11:12.513912+0000","last_undegraded":"2024-04-25T00:11:12.513912+0000","last_fullsized":"2024-04-25T00:11:12.513912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:54.204077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874289+0000","last_change":"2024-04-25T00:11:06.867310+0000","last_active":"2024-04-25T00:11:11.874289+0000","last_peered":"2024-04-25T00:11:11.874289+0000","last_clean":"2024-04-25T00:11:11.874289+0000","last_became_active":"2024-04-25T00:11:06.867010+0000","last_became_peered":"2024-04-25T00:11:06.867010+0000","last_unstale":"2024-04-25T00:11:11.874289+0000","last_undegraded":"2024-04-25T00:11:11.874289+0000","last_fullsized":"2024-04-25T00:11:11.874289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515126+0000","last_change":"2024-04-25T00:11:06.855941+0000","last_active":"2024-04-25T00:11:12.515126+0000","last_peered":"2024-04-25T00:11:12.515126+0000","last_clean":"2024-04-25T00:11:12.515126+0000","last_became_active":"2024-04-25T00:11:06.855737+0000","last_became_peered":"2024-04-25T00:11:06.855737+0000","last_unstale":"2024-04-25T00:11:12.515126+0000","last_undegraded":"2024-04-25T00:11:12.515126+0000","last_fullsized":"2024-04-25T00:11:12.515126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:18.665263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.876458+0000","last_change":"2024-04-25T00:11:06.865551+0000","last_active":"2024-04-25T00:11:11.876458+0000","last_peered":"2024-04-25T00:11:11.876458+0000","last_clean":"2024-04-25T00:11:11.876458+0000","last_became_active":"2024-04-25T00:11:06.865289+0000","last_became_peered":"2024-04-25T00:11:06.865289+0000","last_unstale":"2024-04-25T00:11:11.876458+0000","last_undegraded":"2024-04-25T00:11:11.876458+0000","last_fullsized":"2024-04-25T00:11:11.876458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:51.375329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.516337+0000","last_change":"2024-04-25T00:11:06.863407+0000","last_active":"2024-04-25T00:11:12.516337+0000","last_peered":"2024-04-25T00:11:12.516337+0000","last_clean":"2024-04-25T00:11:12.516337+0000","last_became_active":"2024-04-25T00:11:06.863119+0000","last_became_peered":"2024-04-25T00:11:06.863119+0000","last_unstale":"2024-04-25T00:11:12.516337+0000","last_undegraded":"2024-04-25T00:11:12.516337+0000","last_fullsized":"2024-04-25T00:11:12.516337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:08.978181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513899+0000","last_change":"2024-04-25T00:11:06.854703+0000","last_active":"2024-04-25T00:11:12.513899+0000","last_peered":"2024-04-25T00:11:12.513899+0000","last_clean":"2024-04-25T00:11:12.513899+0000","last_became_active":"2024-04-25T00:11:06.854412+0000","last_became_peered":"2024-04-25T00:11:06.854412+0000","last_unstale":"2024-04-25T00:11:12.513899+0000","last_undegraded":"2024-04-25T00:11:12.513899+0000","last_fullsized":"2024-04-25T00:11:12.513899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:17.037033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514230+0000","last_change":"2024-04-25T00:11:06.850481+0000","last_active":"2024-04-25T00:11:12.514230+0000","last_peered":"2024-04-25T00:11:12.514230+0000","last_clean":"2024-04-25T00:11:12.514230+0000","last_became_active":"2024-04-25T00:11:06.850219+0000","last_became_peered":"2024-04-25T00:11:06.850219+0000","last_unstale":"2024-04-25T00:11:12.514230+0000","last_undegraded":"2024-04-25T00:11:12.514230+0000","last_fullsized":"2024-04-25T00:11:12.514230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.396341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514123+0000","last_change":"2024-04-25T00:11:06.841945+0000","last_active":"2024-04-25T00:11:12.514123+0000","last_peered":"2024-04-25T00:11:12.514123+0000","last_clean":"2024-04-25T00:11:12.514123+0000","last_became_active":"2024-04-25T00:11:06.841645+0000","last_became_peered":"2024-04-25T00:11:06.841645+0000","last_unstale":"2024-04-25T00:11:12.514123+0000","last_undegraded":"2024-04-25T00:11:12.514123+0000","last_fullsized":"2024-04-25T00:11:12.514123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:40.495315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874037+0000","last_change":"2024-04-25T00:11:06.856378+0000","last_active":"2024-04-25T00:11:11.874037+0000","last_peered":"2024-04-25T00:11:11.874037+0000","last_clean":"2024-04-25T00:11:11.874037+0000","last_became_active":"2024-04-25T00:11:06.856108+0000","last_became_peered":"2024-04-25T00:11:06.856108+0000","last_unstale":"2024-04-25T00:11:11.874037+0000","last_undegraded":"2024-04-25T00:11:11.874037+0000","last_fullsized":"2024-04-25T00:11:11.874037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515971+0000","last_change":"2024-04-25T00:11:06.868107+0000","last_active":"2024-04-25T00:11:12.515971+0000","last_peered":"2024-04-25T00:11:12.515971+0000","last_clean":"2024-04-25T00:11:12.515971+0000","last_became_active":"2024-04-25T00:11:06.867822+0000","last_became_peered":"2024-04-25T00:11:06.867822+0000","last_unstale":"2024-04-25T00:11:12.515971+0000","last_undegraded":"2024-04-25T00:11:12.515971+0000","last_fullsized":"2024-04-25T00:11:12.515971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:51.375329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874207+0000","last_change":"2024-04-25T00:11:06.848480+0000","last_active":"2024-04-25T00:11:11.874207+0000","last_peered":"2024-04-25T00:11:11.874207+0000","last_clean":"2024-04-25T00:11:11.874207+0000","last_became_active":"2024-04-25T00:11:06.848228+0000","last_became_peered":"2024-04-25T00:11:06.848228+0000","last_unstale":"2024-04-25T00:11:11.874207+0000","last_undegraded":"2024-04-25T00:11:11.874207+0000","last_fullsized":"2024-04-25T00:11:11.874207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:16.856826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874653+0000","last_change":"2024-04-25T00:11:06.852645+0000","last_active":"2024-04-25T00:11:11.874653+0000","last_peered":"2024-04-25T00:11:11.874653+0000","last_clean":"2024-04-25T00:11:11.874653+0000","last_became_active":"2024-04-25T00:11:06.852364+0000","last_became_peered":"2024-04-25T00:11:06.852364+0000","last_unstale":"2024-04-25T00:11:11.874653+0000","last_undegraded":"2024-04-25T00:11:11.874653+0000","last_fullsized":"2024-04-25T00:11:11.874653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514727+0000","last_change":"2024-04-25T00:11:06.858052+0000","last_active":"2024-04-25T00:11:12.514727+0000","last_peered":"2024-04-25T00:11:12.514727+0000","last_clean":"2024-04-25T00:11:12.514727+0000","last_became_active":"2024-04-25T00:11:06.857402+0000","last_became_peered":"2024-04-25T00:11:06.857402+0000","last_unstale":"2024-04-25T00:11:12.514727+0000","last_undegraded":"2024-04-25T00:11:12.514727+0000","last_fullsized":"2024-04-25T00:11:12.514727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:03.766175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:11.874165+0000","last_change":"2024-04-25T00:11:06.852942+0000","last_active":"2024-04-25T00:11:11.874165+0000","last_peered":"2024-04-25T00:11:11.874165+0000","last_clean":"2024-04-25T00:11:11.874165+0000","last_became_active":"2024-04-25T00:11:06.852671+0000","last_became_peered":"2024-04-25T00:11:06.852671+0000","last_unstale":"2024-04-25T00:11:11.874165+0000","last_undegraded":"2024-04-25T00:11:11.874165+0000","last_fullsized":"2024-04-25T00:11:11.874165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:11.267075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515899+0000","last_change":"2024-04-25T00:11:06.853657+0000","last_active":"2024-04-25T00:11:12.515899+0000","last_peered":"2024-04-25T00:11:12.515899+0000","last_clean":"2024-04-25T00:11:12.515899+0000","last_became_active":"2024-04-25T00:11:06.853342+0000","last_became_peered":"2024-04-25T00:11:06.853342+0000","last_unstale":"2024-04-25T00:11:12.515899+0000","last_undegraded":"2024-04-25T00:11:12.515899+0000","last_fullsized":"2024-04-25T00:11:12.515899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:43.612611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514817+0000","last_change":"2024-04-25T00:11:06.854058+0000","last_active":"2024-04-25T00:11:12.514817+0000","last_peered":"2024-04-25T00:11:12.514817+0000","last_clean":"2024-04-25T00:11:12.514817+0000","last_became_active":"2024-04-25T00:11:06.853803+0000","last_became_peered":"2024-04-25T00:11:06.853803+0000","last_unstale":"2024-04-25T00:11:12.514817+0000","last_undegraded":"2024-04-25T00:11:12.514817+0000","last_fullsized":"2024-04-25T00:11:12.514817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.141492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514199+0000","last_change":"2024-04-25T00:11:06.851312+0000","last_active":"2024-04-25T00:11:12.514199+0000","last_peered":"2024-04-25T00:11:12.514199+0000","last_clean":"2024-04-25T00:11:12.514199+0000","last_became_active":"2024-04-25T00:11:06.851038+0000","last_became_peered":"2024-04-25T00:11:06.851038+0000","last_unstale":"2024-04-25T00:11:12.514199+0000","last_undegraded":"2024-04-25T00:11:12.514199+0000","last_fullsized":"2024-04-25T00:11:12.514199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.219642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514597+0000","last_change":"2024-04-25T00:11:06.860160+0000","last_active":"2024-04-25T00:11:12.514597+0000","last_peered":"2024-04-25T00:11:12.514597+0000","last_clean":"2024-04-25T00:11:12.514597+0000","last_became_active":"2024-04-25T00:11:06.859259+0000","last_became_peered":"2024-04-25T00:11:06.859259+0000","last_unstale":"2024-04-25T00:11:12.514597+0000","last_undegraded":"2024-04-25T00:11:12.514597+0000","last_fullsized":"2024-04-25T00:11:12.514597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:49.414621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.513796+0000","last_change":"2024-04-25T00:11:06.848844+0000","last_active":"2024-04-25T00:11:12.513796+0000","last_peered":"2024-04-25T00:11:12.513796+0000","last_clean":"2024-04-25T00:11:12.513796+0000","last_became_active":"2024-04-25T00:11:06.848597+0000","last_became_peered":"2024-04-25T00:11:06.848597+0000","last_unstale":"2024-04-25T00:11:12.513796+0000","last_undegraded":"2024-04-25T00:11:12.513796+0000","last_fullsized":"2024-04-25T00:11:12.513796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.515039+0000","last_change":"2024-04-25T00:11:06.844764+0000","last_active":"2024-04-25T00:11:12.515039+0000","last_peered":"2024-04-25T00:11:12.515039+0000","last_clean":"2024-04-25T00:11:12.515039+0000","last_became_active":"2024-04-25T00:11:06.844484+0000","last_became_peered":"2024-04-25T00:11:06.844484+0000","last_unstale":"2024-04-25T00:11:12.515039+0000","last_undegraded":"2024-04-25T00:11:12.515039+0000","last_fullsized":"2024-04-25T00:11:12.515039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:35.986742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T00:11:12.514486+0000","last_change":"2024-04-25T00:11:06.860556+0000","last_active":"2024-04-25T00:11:12.514486+0000","last_peered":"2024-04-25T00:11:12.514486+0000","last_clean":"2024-04-25T00:11:12.514486+0000","last_became_active":"2024-04-25T00:11:06.860210+0000","last_became_peered":"2024-04-25T00:11:06.860210+0000","last_unstale":"2024-04-25T00:11:12.514486+0000","last_undegraded":"2024-04-25T00:11:12.514486+0000","last_fullsized":"2024-04-25T00:11:12.514486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:05.805728+0000","last_clean_scrub_stamp":"2024-04-25T00:11:05.805728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:17.037033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:11:30.148 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T00:11:30.148 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T00:11:30.148 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T00:11:30.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T00:11:30.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa35a600640 1 Processor -- start 2024-04-25T00:11:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa35a600640 1 -- start start 2024-04-25T00:11:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa35a600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3540d2ac0 0x7fa3540b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa35a600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3540b7cc0 con 0x7fa3540d1bd0 2024-04-25T00:11:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa35a600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3540b7fa0 con 0x7fa35415e320 2024-04-25T00:11:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa35a600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa3540b8280 con 0x7fa3540d2ac0 2024-04-25T00:11:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa353e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa3540d1bd0 0x7fa3540b8d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52270/0 (socket says 172.21.15.83:52270) 2024-04-25T00:11:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa353e00640 1 -- 172.21.15.83:0/3250350727 learned_addr learned my addr 172.21.15.83:0/3250350727 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa352a00640 1 --2- 172.21.15.83:0/3250350727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3540d2ac0 0x7fa3540b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2854731736 0 0) 0x7fa3540b7cc0 con 0x7fa3540d1bd0 2024-04-25T00:11:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa338003530 con 0x7fa3540d1bd0 2024-04-25T00:11:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3852468526 0 0) 0x7fa3540b7fa0 con 0x7fa35415e320 2024-04-25T00:11:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3540b7cc0 con 0x7fa35415e320 2024-04-25T00:11:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4235515550 0 0) 0x7fa338003530 con 0x7fa3540d1bd0 2024-04-25T00:11:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3540d2ac0 msgr2=0x7fa3540b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa352000640 1 --2- 172.21.15.83:0/3250350727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3540d2ac0 0x7fa3540b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 >> v1:172.21.15.86:6789/0 conn(0x7fa35415e320 legacy=0x7fa3540d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3540b8560 con 0x7fa3540d1bd0 2024-04-25T00:11:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fa34c001a00 con 0x7fa3540d1bd0 2024-04-25T00:11:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 >> v1:172.21.15.83:6789/0 conn(0x7fa3540d1bd0 legacy=0x7fa3540b8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.316+0000 7fa352000640 1 --2- 172.21.15.83:0/3250350727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3540d2ac0 0x7fa338003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa352000640 1 --2- 172.21.15.83:0/3250350727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3380040e0 0x7fa338004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa352000640 1 --2- 172.21.15.83:0/3250350727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa338004d50 0x7fa3380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3540b8280 con 0x7fa338004d50 2024-04-25T00:11:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa33800bab0 con 0x7fa3380040e0 2024-04-25T00:11:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa33800bcb0 con 0x7fa3540d2ac0 2024-04-25T00:11:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa353e00640 1 --2- 172.21.15.83:0/3250350727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa338004d50 0x7fa3380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa352a00640 1 --2- 172.21.15.83:0/3250350727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3380040e0 0x7fa338004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa353400640 1 --2- 172.21.15.83:0/3250350727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3540d2ac0 0x7fa338003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa353e00640 1 -- 172.21.15.83:0/3250350727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3540d2ac0 msgr2=0x7fa338003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa353e00640 1 --2- 172.21.15.83:0/3250350727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3540d2ac0 0x7fa338003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa353e00640 1 -- 172.21.15.83:0/3250350727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3380040e0 msgr2=0x7fa338004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:30.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa353e00640 1 --2- 172.21.15.83:0/3250350727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3380040e0 0x7fa338004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa353e00640 1 -- 172.21.15.83:0/3250350727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa338003530 con 0x7fa338004d50 2024-04-25T00:11:30.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa352a00640 1 --2- 172.21.15.83:0/3250350727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3380040e0 0x7fa338004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:30.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.320+0000 7fa353e00640 1 --2- 172.21.15.83:0/3250350727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa338004d50 0x7fa3380051d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00a7d0 tx=0x7fa34c01fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fa53df6b490834 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:30.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c00a430 con 0x7fa338004d50 2024-04-25T00:11:30.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34c0aa670 con 0x7fa338004d50 2024-04-25T00:11:30.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa352000640 1 -- 172.21.15.83:0/3250350727 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0aaaa0 con 0x7fa338004d50 2024-04-25T00:11:30.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa35a600640 1 -- 172.21.15.83:0/3250350727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa338004d50 msgr2=0x7fa3380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa35a600640 1 --2- 172.21.15.83:0/3250350727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa338004d50 0x7fa3380051d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00a7d0 tx=0x7fa34c01fa40 comp rx=0 tx=0).stop 2024-04-25T00:11:30.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa353e00640 1 -- 172.21.15.83:0/3250350727 reap_dead start 2024-04-25T00:11:30.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa35a600640 1 -- 172.21.15.83:0/3250350727 shutdown_connections 2024-04-25T00:11:30.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa35a600640 1 -- 172.21.15.83:0/3250350727 >> 172.21.15.83:0/3250350727 conn(0x7fa3540d3e80 msgr2=0x7fa3540d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:30.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa35a600640 1 -- 172.21.15.83:0/3250350727 shutdown_connections 2024-04-25T00:11:30.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa35a600640 1 -- 172.21.15.83:0/3250350727 wait complete. 2024-04-25T00:11:30.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa35a600640 1 Processor -- start 2024-04-25T00:11:30.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa35a600640 1 -- start start 2024-04-25T00:11:30.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa35a600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa338004d50 0x7fa3540ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa35a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa35415fa60 0x7fa35415fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa35a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354158c30 0x7fa3541590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa35a600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3540cfe50 con 0x7fa35415fa60 2024-04-25T00:11:30.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa35a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3540cfcd0 con 0x7fa354158c30 2024-04-25T00:11:30.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa35a600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa3540b76e0 con 0x7fa338004d50 2024-04-25T00:11:30.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa352a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa35415fa60 0x7fa35415fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa352a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa35415fa60 0x7fa35415fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36684/0 (socket says 172.21.15.83:36684) 2024-04-25T00:11:30.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa352a00640 1 -- 172.21.15.83:0/391460348 learned_addr learned my addr 172.21.15.83:0/391460348 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:30.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa353e00640 1 --2- 172.21.15.83:0/391460348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354158c30 0x7fa3541590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.324+0000 7fa353400640 1 --2- 172.21.15.83:0/391460348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa338004d50 0x7fa3540ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa352a00640 1 -- 172.21.15.83:0/391460348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa338004d50 msgr2=0x7fa3540ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa352a00640 1 --2- 172.21.15.83:0/391460348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa338004d50 0x7fa3540ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa352a00640 1 -- 172.21.15.83:0/391460348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354158c30 msgr2=0x7fa3541590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa352a00640 1 --2- 172.21.15.83:0/391460348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354158c30 0x7fa3541590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa352a00640 1 -- 172.21.15.83:0/391460348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3540cacb0 con 0x7fa35415fa60 2024-04-25T00:11:30.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa352a00640 1 --2- 172.21.15.83:0/391460348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa35415fa60 0x7fa35415fee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa340011d00 tx=0x7fa34000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:30.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa34b400640 1 -- 172.21.15.83:0/391460348 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34000fcc0 con 0x7fa35415fa60 2024-04-25T00:11:30.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa34b400640 1 -- 172.21.15.83:0/391460348 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34000fe60 con 0x7fa35415fa60 2024-04-25T00:11:30.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa35a600640 1 -- 172.21.15.83:0/391460348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35415e700 con 0x7fa35415fa60 2024-04-25T00:11:30.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa34b400640 1 -- 172.21.15.83:0/391460348 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340004260 con 0x7fa35415fa60 2024-04-25T00:11:30.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa35a600640 1 -- 172.21.15.83:0/391460348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35415eca0 con 0x7fa35415fa60 2024-04-25T00:11:30.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa35a600640 1 -- 172.21.15.83:0/391460348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3540a1df0 con 0x7fa35415fa60 2024-04-25T00:11:30.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa34b400640 1 -- 172.21.15.83:0/391460348 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa340012040 con 0x7fa35415fa60 2024-04-25T00:11:30.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa34b400640 1 --2- 172.21.15.83:0/391460348 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3340678c0 0x7fa334069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa34b400640 1 -- 172.21.15.83:0/391460348 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa340086900 con 0x7fa35415fa60 2024-04-25T00:11:30.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.328+0000 7fa353400640 1 --2- 172.21.15.83:0/391460348 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3340678c0 0x7fa334069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.332+0000 7fa353400640 1 --2- 172.21.15.83:0/391460348 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3340678c0 0x7fa334069d80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa344007fd0 tx=0x7fa344009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:30.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.332+0000 7fa34b400640 1 -- 172.21.15.83:0/391460348 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa340019030 con 0x7fa35415fa60 2024-04-25T00:11:30.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.520+0000 7fa35a600640 1 -- 172.21.15.83:0/391460348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa35404f5f0 con 0x7fa35415fa60 2024-04-25T00:11:30.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.520+0000 7fa34b400640 1 -- 172.21.15.83:0/391460348 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa340057060 con 0x7fa35415fa60 2024-04-25T00:11:30.524 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:30.524 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T00:11:30.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.524+0000 7fa35a600640 1 -- 172.21.15.83:0/391460348 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3340678c0 msgr2=0x7fa334069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.524+0000 7fa35a600640 1 --2- 172.21.15.83:0/391460348 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3340678c0 0x7fa334069d80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa344007fd0 tx=0x7fa344009040 comp rx=0 tx=0).stop 2024-04-25T00:11:30.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.524+0000 7fa35a600640 1 -- 172.21.15.83:0/391460348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa35415fa60 msgr2=0x7fa35415fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.524+0000 7fa35a600640 1 --2- 172.21.15.83:0/391460348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa35415fa60 0x7fa35415fee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa340011d00 tx=0x7fa34000f610 comp rx=0 tx=0).stop 2024-04-25T00:11:30.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.524+0000 7fa35a600640 1 -- 172.21.15.83:0/391460348 shutdown_connections 2024-04-25T00:11:30.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.524+0000 7fa35a600640 1 --2- 172.21.15.83:0/391460348 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3340678c0 0x7fa334069d80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.524+0000 7fa35a600640 1 --2- 172.21.15.83:0/391460348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354158c30 0x7fa3541590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.524+0000 7fa35a600640 1 --2- 172.21.15.83:0/391460348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa35415fa60 0x7fa35415fee0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.524+0000 7fa35a600640 1 --2- 172.21.15.83:0/391460348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa338004d50 0x7fa3540ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.524+0000 7fa35a600640 1 -- 172.21.15.83:0/391460348 >> 172.21.15.83:0/391460348 conn(0x7fa3540d3e80 msgr2=0x7fa35415d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.524+0000 7fa35a600640 1 -- 172.21.15.83:0/391460348 shutdown_connections 2024-04-25T00:11:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.524+0000 7fa35a600640 1 -- 172.21.15.83:0/391460348 wait complete. 2024-04-25T00:11:30.545 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T00:11:30.545 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:30.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fd4e00640 1 Processor -- start 2024-04-25T00:11:30.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fd4e00640 1 -- start start 2024-04-25T00:11:30.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fd4e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6fd006fec0 0x7f6fd00af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fd4e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd00b0290 0x7f6fd00b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fd4e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd00d08b0 con 0x7f6fd015f3c0 2024-04-25T00:11:30.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fd4e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6fd00d0140 con 0x7f6fd006fec0 2024-04-25T00:11:30.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fd4e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd015f7a0 con 0x7f6fd00b0290 2024-04-25T00:11:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fcea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6fd006fec0 0x7f6fd00af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fcfe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd00b0290 0x7f6fd00b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fcfe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd00b0290 0x7f6fd00b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:50140/0 (socket says 172.21.15.83:50140) 2024-04-25T00:11:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fcfe00640 1 -- 172.21.15.83:0/1704988948 learned_addr learned my addr 172.21.15.83:0/1704988948 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fcea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6fd006fec0 0x7f6fd00af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:39340/0 (socket says 172.21.15.83:39340) 2024-04-25T00:11:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fcf400640 1 --1- 172.21.15.83:0/1704988948 >> v1:172.21.15.86:6789/0 conn(0x7f6fd015f3c0 0x7f6fd00ae8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:46926/0 (socket says 172.21.15.83:46926) 2024-04-25T00:11:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fcfe00640 1 -- 172.21.15.83:0/1704988948 >> v2:172.21.15.86:3300/0 conn(0x7f6fd006fec0 msgr2=0x7f6fd00af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fcfe00640 1 --2- 172.21.15.83:0/1704988948 >> v2:172.21.15.86:3300/0 conn(0x7f6fd006fec0 0x7f6fd00af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fcfe00640 1 -- 172.21.15.83:0/1704988948 >> v1:172.21.15.86:6789/0 conn(0x7f6fd015f3c0 legacy=0x7f6fd00ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.680+0000 7f6fcfe00640 1 -- 172.21.15.83:0/1704988948 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd00d2a60 con 0x7f6fd00b0290 2024-04-25T00:11:30.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.684+0000 7f6fcfe00640 1 --2- 172.21.15.83:0/1704988948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd00b0290 0x7f6fd00b0720 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009870 tx=0x7f6fc801c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e573b9db0b9287cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.684+0000 7f6fce000640 1 -- 172.21.15.83:0/1704988948 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc80b0070 con 0x7f6fd00b0290 2024-04-25T00:11:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.684+0000 7f6fce000640 1 -- 172.21.15.83:0/1704988948 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc80ab600 con 0x7f6fd00b0290 2024-04-25T00:11:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.684+0000 7f6fce000640 1 -- 172.21.15.83:0/1704988948 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc80ab9d0 con 0x7f6fd00b0290 2024-04-25T00:11:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.684+0000 7f6fd4e00640 1 -- 172.21.15.83:0/1704988948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd00b0290 msgr2=0x7f6fd00b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.684+0000 7f6fd4e00640 1 --2- 172.21.15.83:0/1704988948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd00b0290 0x7f6fd00b0720 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009870 tx=0x7f6fc801c440 comp rx=0 tx=0).stop 2024-04-25T00:11:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.684+0000 7f6fd4e00640 1 -- 172.21.15.83:0/1704988948 shutdown_connections 2024-04-25T00:11:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.684+0000 7f6fd4e00640 1 --2- 172.21.15.83:0/1704988948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd00b0290 0x7f6fd00b0720 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.684+0000 7f6fd4e00640 1 --2- 172.21.15.83:0/1704988948 >> v2:172.21.15.86:3300/0 conn(0x7f6fd006fec0 0x7f6fd00af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.684+0000 7f6fd4e00640 1 -- 172.21.15.83:0/1704988948 >> 172.21.15.83:0/1704988948 conn(0x7f6fd00d1b60 msgr2=0x7f6fd00d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.684+0000 7f6fd4e00640 1 -- 172.21.15.83:0/1704988948 shutdown_connections 2024-04-25T00:11:30.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.684+0000 7f6fd4e00640 1 -- 172.21.15.83:0/1704988948 wait complete. 2024-04-25T00:11:30.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.684+0000 7f6fd4e00640 1 Processor -- start 2024-04-25T00:11:30.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.684+0000 7f6fd4e00640 1 -- start start 2024-04-25T00:11:30.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fd4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd006fec0 0x7f6fd0156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fd4e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd00b0290 0x7f6fd00c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fd4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd015f3c0 0x7f6fd00c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fd4e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd00b11f0 con 0x7f6fd015f3c0 2024-04-25T00:11:30.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fd4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd00b1070 con 0x7f6fd006fec0 2024-04-25T00:11:30.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fd4e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd00cc400 con 0x7f6fd00b0290 2024-04-25T00:11:30.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fcfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd015f3c0 0x7f6fd00c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fcfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd015f3c0 0x7f6fd00c72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36692/0 (socket says 172.21.15.83:36692) 2024-04-25T00:11:30.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fcfe00640 1 -- 172.21.15.83:0/398356762 learned_addr learned my addr 172.21.15.83:0/398356762 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:30.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fcf400640 1 --2- 172.21.15.83:0/398356762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd006fec0 0x7f6fd0156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fcea00640 1 --2- 172.21.15.83:0/398356762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd00b0290 0x7f6fd00c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fcfe00640 1 -- 172.21.15.83:0/398356762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd00b0290 msgr2=0x7f6fd00c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fcfe00640 1 --2- 172.21.15.83:0/398356762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd00b0290 0x7f6fd00c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fcfe00640 1 -- 172.21.15.83:0/398356762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd006fec0 msgr2=0x7f6fd0156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fcfe00640 1 --2- 172.21.15.83:0/398356762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd006fec0 0x7f6fd0156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fcfe00640 1 -- 172.21.15.83:0/398356762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd0157590 con 0x7f6fd015f3c0 2024-04-25T00:11:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fcfe00640 1 --2- 172.21.15.83:0/398356762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd015f3c0 0x7f6fd00c72d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80ae040 tx=0x7f6fc80191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fc7400640 1 -- 172.21.15.83:0/398356762 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc80b0040 con 0x7f6fd015f3c0 2024-04-25T00:11:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fc7400640 1 -- 172.21.15.83:0/398356762 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc80b4430 con 0x7f6fd015f3c0 2024-04-25T00:11:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fc7400640 1 -- 172.21.15.83:0/398356762 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc80b4800 con 0x7f6fd015f3c0 2024-04-25T00:11:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fd4e00640 1 -- 172.21.15.83:0/398356762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd015a9a0 con 0x7f6fd015f3c0 2024-04-25T00:11:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fd4e00640 1 -- 172.21.15.83:0/398356762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd015af60 con 0x7f6fd015f3c0 2024-04-25T00:11:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.688+0000 7f6fd4e00640 1 -- 172.21.15.83:0/398356762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd004f510 con 0x7f6fd015f3c0 2024-04-25T00:11:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.692+0000 7f6fc7400640 1 -- 172.21.15.83:0/398356762 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6fc80b49a0 con 0x7f6fd015f3c0 2024-04-25T00:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.692+0000 7f6fc7400640 1 --2- 172.21.15.83:0/398356762 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fb00678c0 0x7f6fb0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.692+0000 7f6fc7400640 1 -- 172.21.15.83:0/398356762 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6fc811f8d0 con 0x7f6fd015f3c0 2024-04-25T00:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.692+0000 7f6fcf400640 1 --2- 172.21.15.83:0/398356762 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fb00678c0 0x7f6fb0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.692+0000 7f6fcf400640 1 --2- 172.21.15.83:0/398356762 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fb00678c0 0x7f6fb0069d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00c85a0 tx=0x7f6fc00016e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:30.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.696+0000 7f6fc7400640 1 -- 172.21.15.83:0/398356762 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6fc8016150 con 0x7f6fd015f3c0 2024-04-25T00:11:30.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.856+0000 7f6fd4e00640 1 -- 172.21.15.83:0/398356762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fd015e0c0 con 0x7f6fd015f3c0 2024-04-25T00:11:30.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.856+0000 7f6fc7400640 1 -- 172.21.15.83:0/398356762 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7f6fc80efb90 con 0x7f6fd015f3c0 2024-04-25T00:11:30.861 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:30.861 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.860+0000 7f6fd4e00640 1 -- 172.21.15.83:0/398356762 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fb00678c0 msgr2=0x7f6fb0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.860+0000 7f6fd4e00640 1 --2- 172.21.15.83:0/398356762 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fb00678c0 0x7f6fb0069d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00c85a0 tx=0x7f6fc00016e0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.860+0000 7f6fd4e00640 1 -- 172.21.15.83:0/398356762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd015f3c0 msgr2=0x7f6fd00c72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.860+0000 7f6fd4e00640 1 --2- 172.21.15.83:0/398356762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd015f3c0 0x7f6fd00c72d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80ae040 tx=0x7f6fc80191e0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.860+0000 7f6fd4e00640 1 -- 172.21.15.83:0/398356762 shutdown_connections 2024-04-25T00:11:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.860+0000 7f6fd4e00640 1 --2- 172.21.15.83:0/398356762 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fb00678c0 0x7f6fb0069d80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.860+0000 7f6fd4e00640 1 --2- 172.21.15.83:0/398356762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fd00b0290 0x7f6fd00c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.860+0000 7f6fd4e00640 1 --2- 172.21.15.83:0/398356762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd015f3c0 0x7f6fd00c72d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.860+0000 7f6fd4e00640 1 --2- 172.21.15.83:0/398356762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd006fec0 0x7f6fd0156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.860+0000 7f6fd4e00640 1 -- 172.21.15.83:0/398356762 >> 172.21.15.83:0/398356762 conn(0x7f6fd00d1b60 msgr2=0x7f6fd00cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.860+0000 7f6fd4e00640 1 -- 172.21.15.83:0/398356762 shutdown_connections 2024-04-25T00:11:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:30.860+0000 7f6fd4e00640 1 -- 172.21.15.83:0/398356762 wait complete. 2024-04-25T00:11:30.884 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:11:04.621429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:11:05.189716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:11:08.902215+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:11:30.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:11:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.020+0000 7f6de2a00640 1 Processor -- start 2024-04-25T00:11:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.020+0000 7f6de2a00640 1 -- start start 2024-04-25T00:11:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.020+0000 7f6de2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6ddc15e260 0x7f6ddc0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.020+0000 7f6de2a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6ddc0b4470 0x7f6ddc0ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.020+0000 7f6de2a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ddc15e640 con 0x7f6ddc06f0a0 2024-04-25T00:11:31.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.020+0000 7f6de2a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f6ddc0d1350 con 0x7f6ddc0b4470 2024-04-25T00:11:31.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.020+0000 7f6de2a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6ddc0d0d90 con 0x7f6ddc15e260 2024-04-25T00:11:31.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.024+0000 7f6de0a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6ddc06f0a0 0x7f6ddc06f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52278/0 (socket says 172.21.15.83:52278) 2024-04-25T00:11:31.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.024+0000 7f6de0a00640 1 -- 172.21.15.83:0/2854591472 learned_addr learned my addr 172.21.15.83:0/2854591472 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:31.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.024+0000 7f6ddb400640 1 --2- 172.21.15.83:0/2854591472 >> v2:172.21.15.83:3300/0 conn(0x7f6ddc0b4470 0x7f6ddc0ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.024+0000 7f6ddbe00640 1 --2- 172.21.15.83:0/2854591472 >> v2:172.21.15.86:3300/0 conn(0x7f6ddc15e260 0x7f6ddc0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.024+0000 7f6ddaa00640 1 -- 172.21.15.83:0/2854591472 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1606021093 0 0) 0x7f6ddc15e640 con 0x7f6ddc06f0a0 2024-04-25T00:11:31.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.024+0000 7f6ddb400640 1 -- 172.21.15.83:0/2854591472 >> v2:172.21.15.86:3300/0 conn(0x7f6ddc15e260 msgr2=0x7f6ddc0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:31.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.024+0000 7f6ddb400640 1 --2- 172.21.15.83:0/2854591472 >> v2:172.21.15.86:3300/0 conn(0x7f6ddc15e260 0x7f6ddc0ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.024+0000 7f6ddb400640 1 -- 172.21.15.83:0/2854591472 >> v1:172.21.15.83:6789/0 conn(0x7f6ddc06f0a0 legacy=0x7f6ddc06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.024+0000 7f6ddb400640 1 -- 172.21.15.83:0/2854591472 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc0d1070 con 0x7f6ddc0b4470 2024-04-25T00:11:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.024+0000 7f6ddb400640 1 --2- 172.21.15.83:0/2854591472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ddc0b4470 0x7f6ddc0ceca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc06ef70 tx=0x7f6dc801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e11b35dbfe85a43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.024+0000 7f6ddaa00640 1 -- 172.21.15.83:0/2854591472 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc80b0070 con 0x7f6ddc0b4470 2024-04-25T00:11:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.024+0000 7f6ddaa00640 1 -- 172.21.15.83:0/2854591472 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc80ab600 con 0x7f6ddc0b4470 2024-04-25T00:11:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.024+0000 7f6ddaa00640 1 -- 172.21.15.83:0/2854591472 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc80ab9d0 con 0x7f6ddc0b4470 2024-04-25T00:11:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 -- 172.21.15.83:0/2854591472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ddc0b4470 msgr2=0x7f6ddc0ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 --2- 172.21.15.83:0/2854591472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ddc0b4470 0x7f6ddc0ceca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc06ef70 tx=0x7f6dc801c480 comp rx=0 tx=0).stop 2024-04-25T00:11:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 -- 172.21.15.83:0/2854591472 shutdown_connections 2024-04-25T00:11:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 --2- 172.21.15.83:0/2854591472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ddc0b4470 0x7f6ddc0ceca0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 --2- 172.21.15.83:0/2854591472 >> v2:172.21.15.86:3300/0 conn(0x7f6ddc15e260 0x7f6ddc0ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 -- 172.21.15.83:0/2854591472 >> 172.21.15.83:0/2854591472 conn(0x7f6ddc0b3950 msgr2=0x7f6ddc0b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 -- 172.21.15.83:0/2854591472 shutdown_connections 2024-04-25T00:11:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 -- 172.21.15.83:0/2854591472 wait complete. 2024-04-25T00:11:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 Processor -- start 2024-04-25T00:11:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 -- start start 2024-04-25T00:11:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ddc06f0a0 0x7f6ddc15bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ddc0b4470 0x7f6ddc15b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ddc15e260 0x7f6ddc152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc16af30 con 0x7f6ddc15e260 2024-04-25T00:11:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc16adb0 con 0x7f6ddc0b4470 2024-04-25T00:11:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc16b0b0 con 0x7f6ddc06f0a0 2024-04-25T00:11:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ddc15e260 0x7f6ddc152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ddc15e260 0x7f6ddc152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36702/0 (socket says 172.21.15.83:36702) 2024-04-25T00:11:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de0a00640 1 -- 172.21.15.83:0/1961367431 learned_addr learned my addr 172.21.15.83:0/1961367431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6ddb400640 1 --2- 172.21.15.83:0/1961367431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ddc0b4470 0x7f6ddc15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6ddbe00640 1 --2- 172.21.15.83:0/1961367431 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ddc06f0a0 0x7f6ddc15bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de0a00640 1 -- 172.21.15.83:0/1961367431 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ddc06f0a0 msgr2=0x7f6ddc15bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de0a00640 1 --2- 172.21.15.83:0/1961367431 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ddc06f0a0 0x7f6ddc15bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de0a00640 1 -- 172.21.15.83:0/1961367431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ddc0b4470 msgr2=0x7f6ddc15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de0a00640 1 --2- 172.21.15.83:0/1961367431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ddc0b4470 0x7f6ddc15b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de0a00640 1 -- 172.21.15.83:0/1961367431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc15c560 con 0x7f6ddc15e260 2024-04-25T00:11:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de0a00640 1 --2- 172.21.15.83:0/1961367431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ddc15e260 0x7f6ddc152310 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400c3e0 tx=0x7f6dd400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6dd3e00640 1 -- 172.21.15.83:0/1961367431 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd40074e0 con 0x7f6ddc15e260 2024-04-25T00:11:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6dd3e00640 1 -- 172.21.15.83:0/1961367431 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd40047f0 con 0x7f6ddc15e260 2024-04-25T00:11:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6de2a00640 1 -- 172.21.15.83:0/1961367431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc1535e0 con 0x7f6ddc15e260 2024-04-25T00:11:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.028+0000 7f6dd3e00640 1 -- 172.21.15.83:0/1961367431 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4004be0 con 0x7f6ddc15e260 2024-04-25T00:11:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.032+0000 7f6de2a00640 1 -- 172.21.15.83:0/1961367431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc153ba0 con 0x7f6ddc15e260 2024-04-25T00:11:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.032+0000 7f6de2a00640 1 -- 172.21.15.83:0/1961367431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ddc1543f0 con 0x7f6ddc15e260 2024-04-25T00:11:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.032+0000 7f6dd3e00640 1 -- 172.21.15.83:0/1961367431 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6dd4011070 con 0x7f6ddc15e260 2024-04-25T00:11:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.032+0000 7f6dd3e00640 1 --2- 172.21.15.83:0/1961367431 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dbc0678c0 0x7f6dbc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.032+0000 7f6dd3e00640 1 -- 172.21.15.83:0/1961367431 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6dd4085c60 con 0x7f6ddc15e260 2024-04-25T00:11:31.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.032+0000 7f6ddbe00640 1 --2- 172.21.15.83:0/1961367431 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dbc0678c0 0x7f6dbc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.032+0000 7f6ddbe00640 1 --2- 172.21.15.83:0/1961367431 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dbc0678c0 0x7f6dbc069d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0074a0 tx=0x7f6dcc00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:31.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.036+0000 7f6dd3e00640 1 -- 172.21.15.83:0/1961367431 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6dd40563c0 con 0x7f6ddc15e260 2024-04-25T00:11:31.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.192+0000 7f6de2a00640 1 -- 172.21.15.83:0/1961367431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6ddc04f510 con 0x7f6ddc15e260 2024-04-25T00:11:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.192+0000 7f6dd3e00640 1 -- 172.21.15.83:0/1961367431 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f6dd4055cc0 con 0x7f6ddc15e260 2024-04-25T00:11:31.195 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:11:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.192+0000 7f6de2a00640 1 -- 172.21.15.83:0/1961367431 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dbc0678c0 msgr2=0x7f6dbc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.192+0000 7f6de2a00640 1 --2- 172.21.15.83:0/1961367431 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dbc0678c0 0x7f6dbc069d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0074a0 tx=0x7f6dcc00b040 comp rx=0 tx=0).stop 2024-04-25T00:11:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.192+0000 7f6de2a00640 1 -- 172.21.15.83:0/1961367431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ddc15e260 msgr2=0x7f6ddc152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.192+0000 7f6de2a00640 1 --2- 172.21.15.83:0/1961367431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ddc15e260 0x7f6ddc152310 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400c3e0 tx=0x7f6dd400c8a0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.192+0000 7f6de2a00640 1 -- 172.21.15.83:0/1961367431 shutdown_connections 2024-04-25T00:11:31.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.192+0000 7f6de2a00640 1 --2- 172.21.15.83:0/1961367431 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dbc0678c0 0x7f6dbc069d80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.192+0000 7f6de2a00640 1 --2- 172.21.15.83:0/1961367431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ddc15e260 0x7f6ddc152310 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.192+0000 7f6de2a00640 1 --2- 172.21.15.83:0/1961367431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ddc0b4470 0x7f6ddc15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.192+0000 7f6de2a00640 1 --2- 172.21.15.83:0/1961367431 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ddc06f0a0 0x7f6ddc15bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.192+0000 7f6de2a00640 1 -- 172.21.15.83:0/1961367431 >> 172.21.15.83:0/1961367431 conn(0x7f6ddc0b3950 msgr2=0x7f6ddc0aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:31.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.192+0000 7f6de2a00640 1 -- 172.21.15.83:0/1961367431 shutdown_connections 2024-04-25T00:11:31.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.192+0000 7f6de2a00640 1 -- 172.21.15.83:0/1961367431 wait complete. 2024-04-25T00:11:31.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:11:31.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.348+0000 7f4b3de00640 1 Processor -- start 2024-04-25T00:11:31.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.348+0000 7f4b3de00640 1 -- start start 2024-04-25T00:11:31.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.352+0000 7f4b3de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4b3806e950 0x7f4b3815f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.352+0000 7f4b3de00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4b3815fe50 0x7f4b380ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.352+0000 7f4b3de00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b380ce000 con 0x7f4b38159810 2024-04-25T00:11:31.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.352+0000 7f4b3de00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4b3806e210 con 0x7f4b3815fe50 2024-04-25T00:11:31.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.352+0000 7f4b3de00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4b381602a0 con 0x7f4b3806e950 2024-04-25T00:11:31.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.352+0000 7f4b36a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4b38159810 0x7f4b380cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52294/0 (socket says 172.21.15.83:52294) 2024-04-25T00:11:31.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.352+0000 7f4b36a00640 1 -- 172.21.15.83:0/2957051741 learned_addr learned my addr 172.21.15.83:0/2957051741 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:31.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.352+0000 7f4b37400640 1 --2- 172.21.15.83:0/2957051741 >> v2:172.21.15.83:3300/0 conn(0x7f4b3815fe50 0x7f4b380ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.352+0000 7f4b2e000640 1 --2- 172.21.15.83:0/2957051741 >> v2:172.21.15.86:3300/0 conn(0x7f4b3806e950 0x7f4b3815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.352+0000 7f4b37400640 1 -- 172.21.15.83:0/2957051741 >> v2:172.21.15.86:3300/0 conn(0x7f4b3806e950 msgr2=0x7f4b3815f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:31.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.352+0000 7f4b37400640 1 --2- 172.21.15.83:0/2957051741 >> v2:172.21.15.86:3300/0 conn(0x7f4b3806e950 0x7f4b3815f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.352+0000 7f4b37400640 1 -- 172.21.15.83:0/2957051741 >> v1:172.21.15.83:6789/0 conn(0x7f4b38159810 legacy=0x7f4b380cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.352+0000 7f4b37400640 1 -- 172.21.15.83:0/2957051741 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3806eec0 con 0x7f4b3815fe50 2024-04-25T00:11:31.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.352+0000 7f4b36000640 1 -- 172.21.15.83:0/2957051741 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 593472488 0 0) 0x7f4b380ce000 con 0x7f4b38159810 2024-04-25T00:11:31.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.356+0000 7f4b37400640 1 --2- 172.21.15.83:0/2957051741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3815fe50 0x7f4b380ce4a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4b380709a0 tx=0x7f4b3001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2c30d41fc44326 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:31.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.356+0000 7f4b36000640 1 -- 172.21.15.83:0/2957051741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b300b2070 con 0x7f4b3815fe50 2024-04-25T00:11:31.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.356+0000 7f4b36000640 1 -- 172.21.15.83:0/2957051741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b300a5850 con 0x7f4b3815fe50 2024-04-25T00:11:31.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.356+0000 7f4b36000640 1 -- 172.21.15.83:0/2957051741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b300a5c20 con 0x7f4b3815fe50 2024-04-25T00:11:31.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.356+0000 7f4b3de00640 1 -- 172.21.15.83:0/2957051741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3815fe50 msgr2=0x7f4b380ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.356+0000 7f4b3de00640 1 --2- 172.21.15.83:0/2957051741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3815fe50 0x7f4b380ce4a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4b380709a0 tx=0x7f4b3001c8a0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.356+0000 7f4b3de00640 1 -- 172.21.15.83:0/2957051741 shutdown_connections 2024-04-25T00:11:31.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.356+0000 7f4b3de00640 1 --2- 172.21.15.83:0/2957051741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3815fe50 0x7f4b380ce4a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.356+0000 7f4b3de00640 1 --2- 172.21.15.83:0/2957051741 >> v2:172.21.15.86:3300/0 conn(0x7f4b3806e950 0x7f4b3815f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.356+0000 7f4b3de00640 1 -- 172.21.15.83:0/2957051741 >> 172.21.15.83:0/2957051741 conn(0x7f4b380b1830 msgr2=0x7f4b380b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:31.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.356+0000 7f4b3de00640 1 -- 172.21.15.83:0/2957051741 shutdown_connections 2024-04-25T00:11:31.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.356+0000 7f4b3de00640 1 -- 172.21.15.83:0/2957051741 wait complete. 2024-04-25T00:11:31.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.356+0000 7f4b3de00640 1 Processor -- start 2024-04-25T00:11:31.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.356+0000 7f4b3de00640 1 -- start start 2024-04-25T00:11:31.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b3de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b3806e950 0x7f4b380b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b3de00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b38159810 0x7f4b38150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b3de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3815fe50 0x7f4b380b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b3de00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b38070f20 con 0x7f4b3815fe50 2024-04-25T00:11:31.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b3de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b38070da0 con 0x7f4b3806e950 2024-04-25T00:11:31.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b3de00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4b38168cb0 con 0x7f4b38159810 2024-04-25T00:11:31.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b37400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3815fe50 0x7f4b380b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b37400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3815fe50 0x7f4b380b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36710/0 (socket says 172.21.15.83:36710) 2024-04-25T00:11:31.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b37400640 1 -- 172.21.15.83:0/464826471 learned_addr learned my addr 172.21.15.83:0/464826471 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:31.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b37400640 1 -- 172.21.15.83:0/464826471 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b38159810 msgr2=0x7f4b38150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b37400640 1 --2- 172.21.15.83:0/464826471 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b38159810 0x7f4b38150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b37400640 1 -- 172.21.15.83:0/464826471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b3806e950 msgr2=0x7f4b380b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b37400640 1 --2- 172.21.15.83:0/464826471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b3806e950 0x7f4b380b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b37400640 1 -- 172.21.15.83:0/464826471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b38150cb0 con 0x7f4b3815fe50 2024-04-25T00:11:31.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b37400640 1 --2- 172.21.15.83:0/464826471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3815fe50 0x7f4b380b2360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b3001a4d0 tx=0x7f4b30019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:31.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b2fe00640 1 -- 172.21.15.83:0/464826471 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b300b2050 con 0x7f4b3815fe50 2024-04-25T00:11:31.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b2fe00640 1 -- 172.21.15.83:0/464826471 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b300be070 con 0x7f4b3815fe50 2024-04-25T00:11:31.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b3de00640 1 -- 172.21.15.83:0/464826471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b380b28a0 con 0x7f4b3815fe50 2024-04-25T00:11:31.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b2fe00640 1 -- 172.21.15.83:0/464826471 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b300ada70 con 0x7f4b3815fe50 2024-04-25T00:11:31.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b3de00640 1 -- 172.21.15.83:0/464826471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b380b2b40 con 0x7f4b3815fe50 2024-04-25T00:11:31.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b3de00640 1 -- 172.21.15.83:0/464826471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3804f510 con 0x7f4b3815fe50 2024-04-25T00:11:31.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.360+0000 7f4b2fe00640 1 -- 172.21.15.83:0/464826471 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4b300a56d0 con 0x7f4b3815fe50 2024-04-25T00:11:31.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.364+0000 7f4b2fe00640 1 --2- 172.21.15.83:0/464826471 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b140678c0 0x7f4b14069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.364+0000 7f4b2fe00640 1 -- 172.21.15.83:0/464826471 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4b301212f0 con 0x7f4b3815fe50 2024-04-25T00:11:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.364+0000 7f4b36a00640 1 --2- 172.21.15.83:0/464826471 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b140678c0 0x7f4b14069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.364+0000 7f4b36a00640 1 --2- 172.21.15.83:0/464826471 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b140678c0 0x7f4b14069d80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4b28007fd0 tx=0x7f4b28007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:31.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.368+0000 7f4b2fe00640 1 -- 172.21.15.83:0/464826471 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4b30016520 con 0x7f4b3815fe50 2024-04-25T00:11:31.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.524+0000 7f4b3de00640 1 -- 172.21.15.83:0/464826471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4b38074b60 con 0x7f4b3815fe50 2024-04-25T00:11:31.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.524+0000 7f4b2fe00640 1 -- 172.21.15.83:0/464826471 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f4b300f19d0 con 0x7f4b3815fe50 2024-04-25T00:11:31.526 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.524+0000 7f4b3de00640 1 -- 172.21.15.83:0/464826471 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b140678c0 msgr2=0x7f4b14069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.524+0000 7f4b3de00640 1 --2- 172.21.15.83:0/464826471 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b140678c0 0x7f4b14069d80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4b28007fd0 tx=0x7f4b28007960 comp rx=0 tx=0).stop 2024-04-25T00:11:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.524+0000 7f4b3de00640 1 -- 172.21.15.83:0/464826471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3815fe50 msgr2=0x7f4b380b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.524+0000 7f4b3de00640 1 --2- 172.21.15.83:0/464826471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3815fe50 0x7f4b380b2360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b3001a4d0 tx=0x7f4b30019640 comp rx=0 tx=0).stop 2024-04-25T00:11:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.524+0000 7f4b3de00640 1 -- 172.21.15.83:0/464826471 shutdown_connections 2024-04-25T00:11:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.524+0000 7f4b3de00640 1 --2- 172.21.15.83:0/464826471 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b140678c0 0x7f4b14069d80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.524+0000 7f4b3de00640 1 --2- 172.21.15.83:0/464826471 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b38159810 0x7f4b38150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.524+0000 7f4b3de00640 1 --2- 172.21.15.83:0/464826471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3815fe50 0x7f4b380b2360 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.524+0000 7f4b3de00640 1 --2- 172.21.15.83:0/464826471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b3806e950 0x7f4b380b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.524+0000 7f4b3de00640 1 -- 172.21.15.83:0/464826471 >> 172.21.15.83:0/464826471 conn(0x7f4b380b1830 msgr2=0x7f4b38071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.524+0000 7f4b3de00640 1 -- 172.21.15.83:0/464826471 shutdown_connections 2024-04-25T00:11:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.524+0000 7f4b3de00640 1 -- 172.21.15.83:0/464826471 wait complete. 2024-04-25T00:11:31.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:11:31.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.680+0000 7fb077200640 1 Processor -- start 2024-04-25T00:11:31.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.680+0000 7fb077200640 1 -- start start 2024-04-25T00:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.680+0000 7fb077200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb070160020 0x7fb070166ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.680+0000 7fb077200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0700ce000 con 0x7fb070159810 2024-04-25T00:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.680+0000 7fb077200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb07006eec0 con 0x7fb07006e950 2024-04-25T00:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.680+0000 7fb077200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb07006e210 con 0x7fb070160020 2024-04-25T00:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06fe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb070159810 0x7fb0700cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52296/0 (socket says 172.21.15.83:52296) 2024-04-25T00:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06fe00640 1 -- 172.21.15.83:0/3250941292 learned_addr learned my addr 172.21.15.83:0/3250941292 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb075200640 1 --2- 172.21.15.83:0/3250941292 >> v2:172.21.15.86:3300/0 conn(0x7fb070160020 0x7fb070166ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 -- 172.21.15.83:0/3250941292 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2830252583 0 0) 0x7fb0700ce000 con 0x7fb070159810 2024-04-25T00:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 -- 172.21.15.83:0/3250941292 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb054003530 con 0x7fb070159810 2024-04-25T00:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 -- 172.21.15.83:0/3250941292 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1784894274 0 0) 0x7fb054003530 con 0x7fb070159810 2024-04-25T00:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 -- 172.21.15.83:0/3250941292 >> v2:172.21.15.86:3300/0 conn(0x7fb070160020 msgr2=0x7fb070166ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 --2- 172.21.15.83:0/3250941292 >> v2:172.21.15.86:3300/0 conn(0x7fb070160020 0x7fb070166ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 -- 172.21.15.83:0/3250941292 >> v1:172.21.15.86:6789/0 conn(0x7fb07006e950 legacy=0x7fb07015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 -- 172.21.15.83:0/3250941292 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07006dc50 con 0x7fb070159810 2024-04-25T00:11:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 -- 172.21.15.83:0/3250941292 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fb060001a00 con 0x7fb070159810 2024-04-25T00:11:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 -- 172.21.15.83:0/3250941292 >> v1:172.21.15.83:6789/0 conn(0x7fb070159810 legacy=0x7fb0700cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 --2- 172.21.15.83:0/3250941292 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb054003a30 0x7fb054003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 --2- 172.21.15.83:0/3250941292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb054004810 0x7fb054004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 --2- 172.21.15.83:0/3250941292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054005240 0x7fb05400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 -- 172.21.15.83:0/3250941292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb07006e210 con 0x7fb054004810 2024-04-25T00:11:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 -- 172.21.15.83:0/3250941292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb05400c1a0 con 0x7fb054005240 2024-04-25T00:11:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06ea00640 1 -- 172.21.15.83:0/3250941292 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb05400c3a0 con 0x7fb054003a30 2024-04-25T00:11:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06f400640 1 --2- 172.21.15.83:0/3250941292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb054004810 0x7fb054004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06fe00640 1 --2- 172.21.15.83:0/3250941292 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb054003a30 0x7fb054003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb075200640 1 --2- 172.21.15.83:0/3250941292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054005240 0x7fb05400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06f400640 1 -- 172.21.15.83:0/3250941292 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb054003a30 msgr2=0x7fb054003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:31.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06f400640 1 --2- 172.21.15.83:0/3250941292 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb054003a30 0x7fb054003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06f400640 1 -- 172.21.15.83:0/3250941292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054005240 msgr2=0x7fb05400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:31.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06f400640 1 --2- 172.21.15.83:0/3250941292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054005240 0x7fb05400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06f400640 1 -- 172.21.15.83:0/3250941292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb054003530 con 0x7fb054004810 2024-04-25T00:11:31.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb075200640 1 -- 172.21.15.83:0/3250941292 reap_dead start 2024-04-25T00:11:31.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.684+0000 7fb06fe00640 1 --2- 172.21.15.83:0/3250941292 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb054003a30 0x7fb054003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:31.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.688+0000 7fb06f400640 1 --2- 172.21.15.83:0/3250941292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb054004810 0x7fb054004c90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb05c001780 tx=0x7fb05c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683ae8b3c61cd18b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.688+0000 7fb06ea00640 1 -- 172.21.15.83:0/3250941292 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0013a0 con 0x7fb054004810 2024-04-25T00:11:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.688+0000 7fb06ea00640 1 -- 172.21.15.83:0/3250941292 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb05c0052f0 con 0x7fb054004810 2024-04-25T00:11:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.688+0000 7fb06ea00640 1 -- 172.21.15.83:0/3250941292 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c003840 con 0x7fb054004810 2024-04-25T00:11:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.688+0000 7fb077200640 1 -- 172.21.15.83:0/3250941292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb054004810 msgr2=0x7fb054004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.688+0000 7fb077200640 1 --2- 172.21.15.83:0/3250941292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb054004810 0x7fb054004c90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb05c001780 tx=0x7fb05c01c3e0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.688+0000 7fb077200640 1 -- 172.21.15.83:0/3250941292 shutdown_connections 2024-04-25T00:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.688+0000 7fb077200640 1 --2- 172.21.15.83:0/3250941292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb054004810 0x7fb054004c90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.688+0000 7fb077200640 1 -- 172.21.15.83:0/3250941292 >> 172.21.15.83:0/3250941292 conn(0x7fb0700b1830 msgr2=0x7fb0700b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.688+0000 7fb077200640 1 -- 172.21.15.83:0/3250941292 shutdown_connections 2024-04-25T00:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.688+0000 7fb077200640 1 -- 172.21.15.83:0/3250941292 wait complete. 2024-04-25T00:11:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.688+0000 7fb077200640 1 Processor -- start 2024-04-25T00:11:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.688+0000 7fb077200640 1 -- start start 2024-04-25T00:11:31.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.688+0000 7fb077200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054004810 0x7fb07015e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb077200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0701484d0 0x7fb0701546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb077200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb07015eab0 0x7fb07015ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb077200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb070070f20 con 0x7fb07015eab0 2024-04-25T00:11:31.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb077200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb070070da0 con 0x7fb054004810 2024-04-25T00:11:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb077200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb0700cc8b0 con 0x7fb0701484d0 2024-04-25T00:11:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb075200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb07015eab0 0x7fb07015ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb075200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb07015eab0 0x7fb07015ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36726/0 (socket says 172.21.15.83:36726) 2024-04-25T00:11:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb075200640 1 -- 172.21.15.83:0/765312296 learned_addr learned my addr 172.21.15.83:0/765312296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb06fe00640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054004810 0x7fb07015e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb075200640 1 -- 172.21.15.83:0/765312296 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0701484d0 msgr2=0x7fb0701546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb06f400640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0701484d0 0x7fb0701546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb075200640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0701484d0 0x7fb0701546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb075200640 1 -- 172.21.15.83:0/765312296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054004810 msgr2=0x7fb07015e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb075200640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054004810 0x7fb07015e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb075200640 1 -- 172.21.15.83:0/765312296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb070155040 con 0x7fb07015eab0 2024-04-25T00:11:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb06fe00640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054004810 0x7fb07015e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb06f400640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0701484d0 0x7fb0701546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb075200640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb07015eab0 0x7fb07015ef30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb07006edf0 tx=0x7fb068011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb067e00640 1 -- 172.21.15.83:0/765312296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068019070 con 0x7fb07015eab0 2024-04-25T00:11:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb067e00640 1 -- 172.21.15.83:0/765312296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb068004240 con 0x7fb07015eab0 2024-04-25T00:11:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb077200640 1 -- 172.21.15.83:0/765312296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb070155320 con 0x7fb07015eab0 2024-04-25T00:11:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb067e00640 1 -- 172.21.15.83:0/765312296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06800f6c0 con 0x7fb07015eab0 2024-04-25T00:11:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb077200640 1 -- 172.21.15.83:0/765312296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0701558e0 con 0x7fb07015eab0 2024-04-25T00:11:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb077200640 1 -- 172.21.15.83:0/765312296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07004f510 con 0x7fb07015eab0 2024-04-25T00:11:31.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.692+0000 7fb067e00640 1 -- 172.21.15.83:0/765312296 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb06801c070 con 0x7fb07015eab0 2024-04-25T00:11:31.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.696+0000 7fb067e00640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb0500678c0 0x7fb050069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:31.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.696+0000 7fb067e00640 1 -- 172.21.15.83:0/765312296 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb068086300 con 0x7fb07015eab0 2024-04-25T00:11:31.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.696+0000 7fb06fe00640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb0500678c0 0x7fb050069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:31.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.696+0000 7fb06fe00640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb0500678c0 0x7fb050069d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb06000a450 tx=0x7fb060006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.700+0000 7fb067e00640 1 -- 172.21.15.83:0/765312296 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb068056af0 con 0x7fb07015eab0 2024-04-25T00:11:31.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.852+0000 7fb077200640 1 -- 172.21.15.83:0/765312296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb070072c90 con 0x7fb07015eab0 2024-04-25T00:11:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.852+0000 7fb067e00640 1 -- 172.21.15.83:0/765312296 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fb0680563f0 con 0x7fb07015eab0 2024-04-25T00:11:31.857 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:11:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.856+0000 7fb077200640 1 -- 172.21.15.83:0/765312296 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb0500678c0 msgr2=0x7fb050069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.856+0000 7fb077200640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb0500678c0 0x7fb050069d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb06000a450 tx=0x7fb060006040 comp rx=0 tx=0).stop 2024-04-25T00:11:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.856+0000 7fb077200640 1 -- 172.21.15.83:0/765312296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb07015eab0 msgr2=0x7fb07015ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.856+0000 7fb077200640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb07015eab0 0x7fb07015ef30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb07006edf0 tx=0x7fb068011cf0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.856+0000 7fb077200640 1 -- 172.21.15.83:0/765312296 shutdown_connections 2024-04-25T00:11:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.856+0000 7fb077200640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb0500678c0 0x7fb050069d80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.856+0000 7fb077200640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0701484d0 0x7fb0701546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.856+0000 7fb077200640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb07015eab0 0x7fb07015ef30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.856+0000 7fb077200640 1 --2- 172.21.15.83:0/765312296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054004810 0x7fb07015e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.856+0000 7fb077200640 1 -- 172.21.15.83:0/765312296 >> 172.21.15.83:0/765312296 conn(0x7fb0700b1830 msgr2=0x7fb07014f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.856+0000 7fb077200640 1 -- 172.21.15.83:0/765312296 shutdown_connections 2024-04-25T00:11:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:31.856+0000 7fb077200640 1 -- 172.21.15.83:0/765312296 wait complete. 2024-04-25T00:11:31.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:11:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.012+0000 7f1d6be00640 1 Processor -- start 2024-04-25T00:11:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.012+0000 7f1d6be00640 1 -- start start 2024-04-25T00:11:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.012+0000 7f1d6be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d6406e950 0x7f1d6415f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.012+0000 7f1d6be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1d6415fe50 0x7f1d640ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.012+0000 7f1d6be00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d640ce000 con 0x7f1d64159810 2024-04-25T00:11:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.012+0000 7f1d6be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1d6406e210 con 0x7f1d6415fe50 2024-04-25T00:11:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.012+0000 7f1d6be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6406da00 con 0x7f1d6406e950 2024-04-25T00:11:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d69400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1d64159810 0x7f1d640cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52300/0 (socket says 172.21.15.83:52300) 2024-04-25T00:11:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d69400640 1 -- 172.21.15.83:0/3835040697 learned_addr learned my addr 172.21.15.83:0/3835040697 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d68a00640 1 --2- 172.21.15.83:0/3835040697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d6406e950 0x7f1d6415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d69e00640 1 --2- 172.21.15.83:0/3835040697 >> v2:172.21.15.86:3300/0 conn(0x7f1d6415fe50 0x7f1d640ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 -- 172.21.15.83:0/3835040697 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2416149246 0 0) 0x7f1d640ce000 con 0x7f1d64159810 2024-04-25T00:11:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 -- 172.21.15.83:0/3835040697 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d4c003530 con 0x7f1d64159810 2024-04-25T00:11:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 -- 172.21.15.83:0/3835040697 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2405881219 0 0) 0x7f1d4c003530 con 0x7f1d64159810 2024-04-25T00:11:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 -- 172.21.15.83:0/3835040697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d6406e950 msgr2=0x7f1d6415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 --2- 172.21.15.83:0/3835040697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d6406e950 0x7f1d6415f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 -- 172.21.15.83:0/3835040697 >> v2:172.21.15.86:3300/0 conn(0x7f1d6415fe50 msgr2=0x7f1d640ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 --2- 172.21.15.83:0/3835040697 >> v2:172.21.15.86:3300/0 conn(0x7f1d6415fe50 0x7f1d640ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 -- 172.21.15.83:0/3835040697 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6406eec0 con 0x7f1d64159810 2024-04-25T00:11:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 -- 172.21.15.83:0/3835040697 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f1d58001680 con 0x7f1d64159810 2024-04-25T00:11:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 -- 172.21.15.83:0/3835040697 >> v1:172.21.15.83:6789/0 conn(0x7f1d64159810 legacy=0x7f1d640cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 --2- 172.21.15.83:0/3835040697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d4c003a30 0x7f1d4c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 --2- 172.21.15.83:0/3835040697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c004810 0x7f1d4c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 --2- 172.21.15.83:0/3835040697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d4c0051d0 0x7f1d4c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 -- 172.21.15.83:0/3835040697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6406e210 con 0x7f1d4c003a30 2024-04-25T00:11:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 -- 172.21.15.83:0/3835040697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6406da00 con 0x7f1d4c004810 2024-04-25T00:11:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d63e00640 1 -- 172.21.15.83:0/3835040697 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4c00bfc0 con 0x7f1d4c0051d0 2024-04-25T00:11:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d68a00640 1 --2- 172.21.15.83:0/3835040697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d6406e950 0x7f1d6415f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d69e00640 1 --2- 172.21.15.83:0/3835040697 >> v2:172.21.15.86:3300/0 conn(0x7f1d6415fe50 0x7f1d640ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d69400640 1 --2- 172.21.15.83:0/3835040697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d4c003a30 0x7f1d4c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d69e00640 1 --2- 172.21.15.83:0/3835040697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d4c0051d0 0x7f1d4c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d68a00640 1 --2- 172.21.15.83:0/3835040697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c004810 0x7f1d4c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d69400640 1 -- 172.21.15.83:0/3835040697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d4c0051d0 msgr2=0x7f1d4c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d69400640 1 --2- 172.21.15.83:0/3835040697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d4c0051d0 0x7f1d4c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d69400640 1 -- 172.21.15.83:0/3835040697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c004810 msgr2=0x7f1d4c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d69400640 1 --2- 172.21.15.83:0/3835040697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c004810 0x7f1d4c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d69400640 1 -- 172.21.15.83:0/3835040697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c003530 con 0x7f1d4c003a30 2024-04-25T00:11:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.016+0000 7f1d68a00640 1 --2- 172.21.15.83:0/3835040697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c004810 0x7f1d4c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d69400640 1 --2- 172.21.15.83:0/3835040697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d4c003a30 0x7f1d4c003e50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c2c0 tx=0x7f1d5801cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ca4e79a8a591d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d63e00640 1 -- 172.21.15.83:0/3835040697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58007d60 con 0x7f1d4c003a30 2024-04-25T00:11:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d63e00640 1 -- 172.21.15.83:0/3835040697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d580a9020 con 0x7f1d4c003a30 2024-04-25T00:11:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d63e00640 1 -- 172.21.15.83:0/3835040697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58005ae0 con 0x7f1d4c003a30 2024-04-25T00:11:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d6be00640 1 -- 172.21.15.83:0/3835040697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d4c003a30 msgr2=0x7f1d4c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d6be00640 1 --2- 172.21.15.83:0/3835040697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d4c003a30 0x7f1d4c003e50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c2c0 tx=0x7f1d5801cd40 comp rx=0 tx=0).stop 2024-04-25T00:11:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d69e00640 1 -- 172.21.15.83:0/3835040697 reap_dead start 2024-04-25T00:11:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d6be00640 1 -- 172.21.15.83:0/3835040697 shutdown_connections 2024-04-25T00:11:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d6be00640 1 -- 172.21.15.83:0/3835040697 >> 172.21.15.83:0/3835040697 conn(0x7f1d640b1830 msgr2=0x7f1d640b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d6be00640 1 -- 172.21.15.83:0/3835040697 shutdown_connections 2024-04-25T00:11:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d6be00640 1 -- 172.21.15.83:0/3835040697 wait complete. 2024-04-25T00:11:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d6be00640 1 Processor -- start 2024-04-25T00:11:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d6be00640 1 -- start start 2024-04-25T00:11:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d6be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d4c003a30 0x7f1d64161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6415aec0 0x7f1d641623a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d6be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d64160a60 0x7f1d64164ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d6be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64070f20 con 0x7f1d64160a60 2024-04-25T00:11:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d6be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64070da0 con 0x7f1d6415aec0 2024-04-25T00:11:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.020+0000 7f1d6be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1d64166ca0 con 0x7f1d4c003a30 2024-04-25T00:11:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d69e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d64160a60 0x7f1d64164ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d69e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d64160a60 0x7f1d64164ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36740/0 (socket says 172.21.15.83:36740) 2024-04-25T00:11:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d69e00640 1 -- 172.21.15.83:0/474451795 learned_addr learned my addr 172.21.15.83:0/474451795 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d68a00640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6415aec0 0x7f1d641623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d69400640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d4c003a30 0x7f1d64161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d69e00640 1 -- 172.21.15.83:0/474451795 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d4c003a30 msgr2=0x7f1d64161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d69e00640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d4c003a30 0x7f1d64161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d69e00640 1 -- 172.21.15.83:0/474451795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6415aec0 msgr2=0x7f1d641623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d69e00640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6415aec0 0x7f1d641623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d69e00640 1 -- 172.21.15.83:0/474451795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6415b3a0 con 0x7f1d64160a60 2024-04-25T00:11:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d68a00640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6415aec0 0x7f1d641623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d69e00640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d64160a60 0x7f1d64164ec0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1d6406edd0 tx=0x7f1d5c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d61600640 1 -- 172.21.15.83:0/474451795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c011e60 con 0x7f1d64160a60 2024-04-25T00:11:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d61600640 1 -- 172.21.15.83:0/474451795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d5c01b070 con 0x7f1d64160a60 2024-04-25T00:11:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d61600640 1 -- 172.21.15.83:0/474451795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0039a0 con 0x7f1d64160a60 2024-04-25T00:11:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d6be00640 1 -- 172.21.15.83:0/474451795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6415b680 con 0x7f1d64160a60 2024-04-25T00:11:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d6be00640 1 -- 172.21.15.83:0/474451795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6415bc40 con 0x7f1d64160a60 2024-04-25T00:11:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d6be00640 1 -- 172.21.15.83:0/474451795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d6404f510 con 0x7f1d64160a60 2024-04-25T00:11:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.024+0000 7f1d61600640 1 -- 172.21.15.83:0/474451795 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1d5c003b40 con 0x7f1d64160a60 2024-04-25T00:11:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.028+0000 7f1d61600640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1d440678c0 0x7f1d44069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.028+0000 7f1d61600640 1 -- 172.21.15.83:0/474451795 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1d5c0869e0 con 0x7f1d64160a60 2024-04-25T00:11:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.028+0000 7f1d69400640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1d440678c0 0x7f1d44069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.028+0000 7f1d69400640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1d440678c0 0x7f1d44069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1d580aa000 tx=0x7f1d580b0040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.032+0000 7f1d61600640 1 -- 172.21.15.83:0/474451795 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1d5c025020 con 0x7f1d64160a60 2024-04-25T00:11:32.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.184+0000 7f1d6be00640 1 -- 172.21.15.83:0/474451795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1d64072c90 con 0x7f1d64160a60 2024-04-25T00:11:32.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.184+0000 7f1d61600640 1 -- 172.21.15.83:0/474451795 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f1d5c0571e0 con 0x7f1d64160a60 2024-04-25T00:11:32.188 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:32.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.188+0000 7f1d6be00640 1 -- 172.21.15.83:0/474451795 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1d440678c0 msgr2=0x7f1d44069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.188+0000 7f1d6be00640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1d440678c0 0x7f1d44069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1d580aa000 tx=0x7f1d580b0040 comp rx=0 tx=0).stop 2024-04-25T00:11:32.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.188+0000 7f1d6be00640 1 -- 172.21.15.83:0/474451795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d64160a60 msgr2=0x7f1d64164ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.188+0000 7f1d6be00640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d64160a60 0x7f1d64164ec0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1d6406edd0 tx=0x7f1d5c009f90 comp rx=0 tx=0).stop 2024-04-25T00:11:32.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.188+0000 7f1d6be00640 1 -- 172.21.15.83:0/474451795 shutdown_connections 2024-04-25T00:11:32.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.188+0000 7f1d6be00640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1d440678c0 0x7f1d44069d80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.188+0000 7f1d6be00640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d64160a60 0x7f1d64164ec0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.188+0000 7f1d6be00640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6415aec0 0x7f1d641623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.188+0000 7f1d6be00640 1 --2- 172.21.15.83:0/474451795 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d4c003a30 0x7f1d64161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.188+0000 7f1d6be00640 1 -- 172.21.15.83:0/474451795 >> 172.21.15.83:0/474451795 conn(0x7f1d640b1830 msgr2=0x7f1d64158520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:32.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.188+0000 7f1d6be00640 1 -- 172.21.15.83:0/474451795 shutdown_connections 2024-04-25T00:11:32.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.188+0000 7f1d6be00640 1 -- 172.21.15.83:0/474451795 wait complete. 2024-04-25T00:11:32.208 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:11:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.340+0000 7fa3fe200640 1 Processor -- start 2024-04-25T00:11:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.340+0000 7fa3fe200640 1 -- start start 2024-04-25T00:11:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.340+0000 7fa3fe200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa3f8161dc0 0x7fa3f81621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.340+0000 7fa3fe200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f80b1cc0 con 0x7fa3f806c500 2024-04-25T00:11:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.340+0000 7fa3fe200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f81626e0 con 0x7fa3f815dff0 2024-04-25T00:11:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.340+0000 7fa3fe200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa3f80ae9e0 con 0x7fa3f8161dc0 2024-04-25T00:11:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.340+0000 7fa3f7400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa3f8161dc0 0x7fa3f81621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.340+0000 7fa3f6a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa3f806c500 0x7fa3f815d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52316/0 (socket says 172.21.15.83:52316) 2024-04-25T00:11:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.340+0000 7fa3f6a00640 1 -- 172.21.15.83:0/667721021 learned_addr learned my addr 172.21.15.83:0/667721021 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.344+0000 7fa3f7400640 1 -- 172.21.15.83:0/667721021 >> v1:172.21.15.86:6789/0 conn(0x7fa3f815dff0 legacy=0x7fa3f80cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.344+0000 7fa3f7400640 1 -- 172.21.15.83:0/667721021 >> v1:172.21.15.83:6789/0 conn(0x7fa3f806c500 legacy=0x7fa3f815d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.344+0000 7fa3f7400640 1 -- 172.21.15.83:0/667721021 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f806cd00 con 0x7fa3f8161dc0 2024-04-25T00:11:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.344+0000 7fa3f7400640 1 --2- 172.21.15.83:0/667721021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8161dc0 0x7fa3f81621a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80097c0 tx=0x7fa3e801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d12cce7668708d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.344+0000 7fa3f6000640 1 -- 172.21.15.83:0/667721021 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e80b1070 con 0x7fa3f8161dc0 2024-04-25T00:11:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.344+0000 7fa3f6000640 1 -- 172.21.15.83:0/667721021 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e80ac5f0 con 0x7fa3f8161dc0 2024-04-25T00:11:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.344+0000 7fa3f6000640 1 -- 172.21.15.83:0/667721021 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e80ac9c0 con 0x7fa3f8161dc0 2024-04-25T00:11:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.344+0000 7fa3fe200640 1 -- 172.21.15.83:0/667721021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8161dc0 msgr2=0x7fa3f81621a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.344+0000 7fa3fe200640 1 --2- 172.21.15.83:0/667721021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8161dc0 0x7fa3f81621a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80097c0 tx=0x7fa3e801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.344+0000 7fa3fe200640 1 -- 172.21.15.83:0/667721021 shutdown_connections 2024-04-25T00:11:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.344+0000 7fa3fe200640 1 --2- 172.21.15.83:0/667721021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8161dc0 0x7fa3f81621a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.344+0000 7fa3fe200640 1 -- 172.21.15.83:0/667721021 >> 172.21.15.83:0/667721021 conn(0x7fa3f80d1c70 msgr2=0x7fa3f80d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.344+0000 7fa3fe200640 1 -- 172.21.15.83:0/667721021 shutdown_connections 2024-04-25T00:11:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.344+0000 7fa3fe200640 1 -- 172.21.15.83:0/667721021 wait complete. 2024-04-25T00:11:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3fe200640 1 Processor -- start 2024-04-25T00:11:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3fe200640 1 -- start start 2024-04-25T00:11:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3fe200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3f806c500 0x7fa3f8150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3fe200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3f815dff0 0x7fa3f80c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3fe200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8161dc0 0x7fa3f80c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3fe200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f80d2760 con 0x7fa3f8161dc0 2024-04-25T00:11:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3fe200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f80b3490 con 0x7fa3f815dff0 2024-04-25T00:11:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3fe200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f80d28e0 con 0x7fa3f806c500 2024-04-25T00:11:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3f7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8161dc0 0x7fa3f80c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3f7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8161dc0 0x7fa3f80c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36760/0 (socket says 172.21.15.83:36760) 2024-04-25T00:11:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3f7e00640 1 -- 172.21.15.83:0/3549894249 learned_addr learned my addr 172.21.15.83:0/3549894249 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3f6a00640 1 --2- 172.21.15.83:0/3549894249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3f815dff0 0x7fa3f80c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3f7400640 1 --2- 172.21.15.83:0/3549894249 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3f806c500 0x7fa3f8150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3f7e00640 1 -- 172.21.15.83:0/3549894249 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3f806c500 msgr2=0x7fa3f8150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3f7e00640 1 --2- 172.21.15.83:0/3549894249 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3f806c500 0x7fa3f8150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3f7e00640 1 -- 172.21.15.83:0/3549894249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3f815dff0 msgr2=0x7fa3f80c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3f7e00640 1 --2- 172.21.15.83:0/3549894249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3f815dff0 0x7fa3f80c5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3f7e00640 1 -- 172.21.15.83:0/3549894249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f8150d40 con 0x7fa3f8161dc0 2024-04-25T00:11:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3f7e00640 1 --2- 172.21.15.83:0/3549894249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8161dc0 0x7fa3f80c62c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c770 tx=0x7fa3f000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3ef400640 1 -- 172.21.15.83:0/3549894249 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0003b80 con 0x7fa3f8161dc0 2024-04-25T00:11:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3ef400640 1 -- 172.21.15.83:0/3549894249 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f0003d20 con 0x7fa3f8161dc0 2024-04-25T00:11:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3ef400640 1 -- 172.21.15.83:0/3549894249 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f000aa90 con 0x7fa3f8161dc0 2024-04-25T00:11:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.348+0000 7fa3fe200640 1 -- 172.21.15.83:0/3549894249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f80c6950 con 0x7fa3f8161dc0 2024-04-25T00:11:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.352+0000 7fa3fe200640 1 -- 172.21.15.83:0/3549894249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f815be00 con 0x7fa3f8161dc0 2024-04-25T00:11:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.352+0000 7fa3fe200640 1 -- 172.21.15.83:0/3549894249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f815c000 con 0x7fa3f8161dc0 2024-04-25T00:11:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.352+0000 7fa3ef400640 1 -- 172.21.15.83:0/3549894249 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa3f000ac30 con 0x7fa3f8161dc0 2024-04-25T00:11:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.352+0000 7fa3ef400640 1 --2- 172.21.15.83:0/3549894249 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3d80678c0 0x7fa3d8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.352+0000 7fa3ef400640 1 -- 172.21.15.83:0/3549894249 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa3f0085e30 con 0x7fa3f8161dc0 2024-04-25T00:11:32.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.352+0000 7fa3f7400640 1 --2- 172.21.15.83:0/3549894249 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3d80678c0 0x7fa3d8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.352+0000 7fa3f7400640 1 --2- 172.21.15.83:0/3549894249 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3d80678c0 0x7fa3d8069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80ae000 tx=0x7fa3e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.356+0000 7fa3ef400640 1 -- 172.21.15.83:0/3549894249 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa3f815c000 con 0x7fa3f8161dc0 2024-04-25T00:11:32.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.548+0000 7fa3fe200640 1 -- 172.21.15.83:0/3549894249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3f809fd60 con 0x7fa3f8161dc0 2024-04-25T00:11:32.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.548+0000 7fa3ef400640 1 -- 172.21.15.83:0/3549894249 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7fa3f0055fd0 con 0x7fa3f8161dc0 2024-04-25T00:11:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:32.550 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/1335208265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1335208265},{"type":"v1","addr":"172.21.15.86:6834","nonce":1335208265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/911805587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":911805587},{"type":"v1","addr":"172.21.15.83:6836","nonce":911805587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2309157687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2309157687},{"type":"v1","addr":"172.21.15.83:6837","nonce":2309157687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:11:06.200989+0000","modified":"2024-04-25T00:11:23.462521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1603954817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":1603954817},{"type":"v1","addr":"172.21.15.86:6835","nonce":1603954817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T00:11:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2024-04-25T00:11:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.548+0000 7fa3fe200640 1 -- 172.21.15.83:0/3549894249 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3d80678c0 msgr2=0x7fa3d8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.548+0000 7fa3fe200640 1 --2- 172.21.15.83:0/3549894249 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3d80678c0 0x7fa3d8069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80ae000 tx=0x7fa3e8008040 comp rx=0 tx=0).stop 2024-04-25T00:11:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.548+0000 7fa3fe200640 1 -- 172.21.15.83:0/3549894249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8161dc0 msgr2=0x7fa3f80c62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.548+0000 7fa3fe200640 1 --2- 172.21.15.83:0/3549894249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8161dc0 0x7fa3f80c62c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c770 tx=0x7fa3f000cc30 comp rx=0 tx=0).stop 2024-04-25T00:11:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.548+0000 7fa3fe200640 1 -- 172.21.15.83:0/3549894249 shutdown_connections 2024-04-25T00:11:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.548+0000 7fa3fe200640 1 --2- 172.21.15.83:0/3549894249 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3d80678c0 0x7fa3d8069d80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.548+0000 7fa3fe200640 1 --2- 172.21.15.83:0/3549894249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8161dc0 0x7fa3f80c62c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.548+0000 7fa3fe200640 1 --2- 172.21.15.83:0/3549894249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3f815dff0 0x7fa3f80c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.548+0000 7fa3fe200640 1 --2- 172.21.15.83:0/3549894249 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3f806c500 0x7fa3f8150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.548+0000 7fa3fe200640 1 -- 172.21.15.83:0/3549894249 >> 172.21.15.83:0/3549894249 conn(0x7fa3f80d1c70 msgr2=0x7fa3f80d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.548+0000 7fa3fe200640 1 -- 172.21.15.83:0/3549894249 shutdown_connections 2024-04-25T00:11:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.548+0000 7fa3fe200640 1 -- 172.21.15.83:0/3549894249 wait complete. 2024-04-25T00:11:32.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:32.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb927800640 1 Processor -- start 2024-04-25T00:11:32.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb927800640 1 -- start start 2024-04-25T00:11:32.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb927800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9200d0230 0x7fb9200d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb927800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9200cc9b0 con 0x7fb920163b00 2024-04-25T00:11:32.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb927800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9200ccc90 con 0x7fb92015bf70 2024-04-25T00:11:32.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb927800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb9200d0be0 con 0x7fb9200d0230 2024-04-25T00:11:32.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb925800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb920163b00 0x7fb920165ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52332/0 (socket says 172.21.15.83:52332) 2024-04-25T00:11:32.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb925800640 1 -- 172.21.15.83:0/2122703828 learned_addr learned my addr 172.21.15.83:0/2122703828 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:32.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb917e00640 1 --2- 172.21.15.83:0/2122703828 >> v2:172.21.15.86:3300/0 conn(0x7fb9200d0230 0x7fb9200d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 -- 172.21.15.83:0/2122703828 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2360151073 0 0) 0x7fb9200cc9b0 con 0x7fb920163b00 2024-04-25T00:11:32.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 -- 172.21.15.83:0/2122703828 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb904003530 con 0x7fb920163b00 2024-04-25T00:11:32.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 -- 172.21.15.83:0/2122703828 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1908190261 0 0) 0x7fb904003530 con 0x7fb920163b00 2024-04-25T00:11:32.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 -- 172.21.15.83:0/2122703828 >> v2:172.21.15.86:3300/0 conn(0x7fb9200d0230 msgr2=0x7fb9200d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 --2- 172.21.15.83:0/2122703828 >> v2:172.21.15.86:3300/0 conn(0x7fb9200d0230 0x7fb9200d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 -- 172.21.15.83:0/2122703828 >> v1:172.21.15.86:6789/0 conn(0x7fb92015bf70 legacy=0x7fb9200cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 -- 172.21.15.83:0/2122703828 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9200ccf70 con 0x7fb920163b00 2024-04-25T00:11:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 -- 172.21.15.83:0/2122703828 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fb918001a00 con 0x7fb920163b00 2024-04-25T00:11:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 -- 172.21.15.83:0/2122703828 >> v1:172.21.15.83:6789/0 conn(0x7fb920163b00 legacy=0x7fb920165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 --2- 172.21.15.83:0/2122703828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9040039c0 0x7fb904003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 --2- 172.21.15.83:0/2122703828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb904004810 0x7fb904004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 --2- 172.21.15.83:0/2122703828 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9040051d0 0x7fb90400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 -- 172.21.15.83:0/2122703828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9200d0be0 con 0x7fb904004810 2024-04-25T00:11:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 -- 172.21.15.83:0/2122703828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb90400c070 con 0x7fb9040039c0 2024-04-25T00:11:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb91fe00640 1 -- 172.21.15.83:0/2122703828 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb90400c270 con 0x7fb9040051d0 2024-04-25T00:11:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb917e00640 1 --2- 172.21.15.83:0/2122703828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb904004810 0x7fb904004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb924e00640 1 --2- 172.21.15.83:0/2122703828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9040039c0 0x7fb904003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb925800640 1 --2- 172.21.15.83:0/2122703828 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9040051d0 0x7fb90400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb917e00640 1 -- 172.21.15.83:0/2122703828 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9040051d0 msgr2=0x7fb90400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb917e00640 1 --2- 172.21.15.83:0/2122703828 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9040051d0 0x7fb90400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb917e00640 1 -- 172.21.15.83:0/2122703828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9040039c0 msgr2=0x7fb904003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb917e00640 1 --2- 172.21.15.83:0/2122703828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9040039c0 0x7fb904003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb917e00640 1 -- 172.21.15.83:0/2122703828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb904003530 con 0x7fb904004810 2024-04-25T00:11:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.704+0000 7fb925800640 1 -- 172.21.15.83:0/2122703828 reap_dead start 2024-04-25T00:11:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.708+0000 7fb917e00640 1 --2- 172.21.15.83:0/2122703828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb904004810 0x7fb904004c90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb908009ac0 tx=0x7fb90801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afab25705ba42567 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.708+0000 7fb91fe00640 1 -- 172.21.15.83:0/2122703828 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9080b2070 con 0x7fb904004810 2024-04-25T00:11:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.708+0000 7fb91fe00640 1 -- 172.21.15.83:0/2122703828 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9080a5530 con 0x7fb904004810 2024-04-25T00:11:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.708+0000 7fb91fe00640 1 -- 172.21.15.83:0/2122703828 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908009690 con 0x7fb904004810 2024-04-25T00:11:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.708+0000 7fb927800640 1 -- 172.21.15.83:0/2122703828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb904004810 msgr2=0x7fb904004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.708+0000 7fb927800640 1 --2- 172.21.15.83:0/2122703828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb904004810 0x7fb904004c90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb908009ac0 tx=0x7fb90801ca50 comp rx=0 tx=0).stop 2024-04-25T00:11:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.708+0000 7fb927800640 1 -- 172.21.15.83:0/2122703828 shutdown_connections 2024-04-25T00:11:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.708+0000 7fb927800640 1 --2- 172.21.15.83:0/2122703828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb904004810 0x7fb904004c90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.708+0000 7fb927800640 1 -- 172.21.15.83:0/2122703828 >> 172.21.15.83:0/2122703828 conn(0x7fb9200d1c70 msgr2=0x7fb9200d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.708+0000 7fb927800640 1 -- 172.21.15.83:0/2122703828 shutdown_connections 2024-04-25T00:11:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.708+0000 7fb927800640 1 -- 172.21.15.83:0/2122703828 wait complete. 2024-04-25T00:11:32.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb927800640 1 Processor -- start 2024-04-25T00:11:32.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb927800640 1 -- start start 2024-04-25T00:11:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb927800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb904004810 0x7fb92014d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb927800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92006ce10 0x7fb92006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb927800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9201f7850 0x7fb9201f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb927800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9200d29d0 con 0x7fb92006ce10 2024-04-25T00:11:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb927800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9200cd530 con 0x7fb9201f7850 2024-04-25T00:11:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb927800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb92015c350 con 0x7fb904004810 2024-04-25T00:11:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb917e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92006ce10 0x7fb92006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb917e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92006ce10 0x7fb92006d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36780/0 (socket says 172.21.15.83:36780) 2024-04-25T00:11:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb917e00640 1 -- 172.21.15.83:0/745044674 learned_addr learned my addr 172.21.15.83:0/745044674 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb925800640 1 --2- 172.21.15.83:0/745044674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9201f7850 0x7fb9201f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb924e00640 1 --2- 172.21.15.83:0/745044674 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb904004810 0x7fb92014d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb917e00640 1 -- 172.21.15.83:0/745044674 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb904004810 msgr2=0x7fb92014d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb917e00640 1 --2- 172.21.15.83:0/745044674 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb904004810 0x7fb92014d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb917e00640 1 -- 172.21.15.83:0/745044674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9201f7850 msgr2=0x7fb9201f7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb917e00640 1 --2- 172.21.15.83:0/745044674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9201f7850 0x7fb9201f7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb917e00640 1 -- 172.21.15.83:0/745044674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9201f8560 con 0x7fb92006ce10 2024-04-25T00:11:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb917e00640 1 --2- 172.21.15.83:0/745044674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92006ce10 0x7fb92006d270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb90801c880 tx=0x7fb9080197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb91d600640 1 -- 172.21.15.83:0/745044674 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9080b2040 con 0x7fb92006ce10 2024-04-25T00:11:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb91d600640 1 -- 172.21.15.83:0/745044674 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9080ad440 con 0x7fb92006ce10 2024-04-25T00:11:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb91d600640 1 -- 172.21.15.83:0/745044674 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9080ad7f0 con 0x7fb92006ce10 2024-04-25T00:11:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb927800640 1 -- 172.21.15.83:0/745044674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92009bed0 con 0x7fb92006ce10 2024-04-25T00:11:32.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.712+0000 7fb927800640 1 -- 172.21.15.83:0/745044674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92014d580 con 0x7fb92006ce10 2024-04-25T00:11:32.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.716+0000 7fb927800640 1 -- 172.21.15.83:0/745044674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb92009fd60 con 0x7fb92006ce10 2024-04-25T00:11:32.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.716+0000 7fb91d600640 1 -- 172.21.15.83:0/745044674 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb9080ad990 con 0x7fb92006ce10 2024-04-25T00:11:32.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.716+0000 7fb91d600640 1 --2- 172.21.15.83:0/745044674 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8fc0678c0 0x7fb8fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.716+0000 7fb91d600640 1 -- 172.21.15.83:0/745044674 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb9081205f0 con 0x7fb92006ce10 2024-04-25T00:11:32.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.716+0000 7fb924e00640 1 --2- 172.21.15.83:0/745044674 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8fc0678c0 0x7fb8fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.716+0000 7fb924e00640 1 --2- 172.21.15.83:0/745044674 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8fc0678c0 0x7fb8fc069d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb910005510 tx=0x7fb910009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:32.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.720+0000 7fb91d600640 1 -- 172.21.15.83:0/745044674 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb908016760 con 0x7fb92006ce10 2024-04-25T00:11:32.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.876+0000 7fb927800640 1 -- 172.21.15.83:0/745044674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb92004f5f0 con 0x7fb92006ce10 2024-04-25T00:11:32.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.876+0000 7fb91d600640 1 -- 172.21.15.83:0/745044674 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7fb9080f08b0 con 0x7fb92006ce10 2024-04-25T00:11:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:32.879 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:32.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.876+0000 7fb927800640 1 -- 172.21.15.83:0/745044674 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8fc0678c0 msgr2=0x7fb8fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.876+0000 7fb927800640 1 --2- 172.21.15.83:0/745044674 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8fc0678c0 0x7fb8fc069d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb910005510 tx=0x7fb910009040 comp rx=0 tx=0).stop 2024-04-25T00:11:32.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.876+0000 7fb927800640 1 -- 172.21.15.83:0/745044674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92006ce10 msgr2=0x7fb92006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.876+0000 7fb927800640 1 --2- 172.21.15.83:0/745044674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92006ce10 0x7fb92006d270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb90801c880 tx=0x7fb9080197f0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.876+0000 7fb927800640 1 -- 172.21.15.83:0/745044674 shutdown_connections 2024-04-25T00:11:32.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.876+0000 7fb927800640 1 --2- 172.21.15.83:0/745044674 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8fc0678c0 0x7fb8fc069d80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.876+0000 7fb927800640 1 --2- 172.21.15.83:0/745044674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9201f7850 0x7fb9201f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.876+0000 7fb927800640 1 --2- 172.21.15.83:0/745044674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb92006ce10 0x7fb92006d270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.876+0000 7fb927800640 1 --2- 172.21.15.83:0/745044674 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb904004810 0x7fb92014d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.876+0000 7fb927800640 1 -- 172.21.15.83:0/745044674 >> 172.21.15.83:0/745044674 conn(0x7fb9200d1c70 msgr2=0x7fb92006dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:32.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.880+0000 7fb927800640 1 -- 172.21.15.83:0/745044674 shutdown_connections 2024-04-25T00:11:32.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:32.880+0000 7fb927800640 1 -- 172.21.15.83:0/745044674 wait complete. 2024-04-25T00:11:32.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:11:04.621429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:11:05.189716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:11:08.902215+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:11:32.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:11:33.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.036+0000 7fc0ed600640 1 Processor -- start 2024-04-25T00:11:33.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.036+0000 7fc0ed600640 1 -- start start 2024-04-25T00:11:33.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.036+0000 7fc0ed600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0e8155310 0x7fc0e806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.036+0000 7fc0ed600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc0e815e790 0x7fc0e815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.036+0000 7fc0ed600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0e80719f0 con 0x7fc0e815b1a0 2024-04-25T00:11:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.036+0000 7fc0ed600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e815f610 con 0x7fc0e8155310 2024-04-25T00:11:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.036+0000 7fc0ed600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e8071cd0 con 0x7fc0e815e790 2024-04-25T00:11:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.036+0000 7fc0e7e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0e8155310 0x7fc0e806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.036+0000 7fc0e7e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0e8155310 0x7fc0e806cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36786/0 (socket says 172.21.15.83:36786) 2024-04-25T00:11:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.036+0000 7fc0e7e00640 1 -- 172.21.15.83:0/2760243906 learned_addr learned my addr 172.21.15.83:0/2760243906 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0df400640 1 --2- 172.21.15.83:0/2760243906 >> v2:172.21.15.86:3300/0 conn(0x7fc0e815e790 0x7fc0e815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0e7e00640 1 -- 172.21.15.83:0/2760243906 >> v2:172.21.15.86:3300/0 conn(0x7fc0e815e790 msgr2=0x7fc0e815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0e7e00640 1 --2- 172.21.15.83:0/2760243906 >> v2:172.21.15.86:3300/0 conn(0x7fc0e815e790 0x7fc0e815ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0e7e00640 1 -- 172.21.15.83:0/2760243906 >> v1:172.21.15.86:6789/0 conn(0x7fc0e815b1a0 legacy=0x7fc0e815b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0e7e00640 1 -- 172.21.15.83:0/2760243906 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e8071fb0 con 0x7fc0e8155310 2024-04-25T00:11:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0df400640 1 --2- 172.21.15.83:0/2760243906 >> v2:172.21.15.86:3300/0 conn(0x7fc0e815e790 0x7fc0e815ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0e7e00640 1 --2- 172.21.15.83:0/2760243906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e8155310 0x7fc0e806cc60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80b16c0 tx=0x7fc0d801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c3ce45c527deba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0e7400640 1 -- 172.21.15.83:0/2760243906 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d80b0070 con 0x7fc0e8155310 2024-04-25T00:11:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0e7400640 1 -- 172.21.15.83:0/2760243906 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d80ab600 con 0x7fc0e8155310 2024-04-25T00:11:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0e7400640 1 -- 172.21.15.83:0/2760243906 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d80ab9d0 con 0x7fc0e8155310 2024-04-25T00:11:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0ed600640 1 -- 172.21.15.83:0/2760243906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e8155310 msgr2=0x7fc0e806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0ed600640 1 --2- 172.21.15.83:0/2760243906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e8155310 0x7fc0e806cc60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80b16c0 tx=0x7fc0d801c450 comp rx=0 tx=0).stop 2024-04-25T00:11:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0ed600640 1 -- 172.21.15.83:0/2760243906 shutdown_connections 2024-04-25T00:11:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0ed600640 1 --2- 172.21.15.83:0/2760243906 >> v2:172.21.15.86:3300/0 conn(0x7fc0e815e790 0x7fc0e815ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.040+0000 7fc0ed600640 1 --2- 172.21.15.83:0/2760243906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e8155310 0x7fc0e806cc60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0ed600640 1 -- 172.21.15.83:0/2760243906 >> 172.21.15.83:0/2760243906 conn(0x7fc0e80cf980 msgr2=0x7fc0e80cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0ed600640 1 -- 172.21.15.83:0/2760243906 shutdown_connections 2024-04-25T00:11:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0ed600640 1 -- 172.21.15.83:0/2760243906 wait complete. 2024-04-25T00:11:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0ed600640 1 Processor -- start 2024-04-25T00:11:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0ed600640 1 -- start start 2024-04-25T00:11:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0ed600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e8155310 0x7fc0e80cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0ed600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e815b1a0 0x7fc0e813db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0ed600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0e815e790 0x7fc0e8153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0ed600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e8071740 con 0x7fc0e815b1a0 2024-04-25T00:11:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0ed600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e80715c0 con 0x7fc0e8155310 2024-04-25T00:11:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0ed600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e8168340 con 0x7fc0e815e790 2024-04-25T00:11:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0df400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e815b1a0 0x7fc0e813db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0df400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e815b1a0 0x7fc0e813db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36800/0 (socket says 172.21.15.83:36800) 2024-04-25T00:11:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0df400640 1 -- 172.21.15.83:0/899358353 learned_addr learned my addr 172.21.15.83:0/899358353 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0e7e00640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e8155310 0x7fc0e80cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0ecc00640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0e815e790 0x7fc0e8153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0df400640 1 -- 172.21.15.83:0/899358353 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0e815e790 msgr2=0x7fc0e8153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0df400640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0e815e790 0x7fc0e8153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0df400640 1 -- 172.21.15.83:0/899358353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e8155310 msgr2=0x7fc0e80cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0df400640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e8155310 0x7fc0e80cef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0df400640 1 -- 172.21.15.83:0/899358353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e80cdaf0 con 0x7fc0e815b1a0 2024-04-25T00:11:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0e7e00640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e8155310 0x7fc0e80cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0df400640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e815b1a0 0x7fc0e813db80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000c7c0 tx=0x7fc0d000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.044+0000 7fc0e4c00640 1 -- 172.21.15.83:0/899358353 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0007a30 con 0x7fc0e815b1a0 2024-04-25T00:11:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.048+0000 7fc0e4c00640 1 -- 172.21.15.83:0/899358353 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d0007bd0 con 0x7fc0e815b1a0 2024-04-25T00:11:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.048+0000 7fc0ed600640 1 -- 172.21.15.83:0/899358353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e80cf480 con 0x7fc0e815b1a0 2024-04-25T00:11:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.048+0000 7fc0e4c00640 1 -- 172.21.15.83:0/899358353 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d00113f0 con 0x7fc0e815b1a0 2024-04-25T00:11:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.048+0000 7fc0ed600640 1 -- 172.21.15.83:0/899358353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e814f660 con 0x7fc0e815b1a0 2024-04-25T00:11:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.048+0000 7fc0ed600640 1 -- 172.21.15.83:0/899358353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e804f510 con 0x7fc0e815b1a0 2024-04-25T00:11:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.048+0000 7fc0e4c00640 1 -- 172.21.15.83:0/899358353 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc0d0011590 con 0x7fc0e815b1a0 2024-04-25T00:11:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.048+0000 7fc0e4c00640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc0c40678c0 0x7fc0c4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.048+0000 7fc0e4c00640 1 -- 172.21.15.83:0/899358353 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc0d0086230 con 0x7fc0e815b1a0 2024-04-25T00:11:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.048+0000 7fc0e7e00640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc0c40678c0 0x7fc0c4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.048+0000 7fc0e7e00640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc0c40678c0 0x7fc0c4069d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc0d80ad000 tx=0x7fc0d80a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.052+0000 7fc0e4c00640 1 -- 172.21.15.83:0/899358353 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc0d0015020 con 0x7fc0e815b1a0 2024-04-25T00:11:33.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.208+0000 7fc0ed600640 1 -- 172.21.15.83:0/899358353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc0e80d0740 con 0x7fc0e815b1a0 2024-04-25T00:11:33.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.208+0000 7fc0e4c00640 1 -- 172.21.15.83:0/899358353 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fc0d0056990 con 0x7fc0e815b1a0 2024-04-25T00:11:33.211 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:11:33.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.212+0000 7fc0ed600640 1 -- 172.21.15.83:0/899358353 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc0c40678c0 msgr2=0x7fc0c4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.212+0000 7fc0ed600640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc0c40678c0 0x7fc0c4069d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc0d80ad000 tx=0x7fc0d80a6000 comp rx=0 tx=0).stop 2024-04-25T00:11:33.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.212+0000 7fc0ed600640 1 -- 172.21.15.83:0/899358353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e815b1a0 msgr2=0x7fc0e813db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.212+0000 7fc0ed600640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e815b1a0 0x7fc0e813db80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000c7c0 tx=0x7fc0d000cc80 comp rx=0 tx=0).stop 2024-04-25T00:11:33.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.212+0000 7fc0ed600640 1 -- 172.21.15.83:0/899358353 shutdown_connections 2024-04-25T00:11:33.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.212+0000 7fc0ed600640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc0c40678c0 0x7fc0c4069d80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.212+0000 7fc0ed600640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0e815e790 0x7fc0e8153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.212+0000 7fc0ed600640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e815b1a0 0x7fc0e813db80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.212+0000 7fc0ed600640 1 --2- 172.21.15.83:0/899358353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e8155310 0x7fc0e80cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.212+0000 7fc0ed600640 1 -- 172.21.15.83:0/899358353 >> 172.21.15.83:0/899358353 conn(0x7fc0e80cf980 msgr2=0x7fc0e8070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:33.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.212+0000 7fc0ed600640 1 -- 172.21.15.83:0/899358353 shutdown_connections 2024-04-25T00:11:33.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.212+0000 7fc0ed600640 1 -- 172.21.15.83:0/899358353 wait complete. 2024-04-25T00:11:33.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:11:33.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd15800640 1 Processor -- start 2024-04-25T00:11:33.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd15800640 1 -- start start 2024-04-25T00:11:33.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd15800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd10159810 0x7fcd100cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd15800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd1006e950 0x7fcd1015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd15800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd100ce000 con 0x7fcd1015fe50 2024-04-25T00:11:33.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd15800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fcd1006e210 con 0x7fcd1006e950 2024-04-25T00:11:33.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd15800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1006da00 con 0x7fcd10159810 2024-04-25T00:11:33.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd06000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd1006e950 0x7fcd1015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd0f400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcd1015fe50 0x7fcd1015c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52340/0 (socket says 172.21.15.83:52340) 2024-04-25T00:11:33.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd0f400640 1 -- 172.21.15.83:0/743140906 learned_addr learned my addr 172.21.15.83:0/743140906 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:33.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd0ea00640 1 --2- 172.21.15.83:0/743140906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd10159810 0x7fcd100cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd06000640 1 -- 172.21.15.83:0/743140906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd10159810 msgr2=0x7fcd100cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:33.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd06000640 1 --2- 172.21.15.83:0/743140906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd10159810 0x7fcd100cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd06000640 1 -- 172.21.15.83:0/743140906 >> v1:172.21.15.83:6789/0 conn(0x7fcd1015fe50 legacy=0x7fcd1015c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd06000640 1 -- 172.21.15.83:0/743140906 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1006eec0 con 0x7fcd1006e950 2024-04-25T00:11:33.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.368+0000 7fcd0e000640 1 -- 172.21.15.83:0/743140906 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3509847169 0 0) 0x7fcd100ce000 con 0x7fcd1015fe50 2024-04-25T00:11:33.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.372+0000 7fcd06000640 1 --2- 172.21.15.83:0/743140906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd1006e950 0x7fcd1015f910 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fccf80097c0 tx=0x7fccf801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a7a23758790658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:33.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.372+0000 7fcd0e000640 1 -- 172.21.15.83:0/743140906 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf80b0070 con 0x7fcd1006e950 2024-04-25T00:11:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.372+0000 7fcd0e000640 1 -- 172.21.15.83:0/743140906 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf80ab600 con 0x7fcd1006e950 2024-04-25T00:11:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.372+0000 7fcd0e000640 1 -- 172.21.15.83:0/743140906 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf80ab9d0 con 0x7fcd1006e950 2024-04-25T00:11:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.372+0000 7fcd15800640 1 -- 172.21.15.83:0/743140906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd1006e950 msgr2=0x7fcd1015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.372+0000 7fcd15800640 1 --2- 172.21.15.83:0/743140906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd1006e950 0x7fcd1015f910 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fccf80097c0 tx=0x7fccf801c410 comp rx=0 tx=0).stop 2024-04-25T00:11:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.372+0000 7fcd15800640 1 -- 172.21.15.83:0/743140906 shutdown_connections 2024-04-25T00:11:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.372+0000 7fcd15800640 1 --2- 172.21.15.83:0/743140906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd1006e950 0x7fcd1015f910 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.372+0000 7fcd15800640 1 --2- 172.21.15.83:0/743140906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd10159810 0x7fcd100cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.372+0000 7fcd15800640 1 -- 172.21.15.83:0/743140906 >> 172.21.15.83:0/743140906 conn(0x7fcd100b1830 msgr2=0x7fcd100b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.372+0000 7fcd15800640 1 -- 172.21.15.83:0/743140906 shutdown_connections 2024-04-25T00:11:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.372+0000 7fcd15800640 1 -- 172.21.15.83:0/743140906 wait complete. 2024-04-25T00:11:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd15800640 1 Processor -- start 2024-04-25T00:11:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd15800640 1 -- start start 2024-04-25T00:11:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd15800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1006e950 0x7fcd1013e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd15800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd10159810 0x7fcd100b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd15800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd1015fe50 0x7fcd101532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd15800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd10159d70 con 0x7fcd10159810 2024-04-25T00:11:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd15800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd10159bf0 con 0x7fcd1006e950 2024-04-25T00:11:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd15800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcd10070da0 con 0x7fcd1015fe50 2024-04-25T00:11:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd06000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd10159810 0x7fcd100b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd06000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd10159810 0x7fcd100b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36820/0 (socket says 172.21.15.83:36820) 2024-04-25T00:11:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd06000640 1 -- 172.21.15.83:0/340130666 learned_addr learned my addr 172.21.15.83:0/340130666 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd0ea00640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1006e950 0x7fcd1013e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd0f400640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd1015fe50 0x7fcd101532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd06000640 1 -- 172.21.15.83:0/340130666 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd1015fe50 msgr2=0x7fcd101532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd06000640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd1015fe50 0x7fcd101532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd06000640 1 -- 172.21.15.83:0/340130666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1006e950 msgr2=0x7fcd1013e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd06000640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1006e950 0x7fcd1013e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd06000640 1 -- 172.21.15.83:0/340130666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd100afc80 con 0x7fcd10159810 2024-04-25T00:11:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd0f400640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd1015fe50 0x7fcd101532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd0ea00640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1006e950 0x7fcd1013e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd06000640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd10159810 0x7fcd100b2e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fccf80097c0 tx=0x7fccf801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd07400640 1 -- 172.21.15.83:0/340130666 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf80bc070 con 0x7fcd10159810 2024-04-25T00:11:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd07400640 1 -- 172.21.15.83:0/340130666 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf80b1040 con 0x7fcd10159810 2024-04-25T00:11:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd07400640 1 -- 172.21.15.83:0/340130666 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf80b0050 con 0x7fcd10159810 2024-04-25T00:11:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd15800640 1 -- 172.21.15.83:0/340130666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd100aff60 con 0x7fcd10159810 2024-04-25T00:11:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.376+0000 7fcd15800640 1 -- 172.21.15.83:0/340130666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd100b0520 con 0x7fcd10159810 2024-04-25T00:11:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.380+0000 7fcd15800640 1 -- 172.21.15.83:0/340130666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1004f510 con 0x7fcd10159810 2024-04-25T00:11:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.380+0000 7fcd07400640 1 -- 172.21.15.83:0/340130666 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fccf80bf070 con 0x7fcd10159810 2024-04-25T00:11:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.380+0000 7fcd07400640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccec0678c0 0x7fccec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.380+0000 7fcd07400640 1 -- 172.21.15.83:0/340130666 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fccf811faa0 con 0x7fcd10159810 2024-04-25T00:11:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.380+0000 7fcd0ea00640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccec0678c0 0x7fccec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.380+0000 7fcd0ea00640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccec0678c0 0x7fccec069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcd10153ed0 tx=0x7fcd00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.384+0000 7fcd07400640 1 -- 172.21.15.83:0/340130666 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fccf8016090 con 0x7fcd10159810 2024-04-25T00:11:33.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.540+0000 7fcd15800640 1 -- 172.21.15.83:0/340130666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcd10072c90 con 0x7fcd10159810 2024-04-25T00:11:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.540+0000 7fcd07400640 1 -- 172.21.15.83:0/340130666 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fccf80f0200 con 0x7fcd10159810 2024-04-25T00:11:33.545 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.544+0000 7fcd15800640 1 -- 172.21.15.83:0/340130666 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccec0678c0 msgr2=0x7fccec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.544+0000 7fcd15800640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccec0678c0 0x7fccec069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcd10153ed0 tx=0x7fcd00008040 comp rx=0 tx=0).stop 2024-04-25T00:11:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.544+0000 7fcd15800640 1 -- 172.21.15.83:0/340130666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd10159810 msgr2=0x7fcd100b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.544+0000 7fcd15800640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd10159810 0x7fcd100b2e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fccf80097c0 tx=0x7fccf801ac50 comp rx=0 tx=0).stop 2024-04-25T00:11:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.544+0000 7fcd15800640 1 -- 172.21.15.83:0/340130666 shutdown_connections 2024-04-25T00:11:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.544+0000 7fcd15800640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccec0678c0 0x7fccec069d80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.544+0000 7fcd15800640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd1015fe50 0x7fcd101532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.544+0000 7fcd15800640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd10159810 0x7fcd100b2e60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.544+0000 7fcd15800640 1 --2- 172.21.15.83:0/340130666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1006e950 0x7fcd1013e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.544+0000 7fcd15800640 1 -- 172.21.15.83:0/340130666 >> 172.21.15.83:0/340130666 conn(0x7fcd100b1830 msgr2=0x7fcd10165580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.544+0000 7fcd15800640 1 -- 172.21.15.83:0/340130666 shutdown_connections 2024-04-25T00:11:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.544+0000 7fcd15800640 1 -- 172.21.15.83:0/340130666 wait complete. 2024-04-25T00:11:33.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:11:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.700+0000 7f9eb4e00640 1 Processor -- start 2024-04-25T00:11:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.700+0000 7f9eb4e00640 1 -- start start 2024-04-25T00:11:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.700+0000 7f9eb4e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9eb015a330 0x7f9eb00b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.700+0000 7f9eb4e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9eb015a710 0x7f9eb0070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9eb4e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb0071850 con 0x7f9eb006ddc0 2024-04-25T00:11:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9eb4e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb00cc690 con 0x7f9eb015a330 2024-04-25T00:11:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9eb4e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb00b41b0 con 0x7f9eb015a710 2024-04-25T00:11:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9eade00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9eb015a330 0x7f9eb00b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9eade00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9eb015a330 0x7f9eb00b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36824/0 (socket says 172.21.15.83:36824) 2024-04-25T00:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9eade00640 1 -- 172.21.15.83:0/32881433 learned_addr learned my addr 172.21.15.83:0/32881433 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9ea5400640 1 --2- 172.21.15.83:0/32881433 >> v2:172.21.15.86:3300/0 conn(0x7f9eb015a710 0x7f9eb0070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9eade00640 1 -- 172.21.15.83:0/32881433 >> v2:172.21.15.86:3300/0 conn(0x7f9eb015a710 msgr2=0x7f9eb0070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9eade00640 1 --2- 172.21.15.83:0/32881433 >> v2:172.21.15.86:3300/0 conn(0x7f9eb015a710 0x7f9eb0070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9eade00640 1 -- 172.21.15.83:0/32881433 >> v1:172.21.15.86:6789/0 conn(0x7f9eb006ddc0 legacy=0x7f9eb006e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9eade00640 1 -- 172.21.15.83:0/32881433 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb0071ac0 con 0x7f9eb015a330 2024-04-25T00:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9ea5400640 1 --2- 172.21.15.83:0/32881433 >> v2:172.21.15.86:3300/0 conn(0x7f9eb015a710 0x7f9eb0070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9eade00640 1 --2- 172.21.15.83:0/32881433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eb015a330 0x7f9eb00b4a40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00b2470 tx=0x7f9ea0021380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538caba5a280aaa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9ead400640 1 -- 172.21.15.83:0/32881433 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00b8070 con 0x7f9eb015a330 2024-04-25T00:11:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9ead400640 1 -- 172.21.15.83:0/32881433 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea00b3410 con 0x7f9eb015a330 2024-04-25T00:11:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.704+0000 7f9ead400640 1 -- 172.21.15.83:0/32881433 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00b37e0 con 0x7f9eb015a330 2024-04-25T00:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 -- 172.21.15.83:0/32881433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eb015a330 msgr2=0x7f9eb00b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 --2- 172.21.15.83:0/32881433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eb015a330 0x7f9eb00b4a40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00b2470 tx=0x7f9ea0021380 comp rx=0 tx=0).stop 2024-04-25T00:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 -- 172.21.15.83:0/32881433 shutdown_connections 2024-04-25T00:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 --2- 172.21.15.83:0/32881433 >> v2:172.21.15.86:3300/0 conn(0x7f9eb015a710 0x7f9eb0070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 --2- 172.21.15.83:0/32881433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eb015a330 0x7f9eb00b4a40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 -- 172.21.15.83:0/32881433 >> 172.21.15.83:0/32881433 conn(0x7f9eb00b1830 msgr2=0x7f9eb00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 -- 172.21.15.83:0/32881433 shutdown_connections 2024-04-25T00:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 -- 172.21.15.83:0/32881433 wait complete. 2024-04-25T00:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 Processor -- start 2024-04-25T00:11:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 -- start start 2024-04-25T00:11:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eb006ddc0 0x7f9eb01e5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb015a330 0x7f9eb0149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eb014a300 0x7f9eb014b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb006cb20 con 0x7f9eb014a300 2024-04-25T00:11:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb00b4490 con 0x7f9eb015a330 2024-04-25T00:11:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eb4e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb006cca0 con 0x7f9eb006ddc0 2024-04-25T00:11:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eae800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eb014a300 0x7f9eb014b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eae800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eb014a300 0x7f9eb014b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36840/0 (socket says 172.21.15.83:36840) 2024-04-25T00:11:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eae800640 1 -- 172.21.15.83:0/2143559752 learned_addr learned my addr 172.21.15.83:0/2143559752 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9eade00640 1 --2- 172.21.15.83:0/2143559752 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eb006ddc0 0x7f9eb01e5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.708+0000 7f9ea5400640 1 --2- 172.21.15.83:0/2143559752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb015a330 0x7f9eb0149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9eae800640 1 -- 172.21.15.83:0/2143559752 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eb006ddc0 msgr2=0x7f9eb01e5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9eae800640 1 --2- 172.21.15.83:0/2143559752 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eb006ddc0 0x7f9eb01e5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9eae800640 1 -- 172.21.15.83:0/2143559752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb015a330 msgr2=0x7f9eb0149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9eae800640 1 --2- 172.21.15.83:0/2143559752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb015a330 0x7f9eb0149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9eae800640 1 -- 172.21.15.83:0/2143559752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb014a100 con 0x7f9eb014a300 2024-04-25T00:11:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9eae800640 1 --2- 172.21.15.83:0/2143559752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eb014a300 0x7f9eb014b200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800c3c0 tx=0x7f9ea800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9ea6a00640 1 -- 172.21.15.83:0/2143559752 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8011790 con 0x7f9eb014a300 2024-04-25T00:11:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9ea6a00640 1 -- 172.21.15.83:0/2143559752 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea8017070 con 0x7f9eb014a300 2024-04-25T00:11:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9ea6a00640 1 -- 172.21.15.83:0/2143559752 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8011b70 con 0x7f9eb014a300 2024-04-25T00:11:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9eb4e00640 1 -- 172.21.15.83:0/2143559752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb015bd50 con 0x7f9eb014a300 2024-04-25T00:11:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9eb4e00640 1 -- 172.21.15.83:0/2143559752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb015c310 con 0x7f9eb014a300 2024-04-25T00:11:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9eb4e00640 1 -- 172.21.15.83:0/2143559752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb015c4e0 con 0x7f9eb014a300 2024-04-25T00:11:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9ea6a00640 1 -- 172.21.15.83:0/2143559752 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9ea8011d10 con 0x7f9eb014a300 2024-04-25T00:11:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9ea6a00640 1 --2- 172.21.15.83:0/2143559752 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9e8c0678c0 0x7f9e8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9ea6a00640 1 -- 172.21.15.83:0/2143559752 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9ea8085c80 con 0x7f9eb014a300 2024-04-25T00:11:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.712+0000 7f9eade00640 1 --2- 172.21.15.83:0/2143559752 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9e8c0678c0 0x7f9e8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.716+0000 7f9eade00640 1 --2- 172.21.15.83:0/2143559752 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9e8c0678c0 0x7f9e8c069d80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00b5000 tx=0x7f9ea00ac000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:33.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.716+0000 7f9ea6a00640 1 -- 172.21.15.83:0/2143559752 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9eb015c4e0 con 0x7f9eb014a300 2024-04-25T00:11:33.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.872+0000 7f9eb4e00640 1 -- 172.21.15.83:0/2143559752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9eb004f510 con 0x7f9eb014a300 2024-04-25T00:11:33.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.872+0000 7f9ea6a00640 1 -- 172.21.15.83:0/2143559752 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f9eb014a100 con 0x7f9eb014a300 2024-04-25T00:11:33.875 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:11:33.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.872+0000 7f9eb4e00640 1 -- 172.21.15.83:0/2143559752 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9e8c0678c0 msgr2=0x7f9e8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.872+0000 7f9eb4e00640 1 --2- 172.21.15.83:0/2143559752 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9e8c0678c0 0x7f9e8c069d80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00b5000 tx=0x7f9ea00ac000 comp rx=0 tx=0).stop 2024-04-25T00:11:33.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.876+0000 7f9eb4e00640 1 -- 172.21.15.83:0/2143559752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eb014a300 msgr2=0x7f9eb014b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:33.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.876+0000 7f9eb4e00640 1 --2- 172.21.15.83:0/2143559752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eb014a300 0x7f9eb014b200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800c3c0 tx=0x7f9ea800c880 comp rx=0 tx=0).stop 2024-04-25T00:11:33.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.876+0000 7f9eb4e00640 1 -- 172.21.15.83:0/2143559752 shutdown_connections 2024-04-25T00:11:33.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.876+0000 7f9eb4e00640 1 --2- 172.21.15.83:0/2143559752 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9e8c0678c0 0x7f9e8c069d80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.876+0000 7f9eb4e00640 1 --2- 172.21.15.83:0/2143559752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eb014a300 0x7f9eb014b200 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.876+0000 7f9eb4e00640 1 --2- 172.21.15.83:0/2143559752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eb015a330 0x7f9eb0149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.876+0000 7f9eb4e00640 1 --2- 172.21.15.83:0/2143559752 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eb006ddc0 0x7f9eb01e5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:33.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.876+0000 7f9eb4e00640 1 -- 172.21.15.83:0/2143559752 >> 172.21.15.83:0/2143559752 conn(0x7f9eb00b1830 msgr2=0x7f9eb00cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:33.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.876+0000 7f9eb4e00640 1 -- 172.21.15.83:0/2143559752 shutdown_connections 2024-04-25T00:11:33.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:33.876+0000 7f9eb4e00640 1 -- 172.21.15.83:0/2143559752 wait complete. 2024-04-25T00:11:33.896 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:11:33.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:11:34.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:11:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.032+0000 7fae9aa00640 1 Processor -- start 2024-04-25T00:11:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.032+0000 7fae9aa00640 1 -- start start 2024-04-25T00:11:34.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.032+0000 7fae9aa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae9415f080 0x7fae9406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.032+0000 7fae9aa00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae94072490 con 0x7fae94072040 2024-04-25T00:11:34.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.032+0000 7fae9aa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae94072740 con 0x7fae9415e590 2024-04-25T00:11:34.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.032+0000 7fae9aa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fae940ae6f0 con 0x7fae9415f080 2024-04-25T00:11:34.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.032+0000 7fae98a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fae94072040 0x7fae940af1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52354/0 (socket says 172.21.15.83:52354) 2024-04-25T00:11:34.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.032+0000 7fae98a00640 1 -- 172.21.15.83:0/2926778202 learned_addr learned my addr 172.21.15.83:0/2926778202 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:34.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae8be00640 1 --2- 172.21.15.83:0/2926778202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae9415f080 0x7fae9406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 511724034 0 0) 0x7fae94072490 con 0x7fae94072040 2024-04-25T00:11:34.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae78003530 con 0x7fae94072040 2024-04-25T00:11:34.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2142563251 0 0) 0x7fae94072740 con 0x7fae9415e590 2024-04-25T00:11:34.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae94072490 con 0x7fae9415e590 2024-04-25T00:11:34.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1587965894 0 0) 0x7fae78003530 con 0x7fae94072040 2024-04-25T00:11:34.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae9415f080 msgr2=0x7fae9406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:34.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 --2- 172.21.15.83:0/2926778202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae9415f080 0x7fae9406d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 >> v1:172.21.15.86:6789/0 conn(0x7fae9415e590 legacy=0x7fae9415e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae940ae9d0 con 0x7fae94072040 2024-04-25T00:11:34.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fae8c001a00 con 0x7fae94072040 2024-04-25T00:11:34.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae8be00640 1 --2- 172.21.15.83:0/2926778202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae9415f080 0x7fae9406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:34.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 >> v1:172.21.15.83:6789/0 conn(0x7fae94072040 legacy=0x7fae940af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 --2- 172.21.15.83:0/2926778202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae9415f080 0x7fae78003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 --2- 172.21.15.83:0/2926778202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae78004180 0x7fae78004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 --2- 172.21.15.83:0/2926778202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae78004d50 0x7fae780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae940ae6f0 con 0x7fae78004180 2024-04-25T00:11:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae7800bbe0 con 0x7fae78004d50 2024-04-25T00:11:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fae7800bde0 con 0x7fae9415f080 2024-04-25T00:11:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae8be00640 1 --2- 172.21.15.83:0/2926778202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae78004180 0x7fae78004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae93e00640 1 --2- 172.21.15.83:0/2926778202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae9415f080 0x7fae78003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae98a00640 1 --2- 172.21.15.83:0/2926778202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae78004d50 0x7fae780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae8be00640 1 -- 172.21.15.83:0/2926778202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae9415f080 msgr2=0x7fae78003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae8be00640 1 --2- 172.21.15.83:0/2926778202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae9415f080 0x7fae78003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae8be00640 1 -- 172.21.15.83:0/2926778202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae78004d50 msgr2=0x7fae780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae8be00640 1 --2- 172.21.15.83:0/2926778202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae78004d50 0x7fae780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.036+0000 7fae8be00640 1 -- 172.21.15.83:0/2926778202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae78003530 con 0x7fae78004180 2024-04-25T00:11:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae8be00640 1 --2- 172.21.15.83:0/2926778202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae78004180 0x7fae78004810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009c90 tx=0x7fae7c01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409cbde1627a4ee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c009860 con 0x7fae78004180 2024-04-25T00:11:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7c004030 con 0x7fae78004180 2024-04-25T00:11:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae93400640 1 -- 172.21.15.83:0/2926778202 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c004460 con 0x7fae78004180 2024-04-25T00:11:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae9aa00640 1 -- 172.21.15.83:0/2926778202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae78004180 msgr2=0x7fae78004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae9aa00640 1 --2- 172.21.15.83:0/2926778202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae78004180 0x7fae78004810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009c90 tx=0x7fae7c01cc10 comp rx=0 tx=0).stop 2024-04-25T00:11:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae98a00640 1 -- 172.21.15.83:0/2926778202 reap_dead start 2024-04-25T00:11:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae9aa00640 1 -- 172.21.15.83:0/2926778202 shutdown_connections 2024-04-25T00:11:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae9aa00640 1 -- 172.21.15.83:0/2926778202 >> 172.21.15.83:0/2926778202 conn(0x7fae940b2190 msgr2=0x7fae940b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae9aa00640 1 -- 172.21.15.83:0/2926778202 shutdown_connections 2024-04-25T00:11:34.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae9aa00640 1 -- 172.21.15.83:0/2926778202 wait complete. 2024-04-25T00:11:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae9aa00640 1 Processor -- start 2024-04-25T00:11:34.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae9aa00640 1 -- start start 2024-04-25T00:11:34.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae9aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae78004180 0x7fae941600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.040+0000 7fae9aa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae940b1110 0x7fae94151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae9aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae94158be0 0x7fae94163700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae9aa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae940afa60 con 0x7fae94158be0 2024-04-25T00:11:34.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae9aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae940af8e0 con 0x7fae78004180 2024-04-25T00:11:34.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae9aa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fae940afbe0 con 0x7fae940b1110 2024-04-25T00:11:34.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae98a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae94158be0 0x7fae94163700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae78004180 0x7fae941600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae8be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae940b1110 0x7fae94151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae78004180 0x7fae941600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:39546/0 (socket says 172.21.15.83:39546) 2024-04-25T00:11:34.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae93e00640 1 -- 172.21.15.83:0/4255303732 learned_addr learned my addr 172.21.15.83:0/4255303732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:34.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae93e00640 1 -- 172.21.15.83:0/4255303732 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae940b1110 msgr2=0x7fae94151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae93e00640 1 --2- 172.21.15.83:0/4255303732 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae940b1110 0x7fae94151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae93e00640 1 -- 172.21.15.83:0/4255303732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae94158be0 msgr2=0x7fae94163700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae93e00640 1 --2- 172.21.15.83:0/4255303732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae94158be0 0x7fae94163700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae93e00640 1 -- 172.21.15.83:0/4255303732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae94166c70 con 0x7fae78004180 2024-04-25T00:11:34.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae98a00640 1 --2- 172.21.15.83:0/4255303732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae94158be0 0x7fae94163700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae8be00640 1 --2- 172.21.15.83:0/4255303732 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae940b1110 0x7fae94151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae93e00640 1 --2- 172.21.15.83:0/4255303732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae78004180 0x7fae941600d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae8400f810 tx=0x7fae8400fcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae90c00640 1 -- 172.21.15.83:0/4255303732 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae840052e0 con 0x7fae78004180 2024-04-25T00:11:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae90c00640 1 -- 172.21.15.83:0/4255303732 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae84018070 con 0x7fae78004180 2024-04-25T00:11:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae90c00640 1 -- 172.21.15.83:0/4255303732 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae840054a0 con 0x7fae78004180 2024-04-25T00:11:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae9aa00640 1 -- 172.21.15.83:0/4255303732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae94166f50 con 0x7fae78004180 2024-04-25T00:11:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae9aa00640 1 -- 172.21.15.83:0/4255303732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae941522f0 con 0x7fae78004180 2024-04-25T00:11:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae9aa00640 1 -- 172.21.15.83:0/4255303732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae9404f510 con 0x7fae78004180 2024-04-25T00:11:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.044+0000 7fae90c00640 1 -- 172.21.15.83:0/4255303732 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fae8400d910 con 0x7fae78004180 2024-04-25T00:11:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.048+0000 7fae90c00640 1 --2- 172.21.15.83:0/4255303732 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fae700678c0 0x7fae70069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.048+0000 7fae90c00640 1 -- 172.21.15.83:0/4255303732 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fae84085ea0 con 0x7fae78004180 2024-04-25T00:11:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.048+0000 7fae8be00640 1 --2- 172.21.15.83:0/4255303732 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fae700678c0 0x7fae70069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.048+0000 7fae8be00640 1 --2- 172.21.15.83:0/4255303732 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fae700678c0 0x7fae70069d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0ae000 tx=0x7fae7c0046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:34.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.052+0000 7fae90c00640 1 -- 172.21.15.83:0/4255303732 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fae84056040 con 0x7fae78004180 2024-04-25T00:11:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.208+0000 7fae9aa00640 1 -- 172.21.15.83:0/4255303732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fae940b28e0 con 0x7fae78004180 2024-04-25T00:11:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.208+0000 7fae90c00640 1 -- 172.21.15.83:0/4255303732 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fae8405a340 con 0x7fae78004180 2024-04-25T00:11:34.210 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.208+0000 7fae9aa00640 1 -- 172.21.15.83:0/4255303732 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fae700678c0 msgr2=0x7fae70069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.208+0000 7fae9aa00640 1 --2- 172.21.15.83:0/4255303732 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fae700678c0 0x7fae70069d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0ae000 tx=0x7fae7c0046d0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.208+0000 7fae9aa00640 1 -- 172.21.15.83:0/4255303732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae78004180 msgr2=0x7fae941600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.208+0000 7fae9aa00640 1 --2- 172.21.15.83:0/4255303732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae78004180 0x7fae941600d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae8400f810 tx=0x7fae8400fcd0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.208+0000 7fae9aa00640 1 -- 172.21.15.83:0/4255303732 shutdown_connections 2024-04-25T00:11:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.208+0000 7fae9aa00640 1 --2- 172.21.15.83:0/4255303732 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fae700678c0 0x7fae70069d80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.208+0000 7fae9aa00640 1 --2- 172.21.15.83:0/4255303732 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae940b1110 0x7fae94151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.208+0000 7fae9aa00640 1 --2- 172.21.15.83:0/4255303732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae94158be0 0x7fae94163700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.208+0000 7fae9aa00640 1 --2- 172.21.15.83:0/4255303732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae78004180 0x7fae941600d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.208+0000 7fae9aa00640 1 -- 172.21.15.83:0/4255303732 >> 172.21.15.83:0/4255303732 conn(0x7fae940b2190 msgr2=0x7fae94071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.208+0000 7fae9aa00640 1 -- 172.21.15.83:0/4255303732 shutdown_connections 2024-04-25T00:11:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.208+0000 7fae9aa00640 1 -- 172.21.15.83:0/4255303732 wait complete. 2024-04-25T00:11:34.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:11:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.403+0000 7f8c4f200640 1 Processor -- start 2024-04-25T00:11:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.403+0000 7f8c4f200640 1 -- start start 2024-04-25T00:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.407+0000 7f8c4f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c48144470 0x7f8c4815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.407+0000 7f8c4f200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8c480b34f0 0x7f8c480cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.407+0000 7f8c4f200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c480b3f40 con 0x7f8c480b3970 2024-04-25T00:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.407+0000 7f8c4f200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8c48144850 con 0x7f8c480b34f0 2024-04-25T00:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.407+0000 7f8c4f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8c480b4220 con 0x7f8c48144470 2024-04-25T00:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.407+0000 7f8c4d400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8c480b34f0 0x7f8c480cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.407+0000 7f8c4e800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8c480b3970 0x7f8c480ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48280/0 (socket says 172.21.15.83:48280) 2024-04-25T00:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.407+0000 7f8c4e800640 1 -- 172.21.15.83:0/1743476379 learned_addr learned my addr 172.21.15.83:0/1743476379 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.407+0000 7f8c4de00640 1 --2- 172.21.15.83:0/1743476379 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c48144470 0x7f8c4815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.407+0000 7f8c4d400640 1 -- 172.21.15.83:0/1743476379 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c48144470 msgr2=0x7f8c4815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.407+0000 7f8c4d400640 1 --2- 172.21.15.83:0/1743476379 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c48144470 0x7f8c4815f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.407+0000 7f8c4d400640 1 -- 172.21.15.83:0/1743476379 >> v1:172.21.15.83:6789/0 conn(0x7f8c480b3970 legacy=0x7f8c480ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.407+0000 7f8c4d400640 1 -- 172.21.15.83:0/1743476379 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4815e5e0 con 0x7f8c480b34f0 2024-04-25T00:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.407+0000 7f8c4ca00640 1 -- 172.21.15.83:0/1743476379 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2370425187 0 0) 0x7f8c480b3f40 con 0x7f8c480b3970 2024-04-25T00:11:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.411+0000 7f8c4d400640 1 --2- 172.21.15.83:0/1743476379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c480b34f0 0x7f8c480cfad0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8c380097c0 tx=0x7f8c3801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997d4317778cd447 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.411+0000 7f8c4ca00640 1 -- 172.21.15.83:0/1743476379 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c380b2070 con 0x7f8c480b34f0 2024-04-25T00:11:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.411+0000 7f8c4ca00640 1 -- 172.21.15.83:0/1743476379 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c380ad680 con 0x7f8c480b34f0 2024-04-25T00:11:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.411+0000 7f8c4ca00640 1 -- 172.21.15.83:0/1743476379 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c380ada50 con 0x7f8c480b34f0 2024-04-25T00:11:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.411+0000 7f8c4f200640 1 -- 172.21.15.83:0/1743476379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c480b34f0 msgr2=0x7f8c480cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.411+0000 7f8c4f200640 1 --2- 172.21.15.83:0/1743476379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c480b34f0 0x7f8c480cfad0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8c380097c0 tx=0x7f8c3801c470 comp rx=0 tx=0).stop 2024-04-25T00:11:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.411+0000 7f8c4f200640 1 -- 172.21.15.83:0/1743476379 shutdown_connections 2024-04-25T00:11:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.411+0000 7f8c4f200640 1 --2- 172.21.15.83:0/1743476379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c480b34f0 0x7f8c480cfad0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.411+0000 7f8c4f200640 1 --2- 172.21.15.83:0/1743476379 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c48144470 0x7f8c4815f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.411+0000 7f8c4f200640 1 -- 172.21.15.83:0/1743476379 >> 172.21.15.83:0/1743476379 conn(0x7f8c4806b3b0 msgr2=0x7f8c4806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.411+0000 7f8c4f200640 1 -- 172.21.15.83:0/1743476379 shutdown_connections 2024-04-25T00:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.411+0000 7f8c4f200640 1 -- 172.21.15.83:0/1743476379 wait complete. 2024-04-25T00:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.411+0000 7f8c4f200640 1 Processor -- start 2024-04-25T00:11:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.411+0000 7f8c4f200640 1 -- start start 2024-04-25T00:11:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c480b34f0 0x7f8c48159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c480b3970 0x7f8c4813f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c48144470 0x7f8c481596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c48167860 con 0x7f8c480b34f0 2024-04-25T00:11:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c481676e0 con 0x7f8c480b3970 2024-04-25T00:11:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8c481679e0 con 0x7f8c48144470 2024-04-25T00:11:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c480b34f0 0x7f8c48159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c480b34f0 0x7f8c48159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45964/0 (socket says 172.21.15.83:45964) 2024-04-25T00:11:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4de00640 1 -- 172.21.15.83:0/4010785744 learned_addr learned my addr 172.21.15.83:0/4010785744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4e800640 1 --2- 172.21.15.83:0/4010785744 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c48144470 0x7f8c481596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4d400640 1 --2- 172.21.15.83:0/4010785744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c480b3970 0x7f8c4813f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4de00640 1 -- 172.21.15.83:0/4010785744 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c48144470 msgr2=0x7f8c481596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4de00640 1 --2- 172.21.15.83:0/4010785744 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c48144470 0x7f8c481596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4de00640 1 -- 172.21.15.83:0/4010785744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c480b3970 msgr2=0x7f8c4813f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4de00640 1 --2- 172.21.15.83:0/4010785744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c480b3970 0x7f8c4813f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4de00640 1 -- 172.21.15.83:0/4010785744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c48162590 con 0x7f8c480b34f0 2024-04-25T00:11:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4de00640 1 --2- 172.21.15.83:0/4010785744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c480b34f0 0x7f8c48159f20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00e870 tx=0x7f8c3c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c42000640 1 -- 172.21.15.83:0/4010785744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c017070 con 0x7f8c480b34f0 2024-04-25T00:11:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c42000640 1 -- 172.21.15.83:0/4010785744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c3c00c940 con 0x7f8c480b34f0 2024-04-25T00:11:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4f200640 1 -- 172.21.15.83:0/4010785744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c48162830 con 0x7f8c480b34f0 2024-04-25T00:11:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c42000640 1 -- 172.21.15.83:0/4010785744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00cd10 con 0x7f8c480b34f0 2024-04-25T00:11:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4f200640 1 -- 172.21.15.83:0/4010785744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c481401e0 con 0x7f8c480b34f0 2024-04-25T00:11:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c4f200640 1 -- 172.21.15.83:0/4010785744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c4804f510 con 0x7f8c480b34f0 2024-04-25T00:11:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.415+0000 7f8c42000640 1 -- 172.21.15.83:0/4010785744 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8c3c01a070 con 0x7f8c480b34f0 2024-04-25T00:11:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.419+0000 7f8c42000640 1 --2- 172.21.15.83:0/4010785744 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c2c0678c0 0x7f8c2c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.419+0000 7f8c42000640 1 -- 172.21.15.83:0/4010785744 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8c3c085f10 con 0x7f8c480b34f0 2024-04-25T00:11:34.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.419+0000 7f8c4d400640 1 --2- 172.21.15.83:0/4010785744 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c2c0678c0 0x7f8c2c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.419+0000 7f8c4d400640 1 --2- 172.21.15.83:0/4010785744 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c2c0678c0 0x7f8c2c069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8c380af000 tx=0x7f8c380a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:34.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.423+0000 7f8c42000640 1 -- 172.21.15.83:0/4010785744 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8c3c056710 con 0x7f8c480b34f0 2024-04-25T00:11:34.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.611+0000 7f8c4f200640 1 -- 172.21.15.83:0/4010785744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c4815db60 con 0x7f8c480b34f0 2024-04-25T00:11:34.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.611+0000 7f8c42000640 1 -- 172.21.15.83:0/4010785744 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7f8c3c056010 con 0x7f8c480b34f0 2024-04-25T00:11:34.617 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:34.617 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/1335208265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1335208265},{"type":"v1","addr":"172.21.15.86:6834","nonce":1335208265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/911805587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":911805587},{"type":"v1","addr":"172.21.15.83:6836","nonce":911805587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2309157687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2309157687},{"type":"v1","addr":"172.21.15.83:6837","nonce":2309157687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:11:06.200989+0000","modified":"2024-04-25T00:11:23.462521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1603954817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":1603954817},{"type":"v1","addr":"172.21.15.86:6835","nonce":1603954817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T00:11:34.617 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2024-04-25T00:11:34.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.615+0000 7f8c4f200640 1 -- 172.21.15.83:0/4010785744 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c2c0678c0 msgr2=0x7f8c2c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.615+0000 7f8c4f200640 1 --2- 172.21.15.83:0/4010785744 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c2c0678c0 0x7f8c2c069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8c380af000 tx=0x7f8c380a7000 comp rx=0 tx=0).stop 2024-04-25T00:11:34.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.615+0000 7f8c4f200640 1 -- 172.21.15.83:0/4010785744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c480b34f0 msgr2=0x7f8c48159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.615+0000 7f8c4f200640 1 --2- 172.21.15.83:0/4010785744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c480b34f0 0x7f8c48159f20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00e870 tx=0x7f8c3c00ed30 comp rx=0 tx=0).stop 2024-04-25T00:11:34.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.615+0000 7f8c4f200640 1 -- 172.21.15.83:0/4010785744 shutdown_connections 2024-04-25T00:11:34.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.615+0000 7f8c4f200640 1 --2- 172.21.15.83:0/4010785744 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c2c0678c0 0x7f8c2c069d80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.615+0000 7f8c4f200640 1 --2- 172.21.15.83:0/4010785744 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c48144470 0x7f8c481596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.615+0000 7f8c4f200640 1 --2- 172.21.15.83:0/4010785744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c480b3970 0x7f8c4813f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.615+0000 7f8c4f200640 1 --2- 172.21.15.83:0/4010785744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c480b34f0 0x7f8c48159f20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.615+0000 7f8c4f200640 1 -- 172.21.15.83:0/4010785744 >> 172.21.15.83:0/4010785744 conn(0x7f8c4806b3b0 msgr2=0x7f8c48142900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:34.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.615+0000 7f8c4f200640 1 -- 172.21.15.83:0/4010785744 shutdown_connections 2024-04-25T00:11:34.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.615+0000 7f8c4f200640 1 -- 172.21.15.83:0/4010785744 wait complete. 2024-04-25T00:11:34.639 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47fa800640 1 Processor -- start 2024-04-25T00:11:34.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47fa800640 1 -- start start 2024-04-25T00:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47fa800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47f40cf420 0x7f47f4071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47fa800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f47f406d230 0x7f47f40b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47fa800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47f4072750 con 0x7f47f41635e0 2024-04-25T00:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47fa800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f47f4071b40 con 0x7f47f406d230 2024-04-25T00:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47fa800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f47f4071e20 con 0x7f47f40cf420 2024-04-25T00:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47f3e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f47f406d230 0x7f47f40b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47f3e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f47f406d230 0x7f47f40b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45966/0 (socket says 172.21.15.83:45966) 2024-04-25T00:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47f3e00640 1 -- 172.21.15.83:0/58123677 learned_addr learned my addr 172.21.15.83:0/58123677 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47f3400640 1 --2- 172.21.15.83:0/58123677 >> v2:172.21.15.86:3300/0 conn(0x7f47f40cf420 0x7f47f4071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47f3e00640 1 -- 172.21.15.83:0/58123677 >> v2:172.21.15.86:3300/0 conn(0x7f47f40cf420 msgr2=0x7f47f4071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47f3e00640 1 --2- 172.21.15.83:0/58123677 >> v2:172.21.15.86:3300/0 conn(0x7f47f40cf420 0x7f47f4071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47f3e00640 1 -- 172.21.15.83:0/58123677 >> v1:172.21.15.86:6789/0 conn(0x7f47f41635e0 legacy=0x7f47f406cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.771+0000 7f47f3e00640 1 -- 172.21.15.83:0/58123677 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f4072100 con 0x7f47f406d230 2024-04-25T00:11:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.775+0000 7f47f3e00640 1 --2- 172.21.15.83:0/58123677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47f406d230 0x7f47f40b2e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f47f4073620 tx=0x7f47ec01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ae342b3d10626 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.775+0000 7f47f2000640 1 -- 172.21.15.83:0/58123677 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0a5790 con 0x7f47f406d230 2024-04-25T00:11:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.775+0000 7f47f2000640 1 -- 172.21.15.83:0/58123677 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec0a5930 con 0x7f47f406d230 2024-04-25T00:11:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.775+0000 7f47f2000640 1 -- 172.21.15.83:0/58123677 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0a5d00 con 0x7f47f406d230 2024-04-25T00:11:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.775+0000 7f47fa800640 1 -- 172.21.15.83:0/58123677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47f406d230 msgr2=0x7f47f40b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.775+0000 7f47fa800640 1 --2- 172.21.15.83:0/58123677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47f406d230 0x7f47f40b2e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f47f4073620 tx=0x7f47ec01c9c0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.775+0000 7f47fa800640 1 -- 172.21.15.83:0/58123677 shutdown_connections 2024-04-25T00:11:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.775+0000 7f47fa800640 1 --2- 172.21.15.83:0/58123677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47f406d230 0x7f47f40b2e10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.775+0000 7f47fa800640 1 --2- 172.21.15.83:0/58123677 >> v2:172.21.15.86:3300/0 conn(0x7f47f40cf420 0x7f47f4071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.775+0000 7f47fa800640 1 -- 172.21.15.83:0/58123677 >> 172.21.15.83:0/58123677 conn(0x7f47f40b1830 msgr2=0x7f47f40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.775+0000 7f47fa800640 1 -- 172.21.15.83:0/58123677 shutdown_connections 2024-04-25T00:11:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.775+0000 7f47fa800640 1 -- 172.21.15.83:0/58123677 wait complete. 2024-04-25T00:11:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.775+0000 7f47fa800640 1 Processor -- start 2024-04-25T00:11:34.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.775+0000 7f47fa800640 1 -- start start 2024-04-25T00:11:34.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47fa800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f406d230 0x7f47f4154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47fa800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47f40cf420 0x7f47f4156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47fa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47f41635e0 0x7f47f4152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47fa800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f47f40b3ce0 con 0x7f47f41635e0 2024-04-25T00:11:34.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47fa800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47f40b3b60 con 0x7f47f406d230 2024-04-25T00:11:34.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47fa800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f47f40b3e60 con 0x7f47f40cf420 2024-04-25T00:11:34.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47f41635e0 0x7f47f4152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47f41635e0 0x7f47f4152d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45976/0 (socket says 172.21.15.83:45976) 2024-04-25T00:11:34.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47f3e00640 1 -- 172.21.15.83:0/328807907 learned_addr learned my addr 172.21.15.83:0/328807907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:34.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47f3400640 1 --2- 172.21.15.83:0/328807907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f406d230 0x7f47f4154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47f2a00640 1 --2- 172.21.15.83:0/328807907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47f40cf420 0x7f47f4156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47f3e00640 1 -- 172.21.15.83:0/328807907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47f40cf420 msgr2=0x7f47f4156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47f3e00640 1 --2- 172.21.15.83:0/328807907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47f40cf420 0x7f47f4156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47f3e00640 1 -- 172.21.15.83:0/328807907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f406d230 msgr2=0x7f47f4154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47f3e00640 1 --2- 172.21.15.83:0/328807907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f406d230 0x7f47f4154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47f3e00640 1 -- 172.21.15.83:0/328807907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f4153a70 con 0x7f47f41635e0 2024-04-25T00:11:34.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47f3e00640 1 --2- 172.21.15.83:0/328807907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47f41635e0 0x7f47f4152d60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0195d0 tx=0x7f47ec019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:34.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47eb400640 1 -- 172.21.15.83:0/328807907 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0b8670 con 0x7f47f41635e0 2024-04-25T00:11:34.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47eb400640 1 -- 172.21.15.83:0/328807907 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec0be070 con 0x7f47f41635e0 2024-04-25T00:11:34.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47fa800640 1 -- 172.21.15.83:0/328807907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f4153d50 con 0x7f47f41635e0 2024-04-25T00:11:34.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47eb400640 1 -- 172.21.15.83:0/328807907 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0b8a70 con 0x7f47f41635e0 2024-04-25T00:11:34.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47fa800640 1 -- 172.21.15.83:0/328807907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f4154310 con 0x7f47f41635e0 2024-04-25T00:11:34.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.779+0000 7f47fa800640 1 -- 172.21.15.83:0/328807907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f40737f0 con 0x7f47f41635e0 2024-04-25T00:11:34.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.783+0000 7f47eb400640 1 -- 172.21.15.83:0/328807907 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f47ec0b8c10 con 0x7f47f41635e0 2024-04-25T00:11:34.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.783+0000 7f47eb400640 1 --2- 172.21.15.83:0/328807907 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47d40678f0 0x7f47d4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.783+0000 7f47eb400640 1 -- 172.21.15.83:0/328807907 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f47ec120d10 con 0x7f47f41635e0 2024-04-25T00:11:34.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.783+0000 7f47f3400640 1 --2- 172.21.15.83:0/328807907 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47d40678f0 0x7f47d4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.783+0000 7f47f3400640 1 --2- 172.21.15.83:0/328807907 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47d40678f0 0x7f47d4069db0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47f41591c0 tx=0x7f47e4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:34.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.787+0000 7f47eb400640 1 -- 172.21.15.83:0/328807907 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f47ec126020 con 0x7f47f41635e0 2024-04-25T00:11:34.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.943+0000 7f47fa800640 1 -- 172.21.15.83:0/328807907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47f4067af0 con 0x7f47f41635e0 2024-04-25T00:11:34.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.943+0000 7f47eb400640 1 -- 172.21.15.83:0/328807907 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7f47ec0165e0 con 0x7f47f41635e0 2024-04-25T00:11:34.948 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:34.949 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:34.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.947+0000 7f47fa800640 1 -- 172.21.15.83:0/328807907 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47d40678f0 msgr2=0x7f47d4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.947+0000 7f47fa800640 1 --2- 172.21.15.83:0/328807907 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47d40678f0 0x7f47d4069db0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47f41591c0 tx=0x7f47e4009190 comp rx=0 tx=0).stop 2024-04-25T00:11:34.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.947+0000 7f47fa800640 1 -- 172.21.15.83:0/328807907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47f41635e0 msgr2=0x7f47f4152d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.947+0000 7f47fa800640 1 --2- 172.21.15.83:0/328807907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47f41635e0 0x7f47f4152d60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0195d0 tx=0x7f47ec019760 comp rx=0 tx=0).stop 2024-04-25T00:11:34.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.947+0000 7f47fa800640 1 -- 172.21.15.83:0/328807907 shutdown_connections 2024-04-25T00:11:34.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.947+0000 7f47fa800640 1 --2- 172.21.15.83:0/328807907 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47d40678f0 0x7f47d4069db0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.947+0000 7f47fa800640 1 --2- 172.21.15.83:0/328807907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47f40cf420 0x7f47f4156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.947+0000 7f47fa800640 1 --2- 172.21.15.83:0/328807907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47f41635e0 0x7f47f4152d60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.947+0000 7f47fa800640 1 --2- 172.21.15.83:0/328807907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f406d230 0x7f47f4154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.947+0000 7f47fa800640 1 -- 172.21.15.83:0/328807907 >> 172.21.15.83:0/328807907 conn(0x7f47f40b1830 msgr2=0x7f47f40b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:34.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.947+0000 7f47fa800640 1 -- 172.21.15.83:0/328807907 shutdown_connections 2024-04-25T00:11:34.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:34.947+0000 7f47fa800640 1 -- 172.21.15.83:0/328807907 wait complete. 2024-04-25T00:11:34.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:11:35.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f501e200640 1 Processor -- start 2024-04-25T00:11:35.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f501e200640 1 -- start start 2024-04-25T00:11:35.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f501e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f501815bf70 0x7f50180cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f501e200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50180cfd80 0x7f50180d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f501e200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50180d0770 0x7f50180b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f501e200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f50180d0bc0 con 0x7f50180d0770 2024-04-25T00:11:35.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f501e200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f50180cc9b0 con 0x7f501815bf70 2024-04-25T00:11:35.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f501e200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f50180ccc90 con 0x7f50180cfd80 2024-04-25T00:11:35.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f5017e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50180d0770 0x7f50180b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f5017e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50180d0770 0x7f50180b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45980/0 (socket says 172.21.15.83:45980) 2024-04-25T00:11:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f5017e00640 1 -- 172.21.15.83:0/493709419 learned_addr learned my addr 172.21.15.83:0/493709419 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f5017400640 1 --2- 172.21.15.83:0/493709419 >> v2:172.21.15.86:3300/0 conn(0x7f501815bf70 0x7f50180cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f500ea00640 1 --2- 172.21.15.83:0/493709419 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50180cfd80 0x7f50180d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f5017e00640 1 -- 172.21.15.83:0/493709419 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50180cfd80 msgr2=0x7f50180d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f5017e00640 1 --2- 172.21.15.83:0/493709419 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50180cfd80 0x7f50180d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f5017e00640 1 -- 172.21.15.83:0/493709419 >> v2:172.21.15.86:3300/0 conn(0x7f501815bf70 msgr2=0x7f50180cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f5017e00640 1 --2- 172.21.15.83:0/493709419 >> v2:172.21.15.86:3300/0 conn(0x7f501815bf70 0x7f50180cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f5017e00640 1 -- 172.21.15.83:0/493709419 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50180ccf70 con 0x7f50180d0770 2024-04-25T00:11:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.103+0000 7f5017400640 1 --2- 172.21.15.83:0/493709419 >> v2:172.21.15.86:3300/0 conn(0x7f501815bf70 0x7f50180cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.107+0000 7f5017e00640 1 --2- 172.21.15.83:0/493709419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50180d0770 0x7f50180b4ac0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5018072520 tx=0x7f501001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727b2b099ea20aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.107+0000 7f5016a00640 1 -- 172.21.15.83:0/493709419 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50100a5840 con 0x7f50180d0770 2024-04-25T00:11:35.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.107+0000 7f5016a00640 1 -- 172.21.15.83:0/493709419 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50100a59e0 con 0x7f50180d0770 2024-04-25T00:11:35.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.107+0000 7f5016a00640 1 -- 172.21.15.83:0/493709419 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50100a5db0 con 0x7f50180d0770 2024-04-25T00:11:35.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.107+0000 7f501e200640 1 -- 172.21.15.83:0/493709419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50180d0770 msgr2=0x7f50180b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.107+0000 7f501e200640 1 --2- 172.21.15.83:0/493709419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50180d0770 0x7f50180b4ac0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5018072520 tx=0x7f501001c9c0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.107+0000 7f501e200640 1 -- 172.21.15.83:0/493709419 shutdown_connections 2024-04-25T00:11:35.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.107+0000 7f501e200640 1 --2- 172.21.15.83:0/493709419 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50180cfd80 0x7f50180d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.107+0000 7f501e200640 1 --2- 172.21.15.83:0/493709419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50180d0770 0x7f50180b4ac0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.107+0000 7f501e200640 1 --2- 172.21.15.83:0/493709419 >> v2:172.21.15.86:3300/0 conn(0x7f501815bf70 0x7f50180cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.107+0000 7f501e200640 1 -- 172.21.15.83:0/493709419 >> 172.21.15.83:0/493709419 conn(0x7f50180d1c70 msgr2=0x7f50180d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.107+0000 7f501e200640 1 -- 172.21.15.83:0/493709419 shutdown_connections 2024-04-25T00:11:35.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.107+0000 7f501e200640 1 -- 172.21.15.83:0/493709419 wait complete. 2024-04-25T00:11:35.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f501e200640 1 Processor -- start 2024-04-25T00:11:35.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f501e200640 1 -- start start 2024-04-25T00:11:35.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f501e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50180cfd80 0x7f501815a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f501e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50180d0770 0x7f50180cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f501e200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f501815bf70 0x7f50180cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f501e200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50180cda50 con 0x7f50180cfd80 2024-04-25T00:11:35.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f501e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f501815c350 con 0x7f50180d0770 2024-04-25T00:11:35.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f501e200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f50180c8d40 con 0x7f501815bf70 2024-04-25T00:11:35.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f5017400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50180cfd80 0x7f501815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f5017400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50180cfd80 0x7f501815a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45994/0 (socket says 172.21.15.83:45994) 2024-04-25T00:11:35.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f5017400640 1 -- 172.21.15.83:0/1529502049 learned_addr learned my addr 172.21.15.83:0/1529502049 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:35.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f5017e00640 1 --2- 172.21.15.83:0/1529502049 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f501815bf70 0x7f50180cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f500ea00640 1 --2- 172.21.15.83:0/1529502049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50180d0770 0x7f50180cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f5017400640 1 -- 172.21.15.83:0/1529502049 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f501815bf70 msgr2=0x7f50180cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f5017400640 1 --2- 172.21.15.83:0/1529502049 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f501815bf70 0x7f50180cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f5017400640 1 -- 172.21.15.83:0/1529502049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50180d0770 msgr2=0x7f50180cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f5017400640 1 --2- 172.21.15.83:0/1529502049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50180d0770 0x7f50180cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f5017400640 1 -- 172.21.15.83:0/1529502049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5018148bc0 con 0x7f50180cfd80 2024-04-25T00:11:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f5017400640 1 --2- 172.21.15.83:0/1529502049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50180cfd80 0x7f501815a350 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f500800e840 tx=0x7f500800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f500fe00640 1 -- 172.21.15.83:0/1529502049 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008017070 con 0x7f50180cfd80 2024-04-25T00:11:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f500fe00640 1 -- 172.21.15.83:0/1529502049 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500800c8d0 con 0x7f50180cfd80 2024-04-25T00:11:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f500fe00640 1 -- 172.21.15.83:0/1529502049 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500800f040 con 0x7f50180cfd80 2024-04-25T00:11:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f501e200640 1 -- 172.21.15.83:0/1529502049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5018145fe0 con 0x7f50180cfd80 2024-04-25T00:11:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f501e200640 1 -- 172.21.15.83:0/1529502049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f501815a890 con 0x7f50180cfd80 2024-04-25T00:11:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.111+0000 7f501e200640 1 -- 172.21.15.83:0/1529502049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f501809fd60 con 0x7f50180cfd80 2024-04-25T00:11:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.115+0000 7f500fe00640 1 -- 172.21.15.83:0/1529502049 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f500801a070 con 0x7f50180cfd80 2024-04-25T00:11:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.115+0000 7f500fe00640 1 --2- 172.21.15.83:0/1529502049 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ff40678c0 0x7f4ff4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.115+0000 7f500fe00640 1 -- 172.21.15.83:0/1529502049 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5008085c20 con 0x7f50180cfd80 2024-04-25T00:11:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.115+0000 7f500ea00640 1 --2- 172.21.15.83:0/1529502049 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ff40678c0 0x7f4ff4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.115+0000 7f500ea00640 1 --2- 172.21.15.83:0/1529502049 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ff40678c0 0x7f4ff4069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5018070000 tx=0x7f5000008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.119+0000 7f500fe00640 1 -- 172.21.15.83:0/1529502049 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5008055d30 con 0x7f50180cfd80 2024-04-25T00:11:35.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.307+0000 7f501e200640 1 -- 172.21.15.83:0/1529502049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f501804f5f0 con 0x7f50180cfd80 2024-04-25T00:11:35.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.307+0000 7f500fe00640 1 -- 172.21.15.83:0/1529502049 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7f500805a030 con 0x7f50180cfd80 2024-04-25T00:11:35.311 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:35.311 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/1335208265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1335208265},{"type":"v1","addr":"172.21.15.86:6834","nonce":1335208265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/911805587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":911805587},{"type":"v1","addr":"172.21.15.83:6836","nonce":911805587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2309157687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2309157687},{"type":"v1","addr":"172.21.15.83:6837","nonce":2309157687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:11:06.200989+0000","modified":"2024-04-25T00:11:23.462521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1603954817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":1603954817},{"type":"v1","addr":"172.21.15.86:6835","nonce":1603954817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T00:11:35.311 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2024-04-25T00:11:35.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.307+0000 7f501e200640 1 -- 172.21.15.83:0/1529502049 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ff40678c0 msgr2=0x7f4ff4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.307+0000 7f501e200640 1 --2- 172.21.15.83:0/1529502049 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ff40678c0 0x7f4ff4069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5018070000 tx=0x7f5000008040 comp rx=0 tx=0).stop 2024-04-25T00:11:35.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.307+0000 7f501e200640 1 -- 172.21.15.83:0/1529502049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50180cfd80 msgr2=0x7f501815a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.307+0000 7f501e200640 1 --2- 172.21.15.83:0/1529502049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50180cfd80 0x7f501815a350 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f500800e840 tx=0x7f500800ed00 comp rx=0 tx=0).stop 2024-04-25T00:11:35.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.307+0000 7f501e200640 1 -- 172.21.15.83:0/1529502049 shutdown_connections 2024-04-25T00:11:35.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.307+0000 7f501e200640 1 --2- 172.21.15.83:0/1529502049 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ff40678c0 0x7f4ff4069d80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.307+0000 7f501e200640 1 --2- 172.21.15.83:0/1529502049 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f501815bf70 0x7f50180cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.307+0000 7f501e200640 1 --2- 172.21.15.83:0/1529502049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50180d0770 0x7f50180cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.307+0000 7f501e200640 1 --2- 172.21.15.83:0/1529502049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50180cfd80 0x7f501815a350 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.307+0000 7f501e200640 1 -- 172.21.15.83:0/1529502049 >> 172.21.15.83:0/1529502049 conn(0x7f50180d1c70 msgr2=0x7f50180d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.307+0000 7f501e200640 1 -- 172.21.15.83:0/1529502049 shutdown_connections 2024-04-25T00:11:35.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.307+0000 7f501e200640 1 -- 172.21.15.83:0/1529502049 wait complete. 2024-04-25T00:11:35.733 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:11:06.200989+0000', 'modified': '2024-04-25T00:11:23.462521+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6835/1603954817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6833', 'nonce': 1603954817}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 1603954817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4285, 'qdb_cluster': [4285]} 2024-04-25T00:11:35.733 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:11:35.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:11:35.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.867+0000 7f1764200640 1 Processor -- start 2024-04-25T00:11:35.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.867+0000 7f1764200640 1 -- start start 2024-04-25T00:11:35.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1764200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f175c1446e0 0x7f175c0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1764200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f175c0cb210 con 0x7f175c06ee20 2024-04-25T00:11:35.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1764200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f175c0cb4f0 con 0x7f175c15f350 2024-04-25T00:11:35.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1764200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f175c0715b0 con 0x7f175c1446e0 2024-04-25T00:11:35.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1762200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f175c06ee20 0x7f175c071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48286/0 (socket says 172.21.15.83:48286) 2024-04-25T00:11:35.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1762200640 1 -- 172.21.15.83:0/1154820368 learned_addr learned my addr 172.21.15.83:0/1154820368 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:35.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1761800640 1 --2- 172.21.15.83:0/1154820368 >> v2:172.21.15.86:3300/0 conn(0x7f175c1446e0 0x7f175c0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 -- 172.21.15.83:0/1154820368 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2849189933 0 0) 0x7f175c0cb210 con 0x7f175c06ee20 2024-04-25T00:11:35.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 -- 172.21.15.83:0/1154820368 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1744003530 con 0x7f175c06ee20 2024-04-25T00:11:35.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 -- 172.21.15.83:0/1154820368 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 648061160 0 0) 0x7f1744003530 con 0x7f175c06ee20 2024-04-25T00:11:35.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 -- 172.21.15.83:0/1154820368 >> v2:172.21.15.86:3300/0 conn(0x7f175c1446e0 msgr2=0x7f175c0cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 --2- 172.21.15.83:0/1154820368 >> v2:172.21.15.86:3300/0 conn(0x7f175c1446e0 0x7f175c0cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 -- 172.21.15.83:0/1154820368 >> v1:172.21.15.86:6789/0 conn(0x7f175c15f350 legacy=0x7f175c06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 -- 172.21.15.83:0/1154820368 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175c0717e0 con 0x7f175c06ee20 2024-04-25T00:11:35.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 -- 172.21.15.83:0/1154820368 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f1754001a00 con 0x7f175c06ee20 2024-04-25T00:11:35.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 -- 172.21.15.83:0/1154820368 >> v1:172.21.15.83:6789/0 conn(0x7f175c06ee20 legacy=0x7f175c071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 --2- 172.21.15.83:0/1154820368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17440039c0 0x7f1744003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 --2- 172.21.15.83:0/1154820368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1744004810 0x7f1744004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 --2- 172.21.15.83:0/1154820368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17440051d0 0x7f174400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 -- 172.21.15.83:0/1154820368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f174400c040 con 0x7f1744004810 2024-04-25T00:11:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 -- 172.21.15.83:0/1154820368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f174400c1f0 con 0x7f17440039c0 2024-04-25T00:11:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f175be00640 1 -- 172.21.15.83:0/1154820368 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f174400c3f0 con 0x7f17440051d0 2024-04-25T00:11:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1760e00640 1 --2- 172.21.15.83:0/1154820368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1744004810 0x7f1744004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1761800640 1 --2- 172.21.15.83:0/1154820368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17440039c0 0x7f1744003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1762200640 1 --2- 172.21.15.83:0/1154820368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17440051d0 0x7f174400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1760e00640 1 -- 172.21.15.83:0/1154820368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17440051d0 msgr2=0x7f174400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1760e00640 1 --2- 172.21.15.83:0/1154820368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17440051d0 0x7f174400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1760e00640 1 -- 172.21.15.83:0/1154820368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17440039c0 msgr2=0x7f1744003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1760e00640 1 --2- 172.21.15.83:0/1154820368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17440039c0 0x7f1744003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1760e00640 1 -- 172.21.15.83:0/1154820368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1744003530 con 0x7f1744004810 2024-04-25T00:11:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1762200640 1 --2- 172.21.15.83:0/1154820368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17440051d0 0x7f174400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.871+0000 7f1762200640 1 -- 172.21.15.83:0/1154820368 reap_dead start 2024-04-25T00:11:35.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.875+0000 7f1760e00640 1 --2- 172.21.15.83:0/1154820368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1744004810 0x7f1744004c90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1748001780 tx=0x7f174801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7e0beaa68a29d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.875+0000 7f175be00640 1 -- 172.21.15.83:0/1154820368 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17480013a0 con 0x7f1744004810 2024-04-25T00:11:35.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.875+0000 7f175be00640 1 -- 172.21.15.83:0/1154820368 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17480052f0 con 0x7f1744004810 2024-04-25T00:11:35.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.875+0000 7f175be00640 1 -- 172.21.15.83:0/1154820368 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748003840 con 0x7f1744004810 2024-04-25T00:11:35.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.875+0000 7f1764200640 1 -- 172.21.15.83:0/1154820368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1744004810 msgr2=0x7f1744004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.875+0000 7f1764200640 1 --2- 172.21.15.83:0/1154820368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1744004810 0x7f1744004c90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1748001780 tx=0x7f174801c400 comp rx=0 tx=0).stop 2024-04-25T00:11:35.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.875+0000 7f1764200640 1 -- 172.21.15.83:0/1154820368 shutdown_connections 2024-04-25T00:11:35.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.875+0000 7f1764200640 1 --2- 172.21.15.83:0/1154820368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1744004810 0x7f1744004c90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.875+0000 7f1764200640 1 -- 172.21.15.83:0/1154820368 >> 172.21.15.83:0/1154820368 conn(0x7f175c0cfa60 msgr2=0x7f175c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.875+0000 7f1764200640 1 -- 172.21.15.83:0/1154820368 shutdown_connections 2024-04-25T00:11:35.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.875+0000 7f1764200640 1 -- 172.21.15.83:0/1154820368 wait complete. 2024-04-25T00:11:35.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.875+0000 7f1764200640 1 Processor -- start 2024-04-25T00:11:35.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.875+0000 7f1764200640 1 -- start start 2024-04-25T00:11:35.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1764200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1744004810 0x7f175c141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1764200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f175c1424a0 0x7f175c15cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1764200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f175c0c8db0 0x7f175c0c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1764200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f175c071b50 con 0x7f175c0c8db0 2024-04-25T00:11:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1764200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f175c144ac0 con 0x7f175c1424a0 2024-04-25T00:11:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1764200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f175c0cd630 con 0x7f1744004810 2024-04-25T00:11:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1762200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f175c0c8db0 0x7f175c0c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1762200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f175c0c8db0 0x7f175c0c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46026/0 (socket says 172.21.15.83:46026) 2024-04-25T00:11:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1762200640 1 -- 172.21.15.83:0/2364659002 learned_addr learned my addr 172.21.15.83:0/2364659002 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1760e00640 1 --2- 172.21.15.83:0/2364659002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f175c1424a0 0x7f175c15cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1761800640 1 --2- 172.21.15.83:0/2364659002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1744004810 0x7f175c141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1762200640 1 -- 172.21.15.83:0/2364659002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1744004810 msgr2=0x7f175c141c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1762200640 1 --2- 172.21.15.83:0/2364659002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1744004810 0x7f175c141c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1762200640 1 -- 172.21.15.83:0/2364659002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f175c1424a0 msgr2=0x7f175c15cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1762200640 1 --2- 172.21.15.83:0/2364659002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f175c1424a0 0x7f175c15cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1762200640 1 -- 172.21.15.83:0/2364659002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175c15d5a0 con 0x7f175c0c8db0 2024-04-25T00:11:35.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1762200640 1 --2- 172.21.15.83:0/2364659002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f175c0c8db0 0x7f175c0c9210 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f175c0ca040 tx=0x7f1754011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1759600640 1 -- 172.21.15.83:0/2364659002 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175401c070 con 0x7f175c0c8db0 2024-04-25T00:11:35.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1759600640 1 -- 172.21.15.83:0/2364659002 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17540137a0 con 0x7f175c0c8db0 2024-04-25T00:11:35.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1759600640 1 -- 172.21.15.83:0/2364659002 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754010040 con 0x7f175c0c8db0 2024-04-25T00:11:35.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1764200640 1 -- 172.21.15.83:0/2364659002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f175c15d840 con 0x7f175c0c8db0 2024-04-25T00:11:35.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1764200640 1 -- 172.21.15.83:0/2364659002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f175c15de00 con 0x7f175c0c8db0 2024-04-25T00:11:35.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.879+0000 7f1759600640 1 -- 172.21.15.83:0/2364659002 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f175401f070 con 0x7f175c0c8db0 2024-04-25T00:11:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1759600640 1 --2- 172.21.15.83:0/2364659002 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f173c0678c0 0x7f173c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1759600640 1 -- 172.21.15.83:0/2364659002 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1754088700 con 0x7f175c0c8db0 2024-04-25T00:11:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 Processor -- start 2024-04-25T00:11:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1761800640 1 --2- 172.21.15.83:0/2364659002 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f173c0678c0 0x7f173c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 -- start start 2024-04-25T00:11:35.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1761800640 1 --2- 172.21.15.83:0/2364659002 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f173c0678c0 0x7f173c069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f175c0c99c0 tx=0x7f1750009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fdef0 0x55e6ba1fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fe810 0x55e6ba1b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba1b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e6ba0025e0 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e6b9f89c50 con 0x55e6ba1fe810 2024-04-25T00:11:35.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55e6b9ffe5a0 con 0x55e6ba1fdef0 2024-04-25T00:11:35.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1762200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba1b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1762200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba1b4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46042/0 (socket says 172.21.15.83:46042) 2024-04-25T00:11:35.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1762200640 1 -- 172.21.15.83:0/1408892403 learned_addr learned my addr 172.21.15.83:0/1408892403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1761800640 1 --2- 172.21.15.83:0/1408892403 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fdef0 0x55e6ba1fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1760e00640 1 --2- 172.21.15.83:0/1408892403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fe810 0x55e6ba1b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1762200640 1 -- 172.21.15.83:0/1408892403 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fdef0 msgr2=0x55e6ba1fe2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1762200640 1 --2- 172.21.15.83:0/1408892403 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fdef0 0x55e6ba1fe2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1762200640 1 -- 172.21.15.83:0/1408892403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fe810 msgr2=0x55e6ba1b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1762200640 1 --2- 172.21.15.83:0/1408892403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fe810 0x55e6ba1b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1762200640 1 -- 172.21.15.83:0/1408892403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6b9f87d10 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1762200640 1 --2- 172.21.15.83:0/1408892403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba1b4fa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1754060ea0 tx=0x7f175405cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462ccd32b5f576ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f174e000640 1 -- 172.21.15.83:0/1408892403 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175405f020 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f174e000640 1 -- 172.21.15.83:0/1408892403 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1754054f10 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f174e000640 1 -- 172.21.15.83:0/1408892403 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754057f10 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 -- 172.21.15.83:0/1408892403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 msgr2=0x55e6ba1b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 --2- 172.21.15.83:0/1408892403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba1b4fa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1754060ea0 tx=0x7f175405cba0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 -- 172.21.15.83:0/1408892403 shutdown_connections 2024-04-25T00:11:35.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 --2- 172.21.15.83:0/1408892403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba1b4fa0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 --2- 172.21.15.83:0/1408892403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fe810 0x55e6ba1b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 --2- 172.21.15.83:0/1408892403 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fdef0 0x55e6ba1fe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 -- 172.21.15.83:0/1408892403 >> 172.21.15.83:0/1408892403 conn(0x55e6ba1b0b70 msgr2=0x55e6ba1b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 -- 172.21.15.83:0/1408892403 shutdown_connections 2024-04-25T00:11:35.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.883+0000 7f1767651000 1 -- 172.21.15.83:0/1408892403 wait complete. 2024-04-25T00:11:35.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 1 -- start start 2024-04-25T00:11:35.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 1 Processor -- start 2024-04-25T00:11:35.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba1e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fdef0 0x55e6ba1e3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fe810 0x55e6ba1e39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e6b9fbc520 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e6ba006c10 con 0x55e6ba1fdef0 2024-04-25T00:11:35.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55e6ba0b4e80 con 0x55e6ba1fe810 2024-04-25T00:11:35.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 10 client.0 _ll_register_callbacks cb 0x55e6ba19f5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:11:35.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1761800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba1e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1761800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba1e2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46052/0 (socket says 172.21.15.83:46052) 2024-04-25T00:11:35.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1761800640 1 -- 172.21.15.83:0/1183040968 learned_addr learned my addr 172.21.15.83:0/1183040968 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:35.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1760e00640 1 --2- 172.21.15.83:0/1183040968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fdef0 0x55e6ba1e3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1762200640 1 --2- 172.21.15.83:0/1183040968 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fe810 0x55e6ba1e39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1761800640 1 -- 172.21.15.83:0/1183040968 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fe810 msgr2=0x55e6ba1e39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1761800640 1 --2- 172.21.15.83:0/1183040968 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fe810 0x55e6ba1e39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1761800640 1 -- 172.21.15.83:0/1183040968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fdef0 msgr2=0x55e6ba1e3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1761800640 1 --2- 172.21.15.83:0/1183040968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fdef0 0x55e6ba1e3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1761800640 1 -- 172.21.15.83:0/1183040968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6b9f9a7b0 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1762200640 1 --2- 172.21.15.83:0/1183040968 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fe810 0x55e6ba1e39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:35.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1761800640 1 --2- 172.21.15.83:0/1183040968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba1e2f20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1750003ba0 tx=0x7f17500046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7e317583cb260c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1742a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:11:35.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1742a00640 1 -- 172.21.15.83:0/1183040968 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175001b070 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1742a00640 1 -- 172.21.15.83:0/1183040968 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1750023bd0 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 1 -- 172.21.15.83:0/1183040968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1754054f10 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1742a00640 1 -- 172.21.15.83:0/1183040968 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f1750023dd0 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 10 client.4818 fetch_fsmap learned FSMap version 9 2024-04-25T00:11:35.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 1 -- 172.21.15.83:0/1183040968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55e6b9f89ea0 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1742a00640 1 -- 172.21.15.83:0/1183040968 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1754054f10 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1742a00640 1 -- 172.21.15.83:0/1183040968 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f1750034050 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 10 client.4818 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T00:11:35.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 10 client.4818 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:11:35.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 20 client.4818 populate_metadata read hostname 'smithi083' 2024-04-25T00:11:35.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 1 --2- 172.21.15.83:0/1183040968 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x55e6ba20df90 0x55e6ba210380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 4 client.4818 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:11:35.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1767651000 1 -- 172.21.15.83:0/1183040968 --> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6b9f9dab0 con 0x55e6ba20df90 2024-04-25T00:11:35.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.887+0000 7f1760e00640 1 --2- 172.21.15.83:0/1183040968 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x55e6ba20df90 0x55e6ba210380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1760e00640 1 --2- 172.21.15.83:0/1183040968 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x55e6ba20df90 0x55e6ba210380 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d3f55069cd2f1be server_cookie=ea43d8cdef91d7fb in_seq=0 out_seq=0 2024-04-25T00:11:35.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1742a00640 10 client.4818 ms_handle_connect on v2:172.21.15.86:6833/1603954817 2024-04-25T00:11:35.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1742a00640 1 -- 172.21.15.83:0/1183040968 <== mds.0 v2:172.21.15.86:6833/1603954817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6b9f9dab0 con 0x55e6ba20df90 2024-04-25T00:11:35.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1742a00640 10 client.4818 handle_command_reply: tid=0 2024-04-25T00:11:35.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1767651000 1 client.4818 shutdown 2024-04-25T00:11:35.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1767651000 1 -- 172.21.15.83:0/1183040968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 msgr2=0x55e6ba1e2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1767651000 1 --2- 172.21.15.83:0/1183040968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba1e2f20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1750003ba0 tx=0x7f17500046d0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1767651000 1 -- 172.21.15.83:0/1183040968 shutdown_connections 2024-04-25T00:11:35.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1767651000 1 --2- 172.21.15.83:0/1183040968 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x55e6ba20df90 0x55e6ba210380 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1767651000 1 --2- 172.21.15.83:0/1183040968 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fe810 0x55e6ba1e39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1767651000 1 --2- 172.21.15.83:0/1183040968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fdef0 0x55e6ba1e3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1742a00640 0 client.4818 ms_handle_reset on v2:172.21.15.86:6833/1603954817 2024-04-25T00:11:35.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1767651000 1 --2- 172.21.15.83:0/1183040968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba1e2f20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1767651000 1 -- 172.21.15.83:0/1183040968 >> 172.21.15.83:0/1183040968 conn(0x55e6ba1b0b70 msgr2=0x55e6ba1b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1767651000 1 -- 172.21.15.83:0/1183040968 shutdown_connections 2024-04-25T00:11:35.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1767651000 1 -- 172.21.15.83:0/1183040968 wait complete. 2024-04-25T00:11:35.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.891+0000 7f1767651000 20 client.4818 trim_cache size 0 max 16384 2024-04-25T00:11:35.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.907+0000 7f1767651000 1 Processor -- start 2024-04-25T00:11:35.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 -- start start 2024-04-25T00:11:35.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1b4bc0 0x55e6ba269500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fdef0 0x55e6ba269a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1fe810 0x55e6ba269f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f175c200f60 con 0x55e6ba1fe810 2024-04-25T00:11:35.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f175c0ad6f0 con 0x55e6ba1fdef0 2024-04-25T00:11:35.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f175c200910 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1762200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1fe810 0x55e6ba269f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1762200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1fe810 0x55e6ba269f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46054/0 (socket says 172.21.15.83:46054) 2024-04-25T00:11:35.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1762200640 1 -- 172.21.15.83:0/4284479198 learned_addr learned my addr 172.21.15.83:0/4284479198 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:35.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1762200640 1 -- 172.21.15.83:0/4284479198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1b4bc0 msgr2=0x55e6ba269500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1762200640 1 --2- 172.21.15.83:0/4284479198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1b4bc0 0x55e6ba269500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1762200640 1 -- 172.21.15.83:0/4284479198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fdef0 msgr2=0x55e6ba269a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1762200640 1 --2- 172.21.15.83:0/4284479198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fdef0 0x55e6ba269a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1762200640 1 -- 172.21.15.83:0/4284479198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6b9f8ae80 con 0x55e6ba1fe810 2024-04-25T00:11:35.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1762200640 1 --2- 172.21.15.83:0/4284479198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1fe810 0x55e6ba269f80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1754001d00 tx=0x7f17540587a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165794f246897e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f174d600640 1 -- 172.21.15.83:0/4284479198 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175405f100 con 0x55e6ba1fe810 2024-04-25T00:11:35.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f174d600640 1 -- 172.21.15.83:0/4284479198 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1754056ba0 con 0x55e6ba1fe810 2024-04-25T00:11:35.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f174d600640 1 -- 172.21.15.83:0/4284479198 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754057cf0 con 0x55e6ba1fe810 2024-04-25T00:11:35.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 -- 172.21.15.83:0/4284479198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1fe810 msgr2=0x55e6ba269f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 --2- 172.21.15.83:0/4284479198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1fe810 0x55e6ba269f80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1754001d00 tx=0x7f17540587a0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 -- 172.21.15.83:0/4284479198 shutdown_connections 2024-04-25T00:11:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 --2- 172.21.15.83:0/4284479198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1fe810 0x55e6ba269f80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 --2- 172.21.15.83:0/4284479198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fdef0 0x55e6ba269a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 --2- 172.21.15.83:0/4284479198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1b4bc0 0x55e6ba269500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 -- 172.21.15.83:0/4284479198 >> 172.21.15.83:0/4284479198 conn(0x55e6ba1b0b70 msgr2=0x55e6ba20b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 -- 172.21.15.83:0/4284479198 shutdown_connections 2024-04-25T00:11:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 -- 172.21.15.83:0/4284479198 wait complete. 2024-04-25T00:11:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 -- start start 2024-04-25T00:11:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.911+0000 7f1767651000 1 Processor -- start 2024-04-25T00:11:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba26e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fdef0 0x55e6ba26e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fe810 0x55e6ba210690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e6ba0014d0 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e6b9f8a640 con 0x55e6ba1fe810 2024-04-25T00:11:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55e6b9f9add0 con 0x55e6ba1fdef0 2024-04-25T00:11:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 10 client.0 _ll_register_callbacks cb 0x7f1750009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:11:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1761800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba26e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1761800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba26e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46066/0 (socket says 172.21.15.83:46066) 2024-04-25T00:11:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1761800640 1 -- 172.21.15.83:0/1449894053 learned_addr learned my addr 172.21.15.83:0/1449894053 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1762200640 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fe810 0x55e6ba210690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1760e00640 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fdef0 0x55e6ba26e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1761800640 1 -- 172.21.15.83:0/1449894053 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fdef0 msgr2=0x55e6ba26e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1761800640 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fdef0 0x55e6ba26e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1761800640 1 -- 172.21.15.83:0/1449894053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fe810 msgr2=0x55e6ba210690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1761800640 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fe810 0x55e6ba210690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1761800640 1 -- 172.21.15.83:0/1449894053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6b9fcf760 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1760e00640 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fdef0 0x55e6ba26e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1762200640 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fe810 0x55e6ba210690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1761800640 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba26e050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1750037e10 tx=0x7f1750004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b85b52e6089a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1742a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:11:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1742a00640 1 -- 172.21.15.83:0/1449894053 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175003b030 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1742a00640 1 -- 172.21.15.83:0/1449894053 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1750023bd0 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 -- 172.21.15.83:0/1449894053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1754056ba0 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1742a00640 1 -- 172.21.15.83:0/1449894053 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f1750044050 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 10 client.4824 fetch_fsmap learned FSMap version 9 2024-04-25T00:11:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 -- 172.21.15.83:0/1449894053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55e6ba05b9e0 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1742a00640 1 -- 172.21.15.83:0/1449894053 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1754056ba0 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1742a00640 1 -- 172.21.15.83:0/1449894053 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f1750047020 con 0x55e6ba1b4bc0 2024-04-25T00:11:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 10 client.4824 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T00:11:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 10 client.4824 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:11:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 20 client.4824 populate_metadata read hostname 'smithi083' 2024-04-25T00:11:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x55e6ba210bd0 0x55e6ba210fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 4 client.4824 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:11:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 -- 172.21.15.83:0/1449894053 --> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] -- command(tid 0: {"prefix": "status"}) -- 0x55e6ba00fb20 con 0x55e6ba210bd0 2024-04-25T00:11:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1760e00640 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x55e6ba210bd0 0x55e6ba210fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1760e00640 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x55e6ba210bd0 0x55e6ba210fb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303f3080c816bd14 server_cookie=2964713e5cfc6553 in_seq=0 out_seq=0 2024-04-25T00:11:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1742a00640 10 client.4824 ms_handle_connect on v2:172.21.15.86:6833/1603954817 2024-04-25T00:11:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1742a00640 1 -- 172.21.15.83:0/1449894053 <== mds.0 v2:172.21.15.86:6833/1603954817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55e6ba00fb20 con 0x55e6ba210bd0 2024-04-25T00:11:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1742a00640 10 client.4824 handle_command_reply: tid=0 2024-04-25T00:11:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 client.4824 shutdown 2024-04-25T00:11:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 -- 172.21.15.83:0/1449894053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 msgr2=0x55e6ba26e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba26e050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1750037e10 tx=0x7f1750004740 comp rx=0 tx=0).stop 2024-04-25T00:11:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 -- 172.21.15.83:0/1449894053 shutdown_connections 2024-04-25T00:11:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.86:6833/1603954817,v1:172.21.15.86:6835/1603954817] conn(0x55e6ba210bd0 0x55e6ba210fb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e6ba1fdef0 0x55e6ba26e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6ba1fe810 0x55e6ba210690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1742a00640 0 client.4824 ms_handle_reset on v2:172.21.15.86:6833/1603954817 2024-04-25T00:11:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 --2- 172.21.15.83:0/1449894053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6ba1b4bc0 0x55e6ba26e050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 -- 172.21.15.83:0/1449894053 >> 172.21.15.83:0/1449894053 conn(0x55e6ba1b0b70 msgr2=0x55e6ba1fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 -- 172.21.15.83:0/1449894053 shutdown_connections 2024-04-25T00:11:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.915+0000 7f1767651000 1 -- 172.21.15.83:0/1449894053 wait complete. 2024-04-25T00:11:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.919+0000 7f1767651000 20 client.4824 trim_cache size 0 max 16384 2024-04-25T00:11:35.926 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:11:35.926 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "d58ae83a-ad72-4e18-824a-86e779979c35", 2024-04-25T00:11:35.926 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-25T00:11:35.926 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4285, 2024-04-25T00:11:35.926 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T00:11:35.926 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T00:11:35.926 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T00:11:35.926 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 29.076713129000002, 2024-04-25T00:11:35.926 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 9, 2024-04-25T00:11:35.926 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 24, 2024-04-25T00:11:35.927 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T00:11:35.927 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 35.998306522999997 2024-04-25T00:11:35.927 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:11:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.919+0000 7f1764200640 1 -- 172.21.15.83:0/2364659002 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f173c0678c0 msgr2=0x7f173c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.919+0000 7f1764200640 1 --2- 172.21.15.83:0/2364659002 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f173c0678c0 0x7f173c069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f175c0c99c0 tx=0x7f1750009190 comp rx=0 tx=0).stop 2024-04-25T00:11:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.919+0000 7f1764200640 1 -- 172.21.15.83:0/2364659002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f175c0c8db0 msgr2=0x7f175c0c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.919+0000 7f1764200640 1 --2- 172.21.15.83:0/2364659002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f175c0c8db0 0x7f175c0c9210 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f175c0ca040 tx=0x7f1754011930 comp rx=0 tx=0).stop 2024-04-25T00:11:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.919+0000 7f1764200640 1 -- 172.21.15.83:0/2364659002 shutdown_connections 2024-04-25T00:11:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.919+0000 7f1764200640 1 --2- 172.21.15.83:0/2364659002 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f173c0678c0 0x7f173c069d80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.919+0000 7f1764200640 1 --2- 172.21.15.83:0/2364659002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f175c0c8db0 0x7f175c0c9210 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.919+0000 7f1764200640 1 --2- 172.21.15.83:0/2364659002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f175c1424a0 0x7f175c15cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.919+0000 7f1764200640 1 --2- 172.21.15.83:0/2364659002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1744004810 0x7f175c141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.919+0000 7f1764200640 1 -- 172.21.15.83:0/2364659002 >> 172.21.15.83:0/2364659002 conn(0x7f175c0cfa60 msgr2=0x7f175c15e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.919+0000 7f1764200640 1 -- 172.21.15.83:0/2364659002 shutdown_connections 2024-04-25T00:11:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:35.919+0000 7f1764200640 1 -- 172.21.15.83:0/2364659002 wait complete. 2024-04-25T00:11:35.940 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T00:11:36.240 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T00:11:36.240 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T00:11:36.241 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T00:11:36.241 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T00:11:36.370 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T00:11:36.370 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T00:11:36.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:36.371 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:11:36.380 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T00:11:36.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:36.381 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:11:36.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:36.451 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-25T00:11:36.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:36.620 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:11:36.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:36.629 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-25T00:11:36.694 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2024-04-25T00:11:36.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:11:36.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:36.697 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T00:11:36.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:36.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-25T00:11:36.774 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-04-25T00:11:36.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:11:36.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:36.777 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T00:11:36.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:36.830 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-25T00:11:36.901 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-04-25T00:11:36.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:11:36.904 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-25T00:11:36.904 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:11:36.904 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:11:36.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:11:37.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.035+0000 7f2e3a200640 1 Processor -- start 2024-04-25T00:11:37.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.035+0000 7f2e3a200640 1 -- start start 2024-04-25T00:11:37.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.035+0000 7f2e3a200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e341599f0 0x7f2e340cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:37.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.035+0000 7f2e3a200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e3415c340 0x7f2e3415c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:37.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.035+0000 7f2e3a200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e340ccc60 con 0x7f2e3415ccf0 2024-04-25T00:11:37.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.035+0000 7f2e3a200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2e34159dd0 con 0x7f2e3415c340 2024-04-25T00:11:37.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.035+0000 7f2e3a200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2e3406dba0 con 0x7f2e341599f0 2024-04-25T00:11:37.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.039+0000 7f2e2aa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e3415c340 0x7f2e3415c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:37.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.039+0000 7f2e2aa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e3415c340 0x7f2e3415c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46072/0 (socket says 172.21.15.83:46072) 2024-04-25T00:11:37.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.039+0000 7f2e2aa00640 1 -- 172.21.15.83:0/3355080948 learned_addr learned my addr 172.21.15.83:0/3355080948 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:37.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.039+0000 7f2e2aa00640 1 -- 172.21.15.83:0/3355080948 >> v2:172.21.15.86:3300/0 conn(0x7f2e341599f0 msgr2=0x7f2e340cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:37.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.039+0000 7f2e33400640 1 --2- 172.21.15.83:0/3355080948 >> v2:172.21.15.86:3300/0 conn(0x7f2e341599f0 0x7f2e340cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:37.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.039+0000 7f2e2aa00640 1 --2- 172.21.15.83:0/3355080948 >> v2:172.21.15.86:3300/0 conn(0x7f2e341599f0 0x7f2e340cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:37.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.039+0000 7f2e2aa00640 1 -- 172.21.15.83:0/3355080948 >> v1:172.21.15.86:6789/0 conn(0x7f2e3415ccf0 legacy=0x7f2e3415f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:37.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.039+0000 7f2e2aa00640 1 -- 172.21.15.83:0/3355080948 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3406de80 con 0x7f2e3415c340 2024-04-25T00:11:37.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.039+0000 7f2e33400640 1 --2- 172.21.15.83:0/3355080948 >> v2:172.21.15.86:3300/0 conn(0x7f2e341599f0 0x7f2e340cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.039+0000 7f2e2aa00640 1 --2- 172.21.15.83:0/3355080948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3415c340 0x7f2e3415c7b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009870 tx=0x7f2e1c01ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d347e0a2dd0c98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.039+0000 7f2e32a00640 1 -- 172.21.15.83:0/3355080948 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0b0070 con 0x7f2e3415c340 2024-04-25T00:11:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.039+0000 7f2e32a00640 1 -- 172.21.15.83:0/3355080948 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1c0ab600 con 0x7f2e3415c340 2024-04-25T00:11:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.039+0000 7f2e32a00640 1 -- 172.21.15.83:0/3355080948 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0ab9d0 con 0x7f2e3415c340 2024-04-25T00:11:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 -- 172.21.15.83:0/3355080948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3415c340 msgr2=0x7f2e3415c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 --2- 172.21.15.83:0/3355080948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3415c340 0x7f2e3415c7b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009870 tx=0x7f2e1c01ccd0 comp rx=0 tx=0).stop 2024-04-25T00:11:37.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 -- 172.21.15.83:0/3355080948 shutdown_connections 2024-04-25T00:11:37.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 --2- 172.21.15.83:0/3355080948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3415c340 0x7f2e3415c7b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:37.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 --2- 172.21.15.83:0/3355080948 >> v2:172.21.15.86:3300/0 conn(0x7f2e341599f0 0x7f2e340cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:37.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 -- 172.21.15.83:0/3355080948 >> 172.21.15.83:0/3355080948 conn(0x7f2e340b1830 msgr2=0x7f2e340b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:37.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 -- 172.21.15.83:0/3355080948 shutdown_connections 2024-04-25T00:11:37.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 -- 172.21.15.83:0/3355080948 wait complete. 2024-04-25T00:11:37.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 Processor -- start 2024-04-25T00:11:37.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 -- start start 2024-04-25T00:11:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e341599f0 0x7f2e3414a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e3415c340 0x7f2e341538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3415ccf0 0x7f2e34155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e340cf870 con 0x7f2e3415ccf0 2024-04-25T00:11:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e340cf6f0 con 0x7f2e341599f0 2024-04-25T00:11:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e3a200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2e34168cd0 con 0x7f2e3415c340 2024-04-25T00:11:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e33e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3415ccf0 0x7f2e34155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e33400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e341599f0 0x7f2e3414a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e33e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3415ccf0 0x7f2e34155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46076/0 (socket says 172.21.15.83:46076) 2024-04-25T00:11:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e33e00640 1 -- 172.21.15.83:0/938754342 learned_addr learned my addr 172.21.15.83:0/938754342 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:37.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e2aa00640 1 --2- 172.21.15.83:0/938754342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e3415c340 0x7f2e341538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:37.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e33e00640 1 -- 172.21.15.83:0/938754342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e3415c340 msgr2=0x7f2e341538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:37.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e33e00640 1 --2- 172.21.15.83:0/938754342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e3415c340 0x7f2e341538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:37.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e33e00640 1 -- 172.21.15.83:0/938754342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e341599f0 msgr2=0x7f2e3414a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:37.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e33e00640 1 --2- 172.21.15.83:0/938754342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e341599f0 0x7f2e3414a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:37.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.043+0000 7f2e33e00640 1 -- 172.21.15.83:0/938754342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3414a8c0 con 0x7f2e3415ccf0 2024-04-25T00:11:37.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.047+0000 7f2e33e00640 1 --2- 172.21.15.83:0/938754342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3415ccf0 0x7f2e34155f70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00b970 tx=0x7f2e2c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.047+0000 7f2e2be00640 1 -- 172.21.15.83:0/938754342 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c019070 con 0x7f2e3415ccf0 2024-04-25T00:11:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.047+0000 7f2e2be00640 1 -- 172.21.15.83:0/938754342 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2c00cb40 con 0x7f2e3415ccf0 2024-04-25T00:11:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.047+0000 7f2e2be00640 1 -- 172.21.15.83:0/938754342 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c014440 con 0x7f2e3415ccf0 2024-04-25T00:11:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.047+0000 7f2e3a200640 1 -- 172.21.15.83:0/938754342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e341faab0 con 0x7f2e3415ccf0 2024-04-25T00:11:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.047+0000 7f2e3a200640 1 -- 172.21.15.83:0/938754342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e341fb070 con 0x7f2e3415ccf0 2024-04-25T00:11:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.047+0000 7f2e3a200640 1 -- 172.21.15.83:0/938754342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e340737f0 con 0x7f2e3415ccf0 2024-04-25T00:11:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.047+0000 7f2e2be00640 1 -- 172.21.15.83:0/938754342 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2e2c01c070 con 0x7f2e3415ccf0 2024-04-25T00:11:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.047+0000 7f2e2be00640 1 --2- 172.21.15.83:0/938754342 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e100678c0 0x7f2e10069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.047+0000 7f2e2be00640 1 -- 172.21.15.83:0/938754342 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2e2c085b60 con 0x7f2e3415ccf0 2024-04-25T00:11:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.047+0000 7f2e33400640 1 --2- 172.21.15.83:0/938754342 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e100678c0 0x7f2e10069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.051+0000 7f2e33400640 1 --2- 172.21.15.83:0/938754342 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e100678c0 0x7f2e10069d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400ab60 tx=0x7f2e24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.051+0000 7f2e2be00640 1 -- 172.21.15.83:0/938754342 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2e2c055d00 con 0x7f2e3415ccf0 2024-04-25T00:11:37.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.243+0000 7f2e3a200640 1 -- 172.21.15.83:0/938754342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2e3404f5f0 con 0x7f2e3415ccf0 2024-04-25T00:11:37.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.243+0000 7f2e2be00640 1 -- 172.21.15.83:0/938754342 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f2e2c05a000 con 0x7f2e3415ccf0 2024-04-25T00:11:37.246 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T00:11:37.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.243+0000 7f2e3a200640 1 -- 172.21.15.83:0/938754342 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e100678c0 msgr2=0x7f2e10069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:37.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.243+0000 7f2e3a200640 1 --2- 172.21.15.83:0/938754342 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e100678c0 0x7f2e10069d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400ab60 tx=0x7f2e24008040 comp rx=0 tx=0).stop 2024-04-25T00:11:37.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.243+0000 7f2e3a200640 1 -- 172.21.15.83:0/938754342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3415ccf0 msgr2=0x7f2e34155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:37.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.243+0000 7f2e3a200640 1 --2- 172.21.15.83:0/938754342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3415ccf0 0x7f2e34155f70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00b970 tx=0x7f2e2c00be30 comp rx=0 tx=0).stop 2024-04-25T00:11:37.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.243+0000 7f2e3a200640 1 -- 172.21.15.83:0/938754342 shutdown_connections 2024-04-25T00:11:37.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.243+0000 7f2e3a200640 1 --2- 172.21.15.83:0/938754342 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e100678c0 0x7f2e10069d80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:37.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.243+0000 7f2e3a200640 1 --2- 172.21.15.83:0/938754342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e3415c340 0x7f2e341538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:37.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.243+0000 7f2e3a200640 1 --2- 172.21.15.83:0/938754342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e3415ccf0 0x7f2e34155f70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:37.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.243+0000 7f2e3a200640 1 --2- 172.21.15.83:0/938754342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e341599f0 0x7f2e3414a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:37.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.243+0000 7f2e3a200640 1 -- 172.21.15.83:0/938754342 >> 172.21.15.83:0/938754342 conn(0x7f2e340b1830 msgr2=0x7f2e340b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:37.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.243+0000 7f2e3a200640 1 -- 172.21.15.83:0/938754342 shutdown_connections 2024-04-25T00:11:37.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37.243+0000 7f2e3a200640 1 -- 172.21.15.83:0/938754342 wait complete. 2024-04-25T00:11:37.268 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:11:37.268 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:11:37.268 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-25T00:11:37.268 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:11:37.268 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:11:37.268 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:11:37.268 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:11:37.268 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:11:37.268 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:11:37.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:37.269 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-25T00:11:37.278 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:11:37.278 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:11:37.278 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:11:37.278 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:37.278 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-25T00:11:37.278 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:37.278 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:37.278 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:37.278 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-25T00:11:37.278 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:37.279 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:37.279 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-25T00:11:37.279 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:37.279 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:37.279 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:11:37.279 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:37.279 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:11:37.279 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42437sec preferred_lft 42437sec 2024-04-25T00:11:37.279 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-25T00:11:37.279 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:37.279 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:11:37.279 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:8d:d4:7d:b9 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:37.280 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:11:37.280 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:37.280 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:11:37.280 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:37.280 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:11:37.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:11:37.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2024-04-25T00:11:37.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2024-04-25T00:11:37.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:11:37.280 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:11:37.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:37.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:37.432 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:11:37.450 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-25T00:11:37.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:37.453 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-25T00:11:37.512 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-25T00:11:37.512 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:11:37.512 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:11:37.513 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:11:37.513 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:11:37.513 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:11:37.513 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:11:37.513 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:37.514 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:37.514 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:37.514 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:11:37.514 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:37.515 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:11:37.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:11:37.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:11:37.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:11:37.515 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:11:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:37.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:37.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:37.645 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:11:37.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:37.654 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-04-25T00:11:37.709 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:37.709 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:11:37.709 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:11:37.709 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T00:11:37.709 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:11:37.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:37.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:37.834 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T00:11:37.834 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:37.834 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:11:37.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T00:11:37.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:11:37.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T00:11:37.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T00:11:37.835 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T00:11:37.835 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:11:37.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:37.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:37.972 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:37.972 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:11:37.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-25T00:11:37.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:11:37.972 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:11:38.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:38.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:38.054 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:11:38.055 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:11:38.109 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:11:38.109 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:11:38.109 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:11:38.164 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T00:11:38.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:38.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:11:38.256 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:11:38.257 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:11:38.851 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:11:38.851 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:11:38.851 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:11:38.851 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:11:38.852 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:11:38.856 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:11:38.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:38.856 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:11:38.876 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:11:38.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:11:38.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:38.979 INFO:teuthology.orchestra.run.smithi083.stdout:client4607 2024-04-25T00:11:38.979 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:11:38.979 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:38.988 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:11:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:11:39.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:39.088 INFO:teuthology.orchestra.run.smithi083.stdout:client4607 2024-04-25T00:11:39.089 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:11:39.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:11:39.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:39.144 INFO:teuthology.orchestra.run.smithi083.stdout:client4607 2024-04-25T00:11:39.145 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d58ae83a-ad72-4e18-824a-86e779979c35.client4607/status') 2024-04-25T00:11:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:39.203 INFO:teuthology.orchestra.run.smithi083.stdout:instance: client.4607 (3)172.21.15.83:0/4078877079 2024-04-25T00:11:39.203 INFO:teuthology.orchestra.run.smithi083.stdout:blocklisted: false 2024-04-25T00:11:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records in 2024-04-25T00:11:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records out 2024-04-25T00:11:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:70 bytes copied, 5.9138e-05 s, 1.2 MB/s 2024-04-25T00:11:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:39.212 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-25T00:11:39.212 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:11:39.212 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T00:11:39.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:11:39.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.343+0000 7f98fc600640 1 Processor -- start 2024-04-25T00:11:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.343+0000 7f98fc600640 1 -- start start 2024-04-25T00:11:39.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.347+0000 7f98fc600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98f40cc510 0x7f98f415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:39.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.347+0000 7f98fc600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98f415cf90 0x7f98f40ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:39.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.347+0000 7f98fc600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98f40ccc60 con 0x7f98f41599f0 2024-04-25T00:11:39.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.347+0000 7f98fc600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f98f4159dd0 con 0x7f98f415cf90 2024-04-25T00:11:39.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.347+0000 7f98fc600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f98f406dba0 con 0x7f98f40cc510 2024-04-25T00:11:39.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.347+0000 7f98fa600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98f415cf90 0x7f98f40ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:39.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.347+0000 7f98fa600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98f415cf90 0x7f98f40ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:48596/0 (socket says 172.21.15.83:48596) 2024-04-25T00:11:39.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.347+0000 7f98fa600640 1 -- 172.21.15.83:0/802681665 learned_addr learned my addr 172.21.15.83:0/802681665 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:39.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.347+0000 7f98f9200640 1 --2- 172.21.15.83:0/802681665 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98f40cc510 0x7f98f415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:39.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.347+0000 7f98fa600640 1 -- 172.21.15.83:0/802681665 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98f40cc510 msgr2=0x7f98f415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:39.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.347+0000 7f98fa600640 1 --2- 172.21.15.83:0/802681665 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98f40cc510 0x7f98f415ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:39.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.347+0000 7f98fa600640 1 -- 172.21.15.83:0/802681665 >> v1:172.21.15.86:6789/0 conn(0x7f98f41599f0 legacy=0x7f98f415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:39.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.347+0000 7f98fa600640 1 -- 172.21.15.83:0/802681665 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f406de80 con 0x7f98f415cf90 2024-04-25T00:11:39.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.347+0000 7f98f3e00640 1 -- 172.21.15.83:0/802681665 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2169816806 0 0) 0x7f98e8001120 con 0x7f98f41599f0 2024-04-25T00:11:39.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.351+0000 7f98fa600640 1 --2- 172.21.15.83:0/802681665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f415cf90 0x7f98f40ce6e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009e50 tx=0x7f98ec01c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95611d79e057bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:39.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.351+0000 7f98f3e00640 1 -- 172.21.15.83:0/802681665 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0a56b0 con 0x7f98f415cf90 2024-04-25T00:11:39.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.351+0000 7f98f3e00640 1 -- 172.21.15.83:0/802681665 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ec0a5850 con 0x7f98f415cf90 2024-04-25T00:11:39.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.351+0000 7f98f3e00640 1 -- 172.21.15.83:0/802681665 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0a5c20 con 0x7f98f415cf90 2024-04-25T00:11:39.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.351+0000 7f98fc600640 1 -- 172.21.15.83:0/802681665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f415cf90 msgr2=0x7f98f40ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:39.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.351+0000 7f98fc600640 1 --2- 172.21.15.83:0/802681665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f415cf90 0x7f98f40ce6e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009e50 tx=0x7f98ec01c9a0 comp rx=0 tx=0).stop 2024-04-25T00:11:39.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.351+0000 7f98fc600640 1 -- 172.21.15.83:0/802681665 shutdown_connections 2024-04-25T00:11:39.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.351+0000 7f98fc600640 1 --2- 172.21.15.83:0/802681665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f415cf90 0x7f98f40ce6e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:39.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.351+0000 7f98fc600640 1 --2- 172.21.15.83:0/802681665 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98f40cc510 0x7f98f415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:39.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.351+0000 7f98fc600640 1 -- 172.21.15.83:0/802681665 >> 172.21.15.83:0/802681665 conn(0x7f98f40b1830 msgr2=0x7f98f40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:39.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.351+0000 7f98fc600640 1 -- 172.21.15.83:0/802681665 shutdown_connections 2024-04-25T00:11:39.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.351+0000 7f98fc600640 1 -- 172.21.15.83:0/802681665 wait complete. 2024-04-25T00:11:39.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.351+0000 7f98fc600640 1 Processor -- start 2024-04-25T00:11:39.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.351+0000 7f98fc600640 1 -- start start 2024-04-25T00:11:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98fc600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98f40cc510 0x7f98f414b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98fc600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98f41599f0 0x7f98f4151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98fc600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f415cf90 0x7f98f40b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98fc600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98f40cf5b0 con 0x7f98f41599f0 2024-04-25T00:11:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98fc600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98f40cf430 con 0x7f98f415cf90 2024-04-25T00:11:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98fc600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f98f40cf730 con 0x7f98f40cc510 2024-04-25T00:11:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98f9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98f41599f0 0x7f98f4151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98f9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98f41599f0 0x7f98f4151be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46080/0 (socket says 172.21.15.83:46080) 2024-04-25T00:11:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98f9200640 1 -- 172.21.15.83:0/3272542216 learned_addr learned my addr 172.21.15.83:0/3272542216 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98f9c00640 1 --2- 172.21.15.83:0/3272542216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98f40cc510 0x7f98f414b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98fa600640 1 --2- 172.21.15.83:0/3272542216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f415cf90 0x7f98f40b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98f9200640 1 -- 172.21.15.83:0/3272542216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98f40cc510 msgr2=0x7f98f414b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98f9200640 1 --2- 172.21.15.83:0/3272542216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98f40cc510 0x7f98f414b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98f9200640 1 -- 172.21.15.83:0/3272542216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f415cf90 msgr2=0x7f98f40b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:39.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98f9200640 1 --2- 172.21.15.83:0/3272542216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f415cf90 0x7f98f40b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:39.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98f9200640 1 -- 172.21.15.83:0/3272542216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f414bd40 con 0x7f98f41599f0 2024-04-25T00:11:39.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98f9200640 1 --2- 172.21.15.83:0/3272542216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98f41599f0 0x7f98f4151be0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f98e000e870 tx=0x7f98e000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:39.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98f1600640 1 -- 172.21.15.83:0/3272542216 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0017070 con 0x7f98f41599f0 2024-04-25T00:11:39.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98f1600640 1 -- 172.21.15.83:0/3272542216 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e000c940 con 0x7f98f41599f0 2024-04-25T00:11:39.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98f1600640 1 -- 172.21.15.83:0/3272542216 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e000ccf0 con 0x7f98f41599f0 2024-04-25T00:11:39.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98fc600640 1 -- 172.21.15.83:0/3272542216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f415be90 con 0x7f98f41599f0 2024-04-25T00:11:39.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98fc600640 1 -- 172.21.15.83:0/3272542216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f415b8d0 con 0x7f98f41599f0 2024-04-25T00:11:39.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98fc600640 1 -- 172.21.15.83:0/3272542216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f40737f0 con 0x7f98f41599f0 2024-04-25T00:11:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.355+0000 7f98f1600640 1 -- 172.21.15.83:0/3272542216 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f98e001a070 con 0x7f98f41599f0 2024-04-25T00:11:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.359+0000 7f98f1600640 1 --2- 172.21.15.83:0/3272542216 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98d40678c0 0x7f98d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.359+0000 7f98f1600640 1 -- 172.21.15.83:0/3272542216 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f98e0085ed0 con 0x7f98f41599f0 2024-04-25T00:11:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.359+0000 7f98f9c00640 1 --2- 172.21.15.83:0/3272542216 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98d40678c0 0x7f98d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:39.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.359+0000 7f98f9c00640 1 --2- 172.21.15.83:0/3272542216 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98d40678c0 0x7f98d4069d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f98e8000f70 tx=0x7f98e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:39.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.363+0000 7f98f1600640 1 -- 172.21.15.83:0/3272542216 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f98e00566c0 con 0x7f98f41599f0 2024-04-25T00:11:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.551+0000 7f98fc600640 1 -- 172.21.15.83:0/3272542216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f98f404f5f0 con 0x7f98f41599f0 2024-04-25T00:11:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.551+0000 7f98f1600640 1 -- 172.21.15.83:0/3272542216 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f98e0055fc0 con 0x7f98f41599f0 2024-04-25T00:11:39.557 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T00:11:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.555+0000 7f98fc600640 1 -- 172.21.15.83:0/3272542216 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98d40678c0 msgr2=0x7f98d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.555+0000 7f98fc600640 1 --2- 172.21.15.83:0/3272542216 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98d40678c0 0x7f98d4069d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f98e8000f70 tx=0x7f98e8009040 comp rx=0 tx=0).stop 2024-04-25T00:11:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.555+0000 7f98fc600640 1 -- 172.21.15.83:0/3272542216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98f41599f0 msgr2=0x7f98f4151be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.555+0000 7f98fc600640 1 --2- 172.21.15.83:0/3272542216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98f41599f0 0x7f98f4151be0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f98e000e870 tx=0x7f98e000ec70 comp rx=0 tx=0).stop 2024-04-25T00:11:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.555+0000 7f98fc600640 1 -- 172.21.15.83:0/3272542216 shutdown_connections 2024-04-25T00:11:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.555+0000 7f98fc600640 1 --2- 172.21.15.83:0/3272542216 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98d40678c0 0x7f98d4069d80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.555+0000 7f98fc600640 1 --2- 172.21.15.83:0/3272542216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f415cf90 0x7f98f40b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.555+0000 7f98fc600640 1 --2- 172.21.15.83:0/3272542216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98f41599f0 0x7f98f4151be0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.555+0000 7f98fc600640 1 --2- 172.21.15.83:0/3272542216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98f40cc510 0x7f98f414b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.555+0000 7f98fc600640 1 -- 172.21.15.83:0/3272542216 >> 172.21.15.83:0/3272542216 conn(0x7f98f40b1830 msgr2=0x7f98f4072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.555+0000 7f98fc600640 1 -- 172.21.15.83:0/3272542216 shutdown_connections 2024-04-25T00:11:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39.555+0000 7f98fc600640 1 -- 172.21.15.83:0/3272542216 wait complete. 2024-04-25T00:11:39.577 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:11:39.577 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:11:39.577 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-25T00:11:39.577 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T00:11:39.578 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T00:11:39.578 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:11:39.578 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:11:39.578 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:11:39.578 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T00:11:39.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:39.578 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-25T00:11:39.588 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:11:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:11:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:11:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-25T00:11:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:39.588 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-25T00:11:39.588 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-25T00:11:39.589 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:39.589 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:11:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:11:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42435sec preferred_lft 42435sec 2024-04-25T00:11:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-25T00:11:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:39.590 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:11:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:8d:d4:7d:b9 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:11:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:39.590 INFO:teuthology.orchestra.run.smithi083.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:11:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:11:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:11:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:39.590 INFO:teuthology.orchestra.run.smithi083.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:11:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:11:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-04-25T00:11:39.591 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:39.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:39.591 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:11:39.648 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-25T00:11:39.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:39.650 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-25T00:11:39.704 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-25T00:11:39.704 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:11:39.704 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:11:39.704 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:11:39.704 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:11:39.704 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:11:39.704 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:11:39.705 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:39.705 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:39.705 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:39.705 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:11:39.705 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:39.705 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:11:39.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:11:39.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:11:39.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:11:39.706 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:11:39.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:39.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:39.838 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:11:39.847 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:11:39.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:39.848 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-04-25T00:11:39.897 INFO:teuthology.orchestra.run.smithi083.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T00:11:39.897 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:39.897 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:11:39.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:11:39.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T00:11:39.898 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:11:39.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:40.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:40.022 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T00:11:40.043 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:11:40.044 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:11:40.044 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:11:40.044 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:40.044 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-25T00:11:40.044 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:40.044 INFO:teuthology.orchestra.run.smithi083.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:11:40.044 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 3a:fa:b4:ca:73:80 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T00:11:40.044 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T00:11:40.044 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:40.045 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::38fa:b4ff:feca:7380/64 scope link 2024-04-25T00:11:40.045 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:40.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:40.046 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T00:11:40.108 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:11:40.108 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:11:40.108 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:11:40.109 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:40.109 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-25T00:11:40.109 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:40.109 INFO:teuthology.orchestra.run.smithi083.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:11:40.109 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 3a:fa:b4:ca:73:80 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T00:11:40.109 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T00:11:40.109 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:40.109 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::38fa:b4ff:feca:7380/64 scope link 2024-04-25T00:11:40.109 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:40.110 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T00:11:40.111 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:40.111 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:11:40.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T00:11:40.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T00:11:40.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T00:11:40.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T00:11:40.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T00:11:40.111 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:11:40.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:40.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:40.293 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:40.293 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:11:40.293 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.1 up 2024-04-25T00:11:40.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T00:11:40.294 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:11:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:40.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:40.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:11:40.419 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T00:11:40.428 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T00:11:40.428 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:11:40.428 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T00:11:40.476 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-25T00:11:40.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:40.477 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:11:40.568 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:11:40.569 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:11:40.583 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:11:40.583 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:11:40.583 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:11:40.583 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 2@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:11:40.583 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:11:40.587 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:11:40.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:40.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T00:11:40.606 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:11:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:11:40.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:40.709 INFO:teuthology.orchestra.run.smithi083.stdout:client4616 2024-04-25T00:11:40.709 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:11:40.709 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:40.761 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:11:40.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:11:40.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:40.859 INFO:teuthology.orchestra.run.smithi083.stdout:client4616 2024-04-25T00:11:40.860 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:11:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:11:40.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:40.916 INFO:teuthology.orchestra.run.smithi083.stdout:client4616 2024-04-25T00:11:40.916 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d58ae83a-ad72-4e18-824a-86e779979c35.client4616/status') 2024-04-25T00:11:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:40.975 INFO:teuthology.orchestra.run.smithi083.stdout:instance: client.4616 (3)172.21.15.83:0/4272914929 2024-04-25T00:11:40.975 INFO:teuthology.orchestra.run.smithi083.stdout:blocklisted: false 2024-04-25T00:11:40.975 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records in 2024-04-25T00:11:40.975 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records out 2024-04-25T00:11:40.975 INFO:teuthology.orchestra.run.smithi083.stderr:70 bytes copied, 4.9569e-05 s, 1.4 MB/s 2024-04-25T00:11:40.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:40.983 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-25T00:11:40.983 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:11:40.983 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:11:40.983 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:11:41.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4fc00640 1 Processor -- start 2024-04-25T00:11:41.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4fc00640 1 -- start start 2024-04-25T00:11:41.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4fc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f481599f0 0x7f5f480cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:41.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4fc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5f4815ce90 0x7f5f480ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:41.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4fc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f480ccc60 con 0x7f5f4815c340 2024-04-25T00:11:41.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4fc00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5f48159dd0 con 0x7f5f4815ce90 2024-04-25T00:11:41.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4fc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5f4806dba0 con 0x7f5f481599f0 2024-04-25T00:11:41.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4dc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5f4815ce90 0x7f5f480ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:41.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4dc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5f4815ce90 0x7f5f480ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46092/0 (socket says 172.21.15.83:46092) 2024-04-25T00:11:41.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4dc00640 1 -- 172.21.15.83:0/4118446784 learned_addr learned my addr 172.21.15.83:0/4118446784 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:41.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4d200640 1 --2- 172.21.15.83:0/4118446784 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f481599f0 0x7f5f480cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:41.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4dc00640 1 -- 172.21.15.83:0/4118446784 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f481599f0 msgr2=0x7f5f480cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:41.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4dc00640 1 --2- 172.21.15.83:0/4118446784 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f481599f0 0x7f5f480cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:41.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4dc00640 1 -- 172.21.15.83:0/4118446784 >> v1:172.21.15.86:6789/0 conn(0x7f5f4815c340 legacy=0x7f5f4815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:41.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.115+0000 7f5f4dc00640 1 -- 172.21.15.83:0/4118446784 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4806de80 con 0x7f5f4815ce90 2024-04-25T00:11:41.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.119+0000 7f5f4dc00640 1 --2- 172.21.15.83:0/4118446784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f4815ce90 0x7f5f480ce6e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009e80 tx=0x7f5f4001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f294efe955d5a0f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:41.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.119+0000 7f5f47e00640 1 -- 172.21.15.83:0/4118446784 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f400a5640 con 0x7f5f4815ce90 2024-04-25T00:11:41.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.119+0000 7f5f47e00640 1 -- 172.21.15.83:0/4118446784 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f400a57e0 con 0x7f5f4815ce90 2024-04-25T00:11:41.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.119+0000 7f5f47e00640 1 -- 172.21.15.83:0/4118446784 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f400a5bb0 con 0x7f5f4815ce90 2024-04-25T00:11:41.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.119+0000 7f5f4fc00640 1 -- 172.21.15.83:0/4118446784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f4815ce90 msgr2=0x7f5f480ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:41.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.119+0000 7f5f4fc00640 1 --2- 172.21.15.83:0/4118446784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f4815ce90 0x7f5f480ce6e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009e80 tx=0x7f5f4001c980 comp rx=0 tx=0).stop 2024-04-25T00:11:41.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.119+0000 7f5f4fc00640 1 -- 172.21.15.83:0/4118446784 shutdown_connections 2024-04-25T00:11:41.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.119+0000 7f5f4fc00640 1 --2- 172.21.15.83:0/4118446784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f4815ce90 0x7f5f480ce6e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:41.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.119+0000 7f5f4fc00640 1 --2- 172.21.15.83:0/4118446784 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f481599f0 0x7f5f480cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:41.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.119+0000 7f5f4fc00640 1 -- 172.21.15.83:0/4118446784 >> 172.21.15.83:0/4118446784 conn(0x7f5f480b1830 msgr2=0x7f5f480b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:41.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.119+0000 7f5f4fc00640 1 -- 172.21.15.83:0/4118446784 shutdown_connections 2024-04-25T00:11:41.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.119+0000 7f5f4fc00640 1 -- 172.21.15.83:0/4118446784 wait complete. 2024-04-25T00:11:41.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4fc00640 1 Processor -- start 2024-04-25T00:11:41.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4fc00640 1 -- start start 2024-04-25T00:11:41.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4fc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f481599f0 0x7f5f4814d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:41.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4fc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f4815c340 0x7f5f48152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:41.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f4815ce90 0x7f5f48165760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:41.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4fc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f480cf7c0 con 0x7f5f481599f0 2024-04-25T00:11:41.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f480cf640 con 0x7f5f4815ce90 2024-04-25T00:11:41.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4fc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5f48168cd0 con 0x7f5f4815c340 2024-04-25T00:11:41.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f481599f0 0x7f5f4814d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:41.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f481599f0 0x7f5f4814d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46098/0 (socket says 172.21.15.83:46098) 2024-04-25T00:11:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4d200640 1 -- 172.21.15.83:0/311378971 learned_addr learned my addr 172.21.15.83:0/311378971 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4dc00640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f4815ce90 0x7f5f48165760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f3fe00640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f4815c340 0x7f5f48152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4d200640 1 -- 172.21.15.83:0/311378971 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f4815c340 msgr2=0x7f5f48152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4d200640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f4815c340 0x7f5f48152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4d200640 1 -- 172.21.15.83:0/311378971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f4815ce90 msgr2=0x7f5f48165760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4d200640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f4815ce90 0x7f5f48165760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4d200640 1 -- 172.21.15.83:0/311378971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4814de80 con 0x7f5f481599f0 2024-04-25T00:11:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4dc00640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f4815ce90 0x7f5f48165760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4d200640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f481599f0 0x7f5f4814d940 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800e870 tx=0x7f5f3800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f45600640 1 -- 172.21.15.83:0/311378971 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38017070 con 0x7f5f481599f0 2024-04-25T00:11:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f45600640 1 -- 172.21.15.83:0/311378971 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f3800c940 con 0x7f5f481599f0 2024-04-25T00:11:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4fc00640 1 -- 172.21.15.83:0/311378971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f48152f30 con 0x7f5f481599f0 2024-04-25T00:11:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f45600640 1 -- 172.21.15.83:0/311378971 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800cd10 con 0x7f5f481599f0 2024-04-25T00:11:41.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.123+0000 7f5f4fc00640 1 -- 172.21.15.83:0/311378971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f481534f0 con 0x7f5f481599f0 2024-04-25T00:11:41.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.127+0000 7f5f4fc00640 1 -- 172.21.15.83:0/311378971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f480737f0 con 0x7f5f481599f0 2024-04-25T00:11:41.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.127+0000 7f5f45600640 1 -- 172.21.15.83:0/311378971 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5f3801a070 con 0x7f5f481599f0 2024-04-25T00:11:41.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.127+0000 7f5f45600640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f240678c0 0x7f5f24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:41.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.127+0000 7f5f45600640 1 -- 172.21.15.83:0/311378971 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5f38085f80 con 0x7f5f481599f0 2024-04-25T00:11:41.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.127+0000 7f5f3fe00640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f240678c0 0x7f5f24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:41.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.127+0000 7f5f3fe00640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f240678c0 0x7f5f24069d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5f480b2d80 tx=0x7f5f30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:41.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.131+0000 7f5f45600640 1 -- 172.21.15.83:0/311378971 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5f4814de80 con 0x7f5f481599f0 2024-04-25T00:11:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.323+0000 7f5f4fc00640 1 -- 172.21.15.83:0/311378971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5f48072c40 con 0x7f5f481599f0 2024-04-25T00:11:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.323+0000 7f5f45600640 1 -- 172.21.15.83:0/311378971 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f5f380566e0 con 0x7f5f481599f0 2024-04-25T00:11:41.328 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T00:11:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.327+0000 7f5f4fc00640 1 -- 172.21.15.83:0/311378971 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f240678c0 msgr2=0x7f5f24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.327+0000 7f5f4fc00640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f240678c0 0x7f5f24069d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5f480b2d80 tx=0x7f5f30009040 comp rx=0 tx=0).stop 2024-04-25T00:11:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.327+0000 7f5f4fc00640 1 -- 172.21.15.83:0/311378971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f481599f0 msgr2=0x7f5f4814d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.327+0000 7f5f4fc00640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f481599f0 0x7f5f4814d940 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800e870 tx=0x7f5f3800ed30 comp rx=0 tx=0).stop 2024-04-25T00:11:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.327+0000 7f5f4fc00640 1 -- 172.21.15.83:0/311378971 shutdown_connections 2024-04-25T00:11:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.327+0000 7f5f4fc00640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f240678c0 0x7f5f24069d80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.327+0000 7f5f4fc00640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f4815c340 0x7f5f48152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.327+0000 7f5f4fc00640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f4815ce90 0x7f5f48165760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.327+0000 7f5f4fc00640 1 --2- 172.21.15.83:0/311378971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f481599f0 0x7f5f4814d940 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.327+0000 7f5f4fc00640 1 -- 172.21.15.83:0/311378971 >> 172.21.15.83:0/311378971 conn(0x7f5f480b1830 msgr2=0x7f5f48164e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.327+0000 7f5f4fc00640 1 -- 172.21.15.83:0/311378971 shutdown_connections 2024-04-25T00:11:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:41.327+0000 7f5f4fc00640 1 -- 172.21.15.83:0/311378971 wait complete. 2024-04-25T00:11:41.350 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:11:41.350 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:11:41.350 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T00:11:41.350 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:11:41.350 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:11:41.350 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:11:41.350 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:11:41.350 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:11:41.350 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:11:41.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:41.351 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T00:11:41.361 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:11:41.361 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:11:41.361 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:11:41.361 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:41.361 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T00:11:41.361 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:41.361 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:41.361 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:41.361 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T00:11:41.361 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:41.361 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:41.361 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T00:11:41.361 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:41.362 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:41.362 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:11:41.362 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:41.362 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:11:41.362 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42427sec preferred_lft 42427sec 2024-04-25T00:11:41.362 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T00:11:41.362 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:41.362 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:11:41.362 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:35:63:ea:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:41.362 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:11:41.362 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:41.363 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:11:41.363 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:41.363 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:11:41.364 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:11:41.364 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-04-25T00:11:41.364 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-04-25T00:11:41.364 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:11:41.364 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:11:41.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:41.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:41.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:41.531 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:11:41.549 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T00:11:41.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:41.551 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T00:11:41.621 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T00:11:41.621 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:11:41.621 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:11:41.621 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:11:41.621 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:11:41.621 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:11:41.622 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:11:41.622 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:41.622 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:41.622 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:41.622 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:11:41.622 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:41.622 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:11:41.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:11:41.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:11:41.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:11:41.623 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:41.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:41.763 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T00:11:41.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:41.818 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-25T00:11:41.869 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:41.869 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:11:41.869 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:11:41.869 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T00:11:41.869 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:11:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:41.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:41.994 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T00:11:41.995 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:41.995 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:11:41.995 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T00:11:41.995 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:11:41.995 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T00:11:41.995 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T00:11:41.995 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T00:11:41.995 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:11:42.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:42.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:42.164 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:42.164 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:11:42.164 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T00:11:42.164 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:11:42.164 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:11:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:42.248 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:11:42.248 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:11:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:11:42.256 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:11:42.256 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:11:42.304 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T00:11:42.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:42.304 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:11:42.391 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:11:42.391 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:11:43.002 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:11:43.002 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:11:43.002 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:11:43.002 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with new device syntax: 1@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:11:43.002 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:11:43.006 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:11:43.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:43.006 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:11:43.026 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:11:43.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:11:43.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:43.132 INFO:teuthology.orchestra.run.smithi086.stdout:client4625 2024-04-25T00:11:43.133 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:11:43.133 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:43.143 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:11:43.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:11:43.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:43.240 INFO:teuthology.orchestra.run.smithi086.stdout:client4625 2024-04-25T00:11:43.240 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:11:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:11:43.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:43.295 INFO:teuthology.orchestra.run.smithi086.stdout:client4625 2024-04-25T00:11:43.296 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d58ae83a-ad72-4e18-824a-86e779979c35.client4625/status') 2024-04-25T00:11:43.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:43.394 INFO:teuthology.orchestra.run.smithi086.stdout:instance: client.4625 (3)192.168.0.1:0/1243870647 2024-04-25T00:11:43.394 INFO:teuthology.orchestra.run.smithi086.stdout:blocklisted: false 2024-04-25T00:11:43.394 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records in 2024-04-25T00:11:43.394 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records out 2024-04-25T00:11:43.394 INFO:teuthology.orchestra.run.smithi086.stderr:69 bytes copied, 4.9551e-05 s, 1.4 MB/s 2024-04-25T00:11:43.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:43.402 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-25T00:11:43.402 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:11:43.402 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T00:11:43.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:11:43.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95ee400640 1 Processor -- start 2024-04-25T00:11:43.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95ee400640 1 -- start start 2024-04-25T00:11:43.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95ee400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95e80d01e0 0x7f95e815c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:43.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95ee400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f95e815c7e0 0x7f95e80b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:43.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95ee400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95e81605c0 con 0x7f95e81597d0 2024-04-25T00:11:43.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95ee400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f95e815cff0 con 0x7f95e815c7e0 2024-04-25T00:11:43.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95ee400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f95e815f800 con 0x7f95e80d01e0 2024-04-25T00:11:43.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95eda00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f95e815c7e0 0x7f95e80b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:43.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95eda00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f95e815c7e0 0x7f95e80b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46102/0 (socket says 172.21.15.83:46102) 2024-04-25T00:11:43.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95eda00640 1 -- 172.21.15.83:0/1165205503 learned_addr learned my addr 172.21.15.83:0/1165205503 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:43.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95e7e00640 1 --2- 172.21.15.83:0/1165205503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95e80d01e0 0x7f95e815c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:43.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95eda00640 1 -- 172.21.15.83:0/1165205503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95e80d01e0 msgr2=0x7f95e815c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:43.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95eda00640 1 --2- 172.21.15.83:0/1165205503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95e80d01e0 0x7f95e815c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:43.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95eda00640 1 -- 172.21.15.83:0/1165205503 >> v1:172.21.15.86:6789/0 conn(0x7f95e81597d0 legacy=0x7f95e80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:43.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.547+0000 7f95eda00640 1 -- 172.21.15.83:0/1165205503 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e815fae0 con 0x7f95e815c7e0 2024-04-25T00:11:43.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.551+0000 7f95eda00640 1 --2- 172.21.15.83:0/1165205503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e815c7e0 0x7f95e80b2950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009e50 tx=0x7f95e001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9086f78857b0c6c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:43.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.551+0000 7f95e7400640 1 -- 172.21.15.83:0/1165205503 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e00a57a0 con 0x7f95e815c7e0 2024-04-25T00:11:43.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.551+0000 7f95e7400640 1 -- 172.21.15.83:0/1165205503 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e00a5940 con 0x7f95e815c7e0 2024-04-25T00:11:43.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.551+0000 7f95e7400640 1 -- 172.21.15.83:0/1165205503 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e00a5d10 con 0x7f95e815c7e0 2024-04-25T00:11:43.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.551+0000 7f95ee400640 1 -- 172.21.15.83:0/1165205503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e815c7e0 msgr2=0x7f95e80b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:43.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.551+0000 7f95ee400640 1 --2- 172.21.15.83:0/1165205503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e815c7e0 0x7f95e80b2950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009e50 tx=0x7f95e001c8e0 comp rx=0 tx=0).stop 2024-04-25T00:11:43.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.551+0000 7f95ee400640 1 -- 172.21.15.83:0/1165205503 shutdown_connections 2024-04-25T00:11:43.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.551+0000 7f95ee400640 1 --2- 172.21.15.83:0/1165205503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e815c7e0 0x7f95e80b2950 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:43.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.551+0000 7f95ee400640 1 --2- 172.21.15.83:0/1165205503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95e80d01e0 0x7f95e815c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:43.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.551+0000 7f95ee400640 1 -- 172.21.15.83:0/1165205503 >> 172.21.15.83:0/1165205503 conn(0x7f95e806b3b0 msgr2=0x7f95e806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:43.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.551+0000 7f95ee400640 1 -- 172.21.15.83:0/1165205503 shutdown_connections 2024-04-25T00:11:43.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.551+0000 7f95ee400640 1 -- 172.21.15.83:0/1165205503 wait complete. 2024-04-25T00:11:43.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.551+0000 7f95ee400640 1 Processor -- start 2024-04-25T00:11:43.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.551+0000 7f95ee400640 1 -- start start 2024-04-25T00:11:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ee400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e80d01e0 0x7f95e815a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ee400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95e81597d0 0x7f95e8153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ee400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95e815c7e0 0x7f95e8153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ee400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95e80b3820 con 0x7f95e80d01e0 2024-04-25T00:11:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ee400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95e80b36a0 con 0x7f95e81597d0 2024-04-25T00:11:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ee400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f95e80b39a0 con 0x7f95e815c7e0 2024-04-25T00:11:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ed000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e80d01e0 0x7f95e815a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ed000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e80d01e0 0x7f95e815a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46106/0 (socket says 172.21.15.83:46106) 2024-04-25T00:11:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ed000640 1 -- 172.21.15.83:0/4164542560 learned_addr learned my addr 172.21.15.83:0/4164542560 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95e7e00640 1 --2- 172.21.15.83:0/4164542560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95e81597d0 0x7f95e8153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95eda00640 1 --2- 172.21.15.83:0/4164542560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95e815c7e0 0x7f95e8153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ed000640 1 -- 172.21.15.83:0/4164542560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95e815c7e0 msgr2=0x7f95e8153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ed000640 1 --2- 172.21.15.83:0/4164542560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95e815c7e0 0x7f95e8153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ed000640 1 -- 172.21.15.83:0/4164542560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95e81597d0 msgr2=0x7f95e8153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ed000640 1 --2- 172.21.15.83:0/4164542560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95e81597d0 0x7f95e8153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ed000640 1 -- 172.21.15.83:0/4164542560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e815ae10 con 0x7f95e80d01e0 2024-04-25T00:11:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ed000640 1 --2- 172.21.15.83:0/4164542560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e80d01e0 0x7f95e815a8d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00c3a0 tx=0x7f95dc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95e4c00640 1 -- 172.21.15.83:0/4164542560 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0052d0 con 0x7f95e80d01e0 2024-04-25T00:11:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95e4c00640 1 -- 172.21.15.83:0/4164542560 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95dc005470 con 0x7f95e80d01e0 2024-04-25T00:11:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ee400640 1 -- 172.21.15.83:0/4164542560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e81564e0 con 0x7f95e80d01e0 2024-04-25T00:11:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95e4c00640 1 -- 172.21.15.83:0/4164542560 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc007260 con 0x7f95e80d01e0 2024-04-25T00:11:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ee400640 1 -- 172.21.15.83:0/4164542560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e8156aa0 con 0x7f95e80d01e0 2024-04-25T00:11:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.555+0000 7f95ee400640 1 -- 172.21.15.83:0/4164542560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e804f510 con 0x7f95e80d01e0 2024-04-25T00:11:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.559+0000 7f95e4c00640 1 -- 172.21.15.83:0/4164542560 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f95dc007400 con 0x7f95e80d01e0 2024-04-25T00:11:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.559+0000 7f95e4c00640 1 --2- 172.21.15.83:0/4164542560 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95c80678f0 0x7f95c8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.559+0000 7f95e4c00640 1 -- 172.21.15.83:0/4164542560 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f95dc0854c0 con 0x7f95e80d01e0 2024-04-25T00:11:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.559+0000 7f95e7e00640 1 --2- 172.21.15.83:0/4164542560 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95c80678f0 0x7f95c8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.559+0000 7f95e7e00640 1 --2- 172.21.15.83:0/4164542560 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95c80678f0 0x7f95c8069db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f95e815b530 tx=0x7f95d4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:43.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.563+0000 7f95e4c00640 1 -- 172.21.15.83:0/4164542560 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f95dc055540 con 0x7f95e80d01e0 2024-04-25T00:11:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.751+0000 7f95ee400640 1 -- 172.21.15.83:0/4164542560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f95e80b4d60 con 0x7f95e80d01e0 2024-04-25T00:11:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.751+0000 7f95e4c00640 1 -- 172.21.15.83:0/4164542560 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f95dc059840 con 0x7f95e80d01e0 2024-04-25T00:11:43.756 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T00:11:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.755+0000 7f95ee400640 1 -- 172.21.15.83:0/4164542560 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95c80678f0 msgr2=0x7f95c8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.755+0000 7f95ee400640 1 --2- 172.21.15.83:0/4164542560 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95c80678f0 0x7f95c8069db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f95e815b530 tx=0x7f95d4009190 comp rx=0 tx=0).stop 2024-04-25T00:11:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.755+0000 7f95ee400640 1 -- 172.21.15.83:0/4164542560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e80d01e0 msgr2=0x7f95e815a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.755+0000 7f95ee400640 1 --2- 172.21.15.83:0/4164542560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e80d01e0 0x7f95e815a8d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00c3a0 tx=0x7f95dc00c860 comp rx=0 tx=0).stop 2024-04-25T00:11:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.755+0000 7f95ee400640 1 -- 172.21.15.83:0/4164542560 shutdown_connections 2024-04-25T00:11:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.755+0000 7f95ee400640 1 --2- 172.21.15.83:0/4164542560 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95c80678f0 0x7f95c8069db0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.755+0000 7f95ee400640 1 --2- 172.21.15.83:0/4164542560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95e815c7e0 0x7f95e8153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.755+0000 7f95ee400640 1 --2- 172.21.15.83:0/4164542560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95e81597d0 0x7f95e8153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.755+0000 7f95ee400640 1 --2- 172.21.15.83:0/4164542560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e80d01e0 0x7f95e815a8d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.755+0000 7f95ee400640 1 -- 172.21.15.83:0/4164542560 >> 172.21.15.83:0/4164542560 conn(0x7f95e806b3b0 msgr2=0x7f95e80cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.755+0000 7f95ee400640 1 -- 172.21.15.83:0/4164542560 shutdown_connections 2024-04-25T00:11:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:43.755+0000 7f95ee400640 1 -- 172.21.15.83:0/4164542560 wait complete. 2024-04-25T00:11:43.777 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:11:43.777 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:11:43.777 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T00:11:43.777 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T00:11:43.777 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T00:11:43.777 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:11:43.778 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:11:43.778 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:11:43.778 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T00:11:43.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:43.778 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T00:11:43.786 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:11:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:11:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:11:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T00:11:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:43.787 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T00:11:43.787 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T00:11:43.788 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:11:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:43.788 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:11:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:11:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42425sec preferred_lft 42425sec 2024-04-25T00:11:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T00:11:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:43.788 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:11:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:35:63:ea:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:11:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:43.788 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:11:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:11:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:11:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:11:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:43.789 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:11:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:11:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-04-25T00:11:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:43.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:43.790 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:11:43.850 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T00:11:43.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:43.853 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T00:11:43.913 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T00:11:43.913 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:11:43.913 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:11:43.914 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:11:43.914 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:11:43.914 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:11:43.914 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:11:43.915 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:43.915 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:43.915 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:11:43.915 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:11:43.916 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:43.916 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:11:43.916 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:11:43.916 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:11:43.917 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:11:43.917 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:11:44.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:44.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:44.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:44.049 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T00:11:44.057 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:11:44.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:44.058 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-25T00:11:44.109 INFO:teuthology.orchestra.run.smithi086.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T00:11:44.109 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:44.109 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:11:44.109 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:11:44.109 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T00:11:44.109 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:11:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:44.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:44.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:44.236 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T00:11:44.256 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:11:44.256 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:11:44.257 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:11:44.257 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:44.257 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T00:11:44.257 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:44.257 INFO:teuthology.orchestra.run.smithi086.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:11:44.257 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 72:df:a0:fe:33:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T00:11:44.257 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T00:11:44.257 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:44.257 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::70df:a0ff:fefe:3350/64 scope link 2024-04-25T00:11:44.257 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:44.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:44.259 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T00:11:44.321 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:11:44.321 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:11:44.321 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:11:44.321 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:44.321 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T00:11:44.321 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:44.321 INFO:teuthology.orchestra.run.smithi086.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:11:44.321 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 72:df:a0:fe:33:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T00:11:44.322 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T00:11:44.322 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:44.322 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::70df:a0ff:fefe:3350/64 scope link 2024-04-25T00:11:44.322 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:11:44.324 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T00:11:44.324 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:44.324 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:11:44.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T00:11:44.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T00:11:44.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T00:11:44.325 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T00:11:44.325 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T00:11:44.325 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:11:44.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:44.508 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:11:44.508 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:11:44.508 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.1 up 2024-04-25T00:11:44.508 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T00:11:44.508 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:11:44.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:44.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:44.591 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:11:44.591 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T00:11:44.645 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T00:11:44.645 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:11:44.645 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T00:11:44.695 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-25T00:11:44.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:44.696 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:11:44.784 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:11:44.784 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:11:44.797 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:11:44.797 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:11:44.797 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:11:44.797 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with new device syntax: 3@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:11:44.798 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:11:44.801 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:11:44.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:44.801 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T00:11:44.820 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:11:44.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:11:44.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:44.920 INFO:teuthology.orchestra.run.smithi086.stdout:client4848 2024-04-25T00:11:44.921 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:11:44.921 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:44.929 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:11:45.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:11:45.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:45.028 INFO:teuthology.orchestra.run.smithi086.stdout:client4848 2024-04-25T00:11:45.028 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:11:45.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:11:45.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:45.083 INFO:teuthology.orchestra.run.smithi086.stdout:client4848 2024-04-25T00:11:45.083 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d58ae83a-ad72-4e18-824a-86e779979c35.client4848/status') 2024-04-25T00:11:45.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:11:45.141 INFO:teuthology.orchestra.run.smithi086.stdout:instance: client.4848 (3)172.21.15.86:0/2961624975 2024-04-25T00:11:45.141 INFO:teuthology.orchestra.run.smithi086.stdout:blocklisted: false 2024-04-25T00:11:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records in 2024-04-25T00:11:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records out 2024-04-25T00:11:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:70 bytes copied, 6.0011e-05 s, 1.2 MB/s 2024-04-25T00:11:45.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:11:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:11:45.150 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T00:11:45.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:45.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.303+0000 7f289e200640 1 Processor -- start 2024-04-25T00:11:45.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.303+0000 7f289e200640 1 -- start start 2024-04-25T00:11:45.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f289e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f28981553e0 0x7f289806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f289e200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f289806ef80 con 0x7f28980b4a70 2024-04-25T00:11:45.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f289e200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f289815b510 con 0x7f289806ea00 2024-04-25T00:11:45.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f289e200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f289815b7f0 con 0x7f28981553e0 2024-04-25T00:11:45.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2897e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f28980b4a70 0x7f28980b2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51772/0 (socket says 172.21.15.83:51772) 2024-04-25T00:11:45.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2897e00640 1 -- 172.21.15.83:0/4219649169 learned_addr learned my addr 172.21.15.83:0/4219649169 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:45.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2897400640 1 --2- 172.21.15.83:0/4219649169 >> v2:172.21.15.86:3300/0 conn(0x7f28981553e0 0x7f289806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 335668177 0 0) 0x7f289806ef80 con 0x7f28980b4a70 2024-04-25T00:11:45.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f287c005740 con 0x7f28980b4a70 2024-04-25T00:11:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2609938474 0 0) 0x7f289815b510 con 0x7f289806ea00 2024-04-25T00:11:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f289806ef80 con 0x7f289806ea00 2024-04-25T00:11:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1116454471 0 0) 0x7f287c005740 con 0x7f28980b4a70 2024-04-25T00:11:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 >> v2:172.21.15.86:3300/0 conn(0x7f28981553e0 msgr2=0x7f289806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 --2- 172.21.15.83:0/4219649169 >> v2:172.21.15.86:3300/0 conn(0x7f28981553e0 0x7f289806e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 >> v1:172.21.15.86:6789/0 conn(0x7f289806ea00 legacy=0x7f28980b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289815bad0 con 0x7f28980b4a70 2024-04-25T00:11:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f2890001a00 con 0x7f28980b4a70 2024-04-25T00:11:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2897400640 1 --2- 172.21.15.83:0/4219649169 >> v2:172.21.15.86:3300/0 conn(0x7f28981553e0 0x7f289806e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 >> v1:172.21.15.83:6789/0 conn(0x7f28980b4a70 legacy=0x7f28980b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 --2- 172.21.15.83:0/4219649169 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f287c005d60 0x7f287c006160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 --2- 172.21.15.83:0/4219649169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c006a20 0x7f287c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 --2- 172.21.15.83:0/4219649169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c0073e0 0x7f287c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f289815b7f0 con 0x7f287c0073e0 2024-04-25T00:11:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f287c00e310 con 0x7f287c006a20 2024-04-25T00:11:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f287c00e510 con 0x7f287c005d60 2024-04-25T00:11:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2897e00640 1 --2- 172.21.15.83:0/4219649169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c0073e0 0x7f287c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2897400640 1 --2- 172.21.15.83:0/4219649169 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f287c005d60 0x7f287c006160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f288ea00640 1 --2- 172.21.15.83:0/4219649169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c006a20 0x7f287c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2897e00640 1 -- 172.21.15.83:0/4219649169 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f287c005d60 msgr2=0x7f287c006160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2897e00640 1 --2- 172.21.15.83:0/4219649169 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f287c005d60 0x7f287c006160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2897e00640 1 -- 172.21.15.83:0/4219649169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c006a20 msgr2=0x7f287c006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2897e00640 1 --2- 172.21.15.83:0/4219649169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c006a20 0x7f287c006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.307+0000 7f2897e00640 1 -- 172.21.15.83:0/4219649169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c005740 con 0x7f287c0073e0 2024-04-25T00:11:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.311+0000 7f2897e00640 1 -- 172.21.15.83:0/4219649169 reap_dead start 2024-04-25T00:11:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.311+0000 7f2897e00640 1 --2- 172.21.15.83:0/4219649169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c0073e0 0x7f287c00dbe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2898068bb0 tx=0x7f289001d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e97032bfe74e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.311+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289000a410 con 0x7f287c0073e0 2024-04-25T00:11:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.311+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28900ac640 con 0x7f287c0073e0 2024-04-25T00:11:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.311+0000 7f2896a00640 1 -- 172.21.15.83:0/4219649169 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28900aca30 con 0x7f287c0073e0 2024-04-25T00:11:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.311+0000 7f289e200640 1 -- 172.21.15.83:0/4219649169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c0073e0 msgr2=0x7f287c00dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.311+0000 7f289e200640 1 --2- 172.21.15.83:0/4219649169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c0073e0 0x7f287c00dbe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2898068bb0 tx=0x7f289001d2a0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.311+0000 7f289e200640 1 -- 172.21.15.83:0/4219649169 shutdown_connections 2024-04-25T00:11:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.311+0000 7f289e200640 1 --2- 172.21.15.83:0/4219649169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287c0073e0 0x7f287c00dbe0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.311+0000 7f289e200640 1 -- 172.21.15.83:0/4219649169 >> 172.21.15.83:0/4219649169 conn(0x7f28980b1830 msgr2=0x7f28980b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.311+0000 7f289e200640 1 -- 172.21.15.83:0/4219649169 shutdown_connections 2024-04-25T00:11:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.311+0000 7f289e200640 1 -- 172.21.15.83:0/4219649169 wait complete. 2024-04-25T00:11:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f289e200640 1 Processor -- start 2024-04-25T00:11:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f289e200640 1 -- start start 2024-04-25T00:11:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f289e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c0073e0 0x7f289815a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f289e200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28981f7e10 0x7f28981450f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f289e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28980cafe0 0x7f2898145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f289e200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28981557c0 con 0x7f28980cafe0 2024-04-25T00:11:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f289e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2898070f30 con 0x7f287c0073e0 2024-04-25T00:11:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f289e200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2898155940 con 0x7f28981f7e10 2024-04-25T00:11:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f2897e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28980cafe0 0x7f2898145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f2897e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28980cafe0 0x7f2898145630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48580/0 (socket says 172.21.15.83:48580) 2024-04-25T00:11:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f2897e00640 1 -- 172.21.15.83:0/3967275543 learned_addr learned my addr 172.21.15.83:0/3967275543 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f2897400640 1 --2- 172.21.15.83:0/3967275543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c0073e0 0x7f289815a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f288ea00640 1 --2- 172.21.15.83:0/3967275543 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28981f7e10 0x7f28981450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f2897e00640 1 -- 172.21.15.83:0/3967275543 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28981f7e10 msgr2=0x7f28981450f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f2897e00640 1 --2- 172.21.15.83:0/3967275543 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28981f7e10 0x7f28981450f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f2897e00640 1 -- 172.21.15.83:0/3967275543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c0073e0 msgr2=0x7f289815a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f2897e00640 1 --2- 172.21.15.83:0/3967275543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c0073e0 0x7f289815a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f2897e00640 1 -- 172.21.15.83:0/3967275543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28981f7370 con 0x7f28980cafe0 2024-04-25T00:11:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f2897e00640 1 --2- 172.21.15.83:0/3967275543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28980cafe0 0x7f2898145630 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f289001db40 tx=0x7f287c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:45.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f288fe00640 1 -- 172.21.15.83:0/3967275543 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28900ac4a0 con 0x7f28980cafe0 2024-04-25T00:11:45.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f288fe00640 1 -- 172.21.15.83:0/3967275543 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2890001a00 con 0x7f28980cafe0 2024-04-25T00:11:45.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f289e200640 1 -- 172.21.15.83:0/3967275543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2898157650 con 0x7f28980cafe0 2024-04-25T00:11:45.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f288fe00640 1 -- 172.21.15.83:0/3967275543 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28900ac9f0 con 0x7f28980cafe0 2024-04-25T00:11:45.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f289e200640 1 -- 172.21.15.83:0/3967275543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28981f7610 con 0x7f28980cafe0 2024-04-25T00:11:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.315+0000 7f289e200640 1 -- 172.21.15.83:0/3967275543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28980737f0 con 0x7f28980cafe0 2024-04-25T00:11:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.319+0000 7f288fe00640 1 -- 172.21.15.83:0/3967275543 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f28900b7470 con 0x7f28980cafe0 2024-04-25T00:11:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.319+0000 7f288fe00640 1 --2- 172.21.15.83:0/3967275543 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28740678c0 0x7f2874069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.319+0000 7f288fe00640 1 -- 172.21.15.83:0/3967275543 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f289011f3b0 con 0x7f28980cafe0 2024-04-25T00:11:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.319+0000 7f2897400640 1 --2- 172.21.15.83:0/3967275543 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28740678c0 0x7f2874069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.319+0000 7f2897400640 1 --2- 172.21.15.83:0/3967275543 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28740678c0 0x7f2874069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f28880093f0 tx=0x7f2888009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:45.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.323+0000 7f288fe00640 1 -- 172.21.15.83:0/3967275543 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f28900bc060 con 0x7f28980cafe0 2024-04-25T00:11:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.479+0000 7f289e200640 1 -- 172.21.15.83:0/3967275543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f289804f5f0 con 0x7f28980cafe0 2024-04-25T00:11:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.479+0000 7f288fe00640 1 -- 172.21.15.83:0/3967275543 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7f2890016fb0 con 0x7f28980cafe0 2024-04-25T00:11:45.486 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:45.486 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.483+0000 7f289e200640 1 -- 172.21.15.83:0/3967275543 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28740678c0 msgr2=0x7f2874069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.483+0000 7f289e200640 1 --2- 172.21.15.83:0/3967275543 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28740678c0 0x7f2874069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f28880093f0 tx=0x7f2888009190 comp rx=0 tx=0).stop 2024-04-25T00:11:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.483+0000 7f289e200640 1 -- 172.21.15.83:0/3967275543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28980cafe0 msgr2=0x7f2898145630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.483+0000 7f289e200640 1 --2- 172.21.15.83:0/3967275543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28980cafe0 0x7f2898145630 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f289001db40 tx=0x7f287c007a50 comp rx=0 tx=0).stop 2024-04-25T00:11:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.483+0000 7f289e200640 1 -- 172.21.15.83:0/3967275543 shutdown_connections 2024-04-25T00:11:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.483+0000 7f289e200640 1 --2- 172.21.15.83:0/3967275543 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28740678c0 0x7f2874069d80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.483+0000 7f289e200640 1 --2- 172.21.15.83:0/3967275543 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28981f7e10 0x7f28981450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.483+0000 7f289e200640 1 --2- 172.21.15.83:0/3967275543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28980cafe0 0x7f2898145630 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.483+0000 7f289e200640 1 --2- 172.21.15.83:0/3967275543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287c0073e0 0x7f289815a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.483+0000 7f289e200640 1 -- 172.21.15.83:0/3967275543 >> 172.21.15.83:0/3967275543 conn(0x7f28980b1830 msgr2=0x7f289815c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.483+0000 7f289e200640 1 -- 172.21.15.83:0/3967275543 shutdown_connections 2024-04-25T00:11:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.483+0000 7f289e200640 1 -- 172.21.15.83:0/3967275543 wait complete. 2024-04-25T00:11:45.508 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:11:04.621429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:11:05.189716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:11:08.902215+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:11:45.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:11:45.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd897800640 1 Processor -- start 2024-04-25T00:11:45.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd897800640 1 -- start start 2024-04-25T00:11:45.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd897800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd890159810 0x7fd8900cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd897800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd89006e950 0x7fd89015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd897800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8900ce000 con 0x7fd89015fe50 2024-04-25T00:11:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd897800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd89006e210 con 0x7fd890159810 2024-04-25T00:11:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd897800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd89006da00 con 0x7fd89006e950 2024-04-25T00:11:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd894e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd890159810 0x7fd8900cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd894e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd890159810 0x7fd8900cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48592/0 (socket says 172.21.15.83:48592) 2024-04-25T00:11:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd894e00640 1 -- 172.21.15.83:0/772302663 learned_addr learned my addr 172.21.15.83:0/772302663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:45.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd887e00640 1 --2- 172.21.15.83:0/772302663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd89006e950 0x7fd89015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd894e00640 1 -- 172.21.15.83:0/772302663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd89006e950 msgr2=0x7fd89015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd894e00640 1 --2- 172.21.15.83:0/772302663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd89006e950 0x7fd89015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd894e00640 1 -- 172.21.15.83:0/772302663 >> v1:172.21.15.83:6789/0 conn(0x7fd89015fe50 legacy=0x7fd89015c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd894e00640 1 -- 172.21.15.83:0/772302663 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89006eec0 con 0x7fd890159810 2024-04-25T00:11:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.643+0000 7fd88fe00640 1 -- 172.21.15.83:0/772302663 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3333461437 0 0) 0x7fd8900ce000 con 0x7fd89015fe50 2024-04-25T00:11:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.647+0000 7fd894e00640 1 --2- 172.21.15.83:0/772302663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd890159810 0x7fd8900cc4a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd880009870 tx=0x7fd88001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508f1cd6852172bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:45.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.647+0000 7fd88fe00640 1 -- 172.21.15.83:0/772302663 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8800b1070 con 0x7fd890159810 2024-04-25T00:11:45.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.647+0000 7fd88fe00640 1 -- 172.21.15.83:0/772302663 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8800a8050 con 0x7fd890159810 2024-04-25T00:11:45.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.647+0000 7fd88fe00640 1 -- 172.21.15.83:0/772302663 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8800ac860 con 0x7fd890159810 2024-04-25T00:11:45.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.647+0000 7fd897800640 1 -- 172.21.15.83:0/772302663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd890159810 msgr2=0x7fd8900cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.647+0000 7fd897800640 1 --2- 172.21.15.83:0/772302663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd890159810 0x7fd8900cc4a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd880009870 tx=0x7fd88001c4d0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.647+0000 7fd897800640 1 -- 172.21.15.83:0/772302663 shutdown_connections 2024-04-25T00:11:45.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.647+0000 7fd897800640 1 --2- 172.21.15.83:0/772302663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd89006e950 0x7fd89015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.647+0000 7fd897800640 1 --2- 172.21.15.83:0/772302663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd890159810 0x7fd8900cc4a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.647+0000 7fd897800640 1 -- 172.21.15.83:0/772302663 >> 172.21.15.83:0/772302663 conn(0x7fd8900b1830 msgr2=0x7fd8900b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:45.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.647+0000 7fd897800640 1 -- 172.21.15.83:0/772302663 shutdown_connections 2024-04-25T00:11:45.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.647+0000 7fd897800640 1 -- 172.21.15.83:0/772302663 wait complete. 2024-04-25T00:11:45.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd897800640 1 Processor -- start 2024-04-25T00:11:45.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd897800640 1 -- start start 2024-04-25T00:11:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd897800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89006e950 0x7fd8901507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd897800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd890159810 0x7fd8901553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd897800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd89015fe50 0x7fd8901558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd897800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd890159d70 con 0x7fd89006e950 2024-04-25T00:11:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd897800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd890159bf0 con 0x7fd890159810 2024-04-25T00:11:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd897800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd890070da0 con 0x7fd89015fe50 2024-04-25T00:11:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd894e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89006e950 0x7fd8901507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd894e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89006e950 0x7fd8901507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48604/0 (socket says 172.21.15.83:48604) 2024-04-25T00:11:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd894e00640 1 -- 172.21.15.83:0/3667363631 learned_addr learned my addr 172.21.15.83:0/3667363631 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd887e00640 1 --2- 172.21.15.83:0/3667363631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd890159810 0x7fd8901553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd895800640 1 --2- 172.21.15.83:0/3667363631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd89015fe50 0x7fd8901558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd894e00640 1 -- 172.21.15.83:0/3667363631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd89015fe50 msgr2=0x7fd8901558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd894e00640 1 --2- 172.21.15.83:0/3667363631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd89015fe50 0x7fd8901558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd894e00640 1 -- 172.21.15.83:0/3667363631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd890159810 msgr2=0x7fd8901553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd894e00640 1 --2- 172.21.15.83:0/3667363631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd890159810 0x7fd8901553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd894e00640 1 -- 172.21.15.83:0/3667363631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd890165c70 con 0x7fd89006e950 2024-04-25T00:11:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd894e00640 1 --2- 172.21.15.83:0/3667363631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89006e950 0x7fd8901507a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd88001a100 tx=0x7fd88001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd88d600640 1 -- 172.21.15.83:0/3667363631 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8800b1040 con 0x7fd89006e950 2024-04-25T00:11:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd88d600640 1 -- 172.21.15.83:0/3667363631 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8800bd070 con 0x7fd89006e950 2024-04-25T00:11:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd897800640 1 -- 172.21.15.83:0/3667363631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd890165f10 con 0x7fd89006e950 2024-04-25T00:11:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd88d600640 1 -- 172.21.15.83:0/3667363631 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8800b7660 con 0x7fd89006e950 2024-04-25T00:11:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd897800640 1 -- 172.21.15.83:0/3667363631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89015bc30 con 0x7fd89006e950 2024-04-25T00:11:45.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.651+0000 7fd897800640 1 -- 172.21.15.83:0/3667363631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd89004f510 con 0x7fd89006e950 2024-04-25T00:11:45.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.655+0000 7fd88d600640 1 -- 172.21.15.83:0/3667363631 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd8800b7800 con 0x7fd89006e950 2024-04-25T00:11:45.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.655+0000 7fd88d600640 1 --2- 172.21.15.83:0/3667363631 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd86c0678c0 0x7fd86c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.655+0000 7fd88d600640 1 -- 172.21.15.83:0/3667363631 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd8801202d0 con 0x7fd89006e950 2024-04-25T00:11:45.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.655+0000 7fd887e00640 1 --2- 172.21.15.83:0/3667363631 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd86c0678c0 0x7fd86c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.655+0000 7fd887e00640 1 --2- 172.21.15.83:0/3667363631 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd86c0678c0 0x7fd86c069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd8900b2410 tx=0x7fd878009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:45.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.659+0000 7fd88d600640 1 -- 172.21.15.83:0/3667363631 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd880016150 con 0x7fd89006e950 2024-04-25T00:11:45.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.811+0000 7fd897800640 1 -- 172.21.15.83:0/3667363631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd890072c90 con 0x7fd89006e950 2024-04-25T00:11:45.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.815+0000 7fd88d600640 1 -- 172.21.15.83:0/3667363631 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fd8800f0b50 con 0x7fd89006e950 2024-04-25T00:11:45.818 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:11:45.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.815+0000 7fd897800640 1 -- 172.21.15.83:0/3667363631 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd86c0678c0 msgr2=0x7fd86c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.815+0000 7fd897800640 1 --2- 172.21.15.83:0/3667363631 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd86c0678c0 0x7fd86c069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd8900b2410 tx=0x7fd878009190 comp rx=0 tx=0).stop 2024-04-25T00:11:45.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.815+0000 7fd897800640 1 -- 172.21.15.83:0/3667363631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89006e950 msgr2=0x7fd8901507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.815+0000 7fd897800640 1 --2- 172.21.15.83:0/3667363631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89006e950 0x7fd8901507a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd88001a100 tx=0x7fd88001ad10 comp rx=0 tx=0).stop 2024-04-25T00:11:45.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.815+0000 7fd897800640 1 -- 172.21.15.83:0/3667363631 shutdown_connections 2024-04-25T00:11:45.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.815+0000 7fd897800640 1 --2- 172.21.15.83:0/3667363631 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd86c0678c0 0x7fd86c069d80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.815+0000 7fd897800640 1 --2- 172.21.15.83:0/3667363631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd89015fe50 0x7fd8901558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.815+0000 7fd897800640 1 --2- 172.21.15.83:0/3667363631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd890159810 0x7fd8901553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.815+0000 7fd897800640 1 --2- 172.21.15.83:0/3667363631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89006e950 0x7fd8901507a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.815+0000 7fd897800640 1 -- 172.21.15.83:0/3667363631 >> 172.21.15.83:0/3667363631 conn(0x7fd8900b1830 msgr2=0x7fd89015e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:45.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.815+0000 7fd897800640 1 -- 172.21.15.83:0/3667363631 shutdown_connections 2024-04-25T00:11:45.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.815+0000 7fd897800640 1 -- 172.21.15.83:0/3667363631 wait complete. 2024-04-25T00:11:45.840 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:11:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c68200640 1 Processor -- start 2024-04-25T00:11:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c68200640 1 -- start start 2024-04-25T00:11:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c68200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c6015f140 0x7f9c600b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c68200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c6006b1e0 0x7f9c6006b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c68200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c6015e6a0 con 0x7f9c60155130 2024-04-25T00:11:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c68200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9c6006bb70 con 0x7f9c6006b1e0 2024-04-25T00:11:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c68200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9c60070420 con 0x7f9c6015f140 2024-04-25T00:11:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c65800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9c60155130 0x7f9c6006fc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51780/0 (socket says 172.21.15.83:51780) 2024-04-25T00:11:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c65800640 1 -- 172.21.15.83:0/3453363638 learned_addr learned my addr 172.21.15.83:0/3453363638 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c64e00640 1 --2- 172.21.15.83:0/3453363638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c6015f140 0x7f9c600b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c66200640 1 --2- 172.21.15.83:0/3453363638 >> v2:172.21.15.86:3300/0 conn(0x7f9c6006b1e0 0x7f9c6006b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c5fe00640 1 -- 172.21.15.83:0/3453363638 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2094138208 0 0) 0x7f9c6015e6a0 con 0x7f9c60155130 2024-04-25T00:11:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c5fe00640 1 -- 172.21.15.83:0/3453363638 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c48003530 con 0x7f9c60155130 2024-04-25T00:11:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c64e00640 1 -- 172.21.15.83:0/3453363638 >> v2:172.21.15.86:3300/0 conn(0x7f9c6006b1e0 msgr2=0x7f9c6006b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c64e00640 1 --2- 172.21.15.83:0/3453363638 >> v2:172.21.15.86:3300/0 conn(0x7f9c6006b1e0 0x7f9c6006b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c64e00640 1 -- 172.21.15.83:0/3453363638 >> v1:172.21.15.83:6789/0 conn(0x7f9c60155130 legacy=0x7f9c6006fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.975+0000 7f9c64e00640 1 -- 172.21.15.83:0/3453363638 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6015e940 con 0x7f9c6015f140 2024-04-25T00:11:45.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.979+0000 7f9c64e00640 1 --2- 172.21.15.83:0/3453363638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c6015f140 0x7f9c600b02a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0097c0 tx=0x7f9c4c01cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1dbd953e9c69881 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:45.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.979+0000 7f9c5fe00640 1 -- 172.21.15.83:0/3453363638 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0b1070 con 0x7f9c6015f140 2024-04-25T00:11:45.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.979+0000 7f9c5fe00640 1 -- 172.21.15.83:0/3453363638 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c4c0ac410 con 0x7f9c6015f140 2024-04-25T00:11:45.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.979+0000 7f9c5fe00640 1 -- 172.21.15.83:0/3453363638 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0ac7e0 con 0x7f9c6015f140 2024-04-25T00:11:45.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.979+0000 7f9c68200640 1 -- 172.21.15.83:0/3453363638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c6015f140 msgr2=0x7f9c600b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.979+0000 7f9c68200640 1 --2- 172.21.15.83:0/3453363638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c6015f140 0x7f9c600b02a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0097c0 tx=0x7f9c4c01cf00 comp rx=0 tx=0).stop 2024-04-25T00:11:45.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.979+0000 7f9c68200640 1 -- 172.21.15.83:0/3453363638 shutdown_connections 2024-04-25T00:11:45.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.979+0000 7f9c68200640 1 --2- 172.21.15.83:0/3453363638 >> v2:172.21.15.86:3300/0 conn(0x7f9c6006b1e0 0x7f9c6006b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.979+0000 7f9c68200640 1 --2- 172.21.15.83:0/3453363638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c6015f140 0x7f9c600b02a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.979+0000 7f9c68200640 1 -- 172.21.15.83:0/3453363638 >> 172.21.15.83:0/3453363638 conn(0x7f9c600b2950 msgr2=0x7f9c600b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:45.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.979+0000 7f9c68200640 1 -- 172.21.15.83:0/3453363638 shutdown_connections 2024-04-25T00:11:45.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.979+0000 7f9c68200640 1 -- 172.21.15.83:0/3453363638 wait complete. 2024-04-25T00:11:45.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c68200640 1 Processor -- start 2024-04-25T00:11:45.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c68200640 1 -- start start 2024-04-25T00:11:45.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c68200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c6006b1e0 0x7f9c601544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c68200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c60155130 0x7f9c600b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c68200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c6015f140 0x7f9c6013c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c68200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c600729c0 con 0x7f9c60155130 2024-04-25T00:11:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c68200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c60072840 con 0x7f9c6015f140 2024-04-25T00:11:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c68200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9c60072b40 con 0x7f9c6006b1e0 2024-04-25T00:11:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c64e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c60155130 0x7f9c600b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c64e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c60155130 0x7f9c600b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48616/0 (socket says 172.21.15.83:48616) 2024-04-25T00:11:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c64e00640 1 -- 172.21.15.83:0/29038841 learned_addr learned my addr 172.21.15.83:0/29038841 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c64e00640 1 -- 172.21.15.83:0/29038841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c6006b1e0 msgr2=0x7f9c601544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c64e00640 1 --2- 172.21.15.83:0/29038841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c6006b1e0 0x7f9c601544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c64e00640 1 -- 172.21.15.83:0/29038841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c6015f140 msgr2=0x7f9c6013c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c64e00640 1 --2- 172.21.15.83:0/29038841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c6015f140 0x7f9c6013c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c64e00640 1 -- 172.21.15.83:0/29038841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c60154c30 con 0x7f9c60155130 2024-04-25T00:11:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c64e00640 1 --2- 172.21.15.83:0/29038841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c60155130 0x7f9c600b5200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0af040 tx=0x7f9c4c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:45.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c5d600640 1 -- 172.21.15.83:0/29038841 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0b1040 con 0x7f9c60155130 2024-04-25T00:11:45.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c5d600640 1 -- 172.21.15.83:0/29038841 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c4c0bb070 con 0x7f9c60155130 2024-04-25T00:11:45.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c5d600640 1 -- 172.21.15.83:0/29038841 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0b5410 con 0x7f9c60155130 2024-04-25T00:11:45.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c68200640 1 -- 172.21.15.83:0/29038841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6014c300 con 0x7f9c60155130 2024-04-25T00:11:45.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c68200640 1 -- 172.21.15.83:0/29038841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6014c8c0 con 0x7f9c60155130 2024-04-25T00:11:45.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.983+0000 7f9c68200640 1 -- 172.21.15.83:0/29038841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c6004f510 con 0x7f9c60155130 2024-04-25T00:11:45.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.987+0000 7f9c5d600640 1 -- 172.21.15.83:0/29038841 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9c4c0b55b0 con 0x7f9c60155130 2024-04-25T00:11:45.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.987+0000 7f9c5d600640 1 --2- 172.21.15.83:0/29038841 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9c400678f0 0x7f9c40069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:45.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.987+0000 7f9c5d600640 1 -- 172.21.15.83:0/29038841 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9c4c11fa40 con 0x7f9c60155130 2024-04-25T00:11:45.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.987+0000 7f9c65800640 1 --2- 172.21.15.83:0/29038841 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9c400678f0 0x7f9c40069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:45.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.987+0000 7f9c65800640 1 --2- 172.21.15.83:0/29038841 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9c400678f0 0x7f9c40069db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9c54007fd0 tx=0x7f9c54007900 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:45.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:45.991+0000 7f9c5d600640 1 -- 172.21.15.83:0/29038841 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9c4c016090 con 0x7f9c60155130 2024-04-25T00:11:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.147+0000 7f9c68200640 1 -- 172.21.15.83:0/29038841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9c601556b0 con 0x7f9c60155130 2024-04-25T00:11:46.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.147+0000 7f9c5d600640 1 -- 172.21.15.83:0/29038841 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f9c4c0f02c0 con 0x7f9c60155130 2024-04-25T00:11:46.152 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.151+0000 7f9c68200640 1 -- 172.21.15.83:0/29038841 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9c400678f0 msgr2=0x7f9c40069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.151+0000 7f9c68200640 1 --2- 172.21.15.83:0/29038841 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9c400678f0 0x7f9c40069db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9c54007fd0 tx=0x7f9c54007900 comp rx=0 tx=0).stop 2024-04-25T00:11:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.151+0000 7f9c68200640 1 -- 172.21.15.83:0/29038841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c60155130 msgr2=0x7f9c600b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.151+0000 7f9c68200640 1 --2- 172.21.15.83:0/29038841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c60155130 0x7f9c600b5200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0af040 tx=0x7f9c4c019240 comp rx=0 tx=0).stop 2024-04-25T00:11:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.151+0000 7f9c68200640 1 -- 172.21.15.83:0/29038841 shutdown_connections 2024-04-25T00:11:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.151+0000 7f9c68200640 1 --2- 172.21.15.83:0/29038841 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9c400678f0 0x7f9c40069db0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.151+0000 7f9c68200640 1 --2- 172.21.15.83:0/29038841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c6015f140 0x7f9c6013c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.151+0000 7f9c68200640 1 --2- 172.21.15.83:0/29038841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c60155130 0x7f9c600b5200 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.151+0000 7f9c68200640 1 --2- 172.21.15.83:0/29038841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c6006b1e0 0x7f9c601544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.151+0000 7f9c68200640 1 -- 172.21.15.83:0/29038841 >> 172.21.15.83:0/29038841 conn(0x7f9c600b2950 msgr2=0x7f9c60161b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.151+0000 7f9c68200640 1 -- 172.21.15.83:0/29038841 shutdown_connections 2024-04-25T00:11:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.151+0000 7f9c68200640 1 -- 172.21.15.83:0/29038841 wait complete. 2024-04-25T00:11:46.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:11:46.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.307+0000 7fd6b3400640 1 Processor -- start 2024-04-25T00:11:46.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.307+0000 7fd6b3400640 1 -- start start 2024-04-25T00:11:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.307+0000 7fd6b3400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6ac1555c0 0x7fd6ac15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.307+0000 7fd6b3400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd6ac06e4c0 0x7fd6ac06e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.307+0000 7fd6b3400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6ac15c560 con 0x7fd6ac1551e0 2024-04-25T00:11:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.307+0000 7fd6b3400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd6ac06ee70 con 0x7fd6ac06e4c0 2024-04-25T00:11:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.307+0000 7fd6b3400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ac072460 con 0x7fd6ac1555c0 2024-04-25T00:11:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.311+0000 7fd6b0a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd6ac1551e0 0x7fd6ac071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51782/0 (socket says 172.21.15.83:51782) 2024-04-25T00:11:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.311+0000 7fd6b0a00640 1 -- 172.21.15.83:0/600562372 learned_addr learned my addr 172.21.15.83:0/600562372 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.311+0000 7fd6b1400640 1 --2- 172.21.15.83:0/600562372 >> v2:172.21.15.83:3300/0 conn(0x7fd6ac06e4c0 0x7fd6ac06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.311+0000 7fd6a3e00640 1 --2- 172.21.15.83:0/600562372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6ac1555c0 0x7fd6ac15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.311+0000 7fd6abe00640 1 -- 172.21.15.83:0/600562372 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4122148059 0 0) 0x7fd6ac15c560 con 0x7fd6ac1551e0 2024-04-25T00:11:46.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.311+0000 7fd6b1400640 1 -- 172.21.15.83:0/600562372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6ac1555c0 msgr2=0x7fd6ac15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:46.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.311+0000 7fd6b1400640 1 --2- 172.21.15.83:0/600562372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6ac1555c0 0x7fd6ac15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.311+0000 7fd6b1400640 1 -- 172.21.15.83:0/600562372 >> v1:172.21.15.83:6789/0 conn(0x7fd6ac1551e0 legacy=0x7fd6ac071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.311+0000 7fd6b1400640 1 -- 172.21.15.83:0/600562372 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac15c7d0 con 0x7fd6ac06e4c0 2024-04-25T00:11:46.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.311+0000 7fd6b1400640 1 --2- 172.21.15.83:0/600562372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac06e4c0 0x7fd6ac06e930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009d90 tx=0x7fd6a401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a864e50e4f031f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:46.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.311+0000 7fd6abe00640 1 -- 172.21.15.83:0/600562372 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a40b2070 con 0x7fd6ac06e4c0 2024-04-25T00:11:46.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.311+0000 7fd6abe00640 1 -- 172.21.15.83:0/600562372 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a40a5940 con 0x7fd6ac06e4c0 2024-04-25T00:11:46.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.311+0000 7fd6abe00640 1 -- 172.21.15.83:0/600562372 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a40a5d10 con 0x7fd6ac06e4c0 2024-04-25T00:11:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 -- 172.21.15.83:0/600562372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac06e4c0 msgr2=0x7fd6ac06e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 --2- 172.21.15.83:0/600562372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac06e4c0 0x7fd6ac06e930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009d90 tx=0x7fd6a401c970 comp rx=0 tx=0).stop 2024-04-25T00:11:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 -- 172.21.15.83:0/600562372 shutdown_connections 2024-04-25T00:11:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 --2- 172.21.15.83:0/600562372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac06e4c0 0x7fd6ac06e930 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 --2- 172.21.15.83:0/600562372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6ac1555c0 0x7fd6ac15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 -- 172.21.15.83:0/600562372 >> 172.21.15.83:0/600562372 conn(0x7fd6ac0b1830 msgr2=0x7fd6ac0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 -- 172.21.15.83:0/600562372 shutdown_connections 2024-04-25T00:11:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 -- 172.21.15.83:0/600562372 wait complete. 2024-04-25T00:11:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 Processor -- start 2024-04-25T00:11:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 -- start start 2024-04-25T00:11:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6ac06e4c0 0x7fd6ac0a9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6ac1551e0 0x7fd6ac0a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac1555c0 0x7fd6ac0aa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac15db70 con 0x7fd6ac1555c0 2024-04-25T00:11:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac15d9f0 con 0x7fd6ac06e4c0 2024-04-25T00:11:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b3400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ac15dcf0 con 0x7fd6ac1551e0 2024-04-25T00:11:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac1555c0 0x7fd6ac0aa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac1555c0 0x7fd6ac0aa3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48640/0 (socket says 172.21.15.83:48640) 2024-04-25T00:11:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b1400640 1 -- 172.21.15.83:0/2323712082 learned_addr learned my addr 172.21.15.83:0/2323712082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6a3e00640 1 --2- 172.21.15.83:0/2323712082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6ac1551e0 0x7fd6ac0a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b0a00640 1 --2- 172.21.15.83:0/2323712082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6ac06e4c0 0x7fd6ac0a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b1400640 1 -- 172.21.15.83:0/2323712082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6ac1551e0 msgr2=0x7fd6ac0a9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b1400640 1 --2- 172.21.15.83:0/2323712082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6ac1551e0 0x7fd6ac0a9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b1400640 1 -- 172.21.15.83:0/2323712082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6ac06e4c0 msgr2=0x7fd6ac0a9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b1400640 1 --2- 172.21.15.83:0/2323712082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6ac06e4c0 0x7fd6ac0a9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.315+0000 7fd6b1400640 1 -- 172.21.15.83:0/2323712082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac0b3300 con 0x7fd6ac1555c0 2024-04-25T00:11:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.319+0000 7fd6b1400640 1 --2- 172.21.15.83:0/2323712082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac1555c0 0x7fd6ac0aa3a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd6a401b020 tx=0x7fd6a4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.319+0000 7fd6a9600640 1 -- 172.21.15.83:0/2323712082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a40b2050 con 0x7fd6ac1555c0 2024-04-25T00:11:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.319+0000 7fd6a9600640 1 -- 172.21.15.83:0/2323712082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a40be070 con 0x7fd6ac1555c0 2024-04-25T00:11:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.319+0000 7fd6a9600640 1 -- 172.21.15.83:0/2323712082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a40adad0 con 0x7fd6ac1555c0 2024-04-25T00:11:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.319+0000 7fd6b3400640 1 -- 172.21.15.83:0/2323712082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ac0b4c70 con 0x7fd6ac1555c0 2024-04-25T00:11:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.319+0000 7fd6b3400640 1 -- 172.21.15.83:0/2323712082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ac0b3860 con 0x7fd6ac1555c0 2024-04-25T00:11:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.319+0000 7fd6b3400640 1 -- 172.21.15.83:0/2323712082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ac04f510 con 0x7fd6ac1555c0 2024-04-25T00:11:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.319+0000 7fd6a9600640 1 -- 172.21.15.83:0/2323712082 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd6a40adc70 con 0x7fd6ac1555c0 2024-04-25T00:11:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.319+0000 7fd6a9600640 1 --2- 172.21.15.83:0/2323712082 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd6880678c0 0x7fd688069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.319+0000 7fd6a9600640 1 -- 172.21.15.83:0/2323712082 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd6a4121210 con 0x7fd6ac1555c0 2024-04-25T00:11:46.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.319+0000 7fd6b0a00640 1 --2- 172.21.15.83:0/2323712082 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd6880678c0 0x7fd688069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.323+0000 7fd6b0a00640 1 --2- 172.21.15.83:0/2323712082 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd6880678c0 0x7fd688069d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd69c007fd0 tx=0x7fd69c007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.327+0000 7fd6a9600640 1 -- 172.21.15.83:0/2323712082 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd6ac0b3300 con 0x7fd6ac1555c0 2024-04-25T00:11:46.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.479+0000 7fd6b3400640 1 -- 172.21.15.83:0/2323712082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd6ac0a9700 con 0x7fd6ac1555c0 2024-04-25T00:11:46.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.483+0000 7fd6a9600640 1 -- 172.21.15.83:0/2323712082 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fd6a4016560 con 0x7fd6ac1555c0 2024-04-25T00:11:46.485 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:11:46.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.483+0000 7fd6b3400640 1 -- 172.21.15.83:0/2323712082 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd6880678c0 msgr2=0x7fd688069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.483+0000 7fd6b3400640 1 --2- 172.21.15.83:0/2323712082 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd6880678c0 0x7fd688069d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd69c007fd0 tx=0x7fd69c007d60 comp rx=0 tx=0).stop 2024-04-25T00:11:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.483+0000 7fd6b3400640 1 -- 172.21.15.83:0/2323712082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac1555c0 msgr2=0x7fd6ac0aa3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.483+0000 7fd6b3400640 1 --2- 172.21.15.83:0/2323712082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac1555c0 0x7fd6ac0aa3a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd6a401b020 tx=0x7fd6a4019710 comp rx=0 tx=0).stop 2024-04-25T00:11:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.483+0000 7fd6b3400640 1 -- 172.21.15.83:0/2323712082 shutdown_connections 2024-04-25T00:11:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.483+0000 7fd6b3400640 1 --2- 172.21.15.83:0/2323712082 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd6880678c0 0x7fd688069d80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.483+0000 7fd6b3400640 1 --2- 172.21.15.83:0/2323712082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6ac1551e0 0x7fd6ac0a9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.483+0000 7fd6b3400640 1 --2- 172.21.15.83:0/2323712082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac1555c0 0x7fd6ac0aa3a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.483+0000 7fd6b3400640 1 --2- 172.21.15.83:0/2323712082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6ac06e4c0 0x7fd6ac0a9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.483+0000 7fd6b3400640 1 -- 172.21.15.83:0/2323712082 >> 172.21.15.83:0/2323712082 conn(0x7fd6ac0b1830 msgr2=0x7fd6ac15ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.483+0000 7fd6b3400640 1 -- 172.21.15.83:0/2323712082 shutdown_connections 2024-04-25T00:11:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.483+0000 7fd6b3400640 1 -- 172.21.15.83:0/2323712082 wait complete. 2024-04-25T00:11:46.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:11:46.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.639+0000 7fb70ba00640 1 Processor -- start 2024-04-25T00:11:46.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.639+0000 7fb70ba00640 1 -- start start 2024-04-25T00:11:46.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.643+0000 7fb70ba00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb704159810 0x7fb7040cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.643+0000 7fb70ba00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb704160020 0x7fb7040ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.643+0000 7fb70ba00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7040ce000 con 0x7fb70406e950 2024-04-25T00:11:46.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.643+0000 7fb70ba00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb70406e210 con 0x7fb704160020 2024-04-25T00:11:46.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.643+0000 7fb70ba00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb70406da00 con 0x7fb704159810 2024-04-25T00:11:46.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.643+0000 7fb709a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb704160020 0x7fb7040ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.643+0000 7fb709a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb704160020 0x7fb7040ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48650/0 (socket says 172.21.15.83:48650) 2024-04-25T00:11:46.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.643+0000 7fb709a00640 1 -- 172.21.15.83:0/2675969407 learned_addr learned my addr 172.21.15.83:0/2675969407 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:46.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.643+0000 7fb703e00640 1 --1- 172.21.15.83:0/2675969407 >> v1:172.21.15.83:6789/0 conn(0x7fb70406e950 0x7fb70415f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51792/0 (socket says 172.21.15.83:51792) 2024-04-25T00:11:46.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.643+0000 7fb709000640 1 --2- 172.21.15.83:0/2675969407 >> v2:172.21.15.86:3300/0 conn(0x7fb704159810 0x7fb7040cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.643+0000 7fb709a00640 1 -- 172.21.15.83:0/2675969407 >> v2:172.21.15.86:3300/0 conn(0x7fb704159810 msgr2=0x7fb7040cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:46.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.643+0000 7fb709a00640 1 --2- 172.21.15.83:0/2675969407 >> v2:172.21.15.86:3300/0 conn(0x7fb704159810 0x7fb7040cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.643+0000 7fb709a00640 1 -- 172.21.15.83:0/2675969407 >> v1:172.21.15.83:6789/0 conn(0x7fb70406e950 legacy=0x7fb70415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.643+0000 7fb709a00640 1 -- 172.21.15.83:0/2675969407 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70406eec0 con 0x7fb704160020 2024-04-25T00:11:46.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.647+0000 7fb709a00640 1 --2- 172.21.15.83:0/2675969407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb704160020 0x7fb7040ce4a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb70406db80 tx=0x7fb6fc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c1762c7568ba88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:46.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.647+0000 7fb703400640 1 -- 172.21.15.83:0/2675969407 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0ab660 con 0x7fb704160020 2024-04-25T00:11:46.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.647+0000 7fb703400640 1 -- 172.21.15.83:0/2675969407 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6fc0b0070 con 0x7fb704160020 2024-04-25T00:11:46.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.647+0000 7fb703400640 1 -- 172.21.15.83:0/2675969407 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0aba20 con 0x7fb704160020 2024-04-25T00:11:46.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.647+0000 7fb70ba00640 1 -- 172.21.15.83:0/2675969407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb704160020 msgr2=0x7fb7040ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.647+0000 7fb70ba00640 1 --2- 172.21.15.83:0/2675969407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb704160020 0x7fb7040ce4a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb70406db80 tx=0x7fb6fc01c460 comp rx=0 tx=0).stop 2024-04-25T00:11:46.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.647+0000 7fb70ba00640 1 -- 172.21.15.83:0/2675969407 shutdown_connections 2024-04-25T00:11:46.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.647+0000 7fb70ba00640 1 --2- 172.21.15.83:0/2675969407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb704160020 0x7fb7040ce4a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.647+0000 7fb70ba00640 1 --2- 172.21.15.83:0/2675969407 >> v2:172.21.15.86:3300/0 conn(0x7fb704159810 0x7fb7040cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.647+0000 7fb70ba00640 1 -- 172.21.15.83:0/2675969407 >> 172.21.15.83:0/2675969407 conn(0x7fb7040b1830 msgr2=0x7fb7040b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:46.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.647+0000 7fb70ba00640 1 -- 172.21.15.83:0/2675969407 shutdown_connections 2024-04-25T00:11:46.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.647+0000 7fb70ba00640 1 -- 172.21.15.83:0/2675969407 wait complete. 2024-04-25T00:11:46.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.647+0000 7fb70ba00640 1 Processor -- start 2024-04-25T00:11:46.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.647+0000 7fb70ba00640 1 -- start start 2024-04-25T00:11:46.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb70ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb70406e950 0x7fb70415e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb70ba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb704159810 0x7fb704140700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb70ba00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb704160020 0x7fb7040b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb70ba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb704159d70 con 0x7fb704159810 2024-04-25T00:11:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb70ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb704159bf0 con 0x7fb70406e950 2024-04-25T00:11:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb70ba00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb704070da0 con 0x7fb704160020 2024-04-25T00:11:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb703e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb704159810 0x7fb704140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb703e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb704159810 0x7fb704140700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48654/0 (socket says 172.21.15.83:48654) 2024-04-25T00:11:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb703e00640 1 -- 172.21.15.83:0/4161534948 learned_addr learned my addr 172.21.15.83:0/4161534948 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb709a00640 1 --2- 172.21.15.83:0/4161534948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb704160020 0x7fb7040b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb709000640 1 --2- 172.21.15.83:0/4161534948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb70406e950 0x7fb70415e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb703e00640 1 -- 172.21.15.83:0/4161534948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb704160020 msgr2=0x7fb7040b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb703e00640 1 --2- 172.21.15.83:0/4161534948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb704160020 0x7fb7040b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb703e00640 1 -- 172.21.15.83:0/4161534948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb70406e950 msgr2=0x7fb70415e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb703e00640 1 --2- 172.21.15.83:0/4161534948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb70406e950 0x7fb70415e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb703e00640 1 -- 172.21.15.83:0/4161534948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70415ec20 con 0x7fb704159810 2024-04-25T00:11:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb703e00640 1 --2- 172.21.15.83:0/4161534948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb704159810 0x7fb704140700 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000c370 tx=0x7fb6f000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb700c00640 1 -- 172.21.15.83:0/4161534948 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00074e0 con 0x7fb704159810 2024-04-25T00:11:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb700c00640 1 -- 172.21.15.83:0/4161534948 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f0007680 con 0x7fb704159810 2024-04-25T00:11:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb700c00640 1 -- 172.21.15.83:0/4161534948 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0004a30 con 0x7fb704159810 2024-04-25T00:11:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb70ba00640 1 -- 172.21.15.83:0/4161534948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb704155b20 con 0x7fb704159810 2024-04-25T00:11:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb70ba00640 1 -- 172.21.15.83:0/4161534948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb704154850 con 0x7fb704159810 2024-04-25T00:11:46.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb70ba00640 1 -- 172.21.15.83:0/4161534948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70404f510 con 0x7fb704159810 2024-04-25T00:11:46.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.651+0000 7fb700c00640 1 -- 172.21.15.83:0/4161534948 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb6f001a020 con 0x7fb704159810 2024-04-25T00:11:46.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.655+0000 7fb700c00640 1 --2- 172.21.15.83:0/4161534948 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6e40678c0 0x7fb6e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.655+0000 7fb700c00640 1 -- 172.21.15.83:0/4161534948 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb6f0085760 con 0x7fb704159810 2024-04-25T00:11:46.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.655+0000 7fb709000640 1 --2- 172.21.15.83:0/4161534948 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6e40678c0 0x7fb6e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.655+0000 7fb709000640 1 --2- 172.21.15.83:0/4161534948 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6e40678c0 0x7fb6e4069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009fd0 tx=0x7fb6f4009800 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:46.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.659+0000 7fb700c00640 1 -- 172.21.15.83:0/4161534948 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb6f00557e0 con 0x7fb704159810 2024-04-25T00:11:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.811+0000 7fb70ba00640 1 -- 172.21.15.83:0/4161534948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb704072c90 con 0x7fb704159810 2024-04-25T00:11:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.811+0000 7fb700c00640 1 -- 172.21.15.83:0/4161534948 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fb6f0059ae0 con 0x7fb704159810 2024-04-25T00:11:46.817 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.815+0000 7fb70ba00640 1 -- 172.21.15.83:0/4161534948 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6e40678c0 msgr2=0x7fb6e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.815+0000 7fb70ba00640 1 --2- 172.21.15.83:0/4161534948 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6e40678c0 0x7fb6e4069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009fd0 tx=0x7fb6f4009800 comp rx=0 tx=0).stop 2024-04-25T00:11:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.815+0000 7fb70ba00640 1 -- 172.21.15.83:0/4161534948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb704159810 msgr2=0x7fb704140700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.815+0000 7fb70ba00640 1 --2- 172.21.15.83:0/4161534948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb704159810 0x7fb704140700 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000c370 tx=0x7fb6f000c830 comp rx=0 tx=0).stop 2024-04-25T00:11:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.815+0000 7fb70ba00640 1 -- 172.21.15.83:0/4161534948 shutdown_connections 2024-04-25T00:11:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.815+0000 7fb70ba00640 1 --2- 172.21.15.83:0/4161534948 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6e40678c0 0x7fb6e4069d80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.815+0000 7fb70ba00640 1 --2- 172.21.15.83:0/4161534948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb704160020 0x7fb7040b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.815+0000 7fb70ba00640 1 --2- 172.21.15.83:0/4161534948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb704159810 0x7fb704140700 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.815+0000 7fb70ba00640 1 --2- 172.21.15.83:0/4161534948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb70406e950 0x7fb70415e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.815+0000 7fb70ba00640 1 -- 172.21.15.83:0/4161534948 >> 172.21.15.83:0/4161534948 conn(0x7fb7040b1830 msgr2=0x7fb704158660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.815+0000 7fb70ba00640 1 -- 172.21.15.83:0/4161534948 shutdown_connections 2024-04-25T00:11:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.815+0000 7fb70ba00640 1 -- 172.21.15.83:0/4161534948 wait complete. 2024-04-25T00:11:46.838 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.971+0000 7f2938200640 1 Processor -- start 2024-04-25T00:11:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.971+0000 7f2938200640 1 -- start start 2024-04-25T00:11:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.971+0000 7f2938200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f29300b47e0 0x7f29300b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.971+0000 7f2938200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29300b4c70 con 0x7f29301553e0 2024-04-25T00:11:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.975+0000 7f2938200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f293006ef80 con 0x7f29300b4360 2024-04-25T00:11:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.975+0000 7f2938200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f293015b510 con 0x7f29300b47e0 2024-04-25T00:11:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.975+0000 7f2935800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f29301553e0 0x7f293006e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51802/0 (socket says 172.21.15.83:51802) 2024-04-25T00:11:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.975+0000 7f2935800640 1 -- 172.21.15.83:0/3119917634 learned_addr learned my addr 172.21.15.83:0/3119917634 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.975+0000 7f2936200640 1 --2- 172.21.15.83:0/3119917634 >> v2:172.21.15.86:3300/0 conn(0x7f29300b47e0 0x7f29300b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.975+0000 7f292fe00640 1 -- 172.21.15.83:0/3119917634 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3047864379 0 0) 0x7f29300b4c70 con 0x7f29301553e0 2024-04-25T00:11:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.975+0000 7f292fe00640 1 -- 172.21.15.83:0/3119917634 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2918003530 con 0x7f29301553e0 2024-04-25T00:11:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.975+0000 7f2936200640 1 -- 172.21.15.83:0/3119917634 >> v1:172.21.15.86:6789/0 conn(0x7f29300b4360 legacy=0x7f29300b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.975+0000 7f2936200640 1 -- 172.21.15.83:0/3119917634 >> v1:172.21.15.83:6789/0 conn(0x7f29301553e0 legacy=0x7f293006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.975+0000 7f2936200640 1 -- 172.21.15.83:0/3119917634 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293015b7f0 con 0x7f29300b47e0 2024-04-25T00:11:46.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2936200640 1 --2- 172.21.15.83:0/3119917634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29300b47e0 0x7f29300b3240 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f29280097c0 tx=0x7f292801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4cf2efd68f2667f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:46.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f292fe00640 1 -- 172.21.15.83:0/3119917634 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29280b0070 con 0x7f29300b47e0 2024-04-25T00:11:46.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f292fe00640 1 -- 172.21.15.83:0/3119917634 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29280ab600 con 0x7f29300b47e0 2024-04-25T00:11:46.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f292fe00640 1 -- 172.21.15.83:0/3119917634 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29280ab9d0 con 0x7f29300b47e0 2024-04-25T00:11:46.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 -- 172.21.15.83:0/3119917634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29300b47e0 msgr2=0x7f29300b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 --2- 172.21.15.83:0/3119917634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29300b47e0 0x7f29300b3240 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f29280097c0 tx=0x7f292801c410 comp rx=0 tx=0).stop 2024-04-25T00:11:46.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 -- 172.21.15.83:0/3119917634 shutdown_connections 2024-04-25T00:11:46.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 --2- 172.21.15.83:0/3119917634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29300b47e0 0x7f29300b3240 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 -- 172.21.15.83:0/3119917634 >> 172.21.15.83:0/3119917634 conn(0x7f29300b1830 msgr2=0x7f29300b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:46.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 -- 172.21.15.83:0/3119917634 shutdown_connections 2024-04-25T00:11:46.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 -- 172.21.15.83:0/3119917634 wait complete. 2024-04-25T00:11:46.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 Processor -- start 2024-04-25T00:11:46.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 -- start start 2024-04-25T00:11:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29300b4360 0x7f2930146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29300b47e0 0x7f2930146970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29301553e0 0x7f2930152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2930155940 con 0x7f29301553e0 2024-04-25T00:11:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29301557c0 con 0x7f29300b4360 2024-04-25T00:11:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.979+0000 7f2938200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f293006ebd0 con 0x7f29300b47e0 2024-04-25T00:11:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f2936200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29301553e0 0x7f2930152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f2936200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29301553e0 0x7f2930152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48666/0 (socket says 172.21.15.83:48666) 2024-04-25T00:11:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f2936200640 1 -- 172.21.15.83:0/3707535867 learned_addr learned my addr 172.21.15.83:0/3707535867 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f2935800640 1 --2- 172.21.15.83:0/3707535867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29300b4360 0x7f2930146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f2934e00640 1 --2- 172.21.15.83:0/3707535867 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29300b47e0 0x7f2930146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f2936200640 1 -- 172.21.15.83:0/3707535867 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29300b47e0 msgr2=0x7f2930146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f2936200640 1 --2- 172.21.15.83:0/3707535867 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29300b47e0 0x7f2930146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f2936200640 1 -- 172.21.15.83:0/3707535867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29300b4360 msgr2=0x7f2930146430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f2936200640 1 --2- 172.21.15.83:0/3707535867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29300b4360 0x7f2930146430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f2936200640 1 -- 172.21.15.83:0/3707535867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2930147d20 con 0x7f29301553e0 2024-04-25T00:11:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f2936200640 1 --2- 172.21.15.83:0/3707535867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29301553e0 0x7f2930152520 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f29280ae040 tx=0x7f29280191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:46.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f292d600640 1 -- 172.21.15.83:0/3707535867 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29280b8020 con 0x7f29301553e0 2024-04-25T00:11:46.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f292d600640 1 -- 172.21.15.83:0/3707535867 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29280b4410 con 0x7f29301553e0 2024-04-25T00:11:46.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f2938200640 1 -- 172.21.15.83:0/3707535867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2930153480 con 0x7f29301553e0 2024-04-25T00:11:46.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f292d600640 1 -- 172.21.15.83:0/3707535867 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29280b0040 con 0x7f29301553e0 2024-04-25T00:11:46.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f2938200640 1 -- 172.21.15.83:0/3707535867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293014bcd0 con 0x7f29301553e0 2024-04-25T00:11:46.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f2938200640 1 -- 172.21.15.83:0/3707535867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29300737f0 con 0x7f29301553e0 2024-04-25T00:11:46.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.983+0000 7f292d600640 1 -- 172.21.15.83:0/3707535867 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f29280bb020 con 0x7f29301553e0 2024-04-25T00:11:46.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.987+0000 7f292d600640 1 --2- 172.21.15.83:0/3707535867 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29100678f0 0x7f2910069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:46.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.987+0000 7f292d600640 1 -- 172.21.15.83:0/3707535867 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f292811fb80 con 0x7f29301553e0 2024-04-25T00:11:46.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.987+0000 7f2935800640 1 --2- 172.21.15.83:0/3707535867 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29100678f0 0x7f2910069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.987+0000 7f2935800640 1 --2- 172.21.15.83:0/3707535867 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29100678f0 0x7f2910069db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2924001570 tx=0x7f2924009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:46.987+0000 7f292d600640 1 -- 172.21.15.83:0/3707535867 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f29280c2030 con 0x7f29301553e0 2024-04-25T00:11:47.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.147+0000 7f2938200640 1 -- 172.21.15.83:0/3707535867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f293015e140 con 0x7f29301553e0 2024-04-25T00:11:47.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.147+0000 7f292d600640 1 -- 172.21.15.83:0/3707535867 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18430 (secure 0 0 0) 0x7f2928016090 con 0x7f29301553e0 2024-04-25T00:11:47.152 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:47.152 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:11.262552+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:47.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.151+0000 7f2938200640 1 -- 172.21.15.83:0/3707535867 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29100678f0 msgr2=0x7f2910069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.151+0000 7f2938200640 1 --2- 172.21.15.83:0/3707535867 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29100678f0 0x7f2910069db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2924001570 tx=0x7f2924009040 comp rx=0 tx=0).stop 2024-04-25T00:11:47.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.151+0000 7f2938200640 1 -- 172.21.15.83:0/3707535867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29301553e0 msgr2=0x7f2930152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.151+0000 7f2938200640 1 --2- 172.21.15.83:0/3707535867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29301553e0 0x7f2930152520 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f29280ae040 tx=0x7f29280191b0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.151+0000 7f2938200640 1 -- 172.21.15.83:0/3707535867 shutdown_connections 2024-04-25T00:11:47.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.151+0000 7f2938200640 1 --2- 172.21.15.83:0/3707535867 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29100678f0 0x7f2910069db0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.151+0000 7f2938200640 1 --2- 172.21.15.83:0/3707535867 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29300b47e0 0x7f2930146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.151+0000 7f2938200640 1 --2- 172.21.15.83:0/3707535867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29301553e0 0x7f2930152520 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.151+0000 7f2938200640 1 --2- 172.21.15.83:0/3707535867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29300b4360 0x7f2930146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.151+0000 7f2938200640 1 -- 172.21.15.83:0/3707535867 >> 172.21.15.83:0/3707535867 conn(0x7f29300b1830 msgr2=0x7f2930159390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:47.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.151+0000 7f2938200640 1 -- 172.21.15.83:0/3707535867 shutdown_connections 2024-04-25T00:11:47.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.151+0000 7f2938200640 1 -- 172.21.15.83:0/3707535867 wait complete. 2024-04-25T00:11:47.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:11:04.621429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:11:05.189716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:11:08.902215+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:11:47.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:11:47.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a4400640 1 Processor -- start 2024-04-25T00:11:47.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a4400640 1 -- start start 2024-04-25T00:11:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a4400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f239c159810 0x7f239c0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a4400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f239c06e950 0x7f239c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a4400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f239c0ce000 con 0x7f239c15fe50 2024-04-25T00:11:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a4400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f239c06e210 con 0x7f239c159810 2024-04-25T00:11:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a4400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f239c1602a0 con 0x7f239c06e950 2024-04-25T00:11:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a1a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f239c159810 0x7f239c0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a1a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f239c159810 0x7f239c0cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48674/0 (socket says 172.21.15.83:48674) 2024-04-25T00:11:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a1a00640 1 -- 172.21.15.83:0/3901590059 learned_addr learned my addr 172.21.15.83:0/3901590059 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a1000640 1 --2- 172.21.15.83:0/3901590059 >> v2:172.21.15.86:3300/0 conn(0x7f239c06e950 0x7f239c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a1a00640 1 -- 172.21.15.83:0/3901590059 >> v2:172.21.15.86:3300/0 conn(0x7f239c06e950 msgr2=0x7f239c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:47.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a1a00640 1 --2- 172.21.15.83:0/3901590059 >> v2:172.21.15.86:3300/0 conn(0x7f239c06e950 0x7f239c15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a1a00640 1 -- 172.21.15.83:0/3901590059 >> v1:172.21.15.86:6789/0 conn(0x7f239c15fe50 legacy=0x7f239c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:47.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a1a00640 1 -- 172.21.15.83:0/3901590059 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239c06eec0 con 0x7f239c159810 2024-04-25T00:11:47.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.315+0000 7f23a1000640 1 --2- 172.21.15.83:0/3901590059 >> v2:172.21.15.86:3300/0 conn(0x7f239c06e950 0x7f239c15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:47.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.319+0000 7f23a1a00640 1 --2- 172.21.15.83:0/3901590059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f239c159810 0x7f239c0cc4a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f239c0b4f50 tx=0x7f239001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c539e16a2117c16f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:47.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.319+0000 7f239be00640 1 -- 172.21.15.83:0/3901590059 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23900b0070 con 0x7f239c159810 2024-04-25T00:11:47.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.319+0000 7f239be00640 1 -- 172.21.15.83:0/3901590059 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23900ab600 con 0x7f239c159810 2024-04-25T00:11:47.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.319+0000 7f239be00640 1 -- 172.21.15.83:0/3901590059 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23900ab9d0 con 0x7f239c159810 2024-04-25T00:11:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.319+0000 7f23a4400640 1 -- 172.21.15.83:0/3901590059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f239c159810 msgr2=0x7f239c0cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.319+0000 7f23a4400640 1 --2- 172.21.15.83:0/3901590059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f239c159810 0x7f239c0cc4a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f239c0b4f50 tx=0x7f239001c430 comp rx=0 tx=0).stop 2024-04-25T00:11:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.319+0000 7f23a4400640 1 -- 172.21.15.83:0/3901590059 shutdown_connections 2024-04-25T00:11:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.319+0000 7f23a4400640 1 --2- 172.21.15.83:0/3901590059 >> v2:172.21.15.86:3300/0 conn(0x7f239c06e950 0x7f239c15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.319+0000 7f23a4400640 1 --2- 172.21.15.83:0/3901590059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f239c159810 0x7f239c0cc4a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.319+0000 7f23a4400640 1 -- 172.21.15.83:0/3901590059 >> 172.21.15.83:0/3901590059 conn(0x7f239c0b1830 msgr2=0x7f239c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:47.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a4400640 1 -- 172.21.15.83:0/3901590059 shutdown_connections 2024-04-25T00:11:47.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a4400640 1 -- 172.21.15.83:0/3901590059 wait complete. 2024-04-25T00:11:47.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a4400640 1 Processor -- start 2024-04-25T00:11:47.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a4400640 1 -- start start 2024-04-25T00:11:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a4400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f239c06e950 0x7f239c156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a4400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f239c159810 0x7f239c0cff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a4400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f239c15fe50 0x7f239c142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a4400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f239c159d70 con 0x7f239c06e950 2024-04-25T00:11:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a4400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f239c159bf0 con 0x7f239c15fe50 2024-04-25T00:11:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a4400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f239c070da0 con 0x7f239c159810 2024-04-25T00:11:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f239c06e950 0x7f239c156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f239c06e950 0x7f239c156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48684/0 (socket says 172.21.15.83:48684) 2024-04-25T00:11:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a1a00640 1 -- 172.21.15.83:0/2822644870 learned_addr learned my addr 172.21.15.83:0/2822644870 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a1000640 1 --2- 172.21.15.83:0/2822644870 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f239c159810 0x7f239c0cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a2400640 1 --2- 172.21.15.83:0/2822644870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f239c15fe50 0x7f239c142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a1a00640 1 -- 172.21.15.83:0/2822644870 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f239c159810 msgr2=0x7f239c0cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a1a00640 1 --2- 172.21.15.83:0/2822644870 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f239c159810 0x7f239c0cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a1a00640 1 -- 172.21.15.83:0/2822644870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f239c15fe50 msgr2=0x7f239c142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a1a00640 1 --2- 172.21.15.83:0/2822644870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f239c15fe50 0x7f239c142210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a1a00640 1 -- 172.21.15.83:0/2822644870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239c145bc0 con 0x7f239c06e950 2024-04-25T00:11:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a1a00640 1 --2- 172.21.15.83:0/2822644870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f239c06e950 0x7f239c156ba0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f239001a060 tx=0x7f239001ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f2399600640 1 -- 172.21.15.83:0/2822644870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23900ba020 con 0x7f239c06e950 2024-04-25T00:11:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f2399600640 1 -- 172.21.15.83:0/2822644870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23900b6640 con 0x7f239c06e950 2024-04-25T00:11:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f2399600640 1 -- 172.21.15.83:0/2822644870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23900b0040 con 0x7f239c06e950 2024-04-25T00:11:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.323+0000 7f23a4400640 1 -- 172.21.15.83:0/2822644870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239c150260 con 0x7f239c06e950 2024-04-25T00:11:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.327+0000 7f23a4400640 1 -- 172.21.15.83:0/2822644870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239c145e60 con 0x7f239c06e950 2024-04-25T00:11:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.327+0000 7f23a4400640 1 -- 172.21.15.83:0/2822644870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f239c04f510 con 0x7f239c06e950 2024-04-25T00:11:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.327+0000 7f2399600640 1 -- 172.21.15.83:0/2822644870 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f23900bd020 con 0x7f239c06e950 2024-04-25T00:11:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.327+0000 7f2399600640 1 --2- 172.21.15.83:0/2822644870 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f237c0678c0 0x7f237c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.327+0000 7f2399600640 1 -- 172.21.15.83:0/2822644870 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7f2390120470 con 0x7f239c06e950 2024-04-25T00:11:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.327+0000 7f23a1000640 1 --2- 172.21.15.83:0/2822644870 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f237c0678c0 0x7f237c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.327+0000 7f23a1000640 1 --2- 172.21.15.83:0/2822644870 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f237c0678c0 0x7f237c069d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f239c0b1500 tx=0x7f2388008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.331+0000 7f2399600640 1 -- 172.21.15.83:0/2822644870 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f23900160b0 con 0x7f239c06e950 2024-04-25T00:11:47.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.491+0000 7f23a4400640 1 -- 172.21.15.83:0/2822644870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f239c0727b0 con 0x7f239c06e950 2024-04-25T00:11:47.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.491+0000 7f2399600640 1 -- 172.21.15.83:0/2822644870 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f23900f0c40 con 0x7f239c06e950 2024-04-25T00:11:47.496 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:11:47.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.495+0000 7f23a4400640 1 -- 172.21.15.83:0/2822644870 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f237c0678c0 msgr2=0x7f237c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.495+0000 7f23a4400640 1 --2- 172.21.15.83:0/2822644870 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f237c0678c0 0x7f237c069d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f239c0b1500 tx=0x7f2388008040 comp rx=0 tx=0).stop 2024-04-25T00:11:47.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.495+0000 7f23a4400640 1 -- 172.21.15.83:0/2822644870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f239c06e950 msgr2=0x7f239c156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.495+0000 7f23a4400640 1 --2- 172.21.15.83:0/2822644870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f239c06e950 0x7f239c156ba0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f239001a060 tx=0x7f239001ac70 comp rx=0 tx=0).stop 2024-04-25T00:11:47.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.495+0000 7f23a4400640 1 -- 172.21.15.83:0/2822644870 shutdown_connections 2024-04-25T00:11:47.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.495+0000 7f23a4400640 1 --2- 172.21.15.83:0/2822644870 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f237c0678c0 0x7f237c069d80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.495+0000 7f23a4400640 1 --2- 172.21.15.83:0/2822644870 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f239c159810 0x7f239c0cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.495+0000 7f23a4400640 1 --2- 172.21.15.83:0/2822644870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f239c15fe50 0x7f239c142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.495+0000 7f23a4400640 1 --2- 172.21.15.83:0/2822644870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f239c06e950 0x7f239c156ba0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.495+0000 7f23a4400640 1 -- 172.21.15.83:0/2822644870 >> 172.21.15.83:0/2822644870 conn(0x7f239c0b1830 msgr2=0x7f239c15df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:47.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.495+0000 7f23a4400640 1 -- 172.21.15.83:0/2822644870 shutdown_connections 2024-04-25T00:11:47.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.495+0000 7f23a4400640 1 -- 172.21.15.83:0/2822644870 wait complete. 2024-04-25T00:11:47.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:11:47.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.651+0000 7fbbef200640 1 Processor -- start 2024-04-25T00:11:47.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.651+0000 7fbbef200640 1 -- start start 2024-04-25T00:11:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.651+0000 7fbbef200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbe81551e0 0x7fbbe815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.651+0000 7fbbef200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbbe81555c0 0x7fbbe8071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.651+0000 7fbbef200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbe815c560 con 0x7fbbe806e4c0 2024-04-25T00:11:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.651+0000 7fbbef200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbbe8072460 con 0x7fbbe81555c0 2024-04-25T00:11:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.651+0000 7fbbef200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe815c7d0 con 0x7fbbe81551e0 2024-04-25T00:11:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.651+0000 7fbbdf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbbe81555c0 0x7fbbe8071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.651+0000 7fbbe7e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbe81551e0 0x7fbbe815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.651+0000 7fbbed200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbbe806e4c0 0x7fbbe815e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:57048/0 (socket says 172.21.15.83:57048) 2024-04-25T00:11:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.651+0000 7fbbed200640 1 -- 172.21.15.83:0/866563486 learned_addr learned my addr 172.21.15.83:0/866563486 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:47.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.655+0000 7fbbe7400640 1 -- 172.21.15.83:0/866563486 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3230314229 0 0) 0x7fbbe815c560 con 0x7fbbe806e4c0 2024-04-25T00:11:47.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.655+0000 7fbbe7400640 1 -- 172.21.15.83:0/866563486 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbcc003530 con 0x7fbbe806e4c0 2024-04-25T00:11:47.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.655+0000 7fbbdf400640 1 -- 172.21.15.83:0/866563486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbe81551e0 msgr2=0x7fbbe815bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:47.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.655+0000 7fbbdf400640 1 --2- 172.21.15.83:0/866563486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbe81551e0 0x7fbbe815bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.655+0000 7fbbdf400640 1 -- 172.21.15.83:0/866563486 >> v1:172.21.15.86:6789/0 conn(0x7fbbe806e4c0 legacy=0x7fbbe815e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.655+0000 7fbbdf400640 1 -- 172.21.15.83:0/866563486 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe815cab0 con 0x7fbbe81555c0 2024-04-25T00:11:47.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.655+0000 7fbbe7e00640 1 --2- 172.21.15.83:0/866563486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbe81551e0 0x7fbbe815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.655+0000 7fbbdf400640 1 --2- 172.21.15.83:0/866563486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe81555c0 0x7fbbe8071b70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00097c0 tx=0x7fbbd001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aca01fb5d6904cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.655+0000 7fbbe7400640 1 -- 172.21.15.83:0/866563486 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00b1070 con 0x7fbbe81555c0 2024-04-25T00:11:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.655+0000 7fbbe7400640 1 -- 172.21.15.83:0/866563486 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd00ac620 con 0x7fbbe81555c0 2024-04-25T00:11:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.655+0000 7fbbe7400640 1 -- 172.21.15.83:0/866563486 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00ac9f0 con 0x7fbbe81555c0 2024-04-25T00:11:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 -- 172.21.15.83:0/866563486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe81555c0 msgr2=0x7fbbe8071b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 --2- 172.21.15.83:0/866563486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe81555c0 0x7fbbe8071b70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00097c0 tx=0x7fbbd001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 -- 172.21.15.83:0/866563486 shutdown_connections 2024-04-25T00:11:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 --2- 172.21.15.83:0/866563486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe81555c0 0x7fbbe8071b70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 --2- 172.21.15.83:0/866563486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbe81551e0 0x7fbbe815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 -- 172.21.15.83:0/866563486 >> 172.21.15.83:0/866563486 conn(0x7fbbe80b1830 msgr2=0x7fbbe80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 -- 172.21.15.83:0/866563486 shutdown_connections 2024-04-25T00:11:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 -- 172.21.15.83:0/866563486 wait complete. 2024-04-25T00:11:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 Processor -- start 2024-04-25T00:11:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 -- start start 2024-04-25T00:11:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbe806e4c0 0x7fbbe8166c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe81551e0 0x7fbbe8168180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe81555c0 0x7fbbe80cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8165e00 con 0x7fbbe81555c0 2024-04-25T00:11:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8165c80 con 0x7fbbe81551e0 2024-04-25T00:11:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbef200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe8165f80 con 0x7fbbe806e4c0 2024-04-25T00:11:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbed200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe81555c0 0x7fbbe80cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbed200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe81555c0 0x7fbbe80cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48688/0 (socket says 172.21.15.83:48688) 2024-04-25T00:11:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbed200640 1 -- 172.21.15.83:0/1186495228 learned_addr learned my addr 172.21.15.83:0/1186495228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbdf400640 1 --2- 172.21.15.83:0/1186495228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe81551e0 0x7fbbe8168180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbe7e00640 1 --2- 172.21.15.83:0/1186495228 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbe806e4c0 0x7fbbe8166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbed200640 1 -- 172.21.15.83:0/1186495228 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbe806e4c0 msgr2=0x7fbbe8166c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbed200640 1 --2- 172.21.15.83:0/1186495228 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbe806e4c0 0x7fbbe8166c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbed200640 1 -- 172.21.15.83:0/1186495228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe81551e0 msgr2=0x7fbbe8168180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbed200640 1 --2- 172.21.15.83:0/1186495228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe81551e0 0x7fbbe8168180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.659+0000 7fbbed200640 1 -- 172.21.15.83:0/1186495228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe81f75e0 con 0x7fbbe81555c0 2024-04-25T00:11:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.663+0000 7fbbed200640 1 --2- 172.21.15.83:0/1186495228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe81555c0 0x7fbbe80cdaf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000cb70 tx=0x7fbbe000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.663+0000 7fbbe4c00640 1 -- 172.21.15.83:0/1186495228 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0017070 con 0x7fbbe81555c0 2024-04-25T00:11:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.663+0000 7fbbe4c00640 1 -- 172.21.15.83:0/1186495228 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbe000ad40 con 0x7fbbe81555c0 2024-04-25T00:11:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.663+0000 7fbbe4c00640 1 -- 172.21.15.83:0/1186495228 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0012620 con 0x7fbbe81555c0 2024-04-25T00:11:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.663+0000 7fbbef200640 1 -- 172.21.15.83:0/1186495228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe81f78c0 con 0x7fbbe81555c0 2024-04-25T00:11:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.663+0000 7fbbef200640 1 -- 172.21.15.83:0/1186495228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe81f7e80 con 0x7fbbe81555c0 2024-04-25T00:11:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.663+0000 7fbbef200640 1 -- 172.21.15.83:0/1186495228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe81f80c0 con 0x7fbbe81555c0 2024-04-25T00:11:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.663+0000 7fbbe4c00640 1 -- 172.21.15.83:0/1186495228 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbbe001a070 con 0x7fbbe81555c0 2024-04-25T00:11:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.663+0000 7fbbe4c00640 1 --2- 172.21.15.83:0/1186495228 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbbc40678f0 0x7fbbc4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.663+0000 7fbbe4c00640 1 -- 172.21.15.83:0/1186495228 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7fbbe0085e60 con 0x7fbbe81555c0 2024-04-25T00:11:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.663+0000 7fbbe7e00640 1 --2- 172.21.15.83:0/1186495228 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbbc40678f0 0x7fbbc4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.663+0000 7fbbe7e00640 1 --2- 172.21.15.83:0/1186495228 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbbc40678f0 0x7fbbc4069db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009580 tx=0x7fbbd8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.667+0000 7fbbe4c00640 1 -- 172.21.15.83:0/1186495228 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbbe81f80c0 con 0x7fbbe81555c0 2024-04-25T00:11:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.823+0000 7fbbef200640 1 -- 172.21.15.83:0/1186495228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbbe804f510 con 0x7fbbe81555c0 2024-04-25T00:11:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.823+0000 7fbbe4c00640 1 -- 172.21.15.83:0/1186495228 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fbbe0055eb0 con 0x7fbbe81555c0 2024-04-25T00:11:47.828 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.827+0000 7fbbef200640 1 -- 172.21.15.83:0/1186495228 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbbc40678f0 msgr2=0x7fbbc4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.827+0000 7fbbef200640 1 --2- 172.21.15.83:0/1186495228 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbbc40678f0 0x7fbbc4069db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009580 tx=0x7fbbd8009190 comp rx=0 tx=0).stop 2024-04-25T00:11:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.827+0000 7fbbef200640 1 -- 172.21.15.83:0/1186495228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe81555c0 msgr2=0x7fbbe80cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.827+0000 7fbbef200640 1 --2- 172.21.15.83:0/1186495228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe81555c0 0x7fbbe80cdaf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000cb70 tx=0x7fbbe000a430 comp rx=0 tx=0).stop 2024-04-25T00:11:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.827+0000 7fbbef200640 1 -- 172.21.15.83:0/1186495228 shutdown_connections 2024-04-25T00:11:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.827+0000 7fbbef200640 1 --2- 172.21.15.83:0/1186495228 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbbc40678f0 0x7fbbc4069db0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.827+0000 7fbbef200640 1 --2- 172.21.15.83:0/1186495228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe81555c0 0x7fbbe80cdaf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.827+0000 7fbbef200640 1 --2- 172.21.15.83:0/1186495228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe81551e0 0x7fbbe8168180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.827+0000 7fbbef200640 1 --2- 172.21.15.83:0/1186495228 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbe806e4c0 0x7fbbe8166c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.827+0000 7fbbef200640 1 -- 172.21.15.83:0/1186495228 >> 172.21.15.83:0/1186495228 conn(0x7fbbe80b1830 msgr2=0x7fbbe80b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:47.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.827+0000 7fbbef200640 1 -- 172.21.15.83:0/1186495228 shutdown_connections 2024-04-25T00:11:47.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.827+0000 7fbbef200640 1 -- 172.21.15.83:0/1186495228 wait complete. 2024-04-25T00:11:47.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:11:47.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f8302a00640 1 Processor -- start 2024-04-25T00:11:47.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f8302a00640 1 -- start start 2024-04-25T00:11:47.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f8302a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f82fc1551e0 0x7f82fc15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f8302a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82fc06e4c0 0x7f82fc06e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f8302a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82fc15c560 con 0x7f82fc1555c0 2024-04-25T00:11:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f8302a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f82fc072460 con 0x7f82fc1551e0 2024-04-25T00:11:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f8302a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f82fc15c7d0 con 0x7f82fc06e4c0 2024-04-25T00:11:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f82fbe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f82fc1551e0 0x7f82fc15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f82fbe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f82fc1551e0 0x7f82fc15bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48700/0 (socket says 172.21.15.83:48700) 2024-04-25T00:11:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f82f3400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f82fc1555c0 0x7f82fc071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51804/0 (socket says 172.21.15.83:51804) 2024-04-25T00:11:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f82fbe00640 1 -- 172.21.15.83:0/1790116093 learned_addr learned my addr 172.21.15.83:0/1790116093 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f8300a00640 1 --2- 172.21.15.83:0/1790116093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82fc06e4c0 0x7f82fc06e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f82fbe00640 1 -- 172.21.15.83:0/1790116093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82fc06e4c0 msgr2=0x7f82fc06e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f82fbe00640 1 --2- 172.21.15.83:0/1790116093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82fc06e4c0 0x7f82fc06e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f82fbe00640 1 -- 172.21.15.83:0/1790116093 >> v1:172.21.15.83:6789/0 conn(0x7f82fc1555c0 legacy=0x7f82fc071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.983+0000 7f82fbe00640 1 -- 172.21.15.83:0/1790116093 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc15cab0 con 0x7f82fc1551e0 2024-04-25T00:11:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.987+0000 7f82fbe00640 1 --2- 172.21.15.83:0/1790116093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82fc1551e0 0x7f82fc15bea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0098a0 tx=0x7f82ec01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af81ead236bb1e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.987+0000 7f82fb400640 1 -- 172.21.15.83:0/1790116093 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0b0070 con 0x7f82fc1551e0 2024-04-25T00:11:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.987+0000 7f82fb400640 1 -- 172.21.15.83:0/1790116093 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec0ab600 con 0x7f82fc1551e0 2024-04-25T00:11:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.987+0000 7f82fb400640 1 -- 172.21.15.83:0/1790116093 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0ab9d0 con 0x7f82fc1551e0 2024-04-25T00:11:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.987+0000 7f8302a00640 1 -- 172.21.15.83:0/1790116093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82fc1551e0 msgr2=0x7f82fc15bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.987+0000 7f8302a00640 1 --2- 172.21.15.83:0/1790116093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82fc1551e0 0x7f82fc15bea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0098a0 tx=0x7f82ec01c540 comp rx=0 tx=0).stop 2024-04-25T00:11:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.987+0000 7f8302a00640 1 -- 172.21.15.83:0/1790116093 shutdown_connections 2024-04-25T00:11:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.987+0000 7f8302a00640 1 --2- 172.21.15.83:0/1790116093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82fc06e4c0 0x7f82fc06e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.987+0000 7f8302a00640 1 --2- 172.21.15.83:0/1790116093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82fc1551e0 0x7f82fc15bea0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.987+0000 7f8302a00640 1 -- 172.21.15.83:0/1790116093 >> 172.21.15.83:0/1790116093 conn(0x7f82fc0b1830 msgr2=0x7f82fc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.987+0000 7f8302a00640 1 -- 172.21.15.83:0/1790116093 shutdown_connections 2024-04-25T00:11:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.987+0000 7f8302a00640 1 -- 172.21.15.83:0/1790116093 wait complete. 2024-04-25T00:11:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f8302a00640 1 Processor -- start 2024-04-25T00:11:47.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f8302a00640 1 -- start start 2024-04-25T00:11:47.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f8302a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82fc06e4c0 0x7f82fc15a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f8302a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1551e0 0x7f82fc0ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f8302a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82fc1555c0 0x7f82fc0aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:47.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f8302a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc15dbb0 con 0x7f82fc06e4c0 2024-04-25T00:11:47.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f8302a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc15da30 con 0x7f82fc1551e0 2024-04-25T00:11:47.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f8302a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f82fc15dd30 con 0x7f82fc1555c0 2024-04-25T00:11:47.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f82fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82fc06e4c0 0x7f82fc15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f82fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82fc06e4c0 0x7f82fc15a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48714/0 (socket says 172.21.15.83:48714) 2024-04-25T00:11:47.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f82fbe00640 1 -- 172.21.15.83:0/4141512009 learned_addr learned my addr 172.21.15.83:0/4141512009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:47.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f8300a00640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82fc1555c0 0x7f82fc0aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f82f3400640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1551e0 0x7f82fc0ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:47.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f82fbe00640 1 -- 172.21.15.83:0/4141512009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82fc1555c0 msgr2=0x7f82fc0aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f82fbe00640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82fc1555c0 0x7f82fc0aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f82fbe00640 1 -- 172.21.15.83:0/4141512009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1551e0 msgr2=0x7f82fc0ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f82fbe00640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1551e0 0x7f82fc0ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f82fbe00640 1 -- 172.21.15.83:0/4141512009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc1f6810 con 0x7f82fc06e4c0 2024-04-25T00:11:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f82f3400640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1551e0 0x7f82fc0ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f82fbe00640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82fc06e4c0 0x7f82fc15a030 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f82ec01abf0 tx=0x7f82ec0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f82f8c00640 1 -- 172.21.15.83:0/4141512009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0b9070 con 0x7f82fc06e4c0 2024-04-25T00:11:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f82f8c00640 1 -- 172.21.15.83:0/4141512009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec0ab400 con 0x7f82fc06e4c0 2024-04-25T00:11:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f82f8c00640 1 -- 172.21.15.83:0/4141512009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0b0040 con 0x7f82fc06e4c0 2024-04-25T00:11:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f8302a00640 1 -- 172.21.15.83:0/4141512009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82fc1f6a90 con 0x7f82fc06e4c0 2024-04-25T00:11:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f8302a00640 1 -- 172.21.15.83:0/4141512009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82fc1f7050 con 0x7f82fc06e4c0 2024-04-25T00:11:48.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.991+0000 7f8302a00640 1 -- 172.21.15.83:0/4141512009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82fc04f510 con 0x7f82fc06e4c0 2024-04-25T00:11:48.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.995+0000 7f82f8c00640 1 -- 172.21.15.83:0/4141512009 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f82ec0aa990 con 0x7f82fc06e4c0 2024-04-25T00:11:48.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.995+0000 7f82f8c00640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f82d80678c0 0x7f82d8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.995+0000 7f82f8c00640 1 -- 172.21.15.83:0/4141512009 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7f82ec11f430 con 0x7f82fc06e4c0 2024-04-25T00:11:48.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.995+0000 7f82f3400640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f82d80678c0 0x7f82d8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.995+0000 7f82f3400640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f82d80678c0 0x7f82d8069d80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0b3ae0 tx=0x7f82e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:47.999+0000 7f82f8c00640 1 -- 172.21.15.83:0/4141512009 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f82fc1f6810 con 0x7f82fc06e4c0 2024-04-25T00:11:48.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.155+0000 7f8302a00640 1 -- 172.21.15.83:0/4141512009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f82fc1530c0 con 0x7f82fc06e4c0 2024-04-25T00:11:48.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.159+0000 7f82f8c00640 1 -- 172.21.15.83:0/4141512009 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f82ec016130 con 0x7f82fc06e4c0 2024-04-25T00:11:48.162 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:11:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.159+0000 7f8302a00640 1 -- 172.21.15.83:0/4141512009 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f82d80678c0 msgr2=0x7f82d8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.159+0000 7f8302a00640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f82d80678c0 0x7f82d8069d80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0b3ae0 tx=0x7f82e4009040 comp rx=0 tx=0).stop 2024-04-25T00:11:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.159+0000 7f8302a00640 1 -- 172.21.15.83:0/4141512009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82fc06e4c0 msgr2=0x7f82fc15a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.159+0000 7f8302a00640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82fc06e4c0 0x7f82fc15a030 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f82ec01abf0 tx=0x7f82ec0192e0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.159+0000 7f8302a00640 1 -- 172.21.15.83:0/4141512009 shutdown_connections 2024-04-25T00:11:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.159+0000 7f8302a00640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f82d80678c0 0x7f82d8069d80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.159+0000 7f8302a00640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82fc1555c0 0x7f82fc0aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.159+0000 7f8302a00640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1551e0 0x7f82fc0ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.159+0000 7f8302a00640 1 --2- 172.21.15.83:0/4141512009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82fc06e4c0 0x7f82fc15a030 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.159+0000 7f8302a00640 1 -- 172.21.15.83:0/4141512009 >> 172.21.15.83:0/4141512009 conn(0x7f82fc0b1830 msgr2=0x7f82fc0713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.159+0000 7f8302a00640 1 -- 172.21.15.83:0/4141512009 shutdown_connections 2024-04-25T00:11:48.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.159+0000 7f8302a00640 1 -- 172.21.15.83:0/4141512009 wait complete. 2024-04-25T00:11:48.183 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:11:48.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.319+0000 7fc3cc800640 1 Processor -- start 2024-04-25T00:11:48.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.319+0000 7fc3cc800640 1 -- start start 2024-04-25T00:11:48.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.323+0000 7fc3cc800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc3c406e950 0x7fc3c415f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.323+0000 7fc3cc800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc3c415fe50 0x7fc3c40ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.323+0000 7fc3cc800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c40ce000 con 0x7fc3c4159810 2024-04-25T00:11:48.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.323+0000 7fc3cc800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c406e210 con 0x7fc3c415fe50 2024-04-25T00:11:48.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.323+0000 7fc3cc800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c41602a0 con 0x7fc3c406e950 2024-04-25T00:11:48.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.323+0000 7fc3ca800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc3c415fe50 0x7fc3c40ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.323+0000 7fc3c9e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc3c4159810 0x7fc3c40cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51814/0 (socket says 172.21.15.83:51814) 2024-04-25T00:11:48.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.323+0000 7fc3c9e00640 1 -- 172.21.15.83:0/1089758115 learned_addr learned my addr 172.21.15.83:0/1089758115 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:48.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.323+0000 7fc3ca800640 1 -- 172.21.15.83:0/1089758115 >> v2:172.21.15.86:3300/0 conn(0x7fc3c406e950 msgr2=0x7fc3c415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:48.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.323+0000 7fc3ca800640 1 --2- 172.21.15.83:0/1089758115 >> v2:172.21.15.86:3300/0 conn(0x7fc3c406e950 0x7fc3c415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.323+0000 7fc3ca800640 1 -- 172.21.15.83:0/1089758115 >> v1:172.21.15.83:6789/0 conn(0x7fc3c4159810 legacy=0x7fc3c40cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.323+0000 7fc3ca800640 1 -- 172.21.15.83:0/1089758115 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c406eec0 con 0x7fc3c415fe50 2024-04-25T00:11:48.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.323+0000 7fc3c8a00640 1 -- 172.21.15.83:0/1089758115 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2205738119 0 0) 0x7fc3c40ce000 con 0x7fc3c4159810 2024-04-25T00:11:48.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.327+0000 7fc3ca800640 1 --2- 172.21.15.83:0/1089758115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3c415fe50 0x7fc3c40ce4a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40709a0 tx=0x7fc3c001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131122a5c7af4b32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.327+0000 7fc3c8a00640 1 -- 172.21.15.83:0/1089758115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c00b2070 con 0x7fc3c415fe50 2024-04-25T00:11:48.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.327+0000 7fc3c8a00640 1 -- 172.21.15.83:0/1089758115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3c00a5850 con 0x7fc3c415fe50 2024-04-25T00:11:48.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.327+0000 7fc3c8a00640 1 -- 172.21.15.83:0/1089758115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c00a5c20 con 0x7fc3c415fe50 2024-04-25T00:11:48.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.327+0000 7fc3cc800640 1 -- 172.21.15.83:0/1089758115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3c415fe50 msgr2=0x7fc3c40ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.327+0000 7fc3cc800640 1 --2- 172.21.15.83:0/1089758115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3c415fe50 0x7fc3c40ce4a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40709a0 tx=0x7fc3c001c8a0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.327+0000 7fc3cc800640 1 -- 172.21.15.83:0/1089758115 shutdown_connections 2024-04-25T00:11:48.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.327+0000 7fc3cc800640 1 --2- 172.21.15.83:0/1089758115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3c415fe50 0x7fc3c40ce4a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.327+0000 7fc3cc800640 1 --2- 172.21.15.83:0/1089758115 >> v2:172.21.15.86:3300/0 conn(0x7fc3c406e950 0x7fc3c415f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.327+0000 7fc3cc800640 1 -- 172.21.15.83:0/1089758115 >> 172.21.15.83:0/1089758115 conn(0x7fc3c40b1830 msgr2=0x7fc3c40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:48.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.327+0000 7fc3cc800640 1 -- 172.21.15.83:0/1089758115 shutdown_connections 2024-04-25T00:11:48.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.327+0000 7fc3cc800640 1 -- 172.21.15.83:0/1089758115 wait complete. 2024-04-25T00:11:48.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.327+0000 7fc3cc800640 1 Processor -- start 2024-04-25T00:11:48.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.327+0000 7fc3cc800640 1 -- start start 2024-04-25T00:11:48.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3cc800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3c406e950 0x7fc3c41423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3cc800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3c4159810 0x7fc3c40ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3cc800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3c415fe50 0x7fc3c40ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3cc800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c4070f20 con 0x7fc3c406e950 2024-04-25T00:11:48.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3cc800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c4070da0 con 0x7fc3c415fe50 2024-04-25T00:11:48.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3cc800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c4168cb0 con 0x7fc3c4159810 2024-04-25T00:11:48.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3c9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3c406e950 0x7fc3c41423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3c9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3c406e950 0x7fc3c41423d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48740/0 (socket says 172.21.15.83:48740) 2024-04-25T00:11:48.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3c9e00640 1 -- 172.21.15.83:0/1107249864 learned_addr learned my addr 172.21.15.83:0/1107249864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:48.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3ca800640 1 --2- 172.21.15.83:0/1107249864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3c415fe50 0x7fc3c40ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3c9400640 1 --2- 172.21.15.83:0/1107249864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3c4159810 0x7fc3c40ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3c9e00640 1 -- 172.21.15.83:0/1107249864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3c4159810 msgr2=0x7fc3c40ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3c9e00640 1 --2- 172.21.15.83:0/1107249864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3c4159810 0x7fc3c40ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3c9e00640 1 -- 172.21.15.83:0/1107249864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3c415fe50 msgr2=0x7fc3c40ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3c9e00640 1 --2- 172.21.15.83:0/1107249864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3c415fe50 0x7fc3c40ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3c9e00640 1 -- 172.21.15.83:0/1107249864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c40b2c30 con 0x7fc3c406e950 2024-04-25T00:11:48.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3c9e00640 1 --2- 172.21.15.83:0/1107249864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3c406e950 0x7fc3c41423d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800c550 tx=0x7fc3b800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3be000640 1 -- 172.21.15.83:0/1107249864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8017070 con 0x7fc3c406e950 2024-04-25T00:11:48.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3be000640 1 -- 172.21.15.83:0/1107249864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b8005d10 con 0x7fc3c406e950 2024-04-25T00:11:48.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3cc800640 1 -- 172.21.15.83:0/1107249864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c415b4a0 con 0x7fc3c406e950 2024-04-25T00:11:48.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3be000640 1 -- 172.21.15.83:0/1107249864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8007330 con 0x7fc3c406e950 2024-04-25T00:11:48.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3cc800640 1 -- 172.21.15.83:0/1107249864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c40b2ef0 con 0x7fc3c406e950 2024-04-25T00:11:48.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3cc800640 1 -- 172.21.15.83:0/1107249864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c404f510 con 0x7fc3c406e950 2024-04-25T00:11:48.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.331+0000 7fc3be000640 1 -- 172.21.15.83:0/1107249864 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc3b801a070 con 0x7fc3c406e950 2024-04-25T00:11:48.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.335+0000 7fc3be000640 1 --2- 172.21.15.83:0/1107249864 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc3a80678c0 0x7fc3a8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.335+0000 7fc3be000640 1 -- 172.21.15.83:0/1107249864 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc3b8085550 con 0x7fc3c406e950 2024-04-25T00:11:48.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.335+0000 7fc3c9400640 1 --2- 172.21.15.83:0/1107249864 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc3a80678c0 0x7fc3a8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.335+0000 7fc3c9400640 1 --2- 172.21.15.83:0/1107249864 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc3a80678c0 0x7fc3a8069d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4006fd0 tx=0x7fc3b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.339+0000 7fc3be000640 1 -- 172.21.15.83:0/1107249864 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc3b8055c80 con 0x7fc3c406e950 2024-04-25T00:11:48.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.495+0000 7fc3cc800640 1 -- 172.21.15.83:0/1107249864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc3c4072c90 con 0x7fc3c406e950 2024-04-25T00:11:48.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.495+0000 7fc3be000640 1 -- 172.21.15.83:0/1107249864 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fc3b8055580 con 0x7fc3c406e950 2024-04-25T00:11:48.500 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:48.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.499+0000 7fc3cc800640 1 -- 172.21.15.83:0/1107249864 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc3a80678c0 msgr2=0x7fc3a8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.499+0000 7fc3cc800640 1 --2- 172.21.15.83:0/1107249864 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc3a80678c0 0x7fc3a8069d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4006fd0 tx=0x7fc3b4008040 comp rx=0 tx=0).stop 2024-04-25T00:11:48.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.499+0000 7fc3cc800640 1 -- 172.21.15.83:0/1107249864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3c406e950 msgr2=0x7fc3c41423d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.499+0000 7fc3cc800640 1 --2- 172.21.15.83:0/1107249864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3c406e950 0x7fc3c41423d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800c550 tx=0x7fc3b800ca10 comp rx=0 tx=0).stop 2024-04-25T00:11:48.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.499+0000 7fc3cc800640 1 -- 172.21.15.83:0/1107249864 shutdown_connections 2024-04-25T00:11:48.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.499+0000 7fc3cc800640 1 --2- 172.21.15.83:0/1107249864 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc3a80678c0 0x7fc3a8069d80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.499+0000 7fc3cc800640 1 --2- 172.21.15.83:0/1107249864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3c4159810 0x7fc3c40ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.499+0000 7fc3cc800640 1 --2- 172.21.15.83:0/1107249864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3c415fe50 0x7fc3c40ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.499+0000 7fc3cc800640 1 --2- 172.21.15.83:0/1107249864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3c406e950 0x7fc3c41423d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.499+0000 7fc3cc800640 1 -- 172.21.15.83:0/1107249864 >> 172.21.15.83:0/1107249864 conn(0x7fc3c40b1830 msgr2=0x7fc3c40d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:48.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.499+0000 7fc3cc800640 1 -- 172.21.15.83:0/1107249864 shutdown_connections 2024-04-25T00:11:48.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.499+0000 7fc3cc800640 1 -- 172.21.15.83:0/1107249864 wait complete. 2024-04-25T00:11:48.521 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:48.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.651+0000 7fe66a600640 1 Processor -- start 2024-04-25T00:11:48.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.651+0000 7fe66a600640 1 -- start start 2024-04-25T00:11:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe66a600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe6641553e0 0x7fe66406e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe66a600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe66406ea00 0x7fe6640b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe66a600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6640b48a0 0x7fe66415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe66a600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe66406ef80 con 0x7fe6641553e0 2024-04-25T00:11:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe66a600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe66415b510 con 0x7fe66406ea00 2024-04-25T00:11:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe66a600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe66415b7f0 con 0x7fe6640b48a0 2024-04-25T00:11:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe663400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe6641553e0 0x7fe66406e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe663400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe6641553e0 0x7fe66406e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48746/0 (socket says 172.21.15.83:48746) 2024-04-25T00:11:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe663400640 1 -- 172.21.15.83:0/3863327112 learned_addr learned my addr 172.21.15.83:0/3863327112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe663e00640 1 --2- 172.21.15.83:0/3863327112 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6640b48a0 0x7fe66415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe65aa00640 1 --2- 172.21.15.83:0/3863327112 >> v2:172.21.15.86:3300/0 conn(0x7fe66406ea00 0x7fe6640b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe663400640 1 -- 172.21.15.83:0/3863327112 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6640b48a0 msgr2=0x7fe66415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe663400640 1 --2- 172.21.15.83:0/3863327112 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6640b48a0 0x7fe66415e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe663400640 1 -- 172.21.15.83:0/3863327112 >> v2:172.21.15.86:3300/0 conn(0x7fe66406ea00 msgr2=0x7fe6640b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe663400640 1 --2- 172.21.15.83:0/3863327112 >> v2:172.21.15.86:3300/0 conn(0x7fe66406ea00 0x7fe6640b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe663400640 1 -- 172.21.15.83:0/3863327112 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66415bad0 con 0x7fe6641553e0 2024-04-25T00:11:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.655+0000 7fe65aa00640 1 --2- 172.21.15.83:0/3863327112 >> v2:172.21.15.86:3300/0 conn(0x7fe66406ea00 0x7fe6640b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe663400640 1 --2- 172.21.15.83:0/3863327112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6641553e0 0x7fe66406e4c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe6640cf5b0 tx=0x7fe65401c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc567621af745a35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe662a00640 1 -- 172.21.15.83:0/3863327112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6540a9430 con 0x7fe6641553e0 2024-04-25T00:11:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe662a00640 1 -- 172.21.15.83:0/3863327112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6540ae070 con 0x7fe6641553e0 2024-04-25T00:11:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe662a00640 1 -- 172.21.15.83:0/3863327112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6540a97f0 con 0x7fe6641553e0 2024-04-25T00:11:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe66a600640 1 -- 172.21.15.83:0/3863327112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6641553e0 msgr2=0x7fe66406e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe66a600640 1 --2- 172.21.15.83:0/3863327112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6641553e0 0x7fe66406e4c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe6640cf5b0 tx=0x7fe65401c6a0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe66a600640 1 -- 172.21.15.83:0/3863327112 shutdown_connections 2024-04-25T00:11:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe66a600640 1 --2- 172.21.15.83:0/3863327112 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6640b48a0 0x7fe66415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe66a600640 1 --2- 172.21.15.83:0/3863327112 >> v2:172.21.15.86:3300/0 conn(0x7fe66406ea00 0x7fe6640b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe66a600640 1 --2- 172.21.15.83:0/3863327112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6641553e0 0x7fe66406e4c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe66a600640 1 -- 172.21.15.83:0/3863327112 >> 172.21.15.83:0/3863327112 conn(0x7fe6640b1830 msgr2=0x7fe6640b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe66a600640 1 -- 172.21.15.83:0/3863327112 shutdown_connections 2024-04-25T00:11:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe66a600640 1 -- 172.21.15.83:0/3863327112 wait complete. 2024-04-25T00:11:48.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe66a600640 1 Processor -- start 2024-04-25T00:11:48.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.659+0000 7fe66a600640 1 -- start start 2024-04-25T00:11:48.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe66a600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe66406ea00 0x7fe66414e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe66a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6640b48a0 0x7fe6641f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe66a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6641553e0 0x7fe6641510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe66a600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6640b2cb0 con 0x7fe6641553e0 2024-04-25T00:11:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe66a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6640b2b30 con 0x7fe6640b48a0 2024-04-25T00:11:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe66a600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe6640b2e30 con 0x7fe66406ea00 2024-04-25T00:11:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe663e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6641553e0 0x7fe6641510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe663e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6641553e0 0x7fe6641510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48762/0 (socket says 172.21.15.83:48762) 2024-04-25T00:11:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe663e00640 1 -- 172.21.15.83:0/1447265019 learned_addr learned my addr 172.21.15.83:0/1447265019 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe663400640 1 --2- 172.21.15.83:0/1447265019 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe66406ea00 0x7fe66414e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe65aa00640 1 --2- 172.21.15.83:0/1447265019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6640b48a0 0x7fe6641f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe663e00640 1 -- 172.21.15.83:0/1447265019 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe66406ea00 msgr2=0x7fe66414e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe663e00640 1 --2- 172.21.15.83:0/1447265019 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe66406ea00 0x7fe66414e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe663e00640 1 -- 172.21.15.83:0/1447265019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6640b48a0 msgr2=0x7fe6641f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe663e00640 1 --2- 172.21.15.83:0/1447265019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6640b48a0 0x7fe6641f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe663e00640 1 -- 172.21.15.83:0/1447265019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6641f8730 con 0x7fe6641553e0 2024-04-25T00:11:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe663e00640 1 --2- 172.21.15.83:0/1447265019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6641553e0 0x7fe6641510c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00c7f0 tx=0x7fe65c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe65be00640 1 -- 172.21.15.83:0/1447265019 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c015070 con 0x7fe6641553e0 2024-04-25T00:11:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe65be00640 1 -- 172.21.15.83:0/1447265019 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65c004220 con 0x7fe6641553e0 2024-04-25T00:11:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe65be00640 1 -- 172.21.15.83:0/1447265019 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c003620 con 0x7fe6641553e0 2024-04-25T00:11:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe66a600640 1 -- 172.21.15.83:0/1447265019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6641f7e90 con 0x7fe6641553e0 2024-04-25T00:11:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe66a600640 1 -- 172.21.15.83:0/1447265019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6641f8450 con 0x7fe6641553e0 2024-04-25T00:11:48.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe66a600640 1 -- 172.21.15.83:0/1447265019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6640737f0 con 0x7fe6641553e0 2024-04-25T00:11:48.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.663+0000 7fe65be00640 1 -- 172.21.15.83:0/1447265019 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe65c0039d0 con 0x7fe6641553e0 2024-04-25T00:11:48.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.667+0000 7fe65be00640 1 --2- 172.21.15.83:0/1447265019 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6400678f0 0x7fe640069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.667+0000 7fe65be00640 1 -- 172.21.15.83:0/1447265019 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe65c086090 con 0x7fe6641553e0 2024-04-25T00:11:48.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.667+0000 7fe663400640 1 --2- 172.21.15.83:0/1447265019 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6400678f0 0x7fe640069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.667+0000 7fe663400640 1 --2- 172.21.15.83:0/1447265019 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6400678f0 0x7fe640069db0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe6540ad000 tx=0x7fe6540b4040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.671+0000 7fe65be00640 1 -- 172.21.15.83:0/1447265019 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe65c0561a0 con 0x7fe6641553e0 2024-04-25T00:11:48.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.827+0000 7fe66a600640 1 -- 172.21.15.83:0/1447265019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe66414dd50 con 0x7fe6641553e0 2024-04-25T00:11:48.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.827+0000 7fe65be00640 1 -- 172.21.15.83:0/1447265019 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18820 (secure 0 0 0) 0x7fe65c05a4a0 con 0x7fe6641553e0 2024-04-25T00:11:48.833 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:48.833 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:48.235809+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:48.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.831+0000 7fe66a600640 1 -- 172.21.15.83:0/1447265019 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6400678f0 msgr2=0x7fe640069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.831+0000 7fe66a600640 1 --2- 172.21.15.83:0/1447265019 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6400678f0 0x7fe640069db0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe6540ad000 tx=0x7fe6540b4040 comp rx=0 tx=0).stop 2024-04-25T00:11:48.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.831+0000 7fe66a600640 1 -- 172.21.15.83:0/1447265019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6641553e0 msgr2=0x7fe6641510c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.831+0000 7fe66a600640 1 --2- 172.21.15.83:0/1447265019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6641553e0 0x7fe6641510c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00c7f0 tx=0x7fe65c00ccb0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.831+0000 7fe66a600640 1 -- 172.21.15.83:0/1447265019 shutdown_connections 2024-04-25T00:11:48.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.831+0000 7fe66a600640 1 --2- 172.21.15.83:0/1447265019 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6400678f0 0x7fe640069db0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.831+0000 7fe66a600640 1 --2- 172.21.15.83:0/1447265019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6641553e0 0x7fe6641510c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.831+0000 7fe66a600640 1 --2- 172.21.15.83:0/1447265019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6640b48a0 0x7fe6641f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.831+0000 7fe66a600640 1 --2- 172.21.15.83:0/1447265019 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe66406ea00 0x7fe66414e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.831+0000 7fe66a600640 1 -- 172.21.15.83:0/1447265019 >> 172.21.15.83:0/1447265019 conn(0x7fe6640b1830 msgr2=0x7fe6640cbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:48.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.831+0000 7fe66a600640 1 -- 172.21.15.83:0/1447265019 shutdown_connections 2024-04-25T00:11:48.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.831+0000 7fe66a600640 1 -- 172.21.15.83:0/1447265019 wait complete. 2024-04-25T00:11:48.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:11:04.621429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:11:05.189716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:11:08.902215+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:11:48.855 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:11:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.991+0000 7fda4dc00640 1 Processor -- start 2024-04-25T00:11:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.991+0000 7fda4dc00640 1 -- start start 2024-04-25T00:11:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.995+0000 7fda4dc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda48159810 0x7fda480cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.995+0000 7fda4dc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fda4806e950 0x7fda4815f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.995+0000 7fda4dc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda480ce000 con 0x7fda4815fe50 2024-04-25T00:11:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.995+0000 7fda4dc00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fda4806e210 con 0x7fda4806e950 2024-04-25T00:11:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.995+0000 7fda4dc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fda4806da00 con 0x7fda48159810 2024-04-25T00:11:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.995+0000 7fda3fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fda4806e950 0x7fda4815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.995+0000 7fda3fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fda4806e950 0x7fda4815f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48770/0 (socket says 172.21.15.83:48770) 2024-04-25T00:11:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.995+0000 7fda3fe00640 1 -- 172.21.15.83:0/2179546991 learned_addr learned my addr 172.21.15.83:0/2179546991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.995+0000 7fda46a00640 1 --2- 172.21.15.83:0/2179546991 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda48159810 0x7fda480cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.995+0000 7fda3fe00640 1 -- 172.21.15.83:0/2179546991 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda48159810 msgr2=0x7fda480cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.995+0000 7fda3fe00640 1 --2- 172.21.15.83:0/2179546991 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda48159810 0x7fda480cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.995+0000 7fda3fe00640 1 -- 172.21.15.83:0/2179546991 >> v1:172.21.15.86:6789/0 conn(0x7fda4815fe50 legacy=0x7fda4815c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:49.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.995+0000 7fda3fe00640 1 -- 172.21.15.83:0/2179546991 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4806eec0 con 0x7fda4806e950 2024-04-25T00:11:49.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.995+0000 7fda46a00640 1 --2- 172.21.15.83:0/2179546991 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda48159810 0x7fda480cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:49.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.999+0000 7fda3fe00640 1 --2- 172.21.15.83:0/2179546991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4806e950 0x7fda4815f910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fda30009870 tx=0x7fda3001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209042123b82f3e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.999+0000 7fda46000640 1 -- 172.21.15.83:0/2179546991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda300b2070 con 0x7fda4806e950 2024-04-25T00:11:49.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.999+0000 7fda46000640 1 -- 172.21.15.83:0/2179546991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda300a9070 con 0x7fda4806e950 2024-04-25T00:11:49.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.999+0000 7fda46000640 1 -- 172.21.15.83:0/2179546991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda300ad880 con 0x7fda4806e950 2024-04-25T00:11:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.999+0000 7fda4dc00640 1 -- 172.21.15.83:0/2179546991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4806e950 msgr2=0x7fda4815f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.999+0000 7fda4dc00640 1 --2- 172.21.15.83:0/2179546991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4806e950 0x7fda4815f910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fda30009870 tx=0x7fda3001c490 comp rx=0 tx=0).stop 2024-04-25T00:11:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.999+0000 7fda4dc00640 1 -- 172.21.15.83:0/2179546991 shutdown_connections 2024-04-25T00:11:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.999+0000 7fda4dc00640 1 --2- 172.21.15.83:0/2179546991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4806e950 0x7fda4815f910 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.999+0000 7fda4dc00640 1 --2- 172.21.15.83:0/2179546991 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda48159810 0x7fda480cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.999+0000 7fda4dc00640 1 -- 172.21.15.83:0/2179546991 >> 172.21.15.83:0/2179546991 conn(0x7fda480b1830 msgr2=0x7fda480b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.999+0000 7fda4dc00640 1 -- 172.21.15.83:0/2179546991 shutdown_connections 2024-04-25T00:11:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.999+0000 7fda4dc00640 1 -- 172.21.15.83:0/2179546991 wait complete. 2024-04-25T00:11:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.999+0000 7fda4dc00640 1 Processor -- start 2024-04-25T00:11:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:48.999+0000 7fda4dc00640 1 -- start start 2024-04-25T00:11:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda4dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4806e950 0x7fda481530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda4dc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda48159810 0x7fda4814e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda4dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda4815fe50 0x7fda480b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda4dc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda48159d70 con 0x7fda4806e950 2024-04-25T00:11:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda4dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda48159bf0 con 0x7fda4815fe50 2024-04-25T00:11:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda4dc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fda48070da0 con 0x7fda48159810 2024-04-25T00:11:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda46a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4806e950 0x7fda481530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda46a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4806e950 0x7fda481530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48772/0 (socket says 172.21.15.83:48772) 2024-04-25T00:11:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda46a00640 1 -- 172.21.15.83:0/3450521634 learned_addr learned my addr 172.21.15.83:0/3450521634 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda3fe00640 1 --2- 172.21.15.83:0/3450521634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda48159810 0x7fda4814e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda47400640 1 --2- 172.21.15.83:0/3450521634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda4815fe50 0x7fda480b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda46a00640 1 -- 172.21.15.83:0/3450521634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda48159810 msgr2=0x7fda4814e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda46a00640 1 --2- 172.21.15.83:0/3450521634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda48159810 0x7fda4814e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda46a00640 1 -- 172.21.15.83:0/3450521634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda4815fe50 msgr2=0x7fda480b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda46a00640 1 --2- 172.21.15.83:0/3450521634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda4815fe50 0x7fda480b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda46a00640 1 -- 172.21.15.83:0/3450521634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4814eb50 con 0x7fda4806e950 2024-04-25T00:11:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda46a00640 1 --2- 172.21.15.83:0/3450521634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4806e950 0x7fda481530a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fda3800c7c0 tx=0x7fda3800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda3ea00640 1 -- 172.21.15.83:0/3450521634 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38007a30 con 0x7fda4806e950 2024-04-25T00:11:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda3ea00640 1 -- 172.21.15.83:0/3450521634 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda38007bd0 con 0x7fda4806e950 2024-04-25T00:11:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda3ea00640 1 -- 172.21.15.83:0/3450521634 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda380113f0 con 0x7fda4806e950 2024-04-25T00:11:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda4dc00640 1 -- 172.21.15.83:0/3450521634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda48153780 con 0x7fda4806e950 2024-04-25T00:11:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda4dc00640 1 -- 172.21.15.83:0/3450521634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda48153d40 con 0x7fda4806e950 2024-04-25T00:11:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda4dc00640 1 -- 172.21.15.83:0/3450521634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda4804f510 con 0x7fda4806e950 2024-04-25T00:11:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.003+0000 7fda3ea00640 1 -- 172.21.15.83:0/3450521634 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fda38011590 con 0x7fda4806e950 2024-04-25T00:11:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.007+0000 7fda3ea00640 1 --2- 172.21.15.83:0/3450521634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fda240678c0 0x7fda24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.007+0000 7fda3ea00640 1 -- 172.21.15.83:0/3450521634 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fda38086320 con 0x7fda4806e950 2024-04-25T00:11:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.007+0000 7fda3fe00640 1 --2- 172.21.15.83:0/3450521634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fda240678c0 0x7fda24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.007+0000 7fda3fe00640 1 --2- 172.21.15.83:0/3450521634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fda240678c0 0x7fda24069d80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fda300af000 tx=0x7fda30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.011+0000 7fda3ea00640 1 -- 172.21.15.83:0/3450521634 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fda38015020 con 0x7fda4806e950 2024-04-25T00:11:49.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.163+0000 7fda4dc00640 1 -- 172.21.15.83:0/3450521634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda480cd3a0 con 0x7fda4806e950 2024-04-25T00:11:49.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.163+0000 7fda3ea00640 1 -- 172.21.15.83:0/3450521634 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fda380562e0 con 0x7fda4806e950 2024-04-25T00:11:49.169 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:11:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.167+0000 7fda4dc00640 1 -- 172.21.15.83:0/3450521634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fda240678c0 msgr2=0x7fda24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.167+0000 7fda4dc00640 1 --2- 172.21.15.83:0/3450521634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fda240678c0 0x7fda24069d80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fda300af000 tx=0x7fda30008040 comp rx=0 tx=0).stop 2024-04-25T00:11:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.167+0000 7fda4dc00640 1 -- 172.21.15.83:0/3450521634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4806e950 msgr2=0x7fda481530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.167+0000 7fda4dc00640 1 --2- 172.21.15.83:0/3450521634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4806e950 0x7fda481530a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fda3800c7c0 tx=0x7fda3800cc80 comp rx=0 tx=0).stop 2024-04-25T00:11:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.167+0000 7fda4dc00640 1 -- 172.21.15.83:0/3450521634 shutdown_connections 2024-04-25T00:11:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.167+0000 7fda4dc00640 1 --2- 172.21.15.83:0/3450521634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fda240678c0 0x7fda24069d80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.167+0000 7fda4dc00640 1 --2- 172.21.15.83:0/3450521634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda48159810 0x7fda4814e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.167+0000 7fda4dc00640 1 --2- 172.21.15.83:0/3450521634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda4815fe50 0x7fda480b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.167+0000 7fda4dc00640 1 --2- 172.21.15.83:0/3450521634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4806e950 0x7fda481530a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.167+0000 7fda4dc00640 1 -- 172.21.15.83:0/3450521634 >> 172.21.15.83:0/3450521634 conn(0x7fda480b1830 msgr2=0x7fda480cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:49.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.167+0000 7fda4dc00640 1 -- 172.21.15.83:0/3450521634 shutdown_connections 2024-04-25T00:11:49.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.167+0000 7fda4dc00640 1 -- 172.21.15.83:0/3450521634 wait complete. 2024-04-25T00:11:49.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:11:49.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.323+0000 7f27dae00640 1 Processor -- start 2024-04-25T00:11:49.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.323+0000 7f27dae00640 1 -- start start 2024-04-25T00:11:49.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.323+0000 7f27dae00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f27d406e950 0x7f27d415f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27dae00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27d40ce000 con 0x7f27d4159810 2024-04-25T00:11:49.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27dae00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27d406eec0 con 0x7f27d415fe50 2024-04-25T00:11:49.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27dae00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f27d406e210 con 0x7f27d406e950 2024-04-25T00:11:49.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d3e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f27d4159810 0x7f27d40cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51830/0 (socket says 172.21.15.83:51830) 2024-04-25T00:11:49.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d3e00640 1 -- 172.21.15.83:0/2029983126 learned_addr learned my addr 172.21.15.83:0/2029983126 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:49.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d3400640 1 --2- 172.21.15.83:0/2029983126 >> v2:172.21.15.86:3300/0 conn(0x7f27d406e950 0x7f27d415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2700037647 0 0) 0x7f27d40ce000 con 0x7f27d4159810 2024-04-25T00:11:49.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27b8003530 con 0x7f27d4159810 2024-04-25T00:11:49.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3916090963 0 0) 0x7f27d406eec0 con 0x7f27d415fe50 2024-04-25T00:11:49.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27d40ce000 con 0x7f27d415fe50 2024-04-25T00:11:49.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 776746466 0 0) 0x7f27b8003530 con 0x7f27d4159810 2024-04-25T00:11:49.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 >> v2:172.21.15.86:3300/0 conn(0x7f27d406e950 msgr2=0x7f27d415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:49.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 --2- 172.21.15.83:0/2029983126 >> v2:172.21.15.86:3300/0 conn(0x7f27d406e950 0x7f27d415f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 >> v1:172.21.15.86:6789/0 conn(0x7f27d415fe50 legacy=0x7f27d40ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d406dc50 con 0x7f27d4159810 2024-04-25T00:11:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f27c4001a20 con 0x7f27d4159810 2024-04-25T00:11:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 >> v1:172.21.15.83:6789/0 conn(0x7f27d4159810 legacy=0x7f27d40cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 --2- 172.21.15.83:0/2029983126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27b8003bc0 0x7f27b8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 --2- 172.21.15.83:0/2029983126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27b8004810 0x7f27b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 --2- 172.21.15.83:0/2029983126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27b8005220 0x7f27b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f27d406e210 con 0x7f27b8005220 2024-04-25T00:11:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27b800c170 con 0x7f27b8004810 2024-04-25T00:11:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f27b800c370 con 0x7f27b8003bc0 2024-04-25T00:11:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d8e00640 1 --2- 172.21.15.83:0/2029983126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27b8005220 0x7f27b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d3400640 1 --2- 172.21.15.83:0/2029983126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27b8004810 0x7f27b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d3e00640 1 --2- 172.21.15.83:0/2029983126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27b8003bc0 0x7f27b8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d8e00640 1 -- 172.21.15.83:0/2029983126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27b8003bc0 msgr2=0x7f27b8003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d8e00640 1 --2- 172.21.15.83:0/2029983126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27b8003bc0 0x7f27b8003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d8e00640 1 -- 172.21.15.83:0/2029983126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27b8004810 msgr2=0x7f27b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:49.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d8e00640 1 --2- 172.21.15.83:0/2029983126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27b8004810 0x7f27b8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d8e00640 1 -- 172.21.15.83:0/2029983126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b8003530 con 0x7f27b8005220 2024-04-25T00:11:49.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.327+0000 7f27d3400640 1 --2- 172.21.15.83:0/2029983126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27b8004810 0x7f27b8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:49.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.331+0000 7f27d8e00640 1 -- 172.21.15.83:0/2029983126 reap_dead start 2024-04-25T00:11:49.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.331+0000 7f27d8e00640 1 --2- 172.21.15.83:0/2029983126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27b8005220 0x7f27b800b9d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f27cc007fd0 tx=0x7f27cc01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ddb31ad0be03d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.331+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc001520 con 0x7f27b8005220 2024-04-25T00:11:49.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.331+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27cc0052d0 con 0x7f27b8005220 2024-04-25T00:11:49.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.331+0000 7f27d2a00640 1 -- 172.21.15.83:0/2029983126 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc0a8930 con 0x7f27b8005220 2024-04-25T00:11:49.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.331+0000 7f27dae00640 1 -- 172.21.15.83:0/2029983126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27b8005220 msgr2=0x7f27b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.331+0000 7f27dae00640 1 --2- 172.21.15.83:0/2029983126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27b8005220 0x7f27b800b9d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f27cc007fd0 tx=0x7f27cc01ecd0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.331+0000 7f27dae00640 1 -- 172.21.15.83:0/2029983126 shutdown_connections 2024-04-25T00:11:49.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.331+0000 7f27dae00640 1 --2- 172.21.15.83:0/2029983126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27b8005220 0x7f27b800b9d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.331+0000 7f27dae00640 1 -- 172.21.15.83:0/2029983126 >> 172.21.15.83:0/2029983126 conn(0x7f27d40b1830 msgr2=0x7f27d40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:49.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.331+0000 7f27dae00640 1 -- 172.21.15.83:0/2029983126 shutdown_connections 2024-04-25T00:11:49.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.331+0000 7f27dae00640 1 -- 172.21.15.83:0/2029983126 wait complete. 2024-04-25T00:11:49.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.331+0000 7f27dae00640 1 Processor -- start 2024-04-25T00:11:49.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.331+0000 7f27dae00640 1 -- start start 2024-04-25T00:11:49.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27dae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27b8005220 0x7f27d41f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27dae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d414d660 0x7f27d4150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27dae00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27d4155b20 0x7f27d4142f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27dae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f27d406da00 con 0x7f27b8005220 2024-04-25T00:11:49.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27dae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27d41602a0 con 0x7f27d414d660 2024-04-25T00:11:49.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27dae00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f27d4159bf0 con 0x7f27d4155b20 2024-04-25T00:11:49.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27d3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27b8005220 0x7f27d41f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27d3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27b8005220 0x7f27d41f1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48790/0 (socket says 172.21.15.83:48790) 2024-04-25T00:11:49.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27d3e00640 1 -- 172.21.15.83:0/1465600633 learned_addr learned my addr 172.21.15.83:0/1465600633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:49.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27d3400640 1 --2- 172.21.15.83:0/1465600633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d414d660 0x7f27d4150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27d8e00640 1 --2- 172.21.15.83:0/1465600633 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27d4155b20 0x7f27d4142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27d3e00640 1 -- 172.21.15.83:0/1465600633 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27d4155b20 msgr2=0x7f27d4142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27d3e00640 1 --2- 172.21.15.83:0/1465600633 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27d4155b20 0x7f27d4142f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27d3e00640 1 -- 172.21.15.83:0/1465600633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d414d660 msgr2=0x7f27d4150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27d3e00640 1 --2- 172.21.15.83:0/1465600633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d414d660 0x7f27d4150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27d3e00640 1 -- 172.21.15.83:0/1465600633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d4152b40 con 0x7f27b8005220 2024-04-25T00:11:49.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27d3e00640 1 --2- 172.21.15.83:0/1465600633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27b8005220 0x7f27d41f1f10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f27c40079e0 tx=0x7f27c400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27cbe00640 1 -- 172.21.15.83:0/1465600633 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4007ad0 con 0x7f27b8005220 2024-04-25T00:11:49.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27cbe00640 1 -- 172.21.15.83:0/1465600633 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27c4018070 con 0x7f27b8005220 2024-04-25T00:11:49.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27dae00640 1 -- 172.21.15.83:0/1465600633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d4152e20 con 0x7f27b8005220 2024-04-25T00:11:49.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27cbe00640 1 -- 172.21.15.83:0/1465600633 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4013640 con 0x7f27b8005220 2024-04-25T00:11:49.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27dae00640 1 -- 172.21.15.83:0/1465600633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d41533e0 con 0x7f27b8005220 2024-04-25T00:11:49.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.335+0000 7f27dae00640 1 -- 172.21.15.83:0/1465600633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d404f510 con 0x7f27b8005220 2024-04-25T00:11:49.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.339+0000 7f27cbe00640 1 -- 172.21.15.83:0/1465600633 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f27c40137e0 con 0x7f27b8005220 2024-04-25T00:11:49.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.339+0000 7f27cbe00640 1 --2- 172.21.15.83:0/1465600633 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27b4067a60 0x7f27b4069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.339+0000 7f27cbe00640 1 -- 172.21.15.83:0/1465600633 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f27c40870a0 con 0x7f27b8005220 2024-04-25T00:11:49.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.339+0000 7f27d3400640 1 --2- 172.21.15.83:0/1465600633 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27b4067a60 0x7f27b4069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.339+0000 7f27d3400640 1 --2- 172.21.15.83:0/1465600633 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27b4067a60 0x7f27b4069f20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f27d4143ca0 tx=0x7f27c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.343+0000 7f27cbe00640 1 -- 172.21.15.83:0/1465600633 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f27c4057830 con 0x7f27b8005220 2024-04-25T00:11:49.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.499+0000 7f27dae00640 1 -- 172.21.15.83:0/1465600633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f27d41535b0 con 0x7f27b8005220 2024-04-25T00:11:49.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.499+0000 7f27cbe00640 1 -- 172.21.15.83:0/1465600633 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f27c4057130 con 0x7f27b8005220 2024-04-25T00:11:49.502 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.499+0000 7f27dae00640 1 -- 172.21.15.83:0/1465600633 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27b4067a60 msgr2=0x7f27b4069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.499+0000 7f27dae00640 1 --2- 172.21.15.83:0/1465600633 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27b4067a60 0x7f27b4069f20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f27d4143ca0 tx=0x7f27c0008040 comp rx=0 tx=0).stop 2024-04-25T00:11:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.499+0000 7f27dae00640 1 -- 172.21.15.83:0/1465600633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27b8005220 msgr2=0x7f27d41f1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.499+0000 7f27dae00640 1 --2- 172.21.15.83:0/1465600633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27b8005220 0x7f27d41f1f10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f27c40079e0 tx=0x7f27c400e530 comp rx=0 tx=0).stop 2024-04-25T00:11:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.499+0000 7f27dae00640 1 -- 172.21.15.83:0/1465600633 shutdown_connections 2024-04-25T00:11:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.499+0000 7f27dae00640 1 --2- 172.21.15.83:0/1465600633 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27b4067a60 0x7f27b4069f20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.499+0000 7f27dae00640 1 --2- 172.21.15.83:0/1465600633 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27d4155b20 0x7f27d4142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.499+0000 7f27dae00640 1 --2- 172.21.15.83:0/1465600633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d414d660 0x7f27d4150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.499+0000 7f27dae00640 1 --2- 172.21.15.83:0/1465600633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27b8005220 0x7f27d41f1f10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.499+0000 7f27dae00640 1 -- 172.21.15.83:0/1465600633 >> 172.21.15.83:0/1465600633 conn(0x7f27d40b1830 msgr2=0x7f27d415e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.499+0000 7f27dae00640 1 -- 172.21.15.83:0/1465600633 shutdown_connections 2024-04-25T00:11:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.499+0000 7f27dae00640 1 -- 172.21.15.83:0/1465600633 wait complete. 2024-04-25T00:11:49.523 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:11:49.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.655+0000 7fc246000640 1 Processor -- start 2024-04-25T00:11:49.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.655+0000 7fc246000640 1 -- start start 2024-04-25T00:11:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.655+0000 7fc246000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc240159810 0x7fc2400cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.655+0000 7fc246000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc240160020 0x7fc2400ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.655+0000 7fc246000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2400ce000 con 0x7fc24006e950 2024-04-25T00:11:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.655+0000 7fc246000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc24006e210 con 0x7fc240160020 2024-04-25T00:11:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.655+0000 7fc246000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc24006da00 con 0x7fc240159810 2024-04-25T00:11:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.659+0000 7fc23fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc240160020 0x7fc2400ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.659+0000 7fc23fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc240160020 0x7fc2400ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48794/0 (socket says 172.21.15.83:48794) 2024-04-25T00:11:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.659+0000 7fc23fe00640 1 -- 172.21.15.83:0/79759578 learned_addr learned my addr 172.21.15.83:0/79759578 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.659+0000 7fc23ea00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc24006e950 0x7fc24015f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51834/0 (socket says 172.21.15.83:51834) 2024-04-25T00:11:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.659+0000 7fc23f400640 1 --2- 172.21.15.83:0/79759578 >> v2:172.21.15.86:3300/0 conn(0x7fc240159810 0x7fc2400cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.659+0000 7fc23fe00640 1 -- 172.21.15.83:0/79759578 >> v2:172.21.15.86:3300/0 conn(0x7fc240159810 msgr2=0x7fc2400cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.659+0000 7fc23fe00640 1 --2- 172.21.15.83:0/79759578 >> v2:172.21.15.86:3300/0 conn(0x7fc240159810 0x7fc2400cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.659+0000 7fc23fe00640 1 -- 172.21.15.83:0/79759578 >> v1:172.21.15.83:6789/0 conn(0x7fc24006e950 legacy=0x7fc24015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.659+0000 7fc23fe00640 1 -- 172.21.15.83:0/79759578 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24006eec0 con 0x7fc240160020 2024-04-25T00:11:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.659+0000 7fc23e000640 1 -- 172.21.15.83:0/79759578 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1638998106 0 0) 0x7fc2400ce000 con 0x7fc24006e950 2024-04-25T00:11:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.659+0000 7fc23fe00640 1 --2- 172.21.15.83:0/79759578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc240160020 0x7fc2400ce4a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc24006db80 tx=0x7fc23801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9211248cff5104 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.659+0000 7fc23e000640 1 -- 172.21.15.83:0/79759578 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2380a57f0 con 0x7fc240160020 2024-04-25T00:11:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.659+0000 7fc23e000640 1 -- 172.21.15.83:0/79759578 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2380a5990 con 0x7fc240160020 2024-04-25T00:11:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.659+0000 7fc23e000640 1 -- 172.21.15.83:0/79759578 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2380a5d60 con 0x7fc240160020 2024-04-25T00:11:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 -- 172.21.15.83:0/79759578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc240160020 msgr2=0x7fc2400ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 --2- 172.21.15.83:0/79759578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc240160020 0x7fc2400ce4a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc24006db80 tx=0x7fc23801c930 comp rx=0 tx=0).stop 2024-04-25T00:11:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 -- 172.21.15.83:0/79759578 shutdown_connections 2024-04-25T00:11:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 --2- 172.21.15.83:0/79759578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc240160020 0x7fc2400ce4a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 --2- 172.21.15.83:0/79759578 >> v2:172.21.15.86:3300/0 conn(0x7fc240159810 0x7fc2400cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 -- 172.21.15.83:0/79759578 >> 172.21.15.83:0/79759578 conn(0x7fc2400b1830 msgr2=0x7fc2400b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:49.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 -- 172.21.15.83:0/79759578 shutdown_connections 2024-04-25T00:11:49.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 -- 172.21.15.83:0/79759578 wait complete. 2024-04-25T00:11:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 Processor -- start 2024-04-25T00:11:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 -- start start 2024-04-25T00:11:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc24006e950 0x7fc24014f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc240159810 0x7fc240155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc240160020 0x7fc2401560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc240159d70 con 0x7fc240160020 2024-04-25T00:11:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc240159bf0 con 0x7fc240159810 2024-04-25T00:11:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc246000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc240070da0 con 0x7fc24006e950 2024-04-25T00:11:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc23fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc240160020 0x7fc2401560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc23fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc240160020 0x7fc2401560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48810/0 (socket says 172.21.15.83:48810) 2024-04-25T00:11:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc23fe00640 1 -- 172.21.15.83:0/2408658977 learned_addr learned my addr 172.21.15.83:0/2408658977 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc23ea00640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc240159810 0x7fc240155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc23f400640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc24006e950 0x7fc24014f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc23fe00640 1 -- 172.21.15.83:0/2408658977 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc24006e950 msgr2=0x7fc24014f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc23fe00640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc24006e950 0x7fc24014f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc23fe00640 1 -- 172.21.15.83:0/2408658977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc240159810 msgr2=0x7fc240155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc23fe00640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc240159810 0x7fc240155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc23fe00640 1 -- 172.21.15.83:0/2408658977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2401583b0 con 0x7fc240160020 2024-04-25T00:11:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc23ea00640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc240159810 0x7fc240155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc23fe00640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc240160020 0x7fc2401560a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc23801afe0 tx=0x7fc2380196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.663+0000 7fc237400640 1 -- 172.21.15.83:0/2408658977 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2380bc030 con 0x7fc240160020 2024-04-25T00:11:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.667+0000 7fc237400640 1 -- 172.21.15.83:0/2408658977 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2380b8470 con 0x7fc240160020 2024-04-25T00:11:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.667+0000 7fc246000640 1 -- 172.21.15.83:0/2408658977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24013a200 con 0x7fc240160020 2024-04-25T00:11:49.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.667+0000 7fc237400640 1 -- 172.21.15.83:0/2408658977 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2380b8820 con 0x7fc240160020 2024-04-25T00:11:49.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.667+0000 7fc246000640 1 -- 172.21.15.83:0/2408658977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc240157680 con 0x7fc240160020 2024-04-25T00:11:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.667+0000 7fc246000640 1 -- 172.21.15.83:0/2408658977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc240139da0 con 0x7fc240160020 2024-04-25T00:11:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.667+0000 7fc237400640 1 -- 172.21.15.83:0/2408658977 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc2380bf070 con 0x7fc240160020 2024-04-25T00:11:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.667+0000 7fc237400640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2200678f0 0x7fc220069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.667+0000 7fc237400640 1 -- 172.21.15.83:0/2408658977 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc238120ea0 con 0x7fc240160020 2024-04-25T00:11:49.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.667+0000 7fc23f400640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2200678f0 0x7fc220069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.667+0000 7fc23f400640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2200678f0 0x7fc220069db0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc230009af0 tx=0x7fc230008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.671+0000 7fc237400640 1 -- 172.21.15.83:0/2408658977 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc240139da0 con 0x7fc240160020 2024-04-25T00:11:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.827+0000 7fc246000640 1 -- 172.21.15.83:0/2408658977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc24004f510 con 0x7fc240160020 2024-04-25T00:11:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.827+0000 7fc237400640 1 -- 172.21.15.83:0/2408658977 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fc2380165e0 con 0x7fc240160020 2024-04-25T00:11:49.833 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:11:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.827+0000 7fc246000640 1 -- 172.21.15.83:0/2408658977 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2200678f0 msgr2=0x7fc220069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.827+0000 7fc246000640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2200678f0 0x7fc220069db0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc230009af0 tx=0x7fc230008040 comp rx=0 tx=0).stop 2024-04-25T00:11:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.827+0000 7fc246000640 1 -- 172.21.15.83:0/2408658977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc240160020 msgr2=0x7fc2401560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.827+0000 7fc246000640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc240160020 0x7fc2401560a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc23801afe0 tx=0x7fc2380196d0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.827+0000 7fc246000640 1 -- 172.21.15.83:0/2408658977 shutdown_connections 2024-04-25T00:11:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.827+0000 7fc246000640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2200678f0 0x7fc220069db0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.827+0000 7fc246000640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc240160020 0x7fc2401560a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.827+0000 7fc246000640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc240159810 0x7fc240155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.827+0000 7fc246000640 1 --2- 172.21.15.83:0/2408658977 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc24006e950 0x7fc24014f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.827+0000 7fc246000640 1 -- 172.21.15.83:0/2408658977 >> 172.21.15.83:0/2408658977 conn(0x7fc2400b1830 msgr2=0x7fc2400cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.827+0000 7fc246000640 1 -- 172.21.15.83:0/2408658977 shutdown_connections 2024-04-25T00:11:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.827+0000 7fc246000640 1 -- 172.21.15.83:0/2408658977 wait complete. 2024-04-25T00:11:49.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:11:49.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.983+0000 7fd48e000640 1 Processor -- start 2024-04-25T00:11:49.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd48e000640 1 -- start start 2024-04-25T00:11:49.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd48e000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd48806e950 0x7fd48815f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd48e000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd48815fe50 0x7fd4880ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd48e000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4880ce000 con 0x7fd488159810 2024-04-25T00:11:49.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd48e000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd48806e210 con 0x7fd48806e950 2024-04-25T00:11:49.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd48e000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd48806da00 con 0x7fd48815fe50 2024-04-25T00:11:49.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd487400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd488159810 0x7fd4880cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51846/0 (socket says 172.21.15.83:51846) 2024-04-25T00:11:49.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd487400640 1 -- 172.21.15.83:0/2121187866 learned_addr learned my addr 172.21.15.83:0/2121187866 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:49.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd486a00640 1 --2- 172.21.15.83:0/2121187866 >> v2:172.21.15.83:3300/0 conn(0x7fd48806e950 0x7fd48815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd487e00640 1 --2- 172.21.15.83:0/2121187866 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd48815fe50 0x7fd4880ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd486a00640 1 -- 172.21.15.83:0/2121187866 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd48815fe50 msgr2=0x7fd4880ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd486a00640 1 --2- 172.21.15.83:0/2121187866 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd48815fe50 0x7fd4880ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd486000640 1 -- 172.21.15.83:0/2121187866 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3021674249 0 0) 0x7fd4880ce000 con 0x7fd488159810 2024-04-25T00:11:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd486a00640 1 -- 172.21.15.83:0/2121187866 >> v1:172.21.15.83:6789/0 conn(0x7fd488159810 legacy=0x7fd4880cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.987+0000 7fd486a00640 1 -- 172.21.15.83:0/2121187866 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48806eec0 con 0x7fd48806e950 2024-04-25T00:11:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.991+0000 7fd486a00640 1 --2- 172.21.15.83:0/2121187866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48806e950 0x7fd48815f910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd48806db80 tx=0x7fd47401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78744c7154bdf11c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.991+0000 7fd486000640 1 -- 172.21.15.83:0/2121187866 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740b0070 con 0x7fd48806e950 2024-04-25T00:11:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.991+0000 7fd486000640 1 -- 172.21.15.83:0/2121187866 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4740ab600 con 0x7fd48806e950 2024-04-25T00:11:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.991+0000 7fd486000640 1 -- 172.21.15.83:0/2121187866 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740ab9d0 con 0x7fd48806e950 2024-04-25T00:11:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.991+0000 7fd48e000640 1 -- 172.21.15.83:0/2121187866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48806e950 msgr2=0x7fd48815f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.991+0000 7fd48e000640 1 --2- 172.21.15.83:0/2121187866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48806e950 0x7fd48815f910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd48806db80 tx=0x7fd47401c410 comp rx=0 tx=0).stop 2024-04-25T00:11:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.991+0000 7fd48e000640 1 -- 172.21.15.83:0/2121187866 shutdown_connections 2024-04-25T00:11:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.991+0000 7fd48e000640 1 --2- 172.21.15.83:0/2121187866 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd48815fe50 0x7fd4880ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.991+0000 7fd48e000640 1 --2- 172.21.15.83:0/2121187866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48806e950 0x7fd48815f910 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.991+0000 7fd48e000640 1 -- 172.21.15.83:0/2121187866 >> 172.21.15.83:0/2121187866 conn(0x7fd4880b1830 msgr2=0x7fd4880b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:49.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.991+0000 7fd48e000640 1 -- 172.21.15.83:0/2121187866 shutdown_connections 2024-04-25T00:11:49.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.991+0000 7fd48e000640 1 -- 172.21.15.83:0/2121187866 wait complete. 2024-04-25T00:11:49.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.991+0000 7fd48e000640 1 Processor -- start 2024-04-25T00:11:49.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.991+0000 7fd48e000640 1 -- start start 2024-04-25T00:11:49.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd48e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48806e950 0x7fd4881494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd48e000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd488159810 0x7fd4881583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd48e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd48815fe50 0x7fd48815b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd48e000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd488070f20 con 0x7fd48806e950 2024-04-25T00:11:49.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd48e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd488070da0 con 0x7fd48815fe50 2024-04-25T00:11:49.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd48e000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd488168cb0 con 0x7fd488159810 2024-04-25T00:11:49.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd487400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48806e950 0x7fd4881494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd487400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48806e950 0x7fd4881494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48820/0 (socket says 172.21.15.83:48820) 2024-04-25T00:11:49.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd487400640 1 -- 172.21.15.83:0/1413707824 learned_addr learned my addr 172.21.15.83:0/1413707824 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:49.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd487e00640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd48815fe50 0x7fd48815b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd486a00640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd488159810 0x7fd4881583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd487400640 1 -- 172.21.15.83:0/1413707824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd488159810 msgr2=0x7fd4881583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd487400640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd488159810 0x7fd4881583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd487400640 1 -- 172.21.15.83:0/1413707824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd48815fe50 msgr2=0x7fd48815b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd487400640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd48815fe50 0x7fd48815b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd487400640 1 -- 172.21.15.83:0/1413707824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd488165a20 con 0x7fd48806e950 2024-04-25T00:11:50.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd487e00640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd48815fe50 0x7fd48815b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:50.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd487400640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48806e950 0x7fd4881494e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd478001570 tx=0x7fd47800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:50.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd47f400640 1 -- 172.21.15.83:0/1413707824 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478018070 con 0x7fd48806e950 2024-04-25T00:11:50.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd47f400640 1 -- 172.21.15.83:0/1413707824 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47800a640 con 0x7fd48806e950 2024-04-25T00:11:50.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd47f400640 1 -- 172.21.15.83:0/1413707824 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47800aa30 con 0x7fd48806e950 2024-04-25T00:11:50.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd48e000640 1 -- 172.21.15.83:0/1413707824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd488165d00 con 0x7fd48806e950 2024-04-25T00:11:50.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd48e000640 1 -- 172.21.15.83:0/1413707824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4881662c0 con 0x7fd48806e950 2024-04-25T00:11:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd48e000640 1 -- 172.21.15.83:0/1413707824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48804f510 con 0x7fd48806e950 2024-04-25T00:11:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.995+0000 7fd47f400640 1 -- 172.21.15.83:0/1413707824 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd47800abd0 con 0x7fd48806e950 2024-04-25T00:11:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.999+0000 7fd47f400640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd4680678c0 0x7fd468069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.999+0000 7fd47f400640 1 -- 172.21.15.83:0/1413707824 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd4780856b0 con 0x7fd48806e950 2024-04-25T00:11:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.999+0000 7fd486a00640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd4680678c0 0x7fd468069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:50.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:49.999+0000 7fd486a00640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd4680678c0 0x7fd468069d80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd4740ad000 tx=0x7fd4740a4f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.003+0000 7fd47f400640 1 -- 172.21.15.83:0/1413707824 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd478055830 con 0x7fd48806e950 2024-04-25T00:11:50.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.259+0000 7fd48e000640 1 -- 172.21.15.83:0/1413707824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd4880b4550 con 0x7fd48806e950 2024-04-25T00:11:50.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.259+0000 7fd47f400640 1 -- 172.21.15.83:0/1413707824 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fd478059b30 con 0x7fd48806e950 2024-04-25T00:11:50.265 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:50.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.263+0000 7fd48e000640 1 -- 172.21.15.83:0/1413707824 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd4680678c0 msgr2=0x7fd468069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.263+0000 7fd48e000640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd4680678c0 0x7fd468069d80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd4740ad000 tx=0x7fd4740a4f90 comp rx=0 tx=0).stop 2024-04-25T00:11:50.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.263+0000 7fd48e000640 1 -- 172.21.15.83:0/1413707824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48806e950 msgr2=0x7fd4881494e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.263+0000 7fd48e000640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48806e950 0x7fd4881494e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd478001570 tx=0x7fd47800cd20 comp rx=0 tx=0).stop 2024-04-25T00:11:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.263+0000 7fd48e000640 1 -- 172.21.15.83:0/1413707824 shutdown_connections 2024-04-25T00:11:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.263+0000 7fd48e000640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd4680678c0 0x7fd468069d80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.263+0000 7fd48e000640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd488159810 0x7fd4881583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.263+0000 7fd48e000640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd48815fe50 0x7fd48815b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.263+0000 7fd48e000640 1 --2- 172.21.15.83:0/1413707824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48806e950 0x7fd4881494e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.263+0000 7fd48e000640 1 -- 172.21.15.83:0/1413707824 >> 172.21.15.83:0/1413707824 conn(0x7fd4880b1830 msgr2=0x7fd4880710d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.263+0000 7fd48e000640 1 -- 172.21.15.83:0/1413707824 shutdown_connections 2024-04-25T00:11:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.263+0000 7fd48e000640 1 -- 172.21.15.83:0/1413707824 wait complete. 2024-04-25T00:11:50.286 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:50.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.419+0000 7fb702000640 1 Processor -- start 2024-04-25T00:11:50.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.419+0000 7fb702000640 1 -- start start 2024-04-25T00:11:50.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb702000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb6fc06f010 0x7fb6fc0d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:50.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb702000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb6fc0d0510 0x7fb6fc06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:50.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb702000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6fc0d0950 con 0x7fb6fc0b0310 2024-04-25T00:11:50.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb702000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb6fc070990 con 0x7fb6fc06f010 2024-04-25T00:11:50.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb702000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb6fc06fd70 con 0x7fb6fc0d0510 2024-04-25T00:11:50.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb700c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb6fc06f010 0x7fb6fc0d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:50.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb700c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb6fc06f010 0x7fb6fc0d26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48826/0 (socket says 172.21.15.83:48826) 2024-04-25T00:11:50.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb700c00640 1 -- 172.21.15.83:0/641191043 learned_addr learned my addr 172.21.15.83:0/641191043 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:50.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb6f3e00640 1 --2- 172.21.15.83:0/641191043 >> v2:172.21.15.86:3300/0 conn(0x7fb6fc0d0510 0x7fb6fc06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:50.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb701600640 1 --1- 172.21.15.83:0/641191043 >> v1:172.21.15.83:6789/0 conn(0x7fb6fc0b0310 0x7fb6fc0b0780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51860/0 (socket says 172.21.15.83:51860) 2024-04-25T00:11:50.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb700c00640 1 -- 172.21.15.83:0/641191043 >> v2:172.21.15.86:3300/0 conn(0x7fb6fc0d0510 msgr2=0x7fb6fc06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:50.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb700c00640 1 --2- 172.21.15.83:0/641191043 >> v2:172.21.15.86:3300/0 conn(0x7fb6fc0d0510 0x7fb6fc06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb700c00640 1 -- 172.21.15.83:0/641191043 >> v1:172.21.15.83:6789/0 conn(0x7fb6fc0b0310 legacy=0x7fb6fc0b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb700c00640 1 -- 172.21.15.83:0/641191043 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc070340 con 0x7fb6fc06f010 2024-04-25T00:11:50.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb6fbe00640 1 -- 172.21.15.83:0/641191043 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 778887122 0 0) 0x7fb6f40014a0 con 0x7fb6fc0b0310 2024-04-25T00:11:50.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.423+0000 7fb700c00640 1 --2- 172.21.15.83:0/641191043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6fc06f010 0x7fb6fc0d26e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc071fc0 tx=0x7fb6ec01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3bf3b6cb0c6ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:50.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb6fbe00640 1 -- 172.21.15.83:0/641191043 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0ab680 con 0x7fb6fc06f010 2024-04-25T00:11:50.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb6fbe00640 1 -- 172.21.15.83:0/641191043 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ec0b0070 con 0x7fb6fc06f010 2024-04-25T00:11:50.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb6fbe00640 1 -- 172.21.15.83:0/641191043 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0aba40 con 0x7fb6fc06f010 2024-04-25T00:11:50.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 -- 172.21.15.83:0/641191043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6fc06f010 msgr2=0x7fb6fc0d26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 --2- 172.21.15.83:0/641191043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6fc06f010 0x7fb6fc0d26e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc071fc0 tx=0x7fb6ec01c460 comp rx=0 tx=0).stop 2024-04-25T00:11:50.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 -- 172.21.15.83:0/641191043 shutdown_connections 2024-04-25T00:11:50.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 --2- 172.21.15.83:0/641191043 >> v2:172.21.15.86:3300/0 conn(0x7fb6fc0d0510 0x7fb6fc06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 --2- 172.21.15.83:0/641191043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6fc06f010 0x7fb6fc0d26e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 -- 172.21.15.83:0/641191043 >> 172.21.15.83:0/641191043 conn(0x7fb6fc0d1bc0 msgr2=0x7fb6fc0d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 -- 172.21.15.83:0/641191043 shutdown_connections 2024-04-25T00:11:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 -- 172.21.15.83:0/641191043 wait complete. 2024-04-25T00:11:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 Processor -- start 2024-04-25T00:11:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 -- start start 2024-04-25T00:11:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6fc06f010 0x7fb6fc1565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6fc0b0310 0x7fb6fc0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6fc0d0510 0x7fb6fc0b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc06fab0 con 0x7fb6fc06f010 2024-04-25T00:11:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc06f930 con 0x7fb6fc0b0310 2024-04-25T00:11:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb702000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb6fc1605a0 con 0x7fb6fc0d0510 2024-04-25T00:11:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb700c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6fc06f010 0x7fb6fc1565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb700c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6fc06f010 0x7fb6fc1565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48828/0 (socket says 172.21.15.83:48828) 2024-04-25T00:11:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb700c00640 1 -- 172.21.15.83:0/264943138 learned_addr learned my addr 172.21.15.83:0/264943138 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb6f3e00640 1 --2- 172.21.15.83:0/264943138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6fc0b0310 0x7fb6fc0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.427+0000 7fb701600640 1 --2- 172.21.15.83:0/264943138 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6fc0d0510 0x7fb6fc0b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb700c00640 1 -- 172.21.15.83:0/264943138 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6fc0d0510 msgr2=0x7fb6fc0b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb700c00640 1 --2- 172.21.15.83:0/264943138 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6fc0d0510 0x7fb6fc0b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb700c00640 1 -- 172.21.15.83:0/264943138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6fc0b0310 msgr2=0x7fb6fc0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb700c00640 1 --2- 172.21.15.83:0/264943138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6fc0b0310 0x7fb6fc0caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb700c00640 1 -- 172.21.15.83:0/264943138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc0c8d30 con 0x7fb6fc06f010 2024-04-25T00:11:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb700c00640 1 --2- 172.21.15.83:0/264943138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6fc06f010 0x7fb6fc1565e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0ad040 tx=0x7fb6ec01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb6f9600640 1 -- 172.21.15.83:0/264943138 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0be5f0 con 0x7fb6fc06f010 2024-04-25T00:11:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb6f9600640 1 -- 172.21.15.83:0/264943138 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ec0a7070 con 0x7fb6fc06f010 2024-04-25T00:11:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb6f9600640 1 -- 172.21.15.83:0/264943138 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0be9f0 con 0x7fb6fc06f010 2024-04-25T00:11:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb702000640 1 -- 172.21.15.83:0/264943138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6fc0cb300 con 0x7fb6fc06f010 2024-04-25T00:11:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb702000640 1 -- 172.21.15.83:0/264943138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6fc0caff0 con 0x7fb6fc06f010 2024-04-25T00:11:50.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb702000640 1 -- 172.21.15.83:0/264943138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6fc04f510 con 0x7fb6fc06f010 2024-04-25T00:11:50.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb6f9600640 1 -- 172.21.15.83:0/264943138 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb6ec0b43f0 con 0x7fb6fc06f010 2024-04-25T00:11:50.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb6f9600640 1 --2- 172.21.15.83:0/264943138 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6d80678f0 0x7fb6d8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:50.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.431+0000 7fb6f9600640 1 -- 172.21.15.83:0/264943138 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb6ec1207b0 con 0x7fb6fc06f010 2024-04-25T00:11:50.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.435+0000 7fb6f3e00640 1 --2- 172.21.15.83:0/264943138 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6d80678f0 0x7fb6d8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:50.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.435+0000 7fb6f3e00640 1 --2- 172.21.15.83:0/264943138 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6d80678f0 0x7fb6d8069db0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc15ec30 tx=0x7fb6e4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.435+0000 7fb6f9600640 1 -- 172.21.15.83:0/264943138 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb6ec0c4060 con 0x7fb6fc06f010 2024-04-25T00:11:50.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.595+0000 7fb702000640 1 -- 172.21.15.83:0/264943138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6fc0cdb40 con 0x7fb6fc06f010 2024-04-25T00:11:50.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.595+0000 7fb6f9600640 1 -- 172.21.15.83:0/264943138 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18820 (secure 0 0 0) 0x7fb6ec016110 con 0x7fb6fc06f010 2024-04-25T00:11:50.601 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:50.601 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:48.235809+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:50.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.599+0000 7fb702000640 1 -- 172.21.15.83:0/264943138 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6d80678f0 msgr2=0x7fb6d8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.599+0000 7fb702000640 1 --2- 172.21.15.83:0/264943138 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6d80678f0 0x7fb6d8069db0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc15ec30 tx=0x7fb6e4009190 comp rx=0 tx=0).stop 2024-04-25T00:11:50.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.599+0000 7fb702000640 1 -- 172.21.15.83:0/264943138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6fc06f010 msgr2=0x7fb6fc1565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.599+0000 7fb702000640 1 --2- 172.21.15.83:0/264943138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6fc06f010 0x7fb6fc1565e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0ad040 tx=0x7fb6ec01aca0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.599+0000 7fb702000640 1 -- 172.21.15.83:0/264943138 shutdown_connections 2024-04-25T00:11:50.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.599+0000 7fb702000640 1 --2- 172.21.15.83:0/264943138 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6d80678f0 0x7fb6d8069db0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.599+0000 7fb702000640 1 --2- 172.21.15.83:0/264943138 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6fc0d0510 0x7fb6fc0b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.599+0000 7fb702000640 1 --2- 172.21.15.83:0/264943138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6fc0b0310 0x7fb6fc0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.599+0000 7fb702000640 1 --2- 172.21.15.83:0/264943138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6fc06f010 0x7fb6fc1565e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.599+0000 7fb702000640 1 -- 172.21.15.83:0/264943138 >> 172.21.15.83:0/264943138 conn(0x7fb6fc0d1bc0 msgr2=0x7fb6fc15b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:50.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.599+0000 7fb702000640 1 -- 172.21.15.83:0/264943138 shutdown_connections 2024-04-25T00:11:50.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.599+0000 7fb702000640 1 -- 172.21.15.83:0/264943138 wait complete. 2024-04-25T00:11:50.623 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:11:04.621429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:11:05.189716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:11:08.902215+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:11:50.623 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:11:50.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.759+0000 7fe175c00640 1 Processor -- start 2024-04-25T00:11:50.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.759+0000 7fe175c00640 1 -- start start 2024-04-25T00:11:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.759+0000 7fe175c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe17015e0e0 0x7fe1700d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.759+0000 7fe175c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe17015e4c0 0x7fe1700ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.759+0000 7fe175c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1700b4b10 con 0x7fe1700cea80 2024-04-25T00:11:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.759+0000 7fe175c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe1700d0f50 con 0x7fe17015e4c0 2024-04-25T00:11:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.759+0000 7fe175c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe1700cc310 con 0x7fe17015e0e0 2024-04-25T00:11:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.759+0000 7fe167e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe17015e4c0 0x7fe1700ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.759+0000 7fe16f400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe1700cea80 0x7fe17006c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51866/0 (socket says 172.21.15.83:51866) 2024-04-25T00:11:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.759+0000 7fe16f400640 1 -- 172.21.15.83:0/2836411822 learned_addr learned my addr 172.21.15.83:0/2836411822 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.759+0000 7fe16ea00640 1 --2- 172.21.15.83:0/2836411822 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe17015e0e0 0x7fe1700d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.759+0000 7fe167e00640 1 -- 172.21.15.83:0/2836411822 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe17015e0e0 msgr2=0x7fe1700d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.759+0000 7fe167e00640 1 --2- 172.21.15.83:0/2836411822 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe17015e0e0 0x7fe1700d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe167e00640 1 -- 172.21.15.83:0/2836411822 >> v1:172.21.15.83:6789/0 conn(0x7fe1700cea80 legacy=0x7fe17006c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe16e000640 1 -- 172.21.15.83:0/2836411822 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4056042886 0 0) 0x7fe1680014a0 con 0x7fe1700cea80 2024-04-25T00:11:50.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe167e00640 1 -- 172.21.15.83:0/2836411822 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1700d1180 con 0x7fe17015e4c0 2024-04-25T00:11:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe167e00640 1 --2- 172.21.15.83:0/2836411822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17015e4c0 0x7fe1700ce540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe1580097c0 tx=0x7fe15801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac12e5fe692a74c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe16e000640 1 -- 172.21.15.83:0/2836411822 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580b2070 con 0x7fe17015e4c0 2024-04-25T00:11:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe16e000640 1 -- 172.21.15.83:0/2836411822 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1580ad640 con 0x7fe17015e4c0 2024-04-25T00:11:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe16e000640 1 -- 172.21.15.83:0/2836411822 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580ada10 con 0x7fe17015e4c0 2024-04-25T00:11:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe175c00640 1 -- 172.21.15.83:0/2836411822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17015e4c0 msgr2=0x7fe1700ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe175c00640 1 --2- 172.21.15.83:0/2836411822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17015e4c0 0x7fe1700ce540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe1580097c0 tx=0x7fe15801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe175c00640 1 -- 172.21.15.83:0/2836411822 shutdown_connections 2024-04-25T00:11:50.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe175c00640 1 --2- 172.21.15.83:0/2836411822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17015e4c0 0x7fe1700ce540 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe175c00640 1 --2- 172.21.15.83:0/2836411822 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe17015e0e0 0x7fe1700d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe175c00640 1 -- 172.21.15.83:0/2836411822 >> 172.21.15.83:0/2836411822 conn(0x7fe1700d1cc0 msgr2=0x7fe1700d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:50.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe175c00640 1 -- 172.21.15.83:0/2836411822 shutdown_connections 2024-04-25T00:11:50.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.763+0000 7fe175c00640 1 -- 172.21.15.83:0/2836411822 wait complete. 2024-04-25T00:11:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe175c00640 1 Processor -- start 2024-04-25T00:11:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe175c00640 1 -- start start 2024-04-25T00:11:50.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe175c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1700cea80 0x7fe170153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:50.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe175c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17015e0e0 0x7fe170154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:50.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe175c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe17015e4c0 0x7fe170155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe175c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe17016aed0 con 0x7fe17015e0e0 2024-04-25T00:11:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe175c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1700ce050 con 0x7fe1700cea80 2024-04-25T00:11:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe175c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe17016b050 con 0x7fe17015e4c0 2024-04-25T00:11:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe167e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17015e0e0 0x7fe170154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe16ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1700cea80 0x7fe170153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe167e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17015e0e0 0x7fe170154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48844/0 (socket says 172.21.15.83:48844) 2024-04-25T00:11:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe16f400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe17015e4c0 0x7fe170155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe167e00640 1 -- 172.21.15.83:0/705346794 learned_addr learned my addr 172.21.15.83:0/705346794 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe167e00640 1 -- 172.21.15.83:0/705346794 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe17015e4c0 msgr2=0x7fe170155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe167e00640 1 --2- 172.21.15.83:0/705346794 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe17015e4c0 0x7fe170155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe167e00640 1 -- 172.21.15.83:0/705346794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1700cea80 msgr2=0x7fe170153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe167e00640 1 --2- 172.21.15.83:0/705346794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1700cea80 0x7fe170153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe167e00640 1 -- 172.21.15.83:0/705346794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1700c6560 con 0x7fe17015e0e0 2024-04-25T00:11:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe16ea00640 1 --2- 172.21.15.83:0/705346794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1700cea80 0x7fe170153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe167e00640 1 --2- 172.21.15.83:0/705346794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17015e0e0 0x7fe170154e00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe1580b0040 tx=0x7fe158019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe166a00640 1 -- 172.21.15.83:0/705346794 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580bd070 con 0x7fe17015e0e0 2024-04-25T00:11:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe166a00640 1 -- 172.21.15.83:0/705346794 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe158004030 con 0x7fe17015e0e0 2024-04-25T00:11:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe166a00640 1 -- 172.21.15.83:0/705346794 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580b2040 con 0x7fe17015e0e0 2024-04-25T00:11:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.767+0000 7fe175c00640 1 -- 172.21.15.83:0/705346794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1700c6840 con 0x7fe17015e0e0 2024-04-25T00:11:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.771+0000 7fe175c00640 1 -- 172.21.15.83:0/705346794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe170154600 con 0x7fe17015e0e0 2024-04-25T00:11:50.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.771+0000 7fe175c00640 1 -- 172.21.15.83:0/705346794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe17004f510 con 0x7fe17015e0e0 2024-04-25T00:11:50.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.771+0000 7fe166a00640 1 -- 172.21.15.83:0/705346794 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe1580c0070 con 0x7fe17015e0e0 2024-04-25T00:11:50.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.771+0000 7fe166a00640 1 --2- 172.21.15.83:0/705346794 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe14c0678c0 0x7fe14c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:50.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.771+0000 7fe166a00640 1 -- 172.21.15.83:0/705346794 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe15811f600 con 0x7fe17015e0e0 2024-04-25T00:11:50.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.771+0000 7fe16ea00640 1 --2- 172.21.15.83:0/705346794 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe14c0678c0 0x7fe14c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:50.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.771+0000 7fe16ea00640 1 --2- 172.21.15.83:0/705346794 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe14c0678c0 0x7fe14c069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe160009f50 tx=0x7fe160009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:50.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.775+0000 7fe166a00640 1 -- 172.21.15.83:0/705346794 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe158016090 con 0x7fe17015e0e0 2024-04-25T00:11:50.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.935+0000 7fe175c00640 1 -- 172.21.15.83:0/705346794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe17006d630 con 0x7fe17015e0e0 2024-04-25T00:11:50.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.935+0000 7fe166a00640 1 -- 172.21.15.83:0/705346794 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fe1580ef650 con 0x7fe17015e0e0 2024-04-25T00:11:50.941 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:11:50.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.939+0000 7fe175c00640 1 -- 172.21.15.83:0/705346794 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe14c0678c0 msgr2=0x7fe14c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.939+0000 7fe175c00640 1 --2- 172.21.15.83:0/705346794 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe14c0678c0 0x7fe14c069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe160009f50 tx=0x7fe160009190 comp rx=0 tx=0).stop 2024-04-25T00:11:50.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.939+0000 7fe175c00640 1 -- 172.21.15.83:0/705346794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17015e0e0 msgr2=0x7fe170154e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.939+0000 7fe175c00640 1 --2- 172.21.15.83:0/705346794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17015e0e0 0x7fe170154e00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe1580b0040 tx=0x7fe158019240 comp rx=0 tx=0).stop 2024-04-25T00:11:50.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.939+0000 7fe175c00640 1 -- 172.21.15.83:0/705346794 shutdown_connections 2024-04-25T00:11:50.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.939+0000 7fe175c00640 1 --2- 172.21.15.83:0/705346794 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe14c0678c0 0x7fe14c069d80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.939+0000 7fe175c00640 1 --2- 172.21.15.83:0/705346794 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe17015e4c0 0x7fe170155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.939+0000 7fe175c00640 1 --2- 172.21.15.83:0/705346794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17015e0e0 0x7fe170154e00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.939+0000 7fe175c00640 1 --2- 172.21.15.83:0/705346794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1700cea80 0x7fe170153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.939+0000 7fe175c00640 1 -- 172.21.15.83:0/705346794 >> 172.21.15.83:0/705346794 conn(0x7fe1700d1cc0 msgr2=0x7fe1700b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:50.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.939+0000 7fe175c00640 1 -- 172.21.15.83:0/705346794 shutdown_connections 2024-04-25T00:11:50.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:50.939+0000 7fe175c00640 1 -- 172.21.15.83:0/705346794 wait complete. 2024-04-25T00:11:50.962 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:11:51.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb249a00640 1 Processor -- start 2024-04-25T00:11:51.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb249a00640 1 -- start start 2024-04-25T00:11:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb249a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb244159810 0x7fb2440cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb249a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb244160020 0x7fb2440ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb249a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2440ce000 con 0x7fb24406e950 2024-04-25T00:11:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb249a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb24406e210 con 0x7fb244159810 2024-04-25T00:11:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb249a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb24406da00 con 0x7fb244160020 2024-04-25T00:11:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb242a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb244159810 0x7fb2440cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb242a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb244159810 0x7fb2440cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48848/0 (socket says 172.21.15.83:48848) 2024-04-25T00:11:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb242a00640 1 -- 172.21.15.83:0/900036397 learned_addr learned my addr 172.21.15.83:0/900036397 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb242000640 1 --1- 172.21.15.83:0/900036397 >> v1:172.21.15.83:6789/0 conn(0x7fb24406e950 0x7fb24415f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51872/0 (socket says 172.21.15.83:51872) 2024-04-25T00:11:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb243400640 1 --2- 172.21.15.83:0/900036397 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb244160020 0x7fb2440ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb242a00640 1 -- 172.21.15.83:0/900036397 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb244160020 msgr2=0x7fb2440ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb242a00640 1 --2- 172.21.15.83:0/900036397 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb244160020 0x7fb2440ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb241600640 1 -- 172.21.15.83:0/900036397 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 503934460 0 0) 0x7fb2440ce000 con 0x7fb24406e950 2024-04-25T00:11:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb242a00640 1 -- 172.21.15.83:0/900036397 >> v1:172.21.15.83:6789/0 conn(0x7fb24406e950 legacy=0x7fb24415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.099+0000 7fb242a00640 1 -- 172.21.15.83:0/900036397 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24406eec0 con 0x7fb244159810 2024-04-25T00:11:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.103+0000 7fb242a00640 1 --2- 172.21.15.83:0/900036397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb244159810 0x7fb2440cc4a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb234009e50 tx=0x7fb23401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bd562f5fd6a93d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.103+0000 7fb241600640 1 -- 172.21.15.83:0/900036397 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2340a5830 con 0x7fb244159810 2024-04-25T00:11:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.103+0000 7fb241600640 1 -- 172.21.15.83:0/900036397 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2340a59d0 con 0x7fb244159810 2024-04-25T00:11:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.103+0000 7fb241600640 1 -- 172.21.15.83:0/900036397 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2340a5da0 con 0x7fb244159810 2024-04-25T00:11:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.103+0000 7fb249a00640 1 -- 172.21.15.83:0/900036397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb244159810 msgr2=0x7fb2440cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.103+0000 7fb249a00640 1 --2- 172.21.15.83:0/900036397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb244159810 0x7fb2440cc4a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb234009e50 tx=0x7fb23401c970 comp rx=0 tx=0).stop 2024-04-25T00:11:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.103+0000 7fb249a00640 1 -- 172.21.15.83:0/900036397 shutdown_connections 2024-04-25T00:11:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.103+0000 7fb249a00640 1 --2- 172.21.15.83:0/900036397 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb244160020 0x7fb2440ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.103+0000 7fb249a00640 1 --2- 172.21.15.83:0/900036397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb244159810 0x7fb2440cc4a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.103+0000 7fb249a00640 1 -- 172.21.15.83:0/900036397 >> 172.21.15.83:0/900036397 conn(0x7fb2440b1830 msgr2=0x7fb2440b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.103+0000 7fb249a00640 1 -- 172.21.15.83:0/900036397 shutdown_connections 2024-04-25T00:11:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.103+0000 7fb249a00640 1 -- 172.21.15.83:0/900036397 wait complete. 2024-04-25T00:11:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.103+0000 7fb249a00640 1 Processor -- start 2024-04-25T00:11:51.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.103+0000 7fb249a00640 1 -- start start 2024-04-25T00:11:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb249a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24406e950 0x7fb2440b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb249a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb244159810 0x7fb244152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb249a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb244160020 0x7fb2441530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb249a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb244159d70 con 0x7fb244160020 2024-04-25T00:11:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb249a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb244159bf0 con 0x7fb24406e950 2024-04-25T00:11:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb249a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb244070da0 con 0x7fb244159810 2024-04-25T00:11:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb243400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb244160020 0x7fb2441530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb243400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb244160020 0x7fb2441530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48864/0 (socket says 172.21.15.83:48864) 2024-04-25T00:11:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb243400640 1 -- 172.21.15.83:0/2499600814 learned_addr learned my addr 172.21.15.83:0/2499600814 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb242000640 1 --2- 172.21.15.83:0/2499600814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb244159810 0x7fb244152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb242a00640 1 --2- 172.21.15.83:0/2499600814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24406e950 0x7fb2440b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb243400640 1 -- 172.21.15.83:0/2499600814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb244159810 msgr2=0x7fb244152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb243400640 1 --2- 172.21.15.83:0/2499600814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb244159810 0x7fb244152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb243400640 1 -- 172.21.15.83:0/2499600814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24406e950 msgr2=0x7fb2440b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb243400640 1 --2- 172.21.15.83:0/2499600814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24406e950 0x7fb2440b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb243400640 1 -- 172.21.15.83:0/2499600814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2440b0990 con 0x7fb244160020 2024-04-25T00:11:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb243400640 1 --2- 172.21.15.83:0/2499600814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb244160020 0x7fb2441530f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00e850 tx=0x7fb23c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb23aa00640 1 -- 172.21.15.83:0/2499600814 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c017070 con 0x7fb244160020 2024-04-25T00:11:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb23aa00640 1 -- 172.21.15.83:0/2499600814 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb23c00c940 con 0x7fb244160020 2024-04-25T00:11:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb249a00640 1 -- 172.21.15.83:0/2499600814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2440b0c70 con 0x7fb244160020 2024-04-25T00:11:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb23aa00640 1 -- 172.21.15.83:0/2499600814 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00cd10 con 0x7fb244160020 2024-04-25T00:11:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb249a00640 1 -- 172.21.15.83:0/2499600814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2440b1230 con 0x7fb244160020 2024-04-25T00:11:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb249a00640 1 -- 172.21.15.83:0/2499600814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb24404f510 con 0x7fb244160020 2024-04-25T00:11:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.107+0000 7fb23aa00640 1 -- 172.21.15.83:0/2499600814 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb23c01a070 con 0x7fb244160020 2024-04-25T00:11:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.111+0000 7fb23aa00640 1 --2- 172.21.15.83:0/2499600814 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb2240678c0 0x7fb224069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.111+0000 7fb23aa00640 1 -- 172.21.15.83:0/2499600814 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb23c086380 con 0x7fb244160020 2024-04-25T00:11:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.111+0000 7fb242a00640 1 --2- 172.21.15.83:0/2499600814 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb2240678c0 0x7fb224069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.111+0000 7fb242a00640 1 --2- 172.21.15.83:0/2499600814 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb2240678c0 0x7fb224069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb2340ae000 tx=0x7fb2340a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:51.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.115+0000 7fb23aa00640 1 -- 172.21.15.83:0/2499600814 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb23c0563d0 con 0x7fb244160020 2024-04-25T00:11:51.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.271+0000 7fb249a00640 1 -- 172.21.15.83:0/2499600814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb244072c90 con 0x7fb244160020 2024-04-25T00:11:51.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.271+0000 7fb23aa00640 1 -- 172.21.15.83:0/2499600814 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fb23c05a6d0 con 0x7fb244160020 2024-04-25T00:11:51.274 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.271+0000 7fb249a00640 1 -- 172.21.15.83:0/2499600814 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb2240678c0 msgr2=0x7fb224069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.271+0000 7fb249a00640 1 --2- 172.21.15.83:0/2499600814 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb2240678c0 0x7fb224069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb2340ae000 tx=0x7fb2340a7070 comp rx=0 tx=0).stop 2024-04-25T00:11:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.271+0000 7fb249a00640 1 -- 172.21.15.83:0/2499600814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb244160020 msgr2=0x7fb2441530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.271+0000 7fb249a00640 1 --2- 172.21.15.83:0/2499600814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb244160020 0x7fb2441530f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00e850 tx=0x7fb23c00ed10 comp rx=0 tx=0).stop 2024-04-25T00:11:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.271+0000 7fb249a00640 1 -- 172.21.15.83:0/2499600814 shutdown_connections 2024-04-25T00:11:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.271+0000 7fb249a00640 1 --2- 172.21.15.83:0/2499600814 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb2240678c0 0x7fb224069d80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.271+0000 7fb249a00640 1 --2- 172.21.15.83:0/2499600814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb244159810 0x7fb244152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.271+0000 7fb249a00640 1 --2- 172.21.15.83:0/2499600814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb244160020 0x7fb2441530f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.271+0000 7fb249a00640 1 --2- 172.21.15.83:0/2499600814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb24406e950 0x7fb2440b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.271+0000 7fb249a00640 1 -- 172.21.15.83:0/2499600814 >> 172.21.15.83:0/2499600814 conn(0x7fb2440b1830 msgr2=0x7fb2440d04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:51.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.271+0000 7fb249a00640 1 -- 172.21.15.83:0/2499600814 shutdown_connections 2024-04-25T00:11:51.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.271+0000 7fb249a00640 1 -- 172.21.15.83:0/2499600814 wait complete. 2024-04-25T00:11:51.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:11:51.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1e000640 1 Processor -- start 2024-04-25T00:11:51.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1e000640 1 -- start start 2024-04-25T00:11:51.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1e000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faf18154e50 0x7faf18071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faf18072470 0x7faf18072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1e000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf180b2dd0 0x7faf180b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1e000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7faf18071740 con 0x7faf18154e50 2024-04-25T00:11:51.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1e000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7faf180b3780 con 0x7faf18072470 2024-04-25T00:11:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1e000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7faf18070e10 con 0x7faf180b2dd0 2024-04-25T00:11:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1cc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faf18154e50 0x7faf18071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1cc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faf18154e50 0x7faf18071bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48866/0 (socket says 172.21.15.83:48866) 2024-04-25T00:11:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1cc00640 1 -- 172.21.15.83:0/1117973359 learned_addr learned my addr 172.21.15.83:0/1117973359 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf17e00640 1 --2- 172.21.15.83:0/1117973359 >> v2:172.21.15.86:3300/0 conn(0x7faf18072470 0x7faf18072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1d600640 1 --2- 172.21.15.83:0/1117973359 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf180b2dd0 0x7faf180b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1cc00640 1 -- 172.21.15.83:0/1117973359 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf180b2dd0 msgr2=0x7faf180b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1cc00640 1 --2- 172.21.15.83:0/1117973359 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf180b2dd0 0x7faf180b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1cc00640 1 -- 172.21.15.83:0/1117973359 >> v2:172.21.15.86:3300/0 conn(0x7faf18072470 msgr2=0x7faf18072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1cc00640 1 --2- 172.21.15.83:0/1117973359 >> v2:172.21.15.86:3300/0 conn(0x7faf18072470 0x7faf18072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf1cc00640 1 -- 172.21.15.83:0/1117973359 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf180710f0 con 0x7faf18154e50 2024-04-25T00:11:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.431+0000 7faf17e00640 1 --2- 172.21.15.83:0/1117973359 >> v2:172.21.15.86:3300/0 conn(0x7faf18072470 0x7faf18072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:51.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.435+0000 7faf1cc00640 1 --2- 172.21.15.83:0/1117973359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf18154e50 0x7faf18071bd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faf18074280 tx=0x7faf0c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df3e93ad8dcf935 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.435+0000 7faf17400640 1 -- 172.21.15.83:0/1117973359 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0b0070 con 0x7faf18154e50 2024-04-25T00:11:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.435+0000 7faf17400640 1 -- 172.21.15.83:0/1117973359 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c0ab600 con 0x7faf18154e50 2024-04-25T00:11:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.435+0000 7faf17400640 1 -- 172.21.15.83:0/1117973359 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0ab9d0 con 0x7faf18154e50 2024-04-25T00:11:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.435+0000 7faf1e000640 1 -- 172.21.15.83:0/1117973359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf18154e50 msgr2=0x7faf18071bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.435+0000 7faf1e000640 1 --2- 172.21.15.83:0/1117973359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf18154e50 0x7faf18071bd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faf18074280 tx=0x7faf0c01c520 comp rx=0 tx=0).stop 2024-04-25T00:11:51.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.435+0000 7faf1e000640 1 -- 172.21.15.83:0/1117973359 shutdown_connections 2024-04-25T00:11:51.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.435+0000 7faf1e000640 1 --2- 172.21.15.83:0/1117973359 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf180b2dd0 0x7faf180b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.435+0000 7faf1e000640 1 --2- 172.21.15.83:0/1117973359 >> v2:172.21.15.86:3300/0 conn(0x7faf18072470 0x7faf18072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.435+0000 7faf1e000640 1 --2- 172.21.15.83:0/1117973359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf18154e50 0x7faf18071bd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.435+0000 7faf1e000640 1 -- 172.21.15.83:0/1117973359 >> 172.21.15.83:0/1117973359 conn(0x7faf180cf980 msgr2=0x7faf180cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:51.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.435+0000 7faf1e000640 1 -- 172.21.15.83:0/1117973359 shutdown_connections 2024-04-25T00:11:51.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.435+0000 7faf1e000640 1 -- 172.21.15.83:0/1117973359 wait complete. 2024-04-25T00:11:51.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.435+0000 7faf1e000640 1 Processor -- start 2024-04-25T00:11:51.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1e000640 1 -- start start 2024-04-25T00:11:51.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf18072470 0x7faf18158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf180b2dd0 0x7faf18160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1e000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf18154e50 0x7faf181502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1e000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faf1806d5f0 con 0x7faf18072470 2024-04-25T00:11:51.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf1806d470 con 0x7faf180b2dd0 2024-04-25T00:11:51.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1e000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7faf1806d770 con 0x7faf18154e50 2024-04-25T00:11:51.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf18072470 0x7faf18158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf18072470 0x7faf18158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48870/0 (socket says 172.21.15.83:48870) 2024-04-25T00:11:51.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1cc00640 1 -- 172.21.15.83:0/2909049952 learned_addr learned my addr 172.21.15.83:0/2909049952 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:51.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1cc00640 1 -- 172.21.15.83:0/2909049952 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf18154e50 msgr2=0x7faf181502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1cc00640 1 --2- 172.21.15.83:0/2909049952 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf18154e50 0x7faf181502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1cc00640 1 -- 172.21.15.83:0/2909049952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf180b2dd0 msgr2=0x7faf18160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1cc00640 1 --2- 172.21.15.83:0/2909049952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf180b2dd0 0x7faf18160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1cc00640 1 -- 172.21.15.83:0/2909049952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf18157360 con 0x7faf18072470 2024-04-25T00:11:51.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1cc00640 1 --2- 172.21.15.83:0/2909049952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf18072470 0x7faf18158400 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0ae040 tx=0x7faf0c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:51.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf14c00640 1 -- 172.21.15.83:0/2909049952 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0ba020 con 0x7faf18072470 2024-04-25T00:11:51.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf14c00640 1 -- 172.21.15.83:0/2909049952 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c0b6420 con 0x7faf18072470 2024-04-25T00:11:51.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf14c00640 1 -- 172.21.15.83:0/2909049952 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0b0040 con 0x7faf18072470 2024-04-25T00:11:51.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1e000640 1 -- 172.21.15.83:0/2909049952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf18157640 con 0x7faf18072470 2024-04-25T00:11:51.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1e000640 1 -- 172.21.15.83:0/2909049952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf18157c00 con 0x7faf18072470 2024-04-25T00:11:51.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.439+0000 7faf1e000640 1 -- 172.21.15.83:0/2909049952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf1804f510 con 0x7faf18072470 2024-04-25T00:11:51.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.443+0000 7faf14c00640 1 -- 172.21.15.83:0/2909049952 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faf0c0bd020 con 0x7faf18072470 2024-04-25T00:11:51.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.443+0000 7faf14c00640 1 --2- 172.21.15.83:0/2909049952 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faef80678c0 0x7faef8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.443+0000 7faf14c00640 1 -- 172.21.15.83:0/2909049952 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7faf0c120270 con 0x7faf18072470 2024-04-25T00:11:51.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.443+0000 7faf17e00640 1 --2- 172.21.15.83:0/2909049952 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faef80678c0 0x7faef8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.443+0000 7faf17e00640 1 --2- 172.21.15.83:0/2909049952 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faef80678c0 0x7faef8069d80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf181525a0 tx=0x7faf04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:51.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.447+0000 7faf14c00640 1 -- 172.21.15.83:0/2909049952 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faf0c0c4030 con 0x7faf18072470 2024-04-25T00:11:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.603+0000 7faf1e000640 1 -- 172.21.15.83:0/2909049952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faf1806db80 con 0x7faf18072470 2024-04-25T00:11:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.603+0000 7faf14c00640 1 -- 172.21.15.83:0/2909049952 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7faf0c016110 con 0x7faf18072470 2024-04-25T00:11:51.609 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:11:51.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.607+0000 7faf1e000640 1 -- 172.21.15.83:0/2909049952 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faef80678c0 msgr2=0x7faef8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.607+0000 7faf1e000640 1 --2- 172.21.15.83:0/2909049952 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faef80678c0 0x7faef8069d80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf181525a0 tx=0x7faf04008040 comp rx=0 tx=0).stop 2024-04-25T00:11:51.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.607+0000 7faf1e000640 1 -- 172.21.15.83:0/2909049952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf18072470 msgr2=0x7faf18158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.607+0000 7faf1e000640 1 --2- 172.21.15.83:0/2909049952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf18072470 0x7faf18158400 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0ae040 tx=0x7faf0c01ad60 comp rx=0 tx=0).stop 2024-04-25T00:11:51.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.607+0000 7faf1e000640 1 -- 172.21.15.83:0/2909049952 shutdown_connections 2024-04-25T00:11:51.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.607+0000 7faf1e000640 1 --2- 172.21.15.83:0/2909049952 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faef80678c0 0x7faef8069d80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.607+0000 7faf1e000640 1 --2- 172.21.15.83:0/2909049952 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf18154e50 0x7faf181502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.607+0000 7faf1e000640 1 --2- 172.21.15.83:0/2909049952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf180b2dd0 0x7faf18160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.607+0000 7faf1e000640 1 --2- 172.21.15.83:0/2909049952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf18072470 0x7faf18158400 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.607+0000 7faf1e000640 1 -- 172.21.15.83:0/2909049952 >> 172.21.15.83:0/2909049952 conn(0x7faf180cf980 msgr2=0x7faf180ad860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.607+0000 7faf1e000640 1 -- 172.21.15.83:0/2909049952 shutdown_connections 2024-04-25T00:11:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.607+0000 7faf1e000640 1 -- 172.21.15.83:0/2909049952 wait complete. 2024-04-25T00:11:51.629 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:11:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.763+0000 7ff9a1800640 1 Processor -- start 2024-04-25T00:11:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.763+0000 7ff9a1800640 1 -- start start 2024-04-25T00:11:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.763+0000 7ff9a1800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff99c1555c0 0x7ff99c15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.763+0000 7ff9a1800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff99c06e4c0 0x7ff99c06e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.763+0000 7ff9a1800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff99c15c560 con 0x7ff99c1551e0 2024-04-25T00:11:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.763+0000 7ff9a1800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff99c072460 con 0x7ff99c06e4c0 2024-04-25T00:11:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.763+0000 7ff9a1800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff99c15c7d0 con 0x7ff99c1555c0 2024-04-25T00:11:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.763+0000 7ff99b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff99c06e4c0 0x7ff99c06e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.763+0000 7ff99b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff99c06e4c0 0x7ff99c06e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48874/0 (socket says 172.21.15.83:48874) 2024-04-25T00:11:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.763+0000 7ff99b400640 1 -- 172.21.15.83:0/2743386747 learned_addr learned my addr 172.21.15.83:0/2743386747 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.763+0000 7ff992000640 1 --2- 172.21.15.83:0/2743386747 >> v2:172.21.15.86:3300/0 conn(0x7ff99c1555c0 0x7ff99c15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.763+0000 7ff99b400640 1 -- 172.21.15.83:0/2743386747 >> v2:172.21.15.86:3300/0 conn(0x7ff99c1555c0 msgr2=0x7ff99c15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.763+0000 7ff99b400640 1 --2- 172.21.15.83:0/2743386747 >> v2:172.21.15.86:3300/0 conn(0x7ff99c1555c0 0x7ff99c15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.763+0000 7ff99b400640 1 -- 172.21.15.83:0/2743386747 >> v1:172.21.15.86:6789/0 conn(0x7ff99c1551e0 legacy=0x7ff99c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.767+0000 7ff99b400640 1 -- 172.21.15.83:0/2743386747 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c15cab0 con 0x7ff99c06e4c0 2024-04-25T00:11:51.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.767+0000 7ff99b400640 1 --2- 172.21.15.83:0/2743386747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff99c06e4c0 0x7ff99c06e950 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff99400bb10 tx=0x7ff99401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb82df674c29aa03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:51.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.767+0000 7ff99a000640 1 -- 172.21.15.83:0/2743386747 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940b4070 con 0x7ff99c06e4c0 2024-04-25T00:11:51.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.767+0000 7ff99a000640 1 -- 172.21.15.83:0/2743386747 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9940a79c0 con 0x7ff99c06e4c0 2024-04-25T00:11:51.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.767+0000 7ff99a000640 1 -- 172.21.15.83:0/2743386747 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940a7d90 con 0x7ff99c06e4c0 2024-04-25T00:11:51.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.767+0000 7ff9a1800640 1 -- 172.21.15.83:0/2743386747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff99c06e4c0 msgr2=0x7ff99c06e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.767+0000 7ff9a1800640 1 --2- 172.21.15.83:0/2743386747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff99c06e4c0 0x7ff99c06e950 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff99400bb10 tx=0x7ff99401ebf0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.767+0000 7ff9a1800640 1 -- 172.21.15.83:0/2743386747 shutdown_connections 2024-04-25T00:11:51.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.767+0000 7ff9a1800640 1 --2- 172.21.15.83:0/2743386747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff99c06e4c0 0x7ff99c06e950 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.767+0000 7ff9a1800640 1 --2- 172.21.15.83:0/2743386747 >> v2:172.21.15.86:3300/0 conn(0x7ff99c1555c0 0x7ff99c15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.767+0000 7ff9a1800640 1 -- 172.21.15.83:0/2743386747 >> 172.21.15.83:0/2743386747 conn(0x7ff99c0b1830 msgr2=0x7ff99c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.767+0000 7ff9a1800640 1 -- 172.21.15.83:0/2743386747 shutdown_connections 2024-04-25T00:11:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.767+0000 7ff9a1800640 1 -- 172.21.15.83:0/2743386747 wait complete. 2024-04-25T00:11:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff9a1800640 1 Processor -- start 2024-04-25T00:11:51.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff9a1800640 1 -- start start 2024-04-25T00:11:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff9a1800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff99c06e4c0 0x7ff99c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff9a1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff99c1551e0 0x7ff99c15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff9a1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c1555c0 0x7ff99c143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff9a1800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c06df50 con 0x7ff99c1551e0 2024-04-25T00:11:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff9a1800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c06ddd0 con 0x7ff99c1555c0 2024-04-25T00:11:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff9a1800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff99c06e0d0 con 0x7ff99c06e4c0 2024-04-25T00:11:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff992000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff99c1551e0 0x7ff99c15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff992000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff99c1551e0 0x7ff99c15ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48878/0 (socket says 172.21.15.83:48878) 2024-04-25T00:11:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff992000640 1 -- 172.21.15.83:0/679605313 learned_addr learned my addr 172.21.15.83:0/679605313 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff99b400640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c1555c0 0x7ff99c143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff99aa00640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff99c06e4c0 0x7ff99c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff992000640 1 -- 172.21.15.83:0/679605313 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff99c06e4c0 msgr2=0x7ff99c15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff992000640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff99c06e4c0 0x7ff99c15e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff992000640 1 -- 172.21.15.83:0/679605313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c1555c0 msgr2=0x7ff99c143730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff992000640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c1555c0 0x7ff99c143730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff992000640 1 -- 172.21.15.83:0/679605313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c15d620 con 0x7ff99c1551e0 2024-04-25T00:11:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff99aa00640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff99c06e4c0 0x7ff99c15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff992000640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff99c1551e0 0x7ff99c15ced0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff98400e870 tx=0x7ff98400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff993400640 1 -- 172.21.15.83:0/679605313 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff984017070 con 0x7ff99c1551e0 2024-04-25T00:11:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff993400640 1 -- 172.21.15.83:0/679605313 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff98400c940 con 0x7ff99c1551e0 2024-04-25T00:11:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff993400640 1 -- 172.21.15.83:0/679605313 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98400cd10 con 0x7ff99c1551e0 2024-04-25T00:11:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff9a1800640 1 -- 172.21.15.83:0/679605313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99c15d900 con 0x7ff99c1551e0 2024-04-25T00:11:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.771+0000 7ff9a1800640 1 -- 172.21.15.83:0/679605313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99c15dec0 con 0x7ff99c1551e0 2024-04-25T00:11:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.775+0000 7ff9a1800640 1 -- 172.21.15.83:0/679605313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff99c04f510 con 0x7ff99c1551e0 2024-04-25T00:11:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.775+0000 7ff993400640 1 -- 172.21.15.83:0/679605313 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff98401a070 con 0x7ff99c1551e0 2024-04-25T00:11:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.775+0000 7ff993400640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff9780678c0 0x7ff978069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.775+0000 7ff993400640 1 -- 172.21.15.83:0/679605313 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff9840862e0 con 0x7ff99c1551e0 2024-04-25T00:11:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.775+0000 7ff99aa00640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff9780678c0 0x7ff978069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.775+0000 7ff99aa00640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff9780678c0 0x7ff978069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff98c001e10 tx=0x7ff98c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.779+0000 7ff993400640 1 -- 172.21.15.83:0/679605313 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff984056330 con 0x7ff99c1551e0 2024-04-25T00:11:51.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.935+0000 7ff9a1800640 1 -- 172.21.15.83:0/679605313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff99c06ce30 con 0x7ff99c1551e0 2024-04-25T00:11:51.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.939+0000 7ff993400640 1 -- 172.21.15.83:0/679605313 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7ff98405a630 con 0x7ff99c1551e0 2024-04-25T00:11:51.942 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:51.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.939+0000 7ff9a1800640 1 -- 172.21.15.83:0/679605313 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff9780678c0 msgr2=0x7ff978069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.939+0000 7ff9a1800640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff9780678c0 0x7ff978069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff98c001e10 tx=0x7ff98c009040 comp rx=0 tx=0).stop 2024-04-25T00:11:51.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.939+0000 7ff9a1800640 1 -- 172.21.15.83:0/679605313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff99c1551e0 msgr2=0x7ff99c15ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.939+0000 7ff9a1800640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff99c1551e0 0x7ff99c15ced0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff98400e870 tx=0x7ff98400ed30 comp rx=0 tx=0).stop 2024-04-25T00:11:51.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.939+0000 7ff9a1800640 1 -- 172.21.15.83:0/679605313 shutdown_connections 2024-04-25T00:11:51.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.939+0000 7ff9a1800640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff9780678c0 0x7ff978069d80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.939+0000 7ff9a1800640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c1555c0 0x7ff99c143730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.939+0000 7ff9a1800640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff99c1551e0 0x7ff99c15ced0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.939+0000 7ff9a1800640 1 --2- 172.21.15.83:0/679605313 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff99c06e4c0 0x7ff99c15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.939+0000 7ff9a1800640 1 -- 172.21.15.83:0/679605313 >> 172.21.15.83:0/679605313 conn(0x7ff99c0b1830 msgr2=0x7ff99c14d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:51.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.939+0000 7ff9a1800640 1 -- 172.21.15.83:0/679605313 shutdown_connections 2024-04-25T00:11:51.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:51.939+0000 7ff9a1800640 1 -- 172.21.15.83:0/679605313 wait complete. 2024-04-25T00:11:51.981 INFO:tasks.cephfs_test_runner:Starting test: test_mount_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) That when a client unmounts, the thread count on the MDS goes back 2024-04-25T00:11:51.981 INFO:tasks.cephfs_test_runner:test_mount_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-04-25T00:11:51.982 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.111+0000 7f21a3800640 1 Processor -- start 2024-04-25T00:11:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.111+0000 7f21a3800640 1 -- start start 2024-04-25T00:11:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.115+0000 7f21a3800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f219c15df00 0x7f219c06d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.115+0000 7f21a3800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f219c06e7b0 0x7f219c0716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.115+0000 7f21a3800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f219c06e0c0 con 0x7f219c071bf0 2024-04-25T00:11:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.115+0000 7f21a3800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f219c06f150 con 0x7f219c06e7b0 2024-04-25T00:11:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.115+0000 7f21a3800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f219c06e3a0 con 0x7f219c15df00 2024-04-25T00:11:52.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.115+0000 7f2193e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f219c06e7b0 0x7f219c0716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.115+0000 7f21a0e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f219c15df00 0x7f219c06d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.115+0000 7f21a1800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f219c071bf0 0x7f219c161ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:57086/0 (socket says 172.21.15.83:57086) 2024-04-25T00:11:52.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.115+0000 7f21a1800640 1 -- 172.21.15.83:0/432347671 learned_addr learned my addr 172.21.15.83:0/432347671 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:52.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.115+0000 7f21a0e00640 1 -- 172.21.15.83:0/432347671 >> v2:172.21.15.86:3300/0 conn(0x7f219c06e7b0 msgr2=0x7f219c0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:52.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.115+0000 7f21a0e00640 1 --2- 172.21.15.83:0/432347671 >> v2:172.21.15.86:3300/0 conn(0x7f219c06e7b0 0x7f219c0716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.115+0000 7f219be00640 1 -- 172.21.15.83:0/432347671 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1309126413 0 0) 0x7f219c06e0c0 con 0x7f219c071bf0 2024-04-25T00:11:52.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.115+0000 7f21a0e00640 1 -- 172.21.15.83:0/432347671 >> v1:172.21.15.86:6789/0 conn(0x7f219c071bf0 legacy=0x7f219c161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.115+0000 7f21a0e00640 1 -- 172.21.15.83:0/432347671 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c0ad990 con 0x7f219c15df00 2024-04-25T00:11:52.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.119+0000 7f21a0e00640 1 --2- 172.21.15.83:0/432347671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f219c15df00 0x7f219c06d970 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f218c0097c0 tx=0x7f218c01c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=230e4428ef011476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:52.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.119+0000 7f219be00640 1 -- 172.21.15.83:0/432347671 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c0b0070 con 0x7f219c15df00 2024-04-25T00:11:52.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.119+0000 7f219be00640 1 -- 172.21.15.83:0/432347671 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218c0ab600 con 0x7f219c15df00 2024-04-25T00:11:52.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.119+0000 7f219be00640 1 -- 172.21.15.83:0/432347671 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c0ab9d0 con 0x7f219c15df00 2024-04-25T00:11:52.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.119+0000 7f21a3800640 1 -- 172.21.15.83:0/432347671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f219c15df00 msgr2=0x7f219c06d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.119+0000 7f21a3800640 1 --2- 172.21.15.83:0/432347671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f219c15df00 0x7f219c06d970 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f218c0097c0 tx=0x7f218c01c380 comp rx=0 tx=0).stop 2024-04-25T00:11:52.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.119+0000 7f21a3800640 1 -- 172.21.15.83:0/432347671 shutdown_connections 2024-04-25T00:11:52.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.119+0000 7f21a3800640 1 --2- 172.21.15.83:0/432347671 >> v2:172.21.15.86:3300/0 conn(0x7f219c06e7b0 0x7f219c0716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.119+0000 7f21a3800640 1 --2- 172.21.15.83:0/432347671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f219c15df00 0x7f219c06d970 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.119+0000 7f21a3800640 1 -- 172.21.15.83:0/432347671 >> 172.21.15.83:0/432347671 conn(0x7f219c0b1830 msgr2=0x7f219c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:52.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.119+0000 7f21a3800640 1 -- 172.21.15.83:0/432347671 shutdown_connections 2024-04-25T00:11:52.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.119+0000 7f21a3800640 1 -- 172.21.15.83:0/432347671 wait complete. 2024-04-25T00:11:52.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.119+0000 7f21a3800640 1 Processor -- start 2024-04-25T00:11:52.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.119+0000 7f21a3800640 1 -- start start 2024-04-25T00:11:52.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f21a3800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f219c06e7b0 0x7f219c0cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f21a3800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f219c071bf0 0x7f219c0afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f21a3800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f219c15df00 0x7f219c0b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f21a3800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f219c0b3120 con 0x7f219c071bf0 2024-04-25T00:11:52.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f21a3800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f219c072040 con 0x7f219c06e7b0 2024-04-25T00:11:52.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f21a3800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f219c0b32a0 con 0x7f219c15df00 2024-04-25T00:11:52.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f2193e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f219c071bf0 0x7f219c0afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f2193e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f219c071bf0 0x7f219c0afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48888/0 (socket says 172.21.15.83:48888) 2024-04-25T00:11:52.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f2193e00640 1 -- 172.21.15.83:0/1482137388 learned_addr learned my addr 172.21.15.83:0/1482137388 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:52.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f21a0e00640 1 --2- 172.21.15.83:0/1482137388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f219c06e7b0 0x7f219c0cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f21a1800640 1 --2- 172.21.15.83:0/1482137388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f219c15df00 0x7f219c0b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f2193e00640 1 -- 172.21.15.83:0/1482137388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f219c15df00 msgr2=0x7f219c0b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f2193e00640 1 --2- 172.21.15.83:0/1482137388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f219c15df00 0x7f219c0b0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f2193e00640 1 -- 172.21.15.83:0/1482137388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f219c06e7b0 msgr2=0x7f219c0cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f2193e00640 1 --2- 172.21.15.83:0/1482137388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f219c06e7b0 0x7f219c0cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f2193e00640 1 -- 172.21.15.83:0/1482137388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c0cd8f0 con 0x7f219c071bf0 2024-04-25T00:11:52.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f2193e00640 1 --2- 172.21.15.83:0/1482137388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f219c071bf0 0x7f219c0afc30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f218400e870 tx=0x7f218400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:52.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f2199600640 1 -- 172.21.15.83:0/1482137388 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184017070 con 0x7f219c071bf0 2024-04-25T00:11:52.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f2199600640 1 -- 172.21.15.83:0/1482137388 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218400c940 con 0x7f219c071bf0 2024-04-25T00:11:52.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f2199600640 1 -- 172.21.15.83:0/1482137388 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218400cd10 con 0x7f219c071bf0 2024-04-25T00:11:52.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f21a3800640 1 -- 172.21.15.83:0/1482137388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219c0cdbd0 con 0x7f219c071bf0 2024-04-25T00:11:52.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f21a3800640 1 -- 172.21.15.83:0/1482137388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f219c0ce190 con 0x7f219c071bf0 2024-04-25T00:11:52.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f21a3800640 1 -- 172.21.15.83:0/1482137388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f219c0737f0 con 0x7f219c071bf0 2024-04-25T00:11:52.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.123+0000 7f2199600640 1 -- 172.21.15.83:0/1482137388 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f218401a070 con 0x7f219c071bf0 2024-04-25T00:11:52.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.127+0000 7f2199600640 1 --2- 172.21.15.83:0/1482137388 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f21780678c0 0x7f2178069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.127+0000 7f2199600640 1 -- 172.21.15.83:0/1482137388 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f21840862a0 con 0x7f219c071bf0 2024-04-25T00:11:52.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.127+0000 7f21a0e00640 1 --2- 172.21.15.83:0/1482137388 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f21780678c0 0x7f2178069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.127+0000 7f21a0e00640 1 --2- 172.21.15.83:0/1482137388 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f21780678c0 0x7f2178069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f218c0ad000 tx=0x7f218c0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.131+0000 7f2199600640 1 -- 172.21.15.83:0/1482137388 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2184056390 con 0x7f219c071bf0 2024-04-25T00:11:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.287+0000 7f21a3800640 1 -- 172.21.15.83:0/1482137388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f219c15d730 con 0x7f219c071bf0 2024-04-25T00:11:52.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.287+0000 7f2199600640 1 -- 172.21.15.83:0/1482137388 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18820 (secure 0 0 0) 0x7f218405a690 con 0x7f219c071bf0 2024-04-25T00:11:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:52.293 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:48.235809+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.291+0000 7f21a3800640 1 -- 172.21.15.83:0/1482137388 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f21780678c0 msgr2=0x7f2178069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.291+0000 7f21a3800640 1 --2- 172.21.15.83:0/1482137388 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f21780678c0 0x7f2178069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f218c0ad000 tx=0x7f218c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:11:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.291+0000 7f21a3800640 1 -- 172.21.15.83:0/1482137388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f219c071bf0 msgr2=0x7f219c0afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.291+0000 7f21a3800640 1 --2- 172.21.15.83:0/1482137388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f219c071bf0 0x7f219c0afc30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f218400e870 tx=0x7f218400ed30 comp rx=0 tx=0).stop 2024-04-25T00:11:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.291+0000 7f21a3800640 1 -- 172.21.15.83:0/1482137388 shutdown_connections 2024-04-25T00:11:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.291+0000 7f21a3800640 1 --2- 172.21.15.83:0/1482137388 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f21780678c0 0x7f2178069d80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.291+0000 7f21a3800640 1 --2- 172.21.15.83:0/1482137388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f219c15df00 0x7f219c0b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.291+0000 7f21a3800640 1 --2- 172.21.15.83:0/1482137388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f219c071bf0 0x7f219c0afc30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.291+0000 7f21a3800640 1 --2- 172.21.15.83:0/1482137388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f219c06e7b0 0x7f219c0cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.291+0000 7f21a3800640 1 -- 172.21.15.83:0/1482137388 >> 172.21.15.83:0/1482137388 conn(0x7f219c0b1830 msgr2=0x7f219c0b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.291+0000 7f21a3800640 1 -- 172.21.15.83:0/1482137388 shutdown_connections 2024-04-25T00:11:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.291+0000 7f21a3800640 1 -- 172.21.15.83:0/1482137388 wait complete. 2024-04-25T00:11:52.313 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:11:04.621429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:11:05.189716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:11:08.902215+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:11:52.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:11:52.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.447+0000 7f1295200640 1 Processor -- start 2024-04-25T00:11:52.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.447+0000 7f1295200640 1 -- start start 2024-04-25T00:11:52.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.451+0000 7f1295200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f129015ec40 0x7f12900b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.451+0000 7f1295200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12900719f0 con 0x7f129015e790 2024-04-25T00:11:52.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.451+0000 7f1295200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1290071cd0 con 0x7f1290155310 2024-04-25T00:11:52.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.451+0000 7f1295200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f129015f610 con 0x7f129015ec40 2024-04-25T00:11:52.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.451+0000 7f128ea00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f129015e790 0x7f12900b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51886/0 (socket says 172.21.15.83:51886) 2024-04-25T00:11:52.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.451+0000 7f128ea00640 1 -- 172.21.15.83:0/2594022848 learned_addr learned my addr 172.21.15.83:0/2594022848 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:52.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.451+0000 7f128fe00640 1 --2- 172.21.15.83:0/2594022848 >> v2:172.21.15.83:3300/0 conn(0x7f129015ec40 0x7f12900b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.451+0000 7f128e000640 1 -- 172.21.15.83:0/2594022848 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4286384682 0 0) 0x7f12900719f0 con 0x7f129015e790 2024-04-25T00:11:52.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.451+0000 7f128e000640 1 -- 172.21.15.83:0/2594022848 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1274005740 con 0x7f129015e790 2024-04-25T00:11:52.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.451+0000 7f128fe00640 1 -- 172.21.15.83:0/2594022848 >> v1:172.21.15.86:6789/0 conn(0x7f1290155310 legacy=0x7f129006cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.451+0000 7f128fe00640 1 -- 172.21.15.83:0/2594022848 >> v1:172.21.15.83:6789/0 conn(0x7f129015e790 legacy=0x7f12900b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.451+0000 7f128fe00640 1 -- 172.21.15.83:0/2594022848 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1290071fb0 con 0x7f129015ec40 2024-04-25T00:11:52.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f128fe00640 1 --2- 172.21.15.83:0/2594022848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129015ec40 0x7f12900b34b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1288009d90 tx=0x7f128801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a3d0408cece08b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:52.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f128e000640 1 -- 172.21.15.83:0/2594022848 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12880b2070 con 0x7f129015ec40 2024-04-25T00:11:52.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f128e000640 1 -- 172.21.15.83:0/2594022848 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12880a5870 con 0x7f129015ec40 2024-04-25T00:11:52.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f128e000640 1 -- 172.21.15.83:0/2594022848 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12880a5c40 con 0x7f129015ec40 2024-04-25T00:11:52.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 -- 172.21.15.83:0/2594022848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129015ec40 msgr2=0x7f12900b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 --2- 172.21.15.83:0/2594022848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129015ec40 0x7f12900b34b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1288009d90 tx=0x7f128801c8e0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 -- 172.21.15.83:0/2594022848 shutdown_connections 2024-04-25T00:11:52.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 --2- 172.21.15.83:0/2594022848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129015ec40 0x7f12900b34b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 -- 172.21.15.83:0/2594022848 >> 172.21.15.83:0/2594022848 conn(0x7f12900cf980 msgr2=0x7f12900cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:52.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 -- 172.21.15.83:0/2594022848 shutdown_connections 2024-04-25T00:11:52.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 -- 172.21.15.83:0/2594022848 wait complete. 2024-04-25T00:11:52.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 Processor -- start 2024-04-25T00:11:52.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 -- start start 2024-04-25T00:11:52.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1290155310 0x7f1290148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f129015e790 0x7f129014ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129015ec40 0x7f1290153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1290071740 con 0x7f129015ec40 2024-04-25T00:11:52.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12900715c0 con 0x7f1290155310 2024-04-25T00:11:52.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.455+0000 7f1295200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f129006d4f0 con 0x7f129015e790 2024-04-25T00:11:52.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f128fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129015ec40 0x7f1290153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f128fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129015ec40 0x7f1290153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48908/0 (socket says 172.21.15.83:48908) 2024-04-25T00:11:52.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f128fe00640 1 -- 172.21.15.83:0/3291360497 learned_addr learned my addr 172.21.15.83:0/3291360497 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:52.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f128f400640 1 --2- 172.21.15.83:0/3291360497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1290155310 0x7f1290148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f128ea00640 1 --2- 172.21.15.83:0/3291360497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f129015e790 0x7f129014ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f128fe00640 1 -- 172.21.15.83:0/3291360497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f129015e790 msgr2=0x7f129014ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f128fe00640 1 --2- 172.21.15.83:0/3291360497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f129015e790 0x7f129014ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f128fe00640 1 -- 172.21.15.83:0/3291360497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1290155310 msgr2=0x7f1290148930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f128fe00640 1 --2- 172.21.15.83:0/3291360497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1290155310 0x7f1290148930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f128fe00640 1 -- 172.21.15.83:0/3291360497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129014d770 con 0x7f129015ec40 2024-04-25T00:11:52.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f128fe00640 1 --2- 172.21.15.83:0/3291360497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129015ec40 0x7f1290153000 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f12880b0040 tx=0x7f12900b4200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:52.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f1287400640 1 -- 172.21.15.83:0/3291360497 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12880bc070 con 0x7f129015ec40 2024-04-25T00:11:52.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f1287400640 1 -- 172.21.15.83:0/3291360497 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12880ad440 con 0x7f129015ec40 2024-04-25T00:11:52.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f1295200640 1 -- 172.21.15.83:0/3291360497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129014da50 con 0x7f129015ec40 2024-04-25T00:11:52.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f1287400640 1 -- 172.21.15.83:0/3291360497 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12880b2040 con 0x7f129015ec40 2024-04-25T00:11:52.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f1295200640 1 -- 172.21.15.83:0/3291360497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129014e010 con 0x7f129015ec40 2024-04-25T00:11:52.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f1295200640 1 -- 172.21.15.83:0/3291360497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129004f510 con 0x7f129015ec40 2024-04-25T00:11:52.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f1287400640 1 -- 172.21.15.83:0/3291360497 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f12880b5070 con 0x7f129015ec40 2024-04-25T00:11:52.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.459+0000 7f1287400640 1 --2- 172.21.15.83:0/3291360497 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f12700678f0 0x7f1270069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.463+0000 7f1287400640 1 -- 172.21.15.83:0/3291360497 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1288120530 con 0x7f129015ec40 2024-04-25T00:11:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.463+0000 7f128f400640 1 --2- 172.21.15.83:0/3291360497 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f12700678f0 0x7f1270069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.463+0000 7f128f400640 1 --2- 172.21.15.83:0/3291360497 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f12700678f0 0x7f1270069db0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1280007fd0 tx=0x7f1280009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:52.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.463+0000 7f1287400640 1 -- 172.21.15.83:0/3291360497 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f12880c5070 con 0x7f129015ec40 2024-04-25T00:11:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.619+0000 7f1295200640 1 -- 172.21.15.83:0/3291360497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f129006ce00 con 0x7f129015ec40 2024-04-25T00:11:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.619+0000 7f1287400640 1 -- 172.21.15.83:0/3291360497 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f1288016560 con 0x7f129015ec40 2024-04-25T00:11:52.625 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:11:52.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.623+0000 7f1295200640 1 -- 172.21.15.83:0/3291360497 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f12700678f0 msgr2=0x7f1270069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.623+0000 7f1295200640 1 --2- 172.21.15.83:0/3291360497 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f12700678f0 0x7f1270069db0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1280007fd0 tx=0x7f1280009040 comp rx=0 tx=0).stop 2024-04-25T00:11:52.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.623+0000 7f1295200640 1 -- 172.21.15.83:0/3291360497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129015ec40 msgr2=0x7f1290153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.623+0000 7f1295200640 1 --2- 172.21.15.83:0/3291360497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129015ec40 0x7f1290153000 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f12880b0040 tx=0x7f12900b4200 comp rx=0 tx=0).stop 2024-04-25T00:11:52.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.623+0000 7f1295200640 1 -- 172.21.15.83:0/3291360497 shutdown_connections 2024-04-25T00:11:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.623+0000 7f1295200640 1 --2- 172.21.15.83:0/3291360497 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f12700678f0 0x7f1270069db0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.623+0000 7f1295200640 1 --2- 172.21.15.83:0/3291360497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f129015e790 0x7f129014ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.623+0000 7f1295200640 1 --2- 172.21.15.83:0/3291360497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129015ec40 0x7f1290153000 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.623+0000 7f1295200640 1 --2- 172.21.15.83:0/3291360497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1290155310 0x7f1290148930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.623+0000 7f1295200640 1 -- 172.21.15.83:0/3291360497 >> 172.21.15.83:0/3291360497 conn(0x7f12900cf980 msgr2=0x7f12900b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.623+0000 7f1295200640 1 -- 172.21.15.83:0/3291360497 shutdown_connections 2024-04-25T00:11:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.623+0000 7f1295200640 1 -- 172.21.15.83:0/3291360497 wait complete. 2024-04-25T00:11:52.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:11:52.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.779+0000 7fd043400640 1 Processor -- start 2024-04-25T00:11:52.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.779+0000 7fd043400640 1 -- start start 2024-04-25T00:11:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd043400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd03c1551e0 0x7fd03c15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd043400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd03c1555c0 0x7fd03c071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd043400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd03c06e4c0 0x7fd03c06e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd043400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd03c06ee70 con 0x7fd03c1551e0 2024-04-25T00:11:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd043400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd03c15c560 con 0x7fd03c06e4c0 2024-04-25T00:11:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd043400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd03c072460 con 0x7fd03c1555c0 2024-04-25T00:11:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd040a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd03c1551e0 0x7fd03c15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd040a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd03c1551e0 0x7fd03c15bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48916/0 (socket says 172.21.15.83:48916) 2024-04-25T00:11:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd040a00640 1 -- 172.21.15.83:0/3343273549 learned_addr learned my addr 172.21.15.83:0/3343273549 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:52.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd041400640 1 --2- 172.21.15.83:0/3343273549 >> v2:172.21.15.86:3300/0 conn(0x7fd03c06e4c0 0x7fd03c06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd033e00640 1 --2- 172.21.15.83:0/3343273549 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd03c1555c0 0x7fd03c071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd040a00640 1 -- 172.21.15.83:0/3343273549 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd03c1555c0 msgr2=0x7fd03c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:52.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd040a00640 1 --2- 172.21.15.83:0/3343273549 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd03c1555c0 0x7fd03c071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd040a00640 1 -- 172.21.15.83:0/3343273549 >> v2:172.21.15.86:3300/0 conn(0x7fd03c06e4c0 msgr2=0x7fd03c06e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:52.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd040a00640 1 --2- 172.21.15.83:0/3343273549 >> v2:172.21.15.86:3300/0 conn(0x7fd03c06e4c0 0x7fd03c06e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd040a00640 1 -- 172.21.15.83:0/3343273549 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c15c7d0 con 0x7fd03c1551e0 2024-04-25T00:11:52.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd041400640 1 --2- 172.21.15.83:0/3343273549 >> v2:172.21.15.86:3300/0 conn(0x7fd03c06e4c0 0x7fd03c06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:52.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.783+0000 7fd033e00640 1 --2- 172.21.15.83:0/3343273549 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd03c1555c0 0x7fd03c071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:52.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd040a00640 1 --2- 172.21.15.83:0/3343273549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd03c1551e0 0x7fd03c15bea0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0cc860 tx=0x7fd02c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23326edf7b511282 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:52.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd03be00640 1 -- 172.21.15.83:0/3343273549 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0b1070 con 0x7fd03c1551e0 2024-04-25T00:11:52.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd03be00640 1 -- 172.21.15.83:0/3343273549 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd02c0ac630 con 0x7fd03c1551e0 2024-04-25T00:11:52.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd03be00640 1 -- 172.21.15.83:0/3343273549 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0a8050 con 0x7fd03c1551e0 2024-04-25T00:11:52.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 -- 172.21.15.83:0/3343273549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd03c1551e0 msgr2=0x7fd03c15bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 --2- 172.21.15.83:0/3343273549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd03c1551e0 0x7fd03c15bea0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0cc860 tx=0x7fd02c01c4f0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 -- 172.21.15.83:0/3343273549 shutdown_connections 2024-04-25T00:11:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 --2- 172.21.15.83:0/3343273549 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd03c1555c0 0x7fd03c071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 --2- 172.21.15.83:0/3343273549 >> v2:172.21.15.86:3300/0 conn(0x7fd03c06e4c0 0x7fd03c06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 --2- 172.21.15.83:0/3343273549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd03c1551e0 0x7fd03c15bea0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 -- 172.21.15.83:0/3343273549 >> 172.21.15.83:0/3343273549 conn(0x7fd03c0b1830 msgr2=0x7fd03c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 -- 172.21.15.83:0/3343273549 shutdown_connections 2024-04-25T00:11:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 -- 172.21.15.83:0/3343273549 wait complete. 2024-04-25T00:11:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 Processor -- start 2024-04-25T00:11:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 -- start start 2024-04-25T00:11:52.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd03c06e4c0 0x7fd03c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd03c1551e0 0x7fd03c151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd03c1555c0 0x7fd03c151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c15f280 con 0x7fd03c06e4c0 2024-04-25T00:11:52.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c15f100 con 0x7fd03c1555c0 2024-04-25T00:11:52.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd043400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd03c15f400 con 0x7fd03c1551e0 2024-04-25T00:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd040a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd03c06e4c0 0x7fd03c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd040a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd03c06e4c0 0x7fd03c15a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48918/0 (socket says 172.21.15.83:48918) 2024-04-25T00:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd040a00640 1 -- 172.21.15.83:0/1177624407 learned_addr learned my addr 172.21.15.83:0/1177624407 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.787+0000 7fd033e00640 1 --2- 172.21.15.83:0/1177624407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd03c1551e0 0x7fd03c151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd041400640 1 --2- 172.21.15.83:0/1177624407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd03c1555c0 0x7fd03c151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd040a00640 1 -- 172.21.15.83:0/1177624407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd03c1551e0 msgr2=0x7fd03c151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd040a00640 1 --2- 172.21.15.83:0/1177624407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd03c1551e0 0x7fd03c151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd040a00640 1 -- 172.21.15.83:0/1177624407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd03c1555c0 msgr2=0x7fd03c151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd040a00640 1 --2- 172.21.15.83:0/1177624407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd03c1555c0 0x7fd03c151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd040a00640 1 -- 172.21.15.83:0/1177624407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c14cf10 con 0x7fd03c06e4c0 2024-04-25T00:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd040a00640 1 --2- 172.21.15.83:0/1177624407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd03c06e4c0 0x7fd03c15a370 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0af040 tx=0x7fd02c019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd039600640 1 -- 172.21.15.83:0/1177624407 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0bc070 con 0x7fd03c06e4c0 2024-04-25T00:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd039600640 1 -- 172.21.15.83:0/1177624407 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd02c0ab820 con 0x7fd03c06e4c0 2024-04-25T00:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd043400640 1 -- 172.21.15.83:0/1177624407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03c14d1f0 con 0x7fd03c06e4c0 2024-04-25T00:11:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd039600640 1 -- 172.21.15.83:0/1177624407 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0b1070 con 0x7fd03c06e4c0 2024-04-25T00:11:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd043400640 1 -- 172.21.15.83:0/1177624407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03c152880 con 0x7fd03c06e4c0 2024-04-25T00:11:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd043400640 1 -- 172.21.15.83:0/1177624407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd03c04f510 con 0x7fd03c06e4c0 2024-04-25T00:11:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd039600640 1 -- 172.21.15.83:0/1177624407 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd02c0bf070 con 0x7fd03c06e4c0 2024-04-25T00:11:52.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd039600640 1 --2- 172.21.15.83:0/1177624407 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd0180678c0 0x7fd018069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:52.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd039600640 1 -- 172.21.15.83:0/1177624407 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd02c11fc90 con 0x7fd03c06e4c0 2024-04-25T00:11:52.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.791+0000 7fd033e00640 1 --2- 172.21.15.83:0/1177624407 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd0180678c0 0x7fd018069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:52.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.795+0000 7fd033e00640 1 --2- 172.21.15.83:0/1177624407 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd0180678c0 0x7fd018069d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd024006fd0 tx=0x7fd024008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.795+0000 7fd039600640 1 -- 172.21.15.83:0/1177624407 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd02c016110 con 0x7fd03c06e4c0 2024-04-25T00:11:52.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.951+0000 7fd043400640 1 -- 172.21.15.83:0/1177624407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd03c073e80 con 0x7fd03c06e4c0 2024-04-25T00:11:52.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.951+0000 7fd039600640 1 -- 172.21.15.83:0/1177624407 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fd02c0f0460 con 0x7fd03c06e4c0 2024-04-25T00:11:52.957 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:52.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.955+0000 7fd043400640 1 -- 172.21.15.83:0/1177624407 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd0180678c0 msgr2=0x7fd018069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.955+0000 7fd043400640 1 --2- 172.21.15.83:0/1177624407 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd0180678c0 0x7fd018069d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd024006fd0 tx=0x7fd024008040 comp rx=0 tx=0).stop 2024-04-25T00:11:52.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.955+0000 7fd043400640 1 -- 172.21.15.83:0/1177624407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd03c06e4c0 msgr2=0x7fd03c15a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:52.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.955+0000 7fd043400640 1 --2- 172.21.15.83:0/1177624407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd03c06e4c0 0x7fd03c15a370 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0af040 tx=0x7fd02c019290 comp rx=0 tx=0).stop 2024-04-25T00:11:52.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.955+0000 7fd043400640 1 -- 172.21.15.83:0/1177624407 shutdown_connections 2024-04-25T00:11:52.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.955+0000 7fd043400640 1 --2- 172.21.15.83:0/1177624407 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd0180678c0 0x7fd018069d80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.955+0000 7fd043400640 1 --2- 172.21.15.83:0/1177624407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd03c1551e0 0x7fd03c151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.955+0000 7fd043400640 1 --2- 172.21.15.83:0/1177624407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd03c1555c0 0x7fd03c151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.955+0000 7fd043400640 1 --2- 172.21.15.83:0/1177624407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd03c06e4c0 0x7fd03c15a370 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:52.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.955+0000 7fd043400640 1 -- 172.21.15.83:0/1177624407 >> 172.21.15.83:0/1177624407 conn(0x7fd03c0b1830 msgr2=0x7fd03c14de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:52.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.955+0000 7fd043400640 1 -- 172.21.15.83:0/1177624407 shutdown_connections 2024-04-25T00:11:52.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:52.955+0000 7fd043400640 1 -- 172.21.15.83:0/1177624407 wait complete. 2024-04-25T00:11:52.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:11:53.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.111+0000 7f3b90400640 1 Processor -- start 2024-04-25T00:11:53.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.111+0000 7f3b90400640 1 -- start start 2024-04-25T00:11:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.111+0000 7f3b90400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3b881551e0 0x7f3b8815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.111+0000 7f3b90400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b881555c0 0x7f3b88071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.111+0000 7f3b90400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b8815c560 con 0x7f3b8806e4c0 2024-04-25T00:11:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.111+0000 7f3b90400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3b88072460 con 0x7f3b881551e0 2024-04-25T00:11:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.111+0000 7f3b90400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3b8815c7d0 con 0x7f3b881555c0 2024-04-25T00:11:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.111+0000 7f3b8e400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3b8806e4c0 0x7f3b8815e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51894/0 (socket says 172.21.15.83:51894) 2024-04-25T00:11:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.111+0000 7f3b8e400640 1 -- 172.21.15.83:0/3076754182 learned_addr learned my addr 172.21.15.83:0/3076754182 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.111+0000 7f3b8da00640 1 --2- 172.21.15.83:0/3076754182 >> v2:172.21.15.86:3300/0 conn(0x7f3b881551e0 0x7f3b8815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.111+0000 7f3b8d000640 1 --2- 172.21.15.83:0/3076754182 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b881555c0 0x7f3b88071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.115+0000 7f3b87e00640 1 -- 172.21.15.83:0/3076754182 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2111020884 0 0) 0x7f3b8815c560 con 0x7f3b8806e4c0 2024-04-25T00:11:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.115+0000 7f3b87e00640 1 -- 172.21.15.83:0/3076754182 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b70005740 con 0x7f3b8806e4c0 2024-04-25T00:11:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.115+0000 7f3b87e00640 1 -- 172.21.15.83:0/3076754182 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2620853930 0 0) 0x7f3b70005740 con 0x7f3b8806e4c0 2024-04-25T00:11:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.115+0000 7f3b8da00640 1 -- 172.21.15.83:0/3076754182 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b881555c0 msgr2=0x7f3b88071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.115+0000 7f3b8da00640 1 --2- 172.21.15.83:0/3076754182 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b881555c0 0x7f3b88071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.115+0000 7f3b8da00640 1 -- 172.21.15.83:0/3076754182 >> v1:172.21.15.83:6789/0 conn(0x7f3b8806e4c0 legacy=0x7f3b8815e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.115+0000 7f3b8da00640 1 -- 172.21.15.83:0/3076754182 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8815cab0 con 0x7f3b881551e0 2024-04-25T00:11:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.115+0000 7f3b8d000640 1 --2- 172.21.15.83:0/3076754182 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b881555c0 0x7f3b88071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:53.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.115+0000 7f3b8da00640 1 --2- 172.21.15.83:0/3076754182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b881551e0 0x7f3b8815bea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0097c0 tx=0x7f3b7c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66b68b4f1a0f6370 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:53.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.115+0000 7f3b87e00640 1 -- 172.21.15.83:0/3076754182 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0b0070 con 0x7f3b881551e0 2024-04-25T00:11:53.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.115+0000 7f3b87e00640 1 -- 172.21.15.83:0/3076754182 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7c0ab400 con 0x7f3b881551e0 2024-04-25T00:11:53.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.115+0000 7f3b87e00640 1 -- 172.21.15.83:0/3076754182 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80001a00 con 0x7f3b881551e0 2024-04-25T00:11:53.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 -- 172.21.15.83:0/3076754182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b881551e0 msgr2=0x7f3b8815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 --2- 172.21.15.83:0/3076754182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b881551e0 0x7f3b8815bea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0097c0 tx=0x7f3b7c01c3c0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 -- 172.21.15.83:0/3076754182 shutdown_connections 2024-04-25T00:11:53.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 --2- 172.21.15.83:0/3076754182 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b881555c0 0x7f3b88071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 --2- 172.21.15.83:0/3076754182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b881551e0 0x7f3b8815bea0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 -- 172.21.15.83:0/3076754182 >> 172.21.15.83:0/3076754182 conn(0x7f3b880b1830 msgr2=0x7f3b880b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:53.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 -- 172.21.15.83:0/3076754182 shutdown_connections 2024-04-25T00:11:53.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 -- 172.21.15.83:0/3076754182 wait complete. 2024-04-25T00:11:53.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 Processor -- start 2024-04-25T00:11:53.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 -- start start 2024-04-25T00:11:53.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b8806e4c0 0x7f3b8814d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b881551e0 0x7f3b8814da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b881555c0 0x7f3b881f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8806d530 con 0x7f3b8806e4c0 2024-04-25T00:11:53.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8806d3b0 con 0x7f3b881551e0 2024-04-25T00:11:53.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b90400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3b8806d6b0 con 0x7f3b881555c0 2024-04-25T00:11:53.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b8da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b8806e4c0 0x7f3b8814d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b8da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b8806e4c0 0x7f3b8814d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48930/0 (socket says 172.21.15.83:48930) 2024-04-25T00:11:53.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b8da00640 1 -- 172.21.15.83:0/2796342124 learned_addr learned my addr 172.21.15.83:0/2796342124 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:53.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b8d000640 1 --2- 172.21.15.83:0/2796342124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b881551e0 0x7f3b8814da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b8da00640 1 -- 172.21.15.83:0/2796342124 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b881555c0 msgr2=0x7f3b881f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b8da00640 1 --2- 172.21.15.83:0/2796342124 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b881555c0 0x7f3b881f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b8da00640 1 -- 172.21.15.83:0/2796342124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b881551e0 msgr2=0x7f3b8814da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b8da00640 1 --2- 172.21.15.83:0/2796342124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b881551e0 0x7f3b8814da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b8da00640 1 -- 172.21.15.83:0/2796342124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8814e740 con 0x7f3b8806e4c0 2024-04-25T00:11:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.119+0000 7f3b8d000640 1 --2- 172.21.15.83:0/2796342124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b881551e0 0x7f3b8814da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.123+0000 7f3b8da00640 1 --2- 172.21.15.83:0/2796342124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b8806e4c0 0x7f3b8814d4e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c01aa70 tx=0x7f3b7c01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.123+0000 7f3b85600640 1 -- 172.21.15.83:0/2796342124 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0b0050 con 0x7f3b8806e4c0 2024-04-25T00:11:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.123+0000 7f3b85600640 1 -- 172.21.15.83:0/2796342124 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7c0bc070 con 0x7f3b8806e4c0 2024-04-25T00:11:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.123+0000 7f3b85600640 1 -- 172.21.15.83:0/2796342124 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0b78b0 con 0x7f3b8806e4c0 2024-04-25T00:11:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.123+0000 7f3b90400640 1 -- 172.21.15.83:0/2796342124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8814ea20 con 0x7f3b8806e4c0 2024-04-25T00:11:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.123+0000 7f3b90400640 1 -- 172.21.15.83:0/2796342124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8814efe0 con 0x7f3b8806e4c0 2024-04-25T00:11:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.123+0000 7f3b90400640 1 -- 172.21.15.83:0/2796342124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b8804f510 con 0x7f3b8806e4c0 2024-04-25T00:11:53.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.123+0000 7f3b85600640 1 -- 172.21.15.83:0/2796342124 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3b7c0b7a50 con 0x7f3b8806e4c0 2024-04-25T00:11:53.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.123+0000 7f3b85600640 1 --2- 172.21.15.83:0/2796342124 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3b680678c0 0x7f3b68069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.123+0000 7f3b85600640 1 -- 172.21.15.83:0/2796342124 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3b7c1208a0 con 0x7f3b8806e4c0 2024-04-25T00:11:53.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.123+0000 7f3b8d000640 1 --2- 172.21.15.83:0/2796342124 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3b680678c0 0x7f3b68069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.127+0000 7f3b8d000640 1 --2- 172.21.15.83:0/2796342124 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3b680678c0 0x7f3b68069d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3b88073e50 tx=0x7f3b74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:53.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.127+0000 7f3b85600640 1 -- 172.21.15.83:0/2796342124 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3b7c015fb0 con 0x7f3b8806e4c0 2024-04-25T00:11:53.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.287+0000 7f3b90400640 1 -- 172.21.15.83:0/2796342124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3b8806ce30 con 0x7f3b8806e4c0 2024-04-25T00:11:53.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.287+0000 7f3b85600640 1 -- 172.21.15.83:0/2796342124 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f3b7c0f10f0 con 0x7f3b8806e4c0 2024-04-25T00:11:53.290 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:11:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.287+0000 7f3b90400640 1 -- 172.21.15.83:0/2796342124 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3b680678c0 msgr2=0x7f3b68069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.287+0000 7f3b90400640 1 --2- 172.21.15.83:0/2796342124 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3b680678c0 0x7f3b68069d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3b88073e50 tx=0x7f3b74008040 comp rx=0 tx=0).stop 2024-04-25T00:11:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.287+0000 7f3b90400640 1 -- 172.21.15.83:0/2796342124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b8806e4c0 msgr2=0x7f3b8814d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.287+0000 7f3b90400640 1 --2- 172.21.15.83:0/2796342124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b8806e4c0 0x7f3b8814d4e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c01aa70 tx=0x7f3b7c01ac00 comp rx=0 tx=0).stop 2024-04-25T00:11:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.287+0000 7f3b90400640 1 -- 172.21.15.83:0/2796342124 shutdown_connections 2024-04-25T00:11:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.287+0000 7f3b90400640 1 --2- 172.21.15.83:0/2796342124 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3b680678c0 0x7f3b68069d80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.287+0000 7f3b90400640 1 --2- 172.21.15.83:0/2796342124 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b881555c0 0x7f3b881f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.287+0000 7f3b90400640 1 --2- 172.21.15.83:0/2796342124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b881551e0 0x7f3b8814da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.287+0000 7f3b90400640 1 --2- 172.21.15.83:0/2796342124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b8806e4c0 0x7f3b8814d4e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.287+0000 7f3b90400640 1 -- 172.21.15.83:0/2796342124 >> 172.21.15.83:0/2796342124 conn(0x7f3b880b1830 msgr2=0x7f3b8814a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.287+0000 7f3b90400640 1 -- 172.21.15.83:0/2796342124 shutdown_connections 2024-04-25T00:11:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.287+0000 7f3b90400640 1 -- 172.21.15.83:0/2796342124 wait complete. 2024-04-25T00:11:53.310 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:11:53.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.443+0000 7faedb600640 1 Processor -- start 2024-04-25T00:11:53.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.443+0000 7faedb600640 1 -- start start 2024-04-25T00:11:53.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.447+0000 7faedb600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faed4160020 0x7faed4166ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.447+0000 7faedb600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faed40ce000 con 0x7faed4159810 2024-04-25T00:11:53.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.447+0000 7faedb600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faed406eec0 con 0x7faed406e950 2024-04-25T00:11:53.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.447+0000 7faedb600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7faed406e210 con 0x7faed4160020 2024-04-25T00:11:53.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.447+0000 7faed9600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faed4160020 0x7faed4166ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.447+0000 7faed8c00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7faed4159810 0x7faed40cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51910/0 (socket says 172.21.15.83:51910) 2024-04-25T00:11:53.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.447+0000 7faed8c00640 1 -- 172.21.15.83:0/577276996 learned_addr learned my addr 172.21.15.83:0/577276996 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:53.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.447+0000 7faed3e00640 1 -- 172.21.15.83:0/577276996 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3861194149 0 0) 0x7faed40ce000 con 0x7faed4159810 2024-04-25T00:11:53.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.447+0000 7faed9600640 1 -- 172.21.15.83:0/577276996 >> v1:172.21.15.86:6789/0 conn(0x7faed406e950 legacy=0x7faed415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:53.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.447+0000 7faed9600640 1 -- 172.21.15.83:0/577276996 >> v1:172.21.15.83:6789/0 conn(0x7faed4159810 legacy=0x7faed40cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.447+0000 7faed9600640 1 -- 172.21.15.83:0/577276996 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed406dc50 con 0x7faed4160020 2024-04-25T00:11:53.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faed9600640 1 --2- 172.21.15.83:0/577276996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed4160020 0x7faed4166ca0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faecc009d90 tx=0x7faecc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6724e873339fbc49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:53.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faed3e00640 1 -- 172.21.15.83:0/577276996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0b2070 con 0x7faed4160020 2024-04-25T00:11:53.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faed3e00640 1 -- 172.21.15.83:0/577276996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faecc0a58b0 con 0x7faed4160020 2024-04-25T00:11:53.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faed3e00640 1 -- 172.21.15.83:0/577276996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0a5c80 con 0x7faed4160020 2024-04-25T00:11:53.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 -- 172.21.15.83:0/577276996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed4160020 msgr2=0x7faed4166ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 --2- 172.21.15.83:0/577276996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed4160020 0x7faed4166ca0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faecc009d90 tx=0x7faecc01c8e0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 -- 172.21.15.83:0/577276996 shutdown_connections 2024-04-25T00:11:53.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 --2- 172.21.15.83:0/577276996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed4160020 0x7faed4166ca0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 -- 172.21.15.83:0/577276996 >> 172.21.15.83:0/577276996 conn(0x7faed40b1830 msgr2=0x7faed40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:53.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 -- 172.21.15.83:0/577276996 shutdown_connections 2024-04-25T00:11:53.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 -- 172.21.15.83:0/577276996 wait complete. 2024-04-25T00:11:53.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 Processor -- start 2024-04-25T00:11:53.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 -- start start 2024-04-25T00:11:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed406e950 0x7faed414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faed4159810 0x7faed40b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed4160020 0x7faed4154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faed4169b80 con 0x7faed4160020 2024-04-25T00:11:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faed4169a00 con 0x7faed406e950 2024-04-25T00:11:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.451+0000 7faedb600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7faed4169d00 con 0x7faed4159810 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed4160020 0x7faed4154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed4160020 0x7faed4154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48946/0 (socket says 172.21.15.83:48946) 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed9600640 1 -- 172.21.15.83:0/4955727 learned_addr learned my addr 172.21.15.83:0/4955727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed8c00640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed406e950 0x7faed414feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faecbe00640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faed4159810 0x7faed40b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed9600640 1 -- 172.21.15.83:0/4955727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faed4159810 msgr2=0x7faed40b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed9600640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faed4159810 0x7faed40b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed9600640 1 -- 172.21.15.83:0/4955727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed406e950 msgr2=0x7faed414feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed9600640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed406e950 0x7faed414feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed9600640 1 -- 172.21.15.83:0/4955727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed40b31e0 con 0x7faed4160020 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed8c00640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed406e950 0x7faed414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed9600640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed4160020 0x7faed4154540 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7faecc009d90 tx=0x7faecc019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed1600640 1 -- 172.21.15.83:0/4955727 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0bd020 con 0x7faed4160020 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed1600640 1 -- 172.21.15.83:0/4955727 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faecc0b8670 con 0x7faed4160020 2024-04-25T00:11:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed1600640 1 -- 172.21.15.83:0/4955727 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0b2040 con 0x7faed4160020 2024-04-25T00:11:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faedb600640 1 -- 172.21.15.83:0/4955727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed40b34c0 con 0x7faed4160020 2024-04-25T00:11:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faedb600640 1 -- 172.21.15.83:0/4955727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed40b3a80 con 0x7faed4160020 2024-04-25T00:11:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faedb600640 1 -- 172.21.15.83:0/4955727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed404f510 con 0x7faed4160020 2024-04-25T00:11:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.455+0000 7faed1600640 1 -- 172.21.15.83:0/4955727 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faecc0c0020 con 0x7faed4160020 2024-04-25T00:11:53.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.459+0000 7faed1600640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faeb00678c0 0x7faeb0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.459+0000 7faed1600640 1 -- 172.21.15.83:0/4955727 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7faecc120e90 con 0x7faed4160020 2024-04-25T00:11:53.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.459+0000 7faed8c00640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faeb00678c0 0x7faeb0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.459+0000 7faed8c00640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faeb00678c0 0x7faeb0069d80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faec4000f70 tx=0x7faec4007c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:53.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.463+0000 7faed1600640 1 -- 172.21.15.83:0/4955727 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faecc0c7030 con 0x7faed4160020 2024-04-25T00:11:53.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.615+0000 7faedb600640 1 -- 172.21.15.83:0/4955727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7faed4072c90 con 0x7faed4160020 2024-04-25T00:11:53.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.615+0000 7faed1600640 1 -- 172.21.15.83:0/4955727 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7faecc016560 con 0x7faed4160020 2024-04-25T00:11:53.620 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.619+0000 7faedb600640 1 -- 172.21.15.83:0/4955727 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faeb00678c0 msgr2=0x7faeb0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.619+0000 7faedb600640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faeb00678c0 0x7faeb0069d80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faec4000f70 tx=0x7faec4007c60 comp rx=0 tx=0).stop 2024-04-25T00:11:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.619+0000 7faedb600640 1 -- 172.21.15.83:0/4955727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed4160020 msgr2=0x7faed4154540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.619+0000 7faedb600640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed4160020 0x7faed4154540 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7faecc009d90 tx=0x7faecc019680 comp rx=0 tx=0).stop 2024-04-25T00:11:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.619+0000 7faedb600640 1 -- 172.21.15.83:0/4955727 shutdown_connections 2024-04-25T00:11:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.619+0000 7faedb600640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faeb00678c0 0x7faeb0069d80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.619+0000 7faedb600640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faed4159810 0x7faed40b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.619+0000 7faedb600640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed4160020 0x7faed4154540 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.619+0000 7faedb600640 1 --2- 172.21.15.83:0/4955727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed406e950 0x7faed414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.619+0000 7faedb600640 1 -- 172.21.15.83:0/4955727 >> 172.21.15.83:0/4955727 conn(0x7faed40b1830 msgr2=0x7faed415e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.619+0000 7faedb600640 1 -- 172.21.15.83:0/4955727 shutdown_connections 2024-04-25T00:11:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.619+0000 7faedb600640 1 -- 172.21.15.83:0/4955727 wait complete. 2024-04-25T00:11:53.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close' 2024-04-25T00:11:53.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46f0400640 1 Processor -- start 2024-04-25T00:11:53.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46f0400640 1 -- start start 2024-04-25T00:11:53.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46f0400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46e806e7b0 0x7f46e806d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46f0400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e806e0c0 con 0x7f46e815df00 2024-04-25T00:11:53.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46f0400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e80ad990 con 0x7f46e8071dc0 2024-04-25T00:11:53.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46f0400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f46e806f150 con 0x7f46e806e7b0 2024-04-25T00:11:53.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46eda00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f46e815df00 0x7f46e80716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51918/0 (socket says 172.21.15.83:51918) 2024-04-25T00:11:53.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46eda00640 1 -- 172.21.15.83:0/4097492695 learned_addr learned my addr 172.21.15.83:0/4097492695 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:53.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46ee400640 1 --2- 172.21.15.83:0/4097492695 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46e806e7b0 0x7f46e806d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 43300681 0 0) 0x7f46e806e0c0 con 0x7f46e815df00 2024-04-25T00:11:53.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46d0003530 con 0x7f46e815df00 2024-04-25T00:11:53.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 8414359 0 0) 0x7f46e80ad990 con 0x7f46e8071dc0 2024-04-25T00:11:53.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46e806e0c0 con 0x7f46e8071dc0 2024-04-25T00:11:53.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 129876410 0 0) 0x7f46d0003530 con 0x7f46e815df00 2024-04-25T00:11:53.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46e806e7b0 msgr2=0x7f46e806d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:53.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46e7e00640 1 --2- 172.21.15.83:0/4097492695 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46e806e7b0 0x7f46e806d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 >> v1:172.21.15.86:6789/0 conn(0x7f46e8071dc0 legacy=0x7f46e80ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e80adc70 con 0x7f46e815df00 2024-04-25T00:11:53.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f46dc001680 con 0x7f46e815df00 2024-04-25T00:11:53.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 >> v1:172.21.15.83:6789/0 conn(0x7f46e815df00 legacy=0x7f46e80716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46e7e00640 1 --2- 172.21.15.83:0/4097492695 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46e806e7b0 0x7f46d0003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.771+0000 7f46e7e00640 1 --2- 172.21.15.83:0/4097492695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d0004130 0x7f46d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46e7e00640 1 --2- 172.21.15.83:0/4097492695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d0004d50 0x7f46d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46d000bc00 con 0x7f46d0004130 2024-04-25T00:11:53.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46d000bdb0 con 0x7f46d0004d50 2024-04-25T00:11:53.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f46d000bfb0 con 0x7f46e806e7b0 2024-04-25T00:11:53.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46ed000640 1 --2- 172.21.15.83:0/4097492695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d0004130 0x7f46d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46eda00640 1 --2- 172.21.15.83:0/4097492695 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46e806e7b0 0x7f46d0003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46ee400640 1 --2- 172.21.15.83:0/4097492695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d0004d50 0x7f46d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46ed000640 1 -- 172.21.15.83:0/4097492695 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46e806e7b0 msgr2=0x7f46d0003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:53.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46ed000640 1 --2- 172.21.15.83:0/4097492695 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46e806e7b0 0x7f46d0003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46ed000640 1 -- 172.21.15.83:0/4097492695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d0004d50 msgr2=0x7f46d00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:53.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46ed000640 1 --2- 172.21.15.83:0/4097492695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d0004d50 0x7f46d00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46ed000640 1 -- 172.21.15.83:0/4097492695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d0003530 con 0x7f46d0004130 2024-04-25T00:11:53.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46eda00640 1 --2- 172.21.15.83:0/4097492695 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46e806e7b0 0x7f46d0003b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:53.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46ee400640 1 --2- 172.21.15.83:0/4097492695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d0004d50 0x7f46d00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:53.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46ed000640 1 --2- 172.21.15.83:0/4097492695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d0004130 0x7f46d0004810 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f46d40018d0 tx=0x7f46d401c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26252d960d50a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:53.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4001520 con 0x7f46d0004130 2024-04-25T00:11:53.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d40052f0 con 0x7f46d0004130 2024-04-25T00:11:53.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.775+0000 7f46e7e00640 1 -- 172.21.15.83:0/4097492695 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4003840 con 0x7f46d0004130 2024-04-25T00:11:53.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46f0400640 1 -- 172.21.15.83:0/4097492695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d0004130 msgr2=0x7f46d0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46f0400640 1 --2- 172.21.15.83:0/4097492695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d0004130 0x7f46d0004810 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f46d40018d0 tx=0x7f46d401c580 comp rx=0 tx=0).stop 2024-04-25T00:11:53.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46ee400640 1 -- 172.21.15.83:0/4097492695 reap_dead start 2024-04-25T00:11:53.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46f0400640 1 -- 172.21.15.83:0/4097492695 shutdown_connections 2024-04-25T00:11:53.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46f0400640 1 -- 172.21.15.83:0/4097492695 >> 172.21.15.83:0/4097492695 conn(0x7f46e80b1830 msgr2=0x7f46e80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:53.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46f0400640 1 -- 172.21.15.83:0/4097492695 shutdown_connections 2024-04-25T00:11:53.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46f0400640 1 -- 172.21.15.83:0/4097492695 wait complete. 2024-04-25T00:11:53.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46f0400640 1 Processor -- start 2024-04-25T00:11:53.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46f0400640 1 -- start start 2024-04-25T00:11:53.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46f0400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46d0004130 0x7f46e814f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46f0400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e81502c0 0x7f46e80affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46f0400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e8150c40 0x7f46e81510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46f0400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46e80b32a0 con 0x7f46e81502c0 2024-04-25T00:11:53.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46f0400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46e80b3120 con 0x7f46e8150c40 2024-04-25T00:11:53.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46f0400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f46e8169e80 con 0x7f46d0004130 2024-04-25T00:11:53.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46ed000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e81502c0 0x7f46e80affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46ed000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e81502c0 0x7f46e80affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48968/0 (socket says 172.21.15.83:48968) 2024-04-25T00:11:53.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46ed000640 1 -- 172.21.15.83:0/1997173003 learned_addr learned my addr 172.21.15.83:0/1997173003 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:53.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46ed000640 1 -- 172.21.15.83:0/1997173003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46d0004130 msgr2=0x7f46e814f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46eda00640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46d0004130 0x7f46e814f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46ee400640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e8150c40 0x7f46e81510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46ed000640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46d0004130 0x7f46e814f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46ed000640 1 -- 172.21.15.83:0/1997173003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e8150c40 msgr2=0x7f46e81510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46ed000640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e8150c40 0x7f46e81510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46ed000640 1 -- 172.21.15.83:0/1997173003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e814fb80 con 0x7f46e81502c0 2024-04-25T00:11:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46ee400640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e8150c40 0x7f46e81510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46eda00640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46d0004130 0x7f46e814f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46ed000640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e81502c0 0x7f46e80affb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f46d4000c10 tx=0x7f46d0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.779+0000 7f46e5600640 1 -- 172.21.15.83:0/1997173003 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d40054a0 con 0x7f46e81502c0 2024-04-25T00:11:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.783+0000 7f46e5600640 1 -- 172.21.15.83:0/1997173003 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d4001500 con 0x7f46e81502c0 2024-04-25T00:11:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.783+0000 7f46e5600640 1 -- 172.21.15.83:0/1997173003 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d40067e0 con 0x7f46e81502c0 2024-04-25T00:11:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.783+0000 7f46f0400640 1 -- 172.21.15.83:0/1997173003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e80b04f0 con 0x7f46e81502c0 2024-04-25T00:11:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.783+0000 7f46f0400640 1 -- 172.21.15.83:0/1997173003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e80b0aa0 con 0x7f46e81502c0 2024-04-25T00:11:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.783+0000 7f46f0400640 1 -- 172.21.15.83:0/1997173003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e80737f0 con 0x7f46e81502c0 2024-04-25T00:11:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.783+0000 7f46e5600640 1 -- 172.21.15.83:0/1997173003 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f46d4006980 con 0x7f46e81502c0 2024-04-25T00:11:53.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.783+0000 7f46e5600640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f46c80678f0 0x7f46c8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:53.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.783+0000 7f46e5600640 1 -- 172.21.15.83:0/1997173003 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f46d41203e0 con 0x7f46e81502c0 2024-04-25T00:11:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.783+0000 7f46eda00640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f46c80678f0 0x7f46c8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.787+0000 7f46eda00640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f46c80678f0 0x7f46c8069db0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46dc001ba0 tx=0x7f46dc000f20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:53.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:53.787+0000 7f46e5600640 1 -- 172.21.15.83:0/1997173003 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f46d40bb020 con 0x7f46e81502c0 2024-04-25T00:11:54.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.003+0000 7f46f0400640 1 -- 172.21.15.83:0/1997173003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]} v 0) -- 0x7f46e80cf310 con 0x7f46e81502c0 2024-04-25T00:11:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.267+0000 7f46e5600640 1 -- 172.21.15.83:0/1997173003 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]}]=0 v73) ==== 147+0+0 (secure 0 0 0) 0x7f46d4016290 con 0x7f46e81502c0 2024-04-25T00:11:54.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.267+0000 7f46f0400640 1 -- 172.21.15.83:0/1997173003 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f46c80678f0 msgr2=0x7f46c8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:54.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.267+0000 7f46f0400640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f46c80678f0 0x7f46c8069db0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46dc001ba0 tx=0x7f46dc000f20 comp rx=0 tx=0).stop 2024-04-25T00:11:54.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.267+0000 7f46f0400640 1 -- 172.21.15.83:0/1997173003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e81502c0 msgr2=0x7f46e80affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.267+0000 7f46f0400640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e81502c0 0x7f46e80affb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f46d4000c10 tx=0x7f46d0005840 comp rx=0 tx=0).stop 2024-04-25T00:11:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.267+0000 7f46f0400640 1 -- 172.21.15.83:0/1997173003 shutdown_connections 2024-04-25T00:11:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.267+0000 7f46f0400640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f46c80678f0 0x7f46c8069db0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.267+0000 7f46f0400640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e8150c40 0x7f46e81510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.267+0000 7f46f0400640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e81502c0 0x7f46e80affb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.267+0000 7f46f0400640 1 --2- 172.21.15.83:0/1997173003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46d0004130 0x7f46e814f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.267+0000 7f46f0400640 1 -- 172.21.15.83:0/1997173003 >> 172.21.15.83:0/1997173003 conn(0x7f46e80b1830 msgr2=0x7f46e8152cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.267+0000 7f46f0400640 1 -- 172.21.15.83:0/1997173003 shutdown_connections 2024-04-25T00:11:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.267+0000 7f46f0400640 1 -- 172.21.15.83:0/1997173003 wait complete. 2024-04-25T00:11:54.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:11:54.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.423+0000 7ff59f200640 1 Processor -- start 2024-04-25T00:11:54.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.427+0000 7ff59f200640 1 -- start start 2024-04-25T00:11:54.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.427+0000 7ff59f200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff5a00727b0 0x7ff5a015f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.427+0000 7ff59f200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a0070ca0 con 0x7ff5a0072c10 2024-04-25T00:11:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.427+0000 7ff59f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a01605d0 con 0x7ff5a01455d0 2024-04-25T00:11:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.427+0000 7ff59f200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a00ce9d0 con 0x7ff5a00727b0 2024-04-25T00:11:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.427+0000 7ff597e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff5a00727b0 0x7ff5a015f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.427+0000 7ff59e800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff5a0072c10 0x7ff5a0071430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56890/0 (socket says 172.21.15.83:56890) 2024-04-25T00:11:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.427+0000 7ff59e800640 1 -- 172.21.15.83:0/1523051029 learned_addr learned my addr 172.21.15.83:0/1523051029 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.427+0000 7ff597e00640 1 -- 172.21.15.83:0/1523051029 >> v1:172.21.15.86:6789/0 conn(0x7ff5a01455d0 legacy=0x7ff5a006cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.427+0000 7ff597e00640 1 -- 172.21.15.83:0/1523051029 >> v1:172.21.15.83:6789/0 conn(0x7ff5a0072c10 legacy=0x7ff5a0071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.427+0000 7ff597e00640 1 -- 172.21.15.83:0/1523051029 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a00cec20 con 0x7ff5a00727b0 2024-04-25T00:11:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.427+0000 7ff59d400640 1 -- 172.21.15.83:0/1523051029 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1950704091 0 0) 0x7ff5a0070ca0 con 0x7ff5a0072c10 2024-04-25T00:11:54.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.431+0000 7ff597e00640 1 --2- 172.21.15.83:0/1523051029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a00727b0 0x7ff5a015f800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff5880097c0 tx=0x7ff58801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55671dea741f037c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:54.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.431+0000 7ff59d400640 1 -- 172.21.15.83:0/1523051029 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5880b0070 con 0x7ff5a00727b0 2024-04-25T00:11:54.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.431+0000 7ff59d400640 1 -- 172.21.15.83:0/1523051029 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5880ab600 con 0x7ff5a00727b0 2024-04-25T00:11:54.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.431+0000 7ff59d400640 1 -- 172.21.15.83:0/1523051029 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5880ab9d0 con 0x7ff5a00727b0 2024-04-25T00:11:54.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.431+0000 7ff59f200640 1 -- 172.21.15.83:0/1523051029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a00727b0 msgr2=0x7ff5a015f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:54.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.431+0000 7ff59f200640 1 --2- 172.21.15.83:0/1523051029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a00727b0 0x7ff5a015f800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff5880097c0 tx=0x7ff58801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:11:54.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.431+0000 7ff59f200640 1 -- 172.21.15.83:0/1523051029 shutdown_connections 2024-04-25T00:11:54.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.431+0000 7ff59f200640 1 --2- 172.21.15.83:0/1523051029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a00727b0 0x7ff5a015f800 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:54.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.431+0000 7ff59f200640 1 -- 172.21.15.83:0/1523051029 >> 172.21.15.83:0/1523051029 conn(0x7ff5a00cf980 msgr2=0x7ff5a00cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:54.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.431+0000 7ff59f200640 1 -- 172.21.15.83:0/1523051029 shutdown_connections 2024-04-25T00:11:54.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.431+0000 7ff59f200640 1 -- 172.21.15.83:0/1523051029 wait complete. 2024-04-25T00:11:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.431+0000 7ff59f200640 1 Processor -- start 2024-04-25T00:11:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.431+0000 7ff59f200640 1 -- start start 2024-04-25T00:11:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5a00727b0 0x7ff5a015ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a0072c10 0x7ff5a01668f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a01455d0 0x7ff5a015c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a016a3c0 con 0x7ff5a01455d0 2024-04-25T00:11:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a0160150 con 0x7ff5a0072c10 2024-04-25T00:11:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a016a540 con 0x7ff5a00727b0 2024-04-25T00:11:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a01455d0 0x7ff5a015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a01455d0 0x7ff5a015c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39550/0 (socket says 172.21.15.83:39550) 2024-04-25T00:11:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59e800640 1 -- 172.21.15.83:0/2572222931 learned_addr learned my addr 172.21.15.83:0/2572222931 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff597e00640 1 --2- 172.21.15.83:0/2572222931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a0072c10 0x7ff5a01668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59de00640 1 --2- 172.21.15.83:0/2572222931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5a00727b0 0x7ff5a015ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59e800640 1 -- 172.21.15.83:0/2572222931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5a00727b0 msgr2=0x7ff5a015ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59e800640 1 --2- 172.21.15.83:0/2572222931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5a00727b0 0x7ff5a015ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59e800640 1 -- 172.21.15.83:0/2572222931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a0072c10 msgr2=0x7ff5a01668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59e800640 1 --2- 172.21.15.83:0/2572222931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a0072c10 0x7ff5a01668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59e800640 1 -- 172.21.15.83:0/2572222931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a0167370 con 0x7ff5a01455d0 2024-04-25T00:11:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59e800640 1 --2- 172.21.15.83:0/2572222931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a01455d0 0x7ff5a015c080 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c780 tx=0x7ff59800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff596000640 1 -- 172.21.15.83:0/2572222931 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598018070 con 0x7ff5a01455d0 2024-04-25T00:11:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff596000640 1 -- 172.21.15.83:0/2572222931 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff598004d70 con 0x7ff5a01455d0 2024-04-25T00:11:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59f200640 1 -- 172.21.15.83:0/2572222931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a0164f80 con 0x7ff5a01455d0 2024-04-25T00:11:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff596000640 1 -- 172.21.15.83:0/2572222931 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59800a880 con 0x7ff5a01455d0 2024-04-25T00:11:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59f200640 1 -- 172.21.15.83:0/2572222931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a0165540 con 0x7ff5a01455d0 2024-04-25T00:11:54.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff59f200640 1 -- 172.21.15.83:0/2572222931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a004f510 con 0x7ff5a01455d0 2024-04-25T00:11:54.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.435+0000 7ff596000640 1 -- 172.21.15.83:0/2572222931 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff59801b070 con 0x7ff5a01455d0 2024-04-25T00:11:54.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.439+0000 7ff596000640 1 --2- 172.21.15.83:0/2572222931 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff57c0678c0 0x7ff57c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:54.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.439+0000 7ff596000640 1 -- 172.21.15.83:0/2572222931 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff5980854c0 con 0x7ff5a01455d0 2024-04-25T00:11:54.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.439+0000 7ff59de00640 1 --2- 172.21.15.83:0/2572222931 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff57c0678c0 0x7ff57c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:54.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.439+0000 7ff59de00640 1 --2- 172.21.15.83:0/2572222931 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff57c0678c0 0x7ff57c069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff590000f70 tx=0x7ff590009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:54.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.443+0000 7ff596000640 1 -- 172.21.15.83:0/2572222931 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff5980555b0 con 0x7ff5a01455d0 2024-04-25T00:11:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.599+0000 7ff59f200640 1 -- 172.21.15.83:0/2572222931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff5a006e8e0 con 0x7ff5a01455d0 2024-04-25T00:11:54.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.603+0000 7ff596000640 1 -- 172.21.15.83:0/2572222931 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7ff5980598b0 con 0x7ff5a01455d0 2024-04-25T00:11:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.603+0000 7ff59f200640 1 -- 172.21.15.83:0/2572222931 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff57c0678c0 msgr2=0x7ff57c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.603+0000 7ff59f200640 1 --2- 172.21.15.83:0/2572222931 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff57c0678c0 0x7ff57c069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff590000f70 tx=0x7ff590009040 comp rx=0 tx=0).stop 2024-04-25T00:11:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.603+0000 7ff59f200640 1 -- 172.21.15.83:0/2572222931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a01455d0 msgr2=0x7ff5a015c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.603+0000 7ff59f200640 1 --2- 172.21.15.83:0/2572222931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a01455d0 0x7ff5a015c080 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c780 tx=0x7ff59800cc40 comp rx=0 tx=0).stop 2024-04-25T00:11:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.603+0000 7ff59f200640 1 -- 172.21.15.83:0/2572222931 shutdown_connections 2024-04-25T00:11:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.603+0000 7ff59f200640 1 --2- 172.21.15.83:0/2572222931 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff57c0678c0 0x7ff57c069d80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.603+0000 7ff59f200640 1 --2- 172.21.15.83:0/2572222931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a01455d0 0x7ff5a015c080 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.603+0000 7ff59f200640 1 --2- 172.21.15.83:0/2572222931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a0072c10 0x7ff5a01668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.603+0000 7ff59f200640 1 --2- 172.21.15.83:0/2572222931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5a00727b0 0x7ff5a015ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.603+0000 7ff59f200640 1 -- 172.21.15.83:0/2572222931 >> 172.21.15.83:0/2572222931 conn(0x7ff5a00cf980 msgr2=0x7ff5a015dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.603+0000 7ff59f200640 1 -- 172.21.15.83:0/2572222931 shutdown_connections 2024-04-25T00:11:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:54.603+0000 7ff59f200640 1 -- 172.21.15.83:0/2572222931 wait complete. 2024-04-25T00:11:54.628 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:11:54.629 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:11:54.684 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:11:54.744 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:11:54.745 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:11:54.745 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:54.753 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:11:54.753 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:54.805 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:11:54.805 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:54.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:55.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.039+0000 7f1eb7800640 1 Processor -- start 2024-04-25T00:11:55.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.039+0000 7f1eb7800640 1 -- start start 2024-04-25T00:11:55.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.039+0000 7f1eb7800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1eb01599f0 0x7f1eb00cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.039+0000 7f1eb7800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1eb015c340 0x7f1eb015c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.039+0000 7f1eb7800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eb00ccc60 con 0x7f1eb015ccc0 2024-04-25T00:11:55.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.043+0000 7f1eb7800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb0159dd0 con 0x7f1eb015c340 2024-04-25T00:11:55.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.043+0000 7f1eb7800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb006dba0 con 0x7f1eb01599f0 2024-04-25T00:11:55.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.043+0000 7f1ea7e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1eb015c340 0x7f1eb015c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.043+0000 7f1ea7e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1eb015c340 0x7f1eb015c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39562/0 (socket says 172.21.15.83:39562) 2024-04-25T00:11:55.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.043+0000 7f1ea7e00640 1 -- 172.21.15.83:0/4132253528 learned_addr learned my addr 172.21.15.83:0/4132253528 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:55.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.043+0000 7f1eb4e00640 1 --2- 172.21.15.83:0/4132253528 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1eb01599f0 0x7f1eb00cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.043+0000 7f1ea7e00640 1 -- 172.21.15.83:0/4132253528 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1eb01599f0 msgr2=0x7f1eb00cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:55.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.043+0000 7f1ea7e00640 1 --2- 172.21.15.83:0/4132253528 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1eb01599f0 0x7f1eb00cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.043+0000 7f1ea7e00640 1 -- 172.21.15.83:0/4132253528 >> v1:172.21.15.86:6789/0 conn(0x7f1eb015ccc0 legacy=0x7f1eb015f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:55.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.043+0000 7f1ea7e00640 1 -- 172.21.15.83:0/4132253528 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb006de80 con 0x7f1eb015c340 2024-04-25T00:11:55.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.043+0000 7f1ea7e00640 1 --2- 172.21.15.83:0/4132253528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb015c340 0x7f1eb015c780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1e980098a0 tx=0x7f1e9801c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df945768099f3389 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:55.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.043+0000 7f1eafe00640 1 -- 172.21.15.83:0/4132253528 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e980b0070 con 0x7f1eb015c340 2024-04-25T00:11:55.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.043+0000 7f1eafe00640 1 -- 172.21.15.83:0/4132253528 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e980ab600 con 0x7f1eb015c340 2024-04-25T00:11:55.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.043+0000 7f1eafe00640 1 -- 172.21.15.83:0/4132253528 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e980ab9d0 con 0x7f1eb015c340 2024-04-25T00:11:55.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 -- 172.21.15.83:0/4132253528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb015c340 msgr2=0x7f1eb015c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 --2- 172.21.15.83:0/4132253528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb015c340 0x7f1eb015c780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1e980098a0 tx=0x7f1e9801c290 comp rx=0 tx=0).stop 2024-04-25T00:11:55.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 -- 172.21.15.83:0/4132253528 shutdown_connections 2024-04-25T00:11:55.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 --2- 172.21.15.83:0/4132253528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb015c340 0x7f1eb015c780 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 --2- 172.21.15.83:0/4132253528 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1eb01599f0 0x7f1eb00cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 -- 172.21.15.83:0/4132253528 >> 172.21.15.83:0/4132253528 conn(0x7f1eb00b1830 msgr2=0x7f1eb00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:55.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 -- 172.21.15.83:0/4132253528 shutdown_connections 2024-04-25T00:11:55.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 -- 172.21.15.83:0/4132253528 wait complete. 2024-04-25T00:11:55.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 Processor -- start 2024-04-25T00:11:55.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 -- start start 2024-04-25T00:11:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1eb01599f0 0x7f1eb0157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1eb015c340 0x7f1eb014a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb015ccc0 0x7f1eb0150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb00cf870 con 0x7f1eb015ccc0 2024-04-25T00:11:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb00cf6f0 con 0x7f1eb015c340 2024-04-25T00:11:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb7800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb0168cd0 con 0x7f1eb01599f0 2024-04-25T00:11:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb015ccc0 0x7f1eb0150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb015ccc0 0x7f1eb0150290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39564/0 (socket says 172.21.15.83:39564) 2024-04-25T00:11:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb5800640 1 -- 172.21.15.83:0/2467859846 learned_addr learned my addr 172.21.15.83:0/2467859846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb4e00640 1 --2- 172.21.15.83:0/2467859846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1eb01599f0 0x7f1eb0157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1ea7e00640 1 --2- 172.21.15.83:0/2467859846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1eb015c340 0x7f1eb014a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb5800640 1 -- 172.21.15.83:0/2467859846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1eb01599f0 msgr2=0x7f1eb0157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb5800640 1 --2- 172.21.15.83:0/2467859846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1eb01599f0 0x7f1eb0157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb5800640 1 -- 172.21.15.83:0/2467859846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1eb015c340 msgr2=0x7f1eb014a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb5800640 1 --2- 172.21.15.83:0/2467859846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1eb015c340 0x7f1eb014a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb5800640 1 -- 172.21.15.83:0/2467859846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb0157dd0 con 0x7f1eb015ccc0 2024-04-25T00:11:55.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.047+0000 7f1eb5800640 1 --2- 172.21.15.83:0/2467859846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb015ccc0 0x7f1eb0150290 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800b9a0 tx=0x7f1ea800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:55.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.051+0000 7f1ead600640 1 -- 172.21.15.83:0/2467859846 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8019070 con 0x7f1eb015ccc0 2024-04-25T00:11:55.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.051+0000 7f1ead600640 1 -- 172.21.15.83:0/2467859846 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea800cba0 con 0x7f1eb015ccc0 2024-04-25T00:11:55.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.051+0000 7f1eb7800640 1 -- 172.21.15.83:0/2467859846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb01580b0 con 0x7f1eb015ccc0 2024-04-25T00:11:55.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.051+0000 7f1ead600640 1 -- 172.21.15.83:0/2467859846 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8014440 con 0x7f1eb015ccc0 2024-04-25T00:11:55.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.051+0000 7f1eb7800640 1 -- 172.21.15.83:0/2467859846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb014b290 con 0x7f1eb015ccc0 2024-04-25T00:11:55.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.051+0000 7f1eb7800640 1 -- 172.21.15.83:0/2467859846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb00737f0 con 0x7f1eb015ccc0 2024-04-25T00:11:55.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.051+0000 7f1ead600640 1 -- 172.21.15.83:0/2467859846 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1ea801c070 con 0x7f1eb015ccc0 2024-04-25T00:11:55.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.051+0000 7f1ead600640 1 --2- 172.21.15.83:0/2467859846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1e8c0678c0 0x7f1e8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.051+0000 7f1ead600640 1 -- 172.21.15.83:0/2467859846 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1ea8085da0 con 0x7f1eb015ccc0 2024-04-25T00:11:55.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.051+0000 7f1eb4e00640 1 --2- 172.21.15.83:0/2467859846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1e8c0678c0 0x7f1e8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.055+0000 7f1eb4e00640 1 --2- 172.21.15.83:0/2467859846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1e8c0678c0 0x7f1e8c069d80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000a970 tx=0x7f1ea0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:55.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.055+0000 7f1ead600640 1 -- 172.21.15.83:0/2467859846 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1ea8056440 con 0x7f1eb015ccc0 2024-04-25T00:11:55.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.215+0000 7f1eb7800640 1 -- 172.21.15.83:0/2467859846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1eb004f5f0 con 0x7f1eb015ccc0 2024-04-25T00:11:55.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.215+0000 7f1ead600640 1 -- 172.21.15.83:0/2467859846 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18820 (secure 0 0 0) 0x7f1ea8055d40 con 0x7f1eb015ccc0 2024-04-25T00:11:55.220 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:55.220 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:48.235809+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:55.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.219+0000 7f1eb7800640 1 -- 172.21.15.83:0/2467859846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1e8c0678c0 msgr2=0x7f1e8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.219+0000 7f1eb7800640 1 --2- 172.21.15.83:0/2467859846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1e8c0678c0 0x7f1e8c069d80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000a970 tx=0x7f1ea0008040 comp rx=0 tx=0).stop 2024-04-25T00:11:55.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.219+0000 7f1eb7800640 1 -- 172.21.15.83:0/2467859846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb015ccc0 msgr2=0x7f1eb0150290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.219+0000 7f1eb7800640 1 --2- 172.21.15.83:0/2467859846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb015ccc0 0x7f1eb0150290 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800b9a0 tx=0x7f1ea800be60 comp rx=0 tx=0).stop 2024-04-25T00:11:55.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.219+0000 7f1eb7800640 1 -- 172.21.15.83:0/2467859846 shutdown_connections 2024-04-25T00:11:55.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.219+0000 7f1eb7800640 1 --2- 172.21.15.83:0/2467859846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1e8c0678c0 0x7f1e8c069d80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.219+0000 7f1eb7800640 1 --2- 172.21.15.83:0/2467859846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb015ccc0 0x7f1eb0150290 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.219+0000 7f1eb7800640 1 --2- 172.21.15.83:0/2467859846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1eb015c340 0x7f1eb014a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.219+0000 7f1eb7800640 1 --2- 172.21.15.83:0/2467859846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1eb01599f0 0x7f1eb0157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.219+0000 7f1eb7800640 1 -- 172.21.15.83:0/2467859846 >> 172.21.15.83:0/2467859846 conn(0x7f1eb00b1830 msgr2=0x7f1eb006e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:55.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.219+0000 7f1eb7800640 1 -- 172.21.15.83:0/2467859846 shutdown_connections 2024-04-25T00:11:55.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.219+0000 7f1eb7800640 1 -- 172.21.15.83:0/2467859846 wait complete. 2024-04-25T00:11:55.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:11:04.621429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:11:05.189716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:11:08.902215+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:11:55.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:11:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.375+0000 7fa5c7400640 1 Processor -- start 2024-04-25T00:11:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.375+0000 7fa5c7400640 1 -- start start 2024-04-25T00:11:55.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.379+0000 7fa5c7400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa5c01551e0 0x7fa5c015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.379+0000 7fa5c7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa5c006e4c0 0x7fa5c006e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.379+0000 7fa5c7400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5c015c560 con 0x7fa5c01555c0 2024-04-25T00:11:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.379+0000 7fa5c7400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa5c0072460 con 0x7fa5c01551e0 2024-04-25T00:11:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.379+0000 7fa5c7400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa5c015c7d0 con 0x7fa5c006e4c0 2024-04-25T00:11:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.379+0000 7fa5c4a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa5c01551e0 0x7fa5c015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.379+0000 7fa5c4a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa5c01551e0 0x7fa5c015bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39568/0 (socket says 172.21.15.83:39568) 2024-04-25T00:11:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.379+0000 7fa5c4a00640 1 -- 172.21.15.83:0/1458580143 learned_addr learned my addr 172.21.15.83:0/1458580143 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.379+0000 7fa5c5400640 1 --2- 172.21.15.83:0/1458580143 >> v2:172.21.15.86:3300/0 conn(0x7fa5c006e4c0 0x7fa5c006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.379+0000 7fa5c4a00640 1 -- 172.21.15.83:0/1458580143 >> v2:172.21.15.86:3300/0 conn(0x7fa5c006e4c0 msgr2=0x7fa5c006e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.379+0000 7fa5c4a00640 1 --2- 172.21.15.83:0/1458580143 >> v2:172.21.15.86:3300/0 conn(0x7fa5c006e4c0 0x7fa5c006e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.379+0000 7fa5c4a00640 1 -- 172.21.15.83:0/1458580143 >> v1:172.21.15.86:6789/0 conn(0x7fa5c01555c0 legacy=0x7fa5c0071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.379+0000 7fa5c4a00640 1 -- 172.21.15.83:0/1458580143 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c015cab0 con 0x7fa5c01551e0 2024-04-25T00:11:55.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.383+0000 7fa5c4a00640 1 --2- 172.21.15.83:0/1458580143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c01551e0 0x7fa5c015bea0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400bb10 tx=0x7fa5b401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b908c22c77e7b15a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:55.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.383+0000 7fa5bf400640 1 -- 172.21.15.83:0/1458580143 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40b4070 con 0x7fa5c01551e0 2024-04-25T00:11:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.383+0000 7fa5bf400640 1 -- 172.21.15.83:0/1458580143 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b40a79c0 con 0x7fa5c01551e0 2024-04-25T00:11:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.383+0000 7fa5bf400640 1 -- 172.21.15.83:0/1458580143 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40a7d90 con 0x7fa5c01551e0 2024-04-25T00:11:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.383+0000 7fa5c7400640 1 -- 172.21.15.83:0/1458580143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c01551e0 msgr2=0x7fa5c015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.383+0000 7fa5c7400640 1 --2- 172.21.15.83:0/1458580143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c01551e0 0x7fa5c015bea0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400bb10 tx=0x7fa5b401ebf0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.383+0000 7fa5c7400640 1 -- 172.21.15.83:0/1458580143 shutdown_connections 2024-04-25T00:11:55.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.383+0000 7fa5c7400640 1 --2- 172.21.15.83:0/1458580143 >> v2:172.21.15.86:3300/0 conn(0x7fa5c006e4c0 0x7fa5c006e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.383+0000 7fa5c7400640 1 --2- 172.21.15.83:0/1458580143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c01551e0 0x7fa5c015bea0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.383+0000 7fa5c7400640 1 -- 172.21.15.83:0/1458580143 >> 172.21.15.83:0/1458580143 conn(0x7fa5c00b1830 msgr2=0x7fa5c00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:55.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.383+0000 7fa5c7400640 1 -- 172.21.15.83:0/1458580143 shutdown_connections 2024-04-25T00:11:55.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.383+0000 7fa5c7400640 1 -- 172.21.15.83:0/1458580143 wait complete. 2024-04-25T00:11:55.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.383+0000 7fa5c7400640 1 Processor -- start 2024-04-25T00:11:55.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.383+0000 7fa5c7400640 1 -- start start 2024-04-25T00:11:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c7400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5c006e4c0 0x7fa5c0149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c01551e0 0x7fa5c0159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c01555c0 0x7fa5c015a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c7400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c006df90 con 0x7fa5c01555c0 2024-04-25T00:11:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c7400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c006de10 con 0x7fa5c01551e0 2024-04-25T00:11:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c7400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c006e110 con 0x7fa5c006e4c0 2024-04-25T00:11:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c01555c0 0x7fa5c015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c01555c0 0x7fa5c015a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39578/0 (socket says 172.21.15.83:39578) 2024-04-25T00:11:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c5400640 1 -- 172.21.15.83:0/1357159578 learned_addr learned my addr 172.21.15.83:0/1357159578 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5bfe00640 1 --2- 172.21.15.83:0/1357159578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c01551e0 0x7fa5c0159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c4a00640 1 --2- 172.21.15.83:0/1357159578 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5c006e4c0 0x7fa5c0149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c5400640 1 -- 172.21.15.83:0/1357159578 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5c006e4c0 msgr2=0x7fa5c0149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c5400640 1 --2- 172.21.15.83:0/1357159578 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5c006e4c0 0x7fa5c0149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c5400640 1 -- 172.21.15.83:0/1357159578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c01551e0 msgr2=0x7fa5c0159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c5400640 1 --2- 172.21.15.83:0/1357159578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c01551e0 0x7fa5c0159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c5400640 1 -- 172.21.15.83:0/1357159578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c00b30f0 con 0x7fa5c01555c0 2024-04-25T00:11:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c5400640 1 --2- 172.21.15.83:0/1357159578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c01555c0 0x7fa5c015a310 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800efc0 tx=0x7fa5b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:55.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5bcc00640 1 -- 172.21.15.83:0/1357159578 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8018020 con 0x7fa5c01555c0 2024-04-25T00:11:55.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5bcc00640 1 -- 172.21.15.83:0/1357159578 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b8014480 con 0x7fa5c01555c0 2024-04-25T00:11:55.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5bcc00640 1 -- 172.21.15.83:0/1357159578 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8014870 con 0x7fa5c01555c0 2024-04-25T00:11:55.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c7400640 1 -- 172.21.15.83:0/1357159578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c015e6e0 con 0x7fa5c01555c0 2024-04-25T00:11:55.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c7400640 1 -- 172.21.15.83:0/1357159578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c015eca0 con 0x7fa5c01555c0 2024-04-25T00:11:55.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.387+0000 7fa5c7400640 1 -- 172.21.15.83:0/1357159578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c004f510 con 0x7fa5c01555c0 2024-04-25T00:11:55.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.391+0000 7fa5bcc00640 1 -- 172.21.15.83:0/1357159578 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa5b801b020 con 0x7fa5c01555c0 2024-04-25T00:11:55.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.391+0000 7fa5bcc00640 1 --2- 172.21.15.83:0/1357159578 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5a00678c0 0x7fa5a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.391+0000 7fa5bcc00640 1 -- 172.21.15.83:0/1357159578 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa5b8086e20 con 0x7fa5c01555c0 2024-04-25T00:11:55.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.391+0000 7fa5c4a00640 1 --2- 172.21.15.83:0/1357159578 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5a00678c0 0x7fa5a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.391+0000 7fa5c4a00640 1 --2- 172.21.15.83:0/1357159578 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5a00678c0 0x7fa5a0069d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40b1000 tx=0x7fa5b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.395+0000 7fa5bcc00640 1 -- 172.21.15.83:0/1357159578 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa5b8022030 con 0x7fa5c01555c0 2024-04-25T00:11:55.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.551+0000 7fa5c7400640 1 -- 172.21.15.83:0/1357159578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa5c014a250 con 0x7fa5c01555c0 2024-04-25T00:11:55.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.551+0000 7fa5bcc00640 1 -- 172.21.15.83:0/1357159578 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fa5b8010040 con 0x7fa5c01555c0 2024-04-25T00:11:55.557 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:11:55.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.555+0000 7fa5c7400640 1 -- 172.21.15.83:0/1357159578 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5a00678c0 msgr2=0x7fa5a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.555+0000 7fa5c7400640 1 --2- 172.21.15.83:0/1357159578 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5a00678c0 0x7fa5a0069d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40b1000 tx=0x7fa5b4008040 comp rx=0 tx=0).stop 2024-04-25T00:11:55.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.555+0000 7fa5c7400640 1 -- 172.21.15.83:0/1357159578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c01555c0 msgr2=0x7fa5c015a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.555+0000 7fa5c7400640 1 --2- 172.21.15.83:0/1357159578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c01555c0 0x7fa5c015a310 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800efc0 tx=0x7fa5b800c490 comp rx=0 tx=0).stop 2024-04-25T00:11:55.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.555+0000 7fa5c7400640 1 -- 172.21.15.83:0/1357159578 shutdown_connections 2024-04-25T00:11:55.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.555+0000 7fa5c7400640 1 --2- 172.21.15.83:0/1357159578 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5a00678c0 0x7fa5a0069d80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.555+0000 7fa5c7400640 1 --2- 172.21.15.83:0/1357159578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5c01555c0 0x7fa5c015a310 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.555+0000 7fa5c7400640 1 --2- 172.21.15.83:0/1357159578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c01551e0 0x7fa5c0159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.555+0000 7fa5c7400640 1 --2- 172.21.15.83:0/1357159578 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5c006e4c0 0x7fa5c0149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.555+0000 7fa5c7400640 1 -- 172.21.15.83:0/1357159578 >> 172.21.15.83:0/1357159578 conn(0x7fa5c00b1830 msgr2=0x7fa5c014d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:55.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.555+0000 7fa5c7400640 1 -- 172.21.15.83:0/1357159578 shutdown_connections 2024-04-25T00:11:55.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.555+0000 7fa5c7400640 1 -- 172.21.15.83:0/1357159578 wait complete. 2024-04-25T00:11:55.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:11:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.711+0000 7fd96be00640 1 Processor -- start 2024-04-25T00:11:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.711+0000 7fd96be00640 1 -- start start 2024-04-25T00:11:55.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd96c147460 0x7fd96c072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96be00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd96c071b10 0x7fd96c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd96c0b4a10 con 0x7fd96c070fa0 2024-04-25T00:11:55.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96be00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd96c0b2db0 con 0x7fd96c071b10 2024-04-25T00:11:55.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd96c0b3000 con 0x7fd96c147460 2024-04-25T00:11:55.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd96c071b10 0x7fd96c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd96c071b10 0x7fd96c06d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39582/0 (socket says 172.21.15.83:39582) 2024-04-25T00:11:55.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96b400640 1 -- 172.21.15.83:0/1481339223 learned_addr learned my addr 172.21.15.83:0/1481339223 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:55.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96aa00640 1 --2- 172.21.15.83:0/1481339223 >> v2:172.21.15.86:3300/0 conn(0x7fd96c147460 0x7fd96c072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96b400640 1 -- 172.21.15.83:0/1481339223 >> v2:172.21.15.86:3300/0 conn(0x7fd96c147460 msgr2=0x7fd96c072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:55.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96b400640 1 --2- 172.21.15.83:0/1481339223 >> v2:172.21.15.86:3300/0 conn(0x7fd96c147460 0x7fd96c072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96b400640 1 -- 172.21.15.83:0/1481339223 >> v1:172.21.15.86:6789/0 conn(0x7fd96c070fa0 legacy=0x7fd96c071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96b400640 1 -- 172.21.15.83:0/1481339223 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c0b32e0 con 0x7fd96c071b10 2024-04-25T00:11:55.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96aa00640 1 --2- 172.21.15.83:0/1481339223 >> v2:172.21.15.86:3300/0 conn(0x7fd96c147460 0x7fd96c072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:55.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.715+0000 7fd96b400640 1 --2- 172.21.15.83:0/1481339223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c071b10 0x7fd96c06d9c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0739f0 tx=0x7fd96401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b59176dfebf7418 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:55.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd969600640 1 -- 172.21.15.83:0/1481339223 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640b0070 con 0x7fd96c071b10 2024-04-25T00:11:55.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd969600640 1 -- 172.21.15.83:0/1481339223 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9640ab600 con 0x7fd96c071b10 2024-04-25T00:11:55.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd969600640 1 -- 172.21.15.83:0/1481339223 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640ab9d0 con 0x7fd96c071b10 2024-04-25T00:11:55.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 -- 172.21.15.83:0/1481339223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c071b10 msgr2=0x7fd96c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 --2- 172.21.15.83:0/1481339223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c071b10 0x7fd96c06d9c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0739f0 tx=0x7fd96401c490 comp rx=0 tx=0).stop 2024-04-25T00:11:55.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 -- 172.21.15.83:0/1481339223 shutdown_connections 2024-04-25T00:11:55.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 --2- 172.21.15.83:0/1481339223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c071b10 0x7fd96c06d9c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 --2- 172.21.15.83:0/1481339223 >> v2:172.21.15.86:3300/0 conn(0x7fd96c147460 0x7fd96c072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 -- 172.21.15.83:0/1481339223 >> 172.21.15.83:0/1481339223 conn(0x7fd96c0cfda0 msgr2=0x7fd96c0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:55.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 -- 172.21.15.83:0/1481339223 shutdown_connections 2024-04-25T00:11:55.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 -- 172.21.15.83:0/1481339223 wait complete. 2024-04-25T00:11:55.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 Processor -- start 2024-04-25T00:11:55.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 -- start start 2024-04-25T00:11:55.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd96c070fa0 0x7fd96c0cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c071b10 0x7fd96c15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c147460 0x7fd96c144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c16ada0 con 0x7fd96c147460 2024-04-25T00:11:55.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c06e960 con 0x7fd96c071b10 2024-04-25T00:11:55.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd96c16af20 con 0x7fd96c070fa0 2024-04-25T00:11:55.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c147460 0x7fd96c144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c147460 0x7fd96c144680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39592/0 (socket says 172.21.15.83:39592) 2024-04-25T00:11:55.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96b400640 1 -- 172.21.15.83:0/2534732743 learned_addr learned my addr 172.21.15.83:0/2534732743 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:55.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96aa00640 1 --2- 172.21.15.83:0/2534732743 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd96c070fa0 0x7fd96c0cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.719+0000 7fd96a000640 1 --2- 172.21.15.83:0/2534732743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c071b10 0x7fd96c15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd96b400640 1 -- 172.21.15.83:0/2534732743 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd96c070fa0 msgr2=0x7fd96c0cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd96b400640 1 --2- 172.21.15.83:0/2534732743 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd96c070fa0 0x7fd96c0cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd96b400640 1 -- 172.21.15.83:0/2534732743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c071b10 msgr2=0x7fd96c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd96b400640 1 --2- 172.21.15.83:0/2534732743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c071b10 0x7fd96c15d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd96b400640 1 -- 172.21.15.83:0/2534732743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c13f530 con 0x7fd96c147460 2024-04-25T00:11:55.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd96b400640 1 --2- 172.21.15.83:0/2534732743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c147460 0x7fd96c144680 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd9640ae040 tx=0x7fd964019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:55.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd962a00640 1 -- 172.21.15.83:0/2534732743 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640b0040 con 0x7fd96c147460 2024-04-25T00:11:55.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd962a00640 1 -- 172.21.15.83:0/2534732743 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9640b9070 con 0x7fd96c147460 2024-04-25T00:11:55.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd962a00640 1 -- 172.21.15.83:0/2534732743 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640b4640 con 0x7fd96c147460 2024-04-25T00:11:55.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd96be00640 1 -- 172.21.15.83:0/2534732743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96c1411a0 con 0x7fd96c147460 2024-04-25T00:11:55.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd96be00640 1 -- 172.21.15.83:0/2534732743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96c141760 con 0x7fd96c147460 2024-04-25T00:11:55.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd96be00640 1 -- 172.21.15.83:0/2534732743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd96c04f510 con 0x7fd96c147460 2024-04-25T00:11:55.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd962a00640 1 -- 172.21.15.83:0/2534732743 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd9640b47e0 con 0x7fd96c147460 2024-04-25T00:11:55.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd962a00640 1 --2- 172.21.15.83:0/2534732743 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd94c0678f0 0x7fd94c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:55.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd962a00640 1 -- 172.21.15.83:0/2534732743 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd96411fb60 con 0x7fd96c147460 2024-04-25T00:11:55.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.723+0000 7fd96aa00640 1 --2- 172.21.15.83:0/2534732743 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd94c0678f0 0x7fd94c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:55.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.727+0000 7fd96aa00640 1 --2- 172.21.15.83:0/2534732743 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd94c0678f0 0x7fd94c069db0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd95c003b90 tx=0x7fd95c009890 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:55.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.727+0000 7fd962a00640 1 -- 172.21.15.83:0/2534732743 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd964016110 con 0x7fd96c147460 2024-04-25T00:11:55.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.883+0000 7fd96be00640 1 -- 172.21.15.83:0/2534732743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd96c06ede0 con 0x7fd96c147460 2024-04-25T00:11:55.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.883+0000 7fd962a00640 1 -- 172.21.15.83:0/2534732743 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fd9640efc40 con 0x7fd96c147460 2024-04-25T00:11:55.888 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.887+0000 7fd96be00640 1 -- 172.21.15.83:0/2534732743 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd94c0678f0 msgr2=0x7fd94c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.887+0000 7fd96be00640 1 --2- 172.21.15.83:0/2534732743 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd94c0678f0 0x7fd94c069db0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd95c003b90 tx=0x7fd95c009890 comp rx=0 tx=0).stop 2024-04-25T00:11:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.887+0000 7fd96be00640 1 -- 172.21.15.83:0/2534732743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c147460 msgr2=0x7fd96c144680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.887+0000 7fd96be00640 1 --2- 172.21.15.83:0/2534732743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c147460 0x7fd96c144680 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd9640ae040 tx=0x7fd964019230 comp rx=0 tx=0).stop 2024-04-25T00:11:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.887+0000 7fd96be00640 1 -- 172.21.15.83:0/2534732743 shutdown_connections 2024-04-25T00:11:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.887+0000 7fd96be00640 1 --2- 172.21.15.83:0/2534732743 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd94c0678f0 0x7fd94c069db0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.887+0000 7fd96be00640 1 --2- 172.21.15.83:0/2534732743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c147460 0x7fd96c144680 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.887+0000 7fd96be00640 1 --2- 172.21.15.83:0/2534732743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c071b10 0x7fd96c15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.887+0000 7fd96be00640 1 --2- 172.21.15.83:0/2534732743 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd96c070fa0 0x7fd96c0cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.887+0000 7fd96be00640 1 -- 172.21.15.83:0/2534732743 >> 172.21.15.83:0/2534732743 conn(0x7fd96c0cfda0 msgr2=0x7fd96c071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.887+0000 7fd96be00640 1 -- 172.21.15.83:0/2534732743 shutdown_connections 2024-04-25T00:11:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:55.887+0000 7fd96be00640 1 -- 172.21.15.83:0/2534732743 wait complete. 2024-04-25T00:11:55.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:11:56.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.043+0000 7fbb3f200640 1 Processor -- start 2024-04-25T00:11:56.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.043+0000 7fbb3f200640 1 -- start start 2024-04-25T00:11:56.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.043+0000 7fbb3f200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbb381551e0 0x7fbb3815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.043+0000 7fbb3f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb3806e4c0 0x7fbb3806e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.043+0000 7fbb3f200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb3815c560 con 0x7fbb381555c0 2024-04-25T00:11:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.043+0000 7fbb3f200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbb38072460 con 0x7fbb381551e0 2024-04-25T00:11:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.043+0000 7fbb3f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3815c7d0 con 0x7fbb3806e4c0 2024-04-25T00:11:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.043+0000 7fbb37e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbb381551e0 0x7fbb3815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.043+0000 7fbb37e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbb381551e0 0x7fbb3815bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39608/0 (socket says 172.21.15.83:39608) 2024-04-25T00:11:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.043+0000 7fbb2f400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbb381555c0 0x7fbb38071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56904/0 (socket says 172.21.15.83:56904) 2024-04-25T00:11:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.047+0000 7fbb37e00640 1 -- 172.21.15.83:0/104854637 learned_addr learned my addr 172.21.15.83:0/104854637 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.047+0000 7fbb3d200640 1 --2- 172.21.15.83:0/104854637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb3806e4c0 0x7fbb3806e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.047+0000 7fbb37e00640 1 -- 172.21.15.83:0/104854637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb3806e4c0 msgr2=0x7fbb3806e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.047+0000 7fbb37e00640 1 --2- 172.21.15.83:0/104854637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb3806e4c0 0x7fbb3806e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.047+0000 7fbb37e00640 1 -- 172.21.15.83:0/104854637 >> v1:172.21.15.83:6789/0 conn(0x7fbb381555c0 legacy=0x7fbb38071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.047+0000 7fbb37e00640 1 -- 172.21.15.83:0/104854637 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3815cab0 con 0x7fbb381551e0 2024-04-25T00:11:56.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.047+0000 7fbb3d200640 1 --2- 172.21.15.83:0/104854637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb3806e4c0 0x7fbb3806e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:56.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.047+0000 7fbb37400640 1 -- 172.21.15.83:0/104854637 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2082470494 0 0) 0x7fbb3815c560 con 0x7fbb381555c0 2024-04-25T00:11:56.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.047+0000 7fbb37e00640 1 --2- 172.21.15.83:0/104854637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb381551e0 0x7fbb3815bea0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009e80 tx=0x7fbb2801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2020df461f63da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:56.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.047+0000 7fbb37400640 1 -- 172.21.15.83:0/104854637 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb280a56e0 con 0x7fbb381551e0 2024-04-25T00:11:56.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.047+0000 7fbb37400640 1 -- 172.21.15.83:0/104854637 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb280a5880 con 0x7fbb381551e0 2024-04-25T00:11:56.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.047+0000 7fbb37400640 1 -- 172.21.15.83:0/104854637 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb280a5c50 con 0x7fbb381551e0 2024-04-25T00:11:56.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 -- 172.21.15.83:0/104854637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb381551e0 msgr2=0x7fbb3815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 --2- 172.21.15.83:0/104854637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb381551e0 0x7fbb3815bea0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009e80 tx=0x7fbb2801ca10 comp rx=0 tx=0).stop 2024-04-25T00:11:56.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 -- 172.21.15.83:0/104854637 shutdown_connections 2024-04-25T00:11:56.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 --2- 172.21.15.83:0/104854637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb3806e4c0 0x7fbb3806e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 --2- 172.21.15.83:0/104854637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb381551e0 0x7fbb3815bea0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 -- 172.21.15.83:0/104854637 >> 172.21.15.83:0/104854637 conn(0x7fbb380b1830 msgr2=0x7fbb380b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:56.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 -- 172.21.15.83:0/104854637 shutdown_connections 2024-04-25T00:11:56.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 -- 172.21.15.83:0/104854637 wait complete. 2024-04-25T00:11:56.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 Processor -- start 2024-04-25T00:11:56.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 -- start start 2024-04-25T00:11:56.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb3806e4c0 0x7fbb380b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb381551e0 0x7fbb38151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb381555c0 0x7fbb380ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3815dbb0 con 0x7fbb3806e4c0 2024-04-25T00:11:56.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3815da30 con 0x7fbb381551e0 2024-04-25T00:11:56.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3815dd30 con 0x7fbb381555c0 2024-04-25T00:11:56.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb37e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb3806e4c0 0x7fbb380b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb37e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb3806e4c0 0x7fbb380b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39624/0 (socket says 172.21.15.83:39624) 2024-04-25T00:11:56.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb37e00640 1 -- 172.21.15.83:0/468097814 learned_addr learned my addr 172.21.15.83:0/468097814 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:56.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb3d200640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb381555c0 0x7fbb380ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb2f400640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb381551e0 0x7fbb38151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb37e00640 1 -- 172.21.15.83:0/468097814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb381555c0 msgr2=0x7fbb380ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb37e00640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb381555c0 0x7fbb380ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb37e00640 1 -- 172.21.15.83:0/468097814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb381551e0 msgr2=0x7fbb38151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb37e00640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb381551e0 0x7fbb38151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb37e00640 1 -- 172.21.15.83:0/468097814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb380cee50 con 0x7fbb3806e4c0 2024-04-25T00:11:56.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb2f400640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb381551e0 0x7fbb38151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:56.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.051+0000 7fbb37e00640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb3806e4c0 0x7fbb380b47d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbb2801a640 tx=0x7fbb280197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.055+0000 7fbb34c00640 1 -- 172.21.15.83:0/468097814 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb280bd020 con 0x7fbb3806e4c0 2024-04-25T00:11:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.055+0000 7fbb34c00640 1 -- 172.21.15.83:0/468097814 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb280b88a0 con 0x7fbb3806e4c0 2024-04-25T00:11:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.055+0000 7fbb3f200640 1 -- 172.21.15.83:0/468097814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb380cf130 con 0x7fbb3806e4c0 2024-04-25T00:11:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.055+0000 7fbb34c00640 1 -- 172.21.15.83:0/468097814 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb280b8c90 con 0x7fbb3806e4c0 2024-04-25T00:11:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.055+0000 7fbb3f200640 1 -- 172.21.15.83:0/468097814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb38151610 con 0x7fbb3806e4c0 2024-04-25T00:11:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.055+0000 7fbb3f200640 1 -- 172.21.15.83:0/468097814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3804f510 con 0x7fbb3806e4c0 2024-04-25T00:11:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.055+0000 7fbb34c00640 1 -- 172.21.15.83:0/468097814 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbb280039a0 con 0x7fbb3806e4c0 2024-04-25T00:11:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.055+0000 7fbb34c00640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbb140678f0 0x7fbb14069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.055+0000 7fbb34c00640 1 -- 172.21.15.83:0/468097814 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fbb28120b80 con 0x7fbb3806e4c0 2024-04-25T00:11:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.055+0000 7fbb2f400640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbb140678f0 0x7fbb14069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.055+0000 7fbb2f400640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbb140678f0 0x7fbb14069db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbb20007fd0 tx=0x7fbb20009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.059+0000 7fbb34c00640 1 -- 172.21.15.83:0/468097814 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbb28016600 con 0x7fbb3806e4c0 2024-04-25T00:11:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.211+0000 7fbb3f200640 1 -- 172.21.15.83:0/468097814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbb380cf3c0 con 0x7fbb3806e4c0 2024-04-25T00:11:56.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.215+0000 7fbb34c00640 1 -- 172.21.15.83:0/468097814 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fbb280a5850 con 0x7fbb3806e4c0 2024-04-25T00:11:56.218 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:11:56.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.215+0000 7fbb3f200640 1 -- 172.21.15.83:0/468097814 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbb140678f0 msgr2=0x7fbb14069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.215+0000 7fbb3f200640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbb140678f0 0x7fbb14069db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbb20007fd0 tx=0x7fbb20009040 comp rx=0 tx=0).stop 2024-04-25T00:11:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.215+0000 7fbb3f200640 1 -- 172.21.15.83:0/468097814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb3806e4c0 msgr2=0x7fbb380b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.215+0000 7fbb3f200640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb3806e4c0 0x7fbb380b47d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbb2801a640 tx=0x7fbb280197b0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.215+0000 7fbb3f200640 1 -- 172.21.15.83:0/468097814 shutdown_connections 2024-04-25T00:11:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.215+0000 7fbb3f200640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbb140678f0 0x7fbb14069db0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.215+0000 7fbb3f200640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb381555c0 0x7fbb380ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.215+0000 7fbb3f200640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb381551e0 0x7fbb38151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.215+0000 7fbb3f200640 1 --2- 172.21.15.83:0/468097814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb3806e4c0 0x7fbb380b47d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.215+0000 7fbb3f200640 1 -- 172.21.15.83:0/468097814 >> 172.21.15.83:0/468097814 conn(0x7fbb380b1830 msgr2=0x7fbb3815ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.215+0000 7fbb3f200640 1 -- 172.21.15.83:0/468097814 shutdown_connections 2024-04-25T00:11:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.215+0000 7fbb3f200640 1 -- 172.21.15.83:0/468097814 wait complete. 2024-04-25T00:11:56.239 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:11:56.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc91400640 1 Processor -- start 2024-04-25T00:11:56.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc91400640 1 -- start start 2024-04-25T00:11:56.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc91400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc8c159810 0x7fbc8c0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc91400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbc8c06e950 0x7fbc8c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc91400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc8c0ce000 con 0x7fbc8c15fe50 2024-04-25T00:11:56.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc91400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbc8c06e210 con 0x7fbc8c06e950 2024-04-25T00:11:56.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc91400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbc8c06da00 con 0x7fbc8c159810 2024-04-25T00:11:56.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc8b200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbc8c15fe50 0x7fbc8c15c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56908/0 (socket says 172.21.15.83:56908) 2024-04-25T00:11:56.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc8b200640 1 -- 172.21.15.83:0/3228032193 learned_addr learned my addr 172.21.15.83:0/3228032193 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:56.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc89e00640 1 --2- 172.21.15.83:0/3228032193 >> v2:172.21.15.86:3300/0 conn(0x7fbc8c06e950 0x7fbc8c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc8a800640 1 --2- 172.21.15.83:0/3228032193 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc8c159810 0x7fbc8c0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc89400640 1 -- 172.21.15.83:0/3228032193 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2544047059 0 0) 0x7fbc8c0ce000 con 0x7fbc8c15fe50 2024-04-25T00:11:56.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc89400640 1 -- 172.21.15.83:0/3228032193 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc70003530 con 0x7fbc8c15fe50 2024-04-25T00:11:56.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc8a800640 1 -- 172.21.15.83:0/3228032193 >> v2:172.21.15.86:3300/0 conn(0x7fbc8c06e950 msgr2=0x7fbc8c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:56.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc8a800640 1 --2- 172.21.15.83:0/3228032193 >> v2:172.21.15.86:3300/0 conn(0x7fbc8c06e950 0x7fbc8c15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc8a800640 1 -- 172.21.15.83:0/3228032193 >> v1:172.21.15.83:6789/0 conn(0x7fbc8c15fe50 legacy=0x7fbc8c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc8a800640 1 -- 172.21.15.83:0/3228032193 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c06eec0 con 0x7fbc8c159810 2024-04-25T00:11:56.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.375+0000 7fbc89e00640 1 --2- 172.21.15.83:0/3228032193 >> v2:172.21.15.86:3300/0 conn(0x7fbc8c06e950 0x7fbc8c15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:56.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.379+0000 7fbc8a800640 1 --2- 172.21.15.83:0/3228032193 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc8c159810 0x7fbc8c0cc4a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0097c0 tx=0x7fbc7c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb440ce8544a782b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:56.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.379+0000 7fbc89400640 1 -- 172.21.15.83:0/3228032193 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0b0070 con 0x7fbc8c159810 2024-04-25T00:11:56.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.379+0000 7fbc89400640 1 -- 172.21.15.83:0/3228032193 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc7c0ab600 con 0x7fbc8c159810 2024-04-25T00:11:56.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.379+0000 7fbc89400640 1 -- 172.21.15.83:0/3228032193 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0ab9d0 con 0x7fbc8c159810 2024-04-25T00:11:56.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.379+0000 7fbc91400640 1 -- 172.21.15.83:0/3228032193 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc8c159810 msgr2=0x7fbc8c0cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.379+0000 7fbc91400640 1 --2- 172.21.15.83:0/3228032193 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc8c159810 0x7fbc8c0cc4a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0097c0 tx=0x7fbc7c01c410 comp rx=0 tx=0).stop 2024-04-25T00:11:56.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.379+0000 7fbc91400640 1 -- 172.21.15.83:0/3228032193 shutdown_connections 2024-04-25T00:11:56.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.379+0000 7fbc91400640 1 --2- 172.21.15.83:0/3228032193 >> v2:172.21.15.86:3300/0 conn(0x7fbc8c06e950 0x7fbc8c15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.379+0000 7fbc91400640 1 --2- 172.21.15.83:0/3228032193 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc8c159810 0x7fbc8c0cc4a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.379+0000 7fbc91400640 1 -- 172.21.15.83:0/3228032193 >> 172.21.15.83:0/3228032193 conn(0x7fbc8c0b1830 msgr2=0x7fbc8c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:56.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.379+0000 7fbc91400640 1 -- 172.21.15.83:0/3228032193 shutdown_connections 2024-04-25T00:11:56.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.379+0000 7fbc91400640 1 -- 172.21.15.83:0/3228032193 wait complete. 2024-04-25T00:11:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc91400640 1 Processor -- start 2024-04-25T00:11:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc91400640 1 -- start start 2024-04-25T00:11:56.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc91400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c06e950 0x7fbc8c13e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc91400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc8c159810 0x7fbc8c1511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc91400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8c15fe50 0x7fbc8c151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc91400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c159d70 con 0x7fbc8c06e950 2024-04-25T00:11:56.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc91400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c159bf0 con 0x7fbc8c15fe50 2024-04-25T00:11:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc91400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbc8c070da0 con 0x7fbc8c159810 2024-04-25T00:11:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc8a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c06e950 0x7fbc8c13e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc8a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c06e950 0x7fbc8c13e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39632/0 (socket says 172.21.15.83:39632) 2024-04-25T00:11:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc8a800640 1 -- 172.21.15.83:0/1030831574 learned_addr learned my addr 172.21.15.83:0/1030831574 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc8b200640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8c15fe50 0x7fbc8c151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc89e00640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc8c159810 0x7fbc8c1511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc8a800640 1 -- 172.21.15.83:0/1030831574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc8c159810 msgr2=0x7fbc8c1511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc8a800640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc8c159810 0x7fbc8c1511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc8a800640 1 -- 172.21.15.83:0/1030831574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8c15fe50 msgr2=0x7fbc8c151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc8a800640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8c15fe50 0x7fbc8c151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc8a800640 1 -- 172.21.15.83:0/1030831574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c158970 con 0x7fbc8c06e950 2024-04-25T00:11:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc8b200640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8c15fe50 0x7fbc8c151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc8a800640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c06e950 0x7fbc8c13e270 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0ae040 tx=0x7fbc7c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc82a00640 1 -- 172.21.15.83:0/1030831574 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0b0050 con 0x7fbc8c06e950 2024-04-25T00:11:56.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc82a00640 1 -- 172.21.15.83:0/1030831574 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc7c0b1040 con 0x7fbc8c06e950 2024-04-25T00:11:56.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc82a00640 1 -- 172.21.15.83:0/1030831574 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c006e20 con 0x7fbc8c06e950 2024-04-25T00:11:56.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc91400640 1 -- 172.21.15.83:0/1030831574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8c158c50 con 0x7fbc8c06e950 2024-04-25T00:11:56.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.383+0000 7fbc91400640 1 -- 172.21.15.83:0/1030831574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8c164e70 con 0x7fbc8c06e950 2024-04-25T00:11:56.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.387+0000 7fbc91400640 1 -- 172.21.15.83:0/1030831574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8c04f510 con 0x7fbc8c06e950 2024-04-25T00:11:56.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.387+0000 7fbc82a00640 1 -- 172.21.15.83:0/1030831574 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbc7c0be030 con 0x7fbc8c06e950 2024-04-25T00:11:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.387+0000 7fbc82a00640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc6c0678c0 0x7fbc6c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.387+0000 7fbc82a00640 1 -- 172.21.15.83:0/1030831574 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fbc7c11fc90 con 0x7fbc8c06e950 2024-04-25T00:11:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.387+0000 7fbc89e00640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc6c0678c0 0x7fbc6c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.387+0000 7fbc89e00640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc6c0678c0 0x7fbc6c069d80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009580 tx=0x7fbc78009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:56.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.391+0000 7fbc82a00640 1 -- 172.21.15.83:0/1030831574 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbc7c0c5020 con 0x7fbc8c06e950 2024-04-25T00:11:56.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.547+0000 7fbc91400640 1 -- 172.21.15.83:0/1030831574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbc8c072c90 con 0x7fbc8c06e950 2024-04-25T00:11:56.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.551+0000 7fbc82a00640 1 -- 172.21.15.83:0/1030831574 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fbc7c016090 con 0x7fbc8c06e950 2024-04-25T00:11:56.553 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:56.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.551+0000 7fbc91400640 1 -- 172.21.15.83:0/1030831574 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc6c0678c0 msgr2=0x7fbc6c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.551+0000 7fbc91400640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc6c0678c0 0x7fbc6c069d80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009580 tx=0x7fbc78009190 comp rx=0 tx=0).stop 2024-04-25T00:11:56.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.551+0000 7fbc91400640 1 -- 172.21.15.83:0/1030831574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c06e950 msgr2=0x7fbc8c13e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.551+0000 7fbc91400640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c06e950 0x7fbc8c13e270 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0ae040 tx=0x7fbc7c01ac50 comp rx=0 tx=0).stop 2024-04-25T00:11:56.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.551+0000 7fbc91400640 1 -- 172.21.15.83:0/1030831574 shutdown_connections 2024-04-25T00:11:56.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.551+0000 7fbc91400640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc6c0678c0 0x7fbc6c069d80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.551+0000 7fbc91400640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc8c159810 0x7fbc8c1511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.551+0000 7fbc91400640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8c15fe50 0x7fbc8c151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.551+0000 7fbc91400640 1 --2- 172.21.15.83:0/1030831574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c06e950 0x7fbc8c13e270 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.551+0000 7fbc91400640 1 -- 172.21.15.83:0/1030831574 >> 172.21.15.83:0/1030831574 conn(0x7fbc8c0b1830 msgr2=0x7fbc8c152bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:56.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.551+0000 7fbc91400640 1 -- 172.21.15.83:0/1030831574 shutdown_connections 2024-04-25T00:11:56.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.551+0000 7fbc91400640 1 -- 172.21.15.83:0/1030831574 wait complete. 2024-04-25T00:11:56.574 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:11:56.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.703+0000 7f8c2b600640 1 Processor -- start 2024-04-25T00:11:56.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c2b600640 1 -- start start 2024-04-25T00:11:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c2b600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c241446e0 0x7f8c240cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c2b600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c2406ee20 0x7f8c24071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c2b600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c240cb210 con 0x7f8c2415f350 2024-04-25T00:11:56.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c2b600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8c240715b0 con 0x7f8c241446e0 2024-04-25T00:11:56.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c2b600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8c24144ac0 con 0x7f8c2406ee20 2024-04-25T00:11:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c28c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c241446e0 0x7f8c240cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c29600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c2406ee20 0x7f8c24071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c29600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c2406ee20 0x7f8c24071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:55666/0 (socket says 172.21.15.83:55666) 2024-04-25T00:11:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c1be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8c2415f350 0x7f8c2406e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:32906/0 (socket says 172.21.15.83:32906) 2024-04-25T00:11:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c29600640 1 -- 172.21.15.83:0/2475194203 learned_addr learned my addr 172.21.15.83:0/2475194203 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c28c00640 1 -- 172.21.15.83:0/2475194203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c2406ee20 msgr2=0x7f8c24071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c28c00640 1 --2- 172.21.15.83:0/2475194203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c2406ee20 0x7f8c24071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c28c00640 1 -- 172.21.15.83:0/2475194203 >> v1:172.21.15.86:6789/0 conn(0x7f8c2415f350 legacy=0x7f8c2406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c28c00640 1 -- 172.21.15.83:0/2475194203 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c240cb4f0 con 0x7f8c241446e0 2024-04-25T00:11:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c29600640 1 --2- 172.21.15.83:0/2475194203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c2406ee20 0x7f8c24071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.707+0000 7f8c23e00640 1 -- 172.21.15.83:0/2475194203 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2771488564 0 0) 0x7f8c240cb210 con 0x7f8c2415f350 2024-04-25T00:11:56.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.711+0000 7f8c28c00640 1 --2- 172.21.15.83:0/2475194203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c241446e0 0x7f8c240cab50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8c140097c0 tx=0x7f8c1401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c93e8b052d3e313 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:56.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.711+0000 7f8c23e00640 1 -- 172.21.15.83:0/2475194203 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c140b0070 con 0x7f8c241446e0 2024-04-25T00:11:56.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.711+0000 7f8c23e00640 1 -- 172.21.15.83:0/2475194203 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c140ab600 con 0x7f8c241446e0 2024-04-25T00:11:56.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.711+0000 7f8c23e00640 1 -- 172.21.15.83:0/2475194203 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c140ab9d0 con 0x7f8c241446e0 2024-04-25T00:11:56.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.711+0000 7f8c2b600640 1 -- 172.21.15.83:0/2475194203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c241446e0 msgr2=0x7f8c240cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.711+0000 7f8c2b600640 1 --2- 172.21.15.83:0/2475194203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c241446e0 0x7f8c240cab50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8c140097c0 tx=0x7f8c1401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.711+0000 7f8c2b600640 1 -- 172.21.15.83:0/2475194203 shutdown_connections 2024-04-25T00:11:56.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.711+0000 7f8c2b600640 1 --2- 172.21.15.83:0/2475194203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c2406ee20 0x7f8c24071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.711+0000 7f8c2b600640 1 --2- 172.21.15.83:0/2475194203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c241446e0 0x7f8c240cab50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.711+0000 7f8c2b600640 1 -- 172.21.15.83:0/2475194203 >> 172.21.15.83:0/2475194203 conn(0x7f8c240cfa60 msgr2=0x7f8c240cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:56.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.711+0000 7f8c2b600640 1 -- 172.21.15.83:0/2475194203 shutdown_connections 2024-04-25T00:11:56.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.711+0000 7f8c2b600640 1 -- 172.21.15.83:0/2475194203 wait complete. 2024-04-25T00:11:56.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.711+0000 7f8c2b600640 1 Processor -- start 2024-04-25T00:11:56.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.711+0000 7f8c2b600640 1 -- start start 2024-04-25T00:11:56.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c2b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2406ee20 0x7f8c240b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c2b600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c241446e0 0x7f8c2415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c2b600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c2415f350 0x7f8c240c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c2b600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c24072eb0 con 0x7f8c2415f350 2024-04-25T00:11:56.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c2b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c24072d30 con 0x7f8c2406ee20 2024-04-25T00:11:56.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c2b600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8c24073030 con 0x7f8c241446e0 2024-04-25T00:11:56.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c29600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c2415f350 0x7f8c240c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c29600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c2415f350 0x7f8c240c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39636/0 (socket says 172.21.15.83:39636) 2024-04-25T00:11:56.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c29600640 1 -- 172.21.15.83:0/3596216862 learned_addr learned my addr 172.21.15.83:0/3596216862 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:56.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c28c00640 1 --2- 172.21.15.83:0/3596216862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2406ee20 0x7f8c240b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c1be00640 1 --2- 172.21.15.83:0/3596216862 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c241446e0 0x7f8c2415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c29600640 1 -- 172.21.15.83:0/3596216862 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c241446e0 msgr2=0x7f8c2415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c29600640 1 --2- 172.21.15.83:0/3596216862 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c241446e0 0x7f8c2415b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c29600640 1 -- 172.21.15.83:0/3596216862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2406ee20 msgr2=0x7f8c240b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c29600640 1 --2- 172.21.15.83:0/3596216862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2406ee20 0x7f8c240b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c29600640 1 -- 172.21.15.83:0/3596216862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c241419f0 con 0x7f8c2415f350 2024-04-25T00:11:56.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c29600640 1 --2- 172.21.15.83:0/3596216862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c2415f350 0x7f8c240c8db0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00e340 tx=0x7f8c1c00e800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:56.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c21600640 1 -- 172.21.15.83:0/3596216862 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c01a070 con 0x7f8c2415f350 2024-04-25T00:11:56.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c21600640 1 -- 172.21.15.83:0/3596216862 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c1c00ee90 con 0x7f8c2415f350 2024-04-25T00:11:56.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c21600640 1 -- 172.21.15.83:0/3596216862 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c009600 con 0x7f8c2415f350 2024-04-25T00:11:56.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c2b600640 1 -- 172.21.15.83:0/3596216862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c24141c90 con 0x7f8c2415f350 2024-04-25T00:11:56.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c2b600640 1 -- 172.21.15.83:0/3596216862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c24142250 con 0x7f8c2415f350 2024-04-25T00:11:56.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.715+0000 7f8c2b600640 1 -- 172.21.15.83:0/3596216862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c24068230 con 0x7f8c2415f350 2024-04-25T00:11:56.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.719+0000 7f8c21600640 1 -- 172.21.15.83:0/3596216862 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8c1c0097a0 con 0x7f8c2415f350 2024-04-25T00:11:56.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.719+0000 7f8c21600640 1 --2- 172.21.15.83:0/3596216862 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c000678c0 0x7f8c00069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.719+0000 7f8c21600640 1 -- 172.21.15.83:0/3596216862 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8c1c086530 con 0x7f8c2415f350 2024-04-25T00:11:56.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.719+0000 7f8c28c00640 1 --2- 172.21.15.83:0/3596216862 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c000678c0 0x7f8c00069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.719+0000 7f8c28c00640 1 --2- 172.21.15.83:0/3596216862 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c000678c0 0x7f8c00069d80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8c140ad000 tx=0x7f8c140a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:56.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.723+0000 7f8c21600640 1 -- 172.21.15.83:0/3596216862 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8c1c056610 con 0x7f8c2415f350 2024-04-25T00:11:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.891+0000 7f8c2b600640 1 -- 172.21.15.83:0/3596216862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8c240cdb40 con 0x7f8c2415f350 2024-04-25T00:11:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.891+0000 7f8c21600640 1 -- 172.21.15.83:0/3596216862 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f8c1c05a910 con 0x7f8c2415f350 2024-04-25T00:11:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:11:56.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.891+0000 7f8c2b600640 1 -- 172.21.15.83:0/3596216862 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c000678c0 msgr2=0x7f8c00069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.891+0000 7f8c2b600640 1 --2- 172.21.15.83:0/3596216862 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c000678c0 0x7f8c00069d80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8c140ad000 tx=0x7f8c140a6000 comp rx=0 tx=0).stop 2024-04-25T00:11:56.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.891+0000 7f8c2b600640 1 -- 172.21.15.83:0/3596216862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c2415f350 msgr2=0x7f8c240c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.891+0000 7f8c2b600640 1 --2- 172.21.15.83:0/3596216862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c2415f350 0x7f8c240c8db0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00e340 tx=0x7f8c1c00e800 comp rx=0 tx=0).stop 2024-04-25T00:11:56.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.891+0000 7f8c2b600640 1 -- 172.21.15.83:0/3596216862 shutdown_connections 2024-04-25T00:11:56.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.891+0000 7f8c2b600640 1 --2- 172.21.15.83:0/3596216862 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c000678c0 0x7f8c00069d80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.891+0000 7f8c2b600640 1 --2- 172.21.15.83:0/3596216862 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c241446e0 0x7f8c2415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.891+0000 7f8c2b600640 1 --2- 172.21.15.83:0/3596216862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c2415f350 0x7f8c240c8db0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.891+0000 7f8c2b600640 1 --2- 172.21.15.83:0/3596216862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2406ee20 0x7f8c240b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.891+0000 7f8c2b600640 1 -- 172.21.15.83:0/3596216862 >> 172.21.15.83:0/3596216862 conn(0x7f8c240cfa60 msgr2=0x7f8c240af710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:56.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.895+0000 7f8c2b600640 1 -- 172.21.15.83:0/3596216862 shutdown_connections 2024-04-25T00:11:56.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:56.895+0000 7f8c2b600640 1 -- 172.21.15.83:0/3596216862 wait complete. 2024-04-25T00:11:56.916 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:11:56.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:56.916 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T00:11:56.951 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:11:56.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:56.951 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:11:57.005 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:11:57.006 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:57.015 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:11:57.015 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:57.065 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:11:57.065 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:57.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:57.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.247+0000 7fe9d8600640 1 Processor -- start 2024-04-25T00:11:57.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.247+0000 7fe9d8600640 1 -- start start 2024-04-25T00:11:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d8600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe9d01553e0 0x7fe9d006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d8600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9d006ea00 0x7fe9d00b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d8600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe9d00b48a0 0x7fe9d015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d8600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d006ef80 con 0x7fe9d00b48a0 2024-04-25T00:11:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d8600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d015b510 con 0x7fe9d01553e0 2024-04-25T00:11:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d8600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d015b7f0 con 0x7fe9d006ea00 2024-04-25T00:11:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d6600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe9d00b48a0 0x7fe9d015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d6600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe9d00b48a0 0x7fe9d015e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39642/0 (socket says 172.21.15.83:39642) 2024-04-25T00:11:57.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d6600640 1 -- 172.21.15.83:0/202457149 learned_addr learned my addr 172.21.15.83:0/202457149 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:57.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d5200640 1 --2- 172.21.15.83:0/202457149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9d006ea00 0x7fe9d00b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d5c00640 1 --2- 172.21.15.83:0/202457149 >> v2:172.21.15.86:3300/0 conn(0x7fe9d01553e0 0x7fe9d006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d6600640 1 -- 172.21.15.83:0/202457149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9d006ea00 msgr2=0x7fe9d00b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:57.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d6600640 1 --2- 172.21.15.83:0/202457149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9d006ea00 0x7fe9d00b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d6600640 1 -- 172.21.15.83:0/202457149 >> v2:172.21.15.86:3300/0 conn(0x7fe9d01553e0 msgr2=0x7fe9d006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:57.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d6600640 1 --2- 172.21.15.83:0/202457149 >> v2:172.21.15.86:3300/0 conn(0x7fe9d01553e0 0x7fe9d006e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.251+0000 7fe9d6600640 1 -- 172.21.15.83:0/202457149 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d015bad0 con 0x7fe9d00b48a0 2024-04-25T00:11:57.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9d6600640 1 --2- 172.21.15.83:0/202457149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9d00b48a0 0x7fe9d015e9f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0068b30 tx=0x7fe9c801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1d631dca38762b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:57.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9cfe00640 1 -- 172.21.15.83:0/202457149 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80a9430 con 0x7fe9d00b48a0 2024-04-25T00:11:57.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9cfe00640 1 -- 172.21.15.83:0/202457149 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c80ae070 con 0x7fe9d00b48a0 2024-04-25T00:11:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9cfe00640 1 -- 172.21.15.83:0/202457149 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80a97f0 con 0x7fe9d00b48a0 2024-04-25T00:11:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9d8600640 1 -- 172.21.15.83:0/202457149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9d00b48a0 msgr2=0x7fe9d015e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9d8600640 1 --2- 172.21.15.83:0/202457149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9d00b48a0 0x7fe9d015e9f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0068b30 tx=0x7fe9c801c6a0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9d8600640 1 -- 172.21.15.83:0/202457149 shutdown_connections 2024-04-25T00:11:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9d8600640 1 --2- 172.21.15.83:0/202457149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9d006ea00 0x7fe9d00b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9d8600640 1 --2- 172.21.15.83:0/202457149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9d00b48a0 0x7fe9d015e9f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9d8600640 1 --2- 172.21.15.83:0/202457149 >> v2:172.21.15.86:3300/0 conn(0x7fe9d01553e0 0x7fe9d006e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9d8600640 1 -- 172.21.15.83:0/202457149 >> 172.21.15.83:0/202457149 conn(0x7fe9d00b1830 msgr2=0x7fe9d00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9d8600640 1 -- 172.21.15.83:0/202457149 shutdown_connections 2024-04-25T00:11:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9d8600640 1 -- 172.21.15.83:0/202457149 wait complete. 2024-04-25T00:11:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9d8600640 1 Processor -- start 2024-04-25T00:11:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.255+0000 7fe9d8600640 1 -- start start 2024-04-25T00:11:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d8600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d006ea00 0x7fe9d006db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d8600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9d00b48a0 0x7fe9d014edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d8600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9d01553e0 0x7fe9d014f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d8600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d006d530 con 0x7fe9d00b48a0 2024-04-25T00:11:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d8600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d006d3b0 con 0x7fe9d006ea00 2024-04-25T00:11:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d8600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d006d6b0 con 0x7fe9d01553e0 2024-04-25T00:11:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9d00b48a0 0x7fe9d014edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9d00b48a0 0x7fe9d014edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39658/0 (socket says 172.21.15.83:39658) 2024-04-25T00:11:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d5200640 1 -- 172.21.15.83:0/3270201498 learned_addr learned my addr 172.21.15.83:0/3270201498 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d5c00640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d006ea00 0x7fe9d006db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d6600640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9d01553e0 0x7fe9d014f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d5200640 1 -- 172.21.15.83:0/3270201498 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9d01553e0 msgr2=0x7fe9d014f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d5200640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9d01553e0 0x7fe9d014f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d5200640 1 -- 172.21.15.83:0/3270201498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d006ea00 msgr2=0x7fe9d006db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d5200640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d006ea00 0x7fe9d006db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d5200640 1 -- 172.21.15.83:0/3270201498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d0159370 con 0x7fe9d00b48a0 2024-04-25T00:11:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d6600640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9d01553e0 0x7fe9d014f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d5c00640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d006ea00 0x7fe9d006db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d5200640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9d00b48a0 0x7fe9d014edf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00d830 tx=0x7fe9bc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9cd600640 1 -- 172.21.15.83:0/3270201498 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc018020 con 0x7fe9d00b48a0 2024-04-25T00:11:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9cd600640 1 -- 172.21.15.83:0/3270201498 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9bc0096a0 con 0x7fe9d00b48a0 2024-04-25T00:11:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d8600640 1 -- 172.21.15.83:0/3270201498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d00cd280 con 0x7fe9d00b48a0 2024-04-25T00:11:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9cd600640 1 -- 172.21.15.83:0/3270201498 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc009a50 con 0x7fe9d00b48a0 2024-04-25T00:11:57.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d8600640 1 -- 172.21.15.83:0/3270201498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d01595f0 con 0x7fe9d00b48a0 2024-04-25T00:11:57.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9d8600640 1 -- 172.21.15.83:0/3270201498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d00737f0 con 0x7fe9d00b48a0 2024-04-25T00:11:57.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.259+0000 7fe9cd600640 1 -- 172.21.15.83:0/3270201498 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe9bc01b020 con 0x7fe9d00b48a0 2024-04-25T00:11:57.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.263+0000 7fe9cd600640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9b00678c0 0x7fe9b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.263+0000 7fe9cd600640 1 -- 172.21.15.83:0/3270201498 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe9bc086870 con 0x7fe9d00b48a0 2024-04-25T00:11:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.263+0000 7fe9d5c00640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9b00678c0 0x7fe9b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.263+0000 7fe9d5c00640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9b00678c0 0x7fe9b0069d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400b9f0 tx=0x7fe9c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.267+0000 7fe9cd600640 1 -- 172.21.15.83:0/3270201498 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe9bc022020 con 0x7fe9d00b48a0 2024-04-25T00:11:57.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.423+0000 7fe9d8600640 1 -- 172.21.15.83:0/3270201498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9d004f5f0 con 0x7fe9d00b48a0 2024-04-25T00:11:57.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.427+0000 7fe9cd600640 1 -- 172.21.15.83:0/3270201498 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18820 (secure 0 0 0) 0x7fe9d0159370 con 0x7fe9d00b48a0 2024-04-25T00:11:57.429 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:57.430 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:48.235809+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:57.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.427+0000 7fe9d8600640 1 -- 172.21.15.83:0/3270201498 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9b00678c0 msgr2=0x7fe9b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:57.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.427+0000 7fe9d8600640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9b00678c0 0x7fe9b0069d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400b9f0 tx=0x7fe9c4008040 comp rx=0 tx=0).stop 2024-04-25T00:11:57.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.427+0000 7fe9d8600640 1 -- 172.21.15.83:0/3270201498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9d00b48a0 msgr2=0x7fe9d014edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:57.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.427+0000 7fe9d8600640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9d00b48a0 0x7fe9d014edf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00d830 tx=0x7fe9bc00dcf0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.427+0000 7fe9d8600640 1 -- 172.21.15.83:0/3270201498 shutdown_connections 2024-04-25T00:11:57.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.427+0000 7fe9d8600640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9b00678c0 0x7fe9b0069d80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.427+0000 7fe9d8600640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9d01553e0 0x7fe9d014f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.427+0000 7fe9d8600640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9d00b48a0 0x7fe9d014edf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.427+0000 7fe9d8600640 1 --2- 172.21.15.83:0/3270201498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d006ea00 0x7fe9d006db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.427+0000 7fe9d8600640 1 -- 172.21.15.83:0/3270201498 >> 172.21.15.83:0/3270201498 conn(0x7fe9d00b1830 msgr2=0x7fe9d0163f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:57.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.427+0000 7fe9d8600640 1 -- 172.21.15.83:0/3270201498 shutdown_connections 2024-04-25T00:11:57.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.427+0000 7fe9d8600640 1 -- 172.21.15.83:0/3270201498 wait complete. 2024-04-25T00:11:57.452 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:11:04.621429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:11:05.189716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:11:08.902215+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:11:57.452 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:11:57.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd984000640 1 Processor -- start 2024-04-25T00:11:57.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd984000640 1 -- start start 2024-04-25T00:11:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd984000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd97c159810 0x7fd97c0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd984000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd97c06e950 0x7fd97c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd984000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd97c0ce000 con 0x7fd97c15fe50 2024-04-25T00:11:57.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd984000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd97c06e210 con 0x7fd97c06e950 2024-04-25T00:11:57.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd984000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd97c06da00 con 0x7fd97c159810 2024-04-25T00:11:57.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd982000640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd97c15fe50 0x7fd97c15c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56912/0 (socket says 172.21.15.83:56912) 2024-04-25T00:11:57.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd982000640 1 -- 172.21.15.83:0/3093898994 learned_addr learned my addr 172.21.15.83:0/3093898994 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd980c00640 1 --2- 172.21.15.83:0/3093898994 >> v2:172.21.15.86:3300/0 conn(0x7fd97c06e950 0x7fd97c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd981600640 1 --2- 172.21.15.83:0/3093898994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd97c159810 0x7fd97c0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 -- 172.21.15.83:0/3093898994 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 112051256 0 0) 0x7fd97c0ce000 con 0x7fd97c15fe50 2024-04-25T00:11:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 -- 172.21.15.83:0/3093898994 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd964003530 con 0x7fd97c15fe50 2024-04-25T00:11:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 -- 172.21.15.83:0/3093898994 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 197856389 0 0) 0x7fd964003530 con 0x7fd97c15fe50 2024-04-25T00:11:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 -- 172.21.15.83:0/3093898994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd97c159810 msgr2=0x7fd97c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 --2- 172.21.15.83:0/3093898994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd97c159810 0x7fd97c0cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 -- 172.21.15.83:0/3093898994 >> v2:172.21.15.86:3300/0 conn(0x7fd97c06e950 msgr2=0x7fd97c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 --2- 172.21.15.83:0/3093898994 >> v2:172.21.15.86:3300/0 conn(0x7fd97c06e950 0x7fd97c15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 -- 172.21.15.83:0/3093898994 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c06eec0 con 0x7fd97c15fe50 2024-04-25T00:11:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 -- 172.21.15.83:0/3093898994 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fd974001680 con 0x7fd97c15fe50 2024-04-25T00:11:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 -- 172.21.15.83:0/3093898994 >> v1:172.21.15.83:6789/0 conn(0x7fd97c15fe50 legacy=0x7fd97c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 --2- 172.21.15.83:0/3093898994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd97c159810 0x7fd964003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 --2- 172.21.15.83:0/3093898994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd964003fb0 0x7fd964004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 --2- 172.21.15.83:0/3093898994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd964004d50 0x7fd9640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 -- 172.21.15.83:0/3093898994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c06e210 con 0x7fd964004d50 2024-04-25T00:11:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 -- 172.21.15.83:0/3093898994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c06da00 con 0x7fd964003fb0 2024-04-25T00:11:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd97be00640 1 -- 172.21.15.83:0/3093898994 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd96400ba80 con 0x7fd97c159810 2024-04-25T00:11:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd982000640 1 --2- 172.21.15.83:0/3093898994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd964004d50 0x7fd9640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd980c00640 1 --2- 172.21.15.83:0/3093898994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd964003fb0 0x7fd964004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd981600640 1 --2- 172.21.15.83:0/3093898994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd97c159810 0x7fd964003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd982000640 1 -- 172.21.15.83:0/3093898994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd97c159810 msgr2=0x7fd964003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:57.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd982000640 1 --2- 172.21.15.83:0/3093898994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd97c159810 0x7fd964003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd982000640 1 -- 172.21.15.83:0/3093898994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd964003fb0 msgr2=0x7fd964004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:57.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.587+0000 7fd982000640 1 --2- 172.21.15.83:0/3093898994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd964003fb0 0x7fd964004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.591+0000 7fd982000640 1 -- 172.21.15.83:0/3093898994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd964003530 con 0x7fd964004d50 2024-04-25T00:11:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.591+0000 7fd982000640 1 --2- 172.21.15.83:0/3093898994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd964004d50 0x7fd9640051d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c210 tx=0x7fd97401f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d3bb14b1f29bcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.591+0000 7fd97be00640 1 -- 172.21.15.83:0/3093898994 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974007d60 con 0x7fd964004d50 2024-04-25T00:11:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.591+0000 7fd97be00640 1 -- 172.21.15.83:0/3093898994 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd974005ae0 con 0x7fd964004d50 2024-04-25T00:11:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.591+0000 7fd97be00640 1 -- 172.21.15.83:0/3093898994 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974007070 con 0x7fd964004d50 2024-04-25T00:11:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.591+0000 7fd984000640 1 -- 172.21.15.83:0/3093898994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd964004d50 msgr2=0x7fd9640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.591+0000 7fd984000640 1 --2- 172.21.15.83:0/3093898994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd964004d50 0x7fd9640051d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c210 tx=0x7fd97401f3f0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.591+0000 7fd982000640 1 -- 172.21.15.83:0/3093898994 reap_dead start 2024-04-25T00:11:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.591+0000 7fd984000640 1 -- 172.21.15.83:0/3093898994 shutdown_connections 2024-04-25T00:11:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.591+0000 7fd984000640 1 -- 172.21.15.83:0/3093898994 >> 172.21.15.83:0/3093898994 conn(0x7fd97c0b1830 msgr2=0x7fd97c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.591+0000 7fd984000640 1 -- 172.21.15.83:0/3093898994 shutdown_connections 2024-04-25T00:11:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.591+0000 7fd984000640 1 -- 172.21.15.83:0/3093898994 wait complete. 2024-04-25T00:11:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd984000640 1 Processor -- start 2024-04-25T00:11:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd984000640 1 -- start start 2024-04-25T00:11:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd984000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd964004d50 0x7fd97c14de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd984000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd97c150b10 0x7fd97c155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd984000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd97c1430f0 0x7fd97c143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd984000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c159bf0 con 0x7fd97c150b10 2024-04-25T00:11:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd984000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c06df30 con 0x7fd964004d50 2024-04-25T00:11:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd984000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd97c159d70 con 0x7fd97c1430f0 2024-04-25T00:11:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd980c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd97c150b10 0x7fd97c155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd980c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd97c150b10 0x7fd97c155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39678/0 (socket says 172.21.15.83:39678) 2024-04-25T00:11:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd980c00640 1 -- 172.21.15.83:0/4055738978 learned_addr learned my addr 172.21.15.83:0/4055738978 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd981600640 1 --2- 172.21.15.83:0/4055738978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd964004d50 0x7fd97c14de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd982000640 1 --2- 172.21.15.83:0/4055738978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd97c1430f0 0x7fd97c143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd980c00640 1 -- 172.21.15.83:0/4055738978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd97c1430f0 msgr2=0x7fd97c143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd980c00640 1 --2- 172.21.15.83:0/4055738978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd97c1430f0 0x7fd97c143570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd980c00640 1 -- 172.21.15.83:0/4055738978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd964004d50 msgr2=0x7fd97c14de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd980c00640 1 --2- 172.21.15.83:0/4055738978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd964004d50 0x7fd97c14de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd980c00640 1 -- 172.21.15.83:0/4055738978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c150f90 con 0x7fd97c150b10 2024-04-25T00:11:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd980c00640 1 --2- 172.21.15.83:0/4055738978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd97c150b10 0x7fd97c155bb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd968011610 tx=0x7fd968011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd979600640 1 -- 172.21.15.83:0/4055738978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96800f8f0 con 0x7fd97c150b10 2024-04-25T00:11:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd979600640 1 -- 172.21.15.83:0/4055738978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd96800fa90 con 0x7fd97c150b10 2024-04-25T00:11:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd979600640 1 -- 172.21.15.83:0/4055738978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96800fe40 con 0x7fd97c150b10 2024-04-25T00:11:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd984000640 1 -- 172.21.15.83:0/4055738978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97c152c00 con 0x7fd97c150b10 2024-04-25T00:11:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.595+0000 7fd984000640 1 -- 172.21.15.83:0/4055738978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97c1f20b0 con 0x7fd97c150b10 2024-04-25T00:11:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.599+0000 7fd984000640 1 -- 172.21.15.83:0/4055738978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd97c04f510 con 0x7fd97c150b10 2024-04-25T00:11:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.599+0000 7fd979600640 1 -- 172.21.15.83:0/4055738978 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd968015460 con 0x7fd97c150b10 2024-04-25T00:11:57.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.599+0000 7fd979600640 1 --2- 172.21.15.83:0/4055738978 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd95c0678c0 0x7fd95c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.599+0000 7fd979600640 1 -- 172.21.15.83:0/4055738978 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd968086700 con 0x7fd97c150b10 2024-04-25T00:11:57.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.599+0000 7fd981600640 1 --2- 172.21.15.83:0/4055738978 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd95c0678c0 0x7fd95c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.599+0000 7fd981600640 1 --2- 172.21.15.83:0/4055738978 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd95c0678c0 0x7fd95c069d80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd970007520 tx=0x7fd97000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:57.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.603+0000 7fd979600640 1 -- 172.21.15.83:0/4055738978 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd968019030 con 0x7fd97c150b10 2024-04-25T00:11:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.759+0000 7fd984000640 1 -- 172.21.15.83:0/4055738978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd97c0d01e0 con 0x7fd97c150b10 2024-04-25T00:11:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.759+0000 7fd979600640 1 -- 172.21.15.83:0/4055738978 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fd9680567e0 con 0x7fd97c150b10 2024-04-25T00:11:57.762 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:11:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.759+0000 7fd984000640 1 -- 172.21.15.83:0/4055738978 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd95c0678c0 msgr2=0x7fd95c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.759+0000 7fd984000640 1 --2- 172.21.15.83:0/4055738978 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd95c0678c0 0x7fd95c069d80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd970007520 tx=0x7fd97000b040 comp rx=0 tx=0).stop 2024-04-25T00:11:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.759+0000 7fd984000640 1 -- 172.21.15.83:0/4055738978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd97c150b10 msgr2=0x7fd97c155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.759+0000 7fd984000640 1 --2- 172.21.15.83:0/4055738978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd97c150b10 0x7fd97c155bb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd968011610 tx=0x7fd968011ad0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.759+0000 7fd984000640 1 -- 172.21.15.83:0/4055738978 shutdown_connections 2024-04-25T00:11:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.759+0000 7fd984000640 1 --2- 172.21.15.83:0/4055738978 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd95c0678c0 0x7fd95c069d80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.759+0000 7fd984000640 1 --2- 172.21.15.83:0/4055738978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd97c1430f0 0x7fd97c143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.759+0000 7fd984000640 1 --2- 172.21.15.83:0/4055738978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd97c150b10 0x7fd97c155bb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.759+0000 7fd984000640 1 --2- 172.21.15.83:0/4055738978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd964004d50 0x7fd97c14de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.759+0000 7fd984000640 1 -- 172.21.15.83:0/4055738978 >> 172.21.15.83:0/4055738978 conn(0x7fd97c0b1830 msgr2=0x7fd97c158b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.759+0000 7fd984000640 1 -- 172.21.15.83:0/4055738978 shutdown_connections 2024-04-25T00:11:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.759+0000 7fd984000640 1 -- 172.21.15.83:0/4055738978 wait complete. 2024-04-25T00:11:57.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:11:57.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.919+0000 7f92ab000640 1 Processor -- start 2024-04-25T00:11:57.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.919+0000 7f92ab000640 1 -- start start 2024-04-25T00:11:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.919+0000 7f92ab000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f92a41551e0 0x7f92a415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.919+0000 7f92ab000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92a406e4c0 0x7f92a406e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.919+0000 7f92ab000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92a415c560 con 0x7f92a41555c0 2024-04-25T00:11:57.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.919+0000 7f92ab000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f92a4072460 con 0x7f92a41551e0 2024-04-25T00:11:57.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.919+0000 7f92ab000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f92a415c7d0 con 0x7f92a406e4c0 2024-04-25T00:11:57.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.919+0000 7f92a3e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f92a41551e0 0x7f92a415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.919+0000 7f92a3e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f92a41551e0 0x7f92a415bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39688/0 (socket says 172.21.15.83:39688) 2024-04-25T00:11:57.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.919+0000 7f92a3e00640 1 -- 172.21.15.83:0/2488331752 learned_addr learned my addr 172.21.15.83:0/2488331752 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:57.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.919+0000 7f92a3e00640 1 -- 172.21.15.83:0/2488331752 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92a406e4c0 msgr2=0x7f92a406e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:57.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.919+0000 7f92a3e00640 1 --2- 172.21.15.83:0/2488331752 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92a406e4c0 0x7f92a406e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.919+0000 7f92a3e00640 1 -- 172.21.15.83:0/2488331752 >> v1:172.21.15.86:6789/0 conn(0x7f92a41555c0 legacy=0x7f92a4071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:57.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.919+0000 7f92a3e00640 1 -- 172.21.15.83:0/2488331752 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a415cab0 con 0x7f92a41551e0 2024-04-25T00:11:57.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.923+0000 7f92a3e00640 1 --2- 172.21.15.83:0/2488331752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41551e0 0x7f92a415bea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f92940098a0 tx=0x7f929401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87a3253db1652c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:57.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.923+0000 7f92a3400640 1 -- 172.21.15.83:0/2488331752 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92940b0070 con 0x7f92a41551e0 2024-04-25T00:11:57.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.923+0000 7f92a3400640 1 -- 172.21.15.83:0/2488331752 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92940ab600 con 0x7f92a41551e0 2024-04-25T00:11:57.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.923+0000 7f92a3400640 1 -- 172.21.15.83:0/2488331752 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92940ab9d0 con 0x7f92a41551e0 2024-04-25T00:11:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.923+0000 7f92ab000640 1 -- 172.21.15.83:0/2488331752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41551e0 msgr2=0x7f92a415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.923+0000 7f92ab000640 1 --2- 172.21.15.83:0/2488331752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41551e0 0x7f92a415bea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f92940098a0 tx=0x7f929401c500 comp rx=0 tx=0).stop 2024-04-25T00:11:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.923+0000 7f92ab000640 1 -- 172.21.15.83:0/2488331752 shutdown_connections 2024-04-25T00:11:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.923+0000 7f92ab000640 1 --2- 172.21.15.83:0/2488331752 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92a406e4c0 0x7f92a406e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.923+0000 7f92ab000640 1 --2- 172.21.15.83:0/2488331752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41551e0 0x7f92a415bea0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.923+0000 7f92ab000640 1 -- 172.21.15.83:0/2488331752 >> 172.21.15.83:0/2488331752 conn(0x7f92a40b1830 msgr2=0x7f92a40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.923+0000 7f92ab000640 1 -- 172.21.15.83:0/2488331752 shutdown_connections 2024-04-25T00:11:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.923+0000 7f92ab000640 1 -- 172.21.15.83:0/2488331752 wait complete. 2024-04-25T00:11:57.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92ab000640 1 Processor -- start 2024-04-25T00:11:57.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92ab000640 1 -- start start 2024-04-25T00:11:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92ab000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a406e4c0 0x7f92a41425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92ab000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92a41551e0 0x7f92a41515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92ab000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41555c0 0x7f92a4152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92ab000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92a415dbb0 con 0x7f92a406e4c0 2024-04-25T00:11:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92ab000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92a415da30 con 0x7f92a41555c0 2024-04-25T00:11:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92ab000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f92a415dd30 con 0x7f92a41551e0 2024-04-25T00:11:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92a3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a406e4c0 0x7f92a41425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92a3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a406e4c0 0x7f92a41425e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39704/0 (socket says 172.21.15.83:39704) 2024-04-25T00:11:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92a3e00640 1 -- 172.21.15.83:0/4051083962 learned_addr learned my addr 172.21.15.83:0/4051083962 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f929be00640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92a41551e0 0x7f92a41515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92a9000640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41555c0 0x7f92a4152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92a3e00640 1 -- 172.21.15.83:0/4051083962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92a41551e0 msgr2=0x7f92a41515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:57.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92a3e00640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92a41551e0 0x7f92a41515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92a3e00640 1 -- 172.21.15.83:0/4051083962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41555c0 msgr2=0x7f92a4152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:57.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92a3e00640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41555c0 0x7f92a4152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:57.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92a3e00640 1 -- 172.21.15.83:0/4051083962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a414ed70 con 0x7f92a406e4c0 2024-04-25T00:11:57.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92a9000640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41555c0 0x7f92a4152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:57.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f929be00640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92a41551e0 0x7f92a41515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:57.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92a3e00640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a406e4c0 0x7f92a41425e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f92940ae040 tx=0x7f929401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:57.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92a0c00640 1 -- 172.21.15.83:0/4051083962 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92940b0040 con 0x7f92a406e4c0 2024-04-25T00:11:57.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92a0c00640 1 -- 172.21.15.83:0/4051083962 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92940bc070 con 0x7f92a406e4c0 2024-04-25T00:11:57.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92ab000640 1 -- 172.21.15.83:0/4051083962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a40b48f0 con 0x7f92a406e4c0 2024-04-25T00:11:57.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92a0c00640 1 -- 172.21.15.83:0/4051083962 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92940aa8e0 con 0x7f92a406e4c0 2024-04-25T00:11:57.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92ab000640 1 -- 172.21.15.83:0/4051083962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a414e670 con 0x7f92a406e4c0 2024-04-25T00:11:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.927+0000 7f92ab000640 1 -- 172.21.15.83:0/4051083962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a404f510 con 0x7f92a406e4c0 2024-04-25T00:11:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.931+0000 7f92a0c00640 1 -- 172.21.15.83:0/4051083962 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f92940aaa80 con 0x7f92a406e4c0 2024-04-25T00:11:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.931+0000 7f92a0c00640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92800678f0 0x7f9280069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:57.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.931+0000 7f92a0c00640 1 -- 172.21.15.83:0/4051083962 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9294120790 con 0x7f92a406e4c0 2024-04-25T00:11:57.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.931+0000 7f929be00640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92800678f0 0x7f9280069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:57.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.931+0000 7f929be00640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92800678f0 0x7f9280069db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f92a4142b20 tx=0x7f928c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:57.935+0000 7f92a0c00640 1 -- 172.21.15.83:0/4051083962 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f92a414ed70 con 0x7f92a406e4c0 2024-04-25T00:11:58.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.091+0000 7f92ab000640 1 -- 172.21.15.83:0/4051083962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f92a4151d30 con 0x7f92a406e4c0 2024-04-25T00:11:58.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.091+0000 7f92a0c00640 1 -- 172.21.15.83:0/4051083962 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f92940160f0 con 0x7f92a406e4c0 2024-04-25T00:11:58.094 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.091+0000 7f92ab000640 1 -- 172.21.15.83:0/4051083962 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92800678f0 msgr2=0x7f9280069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.091+0000 7f92ab000640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92800678f0 0x7f9280069db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f92a4142b20 tx=0x7f928c008040 comp rx=0 tx=0).stop 2024-04-25T00:11:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.091+0000 7f92ab000640 1 -- 172.21.15.83:0/4051083962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a406e4c0 msgr2=0x7f92a41425e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.091+0000 7f92ab000640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a406e4c0 0x7f92a41425e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f92940ae040 tx=0x7f929401ad40 comp rx=0 tx=0).stop 2024-04-25T00:11:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.091+0000 7f92ab000640 1 -- 172.21.15.83:0/4051083962 shutdown_connections 2024-04-25T00:11:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.091+0000 7f92ab000640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92800678f0 0x7f9280069db0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.091+0000 7f92ab000640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92a41551e0 0x7f92a41515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.091+0000 7f92ab000640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41555c0 0x7f92a4152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.091+0000 7f92ab000640 1 --2- 172.21.15.83:0/4051083962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a406e4c0 0x7f92a41425e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.091+0000 7f92ab000640 1 -- 172.21.15.83:0/4051083962 >> 172.21.15.83:0/4051083962 conn(0x7f92a40b1830 msgr2=0x7f92a415f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.091+0000 7f92ab000640 1 -- 172.21.15.83:0/4051083962 shutdown_connections 2024-04-25T00:11:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.091+0000 7f92ab000640 1 -- 172.21.15.83:0/4051083962 wait complete. 2024-04-25T00:11:58.115 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:11:58.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.247+0000 7f8203000640 1 Processor -- start 2024-04-25T00:11:58.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.247+0000 7f8203000640 1 -- start start 2024-04-25T00:11:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.251+0000 7f8203000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f81fc159810 0x7f81fc0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.251+0000 7f8203000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81fc06e950 0x7f81fc15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.251+0000 7f8203000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81fc0ce000 con 0x7f81fc15fe50 2024-04-25T00:11:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.251+0000 7f8203000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f81fc06e210 con 0x7f81fc159810 2024-04-25T00:11:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.251+0000 7f8203000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f81fc06da00 con 0x7f81fc06e950 2024-04-25T00:11:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.251+0000 7f81fbe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f81fc159810 0x7f81fc0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.251+0000 7f81fbe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f81fc159810 0x7f81fc0cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39714/0 (socket says 172.21.15.83:39714) 2024-04-25T00:11:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.251+0000 7f81fbe00640 1 -- 172.21.15.83:0/92242729 learned_addr learned my addr 172.21.15.83:0/92242729 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.251+0000 7f81f3400640 1 --2- 172.21.15.83:0/92242729 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81fc06e950 0x7f81fc15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.251+0000 7f81fbe00640 1 -- 172.21.15.83:0/92242729 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81fc06e950 msgr2=0x7f81fc15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.251+0000 7f81fbe00640 1 --2- 172.21.15.83:0/92242729 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81fc06e950 0x7f81fc15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.251+0000 7f81fbe00640 1 -- 172.21.15.83:0/92242729 >> v1:172.21.15.86:6789/0 conn(0x7f81fc15fe50 legacy=0x7f81fc15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.251+0000 7f81fbe00640 1 -- 172.21.15.83:0/92242729 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc06eec0 con 0x7f81fc159810 2024-04-25T00:11:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.251+0000 7f81f3400640 1 --2- 172.21.15.83:0/92242729 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81fc06e950 0x7f81fc15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.255+0000 7f81fbe00640 1 --2- 172.21.15.83:0/92242729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc159810 0x7f81fc0cc4a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009e50 tx=0x7f81ec01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed684e5600406c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.255+0000 7f81fb400640 1 -- 172.21.15.83:0/92242729 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec0a5770 con 0x7f81fc159810 2024-04-25T00:11:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.255+0000 7f81fb400640 1 -- 172.21.15.83:0/92242729 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ec0a5910 con 0x7f81fc159810 2024-04-25T00:11:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.255+0000 7f81fb400640 1 -- 172.21.15.83:0/92242729 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec0a5ce0 con 0x7f81fc159810 2024-04-25T00:11:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.255+0000 7f8203000640 1 -- 172.21.15.83:0/92242729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc159810 msgr2=0x7f81fc0cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.255+0000 7f8203000640 1 --2- 172.21.15.83:0/92242729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc159810 0x7f81fc0cc4a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009e50 tx=0x7f81ec01c9a0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.255+0000 7f8203000640 1 -- 172.21.15.83:0/92242729 shutdown_connections 2024-04-25T00:11:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.255+0000 7f8203000640 1 --2- 172.21.15.83:0/92242729 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81fc06e950 0x7f81fc15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.255+0000 7f8203000640 1 --2- 172.21.15.83:0/92242729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc159810 0x7f81fc0cc4a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.255+0000 7f8203000640 1 -- 172.21.15.83:0/92242729 >> 172.21.15.83:0/92242729 conn(0x7f81fc0b1830 msgr2=0x7f81fc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.255+0000 7f8203000640 1 -- 172.21.15.83:0/92242729 shutdown_connections 2024-04-25T00:11:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.255+0000 7f8203000640 1 -- 172.21.15.83:0/92242729 wait complete. 2024-04-25T00:11:58.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.255+0000 7f8203000640 1 Processor -- start 2024-04-25T00:11:58.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.255+0000 7f8203000640 1 -- start start 2024-04-25T00:11:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8203000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81fc06e950 0x7f81fc14f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8203000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81fc159810 0x7f81fc142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8203000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc15fe50 0x7f81fc0b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8203000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc159d70 con 0x7f81fc15fe50 2024-04-25T00:11:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8203000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc159bf0 con 0x7f81fc06e950 2024-04-25T00:11:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8203000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f81fc070da0 con 0x7f81fc159810 2024-04-25T00:11:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8201000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc15fe50 0x7f81fc0b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f81f3400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81fc159810 0x7f81fc142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8201000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc15fe50 0x7f81fc0b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39724/0 (socket says 172.21.15.83:39724) 2024-04-25T00:11:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8201000640 1 -- 172.21.15.83:0/355937337 learned_addr learned my addr 172.21.15.83:0/355937337 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f81fbe00640 1 --2- 172.21.15.83:0/355937337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81fc06e950 0x7f81fc14f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8201000640 1 -- 172.21.15.83:0/355937337 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81fc159810 msgr2=0x7f81fc142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8201000640 1 --2- 172.21.15.83:0/355937337 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81fc159810 0x7f81fc142600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8201000640 1 -- 172.21.15.83:0/355937337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81fc06e950 msgr2=0x7f81fc14f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8201000640 1 --2- 172.21.15.83:0/355937337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81fc06e950 0x7f81fc14f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8201000640 1 -- 172.21.15.83:0/355937337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc0b0930 con 0x7f81fc15fe50 2024-04-25T00:11:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8201000640 1 --2- 172.21.15.83:0/355937337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc15fe50 0x7f81fc0b03f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f81f400c430 tx=0x7f81f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f81f8c00640 1 -- 172.21.15.83:0/355937337 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4005220 con 0x7f81fc15fe50 2024-04-25T00:11:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f81f8c00640 1 -- 172.21.15.83:0/355937337 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81f40053c0 con 0x7f81fc15fe50 2024-04-25T00:11:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f81f8c00640 1 -- 172.21.15.83:0/355937337 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4007270 con 0x7f81fc15fe50 2024-04-25T00:11:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8203000640 1 -- 172.21.15.83:0/355937337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81fc0b0be0 con 0x7f81fc15fe50 2024-04-25T00:11:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8203000640 1 -- 172.21.15.83:0/355937337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81fc0b11a0 con 0x7f81fc15fe50 2024-04-25T00:11:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f8203000640 1 -- 172.21.15.83:0/355937337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81fc04f510 con 0x7f81fc15fe50 2024-04-25T00:11:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.259+0000 7f81f8c00640 1 -- 172.21.15.83:0/355937337 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f81f4019020 con 0x7f81fc15fe50 2024-04-25T00:11:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.263+0000 7f81f8c00640 1 --2- 172.21.15.83:0/355937337 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81d80678c0 0x7f81d8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.263+0000 7f81f8c00640 1 -- 172.21.15.83:0/355937337 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f81f40852f0 con 0x7f81fc15fe50 2024-04-25T00:11:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.263+0000 7f81fbe00640 1 --2- 172.21.15.83:0/355937337 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81d80678c0 0x7f81d8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.263+0000 7f81fbe00640 1 --2- 172.21.15.83:0/355937337 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81d80678c0 0x7f81d8069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0ae000 tx=0x7f81ec0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.267+0000 7f81f8c00640 1 -- 172.21.15.83:0/355937337 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f81f40553d0 con 0x7f81fc15fe50 2024-04-25T00:11:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.423+0000 7f8203000640 1 -- 172.21.15.83:0/355937337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f81fc072c90 con 0x7f81fc15fe50 2024-04-25T00:11:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.423+0000 7f81f8c00640 1 -- 172.21.15.83:0/355937337 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f81f40596d0 con 0x7f81fc15fe50 2024-04-25T00:11:58.428 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:11:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.427+0000 7f8203000640 1 -- 172.21.15.83:0/355937337 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81d80678c0 msgr2=0x7f81d8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.427+0000 7f8203000640 1 --2- 172.21.15.83:0/355937337 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81d80678c0 0x7f81d8069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0ae000 tx=0x7f81ec0a7070 comp rx=0 tx=0).stop 2024-04-25T00:11:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.427+0000 7f8203000640 1 -- 172.21.15.83:0/355937337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc15fe50 msgr2=0x7f81fc0b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.427+0000 7f8203000640 1 --2- 172.21.15.83:0/355937337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc15fe50 0x7f81fc0b03f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f81f400c430 tx=0x7f81f400c8f0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.427+0000 7f8203000640 1 -- 172.21.15.83:0/355937337 shutdown_connections 2024-04-25T00:11:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.427+0000 7f8203000640 1 --2- 172.21.15.83:0/355937337 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81d80678c0 0x7f81d8069d80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.427+0000 7f8203000640 1 --2- 172.21.15.83:0/355937337 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81fc159810 0x7f81fc142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.427+0000 7f8203000640 1 --2- 172.21.15.83:0/355937337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81fc15fe50 0x7f81fc0b03f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.427+0000 7f8203000640 1 --2- 172.21.15.83:0/355937337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81fc06e950 0x7f81fc14f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.427+0000 7f8203000640 1 -- 172.21.15.83:0/355937337 >> 172.21.15.83:0/355937337 conn(0x7f81fc0b1830 msgr2=0x7f81fc164f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.427+0000 7f8203000640 1 -- 172.21.15.83:0/355937337 shutdown_connections 2024-04-25T00:11:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.427+0000 7f8203000640 1 -- 172.21.15.83:0/355937337 wait complete. 2024-04-25T00:11:58.448 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:11:58.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.583+0000 7fed6d600640 1 Processor -- start 2024-04-25T00:11:58.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.583+0000 7fed6d600640 1 -- start start 2024-04-25T00:11:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.583+0000 7fed6d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed6806e950 0x7fed6815f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.583+0000 7fed6d600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fed6815fe50 0x7fed680ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.583+0000 7fed6d600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed680ce000 con 0x7fed68159810 2024-04-25T00:11:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.583+0000 7fed6d600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fed6806e210 con 0x7fed6815fe50 2024-04-25T00:11:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.583+0000 7fed6d600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fed681602a0 con 0x7fed6806e950 2024-04-25T00:11:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.583+0000 7fed67400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fed6815fe50 0x7fed680ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.583+0000 7fed67400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fed6815fe50 0x7fed680ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39730/0 (socket says 172.21.15.83:39730) 2024-04-25T00:11:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.583+0000 7fed67400640 1 -- 172.21.15.83:0/3563882533 learned_addr learned my addr 172.21.15.83:0/3563882533 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:58.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.587+0000 7fed5e000640 1 --2- 172.21.15.83:0/3563882533 >> v2:172.21.15.86:3300/0 conn(0x7fed6806e950 0x7fed6815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.587+0000 7fed67400640 1 -- 172.21.15.83:0/3563882533 >> v2:172.21.15.86:3300/0 conn(0x7fed6806e950 msgr2=0x7fed6815f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:58.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.587+0000 7fed67400640 1 --2- 172.21.15.83:0/3563882533 >> v2:172.21.15.86:3300/0 conn(0x7fed6806e950 0x7fed6815f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.587+0000 7fed67400640 1 -- 172.21.15.83:0/3563882533 >> v1:172.21.15.83:6789/0 conn(0x7fed68159810 legacy=0x7fed680cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.587+0000 7fed67400640 1 -- 172.21.15.83:0/3563882533 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6806eec0 con 0x7fed6815fe50 2024-04-25T00:11:58.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.587+0000 7fed66000640 1 -- 172.21.15.83:0/3563882533 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3248202220 0 0) 0x7fed680ce000 con 0x7fed68159810 2024-04-25T00:11:58.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.587+0000 7fed67400640 1 --2- 172.21.15.83:0/3563882533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6815fe50 0x7fed680ce4a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fed680709a0 tx=0x7fed6001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414dc511ac543524 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:58.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.587+0000 7fed66000640 1 -- 172.21.15.83:0/3563882533 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed600a5710 con 0x7fed6815fe50 2024-04-25T00:11:58.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.587+0000 7fed66000640 1 -- 172.21.15.83:0/3563882533 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed600a58b0 con 0x7fed6815fe50 2024-04-25T00:11:58.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.587+0000 7fed66000640 1 -- 172.21.15.83:0/3563882533 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed600a5c80 con 0x7fed6815fe50 2024-04-25T00:11:58.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.587+0000 7fed6d600640 1 -- 172.21.15.83:0/3563882533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6815fe50 msgr2=0x7fed680ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.587+0000 7fed6d600640 1 --2- 172.21.15.83:0/3563882533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6815fe50 0x7fed680ce4a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fed680709a0 tx=0x7fed6001c960 comp rx=0 tx=0).stop 2024-04-25T00:11:58.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 -- 172.21.15.83:0/3563882533 shutdown_connections 2024-04-25T00:11:58.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 --2- 172.21.15.83:0/3563882533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6815fe50 0x7fed680ce4a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 --2- 172.21.15.83:0/3563882533 >> v2:172.21.15.86:3300/0 conn(0x7fed6806e950 0x7fed6815f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 -- 172.21.15.83:0/3563882533 >> 172.21.15.83:0/3563882533 conn(0x7fed680b1830 msgr2=0x7fed680b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:58.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 -- 172.21.15.83:0/3563882533 shutdown_connections 2024-04-25T00:11:58.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 -- 172.21.15.83:0/3563882533 wait complete. 2024-04-25T00:11:58.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 Processor -- start 2024-04-25T00:11:58.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 -- start start 2024-04-25T00:11:58.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6806e950 0x7fed681558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed68159810 0x7fed680b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed6815fe50 0x7fed680b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fed68070f20 con 0x7fed6806e950 2024-04-25T00:11:58.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed68070da0 con 0x7fed68159810 2024-04-25T00:11:58.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fed68168cb0 con 0x7fed6815fe50 2024-04-25T00:11:58.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed66a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6806e950 0x7fed681558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed66a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6806e950 0x7fed681558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39740/0 (socket says 172.21.15.83:39740) 2024-04-25T00:11:58.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed66a00640 1 -- 172.21.15.83:0/4229667248 learned_addr learned my addr 172.21.15.83:0/4229667248 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:58.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed5e000640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed68159810 0x7fed680b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed67400640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed6815fe50 0x7fed680b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed66a00640 1 -- 172.21.15.83:0/4229667248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed6815fe50 msgr2=0x7fed680b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed66a00640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed6815fe50 0x7fed680b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed66a00640 1 -- 172.21.15.83:0/4229667248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed68159810 msgr2=0x7fed680b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed66a00640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed68159810 0x7fed680b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed66a00640 1 -- 172.21.15.83:0/4229667248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed68165010 con 0x7fed6806e950 2024-04-25T00:11:58.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed67400640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed6815fe50 0x7fed680b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:58.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed66a00640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6806e950 0x7fed681558a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fed5800c550 tx=0x7fed5800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:58.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed5f400640 1 -- 172.21.15.83:0/4229667248 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed580052d0 con 0x7fed6806e950 2024-04-25T00:11:58.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed5f400640 1 -- 172.21.15.83:0/4229667248 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed58005470 con 0x7fed6806e950 2024-04-25T00:11:58.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed5f400640 1 -- 172.21.15.83:0/4229667248 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58012870 con 0x7fed6806e950 2024-04-25T00:11:58.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.591+0000 7fed6d600640 1 -- 172.21.15.83:0/4229667248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed681652f0 con 0x7fed6806e950 2024-04-25T00:11:58.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.595+0000 7fed6d600640 1 -- 172.21.15.83:0/4229667248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed681658b0 con 0x7fed6806e950 2024-04-25T00:11:58.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.595+0000 7fed6d600640 1 -- 172.21.15.83:0/4229667248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed6804f510 con 0x7fed6806e950 2024-04-25T00:11:58.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.595+0000 7fed5f400640 1 -- 172.21.15.83:0/4229667248 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fed58012a10 con 0x7fed6806e950 2024-04-25T00:11:58.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.595+0000 7fed5f400640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fed440678c0 0x7fed44069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.595+0000 7fed5f400640 1 -- 172.21.15.83:0/4229667248 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fed58085db0 con 0x7fed6806e950 2024-04-25T00:11:58.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.595+0000 7fed5e000640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fed440678c0 0x7fed44069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.595+0000 7fed5e000640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fed440678c0 0x7fed44069d80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fed681566e0 tx=0x7fed50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:58.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.599+0000 7fed5f400640 1 -- 172.21.15.83:0/4229667248 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fed5808a070 con 0x7fed6806e950 2024-04-25T00:11:58.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.767+0000 7fed6d600640 1 -- 172.21.15.83:0/4229667248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fed68072c90 con 0x7fed6806e950 2024-04-25T00:11:58.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.767+0000 7fed5f400640 1 -- 172.21.15.83:0/4229667248 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fed58056460 con 0x7fed6806e950 2024-04-25T00:11:58.772 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:11:58.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.771+0000 7fed6d600640 1 -- 172.21.15.83:0/4229667248 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fed440678c0 msgr2=0x7fed44069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.771+0000 7fed6d600640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fed440678c0 0x7fed44069d80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fed681566e0 tx=0x7fed50008040 comp rx=0 tx=0).stop 2024-04-25T00:11:58.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.771+0000 7fed6d600640 1 -- 172.21.15.83:0/4229667248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6806e950 msgr2=0x7fed681558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.771+0000 7fed6d600640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6806e950 0x7fed681558a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fed5800c550 tx=0x7fed5800ca10 comp rx=0 tx=0).stop 2024-04-25T00:11:58.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.771+0000 7fed6d600640 1 -- 172.21.15.83:0/4229667248 shutdown_connections 2024-04-25T00:11:58.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.771+0000 7fed6d600640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fed440678c0 0x7fed44069d80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.771+0000 7fed6d600640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed6815fe50 0x7fed680b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.771+0000 7fed6d600640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed68159810 0x7fed680b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.771+0000 7fed6d600640 1 --2- 172.21.15.83:0/4229667248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6806e950 0x7fed681558a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.771+0000 7fed6d600640 1 -- 172.21.15.83:0/4229667248 >> 172.21.15.83:0/4229667248 conn(0x7fed680b1830 msgr2=0x7fed6815e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:58.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.771+0000 7fed6d600640 1 -- 172.21.15.83:0/4229667248 shutdown_connections 2024-04-25T00:11:58.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.771+0000 7fed6d600640 1 -- 172.21.15.83:0/4229667248 wait complete. 2024-04-25T00:11:58.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:11:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.927+0000 7f52aec00640 1 Processor -- start 2024-04-25T00:11:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.927+0000 7f52aec00640 1 -- start start 2024-04-25T00:11:58.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.927+0000 7f52aec00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52a806e630 0x7f52a80ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.927+0000 7f52aec00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52a806eac0 0x7f52a80adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.927+0000 7f52aec00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a815e690 con 0x7f52a80b3050 2024-04-25T00:11:58.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.927+0000 7f52aec00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f52a815e970 con 0x7f52a806e630 2024-04-25T00:11:58.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.927+0000 7f52aec00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f52a815ec50 con 0x7f52a806eac0 2024-04-25T00:11:58.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.927+0000 7f529f400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52a806e630 0x7f52a80ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.931+0000 7f529f400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52a806e630 0x7f52a80ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39756/0 (socket says 172.21.15.83:39756) 2024-04-25T00:11:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.931+0000 7f529f400640 1 -- 172.21.15.83:0/15037589 learned_addr learned my addr 172.21.15.83:0/15037589 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.931+0000 7f52acc00640 1 --2- 172.21.15.83:0/15037589 >> v2:172.21.15.86:3300/0 conn(0x7f52a806eac0 0x7f52a80adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.931+0000 7f529f400640 1 -- 172.21.15.83:0/15037589 >> v2:172.21.15.86:3300/0 conn(0x7f52a806eac0 msgr2=0x7f52a80adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.931+0000 7f529f400640 1 --2- 172.21.15.83:0/15037589 >> v2:172.21.15.86:3300/0 conn(0x7f52a806eac0 0x7f52a80adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.931+0000 7f529f400640 1 -- 172.21.15.83:0/15037589 >> v1:172.21.15.86:6789/0 conn(0x7f52a80b3050 legacy=0x7f52a80b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.931+0000 7f529f400640 1 -- 172.21.15.83:0/15037589 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a815ef30 con 0x7f52a806e630 2024-04-25T00:11:58.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.931+0000 7f529f400640 1 --2- 172.21.15.83:0/15037589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a806e630 0x7f52a80ad860 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f52a8162c10 tx=0x7f529001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3ec86bc0b90712 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.931+0000 7f52a7400640 1 -- 172.21.15.83:0/15037589 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52900b1070 con 0x7f52a806e630 2024-04-25T00:11:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.931+0000 7f52a7400640 1 -- 172.21.15.83:0/15037589 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52900ac630 con 0x7f52a806e630 2024-04-25T00:11:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.931+0000 7f52a7400640 1 -- 172.21.15.83:0/15037589 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52900a8050 con 0x7f52a806e630 2024-04-25T00:11:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 -- 172.21.15.83:0/15037589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a806e630 msgr2=0x7f52a80ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 --2- 172.21.15.83:0/15037589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a806e630 0x7f52a80ad860 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f52a8162c10 tx=0x7f529001c4c0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 -- 172.21.15.83:0/15037589 shutdown_connections 2024-04-25T00:11:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 --2- 172.21.15.83:0/15037589 >> v2:172.21.15.86:3300/0 conn(0x7f52a806eac0 0x7f52a80adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 --2- 172.21.15.83:0/15037589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a806e630 0x7f52a80ad860 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 -- 172.21.15.83:0/15037589 >> 172.21.15.83:0/15037589 conn(0x7f52a80b1d50 msgr2=0x7f52a80b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 -- 172.21.15.83:0/15037589 shutdown_connections 2024-04-25T00:11:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 -- 172.21.15.83:0/15037589 wait complete. 2024-04-25T00:11:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 Processor -- start 2024-04-25T00:11:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 -- start start 2024-04-25T00:11:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a806e630 0x7f52a815d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52a806eac0 0x7f52a8150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52a80b3050 0x7f52a815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52a80b4c10 con 0x7f52a806e630 2024-04-25T00:11:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52a80b4a90 con 0x7f52a806eac0 2024-04-25T00:11:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52aec00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f52a80ae6f0 con 0x7f52a80b3050 2024-04-25T00:11:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a806e630 0x7f52a815d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a806e630 0x7f52a815d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39768/0 (socket says 172.21.15.83:39768) 2024-04-25T00:11:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52a7e00640 1 -- 172.21.15.83:0/3180280677 learned_addr learned my addr 172.21.15.83:0/3180280677 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f529f400640 1 --2- 172.21.15.83:0/3180280677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52a806eac0 0x7f52a8150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52acc00640 1 --2- 172.21.15.83:0/3180280677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52a80b3050 0x7f52a815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52a7e00640 1 -- 172.21.15.83:0/3180280677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52a80b3050 msgr2=0x7f52a815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52a7e00640 1 --2- 172.21.15.83:0/3180280677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52a80b3050 0x7f52a815a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52a7e00640 1 -- 172.21.15.83:0/3180280677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52a806eac0 msgr2=0x7f52a8150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52a7e00640 1 --2- 172.21.15.83:0/3180280677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52a806eac0 0x7f52a8150940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52a7e00640 1 -- 172.21.15.83:0/3180280677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a80725d0 con 0x7f52a806e630 2024-04-25T00:11:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.935+0000 7f52a7e00640 1 --2- 172.21.15.83:0/3180280677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a806e630 0x7f52a815d750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f529800c790 tx=0x7f529800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.939+0000 7f52a4c00640 1 -- 172.21.15.83:0/3180280677 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298016020 con 0x7f52a806e630 2024-04-25T00:11:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.939+0000 7f52a4c00640 1 -- 172.21.15.83:0/3180280677 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5298006d60 con 0x7f52a806e630 2024-04-25T00:11:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.939+0000 7f52a4c00640 1 -- 172.21.15.83:0/3180280677 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298003bc0 con 0x7f52a806e630 2024-04-25T00:11:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.939+0000 7f52aec00640 1 -- 172.21.15.83:0/3180280677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a815be60 con 0x7f52a806e630 2024-04-25T00:11:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.939+0000 7f52aec00640 1 -- 172.21.15.83:0/3180280677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a8156fe0 con 0x7f52a806e630 2024-04-25T00:11:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.939+0000 7f52aec00640 1 -- 172.21.15.83:0/3180280677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52a8068230 con 0x7f52a806e630 2024-04-25T00:11:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.939+0000 7f52a4c00640 1 -- 172.21.15.83:0/3180280677 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f529800a850 con 0x7f52a806e630 2024-04-25T00:11:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.939+0000 7f52a4c00640 1 --2- 172.21.15.83:0/3180280677 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f52840678f0 0x7f5284069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.939+0000 7f52a4c00640 1 -- 172.21.15.83:0/3180280677 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5298085bd0 con 0x7f52a806e630 2024-04-25T00:11:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.939+0000 7f529f400640 1 --2- 172.21.15.83:0/3180280677 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f52840678f0 0x7f5284069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.939+0000 7f529f400640 1 --2- 172.21.15.83:0/3180280677 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f52840678f0 0x7f5284069db0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f52900ae000 tx=0x7f5290008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:58.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:58.943+0000 7f52a4c00640 1 -- 172.21.15.83:0/3180280677 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5298020020 con 0x7f52a806e630 2024-04-25T00:11:59.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.115+0000 7f52aec00640 1 -- 172.21.15.83:0/3180280677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f52a806fba0 con 0x7f52a806e630 2024-04-25T00:11:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.115+0000 7f52a4c00640 1 -- 172.21.15.83:0/3180280677 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f5298055d50 con 0x7f52a806e630 2024-04-25T00:11:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:11:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.115+0000 7f52aec00640 1 -- 172.21.15.83:0/3180280677 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f52840678f0 msgr2=0x7f5284069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.115+0000 7f52aec00640 1 --2- 172.21.15.83:0/3180280677 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f52840678f0 0x7f5284069db0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f52900ae000 tx=0x7f5290008040 comp rx=0 tx=0).stop 2024-04-25T00:11:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.115+0000 7f52aec00640 1 -- 172.21.15.83:0/3180280677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a806e630 msgr2=0x7f52a815d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.115+0000 7f52aec00640 1 --2- 172.21.15.83:0/3180280677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a806e630 0x7f52a815d750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f529800c790 tx=0x7f529800cc50 comp rx=0 tx=0).stop 2024-04-25T00:11:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.115+0000 7f52aec00640 1 -- 172.21.15.83:0/3180280677 shutdown_connections 2024-04-25T00:11:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.115+0000 7f52aec00640 1 --2- 172.21.15.83:0/3180280677 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f52840678f0 0x7f5284069db0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.115+0000 7f52aec00640 1 --2- 172.21.15.83:0/3180280677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52a80b3050 0x7f52a815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.115+0000 7f52aec00640 1 --2- 172.21.15.83:0/3180280677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52a806eac0 0x7f52a8150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.115+0000 7f52aec00640 1 --2- 172.21.15.83:0/3180280677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a806e630 0x7f52a815d750 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.115+0000 7f52aec00640 1 -- 172.21.15.83:0/3180280677 >> 172.21.15.83:0/3180280677 conn(0x7f52a80b1d50 msgr2=0x7f52a80b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.115+0000 7f52aec00640 1 -- 172.21.15.83:0/3180280677 shutdown_connections 2024-04-25T00:11:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.115+0000 7f52aec00640 1 -- 172.21.15.83:0/3180280677 wait complete. 2024-04-25T00:11:59.139 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T00:11:59.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:59.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T00:11:59.173 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:11:59.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:59.173 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:11:59.238 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:11:59.283 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:59.301 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:11:59.301 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:59.361 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:11:59.361 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:59.419 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:59.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.595+0000 7f2d2e000640 1 Processor -- start 2024-04-25T00:11:59.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.595+0000 7f2d2e000640 1 -- start start 2024-04-25T00:11:59.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.595+0000 7f2d2e000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2d28164cb0 0x7f2d281670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:59.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.595+0000 7f2d2e000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d280ccc60 con 0x7f2d281599f0 2024-04-25T00:11:59.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.595+0000 7f2d2e000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2806dba0 con 0x7f2d280cc510 2024-04-25T00:11:59.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.595+0000 7f2d2e000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2d28159dd0 con 0x7f2d28164cb0 2024-04-25T00:11:59.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.595+0000 7f2d27400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2d281599f0 0x7f2d2815c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56932/0 (socket says 172.21.15.83:56932) 2024-04-25T00:11:59.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.595+0000 7f2d27400640 1 -- 172.21.15.83:0/3888600966 learned_addr learned my addr 172.21.15.83:0/3888600966 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:11:59.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.595+0000 7f2d27e00640 1 --2- 172.21.15.83:0/3888600966 >> v2:172.21.15.83:3300/0 conn(0x7f2d28164cb0 0x7f2d281670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:59.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.595+0000 7f2d27e00640 1 -- 172.21.15.83:0/3888600966 >> v1:172.21.15.86:6789/0 conn(0x7f2d280cc510 legacy=0x7f2d2815ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.595+0000 7f2d27e00640 1 -- 172.21.15.83:0/3888600966 >> v1:172.21.15.83:6789/0 conn(0x7f2d281599f0 legacy=0x7f2d2815c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.595+0000 7f2d27e00640 1 -- 172.21.15.83:0/3888600966 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2806de80 con 0x7f2d28164cb0 2024-04-25T00:11:59.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.595+0000 7f2d26000640 1 -- 172.21.15.83:0/3888600966 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3880078143 0 0) 0x7f2d180014a0 con 0x7f2d281599f0 2024-04-25T00:11:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.599+0000 7f2d27e00640 1 --2- 172.21.15.83:0/3888600966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d28164cb0 0x7f2d281670a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2d200097f0 tx=0x7f2d2001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2a643a158ebd35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.599+0000 7f2d26000640 1 -- 172.21.15.83:0/3888600966 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d200b0070 con 0x7f2d28164cb0 2024-04-25T00:11:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.599+0000 7f2d26000640 1 -- 172.21.15.83:0/3888600966 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d200ab600 con 0x7f2d28164cb0 2024-04-25T00:11:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.599+0000 7f2d26000640 1 -- 172.21.15.83:0/3888600966 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d200ab9d0 con 0x7f2d28164cb0 2024-04-25T00:11:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.599+0000 7f2d2e000640 1 -- 172.21.15.83:0/3888600966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d28164cb0 msgr2=0x7f2d281670a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.599+0000 7f2d2e000640 1 --2- 172.21.15.83:0/3888600966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d28164cb0 0x7f2d281670a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2d200097f0 tx=0x7f2d2001c410 comp rx=0 tx=0).stop 2024-04-25T00:11:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.599+0000 7f2d2e000640 1 -- 172.21.15.83:0/3888600966 shutdown_connections 2024-04-25T00:11:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.599+0000 7f2d2e000640 1 --2- 172.21.15.83:0/3888600966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d28164cb0 0x7f2d281670a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.599+0000 7f2d2e000640 1 -- 172.21.15.83:0/3888600966 >> 172.21.15.83:0/3888600966 conn(0x7f2d280b1830 msgr2=0x7f2d280b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.599+0000 7f2d2e000640 1 -- 172.21.15.83:0/3888600966 shutdown_connections 2024-04-25T00:11:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.599+0000 7f2d2e000640 1 -- 172.21.15.83:0/3888600966 wait complete. 2024-04-25T00:11:59.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d2e000640 1 Processor -- start 2024-04-25T00:11:59.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d2e000640 1 -- start start 2024-04-25T00:11:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d2e000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d280cc510 0x7f2d280b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d2e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d281599f0 0x7f2d28152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d2e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d28164cb0 0x7f2d2814bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d2e000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d280ce240 con 0x7f2d281599f0 2024-04-25T00:11:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d2e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d280ce0c0 con 0x7f2d28164cb0 2024-04-25T00:11:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d2e000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2d280cf6f0 con 0x7f2d280cc510 2024-04-25T00:11:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d281599f0 0x7f2d28152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d281599f0 0x7f2d28152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39776/0 (socket says 172.21.15.83:39776) 2024-04-25T00:11:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d26a00640 1 -- 172.21.15.83:0/1419284242 learned_addr learned my addr 172.21.15.83:0/1419284242 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d27e00640 1 --2- 172.21.15.83:0/1419284242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d28164cb0 0x7f2d2814bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d27400640 1 --2- 172.21.15.83:0/1419284242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d280cc510 0x7f2d280b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d26a00640 1 -- 172.21.15.83:0/1419284242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d280cc510 msgr2=0x7f2d280b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d26a00640 1 --2- 172.21.15.83:0/1419284242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d280cc510 0x7f2d280b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d26a00640 1 -- 172.21.15.83:0/1419284242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d28164cb0 msgr2=0x7f2d2814bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d26a00640 1 --2- 172.21.15.83:0/1419284242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d28164cb0 0x7f2d2814bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d26a00640 1 -- 172.21.15.83:0/1419284242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2814c180 con 0x7f2d281599f0 2024-04-25T00:11:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d26a00640 1 --2- 172.21.15.83:0/1419284242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d281599f0 0x7f2d28152570 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c500 tx=0x7f2d1400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d1f400640 1 -- 172.21.15.83:0/1419284242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14016020 con 0x7f2d281599f0 2024-04-25T00:11:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d1f400640 1 -- 172.21.15.83:0/1419284242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d14005220 con 0x7f2d281599f0 2024-04-25T00:11:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d2e000640 1 -- 172.21.15.83:0/1419284242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d28152ca0 con 0x7f2d281599f0 2024-04-25T00:11:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d1f400640 1 -- 172.21.15.83:0/1419284242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14007030 con 0x7f2d281599f0 2024-04-25T00:11:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.603+0000 7f2d2e000640 1 -- 172.21.15.83:0/1419284242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d28153260 con 0x7f2d281599f0 2024-04-25T00:11:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.607+0000 7f2d2e000640 1 -- 172.21.15.83:0/1419284242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d280737f0 con 0x7f2d281599f0 2024-04-25T00:11:59.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.607+0000 7f2d1f400640 1 -- 172.21.15.83:0/1419284242 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2d140071f0 con 0x7f2d281599f0 2024-04-25T00:11:59.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.607+0000 7f2d1f400640 1 --2- 172.21.15.83:0/1419284242 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2d080678f0 0x7f2d08069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:59.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.607+0000 7f2d1f400640 1 -- 172.21.15.83:0/1419284242 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2d14085870 con 0x7f2d281599f0 2024-04-25T00:11:59.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.607+0000 7f2d27400640 1 --2- 172.21.15.83:0/1419284242 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2d080678f0 0x7f2d08069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:59.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.607+0000 7f2d27400640 1 --2- 172.21.15.83:0/1419284242 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2d080678f0 0x7f2d08069db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2d180012f0 tx=0x7f2d18001260 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.611+0000 7f2d1f400640 1 -- 172.21.15.83:0/1419284242 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2d14020020 con 0x7f2d281599f0 2024-04-25T00:11:59.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.767+0000 7f2d2e000640 1 -- 172.21.15.83:0/1419284242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d2804f5f0 con 0x7f2d281599f0 2024-04-25T00:11:59.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.767+0000 7f2d1f400640 1 -- 172.21.15.83:0/1419284242 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18820 (secure 0 0 0) 0x7f2d2814c180 con 0x7f2d281599f0 2024-04-25T00:11:59.773 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:11:59.774 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:48.235809+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:59.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.771+0000 7f2d2e000640 1 -- 172.21.15.83:0/1419284242 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2d080678f0 msgr2=0x7f2d08069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.771+0000 7f2d2e000640 1 --2- 172.21.15.83:0/1419284242 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2d080678f0 0x7f2d08069db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2d180012f0 tx=0x7f2d18001260 comp rx=0 tx=0).stop 2024-04-25T00:11:59.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.771+0000 7f2d2e000640 1 -- 172.21.15.83:0/1419284242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d281599f0 msgr2=0x7f2d28152570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.771+0000 7f2d2e000640 1 --2- 172.21.15.83:0/1419284242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d281599f0 0x7f2d28152570 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c500 tx=0x7f2d1400c9c0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.771+0000 7f2d2e000640 1 -- 172.21.15.83:0/1419284242 shutdown_connections 2024-04-25T00:11:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.771+0000 7f2d2e000640 1 --2- 172.21.15.83:0/1419284242 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2d080678f0 0x7f2d08069db0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.771+0000 7f2d2e000640 1 --2- 172.21.15.83:0/1419284242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d28164cb0 0x7f2d2814bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.771+0000 7f2d2e000640 1 --2- 172.21.15.83:0/1419284242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d281599f0 0x7f2d28152570 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.771+0000 7f2d2e000640 1 --2- 172.21.15.83:0/1419284242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d280cc510 0x7f2d280b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.771+0000 7f2d2e000640 1 -- 172.21.15.83:0/1419284242 >> 172.21.15.83:0/1419284242 conn(0x7f2d280b1830 msgr2=0x7f2d2806e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.771+0000 7f2d2e000640 1 -- 172.21.15.83:0/1419284242 shutdown_connections 2024-04-25T00:11:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.771+0000 7f2d2e000640 1 -- 172.21.15.83:0/1419284242 wait complete. 2024-04-25T00:11:59.794 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:11:04.621429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:11:05.189716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:11:08.902215+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:11:59.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:11:59.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1b3000640 1 Processor -- start 2024-04-25T00:11:59.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1b3000640 1 -- start start 2024-04-25T00:11:59.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1b3000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc1ac15de90 0x7fc1ac071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:59.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1b3000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc1ac06e840 0x7fc1ac071c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:59.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1b3000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1ac06dcf0 con 0x7fc1ac15e270 2024-04-25T00:11:59.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1b3000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc1ac0b4460 con 0x7fc1ac06e840 2024-04-25T00:11:59.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1b3000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc1ac0b2ce0 con 0x7fc1ac15de90 2024-04-25T00:11:59.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1b1000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc1ac06e840 0x7fc1ac071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:59.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1b1000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc1ac06e840 0x7fc1ac071c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39782/0 (socket says 172.21.15.83:39782) 2024-04-25T00:11:59.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1b1000640 1 -- 172.21.15.83:0/3627995529 learned_addr learned my addr 172.21.15.83:0/3627995529 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:59.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1abe00640 1 --2- 172.21.15.83:0/3627995529 >> v2:172.21.15.86:3300/0 conn(0x7fc1ac15de90 0x7fc1ac071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:59.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1b1000640 1 -- 172.21.15.83:0/3627995529 >> v2:172.21.15.86:3300/0 conn(0x7fc1ac15de90 msgr2=0x7fc1ac071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:59.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1b1000640 1 --2- 172.21.15.83:0/3627995529 >> v2:172.21.15.86:3300/0 conn(0x7fc1ac15de90 0x7fc1ac071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1b1000640 1 -- 172.21.15.83:0/3627995529 >> v1:172.21.15.86:6789/0 conn(0x7fc1ac15e270 legacy=0x7fc1ac161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.931+0000 7fc1b1000640 1 -- 172.21.15.83:0/3627995529 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac06dfd0 con 0x7fc1ac06e840 2024-04-25T00:11:59.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.935+0000 7fc1b1000640 1 --2- 172.21.15.83:0/3627995529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac06e840 0x7fc1ac071c80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0709f0 tx=0x7fc1a401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68db425f8259c1e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:59.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.935+0000 7fc1ab400640 1 -- 172.21.15.83:0/3627995529 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a40a5710 con 0x7fc1ac06e840 2024-04-25T00:11:59.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.935+0000 7fc1ab400640 1 -- 172.21.15.83:0/3627995529 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1a40a58b0 con 0x7fc1ac06e840 2024-04-25T00:11:59.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.935+0000 7fc1ab400640 1 -- 172.21.15.83:0/3627995529 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a40a5c80 con 0x7fc1ac06e840 2024-04-25T00:11:59.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.935+0000 7fc1b3000640 1 -- 172.21.15.83:0/3627995529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac06e840 msgr2=0x7fc1ac071c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.935+0000 7fc1b3000640 1 --2- 172.21.15.83:0/3627995529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac06e840 0x7fc1ac071c80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0709f0 tx=0x7fc1a401c960 comp rx=0 tx=0).stop 2024-04-25T00:11:59.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.935+0000 7fc1b3000640 1 -- 172.21.15.83:0/3627995529 shutdown_connections 2024-04-25T00:11:59.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.935+0000 7fc1b3000640 1 --2- 172.21.15.83:0/3627995529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac06e840 0x7fc1ac071c80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.935+0000 7fc1b3000640 1 --2- 172.21.15.83:0/3627995529 >> v2:172.21.15.86:3300/0 conn(0x7fc1ac15de90 0x7fc1ac071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.935+0000 7fc1b3000640 1 -- 172.21.15.83:0/3627995529 >> 172.21.15.83:0/3627995529 conn(0x7fc1ac0b1830 msgr2=0x7fc1ac0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:59.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.935+0000 7fc1b3000640 1 -- 172.21.15.83:0/3627995529 shutdown_connections 2024-04-25T00:11:59.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.935+0000 7fc1b3000640 1 -- 172.21.15.83:0/3627995529 wait complete. 2024-04-25T00:11:59.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1b3000640 1 Processor -- start 2024-04-25T00:11:59.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1b3000640 1 -- start start 2024-04-25T00:11:59.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1b3000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac06e840 0x7fc1ac1460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:59.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1b3000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac15de90 0x7fc1ac15e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:59.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1b3000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc1ac15e270 0x7fc1ac15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:59.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1b3000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac16ad60 con 0x7fc1ac06e840 2024-04-25T00:11:59.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1b3000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac0ad860 con 0x7fc1ac15de90 2024-04-25T00:11:59.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1b3000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ac16aee0 con 0x7fc1ac15e270 2024-04-25T00:11:59.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac06e840 0x7fc1ac1460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:59.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac06e840 0x7fc1ac1460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39790/0 (socket says 172.21.15.83:39790) 2024-04-25T00:11:59.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1abe00640 1 -- 172.21.15.83:0/2268025764 learned_addr learned my addr 172.21.15.83:0/2268025764 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:11:59.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1a3400640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac15de90 0x7fc1ac15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:59.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1b1000640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc1ac15e270 0x7fc1ac15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:59.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1abe00640 1 -- 172.21.15.83:0/2268025764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc1ac15e270 msgr2=0x7fc1ac15ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1abe00640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc1ac15e270 0x7fc1ac15ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1abe00640 1 -- 172.21.15.83:0/2268025764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac15de90 msgr2=0x7fc1ac15e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1abe00640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac15de90 0x7fc1ac15e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1abe00640 1 -- 172.21.15.83:0/2268025764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac146810 con 0x7fc1ac06e840 2024-04-25T00:11:59.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1b1000640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc1ac15e270 0x7fc1ac15ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:59.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1a3400640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac15de90 0x7fc1ac15e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:59.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1abe00640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac06e840 0x7fc1ac1460a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00e9e0 tx=0x7fc19c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:59.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1a8c00640 1 -- 172.21.15.83:0/2268025764 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c017070 con 0x7fc1ac06e840 2024-04-25T00:11:59.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1a8c00640 1 -- 172.21.15.83:0/2268025764 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc19c00c9d0 con 0x7fc1ac06e840 2024-04-25T00:11:59.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1b3000640 1 -- 172.21.15.83:0/2268025764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ac146af0 con 0x7fc1ac06e840 2024-04-25T00:11:59.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1a8c00640 1 -- 172.21.15.83:0/2268025764 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00cda0 con 0x7fc1ac06e840 2024-04-25T00:11:59.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1b3000640 1 -- 172.21.15.83:0/2268025764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ac1470b0 con 0x7fc1ac06e840 2024-04-25T00:11:59.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.939+0000 7fc1b3000640 1 -- 172.21.15.83:0/2268025764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ac04f510 con 0x7fc1ac06e840 2024-04-25T00:11:59.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.943+0000 7fc1a8c00640 1 -- 172.21.15.83:0/2268025764 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc19c01a070 con 0x7fc1ac06e840 2024-04-25T00:11:59.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.943+0000 7fc1a8c00640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc1880678c0 0x7fc188069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:59.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.943+0000 7fc1a8c00640 1 -- 172.21.15.83:0/2268025764 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc19c086590 con 0x7fc1ac06e840 2024-04-25T00:11:59.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.943+0000 7fc1a3400640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc1880678c0 0x7fc188069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:59.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.943+0000 7fc1a3400640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc1880678c0 0x7fc188069d80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac15a1b0 tx=0x7fc194009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:59.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:11:59.947+0000 7fc1a8c00640 1 -- 172.21.15.83:0/2268025764 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc19c056c40 con 0x7fc1ac06e840 2024-04-25T00:12:00.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.103+0000 7fc1b3000640 1 -- 172.21.15.83:0/2268025764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc1ac162020 con 0x7fc1ac06e840 2024-04-25T00:12:00.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.103+0000 7fc1a8c00640 1 -- 172.21.15.83:0/2268025764 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fc19c056540 con 0x7fc1ac06e840 2024-04-25T00:12:00.108 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:12:00.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.107+0000 7fc1b3000640 1 -- 172.21.15.83:0/2268025764 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc1880678c0 msgr2=0x7fc188069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.107+0000 7fc1b3000640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc1880678c0 0x7fc188069d80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac15a1b0 tx=0x7fc194009040 comp rx=0 tx=0).stop 2024-04-25T00:12:00.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.107+0000 7fc1b3000640 1 -- 172.21.15.83:0/2268025764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac06e840 msgr2=0x7fc1ac1460a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.107+0000 7fc1b3000640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac06e840 0x7fc1ac1460a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00e9e0 tx=0x7fc19c00eea0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.107+0000 7fc1b3000640 1 -- 172.21.15.83:0/2268025764 shutdown_connections 2024-04-25T00:12:00.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.107+0000 7fc1b3000640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc1880678c0 0x7fc188069d80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.107+0000 7fc1b3000640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc1ac15e270 0x7fc1ac15ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.107+0000 7fc1b3000640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac15de90 0x7fc1ac15e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.107+0000 7fc1b3000640 1 --2- 172.21.15.83:0/2268025764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac06e840 0x7fc1ac1460a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.107+0000 7fc1b3000640 1 -- 172.21.15.83:0/2268025764 >> 172.21.15.83:0/2268025764 conn(0x7fc1ac0b1830 msgr2=0x7fc1ac1514b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:00.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.107+0000 7fc1b3000640 1 -- 172.21.15.83:0/2268025764 shutdown_connections 2024-04-25T00:12:00.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.107+0000 7fc1b3000640 1 -- 172.21.15.83:0/2268025764 wait complete. 2024-04-25T00:12:00.129 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:12:00.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f2476a00640 1 Processor -- start 2024-04-25T00:12:00.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f2476a00640 1 -- start start 2024-04-25T00:12:00.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f2476a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f24701551e0 0x7f247015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f2476a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f24701555c0 0x7f2470071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f2476a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f247015c560 con 0x7f247006e4c0 2024-04-25T00:12:00.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f2476a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2470072460 con 0x7f24701555c0 2024-04-25T00:12:00.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f2476a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f247015c7d0 con 0x7f24701551e0 2024-04-25T00:12:00.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f2467400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f24701555c0 0x7f2470071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f2467400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f24701555c0 0x7f2470071b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39800/0 (socket says 172.21.15.83:39800) 2024-04-25T00:12:00.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f2467400640 1 -- 172.21.15.83:0/170499102 learned_addr learned my addr 172.21.15.83:0/170499102 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:00.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f246fe00640 1 --2- 172.21.15.83:0/170499102 >> v2:172.21.15.86:3300/0 conn(0x7f24701551e0 0x7f247015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f2467400640 1 -- 172.21.15.83:0/170499102 >> v2:172.21.15.86:3300/0 conn(0x7f24701551e0 msgr2=0x7f247015bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:00.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f2467400640 1 --2- 172.21.15.83:0/170499102 >> v2:172.21.15.86:3300/0 conn(0x7f24701551e0 0x7f247015bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f2467400640 1 -- 172.21.15.83:0/170499102 >> v1:172.21.15.86:6789/0 conn(0x7f247006e4c0 legacy=0x7f247015e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.263+0000 7f2467400640 1 -- 172.21.15.83:0/170499102 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247015cab0 con 0x7f24701555c0 2024-04-25T00:12:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.267+0000 7f2467400640 1 --2- 172.21.15.83:0/170499102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701555c0 0x7f2470071b70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2458009870 tx=0x7f245801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a4988bdba6d60b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.267+0000 7f246f400640 1 -- 172.21.15.83:0/170499102 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24580ab640 con 0x7f24701555c0 2024-04-25T00:12:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.267+0000 7f246f400640 1 -- 172.21.15.83:0/170499102 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24580b0070 con 0x7f24701555c0 2024-04-25T00:12:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.267+0000 7f246f400640 1 -- 172.21.15.83:0/170499102 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24580aba00 con 0x7f24701555c0 2024-04-25T00:12:00.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.267+0000 7f2476a00640 1 -- 172.21.15.83:0/170499102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701555c0 msgr2=0x7f2470071b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.267+0000 7f2476a00640 1 --2- 172.21.15.83:0/170499102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701555c0 0x7f2470071b70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2458009870 tx=0x7f245801c520 comp rx=0 tx=0).stop 2024-04-25T00:12:00.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.267+0000 7f2476a00640 1 -- 172.21.15.83:0/170499102 shutdown_connections 2024-04-25T00:12:00.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.267+0000 7f2476a00640 1 --2- 172.21.15.83:0/170499102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701555c0 0x7f2470071b70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.267+0000 7f2476a00640 1 --2- 172.21.15.83:0/170499102 >> v2:172.21.15.86:3300/0 conn(0x7f24701551e0 0x7f247015bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.267+0000 7f2476a00640 1 -- 172.21.15.83:0/170499102 >> 172.21.15.83:0/170499102 conn(0x7f24700b1830 msgr2=0x7f24700b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:00.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.267+0000 7f2476a00640 1 -- 172.21.15.83:0/170499102 shutdown_connections 2024-04-25T00:12:00.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.267+0000 7f2476a00640 1 -- 172.21.15.83:0/170499102 wait complete. 2024-04-25T00:12:00.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2476a00640 1 Processor -- start 2024-04-25T00:12:00.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2476a00640 1 -- start start 2024-04-25T00:12:00.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2476a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f247006e4c0 0x7f247015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2476a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701551e0 0x7f24700cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2476a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24701555c0 0x7f24700cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2476a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2470165e00 con 0x7f24701551e0 2024-04-25T00:12:00.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2476a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2470165c80 con 0x7f24701555c0 2024-04-25T00:12:00.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2476a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2470165f80 con 0x7f247006e4c0 2024-04-25T00:12:00.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2467400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701551e0 0x7f24700cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2467400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701551e0 0x7f24700cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39812/0 (socket says 172.21.15.83:39812) 2024-04-25T00:12:00.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2467400640 1 -- 172.21.15.83:0/2410866563 learned_addr learned my addr 172.21.15.83:0/2410866563 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f246fe00640 1 --2- 172.21.15.83:0/2410866563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f247006e4c0 0x7f247015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2474a00640 1 --2- 172.21.15.83:0/2410866563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24701555c0 0x7f24700cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2467400640 1 -- 172.21.15.83:0/2410866563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f247006e4c0 msgr2=0x7f247015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2467400640 1 --2- 172.21.15.83:0/2410866563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f247006e4c0 0x7f247015a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2467400640 1 -- 172.21.15.83:0/2410866563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24701555c0 msgr2=0x7f24700cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2467400640 1 --2- 172.21.15.83:0/2410866563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24701555c0 0x7f24700cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2467400640 1 -- 172.21.15.83:0/2410866563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2470071bc0 con 0x7f24701551e0 2024-04-25T00:12:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2467400640 1 --2- 172.21.15.83:0/2410866563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701551e0 0x7f24700cd000 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2458019130 tx=0x7f245801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f246cc00640 1 -- 172.21.15.83:0/2410866563 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24580be3f0 con 0x7f24701551e0 2024-04-25T00:12:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f246cc00640 1 -- 172.21.15.83:0/2410866563 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24580be590 con 0x7f24701551e0 2024-04-25T00:12:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f246cc00640 1 -- 172.21.15.83:0/2410866563 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24580be980 con 0x7f24701551e0 2024-04-25T00:12:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2476a00640 1 -- 172.21.15.83:0/2410866563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24700cdf10 con 0x7f24701551e0 2024-04-25T00:12:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2476a00640 1 -- 172.21.15.83:0/2410866563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24701344f0 con 0x7f24701551e0 2024-04-25T00:12:00.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.271+0000 7f2476a00640 1 -- 172.21.15.83:0/2410866563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f247004f510 con 0x7f24701551e0 2024-04-25T00:12:00.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.275+0000 7f246cc00640 1 -- 172.21.15.83:0/2410866563 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f24580b43f0 con 0x7f24701551e0 2024-04-25T00:12:00.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.275+0000 7f246cc00640 1 --2- 172.21.15.83:0/2410866563 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f244c0678c0 0x7f244c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.275+0000 7f246cc00640 1 -- 172.21.15.83:0/2410866563 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2458120110 con 0x7f24701551e0 2024-04-25T00:12:00.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.275+0000 7f246fe00640 1 --2- 172.21.15.83:0/2410866563 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f244c0678c0 0x7f244c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.275+0000 7f246fe00640 1 --2- 172.21.15.83:0/2410866563 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f244c0678c0 0x7f244c069d80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2470134320 tx=0x7f2460009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:00.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.279+0000 7f246cc00640 1 -- 172.21.15.83:0/2410866563 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2458016110 con 0x7f24701551e0 2024-04-25T00:12:00.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.439+0000 7f2476a00640 1 -- 172.21.15.83:0/2410866563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f247006ce30 con 0x7f24701551e0 2024-04-25T00:12:00.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.439+0000 7f246cc00640 1 -- 172.21.15.83:0/2410866563 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f24580f01e0 con 0x7f24701551e0 2024-04-25T00:12:00.443 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:12:00.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.439+0000 7f2476a00640 1 -- 172.21.15.83:0/2410866563 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f244c0678c0 msgr2=0x7f244c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.439+0000 7f2476a00640 1 --2- 172.21.15.83:0/2410866563 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f244c0678c0 0x7f244c069d80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2470134320 tx=0x7f2460009190 comp rx=0 tx=0).stop 2024-04-25T00:12:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.439+0000 7f2476a00640 1 -- 172.21.15.83:0/2410866563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701551e0 msgr2=0x7f24700cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.439+0000 7f2476a00640 1 --2- 172.21.15.83:0/2410866563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701551e0 0x7f24700cd000 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2458019130 tx=0x7f245801ad60 comp rx=0 tx=0).stop 2024-04-25T00:12:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.439+0000 7f2476a00640 1 -- 172.21.15.83:0/2410866563 shutdown_connections 2024-04-25T00:12:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.439+0000 7f2476a00640 1 --2- 172.21.15.83:0/2410866563 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f244c0678c0 0x7f244c069d80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.439+0000 7f2476a00640 1 --2- 172.21.15.83:0/2410866563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24701555c0 0x7f24700cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.439+0000 7f2476a00640 1 --2- 172.21.15.83:0/2410866563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701551e0 0x7f24700cd000 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.439+0000 7f2476a00640 1 --2- 172.21.15.83:0/2410866563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f247006e4c0 0x7f247015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.439+0000 7f2476a00640 1 -- 172.21.15.83:0/2410866563 >> 172.21.15.83:0/2410866563 conn(0x7f24700b1830 msgr2=0x7f24700713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.439+0000 7f2476a00640 1 -- 172.21.15.83:0/2410866563 shutdown_connections 2024-04-25T00:12:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.439+0000 7f2476a00640 1 -- 172.21.15.83:0/2410866563 wait complete. 2024-04-25T00:12:00.464 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:12:00.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb957600640 1 Processor -- start 2024-04-25T00:12:00.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb957600640 1 -- start start 2024-04-25T00:12:00.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb957600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb950160020 0x7fb950166ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb957600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9500ce000 con 0x7fb95006e950 2024-04-25T00:12:00.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb957600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb95006eec0 con 0x7fb950159810 2024-04-25T00:12:00.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb957600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb95006e210 con 0x7fb950160020 2024-04-25T00:12:00.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb947e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb95006e950 0x7fb95015f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56942/0 (socket says 172.21.15.83:56942) 2024-04-25T00:12:00.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb947e00640 1 -- 172.21.15.83:0/684655441 learned_addr learned my addr 172.21.15.83:0/684655441 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:00.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb955600640 1 --2- 172.21.15.83:0/684655441 >> v2:172.21.15.86:3300/0 conn(0x7fb950160020 0x7fb950166ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb94fe00640 1 -- 172.21.15.83:0/684655441 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3170326966 0 0) 0x7fb9500ce000 con 0x7fb95006e950 2024-04-25T00:12:00.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb94fe00640 1 -- 172.21.15.83:0/684655441 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb934003530 con 0x7fb95006e950 2024-04-25T00:12:00.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb94fe00640 1 -- 172.21.15.83:0/684655441 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3761470750 0 0) 0x7fb934003530 con 0x7fb95006e950 2024-04-25T00:12:00.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb955600640 1 -- 172.21.15.83:0/684655441 >> v1:172.21.15.86:6789/0 conn(0x7fb950159810 legacy=0x7fb9500cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb955600640 1 -- 172.21.15.83:0/684655441 >> v1:172.21.15.83:6789/0 conn(0x7fb95006e950 legacy=0x7fb95015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.599+0000 7fb955600640 1 -- 172.21.15.83:0/684655441 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95006dc50 con 0x7fb950160020 2024-04-25T00:12:00.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.603+0000 7fb955600640 1 --2- 172.21.15.83:0/684655441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb950160020 0x7fb950166ca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb948009d90 tx=0x7fb94801c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c960e0f649224829 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:00.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.603+0000 7fb94fe00640 1 -- 172.21.15.83:0/684655441 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9480b2070 con 0x7fb950160020 2024-04-25T00:12:00.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.603+0000 7fb94fe00640 1 -- 172.21.15.83:0/684655441 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9480a5550 con 0x7fb950160020 2024-04-25T00:12:00.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.603+0000 7fb94fe00640 1 -- 172.21.15.83:0/684655441 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938001680 con 0x7fb950160020 2024-04-25T00:12:00.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.603+0000 7fb957600640 1 -- 172.21.15.83:0/684655441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb950160020 msgr2=0x7fb950166ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.603+0000 7fb957600640 1 --2- 172.21.15.83:0/684655441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb950160020 0x7fb950166ca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb948009d90 tx=0x7fb94801c800 comp rx=0 tx=0).stop 2024-04-25T00:12:00.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.603+0000 7fb957600640 1 -- 172.21.15.83:0/684655441 shutdown_connections 2024-04-25T00:12:00.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.603+0000 7fb957600640 1 --2- 172.21.15.83:0/684655441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb950160020 0x7fb950166ca0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.603+0000 7fb957600640 1 -- 172.21.15.83:0/684655441 >> 172.21.15.83:0/684655441 conn(0x7fb9500b1830 msgr2=0x7fb9500b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:00.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.603+0000 7fb957600640 1 -- 172.21.15.83:0/684655441 shutdown_connections 2024-04-25T00:12:00.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.603+0000 7fb957600640 1 -- 172.21.15.83:0/684655441 wait complete. 2024-04-25T00:12:00.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb957600640 1 Processor -- start 2024-04-25T00:12:00.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb957600640 1 -- start start 2024-04-25T00:12:00.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb957600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95006e950 0x7fb950157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb957600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb950159810 0x7fb950157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb957600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb950160020 0x7fb95015aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb957600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb950169b80 con 0x7fb95006e950 2024-04-25T00:12:00.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb957600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb950169a00 con 0x7fb950160020 2024-04-25T00:12:00.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb957600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb950169d00 con 0x7fb950159810 2024-04-25T00:12:00.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb954c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95006e950 0x7fb950157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb954c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95006e950 0x7fb950157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39824/0 (socket says 172.21.15.83:39824) 2024-04-25T00:12:00.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb954c00640 1 -- 172.21.15.83:0/668642564 learned_addr learned my addr 172.21.15.83:0/668642564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb947e00640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb950159810 0x7fb950157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb955600640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb950160020 0x7fb95015aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb954c00640 1 -- 172.21.15.83:0/668642564 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb950159810 msgr2=0x7fb950157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb954c00640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb950159810 0x7fb950157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb954c00640 1 -- 172.21.15.83:0/668642564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb950160020 msgr2=0x7fb95015aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb954c00640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb950160020 0x7fb95015aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb954c00640 1 -- 172.21.15.83:0/668642564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9501564b0 con 0x7fb95006e950 2024-04-25T00:12:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb947e00640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb950159810 0x7fb950157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb954c00640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95006e950 0x7fb950157230 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb94000c500 tx=0x7fb94000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb94d600640 1 -- 172.21.15.83:0/668642564 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940016020 con 0x7fb95006e950 2024-04-25T00:12:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb94d600640 1 -- 172.21.15.83:0/668642564 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb940005220 con 0x7fb95006e950 2024-04-25T00:12:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb94d600640 1 -- 172.21.15.83:0/668642564 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940007030 con 0x7fb95006e950 2024-04-25T00:12:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb957600640 1 -- 172.21.15.83:0/668642564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb950158600 con 0x7fb95006e950 2024-04-25T00:12:00.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.607+0000 7fb957600640 1 -- 172.21.15.83:0/668642564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb950156a30 con 0x7fb95006e950 2024-04-25T00:12:00.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.611+0000 7fb957600640 1 -- 172.21.15.83:0/668642564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb95004f510 con 0x7fb95006e950 2024-04-25T00:12:00.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.611+0000 7fb94d600640 1 -- 172.21.15.83:0/668642564 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb9400071f0 con 0x7fb95006e950 2024-04-25T00:12:00.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.611+0000 7fb94d600640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb92c0678c0 0x7fb92c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.611+0000 7fb94d600640 1 -- 172.21.15.83:0/668642564 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb940085a90 con 0x7fb95006e950 2024-04-25T00:12:00.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.611+0000 7fb947e00640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb92c0678c0 0x7fb92c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.611+0000 7fb947e00640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb92c0678c0 0x7fb92c069d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb9500af680 tx=0x7fb938009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:00.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.615+0000 7fb94d600640 1 -- 172.21.15.83:0/668642564 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb940020020 con 0x7fb95006e950 2024-04-25T00:12:00.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.771+0000 7fb957600640 1 -- 172.21.15.83:0/668642564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb950166fc0 con 0x7fb95006e950 2024-04-25T00:12:00.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.771+0000 7fb94d600640 1 -- 172.21.15.83:0/668642564 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fb940056140 con 0x7fb95006e950 2024-04-25T00:12:00.775 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:12:00.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.771+0000 7fb957600640 1 -- 172.21.15.83:0/668642564 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb92c0678c0 msgr2=0x7fb92c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.771+0000 7fb957600640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb92c0678c0 0x7fb92c069d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb9500af680 tx=0x7fb938009040 comp rx=0 tx=0).stop 2024-04-25T00:12:00.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.771+0000 7fb957600640 1 -- 172.21.15.83:0/668642564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95006e950 msgr2=0x7fb950157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.771+0000 7fb957600640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95006e950 0x7fb950157230 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb94000c500 tx=0x7fb94000c9c0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.771+0000 7fb957600640 1 -- 172.21.15.83:0/668642564 shutdown_connections 2024-04-25T00:12:00.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.771+0000 7fb957600640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb92c0678c0 0x7fb92c069d80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.771+0000 7fb957600640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb950159810 0x7fb950157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.771+0000 7fb957600640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb950160020 0x7fb95015aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.771+0000 7fb957600640 1 --2- 172.21.15.83:0/668642564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95006e950 0x7fb950157230 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.771+0000 7fb957600640 1 -- 172.21.15.83:0/668642564 >> 172.21.15.83:0/668642564 conn(0x7fb9500b1830 msgr2=0x7fb95015c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:00.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.771+0000 7fb957600640 1 -- 172.21.15.83:0/668642564 shutdown_connections 2024-04-25T00:12:00.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.771+0000 7fb957600640 1 -- 172.21.15.83:0/668642564 wait complete. 2024-04-25T00:12:00.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:12:00.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.927+0000 7f6430c00640 1 Processor -- start 2024-04-25T00:12:00.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.927+0000 7f6430c00640 1 -- start start 2024-04-25T00:12:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.931+0000 7f6430c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f642c1455d0 0x7f642c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.931+0000 7f6430c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f642c15ff10 0x7f642c071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.931+0000 7f6430c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f642c070ca0 con 0x7f642c0727b0 2024-04-25T00:12:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.931+0000 7f6430c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f642c0ce9d0 con 0x7f642c1455d0 2024-04-25T00:12:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.931+0000 7f6430c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f642c070fa0 con 0x7f642c15ff10 2024-04-25T00:12:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.931+0000 7f642b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f642c1455d0 0x7f642c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.931+0000 7f642b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f642c1455d0 0x7f642c06cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39832/0 (socket says 172.21.15.83:39832) 2024-04-25T00:12:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.931+0000 7f642b400640 1 -- 172.21.15.83:0/1052952296 learned_addr learned my addr 172.21.15.83:0/1052952296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.931+0000 7f642be00640 1 --2- 172.21.15.83:0/1052952296 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f642c15ff10 0x7f642c071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.931+0000 7f642b400640 1 -- 172.21.15.83:0/1052952296 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f642c15ff10 msgr2=0x7f642c071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.931+0000 7f642b400640 1 --2- 172.21.15.83:0/1052952296 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f642c15ff10 0x7f642c071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.931+0000 7f642b400640 1 -- 172.21.15.83:0/1052952296 >> v1:172.21.15.86:6789/0 conn(0x7f642c0727b0 legacy=0x7f642c15f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.931+0000 7f642b400640 1 -- 172.21.15.83:0/1052952296 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642c06d1a0 con 0x7f642c1455d0 2024-04-25T00:12:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.935+0000 7f642b400640 1 --2- 172.21.15.83:0/1052952296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f642c1455d0 0x7f642c06cc60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f641c009e50 tx=0x7f642c072180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a23bb12570ee750 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:00.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.935+0000 7f642aa00640 1 -- 172.21.15.83:0/1052952296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c0a5700 con 0x7f642c1455d0 2024-04-25T00:12:00.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.935+0000 7f642aa00640 1 -- 172.21.15.83:0/1052952296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f641c0a58a0 con 0x7f642c1455d0 2024-04-25T00:12:00.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.935+0000 7f642aa00640 1 -- 172.21.15.83:0/1052952296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c0a5c70 con 0x7f642c1455d0 2024-04-25T00:12:00.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.935+0000 7f6430c00640 1 -- 172.21.15.83:0/1052952296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f642c1455d0 msgr2=0x7f642c06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.935+0000 7f6430c00640 1 --2- 172.21.15.83:0/1052952296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f642c1455d0 0x7f642c06cc60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f641c009e50 tx=0x7f642c072180 comp rx=0 tx=0).stop 2024-04-25T00:12:00.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.935+0000 7f6430c00640 1 -- 172.21.15.83:0/1052952296 shutdown_connections 2024-04-25T00:12:00.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.935+0000 7f6430c00640 1 --2- 172.21.15.83:0/1052952296 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f642c15ff10 0x7f642c071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.935+0000 7f6430c00640 1 --2- 172.21.15.83:0/1052952296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f642c1455d0 0x7f642c06cc60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.935+0000 7f6430c00640 1 -- 172.21.15.83:0/1052952296 >> 172.21.15.83:0/1052952296 conn(0x7f642c0cf980 msgr2=0x7f642c0cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:00.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.935+0000 7f6430c00640 1 -- 172.21.15.83:0/1052952296 shutdown_connections 2024-04-25T00:12:00.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.935+0000 7f6430c00640 1 -- 172.21.15.83:0/1052952296 wait complete. 2024-04-25T00:12:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.935+0000 7f6430c00640 1 Processor -- start 2024-04-25T00:12:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.935+0000 7f6430c00640 1 -- start start 2024-04-25T00:12:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f6430c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f642c0727b0 0x7f642c0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f6430c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f642c1455d0 0x7f642c13e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f6430c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f642c15ff10 0x7f642c15b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f6430c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f642c168ba0 con 0x7f642c0727b0 2024-04-25T00:12:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f6430c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f642c0cf1e0 con 0x7f642c15ff10 2024-04-25T00:12:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f6430c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f642c168d20 con 0x7f642c1455d0 2024-04-25T00:12:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f642b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f642c0727b0 0x7f642c0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f642b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f642c0727b0 0x7f642c0b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39846/0 (socket says 172.21.15.83:39846) 2024-04-25T00:12:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f642b400640 1 -- 172.21.15.83:0/112121870 learned_addr learned my addr 172.21.15.83:0/112121870 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:00.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f6422a00640 1 --2- 172.21.15.83:0/112121870 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f642c1455d0 0x7f642c13e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f642be00640 1 --2- 172.21.15.83:0/112121870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f642c15ff10 0x7f642c15b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f642b400640 1 -- 172.21.15.83:0/112121870 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f642c1455d0 msgr2=0x7f642c13e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f642b400640 1 --2- 172.21.15.83:0/112121870 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f642c1455d0 0x7f642c13e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f642b400640 1 -- 172.21.15.83:0/112121870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f642c15ff10 msgr2=0x7f642c15b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f642b400640 1 --2- 172.21.15.83:0/112121870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f642c15ff10 0x7f642c15b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f642b400640 1 -- 172.21.15.83:0/112121870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642c13ea70 con 0x7f642c0727b0 2024-04-25T00:12:00.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f642b400640 1 --2- 172.21.15.83:0/112121870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f642c0727b0 0x7f642c0b3c70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f641c01a660 tx=0x7f641c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:00.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f6423e00640 1 -- 172.21.15.83:0/112121870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c0bc020 con 0x7f642c0727b0 2024-04-25T00:12:00.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f6423e00640 1 -- 172.21.15.83:0/112121870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f641c0b8670 con 0x7f642c0727b0 2024-04-25T00:12:00.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f6423e00640 1 -- 172.21.15.83:0/112121870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c0b8a60 con 0x7f642c0727b0 2024-04-25T00:12:00.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f6430c00640 1 -- 172.21.15.83:0/112121870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642c13fd90 con 0x7f642c0727b0 2024-04-25T00:12:00.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f6430c00640 1 -- 172.21.15.83:0/112121870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f642c13efd0 con 0x7f642c0727b0 2024-04-25T00:12:00.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f6430c00640 1 -- 172.21.15.83:0/112121870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f642c04f510 con 0x7f642c0727b0 2024-04-25T00:12:00.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.939+0000 7f6423e00640 1 -- 172.21.15.83:0/112121870 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f641c0bf020 con 0x7f642c0727b0 2024-04-25T00:12:00.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.943+0000 7f6423e00640 1 --2- 172.21.15.83:0/112121870 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f64080678c0 0x7f6408069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.943+0000 7f6423e00640 1 -- 172.21.15.83:0/112121870 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f641c120fe0 con 0x7f642c0727b0 2024-04-25T00:12:00.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.943+0000 7f6422a00640 1 --2- 172.21.15.83:0/112121870 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f64080678c0 0x7f6408069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.943+0000 7f6422a00640 1 --2- 172.21.15.83:0/112121870 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f64080678c0 0x7f6408069d80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f642c0ada10 tx=0x7f6414009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:00.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:00.947+0000 7f6423e00640 1 -- 172.21.15.83:0/112121870 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f641c0c6030 con 0x7f642c0727b0 2024-04-25T00:12:01.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.099+0000 7f6430c00640 1 -- 172.21.15.83:0/112121870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f642c06e8e0 con 0x7f642c0727b0 2024-04-25T00:12:01.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.099+0000 7f6423e00640 1 -- 172.21.15.83:0/112121870 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f642c13ea70 con 0x7f642c0727b0 2024-04-25T00:12:01.104 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:12:01.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.103+0000 7f6430c00640 1 -- 172.21.15.83:0/112121870 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f64080678c0 msgr2=0x7f6408069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.103+0000 7f6430c00640 1 --2- 172.21.15.83:0/112121870 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f64080678c0 0x7f6408069d80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f642c0ada10 tx=0x7f6414009040 comp rx=0 tx=0).stop 2024-04-25T00:12:01.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.103+0000 7f6430c00640 1 -- 172.21.15.83:0/112121870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f642c0727b0 msgr2=0x7f642c0b3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.103+0000 7f6430c00640 1 --2- 172.21.15.83:0/112121870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f642c0727b0 0x7f642c0b3c70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f641c01a660 tx=0x7f641c0197d0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.103+0000 7f6430c00640 1 -- 172.21.15.83:0/112121870 shutdown_connections 2024-04-25T00:12:01.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.103+0000 7f6430c00640 1 --2- 172.21.15.83:0/112121870 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f64080678c0 0x7f6408069d80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.103+0000 7f6430c00640 1 --2- 172.21.15.83:0/112121870 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f642c1455d0 0x7f642c13e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.103+0000 7f6430c00640 1 --2- 172.21.15.83:0/112121870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f642c15ff10 0x7f642c15b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.103+0000 7f6430c00640 1 --2- 172.21.15.83:0/112121870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f642c0727b0 0x7f642c0b3c70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.103+0000 7f6430c00640 1 -- 172.21.15.83:0/112121870 >> 172.21.15.83:0/112121870 conn(0x7f642c0cf980 msgr2=0x7f642c0adb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:01.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.103+0000 7f6430c00640 1 -- 172.21.15.83:0/112121870 shutdown_connections 2024-04-25T00:12:01.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.103+0000 7f6430c00640 1 -- 172.21.15.83:0/112121870 wait complete. 2024-04-25T00:12:01.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:12:01.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.255+0000 7f5bb0600640 1 Processor -- start 2024-04-25T00:12:01.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.255+0000 7f5bb0600640 1 -- start start 2024-04-25T00:12:01.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.255+0000 7f5bb0600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ba80cb640 0x7f5ba806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.255+0000 7f5bb0600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ba8070d10 0x7f5ba8165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.255+0000 7f5bb0600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ba80711a0 con 0x7f5ba806cfd0 2024-04-25T00:12:01.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.255+0000 7f5bb0600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5ba806e160 con 0x7f5ba80cb640 2024-04-25T00:12:01.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.255+0000 7f5bb0600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba806ea10 con 0x7f5ba8070d10 2024-04-25T00:12:01.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.255+0000 7f5bad200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ba80cb640 0x7f5ba806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.255+0000 7f5badc00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5ba806cfd0 0x7f5ba806d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56954/0 (socket says 172.21.15.83:56954) 2024-04-25T00:12:01.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.255+0000 7f5badc00640 1 -- 172.21.15.83:0/3381302620 learned_addr learned my addr 172.21.15.83:0/3381302620 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:01.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.259+0000 7f5bae600640 1 --2- 172.21.15.83:0/3381302620 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ba8070d10 0x7f5ba8165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.259+0000 7f5bad200640 1 -- 172.21.15.83:0/3381302620 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ba8070d10 msgr2=0x7f5ba8165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:01.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.259+0000 7f5bad200640 1 --2- 172.21.15.83:0/3381302620 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ba8070d10 0x7f5ba8165930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.259+0000 7f5bad200640 1 -- 172.21.15.83:0/3381302620 >> v1:172.21.15.83:6789/0 conn(0x7f5ba806cfd0 legacy=0x7f5ba806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.259+0000 7f5bad200640 1 -- 172.21.15.83:0/3381302620 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba806ecf0 con 0x7f5ba80cb640 2024-04-25T00:12:01.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.259+0000 7f5ba7e00640 1 -- 172.21.15.83:0/3381302620 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3365236600 0 0) 0x7f5b9c001120 con 0x7f5ba806cfd0 2024-04-25T00:12:01.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.259+0000 7f5bad200640 1 --2- 172.21.15.83:0/3381302620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80cb640 0x7f5ba806dac0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5b940097c0 tx=0x7f5b9401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3be49cdb4e2199 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.259+0000 7f5ba7e00640 1 -- 172.21.15.83:0/3381302620 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b940b1070 con 0x7f5ba80cb640 2024-04-25T00:12:01.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.259+0000 7f5ba7e00640 1 -- 172.21.15.83:0/3381302620 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b940ac410 con 0x7f5ba80cb640 2024-04-25T00:12:01.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.259+0000 7f5ba7e00640 1 -- 172.21.15.83:0/3381302620 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b940ac7e0 con 0x7f5ba80cb640 2024-04-25T00:12:01.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 -- 172.21.15.83:0/3381302620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80cb640 msgr2=0x7f5ba806dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 --2- 172.21.15.83:0/3381302620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80cb640 0x7f5ba806dac0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5b940097c0 tx=0x7f5b9401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 -- 172.21.15.83:0/3381302620 shutdown_connections 2024-04-25T00:12:01.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 --2- 172.21.15.83:0/3381302620 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ba8070d10 0x7f5ba8165930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 --2- 172.21.15.83:0/3381302620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80cb640 0x7f5ba806dac0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 -- 172.21.15.83:0/3381302620 >> 172.21.15.83:0/3381302620 conn(0x7f5ba80cfa60 msgr2=0x7f5ba80cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:01.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 -- 172.21.15.83:0/3381302620 shutdown_connections 2024-04-25T00:12:01.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 -- 172.21.15.83:0/3381302620 wait complete. 2024-04-25T00:12:01.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 Processor -- start 2024-04-25T00:12:01.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 -- start start 2024-04-25T00:12:01.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ba806cfd0 0x7f5ba815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ba8070d10 0x7f5ba80cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80cb640 0x7f5ba80cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba8168a20 con 0x7f5ba80cb640 2024-04-25T00:12:01.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba81688a0 con 0x7f5ba8070d10 2024-04-25T00:12:01.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bb0600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba8168ba0 con 0x7f5ba806cfd0 2024-04-25T00:12:01.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bae600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80cb640 0x7f5ba80cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bae600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80cb640 0x7f5ba80cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39862/0 (socket says 172.21.15.83:39862) 2024-04-25T00:12:01.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bae600640 1 -- 172.21.15.83:0/2914552005 learned_addr learned my addr 172.21.15.83:0/2914552005 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:01.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5badc00640 1 --2- 172.21.15.83:0/2914552005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ba806cfd0 0x7f5ba815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bad200640 1 --2- 172.21.15.83:0/2914552005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ba8070d10 0x7f5ba80cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bae600640 1 -- 172.21.15.83:0/2914552005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ba806cfd0 msgr2=0x7f5ba815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bae600640 1 --2- 172.21.15.83:0/2914552005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ba806cfd0 0x7f5ba815b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bae600640 1 -- 172.21.15.83:0/2914552005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ba8070d10 msgr2=0x7f5ba80cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bae600640 1 --2- 172.21.15.83:0/2914552005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ba8070d10 0x7f5ba80cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bae600640 1 -- 172.21.15.83:0/2914552005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba8143eb0 con 0x7f5ba80cb640 2024-04-25T00:12:01.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.263+0000 7f5bae600640 1 --2- 172.21.15.83:0/2914552005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80cb640 0x7f5ba80cca20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0007e60 tx=0x7f5ba000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.267+0000 7f5ba5600640 1 -- 172.21.15.83:0/2914552005 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0014420 con 0x7f5ba80cb640 2024-04-25T00:12:01.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.267+0000 7f5ba5600640 1 -- 172.21.15.83:0/2914552005 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba0019070 con 0x7f5ba80cb640 2024-04-25T00:12:01.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.267+0000 7f5ba5600640 1 -- 172.21.15.83:0/2914552005 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0014820 con 0x7f5ba80cb640 2024-04-25T00:12:01.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.267+0000 7f5bb0600640 1 -- 172.21.15.83:0/2914552005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba80b3f70 con 0x7f5ba80cb640 2024-04-25T00:12:01.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.267+0000 7f5bb0600640 1 -- 172.21.15.83:0/2914552005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba80b4530 con 0x7f5ba80cb640 2024-04-25T00:12:01.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.267+0000 7f5bb0600640 1 -- 172.21.15.83:0/2914552005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba8068230 con 0x7f5ba80cb640 2024-04-25T00:12:01.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.267+0000 7f5ba5600640 1 -- 172.21.15.83:0/2914552005 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5ba00149c0 con 0x7f5ba80cb640 2024-04-25T00:12:01.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.267+0000 7f5ba5600640 1 --2- 172.21.15.83:0/2914552005 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5b880678c0 0x7f5b88069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.267+0000 7f5ba5600640 1 -- 172.21.15.83:0/2914552005 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5ba0086890 con 0x7f5ba80cb640 2024-04-25T00:12:01.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.267+0000 7f5badc00640 1 --2- 172.21.15.83:0/2914552005 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5b880678c0 0x7f5b88069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.271+0000 7f5badc00640 1 --2- 172.21.15.83:0/2914552005 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5b880678c0 0x7f5b88069d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c000f70 tx=0x7f5b9c007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.271+0000 7f5ba5600640 1 -- 172.21.15.83:0/2914552005 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5ba0056850 con 0x7f5ba80cb640 2024-04-25T00:12:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.443+0000 7f5bb0600640 1 -- 172.21.15.83:0/2914552005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5ba815dbc0 con 0x7f5ba80cb640 2024-04-25T00:12:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.443+0000 7f5ba5600640 1 -- 172.21.15.83:0/2914552005 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f5ba005ab50 con 0x7f5ba80cb640 2024-04-25T00:12:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:12:01.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.443+0000 7f5bb0600640 1 -- 172.21.15.83:0/2914552005 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5b880678c0 msgr2=0x7f5b88069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.443+0000 7f5bb0600640 1 --2- 172.21.15.83:0/2914552005 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5b880678c0 0x7f5b88069d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c000f70 tx=0x7f5b9c007df0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.443+0000 7f5bb0600640 1 -- 172.21.15.83:0/2914552005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80cb640 msgr2=0x7f5ba80cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.443+0000 7f5bb0600640 1 --2- 172.21.15.83:0/2914552005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80cb640 0x7f5ba80cca20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0007e60 tx=0x7f5ba000e4b0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.443+0000 7f5bb0600640 1 -- 172.21.15.83:0/2914552005 shutdown_connections 2024-04-25T00:12:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.443+0000 7f5bb0600640 1 --2- 172.21.15.83:0/2914552005 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5b880678c0 0x7f5b88069d80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.443+0000 7f5bb0600640 1 --2- 172.21.15.83:0/2914552005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ba80cb640 0x7f5ba80cca20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.443+0000 7f5bb0600640 1 --2- 172.21.15.83:0/2914552005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ba8070d10 0x7f5ba80cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.443+0000 7f5bb0600640 1 --2- 172.21.15.83:0/2914552005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ba806cfd0 0x7f5ba815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.443+0000 7f5bb0600640 1 -- 172.21.15.83:0/2914552005 >> 172.21.15.83:0/2914552005 conn(0x7f5ba80cfa60 msgr2=0x7f5ba815ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.443+0000 7f5bb0600640 1 -- 172.21.15.83:0/2914552005 shutdown_connections 2024-04-25T00:12:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.443+0000 7f5bb0600640 1 -- 172.21.15.83:0/2914552005 wait complete. 2024-04-25T00:12:01.468 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T00:12:01.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:01.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T00:12:01.492 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:12:01.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:01.492 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T00:12:01.545 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:12:01.545 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:01.555 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:12:01.555 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:01.606 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:12:01.606 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:01.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.787+0000 7f103b200640 1 Processor -- start 2024-04-25T00:12:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.787+0000 7f103b200640 1 -- start start 2024-04-25T00:12:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.787+0000 7f103b200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f10341553e0 0x7f103406e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.787+0000 7f103b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340b4a70 0x7f10340b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.787+0000 7f103b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f103406ef80 con 0x7f103406ea00 2024-04-25T00:12:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.787+0000 7f103b200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f103415b510 con 0x7f10341553e0 2024-04-25T00:12:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.787+0000 7f103b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f103415b7f0 con 0x7f10340b4a70 2024-04-25T00:12:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.791+0000 7f1033e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f10341553e0 0x7f103406e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.791+0000 7f1033e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f10341553e0 0x7f103406e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39876/0 (socket says 172.21.15.83:39876) 2024-04-25T00:12:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.791+0000 7f1033e00640 1 -- 172.21.15.83:0/4089887836 learned_addr learned my addr 172.21.15.83:0/4089887836 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.791+0000 7f1039200640 1 --2- 172.21.15.83:0/4089887836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340b4a70 0x7f10340b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.791+0000 7f1033e00640 1 -- 172.21.15.83:0/4089887836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340b4a70 msgr2=0x7f10340b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.791+0000 7f1033e00640 1 --2- 172.21.15.83:0/4089887836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340b4a70 0x7f10340b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.791+0000 7f1033e00640 1 -- 172.21.15.83:0/4089887836 >> v1:172.21.15.86:6789/0 conn(0x7f103406ea00 legacy=0x7f10340b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.791+0000 7f1033e00640 1 -- 172.21.15.83:0/4089887836 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103415bad0 con 0x7f10341553e0 2024-04-25T00:12:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.791+0000 7f1033e00640 1 --2- 172.21.15.83:0/4089887836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10341553e0 0x7f103406e4c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1024009e50 tx=0x7f102401ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be43d47bb56a19c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.791+0000 7f1033400640 1 -- 172.21.15.83:0/4089887836 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10240a7990 con 0x7f10341553e0 2024-04-25T00:12:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.791+0000 7f1033400640 1 -- 172.21.15.83:0/4089887836 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10240a7b30 con 0x7f10341553e0 2024-04-25T00:12:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.791+0000 7f1033400640 1 -- 172.21.15.83:0/4089887836 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10240af3f0 con 0x7f10341553e0 2024-04-25T00:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 -- 172.21.15.83:0/4089887836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10341553e0 msgr2=0x7f103406e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 --2- 172.21.15.83:0/4089887836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10341553e0 0x7f103406e4c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1024009e50 tx=0x7f102401ebb0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 -- 172.21.15.83:0/4089887836 shutdown_connections 2024-04-25T00:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 --2- 172.21.15.83:0/4089887836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340b4a70 0x7f10340b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 --2- 172.21.15.83:0/4089887836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10341553e0 0x7f103406e4c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 -- 172.21.15.83:0/4089887836 >> 172.21.15.83:0/4089887836 conn(0x7f10340b1830 msgr2=0x7f10340b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 -- 172.21.15.83:0/4089887836 shutdown_connections 2024-04-25T00:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 -- 172.21.15.83:0/4089887836 wait complete. 2024-04-25T00:12:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 Processor -- start 2024-04-25T00:12:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 -- start start 2024-04-25T00:12:01.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103406ea00 0x7f1034142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340b4a70 0x7f103415cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10341553e0 0x7f103415d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f10340b3a00 con 0x7f10341553e0 2024-04-25T00:12:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10340b3880 con 0x7f103406ea00 2024-04-25T00:12:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f103b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f10340b3b80 con 0x7f10340b4a70 2024-04-25T00:12:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f1039200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10341553e0 0x7f103415d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f1039200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10341553e0 0x7f103415d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39892/0 (socket says 172.21.15.83:39892) 2024-04-25T00:12:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f1039200640 1 -- 172.21.15.83:0/1386889051 learned_addr learned my addr 172.21.15.83:0/1386889051 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f1033e00640 1 --2- 172.21.15.83:0/1386889051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103406ea00 0x7f1034142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f102b400640 1 --2- 172.21.15.83:0/1386889051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340b4a70 0x7f103415cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f1039200640 1 -- 172.21.15.83:0/1386889051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340b4a70 msgr2=0x7f103415cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f1039200640 1 --2- 172.21.15.83:0/1386889051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340b4a70 0x7f103415cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f1039200640 1 -- 172.21.15.83:0/1386889051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103406ea00 msgr2=0x7f1034142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f1039200640 1 --2- 172.21.15.83:0/1386889051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103406ea00 0x7f1034142470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.795+0000 7f1039200640 1 -- 172.21.15.83:0/1386889051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103415a090 con 0x7f10341553e0 2024-04-25T00:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.799+0000 7f1039200640 1 --2- 172.21.15.83:0/1386889051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10341553e0 0x7f103415d2d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f102c00d590 tx=0x7f102c00da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.799+0000 7f1030c00640 1 -- 172.21.15.83:0/1386889051 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c0125f0 con 0x7f10341553e0 2024-04-25T00:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.799+0000 7f1030c00640 1 -- 172.21.15.83:0/1386889051 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f102c019070 con 0x7f10341553e0 2024-04-25T00:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.799+0000 7f1030c00640 1 -- 172.21.15.83:0/1386889051 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c0129f0 con 0x7f10341553e0 2024-04-25T00:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.799+0000 7f103b200640 1 -- 172.21.15.83:0/1386889051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103415a370 con 0x7f10341553e0 2024-04-25T00:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.799+0000 7f103b200640 1 -- 172.21.15.83:0/1386889051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103415a930 con 0x7f10341553e0 2024-04-25T00:12:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.799+0000 7f103b200640 1 -- 172.21.15.83:0/1386889051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10340737f0 con 0x7f10341553e0 2024-04-25T00:12:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.799+0000 7f1030c00640 1 -- 172.21.15.83:0/1386889051 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f102c012b90 con 0x7f10341553e0 2024-04-25T00:12:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.799+0000 7f1030c00640 1 --2- 172.21.15.83:0/1386889051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f10100678c0 0x7f1010069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.799+0000 7f1030c00640 1 -- 172.21.15.83:0/1386889051 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f102c086330 con 0x7f10341553e0 2024-04-25T00:12:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.799+0000 7f1033e00640 1 --2- 172.21.15.83:0/1386889051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f10100678c0 0x7f1010069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.803+0000 7f1033e00640 1 --2- 172.21.15.83:0/1386889051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f10100678c0 0x7f1010069d80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f10240b0000 tx=0x7f102400d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.803+0000 7f1030c00640 1 -- 172.21.15.83:0/1386889051 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f102c056410 con 0x7f10341553e0 2024-04-25T00:12:01.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.963+0000 7f103b200640 1 -- 172.21.15.83:0/1386889051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f103404f5f0 con 0x7f10341553e0 2024-04-25T00:12:01.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.963+0000 7f1030c00640 1 -- 172.21.15.83:0/1386889051 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18820 (secure 0 0 0) 0x7f102c05a710 con 0x7f10341553e0 2024-04-25T00:12:01.969 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:01.969 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:48.235809+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:01.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.967+0000 7f103b200640 1 -- 172.21.15.83:0/1386889051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f10100678c0 msgr2=0x7f1010069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.967+0000 7f103b200640 1 --2- 172.21.15.83:0/1386889051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f10100678c0 0x7f1010069d80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f10240b0000 tx=0x7f102400d040 comp rx=0 tx=0).stop 2024-04-25T00:12:01.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.967+0000 7f103b200640 1 -- 172.21.15.83:0/1386889051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10341553e0 msgr2=0x7f103415d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.967+0000 7f103b200640 1 --2- 172.21.15.83:0/1386889051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10341553e0 0x7f103415d2d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f102c00d590 tx=0x7f102c00da50 comp rx=0 tx=0).stop 2024-04-25T00:12:01.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.967+0000 7f103b200640 1 -- 172.21.15.83:0/1386889051 shutdown_connections 2024-04-25T00:12:01.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.967+0000 7f103b200640 1 --2- 172.21.15.83:0/1386889051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f10100678c0 0x7f1010069d80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.967+0000 7f103b200640 1 --2- 172.21.15.83:0/1386889051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340b4a70 0x7f103415cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.967+0000 7f103b200640 1 --2- 172.21.15.83:0/1386889051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10341553e0 0x7f103415d2d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.967+0000 7f103b200640 1 --2- 172.21.15.83:0/1386889051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103406ea00 0x7f1034142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.967+0000 7f103b200640 1 -- 172.21.15.83:0/1386889051 >> 172.21.15.83:0/1386889051 conn(0x7f10340b1830 msgr2=0x7f1034071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:01.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.967+0000 7f103b200640 1 -- 172.21.15.83:0/1386889051 shutdown_connections 2024-04-25T00:12:01.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:01.967+0000 7f103b200640 1 -- 172.21.15.83:0/1386889051 wait complete. 2024-04-25T00:12:01.990 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:11:04.621429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:11:05.189716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:11:08.902215+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:12:01.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:12:02.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.123+0000 7fd317e00640 1 Processor -- start 2024-04-25T00:12:02.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.123+0000 7fd317e00640 1 -- start start 2024-04-25T00:12:02.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd317e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd318155310 0x7fd31806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd317e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd31815e790 0x7fd31815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd317e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3180719f0 con 0x7fd31815b1a0 2024-04-25T00:12:02.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd317e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd31815f610 con 0x7fd31815e790 2024-04-25T00:12:02.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd317e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd318071cd0 con 0x7fd318155310 2024-04-25T00:12:02.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd316000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd31815e790 0x7fd31815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd316000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd31815e790 0x7fd31815ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39898/0 (socket says 172.21.15.83:39898) 2024-04-25T00:12:02.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd316000640 1 -- 172.21.15.83:0/730650725 learned_addr learned my addr 172.21.15.83:0/730650725 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:02.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd316a00640 1 --2- 172.21.15.83:0/730650725 >> v2:172.21.15.86:3300/0 conn(0x7fd318155310 0x7fd31806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd316000640 1 -- 172.21.15.83:0/730650725 >> v2:172.21.15.86:3300/0 conn(0x7fd318155310 msgr2=0x7fd31806cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:02.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd316000640 1 --2- 172.21.15.83:0/730650725 >> v2:172.21.15.86:3300/0 conn(0x7fd318155310 0x7fd31806cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd316000640 1 -- 172.21.15.83:0/730650725 >> v1:172.21.15.86:6789/0 conn(0x7fd31815b1a0 legacy=0x7fd31815b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd316000640 1 -- 172.21.15.83:0/730650725 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd318071fb0 con 0x7fd31815e790 2024-04-25T00:12:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd316000640 1 --2- 172.21.15.83:0/730650725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd31815e790 0x7fd31815ec40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd31806ef00 tx=0x7fd30401eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8804c193c070c633 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd315600640 1 -- 172.21.15.83:0/730650725 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3040b4070 con 0x7fd31815e790 2024-04-25T00:12:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd315600640 1 -- 172.21.15.83:0/730650725 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3040a7930 con 0x7fd31815e790 2024-04-25T00:12:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.127+0000 7fd315600640 1 -- 172.21.15.83:0/730650725 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3040a7d00 con 0x7fd31815e790 2024-04-25T00:12:02.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 -- 172.21.15.83:0/730650725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd31815e790 msgr2=0x7fd31815ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 --2- 172.21.15.83:0/730650725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd31815e790 0x7fd31815ec40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd31806ef00 tx=0x7fd30401eb60 comp rx=0 tx=0).stop 2024-04-25T00:12:02.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 -- 172.21.15.83:0/730650725 shutdown_connections 2024-04-25T00:12:02.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 --2- 172.21.15.83:0/730650725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd31815e790 0x7fd31815ec40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 --2- 172.21.15.83:0/730650725 >> v2:172.21.15.86:3300/0 conn(0x7fd318155310 0x7fd31806cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 -- 172.21.15.83:0/730650725 >> 172.21.15.83:0/730650725 conn(0x7fd3180cf980 msgr2=0x7fd3180cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:02.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 -- 172.21.15.83:0/730650725 shutdown_connections 2024-04-25T00:12:02.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 -- 172.21.15.83:0/730650725 wait complete. 2024-04-25T00:12:02.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 Processor -- start 2024-04-25T00:12:02.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 -- start start 2024-04-25T00:12:02.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318155310 0x7fd31813e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd31815b1a0 0x7fd31813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd31815e790 0x7fd3181403b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd318071740 con 0x7fd318155310 2024-04-25T00:12:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3180715c0 con 0x7fd31815b1a0 2024-04-25T00:12:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd31806dd00 con 0x7fd31815e790 2024-04-25T00:12:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd316a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318155310 0x7fd31813e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd316a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318155310 0x7fd31813e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39906/0 (socket says 172.21.15.83:39906) 2024-04-25T00:12:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd316a00640 1 -- 172.21.15.83:0/1926690354 learned_addr learned my addr 172.21.15.83:0/1926690354 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd316000640 1 --2- 172.21.15.83:0/1926690354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd31815b1a0 0x7fd31813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd317400640 1 --2- 172.21.15.83:0/1926690354 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd31815e790 0x7fd3181403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd316a00640 1 -- 172.21.15.83:0/1926690354 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd31815e790 msgr2=0x7fd3181403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd316a00640 1 --2- 172.21.15.83:0/1926690354 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd31815e790 0x7fd3181403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd316a00640 1 -- 172.21.15.83:0/1926690354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd31815b1a0 msgr2=0x7fd31813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd316a00640 1 --2- 172.21.15.83:0/1926690354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd31815b1a0 0x7fd31813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd316a00640 1 -- 172.21.15.83:0/1926690354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31813e8a0 con 0x7fd318155310 2024-04-25T00:12:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.131+0000 7fd316a00640 1 --2- 172.21.15.83:0/1926690354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318155310 0x7fd31813e360 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd30800d800 tx=0x7fd30800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.135+0000 7fd30ea00640 1 -- 172.21.15.83:0/1926690354 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30800b6c0 con 0x7fd318155310 2024-04-25T00:12:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.135+0000 7fd30ea00640 1 -- 172.21.15.83:0/1926690354 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd308019070 con 0x7fd318155310 2024-04-25T00:12:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.135+0000 7fd30ea00640 1 -- 172.21.15.83:0/1926690354 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30800ba60 con 0x7fd318155310 2024-04-25T00:12:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.135+0000 7fd317e00640 1 -- 172.21.15.83:0/1926690354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd318140be0 con 0x7fd318155310 2024-04-25T00:12:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.135+0000 7fd317e00640 1 -- 172.21.15.83:0/1926690354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3181411a0 con 0x7fd318155310 2024-04-25T00:12:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.135+0000 7fd317e00640 1 -- 172.21.15.83:0/1926690354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd31804f510 con 0x7fd318155310 2024-04-25T00:12:02.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.135+0000 7fd30ea00640 1 -- 172.21.15.83:0/1926690354 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd30800bc00 con 0x7fd318155310 2024-04-25T00:12:02.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.135+0000 7fd30ea00640 1 --2- 172.21.15.83:0/1926690354 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd2f80678f0 0x7fd2f8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.135+0000 7fd30ea00640 1 -- 172.21.15.83:0/1926690354 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd308086470 con 0x7fd318155310 2024-04-25T00:12:02.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.135+0000 7fd316000640 1 --2- 172.21.15.83:0/1926690354 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd2f80678f0 0x7fd2f8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.135+0000 7fd316000640 1 --2- 172.21.15.83:0/1926690354 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd2f80678f0 0x7fd2f8069db0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd3040b1000 tx=0x7fd304008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:02.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.139+0000 7fd30ea00640 1 -- 172.21.15.83:0/1926690354 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd308056ba0 con 0x7fd318155310 2024-04-25T00:12:02.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.299+0000 7fd317e00640 1 -- 172.21.15.83:0/1926690354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3180d0740 con 0x7fd318155310 2024-04-25T00:12:02.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.299+0000 7fd30ea00640 1 -- 172.21.15.83:0/1926690354 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fd3080564a0 con 0x7fd318155310 2024-04-25T00:12:02.304 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:12:02.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.303+0000 7fd317e00640 1 -- 172.21.15.83:0/1926690354 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd2f80678f0 msgr2=0x7fd2f8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.303+0000 7fd317e00640 1 --2- 172.21.15.83:0/1926690354 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd2f80678f0 0x7fd2f8069db0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd3040b1000 tx=0x7fd304008040 comp rx=0 tx=0).stop 2024-04-25T00:12:02.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.303+0000 7fd317e00640 1 -- 172.21.15.83:0/1926690354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318155310 msgr2=0x7fd31813e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.303+0000 7fd317e00640 1 --2- 172.21.15.83:0/1926690354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318155310 0x7fd31813e360 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd30800d800 tx=0x7fd30800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.303+0000 7fd317e00640 1 -- 172.21.15.83:0/1926690354 shutdown_connections 2024-04-25T00:12:02.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.303+0000 7fd317e00640 1 --2- 172.21.15.83:0/1926690354 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd2f80678f0 0x7fd2f8069db0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.303+0000 7fd317e00640 1 --2- 172.21.15.83:0/1926690354 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd31815e790 0x7fd3181403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.303+0000 7fd317e00640 1 --2- 172.21.15.83:0/1926690354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd31815b1a0 0x7fd31813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.303+0000 7fd317e00640 1 --2- 172.21.15.83:0/1926690354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318155310 0x7fd31813e360 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.303+0000 7fd317e00640 1 -- 172.21.15.83:0/1926690354 >> 172.21.15.83:0/1926690354 conn(0x7fd3180cf980 msgr2=0x7fd318159c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:02.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.303+0000 7fd317e00640 1 -- 172.21.15.83:0/1926690354 shutdown_connections 2024-04-25T00:12:02.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.303+0000 7fd317e00640 1 -- 172.21.15.83:0/1926690354 wait complete. 2024-04-25T00:12:02.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:12:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.459+0000 7f758a600640 1 Processor -- start 2024-04-25T00:12:02.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f758a600640 1 -- start start 2024-04-25T00:12:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f758a600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f758415bee0 0x7f75840cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f758a600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75840d0860 0x7f75840b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f758a600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75840b1120 con 0x7f75840cd140 2024-04-25T00:12:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f758a600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f75840b1470 con 0x7f758415bee0 2024-04-25T00:12:02.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f758a600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f75840cd690 con 0x7f75840d0860 2024-04-25T00:12:02.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f7589200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f758415bee0 0x7f75840cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f757be00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f75840cd140 0x7f75840d0150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56970/0 (socket says 172.21.15.83:56970) 2024-04-25T00:12:02.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f757be00640 1 -- 172.21.15.83:0/3468616024 learned_addr learned my addr 172.21.15.83:0/3468616024 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:02.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f7589c00640 1 --2- 172.21.15.83:0/3468616024 >> v2:172.21.15.86:3300/0 conn(0x7f75840d0860 0x7f75840b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f7589200640 1 -- 172.21.15.83:0/3468616024 >> v2:172.21.15.86:3300/0 conn(0x7f75840d0860 msgr2=0x7f75840b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:02.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f7589200640 1 --2- 172.21.15.83:0/3468616024 >> v2:172.21.15.86:3300/0 conn(0x7f75840d0860 0x7f75840b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f7589200640 1 -- 172.21.15.83:0/3468616024 >> v1:172.21.15.83:6789/0 conn(0x7f75840cd140 legacy=0x7f75840d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f7589200640 1 -- 172.21.15.83:0/3468616024 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75840cd970 con 0x7f758415bee0 2024-04-25T00:12:02.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.463+0000 7f7589c00640 1 --2- 172.21.15.83:0/3468616024 >> v2:172.21.15.86:3300/0 conn(0x7f75840d0860 0x7f75840b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:02.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.467+0000 7f7589200640 1 --2- 172.21.15.83:0/3468616024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f758415bee0 0x7f75840cf900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7584071ca0 tx=0x7f757401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e0021781d60809 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.467+0000 7f7583e00640 1 -- 172.21.15.83:0/3468616024 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740aa600 con 0x7f758415bee0 2024-04-25T00:12:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.467+0000 7f7583e00640 1 -- 172.21.15.83:0/3468616024 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75740af070 con 0x7f758415bee0 2024-04-25T00:12:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.467+0000 7f7583e00640 1 -- 172.21.15.83:0/3468616024 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740aa9c0 con 0x7f758415bee0 2024-04-25T00:12:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.467+0000 7f758a600640 1 -- 172.21.15.83:0/3468616024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f758415bee0 msgr2=0x7f75840cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.467+0000 7f758a600640 1 --2- 172.21.15.83:0/3468616024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f758415bee0 0x7f75840cf900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7584071ca0 tx=0x7f757401c480 comp rx=0 tx=0).stop 2024-04-25T00:12:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.467+0000 7f758a600640 1 -- 172.21.15.83:0/3468616024 shutdown_connections 2024-04-25T00:12:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.467+0000 7f758a600640 1 --2- 172.21.15.83:0/3468616024 >> v2:172.21.15.86:3300/0 conn(0x7f75840d0860 0x7f75840b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.467+0000 7f758a600640 1 --2- 172.21.15.83:0/3468616024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f758415bee0 0x7f75840cf900 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.467+0000 7f758a600640 1 -- 172.21.15.83:0/3468616024 >> 172.21.15.83:0/3468616024 conn(0x7f75840d1b70 msgr2=0x7f75840d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.467+0000 7f758a600640 1 -- 172.21.15.83:0/3468616024 shutdown_connections 2024-04-25T00:12:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.467+0000 7f758a600640 1 -- 172.21.15.83:0/3468616024 wait complete. 2024-04-25T00:12:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.467+0000 7f758a600640 1 Processor -- start 2024-04-25T00:12:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.467+0000 7f758a600640 1 -- start start 2024-04-25T00:12:02.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f758a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75840cd140 0x7f75840c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f758a600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75840d0860 0x7f758415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f758a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f758415bee0 0x7f758415cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f758a600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75840b2a60 con 0x7f758415bee0 2024-04-25T00:12:02.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f758a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75840b28e0 con 0x7f75840cd140 2024-04-25T00:12:02.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f758a600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f75840b2be0 con 0x7f75840d0860 2024-04-25T00:12:02.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7589c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f758415bee0 0x7f758415cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7589c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f758415bee0 0x7f758415cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39928/0 (socket says 172.21.15.83:39928) 2024-04-25T00:12:02.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7589c00640 1 -- 172.21.15.83:0/2191918929 learned_addr learned my addr 172.21.15.83:0/2191918929 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:02.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7589c00640 1 -- 172.21.15.83:0/2191918929 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75840d0860 msgr2=0x7f758415c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7589c00640 1 --2- 172.21.15.83:0/2191918929 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75840d0860 0x7f758415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7589c00640 1 -- 172.21.15.83:0/2191918929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75840cd140 msgr2=0x7f75840c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7589200640 1 --2- 172.21.15.83:0/2191918929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75840cd140 0x7f75840c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7589c00640 1 --2- 172.21.15.83:0/2191918929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75840cd140 0x7f75840c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7589c00640 1 -- 172.21.15.83:0/2191918929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75840c7e40 con 0x7f758415bee0 2024-04-25T00:12:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7589200640 1 --2- 172.21.15.83:0/2191918929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75840cd140 0x7f75840c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7589c00640 1 --2- 172.21.15.83:0/2191918929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f758415bee0 0x7f758415cb10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f757c00cd50 tx=0x7f757c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:02.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7581600640 1 -- 172.21.15.83:0/2191918929 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c00f680 con 0x7f758415bee0 2024-04-25T00:12:02.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7581600640 1 -- 172.21.15.83:0/2191918929 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c00f840 con 0x7f758415bee0 2024-04-25T00:12:02.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7581600640 1 -- 172.21.15.83:0/2191918929 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c00fc30 con 0x7f758415bee0 2024-04-25T00:12:02.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f758a600640 1 -- 172.21.15.83:0/2191918929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75840cb730 con 0x7f758415bee0 2024-04-25T00:12:02.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f758a600640 1 -- 172.21.15.83:0/2191918929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7584148310 con 0x7f758415bee0 2024-04-25T00:12:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f758a600640 1 -- 172.21.15.83:0/2191918929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758404f510 con 0x7f758415bee0 2024-04-25T00:12:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.471+0000 7f7581600640 1 -- 172.21.15.83:0/2191918929 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f757c00e3e0 con 0x7f758415bee0 2024-04-25T00:12:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.475+0000 7f7581600640 1 --2- 172.21.15.83:0/2191918929 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f75600678c0 0x7f7560069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.475+0000 7f7581600640 1 -- 172.21.15.83:0/2191918929 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f757c086ed0 con 0x7f758415bee0 2024-04-25T00:12:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.475+0000 7f7589200640 1 --2- 172.21.15.83:0/2191918929 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f75600678c0 0x7f7560069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.475+0000 7f7589200640 1 --2- 172.21.15.83:0/2191918929 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f75600678c0 0x7f7560069d80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f75740ae000 tx=0x7f7574008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.479+0000 7f7581600640 1 -- 172.21.15.83:0/2191918929 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f757c014030 con 0x7f758415bee0 2024-04-25T00:12:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.631+0000 7f758a600640 1 -- 172.21.15.83:0/2191918929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f75840b1ad0 con 0x7f758415bee0 2024-04-25T00:12:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.631+0000 7f7581600640 1 -- 172.21.15.83:0/2191918929 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f757c0574e0 con 0x7f758415bee0 2024-04-25T00:12:02.638 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:12:02.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.635+0000 7f758a600640 1 -- 172.21.15.83:0/2191918929 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f75600678c0 msgr2=0x7f7560069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.635+0000 7f758a600640 1 --2- 172.21.15.83:0/2191918929 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f75600678c0 0x7f7560069d80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f75740ae000 tx=0x7f7574008040 comp rx=0 tx=0).stop 2024-04-25T00:12:02.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.635+0000 7f758a600640 1 -- 172.21.15.83:0/2191918929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f758415bee0 msgr2=0x7f758415cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.635+0000 7f758a600640 1 --2- 172.21.15.83:0/2191918929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f758415bee0 0x7f758415cb10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f757c00cd50 tx=0x7f757c00a650 comp rx=0 tx=0).stop 2024-04-25T00:12:02.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.635+0000 7f758a600640 1 -- 172.21.15.83:0/2191918929 shutdown_connections 2024-04-25T00:12:02.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.635+0000 7f758a600640 1 --2- 172.21.15.83:0/2191918929 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f75600678c0 0x7f7560069d80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.635+0000 7f758a600640 1 --2- 172.21.15.83:0/2191918929 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75840d0860 0x7f758415c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.635+0000 7f758a600640 1 --2- 172.21.15.83:0/2191918929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f758415bee0 0x7f758415cb10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.635+0000 7f758a600640 1 --2- 172.21.15.83:0/2191918929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75840cd140 0x7f75840c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.635+0000 7f758a600640 1 -- 172.21.15.83:0/2191918929 >> 172.21.15.83:0/2191918929 conn(0x7f75840d1b70 msgr2=0x7f75840cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.635+0000 7f758a600640 1 -- 172.21.15.83:0/2191918929 shutdown_connections 2024-04-25T00:12:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.635+0000 7f758a600640 1 -- 172.21.15.83:0/2191918929 wait complete. 2024-04-25T00:12:02.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:12:02.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.790+0000 7f6d95600640 1 Processor -- start 2024-04-25T00:12:02.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.790+0000 7f6d95600640 1 -- start start 2024-04-25T00:12:02.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.790+0000 7f6d95600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6d90160020 0x7f6d90166ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.790+0000 7f6d95600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d900ce000 con 0x7f6d9006e950 2024-04-25T00:12:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.790+0000 7f6d95600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d9006eec0 con 0x7f6d90159810 2024-04-25T00:12:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.790+0000 7f6d95600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f6d9006e210 con 0x7f6d90160020 2024-04-25T00:12:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.790+0000 7f6d8f400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6d90160020 0x7f6d90166ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.790+0000 7f6d8f400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6d90160020 0x7f6d90166ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39944/0 (socket says 172.21.15.83:39944) 2024-04-25T00:12:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.790+0000 7f6d8f400640 1 -- 172.21.15.83:0/998481738 learned_addr learned my addr 172.21.15.83:0/998481738 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.794+0000 7f6d8f400640 1 -- 172.21.15.83:0/998481738 >> v1:172.21.15.86:6789/0 conn(0x7f6d90159810 legacy=0x7f6d900cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.794+0000 7f6d8f400640 1 -- 172.21.15.83:0/998481738 >> v1:172.21.15.83:6789/0 conn(0x7f6d9006e950 legacy=0x7f6d9015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.794+0000 7f6d8f400640 1 -- 172.21.15.83:0/998481738 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9006dc50 con 0x7f6d90160020 2024-04-25T00:12:02.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.794+0000 7f6d8f400640 1 --2- 172.21.15.83:0/998481738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d90160020 0x7f6d90166ca0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009870 tx=0x7f6d8801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcc22c21ecd08e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:02.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.794+0000 7f6d8d600640 1 -- 172.21.15.83:0/998481738 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d880b1070 con 0x7f6d90160020 2024-04-25T00:12:02.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.794+0000 7f6d8d600640 1 -- 172.21.15.83:0/998481738 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d880a8050 con 0x7f6d90160020 2024-04-25T00:12:02.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.794+0000 7f6d8d600640 1 -- 172.21.15.83:0/998481738 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d880ac860 con 0x7f6d90160020 2024-04-25T00:12:02.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.794+0000 7f6d95600640 1 -- 172.21.15.83:0/998481738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d90160020 msgr2=0x7f6d90166ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.794+0000 7f6d95600640 1 --2- 172.21.15.83:0/998481738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d90160020 0x7f6d90166ca0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009870 tx=0x7f6d8801c4d0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.794+0000 7f6d95600640 1 -- 172.21.15.83:0/998481738 shutdown_connections 2024-04-25T00:12:02.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.794+0000 7f6d95600640 1 --2- 172.21.15.83:0/998481738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d90160020 0x7f6d90166ca0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.794+0000 7f6d95600640 1 -- 172.21.15.83:0/998481738 >> 172.21.15.83:0/998481738 conn(0x7f6d900b1830 msgr2=0x7f6d900b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:02.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d95600640 1 -- 172.21.15.83:0/998481738 shutdown_connections 2024-04-25T00:12:02.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d95600640 1 -- 172.21.15.83:0/998481738 wait complete. 2024-04-25T00:12:02.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d95600640 1 Processor -- start 2024-04-25T00:12:02.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d95600640 1 -- start start 2024-04-25T00:12:02.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d95600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d9006e950 0x7f6d9014ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d95600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d90159810 0x7f6d901559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d95600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d90160020 0x7f6d90156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d95600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d90169b80 con 0x7f6d9006e950 2024-04-25T00:12:02.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d95600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d90169a00 con 0x7f6d90160020 2024-04-25T00:12:02.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d95600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6d90169d00 con 0x7f6d90159810 2024-04-25T00:12:02.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d8ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d9006e950 0x7f6d9014ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d8ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d9006e950 0x7f6d9014ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39952/0 (socket says 172.21.15.83:39952) 2024-04-25T00:12:02.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d8ea00640 1 -- 172.21.15.83:0/3273790641 learned_addr learned my addr 172.21.15.83:0/3273790641 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:02.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d8f400640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d90160020 0x7f6d90156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d8e000640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d90159810 0x7f6d901559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d8ea00640 1 -- 172.21.15.83:0/3273790641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d90159810 msgr2=0x7f6d901559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d8ea00640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d90159810 0x7f6d901559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d8ea00640 1 -- 172.21.15.83:0/3273790641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d90160020 msgr2=0x7f6d90156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d8ea00640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d90160020 0x7f6d90156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d8ea00640 1 -- 172.21.15.83:0/3273790641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9015b260 con 0x7f6d9006e950 2024-04-25T00:12:02.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d8e000640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d90159810 0x7f6d901559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:02.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d8ea00640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d9006e950 0x7f6d9014ba50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000c790 tx=0x7f6d8000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:02.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d86a00640 1 -- 172.21.15.83:0/3273790641 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80016020 con 0x7f6d9006e950 2024-04-25T00:12:02.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d86a00640 1 -- 172.21.15.83:0/3273790641 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d80006d60 con 0x7f6d9006e950 2024-04-25T00:12:02.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d86a00640 1 -- 172.21.15.83:0/3273790641 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80003bc0 con 0x7f6d9006e950 2024-04-25T00:12:02.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d95600640 1 -- 172.21.15.83:0/3273790641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9015b540 con 0x7f6d9006e950 2024-04-25T00:12:02.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.798+0000 7f6d95600640 1 -- 172.21.15.83:0/3273790641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d901527f0 con 0x7f6d9006e950 2024-04-25T00:12:02.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.802+0000 7f6d95600640 1 -- 172.21.15.83:0/3273790641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d901529f0 con 0x7f6d9006e950 2024-04-25T00:12:02.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.802+0000 7f6d86a00640 1 -- 172.21.15.83:0/3273790641 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6d8000a850 con 0x7f6d9006e950 2024-04-25T00:12:02.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.802+0000 7f6d86a00640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6d700678c0 0x7f6d70069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.802+0000 7f6d86a00640 1 -- 172.21.15.83:0/3273790641 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6d80085e40 con 0x7f6d9006e950 2024-04-25T00:12:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.802+0000 7f6d8e000640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6d700678c0 0x7f6d70069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.802+0000 7f6d8e000640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6d700678c0 0x7f6d70069d80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6d90156180 tx=0x7f6d7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:02.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.806+0000 7f6d86a00640 1 -- 172.21.15.83:0/3273790641 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6d901529f0 con 0x7f6d9006e950 2024-04-25T00:12:02.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.966+0000 7f6d95600640 1 -- 172.21.15.83:0/3273790641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6d9004f510 con 0x7f6d9006e950 2024-04-25T00:12:02.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.966+0000 7f6d86a00640 1 -- 172.21.15.83:0/3273790641 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f6d80020020 con 0x7f6d9006e950 2024-04-25T00:12:02.973 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:12:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.970+0000 7f6d95600640 1 -- 172.21.15.83:0/3273790641 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6d700678c0 msgr2=0x7f6d70069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.970+0000 7f6d95600640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6d700678c0 0x7f6d70069d80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6d90156180 tx=0x7f6d7c009040 comp rx=0 tx=0).stop 2024-04-25T00:12:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.970+0000 7f6d95600640 1 -- 172.21.15.83:0/3273790641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d9006e950 msgr2=0x7f6d9014ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.970+0000 7f6d95600640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d9006e950 0x7f6d9014ba50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000c790 tx=0x7f6d8000cc50 comp rx=0 tx=0).stop 2024-04-25T00:12:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.970+0000 7f6d95600640 1 -- 172.21.15.83:0/3273790641 shutdown_connections 2024-04-25T00:12:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.970+0000 7f6d95600640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6d700678c0 0x7f6d70069d80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.970+0000 7f6d95600640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d90159810 0x7f6d901559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.970+0000 7f6d95600640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d90160020 0x7f6d90156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.970+0000 7f6d95600640 1 --2- 172.21.15.83:0/3273790641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d9006e950 0x7f6d9014ba50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.970+0000 7f6d95600640 1 -- 172.21.15.83:0/3273790641 >> 172.21.15.83:0/3273790641 conn(0x7f6d900b1830 msgr2=0x7f6d901512f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.970+0000 7f6d95600640 1 -- 172.21.15.83:0/3273790641 shutdown_connections 2024-04-25T00:12:02.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:02.970+0000 7f6d95600640 1 -- 172.21.15.83:0/3273790641 wait complete. 2024-04-25T00:12:02.994 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:12:03.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa10b200640 1 Processor -- start 2024-04-25T00:12:03.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa10b200640 1 -- start start 2024-04-25T00:12:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa10b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa104159810 0x7fa1040cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa10b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa104160020 0x7fa1040ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa10b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1040ce000 con 0x7fa10406e950 2024-04-25T00:12:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa10b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa10406e210 con 0x7fa104159810 2024-04-25T00:12:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa10b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa10406da00 con 0x7fa104160020 2024-04-25T00:12:03.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa103e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa104159810 0x7fa1040cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa109200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa104160020 0x7fa1040ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa103e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa104159810 0x7fa1040cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:47806/0 (socket says 172.21.15.83:47806) 2024-04-25T00:12:03.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa103e00640 1 -- 172.21.15.83:0/1093979270 learned_addr learned my addr 172.21.15.83:0/1093979270 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:03.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa0fb400640 1 --1- 172.21.15.83:0/1093979270 >> v1:172.21.15.86:6789/0 conn(0x7fa10406e950 0x7fa10415f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:32998/0 (socket says 172.21.15.83:32998) 2024-04-25T00:12:03.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa103e00640 1 -- 172.21.15.83:0/1093979270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa104160020 msgr2=0x7fa1040ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:03.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa103e00640 1 --2- 172.21.15.83:0/1093979270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa104160020 0x7fa1040ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa103e00640 1 -- 172.21.15.83:0/1093979270 >> v1:172.21.15.86:6789/0 conn(0x7fa10406e950 legacy=0x7fa10415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa103e00640 1 -- 172.21.15.83:0/1093979270 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10406eec0 con 0x7fa104159810 2024-04-25T00:12:03.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa109200640 1 --2- 172.21.15.83:0/1093979270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa104160020 0x7fa1040ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:03.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.138+0000 7fa103400640 1 -- 172.21.15.83:0/1093979270 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3296386180 0 0) 0x7fa1040ce000 con 0x7fa10406e950 2024-04-25T00:12:03.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.142+0000 7fa103e00640 1 --2- 172.21.15.83:0/1093979270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa104159810 0x7fa1040cc4a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009870 tx=0x7fa0f401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92ba07791b9b7606 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:03.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.142+0000 7fa103400640 1 -- 172.21.15.83:0/1093979270 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40aa600 con 0x7fa104159810 2024-04-25T00:12:03.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.142+0000 7fa103400640 1 -- 172.21.15.83:0/1093979270 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f40af070 con 0x7fa104159810 2024-04-25T00:12:03.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.142+0000 7fa103400640 1 -- 172.21.15.83:0/1093979270 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40aa9c0 con 0x7fa104159810 2024-04-25T00:12:03.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.142+0000 7fa10b200640 1 -- 172.21.15.83:0/1093979270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa104159810 msgr2=0x7fa1040cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.142+0000 7fa10b200640 1 --2- 172.21.15.83:0/1093979270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa104159810 0x7fa1040cc4a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009870 tx=0x7fa0f401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.142+0000 7fa10b200640 1 -- 172.21.15.83:0/1093979270 shutdown_connections 2024-04-25T00:12:03.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.142+0000 7fa10b200640 1 --2- 172.21.15.83:0/1093979270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa104160020 0x7fa1040ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.142+0000 7fa10b200640 1 --2- 172.21.15.83:0/1093979270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa104159810 0x7fa1040cc4a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.142+0000 7fa10b200640 1 -- 172.21.15.83:0/1093979270 >> 172.21.15.83:0/1093979270 conn(0x7fa1040b1830 msgr2=0x7fa1040b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:03.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.142+0000 7fa10b200640 1 -- 172.21.15.83:0/1093979270 shutdown_connections 2024-04-25T00:12:03.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.142+0000 7fa10b200640 1 -- 172.21.15.83:0/1093979270 wait complete. 2024-04-25T00:12:03.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.142+0000 7fa10b200640 1 Processor -- start 2024-04-25T00:12:03.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa10b200640 1 -- start start 2024-04-25T00:12:03.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa10b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10406e950 0x7fa1041509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa10b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa104159810 0x7fa1041665b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa10b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa104160020 0x7fa1040b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa10b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa104159d70 con 0x7fa10406e950 2024-04-25T00:12:03.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa10b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa104159bf0 con 0x7fa104159810 2024-04-25T00:12:03.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa10b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa104070da0 con 0x7fa104160020 2024-04-25T00:12:03.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa103e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10406e950 0x7fa1041509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa103e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10406e950 0x7fa1041509c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39966/0 (socket says 172.21.15.83:39966) 2024-04-25T00:12:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa103e00640 1 -- 172.21.15.83:0/2634928132 learned_addr learned my addr 172.21.15.83:0/2634928132 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa0fb400640 1 --2- 172.21.15.83:0/2634928132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa104159810 0x7fa1041665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa109200640 1 --2- 172.21.15.83:0/2634928132 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa104160020 0x7fa1040b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa103e00640 1 -- 172.21.15.83:0/2634928132 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa104160020 msgr2=0x7fa1040b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa103e00640 1 --2- 172.21.15.83:0/2634928132 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa104160020 0x7fa1040b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa103e00640 1 -- 172.21.15.83:0/2634928132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa104159810 msgr2=0x7fa1041665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa103e00640 1 --2- 172.21.15.83:0/2634928132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa104159810 0x7fa1041665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa103e00640 1 -- 172.21.15.83:0/2634928132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1040b0b00 con 0x7fa10406e950 2024-04-25T00:12:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa103e00640 1 --2- 172.21.15.83:0/2634928132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10406e950 0x7fa1041509c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa0f401a0d0 tx=0x7fa0f401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa100c00640 1 -- 172.21.15.83:0/2634928132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40bd5f0 con 0x7fa10406e950 2024-04-25T00:12:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa100c00640 1 -- 172.21.15.83:0/2634928132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f40bd7b0 con 0x7fa10406e950 2024-04-25T00:12:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa100c00640 1 -- 172.21.15.83:0/2634928132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40bdba0 con 0x7fa10406e950 2024-04-25T00:12:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa10b200640 1 -- 172.21.15.83:0/2634928132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa104151050 con 0x7fa10406e950 2024-04-25T00:12:03.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa10b200640 1 -- 172.21.15.83:0/2634928132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa104151610 con 0x7fa10406e950 2024-04-25T00:12:03.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.146+0000 7fa10b200640 1 -- 172.21.15.83:0/2634928132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa10404f510 con 0x7fa10406e950 2024-04-25T00:12:03.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.150+0000 7fa100c00640 1 -- 172.21.15.83:0/2634928132 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa0f40a9070 con 0x7fa10406e950 2024-04-25T00:12:03.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.150+0000 7fa100c00640 1 --2- 172.21.15.83:0/2634928132 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa0e00678c0 0x7fa0e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.150+0000 7fa100c00640 1 -- 172.21.15.83:0/2634928132 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa0f4120a40 con 0x7fa10406e950 2024-04-25T00:12:03.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.150+0000 7fa0fb400640 1 --2- 172.21.15.83:0/2634928132 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa0e00678c0 0x7fa0e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.150+0000 7fa0fb400640 1 --2- 172.21.15.83:0/2634928132 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa0e00678c0 0x7fa0e0069d80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa104153340 tx=0x7fa0ec007e10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:03.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.154+0000 7fa100c00640 1 -- 172.21.15.83:0/2634928132 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa0f40c3060 con 0x7fa10406e950 2024-04-25T00:12:03.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.310+0000 7fa10b200640 1 -- 172.21.15.83:0/2634928132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa104072c90 con 0x7fa10406e950 2024-04-25T00:12:03.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.310+0000 7fa100c00640 1 -- 172.21.15.83:0/2634928132 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fa0f4016150 con 0x7fa10406e950 2024-04-25T00:12:03.317 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:12:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.314+0000 7fa10b200640 1 -- 172.21.15.83:0/2634928132 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa0e00678c0 msgr2=0x7fa0e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.314+0000 7fa10b200640 1 --2- 172.21.15.83:0/2634928132 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa0e00678c0 0x7fa0e0069d80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa104153340 tx=0x7fa0ec007e10 comp rx=0 tx=0).stop 2024-04-25T00:12:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.314+0000 7fa10b200640 1 -- 172.21.15.83:0/2634928132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10406e950 msgr2=0x7fa1041509c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.314+0000 7fa10b200640 1 --2- 172.21.15.83:0/2634928132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10406e950 0x7fa1041509c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa0f401a0d0 tx=0x7fa0f401ace0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.314+0000 7fa10b200640 1 -- 172.21.15.83:0/2634928132 shutdown_connections 2024-04-25T00:12:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.314+0000 7fa10b200640 1 --2- 172.21.15.83:0/2634928132 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa0e00678c0 0x7fa0e0069d80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.314+0000 7fa10b200640 1 --2- 172.21.15.83:0/2634928132 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa104160020 0x7fa1040b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.314+0000 7fa10b200640 1 --2- 172.21.15.83:0/2634928132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa104159810 0x7fa1041665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.314+0000 7fa10b200640 1 --2- 172.21.15.83:0/2634928132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10406e950 0x7fa1041509c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.314+0000 7fa10b200640 1 -- 172.21.15.83:0/2634928132 >> 172.21.15.83:0/2634928132 conn(0x7fa1040b1830 msgr2=0x7fa10406cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:03.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.314+0000 7fa10b200640 1 -- 172.21.15.83:0/2634928132 shutdown_connections 2024-04-25T00:12:03.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.314+0000 7fa10b200640 1 -- 172.21.15.83:0/2634928132 wait complete. 2024-04-25T00:12:03.338 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:12:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6c3400640 1 Processor -- start 2024-04-25T00:12:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6c3400640 1 -- start start 2024-04-25T00:12:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6c3400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6bc0cb640 0x7ff6bc06dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6c3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff6bc070d10 0x7ff6bc165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6c3400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6bc0711a0 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6c3400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff6bc06e160 con 0x7ff6bc070d10 2024-04-25T00:12:03.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6c3400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff6bc06ea10 con 0x7ff6bc0cb640 2024-04-25T00:12:03.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6c0a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff6bc06cfd0 0x7ff6bc06d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56992/0 (socket says 172.21.15.83:56992) 2024-04-25T00:12:03.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6c0a00640 1 -- 172.21.15.83:0/15570611 learned_addr learned my addr 172.21.15.83:0/15570611 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:03.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6c1400640 1 --2- 172.21.15.83:0/15570611 >> v2:172.21.15.86:3300/0 conn(0x7ff6bc070d10 0x7ff6bc165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6bbe00640 1 --2- 172.21.15.83:0/15570611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6bc0cb640 0x7ff6bc06dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6bb400640 1 -- 172.21.15.83:0/15570611 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1844271496 0 0) 0x7ff6b0001120 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6bb400640 1 -- 172.21.15.83:0/15570611 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6a4003530 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6c1400640 1 -- 172.21.15.83:0/15570611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6bc0cb640 msgr2=0x7ff6bc06dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:03.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6c1400640 1 --2- 172.21.15.83:0/15570611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6bc0cb640 0x7ff6bc06dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6c1400640 1 -- 172.21.15.83:0/15570611 >> v1:172.21.15.83:6789/0 conn(0x7ff6bc06cfd0 legacy=0x7ff6bc06d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.470+0000 7ff6c1400640 1 -- 172.21.15.83:0/15570611 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6bc06ecf0 con 0x7ff6bc070d10 2024-04-25T00:12:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.474+0000 7ff6c1400640 1 --2- 172.21.15.83:0/15570611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6bc070d10 0x7ff6bc165930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009d90 tx=0x7ff6b401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dc710ae19159525 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.474+0000 7ff6bb400640 1 -- 172.21.15.83:0/15570611 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40b2070 con 0x7ff6bc070d10 2024-04-25T00:12:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.474+0000 7ff6bb400640 1 -- 172.21.15.83:0/15570611 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b40a5940 con 0x7ff6bc070d10 2024-04-25T00:12:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.474+0000 7ff6bb400640 1 -- 172.21.15.83:0/15570611 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40a5d10 con 0x7ff6bc070d10 2024-04-25T00:12:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.474+0000 7ff6c3400640 1 -- 172.21.15.83:0/15570611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6bc070d10 msgr2=0x7ff6bc165930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.474+0000 7ff6c3400640 1 --2- 172.21.15.83:0/15570611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6bc070d10 0x7ff6bc165930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009d90 tx=0x7ff6b401c970 comp rx=0 tx=0).stop 2024-04-25T00:12:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.474+0000 7ff6c3400640 1 -- 172.21.15.83:0/15570611 shutdown_connections 2024-04-25T00:12:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.474+0000 7ff6c3400640 1 --2- 172.21.15.83:0/15570611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6bc070d10 0x7ff6bc165930 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.474+0000 7ff6c3400640 1 --2- 172.21.15.83:0/15570611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6bc0cb640 0x7ff6bc06dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.474+0000 7ff6c3400640 1 -- 172.21.15.83:0/15570611 >> 172.21.15.83:0/15570611 conn(0x7ff6bc0cfa60 msgr2=0x7ff6bc0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.474+0000 7ff6c3400640 1 -- 172.21.15.83:0/15570611 shutdown_connections 2024-04-25T00:12:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.474+0000 7ff6c3400640 1 -- 172.21.15.83:0/15570611 wait complete. 2024-04-25T00:12:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c3400640 1 Processor -- start 2024-04-25T00:12:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c3400640 1 -- start start 2024-04-25T00:12:03.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6bc06cfd0 0x7ff6bc0cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c3400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6bc070d10 0x7ff6bc15caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6bc0cb640 0x7ff6bc0cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c3400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc168a20 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c3400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1688a0 con 0x7ff6bc0cb640 2024-04-25T00:12:03.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c3400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff6bc168ba0 con 0x7ff6bc070d10 2024-04-25T00:12:03.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6bc06cfd0 0x7ff6bc0cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6bc06cfd0 0x7ff6bc0cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39972/0 (socket says 172.21.15.83:39972) 2024-04-25T00:12:03.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c0a00640 1 -- 172.21.15.83:0/3786876791 learned_addr learned my addr 172.21.15.83:0/3786876791 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:03.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c1400640 1 --2- 172.21.15.83:0/3786876791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6bc0cb640 0x7ff6bc0cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6bbe00640 1 --2- 172.21.15.83:0/3786876791 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6bc070d10 0x7ff6bc15caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c0a00640 1 -- 172.21.15.83:0/3786876791 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6bc070d10 msgr2=0x7ff6bc15caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c0a00640 1 --2- 172.21.15.83:0/3786876791 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6bc070d10 0x7ff6bc15caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c0a00640 1 -- 172.21.15.83:0/3786876791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6bc0cb640 msgr2=0x7ff6bc0cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c0a00640 1 --2- 172.21.15.83:0/3786876791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6bc0cb640 0x7ff6bc0cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c0a00640 1 -- 172.21.15.83:0/3786876791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6bc15d030 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c0a00640 1 --2- 172.21.15.83:0/3786876791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6bc06cfd0 0x7ff6bc0cf120 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000c3c0 tx=0x7ff6b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:03.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6b8c00640 1 -- 172.21.15.83:0/3786876791 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0017070 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6b8c00640 1 -- 172.21.15.83:0/3786876791 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b0005cf0 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c3400640 1 -- 172.21.15.83:0/3786876791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6bc135bd0 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6b8c00640 1 -- 172.21.15.83:0/3786876791 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0007310 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c3400640 1 -- 172.21.15.83:0/3786876791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6bc0ce360 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.478+0000 7ff6c3400640 1 -- 172.21.15.83:0/3786876791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6bc068230 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.482+0000 7ff6b8c00640 1 -- 172.21.15.83:0/3786876791 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff6b001a070 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.482+0000 7ff6b8c00640 1 --2- 172.21.15.83:0/3786876791 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff69c0678c0 0x7ff69c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.482+0000 7ff6b8c00640 1 -- 172.21.15.83:0/3786876791 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff6b00853d0 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.482+0000 7ff6bbe00640 1 --2- 172.21.15.83:0/3786876791 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff69c0678c0 0x7ff69c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.482+0000 7ff6bbe00640 1 --2- 172.21.15.83:0/3786876791 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff69c0678c0 0x7ff69c069d80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc1fa8c0 tx=0x7ff6a8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:03.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.486+0000 7ff6b8c00640 1 -- 172.21.15.83:0/3786876791 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff6bc0711a0 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.654+0000 7ff6c3400640 1 -- 172.21.15.83:0/3786876791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff6bc15d7b0 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.654+0000 7ff6b8c00640 1 -- 172.21.15.83:0/3786876791 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7ff6b00554b0 con 0x7ff6bc06cfd0 2024-04-25T00:12:03.659 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:12:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.654+0000 7ff6c3400640 1 -- 172.21.15.83:0/3786876791 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff69c0678c0 msgr2=0x7ff69c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.654+0000 7ff6c3400640 1 --2- 172.21.15.83:0/3786876791 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff69c0678c0 0x7ff69c069d80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc1fa8c0 tx=0x7ff6a8009190 comp rx=0 tx=0).stop 2024-04-25T00:12:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.654+0000 7ff6c3400640 1 -- 172.21.15.83:0/3786876791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6bc06cfd0 msgr2=0x7ff6bc0cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.654+0000 7ff6c3400640 1 --2- 172.21.15.83:0/3786876791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6bc06cfd0 0x7ff6bc0cf120 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000c3c0 tx=0x7ff6b000c880 comp rx=0 tx=0).stop 2024-04-25T00:12:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.654+0000 7ff6c3400640 1 -- 172.21.15.83:0/3786876791 shutdown_connections 2024-04-25T00:12:03.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.654+0000 7ff6c3400640 1 --2- 172.21.15.83:0/3786876791 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff69c0678c0 0x7ff69c069d80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.654+0000 7ff6c3400640 1 --2- 172.21.15.83:0/3786876791 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6bc070d10 0x7ff6bc15caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.654+0000 7ff6c3400640 1 --2- 172.21.15.83:0/3786876791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6bc0cb640 0x7ff6bc0cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.654+0000 7ff6c3400640 1 --2- 172.21.15.83:0/3786876791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6bc06cfd0 0x7ff6bc0cf120 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.654+0000 7ff6c3400640 1 -- 172.21.15.83:0/3786876791 >> 172.21.15.83:0/3786876791 conn(0x7ff6bc0cfa60 msgr2=0x7ff6bc13a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:03.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.658+0000 7ff6c3400640 1 -- 172.21.15.83:0/3786876791 shutdown_connections 2024-04-25T00:12:03.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.658+0000 7ff6c3400640 1 -- 172.21.15.83:0/3786876791 wait complete. 2024-04-25T00:12:03.681 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T00:12:03.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:03.681 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T00:12:03.708 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:12:03.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:03.709 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T00:12:03.759 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:12:03.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:12:03.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f987b200640 1 Processor -- start 2024-04-25T00:12:03.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f987b200640 1 -- start start 2024-04-25T00:12:03.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f987b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98741599f0 0x7f98740cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f987b200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f987415c340 0x7f987415c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f987b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f987415ccf0 0x7f987415f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f987b200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9874159dd0 con 0x7f987415c340 2024-04-25T00:12:03.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f987b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f98740ccc60 con 0x7f98741599f0 2024-04-25T00:12:03.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f987b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f987406dba0 con 0x7f987415ccf0 2024-04-25T00:12:03.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f986b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f987415c340 0x7f987415c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f986b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f987415c340 0x7f987415c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39974/0 (socket says 172.21.15.83:39974) 2024-04-25T00:12:03.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f986b400640 1 -- 172.21.15.83:0/2401079696 learned_addr learned my addr 172.21.15.83:0/2401079696 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:03.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f9879200640 1 --2- 172.21.15.83:0/2401079696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f987415ccf0 0x7f987415f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f9873e00640 1 --2- 172.21.15.83:0/2401079696 >> v2:172.21.15.86:3300/0 conn(0x7f98741599f0 0x7f98740cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f986b400640 1 -- 172.21.15.83:0/2401079696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f987415ccf0 msgr2=0x7f987415f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:03.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f986b400640 1 --2- 172.21.15.83:0/2401079696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f987415ccf0 0x7f987415f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f986b400640 1 -- 172.21.15.83:0/2401079696 >> v2:172.21.15.86:3300/0 conn(0x7f98741599f0 msgr2=0x7f98740cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:03.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f986b400640 1 --2- 172.21.15.83:0/2401079696 >> v2:172.21.15.86:3300/0 conn(0x7f98741599f0 0x7f98740cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.890+0000 7f986b400640 1 -- 172.21.15.83:0/2401079696 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987406de80 con 0x7f987415c340 2024-04-25T00:12:03.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.894+0000 7f986b400640 1 --2- 172.21.15.83:0/2401079696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f987415c340 0x7f987415c7b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f985c009870 tx=0x7f985c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6466b19f9594ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:03.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.894+0000 7f9873400640 1 -- 172.21.15.83:0/2401079696 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c0b2070 con 0x7f987415c340 2024-04-25T00:12:03.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.894+0000 7f9873400640 1 -- 172.21.15.83:0/2401079696 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f985c0a9070 con 0x7f987415c340 2024-04-25T00:12:03.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.894+0000 7f9873400640 1 -- 172.21.15.83:0/2401079696 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c0ad880 con 0x7f987415c340 2024-04-25T00:12:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.894+0000 7f987b200640 1 -- 172.21.15.83:0/2401079696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f987415c340 msgr2=0x7f987415c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.894+0000 7f987b200640 1 --2- 172.21.15.83:0/2401079696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f987415c340 0x7f987415c7b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f985c009870 tx=0x7f985c01c490 comp rx=0 tx=0).stop 2024-04-25T00:12:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.894+0000 7f987b200640 1 -- 172.21.15.83:0/2401079696 shutdown_connections 2024-04-25T00:12:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.894+0000 7f987b200640 1 --2- 172.21.15.83:0/2401079696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f987415ccf0 0x7f987415f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.894+0000 7f987b200640 1 --2- 172.21.15.83:0/2401079696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f987415c340 0x7f987415c7b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.894+0000 7f987b200640 1 --2- 172.21.15.83:0/2401079696 >> v2:172.21.15.86:3300/0 conn(0x7f98741599f0 0x7f98740cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.894+0000 7f987b200640 1 -- 172.21.15.83:0/2401079696 >> 172.21.15.83:0/2401079696 conn(0x7f98740b1830 msgr2=0x7f98740b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.894+0000 7f987b200640 1 -- 172.21.15.83:0/2401079696 shutdown_connections 2024-04-25T00:12:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.894+0000 7f987b200640 1 -- 172.21.15.83:0/2401079696 wait complete. 2024-04-25T00:12:03.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f987b200640 1 Processor -- start 2024-04-25T00:12:03.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f987b200640 1 -- start start 2024-04-25T00:12:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f987b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98741599f0 0x7f98741faa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f987b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f987415c340 0x7f98740b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f987b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f987415ccf0 0x7f987414c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f987b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98740ce860 con 0x7f987415ccf0 2024-04-25T00:12:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f987b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98740ce6e0 con 0x7f98741599f0 2024-04-25T00:12:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f987b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f98740ce9e0 con 0x7f987415c340 2024-04-25T00:12:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f9879200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f987415ccf0 0x7f987414c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f9879200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f987415ccf0 0x7f987414c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39990/0 (socket says 172.21.15.83:39990) 2024-04-25T00:12:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f9879200640 1 -- 172.21.15.83:0/1623453006 learned_addr learned my addr 172.21.15.83:0/1623453006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f986b400640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f987415c340 0x7f98740b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f9873e00640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98741599f0 0x7f98741faa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f9879200640 1 -- 172.21.15.83:0/1623453006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f987415c340 msgr2=0x7f98740b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f9879200640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f987415c340 0x7f98740b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f9879200640 1 -- 172.21.15.83:0/1623453006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98741599f0 msgr2=0x7f98741faa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f9879200640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98741599f0 0x7f98741faa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f9879200640 1 -- 172.21.15.83:0/1623453006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9874155f00 con 0x7f987415ccf0 2024-04-25T00:12:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f9873e00640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98741599f0 0x7f98741faa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f9879200640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f987415ccf0 0x7f987414c130 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f986c007e60 tx=0x7f986c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f9870c00640 1 -- 172.21.15.83:0/1623453006 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c014420 con 0x7f987415ccf0 2024-04-25T00:12:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f9870c00640 1 -- 172.21.15.83:0/1623453006 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986c019070 con 0x7f987415ccf0 2024-04-25T00:12:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f9870c00640 1 -- 172.21.15.83:0/1623453006 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c014820 con 0x7f987415ccf0 2024-04-25T00:12:03.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f987b200640 1 -- 172.21.15.83:0/1623453006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98741561e0 con 0x7f987415ccf0 2024-04-25T00:12:03.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.898+0000 7f987b200640 1 -- 172.21.15.83:0/1623453006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98741567a0 con 0x7f987415ccf0 2024-04-25T00:12:03.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.902+0000 7f987b200640 1 -- 172.21.15.83:0/1623453006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9874166af0 con 0x7f987415ccf0 2024-04-25T00:12:03.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.902+0000 7f9870c00640 1 -- 172.21.15.83:0/1623453006 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f986c0149c0 con 0x7f987415ccf0 2024-04-25T00:12:03.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.902+0000 7f9870c00640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98500678c0 0x7f9850069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:03.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.902+0000 7f9870c00640 1 -- 172.21.15.83:0/1623453006 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f986c086780 con 0x7f987415ccf0 2024-04-25T00:12:03.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.902+0000 7f9873e00640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98500678c0 0x7f9850069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:03.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.902+0000 7f9873e00640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98500678c0 0x7f9850069d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9864004660 tx=0x7f9864008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:03.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:03.906+0000 7f9870c00640 1 -- 172.21.15.83:0/1623453006 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f986c056f50 con 0x7f987415ccf0 2024-04-25T00:12:04.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:12:04.056 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:12:04.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.098+0000 7f987b200640 1 -- 172.21.15.83:0/1623453006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f987404f5f0 con 0x7f987415ccf0 2024-04-25T00:12:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.098+0000 7f9870c00640 1 -- 172.21.15.83:0/1623453006 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7f986c056850 con 0x7f987415ccf0 2024-04-25T00:12:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:04.105 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/1335208265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1335208265},{"type":"v1","addr":"172.21.15.86:6834","nonce":1335208265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/911805587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":911805587},{"type":"v1","addr":"172.21.15.83:6836","nonce":911805587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2309157687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2309157687},{"type":"v1","addr":"172.21.15.83:6837","nonce":2309157687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:11:06.200989+0000","modified":"2024-04-25T00:11:23.462521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1603954817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":1603954817},{"type":"v1","addr":"172.21.15.86:6835","nonce":1603954817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T00:12:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2024-04-25T00:12:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.102+0000 7f987b200640 1 -- 172.21.15.83:0/1623453006 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98500678c0 msgr2=0x7f9850069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.102+0000 7f987b200640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98500678c0 0x7f9850069d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9864004660 tx=0x7f9864008040 comp rx=0 tx=0).stop 2024-04-25T00:12:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.102+0000 7f987b200640 1 -- 172.21.15.83:0/1623453006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f987415ccf0 msgr2=0x7f987414c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.102+0000 7f987b200640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f987415ccf0 0x7f987414c130 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f986c007e60 tx=0x7f986c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.102+0000 7f987b200640 1 -- 172.21.15.83:0/1623453006 shutdown_connections 2024-04-25T00:12:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.102+0000 7f987b200640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98500678c0 0x7f9850069d80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.102+0000 7f987b200640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f987415c340 0x7f98740b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.102+0000 7f987b200640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f987415ccf0 0x7f987414c130 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.102+0000 7f987b200640 1 --2- 172.21.15.83:0/1623453006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98741599f0 0x7f98741faa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.102+0000 7f987b200640 1 -- 172.21.15.83:0/1623453006 >> 172.21.15.83:0/1623453006 conn(0x7f98740b1830 msgr2=0x7f98740b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.102+0000 7f987b200640 1 -- 172.21.15.83:0/1623453006 shutdown_connections 2024-04-25T00:12:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.102+0000 7f987b200640 1 -- 172.21.15.83:0/1623453006 wait complete. 2024-04-25T00:12:04.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:04.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.298+0000 7f2f73e00640 1 Processor -- start 2024-04-25T00:12:04.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.298+0000 7f2f73e00640 1 -- start start 2024-04-25T00:12:04.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.302+0000 7f2f73e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2f6c0b4360 0x7f2f6c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.302+0000 7f2f73e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f6c0b2b30 0x7f2f6c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.302+0000 7f2f73e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f6c06ef80 con 0x7f2f6c1553e0 2024-04-25T00:12:04.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.302+0000 7f2f73e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2f6c15b510 con 0x7f2f6c0b4360 2024-04-25T00:12:04.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.302+0000 7f2f73e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6c15b7f0 con 0x7f2f6c0b2b30 2024-04-25T00:12:04.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.302+0000 7f2f70a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2f6c0b4360 0x7f2f6c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.302+0000 7f2f71400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2f6c1553e0 0x7f2f6c06e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46012/0 (socket says 172.21.15.83:46012) 2024-04-25T00:12:04.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.302+0000 7f2f71400640 1 -- 172.21.15.83:0/1977825319 learned_addr learned my addr 172.21.15.83:0/1977825319 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:04.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.302+0000 7f2f70a00640 1 -- 172.21.15.83:0/1977825319 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f6c0b2b30 msgr2=0x7f2f6c0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:04.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.302+0000 7f2f70a00640 1 --2- 172.21.15.83:0/1977825319 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f6c0b2b30 0x7f2f6c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.302+0000 7f2f70a00640 1 -- 172.21.15.83:0/1977825319 >> v1:172.21.15.83:6789/0 conn(0x7f2f6c1553e0 legacy=0x7f2f6c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.302+0000 7f2f70a00640 1 -- 172.21.15.83:0/1977825319 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c15bad0 con 0x7f2f6c0b4360 2024-04-25T00:12:04.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.302+0000 7f2f6be00640 1 -- 172.21.15.83:0/1977825319 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268995107 0 0) 0x7f2f6c06ef80 con 0x7f2f6c1553e0 2024-04-25T00:12:04.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.306+0000 7f2f70a00640 1 --2- 172.21.15.83:0/1977825319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f6c0b4360 0x7f2f6c0b47e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2f580097c0 tx=0x7f2f5801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52003021ccea1bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:04.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f6be00640 1 -- 172.21.15.83:0/1977825319 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f580b0070 con 0x7f2f6c0b4360 2024-04-25T00:12:04.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f6be00640 1 -- 172.21.15.83:0/1977825319 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f580ab600 con 0x7f2f6c0b4360 2024-04-25T00:12:04.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f6be00640 1 -- 172.21.15.83:0/1977825319 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f580a4e60 con 0x7f2f6c0b4360 2024-04-25T00:12:04.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 -- 172.21.15.83:0/1977825319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f6c0b4360 msgr2=0x7f2f6c0b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 --2- 172.21.15.83:0/1977825319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f6c0b4360 0x7f2f6c0b47e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2f580097c0 tx=0x7f2f5801c410 comp rx=0 tx=0).stop 2024-04-25T00:12:04.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 -- 172.21.15.83:0/1977825319 shutdown_connections 2024-04-25T00:12:04.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 --2- 172.21.15.83:0/1977825319 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f6c0b2b30 0x7f2f6c0b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 --2- 172.21.15.83:0/1977825319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f6c0b4360 0x7f2f6c0b47e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 -- 172.21.15.83:0/1977825319 >> 172.21.15.83:0/1977825319 conn(0x7f2f6c0b1830 msgr2=0x7f2f6c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:04.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 -- 172.21.15.83:0/1977825319 shutdown_connections 2024-04-25T00:12:04.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 -- 172.21.15.83:0/1977825319 wait complete. 2024-04-25T00:12:04.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 Processor -- start 2024-04-25T00:12:04.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 -- start start 2024-04-25T00:12:04.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f6c0b2b30 0x7f2f6c156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f6c0b4360 0x7f2f6c14dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f6c1553e0 0x7f2f6c158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c06ebd0 con 0x7f2f6c0b4360 2024-04-25T00:12:04.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c167d90 con 0x7f2f6c0b2b30 2024-04-25T00:12:04.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f73e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6c157370 con 0x7f2f6c1553e0 2024-04-25T00:12:04.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f70a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f6c0b4360 0x7f2f6c14dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f70a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f6c0b4360 0x7f2f6c14dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60476/0 (socket says 172.21.15.83:60476) 2024-04-25T00:12:04.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f70a00640 1 -- 172.21.15.83:0/1878742047 learned_addr learned my addr 172.21.15.83:0/1878742047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:04.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f71400640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f6c0b2b30 0x7f2f6c156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.310+0000 7f2f71e00640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f6c1553e0 0x7f2f6c158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f70a00640 1 -- 172.21.15.83:0/1878742047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f6c1553e0 msgr2=0x7f2f6c158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f70a00640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f6c1553e0 0x7f2f6c158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f70a00640 1 -- 172.21.15.83:0/1878742047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f6c0b2b30 msgr2=0x7f2f6c156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f70a00640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f6c0b2b30 0x7f2f6c156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f70a00640 1 -- 172.21.15.83:0/1878742047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c157650 con 0x7f2f6c0b4360 2024-04-25T00:12:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f71e00640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f6c1553e0 0x7f2f6c158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f70a00640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f6c0b4360 0x7f2f6c14dae0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2f5801aac0 tx=0x7f2f5801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f69600640 1 -- 172.21.15.83:0/1878742047 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f580b0050 con 0x7f2f6c0b4360 2024-04-25T00:12:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f69600640 1 -- 172.21.15.83:0/1878742047 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f580bc070 con 0x7f2f6c0b4360 2024-04-25T00:12:04.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f73e00640 1 -- 172.21.15.83:0/1878742047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c157930 con 0x7f2f6c0b4360 2024-04-25T00:12:04.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f69600640 1 -- 172.21.15.83:0/1878742047 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f580b5830 con 0x7f2f6c0b4360 2024-04-25T00:12:04.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f73e00640 1 -- 172.21.15.83:0/1878742047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c157ef0 con 0x7f2f6c0b4360 2024-04-25T00:12:04.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f73e00640 1 -- 172.21.15.83:0/1878742047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6c04f5f0 con 0x7f2f6c0b4360 2024-04-25T00:12:04.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f69600640 1 -- 172.21.15.83:0/1878742047 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2f580b59d0 con 0x7f2f6c0b4360 2024-04-25T00:12:04.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f69600640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2f4c0678c0 0x7f2f4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f69600640 1 -- 172.21.15.83:0/1878742047 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2f58120880 con 0x7f2f6c0b4360 2024-04-25T00:12:04.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.314+0000 7f2f71400640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2f4c0678c0 0x7f2f4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.318+0000 7f2f71400640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2f4c0678c0 0x7f2f4c069d80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2f600012f0 tx=0x7f2f60001770 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.318+0000 7f2f69600640 1 -- 172.21.15.83:0/1878742047 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2f58016090 con 0x7f2f6c0b4360 2024-04-25T00:12:04.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.482+0000 7f2f73e00640 1 -- 172.21.15.83:0/1878742047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f6c06ce10 con 0x7f2f6c0b4360 2024-04-25T00:12:04.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.482+0000 7f2f69600640 1 -- 172.21.15.83:0/1878742047 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18820 (secure 0 0 0) 0x7f2f580f0f20 con 0x7f2f6c0b4360 2024-04-25T00:12:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:04.488 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:48.235809+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:04.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.486+0000 7f2f73e00640 1 -- 172.21.15.83:0/1878742047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2f4c0678c0 msgr2=0x7f2f4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.486+0000 7f2f73e00640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2f4c0678c0 0x7f2f4c069d80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2f600012f0 tx=0x7f2f60001770 comp rx=0 tx=0).stop 2024-04-25T00:12:04.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.486+0000 7f2f73e00640 1 -- 172.21.15.83:0/1878742047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f6c0b4360 msgr2=0x7f2f6c14dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.486+0000 7f2f73e00640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f6c0b4360 0x7f2f6c14dae0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2f5801aac0 tx=0x7f2f5801ac50 comp rx=0 tx=0).stop 2024-04-25T00:12:04.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.486+0000 7f2f73e00640 1 -- 172.21.15.83:0/1878742047 shutdown_connections 2024-04-25T00:12:04.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.486+0000 7f2f73e00640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2f4c0678c0 0x7f2f4c069d80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.486+0000 7f2f73e00640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f6c1553e0 0x7f2f6c158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.486+0000 7f2f73e00640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f6c0b4360 0x7f2f6c14dae0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.486+0000 7f2f73e00640 1 --2- 172.21.15.83:0/1878742047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f6c0b2b30 0x7f2f6c156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.486+0000 7f2f73e00640 1 -- 172.21.15.83:0/1878742047 >> 172.21.15.83:0/1878742047 conn(0x7f2f6c0b1830 msgr2=0x7f2f6c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.486+0000 7f2f73e00640 1 -- 172.21.15.83:0/1878742047 shutdown_connections 2024-04-25T00:12:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.486+0000 7f2f73e00640 1 -- 172.21.15.83:0/1878742047 wait complete. 2024-04-25T00:12:04.509 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:11:04.621429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:11:05.189716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:11:08.902215+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:12:04.510 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:12:04.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71cdc00640 1 Processor -- start 2024-04-25T00:12:04.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71cdc00640 1 -- start start 2024-04-25T00:12:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71cdc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71c81551e0 0x7f71c815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71cdc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71c81555c0 0x7f71c8071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71cdc00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71c815c560 con 0x7f71c806e4c0 2024-04-25T00:12:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71cdc00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f71c8072460 con 0x7f71c81555c0 2024-04-25T00:12:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71cdc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f71c815c7d0 con 0x7f71c81551e0 2024-04-25T00:12:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71c6000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71c81555c0 0x7f71c8071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71c7400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f71c806e4c0 0x7f71c815e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46026/0 (socket says 172.21.15.83:46026) 2024-04-25T00:12:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71c7400640 1 -- 172.21.15.83:0/2505072851 learned_addr learned my addr 172.21.15.83:0/2505072851 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71c6a00640 1 --2- 172.21.15.83:0/2505072851 >> v2:172.21.15.86:3300/0 conn(0x7f71c81551e0 0x7f71c815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71c6000640 1 -- 172.21.15.83:0/2505072851 >> v2:172.21.15.86:3300/0 conn(0x7f71c81551e0 msgr2=0x7f71c815bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71c6000640 1 --2- 172.21.15.83:0/2505072851 >> v2:172.21.15.86:3300/0 conn(0x7f71c81551e0 0x7f71c815bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71c6000640 1 -- 172.21.15.83:0/2505072851 >> v1:172.21.15.83:6789/0 conn(0x7f71c806e4c0 legacy=0x7f71c815e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71c6000640 1 -- 172.21.15.83:0/2505072851 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c815cab0 con 0x7f71c81555c0 2024-04-25T00:12:04.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71c5600640 1 -- 172.21.15.83:0/2505072851 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865526348 0 0) 0x7f71c815c560 con 0x7f71c806e4c0 2024-04-25T00:12:04.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.646+0000 7f71c6a00640 1 --2- 172.21.15.83:0/2505072851 >> v2:172.21.15.86:3300/0 conn(0x7f71c81551e0 0x7f71c815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:04.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.650+0000 7f71c6000640 1 --2- 172.21.15.83:0/2505072851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c81555c0 0x7f71c8071b70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f71b80097c0 tx=0x7f71b801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a83ebccabcd73b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:04.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.650+0000 7f71c5600640 1 -- 172.21.15.83:0/2505072851 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b80b2070 con 0x7f71c81555c0 2024-04-25T00:12:04.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.650+0000 7f71c5600640 1 -- 172.21.15.83:0/2505072851 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b80ad680 con 0x7f71c81555c0 2024-04-25T00:12:04.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.650+0000 7f71c5600640 1 -- 172.21.15.83:0/2505072851 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b80ada50 con 0x7f71c81555c0 2024-04-25T00:12:04.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.650+0000 7f71cdc00640 1 -- 172.21.15.83:0/2505072851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c81555c0 msgr2=0x7f71c8071b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.650+0000 7f71cdc00640 1 --2- 172.21.15.83:0/2505072851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c81555c0 0x7f71c8071b70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f71b80097c0 tx=0x7f71b801c460 comp rx=0 tx=0).stop 2024-04-25T00:12:04.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.650+0000 7f71cdc00640 1 -- 172.21.15.83:0/2505072851 shutdown_connections 2024-04-25T00:12:04.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.650+0000 7f71cdc00640 1 --2- 172.21.15.83:0/2505072851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c81555c0 0x7f71c8071b70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.650+0000 7f71cdc00640 1 --2- 172.21.15.83:0/2505072851 >> v2:172.21.15.86:3300/0 conn(0x7f71c81551e0 0x7f71c815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.650+0000 7f71cdc00640 1 -- 172.21.15.83:0/2505072851 >> 172.21.15.83:0/2505072851 conn(0x7f71c80b1830 msgr2=0x7f71c80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:04.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.650+0000 7f71cdc00640 1 -- 172.21.15.83:0/2505072851 shutdown_connections 2024-04-25T00:12:04.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.650+0000 7f71cdc00640 1 -- 172.21.15.83:0/2505072851 wait complete. 2024-04-25T00:12:04.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.650+0000 7f71cdc00640 1 Processor -- start 2024-04-25T00:12:04.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.650+0000 7f71cdc00640 1 -- start start 2024-04-25T00:12:04.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71cdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c806e4c0 0x7f71c81f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71cdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c81551e0 0x7f71c814e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71cdc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f71c81555c0 0x7f71c8159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71cdc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71c806d530 con 0x7f71c806e4c0 2024-04-25T00:12:04.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71cdc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71c806d3b0 con 0x7f71c81551e0 2024-04-25T00:12:04.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71cdc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f71c806d6b0 con 0x7f71c81555c0 2024-04-25T00:12:04.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71c6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c806e4c0 0x7f71c81f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71c6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c806e4c0 0x7f71c81f7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60490/0 (socket says 172.21.15.83:60490) 2024-04-25T00:12:04.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71c6a00640 1 -- 172.21.15.83:0/886155292 learned_addr learned my addr 172.21.15.83:0/886155292 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:04.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71c6a00640 1 -- 172.21.15.83:0/886155292 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f71c81555c0 msgr2=0x7f71c8159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71c6a00640 1 --2- 172.21.15.83:0/886155292 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f71c81555c0 0x7f71c8159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71c6a00640 1 -- 172.21.15.83:0/886155292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c81551e0 msgr2=0x7f71c814e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71c6a00640 1 --2- 172.21.15.83:0/886155292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c81551e0 0x7f71c814e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71c6a00640 1 -- 172.21.15.83:0/886155292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c80cfda0 con 0x7f71c806e4c0 2024-04-25T00:12:04.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71c6a00640 1 --2- 172.21.15.83:0/886155292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c806e4c0 0x7f71c81f7200 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f71b000e810 tx=0x7f71b000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:04.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71bea00640 1 -- 172.21.15.83:0/886155292 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0017070 con 0x7f71c806e4c0 2024-04-25T00:12:04.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71bea00640 1 -- 172.21.15.83:0/886155292 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b000c8d0 con 0x7f71c806e4c0 2024-04-25T00:12:04.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71bea00640 1 -- 172.21.15.83:0/886155292 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b000f040 con 0x7f71c806e4c0 2024-04-25T00:12:04.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71cdc00640 1 -- 172.21.15.83:0/886155292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c814db90 con 0x7f71c806e4c0 2024-04-25T00:12:04.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71cdc00640 1 -- 172.21.15.83:0/886155292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c80cd740 con 0x7f71c806e4c0 2024-04-25T00:12:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71cdc00640 1 -- 172.21.15.83:0/886155292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c804f510 con 0x7f71c806e4c0 2024-04-25T00:12:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.654+0000 7f71bea00640 1 -- 172.21.15.83:0/886155292 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f71b001a070 con 0x7f71c806e4c0 2024-04-25T00:12:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.658+0000 7f71bea00640 1 --2- 172.21.15.83:0/886155292 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f71a40678f0 0x7f71a4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.658+0000 7f71bea00640 1 -- 172.21.15.83:0/886155292 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f71b0085c50 con 0x7f71c806e4c0 2024-04-25T00:12:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.658+0000 7f71c6000640 1 --2- 172.21.15.83:0/886155292 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f71a40678f0 0x7f71a4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.658+0000 7f71c6000640 1 --2- 172.21.15.83:0/886155292 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f71a40678f0 0x7f71a4069db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f71b80af000 tx=0x7f71b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:04.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.662+0000 7f71bea00640 1 -- 172.21.15.83:0/886155292 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f71b0056380 con 0x7f71c806e4c0 2024-04-25T00:12:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.814+0000 7f71cdc00640 1 -- 172.21.15.83:0/886155292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f71c806ce30 con 0x7f71c806e4c0 2024-04-25T00:12:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.814+0000 7f71bea00640 1 -- 172.21.15.83:0/886155292 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f71b0055c80 con 0x7f71c806e4c0 2024-04-25T00:12:04.821 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:12:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.818+0000 7f71cdc00640 1 -- 172.21.15.83:0/886155292 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f71a40678f0 msgr2=0x7f71a4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.818+0000 7f71cdc00640 1 --2- 172.21.15.83:0/886155292 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f71a40678f0 0x7f71a4069db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f71b80af000 tx=0x7f71b8008040 comp rx=0 tx=0).stop 2024-04-25T00:12:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.818+0000 7f71cdc00640 1 -- 172.21.15.83:0/886155292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c806e4c0 msgr2=0x7f71c81f7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.818+0000 7f71cdc00640 1 --2- 172.21.15.83:0/886155292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c806e4c0 0x7f71c81f7200 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f71b000e810 tx=0x7f71b000ecd0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.818+0000 7f71cdc00640 1 -- 172.21.15.83:0/886155292 shutdown_connections 2024-04-25T00:12:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.818+0000 7f71cdc00640 1 --2- 172.21.15.83:0/886155292 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f71a40678f0 0x7f71a4069db0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.818+0000 7f71cdc00640 1 --2- 172.21.15.83:0/886155292 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f71c81555c0 0x7f71c8159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.818+0000 7f71cdc00640 1 --2- 172.21.15.83:0/886155292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c81551e0 0x7f71c814e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.818+0000 7f71cdc00640 1 --2- 172.21.15.83:0/886155292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c806e4c0 0x7f71c81f7200 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.818+0000 7f71cdc00640 1 -- 172.21.15.83:0/886155292 >> 172.21.15.83:0/886155292 conn(0x7f71c80b1830 msgr2=0x7f71c80afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.818+0000 7f71cdc00640 1 -- 172.21.15.83:0/886155292 shutdown_connections 2024-04-25T00:12:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.818+0000 7f71cdc00640 1 -- 172.21.15.83:0/886155292 wait complete. 2024-04-25T00:12:04.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:12:04.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.974+0000 7f9907400640 1 Processor -- start 2024-04-25T00:12:04.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.974+0000 7f9907400640 1 -- start start 2024-04-25T00:12:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.974+0000 7f9907400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9900159810 0x7f99000cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.974+0000 7f9907400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9900160020 0x7f99000ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.974+0000 7f9907400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99000ce000 con 0x7f990006e950 2024-04-25T00:12:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.974+0000 7f9907400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f990006e210 con 0x7f9900159810 2024-04-25T00:12:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.974+0000 7f9907400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f990006da00 con 0x7f9900160020 2024-04-25T00:12:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.974+0000 7f98ffe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f990006e950 0x7f990015f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46038/0 (socket says 172.21.15.83:46038) 2024-04-25T00:12:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.974+0000 7f98ffe00640 1 -- 172.21.15.83:0/4207419207 learned_addr learned my addr 172.21.15.83:0/4207419207 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.974+0000 7f9904a00640 1 --2- 172.21.15.83:0/4207419207 >> v2:172.21.15.83:3300/0 conn(0x7f9900159810 0x7f99000cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.978+0000 7f9905400640 1 --2- 172.21.15.83:0/4207419207 >> v2:172.21.15.86:3300/0 conn(0x7f9900160020 0x7f99000ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.978+0000 7f98ff400640 1 -- 172.21.15.83:0/4207419207 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2438904278 0 0) 0x7f99000ce000 con 0x7f990006e950 2024-04-25T00:12:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.978+0000 7f98ff400640 1 -- 172.21.15.83:0/4207419207 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98e8003530 con 0x7f990006e950 2024-04-25T00:12:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.978+0000 7f9904a00640 1 -- 172.21.15.83:0/4207419207 >> v2:172.21.15.86:3300/0 conn(0x7f9900160020 msgr2=0x7f99000ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.978+0000 7f9904a00640 1 --2- 172.21.15.83:0/4207419207 >> v2:172.21.15.86:3300/0 conn(0x7f9900160020 0x7f99000ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.978+0000 7f9904a00640 1 -- 172.21.15.83:0/4207419207 >> v1:172.21.15.83:6789/0 conn(0x7f990006e950 legacy=0x7f990015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.978+0000 7f9904a00640 1 -- 172.21.15.83:0/4207419207 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990006eec0 con 0x7f9900159810 2024-04-25T00:12:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.978+0000 7f9904a00640 1 --2- 172.21.15.83:0/4207419207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9900159810 0x7f99000cc4a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f990006db80 tx=0x7f98f401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61f370f9e35a0fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:04.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.978+0000 7f98ff400640 1 -- 172.21.15.83:0/4207419207 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f40b1070 con 0x7f9900159810 2024-04-25T00:12:04.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.978+0000 7f98ff400640 1 -- 172.21.15.83:0/4207419207 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f40ac600 con 0x7f9900159810 2024-04-25T00:12:04.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.978+0000 7f98ff400640 1 -- 172.21.15.83:0/4207419207 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f40ac9d0 con 0x7f9900159810 2024-04-25T00:12:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 -- 172.21.15.83:0/4207419207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9900159810 msgr2=0x7f99000cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 --2- 172.21.15.83:0/4207419207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9900159810 0x7f99000cc4a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f990006db80 tx=0x7f98f401c3a0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 -- 172.21.15.83:0/4207419207 shutdown_connections 2024-04-25T00:12:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 --2- 172.21.15.83:0/4207419207 >> v2:172.21.15.86:3300/0 conn(0x7f9900160020 0x7f99000ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 --2- 172.21.15.83:0/4207419207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9900159810 0x7f99000cc4a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 -- 172.21.15.83:0/4207419207 >> 172.21.15.83:0/4207419207 conn(0x7f99000b1830 msgr2=0x7f99000b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 -- 172.21.15.83:0/4207419207 shutdown_connections 2024-04-25T00:12:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 -- 172.21.15.83:0/4207419207 wait complete. 2024-04-25T00:12:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 Processor -- start 2024-04-25T00:12:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 -- start start 2024-04-25T00:12:04.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f990006e950 0x7f99001568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9900159810 0x7f9900154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9900160020 0x7f9900153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9900159d70 con 0x7f9900159810 2024-04-25T00:12:04.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9900159bf0 con 0x7f990006e950 2024-04-25T00:12:04.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9900070da0 con 0x7f9900160020 2024-04-25T00:12:04.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f98ffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9900159810 0x7f9900154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f98ffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9900159810 0x7f9900154650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60504/0 (socket says 172.21.15.83:60504) 2024-04-25T00:12:04.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f98ffe00640 1 -- 172.21.15.83:0/2680038620 learned_addr learned my addr 172.21.15.83:0/2680038620 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:04.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9904a00640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f990006e950 0x7f99001568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9905400640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9900160020 0x7f9900153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f98ffe00640 1 -- 172.21.15.83:0/2680038620 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9900160020 msgr2=0x7f9900153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f98ffe00640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9900160020 0x7f9900153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f98ffe00640 1 -- 172.21.15.83:0/2680038620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f990006e950 msgr2=0x7f99001568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f98ffe00640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f990006e950 0x7f99001568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f98ffe00640 1 -- 172.21.15.83:0/2680038620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990015f410 con 0x7f9900159810 2024-04-25T00:12:04.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9904a00640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f990006e950 0x7f99001568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:04.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f98ffe00640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9900159810 0x7f9900154650 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00c3c0 tx=0x7f98ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:04.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f98fcc00640 1 -- 172.21.15.83:0/2680038620 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec017070 con 0x7f9900159810 2024-04-25T00:12:04.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f98fcc00640 1 -- 172.21.15.83:0/2680038620 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ec005cf0 con 0x7f9900159810 2024-04-25T00:12:04.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.982+0000 7f9907400640 1 -- 172.21.15.83:0/2680038620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9900152d20 con 0x7f9900159810 2024-04-25T00:12:04.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.986+0000 7f98fcc00640 1 -- 172.21.15.83:0/2680038620 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0072f0 con 0x7f9900159810 2024-04-25T00:12:04.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.986+0000 7f9907400640 1 -- 172.21.15.83:0/2680038620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99001532e0 con 0x7f9900159810 2024-04-25T00:12:04.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.986+0000 7f9907400640 1 -- 172.21.15.83:0/2680038620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990004f510 con 0x7f9900159810 2024-04-25T00:12:04.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.986+0000 7f98fcc00640 1 -- 172.21.15.83:0/2680038620 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f98ec01a070 con 0x7f9900159810 2024-04-25T00:12:04.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.986+0000 7f98fcc00640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98e00678c0 0x7f98e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.986+0000 7f98fcc00640 1 -- 172.21.15.83:0/2680038620 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f98ec085450 con 0x7f9900159810 2024-04-25T00:12:04.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.986+0000 7f9904a00640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98e00678c0 0x7f98e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.986+0000 7f9904a00640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98e00678c0 0x7f98e0069d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f98f40ae000 tx=0x7f98f40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:04.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:04.990+0000 7f98fcc00640 1 -- 172.21.15.83:0/2680038620 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f98ec055b80 con 0x7f9900159810 2024-04-25T00:12:05.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.146+0000 7f9907400640 1 -- 172.21.15.83:0/2680038620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9900072c90 con 0x7f9900159810 2024-04-25T00:12:05.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.146+0000 7f98fcc00640 1 -- 172.21.15.83:0/2680038620 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f98ec055480 con 0x7f9900159810 2024-04-25T00:12:05.150 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:12:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.146+0000 7f9907400640 1 -- 172.21.15.83:0/2680038620 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98e00678c0 msgr2=0x7f98e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.146+0000 7f9907400640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98e00678c0 0x7f98e0069d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f98f40ae000 tx=0x7f98f40a7000 comp rx=0 tx=0).stop 2024-04-25T00:12:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.146+0000 7f9907400640 1 -- 172.21.15.83:0/2680038620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9900159810 msgr2=0x7f9900154650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.146+0000 7f9907400640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9900159810 0x7f9900154650 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00c3c0 tx=0x7f98ec00c880 comp rx=0 tx=0).stop 2024-04-25T00:12:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.146+0000 7f9907400640 1 -- 172.21.15.83:0/2680038620 shutdown_connections 2024-04-25T00:12:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.146+0000 7f9907400640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98e00678c0 0x7f98e0069d80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.146+0000 7f9907400640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9900160020 0x7f9900153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.146+0000 7f9907400640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9900159810 0x7f9900154650 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.146+0000 7f9907400640 1 --2- 172.21.15.83:0/2680038620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f990006e950 0x7f99001568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.146+0000 7f9907400640 1 -- 172.21.15.83:0/2680038620 >> 172.21.15.83:0/2680038620 conn(0x7f99000b1830 msgr2=0x7f99000d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:05.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.146+0000 7f9907400640 1 -- 172.21.15.83:0/2680038620 shutdown_connections 2024-04-25T00:12:05.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.146+0000 7f9907400640 1 -- 172.21.15.83:0/2680038620 wait complete. 2024-04-25T00:12:05.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:12:05.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.306+0000 7f43a0800640 1 Processor -- start 2024-04-25T00:12:05.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.306+0000 7f43a0800640 1 -- start start 2024-04-25T00:12:05.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.306+0000 7f43a0800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f439815e040 0x7f43980d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.306+0000 7f43a0800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43980d1150 con 0x7f439806cc10 2024-04-25T00:12:05.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.306+0000 7f43a0800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43980cc530 con 0x7f439815e420 2024-04-25T00:12:05.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.306+0000 7f43a0800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f43980d0f20 con 0x7f439815e040 2024-04-25T00:12:05.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.306+0000 7f439e800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f439806cc10 0x7f43980cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46044/0 (socket says 172.21.15.83:46044) 2024-04-25T00:12:05.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.306+0000 7f439e800640 1 -- 172.21.15.83:0/4263093975 learned_addr learned my addr 172.21.15.83:0/4263093975 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:05.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.306+0000 7f439de00640 1 --2- 172.21.15.83:0/4263093975 >> v2:172.21.15.86:3300/0 conn(0x7f439815e040 0x7f43980d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.306+0000 7f439de00640 1 -- 172.21.15.83:0/4263093975 >> v1:172.21.15.86:6789/0 conn(0x7f439815e420 legacy=0x7f439806c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.306+0000 7f439de00640 1 -- 172.21.15.83:0/4263093975 >> v1:172.21.15.83:6789/0 conn(0x7f439806cc10 legacy=0x7f43980cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.306+0000 7f439de00640 1 -- 172.21.15.83:0/4263093975 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43980cc810 con 0x7f439815e040 2024-04-25T00:12:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.310+0000 7f439de00640 1 --2- 172.21.15.83:0/4263093975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439815e040 0x7f43980d09e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f438c0097c0 tx=0x7f438c01ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80413fd076863139 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.310+0000 7f439ca00640 1 -- 172.21.15.83:0/4263093975 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c0b5070 con 0x7f439815e040 2024-04-25T00:12:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.310+0000 7f439ca00640 1 -- 172.21.15.83:0/4263093975 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f438c0a8990 con 0x7f439815e040 2024-04-25T00:12:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.310+0000 7f439ca00640 1 -- 172.21.15.83:0/4263093975 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c0a8d60 con 0x7f439815e040 2024-04-25T00:12:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.310+0000 7f43a0800640 1 -- 172.21.15.83:0/4263093975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439815e040 msgr2=0x7f43980d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.310+0000 7f43a0800640 1 --2- 172.21.15.83:0/4263093975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439815e040 0x7f43980d09e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f438c0097c0 tx=0x7f438c01ec90 comp rx=0 tx=0).stop 2024-04-25T00:12:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.310+0000 7f43a0800640 1 -- 172.21.15.83:0/4263093975 shutdown_connections 2024-04-25T00:12:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.310+0000 7f43a0800640 1 --2- 172.21.15.83:0/4263093975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439815e040 0x7f43980d09e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.310+0000 7f43a0800640 1 -- 172.21.15.83:0/4263093975 >> 172.21.15.83:0/4263093975 conn(0x7f43980d1c90 msgr2=0x7f43980d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.310+0000 7f43a0800640 1 -- 172.21.15.83:0/4263093975 shutdown_connections 2024-04-25T00:12:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.310+0000 7f43a0800640 1 -- 172.21.15.83:0/4263093975 wait complete. 2024-04-25T00:12:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f43a0800640 1 Processor -- start 2024-04-25T00:12:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f43a0800640 1 -- start start 2024-04-25T00:12:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f43a0800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f439806cc10 0x7f43980c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f43a0800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439815e040 0x7f439814f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f43a0800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439815e420 0x7f439814fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f43a0800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43980cc2e0 con 0x7f439806cc10 2024-04-25T00:12:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f43a0800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f439806d2d0 con 0x7f439815e420 2024-04-25T00:12:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f43a0800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f43980ccb80 con 0x7f439815e040 2024-04-25T00:12:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f439de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f439806cc10 0x7f43980c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f439de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f439806cc10 0x7f43980c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60516/0 (socket says 172.21.15.83:60516) 2024-04-25T00:12:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f439de00640 1 -- 172.21.15.83:0/1927806030 learned_addr learned my addr 172.21.15.83:0/1927806030 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f439e800640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439815e420 0x7f439814fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f439d400640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439815e040 0x7f439814f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f439de00640 1 -- 172.21.15.83:0/1927806030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439815e040 msgr2=0x7f439814f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f439de00640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439815e040 0x7f439814f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f439de00640 1 -- 172.21.15.83:0/1927806030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439815e420 msgr2=0x7f439814fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f439de00640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439815e420 0x7f439814fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f439de00640 1 -- 172.21.15.83:0/1927806030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439809d9a0 con 0x7f439806cc10 2024-04-25T00:12:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f439e800640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439815e420 0x7f439814fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f439de00640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f439806cc10 0x7f43980c9fa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f438c0b3040 tx=0x7f438c01ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f4392000640 1 -- 172.21.15.83:0/1927806030 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c0bf070 con 0x7f439806cc10 2024-04-25T00:12:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f4392000640 1 -- 172.21.15.83:0/1927806030 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f438c0b0900 con 0x7f439806cc10 2024-04-25T00:12:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f43a0800640 1 -- 172.21.15.83:0/1927806030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439809dc80 con 0x7f439806cc10 2024-04-25T00:12:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f4392000640 1 -- 172.21.15.83:0/1927806030 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c0b5040 con 0x7f439806cc10 2024-04-25T00:12:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f43a0800640 1 -- 172.21.15.83:0/1927806030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439809e520 con 0x7f439806cc10 2024-04-25T00:12:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.314+0000 7f43a0800640 1 -- 172.21.15.83:0/1927806030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f439809e070 con 0x7f439806cc10 2024-04-25T00:12:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.318+0000 7f4392000640 1 -- 172.21.15.83:0/1927806030 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f438c0c2070 con 0x7f439806cc10 2024-04-25T00:12:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.318+0000 7f4392000640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f437c0678c0 0x7f437c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.318+0000 7f4392000640 1 -- 172.21.15.83:0/1927806030 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f438c122990 con 0x7f439806cc10 2024-04-25T00:12:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.318+0000 7f439d400640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f437c0678c0 0x7f437c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.318+0000 7f439d400640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f437c0678c0 0x7f437c069d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f43980b6630 tx=0x7f4388009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.322+0000 7f4392000640 1 -- 172.21.15.83:0/1927806030 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f438c0187f0 con 0x7f439806cc10 2024-04-25T00:12:05.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.478+0000 7f43a0800640 1 -- 172.21.15.83:0/1927806030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f439809de50 con 0x7f439806cc10 2024-04-25T00:12:05.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.478+0000 7f4392000640 1 -- 172.21.15.83:0/1927806030 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f438c0f30c0 con 0x7f439806cc10 2024-04-25T00:12:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:12:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.478+0000 7f43a0800640 1 -- 172.21.15.83:0/1927806030 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f437c0678c0 msgr2=0x7f437c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.478+0000 7f43a0800640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f437c0678c0 0x7f437c069d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f43980b6630 tx=0x7f4388009040 comp rx=0 tx=0).stop 2024-04-25T00:12:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.478+0000 7f43a0800640 1 -- 172.21.15.83:0/1927806030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f439806cc10 msgr2=0x7f43980c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.478+0000 7f43a0800640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f439806cc10 0x7f43980c9fa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f438c0b3040 tx=0x7f438c01ba30 comp rx=0 tx=0).stop 2024-04-25T00:12:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.478+0000 7f43a0800640 1 -- 172.21.15.83:0/1927806030 shutdown_connections 2024-04-25T00:12:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.478+0000 7f43a0800640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f437c0678c0 0x7f437c069d80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.478+0000 7f43a0800640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439815e040 0x7f439814f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.478+0000 7f43a0800640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439815e420 0x7f439814fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.478+0000 7f43a0800640 1 --2- 172.21.15.83:0/1927806030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f439806cc10 0x7f43980c9fa0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.478+0000 7f43a0800640 1 -- 172.21.15.83:0/1927806030 >> 172.21.15.83:0/1927806030 conn(0x7f43980d1c90 msgr2=0x7f43980d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.478+0000 7f43a0800640 1 -- 172.21.15.83:0/1927806030 shutdown_connections 2024-04-25T00:12:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.478+0000 7f43a0800640 1 -- 172.21.15.83:0/1927806030 wait complete. 2024-04-25T00:12:05.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:12:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2027200640 1 Processor -- start 2024-04-25T00:12:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2027200640 1 -- start start 2024-04-25T00:12:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2027200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f202815e0e0 0x7f20280d4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2027200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20280cf4c0 con 0x7f20280b2c10 2024-04-25T00:12:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2027200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20280cf760 con 0x7f20280d2a60 2024-04-25T00:12:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2027200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f20280b0b70 con 0x7f202815e0e0 2024-04-25T00:12:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f201d400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f20280b2c10 0x7f20280d2350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46058/0 (socket says 172.21.15.83:46058) 2024-04-25T00:12:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f201d400640 1 -- 172.21.15.83:0/2620146613 learned_addr learned my addr 172.21.15.83:0/2620146613 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025e00640 1 --2- 172.21.15.83:0/2620146613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f202815e0e0 0x7f20280d4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3058489653 0 0) 0x7f20280cf4c0 con 0x7f20280b2c10 2024-04-25T00:12:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f200c003530 con 0x7f20280b2c10 2024-04-25T00:12:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3225930331 0 0) 0x7f20280cf760 con 0x7f20280d2a60 2024-04-25T00:12:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20280cf4c0 con 0x7f20280d2a60 2024-04-25T00:12:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1978464288 0 0) 0x7f200c003530 con 0x7f20280b2c10 2024-04-25T00:12:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f202815e0e0 msgr2=0x7f20280d4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 --2- 172.21.15.83:0/2620146613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f202815e0e0 0x7f20280d4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 >> v1:172.21.15.86:6789/0 conn(0x7f20280d2a60 legacy=0x7f20280b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20280cfa40 con 0x7f20280b2c10 2024-04-25T00:12:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f2010001680 con 0x7f20280b2c10 2024-04-25T00:12:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 >> v1:172.21.15.83:6789/0 conn(0x7f20280b2c10 legacy=0x7f20280d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 --2- 172.21.15.83:0/2620146613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c003b80 0x7f200c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 --2- 172.21.15.83:0/2620146613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f202815e0e0 0x7f200c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 --2- 172.21.15.83:0/2620146613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200c004d50 0x7f200c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20280b0b70 con 0x7f200c003b80 2024-04-25T00:12:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f200c00ba80 con 0x7f200c004d50 2024-04-25T00:12:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f200c00bc80 con 0x7f202815e0e0 2024-04-25T00:12:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025e00640 1 --2- 172.21.15.83:0/2620146613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c003b80 0x7f200c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f201d400640 1 --2- 172.21.15.83:0/2620146613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f202815e0e0 0x7f200c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2026800640 1 --2- 172.21.15.83:0/2620146613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200c004d50 0x7f200c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025e00640 1 -- 172.21.15.83:0/2620146613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f202815e0e0 msgr2=0x7f200c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025e00640 1 --2- 172.21.15.83:0/2620146613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f202815e0e0 0x7f200c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025e00640 1 -- 172.21.15.83:0/2620146613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200c004d50 msgr2=0x7f200c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025e00640 1 --2- 172.21.15.83:0/2620146613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200c004d50 0x7f200c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.642+0000 7f2025e00640 1 -- 172.21.15.83:0/2620146613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c003530 con 0x7f200c003b80 2024-04-25T00:12:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.646+0000 7f2025e00640 1 --2- 172.21.15.83:0/2620146613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c003b80 0x7f200c003f80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2018009ae0 tx=0x7f201801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5880f4f412a10be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.646+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180096b0 con 0x7f200c003b80 2024-04-25T00:12:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.646+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20180a6450 con 0x7f200c003b80 2024-04-25T00:12:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.646+0000 7f2025400640 1 -- 172.21.15.83:0/2620146613 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180a6840 con 0x7f200c003b80 2024-04-25T00:12:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.646+0000 7f2027200640 1 -- 172.21.15.83:0/2620146613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c003b80 msgr2=0x7f200c003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.646+0000 7f2027200640 1 --2- 172.21.15.83:0/2620146613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c003b80 0x7f200c003f80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2018009ae0 tx=0x7f201801ca70 comp rx=0 tx=0).stop 2024-04-25T00:12:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.646+0000 7f2026800640 1 -- 172.21.15.83:0/2620146613 reap_dead start 2024-04-25T00:12:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.646+0000 7f2027200640 1 -- 172.21.15.83:0/2620146613 shutdown_connections 2024-04-25T00:12:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.646+0000 7f2027200640 1 -- 172.21.15.83:0/2620146613 >> 172.21.15.83:0/2620146613 conn(0x7f20280d3d70 msgr2=0x7f20280d4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.646+0000 7f2027200640 1 -- 172.21.15.83:0/2620146613 shutdown_connections 2024-04-25T00:12:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.646+0000 7f2027200640 1 -- 172.21.15.83:0/2620146613 wait complete. 2024-04-25T00:12:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.646+0000 7f2027200640 1 Processor -- start 2024-04-25T00:12:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.646+0000 7f2027200640 1 -- start start 2024-04-25T00:12:05.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f2027200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200c003b80 0x7f20280cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f2027200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20280b5380 0x7f20280cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f2027200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f20280b4040 0x7f20280b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f2027200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20280b22b0 con 0x7f20280b5380 2024-04-25T00:12:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f2027200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20280b2130 con 0x7f200c003b80 2024-04-25T00:12:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f2027200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2028169ca0 con 0x7f20280b4040 2024-04-25T00:12:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f201d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20280b5380 0x7f20280cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f201d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20280b5380 0x7f20280cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60532/0 (socket says 172.21.15.83:60532) 2024-04-25T00:12:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f201d400640 1 -- 172.21.15.83:0/1918758195 learned_addr learned my addr 172.21.15.83:0/1918758195 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f2025e00640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200c003b80 0x7f20280cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f2026800640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f20280b4040 0x7f20280b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f201d400640 1 -- 172.21.15.83:0/1918758195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f20280b4040 msgr2=0x7f20280b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f201d400640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f20280b4040 0x7f20280b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f201d400640 1 -- 172.21.15.83:0/1918758195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200c003b80 msgr2=0x7f20280cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f201d400640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200c003b80 0x7f20280cbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f201d400640 1 -- 172.21.15.83:0/1918758195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20280cc0e0 con 0x7f20280b5380 2024-04-25T00:12:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f2026800640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f20280b4040 0x7f20280b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:05.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f201d400640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20280b5380 0x7f20280cd7e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f201000e7b0 tx=0x7f201000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:05.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f201ea00640 1 -- 172.21.15.83:0/1918758195 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201000ee60 con 0x7f20280b5380 2024-04-25T00:12:05.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f201ea00640 1 -- 172.21.15.83:0/1918758195 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2010018070 con 0x7f20280b5380 2024-04-25T00:12:05.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f201ea00640 1 -- 172.21.15.83:0/1918758195 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201000cb10 con 0x7f20280b5380 2024-04-25T00:12:05.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f2027200640 1 -- 172.21.15.83:0/1918758195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20280cc3c0 con 0x7f20280b5380 2024-04-25T00:12:05.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f2027200640 1 -- 172.21.15.83:0/1918758195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20280cc980 con 0x7f20280b5380 2024-04-25T00:12:05.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f2027200640 1 -- 172.21.15.83:0/1918758195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202804f510 con 0x7f20280b5380 2024-04-25T00:12:05.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.650+0000 7f201ea00640 1 -- 172.21.15.83:0/1918758195 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f201000ccb0 con 0x7f20280b5380 2024-04-25T00:12:05.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.654+0000 7f201ea00640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f20040678f0 0x7f2004069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.654+0000 7f201ea00640 1 -- 172.21.15.83:0/1918758195 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2010085ec0 con 0x7f20280b5380 2024-04-25T00:12:05.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.654+0000 7f2025e00640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f20040678f0 0x7f2004069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.654+0000 7f2025e00640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f20040678f0 0x7f2004069db0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f20180ad000 tx=0x7f20180046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.658+0000 7f201ea00640 1 -- 172.21.15.83:0/1918758195 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2010056560 con 0x7f20280b5380 2024-04-25T00:12:05.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.814+0000 7f2027200640 1 -- 172.21.15.83:0/1918758195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f20280ccb80 con 0x7f20280b5380 2024-04-25T00:12:05.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.814+0000 7f201ea00640 1 -- 172.21.15.83:0/1918758195 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f2010055e60 con 0x7f20280b5380 2024-04-25T00:12:05.820 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:12:05.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.818+0000 7f2027200640 1 -- 172.21.15.83:0/1918758195 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f20040678f0 msgr2=0x7f2004069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.818+0000 7f2027200640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f20040678f0 0x7f2004069db0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f20180ad000 tx=0x7f20180046d0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.818+0000 7f2027200640 1 -- 172.21.15.83:0/1918758195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20280b5380 msgr2=0x7f20280cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.818+0000 7f2027200640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20280b5380 0x7f20280cd7e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f201000e7b0 tx=0x7f201000ec70 comp rx=0 tx=0).stop 2024-04-25T00:12:05.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.818+0000 7f2027200640 1 -- 172.21.15.83:0/1918758195 shutdown_connections 2024-04-25T00:12:05.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.818+0000 7f2027200640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f20040678f0 0x7f2004069db0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.818+0000 7f2027200640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f20280b4040 0x7f20280b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.818+0000 7f2027200640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20280b5380 0x7f20280cd7e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.818+0000 7f2027200640 1 --2- 172.21.15.83:0/1918758195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200c003b80 0x7f20280cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.818+0000 7f2027200640 1 -- 172.21.15.83:0/1918758195 >> 172.21.15.83:0/1918758195 conn(0x7f20280d3d70 msgr2=0x7f20280b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:05.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.818+0000 7f2027200640 1 -- 172.21.15.83:0/1918758195 shutdown_connections 2024-04-25T00:12:05.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.818+0000 7f2027200640 1 -- 172.21.15.83:0/1918758195 wait complete. 2024-04-25T00:12:05.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:12:05.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 Processor -- start 2024-04-25T00:12:05.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 -- start start 2024-04-25T00:12:05.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf3c1599f0 0x7fbf3c0cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf3c15ce90 0x7fbf3c0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf3c0ccc60 con 0x7fbf3c15c340 2024-04-25T00:12:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbf3c159dd0 con 0x7fbf3c15ce90 2024-04-25T00:12:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3c06dba0 con 0x7fbf3c1599f0 2024-04-25T00:12:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf3aa00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbf3c15c340 0x7fbf3c15c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46066/0 (socket says 172.21.15.83:46066) 2024-04-25T00:12:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf3aa00640 1 -- 172.21.15.83:0/2925810449 learned_addr learned my addr 172.21.15.83:0/2925810449 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf3be00640 1 --2- 172.21.15.83:0/2925810449 >> v2:172.21.15.83:3300/0 conn(0x7fbf3c15ce90 0x7fbf3c0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf3b400640 1 --2- 172.21.15.83:0/2925810449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf3c1599f0 0x7fbf3c0cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf3a000640 1 -- 172.21.15.83:0/2925810449 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1657575407 0 0) 0x7fbf28001120 con 0x7fbf3c15c340 2024-04-25T00:12:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf3a000640 1 -- 172.21.15.83:0/2925810449 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf20003530 con 0x7fbf3c15c340 2024-04-25T00:12:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf3be00640 1 -- 172.21.15.83:0/2925810449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf3c1599f0 msgr2=0x7fbf3c0cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf3be00640 1 --2- 172.21.15.83:0/2925810449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf3c1599f0 0x7fbf3c0cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf3be00640 1 -- 172.21.15.83:0/2925810449 >> v1:172.21.15.83:6789/0 conn(0x7fbf3c15c340 legacy=0x7fbf3c15c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf3be00640 1 -- 172.21.15.83:0/2925810449 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c06de80 con 0x7fbf3c15ce90 2024-04-25T00:12:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf3be00640 1 --2- 172.21.15.83:0/2925810449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3c15ce90 0x7fbf3c0ce6e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf340097f0 tx=0x7fbf3401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e559ed43d06bdd67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf3a000640 1 -- 172.21.15.83:0/2925810449 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340b0070 con 0x7fbf3c15ce90 2024-04-25T00:12:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf3a000640 1 -- 172.21.15.83:0/2925810449 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf340ab600 con 0x7fbf3c15ce90 2024-04-25T00:12:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf3a000640 1 -- 172.21.15.83:0/2925810449 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340ab9d0 con 0x7fbf3c15ce90 2024-04-25T00:12:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 -- 172.21.15.83:0/2925810449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3c15ce90 msgr2=0x7fbf3c0ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 --2- 172.21.15.83:0/2925810449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3c15ce90 0x7fbf3c0ce6e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf340097f0 tx=0x7fbf3401c410 comp rx=0 tx=0).stop 2024-04-25T00:12:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 -- 172.21.15.83:0/2925810449 shutdown_connections 2024-04-25T00:12:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 --2- 172.21.15.83:0/2925810449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3c15ce90 0x7fbf3c0ce6e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 --2- 172.21.15.83:0/2925810449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf3c1599f0 0x7fbf3c0cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 -- 172.21.15.83:0/2925810449 >> 172.21.15.83:0/2925810449 conn(0x7fbf3c0b1830 msgr2=0x7fbf3c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 -- 172.21.15.83:0/2925810449 shutdown_connections 2024-04-25T00:12:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.974+0000 7fbf42600640 1 -- 172.21.15.83:0/2925810449 wait complete. 2024-04-25T00:12:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf42600640 1 Processor -- start 2024-04-25T00:12:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf42600640 1 -- start start 2024-04-25T00:12:05.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf42600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3c1599f0 0x7fbf3c150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf42600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c15c340 0x7fbf3c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf42600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf3c15ce90 0x7fbf3c15b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf42600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c0cf7c0 con 0x7fbf3c1599f0 2024-04-25T00:12:05.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf42600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c0cf640 con 0x7fbf3c15c340 2024-04-25T00:12:05.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf42600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3c168cd0 con 0x7fbf3c15ce90 2024-04-25T00:12:05.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf3b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3c1599f0 0x7fbf3c150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf3b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3c1599f0 0x7fbf3c150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60560/0 (socket says 172.21.15.83:60560) 2024-04-25T00:12:05.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf3b400640 1 -- 172.21.15.83:0/3193370879 learned_addr learned my addr 172.21.15.83:0/3193370879 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:05.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf3aa00640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c15c340 0x7fbf3c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf3be00640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf3c15ce90 0x7fbf3c15b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf3b400640 1 -- 172.21.15.83:0/3193370879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf3c15ce90 msgr2=0x7fbf3c15b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf3b400640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf3c15ce90 0x7fbf3c15b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf3b400640 1 -- 172.21.15.83:0/3193370879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c15c340 msgr2=0x7fbf3c15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf3b400640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c15c340 0x7fbf3c15afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf3b400640 1 -- 172.21.15.83:0/3193370879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c06ef60 con 0x7fbf3c1599f0 2024-04-25T00:12:05.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf3be00640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf3c15ce90 0x7fbf3c15b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:05.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf3aa00640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c15c340 0x7fbf3c15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:05.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf3b400640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3c1599f0 0x7fbf3c150c10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c007e60 tx=0x7fbf2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:05.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf33400640 1 -- 172.21.15.83:0/3193370879 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c014620 con 0x7fbf3c1599f0 2024-04-25T00:12:05.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf33400640 1 -- 172.21.15.83:0/3193370879 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf2c019070 con 0x7fbf3c1599f0 2024-04-25T00:12:05.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf42600640 1 -- 172.21.15.83:0/3193370879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3c1529e0 con 0x7fbf3c1599f0 2024-04-25T00:12:05.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf33400640 1 -- 172.21.15.83:0/3193370879 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c014a20 con 0x7fbf3c1599f0 2024-04-25T00:12:05.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.978+0000 7fbf42600640 1 -- 172.21.15.83:0/3193370879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3c152fa0 con 0x7fbf3c1599f0 2024-04-25T00:12:05.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.982+0000 7fbf42600640 1 -- 172.21.15.83:0/3193370879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf3c0737f0 con 0x7fbf3c1599f0 2024-04-25T00:12:05.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.982+0000 7fbf33400640 1 -- 172.21.15.83:0/3193370879 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbf2c014bc0 con 0x7fbf3c1599f0 2024-04-25T00:12:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.982+0000 7fbf33400640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbf1c0678f0 0x7fbf1c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.982+0000 7fbf33400640 1 -- 172.21.15.83:0/3193370879 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fbf2c086990 con 0x7fbf3c1599f0 2024-04-25T00:12:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.982+0000 7fbf3aa00640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbf1c0678f0 0x7fbf1c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.982+0000 7fbf3aa00640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbf1c0678f0 0x7fbf1c069db0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c06d6f0 tx=0x7fbf28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:05.986+0000 7fbf33400640 1 -- 172.21.15.83:0/3193370879 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbf2c0570c0 con 0x7fbf3c1599f0 2024-04-25T00:12:06.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.174+0000 7fbf42600640 1 -- 172.21.15.83:0/3193370879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf3c04f5f0 con 0x7fbf3c1599f0 2024-04-25T00:12:06.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.174+0000 7fbf33400640 1 -- 172.21.15.83:0/3193370879 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7fbf2c0569c0 con 0x7fbf3c1599f0 2024-04-25T00:12:06.180 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:06.180 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/1335208265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1335208265},{"type":"v1","addr":"172.21.15.86:6834","nonce":1335208265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/911805587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":911805587},{"type":"v1","addr":"172.21.15.83:6836","nonce":911805587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2309157687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2309157687},{"type":"v1","addr":"172.21.15.83:6837","nonce":2309157687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:11:06.200989+0000","modified":"2024-04-25T00:11:23.462521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1603954817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":1603954817},{"type":"v1","addr":"172.21.15.86:6835","nonce":1603954817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T00:12:06.180 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2024-04-25T00:12:06.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.174+0000 7fbf42600640 1 -- 172.21.15.83:0/3193370879 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbf1c0678f0 msgr2=0x7fbf1c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.174+0000 7fbf42600640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbf1c0678f0 0x7fbf1c069db0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c06d6f0 tx=0x7fbf28009040 comp rx=0 tx=0).stop 2024-04-25T00:12:06.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.174+0000 7fbf42600640 1 -- 172.21.15.83:0/3193370879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3c1599f0 msgr2=0x7fbf3c150c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.174+0000 7fbf42600640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3c1599f0 0x7fbf3c150c10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c007e60 tx=0x7fbf2c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.178+0000 7fbf42600640 1 -- 172.21.15.83:0/3193370879 shutdown_connections 2024-04-25T00:12:06.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.178+0000 7fbf42600640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbf1c0678f0 0x7fbf1c069db0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.178+0000 7fbf42600640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf3c15ce90 0x7fbf3c15b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.178+0000 7fbf42600640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c15c340 0x7fbf3c15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.178+0000 7fbf42600640 1 --2- 172.21.15.83:0/3193370879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3c1599f0 0x7fbf3c150c10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.178+0000 7fbf42600640 1 -- 172.21.15.83:0/3193370879 >> 172.21.15.83:0/3193370879 conn(0x7fbf3c0b1830 msgr2=0x7fbf3c153d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:06.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.178+0000 7fbf42600640 1 -- 172.21.15.83:0/3193370879 shutdown_connections 2024-04-25T00:12:06.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.178+0000 7fbf42600640 1 -- 172.21.15.83:0/3193370879 wait complete. 2024-04-25T00:12:06.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:06.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce8a000640 1 Processor -- start 2024-04-25T00:12:06.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce8a000640 1 -- start start 2024-04-25T00:12:06.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce8a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce8415ada0 0x7fce84072930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce8a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce840cfce0 0x7fce84071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce8a000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce840cf850 con 0x7fce84072130 2024-04-25T00:12:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce8a000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fce840cea90 con 0x7fce8415ada0 2024-04-25T00:12:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce8a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fce840ced70 con 0x7fce840cfce0 2024-04-25T00:12:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce88c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce8415ada0 0x7fce84072930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce83e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce840cfce0 0x7fce84071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce88c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce8415ada0 0x7fce84072930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:43806/0 (socket says 172.21.15.83:43806) 2024-04-25T00:12:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce88c00640 1 -- 172.21.15.83:0/1433014574 learned_addr learned my addr 172.21.15.83:0/1433014574 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce89600640 1 --1- 172.21.15.83:0/1433014574 >> v1:172.21.15.86:6789/0 conn(0x7fce84072130 0x7fce8415e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:40432/0 (socket says 172.21.15.83:40432) 2024-04-25T00:12:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce83e00640 1 -- 172.21.15.83:0/1433014574 >> v2:172.21.15.86:3300/0 conn(0x7fce8415ada0 msgr2=0x7fce84072930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce83e00640 1 --2- 172.21.15.83:0/1433014574 >> v2:172.21.15.86:3300/0 conn(0x7fce8415ada0 0x7fce84072930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce83e00640 1 -- 172.21.15.83:0/1433014574 >> v1:172.21.15.86:6789/0 conn(0x7fce84072130 legacy=0x7fce8415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce83e00640 1 -- 172.21.15.83:0/1433014574 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce840cf050 con 0x7fce840cfce0 2024-04-25T00:12:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.338+0000 7fce88c00640 1 --2- 172.21.15.83:0/1433014574 >> v2:172.21.15.86:3300/0 conn(0x7fce8415ada0 0x7fce84072930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.342+0000 7fce83e00640 1 --2- 172.21.15.83:0/1433014574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce840cfce0 0x7fce84071bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fce700097c0 tx=0x7fce7001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30c654839696397 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.342+0000 7fce83400640 1 -- 172.21.15.83:0/1433014574 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce700b0070 con 0x7fce840cfce0 2024-04-25T00:12:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.342+0000 7fce83400640 1 -- 172.21.15.83:0/1433014574 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce700a4e50 con 0x7fce840cfce0 2024-04-25T00:12:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.342+0000 7fce83400640 1 -- 172.21.15.83:0/1433014574 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce700ab820 con 0x7fce840cfce0 2024-04-25T00:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.342+0000 7fce8a000640 1 -- 172.21.15.83:0/1433014574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce840cfce0 msgr2=0x7fce84071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.342+0000 7fce8a000640 1 --2- 172.21.15.83:0/1433014574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce840cfce0 0x7fce84071bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fce700097c0 tx=0x7fce7001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.342+0000 7fce8a000640 1 -- 172.21.15.83:0/1433014574 shutdown_connections 2024-04-25T00:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.342+0000 7fce8a000640 1 --2- 172.21.15.83:0/1433014574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce840cfce0 0x7fce84071bf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.342+0000 7fce8a000640 1 --2- 172.21.15.83:0/1433014574 >> v2:172.21.15.86:3300/0 conn(0x7fce8415ada0 0x7fce84072930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.342+0000 7fce8a000640 1 -- 172.21.15.83:0/1433014574 >> 172.21.15.83:0/1433014574 conn(0x7fce8406c2f0 msgr2=0x7fce8406c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.342+0000 7fce8a000640 1 -- 172.21.15.83:0/1433014574 shutdown_connections 2024-04-25T00:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.342+0000 7fce8a000640 1 -- 172.21.15.83:0/1433014574 wait complete. 2024-04-25T00:12:06.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce8a000640 1 Processor -- start 2024-04-25T00:12:06.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce8a000640 1 -- start start 2024-04-25T00:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce8a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce84072130 0x7fce8415b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce8a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce840cfce0 0x7fce841653c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce8a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8415ada0 0x7fce84159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce8a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fce84072540 con 0x7fce840cfce0 2024-04-25T00:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce8a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce8406ef20 con 0x7fce8415ada0 2024-04-25T00:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce8a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fce8406b1e0 con 0x7fce84072130 2024-04-25T00:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce840cfce0 0x7fce841653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce840cfce0 0x7fce841653c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60574/0 (socket says 172.21.15.83:60574) 2024-04-25T00:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce83e00640 1 -- 172.21.15.83:0/781289671 learned_addr learned my addr 172.21.15.83:0/781289671 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce88c00640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce84072130 0x7fce8415b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce89600640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8415ada0 0x7fce84159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce83e00640 1 -- 172.21.15.83:0/781289671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce84072130 msgr2=0x7fce8415b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce83e00640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce84072130 0x7fce8415b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce83e00640 1 -- 172.21.15.83:0/781289671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8415ada0 msgr2=0x7fce84159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce83e00640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8415ada0 0x7fce84159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce83e00640 1 -- 172.21.15.83:0/781289671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8415b970 con 0x7fce840cfce0 2024-04-25T00:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce89600640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8415ada0 0x7fce84159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce83e00640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce840cfce0 0x7fce841653c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fce700ae040 tx=0x7fce70019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce80c00640 1 -- 172.21.15.83:0/781289671 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce700ba070 con 0x7fce840cfce0 2024-04-25T00:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce80c00640 1 -- 172.21.15.83:0/781289671 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce700abdf0 con 0x7fce840cfce0 2024-04-25T00:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce80c00640 1 -- 172.21.15.83:0/781289671 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce700b0040 con 0x7fce840cfce0 2024-04-25T00:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce8a000640 1 -- 172.21.15.83:0/781289671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8415bc50 con 0x7fce840cfce0 2024-04-25T00:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce8a000640 1 -- 172.21.15.83:0/781289671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8415c210 con 0x7fce840cfce0 2024-04-25T00:12:06.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.346+0000 7fce8a000640 1 -- 172.21.15.83:0/781289671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce8404f510 con 0x7fce840cfce0 2024-04-25T00:12:06.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.350+0000 7fce80c00640 1 -- 172.21.15.83:0/781289671 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fce700b4630 con 0x7fce840cfce0 2024-04-25T00:12:06.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.350+0000 7fce80c00640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fce640678f0 0x7fce64069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:06.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.350+0000 7fce80c00640 1 -- 172.21.15.83:0/781289671 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fce7011f130 con 0x7fce840cfce0 2024-04-25T00:12:06.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.350+0000 7fce88c00640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fce640678f0 0x7fce64069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:06.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.350+0000 7fce88c00640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fce640678f0 0x7fce64069db0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fce78006cb0 tx=0x7fce78009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.354+0000 7fce80c00640 1 -- 172.21.15.83:0/781289671 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fce70016090 con 0x7fce840cfce0 2024-04-25T00:12:06.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.510+0000 7fce8a000640 1 -- 172.21.15.83:0/781289671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce84156e30 con 0x7fce840cfce0 2024-04-25T00:12:06.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.514+0000 7fce80c00640 1 -- 172.21.15.83:0/781289671 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18820 (secure 0 0 0) 0x7fce700ef070 con 0x7fce840cfce0 2024-04-25T00:12:06.514 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:06.514 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:48.235809+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:06.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.514+0000 7fce8a000640 1 -- 172.21.15.83:0/781289671 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fce640678f0 msgr2=0x7fce64069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.514+0000 7fce8a000640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fce640678f0 0x7fce64069db0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fce78006cb0 tx=0x7fce78009240 comp rx=0 tx=0).stop 2024-04-25T00:12:06.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.514+0000 7fce8a000640 1 -- 172.21.15.83:0/781289671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce840cfce0 msgr2=0x7fce841653c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.514+0000 7fce8a000640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce840cfce0 0x7fce841653c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fce700ae040 tx=0x7fce70019240 comp rx=0 tx=0).stop 2024-04-25T00:12:06.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.514+0000 7fce8a000640 1 -- 172.21.15.83:0/781289671 shutdown_connections 2024-04-25T00:12:06.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.514+0000 7fce8a000640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fce640678f0 0x7fce64069db0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.514+0000 7fce8a000640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8415ada0 0x7fce84159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.514+0000 7fce8a000640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce840cfce0 0x7fce841653c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.514+0000 7fce8a000640 1 --2- 172.21.15.83:0/781289671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce84072130 0x7fce8415b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.514+0000 7fce8a000640 1 -- 172.21.15.83:0/781289671 >> 172.21.15.83:0/781289671 conn(0x7fce8406c2f0 msgr2=0x7fce8406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:06.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.514+0000 7fce8a000640 1 -- 172.21.15.83:0/781289671 shutdown_connections 2024-04-25T00:12:06.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.514+0000 7fce8a000640 1 -- 172.21.15.83:0/781289671 wait complete. 2024-04-25T00:12:06.536 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:12:06.536 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:12:06.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.670+0000 7f49a4a00640 1 Processor -- start 2024-04-25T00:12:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.670+0000 7f49a4a00640 1 -- start start 2024-04-25T00:12:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.670+0000 7f49a4a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f49a00b4360 0x7f49a00b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.670+0000 7f49a4a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49a006ef80 con 0x7f49a01553e0 2024-04-25T00:12:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.670+0000 7f49a4a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49a015b510 con 0x7f49a00b2b30 2024-04-25T00:12:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.670+0000 7f49a4a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f49a015b7f0 con 0x7f49a00b4360 2024-04-25T00:12:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.670+0000 7f499de00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f49a01553e0 0x7f49a006e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46072/0 (socket says 172.21.15.83:46072) 2024-04-25T00:12:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.670+0000 7f499de00640 1 -- 172.21.15.83:0/2015016871 learned_addr learned my addr 172.21.15.83:0/2015016871 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.670+0000 7f499d400640 1 --2- 172.21.15.83:0/2015016871 >> v2:172.21.15.83:3300/0 conn(0x7f49a00b4360 0x7f49a00b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:06.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.670+0000 7f499ca00640 1 -- 172.21.15.83:0/2015016871 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1299312898 0 0) 0x7f49a006ef80 con 0x7f49a01553e0 2024-04-25T00:12:06.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.670+0000 7f499ca00640 1 -- 172.21.15.83:0/2015016871 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4984003530 con 0x7f49a01553e0 2024-04-25T00:12:06.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.670+0000 7f499d400640 1 -- 172.21.15.83:0/2015016871 >> v1:172.21.15.86:6789/0 conn(0x7f49a00b2b30 legacy=0x7f49a00b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.670+0000 7f499d400640 1 -- 172.21.15.83:0/2015016871 >> v1:172.21.15.83:6789/0 conn(0x7f49a01553e0 legacy=0x7f49a006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.670+0000 7f499d400640 1 -- 172.21.15.83:0/2015016871 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a015bad0 con 0x7f49a00b4360 2024-04-25T00:12:06.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.674+0000 7f499d400640 1 --2- 172.21.15.83:0/2015016871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a00b4360 0x7f49a00b47e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f498c0097c0 tx=0x7f498c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bf77a08b916a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:06.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.674+0000 7f499ca00640 1 -- 172.21.15.83:0/2015016871 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c0b0070 con 0x7f49a00b4360 2024-04-25T00:12:06.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.674+0000 7f499ca00640 1 -- 172.21.15.83:0/2015016871 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f498c0ab600 con 0x7f49a00b4360 2024-04-25T00:12:06.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.674+0000 7f499ca00640 1 -- 172.21.15.83:0/2015016871 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c0ab9b0 con 0x7f49a00b4360 2024-04-25T00:12:06.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.674+0000 7f49a4a00640 1 -- 172.21.15.83:0/2015016871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a00b4360 msgr2=0x7f49a00b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.674+0000 7f49a4a00640 1 --2- 172.21.15.83:0/2015016871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a00b4360 0x7f49a00b47e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f498c0097c0 tx=0x7f498c01c410 comp rx=0 tx=0).stop 2024-04-25T00:12:06.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.674+0000 7f49a4a00640 1 -- 172.21.15.83:0/2015016871 shutdown_connections 2024-04-25T00:12:06.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.674+0000 7f49a4a00640 1 --2- 172.21.15.83:0/2015016871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a00b4360 0x7f49a00b47e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.674+0000 7f49a4a00640 1 -- 172.21.15.83:0/2015016871 >> 172.21.15.83:0/2015016871 conn(0x7f49a00b1830 msgr2=0x7f49a00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:06.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.674+0000 7f49a4a00640 1 -- 172.21.15.83:0/2015016871 shutdown_connections 2024-04-25T00:12:06.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.674+0000 7f49a4a00640 1 -- 172.21.15.83:0/2015016871 wait complete. 2024-04-25T00:12:06.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f49a4a00640 1 Processor -- start 2024-04-25T00:12:06.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f49a4a00640 1 -- start start 2024-04-25T00:12:06.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f49a4a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49a00b2b30 0x7f49a00ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:06.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f49a4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a00b4360 0x7f49a0150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:06.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f49a4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a01553e0 0x7f49a0151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:06.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f49a4a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49a0155940 con 0x7f49a01553e0 2024-04-25T00:12:06.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f49a4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49a01557c0 con 0x7f49a00b4360 2024-04-25T00:12:06.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f49a4a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f49a006ebd0 con 0x7f49a00b2b30 2024-04-25T00:12:06.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f499e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a01553e0 0x7f49a0151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:06.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f499e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a01553e0 0x7f49a0151520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60598/0 (socket says 172.21.15.83:60598) 2024-04-25T00:12:06.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f499e800640 1 -- 172.21.15.83:0/3283871997 learned_addr learned my addr 172.21.15.83:0/3283871997 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:06.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f499de00640 1 --2- 172.21.15.83:0/3283871997 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49a00b2b30 0x7f49a00ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:06.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f499d400640 1 --2- 172.21.15.83:0/3283871997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a00b4360 0x7f49a0150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:06.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f499e800640 1 -- 172.21.15.83:0/3283871997 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49a00b2b30 msgr2=0x7f49a00ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f499e800640 1 --2- 172.21.15.83:0/3283871997 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49a00b2b30 0x7f49a00ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f499e800640 1 -- 172.21.15.83:0/3283871997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a00b4360 msgr2=0x7f49a0150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f499e800640 1 --2- 172.21.15.83:0/3283871997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a00b4360 0x7f49a0150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f499e800640 1 -- 172.21.15.83:0/3283871997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a00c8eb0 con 0x7f49a01553e0 2024-04-25T00:12:06.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f499e800640 1 --2- 172.21.15.83:0/3283871997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a01553e0 0x7f49a0151520 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f499800c370 tx=0x7f499800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f4996000640 1 -- 172.21.15.83:0/3283871997 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998005220 con 0x7f49a01553e0 2024-04-25T00:12:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f4996000640 1 -- 172.21.15.83:0/3283871997 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49980053c0 con 0x7f49a01553e0 2024-04-25T00:12:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f4996000640 1 -- 172.21.15.83:0/3283871997 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998007270 con 0x7f49a01553e0 2024-04-25T00:12:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f49a4a00640 1 -- 172.21.15.83:0/3283871997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a0151b30 con 0x7f49a01553e0 2024-04-25T00:12:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.678+0000 7f49a4a00640 1 -- 172.21.15.83:0/3283871997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a015a7a0 con 0x7f49a01553e0 2024-04-25T00:12:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.682+0000 7f49a4a00640 1 -- 172.21.15.83:0/3283871997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a015a2b0 con 0x7f49a01553e0 2024-04-25T00:12:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.682+0000 7f4996000640 1 -- 172.21.15.83:0/3283871997 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4998019020 con 0x7f49a01553e0 2024-04-25T00:12:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.682+0000 7f4996000640 1 --2- 172.21.15.83:0/3283871997 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f49800678c0 0x7f4980069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.682+0000 7f4996000640 1 -- 172.21.15.83:0/3283871997 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f4998085220 con 0x7f49a01553e0 2024-04-25T00:12:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.682+0000 7f499de00640 1 --2- 172.21.15.83:0/3283871997 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f49800678c0 0x7f4980069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:06.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.682+0000 7f499de00640 1 --2- 172.21.15.83:0/3283871997 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f49800678c0 0x7f4980069d80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4990004fd0 tx=0x7f49900017a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:06.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.686+0000 7f4996000640 1 -- 172.21.15.83:0/3283871997 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4998055300 con 0x7f49a01553e0 2024-04-25T00:12:06.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.874+0000 7f49a4a00640 1 -- 172.21.15.83:0/3283871997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49a004f5f0 con 0x7f49a01553e0 2024-04-25T00:12:06.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.874+0000 7f4996000640 1 -- 172.21.15.83:0/3283871997 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7f4998059600 con 0x7f49a01553e0 2024-04-25T00:12:06.876 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:06.876 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/1335208265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1335208265},{"type":"v1","addr":"172.21.15.86:6834","nonce":1335208265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/911805587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":911805587},{"type":"v1","addr":"172.21.15.83:6836","nonce":911805587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2309157687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2309157687},{"type":"v1","addr":"172.21.15.83:6837","nonce":2309157687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:11:06.200989+0000","modified":"2024-04-25T00:11:23.462521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1603954817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":1603954817},{"type":"v1","addr":"172.21.15.86:6835","nonce":1603954817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T00:12:06.876 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2024-04-25T00:12:06.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.878+0000 7f49a4a00640 1 -- 172.21.15.83:0/3283871997 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f49800678c0 msgr2=0x7f4980069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.878+0000 7f49a4a00640 1 --2- 172.21.15.83:0/3283871997 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f49800678c0 0x7f4980069d80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4990004fd0 tx=0x7f49900017a0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.878+0000 7f49a4a00640 1 -- 172.21.15.83:0/3283871997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a01553e0 msgr2=0x7f49a0151520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.878+0000 7f49a4a00640 1 --2- 172.21.15.83:0/3283871997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a01553e0 0x7f49a0151520 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f499800c370 tx=0x7f499800c830 comp rx=0 tx=0).stop 2024-04-25T00:12:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.878+0000 7f49a4a00640 1 -- 172.21.15.83:0/3283871997 shutdown_connections 2024-04-25T00:12:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.878+0000 7f49a4a00640 1 --2- 172.21.15.83:0/3283871997 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f49800678c0 0x7f4980069d80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.878+0000 7f49a4a00640 1 --2- 172.21.15.83:0/3283871997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a01553e0 0x7f49a0151520 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.878+0000 7f49a4a00640 1 --2- 172.21.15.83:0/3283871997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a00b4360 0x7f49a0150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.878+0000 7f49a4a00640 1 --2- 172.21.15.83:0/3283871997 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49a00b2b30 0x7f49a00ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.878+0000 7f49a4a00640 1 -- 172.21.15.83:0/3283871997 >> 172.21.15.83:0/3283871997 conn(0x7f49a00b1830 msgr2=0x7f49a006e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.878+0000 7f49a4a00640 1 -- 172.21.15.83:0/3283871997 shutdown_connections 2024-04-25T00:12:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:06.878+0000 7f49a4a00640 1 -- 172.21.15.83:0/3283871997 wait complete. 2024-04-25T00:12:06.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:12:07.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75d6e00640 1 Processor -- start 2024-04-25T00:12:07.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75d6e00640 1 -- start start 2024-04-25T00:12:07.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75d6e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d01599f0 0x7f75d00cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75d6e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75d015c340 0x7f75d015c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75d6e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d00ccc60 con 0x7f75d015ccc0 2024-04-25T00:12:07.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75d6e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f75d0159dd0 con 0x7f75d015c340 2024-04-25T00:12:07.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75d6e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f75d006dba0 con 0x7f75d01599f0 2024-04-25T00:12:07.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75d4e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f75d015ccc0 0x7f75d015f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46080/0 (socket says 172.21.15.83:46080) 2024-04-25T00:12:07.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75c7400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75d015c340 0x7f75d015c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75d4e00640 1 -- 172.21.15.83:0/4044116952 learned_addr learned my addr 172.21.15.83:0/4044116952 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:07.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75cfe00640 1 --2- 172.21.15.83:0/4044116952 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d01599f0 0x7f75d00cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75c7400640 1 -- 172.21.15.83:0/4044116952 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d01599f0 msgr2=0x7f75d00cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75c7400640 1 --2- 172.21.15.83:0/4044116952 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d01599f0 0x7f75d00cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75c7400640 1 -- 172.21.15.83:0/4044116952 >> v1:172.21.15.83:6789/0 conn(0x7f75d015ccc0 legacy=0x7f75d015f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.038+0000 7f75c7400640 1 -- 172.21.15.83:0/4044116952 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d006de80 con 0x7f75d015c340 2024-04-25T00:12:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.034+0000 7f75cf400640 1 -- 172.21.15.83:0/4044116952 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 313360380 0 0) 0x7f75c80014a0 con 0x7f75d015ccc0 2024-04-25T00:12:07.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.038+0000 7f75c7400640 1 --2- 172.21.15.83:0/4044116952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d015c340 0x7f75d015c780 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f75b80097f0 tx=0x7f75b801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a98ef1292b06a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:07.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.038+0000 7f75cf400640 1 -- 172.21.15.83:0/4044116952 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b80b0070 con 0x7f75d015c340 2024-04-25T00:12:07.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.038+0000 7f75cf400640 1 -- 172.21.15.83:0/4044116952 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b80ab600 con 0x7f75d015c340 2024-04-25T00:12:07.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.038+0000 7f75cf400640 1 -- 172.21.15.83:0/4044116952 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b80ab9d0 con 0x7f75d015c340 2024-04-25T00:12:07.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.038+0000 7f75d6e00640 1 -- 172.21.15.83:0/4044116952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d015c340 msgr2=0x7f75d015c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.038+0000 7f75d6e00640 1 --2- 172.21.15.83:0/4044116952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d015c340 0x7f75d015c780 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f75b80097f0 tx=0x7f75b801c410 comp rx=0 tx=0).stop 2024-04-25T00:12:07.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.038+0000 7f75d6e00640 1 -- 172.21.15.83:0/4044116952 shutdown_connections 2024-04-25T00:12:07.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.038+0000 7f75d6e00640 1 --2- 172.21.15.83:0/4044116952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d015c340 0x7f75d015c780 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.038+0000 7f75d6e00640 1 --2- 172.21.15.83:0/4044116952 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d01599f0 0x7f75d00cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.038+0000 7f75d6e00640 1 -- 172.21.15.83:0/4044116952 >> 172.21.15.83:0/4044116952 conn(0x7f75d00b1830 msgr2=0x7f75d00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:07.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.038+0000 7f75d6e00640 1 -- 172.21.15.83:0/4044116952 shutdown_connections 2024-04-25T00:12:07.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.038+0000 7f75d6e00640 1 -- 172.21.15.83:0/4044116952 wait complete. 2024-04-25T00:12:07.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75d6e00640 1 Processor -- start 2024-04-25T00:12:07.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75d6e00640 1 -- start start 2024-04-25T00:12:07.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75d6e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d01599f0 0x7f75d0157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75d6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d015c340 0x7f75d01528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75d6e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d015ccc0 0x7f75d014f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75d6e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75d00cf870 con 0x7f75d01599f0 2024-04-25T00:12:07.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75d6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75d00cf6f0 con 0x7f75d015c340 2024-04-25T00:12:07.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75d6e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f75d0168cd0 con 0x7f75d015ccc0 2024-04-25T00:12:07.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d01599f0 0x7f75d0157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d01599f0 0x7f75d0157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60622/0 (socket says 172.21.15.83:60622) 2024-04-25T00:12:07.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75cfe00640 1 -- 172.21.15.83:0/1047831236 learned_addr learned my addr 172.21.15.83:0/1047831236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75d4e00640 1 --2- 172.21.15.83:0/1047831236 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d015ccc0 0x7f75d014f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75c7400640 1 --2- 172.21.15.83:0/1047831236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d015c340 0x7f75d01528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75cfe00640 1 -- 172.21.15.83:0/1047831236 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d015ccc0 msgr2=0x7f75d014f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75cfe00640 1 --2- 172.21.15.83:0/1047831236 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d015ccc0 0x7f75d014f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75cfe00640 1 -- 172.21.15.83:0/1047831236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d015c340 msgr2=0x7f75d01528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75cfe00640 1 --2- 172.21.15.83:0/1047831236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d015c340 0x7f75d01528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75cfe00640 1 -- 172.21.15.83:0/1047831236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d00ad860 con 0x7f75d01599f0 2024-04-25T00:12:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75cfe00640 1 --2- 172.21.15.83:0/1047831236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d01599f0 0x7f75d0157d80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f75c000e9b0 tx=0x7f75c000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75ccc00640 1 -- 172.21.15.83:0/1047831236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0017070 con 0x7f75d01599f0 2024-04-25T00:12:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75ccc00640 1 -- 172.21.15.83:0/1047831236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c000c9a0 con 0x7f75d01599f0 2024-04-25T00:12:07.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75d6e00640 1 -- 172.21.15.83:0/1047831236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d00adb40 con 0x7f75d01599f0 2024-04-25T00:12:07.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75ccc00640 1 -- 172.21.15.83:0/1047831236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c000cd70 con 0x7f75d01599f0 2024-04-25T00:12:07.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.042+0000 7f75d6e00640 1 -- 172.21.15.83:0/1047831236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d01582c0 con 0x7f75d01599f0 2024-04-25T00:12:07.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.046+0000 7f75d6e00640 1 -- 172.21.15.83:0/1047831236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d00737f0 con 0x7f75d01599f0 2024-04-25T00:12:07.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.046+0000 7f75ccc00640 1 -- 172.21.15.83:0/1047831236 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f75c001a070 con 0x7f75d01599f0 2024-04-25T00:12:07.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.046+0000 7f75ccc00640 1 --2- 172.21.15.83:0/1047831236 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f75ac0678c0 0x7f75ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.046+0000 7f75ccc00640 1 -- 172.21.15.83:0/1047831236 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f75c0086480 con 0x7f75d01599f0 2024-04-25T00:12:07.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.046+0000 7f75c7400640 1 --2- 172.21.15.83:0/1047831236 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f75ac0678c0 0x7f75ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.046+0000 7f75c7400640 1 --2- 172.21.15.83:0/1047831236 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f75ac0678c0 0x7f75ac069d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f75b80ad000 tx=0x7f75b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:07.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.050+0000 7f75ccc00640 1 -- 172.21.15.83:0/1047831236 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f75c0056560 con 0x7f75d01599f0 2024-04-25T00:12:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.238+0000 7f75d6e00640 1 -- 172.21.15.83:0/1047831236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75d015be80 con 0x7f75d01599f0 2024-04-25T00:12:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.242+0000 7f75ccc00640 1 -- 172.21.15.83:0/1047831236 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7f75c005a860 con 0x7f75d01599f0 2024-04-25T00:12:07.242 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:07.242 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/1335208265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1335208265},{"type":"v1","addr":"172.21.15.86:6834","nonce":1335208265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/911805587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":911805587},{"type":"v1","addr":"172.21.15.83:6836","nonce":911805587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2309157687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2309157687},{"type":"v1","addr":"172.21.15.83:6837","nonce":2309157687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:11:06.200989+0000","modified":"2024-04-25T00:11:23.462521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1603954817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":1603954817},{"type":"v1","addr":"172.21.15.86:6835","nonce":1603954817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T00:12:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2024-04-25T00:12:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.242+0000 7f75d6e00640 1 -- 172.21.15.83:0/1047831236 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f75ac0678c0 msgr2=0x7f75ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.242+0000 7f75d6e00640 1 --2- 172.21.15.83:0/1047831236 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f75ac0678c0 0x7f75ac069d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f75b80ad000 tx=0x7f75b8008040 comp rx=0 tx=0).stop 2024-04-25T00:12:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.242+0000 7f75d6e00640 1 -- 172.21.15.83:0/1047831236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d01599f0 msgr2=0x7f75d0157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.242+0000 7f75d6e00640 1 --2- 172.21.15.83:0/1047831236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d01599f0 0x7f75d0157d80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f75c000e9b0 tx=0x7f75c000ee70 comp rx=0 tx=0).stop 2024-04-25T00:12:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.242+0000 7f75d6e00640 1 -- 172.21.15.83:0/1047831236 shutdown_connections 2024-04-25T00:12:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.242+0000 7f75d6e00640 1 --2- 172.21.15.83:0/1047831236 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f75ac0678c0 0x7f75ac069d80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.242+0000 7f75d6e00640 1 --2- 172.21.15.83:0/1047831236 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d015ccc0 0x7f75d014f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.242+0000 7f75d6e00640 1 --2- 172.21.15.83:0/1047831236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d015c340 0x7f75d01528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.242+0000 7f75d6e00640 1 --2- 172.21.15.83:0/1047831236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d01599f0 0x7f75d0157d80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.242+0000 7f75d6e00640 1 -- 172.21.15.83:0/1047831236 >> 172.21.15.83:0/1047831236 conn(0x7f75d00b1830 msgr2=0x7f75d006e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.242+0000 7f75d6e00640 1 -- 172.21.15.83:0/1047831236 shutdown_connections 2024-04-25T00:12:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.242+0000 7f75d6e00640 1 -- 172.21.15.83:0/1047831236 wait complete. 2024-04-25T00:12:07.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:07.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f4022a00640 1 Processor -- start 2024-04-25T00:12:07.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f4022a00640 1 -- start start 2024-04-25T00:12:07.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f4022a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f401c161dc0 0x7f401c1621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f4022a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f401c15dff0 0x7f401c0cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f4022a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f401c0b1cc0 con 0x7f401c06c500 2024-04-25T00:12:07.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f4022a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f401c1626e0 con 0x7f401c161dc0 2024-04-25T00:12:07.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f4022a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f401c0ae9e0 con 0x7f401c15dff0 2024-04-25T00:12:07.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f401be00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f401c161dc0 0x7f401c1621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f401be00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f401c161dc0 0x7f401c1621a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60628/0 (socket says 172.21.15.83:60628) 2024-04-25T00:12:07.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f401be00640 1 -- 172.21.15.83:0/2051734889 learned_addr learned my addr 172.21.15.83:0/2051734889 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:07.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f4020a00640 1 --2- 172.21.15.83:0/2051734889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f401c15dff0 0x7f401c0cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f401be00640 1 -- 172.21.15.83:0/2051734889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f401c15dff0 msgr2=0x7f401c0cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:07.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f401be00640 1 --2- 172.21.15.83:0/2051734889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f401c15dff0 0x7f401c0cff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f401be00640 1 -- 172.21.15.83:0/2051734889 >> v1:172.21.15.86:6789/0 conn(0x7f401c06c500 legacy=0x7f401c15d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.398+0000 7f401be00640 1 -- 172.21.15.83:0/2051734889 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401c06cd00 con 0x7f401c161dc0 2024-04-25T00:12:07.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.402+0000 7f401be00640 1 --2- 172.21.15.83:0/2051734889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f401c161dc0 0x7f401c1621a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f400c009e50 tx=0x7f400c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233d1a1cf4070ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:07.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.402+0000 7f401aa00640 1 -- 172.21.15.83:0/2051734889 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c0a5770 con 0x7f401c161dc0 2024-04-25T00:12:07.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.402+0000 7f401aa00640 1 -- 172.21.15.83:0/2051734889 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f400c0a5910 con 0x7f401c161dc0 2024-04-25T00:12:07.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.402+0000 7f401aa00640 1 -- 172.21.15.83:0/2051734889 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c0a5ce0 con 0x7f401c161dc0 2024-04-25T00:12:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.402+0000 7f4022a00640 1 -- 172.21.15.83:0/2051734889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f401c161dc0 msgr2=0x7f401c1621a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.402+0000 7f4022a00640 1 --2- 172.21.15.83:0/2051734889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f401c161dc0 0x7f401c1621a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f400c009e50 tx=0x7f400c01ca30 comp rx=0 tx=0).stop 2024-04-25T00:12:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.402+0000 7f4022a00640 1 -- 172.21.15.83:0/2051734889 shutdown_connections 2024-04-25T00:12:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.402+0000 7f4022a00640 1 --2- 172.21.15.83:0/2051734889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f401c15dff0 0x7f401c0cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.402+0000 7f4022a00640 1 --2- 172.21.15.83:0/2051734889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f401c161dc0 0x7f401c1621a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.402+0000 7f4022a00640 1 -- 172.21.15.83:0/2051734889 >> 172.21.15.83:0/2051734889 conn(0x7f401c0d1c70 msgr2=0x7f401c0d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.402+0000 7f4022a00640 1 -- 172.21.15.83:0/2051734889 shutdown_connections 2024-04-25T00:12:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.402+0000 7f4022a00640 1 -- 172.21.15.83:0/2051734889 wait complete. 2024-04-25T00:12:07.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4022a00640 1 Processor -- start 2024-04-25T00:12:07.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4022a00640 1 -- start start 2024-04-25T00:12:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4022a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f401c06c500 0x7f401c150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4022a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f401c15dff0 0x7f401c06f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4022a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f401c161dc0 0x7f401c06f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4022a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f401c0d2760 con 0x7f401c15dff0 2024-04-25T00:12:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4022a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f401c0b3490 con 0x7f401c161dc0 2024-04-25T00:12:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4022a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f401c0d28e0 con 0x7f401c06c500 2024-04-25T00:12:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f401b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f401c15dff0 0x7f401c06f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f401b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f401c15dff0 0x7f401c06f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60640/0 (socket says 172.21.15.83:60640) 2024-04-25T00:12:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f401b400640 1 -- 172.21.15.83:0/2803717107 learned_addr learned my addr 172.21.15.83:0/2803717107 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f401be00640 1 --2- 172.21.15.83:0/2803717107 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f401c06c500 0x7f401c150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4020a00640 1 --2- 172.21.15.83:0/2803717107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f401c161dc0 0x7f401c06f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f401b400640 1 -- 172.21.15.83:0/2803717107 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f401c06c500 msgr2=0x7f401c150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f401b400640 1 --2- 172.21.15.83:0/2803717107 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f401c06c500 0x7f401c150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f401b400640 1 -- 172.21.15.83:0/2803717107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f401c161dc0 msgr2=0x7f401c06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f401b400640 1 --2- 172.21.15.83:0/2803717107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f401c161dc0 0x7f401c06f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f401b400640 1 -- 172.21.15.83:0/2803717107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401c0c6c50 con 0x7f401c15dff0 2024-04-25T00:12:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f401b400640 1 --2- 172.21.15.83:0/2803717107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f401c15dff0 0x7f401c06f4a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f400800c3f0 tx=0x7f400800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4013e00640 1 -- 172.21.15.83:0/2803717107 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008017070 con 0x7f401c15dff0 2024-04-25T00:12:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4013e00640 1 -- 172.21.15.83:0/2803717107 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40080052f0 con 0x7f401c15dff0 2024-04-25T00:12:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4013e00640 1 -- 172.21.15.83:0/2803717107 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008003860 con 0x7f401c15dff0 2024-04-25T00:12:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4022a00640 1 -- 172.21.15.83:0/2803717107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401c0c5e90 con 0x7f401c15dff0 2024-04-25T00:12:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4022a00640 1 -- 172.21.15.83:0/2803717107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f401c0c6450 con 0x7f401c15dff0 2024-04-25T00:12:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.406+0000 7f4022a00640 1 -- 172.21.15.83:0/2803717107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f401c09fd60 con 0x7f401c15dff0 2024-04-25T00:12:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.410+0000 7f4013e00640 1 -- 172.21.15.83:0/2803717107 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f400801a070 con 0x7f401c15dff0 2024-04-25T00:12:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.410+0000 7f4013e00640 1 --2- 172.21.15.83:0/2803717107 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3ffc0678c0 0x7f3ffc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.410+0000 7f4013e00640 1 -- 172.21.15.83:0/2803717107 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f4008085280 con 0x7f401c15dff0 2024-04-25T00:12:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.410+0000 7f401be00640 1 --2- 172.21.15.83:0/2803717107 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3ffc0678c0 0x7f3ffc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.410+0000 7f401be00640 1 --2- 172.21.15.83:0/2803717107 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3ffc0678c0 0x7f3ffc069d80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f400c0ae000 tx=0x7f400c0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.414+0000 7f4013e00640 1 -- 172.21.15.83:0/2803717107 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f40080552d0 con 0x7f401c15dff0 2024-04-25T00:12:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.570+0000 7f4022a00640 1 -- 172.21.15.83:0/2803717107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f401c152680 con 0x7f401c15dff0 2024-04-25T00:12:07.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.570+0000 7f4013e00640 1 -- 172.21.15.83:0/2803717107 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18820 (secure 0 0 0) 0x7f40080595d0 con 0x7f401c15dff0 2024-04-25T00:12:07.572 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:07.572 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:11:48.235809+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:11:04.621429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:11:05.189716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:11:08.902215+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:07.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.574+0000 7f4022a00640 1 -- 172.21.15.83:0/2803717107 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3ffc0678c0 msgr2=0x7f3ffc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.574+0000 7f4022a00640 1 --2- 172.21.15.83:0/2803717107 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3ffc0678c0 0x7f3ffc069d80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f400c0ae000 tx=0x7f400c0a7070 comp rx=0 tx=0).stop 2024-04-25T00:12:07.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.574+0000 7f4022a00640 1 -- 172.21.15.83:0/2803717107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f401c15dff0 msgr2=0x7f401c06f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.574+0000 7f4022a00640 1 --2- 172.21.15.83:0/2803717107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f401c15dff0 0x7f401c06f4a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f400800c3f0 tx=0x7f400800c7f0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.574+0000 7f4022a00640 1 -- 172.21.15.83:0/2803717107 shutdown_connections 2024-04-25T00:12:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.574+0000 7f4022a00640 1 --2- 172.21.15.83:0/2803717107 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3ffc0678c0 0x7f3ffc069d80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.574+0000 7f4022a00640 1 --2- 172.21.15.83:0/2803717107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f401c161dc0 0x7f401c06f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.574+0000 7f4022a00640 1 --2- 172.21.15.83:0/2803717107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f401c15dff0 0x7f401c06f4a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.574+0000 7f4022a00640 1 --2- 172.21.15.83:0/2803717107 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f401c06c500 0x7f401c150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.574+0000 7f4022a00640 1 -- 172.21.15.83:0/2803717107 >> 172.21.15.83:0/2803717107 conn(0x7f401c0d1c70 msgr2=0x7f401c0d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.574+0000 7f4022a00640 1 -- 172.21.15.83:0/2803717107 shutdown_connections 2024-04-25T00:12:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.574+0000 7f4022a00640 1 -- 172.21.15.83:0/2803717107 wait complete. 2024-04-25T00:12:07.594 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:12:07.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.726+0000 7f2e2fe00640 1 Processor -- start 2024-04-25T00:12:07.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.726+0000 7f2e2fe00640 1 -- start start 2024-04-25T00:12:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e280cb640 0x7f2e2806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2fe00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e280711a0 con 0x7f2e28070d10 2024-04-25T00:12:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e2806e160 con 0x7f2e2806cfd0 2024-04-25T00:12:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2fe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2e2806ea10 con 0x7f2e280cb640 2024-04-25T00:12:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2de00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2e28070d10 0x7f2e28165930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46090/0 (socket says 172.21.15.83:46090) 2024-04-25T00:12:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2de00640 1 -- 172.21.15.83:0/2288591407 learned_addr learned my addr 172.21.15.83:0/2288591407 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2ca00640 1 --2- 172.21.15.83:0/2288591407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e280cb640 0x7f2e2806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4049600021 0 0) 0x7f2e200014a0 con 0x7f2e28070d10 2024-04-25T00:12:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e10003530 con 0x7f2e28070d10 2024-04-25T00:12:07.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1748070816 0 0) 0x7f2e2806e160 con 0x7f2e2806cfd0 2024-04-25T00:12:07.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e200014a0 con 0x7f2e2806cfd0 2024-04-25T00:12:07.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4160320947 0 0) 0x7f2e10003530 con 0x7f2e28070d10 2024-04-25T00:12:07.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e280cb640 msgr2=0x7f2e2806dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:07.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 --2- 172.21.15.83:0/2288591407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e280cb640 0x7f2e2806dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 >> v1:172.21.15.86:6789/0 conn(0x7f2e2806cfd0 legacy=0x7f2e2806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2806ecf0 con 0x7f2e28070d10 2024-04-25T00:12:07.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f2e20001bd0 con 0x7f2e28070d10 2024-04-25T00:12:07.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 >> v1:172.21.15.83:6789/0 conn(0x7f2e28070d10 legacy=0x7f2e28165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 --2- 172.21.15.83:0/2288591407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10003b80 0x7f2e10003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 --2- 172.21.15.83:0/2288591407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e10004810 0x7f2e10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 --2- 172.21.15.83:0/2288591407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e280cb640 0x7f2e100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2806ea10 con 0x7f2e10004810 2024-04-25T00:12:07.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1000ba80 con 0x7f2e10003b80 2024-04-25T00:12:07.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2e1000bc80 con 0x7f2e280cb640 2024-04-25T00:12:07.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2ca00640 1 --2- 172.21.15.83:0/2288591407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e10004810 0x7f2e10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2d400640 1 --2- 172.21.15.83:0/2288591407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10003b80 0x7f2e10003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2de00640 1 --2- 172.21.15.83:0/2288591407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e280cb640 0x7f2e100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2ca00640 1 -- 172.21.15.83:0/2288591407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e280cb640 msgr2=0x7f2e100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:07.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2ca00640 1 --2- 172.21.15.83:0/2288591407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e280cb640 0x7f2e100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2ca00640 1 -- 172.21.15.83:0/2288591407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10003b80 msgr2=0x7f2e10003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:07.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2ca00640 1 --2- 172.21.15.83:0/2288591407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10003b80 0x7f2e10003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2ca00640 1 -- 172.21.15.83:0/2288591407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e10003530 con 0x7f2e10004810 2024-04-25T00:12:07.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.730+0000 7f2e2d400640 1 --2- 172.21.15.83:0/2288591407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10003b80 0x7f2e10003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:07.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.734+0000 7f2e2ca00640 1 --2- 172.21.15.83:0/2288591407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e10004810 0x7f2e10004c90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009ae0 tx=0x7f2e1401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb439f3e5c1f1cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:07.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.734+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e140b2070 con 0x7f2e10004810 2024-04-25T00:12:07.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.734+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e140a5660 con 0x7f2e10004810 2024-04-25T00:12:07.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.734+0000 7f2e27e00640 1 -- 172.21.15.83:0/2288591407 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e140096b0 con 0x7f2e10004810 2024-04-25T00:12:07.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.734+0000 7f2e2fe00640 1 -- 172.21.15.83:0/2288591407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e10004810 msgr2=0x7f2e10004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.734+0000 7f2e2fe00640 1 --2- 172.21.15.83:0/2288591407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e10004810 0x7f2e10004c90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009ae0 tx=0x7f2e1401ca70 comp rx=0 tx=0).stop 2024-04-25T00:12:07.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.734+0000 7f2e2de00640 1 -- 172.21.15.83:0/2288591407 reap_dead start 2024-04-25T00:12:07.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.734+0000 7f2e2fe00640 1 -- 172.21.15.83:0/2288591407 shutdown_connections 2024-04-25T00:12:07.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.734+0000 7f2e2fe00640 1 -- 172.21.15.83:0/2288591407 >> 172.21.15.83:0/2288591407 conn(0x7f2e280cfa60 msgr2=0x7f2e280cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:07.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.734+0000 7f2e2fe00640 1 -- 172.21.15.83:0/2288591407 shutdown_connections 2024-04-25T00:12:07.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.734+0000 7f2e2fe00640 1 -- 172.21.15.83:0/2288591407 wait complete. 2024-04-25T00:12:07.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2fe00640 1 Processor -- start 2024-04-25T00:12:07.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2fe00640 1 -- start start 2024-04-25T00:12:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e10004810 0x7f2e280cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e2813cec0 0x7f2e2813ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e280c8db0 0x7f2e280c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e281458e0 con 0x7f2e280c8db0 2024-04-25T00:12:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e28145760 con 0x7f2e2813cec0 2024-04-25T00:12:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2fe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2e2806dac0 con 0x7f2e10004810 2024-04-25T00:12:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e280c8db0 0x7f2e280c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e280c8db0 0x7f2e280c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60660/0 (socket says 172.21.15.83:60660) 2024-04-25T00:12:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2de00640 1 -- 172.21.15.83:0/412753778 learned_addr learned my addr 172.21.15.83:0/412753778 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2de00640 1 -- 172.21.15.83:0/412753778 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e10004810 msgr2=0x7f2e280cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2de00640 1 --2- 172.21.15.83:0/412753778 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e10004810 0x7f2e280cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2de00640 1 -- 172.21.15.83:0/412753778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e2813cec0 msgr2=0x7f2e2813ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2de00640 1 --2- 172.21.15.83:0/412753778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e2813cec0 0x7f2e2813ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2de00640 1 -- 172.21.15.83:0/412753778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2813d350 con 0x7f2e280c8db0 2024-04-25T00:12:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2de00640 1 --2- 172.21.15.83:0/412753778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e280c8db0 0x7f2e280c9260 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2e28073d40 tx=0x7f2e2000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e25600640 1 -- 172.21.15.83:0/412753778 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20007ad0 con 0x7f2e280c8db0 2024-04-25T00:12:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e25600640 1 -- 172.21.15.83:0/412753778 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e20018070 con 0x7f2e280c8db0 2024-04-25T00:12:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2fe00640 1 -- 172.21.15.83:0/412753778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2813d630 con 0x7f2e280c8db0 2024-04-25T00:12:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e25600640 1 -- 172.21.15.83:0/412753778 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20013420 con 0x7f2e280c8db0 2024-04-25T00:12:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2fe00640 1 -- 172.21.15.83:0/412753778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e2813dbf0 con 0x7f2e280c8db0 2024-04-25T00:12:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.738+0000 7f2e2fe00640 1 -- 172.21.15.83:0/412753778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e28068230 con 0x7f2e280c8db0 2024-04-25T00:12:07.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.742+0000 7f2e25600640 1 -- 172.21.15.83:0/412753778 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2e200135c0 con 0x7f2e280c8db0 2024-04-25T00:12:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.742+0000 7f2e25600640 1 --2- 172.21.15.83:0/412753778 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e08067a90 0x7f2e08069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.742+0000 7f2e25600640 1 -- 172.21.15.83:0/412753778 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2e20086520 con 0x7f2e280c8db0 2024-04-25T00:12:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.742+0000 7f2e2d400640 1 --2- 172.21.15.83:0/412753778 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e08067a90 0x7f2e08069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.742+0000 7f2e2d400640 1 --2- 172.21.15.83:0/412753778 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e08067a90 0x7f2e08069f50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c007af0 tx=0x7f2e1c007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:07.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.746+0000 7f2e25600640 1 -- 172.21.15.83:0/412753778 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2e280711a0 con 0x7f2e280c8db0 2024-04-25T00:12:07.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.934+0000 7f2e2fe00640 1 -- 172.21.15.83:0/412753778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2e2806e160 con 0x7f2e280c8db0 2024-04-25T00:12:07.935 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-25T00:12:07.934+0000 7efeeee00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:12:07.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.938+0000 7f2e25600640 1 -- 172.21.15.83:0/412753778 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f2e20056c50 con 0x7f2e280c8db0 2024-04-25T00:12:07.938 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:12:07.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.938+0000 7f2e2fe00640 1 -- 172.21.15.83:0/412753778 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e08067a90 msgr2=0x7f2e08069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.938+0000 7f2e2fe00640 1 --2- 172.21.15.83:0/412753778 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e08067a90 0x7f2e08069f50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c007af0 tx=0x7f2e1c007920 comp rx=0 tx=0).stop 2024-04-25T00:12:07.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.938+0000 7f2e2fe00640 1 -- 172.21.15.83:0/412753778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e280c8db0 msgr2=0x7f2e280c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:07.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.938+0000 7f2e2fe00640 1 --2- 172.21.15.83:0/412753778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e280c8db0 0x7f2e280c9260 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2e28073d40 tx=0x7f2e2000e530 comp rx=0 tx=0).stop 2024-04-25T00:12:07.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.938+0000 7f2e2fe00640 1 -- 172.21.15.83:0/412753778 shutdown_connections 2024-04-25T00:12:07.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.938+0000 7f2e2fe00640 1 --2- 172.21.15.83:0/412753778 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e08067a90 0x7f2e08069f50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.938+0000 7f2e2fe00640 1 --2- 172.21.15.83:0/412753778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e280c8db0 0x7f2e280c9260 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.938+0000 7f2e2fe00640 1 --2- 172.21.15.83:0/412753778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e2813cec0 0x7f2e2813ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.938+0000 7f2e2fe00640 1 --2- 172.21.15.83:0/412753778 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e10004810 0x7f2e280cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:07.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.938+0000 7f2e2fe00640 1 -- 172.21.15.83:0/412753778 >> 172.21.15.83:0/412753778 conn(0x7f2e280cfa60 msgr2=0x7f2e280b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:07.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.938+0000 7f2e2fe00640 1 -- 172.21.15.83:0/412753778 shutdown_connections 2024-04-25T00:12:07.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:07.938+0000 7f2e2fe00640 1 -- 172.21.15.83:0/412753778 wait complete. 2024-04-25T00:12:07.949 INFO:tasks.ceph.mds.b.smithi086.stderr: -6754> 2024-04-25T00:10:59.916+0000 7f7771c00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T00:12:07.952 INFO:tasks.ceph.mds.b.smithi086.stderr: -5485> 2024-04-25T00:11:10.928+0000 7f776a400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:12:07.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:12:08.004 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2024-04-25T00:12:08.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.094+0000 7f2cad800640 1 Processor -- start 2024-04-25T00:12:08.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.094+0000 7f2cad800640 1 -- start start 2024-04-25T00:12:08.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.098+0000 7f2cad800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ca81551e0 0x7f2ca815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:08.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.098+0000 7f2cad800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ca806e4c0 0x7f2ca806e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:08.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.098+0000 7f2cad800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca815c560 con 0x7f2ca81555c0 2024-04-25T00:12:08.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.098+0000 7f2cad800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca806ee70 con 0x7f2ca806e4c0 2024-04-25T00:12:08.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.098+0000 7f2cad800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca8072460 con 0x7f2ca81551e0 2024-04-25T00:12:08.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.098+0000 7f2ca7400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ca806e4c0 0x7f2ca806e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:08.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.098+0000 7f2ca7400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ca806e4c0 0x7f2ca806e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60668/0 (socket says 172.21.15.83:60668) 2024-04-25T00:12:08.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.098+0000 7f2ca7400640 1 -- 172.21.15.83:0/3767918099 learned_addr learned my addr 172.21.15.83:0/3767918099 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:08.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.098+0000 7f2ca6a00640 1 --2- 172.21.15.83:0/3767918099 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ca81551e0 0x7f2ca815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:08.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.098+0000 7f2ca7400640 1 -- 172.21.15.83:0/3767918099 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ca81551e0 msgr2=0x7f2ca815bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:08.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.098+0000 7f2ca7400640 1 --2- 172.21.15.83:0/3767918099 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ca81551e0 0x7f2ca815bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.098+0000 7f2ca7400640 1 -- 172.21.15.83:0/3767918099 >> v1:172.21.15.86:6789/0 conn(0x7f2ca81555c0 legacy=0x7f2ca8071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:08.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.098+0000 7f2ca7400640 1 -- 172.21.15.83:0/3767918099 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca815c7d0 con 0x7f2ca806e4c0 2024-04-25T00:12:08.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.102+0000 7f2ca7400640 1 --2- 172.21.15.83:0/3767918099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca806e4c0 0x7f2ca806e930 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000d150 tx=0x7f2ca001ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561c1529c2f3fb11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.102+0000 7f2ca6000640 1 -- 172.21.15.83:0/3767918099 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000b2d0 con 0x7f2ca806e4c0 2024-04-25T00:12:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.102+0000 7f2ca6000640 1 -- 172.21.15.83:0/3767918099 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca000b470 con 0x7f2ca806e4c0 2024-04-25T00:12:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.102+0000 7f2ca6000640 1 -- 172.21.15.83:0/3767918099 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00a7b30 con 0x7f2ca806e4c0 2024-04-25T00:12:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.102+0000 7f2cad800640 1 -- 172.21.15.83:0/3767918099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca806e4c0 msgr2=0x7f2ca806e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.102+0000 7f2cad800640 1 --2- 172.21.15.83:0/3767918099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca806e4c0 0x7f2ca806e930 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000d150 tx=0x7f2ca001ec30 comp rx=0 tx=0).stop 2024-04-25T00:12:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.102+0000 7f2cad800640 1 -- 172.21.15.83:0/3767918099 shutdown_connections 2024-04-25T00:12:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.102+0000 7f2cad800640 1 --2- 172.21.15.83:0/3767918099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca806e4c0 0x7f2ca806e930 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.102+0000 7f2cad800640 1 --2- 172.21.15.83:0/3767918099 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ca81551e0 0x7f2ca815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.102+0000 7f2cad800640 1 -- 172.21.15.83:0/3767918099 >> 172.21.15.83:0/3767918099 conn(0x7f2ca80b1830 msgr2=0x7f2ca80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:08.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.102+0000 7f2cad800640 1 -- 172.21.15.83:0/3767918099 shutdown_connections 2024-04-25T00:12:08.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.102+0000 7f2cad800640 1 -- 172.21.15.83:0/3767918099 wait complete. 2024-04-25T00:12:08.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.102+0000 7f2cad800640 1 Processor -- start 2024-04-25T00:12:08.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.102+0000 7f2cad800640 1 -- start start 2024-04-25T00:12:08.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2cad800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ca806e4c0 0x7f2ca815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:08.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2cad800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ca81551e0 0x7f2ca815d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:08.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2cad800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca81555c0 0x7f2ca815d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:08.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2cad800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca806df50 con 0x7f2ca81555c0 2024-04-25T00:12:08.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2cad800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca806ddd0 con 0x7f2ca81551e0 2024-04-25T00:12:08.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2cad800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca806e0d0 con 0x7f2ca806e4c0 2024-04-25T00:12:08.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2ca7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca81555c0 0x7f2ca815d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:08.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2ca7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca81555c0 0x7f2ca815d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60682/0 (socket says 172.21.15.83:60682) 2024-04-25T00:12:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2ca7400640 1 -- 172.21.15.83:0/876245846 learned_addr learned my addr 172.21.15.83:0/876245846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2ca6a00640 1 --2- 172.21.15.83:0/876245846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ca806e4c0 0x7f2ca815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2c9e000640 1 --2- 172.21.15.83:0/876245846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ca81551e0 0x7f2ca815d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2ca7400640 1 -- 172.21.15.83:0/876245846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ca806e4c0 msgr2=0x7f2ca815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2ca7400640 1 --2- 172.21.15.83:0/876245846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ca806e4c0 0x7f2ca815e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2ca7400640 1 -- 172.21.15.83:0/876245846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ca81551e0 msgr2=0x7f2ca815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2ca7400640 1 --2- 172.21.15.83:0/876245846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ca81551e0 0x7f2ca815d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2ca7400640 1 -- 172.21.15.83:0/876245846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca8154760 con 0x7f2ca81555c0 2024-04-25T00:12:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2ca7400640 1 --2- 172.21.15.83:0/876245846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca81555c0 0x7f2ca815d5a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2ca001c860 tx=0x7f2ca001b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:08.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2c9f400640 1 -- 172.21.15.83:0/876245846 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0009800 con 0x7f2ca81555c0 2024-04-25T00:12:08.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2c9f400640 1 -- 172.21.15.83:0/876245846 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca00bf070 con 0x7f2ca81555c0 2024-04-25T00:12:08.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2c9f400640 1 -- 172.21.15.83:0/876245846 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0009c20 con 0x7f2ca81555c0 2024-04-25T00:12:08.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2cad800640 1 -- 172.21.15.83:0/876245846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca8154480 con 0x7f2ca81555c0 2024-04-25T00:12:08.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2cad800640 1 -- 172.21.15.83:0/876245846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca8147b10 con 0x7f2ca81555c0 2024-04-25T00:12:08.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2cad800640 1 -- 172.21.15.83:0/876245846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca804f510 con 0x7f2ca81555c0 2024-04-25T00:12:08.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.106+0000 7f2c9f400640 1 -- 172.21.15.83:0/876245846 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2ca000b2e0 con 0x7f2ca81555c0 2024-04-25T00:12:08.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.110+0000 7f2c9f400640 1 --2- 172.21.15.83:0/876245846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2c840678f0 0x7f2c84069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:08.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.110+0000 7f2c9f400640 1 -- 172.21.15.83:0/876245846 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f2ca0123130 con 0x7f2ca81555c0 2024-04-25T00:12:08.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.110+0000 7f2ca6a00640 1 --2- 172.21.15.83:0/876245846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2c840678f0 0x7f2c84069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:08.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.110+0000 7f2ca6a00640 1 --2- 172.21.15.83:0/876245846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2c840678f0 0x7f2c84069db0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2c98007a60 tx=0x7f2c98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:08.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.114+0000 7f2c9f400640 1 -- 172.21.15.83:0/876245846 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2ca0018820 con 0x7f2ca81555c0 2024-04-25T00:12:08.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.302+0000 7f2cad800640 1 -- 172.21.15.83:0/876245846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2ca80b4dd0 con 0x7f2ca81555c0 2024-04-25T00:12:08.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.938+0000 7f2c9f400640 1 -- 172.21.15.83:0/876245846 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f2ca00f3280 con 0x7f2ca81555c0 2024-04-25T00:12:08.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.942+0000 7f2cad800640 1 -- 172.21.15.83:0/876245846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2c840678f0 msgr2=0x7f2c84069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:08.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.942+0000 7f2cad800640 1 --2- 172.21.15.83:0/876245846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2c840678f0 0x7f2c84069db0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2c98007a60 tx=0x7f2c98008040 comp rx=0 tx=0).stop 2024-04-25T00:12:08.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.942+0000 7f2cad800640 1 -- 172.21.15.83:0/876245846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca81555c0 msgr2=0x7f2ca815d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:08.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.942+0000 7f2cad800640 1 --2- 172.21.15.83:0/876245846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca81555c0 0x7f2ca815d5a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2ca001c860 tx=0x7f2ca001b9d0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.942+0000 7f2cad800640 1 -- 172.21.15.83:0/876245846 shutdown_connections 2024-04-25T00:12:08.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.942+0000 7f2cad800640 1 --2- 172.21.15.83:0/876245846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2c840678f0 0x7f2c84069db0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.942+0000 7f2cad800640 1 --2- 172.21.15.83:0/876245846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca81555c0 0x7f2ca815d5a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.942+0000 7f2cad800640 1 --2- 172.21.15.83:0/876245846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ca81551e0 0x7f2ca815d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.942+0000 7f2cad800640 1 --2- 172.21.15.83:0/876245846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ca806e4c0 0x7f2ca815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.942+0000 7f2cad800640 1 -- 172.21.15.83:0/876245846 >> 172.21.15.83:0/876245846 conn(0x7f2ca80b1830 msgr2=0x7f2ca814ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:08.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.942+0000 7f2cad800640 1 -- 172.21.15.83:0/876245846 shutdown_connections 2024-04-25T00:12:08.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:08.942+0000 7f2cad800640 1 -- 172.21.15.83:0/876245846 wait complete. 2024-04-25T00:12:08.962 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:12:09.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.102+0000 7f89d1600640 1 Processor -- start 2024-04-25T00:12:09.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.102+0000 7f89d1600640 1 -- start start 2024-04-25T00:12:09.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.102+0000 7f89d1600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc1455d0 0x7f89cc06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.102+0000 7f89d1600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89cc0ce610 con 0x7f89cc0ce080 2024-04-25T00:12:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.102+0000 7f89d1600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89cc0ce8f0 con 0x7f89cc0d0450 2024-04-25T00:12:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.102+0000 7f89d1600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f89cc0cebd0 con 0x7f89cc1455d0 2024-04-25T00:12:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.102+0000 7f89ca000640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f89cc0ce080 0x7f89cc0cfd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46096/0 (socket says 172.21.15.83:46096) 2024-04-25T00:12:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.102+0000 7f89ca000640 1 -- 172.21.15.83:0/677693230 learned_addr learned my addr 172.21.15.83:0/677693230 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.102+0000 7f89caa00640 1 --2- 172.21.15.83:0/677693230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc1455d0 0x7f89cc06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.106+0000 7f89c9600640 1 -- 172.21.15.83:0/677693230 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1876786992 0 0) 0x7f89cc0ce610 con 0x7f89cc0ce080 2024-04-25T00:12:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.106+0000 7f89c9600640 1 -- 172.21.15.83:0/677693230 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89b0003530 con 0x7f89cc0ce080 2024-04-25T00:12:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.106+0000 7f89c9600640 1 -- 172.21.15.83:0/677693230 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 567302625 0 0) 0x7f89cc0ce8f0 con 0x7f89cc0d0450 2024-04-25T00:12:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.106+0000 7f89c9600640 1 -- 172.21.15.83:0/677693230 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89cc0ce610 con 0x7f89cc0d0450 2024-04-25T00:12:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.106+0000 7f89caa00640 1 -- 172.21.15.83:0/677693230 >> v1:172.21.15.86:6789/0 conn(0x7f89cc0d0450 legacy=0x7f89cc0b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.106+0000 7f89c9600640 1 -- 172.21.15.83:0/677693230 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1782128369 0 0) 0x7f89b0003530 con 0x7f89cc0ce080 2024-04-25T00:12:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.106+0000 7f89caa00640 1 -- 172.21.15.83:0/677693230 >> v1:172.21.15.83:6789/0 conn(0x7f89cc0ce080 legacy=0x7f89cc0cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.106+0000 7f89caa00640 1 -- 172.21.15.83:0/677693230 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89cc0ceeb0 con 0x7f89cc1455d0 2024-04-25T00:12:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.106+0000 7f89caa00640 1 --2- 172.21.15.83:0/677693230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc1455d0 0x7f89cc06d7e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0097f0 tx=0x7f89bc01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d473302a5b1109e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.106+0000 7f89c9600640 1 -- 172.21.15.83:0/677693230 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc0b0070 con 0x7f89cc1455d0 2024-04-25T00:12:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.106+0000 7f89c9600640 1 -- 172.21.15.83:0/677693230 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89bc0ab400 con 0x7f89cc1455d0 2024-04-25T00:12:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.106+0000 7f89c9600640 1 -- 172.21.15.83:0/677693230 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8001680 con 0x7f89cc1455d0 2024-04-25T00:12:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 -- 172.21.15.83:0/677693230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc1455d0 msgr2=0x7f89cc06d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 --2- 172.21.15.83:0/677693230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc1455d0 0x7f89cc06d7e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0097f0 tx=0x7f89bc01c400 comp rx=0 tx=0).stop 2024-04-25T00:12:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 -- 172.21.15.83:0/677693230 shutdown_connections 2024-04-25T00:12:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 --2- 172.21.15.83:0/677693230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc1455d0 0x7f89cc06d7e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 -- 172.21.15.83:0/677693230 >> 172.21.15.83:0/677693230 conn(0x7f89cc06b3b0 msgr2=0x7f89cc06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 -- 172.21.15.83:0/677693230 shutdown_connections 2024-04-25T00:12:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 -- 172.21.15.83:0/677693230 wait complete. 2024-04-25T00:12:09.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 Processor -- start 2024-04-25T00:12:09.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 -- start start 2024-04-25T00:12:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc0ce080 0x7f89cc13baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc0d0450 0x7f89cc1404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89cc1455d0 0x7f89cc140a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc0b3bd0 con 0x7f89cc1455d0 2024-04-25T00:12:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc0b3a50 con 0x7f89cc0d0450 2024-04-25T00:12:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89d1600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f89cc0b3d50 con 0x7f89cc0ce080 2024-04-25T00:12:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89cb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89cc1455d0 0x7f89cc140a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89cb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89cc1455d0 0x7f89cc140a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60690/0 (socket says 172.21.15.83:60690) 2024-04-25T00:12:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89cb400640 1 -- 172.21.15.83:0/1295136350 learned_addr learned my addr 172.21.15.83:0/1295136350 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89ca000640 1 --2- 172.21.15.83:0/1295136350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc0d0450 0x7f89cc1404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89caa00640 1 --2- 172.21.15.83:0/1295136350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc0ce080 0x7f89cc13baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89cb400640 1 -- 172.21.15.83:0/1295136350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc0ce080 msgr2=0x7f89cc13baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89cb400640 1 --2- 172.21.15.83:0/1295136350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc0ce080 0x7f89cc13baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89cb400640 1 -- 172.21.15.83:0/1295136350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc0d0450 msgr2=0x7f89cc1404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89cb400640 1 --2- 172.21.15.83:0/1295136350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc0d0450 0x7f89cc1404e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.110+0000 7f89cb400640 1 -- 172.21.15.83:0/1295136350 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89cc15fc10 con 0x7f89cc1455d0 2024-04-25T00:12:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.114+0000 7f89cb400640 1 --2- 172.21.15.83:0/1295136350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89cc1455d0 0x7f89cc140a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f89c400c730 tx=0x7f89c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.114+0000 7f89c2a00640 1 -- 172.21.15.83:0/1295136350 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4003b80 con 0x7f89cc1455d0 2024-04-25T00:12:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.114+0000 7f89c2a00640 1 -- 172.21.15.83:0/1295136350 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c4003d20 con 0x7f89cc1455d0 2024-04-25T00:12:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.114+0000 7f89c2a00640 1 -- 172.21.15.83:0/1295136350 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c400aa90 con 0x7f89cc1455d0 2024-04-25T00:12:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.114+0000 7f89d1600640 1 -- 172.21.15.83:0/1295136350 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89cc13afc0 con 0x7f89cc1455d0 2024-04-25T00:12:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.114+0000 7f89d1600640 1 -- 172.21.15.83:0/1295136350 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89cc13b580 con 0x7f89cc1455d0 2024-04-25T00:12:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.114+0000 7f89d1600640 1 -- 172.21.15.83:0/1295136350 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89cc13b780 con 0x7f89cc1455d0 2024-04-25T00:12:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.114+0000 7f89c2a00640 1 -- 172.21.15.83:0/1295136350 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f89c400ac30 con 0x7f89cc1455d0 2024-04-25T00:12:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.114+0000 7f89c2a00640 1 --2- 172.21.15.83:0/1295136350 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f89ac0678f0 0x7f89ac069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.114+0000 7f89c2a00640 1 -- 172.21.15.83:0/1295136350 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f89c4086140 con 0x7f89cc1455d0 2024-04-25T00:12:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.114+0000 7f89caa00640 1 --2- 172.21.15.83:0/1295136350 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f89ac0678f0 0x7f89ac069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.114+0000 7f89caa00640 1 --2- 172.21.15.83:0/1295136350 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f89ac0678f0 0x7f89ac069db0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0ad000 tx=0x7f89bc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.118+0000 7f89c2a00640 1 -- 172.21.15.83:0/1295136350 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f89cc13b780 con 0x7f89cc1455d0 2024-04-25T00:12:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.274+0000 7f89d1600640 1 -- 172.21.15.83:0/1295136350 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f89cc0736f0 con 0x7f89cc1455d0 2024-04-25T00:12:09.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.942+0000 7f89c2a00640 1 -- 172.21.15.83:0/1295136350 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7f89c40566f0 con 0x7f89cc1455d0 2024-04-25T00:12:09.942 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:12:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.942+0000 7f89d1600640 1 -- 172.21.15.83:0/1295136350 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f89ac0678f0 msgr2=0x7f89ac069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.942+0000 7f89d1600640 1 --2- 172.21.15.83:0/1295136350 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f89ac0678f0 0x7f89ac069db0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0ad000 tx=0x7f89bc0a6000 comp rx=0 tx=0).stop 2024-04-25T00:12:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.942+0000 7f89d1600640 1 -- 172.21.15.83:0/1295136350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89cc1455d0 msgr2=0x7f89cc140a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.942+0000 7f89d1600640 1 --2- 172.21.15.83:0/1295136350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89cc1455d0 0x7f89cc140a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f89c400c730 tx=0x7f89c400cbf0 comp rx=0 tx=0).stop 2024-04-25T00:12:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.942+0000 7f89d1600640 1 -- 172.21.15.83:0/1295136350 shutdown_connections 2024-04-25T00:12:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.942+0000 7f89d1600640 1 --2- 172.21.15.83:0/1295136350 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f89ac0678f0 0x7f89ac069db0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.942+0000 7f89d1600640 1 --2- 172.21.15.83:0/1295136350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89cc1455d0 0x7f89cc140a20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.942+0000 7f89d1600640 1 --2- 172.21.15.83:0/1295136350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc0d0450 0x7f89cc1404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.942+0000 7f89d1600640 1 --2- 172.21.15.83:0/1295136350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc0ce080 0x7f89cc13baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.942+0000 7f89d1600640 1 -- 172.21.15.83:0/1295136350 >> 172.21.15.83:0/1295136350 conn(0x7f89cc06b3b0 msgr2=0x7f89cc142cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.942+0000 7f89d1600640 1 -- 172.21.15.83:0/1295136350 shutdown_connections 2024-04-25T00:12:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:09.946+0000 7f89d1600640 1 -- 172.21.15.83:0/1295136350 wait complete. 2024-04-25T00:12:09.967 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:12:10.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99ef600640 1 Processor -- start 2024-04-25T00:12:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99ef600640 1 -- start start 2024-04-25T00:12:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99ef600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f99e81455d0 0x7f99e806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99ef600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99e80ce080 0x7f99e80cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99ef600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99e80ce610 con 0x7f99e80d0280 2024-04-25T00:12:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99ef600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f99e80d06f0 con 0x7f99e81455d0 2024-04-25T00:12:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99ef600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f99e80ce8f0 con 0x7f99e80ce080 2024-04-25T00:12:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99ecc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f99e81455d0 0x7f99e806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99ed600640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f99e80d0280 0x7f99e815f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46108/0 (socket says 172.21.15.83:46108) 2024-04-25T00:12:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99ed600640 1 -- 172.21.15.83:0/3433022713 learned_addr learned my addr 172.21.15.83:0/3433022713 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99e7e00640 1 --2- 172.21.15.83:0/3433022713 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99e80ce080 0x7f99e80cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99e7400640 1 -- 172.21.15.83:0/3433022713 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 549416862 0 0) 0x7f99e80ce610 con 0x7f99e80d0280 2024-04-25T00:12:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99e7400640 1 -- 172.21.15.83:0/3433022713 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99d0003530 con 0x7f99e80d0280 2024-04-25T00:12:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99ecc00640 1 -- 172.21.15.83:0/3433022713 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99e80ce080 msgr2=0x7f99e80cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99ecc00640 1 --2- 172.21.15.83:0/3433022713 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99e80ce080 0x7f99e80cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99ecc00640 1 -- 172.21.15.83:0/3433022713 >> v1:172.21.15.83:6789/0 conn(0x7f99e80d0280 legacy=0x7f99e815f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99ecc00640 1 -- 172.21.15.83:0/3433022713 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e80cebd0 con 0x7f99e81455d0 2024-04-25T00:12:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.110+0000 7f99e7e00640 1 --2- 172.21.15.83:0/3433022713 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99e80ce080 0x7f99e80cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.114+0000 7f99ecc00640 1 --2- 172.21.15.83:0/3433022713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99e81455d0 0x7f99e806d7e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0097c0 tx=0x7f99dc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6023f60632fe85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.114+0000 7f99e7400640 1 -- 172.21.15.83:0/3433022713 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc0aa600 con 0x7f99e81455d0 2024-04-25T00:12:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.114+0000 7f99e7400640 1 -- 172.21.15.83:0/3433022713 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99dc0af070 con 0x7f99e81455d0 2024-04-25T00:12:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.114+0000 7f99e7400640 1 -- 172.21.15.83:0/3433022713 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc0aa9a0 con 0x7f99e81455d0 2024-04-25T00:12:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.114+0000 7f99ef600640 1 -- 172.21.15.83:0/3433022713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99e81455d0 msgr2=0x7f99e806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.114+0000 7f99ef600640 1 --2- 172.21.15.83:0/3433022713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99e81455d0 0x7f99e806d7e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0097c0 tx=0x7f99dc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:12:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.114+0000 7f99ef600640 1 -- 172.21.15.83:0/3433022713 shutdown_connections 2024-04-25T00:12:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.114+0000 7f99ef600640 1 --2- 172.21.15.83:0/3433022713 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99e80ce080 0x7f99e80cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.114+0000 7f99ef600640 1 --2- 172.21.15.83:0/3433022713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99e81455d0 0x7f99e806d7e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.114+0000 7f99ef600640 1 -- 172.21.15.83:0/3433022713 >> 172.21.15.83:0/3433022713 conn(0x7f99e806b3b0 msgr2=0x7f99e806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:10.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.114+0000 7f99ef600640 1 -- 172.21.15.83:0/3433022713 shutdown_connections 2024-04-25T00:12:10.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.114+0000 7f99ef600640 1 -- 172.21.15.83:0/3433022713 wait complete. 2024-04-25T00:12:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.114+0000 7f99ef600640 1 Processor -- start 2024-04-25T00:12:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.114+0000 7f99ef600640 1 -- start start 2024-04-25T00:12:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99ef600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e80ce080 0x7f99e81586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99ef600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99e80d0280 0x7f99e813b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99ef600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99e81455d0 0x7f99e815aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99ef600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8145b30 con 0x7f99e80d0280 2024-04-25T00:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99ef600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99e81459b0 con 0x7f99e80ce080 2024-04-25T00:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99ef600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f99e8168cd0 con 0x7f99e81455d0 2024-04-25T00:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99ecc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e80ce080 0x7f99e81586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99e7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99e80d0280 0x7f99e813b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99ecc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e80ce080 0x7f99e81586f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:43876/0 (socket says 172.21.15.83:43876) 2024-04-25T00:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99ecc00640 1 -- 172.21.15.83:0/2441282608 learned_addr learned my addr 172.21.15.83:0/2441282608 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99ed600640 1 --2- 172.21.15.83:0/2441282608 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99e81455d0 0x7f99e815aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99e7e00640 1 -- 172.21.15.83:0/2441282608 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99e81455d0 msgr2=0x7f99e815aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99e7e00640 1 --2- 172.21.15.83:0/2441282608 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99e81455d0 0x7f99e815aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99e7e00640 1 -- 172.21.15.83:0/2441282608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e80ce080 msgr2=0x7f99e81586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99e7e00640 1 --2- 172.21.15.83:0/2441282608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e80ce080 0x7f99e81586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99e7e00640 1 -- 172.21.15.83:0/2441282608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e815c930 con 0x7f99e80d0280 2024-04-25T00:12:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99ecc00640 1 --2- 172.21.15.83:0/2441282608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e80ce080 0x7f99e81586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99e7e00640 1 --2- 172.21.15.83:0/2441282608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99e80d0280 0x7f99e813b2b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f99d400d5c0 tx=0x7f99d400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99e4c00640 1 -- 172.21.15.83:0/2441282608 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d40125f0 con 0x7f99e80d0280 2024-04-25T00:12:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99e4c00640 1 -- 172.21.15.83:0/2441282608 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d4019070 con 0x7f99e80d0280 2024-04-25T00:12:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.118+0000 7f99e4c00640 1 -- 172.21.15.83:0/2441282608 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4012990 con 0x7f99e80d0280 2024-04-25T00:12:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.122+0000 7f99ef600640 1 -- 172.21.15.83:0/2441282608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e815cc10 con 0x7f99e80d0280 2024-04-25T00:12:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.122+0000 7f99ef600640 1 -- 172.21.15.83:0/2441282608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e815d1d0 con 0x7f99e80d0280 2024-04-25T00:12:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.122+0000 7f99ef600640 1 -- 172.21.15.83:0/2441282608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e80736f0 con 0x7f99e80d0280 2024-04-25T00:12:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.122+0000 7f99e4c00640 1 -- 172.21.15.83:0/2441282608 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f99d4012b30 con 0x7f99e80d0280 2024-04-25T00:12:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.126+0000 7f99e4c00640 1 --2- 172.21.15.83:0/2441282608 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99c8067910 0x7f99c8069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.126+0000 7f99e4c00640 1 -- 172.21.15.83:0/2441282608 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6383+0+0 (secure 0 0 0) 0x7f99d4086030 con 0x7f99e80d0280 2024-04-25T00:12:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.126+0000 7f99ecc00640 1 --2- 172.21.15.83:0/2441282608 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99c8067910 0x7f99c8069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.126+0000 7f99ecc00640 1 --2- 172.21.15.83:0/2441282608 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99c8067910 0x7f99c8069dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0097c0 tx=0x7f99dc0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.130+0000 7f99e4c00640 1 -- 172.21.15.83:0/2441282608 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f99d4056360 con 0x7f99e80d0280 2024-04-25T00:12:10.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.286+0000 7f99ef600640 1 -- 172.21.15.83:0/2441282608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f99e804f5f0 con 0x7f99e80d0280 2024-04-25T00:12:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.942+0000 7f99e4c00640 1 -- 172.21.15.83:0/2441282608 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7f99d405a660 con 0x7f99e80d0280 2024-04-25T00:12:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-25T00:12:10.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.946+0000 7f99ef600640 1 -- 172.21.15.83:0/2441282608 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99c8067910 msgr2=0x7f99c8069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:10.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.946+0000 7f99ef600640 1 --2- 172.21.15.83:0/2441282608 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99c8067910 0x7f99c8069dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0097c0 tx=0x7f99dc0a7000 comp rx=0 tx=0).stop 2024-04-25T00:12:10.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.946+0000 7f99ef600640 1 -- 172.21.15.83:0/2441282608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99e80d0280 msgr2=0x7f99e813b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:10.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.946+0000 7f99ef600640 1 --2- 172.21.15.83:0/2441282608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99e80d0280 0x7f99e813b2b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f99d400d5c0 tx=0x7f99d400da80 comp rx=0 tx=0).stop 2024-04-25T00:12:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.946+0000 7f99ef600640 1 -- 172.21.15.83:0/2441282608 shutdown_connections 2024-04-25T00:12:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.946+0000 7f99ef600640 1 --2- 172.21.15.83:0/2441282608 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99c8067910 0x7f99c8069dd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.946+0000 7f99ef600640 1 --2- 172.21.15.83:0/2441282608 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99e81455d0 0x7f99e815aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.946+0000 7f99ef600640 1 --2- 172.21.15.83:0/2441282608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99e80d0280 0x7f99e813b2b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.946+0000 7f99ef600640 1 --2- 172.21.15.83:0/2441282608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99e80ce080 0x7f99e81586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.946+0000 7f99ef600640 1 -- 172.21.15.83:0/2441282608 >> 172.21.15.83:0/2441282608 conn(0x7f99e806b3b0 msgr2=0x7f99e8144000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.946+0000 7f99ef600640 1 -- 172.21.15.83:0/2441282608 shutdown_connections 2024-04-25T00:12:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:10.946+0000 7f99ef600640 1 -- 172.21.15.83:0/2441282608 wait complete. 2024-04-25T00:12:10.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T00:12:11.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbfe00640 1 Processor -- start 2024-04-25T00:12:11.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbfe00640 1 -- start start 2024-04-25T00:12:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbbb81455d0 0x7fbbb806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbfe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbbb80ce080 0x7fbbb80cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbfe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbb80ce610 con 0x7fbbb80d0280 2024-04-25T00:12:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbfe00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbbb80d06d0 con 0x7fbbb80ce080 2024-04-25T00:12:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbfe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbbb80ce8f0 con 0x7fbbb81455d0 2024-04-25T00:12:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbca00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbbb80ce080 0x7fbbb80cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbca00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbbb80ce080 0x7fbbb80cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60714/0 (socket says 172.21.15.83:60714) 2024-04-25T00:12:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbca00640 1 -- 172.21.15.83:0/1156872883 learned_addr learned my addr 172.21.15.83:0/1156872883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:11.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbd400640 1 --2- 172.21.15.83:0/1156872883 >> v2:172.21.15.86:3300/0 conn(0x7fbbb81455d0 0x7fbbb806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:11.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbca00640 1 -- 172.21.15.83:0/1156872883 >> v2:172.21.15.86:3300/0 conn(0x7fbbb81455d0 msgr2=0x7fbbb806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:11.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbca00640 1 --2- 172.21.15.83:0/1156872883 >> v2:172.21.15.86:3300/0 conn(0x7fbbb81455d0 0x7fbbb806d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:11.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbca00640 1 -- 172.21.15.83:0/1156872883 >> v1:172.21.15.86:6789/0 conn(0x7fbbb80d0280 legacy=0x7fbbb815f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:11.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.114+0000 7fbbbca00640 1 -- 172.21.15.83:0/1156872883 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb80cebd0 con 0x7fbbb80ce080 2024-04-25T00:12:11.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.118+0000 7fbbbca00640 1 --2- 172.21.15.83:0/1156872883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb80ce080 0x7fbbb80cfd40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80d0a20 tx=0x7fbba401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ab9755b40bda9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:11.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.118+0000 7fbbb7e00640 1 -- 172.21.15.83:0/1156872883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba40a6040 con 0x7fbbb80ce080 2024-04-25T00:12:11.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.118+0000 7fbbb7e00640 1 -- 172.21.15.83:0/1156872883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba40af070 con 0x7fbbb80ce080 2024-04-25T00:12:11.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.118+0000 7fbbb7e00640 1 -- 172.21.15.83:0/1156872883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba40aa860 con 0x7fbbb80ce080 2024-04-25T00:12:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.118+0000 7fbbbfe00640 1 -- 172.21.15.83:0/1156872883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb80ce080 msgr2=0x7fbbb80cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.118+0000 7fbbbfe00640 1 --2- 172.21.15.83:0/1156872883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb80ce080 0x7fbbb80cfd40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80d0a20 tx=0x7fbba401c4e0 comp rx=0 tx=0).stop 2024-04-25T00:12:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.118+0000 7fbbbfe00640 1 -- 172.21.15.83:0/1156872883 shutdown_connections 2024-04-25T00:12:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.118+0000 7fbbbfe00640 1 --2- 172.21.15.83:0/1156872883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb80ce080 0x7fbbb80cfd40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.118+0000 7fbbbfe00640 1 --2- 172.21.15.83:0/1156872883 >> v2:172.21.15.86:3300/0 conn(0x7fbbb81455d0 0x7fbbb806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.118+0000 7fbbbfe00640 1 -- 172.21.15.83:0/1156872883 >> 172.21.15.83:0/1156872883 conn(0x7fbbb806b3b0 msgr2=0x7fbbb806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.118+0000 7fbbbfe00640 1 -- 172.21.15.83:0/1156872883 shutdown_connections 2024-04-25T00:12:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.118+0000 7fbbbfe00640 1 -- 172.21.15.83:0/1156872883 wait complete. 2024-04-25T00:12:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbfe00640 1 Processor -- start 2024-04-25T00:12:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbfe00640 1 -- start start 2024-04-25T00:12:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb80ce080 0x7fbbb8157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbfe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbb80d0280 0x7fbbb813b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb81455d0 0x7fbbb8072530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbfe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb8145b30 con 0x7fbbb81455d0 2024-04-25T00:12:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb81459b0 con 0x7fbbb80ce080 2024-04-25T00:12:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbfe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb8168cd0 con 0x7fbbb80d0280 2024-04-25T00:12:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb80ce080 0x7fbbb8157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb81455d0 0x7fbbb8072530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbb80d0280 0x7fbbb813b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:11.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb80ce080 0x7fbbb8157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:43892/0 (socket says 172.21.15.83:43892) 2024-04-25T00:12:11.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbd400640 1 -- 172.21.15.83:0/4121960575 learned_addr learned my addr 172.21.15.83:0/4121960575 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbde00640 1 -- 172.21.15.83:0/4121960575 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbb80d0280 msgr2=0x7fbbb813b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbde00640 1 --2- 172.21.15.83:0/4121960575 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbb80d0280 0x7fbbb813b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbde00640 1 -- 172.21.15.83:0/4121960575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb80ce080 msgr2=0x7fbbb8157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbde00640 1 --2- 172.21.15.83:0/4121960575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb80ce080 0x7fbbb8157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbde00640 1 -- 172.21.15.83:0/4121960575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb8156560 con 0x7fbbb81455d0 2024-04-25T00:12:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbd400640 1 --2- 172.21.15.83:0/4121960575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb80ce080 0x7fbbb8157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbde00640 1 --2- 172.21.15.83:0/4121960575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb81455d0 0x7fbbb8072530 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000c410 tx=0x7fbbb000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbb5600640 1 -- 172.21.15.83:0/4121960575 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0017070 con 0x7fbbb81455d0 2024-04-25T00:12:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbb5600640 1 -- 172.21.15.83:0/4121960575 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbb0005230 con 0x7fbbb81455d0 2024-04-25T00:12:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbb5600640 1 -- 172.21.15.83:0/4121960575 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0004590 con 0x7fbbb81455d0 2024-04-25T00:12:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbfe00640 1 -- 172.21.15.83:0/4121960575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb8156840 con 0x7fbbb81455d0 2024-04-25T00:12:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbfe00640 1 -- 172.21.15.83:0/4121960575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb8156e00 con 0x7fbbb81455d0 2024-04-25T00:12:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.122+0000 7fbbbfe00640 1 -- 172.21.15.83:0/4121960575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb80736f0 con 0x7fbbb81455d0 2024-04-25T00:12:11.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.126+0000 7fbbb5600640 1 -- 172.21.15.83:0/4121960575 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbbb001a070 con 0x7fbbb81455d0 2024-04-25T00:12:11.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.126+0000 7fbbb5600640 1 --2- 172.21.15.83:0/4121960575 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbb980678c0 0x7fbb98069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:11.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.126+0000 7fbbb5600640 1 -- 172.21.15.83:0/4121960575 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7fbbb0084f60 con 0x7fbbb81455d0 2024-04-25T00:12:11.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.126+0000 7fbbbd400640 1 --2- 172.21.15.83:0/4121960575 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbb980678c0 0x7fbb98069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:11.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.126+0000 7fbbbd400640 1 --2- 172.21.15.83:0/4121960575 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbb980678c0 0x7fbb98069d80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbbac007860 tx=0x7fbbac0098b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:11.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.130+0000 7fbbb5600640 1 -- 172.21.15.83:0/4121960575 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbbb0055320 con 0x7fbbb81455d0 2024-04-25T00:12:11.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.286+0000 7fbbbfe00640 1 -- 172.21.15.83:0/4121960575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbbb804f5f0 con 0x7fbbb81455d0 2024-04-25T00:12:11.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.946+0000 7fbbb5600640 1 -- 172.21.15.83:0/4121960575 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7fbbb0059620 con 0x7fbbb81455d0 2024-04-25T00:12:11.946 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data_ec' removed 2024-04-25T00:12:11.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.946+0000 7fbbbfe00640 1 -- 172.21.15.83:0/4121960575 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbb980678c0 msgr2=0x7fbb98069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:11.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.946+0000 7fbbbfe00640 1 --2- 172.21.15.83:0/4121960575 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbb980678c0 0x7fbb98069d80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbbac007860 tx=0x7fbbac0098b0 comp rx=0 tx=0).stop 2024-04-25T00:12:11.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.946+0000 7fbbbfe00640 1 -- 172.21.15.83:0/4121960575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb81455d0 msgr2=0x7fbbb8072530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.946+0000 7fbbbfe00640 1 --2- 172.21.15.83:0/4121960575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb81455d0 0x7fbbb8072530 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000c410 tx=0x7fbbb000c8d0 comp rx=0 tx=0).stop 2024-04-25T00:12:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.946+0000 7fbbbfe00640 1 -- 172.21.15.83:0/4121960575 shutdown_connections 2024-04-25T00:12:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.946+0000 7fbbbfe00640 1 --2- 172.21.15.83:0/4121960575 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbb980678c0 0x7fbb98069d80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.946+0000 7fbbbfe00640 1 --2- 172.21.15.83:0/4121960575 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbb80d0280 0x7fbbb813b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.946+0000 7fbbbfe00640 1 --2- 172.21.15.83:0/4121960575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb81455d0 0x7fbbb8072530 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.946+0000 7fbbbfe00640 1 --2- 172.21.15.83:0/4121960575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb80ce080 0x7fbbb8157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.946+0000 7fbbbfe00640 1 -- 172.21.15.83:0/4121960575 >> 172.21.15.83:0/4121960575 conn(0x7fbbb806b3b0 msgr2=0x7fbbb80b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.946+0000 7fbbbfe00640 1 -- 172.21.15.83:0/4121960575 shutdown_connections 2024-04-25T00:12:11.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:11.950+0000 7fbbbfe00640 1 -- 172.21.15.83:0/4121960575 wait complete. 2024-04-25T00:12:11.979 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:12:11.979 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:12:11.979 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:12:11.979 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:12:11.980 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:12:11.980 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:12:11.980 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:12:11.980 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:12:11.980 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:12:11.980 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:12:11.981 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:12:11.981 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:12:11.981 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:12:11.981 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:12:11.981 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:12:11.981 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:12:11.992 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:12:11.992 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:12:11.994 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:12:11.994 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:12:11.995 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:12:11.995 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:12:11.996 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:12:11.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:12:12.039 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:12:12.039 DEBUG:teuthology.parallel:result is None 2024-04-25T00:12:12.040 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:12:12.040 DEBUG:teuthology.parallel:result is None 2024-04-25T00:12:12.041 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:12:12.041 DEBUG:teuthology.parallel:result is None 2024-04-25T00:12:12.042 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:12:12.042 DEBUG:teuthology.parallel:result is None 2024-04-25T00:12:12.042 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:12:12.128 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-25T00:12:12.129 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-25T00:12:12.129 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2024-04-25T00:12:12.130 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2024-04-25T00:12:12.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.186+0000 7f9fdbc00640 1 Processor -- start 2024-04-25T00:12:12.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.186+0000 7f9fdbc00640 1 -- start start 2024-04-25T00:12:12.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.190+0000 7f9fdbc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fd40cb640 0x7f9fd406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:12.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.190+0000 7f9fdbc00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fd40711a0 con 0x7f9fd4070d10 2024-04-25T00:12:12.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.190+0000 7f9fdbc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fd406e160 con 0x7f9fd406cfd0 2024-04-25T00:12:12.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.190+0000 7f9fdbc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd406ea10 con 0x7f9fd40cb640 2024-04-25T00:12:12.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.190+0000 7f9fd9c00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9fd4070d10 0x7f9fd4165930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46140/0 (socket says 172.21.15.83:46140) 2024-04-25T00:12:12.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.190+0000 7f9fd9c00640 1 -- 172.21.15.83:0/1889509908 learned_addr learned my addr 172.21.15.83:0/1889509908 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:12.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.190+0000 7f9fd3e00640 1 --2- 172.21.15.83:0/1889509908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fd40cb640 0x7f9fd406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:12.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.190+0000 7f9fd3400640 1 -- 172.21.15.83:0/1889509908 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3100686848 0 0) 0x7f9fcc0014a0 con 0x7f9fd4070d10 2024-04-25T00:12:12.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.190+0000 7f9fd3400640 1 -- 172.21.15.83:0/1889509908 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fb8003530 con 0x7f9fd4070d10 2024-04-25T00:12:12.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.190+0000 7f9fd3400640 1 -- 172.21.15.83:0/1889509908 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3365110417 0 0) 0x7f9fd406e160 con 0x7f9fd406cfd0 2024-04-25T00:12:12.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.190+0000 7f9fd3400640 1 -- 172.21.15.83:0/1889509908 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fcc0014a0 con 0x7f9fd406cfd0 2024-04-25T00:12:12.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.190+0000 7f9fd3e00640 1 -- 172.21.15.83:0/1889509908 >> v1:172.21.15.86:6789/0 conn(0x7f9fd406cfd0 legacy=0x7f9fd406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:12.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.190+0000 7f9fd3e00640 1 -- 172.21.15.83:0/1889509908 >> v1:172.21.15.83:6789/0 conn(0x7f9fd4070d10 legacy=0x7f9fd4165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:12.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.190+0000 7f9fd3e00640 1 -- 172.21.15.83:0/1889509908 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd406ecf0 con 0x7f9fd40cb640 2024-04-25T00:12:12.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.194+0000 7f9fd3e00640 1 --2- 172.21.15.83:0/1889509908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fd40cb640 0x7f9fd406dac0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00097f0 tx=0x7f9fc001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ac3679c1a8320f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:12.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.194+0000 7f9fd3400640 1 -- 172.21.15.83:0/1889509908 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00ab5f0 con 0x7f9fd40cb640 2024-04-25T00:12:12.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.194+0000 7f9fd3400640 1 -- 172.21.15.83:0/1889509908 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fc00b0070 con 0x7f9fd40cb640 2024-04-25T00:12:12.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.194+0000 7f9fd3400640 1 -- 172.21.15.83:0/1889509908 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00ab9b0 con 0x7f9fd40cb640 2024-04-25T00:12:12.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.194+0000 7f9fdbc00640 1 -- 172.21.15.83:0/1889509908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fd40cb640 msgr2=0x7f9fd406dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:12.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.194+0000 7f9fdbc00640 1 --2- 172.21.15.83:0/1889509908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fd40cb640 0x7f9fd406dac0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00097f0 tx=0x7f9fc001c4e0 comp rx=0 tx=0).stop 2024-04-25T00:12:12.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.194+0000 7f9fdbc00640 1 -- 172.21.15.83:0/1889509908 shutdown_connections 2024-04-25T00:12:12.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.194+0000 7f9fdbc00640 1 --2- 172.21.15.83:0/1889509908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fd40cb640 0x7f9fd406dac0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:12.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.194+0000 7f9fdbc00640 1 -- 172.21.15.83:0/1889509908 >> 172.21.15.83:0/1889509908 conn(0x7f9fd40cfa60 msgr2=0x7f9fd40cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:12.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.194+0000 7f9fdbc00640 1 -- 172.21.15.83:0/1889509908 shutdown_connections 2024-04-25T00:12:12.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.194+0000 7f9fdbc00640 1 -- 172.21.15.83:0/1889509908 wait complete. 2024-04-25T00:12:12.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.194+0000 7f9fdbc00640 1 Processor -- start 2024-04-25T00:12:12.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.194+0000 7f9fdbc00640 1 -- start start 2024-04-25T00:12:12.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fdbc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd406cfd0 0x7f9fd415a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:12.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fdbc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fd4070d10 0x7f9fd415d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:12.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fdbc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fd40cb640 0x7f9fd415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:12.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fdbc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd4169c00 con 0x7f9fd40cb640 2024-04-25T00:12:12.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fdbc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd4169a80 con 0x7f9fd406cfd0 2024-04-25T00:12:12.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fdbc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd4169d80 con 0x7f9fd4070d10 2024-04-25T00:12:12.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fd40cb640 0x7f9fd415dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:12.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fd40cb640 0x7f9fd415dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60732/0 (socket says 172.21.15.83:60732) 2024-04-25T00:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd9c00640 1 -- 172.21.15.83:0/2773294017 learned_addr learned my addr 172.21.15.83:0/2773294017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd3e00640 1 --2- 172.21.15.83:0/2773294017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fd4070d10 0x7f9fd415d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd9200640 1 --2- 172.21.15.83:0/2773294017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd406cfd0 0x7f9fd415a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd9c00640 1 -- 172.21.15.83:0/2773294017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fd4070d10 msgr2=0x7f9fd415d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd9c00640 1 --2- 172.21.15.83:0/2773294017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fd4070d10 0x7f9fd415d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd9c00640 1 -- 172.21.15.83:0/2773294017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd406cfd0 msgr2=0x7f9fd415a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd9c00640 1 --2- 172.21.15.83:0/2773294017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd406cfd0 0x7f9fd415a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd9c00640 1 -- 172.21.15.83:0/2773294017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd413a150 con 0x7f9fd40cb640 2024-04-25T00:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd9c00640 1 --2- 172.21.15.83:0/2773294017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fd40cb640 0x7f9fd415dd80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc001fa0 tx=0x7f9fcc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd0c00640 1 -- 172.21.15.83:0/2773294017 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc017070 con 0x7f9fd40cb640 2024-04-25T00:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd0c00640 1 -- 172.21.15.83:0/2773294017 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fcc012400 con 0x7f9fd40cb640 2024-04-25T00:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fdbc00640 1 -- 172.21.15.83:0/2773294017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd413a430 con 0x7f9fd40cb640 2024-04-25T00:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd0c00640 1 -- 172.21.15.83:0/2773294017 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0127f0 con 0x7f9fd40cb640 2024-04-25T00:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fdbc00640 1 -- 172.21.15.83:0/2773294017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd413a9f0 con 0x7f9fd40cb640 2024-04-25T00:12:12.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fdbc00640 1 -- 172.21.15.83:0/2773294017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd4068230 con 0x7f9fd40cb640 2024-04-25T00:12:12.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.198+0000 7f9fd0c00640 1 -- 172.21.15.83:0/2773294017 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9fcc01a070 con 0x7f9fd40cb640 2024-04-25T00:12:12.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.202+0000 7f9fd0c00640 1 --2- 172.21.15.83:0/2773294017 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9fb40678c0 0x7f9fb4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:12.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.202+0000 7f9fd0c00640 1 -- 172.21.15.83:0/2773294017 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5335+0+0 (secure 0 0 0) 0x7f9fcc085af0 con 0x7f9fd40cb640 2024-04-25T00:12:12.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.202+0000 7f9fd9200640 1 --2- 172.21.15.83:0/2773294017 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9fb40678c0 0x7f9fb4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:12.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.202+0000 7f9fd9200640 1 --2- 172.21.15.83:0/2773294017 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9fb40678c0 0x7f9fb4069d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4007fd0 tx=0x7f9fc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:12.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.206+0000 7f9fd0c00640 1 -- 172.21.15.83:0/2773294017 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9fd40711a0 con 0x7f9fd40cb640 2024-04-25T00:12:12.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.374+0000 7f9fdbc00640 1 -- 172.21.15.83:0/2773294017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9fd413ce00 con 0x7f9fd40cb640 2024-04-25T00:12:12.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.974+0000 7f9fd0c00640 1 -- 172.21.15.83:0/2773294017 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7f9fcc056170 con 0x7f9fd40cb640 2024-04-25T00:12:12.979 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-25T00:12:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.978+0000 7f9fdbc00640 1 -- 172.21.15.83:0/2773294017 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9fb40678c0 msgr2=0x7f9fb4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.978+0000 7f9fdbc00640 1 --2- 172.21.15.83:0/2773294017 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9fb40678c0 0x7f9fb4069d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4007fd0 tx=0x7f9fc4009040 comp rx=0 tx=0).stop 2024-04-25T00:12:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.978+0000 7f9fdbc00640 1 -- 172.21.15.83:0/2773294017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fd40cb640 msgr2=0x7f9fd415dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.978+0000 7f9fdbc00640 1 --2- 172.21.15.83:0/2773294017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fd40cb640 0x7f9fd415dd80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc001fa0 tx=0x7f9fcc00a430 comp rx=0 tx=0).stop 2024-04-25T00:12:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.982+0000 7f9fdbc00640 1 -- 172.21.15.83:0/2773294017 shutdown_connections 2024-04-25T00:12:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.982+0000 7f9fdbc00640 1 --2- 172.21.15.83:0/2773294017 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9fb40678c0 0x7f9fb4069d80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.982+0000 7f9fdbc00640 1 --2- 172.21.15.83:0/2773294017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fd4070d10 0x7f9fd415d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.982+0000 7f9fdbc00640 1 --2- 172.21.15.83:0/2773294017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fd40cb640 0x7f9fd415dd80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.982+0000 7f9fdbc00640 1 --2- 172.21.15.83:0/2773294017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd406cfd0 0x7f9fd415a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.982+0000 7f9fdbc00640 1 -- 172.21.15.83:0/2773294017 >> 172.21.15.83:0/2773294017 conn(0x7f9fd40cfa60 msgr2=0x7f9fd40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:12.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.982+0000 7f9fdbc00640 1 -- 172.21.15.83:0/2773294017 shutdown_connections 2024-04-25T00:12:12.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:12.982+0000 7f9fdbc00640 1 -- 172.21.15.83:0/2773294017 wait complete. 2024-04-25T00:12:13.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:12:13.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9bd000640 1 Processor -- start 2024-04-25T00:12:13.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9bd000640 1 -- start start 2024-04-25T00:12:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9bd000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe9b81553e0 0x7fe9b806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9bd000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9b80b4a70 0x7fe9b80b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9bd000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b806ef80 con 0x7fe9b806ea00 2024-04-25T00:12:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9bd000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe9b815b510 con 0x7fe9b81553e0 2024-04-25T00:12:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9bd000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b815b7f0 con 0x7fe9b80b4a70 2024-04-25T00:12:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9b5e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe9b81553e0 0x7fe9b806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9b5400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe9b806ea00 0x7fe9b80b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46150/0 (socket says 172.21.15.83:46150) 2024-04-25T00:12:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9b5400640 1 -- 172.21.15.83:0/4168958287 learned_addr learned my addr 172.21.15.83:0/4168958287 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9b6800640 1 --2- 172.21.15.83:0/4168958287 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9b80b4a70 0x7fe9b80b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9b5e00640 1 -- 172.21.15.83:0/4168958287 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9b80b4a70 msgr2=0x7fe9b80b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9b5e00640 1 --2- 172.21.15.83:0/4168958287 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9b80b4a70 0x7fe9b80b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9b5e00640 1 -- 172.21.15.83:0/4168958287 >> v1:172.21.15.83:6789/0 conn(0x7fe9b806ea00 legacy=0x7fe9b80b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9b5e00640 1 -- 172.21.15.83:0/4168958287 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b815bad0 con 0x7fe9b81553e0 2024-04-25T00:12:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.142+0000 7fe9b4a00640 1 -- 172.21.15.83:0/4168958287 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1024067553 0 0) 0x7fe9b806ef80 con 0x7fe9b806ea00 2024-04-25T00:12:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.146+0000 7fe9b5e00640 1 --2- 172.21.15.83:0/4168958287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9b81553e0 0x7fe9b806e4c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009d90 tx=0x7fe9a801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f688a0ad2f5f16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.146+0000 7fe9b4a00640 1 -- 172.21.15.83:0/4168958287 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80b2070 con 0x7fe9b81553e0 2024-04-25T00:12:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.146+0000 7fe9b4a00640 1 -- 172.21.15.83:0/4168958287 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a80a58b0 con 0x7fe9b81553e0 2024-04-25T00:12:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.146+0000 7fe9b4a00640 1 -- 172.21.15.83:0/4168958287 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80a5c80 con 0x7fe9b81553e0 2024-04-25T00:12:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.146+0000 7fe9bd000640 1 -- 172.21.15.83:0/4168958287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9b81553e0 msgr2=0x7fe9b806e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.146+0000 7fe9bd000640 1 --2- 172.21.15.83:0/4168958287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9b81553e0 0x7fe9b806e4c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009d90 tx=0x7fe9a801c8e0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.146+0000 7fe9bd000640 1 -- 172.21.15.83:0/4168958287 shutdown_connections 2024-04-25T00:12:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.146+0000 7fe9bd000640 1 --2- 172.21.15.83:0/4168958287 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9b80b4a70 0x7fe9b80b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.146+0000 7fe9bd000640 1 --2- 172.21.15.83:0/4168958287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9b81553e0 0x7fe9b806e4c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.146+0000 7fe9bd000640 1 -- 172.21.15.83:0/4168958287 >> 172.21.15.83:0/4168958287 conn(0x7fe9b80b1830 msgr2=0x7fe9b80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.146+0000 7fe9bd000640 1 -- 172.21.15.83:0/4168958287 shutdown_connections 2024-04-25T00:12:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.146+0000 7fe9bd000640 1 -- 172.21.15.83:0/4168958287 wait complete. 2024-04-25T00:12:13.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9bd000640 1 Processor -- start 2024-04-25T00:12:13.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9bd000640 1 -- start start 2024-04-25T00:12:13.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9bd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9b806ea00 0x7fe9b8159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9bd000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9b80b4a70 0x7fe9b815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9bd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9b81553e0 0x7fe9b8156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9bd000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b80b3a00 con 0x7fe9b806ea00 2024-04-25T00:12:13.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9bd000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b80b3880 con 0x7fe9b81553e0 2024-04-25T00:12:13.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9bd000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b80b3b80 con 0x7fe9b80b4a70 2024-04-25T00:12:13.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9b5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9b806ea00 0x7fe9b8159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9b5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9b806ea00 0x7fe9b8159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60752/0 (socket says 172.21.15.83:60752) 2024-04-25T00:12:13.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9b5e00640 1 -- 172.21.15.83:0/2909046332 learned_addr learned my addr 172.21.15.83:0/2909046332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9b6800640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9b81553e0 0x7fe9b8156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9b5e00640 1 -- 172.21.15.83:0/2909046332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9b80b4a70 msgr2=0x7fe9b815ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9b5400640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9b80b4a70 0x7fe9b815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9b5e00640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9b80b4a70 0x7fe9b815ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9b5e00640 1 -- 172.21.15.83:0/2909046332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9b81553e0 msgr2=0x7fe9b8156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9b5e00640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9b81553e0 0x7fe9b8156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9b5e00640 1 -- 172.21.15.83:0/2909046332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b815b150 con 0x7fe9b806ea00 2024-04-25T00:12:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9b6800640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9b81553e0 0x7fe9b8156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9b5400640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9b80b4a70 0x7fe9b815ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9b5e00640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9b806ea00 0x7fe9b8159590 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe9a801a510 tx=0x7fe9a8019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9ae000640 1 -- 172.21.15.83:0/2909046332 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80bd020 con 0x7fe9b806ea00 2024-04-25T00:12:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9ae000640 1 -- 172.21.15.83:0/2909046332 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a80b8670 con 0x7fe9b806ea00 2024-04-25T00:12:13.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9ae000640 1 -- 172.21.15.83:0/2909046332 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80b2040 con 0x7fe9b806ea00 2024-04-25T00:12:13.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9bd000640 1 -- 172.21.15.83:0/2909046332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b8159b70 con 0x7fe9b806ea00 2024-04-25T00:12:13.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9bd000640 1 -- 172.21.15.83:0/2909046332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b815a130 con 0x7fe9b806ea00 2024-04-25T00:12:13.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.150+0000 7fe9bd000640 1 -- 172.21.15.83:0/2909046332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b80737f0 con 0x7fe9b806ea00 2024-04-25T00:12:13.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.154+0000 7fe9ae000640 1 -- 172.21.15.83:0/2909046332 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe9a80c0020 con 0x7fe9b806ea00 2024-04-25T00:12:13.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.154+0000 7fe9ae000640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9980678c0 0x7fe998069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.154+0000 7fe9ae000640 1 -- 172.21.15.83:0/2909046332 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fe9a8120ba0 con 0x7fe9b806ea00 2024-04-25T00:12:13.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.154+0000 7fe9b5400640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9980678c0 0x7fe998069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.154+0000 7fe9b5400640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9980678c0 0x7fe998069d80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4007fd0 tx=0x7fe9a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:13.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.158+0000 7fe9ae000640 1 -- 172.21.15.83:0/2909046332 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe9a80c7030 con 0x7fe9b806ea00 2024-04-25T00:12:13.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.366+0000 7fe9bd000640 1 -- 172.21.15.83:0/2909046332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe9b804f5f0 con 0x7fe9b806ea00 2024-04-25T00:12:13.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.366+0000 7fe9ae000640 1 -- 172.21.15.83:0/2909046332 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7fe9a8016560 con 0x7fe9b806ea00 2024-04-25T00:12:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:13.369 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:12:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-25T00:12:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-25T00:12:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmytlRFRAA3vHLqK3vHSB+8+TZHAuj7Q==", 2024-04-25T00:12:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:12:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:12:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-25T00:12:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm088MIRAApJvvePXyFoO8JnVErd57zQ==", 2024-04-25T00:12:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:12:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:12:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-25T00:12:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmCqZXHBAAPy5JGbEU11i4gc3gFtbGIQ==", 2024-04-25T00:12:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:12:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:12:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-25T00:12:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmiJZGKBAA5CBwH/cA+LOAj+inS/Hx2g==", 2024-04-25T00:12:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:12:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:12:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-25T00:12:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBhnylmMOiqLRAAdqrCTz4KfghoJ7cZc+b4qg==", 2024-04-25T00:12:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-25T00:12:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBknylmlp8OOBAAl+y4vBrCzDyV/ne9qICQ+w==", 2024-04-25T00:12:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-25T00:12:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBonylm85MOBhAA3ciY1UT/Qg4Pe72OA0IvnA==", 2024-04-25T00:12:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-25T00:12:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBrnylmnAbDEhAAkCQXjbiKINiftX71HBKvQA==", 2024-04-25T00:12:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.378 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-25T00:12:13.378 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBxnylmxhy2KxAAs6qjxZA+Fj5umHU/4FVqGw==", 2024-04-25T00:12:13.378 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.379 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:13.379 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:13.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.379 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-25T00:12:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB0nylmGfSlNRAA45tuuh+5ISOMlhaFdCE5rQ==", 2024-04-25T00:12:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-25T00:12:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB4nylmouyBARAAz4s4S1BSiDU9YxQ/Vj0d0A==", 2024-04-25T00:12:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-25T00:12:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB7nylmtIh0CxAApWJLvC8Zlt298VJ55gPrqw==", 2024-04-25T00:12:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-25T00:12:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmOqooLBAApte+X6YkRmrpe12YdjYekg==", 2024-04-25T00:12:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:12:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:12:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:12:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-25T00:12:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmDH93NRAASQzJNC3nI0NeU8cv951eBA==", 2024-04-25T00:12:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:12:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:12:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:12:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-25T00:12:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmHw3+MRAAciXRmexPxmrf7U2jKwpZQQ==", 2024-04-25T00:12:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:12:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:12:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:12:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-25T00:12:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBfnylmIsMCABAAu+yzPRupFFqA/yH4GSZ+8w==", 2024-04-25T00:12:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:12:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:12:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:12:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-25T00:12:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdnylmZ7SKNRAArVHYbh7fBiOUNRTenIwT1w==", 2024-04-25T00:12:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:12:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:12:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:12:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-mds", 2024-04-25T00:12:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCBnylm3tmUGBAAiSckIMS+BRSKzwMJNllyow==", 2024-04-25T00:12:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T00:12:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T00:12:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCBnylmCeSUGBAA27pITF5k+IuxnjVcK/kZCw==", 2024-04-25T00:12:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T00:12:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-osd", 2024-04-25T00:12:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCBnylmzOyUGBAAjmJ3hjULKw7z0hxc0tKbCw==", 2024-04-25T00:12:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T00:12:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T00:12:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCBnylm+PSUGBAA4VSPuTRs2lnod5hTU2O4Rw==", 2024-04-25T00:12:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T00:12:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T00:12:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCBnylmc/2UGBAAP3/JOO3A9Ocu5nJBtQwM+Q==", 2024-04-25T00:12:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T00:12:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T00:12:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCBnylmDQaVGBAAN8D6f9LiyXWXGok6DCf4QQ==", 2024-04-25T00:12:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T00:12:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-25T00:12:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm6TMqERAAiPL81HRFrlz4ku20HemS4w==", 2024-04-25T00:12:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:12:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-25T00:12:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-25T00:12:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm9VRpDRAA5XUtYwjiWBpF1ObJ/Y5g0w==", 2024-04-25T00:12:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:12:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-25T00:12:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T00:12:13.391 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:12:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.370+0000 7fe9bd000640 1 -- 172.21.15.83:0/2909046332 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9980678c0 msgr2=0x7fe998069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.370+0000 7fe9bd000640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9980678c0 0x7fe998069d80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4007fd0 tx=0x7fe9a4009040 comp rx=0 tx=0).stop 2024-04-25T00:12:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.370+0000 7fe9bd000640 1 -- 172.21.15.83:0/2909046332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9b806ea00 msgr2=0x7fe9b8159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.370+0000 7fe9bd000640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9b806ea00 0x7fe9b8159590 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe9a801a510 tx=0x7fe9a8019680 comp rx=0 tx=0).stop 2024-04-25T00:12:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.370+0000 7fe9bd000640 1 -- 172.21.15.83:0/2909046332 shutdown_connections 2024-04-25T00:12:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.370+0000 7fe9bd000640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9980678c0 0x7fe998069d80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.370+0000 7fe9bd000640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9b80b4a70 0x7fe9b815ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.370+0000 7fe9bd000640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9b81553e0 0x7fe9b8156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.370+0000 7fe9bd000640 1 --2- 172.21.15.83:0/2909046332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9b806ea00 0x7fe9b8159590 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.370+0000 7fe9bd000640 1 -- 172.21.15.83:0/2909046332 >> 172.21.15.83:0/2909046332 conn(0x7fe9b80b1830 msgr2=0x7fe9b8071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:13.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.370+0000 7fe9bd000640 1 -- 172.21.15.83:0/2909046332 shutdown_connections 2024-04-25T00:12:13.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.370+0000 7fe9bd000640 1 -- 172.21.15.83:0/2909046332 wait complete. 2024-04-25T00:12:13.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T00:12:13.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.530+0000 7fc68fa00640 1 Processor -- start 2024-04-25T00:12:13.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.530+0000 7fc68fa00640 1 -- start start 2024-04-25T00:12:13.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc68fa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc68806cfd0 0x7fc688070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc68fa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6880cb640 0x7fc688165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc68fa00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc68806e160 con 0x7fc68806d3b0 2024-04-25T00:12:13.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc68fa00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc68806df20 con 0x7fc68806cfd0 2024-04-25T00:12:13.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc68fa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc68806ea10 con 0x7fc6880cb640 2024-04-25T00:12:13.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc67fe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc68806d3b0 0x7fc68806d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46152/0 (socket says 172.21.15.83:46152) 2024-04-25T00:12:13.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc67fe00640 1 -- 172.21.15.83:0/1321510560 learned_addr learned my addr 172.21.15.83:0/1321510560 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:13.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc68d000640 1 --2- 172.21.15.83:0/1321510560 >> v2:172.21.15.83:3300/0 conn(0x7fc68806cfd0 0x7fc688070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc68da00640 1 --2- 172.21.15.83:0/1321510560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6880cb640 0x7fc688165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc68d000640 1 -- 172.21.15.83:0/1321510560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6880cb640 msgr2=0x7fc688165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:13.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc68d000640 1 --2- 172.21.15.83:0/1321510560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6880cb640 0x7fc688165930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc68d000640 1 -- 172.21.15.83:0/1321510560 >> v1:172.21.15.83:6789/0 conn(0x7fc68806d3b0 legacy=0x7fc68806d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc68d000640 1 -- 172.21.15.83:0/1321510560 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68806ecf0 con 0x7fc68806cfd0 2024-04-25T00:12:13.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc687e00640 1 -- 172.21.15.83:0/1321510560 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4293221691 0 0) 0x7fc68806e160 con 0x7fc68806d3b0 2024-04-25T00:12:13.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc68d000640 1 --2- 172.21.15.83:0/1321510560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc68806cfd0 0x7fc688070d10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc6780097c0 tx=0x7fc67801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df70fdef689e6b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:13.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.534+0000 7fc687e00640 1 -- 172.21.15.83:0/1321510560 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780b0070 con 0x7fc68806cfd0 2024-04-25T00:12:13.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc687e00640 1 -- 172.21.15.83:0/1321510560 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6780ab600 con 0x7fc68806cfd0 2024-04-25T00:12:13.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc687e00640 1 -- 172.21.15.83:0/1321510560 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780ab9d0 con 0x7fc68806cfd0 2024-04-25T00:12:13.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 -- 172.21.15.83:0/1321510560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc68806cfd0 msgr2=0x7fc688070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 --2- 172.21.15.83:0/1321510560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc68806cfd0 0x7fc688070d10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc6780097c0 tx=0x7fc67801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 -- 172.21.15.83:0/1321510560 shutdown_connections 2024-04-25T00:12:13.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 --2- 172.21.15.83:0/1321510560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6880cb640 0x7fc688165930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 --2- 172.21.15.83:0/1321510560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc68806cfd0 0x7fc688070d10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 -- 172.21.15.83:0/1321510560 >> 172.21.15.83:0/1321510560 conn(0x7fc6880cfa60 msgr2=0x7fc6880cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:13.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 -- 172.21.15.83:0/1321510560 shutdown_connections 2024-04-25T00:12:13.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 -- 172.21.15.83:0/1321510560 wait complete. 2024-04-25T00:12:13.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 Processor -- start 2024-04-25T00:12:13.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 -- start start 2024-04-25T00:12:13.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc68806cfd0 0x7fc688139540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc68806d3b0 0x7fc68815ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6880cb640 0x7fc688139b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc6881688a0 con 0x7fc68806cfd0 2024-04-25T00:12:13.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc688145970 con 0x7fc68806d3b0 2024-04-25T00:12:13.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68fa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc688168a20 con 0x7fc6880cb640 2024-04-25T00:12:13.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc68806cfd0 0x7fc688139540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc68806cfd0 0x7fc688139540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60764/0 (socket says 172.21.15.83:60764) 2024-04-25T00:12:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68d000640 1 -- 172.21.15.83:0/4068331066 learned_addr learned my addr 172.21.15.83:0/4068331066 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc67fe00640 1 --2- 172.21.15.83:0/4068331066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc68806d3b0 0x7fc68815ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68da00640 1 --2- 172.21.15.83:0/4068331066 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6880cb640 0x7fc688139b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68d000640 1 -- 172.21.15.83:0/4068331066 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6880cb640 msgr2=0x7fc688139b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68d000640 1 --2- 172.21.15.83:0/4068331066 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6880cb640 0x7fc688139b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68d000640 1 -- 172.21.15.83:0/4068331066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc68806d3b0 msgr2=0x7fc68815ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68d000640 1 --2- 172.21.15.83:0/4068331066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc68806d3b0 0x7fc68815ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.538+0000 7fc68d000640 1 -- 172.21.15.83:0/4068331066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6880cec40 con 0x7fc68806cfd0 2024-04-25T00:12:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.542+0000 7fc68d000640 1 --2- 172.21.15.83:0/4068331066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc68806cfd0 0x7fc688139540 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc6780097c0 tx=0x7fc67801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.542+0000 7fc685600640 1 -- 172.21.15.83:0/4068331066 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780b0050 con 0x7fc68806cfd0 2024-04-25T00:12:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.542+0000 7fc685600640 1 -- 172.21.15.83:0/4068331066 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6780bc070 con 0x7fc68806cfd0 2024-04-25T00:12:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.542+0000 7fc68fa00640 1 -- 172.21.15.83:0/4068331066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6880cd7b0 con 0x7fc68806cfd0 2024-04-25T00:12:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.542+0000 7fc685600640 1 -- 172.21.15.83:0/4068331066 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780b5830 con 0x7fc68806cfd0 2024-04-25T00:12:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.542+0000 7fc68fa00640 1 -- 172.21.15.83:0/4068331066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68815b180 con 0x7fc68806cfd0 2024-04-25T00:12:13.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.542+0000 7fc68fa00640 1 -- 172.21.15.83:0/4068331066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc688068230 con 0x7fc68806cfd0 2024-04-25T00:12:13.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.542+0000 7fc685600640 1 -- 172.21.15.83:0/4068331066 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc6780b59d0 con 0x7fc68806cfd0 2024-04-25T00:12:13.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.542+0000 7fc685600640 1 --2- 172.21.15.83:0/4068331066 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc6640678c0 0x7fc664069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.542+0000 7fc685600640 1 -- 172.21.15.83:0/4068331066 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fc6781202d0 con 0x7fc68806cfd0 2024-04-25T00:12:13.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.542+0000 7fc67fe00640 1 --2- 172.21.15.83:0/4068331066 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc6640678c0 0x7fc664069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.546+0000 7fc67fe00640 1 --2- 172.21.15.83:0/4068331066 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc6640678c0 0x7fc664069d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc6880c9bb0 tx=0x7fc670007b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:13.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.546+0000 7fc685600640 1 -- 172.21.15.83:0/4068331066 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc678016090 con 0x7fc68806cfd0 2024-04-25T00:12:13.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.758+0000 7fc68fa00640 1 -- 172.21.15.83:0/4068331066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fc68813af80 con 0x7fc68806cfd0 2024-04-25T00:12:13.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.758+0000 7fc685600640 1 -- 172.21.15.83:0/4068331066 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fc6780f0fc0 con 0x7fc68806cfd0 2024-04-25T00:12:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.762+0000 7fc68fa00640 1 -- 172.21.15.83:0/4068331066 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc6640678c0 msgr2=0x7fc664069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.762+0000 7fc68fa00640 1 --2- 172.21.15.83:0/4068331066 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc6640678c0 0x7fc664069d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc6880c9bb0 tx=0x7fc670007b60 comp rx=0 tx=0).stop 2024-04-25T00:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.762+0000 7fc68fa00640 1 -- 172.21.15.83:0/4068331066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc68806cfd0 msgr2=0x7fc688139540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.762+0000 7fc68fa00640 1 --2- 172.21.15.83:0/4068331066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc68806cfd0 0x7fc688139540 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc6780097c0 tx=0x7fc67801ace0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.762+0000 7fc68fa00640 1 -- 172.21.15.83:0/4068331066 shutdown_connections 2024-04-25T00:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.762+0000 7fc68fa00640 1 --2- 172.21.15.83:0/4068331066 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc6640678c0 0x7fc664069d80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.762+0000 7fc68fa00640 1 --2- 172.21.15.83:0/4068331066 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6880cb640 0x7fc688139b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.762+0000 7fc68fa00640 1 --2- 172.21.15.83:0/4068331066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc68806d3b0 0x7fc68815ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.762+0000 7fc68fa00640 1 --2- 172.21.15.83:0/4068331066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc68806cfd0 0x7fc688139540 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.762+0000 7fc68fa00640 1 -- 172.21.15.83:0/4068331066 >> 172.21.15.83:0/4068331066 conn(0x7fc6880cfa60 msgr2=0x7fc68813e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.762+0000 7fc68fa00640 1 -- 172.21.15.83:0/4068331066 shutdown_connections 2024-04-25T00:12:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.762+0000 7fc68fa00640 1 -- 172.21.15.83:0/4068331066 wait complete. 2024-04-25T00:12:13.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T00:12:13.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f2424a00640 1 Processor -- start 2024-04-25T00:12:13.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f2424a00640 1 -- start start 2024-04-25T00:12:13.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f2424a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f242006cfd0 0x7f2420070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f2424a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f24200cb640 0x7f2420165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f2424a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f242006e160 con 0x7f242006d3b0 2024-04-25T00:12:13.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f2424a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f242006df20 con 0x7f24200cb640 2024-04-25T00:12:13.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f2424a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f242006ea10 con 0x7f242006cfd0 2024-04-25T00:12:13.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f241e800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f24200cb640 0x7f2420165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f241e800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f24200cb640 0x7f2420165930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60766/0 (socket says 172.21.15.83:60766) 2024-04-25T00:12:13.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f241e800640 1 -- 172.21.15.83:0/3719557276 learned_addr learned my addr 172.21.15.83:0/3719557276 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:13.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f241de00640 1 --2- 172.21.15.83:0/3719557276 >> v2:172.21.15.86:3300/0 conn(0x7f242006cfd0 0x7f2420070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f241e800640 1 -- 172.21.15.83:0/3719557276 >> v2:172.21.15.86:3300/0 conn(0x7f242006cfd0 msgr2=0x7f2420070d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:13.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f241e800640 1 --2- 172.21.15.83:0/3719557276 >> v2:172.21.15.86:3300/0 conn(0x7f242006cfd0 0x7f2420070d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f241e800640 1 -- 172.21.15.83:0/3719557276 >> v1:172.21.15.86:6789/0 conn(0x7f242006d3b0 legacy=0x7f242006d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.918+0000 7f241e800640 1 -- 172.21.15.83:0/3719557276 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242006ecf0 con 0x7f24200cb640 2024-04-25T00:12:13.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.922+0000 7f241e800640 1 --2- 172.21.15.83:0/3719557276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24200cb640 0x7f2420165930 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f242006c300 tx=0x7f241801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee24772eff8b46ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:13.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.922+0000 7f241d400640 1 -- 172.21.15.83:0/3719557276 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24180a56c0 con 0x7f24200cb640 2024-04-25T00:12:13.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.922+0000 7f241d400640 1 -- 172.21.15.83:0/3719557276 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24180a5860 con 0x7f24200cb640 2024-04-25T00:12:13.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.922+0000 7f241d400640 1 -- 172.21.15.83:0/3719557276 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24180a5c30 con 0x7f24200cb640 2024-04-25T00:12:13.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.922+0000 7f2424a00640 1 -- 172.21.15.83:0/3719557276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24200cb640 msgr2=0x7f2420165930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.922+0000 7f2424a00640 1 --2- 172.21.15.83:0/3719557276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24200cb640 0x7f2420165930 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f242006c300 tx=0x7f241801c9f0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.922+0000 7f2424a00640 1 -- 172.21.15.83:0/3719557276 shutdown_connections 2024-04-25T00:12:13.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.922+0000 7f2424a00640 1 --2- 172.21.15.83:0/3719557276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24200cb640 0x7f2420165930 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.922+0000 7f2424a00640 1 --2- 172.21.15.83:0/3719557276 >> v2:172.21.15.86:3300/0 conn(0x7f242006cfd0 0x7f2420070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.922+0000 7f2424a00640 1 -- 172.21.15.83:0/3719557276 >> 172.21.15.83:0/3719557276 conn(0x7f24200cfa60 msgr2=0x7f24200cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:13.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.922+0000 7f2424a00640 1 -- 172.21.15.83:0/3719557276 shutdown_connections 2024-04-25T00:12:13.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.922+0000 7f2424a00640 1 -- 172.21.15.83:0/3719557276 wait complete. 2024-04-25T00:12:13.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2424a00640 1 Processor -- start 2024-04-25T00:12:13.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2424a00640 1 -- start start 2024-04-25T00:12:13.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2424a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f242006cfd0 0x7f242013e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2424a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f242006d3b0 0x7f24200c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2424a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24200cb640 0x7f24200c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2424a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f24201688a0 con 0x7f242006d3b0 2024-04-25T00:12:13.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2424a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2420145970 con 0x7f24200cb640 2024-04-25T00:12:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2424a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2420168a20 con 0x7f242006cfd0 2024-04-25T00:12:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2415400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f242006d3b0 0x7f24200c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f241de00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f242006cfd0 0x7f242013e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2415400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f242006d3b0 0x7f24200c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60778/0 (socket says 172.21.15.83:60778) 2024-04-25T00:12:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2415400640 1 -- 172.21.15.83:0/4101731474 learned_addr learned my addr 172.21.15.83:0/4101731474 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f241e800640 1 --2- 172.21.15.83:0/4101731474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24200cb640 0x7f24200c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2415400640 1 -- 172.21.15.83:0/4101731474 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f242006cfd0 msgr2=0x7f242013e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2415400640 1 --2- 172.21.15.83:0/4101731474 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f242006cfd0 0x7f242013e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2415400640 1 -- 172.21.15.83:0/4101731474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24200cb640 msgr2=0x7f24200c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2415400640 1 --2- 172.21.15.83:0/4101731474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24200cb640 0x7f24200c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2415400640 1 -- 172.21.15.83:0/4101731474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242013ea10 con 0x7f242006d3b0 2024-04-25T00:12:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2415400640 1 --2- 172.21.15.83:0/4101731474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f242006d3b0 0x7f24200c8db0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f240800c3a0 tx=0x7f240800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:13.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2416a00640 1 -- 172.21.15.83:0/4101731474 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24080052d0 con 0x7f242006d3b0 2024-04-25T00:12:13.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2416a00640 1 -- 172.21.15.83:0/4101731474 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2408005470 con 0x7f242006d3b0 2024-04-25T00:12:13.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2416a00640 1 -- 172.21.15.83:0/4101731474 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408007260 con 0x7f242006d3b0 2024-04-25T00:12:13.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2424a00640 1 -- 172.21.15.83:0/4101731474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24200b3b00 con 0x7f242006d3b0 2024-04-25T00:12:13.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.926+0000 7f2424a00640 1 -- 172.21.15.83:0/4101731474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24200b40c0 con 0x7f242006d3b0 2024-04-25T00:12:13.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.930+0000 7f2424a00640 1 -- 172.21.15.83:0/4101731474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2420068230 con 0x7f242006d3b0 2024-04-25T00:12:13.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.930+0000 7f2416a00640 1 -- 172.21.15.83:0/4101731474 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2408007400 con 0x7f242006d3b0 2024-04-25T00:12:13.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.930+0000 7f2416a00640 1 --2- 172.21.15.83:0/4101731474 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f23fc0678c0 0x7f23fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.930+0000 7f2416a00640 1 -- 172.21.15.83:0/4101731474 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f2408084f30 con 0x7f242006d3b0 2024-04-25T00:12:13.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.930+0000 7f241de00640 1 --2- 172.21.15.83:0/4101731474 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f23fc0678c0 0x7f23fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.930+0000 7f241de00640 1 --2- 172.21.15.83:0/4101731474 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f23fc0678c0 0x7f23fc069d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2410009ac0 tx=0x7f2410009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:13.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:13.934+0000 7f2416a00640 1 -- 172.21.15.83:0/4101731474 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f242013ea10 con 0x7f242006d3b0 2024-04-25T00:12:14.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.146+0000 7f2424a00640 1 -- 172.21.15.83:0/4101731474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f2420074a10 con 0x7f242006d3b0 2024-04-25T00:12:14.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.146+0000 7f2416a00640 1 -- 172.21.15.83:0/4101731474 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f2408055540 con 0x7f242006d3b0 2024-04-25T00:12:14.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.150+0000 7f2424a00640 1 -- 172.21.15.83:0/4101731474 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f23fc0678c0 msgr2=0x7f23fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.150+0000 7f2424a00640 1 --2- 172.21.15.83:0/4101731474 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f23fc0678c0 0x7f23fc069d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2410009ac0 tx=0x7f2410009190 comp rx=0 tx=0).stop 2024-04-25T00:12:14.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.150+0000 7f2424a00640 1 -- 172.21.15.83:0/4101731474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f242006d3b0 msgr2=0x7f24200c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.150+0000 7f2424a00640 1 --2- 172.21.15.83:0/4101731474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f242006d3b0 0x7f24200c8db0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f240800c3a0 tx=0x7f240800c860 comp rx=0 tx=0).stop 2024-04-25T00:12:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.150+0000 7f2424a00640 1 -- 172.21.15.83:0/4101731474 shutdown_connections 2024-04-25T00:12:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.150+0000 7f2424a00640 1 --2- 172.21.15.83:0/4101731474 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f23fc0678c0 0x7f23fc069d80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.150+0000 7f2424a00640 1 --2- 172.21.15.83:0/4101731474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24200cb640 0x7f24200c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.150+0000 7f2424a00640 1 --2- 172.21.15.83:0/4101731474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f242006d3b0 0x7f24200c8db0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.150+0000 7f2424a00640 1 --2- 172.21.15.83:0/4101731474 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f242006cfd0 0x7f242013e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.150+0000 7f2424a00640 1 -- 172.21.15.83:0/4101731474 >> 172.21.15.83:0/4101731474 conn(0x7f24200cfa60 msgr2=0x7f242015e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.150+0000 7f2424a00640 1 -- 172.21.15.83:0/4101731474 shutdown_connections 2024-04-25T00:12:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.150+0000 7f2424a00640 1 -- 172.21.15.83:0/4101731474 wait complete. 2024-04-25T00:12:14.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T00:12:14.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f287f200640 1 Processor -- start 2024-04-25T00:12:14.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f287f200640 1 -- start start 2024-04-25T00:12:14.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f287f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f28781446e0 0x7f28780cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f287f200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f287806ee20 0x7f2878071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f287f200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28780cb210 con 0x7f287815f350 2024-04-25T00:12:14.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f287f200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f28780715b0 con 0x7f287806ee20 2024-04-25T00:12:14.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f287f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2878144ac0 con 0x7f28781446e0 2024-04-25T00:12:14.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f287d200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f287806ee20 0x7f2878071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f286f400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f287815f350 0x7f287806e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56918/0 (socket says 172.21.15.83:56918) 2024-04-25T00:12:14.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f286f400640 1 -- 172.21.15.83:0/250717547 learned_addr learned my addr 172.21.15.83:0/250717547 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:14.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f2877e00640 1 --2- 172.21.15.83:0/250717547 >> v2:172.21.15.86:3300/0 conn(0x7f28781446e0 0x7f28780cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f287d200640 1 -- 172.21.15.83:0/250717547 >> v2:172.21.15.86:3300/0 conn(0x7f28781446e0 msgr2=0x7f28780cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:14.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f287d200640 1 --2- 172.21.15.83:0/250717547 >> v2:172.21.15.86:3300/0 conn(0x7f28781446e0 0x7f28780cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f287d200640 1 -- 172.21.15.83:0/250717547 >> v1:172.21.15.83:6789/0 conn(0x7f287815f350 legacy=0x7f287806e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f287d200640 1 -- 172.21.15.83:0/250717547 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28780cb4f0 con 0x7f287806ee20 2024-04-25T00:12:14.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.306+0000 7f2877400640 1 -- 172.21.15.83:0/250717547 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2937107877 0 0) 0x7f28780cb210 con 0x7f287815f350 2024-04-25T00:12:14.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.310+0000 7f287d200640 1 --2- 172.21.15.83:0/250717547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287806ee20 0x7f2878071fe0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f287804fb60 tx=0x7f287001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b104599266585a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.310+0000 7f2877400640 1 -- 172.21.15.83:0/250717547 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28700b2070 con 0x7f287806ee20 2024-04-25T00:12:14.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.310+0000 7f2877400640 1 -- 172.21.15.83:0/250717547 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28700a5880 con 0x7f287806ee20 2024-04-25T00:12:14.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.310+0000 7f2877400640 1 -- 172.21.15.83:0/250717547 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28700a5c50 con 0x7f287806ee20 2024-04-25T00:12:14.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.310+0000 7f287f200640 1 -- 172.21.15.83:0/250717547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287806ee20 msgr2=0x7f2878071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.310+0000 7f287f200640 1 --2- 172.21.15.83:0/250717547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287806ee20 0x7f2878071fe0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f287804fb60 tx=0x7f287001c930 comp rx=0 tx=0).stop 2024-04-25T00:12:14.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.310+0000 7f287f200640 1 -- 172.21.15.83:0/250717547 shutdown_connections 2024-04-25T00:12:14.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.310+0000 7f287f200640 1 --2- 172.21.15.83:0/250717547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287806ee20 0x7f2878071fe0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.310+0000 7f287f200640 1 --2- 172.21.15.83:0/250717547 >> v2:172.21.15.86:3300/0 conn(0x7f28781446e0 0x7f28780cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.310+0000 7f287f200640 1 -- 172.21.15.83:0/250717547 >> 172.21.15.83:0/250717547 conn(0x7f28780cfa60 msgr2=0x7f28780cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:14.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.310+0000 7f287f200640 1 -- 172.21.15.83:0/250717547 shutdown_connections 2024-04-25T00:12:14.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.310+0000 7f287f200640 1 -- 172.21.15.83:0/250717547 wait complete. 2024-04-25T00:12:14.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f287f200640 1 Processor -- start 2024-04-25T00:12:14.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f287f200640 1 -- start start 2024-04-25T00:12:14.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f287f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287806ee20 0x7f2878139d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f287f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28781446e0 0x7f2878137a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f287f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287815f350 0x7f28781f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f287f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2878072eb0 con 0x7f287806ee20 2024-04-25T00:12:14.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f287f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2878072d30 con 0x7f287815f350 2024-04-25T00:12:14.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f287f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2878073030 con 0x7f28781446e0 2024-04-25T00:12:14.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f2877e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287806ee20 0x7f2878139d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f2877e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287806ee20 0x7f2878139d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34196/0 (socket says 172.21.15.83:34196) 2024-04-25T00:12:14.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f2877e00640 1 -- 172.21.15.83:0/1761819693 learned_addr learned my addr 172.21.15.83:0/1761819693 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:14.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f2877e00640 1 -- 172.21.15.83:0/1761819693 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28781446e0 msgr2=0x7f2878137a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f2877e00640 1 --2- 172.21.15.83:0/1761819693 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28781446e0 0x7f2878137a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f287d200640 1 --2- 172.21.15.83:0/1761819693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287815f350 0x7f28781f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f2877e00640 1 -- 172.21.15.83:0/1761819693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287815f350 msgr2=0x7f28781f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f2877e00640 1 --2- 172.21.15.83:0/1761819693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287815f350 0x7f28781f6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f2877e00640 1 -- 172.21.15.83:0/1761819693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2878137fb0 con 0x7f287806ee20 2024-04-25T00:12:14.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f2877e00640 1 --2- 172.21.15.83:0/1761819693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287806ee20 0x7f2878139d00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f286800d800 tx=0x7f286800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f2874c00640 1 -- 172.21.15.83:0/1761819693 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286800b6c0 con 0x7f287806ee20 2024-04-25T00:12:14.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f2874c00640 1 -- 172.21.15.83:0/1761819693 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2868019070 con 0x7f287806ee20 2024-04-25T00:12:14.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f287f200640 1 -- 172.21.15.83:0/1761819693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28781f7180 con 0x7f287806ee20 2024-04-25T00:12:14.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f2874c00640 1 -- 172.21.15.83:0/1761819693 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286800bae0 con 0x7f287806ee20 2024-04-25T00:12:14.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f287f200640 1 -- 172.21.15.83:0/1761819693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2878137270 con 0x7f287806ee20 2024-04-25T00:12:14.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.314+0000 7f287f200640 1 -- 172.21.15.83:0/1761819693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2878137060 con 0x7f287806ee20 2024-04-25T00:12:14.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.318+0000 7f2874c00640 1 -- 172.21.15.83:0/1761819693 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f286800bc80 con 0x7f287806ee20 2024-04-25T00:12:14.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.318+0000 7f2874c00640 1 --2- 172.21.15.83:0/1761819693 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28540678c0 0x7f2854069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.318+0000 7f2874c00640 1 -- 172.21.15.83:0/1761819693 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f2868085ff0 con 0x7f287806ee20 2024-04-25T00:12:14.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.318+0000 7f286f400640 1 --2- 172.21.15.83:0/1761819693 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28540678c0 0x7f2854069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.318+0000 7f286f400640 1 --2- 172.21.15.83:0/1761819693 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28540678c0 0x7f2854069d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f28600011f0 tx=0x7f2860007930 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.322+0000 7f2874c00640 1 -- 172.21.15.83:0/1761819693 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2868056d00 con 0x7f287806ee20 2024-04-25T00:12:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.534+0000 7f287f200640 1 -- 172.21.15.83:0/1761819693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f2878068230 con 0x7f287806ee20 2024-04-25T00:12:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.538+0000 7f2874c00640 1 -- 172.21.15.83:0/1761819693 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f2868056600 con 0x7f287806ee20 2024-04-25T00:12:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.538+0000 7f287f200640 1 -- 172.21.15.83:0/1761819693 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28540678c0 msgr2=0x7f2854069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.538+0000 7f287f200640 1 --2- 172.21.15.83:0/1761819693 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28540678c0 0x7f2854069d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f28600011f0 tx=0x7f2860007930 comp rx=0 tx=0).stop 2024-04-25T00:12:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.538+0000 7f287f200640 1 -- 172.21.15.83:0/1761819693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287806ee20 msgr2=0x7f2878139d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.538+0000 7f287f200640 1 --2- 172.21.15.83:0/1761819693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287806ee20 0x7f2878139d00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f286800d800 tx=0x7f286800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.538+0000 7f287f200640 1 -- 172.21.15.83:0/1761819693 shutdown_connections 2024-04-25T00:12:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.538+0000 7f287f200640 1 --2- 172.21.15.83:0/1761819693 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28540678c0 0x7f2854069d80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.538+0000 7f287f200640 1 --2- 172.21.15.83:0/1761819693 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28781446e0 0x7f2878137a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.538+0000 7f287f200640 1 --2- 172.21.15.83:0/1761819693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287815f350 0x7f28781f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.538+0000 7f287f200640 1 --2- 172.21.15.83:0/1761819693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f287806ee20 0x7f2878139d00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.538+0000 7f287f200640 1 -- 172.21.15.83:0/1761819693 >> 172.21.15.83:0/1761819693 conn(0x7f28780cfa60 msgr2=0x7f28780cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.538+0000 7f287f200640 1 -- 172.21.15.83:0/1761819693 shutdown_connections 2024-04-25T00:12:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.538+0000 7f287f200640 1 -- 172.21.15.83:0/1761819693 wait complete. 2024-04-25T00:12:14.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T00:12:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.694+0000 7f0b7fa00640 1 Processor -- start 2024-04-25T00:12:14.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.694+0000 7f0b7fa00640 1 -- start start 2024-04-25T00:12:14.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.698+0000 7f0b7fa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b781446e0 0x7f0b780cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.698+0000 7f0b7fa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0b7815f350 0x7f0b7806e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.698+0000 7f0b7fa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b78072ff0 con 0x7f0b7806ec50 2024-04-25T00:12:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.698+0000 7f0b7fa00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0b780715b0 con 0x7f0b7815f350 2024-04-25T00:12:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.698+0000 7f0b7fa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0b78144ac0 con 0x7f0b781446e0 2024-04-25T00:12:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.698+0000 7f0b6fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0b7815f350 0x7f0b7806e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.698+0000 7f0b6fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0b7815f350 0x7f0b7806e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34208/0 (socket says 172.21.15.83:34208) 2024-04-25T00:12:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.698+0000 7f0b6fe00640 1 -- 172.21.15.83:0/4167929903 learned_addr learned my addr 172.21.15.83:0/4167929903 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.698+0000 7f0b6fe00640 1 -- 172.21.15.83:0/4167929903 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b781446e0 msgr2=0x7f0b780cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.698+0000 7f0b7d000640 1 --2- 172.21.15.83:0/4167929903 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b781446e0 0x7f0b780cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.698+0000 7f0b6fe00640 1 --2- 172.21.15.83:0/4167929903 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b781446e0 0x7f0b780cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.698+0000 7f0b6fe00640 1 -- 172.21.15.83:0/4167929903 >> v1:172.21.15.86:6789/0 conn(0x7f0b7806ec50 legacy=0x7f0b7815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.698+0000 7f0b6fe00640 1 -- 172.21.15.83:0/4167929903 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b780cb210 con 0x7f0b7815f350 2024-04-25T00:12:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.698+0000 7f0b7d000640 1 --2- 172.21.15.83:0/4167929903 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b781446e0 0x7f0b780cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.702+0000 7f0b6fe00640 1 --2- 172.21.15.83:0/4167929903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7815f350 0x7f0b7806e710 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000bb10 tx=0x7f0b6001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ff9b2c204d1214 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.702+0000 7f0b77e00640 1 -- 172.21.15.83:0/4167929903 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b600b4070 con 0x7f0b7815f350 2024-04-25T00:12:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.702+0000 7f0b77e00640 1 -- 172.21.15.83:0/4167929903 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b600a7a80 con 0x7f0b7815f350 2024-04-25T00:12:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.702+0000 7f0b77e00640 1 -- 172.21.15.83:0/4167929903 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b600a7e50 con 0x7f0b7815f350 2024-04-25T00:12:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.702+0000 7f0b7fa00640 1 -- 172.21.15.83:0/4167929903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7815f350 msgr2=0x7f0b7806e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.702+0000 7f0b7fa00640 1 --2- 172.21.15.83:0/4167929903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7815f350 0x7f0b7806e710 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000bb10 tx=0x7f0b6001ebf0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.702+0000 7f0b7fa00640 1 -- 172.21.15.83:0/4167929903 shutdown_connections 2024-04-25T00:12:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.702+0000 7f0b7fa00640 1 --2- 172.21.15.83:0/4167929903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7815f350 0x7f0b7806e710 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.702+0000 7f0b7fa00640 1 --2- 172.21.15.83:0/4167929903 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b781446e0 0x7f0b780cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.702+0000 7f0b7fa00640 1 -- 172.21.15.83:0/4167929903 >> 172.21.15.83:0/4167929903 conn(0x7f0b780cfa60 msgr2=0x7f0b780cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.702+0000 7f0b7fa00640 1 -- 172.21.15.83:0/4167929903 shutdown_connections 2024-04-25T00:12:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.702+0000 7f0b7fa00640 1 -- 172.21.15.83:0/4167929903 wait complete. 2024-04-25T00:12:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.702+0000 7f0b7fa00640 1 Processor -- start 2024-04-25T00:12:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.702+0000 7f0b7fa00640 1 -- start start 2024-04-25T00:12:14.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7fa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7806ec50 0x7f0b781f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7fa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b781446e0 0x7f0b780c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7815f350 0x7f0b781faf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7fa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7806d600 con 0x7f0b7806ec50 2024-04-25T00:12:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b780cd630 con 0x7f0b7815f350 2024-04-25T00:12:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7fa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7806d780 con 0x7f0b781446e0 2024-04-25T00:12:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7806ec50 0x7f0b781f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7806ec50 0x7f0b781f7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34214/0 (socket says 172.21.15.83:34214) 2024-04-25T00:12:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7d000640 1 -- 172.21.15.83:0/3718191731 learned_addr learned my addr 172.21.15.83:0/3718191731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b6fe00640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b781446e0 0x7f0b780c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7da00640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7815f350 0x7f0b781faf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7d000640 1 -- 172.21.15.83:0/3718191731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b781446e0 msgr2=0x7f0b780c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7d000640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b781446e0 0x7f0b780c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7d000640 1 -- 172.21.15.83:0/3718191731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7815f350 msgr2=0x7f0b781faf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7d000640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7815f350 0x7f0b781faf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7d000640 1 -- 172.21.15.83:0/3718191731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b781fa740 con 0x7f0b7806ec50 2024-04-25T00:12:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7da00640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7815f350 0x7f0b781faf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7d000640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7806ec50 0x7f0b781f7170 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800c7c0 tx=0x7f0b6800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b75600640 1 -- 172.21.15.83:0/3718191731 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68007a30 con 0x7f0b7806ec50 2024-04-25T00:12:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b75600640 1 -- 172.21.15.83:0/3718191731 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b68007bd0 con 0x7f0b7806ec50 2024-04-25T00:12:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b75600640 1 -- 172.21.15.83:0/3718191731 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b680113f0 con 0x7f0b7806ec50 2024-04-25T00:12:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7fa00640 1 -- 172.21.15.83:0/3718191731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b781f76b0 con 0x7f0b7806ec50 2024-04-25T00:12:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7fa00640 1 -- 172.21.15.83:0/3718191731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b781f7c60 con 0x7f0b7806ec50 2024-04-25T00:12:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b7fa00640 1 -- 172.21.15.83:0/3718191731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b78068230 con 0x7f0b7806ec50 2024-04-25T00:12:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.706+0000 7f0b75600640 1 -- 172.21.15.83:0/3718191731 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0b68011590 con 0x7f0b7806ec50 2024-04-25T00:12:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.710+0000 7f0b75600640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0b540678c0 0x7f0b54069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.710+0000 7f0b75600640 1 -- 172.21.15.83:0/3718191731 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f0b68085c40 con 0x7f0b7806ec50 2024-04-25T00:12:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.710+0000 7f0b6fe00640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0b540678c0 0x7f0b54069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.710+0000 7f0b6fe00640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0b540678c0 0x7f0b54069d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0b600b1000 tx=0x7f0b600a9070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.714+0000 7f0b75600640 1 -- 172.21.15.83:0/3718191731 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0b68015020 con 0x7f0b7806ec50 2024-04-25T00:12:14.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.922+0000 7f0b7fa00640 1 -- 172.21.15.83:0/3718191731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f0b78132f90 con 0x7f0b7806ec50 2024-04-25T00:12:14.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.926+0000 7f0b75600640 1 -- 172.21.15.83:0/3718191731 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f0b680562e0 con 0x7f0b7806ec50 2024-04-25T00:12:14.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.926+0000 7f0b7fa00640 1 -- 172.21.15.83:0/3718191731 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0b540678c0 msgr2=0x7f0b54069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.926+0000 7f0b7fa00640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0b540678c0 0x7f0b54069d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0b600b1000 tx=0x7f0b600a9070 comp rx=0 tx=0).stop 2024-04-25T00:12:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.926+0000 7f0b7fa00640 1 -- 172.21.15.83:0/3718191731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7806ec50 msgr2=0x7f0b781f7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.926+0000 7f0b7fa00640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7806ec50 0x7f0b781f7170 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800c7c0 tx=0x7f0b6800cc80 comp rx=0 tx=0).stop 2024-04-25T00:12:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.926+0000 7f0b7fa00640 1 -- 172.21.15.83:0/3718191731 shutdown_connections 2024-04-25T00:12:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.926+0000 7f0b7fa00640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0b540678c0 0x7f0b54069d80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.926+0000 7f0b7fa00640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b781446e0 0x7f0b780c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.926+0000 7f0b7fa00640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7815f350 0x7f0b781faf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.926+0000 7f0b7fa00640 1 --2- 172.21.15.83:0/3718191731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7806ec50 0x7f0b781f7170 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.926+0000 7f0b7fa00640 1 -- 172.21.15.83:0/3718191731 >> 172.21.15.83:0/3718191731 conn(0x7f0b780cfa60 msgr2=0x7f0b7813c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.926+0000 7f0b7fa00640 1 -- 172.21.15.83:0/3718191731 shutdown_connections 2024-04-25T00:12:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:14.926+0000 7f0b7fa00640 1 -- 172.21.15.83:0/3718191731 wait complete. 2024-04-25T00:12:14.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T00:12:15.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd970a00640 1 Processor -- start 2024-04-25T00:12:15.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd970a00640 1 -- start start 2024-04-25T00:12:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd970a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd96c1446e0 0x7fd96c0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd970a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd96c0cb210 con 0x7fd96c15f350 2024-04-25T00:12:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd970a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd96c0cb4f0 con 0x7fd96c06ee20 2024-04-25T00:12:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd970a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd96c0715b0 con 0x7fd96c1446e0 2024-04-25T00:12:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd969400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd96c15f350 0x7fd96c06e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56924/0 (socket says 172.21.15.83:56924) 2024-04-25T00:12:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd969400640 1 -- 172.21.15.83:0/1145464995 learned_addr learned my addr 172.21.15.83:0/1145464995 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd969e00640 1 --2- 172.21.15.83:0/1145464995 >> v2:172.21.15.86:3300/0 conn(0x7fd96c1446e0 0x7fd96c0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1281377216 0 0) 0x7fd96c0cb210 con 0x7fd96c15f350 2024-04-25T00:12:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd950003530 con 0x7fd96c15f350 2024-04-25T00:12:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1672416465 0 0) 0x7fd96c0cb4f0 con 0x7fd96c06ee20 2024-04-25T00:12:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd96c0cb210 con 0x7fd96c06ee20 2024-04-25T00:12:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2638423311 0 0) 0x7fd950003530 con 0x7fd96c15f350 2024-04-25T00:12:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 >> v2:172.21.15.86:3300/0 conn(0x7fd96c1446e0 msgr2=0x7fd96c0cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 --2- 172.21.15.83:0/1145464995 >> v2:172.21.15.86:3300/0 conn(0x7fd96c1446e0 0x7fd96c0cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 >> v1:172.21.15.86:6789/0 conn(0x7fd96c06ee20 legacy=0x7fd96c071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c0717e0 con 0x7fd96c15f350 2024-04-25T00:12:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fd958001680 con 0x7fd96c15f350 2024-04-25T00:12:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 >> v1:172.21.15.83:6789/0 conn(0x7fd96c15f350 legacy=0x7fd96c06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 --2- 172.21.15.83:0/1145464995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd950003b50 0x7fd950003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 --2- 172.21.15.83:0/1145464995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd950004810 0x7fd950004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 --2- 172.21.15.83:0/1145464995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9500051d0 0x7fd95000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd95000c020 con 0x7fd950003b50 2024-04-25T00:12:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd95000c1d0 con 0x7fd950004810 2024-04-25T00:12:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd95000c3d0 con 0x7fd9500051d0 2024-04-25T00:12:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd969e00640 1 --2- 172.21.15.83:0/1145464995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd950003b50 0x7fd950003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd969400640 1 --2- 172.21.15.83:0/1145464995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd950004810 0x7fd950004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd96a800640 1 --2- 172.21.15.83:0/1145464995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9500051d0 0x7fd95000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd969e00640 1 -- 172.21.15.83:0/1145464995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9500051d0 msgr2=0x7fd95000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd969e00640 1 --2- 172.21.15.83:0/1145464995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9500051d0 0x7fd95000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd969e00640 1 -- 172.21.15.83:0/1145464995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd950004810 msgr2=0x7fd950004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd969e00640 1 --2- 172.21.15.83:0/1145464995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd950004810 0x7fd950004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd969e00640 1 -- 172.21.15.83:0/1145464995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd950003530 con 0x7fd950003b50 2024-04-25T00:12:15.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd969400640 1 --2- 172.21.15.83:0/1145464995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd950004810 0x7fd950004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:15.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.086+0000 7fd96a800640 1 -- 172.21.15.83:0/1145464995 reap_dead start 2024-04-25T00:12:15.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.090+0000 7fd969e00640 1 --2- 172.21.15.83:0/1145464995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd950003b50 0x7fd950003f50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd95c009ae0 tx=0x7fd95c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328eb737a50313b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:15.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.090+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0096b0 con 0x7fd950003b50 2024-04-25T00:12:15.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.090+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd95c0a6450 con 0x7fd950003b50 2024-04-25T00:12:15.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.090+0000 7fd968a00640 1 -- 172.21.15.83:0/1145464995 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0a6840 con 0x7fd950003b50 2024-04-25T00:12:15.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.090+0000 7fd970a00640 1 -- 172.21.15.83:0/1145464995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd950003b50 msgr2=0x7fd950003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.090+0000 7fd970a00640 1 --2- 172.21.15.83:0/1145464995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd950003b50 0x7fd950003f50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd95c009ae0 tx=0x7fd95c01ca30 comp rx=0 tx=0).stop 2024-04-25T00:12:15.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.090+0000 7fd970a00640 1 -- 172.21.15.83:0/1145464995 shutdown_connections 2024-04-25T00:12:15.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.090+0000 7fd970a00640 1 --2- 172.21.15.83:0/1145464995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd950003b50 0x7fd950003f50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.090+0000 7fd970a00640 1 -- 172.21.15.83:0/1145464995 >> 172.21.15.83:0/1145464995 conn(0x7fd96c0cfa60 msgr2=0x7fd96c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:15.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.090+0000 7fd970a00640 1 -- 172.21.15.83:0/1145464995 shutdown_connections 2024-04-25T00:12:15.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.090+0000 7fd970a00640 1 -- 172.21.15.83:0/1145464995 wait complete. 2024-04-25T00:12:15.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd970a00640 1 Processor -- start 2024-04-25T00:12:15.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd970a00640 1 -- start start 2024-04-25T00:12:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd970a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd950003b50 0x7fd96c140920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd970a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c13bf50 0x7fd96c13ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd970a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c13f2c0 0x7fd96c1f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd970a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c06eca0 con 0x7fd96c13bf50 2024-04-25T00:12:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd970a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c06eb20 con 0x7fd96c13f2c0 2024-04-25T00:12:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd970a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd96c15bb90 con 0x7fd950003b50 2024-04-25T00:12:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd969400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c13bf50 0x7fd96c13ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd969400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c13bf50 0x7fd96c13ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34232/0 (socket says 172.21.15.83:34232) 2024-04-25T00:12:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd969400640 1 -- 172.21.15.83:0/2788397104 learned_addr learned my addr 172.21.15.83:0/2788397104 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd969e00640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd950003b50 0x7fd96c140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd969400640 1 -- 172.21.15.83:0/2788397104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd950003b50 msgr2=0x7fd96c140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd96a800640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c13f2c0 0x7fd96c1f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd969400640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd950003b50 0x7fd96c140920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd969400640 1 -- 172.21.15.83:0/2788397104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c13f2c0 msgr2=0x7fd96c1f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd969400640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c13f2c0 0x7fd96c1f5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd969400640 1 -- 172.21.15.83:0/2788397104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c13cc40 con 0x7fd96c13bf50 2024-04-25T00:12:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd969e00640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd950003b50 0x7fd96c140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd969400640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c13bf50 0x7fd96c13ed80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd958001b40 tx=0x7fd95800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd962000640 1 -- 172.21.15.83:0/2788397104 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95800c690 con 0x7fd96c13bf50 2024-04-25T00:12:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd962000640 1 -- 172.21.15.83:0/2788397104 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd958018070 con 0x7fd96c13bf50 2024-04-25T00:12:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd970a00640 1 -- 172.21.15.83:0/2788397104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96c13f990 con 0x7fd96c13bf50 2024-04-25T00:12:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd962000640 1 -- 172.21.15.83:0/2788397104 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95800ca70 con 0x7fd96c13bf50 2024-04-25T00:12:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.094+0000 7fd970a00640 1 -- 172.21.15.83:0/2788397104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96c13cee0 con 0x7fd96c13bf50 2024-04-25T00:12:15.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.098+0000 7fd970a00640 1 -- 172.21.15.83:0/2788397104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd96c068230 con 0x7fd96c13bf50 2024-04-25T00:12:15.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.098+0000 7fd962000640 1 -- 172.21.15.83:0/2788397104 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd95800cc10 con 0x7fd96c13bf50 2024-04-25T00:12:15.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.098+0000 7fd962000640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd94c0678f0 0x7fd94c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.098+0000 7fd962000640 1 -- 172.21.15.83:0/2788397104 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fd958085580 con 0x7fd96c13bf50 2024-04-25T00:12:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.098+0000 7fd969e00640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd94c0678f0 0x7fd94c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.098+0000 7fd969e00640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd94c0678f0 0x7fd94c069db0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0ad000 tx=0x7fd95c0a9040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.102+0000 7fd962000640 1 -- 172.21.15.83:0/2788397104 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd9580561e0 con 0x7fd96c13bf50 2024-04-25T00:12:15.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.310+0000 7fd970a00640 1 -- 172.21.15.83:0/2788397104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fd96c072dd0 con 0x7fd96c13bf50 2024-04-25T00:12:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.314+0000 7fd962000640 1 -- 172.21.15.83:0/2788397104 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7fd958055ae0 con 0x7fd96c13bf50 2024-04-25T00:12:15.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.314+0000 7fd970a00640 1 -- 172.21.15.83:0/2788397104 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd94c0678f0 msgr2=0x7fd94c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.314+0000 7fd970a00640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd94c0678f0 0x7fd94c069db0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0ad000 tx=0x7fd95c0a9040 comp rx=0 tx=0).stop 2024-04-25T00:12:15.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.314+0000 7fd970a00640 1 -- 172.21.15.83:0/2788397104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c13bf50 msgr2=0x7fd96c13ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.314+0000 7fd970a00640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c13bf50 0x7fd96c13ed80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd958001b40 tx=0x7fd95800ec10 comp rx=0 tx=0).stop 2024-04-25T00:12:15.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.314+0000 7fd970a00640 1 -- 172.21.15.83:0/2788397104 shutdown_connections 2024-04-25T00:12:15.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.314+0000 7fd970a00640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd94c0678f0 0x7fd94c069db0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.314+0000 7fd970a00640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c13f2c0 0x7fd96c1f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.314+0000 7fd970a00640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96c13bf50 0x7fd96c13ed80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.314+0000 7fd970a00640 1 --2- 172.21.15.83:0/2788397104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd950003b50 0x7fd96c140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.314+0000 7fd970a00640 1 -- 172.21.15.83:0/2788397104 >> 172.21.15.83:0/2788397104 conn(0x7fd96c0cfa60 msgr2=0x7fd96c0ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:15.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.314+0000 7fd970a00640 1 -- 172.21.15.83:0/2788397104 shutdown_connections 2024-04-25T00:12:15.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.314+0000 7fd970a00640 1 -- 172.21.15.83:0/2788397104 wait complete. 2024-04-25T00:12:15.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T00:12:15.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.470+0000 7f1669000640 1 Processor -- start 2024-04-25T00:12:15.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.470+0000 7f1669000640 1 -- start start 2024-04-25T00:12:15.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.474+0000 7f1669000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f16640cb640 0x7f166406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.474+0000 7f1669000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1664070d10 0x7f1664165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.474+0000 7f1669000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16640711a0 con 0x7f166406cfd0 2024-04-25T00:12:15.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.474+0000 7f1669000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f166406e160 con 0x7f16640cb640 2024-04-25T00:12:15.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.474+0000 7f1669000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f166406ea10 con 0x7f1664070d10 2024-04-25T00:12:15.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.474+0000 7f1662800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1664070d10 0x7f1664165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.474+0000 7f1659400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f16640cb640 0x7f166406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.474+0000 7f1659400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f16640cb640 0x7f166406dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:46904/0 (socket says 172.21.15.83:46904) 2024-04-25T00:12:15.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.474+0000 7f1659400640 1 -- 172.21.15.83:0/3636458721 learned_addr learned my addr 172.21.15.83:0/3636458721 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:15.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.474+0000 7f1659400640 1 -- 172.21.15.83:0/3636458721 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1664070d10 msgr2=0x7f1664165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:15.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.474+0000 7f1659400640 1 --2- 172.21.15.83:0/3636458721 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1664070d10 0x7f1664165930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.474+0000 7f1659400640 1 -- 172.21.15.83:0/3636458721 >> v1:172.21.15.86:6789/0 conn(0x7f166406cfd0 legacy=0x7f166406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.474+0000 7f1659400640 1 -- 172.21.15.83:0/3636458721 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166406ecf0 con 0x7f16640cb640 2024-04-25T00:12:15.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.474+0000 7f1662800640 1 --2- 172.21.15.83:0/3636458721 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1664070d10 0x7f1664165930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:15.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1659400640 1 --2- 172.21.15.83:0/3636458721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16640cb640 0x7f166406dac0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f164c009870 tx=0x7f164c01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb639d96a473a6a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:15.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1661400640 1 -- 172.21.15.83:0/3636458721 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c0a7070 con 0x7f16640cb640 2024-04-25T00:12:15.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1661400640 1 -- 172.21.15.83:0/3636458721 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f164c0b0070 con 0x7f16640cb640 2024-04-25T00:12:15.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1661400640 1 -- 172.21.15.83:0/3636458721 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c0ab890 con 0x7f16640cb640 2024-04-25T00:12:15.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 -- 172.21.15.83:0/3636458721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16640cb640 msgr2=0x7f166406dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 --2- 172.21.15.83:0/3636458721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16640cb640 0x7f166406dac0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f164c009870 tx=0x7f164c01c560 comp rx=0 tx=0).stop 2024-04-25T00:12:15.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 -- 172.21.15.83:0/3636458721 shutdown_connections 2024-04-25T00:12:15.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 --2- 172.21.15.83:0/3636458721 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1664070d10 0x7f1664165930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 --2- 172.21.15.83:0/3636458721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16640cb640 0x7f166406dac0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 -- 172.21.15.83:0/3636458721 >> 172.21.15.83:0/3636458721 conn(0x7f16640cfa60 msgr2=0x7f16640cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:15.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 -- 172.21.15.83:0/3636458721 shutdown_connections 2024-04-25T00:12:15.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 -- 172.21.15.83:0/3636458721 wait complete. 2024-04-25T00:12:15.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 Processor -- start 2024-04-25T00:12:15.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 -- start start 2024-04-25T00:12:15.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166406cfd0 0x7f166415b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1664070d10 0x7f16640c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640cb640 0x7f166415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1664168a20 con 0x7f16640cb640 2024-04-25T00:12:15.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16641688a0 con 0x7f166406cfd0 2024-04-25T00:12:15.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1669000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1664168ba0 con 0x7f1664070d10 2024-04-25T00:12:15.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.478+0000 7f1662800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640cb640 0x7f166415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f1662800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640cb640 0x7f166415d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34236/0 (socket says 172.21.15.83:34236) 2024-04-25T00:12:15.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f1662800640 1 -- 172.21.15.83:0/421043405 learned_addr learned my addr 172.21.15.83:0/421043405 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:15.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f1659400640 1 --2- 172.21.15.83:0/421043405 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1664070d10 0x7f16640c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f1661e00640 1 --2- 172.21.15.83:0/421043405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166406cfd0 0x7f166415b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f1662800640 1 -- 172.21.15.83:0/421043405 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1664070d10 msgr2=0x7f16640c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f1662800640 1 --2- 172.21.15.83:0/421043405 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1664070d10 0x7f16640c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f1662800640 1 -- 172.21.15.83:0/421043405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166406cfd0 msgr2=0x7f166415b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f1662800640 1 --2- 172.21.15.83:0/421043405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166406cfd0 0x7f166415b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f1662800640 1 -- 172.21.15.83:0/421043405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1664135050 con 0x7f16640cb640 2024-04-25T00:12:15.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f1662800640 1 --2- 172.21.15.83:0/421043405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640cb640 0x7f166415d9a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f165c00efc0 tx=0x7f165c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:15.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f165aa00640 1 -- 172.21.15.83:0/421043405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c018030 con 0x7f16640cb640 2024-04-25T00:12:15.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f165aa00640 1 -- 172.21.15.83:0/421043405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165c00cdf0 con 0x7f16640cb640 2024-04-25T00:12:15.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f1669000640 1 -- 172.21.15.83:0/421043405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1664135b00 con 0x7f16640cb640 2024-04-25T00:12:15.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f165aa00640 1 -- 172.21.15.83:0/421043405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c003bc0 con 0x7f16640cb640 2024-04-25T00:12:15.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f1669000640 1 -- 172.21.15.83:0/421043405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16641355a0 con 0x7f16640cb640 2024-04-25T00:12:15.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f1669000640 1 -- 172.21.15.83:0/421043405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1664068230 con 0x7f16640cb640 2024-04-25T00:12:15.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f165aa00640 1 -- 172.21.15.83:0/421043405 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f165c01b070 con 0x7f16640cb640 2024-04-25T00:12:15.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f165aa00640 1 --2- 172.21.15.83:0/421043405 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16400678c0 0x7f1640069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f165aa00640 1 -- 172.21.15.83:0/421043405 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f165c0863a0 con 0x7f16640cb640 2024-04-25T00:12:15.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.482+0000 7f1661e00640 1 --2- 172.21.15.83:0/421043405 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16400678c0 0x7f1640069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.486+0000 7f1661e00640 1 --2- 172.21.15.83:0/421043405 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16400678c0 0x7f1640069d80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1664073170 tx=0x7f1654009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:15.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.486+0000 7f165aa00640 1 -- 172.21.15.83:0/421043405 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f165c022070 con 0x7f16640cb640 2024-04-25T00:12:15.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.698+0000 7f1669000640 1 -- 172.21.15.83:0/421043405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f16640aff50 con 0x7f16640cb640 2024-04-25T00:12:15.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.702+0000 7f165aa00640 1 -- 172.21.15.83:0/421043405 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f1664135b00 con 0x7f16640cb640 2024-04-25T00:12:15.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.706+0000 7f1669000640 1 -- 172.21.15.83:0/421043405 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16400678c0 msgr2=0x7f1640069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.706+0000 7f1669000640 1 --2- 172.21.15.83:0/421043405 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16400678c0 0x7f1640069d80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1664073170 tx=0x7f1654009040 comp rx=0 tx=0).stop 2024-04-25T00:12:15.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.706+0000 7f1669000640 1 -- 172.21.15.83:0/421043405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640cb640 msgr2=0x7f166415d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.706+0000 7f1669000640 1 --2- 172.21.15.83:0/421043405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640cb640 0x7f166415d9a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f165c00efc0 tx=0x7f165c00c490 comp rx=0 tx=0).stop 2024-04-25T00:12:15.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.706+0000 7f1669000640 1 -- 172.21.15.83:0/421043405 shutdown_connections 2024-04-25T00:12:15.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.706+0000 7f1669000640 1 --2- 172.21.15.83:0/421043405 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16400678c0 0x7f1640069d80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.706+0000 7f1669000640 1 --2- 172.21.15.83:0/421043405 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1664070d10 0x7f16640c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.706+0000 7f1669000640 1 --2- 172.21.15.83:0/421043405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16640cb640 0x7f166415d9a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.706+0000 7f1669000640 1 --2- 172.21.15.83:0/421043405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166406cfd0 0x7f166415b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.706+0000 7f1669000640 1 -- 172.21.15.83:0/421043405 >> 172.21.15.83:0/421043405 conn(0x7f16640cfa60 msgr2=0x7f16640b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:15.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.706+0000 7f1669000640 1 -- 172.21.15.83:0/421043405 shutdown_connections 2024-04-25T00:12:15.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.706+0000 7f1669000640 1 -- 172.21.15.83:0/421043405 wait complete. 2024-04-25T00:12:15.724 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:15.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f17800640 1 Processor -- start 2024-04-25T00:12:15.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f17800640 1 -- start start 2024-04-25T00:12:15.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f17800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f1006f9e0 0x7f1f1006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f17800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1f1006c2f0 0x7f1f100d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f17800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f10070300 con 0x7f1f100d0e60 2024-04-25T00:12:15.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f17800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1f100ccda0 con 0x7f1f1006c2f0 2024-04-25T00:12:15.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f17800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1f100cc470 con 0x7f1f1006f9e0 2024-04-25T00:12:15.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f0fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1f1006c2f0 0x7f1f100d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f0fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1f1006c2f0 0x7f1f100d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34238/0 (socket says 172.21.15.83:34238) 2024-04-25T00:12:15.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f0fe00640 1 -- 172.21.15.83:0/2983588928 learned_addr learned my addr 172.21.15.83:0/2983588928 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:15.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f14e00640 1 --2- 172.21.15.83:0/2983588928 >> v2:172.21.15.86:3300/0 conn(0x7f1f1006f9e0 0x7f1f1006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f0fe00640 1 -- 172.21.15.83:0/2983588928 >> v2:172.21.15.86:3300/0 conn(0x7f1f1006f9e0 msgr2=0x7f1f1006fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:15.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f0fe00640 1 --2- 172.21.15.83:0/2983588928 >> v2:172.21.15.86:3300/0 conn(0x7f1f1006f9e0 0x7f1f1006fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f0fe00640 1 -- 172.21.15.83:0/2983588928 >> v1:172.21.15.86:6789/0 conn(0x7f1f100d0e60 legacy=0x7f1f1015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.858+0000 7f1f0fe00640 1 -- 172.21.15.83:0/2983588928 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f100cc750 con 0x7f1f1006c2f0 2024-04-25T00:12:15.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.862+0000 7f1f0fe00640 1 --2- 172.21.15.83:0/2983588928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f1006c2f0 0x7f1f100d0920 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1f10071aa0 tx=0x7f1efc01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23682adc8a448d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:15.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.862+0000 7f1f0f400640 1 -- 172.21.15.83:0/2983588928 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0b5070 con 0x7f1f1006c2f0 2024-04-25T00:12:15.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.862+0000 7f1f0f400640 1 -- 172.21.15.83:0/2983588928 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1efc0a8a10 con 0x7f1f1006c2f0 2024-04-25T00:12:15.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.862+0000 7f1f0f400640 1 -- 172.21.15.83:0/2983588928 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0a8de0 con 0x7f1f1006c2f0 2024-04-25T00:12:15.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.862+0000 7f1f17800640 1 -- 172.21.15.83:0/2983588928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f1006c2f0 msgr2=0x7f1f100d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.862+0000 7f1f17800640 1 --2- 172.21.15.83:0/2983588928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f1006c2f0 0x7f1f100d0920 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1f10071aa0 tx=0x7f1efc01ed10 comp rx=0 tx=0).stop 2024-04-25T00:12:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.862+0000 7f1f17800640 1 -- 172.21.15.83:0/2983588928 shutdown_connections 2024-04-25T00:12:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.862+0000 7f1f17800640 1 --2- 172.21.15.83:0/2983588928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f1006c2f0 0x7f1f100d0920 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.862+0000 7f1f17800640 1 --2- 172.21.15.83:0/2983588928 >> v2:172.21.15.86:3300/0 conn(0x7f1f1006f9e0 0x7f1f1006fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.862+0000 7f1f17800640 1 -- 172.21.15.83:0/2983588928 >> 172.21.15.83:0/2983588928 conn(0x7f1f100d1c80 msgr2=0x7f1f100d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.862+0000 7f1f17800640 1 -- 172.21.15.83:0/2983588928 shutdown_connections 2024-04-25T00:12:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.862+0000 7f1f17800640 1 -- 172.21.15.83:0/2983588928 wait complete. 2024-04-25T00:12:15.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f17800640 1 Processor -- start 2024-04-25T00:12:15.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f17800640 1 -- start start 2024-04-25T00:12:15.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f17800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f1006c2f0 0x7f1f1015b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f17800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1006f9e0 0x7f1f101e3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f17800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f100d0e60 0x7f1f1015c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f17800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1006f4b0 con 0x7f1f1006c2f0 2024-04-25T00:12:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f17800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1006f330 con 0x7f1f1006f9e0 2024-04-25T00:12:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f17800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1f1015e000 con 0x7f1f100d0e60 2024-04-25T00:12:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f14e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f1006c2f0 0x7f1f1015b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f14e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f1006c2f0 0x7f1f1015b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34242/0 (socket says 172.21.15.83:34242) 2024-04-25T00:12:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f14e00640 1 -- 172.21.15.83:0/1943244989 learned_addr learned my addr 172.21.15.83:0/1943244989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f14e00640 1 -- 172.21.15.83:0/1943244989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f100d0e60 msgr2=0x7f1f1015c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f14e00640 1 --2- 172.21.15.83:0/1943244989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f100d0e60 0x7f1f1015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f14e00640 1 -- 172.21.15.83:0/1943244989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1006f9e0 msgr2=0x7f1f101e3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f0fe00640 1 --2- 172.21.15.83:0/1943244989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1006f9e0 0x7f1f101e3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f14e00640 1 --2- 172.21.15.83:0/1943244989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1006f9e0 0x7f1f101e3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f14e00640 1 -- 172.21.15.83:0/1943244989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f100b4db0 con 0x7f1f1006c2f0 2024-04-25T00:12:15.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f0fe00640 1 --2- 172.21.15.83:0/1943244989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1006f9e0 0x7f1f101e3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:15.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f14e00640 1 --2- 172.21.15.83:0/1943244989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f1006c2f0 0x7f1f1015b6f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1f00007ef0 tx=0x7f1f0000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:15.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f0cc00640 1 -- 172.21.15.83:0/1943244989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00014620 con 0x7f1f1006c2f0 2024-04-25T00:12:15.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f0cc00640 1 -- 172.21.15.83:0/1943244989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f00019070 con 0x7f1f1006c2f0 2024-04-25T00:12:15.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f17800640 1 -- 172.21.15.83:0/1943244989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1015bc30 con 0x7f1f1006c2f0 2024-04-25T00:12:15.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f0cc00640 1 -- 172.21.15.83:0/1943244989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00014a00 con 0x7f1f1006c2f0 2024-04-25T00:12:15.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f17800640 1 -- 172.21.15.83:0/1943244989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f101e49d0 con 0x7f1f1006c2f0 2024-04-25T00:12:15.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.866+0000 7f1f17800640 1 -- 172.21.15.83:0/1943244989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f1009fdd0 con 0x7f1f1006c2f0 2024-04-25T00:12:15.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.870+0000 7f1f0cc00640 1 -- 172.21.15.83:0/1943244989 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1f00014ba0 con 0x7f1f1006c2f0 2024-04-25T00:12:15.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.870+0000 7f1f0cc00640 1 --2- 172.21.15.83:0/1943244989 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1ef00678c0 0x7f1ef0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.870+0000 7f1f0cc00640 1 -- 172.21.15.83:0/1943244989 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f1f00086420 con 0x7f1f1006c2f0 2024-04-25T00:12:15.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.870+0000 7f1f0fe00640 1 --2- 172.21.15.83:0/1943244989 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1ef00678c0 0x7f1ef0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.870+0000 7f1f0fe00640 1 --2- 172.21.15.83:0/1943244989 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1ef00678c0 0x7f1ef0069d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0b2000 tx=0x7f1efc0aa070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:15.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:15.874+0000 7f1f0cc00640 1 -- 172.21.15.83:0/1943244989 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1f00057110 con 0x7f1f1006c2f0 2024-04-25T00:12:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.030+0000 7f1f17800640 1 -- 172.21.15.83:0/1943244989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f1004f5f0 con 0x7f1f1006c2f0 2024-04-25T00:12:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.030+0000 7f1f0cc00640 1 -- 172.21.15.83:0/1943244989 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+13080 (secure 0 0 0) 0x7f1f00056a10 con 0x7f1f1006c2f0 2024-04-25T00:12:16.032 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:16.032 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:12.989205+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:16.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.034+0000 7f1f17800640 1 -- 172.21.15.83:0/1943244989 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1ef00678c0 msgr2=0x7f1ef0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.034+0000 7f1f17800640 1 --2- 172.21.15.83:0/1943244989 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1ef00678c0 0x7f1ef0069d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0b2000 tx=0x7f1efc0aa070 comp rx=0 tx=0).stop 2024-04-25T00:12:16.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.034+0000 7f1f17800640 1 -- 172.21.15.83:0/1943244989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f1006c2f0 msgr2=0x7f1f1015b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.034+0000 7f1f17800640 1 --2- 172.21.15.83:0/1943244989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f1006c2f0 0x7f1f1015b6f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1f00007ef0 tx=0x7f1f0000e4b0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.034+0000 7f1f17800640 1 -- 172.21.15.83:0/1943244989 shutdown_connections 2024-04-25T00:12:16.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.034+0000 7f1f17800640 1 --2- 172.21.15.83:0/1943244989 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1ef00678c0 0x7f1ef0069d80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.034+0000 7f1f17800640 1 --2- 172.21.15.83:0/1943244989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f100d0e60 0x7f1f1015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.034+0000 7f1f17800640 1 --2- 172.21.15.83:0/1943244989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1006f9e0 0x7f1f101e3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.034+0000 7f1f17800640 1 --2- 172.21.15.83:0/1943244989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f1006c2f0 0x7f1f1015b6f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.034+0000 7f1f17800640 1 -- 172.21.15.83:0/1943244989 >> 172.21.15.83:0/1943244989 conn(0x7f1f100d1c80 msgr2=0x7f1f100d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:16.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.034+0000 7f1f17800640 1 -- 172.21.15.83:0/1943244989 shutdown_connections 2024-04-25T00:12:16.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.034+0000 7f1f17800640 1 -- 172.21.15.83:0/1943244989 wait complete. 2024-04-25T00:12:16.053 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:12:16.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:12:16.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.190+0000 7f7b17200640 1 Processor -- start 2024-04-25T00:12:16.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.190+0000 7f7b17200640 1 -- start start 2024-04-25T00:12:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.194+0000 7f7b17200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7b1815e030 0x7f7b180d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.194+0000 7f7b17200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b1815e410 con 0x7f7b180cd970 2024-04-25T00:12:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.194+0000 7f7b17200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b180cfba0 con 0x7f7b180cec40 2024-04-25T00:12:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.194+0000 7f7b17200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7b180cdda0 con 0x7f7b1815e030 2024-04-25T00:12:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.194+0000 7f7b16800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7b180cd970 0x7f7b1806d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56928/0 (socket says 172.21.15.83:56928) 2024-04-25T00:12:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.194+0000 7f7b15e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7b1815e030 0x7f7b180d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.194+0000 7f7b16800640 1 -- 172.21.15.83:0/3231180749 learned_addr learned my addr 172.21.15.83:0/3231180749 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.194+0000 7f7b15e00640 1 -- 172.21.15.83:0/3231180749 >> v1:172.21.15.86:6789/0 conn(0x7f7b180cec40 legacy=0x7f7b1806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.194+0000 7f7b15e00640 1 -- 172.21.15.83:0/3231180749 >> v1:172.21.15.83:6789/0 conn(0x7f7b180cd970 legacy=0x7f7b1806d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.194+0000 7f7b15e00640 1 -- 172.21.15.83:0/3231180749 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1806c390 con 0x7f7b1815e030 2024-04-25T00:12:16.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.194+0000 7f7b15e00640 1 --2- 172.21.15.83:0/3231180749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b1815e030 0x7f7b180d0150 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7b080097c0 tx=0x7f7b0801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6209af1f4b9fbac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:16.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b15400640 1 -- 172.21.15.83:0/3231180749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b080b0070 con 0x7f7b1815e030 2024-04-25T00:12:16.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b15400640 1 -- 172.21.15.83:0/3231180749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b080ab600 con 0x7f7b1815e030 2024-04-25T00:12:16.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b15400640 1 -- 172.21.15.83:0/3231180749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b080ab9d0 con 0x7f7b1815e030 2024-04-25T00:12:16.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 -- 172.21.15.83:0/3231180749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b1815e030 msgr2=0x7f7b180d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 --2- 172.21.15.83:0/3231180749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b1815e030 0x7f7b180d0150 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7b080097c0 tx=0x7f7b0801c460 comp rx=0 tx=0).stop 2024-04-25T00:12:16.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 -- 172.21.15.83:0/3231180749 shutdown_connections 2024-04-25T00:12:16.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 --2- 172.21.15.83:0/3231180749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b1815e030 0x7f7b180d0150 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 -- 172.21.15.83:0/3231180749 >> 172.21.15.83:0/3231180749 conn(0x7f7b180d1b70 msgr2=0x7f7b180d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:16.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 -- 172.21.15.83:0/3231180749 shutdown_connections 2024-04-25T00:12:16.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 -- 172.21.15.83:0/3231180749 wait complete. 2024-04-25T00:12:16.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 Processor -- start 2024-04-25T00:12:16.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 -- start start 2024-04-25T00:12:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b180cd970 0x7f7b180c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7b180cec40 0x7f7b180c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b1815e030 0x7f7b1814f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b180b10a0 con 0x7f7b180cd970 2024-04-25T00:12:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b180d0aa0 con 0x7f7b1815e030 2024-04-25T00:12:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b17200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1806df70 con 0x7f7b180cec40 2024-04-25T00:12:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b15e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b180cd970 0x7f7b180c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b15e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b180cd970 0x7f7b180c5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34266/0 (socket says 172.21.15.83:34266) 2024-04-25T00:12:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b15e00640 1 -- 172.21.15.83:0/3650374249 learned_addr learned my addr 172.21.15.83:0/3650374249 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b16800640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b1815e030 0x7f7b1814f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b0d400640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7b180cec40 0x7f7b180c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b15e00640 1 -- 172.21.15.83:0/3650374249 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7b180cec40 msgr2=0x7f7b180c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b15e00640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7b180cec40 0x7f7b180c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b15e00640 1 -- 172.21.15.83:0/3650374249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b1815e030 msgr2=0x7f7b1814f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b15e00640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b1815e030 0x7f7b1814f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b15e00640 1 -- 172.21.15.83:0/3650374249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b18150470 con 0x7f7b180cd970 2024-04-25T00:12:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.198+0000 7f7b0d400640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7b180cec40 0x7f7b180c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.202+0000 7f7b15e00640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b180cd970 0x7f7b180c5e50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7b08019070 tx=0x7f7b0801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.202+0000 7f7b0ea00640 1 -- 172.21.15.83:0/3650374249 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b080b0050 con 0x7f7b180cd970 2024-04-25T00:12:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.202+0000 7f7b0ea00640 1 -- 172.21.15.83:0/3650374249 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b080bc070 con 0x7f7b180cd970 2024-04-25T00:12:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.202+0000 7f7b0ea00640 1 -- 172.21.15.83:0/3650374249 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b080b5a30 con 0x7f7b180cd970 2024-04-25T00:12:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.202+0000 7f7b17200640 1 -- 172.21.15.83:0/3650374249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b18150750 con 0x7f7b180cd970 2024-04-25T00:12:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.202+0000 7f7b17200640 1 -- 172.21.15.83:0/3650374249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b18150d10 con 0x7f7b180cd970 2024-04-25T00:12:16.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.202+0000 7f7b17200640 1 -- 172.21.15.83:0/3650374249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b1804f510 con 0x7f7b180cd970 2024-04-25T00:12:16.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.202+0000 7f7b0ea00640 1 -- 172.21.15.83:0/3650374249 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7b080ab4c0 con 0x7f7b180cd970 2024-04-25T00:12:16.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.202+0000 7f7b0ea00640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7af40678c0 0x7f7af4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.202+0000 7f7b0ea00640 1 -- 172.21.15.83:0/3650374249 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f7b081203a0 con 0x7f7b180cd970 2024-04-25T00:12:16.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.206+0000 7f7b0d400640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7af40678c0 0x7f7af4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.206+0000 7f7b0d400640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7af40678c0 0x7f7af4069d80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7b181559a0 tx=0x7f7b00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:16.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.206+0000 7f7b0ea00640 1 -- 172.21.15.83:0/3650374249 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7b08016050 con 0x7f7b180cd970 2024-04-25T00:12:16.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.362+0000 7f7b17200640 1 -- 172.21.15.83:0/3650374249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7b180b79f0 con 0x7f7b180cd970 2024-04-25T00:12:16.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.362+0000 7f7b0ea00640 1 -- 172.21.15.83:0/3650374249 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f7b080f1090 con 0x7f7b180cd970 2024-04-25T00:12:16.364 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:12:16.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.366+0000 7f7b17200640 1 -- 172.21.15.83:0/3650374249 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7af40678c0 msgr2=0x7f7af4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.366+0000 7f7b17200640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7af40678c0 0x7f7af4069d80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7b181559a0 tx=0x7f7b00009040 comp rx=0 tx=0).stop 2024-04-25T00:12:16.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.366+0000 7f7b17200640 1 -- 172.21.15.83:0/3650374249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b180cd970 msgr2=0x7f7b180c5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.366+0000 7f7b17200640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b180cd970 0x7f7b180c5e50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7b08019070 tx=0x7f7b0801aca0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.366+0000 7f7b17200640 1 -- 172.21.15.83:0/3650374249 shutdown_connections 2024-04-25T00:12:16.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.366+0000 7f7b17200640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7af40678c0 0x7f7af4069d80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.366+0000 7f7b17200640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7b180cec40 0x7f7b180c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.366+0000 7f7b17200640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b1815e030 0x7f7b1814f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.366+0000 7f7b17200640 1 --2- 172.21.15.83:0/3650374249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b180cd970 0x7f7b180c5e50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.366+0000 7f7b17200640 1 -- 172.21.15.83:0/3650374249 >> 172.21.15.83:0/3650374249 conn(0x7f7b180d1b70 msgr2=0x7f7b1806d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:16.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.366+0000 7f7b17200640 1 -- 172.21.15.83:0/3650374249 shutdown_connections 2024-04-25T00:12:16.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.366+0000 7f7b17200640 1 -- 172.21.15.83:0/3650374249 wait complete. 2024-04-25T00:12:16.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:12:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa855e00640 1 Processor -- start 2024-04-25T00:12:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa855e00640 1 -- start start 2024-04-25T00:12:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa855e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa85013ef10 0x7fa8500d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa855e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa85015cd90 0x7fa85006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa855e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8500cf170 con 0x7fa85006b1e0 2024-04-25T00:12:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa855e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa85015d1d0 con 0x7fa85013ef10 2024-04-25T00:12:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa855e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa8500cf3f0 con 0x7fa85015cd90 2024-04-25T00:12:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84e000640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa85006b1e0 0x7fa85015c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56938/0 (socket says 172.21.15.83:56938) 2024-04-25T00:12:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84e000640 1 -- 172.21.15.83:0/2692003374 learned_addr learned my addr 172.21.15.83:0/2692003374 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84ea00640 1 --2- 172.21.15.83:0/2692003374 >> v2:172.21.15.86:3300/0 conn(0x7fa85013ef10 0x7fa8500d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84f400640 1 --2- 172.21.15.83:0/2692003374 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa85015cd90 0x7fa85006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 -- 172.21.15.83:0/2692003374 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 909972461 0 0) 0x7fa83c001120 con 0x7fa85006b1e0 2024-04-25T00:12:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 -- 172.21.15.83:0/2692003374 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa834003530 con 0x7fa85006b1e0 2024-04-25T00:12:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 -- 172.21.15.83:0/2692003374 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 643552114 0 0) 0x7fa834003530 con 0x7fa85006b1e0 2024-04-25T00:12:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 -- 172.21.15.83:0/2692003374 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa85015cd90 msgr2=0x7fa85006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 --2- 172.21.15.83:0/2692003374 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa85015cd90 0x7fa85006b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 -- 172.21.15.83:0/2692003374 >> v2:172.21.15.86:3300/0 conn(0x7fa85013ef10 msgr2=0x7fa8500d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 --2- 172.21.15.83:0/2692003374 >> v2:172.21.15.86:3300/0 conn(0x7fa85013ef10 0x7fa8500d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 -- 172.21.15.83:0/2692003374 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8500cf6b0 con 0x7fa85006b1e0 2024-04-25T00:12:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 -- 172.21.15.83:0/2692003374 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fa83c001850 con 0x7fa85006b1e0 2024-04-25T00:12:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 -- 172.21.15.83:0/2692003374 >> v1:172.21.15.83:6789/0 conn(0x7fa85006b1e0 legacy=0x7fa85015c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 --2- 172.21.15.83:0/2692003374 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa85015cd90 0x7fa8340039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 --2- 172.21.15.83:0/2692003374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa834003f40 0x7fa834004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 --2- 172.21.15.83:0/2692003374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa834004d50 0x7fa8340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 -- 172.21.15.83:0/2692003374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa85015d1d0 con 0x7fa834003f40 2024-04-25T00:12:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 -- 172.21.15.83:0/2692003374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8500cf3f0 con 0x7fa834004d50 2024-04-25T00:12:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84d600640 1 -- 172.21.15.83:0/2692003374 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa83400b9d0 con 0x7fa85015cd90 2024-04-25T00:12:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84e000640 1 --2- 172.21.15.83:0/2692003374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa834003f40 0x7fa834004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84ea00640 1 --2- 172.21.15.83:0/2692003374 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa85015cd90 0x7fa8340039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84f400640 1 --2- 172.21.15.83:0/2692003374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa834004d50 0x7fa8340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84e000640 1 -- 172.21.15.83:0/2692003374 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa85015cd90 msgr2=0x7fa8340039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:16.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84e000640 1 --2- 172.21.15.83:0/2692003374 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa85015cd90 0x7fa8340039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84e000640 1 -- 172.21.15.83:0/2692003374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa834004d50 msgr2=0x7fa8340051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:16.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84e000640 1 --2- 172.21.15.83:0/2692003374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa834004d50 0x7fa8340051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.522+0000 7fa84e000640 1 -- 172.21.15.83:0/2692003374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa834003530 con 0x7fa834003f40 2024-04-25T00:12:16.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.526+0000 7fa84e000640 1 --2- 172.21.15.83:0/2692003374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa834003f40 0x7fa834004810 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c4c0 tx=0x7fa83c01f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c104b9e43680be62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.526+0000 7fa84d600640 1 -- 172.21.15.83:0/2692003374 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c00c0e0 con 0x7fa834003f40 2024-04-25T00:12:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.526+0000 7fa84d600640 1 -- 172.21.15.83:0/2692003374 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83c005b00 con 0x7fa834003f40 2024-04-25T00:12:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.526+0000 7fa84d600640 1 -- 172.21.15.83:0/2692003374 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c007070 con 0x7fa834003f40 2024-04-25T00:12:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.526+0000 7fa855e00640 1 -- 172.21.15.83:0/2692003374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa834003f40 msgr2=0x7fa834004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.526+0000 7fa855e00640 1 --2- 172.21.15.83:0/2692003374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa834003f40 0x7fa834004810 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c4c0 tx=0x7fa83c01f3e0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.526+0000 7fa84f400640 1 -- 172.21.15.83:0/2692003374 reap_dead start 2024-04-25T00:12:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.526+0000 7fa855e00640 1 -- 172.21.15.83:0/2692003374 shutdown_connections 2024-04-25T00:12:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.526+0000 7fa855e00640 1 -- 172.21.15.83:0/2692003374 >> 172.21.15.83:0/2692003374 conn(0x7fa8500b2840 msgr2=0x7fa8500b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.526+0000 7fa855e00640 1 -- 172.21.15.83:0/2692003374 shutdown_connections 2024-04-25T00:12:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.526+0000 7fa855e00640 1 -- 172.21.15.83:0/2692003374 wait complete. 2024-04-25T00:12:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa855e00640 1 Processor -- start 2024-04-25T00:12:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa855e00640 1 -- start start 2024-04-25T00:12:16.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa855e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa834003f40 0x7fa85013c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa855e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8500b4650 0x7fa85015ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa855e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8500b4ad0 0x7fa85015b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa855e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa85006b660 con 0x7fa8500b4ad0 2024-04-25T00:12:16.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa855e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8500cfc70 con 0x7fa834003f40 2024-04-25T00:12:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa855e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa85006b7e0 con 0x7fa8500b4650 2024-04-25T00:12:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa84f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8500b4ad0 0x7fa85015b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa84f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8500b4ad0 0x7fa85015b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34282/0 (socket says 172.21.15.83:34282) 2024-04-25T00:12:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa84ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa834003f40 0x7fa85013c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa84e000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8500b4650 0x7fa85015ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa84f400640 1 -- 172.21.15.83:0/489099295 learned_addr learned my addr 172.21.15.83:0/489099295 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa84f400640 1 -- 172.21.15.83:0/489099295 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8500b4650 msgr2=0x7fa85015ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa84f400640 1 --2- 172.21.15.83:0/489099295 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8500b4650 0x7fa85015ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa84f400640 1 -- 172.21.15.83:0/489099295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa834003f40 msgr2=0x7fa85013c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa84f400640 1 --2- 172.21.15.83:0/489099295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa834003f40 0x7fa85013c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa84f400640 1 -- 172.21.15.83:0/489099295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8500b5030 con 0x7fa8500b4ad0 2024-04-25T00:12:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa84f400640 1 --2- 172.21.15.83:0/489099295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8500b4ad0 0x7fa85015b340 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa848007d10 tx=0x7fa8480115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa847400640 1 -- 172.21.15.83:0/489099295 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848011e40 con 0x7fa8500b4ad0 2024-04-25T00:12:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa847400640 1 -- 172.21.15.83:0/489099295 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84801b070 con 0x7fa8500b4ad0 2024-04-25T00:12:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa847400640 1 -- 172.21.15.83:0/489099295 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480039a0 con 0x7fa8500b4ad0 2024-04-25T00:12:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa855e00640 1 -- 172.21.15.83:0/489099295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8500b5310 con 0x7fa8500b4ad0 2024-04-25T00:12:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.530+0000 7fa855e00640 1 -- 172.21.15.83:0/489099295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa850148f00 con 0x7fa8500b4ad0 2024-04-25T00:12:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.534+0000 7fa855e00640 1 -- 172.21.15.83:0/489099295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa850149100 con 0x7fa8500b4ad0 2024-04-25T00:12:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.534+0000 7fa847400640 1 -- 172.21.15.83:0/489099295 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa848003b40 con 0x7fa8500b4ad0 2024-04-25T00:12:16.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.534+0000 7fa847400640 1 --2- 172.21.15.83:0/489099295 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8300678c0 0x7fa830069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.534+0000 7fa847400640 1 -- 172.21.15.83:0/489099295 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fa848086170 con 0x7fa8500b4ad0 2024-04-25T00:12:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.534+0000 7fa84ea00640 1 --2- 172.21.15.83:0/489099295 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8300678c0 0x7fa830069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.534+0000 7fa84ea00640 1 --2- 172.21.15.83:0/489099295 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8300678c0 0x7fa830069d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa840009be0 tx=0x7fa840009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:16.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.538+0000 7fa847400640 1 -- 172.21.15.83:0/489099295 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa850149100 con 0x7fa8500b4ad0 2024-04-25T00:12:16.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.698+0000 7fa855e00640 1 -- 172.21.15.83:0/489099295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa85004f5f0 con 0x7fa8500b4ad0 2024-04-25T00:12:16.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.698+0000 7fa847400640 1 -- 172.21.15.83:0/489099295 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) ==== 84+0+65 (secure 0 0 0) 0x7fa848025020 con 0x7fa8500b4ad0 2024-04-25T00:12:16.700 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:16.700 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-25T00:12:16.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:16.701 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-25T00:12:16.701 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-25T00:12:16.701 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:16.701 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-25T00:12:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.702+0000 7fa855e00640 1 -- 172.21.15.83:0/489099295 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8300678c0 msgr2=0x7fa830069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.702+0000 7fa855e00640 1 --2- 172.21.15.83:0/489099295 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8300678c0 0x7fa830069d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa840009be0 tx=0x7fa840009190 comp rx=0 tx=0).stop 2024-04-25T00:12:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.702+0000 7fa855e00640 1 -- 172.21.15.83:0/489099295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8500b4ad0 msgr2=0x7fa85015b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.702+0000 7fa855e00640 1 --2- 172.21.15.83:0/489099295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8500b4ad0 0x7fa85015b340 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa848007d10 tx=0x7fa8480115a0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.702+0000 7fa855e00640 1 -- 172.21.15.83:0/489099295 shutdown_connections 2024-04-25T00:12:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.702+0000 7fa855e00640 1 --2- 172.21.15.83:0/489099295 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8300678c0 0x7fa830069d80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.702+0000 7fa855e00640 1 --2- 172.21.15.83:0/489099295 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8500b4650 0x7fa85015ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.702+0000 7fa855e00640 1 --2- 172.21.15.83:0/489099295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8500b4ad0 0x7fa85015b340 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.702+0000 7fa855e00640 1 --2- 172.21.15.83:0/489099295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa834003f40 0x7fa85013c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.702+0000 7fa855e00640 1 -- 172.21.15.83:0/489099295 >> 172.21.15.83:0/489099295 conn(0x7fa8500b2840 msgr2=0x7fa85015eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.702+0000 7fa855e00640 1 -- 172.21.15.83:0/489099295 shutdown_connections 2024-04-25T00:12:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.702+0000 7fa855e00640 1 -- 172.21.15.83:0/489099295 wait complete. 2024-04-25T00:12:16.721 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:12:16.722 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:12:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.858+0000 7fbdafa00640 1 Processor -- start 2024-04-25T00:12:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.858+0000 7fbdafa00640 1 -- start start 2024-04-25T00:12:16.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdafa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbda81445e0 0x7fbda80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdafa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbda8072120 0x7fbda815bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdafa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbda815c1a0 0x7fbda80cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdafa00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbda815b7d0 con 0x7fbda81445e0 2024-04-25T00:12:16.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdafa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbda815b480 con 0x7fbda815c1a0 2024-04-25T00:12:16.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdafa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbda81449c0 con 0x7fbda8072120 2024-04-25T00:12:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdad000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbda81445e0 0x7fbda80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdad000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbda81445e0 0x7fbda80b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34294/0 (socket says 172.21.15.83:34294) 2024-04-25T00:12:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdad000640 1 -- 172.21.15.83:0/1228067130 learned_addr learned my addr 172.21.15.83:0/1228067130 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbda7e00640 1 --2- 172.21.15.83:0/1228067130 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbda8072120 0x7fbda815bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdada00640 1 --2- 172.21.15.83:0/1228067130 >> v2:172.21.15.86:3300/0 conn(0x7fbda815c1a0 0x7fbda80cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdad000640 1 -- 172.21.15.83:0/1228067130 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbda8072120 msgr2=0x7fbda815bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdad000640 1 --2- 172.21.15.83:0/1228067130 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbda8072120 0x7fbda815bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdad000640 1 -- 172.21.15.83:0/1228067130 >> v2:172.21.15.86:3300/0 conn(0x7fbda815c1a0 msgr2=0x7fbda80cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdad000640 1 --2- 172.21.15.83:0/1228067130 >> v2:172.21.15.86:3300/0 conn(0x7fbda815c1a0 0x7fbda80cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdad000640 1 -- 172.21.15.83:0/1228067130 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda8072910 con 0x7fbda81445e0 2024-04-25T00:12:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.862+0000 7fbdada00640 1 --2- 172.21.15.83:0/1228067130 >> v2:172.21.15.86:3300/0 conn(0x7fbda815c1a0 0x7fbda80cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbdad000640 1 --2- 172.21.15.83:0/1228067130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81445e0 0x7fbda80b4330 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbda8144bd0 tx=0x7fbd9801eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cf7fb57a567eb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbda7400640 1 -- 172.21.15.83:0/1228067130 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800b2d0 con 0x7fbda81445e0 2024-04-25T00:12:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbda7400640 1 -- 172.21.15.83:0/1228067130 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9800b470 con 0x7fbda81445e0 2024-04-25T00:12:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbda7400640 1 -- 172.21.15.83:0/1228067130 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd980a79f0 con 0x7fbda81445e0 2024-04-25T00:12:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbdafa00640 1 -- 172.21.15.83:0/1228067130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81445e0 msgr2=0x7fbda80b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbdafa00640 1 --2- 172.21.15.83:0/1228067130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81445e0 0x7fbda80b4330 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbda8144bd0 tx=0x7fbd9801eb40 comp rx=0 tx=0).stop 2024-04-25T00:12:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbdafa00640 1 -- 172.21.15.83:0/1228067130 shutdown_connections 2024-04-25T00:12:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbdafa00640 1 --2- 172.21.15.83:0/1228067130 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbda8072120 0x7fbda815bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbdafa00640 1 --2- 172.21.15.83:0/1228067130 >> v2:172.21.15.86:3300/0 conn(0x7fbda815c1a0 0x7fbda80cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbdafa00640 1 --2- 172.21.15.83:0/1228067130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81445e0 0x7fbda80b4330 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbdafa00640 1 -- 172.21.15.83:0/1228067130 >> 172.21.15.83:0/1228067130 conn(0x7fbda806b3b0 msgr2=0x7fbda806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbdafa00640 1 -- 172.21.15.83:0/1228067130 shutdown_connections 2024-04-25T00:12:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbdafa00640 1 -- 172.21.15.83:0/1228067130 wait complete. 2024-04-25T00:12:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbdafa00640 1 Processor -- start 2024-04-25T00:12:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.866+0000 7fbdafa00640 1 -- start start 2024-04-25T00:12:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbdafa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbda8072120 0x7fbda80cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbdafa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81445e0 0x7fbda8142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbdafa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbda8143b10 0x7fbda813b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbdafa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbda80726a0 con 0x7fbda81445e0 2024-04-25T00:12:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbdafa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbda8072520 con 0x7fbda8143b10 2024-04-25T00:12:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbdafa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbda8165e00 con 0x7fbda8072120 2024-04-25T00:12:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbda7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81445e0 0x7fbda8142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbda7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81445e0 0x7fbda8142a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34310/0 (socket says 172.21.15.83:34310) 2024-04-25T00:12:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbda7e00640 1 -- 172.21.15.83:0/3522585848 learned_addr learned my addr 172.21.15.83:0/3522585848 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbdad000640 1 --2- 172.21.15.83:0/3522585848 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbda8072120 0x7fbda80cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbdada00640 1 --2- 172.21.15.83:0/3522585848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbda8143b10 0x7fbda813b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbda7e00640 1 -- 172.21.15.83:0/3522585848 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbda8072120 msgr2=0x7fbda80cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbda7e00640 1 --2- 172.21.15.83:0/3522585848 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbda8072120 0x7fbda80cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbda7e00640 1 -- 172.21.15.83:0/3522585848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbda8143b10 msgr2=0x7fbda813b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbda7e00640 1 --2- 172.21.15.83:0/3522585848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbda8143b10 0x7fbda813b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbda7e00640 1 -- 172.21.15.83:0/3522585848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda813a120 con 0x7fbda81445e0 2024-04-25T00:12:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbda7e00640 1 --2- 172.21.15.83:0/3522585848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81445e0 0x7fbda8142a00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400d5f0 tx=0x7fbd9400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:16.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbda4c00640 1 -- 172.21.15.83:0/3522585848 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94018020 con 0x7fbda81445e0 2024-04-25T00:12:16.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbda4c00640 1 -- 172.21.15.83:0/3522585848 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd94014620 con 0x7fbda81445e0 2024-04-25T00:12:16.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbda4c00640 1 -- 172.21.15.83:0/3522585848 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94014a10 con 0x7fbda81445e0 2024-04-25T00:12:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbdafa00640 1 -- 172.21.15.83:0/3522585848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda813a400 con 0x7fbda81445e0 2024-04-25T00:12:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbdafa00640 1 -- 172.21.15.83:0/3522585848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda813a9c0 con 0x7fbda81445e0 2024-04-25T00:12:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbdafa00640 1 -- 172.21.15.83:0/3522585848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda80736f0 con 0x7fbda81445e0 2024-04-25T00:12:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.870+0000 7fbda4c00640 1 -- 172.21.15.83:0/3522585848 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbd9401b020 con 0x7fbda81445e0 2024-04-25T00:12:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.874+0000 7fbda4c00640 1 --2- 172.21.15.83:0/3522585848 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd880678c0 0x7fbd88069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.874+0000 7fbda4c00640 1 -- 172.21.15.83:0/3522585848 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fbd94085f30 con 0x7fbda81445e0 2024-04-25T00:12:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.874+0000 7fbdad000640 1 --2- 172.21.15.83:0/3522585848 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd880678c0 0x7fbd88069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.874+0000 7fbdad000640 1 --2- 172.21.15.83:0/3522585848 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd880678c0 0x7fbd88069d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbd980b0000 tx=0x7fbd980a9070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:16.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:16.878+0000 7fbda4c00640 1 -- 172.21.15.83:0/3522585848 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbd94022020 con 0x7fbda81445e0 2024-04-25T00:12:17.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.034+0000 7fbdafa00640 1 -- 172.21.15.83:0/3522585848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fbda80719f0 con 0x7fbda81445e0 2024-04-25T00:12:17.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.158+0000 7fbda4c00640 1 -- 172.21.15.83:0/3522585848 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) ==== 138+0+0 (secure 0 0 0) 0x7fbd94056540 con 0x7fbda81445e0 2024-04-25T00:12:17.160 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-25T00:12:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.162+0000 7fbdafa00640 1 -- 172.21.15.83:0/3522585848 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd880678c0 msgr2=0x7fbd88069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.162+0000 7fbdafa00640 1 --2- 172.21.15.83:0/3522585848 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd880678c0 0x7fbd88069d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbd980b0000 tx=0x7fbd980a9070 comp rx=0 tx=0).stop 2024-04-25T00:12:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.162+0000 7fbdafa00640 1 -- 172.21.15.83:0/3522585848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81445e0 msgr2=0x7fbda8142a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.162+0000 7fbdafa00640 1 --2- 172.21.15.83:0/3522585848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81445e0 0x7fbda8142a00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400d5f0 tx=0x7fbd9400dab0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.162+0000 7fbdafa00640 1 -- 172.21.15.83:0/3522585848 shutdown_connections 2024-04-25T00:12:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.162+0000 7fbdafa00640 1 --2- 172.21.15.83:0/3522585848 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd880678c0 0x7fbd88069d80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.162+0000 7fbdafa00640 1 --2- 172.21.15.83:0/3522585848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbda8143b10 0x7fbda813b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.162+0000 7fbdafa00640 1 --2- 172.21.15.83:0/3522585848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda81445e0 0x7fbda8142a00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.162+0000 7fbdafa00640 1 --2- 172.21.15.83:0/3522585848 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbda8072120 0x7fbda80cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.162+0000 7fbdafa00640 1 -- 172.21.15.83:0/3522585848 >> 172.21.15.83:0/3522585848 conn(0x7fbda806b3b0 msgr2=0x7fbda8140c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.162+0000 7fbdafa00640 1 -- 172.21.15.83:0/3522585848 shutdown_connections 2024-04-25T00:12:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.162+0000 7fbdafa00640 1 -- 172.21.15.83:0/3522585848 wait complete. 2024-04-25T00:12:17.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:12:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.326+0000 7f88a3e00640 1 Processor -- start 2024-04-25T00:12:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.326+0000 7f88a3e00640 1 -- start start 2024-04-25T00:12:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.330+0000 7f88a3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f88a4145500 0x7f88a415f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.330+0000 7f88a3e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88a4071c50 0x7f88a406d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.330+0000 7f88a3e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88a40cee10 con 0x7f88a415fdb0 2024-04-25T00:12:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.330+0000 7f88a3e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f88a4160210 con 0x7f88a4145500 2024-04-25T00:12:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.330+0000 7f88a3e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f88a406cd20 con 0x7f88a4071c50 2024-04-25T00:12:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.330+0000 7f88a2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f88a4145500 0x7f88a415f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.330+0000 7f889a000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f88a415fdb0 0x7f88a406d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:53934/0 (socket says 172.21.15.83:53934) 2024-04-25T00:12:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.330+0000 7f889a000640 1 -- 172.21.15.83:0/2956450238 learned_addr learned my addr 172.21.15.83:0/2956450238 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.330+0000 7f88a3400640 1 --2- 172.21.15.83:0/2956450238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88a4071c50 0x7f88a406d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.330+0000 7f88a2a00640 1 -- 172.21.15.83:0/2956450238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88a4071c50 msgr2=0x7f88a406d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.330+0000 7f88a2a00640 1 --2- 172.21.15.83:0/2956450238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88a4071c50 0x7f88a406d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.330+0000 7f88a2a00640 1 -- 172.21.15.83:0/2956450238 >> v1:172.21.15.86:6789/0 conn(0x7f88a415fdb0 legacy=0x7f88a406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.330+0000 7f88a2a00640 1 -- 172.21.15.83:0/2956450238 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a40cf0f0 con 0x7f88a4145500 2024-04-25T00:12:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.330+0000 7f88a2000640 1 -- 172.21.15.83:0/2956450238 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2912046229 0 0) 0x7f88a40cee10 con 0x7f88a415fdb0 2024-04-25T00:12:17.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a2a00640 1 --2- 172.21.15.83:0/2956450238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4145500 0x7f88a415f870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f88940097c0 tx=0x7f889401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d48012277652e061 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:17.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a2000640 1 -- 172.21.15.83:0/2956450238 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88940b2070 con 0x7f88a4145500 2024-04-25T00:12:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a2000640 1 -- 172.21.15.83:0/2956450238 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88940ad640 con 0x7f88a4145500 2024-04-25T00:12:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a2000640 1 -- 172.21.15.83:0/2956450238 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88940ada10 con 0x7f88a4145500 2024-04-25T00:12:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 -- 172.21.15.83:0/2956450238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4145500 msgr2=0x7f88a415f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 --2- 172.21.15.83:0/2956450238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4145500 0x7f88a415f870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f88940097c0 tx=0x7f889401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 -- 172.21.15.83:0/2956450238 shutdown_connections 2024-04-25T00:12:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 --2- 172.21.15.83:0/2956450238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88a4071c50 0x7f88a406d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 --2- 172.21.15.83:0/2956450238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4145500 0x7f88a415f870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 -- 172.21.15.83:0/2956450238 >> 172.21.15.83:0/2956450238 conn(0x7f88a406b1e0 msgr2=0x7f88a406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 -- 172.21.15.83:0/2956450238 shutdown_connections 2024-04-25T00:12:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 -- 172.21.15.83:0/2956450238 wait complete. 2024-04-25T00:12:17.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 Processor -- start 2024-04-25T00:12:17.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 -- start start 2024-04-25T00:12:17.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88a4071c50 0x7f88a41562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:17.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a4145500 0x7f88a4153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:17.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a415fdb0 0x7f88a4158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88a40ceb20 con 0x7f88a4145500 2024-04-25T00:12:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88a40ce9a0 con 0x7f88a415fdb0 2024-04-25T00:12:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.334+0000 7f88a3e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f88a4072060 con 0x7f88a4071c50 2024-04-25T00:12:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f88a2a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88a4071c50 0x7f88a41562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f889a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a4145500 0x7f88a4153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f88a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a415fdb0 0x7f88a4158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f88a2a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88a4071c50 0x7f88a41562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:42994/0 (socket says 172.21.15.83:42994) 2024-04-25T00:12:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f88a2a00640 1 -- 172.21.15.83:0/175061688 learned_addr learned my addr 172.21.15.83:0/175061688 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f88a3400640 1 -- 172.21.15.83:0/175061688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88a4071c50 msgr2=0x7f88a41562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f88a3400640 1 --2- 172.21.15.83:0/175061688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88a4071c50 0x7f88a41562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f88a3400640 1 -- 172.21.15.83:0/175061688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a4145500 msgr2=0x7f88a4153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f88a3400640 1 --2- 172.21.15.83:0/175061688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a4145500 0x7f88a4153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f88a3400640 1 -- 172.21.15.83:0/175061688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a406e730 con 0x7f88a415fdb0 2024-04-25T00:12:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f889a000640 1 --2- 172.21.15.83:0/175061688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a4145500 0x7f88a4153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:17.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f88a3400640 1 --2- 172.21.15.83:0/175061688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a415fdb0 0x7f88a4158720 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f889c00efc0 tx=0x7f889c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:17.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f889b400640 1 -- 172.21.15.83:0/175061688 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c018020 con 0x7f88a415fdb0 2024-04-25T00:12:17.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f889b400640 1 -- 172.21.15.83:0/175061688 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f889c014480 con 0x7f88a415fdb0 2024-04-25T00:12:17.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f88a3e00640 1 -- 172.21.15.83:0/175061688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a40726b0 con 0x7f88a415fdb0 2024-04-25T00:12:17.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f889b400640 1 -- 172.21.15.83:0/175061688 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c009b10 con 0x7f88a415fdb0 2024-04-25T00:12:17.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f88a3e00640 1 -- 172.21.15.83:0/175061688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a41548d0 con 0x7f88a415fdb0 2024-04-25T00:12:17.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f88a3e00640 1 -- 172.21.15.83:0/175061688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a404f510 con 0x7f88a415fdb0 2024-04-25T00:12:17.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f889b400640 1 -- 172.21.15.83:0/175061688 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f889c01b020 con 0x7f88a415fdb0 2024-04-25T00:12:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.338+0000 7f889b400640 1 --2- 172.21.15.83:0/175061688 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88800678c0 0x7f8880069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.342+0000 7f889b400640 1 -- 172.21.15.83:0/175061688 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5619+0+0 (secure 0 0 0) 0x7f889c086810 con 0x7f88a415fdb0 2024-04-25T00:12:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.342+0000 7f88a2a00640 1 --2- 172.21.15.83:0/175061688 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88800678c0 0x7f8880069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:17.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.342+0000 7f88a2a00640 1 --2- 172.21.15.83:0/175061688 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88800678c0 0x7f8880069d80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f88940097c0 tx=0x7f8894008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:17.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.346+0000 7f889b400640 1 -- 172.21.15.83:0/175061688 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f889c022030 con 0x7f88a415fdb0 2024-04-25T00:12:17.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:17.502+0000 7f88a3e00640 1 -- 172.21.15.83:0/175061688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f88a40b0630 con 0x7f88a415fdb0 2024-04-25T00:12:18.176 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-25T00:12:18.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.174+0000 7f889b400640 1 -- 172.21.15.83:0/175061688 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) ==== 170+0+0 (secure 0 0 0) 0x7f889c010040 con 0x7f88a415fdb0 2024-04-25T00:12:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.178+0000 7f88a3e00640 1 -- 172.21.15.83:0/175061688 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88800678c0 msgr2=0x7f8880069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.178+0000 7f88a3e00640 1 --2- 172.21.15.83:0/175061688 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88800678c0 0x7f8880069d80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f88940097c0 tx=0x7f8894008040 comp rx=0 tx=0).stop 2024-04-25T00:12:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.178+0000 7f88a3e00640 1 -- 172.21.15.83:0/175061688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a415fdb0 msgr2=0x7f88a4158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.178+0000 7f88a3e00640 1 --2- 172.21.15.83:0/175061688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a415fdb0 0x7f88a4158720 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f889c00efc0 tx=0x7f889c00c490 comp rx=0 tx=0).stop 2024-04-25T00:12:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.182+0000 7f88a3e00640 1 -- 172.21.15.83:0/175061688 shutdown_connections 2024-04-25T00:12:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.182+0000 7f88a3e00640 1 --2- 172.21.15.83:0/175061688 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88800678c0 0x7f8880069d80 secure :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f88940097c0 tx=0x7f8894008040 comp rx=0 tx=0).stop 2024-04-25T00:12:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.182+0000 7f88a3e00640 1 --2- 172.21.15.83:0/175061688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a415fdb0 0x7f88a4158720 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.182+0000 7f88a3e00640 1 --2- 172.21.15.83:0/175061688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a4145500 0x7f88a4153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.182+0000 7f88a3e00640 1 --2- 172.21.15.83:0/175061688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88a4071c50 0x7f88a41562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.182+0000 7f88a3e00640 1 -- 172.21.15.83:0/175061688 >> 172.21.15.83:0/175061688 conn(0x7f88a406b1e0 msgr2=0x7f88a40b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:18.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.186+0000 7f88a3e00640 1 -- 172.21.15.83:0/175061688 shutdown_connections 2024-04-25T00:12:18.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.190+0000 7f88a3e00640 1 -- 172.21.15.83:0/175061688 wait complete. 2024-04-25T00:12:18.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:12:18.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.358+0000 7f2d11200640 1 Processor -- start 2024-04-25T00:12:18.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.358+0000 7f2d11200640 1 -- start start 2024-04-25T00:12:18.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.358+0000 7f2d11200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2d0c1551e0 0x7f2d0c15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:18.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.358+0000 7f2d11200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0c06e4c0 0x7f2d0c06e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:18.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.358+0000 7f2d11200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d0c15c560 con 0x7f2d0c1555c0 2024-04-25T00:12:18.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.358+0000 7f2d11200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2d0c072460 con 0x7f2d0c1551e0 2024-04-25T00:12:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.358+0000 7f2d11200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2d0c15c7d0 con 0x7f2d0c06e4c0 2024-04-25T00:12:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.358+0000 7f2d09e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2d0c1551e0 0x7f2d0c15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.358+0000 7f2d09e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2d0c1551e0 0x7f2d0c15bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34328/0 (socket says 172.21.15.83:34328) 2024-04-25T00:12:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.358+0000 7f2d09e00640 1 -- 172.21.15.83:0/1096657468 learned_addr learned my addr 172.21.15.83:0/1096657468 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.358+0000 7f2d0a800640 1 --2- 172.21.15.83:0/1096657468 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0c06e4c0 0x7f2d0c06e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d09e00640 1 -- 172.21.15.83:0/1096657468 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0c06e4c0 msgr2=0x7f2d0c06e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d09e00640 1 --2- 172.21.15.83:0/1096657468 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0c06e4c0 0x7f2d0c06e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d09e00640 1 -- 172.21.15.83:0/1096657468 >> v1:172.21.15.86:6789/0 conn(0x7f2d0c1555c0 legacy=0x7f2d0c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d09e00640 1 -- 172.21.15.83:0/1096657468 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c15cab0 con 0x7f2d0c1551e0 2024-04-25T00:12:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d0a800640 1 --2- 172.21.15.83:0/1096657468 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0c06e4c0 0x7f2d0c06e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d09e00640 1 --2- 172.21.15.83:0/1096657468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d0c1551e0 0x7f2d0c15bea0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0098a0 tx=0x7f2cfc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a6873637560f52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d09400640 1 -- 172.21.15.83:0/1096657468 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc0b0070 con 0x7f2d0c1551e0 2024-04-25T00:12:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d09400640 1 -- 172.21.15.83:0/1096657468 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cfc0ab600 con 0x7f2d0c1551e0 2024-04-25T00:12:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d09400640 1 -- 172.21.15.83:0/1096657468 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc0ab9d0 con 0x7f2d0c1551e0 2024-04-25T00:12:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d11200640 1 -- 172.21.15.83:0/1096657468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d0c1551e0 msgr2=0x7f2d0c15bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d11200640 1 --2- 172.21.15.83:0/1096657468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d0c1551e0 0x7f2d0c15bea0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0098a0 tx=0x7f2cfc01c500 comp rx=0 tx=0).stop 2024-04-25T00:12:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d11200640 1 -- 172.21.15.83:0/1096657468 shutdown_connections 2024-04-25T00:12:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d11200640 1 --2- 172.21.15.83:0/1096657468 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0c06e4c0 0x7f2d0c06e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d11200640 1 --2- 172.21.15.83:0/1096657468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d0c1551e0 0x7f2d0c15bea0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d11200640 1 -- 172.21.15.83:0/1096657468 >> 172.21.15.83:0/1096657468 conn(0x7f2d0c0b1830 msgr2=0x7f2d0c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d11200640 1 -- 172.21.15.83:0/1096657468 shutdown_connections 2024-04-25T00:12:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.362+0000 7f2d11200640 1 -- 172.21.15.83:0/1096657468 wait complete. 2024-04-25T00:12:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d11200640 1 Processor -- start 2024-04-25T00:12:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d11200640 1 -- start start 2024-04-25T00:12:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d11200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0c06e4c0 0x7f2d0c14db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d11200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c1551e0 0x7f2d0c15f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d11200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d0c1555c0 0x7f2d0c157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d11200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c15dbb0 con 0x7f2d0c1555c0 2024-04-25T00:12:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d11200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c15da30 con 0x7f2d0c1551e0 2024-04-25T00:12:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d11200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2d0c15dd30 con 0x7f2d0c06e4c0 2024-04-25T00:12:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d0a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d0c1555c0 0x7f2d0c157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:18.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d09e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0c06e4c0 0x7f2d0c14db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:18.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d0a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d0c1555c0 0x7f2d0c157790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34344/0 (socket says 172.21.15.83:34344) 2024-04-25T00:12:18.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d0a800640 1 -- 172.21.15.83:0/4009899228 learned_addr learned my addr 172.21.15.83:0/4009899228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:18.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d01400640 1 --2- 172.21.15.83:0/4009899228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c1551e0 0x7f2d0c15f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:18.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d0a800640 1 -- 172.21.15.83:0/4009899228 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0c06e4c0 msgr2=0x7f2d0c14db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:18.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d0a800640 1 --2- 172.21.15.83:0/4009899228 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0c06e4c0 0x7f2d0c14db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:18.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d0a800640 1 -- 172.21.15.83:0/4009899228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c1551e0 msgr2=0x7f2d0c15f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:18.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d0a800640 1 --2- 172.21.15.83:0/4009899228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c1551e0 0x7f2d0c15f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:18.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d0a800640 1 -- 172.21.15.83:0/4009899228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c15f630 con 0x7f2d0c1555c0 2024-04-25T00:12:18.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d0a800640 1 --2- 172.21.15.83:0/4009899228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d0c1555c0 0x7f2d0c157790 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400cf20 tx=0x7f2d0400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:18.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d02a00640 1 -- 172.21.15.83:0/4009899228 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0401a070 con 0x7f2d0c1555c0 2024-04-25T00:12:18.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d02a00640 1 -- 172.21.15.83:0/4009899228 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0400d840 con 0x7f2d0c1555c0 2024-04-25T00:12:18.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d11200640 1 -- 172.21.15.83:0/4009899228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0c15e330 con 0x7f2d0c1555c0 2024-04-25T00:12:18.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d02a00640 1 -- 172.21.15.83:0/4009899228 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0400dc30 con 0x7f2d0c1555c0 2024-04-25T00:12:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.366+0000 7f2d11200640 1 -- 172.21.15.83:0/4009899228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0c15e8f0 con 0x7f2d0c1555c0 2024-04-25T00:12:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.370+0000 7f2d11200640 1 -- 172.21.15.83:0/4009899228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d0c04f510 con 0x7f2d0c1555c0 2024-04-25T00:12:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.370+0000 7f2d02a00640 1 -- 172.21.15.83:0/4009899228 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2d0401d070 con 0x7f2d0c1555c0 2024-04-25T00:12:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.370+0000 7f2d02a00640 1 --2- 172.21.15.83:0/4009899228 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2ce80678c0 0x7f2ce8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.370+0000 7f2d02a00640 1 -- 172.21.15.83:0/4009899228 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6001+0+0 (secure 0 0 0) 0x7f2d04086ab0 con 0x7f2d0c1555c0 2024-04-25T00:12:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.370+0000 7f2d09e00640 1 --2- 172.21.15.83:0/4009899228 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2ce80678c0 0x7f2ce8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:18.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.374+0000 7f2d09e00640 1 --2- 172.21.15.83:0/4009899228 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2ce80678c0 0x7f2ce8069d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0ad000 tx=0x7f2cfc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:18.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.374+0000 7f2d02a00640 1 -- 172.21.15.83:0/4009899228 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2d04024020 con 0x7f2d0c1555c0 2024-04-25T00:12:18.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:18.562+0000 7f2d11200640 1 -- 172.21.15.83:0/4009899228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2d0c06ce30 con 0x7f2d0c1555c0 2024-04-25T00:12:19.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.186+0000 7f2d02a00640 1 -- 172.21.15.83:0/4009899228 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f2d04056e50 con 0x7f2d0c1555c0 2024-04-25T00:12:19.188 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T00:12:19.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.190+0000 7f2d11200640 1 -- 172.21.15.83:0/4009899228 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2ce80678c0 msgr2=0x7f2ce8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.190+0000 7f2d11200640 1 --2- 172.21.15.83:0/4009899228 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2ce80678c0 0x7f2ce8069d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0ad000 tx=0x7f2cfc0a6000 comp rx=0 tx=0).stop 2024-04-25T00:12:19.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.190+0000 7f2d11200640 1 -- 172.21.15.83:0/4009899228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d0c1555c0 msgr2=0x7f2d0c157790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.190+0000 7f2d11200640 1 --2- 172.21.15.83:0/4009899228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d0c1555c0 0x7f2d0c157790 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400cf20 tx=0x7f2d0400a7d0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.190+0000 7f2d11200640 1 -- 172.21.15.83:0/4009899228 shutdown_connections 2024-04-25T00:12:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.190+0000 7f2d11200640 1 --2- 172.21.15.83:0/4009899228 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2ce80678c0 0x7f2ce8069d80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.190+0000 7f2d11200640 1 --2- 172.21.15.83:0/4009899228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d0c1555c0 0x7f2d0c157790 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.190+0000 7f2d11200640 1 --2- 172.21.15.83:0/4009899228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c1551e0 0x7f2d0c15f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.190+0000 7f2d11200640 1 --2- 172.21.15.83:0/4009899228 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0c06e4c0 0x7f2d0c14db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.190+0000 7f2d11200640 1 -- 172.21.15.83:0/4009899228 >> 172.21.15.83:0/4009899228 conn(0x7f2d0c0b1830 msgr2=0x7f2d0c06e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:19.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.190+0000 7f2d11200640 1 -- 172.21.15.83:0/4009899228 shutdown_connections 2024-04-25T00:12:19.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.190+0000 7f2d11200640 1 -- 172.21.15.83:0/4009899228 wait complete. 2024-04-25T00:12:19.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.362+0000 7fed40a00640 1 Processor -- start 2024-04-25T00:12:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.362+0000 7fed40a00640 1 -- start start 2024-04-25T00:12:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.362+0000 7fed40a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed3c15df80 0x7fed3c0d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.362+0000 7fed40a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed3c0d1360 con 0x7fed3c0d2750 2024-04-25T00:12:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.362+0000 7fed40a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed3c0d1640 con 0x7fed3c15e360 2024-04-25T00:12:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.362+0000 7fed40a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fed3c0b3250 con 0x7fed3c15df80 2024-04-25T00:12:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3be00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fed3c0d2750 0x7fed3c0b3d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56944/0 (socket says 172.21.15.83:56944) 2024-04-25T00:12:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3be00640 1 -- 172.21.15.83:0/3772466978 learned_addr learned my addr 172.21.15.83:0/3772466978 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3b400640 1 --2- 172.21.15.83:0/3772466978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed3c15df80 0x7fed3c0d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 -- 172.21.15.83:0/3772466978 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3381060089 0 0) 0x7fed3c0d1360 con 0x7fed3c0d2750 2024-04-25T00:12:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 -- 172.21.15.83:0/3772466978 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed20003530 con 0x7fed3c0d2750 2024-04-25T00:12:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 -- 172.21.15.83:0/3772466978 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2137685779 0 0) 0x7fed20003530 con 0x7fed3c0d2750 2024-04-25T00:12:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 -- 172.21.15.83:0/3772466978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed3c15df80 msgr2=0x7fed3c0d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 --2- 172.21.15.83:0/3772466978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed3c15df80 0x7fed3c0d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 -- 172.21.15.83:0/3772466978 >> v1:172.21.15.86:6789/0 conn(0x7fed3c15e360 legacy=0x7fed3c0d2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 -- 172.21.15.83:0/3772466978 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3c0b3530 con 0x7fed3c0d2750 2024-04-25T00:12:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 -- 172.21.15.83:0/3772466978 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fed34001a00 con 0x7fed3c0d2750 2024-04-25T00:12:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 -- 172.21.15.83:0/3772466978 >> v1:172.21.15.83:6789/0 conn(0x7fed3c0d2750 legacy=0x7fed3c0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 --2- 172.21.15.83:0/3772466978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed3c15df80 0x7fed200039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 --2- 172.21.15.83:0/3772466978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed20003f30 0x7fed20004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 --2- 172.21.15.83:0/3772466978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed20004d50 0x7fed200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 -- 172.21.15.83:0/3772466978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c0b3250 con 0x7fed20004d50 2024-04-25T00:12:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 -- 172.21.15.83:0/3772466978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed2000ba30 con 0x7fed20003f30 2024-04-25T00:12:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3a000640 1 -- 172.21.15.83:0/3772466978 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fed2000bc30 con 0x7fed3c15df80 2024-04-25T00:12:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3be00640 1 --2- 172.21.15.83:0/3772466978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed20004d50 0x7fed200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3aa00640 1 --2- 172.21.15.83:0/3772466978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed20003f30 0x7fed20004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3b400640 1 --2- 172.21.15.83:0/3772466978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed3c15df80 0x7fed200039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3be00640 1 -- 172.21.15.83:0/3772466978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed3c15df80 msgr2=0x7fed200039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3be00640 1 --2- 172.21.15.83:0/3772466978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed3c15df80 0x7fed200039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3be00640 1 -- 172.21.15.83:0/3772466978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed20003f30 msgr2=0x7fed20004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3be00640 1 --2- 172.21.15.83:0/3772466978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed20003f30 0x7fed20004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.366+0000 7fed3be00640 1 -- 172.21.15.83:0/3772466978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed20003530 con 0x7fed20004d50 2024-04-25T00:12:19.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.370+0000 7fed3be00640 1 --2- 172.21.15.83:0/3772466978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed20004d50 0x7fed200051d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0ce380 tx=0x7fed3401fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d78ab81c4a6800e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:19.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.370+0000 7fed3a000640 1 -- 172.21.15.83:0/3772466978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3400a4c0 con 0x7fed20004d50 2024-04-25T00:12:19.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.370+0000 7fed3a000640 1 -- 172.21.15.83:0/3772466978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed340aa6e0 con 0x7fed20004d50 2024-04-25T00:12:19.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.370+0000 7fed3a000640 1 -- 172.21.15.83:0/3772466978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340aab10 con 0x7fed20004d50 2024-04-25T00:12:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.370+0000 7fed40a00640 1 -- 172.21.15.83:0/3772466978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed20004d50 msgr2=0x7fed200051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.370+0000 7fed40a00640 1 --2- 172.21.15.83:0/3772466978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed20004d50 0x7fed200051d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0ce380 tx=0x7fed3401fb20 comp rx=0 tx=0).stop 2024-04-25T00:12:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.370+0000 7fed3be00640 1 -- 172.21.15.83:0/3772466978 reap_dead start 2024-04-25T00:12:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.370+0000 7fed40a00640 1 -- 172.21.15.83:0/3772466978 shutdown_connections 2024-04-25T00:12:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.370+0000 7fed40a00640 1 -- 172.21.15.83:0/3772466978 >> 172.21.15.83:0/3772466978 conn(0x7fed3c0d3d80 msgr2=0x7fed3c0d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.370+0000 7fed40a00640 1 -- 172.21.15.83:0/3772466978 shutdown_connections 2024-04-25T00:12:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.370+0000 7fed40a00640 1 -- 172.21.15.83:0/3772466978 wait complete. 2024-04-25T00:12:19.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.370+0000 7fed40a00640 1 Processor -- start 2024-04-25T00:12:19.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.370+0000 7fed40a00640 1 -- start start 2024-04-25T00:12:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed40a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed20004d50 0x7fed3c1611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed40a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed3c158730 0x7fed3c15e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed40a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed3c15eca0 0x7fed3c15f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed40a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c0d1e70 con 0x7fed3c158730 2024-04-25T00:12:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed40a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c0d1cf0 con 0x7fed20004d50 2024-04-25T00:12:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed40a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fed3c0d48a0 con 0x7fed3c15eca0 2024-04-25T00:12:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed3aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed3c158730 0x7fed3c15e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed3aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed3c158730 0x7fed3c15e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34368/0 (socket says 172.21.15.83:34368) 2024-04-25T00:12:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed3aa00640 1 -- 172.21.15.83:0/2822973480 learned_addr learned my addr 172.21.15.83:0/2822973480 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed3b400640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed20004d50 0x7fed3c1611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed3be00640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed3c15eca0 0x7fed3c15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed3aa00640 1 -- 172.21.15.83:0/2822973480 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed3c15eca0 msgr2=0x7fed3c15f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed3aa00640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed3c15eca0 0x7fed3c15f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed3aa00640 1 -- 172.21.15.83:0/2822973480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed20004d50 msgr2=0x7fed3c1611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed3aa00640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed20004d50 0x7fed3c1611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed3aa00640 1 -- 172.21.15.83:0/2822973480 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3c1606f0 con 0x7fed3c158730 2024-04-25T00:12:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed3be00640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed3c15eca0 0x7fed3c15f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed3aa00640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed3c158730 0x7fed3c15e760 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00f7b0 tx=0x7fed2c00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed33400640 1 -- 172.21.15.83:0/2822973480 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c0050d0 con 0x7fed3c158730 2024-04-25T00:12:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed33400640 1 -- 172.21.15.83:0/2822973480 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed2c018070 con 0x7fed3c158730 2024-04-25T00:12:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed40a00640 1 -- 172.21.15.83:0/2822973480 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3c0c79e0 con 0x7fed3c158730 2024-04-25T00:12:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed33400640 1 -- 172.21.15.83:0/2822973480 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c00d730 con 0x7fed3c158730 2024-04-25T00:12:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed40a00640 1 -- 172.21.15.83:0/2822973480 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3c0c7fa0 con 0x7fed3c158730 2024-04-25T00:12:19.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed40a00640 1 -- 172.21.15.83:0/2822973480 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed3c04f510 con 0x7fed3c158730 2024-04-25T00:12:19.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.374+0000 7fed33400640 1 -- 172.21.15.83:0/2822973480 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fed2c00d8d0 con 0x7fed3c158730 2024-04-25T00:12:19.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.378+0000 7fed33400640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fed180678f0 0x7fed18069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.378+0000 7fed33400640 1 -- 172.21.15.83:0/2822973480 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7fed2c085c40 con 0x7fed3c158730 2024-04-25T00:12:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.378+0000 7fed3b400640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fed180678f0 0x7fed18069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.378+0000 7fed3b400640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fed180678f0 0x7fed18069db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0a1cd0 tx=0x7fed24009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.382+0000 7fed33400640 1 -- 172.21.15.83:0/2822973480 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fed2c056680 con 0x7fed3c158730 2024-04-25T00:12:19.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.542+0000 7fed40a00640 1 -- 172.21.15.83:0/2822973480 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed3c0b5650 con 0x7fed3c158730 2024-04-25T00:12:19.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.542+0000 7fed33400640 1 -- 172.21.15.83:0/2822973480 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16871 (secure 0 0 0) 0x7fed2c055f80 con 0x7fed3c158730 2024-04-25T00:12:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:19.544 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":37,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:18.565779+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:17.079304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:17.548692+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.542+0000 7fed40a00640 1 -- 172.21.15.83:0/2822973480 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fed180678f0 msgr2=0x7fed18069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.542+0000 7fed40a00640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fed180678f0 0x7fed18069db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0a1cd0 tx=0x7fed24009190 comp rx=0 tx=0).stop 2024-04-25T00:12:19.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.542+0000 7fed40a00640 1 -- 172.21.15.83:0/2822973480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed3c158730 msgr2=0x7fed3c15e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.542+0000 7fed40a00640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed3c158730 0x7fed3c15e760 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00f7b0 tx=0x7fed2c00fbb0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.542+0000 7fed40a00640 1 -- 172.21.15.83:0/2822973480 shutdown_connections 2024-04-25T00:12:19.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.542+0000 7fed40a00640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fed180678f0 0x7fed18069db0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.542+0000 7fed40a00640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed3c15eca0 0x7fed3c15f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.542+0000 7fed40a00640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed3c158730 0x7fed3c15e760 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.542+0000 7fed40a00640 1 --2- 172.21.15.83:0/2822973480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed20004d50 0x7fed3c1611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.542+0000 7fed40a00640 1 -- 172.21.15.83:0/2822973480 >> 172.21.15.83:0/2822973480 conn(0x7fed3c0d3d80 msgr2=0x7fed3c06bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:19.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.542+0000 7fed40a00640 1 -- 172.21.15.83:0/2822973480 shutdown_connections 2024-04-25T00:12:19.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.542+0000 7fed40a00640 1 -- 172.21.15.83:0/2822973480 wait complete. 2024-04-25T00:12:19.564 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:19.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.698+0000 7fa6ca600640 1 Processor -- start 2024-04-25T00:12:19.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.698+0000 7fa6ca600640 1 -- start start 2024-04-25T00:12:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.698+0000 7fa6ca600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa6c415df00 0x7fa6c406d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.698+0000 7fa6ca600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6c406e0c0 con 0x7fa6c4071dc0 2024-04-25T00:12:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.698+0000 7fa6ca600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6c40ad990 con 0x7fa6c406e7b0 2024-04-25T00:12:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.698+0000 7fa6ca600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa6c406f150 con 0x7fa6c415df00 2024-04-25T00:12:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.698+0000 7fa6c3400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa6c415df00 0x7fa6c406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.698+0000 7fa6c3e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa6c4071dc0 0x7fa6c40ae470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56948/0 (socket says 172.21.15.83:56948) 2024-04-25T00:12:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.698+0000 7fa6c3e00640 1 -- 172.21.15.83:0/2148149693 learned_addr learned my addr 172.21.15.83:0/2148149693 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.698+0000 7fa6c3400640 1 -- 172.21.15.83:0/2148149693 >> v1:172.21.15.86:6789/0 conn(0x7fa6c406e7b0 legacy=0x7fa6c40716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.698+0000 7fa6c3400640 1 -- 172.21.15.83:0/2148149693 >> v1:172.21.15.83:6789/0 conn(0x7fa6c4071dc0 legacy=0x7fa6c40ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.698+0000 7fa6c3400640 1 -- 172.21.15.83:0/2148149693 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c40adc70 con 0x7fa6c415df00 2024-04-25T00:12:19.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.702+0000 7fa6c3400640 1 --2- 172.21.15.83:0/2148149693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6c415df00 0x7fa6c406d970 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40097c0 tx=0x7fa6b401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f145110615b41cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:19.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.702+0000 7fa6c2a00640 1 -- 172.21.15.83:0/2148149693 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40b0070 con 0x7fa6c415df00 2024-04-25T00:12:19.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.702+0000 7fa6c2a00640 1 -- 172.21.15.83:0/2148149693 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b40ab600 con 0x7fa6c415df00 2024-04-25T00:12:19.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.702+0000 7fa6c2a00640 1 -- 172.21.15.83:0/2148149693 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40ab9d0 con 0x7fa6c415df00 2024-04-25T00:12:19.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.702+0000 7fa6ca600640 1 -- 172.21.15.83:0/2148149693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6c415df00 msgr2=0x7fa6c406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.702+0000 7fa6ca600640 1 --2- 172.21.15.83:0/2148149693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6c415df00 0x7fa6c406d970 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40097c0 tx=0x7fa6b401c340 comp rx=0 tx=0).stop 2024-04-25T00:12:19.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.702+0000 7fa6ca600640 1 -- 172.21.15.83:0/2148149693 shutdown_connections 2024-04-25T00:12:19.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.702+0000 7fa6ca600640 1 --2- 172.21.15.83:0/2148149693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6c415df00 0x7fa6c406d970 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.702+0000 7fa6ca600640 1 -- 172.21.15.83:0/2148149693 >> 172.21.15.83:0/2148149693 conn(0x7fa6c40b1830 msgr2=0x7fa6c40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:19.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.702+0000 7fa6ca600640 1 -- 172.21.15.83:0/2148149693 shutdown_connections 2024-04-25T00:12:19.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.702+0000 7fa6ca600640 1 -- 172.21.15.83:0/2148149693 wait complete. 2024-04-25T00:12:19.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6ca600640 1 Processor -- start 2024-04-25T00:12:19.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6ca600640 1 -- start start 2024-04-25T00:12:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6ca600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6c406e7b0 0x7fa6c4148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6ca600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6c4071dc0 0x7fa6c4152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6ca600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6c415df00 0x7fa6c41530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6ca600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c416c010 con 0x7fa6c415df00 2024-04-25T00:12:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6ca600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c416be90 con 0x7fa6c4071dc0 2024-04-25T00:12:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6ca600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c416c190 con 0x7fa6c406e7b0 2024-04-25T00:12:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6c415df00 0x7fa6c41530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6c415df00 0x7fa6c41530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34398/0 (socket says 172.21.15.83:34398) 2024-04-25T00:12:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6c3e00640 1 -- 172.21.15.83:0/743831409 learned_addr learned my addr 172.21.15.83:0/743831409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6c3400640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6c406e7b0 0x7fa6c4148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6c3e00640 1 -- 172.21.15.83:0/743831409 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6c406e7b0 msgr2=0x7fa6c4148ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6c3e00640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6c406e7b0 0x7fa6c4148ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6c3e00640 1 -- 172.21.15.83:0/743831409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6c4071dc0 msgr2=0x7fa6c4152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6bbe00640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6c4071dc0 0x7fa6c4152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6c3e00640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6c4071dc0 0x7fa6c4152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6c3e00640 1 -- 172.21.15.83:0/743831409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c40b4d40 con 0x7fa6c415df00 2024-04-25T00:12:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6c3400640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6c406e7b0 0x7fa6c4148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6bbe00640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6c4071dc0 0x7fa6c4152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6c3e00640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6c415df00 0x7fa6c41530d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00ca20 tx=0x7fa6bc00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6bb400640 1 -- 172.21.15.83:0/743831409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc017070 con 0x7fa6c415df00 2024-04-25T00:12:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6bb400640 1 -- 172.21.15.83:0/743831409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6bc00abd0 con 0x7fa6c415df00 2024-04-25T00:12:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6ca600640 1 -- 172.21.15.83:0/743831409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c41f8ad0 con 0x7fa6c415df00 2024-04-25T00:12:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6bb400640 1 -- 172.21.15.83:0/743831409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc012400 con 0x7fa6c415df00 2024-04-25T00:12:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6ca600640 1 -- 172.21.15.83:0/743831409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c41f9050 con 0x7fa6c415df00 2024-04-25T00:12:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.706+0000 7fa6ca600640 1 -- 172.21.15.83:0/743831409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c41f9b00 con 0x7fa6c415df00 2024-04-25T00:12:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.710+0000 7fa6bb400640 1 -- 172.21.15.83:0/743831409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa6bc01a070 con 0x7fa6c415df00 2024-04-25T00:12:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.710+0000 7fa6bb400640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa6a00678f0 0x7fa6a0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.710+0000 7fa6bb400640 1 -- 172.21.15.83:0/743831409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7fa6bc085970 con 0x7fa6c415df00 2024-04-25T00:12:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.710+0000 7fa6c3400640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa6a00678f0 0x7fa6a0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.710+0000 7fa6c3400640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa6a00678f0 0x7fa6a0069db0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40ad000 tx=0x7fa6b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:19.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.714+0000 7fa6bb400640 1 -- 172.21.15.83:0/743831409 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa6bc055cc0 con 0x7fa6c415df00 2024-04-25T00:12:19.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.870+0000 7fa6ca600640 1 -- 172.21.15.83:0/743831409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6c404f5f0 con 0x7fa6c415df00 2024-04-25T00:12:19.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.870+0000 7fa6bb400640 1 -- 172.21.15.83:0/743831409 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16871 (secure 0 0 0) 0x7fa6bc059fc0 con 0x7fa6c415df00 2024-04-25T00:12:19.872 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:19.872 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":37,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:18.565779+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:17.079304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:17.548692+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:19.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.874+0000 7fa6ca600640 1 -- 172.21.15.83:0/743831409 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa6a00678f0 msgr2=0x7fa6a0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.874+0000 7fa6ca600640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa6a00678f0 0x7fa6a0069db0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40ad000 tx=0x7fa6b4008040 comp rx=0 tx=0).stop 2024-04-25T00:12:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.874+0000 7fa6ca600640 1 -- 172.21.15.83:0/743831409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6c415df00 msgr2=0x7fa6c41530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.874+0000 7fa6ca600640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6c415df00 0x7fa6c41530d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00ca20 tx=0x7fa6bc00cee0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.874+0000 7fa6ca600640 1 -- 172.21.15.83:0/743831409 shutdown_connections 2024-04-25T00:12:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.874+0000 7fa6ca600640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa6a00678f0 0x7fa6a0069db0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.874+0000 7fa6ca600640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6c415df00 0x7fa6c41530d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.874+0000 7fa6ca600640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6c4071dc0 0x7fa6c4152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.874+0000 7fa6ca600640 1 --2- 172.21.15.83:0/743831409 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6c406e7b0 0x7fa6c4148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.874+0000 7fa6ca600640 1 -- 172.21.15.83:0/743831409 >> 172.21.15.83:0/743831409 conn(0x7fa6c40b1830 msgr2=0x7fa6c40ae470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.874+0000 7fa6ca600640 1 -- 172.21.15.83:0/743831409 shutdown_connections 2024-04-25T00:12:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:19.874+0000 7fa6ca600640 1 -- 172.21.15.83:0/743831409 wait complete. 2024-04-25T00:12:19.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:12:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.030+0000 7fe08d000640 1 Processor -- start 2024-04-25T00:12:20.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.030+0000 7fe08d000640 1 -- start start 2024-04-25T00:12:20.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.030+0000 7fe08d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe0881555c0 0x7fe08815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:20.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.030+0000 7fe08d000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe08806e4c0 0x7fe08806e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:20.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.030+0000 7fe08d000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe08815c560 con 0x7fe0881551e0 2024-04-25T00:12:20.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.030+0000 7fe08d000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe088072460 con 0x7fe08806e4c0 2024-04-25T00:12:20.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.030+0000 7fe08d000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe08815c7d0 con 0x7fe0881555c0 2024-04-25T00:12:20.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.030+0000 7fe086800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe08806e4c0 0x7fe08806e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:20.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.030+0000 7fe086800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe08806e4c0 0x7fe08806e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34400/0 (socket says 172.21.15.83:34400) 2024-04-25T00:12:20.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.030+0000 7fe086800640 1 -- 172.21.15.83:0/1887996909 learned_addr learned my addr 172.21.15.83:0/1887996909 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:20.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.030+0000 7fe085400640 1 --2- 172.21.15.83:0/1887996909 >> v2:172.21.15.86:3300/0 conn(0x7fe0881555c0 0x7fe08815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:20.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.034+0000 7fe086800640 1 -- 172.21.15.83:0/1887996909 >> v2:172.21.15.86:3300/0 conn(0x7fe0881555c0 msgr2=0x7fe08815bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:20.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.034+0000 7fe086800640 1 --2- 172.21.15.83:0/1887996909 >> v2:172.21.15.86:3300/0 conn(0x7fe0881555c0 0x7fe08815bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:20.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.034+0000 7fe086800640 1 -- 172.21.15.83:0/1887996909 >> v1:172.21.15.86:6789/0 conn(0x7fe0881551e0 legacy=0x7fe088071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:20.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.034+0000 7fe086800640 1 -- 172.21.15.83:0/1887996909 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08815cab0 con 0x7fe08806e4c0 2024-04-25T00:12:20.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.034+0000 7fe086800640 1 --2- 172.21.15.83:0/1887996909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08806e4c0 0x7fe08806e950 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe080009e50 tx=0x7fe08001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57301efb53c5073b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:20.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.034+0000 7fe084a00640 1 -- 172.21.15.83:0/1887996909 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0800a56c0 con 0x7fe08806e4c0 2024-04-25T00:12:20.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.034+0000 7fe084a00640 1 -- 172.21.15.83:0/1887996909 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0800a5860 con 0x7fe08806e4c0 2024-04-25T00:12:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.034+0000 7fe084a00640 1 -- 172.21.15.83:0/1887996909 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0800a5c30 con 0x7fe08806e4c0 2024-04-25T00:12:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.034+0000 7fe08d000640 1 -- 172.21.15.83:0/1887996909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08806e4c0 msgr2=0x7fe08806e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.034+0000 7fe08d000640 1 --2- 172.21.15.83:0/1887996909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08806e4c0 0x7fe08806e950 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe080009e50 tx=0x7fe08001c9f0 comp rx=0 tx=0).stop 2024-04-25T00:12:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 -- 172.21.15.83:0/1887996909 shutdown_connections 2024-04-25T00:12:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 --2- 172.21.15.83:0/1887996909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08806e4c0 0x7fe08806e950 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 --2- 172.21.15.83:0/1887996909 >> v2:172.21.15.86:3300/0 conn(0x7fe0881555c0 0x7fe08815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 -- 172.21.15.83:0/1887996909 >> 172.21.15.83:0/1887996909 conn(0x7fe0880b1830 msgr2=0x7fe0880b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 -- 172.21.15.83:0/1887996909 shutdown_connections 2024-04-25T00:12:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 -- 172.21.15.83:0/1887996909 wait complete. 2024-04-25T00:12:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 Processor -- start 2024-04-25T00:12:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 -- start start 2024-04-25T00:12:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe08806e4c0 0x7fe0880b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0881551e0 0x7fe088150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0881555c0 0x7fe0881470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe08815db70 con 0x7fe0881551e0 2024-04-25T00:12:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe08815d9f0 con 0x7fe0881555c0 2024-04-25T00:12:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe08815dcf0 con 0x7fe08806e4c0 2024-04-25T00:12:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe085400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0881551e0 0x7fe088150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe085400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0881551e0 0x7fe088150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34402/0 (socket says 172.21.15.83:34402) 2024-04-25T00:12:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe085400640 1 -- 172.21.15.83:0/369621906 learned_addr learned my addr 172.21.15.83:0/369621906 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe086800640 1 --2- 172.21.15.83:0/369621906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0881555c0 0x7fe0881470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe085e00640 1 --2- 172.21.15.83:0/369621906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe08806e4c0 0x7fe0880b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe085400640 1 -- 172.21.15.83:0/369621906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe08806e4c0 msgr2=0x7fe0880b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe085400640 1 --2- 172.21.15.83:0/369621906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe08806e4c0 0x7fe0880b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe085400640 1 -- 172.21.15.83:0/369621906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0881555c0 msgr2=0x7fe0881470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe085400640 1 --2- 172.21.15.83:0/369621906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0881555c0 0x7fe0881470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe085400640 1 -- 172.21.15.83:0/369621906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0880b3a10 con 0x7fe0881551e0 2024-04-25T00:12:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe085400640 1 --2- 172.21.15.83:0/369621906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0881551e0 0x7fe088150530 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe07400e870 tx=0x7fe07400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe07e000640 1 -- 172.21.15.83:0/369621906 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074017070 con 0x7fe0881551e0 2024-04-25T00:12:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe07e000640 1 -- 172.21.15.83:0/369621906 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe07400c940 con 0x7fe0881551e0 2024-04-25T00:12:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe07e000640 1 -- 172.21.15.83:0/369621906 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07400cd10 con 0x7fe0881551e0 2024-04-25T00:12:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.038+0000 7fe08d000640 1 -- 172.21.15.83:0/369621906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0880b3cf0 con 0x7fe0881551e0 2024-04-25T00:12:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.042+0000 7fe08d000640 1 -- 172.21.15.83:0/369621906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0880b42b0 con 0x7fe0881551e0 2024-04-25T00:12:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.042+0000 7fe08d000640 1 -- 172.21.15.83:0/369621906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe088150a70 con 0x7fe0881551e0 2024-04-25T00:12:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.042+0000 7fe07e000640 1 -- 172.21.15.83:0/369621906 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe07401a070 con 0x7fe0881551e0 2024-04-25T00:12:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.042+0000 7fe07e000640 1 --2- 172.21.15.83:0/369621906 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe0680678c0 0x7fe068069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.042+0000 7fe07e000640 1 -- 172.21.15.83:0/369621906 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7fe074085c60 con 0x7fe0881551e0 2024-04-25T00:12:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.042+0000 7fe085e00640 1 --2- 172.21.15.83:0/369621906 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe0680678c0 0x7fe068069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:20.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.042+0000 7fe085e00640 1 --2- 172.21.15.83:0/369621906 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe0680678c0 0x7fe068069d80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe088148af0 tx=0x7fe078009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:20.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.046+0000 7fe07e000640 1 -- 172.21.15.83:0/369621906 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe074055fb0 con 0x7fe0881551e0 2024-04-25T00:12:20.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:20.238+0000 7fe08d000640 1 -- 172.21.15.83:0/369621906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe08806ce30 con 0x7fe0881551e0 2024-04-25T00:12:21.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.190+0000 7fe07e000640 1 -- 172.21.15.83:0/369621906 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fe07405a2b0 con 0x7fe0881551e0 2024-04-25T00:12:21.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.194+0000 7fe08d000640 1 -- 172.21.15.83:0/369621906 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe0680678c0 msgr2=0x7fe068069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:21.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.194+0000 7fe08d000640 1 --2- 172.21.15.83:0/369621906 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe0680678c0 0x7fe068069d80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe088148af0 tx=0x7fe078009040 comp rx=0 tx=0).stop 2024-04-25T00:12:21.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.194+0000 7fe08d000640 1 -- 172.21.15.83:0/369621906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0881551e0 msgr2=0x7fe088150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:21.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.194+0000 7fe08d000640 1 --2- 172.21.15.83:0/369621906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0881551e0 0x7fe088150530 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe07400e870 tx=0x7fe07400ed30 comp rx=0 tx=0).stop 2024-04-25T00:12:21.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.194+0000 7fe08d000640 1 -- 172.21.15.83:0/369621906 shutdown_connections 2024-04-25T00:12:21.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.194+0000 7fe08d000640 1 --2- 172.21.15.83:0/369621906 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe0680678c0 0x7fe068069d80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:21.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.194+0000 7fe08d000640 1 --2- 172.21.15.83:0/369621906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0881555c0 0x7fe0881470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:21.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.194+0000 7fe08d000640 1 --2- 172.21.15.83:0/369621906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0881551e0 0x7fe088150530 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:21.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.194+0000 7fe08d000640 1 --2- 172.21.15.83:0/369621906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe08806e4c0 0x7fe0880b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:21.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.194+0000 7fe08d000640 1 -- 172.21.15.83:0/369621906 >> 172.21.15.83:0/369621906 conn(0x7fe0880b1830 msgr2=0x7fe088071540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:21.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.194+0000 7fe08d000640 1 -- 172.21.15.83:0/369621906 shutdown_connections 2024-04-25T00:12:21.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.194+0000 7fe08d000640 1 -- 172.21.15.83:0/369621906 wait complete. 2024-04-25T00:12:21.219 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:12:21.219 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:12:21.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.358+0000 7f856b600640 1 Processor -- start 2024-04-25T00:12:21.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.358+0000 7f856b600640 1 -- start start 2024-04-25T00:12:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f856b600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f856406e950 0x7f856415f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f856b600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f856415fe50 0x7f85640ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f856b600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85640ce000 con 0x7f8564159810 2024-04-25T00:12:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f856b600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f856406e210 con 0x7f856415fe50 2024-04-25T00:12:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f856b600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f856406da00 con 0x7f856406e950 2024-04-25T00:12:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f8568c00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8564159810 0x7f85640cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56958/0 (socket says 172.21.15.83:56958) 2024-04-25T00:12:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f8568c00640 1 -- 172.21.15.83:0/402921318 learned_addr learned my addr 172.21.15.83:0/402921318 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f8563e00640 1 --2- 172.21.15.83:0/402921318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f856406e950 0x7f856415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:21.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f8569600640 1 --2- 172.21.15.83:0/402921318 >> v2:172.21.15.86:3300/0 conn(0x7f856415fe50 0x7f85640ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:21.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f8563400640 1 -- 172.21.15.83:0/402921318 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2634465464 0 0) 0x7f85640ce000 con 0x7f8564159810 2024-04-25T00:12:21.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f8563400640 1 -- 172.21.15.83:0/402921318 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8548003530 con 0x7f8564159810 2024-04-25T00:12:21.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f8563e00640 1 -- 172.21.15.83:0/402921318 >> v2:172.21.15.86:3300/0 conn(0x7f856415fe50 msgr2=0x7f85640ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:21.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f8563e00640 1 --2- 172.21.15.83:0/402921318 >> v2:172.21.15.86:3300/0 conn(0x7f856415fe50 0x7f85640ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:21.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f8563e00640 1 -- 172.21.15.83:0/402921318 >> v1:172.21.15.83:6789/0 conn(0x7f8564159810 legacy=0x7f85640cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:21.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.362+0000 7f8563e00640 1 -- 172.21.15.83:0/402921318 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856406eec0 con 0x7f856406e950 2024-04-25T00:12:21.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.366+0000 7f8563e00640 1 --2- 172.21.15.83:0/402921318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f856406e950 0x7f856415f910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f85500097c0 tx=0x7f855001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64e52f3e7171ea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:21.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.366+0000 7f8563400640 1 -- 172.21.15.83:0/402921318 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85500b0070 con 0x7f856406e950 2024-04-25T00:12:21.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.366+0000 7f8563400640 1 -- 172.21.15.83:0/402921318 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85500ab600 con 0x7f856406e950 2024-04-25T00:12:21.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.366+0000 7f8563400640 1 -- 172.21.15.83:0/402921318 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85500ab9d0 con 0x7f856406e950 2024-04-25T00:12:21.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.366+0000 7f856b600640 1 -- 172.21.15.83:0/402921318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f856406e950 msgr2=0x7f856415f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:21.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.366+0000 7f856b600640 1 --2- 172.21.15.83:0/402921318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f856406e950 0x7f856415f910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f85500097c0 tx=0x7f855001c410 comp rx=0 tx=0).stop 2024-04-25T00:12:21.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.366+0000 7f856b600640 1 -- 172.21.15.83:0/402921318 shutdown_connections 2024-04-25T00:12:21.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.366+0000 7f856b600640 1 --2- 172.21.15.83:0/402921318 >> v2:172.21.15.86:3300/0 conn(0x7f856415fe50 0x7f85640ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:21.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.366+0000 7f856b600640 1 --2- 172.21.15.83:0/402921318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f856406e950 0x7f856415f910 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:21.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.366+0000 7f856b600640 1 -- 172.21.15.83:0/402921318 >> 172.21.15.83:0/402921318 conn(0x7f85640b1830 msgr2=0x7f85640b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:21.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.366+0000 7f856b600640 1 -- 172.21.15.83:0/402921318 shutdown_connections 2024-04-25T00:12:21.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.366+0000 7f856b600640 1 -- 172.21.15.83:0/402921318 wait complete. 2024-04-25T00:12:21.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.366+0000 7f856b600640 1 Processor -- start 2024-04-25T00:12:21.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.366+0000 7f856b600640 1 -- start start 2024-04-25T00:12:21.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f856b600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f856406e950 0x7f85640b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:21.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f856b600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8564159810 0x7f8564166220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:21.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f856b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f856415fe50 0x7f85640b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:21.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f856b600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8564070f20 con 0x7f856406e950 2024-04-25T00:12:21.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f856b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8564070da0 con 0x7f856415fe50 2024-04-25T00:12:21.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f856b600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8564168cb0 con 0x7f8564159810 2024-04-25T00:12:21.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8568c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f856406e950 0x7f85640b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:21.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8568c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f856406e950 0x7f85640b4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34408/0 (socket says 172.21.15.83:34408) 2024-04-25T00:12:21.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8568c00640 1 -- 172.21.15.83:0/3330816123 learned_addr learned my addr 172.21.15.83:0/3330816123 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:21.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8569600640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f856415fe50 0x7f85640b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:21.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8563e00640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8564159810 0x7f8564166220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:21.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8568c00640 1 -- 172.21.15.83:0/3330816123 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8564159810 msgr2=0x7f8564166220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:21.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8568c00640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8564159810 0x7f8564166220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:21.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8568c00640 1 -- 172.21.15.83:0/3330816123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f856415fe50 msgr2=0x7f85640b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:21.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8568c00640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f856415fe50 0x7f85640b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:21.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8568c00640 1 -- 172.21.15.83:0/3330816123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8564158010 con 0x7f856406e950 2024-04-25T00:12:21.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8569600640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f856415fe50 0x7f85640b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:21.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8568c00640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f856406e950 0x7f85640b4500 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f855400cb70 tx=0x7f855400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:21.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8560c00640 1 -- 172.21.15.83:0/3330816123 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554017070 con 0x7f856406e950 2024-04-25T00:12:21.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8560c00640 1 -- 172.21.15.83:0/3330816123 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855400ad40 con 0x7f856406e950 2024-04-25T00:12:21.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f856b600640 1 -- 172.21.15.83:0/3330816123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85641582f0 con 0x7f856406e950 2024-04-25T00:12:21.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8560c00640 1 -- 172.21.15.83:0/3330816123 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554012620 con 0x7f856406e950 2024-04-25T00:12:21.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f856b600640 1 -- 172.21.15.83:0/3330816123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8564165d00 con 0x7f856406e950 2024-04-25T00:12:21.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f856b600640 1 -- 172.21.15.83:0/3330816123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f856404f510 con 0x7f856406e950 2024-04-25T00:12:21.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.370+0000 7f8560c00640 1 -- 172.21.15.83:0/3330816123 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f855401a070 con 0x7f856406e950 2024-04-25T00:12:21.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.374+0000 7f8560c00640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85440678c0 0x7f8544069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:21.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.374+0000 7f8560c00640 1 -- 172.21.15.83:0/3330816123 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f8554085c20 con 0x7f856406e950 2024-04-25T00:12:21.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.374+0000 7f8563e00640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85440678c0 0x7f8544069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:21.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.374+0000 7f8563e00640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85440678c0 0x7f8544069d80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f85500ad000 tx=0x7f8550008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:21.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.378+0000 7f8560c00640 1 -- 172.21.15.83:0/3330816123 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8554055ee0 con 0x7f856406e950 2024-04-25T00:12:21.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:21.570+0000 7f856b600640 1 -- 172.21.15.83:0/3330816123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8564072c90 con 0x7f856406e950 2024-04-25T00:12:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.198+0000 7f8560c00640 1 -- 172.21.15.83:0/3330816123 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f855405a1e0 con 0x7f856406e950 2024-04-25T00:12:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.202+0000 7f856b600640 1 -- 172.21.15.83:0/3330816123 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85440678c0 msgr2=0x7f8544069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.202+0000 7f856b600640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85440678c0 0x7f8544069d80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f85500ad000 tx=0x7f8550008040 comp rx=0 tx=0).stop 2024-04-25T00:12:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.202+0000 7f856b600640 1 -- 172.21.15.83:0/3330816123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f856406e950 msgr2=0x7f85640b4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.202+0000 7f856b600640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f856406e950 0x7f85640b4500 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f855400cb70 tx=0x7f855400a430 comp rx=0 tx=0).stop 2024-04-25T00:12:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.202+0000 7f856b600640 1 -- 172.21.15.83:0/3330816123 shutdown_connections 2024-04-25T00:12:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.202+0000 7f856b600640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85440678c0 0x7f8544069d80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.202+0000 7f856b600640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8564159810 0x7f8564166220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.202+0000 7f856b600640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f856415fe50 0x7f85640b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.202+0000 7f856b600640 1 --2- 172.21.15.83:0/3330816123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f856406e950 0x7f85640b4500 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.202+0000 7f856b600640 1 -- 172.21.15.83:0/3330816123 >> 172.21.15.83:0/3330816123 conn(0x7f85640b1830 msgr2=0x7f8564071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.202+0000 7f856b600640 1 -- 172.21.15.83:0/3330816123 shutdown_connections 2024-04-25T00:12:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.202+0000 7f856b600640 1 -- 172.21.15.83:0/3330816123 wait complete. 2024-04-25T00:12:22.223 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:12:22.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:12:22.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.358+0000 7ff043e00640 1 Processor -- start 2024-04-25T00:12:22.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.358+0000 7ff043e00640 1 -- start start 2024-04-25T00:12:22.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff043e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03c0cc510 0x7ff03c15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff043e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff03c0ccc60 con 0x7ff03c15cf90 2024-04-25T00:12:22.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff043e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff03c06dba0 con 0x7ff03c1599f0 2024-04-25T00:12:22.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff043e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff03c159dd0 con 0x7ff03c0cc510 2024-04-25T00:12:22.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff041e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff03c15cf90 0x7ff03c0ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56960/0 (socket says 172.21.15.83:56960) 2024-04-25T00:12:22.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff041e00640 1 -- 172.21.15.83:0/3547755271 learned_addr learned my addr 172.21.15.83:0/3547755271 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:22.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff040a00640 1 --2- 172.21.15.83:0/3547755271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03c0cc510 0x7ff03c15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 205990490 0 0) 0x7ff0340014a0 con 0x7ff03c15cf90 2024-04-25T00:12:22.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff024003530 con 0x7ff03c15cf90 2024-04-25T00:12:22.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2688299952 0 0) 0x7ff03c06dba0 con 0x7ff03c1599f0 2024-04-25T00:12:22.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0340014a0 con 0x7ff03c1599f0 2024-04-25T00:12:22.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2962003627 0 0) 0x7ff024003530 con 0x7ff03c15cf90 2024-04-25T00:12:22.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03c0cc510 msgr2=0x7ff03c15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:22.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 --2- 172.21.15.83:0/3547755271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03c0cc510 0x7ff03c15ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 >> v1:172.21.15.86:6789/0 conn(0x7ff03c1599f0 legacy=0x7ff03c15c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c06de80 con 0x7ff03c15cf90 2024-04-25T00:12:22.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7ff034001b50 con 0x7ff03c15cf90 2024-04-25T00:12:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 >> v1:172.21.15.83:6789/0 conn(0x7ff03c15cf90 legacy=0x7ff03c0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 --2- 172.21.15.83:0/3547755271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff024003b80 0x7ff024003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 --2- 172.21.15.83:0/3547755271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff024004810 0x7ff024004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 --2- 172.21.15.83:0/3547755271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03c0cc510 0x7ff024005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff02400bb60 con 0x7ff024004810 2024-04-25T00:12:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff02400bce0 con 0x7ff024003b80 2024-04-25T00:12:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff02400bee0 con 0x7ff03c0cc510 2024-04-25T00:12:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff040a00640 1 --2- 172.21.15.83:0/3547755271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff024004810 0x7ff024004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff041400640 1 --2- 172.21.15.83:0/3547755271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff024003b80 0x7ff024003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff041e00640 1 --2- 172.21.15.83:0/3547755271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03c0cc510 0x7ff024005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff040a00640 1 -- 172.21.15.83:0/3547755271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03c0cc510 msgr2=0x7ff024005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff040a00640 1 --2- 172.21.15.83:0/3547755271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03c0cc510 0x7ff024005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff040a00640 1 -- 172.21.15.83:0/3547755271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff024003b80 msgr2=0x7ff024003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:22.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff040a00640 1 --2- 172.21.15.83:0/3547755271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff024003b80 0x7ff024003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff040a00640 1 -- 172.21.15.83:0/3547755271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff024003530 con 0x7ff024004810 2024-04-25T00:12:22.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.362+0000 7ff041e00640 1 --2- 172.21.15.83:0/3547755271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03c0cc510 0x7ff024005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.366+0000 7ff040a00640 1 --2- 172.21.15.83:0/3547755271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff024004810 0x7ff024004c90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff028009ac0 tx=0x7ff02801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c45b710d8defce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.366+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff028009690 con 0x7ff024004810 2024-04-25T00:12:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.366+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0280a6430 con 0x7ff024004810 2024-04-25T00:12:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.366+0000 7ff03be00640 1 -- 172.21.15.83:0/3547755271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0280a6800 con 0x7ff024004810 2024-04-25T00:12:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.366+0000 7ff043e00640 1 -- 172.21.15.83:0/3547755271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff024004810 msgr2=0x7ff024004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.366+0000 7ff043e00640 1 --2- 172.21.15.83:0/3547755271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff024004810 0x7ff024004c90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff028009ac0 tx=0x7ff02801ca50 comp rx=0 tx=0).stop 2024-04-25T00:12:22.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.366+0000 7ff041e00640 1 -- 172.21.15.83:0/3547755271 reap_dead start 2024-04-25T00:12:22.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.366+0000 7ff043e00640 1 -- 172.21.15.83:0/3547755271 shutdown_connections 2024-04-25T00:12:22.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.366+0000 7ff043e00640 1 -- 172.21.15.83:0/3547755271 >> 172.21.15.83:0/3547755271 conn(0x7ff03c0b1830 msgr2=0x7ff03c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:22.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.366+0000 7ff043e00640 1 -- 172.21.15.83:0/3547755271 shutdown_connections 2024-04-25T00:12:22.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.366+0000 7ff043e00640 1 -- 172.21.15.83:0/3547755271 wait complete. 2024-04-25T00:12:22.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff043e00640 1 Processor -- start 2024-04-25T00:12:22.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff043e00640 1 -- start start 2024-04-25T00:12:22.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff043e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff024004810 0x7ff03c14bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff043e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff03c14c2e0 0x7ff03c15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff043e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03c15b4e0 0x7ff03c160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff043e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c0cf780 con 0x7ff024004810 2024-04-25T00:12:22.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff043e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c0cf600 con 0x7ff03c14c2e0 2024-04-25T00:12:22.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff043e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff03c0cf900 con 0x7ff03c15b4e0 2024-04-25T00:12:22.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff041400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff024004810 0x7ff03c14bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff041400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff024004810 0x7ff03c14bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34432/0 (socket says 172.21.15.83:34432) 2024-04-25T00:12:22.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff041400640 1 -- 172.21.15.83:0/2746637147 learned_addr learned my addr 172.21.15.83:0/2746637147 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:22.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff041400640 1 -- 172.21.15.83:0/2746637147 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03c15b4e0 msgr2=0x7ff03c160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff041400640 1 --2- 172.21.15.83:0/2746637147 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03c15b4e0 0x7ff03c160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff041400640 1 -- 172.21.15.83:0/2746637147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff03c14c2e0 msgr2=0x7ff03c15afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff041400640 1 --2- 172.21.15.83:0/2746637147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff03c14c2e0 0x7ff03c15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff041400640 1 -- 172.21.15.83:0/2746637147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c159f00 con 0x7ff024004810 2024-04-25T00:12:22.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff041400640 1 --2- 172.21.15.83:0/2746637147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff024004810 0x7ff03c14bda0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff03000f7e0 tx=0x7ff03000fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:22.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff039600640 1 -- 172.21.15.83:0/2746637147 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0300052e0 con 0x7ff024004810 2024-04-25T00:12:22.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff039600640 1 -- 172.21.15.83:0/2746637147 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff030018070 con 0x7ff024004810 2024-04-25T00:12:22.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff043e00640 1 -- 172.21.15.83:0/2746637147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03c15a1e0 con 0x7ff024004810 2024-04-25T00:12:22.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff039600640 1 -- 172.21.15.83:0/2746637147 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0300054a0 con 0x7ff024004810 2024-04-25T00:12:22.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff043e00640 1 -- 172.21.15.83:0/2746637147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03c15a7a0 con 0x7ff024004810 2024-04-25T00:12:22.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.370+0000 7ff043e00640 1 -- 172.21.15.83:0/2746637147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff03c15b900 con 0x7ff024004810 2024-04-25T00:12:22.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.374+0000 7ff039600640 1 -- 172.21.15.83:0/2746637147 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff03000d8f0 con 0x7ff024004810 2024-04-25T00:12:22.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.374+0000 7ff039600640 1 --2- 172.21.15.83:0/2746637147 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff01c0678c0 0x7ff01c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.374+0000 7ff039600640 1 -- 172.21.15.83:0/2746637147 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7ff030085d20 con 0x7ff024004810 2024-04-25T00:12:22.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.374+0000 7ff040a00640 1 --2- 172.21.15.83:0/2746637147 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff01c0678c0 0x7ff01c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.374+0000 7ff040a00640 1 --2- 172.21.15.83:0/2746637147 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff01c0678c0 0x7ff01c069d80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff028009ac0 tx=0x7ff028004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:22.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.378+0000 7ff039600640 1 -- 172.21.15.83:0/2746637147 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff03001a310 con 0x7ff024004810 2024-04-25T00:12:22.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.566+0000 7ff043e00640 1 -- 172.21.15.83:0/2746637147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff03c04f5f0 con 0x7ff024004810 2024-04-25T00:12:22.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.566+0000 7ff039600640 1 -- 172.21.15.83:0/2746637147 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7ff030056760 con 0x7ff024004810 2024-04-25T00:12:22.568 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:22.568 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4277921554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":4277921554},{"type":"v1","addr":"172.21.15.86:6835","nonce":4277921554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/4145334103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":4145334103},{"type":"v1","addr":"172.21.15.86:6834","nonce":4145334103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3441801758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3441801758},{"type":"v1","addr":"172.21.15.83:6836","nonce":3441801758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:12:18.566138+0000","modified":"2024-04-25T00:12:21.572278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5328},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5328":{"gid":5328,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2704243724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2704243724},{"type":"v1","addr":"172.21.15.83:6837","nonce":2704243724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5328,"qdb_cluster":[5328]},"id":2}]} 2024-04-25T00:12:22.569 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 18 2024-04-25T00:12:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.570+0000 7ff043e00640 1 -- 172.21.15.83:0/2746637147 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff01c0678c0 msgr2=0x7ff01c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.570+0000 7ff043e00640 1 --2- 172.21.15.83:0/2746637147 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff01c0678c0 0x7ff01c069d80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff028009ac0 tx=0x7ff028004740 comp rx=0 tx=0).stop 2024-04-25T00:12:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.570+0000 7ff043e00640 1 -- 172.21.15.83:0/2746637147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff024004810 msgr2=0x7ff03c14bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.570+0000 7ff043e00640 1 --2- 172.21.15.83:0/2746637147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff024004810 0x7ff03c14bda0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff03000f7e0 tx=0x7ff03000fca0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.570+0000 7ff043e00640 1 -- 172.21.15.83:0/2746637147 shutdown_connections 2024-04-25T00:12:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.570+0000 7ff043e00640 1 --2- 172.21.15.83:0/2746637147 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff01c0678c0 0x7ff01c069d80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.570+0000 7ff043e00640 1 --2- 172.21.15.83:0/2746637147 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03c15b4e0 0x7ff03c160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.570+0000 7ff043e00640 1 --2- 172.21.15.83:0/2746637147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff03c14c2e0 0x7ff03c15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.570+0000 7ff043e00640 1 --2- 172.21.15.83:0/2746637147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff024004810 0x7ff03c14bda0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.570+0000 7ff043e00640 1 -- 172.21.15.83:0/2746637147 >> 172.21.15.83:0/2746637147 conn(0x7ff03c0b1830 msgr2=0x7ff03c0b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.570+0000 7ff043e00640 1 -- 172.21.15.83:0/2746637147 shutdown_connections 2024-04-25T00:12:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.570+0000 7ff043e00640 1 -- 172.21.15.83:0/2746637147 wait complete. 2024-04-25T00:12:22.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.722+0000 7f205c800640 1 Processor -- start 2024-04-25T00:12:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.722+0000 7f205c800640 1 -- start start 2024-04-25T00:12:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.726+0000 7f205c800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f20541599f0 0x7f20540cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.726+0000 7f205c800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f205415c340 0x7f205415c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.726+0000 7f205c800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20540ccc60 con 0x7f205415ccf0 2024-04-25T00:12:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.726+0000 7f205c800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2054159dd0 con 0x7f20541599f0 2024-04-25T00:12:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.726+0000 7f205c800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f205406dba0 con 0x7f205415c340 2024-04-25T00:12:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.726+0000 7f2059e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f20541599f0 0x7f20540cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.726+0000 7f2059e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f20541599f0 0x7f20540cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34440/0 (socket says 172.21.15.83:34440) 2024-04-25T00:12:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.726+0000 7f2059e00640 1 -- 172.21.15.83:0/1865685741 learned_addr learned my addr 172.21.15.83:0/1865685741 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.726+0000 7f2059400640 1 --2- 172.21.15.83:0/1865685741 >> v2:172.21.15.86:3300/0 conn(0x7f205415c340 0x7f205415c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.726+0000 7f2059e00640 1 -- 172.21.15.83:0/1865685741 >> v2:172.21.15.86:3300/0 conn(0x7f205415c340 msgr2=0x7f205415c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.726+0000 7f2059e00640 1 --2- 172.21.15.83:0/1865685741 >> v2:172.21.15.86:3300/0 conn(0x7f205415c340 0x7f205415c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.726+0000 7f2059e00640 1 -- 172.21.15.83:0/1865685741 >> v1:172.21.15.86:6789/0 conn(0x7f205415ccf0 legacy=0x7f205415f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.726+0000 7f2059e00640 1 -- 172.21.15.83:0/1865685741 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205406de80 con 0x7f20541599f0 2024-04-25T00:12:22.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f2059e00640 1 --2- 172.21.15.83:0/1865685741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20541599f0 0x7f20540cc510 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2048009870 tx=0x7f204801ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0ece2d85cca4e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f2058a00640 1 -- 172.21.15.83:0/1865685741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20480b0070 con 0x7f20541599f0 2024-04-25T00:12:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f2058a00640 1 -- 172.21.15.83:0/1865685741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20480ab600 con 0x7f20541599f0 2024-04-25T00:12:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f2058a00640 1 -- 172.21.15.83:0/1865685741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20480ab9d0 con 0x7f20541599f0 2024-04-25T00:12:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f205c800640 1 -- 172.21.15.83:0/1865685741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20541599f0 msgr2=0x7f20540cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f205c800640 1 --2- 172.21.15.83:0/1865685741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20541599f0 0x7f20540cc510 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2048009870 tx=0x7f204801ccd0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f205c800640 1 -- 172.21.15.83:0/1865685741 shutdown_connections 2024-04-25T00:12:22.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f205c800640 1 --2- 172.21.15.83:0/1865685741 >> v2:172.21.15.86:3300/0 conn(0x7f205415c340 0x7f205415c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f205c800640 1 --2- 172.21.15.83:0/1865685741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20541599f0 0x7f20540cc510 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f205c800640 1 -- 172.21.15.83:0/1865685741 >> 172.21.15.83:0/1865685741 conn(0x7f20540b1830 msgr2=0x7f20540b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:22.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f205c800640 1 -- 172.21.15.83:0/1865685741 shutdown_connections 2024-04-25T00:12:22.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f205c800640 1 -- 172.21.15.83:0/1865685741 wait complete. 2024-04-25T00:12:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f205c800640 1 Processor -- start 2024-04-25T00:12:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f205c800640 1 -- start start 2024-04-25T00:12:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.730+0000 7f205c800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f20541599f0 0x7f20540b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f205c800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f205415c340 0x7f20540b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f205c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f205415ccf0 0x7f20541f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f205c800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20540cf870 con 0x7f205415c340 2024-04-25T00:12:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f205c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20540cf6f0 con 0x7f205415ccf0 2024-04-25T00:12:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f205c800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2054168cd0 con 0x7f20541599f0 2024-04-25T00:12:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f2059400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f205415c340 0x7f20540b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f2059400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f205415c340 0x7f20540b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34442/0 (socket says 172.21.15.83:34442) 2024-04-25T00:12:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f2059400640 1 -- 172.21.15.83:0/1965468166 learned_addr learned my addr 172.21.15.83:0/1965468166 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f205a800640 1 --2- 172.21.15.83:0/1965468166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f205415ccf0 0x7f20541f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f2059e00640 1 --2- 172.21.15.83:0/1965468166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f20541599f0 0x7f20540b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f2059400640 1 -- 172.21.15.83:0/1965468166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f20541599f0 msgr2=0x7f20540b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f2059400640 1 --2- 172.21.15.83:0/1965468166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f20541599f0 0x7f20540b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f2059400640 1 -- 172.21.15.83:0/1965468166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f205415ccf0 msgr2=0x7f20541f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f2059400640 1 --2- 172.21.15.83:0/1965468166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f205415ccf0 0x7f20541f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f2059400640 1 -- 172.21.15.83:0/1965468166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20540b41b0 con 0x7f205415c340 2024-04-25T00:12:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f2059400640 1 --2- 172.21.15.83:0/1965468166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f205415c340 0x7f20540b3c70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f204400d830 tx=0x7f204400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f204e000640 1 -- 172.21.15.83:0/1965468166 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204400b6c0 con 0x7f205415c340 2024-04-25T00:12:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f204e000640 1 -- 172.21.15.83:0/1965468166 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2044019070 con 0x7f205415c340 2024-04-25T00:12:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f205c800640 1 -- 172.21.15.83:0/1965468166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20540b4480 con 0x7f205415c340 2024-04-25T00:12:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f204e000640 1 -- 172.21.15.83:0/1965468166 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204400bae0 con 0x7f205415c340 2024-04-25T00:12:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f205c800640 1 -- 172.21.15.83:0/1965468166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20540b4a40 con 0x7f205415c340 2024-04-25T00:12:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f205c800640 1 -- 172.21.15.83:0/1965468166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20540737f0 con 0x7f205415c340 2024-04-25T00:12:22.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.734+0000 7f204e000640 1 -- 172.21.15.83:0/1965468166 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f204401e460 con 0x7f205415c340 2024-04-25T00:12:22.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.738+0000 7f204e000640 1 --2- 172.21.15.83:0/1965468166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f20380678c0 0x7f2038069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.738+0000 7f204e000640 1 -- 172.21.15.83:0/1965468166 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f2044086610 con 0x7f205415c340 2024-04-25T00:12:22.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.738+0000 7f2059e00640 1 --2- 172.21.15.83:0/1965468166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f20380678c0 0x7f2038069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.738+0000 7f2059e00640 1 --2- 172.21.15.83:0/1965468166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f20380678c0 0x7f2038069d80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f20480ad000 tx=0x7f2048004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:22.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.742+0000 7f204e000640 1 -- 172.21.15.83:0/1965468166 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2044056f20 con 0x7f205415c340 2024-04-25T00:12:22.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.898+0000 7f205c800640 1 -- 172.21.15.83:0/1965468166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f205404f5f0 con 0x7f205415c340 2024-04-25T00:12:22.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.898+0000 7f204e000640 1 -- 172.21.15.83:0/1965468166 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16858 (secure 0 0 0) 0x7f2044056820 con 0x7f205415c340 2024-04-25T00:12:22.900 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:22.900 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:20.181255+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:17.079304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:17.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.902+0000 7f205c800640 1 -- 172.21.15.83:0/1965468166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f20380678c0 msgr2=0x7f2038069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.902+0000 7f205c800640 1 --2- 172.21.15.83:0/1965468166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f20380678c0 0x7f2038069d80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f20480ad000 tx=0x7f2048004740 comp rx=0 tx=0).stop 2024-04-25T00:12:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.902+0000 7f205c800640 1 -- 172.21.15.83:0/1965468166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f205415c340 msgr2=0x7f20540b3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.902+0000 7f205c800640 1 --2- 172.21.15.83:0/1965468166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f205415c340 0x7f20540b3c70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f204400d830 tx=0x7f204400dcf0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.902+0000 7f205c800640 1 -- 172.21.15.83:0/1965468166 shutdown_connections 2024-04-25T00:12:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.902+0000 7f205c800640 1 --2- 172.21.15.83:0/1965468166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f20380678c0 0x7f2038069d80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.902+0000 7f205c800640 1 --2- 172.21.15.83:0/1965468166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f205415ccf0 0x7f20541f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.902+0000 7f205c800640 1 --2- 172.21.15.83:0/1965468166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f205415c340 0x7f20540b3c70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.902+0000 7f205c800640 1 --2- 172.21.15.83:0/1965468166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f20541599f0 0x7f20540b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.902+0000 7f205c800640 1 -- 172.21.15.83:0/1965468166 >> 172.21.15.83:0/1965468166 conn(0x7f20540b1830 msgr2=0x7f205406cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.902+0000 7f205c800640 1 -- 172.21.15.83:0/1965468166 shutdown_connections 2024-04-25T00:12:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:22.902+0000 7f205c800640 1 -- 172.21.15.83:0/1965468166 wait complete. 2024-04-25T00:12:22.922 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:12:22.922 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:12:22.982 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-04-25T00:12:22.982+0000 7fc2de800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:12:23.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff52d600640 1 Processor -- start 2024-04-25T00:12:23.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff52d600640 1 -- start start 2024-04-25T00:12:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff52d600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52815e110 0x7ff5280ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff52d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff52806fdc0 0x7ff5280ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff52d600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5280d0ad0 con 0x7ff52815e4f0 2024-04-25T00:12:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff52d600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff5280d0db0 con 0x7ff52806fdc0 2024-04-25T00:12:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff52d600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff5280cc3b0 con 0x7ff52815e110 2024-04-25T00:12:23.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff527e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52815e110 0x7ff5280ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff52cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff52806fdc0 0x7ff5280ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff52cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff52806fdc0 0x7ff5280ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:47148/0 (socket says 172.21.15.83:47148) 2024-04-25T00:12:23.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff52cc00640 1 -- 172.21.15.83:0/2140097497 learned_addr learned my addr 172.21.15.83:0/2140097497 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:23.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff52cc00640 1 -- 172.21.15.83:0/2140097497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52815e110 msgr2=0x7ff5280ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff52cc00640 1 --2- 172.21.15.83:0/2140097497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52815e110 0x7ff5280ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff52cc00640 1 -- 172.21.15.83:0/2140097497 >> v1:172.21.15.86:6789/0 conn(0x7ff52815e4f0 legacy=0x7ff5280ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.058+0000 7ff52cc00640 1 -- 172.21.15.83:0/2140097497 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5280d0760 con 0x7ff52806fdc0 2024-04-25T00:12:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.062+0000 7ff52cc00640 1 --2- 172.21.15.83:0/2140097497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52806fdc0 0x7ff5280ccbb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff520009870 tx=0x7ff52001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa7bd34c24edb408 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.062+0000 7ff527400640 1 -- 172.21.15.83:0/2140097497 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5200b0070 con 0x7ff52806fdc0 2024-04-25T00:12:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.062+0000 7ff527400640 1 -- 172.21.15.83:0/2140097497 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5200ab600 con 0x7ff52806fdc0 2024-04-25T00:12:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.062+0000 7ff527400640 1 -- 172.21.15.83:0/2140097497 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5200ab9d0 con 0x7ff52806fdc0 2024-04-25T00:12:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.062+0000 7ff52d600640 1 -- 172.21.15.83:0/2140097497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52806fdc0 msgr2=0x7ff5280ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.062+0000 7ff52d600640 1 --2- 172.21.15.83:0/2140097497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52806fdc0 0x7ff5280ccbb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff520009870 tx=0x7ff52001c4d0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.062+0000 7ff52d600640 1 -- 172.21.15.83:0/2140097497 shutdown_connections 2024-04-25T00:12:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.062+0000 7ff52d600640 1 --2- 172.21.15.83:0/2140097497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52806fdc0 0x7ff5280ccbb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.062+0000 7ff52d600640 1 --2- 172.21.15.83:0/2140097497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52815e110 0x7ff5280ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.062+0000 7ff52d600640 1 -- 172.21.15.83:0/2140097497 >> 172.21.15.83:0/2140097497 conn(0x7ff5280d1bc0 msgr2=0x7ff5280d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.062+0000 7ff52d600640 1 -- 172.21.15.83:0/2140097497 shutdown_connections 2024-04-25T00:12:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.062+0000 7ff52d600640 1 -- 172.21.15.83:0/2140097497 wait complete. 2024-04-25T00:12:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff52d600640 1 Processor -- start 2024-04-25T00:12:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff52d600640 1 -- start start 2024-04-25T00:12:23.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff52d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff52806fdc0 0x7ff528158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff52d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52815e110 0x7ff52814ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff52d600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52815e4f0 0x7ff52815c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff52d600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff5280cdc90 con 0x7ff52806fdc0 2024-04-25T00:12:23.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff52d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5280cdb10 con 0x7ff52815e110 2024-04-25T00:12:23.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff52d600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff5280cde10 con 0x7ff52815e4f0 2024-04-25T00:12:23.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff527e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff52806fdc0 0x7ff528158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff527e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff52806fdc0 0x7ff528158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34450/0 (socket says 172.21.15.83:34450) 2024-04-25T00:12:23.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff527e00640 1 -- 172.21.15.83:0/2610990568 learned_addr learned my addr 172.21.15.83:0/2610990568 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:23.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff52cc00640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52815e4f0 0x7ff52815c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff51f400640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52815e110 0x7ff52814ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff527e00640 1 -- 172.21.15.83:0/2610990568 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52815e4f0 msgr2=0x7ff52815c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff527e00640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52815e4f0 0x7ff52815c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff527e00640 1 -- 172.21.15.83:0/2610990568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52815e110 msgr2=0x7ff52814ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff527e00640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52815e110 0x7ff52814ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff527e00640 1 -- 172.21.15.83:0/2610990568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52814d410 con 0x7ff52806fdc0 2024-04-25T00:12:23.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff52cc00640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52815e4f0 0x7ff52815c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:23.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff527e00640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff52806fdc0 0x7ff528158810 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff51800e8a0 tx=0x7ff51800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:23.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff524c00640 1 -- 172.21.15.83:0/2610990568 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518017070 con 0x7ff52806fdc0 2024-04-25T00:12:23.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff524c00640 1 -- 172.21.15.83:0/2610990568 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff51800c940 con 0x7ff52806fdc0 2024-04-25T00:12:23.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff52d600640 1 -- 172.21.15.83:0/2610990568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52815a1c0 con 0x7ff52806fdc0 2024-04-25T00:12:23.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff524c00640 1 -- 172.21.15.83:0/2610990568 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51800ccf0 con 0x7ff52806fdc0 2024-04-25T00:12:23.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.066+0000 7ff52d600640 1 -- 172.21.15.83:0/2610990568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5281604f0 con 0x7ff52806fdc0 2024-04-25T00:12:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.070+0000 7ff52d600640 1 -- 172.21.15.83:0/2610990568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff52804f510 con 0x7ff52806fdc0 2024-04-25T00:12:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.070+0000 7ff524c00640 1 -- 172.21.15.83:0/2610990568 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff51801a070 con 0x7ff52806fdc0 2024-04-25T00:12:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.070+0000 7ff524c00640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5040678f0 0x7ff504069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.070+0000 7ff524c00640 1 -- 172.21.15.83:0/2610990568 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7ff518085ca0 con 0x7ff52806fdc0 2024-04-25T00:12:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.070+0000 7ff51f400640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5040678f0 0x7ff504069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.070+0000 7ff51f400640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5040678f0 0x7ff504069db0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff52815d420 tx=0x7ff510009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:23.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.074+0000 7ff524c00640 1 -- 172.21.15.83:0/2610990568 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff52814d410 con 0x7ff52806fdc0 2024-04-25T00:12:23.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.230+0000 7ff52d600640 1 -- 172.21.15.83:0/2610990568 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff528157770 con 0x7ff5040678f0 2024-04-25T00:12:23.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.238+0000 7ff524c00640 1 -- 172.21.15.83:0/2610990568 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251890 (secure 0 0 0) 0x7ff528157770 con 0x7ff5040678f0 2024-04-25T00:12:23.242 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:12:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.242+0000 7ff52d600640 1 -- 172.21.15.83:0/2610990568 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5040678f0 msgr2=0x7ff504069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.242+0000 7ff52d600640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5040678f0 0x7ff504069db0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff52815d420 tx=0x7ff510009040 comp rx=0 tx=0).stop 2024-04-25T00:12:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.242+0000 7ff52d600640 1 -- 172.21.15.83:0/2610990568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff52806fdc0 msgr2=0x7ff528158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.242+0000 7ff52d600640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff52806fdc0 0x7ff528158810 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff51800e8a0 tx=0x7ff51800eca0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.242+0000 7ff52d600640 1 -- 172.21.15.83:0/2610990568 shutdown_connections 2024-04-25T00:12:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.242+0000 7ff52d600640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5040678f0 0x7ff504069db0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.242+0000 7ff52d600640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52815e4f0 0x7ff52815c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.242+0000 7ff52d600640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52815e110 0x7ff52814ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.242+0000 7ff52d600640 1 --2- 172.21.15.83:0/2610990568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff52806fdc0 0x7ff528158810 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.242+0000 7ff52d600640 1 -- 172.21.15.83:0/2610990568 >> 172.21.15.83:0/2610990568 conn(0x7ff5280d1bc0 msgr2=0x7ff52806d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:23.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.242+0000 7ff52d600640 1 -- 172.21.15.83:0/2610990568 shutdown_connections 2024-04-25T00:12:23.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.242+0000 7ff52d600640 1 -- 172.21.15.83:0/2610990568 wait complete. 2024-04-25T00:12:23.262 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-04-25T00:12:22.459968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220584,"kb_used_data":4392,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754136,"statfs":{"total":773094113280,"available":772868235264,"internally_reserved":0,"allocated":4497408,"data_stored":2590152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.298861"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.204387+0000","last_change":"2024-04-25T00:12:19.198325+0000","last_active":"2024-04-25T00:12:19.204387+0000","last_peered":"2024-04-25T00:12:19.204387+0000","last_clean":"2024-04-25T00:12:19.204387+0000","last_became_active":"2024-04-25T00:12:19.198064+0000","last_became_peered":"2024-04-25T00:12:19.198064+0000","last_unstale":"2024-04-25T00:12:19.204387+0000","last_undegraded":"2024-04-25T00:12:19.204387+0000","last_fullsized":"2024-04-25T00:12:19.204387+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:30.398208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.205023+0000","last_change":"2024-04-25T00:12:19.198635+0000","last_active":"2024-04-25T00:12:19.205023+0000","last_peered":"2024-04-25T00:12:19.205023+0000","last_clean":"2024-04-25T00:12:19.205023+0000","last_became_active":"2024-04-25T00:12:19.198374+0000","last_became_peered":"2024-04-25T00:12:19.198374+0000","last_unstale":"2024-04-25T00:12:19.205023+0000","last_undegraded":"2024-04-25T00:12:19.205023+0000","last_fullsized":"2024-04-25T00:12:19.205023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:41.297808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.217606+0000","last_change":"2024-04-25T00:12:19.216865+0000","last_active":"2024-04-25T00:12:19.217606+0000","last_peered":"2024-04-25T00:12:19.217606+0000","last_clean":"2024-04-25T00:12:19.217606+0000","last_became_active":"2024-04-25T00:12:19.216557+0000","last_became_peered":"2024-04-25T00:12:19.216557+0000","last_unstale":"2024-04-25T00:12:19.217606+0000","last_undegraded":"2024-04-25T00:12:19.217606+0000","last_fullsized":"2024-04-25T00:12:19.217606+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:30.684139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.209148+0000","last_change":"2024-04-25T00:12:19.204017+0000","last_active":"2024-04-25T00:12:19.209148+0000","last_peered":"2024-04-25T00:12:19.209148+0000","last_clean":"2024-04-25T00:12:19.209148+0000","last_became_active":"2024-04-25T00:12:19.203770+0000","last_became_peered":"2024-04-25T00:12:19.203770+0000","last_unstale":"2024-04-25T00:12:19.209148+0000","last_undegraded":"2024-04-25T00:12:19.209148+0000","last_fullsized":"2024-04-25T00:12:19.209148+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:41.771920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.203624+0000","last_change":"2024-04-25T00:12:19.193889+0000","last_active":"2024-04-25T00:12:19.203624+0000","last_peered":"2024-04-25T00:12:19.203624+0000","last_clean":"2024-04-25T00:12:19.203624+0000","last_became_active":"2024-04-25T00:12:19.193690+0000","last_became_peered":"2024-04-25T00:12:19.193690+0000","last_unstale":"2024-04-25T00:12:19.203624+0000","last_undegraded":"2024-04-25T00:12:19.203624+0000","last_fullsized":"2024-04-25T00:12:19.203624+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:31.009303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.202778+0000","last_change":"2024-04-25T00:12:19.200943+0000","last_active":"2024-04-25T00:12:19.202778+0000","last_peered":"2024-04-25T00:12:19.202778+0000","last_clean":"2024-04-25T00:12:19.202778+0000","last_became_active":"2024-04-25T00:12:19.200683+0000","last_became_peered":"2024-04-25T00:12:19.200683+0000","last_unstale":"2024-04-25T00:12:19.202778+0000","last_undegraded":"2024-04-25T00:12:19.202778+0000","last_fullsized":"2024-04-25T00:12:19.202778+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:30.684139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.210170+0000","last_change":"2024-04-25T00:12:19.205772+0000","last_active":"2024-04-25T00:12:19.210170+0000","last_peered":"2024-04-25T00:12:19.210170+0000","last_clean":"2024-04-25T00:12:19.210170+0000","last_became_active":"2024-04-25T00:12:19.205315+0000","last_became_peered":"2024-04-25T00:12:19.205315+0000","last_unstale":"2024-04-25T00:12:19.210170+0000","last_undegraded":"2024-04-25T00:12:19.210170+0000","last_fullsized":"2024-04-25T00:12:19.210170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:17.550259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.209386+0000","last_change":"2024-04-25T00:12:19.206007+0000","last_active":"2024-04-25T00:12:19.209386+0000","last_peered":"2024-04-25T00:12:19.209386+0000","last_clean":"2024-04-25T00:12:19.209386+0000","last_became_active":"2024-04-25T00:12:19.205684+0000","last_became_peered":"2024-04-25T00:12:19.205684+0000","last_unstale":"2024-04-25T00:12:19.209386+0000","last_undegraded":"2024-04-25T00:12:19.209386+0000","last_fullsized":"2024-04-25T00:12:19.209386+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:01.841718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.211981+0000","last_change":"2024-04-25T00:12:19.206482+0000","last_active":"2024-04-25T00:12:19.211981+0000","last_peered":"2024-04-25T00:12:19.211981+0000","last_clean":"2024-04-25T00:12:19.211981+0000","last_became_active":"2024-04-25T00:12:19.206203+0000","last_became_peered":"2024-04-25T00:12:19.206203+0000","last_unstale":"2024-04-25T00:12:19.211981+0000","last_undegraded":"2024-04-25T00:12:19.211981+0000","last_fullsized":"2024-04-25T00:12:19.211981+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:54.046773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.220065+0000","last_change":"2024-04-25T00:12:19.216916+0000","last_active":"2024-04-25T00:12:19.220065+0000","last_peered":"2024-04-25T00:12:19.220065+0000","last_clean":"2024-04-25T00:12:19.220065+0000","last_became_active":"2024-04-25T00:12:19.216608+0000","last_became_peered":"2024-04-25T00:12:19.216608+0000","last_unstale":"2024-04-25T00:12:19.220065+0000","last_undegraded":"2024-04-25T00:12:19.220065+0000","last_fullsized":"2024-04-25T00:12:19.220065+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:59.322277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.219955+0000","last_change":"2024-04-25T00:12:19.214605+0000","last_active":"2024-04-25T00:12:19.219955+0000","last_peered":"2024-04-25T00:12:19.219955+0000","last_clean":"2024-04-25T00:12:19.219955+0000","last_became_active":"2024-04-25T00:12:19.214295+0000","last_became_peered":"2024-04-25T00:12:19.214295+0000","last_unstale":"2024-04-25T00:12:19.219955+0000","last_undegraded":"2024-04-25T00:12:19.219955+0000","last_fullsized":"2024-04-25T00:12:19.219955+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:30.398208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.200733+0000","last_change":"2024-04-25T00:12:19.195321+0000","last_active":"2024-04-25T00:12:19.200733+0000","last_peered":"2024-04-25T00:12:19.200733+0000","last_clean":"2024-04-25T00:12:19.200733+0000","last_became_active":"2024-04-25T00:12:19.195023+0000","last_became_peered":"2024-04-25T00:12:19.195023+0000","last_unstale":"2024-04-25T00:12:19.200733+0000","last_undegraded":"2024-04-25T00:12:19.200733+0000","last_fullsized":"2024-04-25T00:12:19.200733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:12.177125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.206284+0000","last_change":"2024-04-25T00:12:19.201761+0000","last_active":"2024-04-25T00:12:19.206284+0000","last_peered":"2024-04-25T00:12:19.206284+0000","last_clean":"2024-04-25T00:12:19.206284+0000","last_became_active":"2024-04-25T00:12:19.201481+0000","last_became_peered":"2024-04-25T00:12:19.201481+0000","last_unstale":"2024-04-25T00:12:19.206284+0000","last_undegraded":"2024-04-25T00:12:19.206284+0000","last_fullsized":"2024-04-25T00:12:19.206284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:30.541480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.207720+0000","last_change":"2024-04-25T00:12:19.199884+0000","last_active":"2024-04-25T00:12:19.207720+0000","last_peered":"2024-04-25T00:12:19.207720+0000","last_clean":"2024-04-25T00:12:19.207720+0000","last_became_active":"2024-04-25T00:12:19.199418+0000","last_became_peered":"2024-04-25T00:12:19.199418+0000","last_unstale":"2024-04-25T00:12:19.207720+0000","last_undegraded":"2024-04-25T00:12:19.207720+0000","last_fullsized":"2024-04-25T00:12:19.207720+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:45.148333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.210152+0000","last_change":"2024-04-25T00:12:19.205614+0000","last_active":"2024-04-25T00:12:19.210152+0000","last_peered":"2024-04-25T00:12:19.210152+0000","last_clean":"2024-04-25T00:12:19.210152+0000","last_became_active":"2024-04-25T00:12:19.205168+0000","last_became_peered":"2024-04-25T00:12:19.205168+0000","last_unstale":"2024-04-25T00:12:19.210152+0000","last_undegraded":"2024-04-25T00:12:19.210152+0000","last_fullsized":"2024-04-25T00:12:19.210152+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:52.162568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.215625+0000","last_change":"2024-04-25T00:12:19.211831+0000","last_active":"2024-04-25T00:12:19.215625+0000","last_peered":"2024-04-25T00:12:19.215625+0000","last_clean":"2024-04-25T00:12:19.215625+0000","last_became_active":"2024-04-25T00:12:19.211563+0000","last_became_peered":"2024-04-25T00:12:19.211563+0000","last_unstale":"2024-04-25T00:12:19.215625+0000","last_undegraded":"2024-04-25T00:12:19.215625+0000","last_fullsized":"2024-04-25T00:12:19.215625+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:52.162568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.214784+0000","last_change":"2024-04-25T00:12:19.209242+0000","last_active":"2024-04-25T00:12:19.214784+0000","last_peered":"2024-04-25T00:12:19.214784+0000","last_clean":"2024-04-25T00:12:19.214784+0000","last_became_active":"2024-04-25T00:12:19.208992+0000","last_became_peered":"2024-04-25T00:12:19.208992+0000","last_unstale":"2024-04-25T00:12:19.214784+0000","last_undegraded":"2024-04-25T00:12:19.214784+0000","last_fullsized":"2024-04-25T00:12:19.214784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:59.846695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.207002+0000","last_change":"2024-04-25T00:12:19.204643+0000","last_active":"2024-04-25T00:12:19.207002+0000","last_peered":"2024-04-25T00:12:19.207002+0000","last_clean":"2024-04-25T00:12:19.207002+0000","last_became_active":"2024-04-25T00:12:19.204373+0000","last_became_peered":"2024-04-25T00:12:19.204373+0000","last_unstale":"2024-04-25T00:12:19.207002+0000","last_undegraded":"2024-04-25T00:12:19.207002+0000","last_fullsized":"2024-04-25T00:12:19.207002+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:54.317722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.202767+0000","last_change":"2024-04-25T00:12:19.197380+0000","last_active":"2024-04-25T00:12:19.202767+0000","last_peered":"2024-04-25T00:12:19.202767+0000","last_clean":"2024-04-25T00:12:19.202767+0000","last_became_active":"2024-04-25T00:12:19.195729+0000","last_became_peered":"2024-04-25T00:12:19.195729+0000","last_unstale":"2024-04-25T00:12:19.202767+0000","last_undegraded":"2024-04-25T00:12:19.202767+0000","last_fullsized":"2024-04-25T00:12:19.202767+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:04.436136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.197799+0000","last_change":"2024-04-25T00:12:19.193388+0000","last_active":"2024-04-25T00:12:19.197799+0000","last_peered":"2024-04-25T00:12:19.197799+0000","last_clean":"2024-04-25T00:12:19.197799+0000","last_became_active":"2024-04-25T00:12:19.193178+0000","last_became_peered":"2024-04-25T00:12:19.193178+0000","last_unstale":"2024-04-25T00:12:19.197799+0000","last_undegraded":"2024-04-25T00:12:19.197799+0000","last_fullsized":"2024-04-25T00:12:19.197799+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:12.177125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.215469+0000","last_change":"2024-04-25T00:12:19.208286+0000","last_active":"2024-04-25T00:12:19.215469+0000","last_peered":"2024-04-25T00:12:19.215469+0000","last_clean":"2024-04-25T00:12:19.215469+0000","last_became_active":"2024-04-25T00:12:19.206423+0000","last_became_peered":"2024-04-25T00:12:19.206423+0000","last_unstale":"2024-04-25T00:12:19.215469+0000","last_undegraded":"2024-04-25T00:12:19.215469+0000","last_fullsized":"2024-04-25T00:12:19.215469+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:37:13.047475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.204037+0000","last_change":"2024-04-25T00:12:19.196257+0000","last_active":"2024-04-25T00:12:19.204037+0000","last_peered":"2024-04-25T00:12:19.204037+0000","last_clean":"2024-04-25T00:12:19.204037+0000","last_became_active":"2024-04-25T00:12:19.195953+0000","last_became_peered":"2024-04-25T00:12:19.195953+0000","last_unstale":"2024-04-25T00:12:19.204037+0000","last_undegraded":"2024-04-25T00:12:19.204037+0000","last_fullsized":"2024-04-25T00:12:19.204037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:39.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.201493+0000","last_change":"2024-04-25T00:12:19.192634+0000","last_active":"2024-04-25T00:12:19.201493+0000","last_peered":"2024-04-25T00:12:19.201493+0000","last_clean":"2024-04-25T00:12:19.201493+0000","last_became_active":"2024-04-25T00:12:19.192136+0000","last_became_peered":"2024-04-25T00:12:19.192136+0000","last_unstale":"2024-04-25T00:12:19.201493+0000","last_undegraded":"2024-04-25T00:12:19.201493+0000","last_fullsized":"2024-04-25T00:12:19.201493+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:55.625707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.203247+0000","last_change":"2024-04-25T00:12:19.201203+0000","last_active":"2024-04-25T00:12:19.203247+0000","last_peered":"2024-04-25T00:12:19.203247+0000","last_clean":"2024-04-25T00:12:19.203247+0000","last_became_active":"2024-04-25T00:12:19.200946+0000","last_became_peered":"2024-04-25T00:12:19.200946+0000","last_unstale":"2024-04-25T00:12:19.203247+0000","last_undegraded":"2024-04-25T00:12:19.203247+0000","last_fullsized":"2024-04-25T00:12:19.203247+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:08.524271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.197600+0000","last_change":"2024-04-25T00:12:19.193232+0000","last_active":"2024-04-25T00:12:19.197600+0000","last_peered":"2024-04-25T00:12:19.197600+0000","last_clean":"2024-04-25T00:12:19.197600+0000","last_became_active":"2024-04-25T00:12:19.192904+0000","last_became_peered":"2024-04-25T00:12:19.192904+0000","last_unstale":"2024-04-25T00:12:19.197600+0000","last_undegraded":"2024-04-25T00:12:19.197600+0000","last_fullsized":"2024-04-25T00:12:19.197600+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:39.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.215568+0000","last_change":"2024-04-25T00:12:19.214294+0000","last_active":"2024-04-25T00:12:19.215568+0000","last_peered":"2024-04-25T00:12:19.215568+0000","last_clean":"2024-04-25T00:12:19.215568+0000","last_became_active":"2024-04-25T00:12:19.214081+0000","last_became_peered":"2024-04-25T00:12:19.214081+0000","last_unstale":"2024-04-25T00:12:19.215568+0000","last_undegraded":"2024-04-25T00:12:19.215568+0000","last_fullsized":"2024-04-25T00:12:19.215568+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:12.784794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.187756+0000","last_change":"2024-04-25T00:12:19.181736+0000","last_active":"2024-04-25T00:12:19.187756+0000","last_peered":"2024-04-25T00:12:19.187756+0000","last_clean":"2024-04-25T00:12:19.187756+0000","last_became_active":"2024-04-25T00:12:19.181446+0000","last_became_peered":"2024-04-25T00:12:19.181446+0000","last_unstale":"2024-04-25T00:12:19.187756+0000","last_undegraded":"2024-04-25T00:12:19.187756+0000","last_fullsized":"2024-04-25T00:12:19.187756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:47.187867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.210239+0000","last_change":"2024-04-25T00:12:19.203769+0000","last_active":"2024-04-25T00:12:19.210239+0000","last_peered":"2024-04-25T00:12:19.210239+0000","last_clean":"2024-04-25T00:12:19.210239+0000","last_became_active":"2024-04-25T00:12:19.203324+0000","last_became_peered":"2024-04-25T00:12:19.203324+0000","last_unstale":"2024-04-25T00:12:19.210239+0000","last_undegraded":"2024-04-25T00:12:19.210239+0000","last_fullsized":"2024-04-25T00:12:19.210239+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:15.513443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.205431+0000","last_change":"2024-04-25T00:12:19.196097+0000","last_active":"2024-04-25T00:12:19.205431+0000","last_peered":"2024-04-25T00:12:19.205431+0000","last_clean":"2024-04-25T00:12:19.205431+0000","last_became_active":"2024-04-25T00:12:19.195829+0000","last_became_peered":"2024-04-25T00:12:19.195829+0000","last_unstale":"2024-04-25T00:12:19.205431+0000","last_undegraded":"2024-04-25T00:12:19.205431+0000","last_fullsized":"2024-04-25T00:12:19.205431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:45.148333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.206783+0000","last_change":"2024-04-25T00:12:19.199560+0000","last_active":"2024-04-25T00:12:19.206783+0000","last_peered":"2024-04-25T00:12:19.206783+0000","last_clean":"2024-04-25T00:12:19.206783+0000","last_became_active":"2024-04-25T00:12:19.199247+0000","last_became_peered":"2024-04-25T00:12:19.199247+0000","last_unstale":"2024-04-25T00:12:19.206783+0000","last_undegraded":"2024-04-25T00:12:19.206783+0000","last_fullsized":"2024-04-25T00:12:19.206783+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:42.766192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.208319+0000","last_change":"2024-04-25T00:12:19.197092+0000","last_active":"2024-04-25T00:12:19.208319+0000","last_peered":"2024-04-25T00:12:19.208319+0000","last_clean":"2024-04-25T00:12:19.208319+0000","last_became_active":"2024-04-25T00:12:19.196828+0000","last_became_peered":"2024-04-25T00:12:19.196828+0000","last_unstale":"2024-04-25T00:12:19.208319+0000","last_undegraded":"2024-04-25T00:12:19.208319+0000","last_fullsized":"2024-04-25T00:12:19.208319+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:30.541480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.199697+0000","last_change":"2024-04-25T00:12:19.192306+0000","last_active":"2024-04-25T00:12:19.199697+0000","last_peered":"2024-04-25T00:12:19.199697+0000","last_clean":"2024-04-25T00:12:19.199697+0000","last_became_active":"2024-04-25T00:12:19.192018+0000","last_became_peered":"2024-04-25T00:12:19.192018+0000","last_unstale":"2024-04-25T00:12:19.199697+0000","last_undegraded":"2024-04-25T00:12:19.199697+0000","last_fullsized":"2024-04-25T00:12:19.199697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:54.046773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.215786+0000","last_change":"2024-04-25T00:12:19.210776+0000","last_active":"2024-04-25T00:12:19.215786+0000","last_peered":"2024-04-25T00:12:19.215786+0000","last_clean":"2024-04-25T00:12:19.215786+0000","last_became_active":"2024-04-25T00:12:19.210489+0000","last_became_peered":"2024-04-25T00:12:19.210489+0000","last_unstale":"2024-04-25T00:12:19.215786+0000","last_undegraded":"2024-04-25T00:12:19.215786+0000","last_fullsized":"2024-04-25T00:12:19.215786+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:32.234020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.211602+0000","last_change":"2024-04-25T00:12:18.195226+0000","last_active":"2024-04-25T00:12:19.211602+0000","last_peered":"2024-04-25T00:12:19.211602+0000","last_clean":"2024-04-25T00:12:19.211602+0000","last_became_active":"2024-04-25T00:12:18.194964+0000","last_became_peered":"2024-04-25T00:12:18.194964+0000","last_unstale":"2024-04-25T00:12:19.211602+0000","last_undegraded":"2024-04-25T00:12:19.211602+0000","last_fullsized":"2024-04-25T00:12:19.211602+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:59.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.218209+0000","last_change":"2024-04-25T00:12:19.216346+0000","last_active":"2024-04-25T00:12:19.218209+0000","last_peered":"2024-04-25T00:12:19.218209+0000","last_clean":"2024-04-25T00:12:19.218209+0000","last_became_active":"2024-04-25T00:12:19.216086+0000","last_became_peered":"2024-04-25T00:12:19.216086+0000","last_unstale":"2024-04-25T00:12:19.218209+0000","last_undegraded":"2024-04-25T00:12:19.218209+0000","last_fullsized":"2024-04-25T00:12:19.218209+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:30.541480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.216349+0000","last_change":"2024-04-25T00:12:18.199602+0000","last_active":"2024-04-25T00:12:19.216349+0000","last_peered":"2024-04-25T00:12:19.216349+0000","last_clean":"2024-04-25T00:12:19.216349+0000","last_became_active":"2024-04-25T00:12:18.197254+0000","last_became_peered":"2024-04-25T00:12:18.197254+0000","last_unstale":"2024-04-25T00:12:19.216349+0000","last_undegraded":"2024-04-25T00:12:19.216349+0000","last_fullsized":"2024-04-25T00:12:19.216349+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:44.748128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.209584+0000","last_change":"2024-04-25T00:12:19.197752+0000","last_active":"2024-04-25T00:12:19.209584+0000","last_peered":"2024-04-25T00:12:19.209584+0000","last_clean":"2024-04-25T00:12:19.209584+0000","last_became_active":"2024-04-25T00:12:19.197336+0000","last_became_peered":"2024-04-25T00:12:19.197336+0000","last_unstale":"2024-04-25T00:12:19.209584+0000","last_undegraded":"2024-04-25T00:12:19.209584+0000","last_fullsized":"2024-04-25T00:12:19.209584+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:12.177125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.215658+0000","last_change":"2024-04-25T00:12:18.194323+0000","last_active":"2024-04-25T00:12:19.215658+0000","last_peered":"2024-04-25T00:12:19.215658+0000","last_clean":"2024-04-25T00:12:19.215658+0000","last_became_active":"2024-04-25T00:12:18.193587+0000","last_became_peered":"2024-04-25T00:12:18.193587+0000","last_unstale":"2024-04-25T00:12:19.215658+0000","last_undegraded":"2024-04-25T00:12:19.215658+0000","last_fullsized":"2024-04-25T00:12:19.215658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:53.311425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.199995+0000","last_change":"2024-04-25T00:12:19.186797+0000","last_active":"2024-04-25T00:12:19.199995+0000","last_peered":"2024-04-25T00:12:19.199995+0000","last_clean":"2024-04-25T00:12:19.199995+0000","last_became_active":"2024-04-25T00:12:19.186615+0000","last_became_peered":"2024-04-25T00:12:19.186615+0000","last_unstale":"2024-04-25T00:12:19.199995+0000","last_undegraded":"2024-04-25T00:12:19.199995+0000","last_fullsized":"2024-04-25T00:12:19.199995+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:17.550259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.209232+0000","last_change":"2024-04-25T00:12:18.197408+0000","last_active":"2024-04-25T00:12:19.209232+0000","last_peered":"2024-04-25T00:12:19.209232+0000","last_clean":"2024-04-25T00:12:19.209232+0000","last_became_active":"2024-04-25T00:12:18.197140+0000","last_became_peered":"2024-04-25T00:12:18.197140+0000","last_unstale":"2024-04-25T00:12:19.209232+0000","last_undegraded":"2024-04-25T00:12:19.209232+0000","last_fullsized":"2024-04-25T00:12:19.209232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:44.748128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.200653+0000","last_change":"2024-04-25T00:12:19.194230+0000","last_active":"2024-04-25T00:12:19.200653+0000","last_peered":"2024-04-25T00:12:19.200653+0000","last_clean":"2024-04-25T00:12:19.200653+0000","last_became_active":"2024-04-25T00:12:19.194014+0000","last_became_peered":"2024-04-25T00:12:19.194014+0000","last_unstale":"2024-04-25T00:12:19.200653+0000","last_undegraded":"2024-04-25T00:12:19.200653+0000","last_fullsized":"2024-04-25T00:12:19.200653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:39.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.214347+0000","last_change":"2024-04-25T00:12:18.199830+0000","last_active":"2024-04-25T00:12:19.214347+0000","last_peered":"2024-04-25T00:12:19.214347+0000","last_clean":"2024-04-25T00:12:19.214347+0000","last_became_active":"2024-04-25T00:12:18.196815+0000","last_became_peered":"2024-04-25T00:12:18.196815+0000","last_unstale":"2024-04-25T00:12:19.214347+0000","last_undegraded":"2024-04-25T00:12:19.214347+0000","last_fullsized":"2024-04-25T00:12:19.214347+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:53.040476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.210127+0000","last_change":"2024-04-25T00:12:19.205500+0000","last_active":"2024-04-25T00:12:19.210127+0000","last_peered":"2024-04-25T00:12:19.210127+0000","last_clean":"2024-04-25T00:12:19.210127+0000","last_became_active":"2024-04-25T00:12:19.204988+0000","last_became_peered":"2024-04-25T00:12:19.204988+0000","last_unstale":"2024-04-25T00:12:19.210127+0000","last_undegraded":"2024-04-25T00:12:19.210127+0000","last_fullsized":"2024-04-25T00:12:19.210127+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:32.234020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.218749+0000","last_change":"2024-04-25T00:12:18.197636+0000","last_active":"2024-04-25T00:12:19.218749+0000","last_peered":"2024-04-25T00:12:19.218749+0000","last_clean":"2024-04-25T00:12:19.218749+0000","last_became_active":"2024-04-25T00:12:18.197386+0000","last_became_peered":"2024-04-25T00:12:18.197386+0000","last_unstale":"2024-04-25T00:12:19.218749+0000","last_undegraded":"2024-04-25T00:12:19.218749+0000","last_fullsized":"2024-04-25T00:12:19.218749+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:44.142036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.216059+0000","last_change":"2024-04-25T00:12:19.212366+0000","last_active":"2024-04-25T00:12:19.216059+0000","last_peered":"2024-04-25T00:12:19.216059+0000","last_clean":"2024-04-25T00:12:19.216059+0000","last_became_active":"2024-04-25T00:12:19.212101+0000","last_became_peered":"2024-04-25T00:12:19.212101+0000","last_unstale":"2024-04-25T00:12:19.216059+0000","last_undegraded":"2024-04-25T00:12:19.216059+0000","last_fullsized":"2024-04-25T00:12:19.216059+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:55.625707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.182436+0000","last_change":"2024-04-25T00:12:18.196175+0000","last_active":"2024-04-25T00:12:19.182436+0000","last_peered":"2024-04-25T00:12:19.182436+0000","last_clean":"2024-04-25T00:12:19.182436+0000","last_became_active":"2024-04-25T00:12:18.195845+0000","last_became_peered":"2024-04-25T00:12:18.195845+0000","last_unstale":"2024-04-25T00:12:19.182436+0000","last_undegraded":"2024-04-25T00:12:19.182436+0000","last_fullsized":"2024-04-25T00:12:19.182436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:41.759894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.205957+0000","last_change":"2024-04-25T00:12:19.199629+0000","last_active":"2024-04-25T00:12:19.205957+0000","last_peered":"2024-04-25T00:12:19.205957+0000","last_clean":"2024-04-25T00:12:19.205957+0000","last_became_active":"2024-04-25T00:12:19.199394+0000","last_became_peered":"2024-04-25T00:12:19.199394+0000","last_unstale":"2024-04-25T00:12:19.205957+0000","last_undegraded":"2024-04-25T00:12:19.205957+0000","last_fullsized":"2024-04-25T00:12:19.205957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:54.297255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.187567+0000","last_change":"2024-04-25T00:12:18.195857+0000","last_active":"2024-04-25T00:12:19.187567+0000","last_peered":"2024-04-25T00:12:19.187567+0000","last_clean":"2024-04-25T00:12:19.187567+0000","last_became_active":"2024-04-25T00:12:18.194730+0000","last_became_peered":"2024-04-25T00:12:18.194730+0000","last_unstale":"2024-04-25T00:12:19.187567+0000","last_undegraded":"2024-04-25T00:12:19.187567+0000","last_fullsized":"2024-04-25T00:12:19.187567+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:15.505830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.206778+0000","last_change":"2024-04-25T00:12:19.203206+0000","last_active":"2024-04-25T00:12:19.206778+0000","last_peered":"2024-04-25T00:12:19.206778+0000","last_clean":"2024-04-25T00:12:19.206778+0000","last_became_active":"2024-04-25T00:12:19.201288+0000","last_became_peered":"2024-04-25T00:12:19.201288+0000","last_unstale":"2024-04-25T00:12:19.206778+0000","last_undegraded":"2024-04-25T00:12:19.206778+0000","last_fullsized":"2024-04-25T00:12:19.206778+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:42.060401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.179910+0000","last_change":"2024-04-25T00:12:18.192334+0000","last_active":"2024-04-25T00:12:19.179910+0000","last_peered":"2024-04-25T00:12:19.179910+0000","last_clean":"2024-04-25T00:12:19.179910+0000","last_became_active":"2024-04-25T00:12:18.192051+0000","last_became_peered":"2024-04-25T00:12:18.192051+0000","last_unstale":"2024-04-25T00:12:19.179910+0000","last_undegraded":"2024-04-25T00:12:19.179910+0000","last_fullsized":"2024-04-25T00:12:19.179910+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:31.227722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.208746+0000","last_change":"2024-04-25T00:12:19.197804+0000","last_active":"2024-04-25T00:12:19.208746+0000","last_peered":"2024-04-25T00:12:19.208746+0000","last_clean":"2024-04-25T00:12:19.208746+0000","last_became_active":"2024-04-25T00:12:19.196950+0000","last_became_peered":"2024-04-25T00:12:19.196950+0000","last_unstale":"2024-04-25T00:12:19.208746+0000","last_undegraded":"2024-04-25T00:12:19.208746+0000","last_fullsized":"2024-04-25T00:12:19.208746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:26.110172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.190601+0000","last_change":"2024-04-25T00:12:18.199725+0000","last_active":"2024-04-25T00:12:19.190601+0000","last_peered":"2024-04-25T00:12:19.190601+0000","last_clean":"2024-04-25T00:12:19.190601+0000","last_became_active":"2024-04-25T00:12:18.196685+0000","last_became_peered":"2024-04-25T00:12:18.196685+0000","last_unstale":"2024-04-25T00:12:19.190601+0000","last_undegraded":"2024-04-25T00:12:19.190601+0000","last_fullsized":"2024-04-25T00:12:19.190601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:50.829524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.204957+0000","last_change":"2024-04-25T00:12:19.199143+0000","last_active":"2024-04-25T00:12:19.204957+0000","last_peered":"2024-04-25T00:12:19.204957+0000","last_clean":"2024-04-25T00:12:19.204957+0000","last_became_active":"2024-04-25T00:12:19.198850+0000","last_became_peered":"2024-04-25T00:12:19.198850+0000","last_unstale":"2024-04-25T00:12:19.204957+0000","last_undegraded":"2024-04-25T00:12:19.204957+0000","last_fullsized":"2024-04-25T00:12:19.204957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:45.148333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.220007+0000","last_change":"2024-04-25T00:12:18.197529+0000","last_active":"2024-04-25T00:12:19.220007+0000","last_peered":"2024-04-25T00:12:19.220007+0000","last_clean":"2024-04-25T00:12:19.220007+0000","last_became_active":"2024-04-25T00:12:18.197265+0000","last_became_peered":"2024-04-25T00:12:18.197265+0000","last_unstale":"2024-04-25T00:12:19.220007+0000","last_undegraded":"2024-04-25T00:12:19.220007+0000","last_fullsized":"2024-04-25T00:12:19.220007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:11.778497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.211043+0000","last_change":"2024-04-25T00:12:19.206496+0000","last_active":"2024-04-25T00:12:19.211043+0000","last_peered":"2024-04-25T00:12:19.211043+0000","last_clean":"2024-04-25T00:12:19.211043+0000","last_became_active":"2024-04-25T00:12:19.206233+0000","last_became_peered":"2024-04-25T00:12:19.206233+0000","last_unstale":"2024-04-25T00:12:19.211043+0000","last_undegraded":"2024-04-25T00:12:19.211043+0000","last_fullsized":"2024-04-25T00:12:19.211043+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:42.766192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.222844+0000","last_change":"2024-04-25T00:12:18.183408+0000","last_active":"2024-04-25T00:12:19.222844+0000","last_peered":"2024-04-25T00:12:19.222844+0000","last_clean":"2024-04-25T00:12:19.222844+0000","last_became_active":"2024-04-25T00:12:18.183090+0000","last_became_peered":"2024-04-25T00:12:18.183090+0000","last_unstale":"2024-04-25T00:12:19.222844+0000","last_undegraded":"2024-04-25T00:12:19.222844+0000","last_fullsized":"2024-04-25T00:12:19.222844+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:58.840397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.211965+0000","last_change":"2024-04-25T00:12:19.207118+0000","last_active":"2024-04-25T00:12:19.211965+0000","last_peered":"2024-04-25T00:12:19.211965+0000","last_clean":"2024-04-25T00:12:19.211965+0000","last_became_active":"2024-04-25T00:12:19.206483+0000","last_became_peered":"2024-04-25T00:12:19.206483+0000","last_unstale":"2024-04-25T00:12:19.211965+0000","last_undegraded":"2024-04-25T00:12:19.211965+0000","last_fullsized":"2024-04-25T00:12:19.211965+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:55.625707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.213299+0000","last_change":"2024-04-25T00:12:18.194759+0000","last_active":"2024-04-25T00:12:19.213299+0000","last_peered":"2024-04-25T00:12:19.213299+0000","last_clean":"2024-04-25T00:12:19.213299+0000","last_became_active":"2024-04-25T00:12:18.194483+0000","last_became_peered":"2024-04-25T00:12:18.194483+0000","last_unstale":"2024-04-25T00:12:19.213299+0000","last_undegraded":"2024-04-25T00:12:19.213299+0000","last_fullsized":"2024-04-25T00:12:19.213299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:51.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.204992+0000","last_change":"2024-04-25T00:12:19.198615+0000","last_active":"2024-04-25T00:12:19.204992+0000","last_peered":"2024-04-25T00:12:19.204992+0000","last_clean":"2024-04-25T00:12:19.204992+0000","last_became_active":"2024-04-25T00:12:19.198052+0000","last_became_peered":"2024-04-25T00:12:19.198052+0000","last_unstale":"2024-04-25T00:12:19.204992+0000","last_undegraded":"2024-04-25T00:12:19.204992+0000","last_fullsized":"2024-04-25T00:12:19.204992+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:59.322277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.181867+0000","last_change":"2024-04-25T00:12:18.200632+0000","last_active":"2024-04-25T00:12:19.181867+0000","last_peered":"2024-04-25T00:12:19.181867+0000","last_clean":"2024-04-25T00:12:19.181867+0000","last_became_active":"2024-04-25T00:12:18.200339+0000","last_became_peered":"2024-04-25T00:12:18.200339+0000","last_unstale":"2024-04-25T00:12:19.181867+0000","last_undegraded":"2024-04-25T00:12:19.181867+0000","last_fullsized":"2024-04-25T00:12:19.181867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:11.170827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.207239+0000","last_change":"2024-04-25T00:12:19.195117+0000","last_active":"2024-04-25T00:12:19.207239+0000","last_peered":"2024-04-25T00:12:19.207239+0000","last_clean":"2024-04-25T00:12:19.207239+0000","last_became_active":"2024-04-25T00:12:19.194759+0000","last_became_peered":"2024-04-25T00:12:19.194759+0000","last_unstale":"2024-04-25T00:12:19.207239+0000","last_undegraded":"2024-04-25T00:12:19.207239+0000","last_fullsized":"2024-04-25T00:12:19.207239+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:30.684139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.176894+0000","last_change":"2024-04-25T00:12:18.196345+0000","last_active":"2024-04-25T00:12:19.176894+0000","last_peered":"2024-04-25T00:12:19.176894+0000","last_clean":"2024-04-25T00:12:19.176894+0000","last_became_active":"2024-04-25T00:12:18.196026+0000","last_became_peered":"2024-04-25T00:12:18.196026+0000","last_unstale":"2024-04-25T00:12:19.176894+0000","last_undegraded":"2024-04-25T00:12:19.176894+0000","last_fullsized":"2024-04-25T00:12:19.176894+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:44.748128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.204440+0000","last_change":"2024-04-25T00:12:19.198292+0000","last_active":"2024-04-25T00:12:19.204440+0000","last_peered":"2024-04-25T00:12:19.204440+0000","last_clean":"2024-04-25T00:12:19.204440+0000","last_became_active":"2024-04-25T00:12:19.197980+0000","last_became_peered":"2024-04-25T00:12:19.197980+0000","last_unstale":"2024-04-25T00:12:19.204440+0000","last_undegraded":"2024-04-25T00:12:19.204440+0000","last_fullsized":"2024-04-25T00:12:19.204440+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:50.934891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.220633+0000","last_change":"2024-04-25T00:12:18.195596+0000","last_active":"2024-04-25T00:12:19.220633+0000","last_peered":"2024-04-25T00:12:19.220633+0000","last_clean":"2024-04-25T00:12:19.220633+0000","last_became_active":"2024-04-25T00:12:18.194339+0000","last_became_peered":"2024-04-25T00:12:18.194339+0000","last_unstale":"2024-04-25T00:12:19.220633+0000","last_undegraded":"2024-04-25T00:12:19.220633+0000","last_fullsized":"2024-04-25T00:12:19.220633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:53.040476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.200712+0000","last_change":"2024-04-25T00:12:19.187654+0000","last_active":"2024-04-25T00:12:19.200712+0000","last_peered":"2024-04-25T00:12:19.200712+0000","last_clean":"2024-04-25T00:12:19.200712+0000","last_became_active":"2024-04-25T00:12:19.187432+0000","last_became_peered":"2024-04-25T00:12:19.187432+0000","last_unstale":"2024-04-25T00:12:19.200712+0000","last_undegraded":"2024-04-25T00:12:19.200712+0000","last_fullsized":"2024-04-25T00:12:19.200712+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:37:13.047475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.219455+0000","last_change":"2024-04-25T00:12:18.200832+0000","last_active":"2024-04-25T00:12:19.219455+0000","last_peered":"2024-04-25T00:12:19.219455+0000","last_clean":"2024-04-25T00:12:19.219455+0000","last_became_active":"2024-04-25T00:12:18.200533+0000","last_became_peered":"2024-04-25T00:12:18.200533+0000","last_unstale":"2024-04-25T00:12:19.219455+0000","last_undegraded":"2024-04-25T00:12:19.219455+0000","last_fullsized":"2024-04-25T00:12:19.219455+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:54.619409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.215241+0000","last_change":"2024-04-25T00:12:19.203611+0000","last_active":"2024-04-25T00:12:19.215241+0000","last_peered":"2024-04-25T00:12:19.215241+0000","last_clean":"2024-04-25T00:12:19.215241+0000","last_became_active":"2024-04-25T00:12:19.203283+0000","last_became_peered":"2024-04-25T00:12:19.203283+0000","last_unstale":"2024-04-25T00:12:19.215241+0000","last_undegraded":"2024-04-25T00:12:19.215241+0000","last_fullsized":"2024-04-25T00:12:19.215241+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:26.110172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.220378+0000","last_change":"2024-04-25T00:12:18.199437+0000","last_active":"2024-04-25T00:12:19.220378+0000","last_peered":"2024-04-25T00:12:19.220378+0000","last_clean":"2024-04-25T00:12:19.220378+0000","last_became_active":"2024-04-25T00:12:18.199140+0000","last_became_peered":"2024-04-25T00:12:18.199140+0000","last_unstale":"2024-04-25T00:12:19.220378+0000","last_undegraded":"2024-04-25T00:12:19.220378+0000","last_fullsized":"2024-04-25T00:12:19.220378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:46.181569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.215975+0000","last_change":"2024-04-25T00:12:19.212030+0000","last_active":"2024-04-25T00:12:19.215975+0000","last_peered":"2024-04-25T00:12:19.215975+0000","last_clean":"2024-04-25T00:12:19.215975+0000","last_became_active":"2024-04-25T00:12:19.207986+0000","last_became_peered":"2024-04-25T00:12:19.207986+0000","last_unstale":"2024-04-25T00:12:19.215975+0000","last_undegraded":"2024-04-25T00:12:19.215975+0000","last_fullsized":"2024-04-25T00:12:19.215975+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:54.317722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.211293+0000","last_change":"2024-04-25T00:12:18.197059+0000","last_active":"2024-04-25T00:12:19.211293+0000","last_peered":"2024-04-25T00:12:19.211293+0000","last_clean":"2024-04-25T00:12:19.211293+0000","last_became_active":"2024-04-25T00:12:18.196783+0000","last_became_peered":"2024-04-25T00:12:18.196783+0000","last_unstale":"2024-04-25T00:12:19.211293+0000","last_undegraded":"2024-04-25T00:12:19.211293+0000","last_fullsized":"2024-04-25T00:12:19.211293+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:02:57.968512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.191460+0000","last_change":"2024-04-25T00:12:19.188997+0000","last_active":"2024-04-25T00:12:19.191460+0000","last_peered":"2024-04-25T00:12:19.191460+0000","last_clean":"2024-04-25T00:12:19.191460+0000","last_became_active":"2024-04-25T00:12:19.188631+0000","last_became_peered":"2024-04-25T00:12:19.188631+0000","last_unstale":"2024-04-25T00:12:19.191460+0000","last_undegraded":"2024-04-25T00:12:19.191460+0000","last_fullsized":"2024-04-25T00:12:19.191460+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:16.933678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.182143+0000","last_change":"2024-04-25T00:12:18.194583+0000","last_active":"2024-04-25T00:12:19.182143+0000","last_peered":"2024-04-25T00:12:19.182143+0000","last_clean":"2024-04-25T00:12:19.182143+0000","last_became_active":"2024-04-25T00:12:18.194237+0000","last_became_peered":"2024-04-25T00:12:18.194237+0000","last_unstale":"2024-04-25T00:12:19.182143+0000","last_undegraded":"2024-04-25T00:12:19.182143+0000","last_fullsized":"2024-04-25T00:12:19.182143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:44.142036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.211894+0000","last_change":"2024-04-25T00:12:19.203056+0000","last_active":"2024-04-25T00:12:19.211894+0000","last_peered":"2024-04-25T00:12:19.211894+0000","last_clean":"2024-04-25T00:12:19.211894+0000","last_became_active":"2024-04-25T00:12:19.202412+0000","last_became_peered":"2024-04-25T00:12:19.202412+0000","last_unstale":"2024-04-25T00:12:19.211894+0000","last_undegraded":"2024-04-25T00:12:19.211894+0000","last_fullsized":"2024-04-25T00:12:19.211894+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:15.513443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.206426+0000","last_change":"2024-04-25T00:12:18.195116+0000","last_active":"2024-04-25T00:12:19.206426+0000","last_peered":"2024-04-25T00:12:19.206426+0000","last_clean":"2024-04-25T00:12:19.206426+0000","last_became_active":"2024-04-25T00:12:18.194535+0000","last_became_peered":"2024-04-25T00:12:18.194535+0000","last_unstale":"2024-04-25T00:12:19.206426+0000","last_undegraded":"2024-04-25T00:12:19.206426+0000","last_fullsized":"2024-04-25T00:12:19.206426+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:23.278925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.210181+0000","last_change":"2024-04-25T00:12:19.203384+0000","last_active":"2024-04-25T00:12:19.210181+0000","last_peered":"2024-04-25T00:12:19.210181+0000","last_clean":"2024-04-25T00:12:19.210181+0000","last_became_active":"2024-04-25T00:12:19.203106+0000","last_became_peered":"2024-04-25T00:12:19.203106+0000","last_unstale":"2024-04-25T00:12:19.210181+0000","last_undegraded":"2024-04-25T00:12:19.210181+0000","last_fullsized":"2024-04-25T00:12:19.210181+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:42.766192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.176385+0000","last_change":"2024-04-25T00:12:18.197429+0000","last_active":"2024-04-25T00:12:19.176385+0000","last_peered":"2024-04-25T00:12:19.176385+0000","last_clean":"2024-04-25T00:12:19.176385+0000","last_became_active":"2024-04-25T00:12:18.197059+0000","last_became_peered":"2024-04-25T00:12:18.197059+0000","last_unstale":"2024-04-25T00:12:19.176385+0000","last_undegraded":"2024-04-25T00:12:19.176385+0000","last_fullsized":"2024-04-25T00:12:19.176385+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:34.858621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.209059+0000","last_change":"2024-04-25T00:12:19.204504+0000","last_active":"2024-04-25T00:12:19.209059+0000","last_peered":"2024-04-25T00:12:19.209059+0000","last_clean":"2024-04-25T00:12:19.209059+0000","last_became_active":"2024-04-25T00:12:19.204208+0000","last_became_peered":"2024-04-25T00:12:19.204208+0000","last_unstale":"2024-04-25T00:12:19.209059+0000","last_undegraded":"2024-04-25T00:12:19.209059+0000","last_fullsized":"2024-04-25T00:12:19.209059+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:59.846695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.181371+0000","last_change":"2024-04-25T00:12:18.192631+0000","last_active":"2024-04-25T00:12:19.181371+0000","last_peered":"2024-04-25T00:12:19.181371+0000","last_clean":"2024-04-25T00:12:19.181371+0000","last_became_active":"2024-04-25T00:12:18.192356+0000","last_became_peered":"2024-04-25T00:12:18.192356+0000","last_unstale":"2024-04-25T00:12:19.181371+0000","last_undegraded":"2024-04-25T00:12:19.181371+0000","last_fullsized":"2024-04-25T00:12:19.181371+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:41.759894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.206036+0000","last_change":"2024-04-25T00:12:19.199805+0000","last_active":"2024-04-25T00:12:19.206036+0000","last_peered":"2024-04-25T00:12:19.206036+0000","last_clean":"2024-04-25T00:12:19.206036+0000","last_became_active":"2024-04-25T00:12:19.199424+0000","last_became_peered":"2024-04-25T00:12:19.199424+0000","last_unstale":"2024-04-25T00:12:19.206036+0000","last_undegraded":"2024-04-25T00:12:19.206036+0000","last_fullsized":"2024-04-25T00:12:19.206036+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:22.932843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.176325+0000","last_change":"2024-04-25T00:12:18.191443+0000","last_active":"2024-04-25T00:12:19.176325+0000","last_peered":"2024-04-25T00:12:19.176325+0000","last_clean":"2024-04-25T00:12:19.176325+0000","last_became_active":"2024-04-25T00:12:18.191178+0000","last_became_peered":"2024-04-25T00:12:18.191178+0000","last_unstale":"2024-04-25T00:12:19.176325+0000","last_undegraded":"2024-04-25T00:12:19.176325+0000","last_fullsized":"2024-04-25T00:12:19.176325+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:53.804182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.212708+0000","last_change":"2024-04-25T00:12:19.207351+0000","last_active":"2024-04-25T00:12:19.212708+0000","last_peered":"2024-04-25T00:12:19.212708+0000","last_clean":"2024-04-25T00:12:19.212708+0000","last_became_active":"2024-04-25T00:12:19.207118+0000","last_became_peered":"2024-04-25T00:12:19.207118+0000","last_unstale":"2024-04-25T00:12:19.212708+0000","last_undegraded":"2024-04-25T00:12:19.212708+0000","last_fullsized":"2024-04-25T00:12:19.212708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:51.835822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.219564+0000","last_change":"2024-04-25T00:12:18.196809+0000","last_active":"2024-04-25T00:12:19.219564+0000","last_peered":"2024-04-25T00:12:19.219564+0000","last_clean":"2024-04-25T00:12:19.219564+0000","last_became_active":"2024-04-25T00:12:18.196483+0000","last_became_peered":"2024-04-25T00:12:18.196483+0000","last_unstale":"2024-04-25T00:12:19.219564+0000","last_undegraded":"2024-04-25T00:12:19.219564+0000","last_fullsized":"2024-04-25T00:12:19.219564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:29.677841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.206775+0000","last_change":"2024-04-25T00:12:19.195807+0000","last_active":"2024-04-25T00:12:19.206775+0000","last_peered":"2024-04-25T00:12:19.206775+0000","last_clean":"2024-04-25T00:12:19.206775+0000","last_became_active":"2024-04-25T00:12:19.195424+0000","last_became_peered":"2024-04-25T00:12:19.195424+0000","last_unstale":"2024-04-25T00:12:19.206775+0000","last_undegraded":"2024-04-25T00:12:19.206775+0000","last_fullsized":"2024-04-25T00:12:19.206775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:12.784794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.176113+0000","last_change":"2024-04-25T00:12:18.198414+0000","last_active":"2024-04-25T00:12:19.176113+0000","last_peered":"2024-04-25T00:12:19.176113+0000","last_clean":"2024-04-25T00:12:19.176113+0000","last_became_active":"2024-04-25T00:12:18.198120+0000","last_became_peered":"2024-04-25T00:12:18.198120+0000","last_unstale":"2024-04-25T00:12:19.176113+0000","last_undegraded":"2024-04-25T00:12:19.176113+0000","last_fullsized":"2024-04-25T00:12:19.176113+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:46.460272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.213192+0000","last_change":"2024-04-25T00:12:19.210555+0000","last_active":"2024-04-25T00:12:19.213192+0000","last_peered":"2024-04-25T00:12:19.213192+0000","last_clean":"2024-04-25T00:12:19.213192+0000","last_became_active":"2024-04-25T00:12:19.208885+0000","last_became_peered":"2024-04-25T00:12:19.208885+0000","last_unstale":"2024-04-25T00:12:19.213192+0000","last_undegraded":"2024-04-25T00:12:19.213192+0000","last_fullsized":"2024-04-25T00:12:19.213192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:38.281454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.189016+0000","last_change":"2024-04-25T00:12:18.197831+0000","last_active":"2024-04-25T00:12:19.189016+0000","last_peered":"2024-04-25T00:12:19.189016+0000","last_clean":"2024-04-25T00:12:19.189016+0000","last_became_active":"2024-04-25T00:12:18.197504+0000","last_became_peered":"2024-04-25T00:12:18.197504+0000","last_unstale":"2024-04-25T00:12:19.189016+0000","last_undegraded":"2024-04-25T00:12:19.189016+0000","last_fullsized":"2024-04-25T00:12:19.189016+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:59.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.211945+0000","last_change":"2024-04-25T00:12:19.205314+0000","last_active":"2024-04-25T00:12:19.211945+0000","last_peered":"2024-04-25T00:12:19.211945+0000","last_clean":"2024-04-25T00:12:19.211945+0000","last_became_active":"2024-04-25T00:12:19.205043+0000","last_became_peered":"2024-04-25T00:12:19.205043+0000","last_unstale":"2024-04-25T00:12:19.211945+0000","last_undegraded":"2024-04-25T00:12:19.211945+0000","last_fullsized":"2024-04-25T00:12:19.211945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:54.317722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.189637+0000","last_change":"2024-04-25T00:12:18.197556+0000","last_active":"2024-04-25T00:12:19.189637+0000","last_peered":"2024-04-25T00:12:19.189637+0000","last_clean":"2024-04-25T00:12:19.189637+0000","last_became_active":"2024-04-25T00:12:18.197268+0000","last_became_peered":"2024-04-25T00:12:18.197268+0000","last_unstale":"2024-04-25T00:12:19.189637+0000","last_undegraded":"2024-04-25T00:12:19.189637+0000","last_fullsized":"2024-04-25T00:12:19.189637+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:46.181569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.198533+0000","last_change":"2024-04-25T00:12:19.192310+0000","last_active":"2024-04-25T00:12:19.198533+0000","last_peered":"2024-04-25T00:12:19.198533+0000","last_clean":"2024-04-25T00:12:19.198533+0000","last_became_active":"2024-04-25T00:12:19.191888+0000","last_became_peered":"2024-04-25T00:12:19.191888+0000","last_unstale":"2024-04-25T00:12:19.198533+0000","last_undegraded":"2024-04-25T00:12:19.198533+0000","last_fullsized":"2024-04-25T00:12:19.198533+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:55.625707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.176533+0000","last_change":"2024-04-25T00:12:18.192181+0000","last_active":"2024-04-25T00:12:19.176533+0000","last_peered":"2024-04-25T00:12:19.176533+0000","last_clean":"2024-04-25T00:12:19.176533+0000","last_became_active":"2024-04-25T00:12:18.191908+0000","last_became_peered":"2024-04-25T00:12:18.191908+0000","last_unstale":"2024-04-25T00:12:19.176533+0000","last_undegraded":"2024-04-25T00:12:19.176533+0000","last_fullsized":"2024-04-25T00:12:19.176533+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:15.505830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.216038+0000","last_change":"2024-04-25T00:12:19.212224+0000","last_active":"2024-04-25T00:12:19.216038+0000","last_peered":"2024-04-25T00:12:19.216038+0000","last_clean":"2024-04-25T00:12:19.216038+0000","last_became_active":"2024-04-25T00:12:19.209567+0000","last_became_peered":"2024-04-25T00:12:19.209567+0000","last_unstale":"2024-04-25T00:12:19.216038+0000","last_undegraded":"2024-04-25T00:12:19.216038+0000","last_fullsized":"2024-04-25T00:12:19.216038+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:54.046773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.188807+0000","last_change":"2024-04-25T00:12:18.196698+0000","last_active":"2024-04-25T00:12:19.188807+0000","last_peered":"2024-04-25T00:12:19.188807+0000","last_clean":"2024-04-25T00:12:19.188807+0000","last_became_active":"2024-04-25T00:12:18.196413+0000","last_became_peered":"2024-04-25T00:12:18.196413+0000","last_unstale":"2024-04-25T00:12:19.188807+0000","last_undegraded":"2024-04-25T00:12:19.188807+0000","last_fullsized":"2024-04-25T00:12:19.188807+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:59.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.202792+0000","last_change":"2024-04-25T00:12:19.195164+0000","last_active":"2024-04-25T00:12:19.202792+0000","last_peered":"2024-04-25T00:12:19.202792+0000","last_clean":"2024-04-25T00:12:19.202792+0000","last_became_active":"2024-04-25T00:12:19.194892+0000","last_became_peered":"2024-04-25T00:12:19.194892+0000","last_unstale":"2024-04-25T00:12:19.202792+0000","last_undegraded":"2024-04-25T00:12:19.202792+0000","last_fullsized":"2024-04-25T00:12:19.202792+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:51.835822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.179458+0000","last_change":"2024-04-25T00:12:18.192155+0000","last_active":"2024-04-25T00:12:19.179458+0000","last_peered":"2024-04-25T00:12:19.179458+0000","last_clean":"2024-04-25T00:12:19.179458+0000","last_became_active":"2024-04-25T00:12:18.191865+0000","last_became_peered":"2024-04-25T00:12:18.191865+0000","last_unstale":"2024-04-25T00:12:19.179458+0000","last_undegraded":"2024-04-25T00:12:19.179458+0000","last_fullsized":"2024-04-25T00:12:19.179458+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:14.507145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.219861+0000","last_change":"2024-04-25T00:12:19.214000+0000","last_active":"2024-04-25T00:12:19.219861+0000","last_peered":"2024-04-25T00:12:19.219861+0000","last_clean":"2024-04-25T00:12:19.219861+0000","last_became_active":"2024-04-25T00:12:19.212643+0000","last_became_peered":"2024-04-25T00:12:19.212643+0000","last_unstale":"2024-04-25T00:12:19.219861+0000","last_undegraded":"2024-04-25T00:12:19.219861+0000","last_fullsized":"2024-04-25T00:12:19.219861+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:50.934891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.182179+0000","last_change":"2024-04-25T00:12:18.194533+0000","last_active":"2024-04-25T00:12:19.182179+0000","last_peered":"2024-04-25T00:12:19.182179+0000","last_clean":"2024-04-25T00:12:19.182179+0000","last_became_active":"2024-04-25T00:12:18.194243+0000","last_became_peered":"2024-04-25T00:12:18.194243+0000","last_unstale":"2024-04-25T00:12:19.182179+0000","last_undegraded":"2024-04-25T00:12:19.182179+0000","last_fullsized":"2024-04-25T00:12:19.182179+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:53.311425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":200,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.184505+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:12:19.184505+0000","last_peered":"2024-04-25T00:12:19.184505+0000","last_clean":"2024-04-25T00:12:19.184505+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:12:19.184505+0000","last_undegraded":"2024-04-25T00:12:19.184505+0000","last_fullsized":"2024-04-25T00:12:19.184505+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:12:23.267 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:12:23.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.438+0000 7efe0de00640 1 Processor -- start 2024-04-25T00:12:23.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.438+0000 7efe0de00640 1 -- start start 2024-04-25T00:12:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.438+0000 7efe0de00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe081553e0 0x7efe0806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.438+0000 7efe0de00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe0806ef80 con 0x7efe0806ea00 2024-04-25T00:12:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.438+0000 7efe0de00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe0815b510 con 0x7efe080b4a70 2024-04-25T00:12:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.438+0000 7efe0de00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efe0815b7f0 con 0x7efe081553e0 2024-04-25T00:12:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.438+0000 7efdffe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efe0806ea00 0x7efe080b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56974/0 (socket says 172.21.15.83:56974) 2024-04-25T00:12:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.438+0000 7efdffe00640 1 -- 172.21.15.83:0/3511992983 learned_addr learned my addr 172.21.15.83:0/3511992983 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.442+0000 7efe06a00640 1 --2- 172.21.15.83:0/3511992983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe081553e0 0x7efe0806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.442+0000 7efe06000640 1 -- 172.21.15.83:0/3511992983 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2745836370 0 0) 0x7efe0806ef80 con 0x7efe0806ea00 2024-04-25T00:12:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.442+0000 7efe06000640 1 -- 172.21.15.83:0/3511992983 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdec003530 con 0x7efe0806ea00 2024-04-25T00:12:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.442+0000 7efe06000640 1 -- 172.21.15.83:0/3511992983 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 805356262 0 0) 0x7efe0815b510 con 0x7efe080b4a70 2024-04-25T00:12:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.442+0000 7efe06000640 1 -- 172.21.15.83:0/3511992983 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe0806ef80 con 0x7efe080b4a70 2024-04-25T00:12:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.442+0000 7efe06000640 1 -- 172.21.15.83:0/3511992983 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 804988667 0 0) 0x7efdec003530 con 0x7efe0806ea00 2024-04-25T00:12:23.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.442+0000 7efe06a00640 1 -- 172.21.15.83:0/3511992983 >> v1:172.21.15.86:6789/0 conn(0x7efe080b4a70 legacy=0x7efe080b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.442+0000 7efe06a00640 1 -- 172.21.15.83:0/3511992983 >> v1:172.21.15.83:6789/0 conn(0x7efe0806ea00 legacy=0x7efe080b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.442+0000 7efe06a00640 1 -- 172.21.15.83:0/3511992983 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0815bad0 con 0x7efe081553e0 2024-04-25T00:12:23.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.442+0000 7efe06a00640 1 --2- 172.21.15.83:0/3511992983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe081553e0 0x7efe0806e4c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efdf80097f0 tx=0x7efdf801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97749be3f37d22ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:23.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.442+0000 7efe06000640 1 -- 172.21.15.83:0/3511992983 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf80b0070 con 0x7efe081553e0 2024-04-25T00:12:23.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.442+0000 7efe06000640 1 -- 172.21.15.83:0/3511992983 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf80ab400 con 0x7efe081553e0 2024-04-25T00:12:23.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.442+0000 7efe06000640 1 -- 172.21.15.83:0/3511992983 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0001600 con 0x7efe081553e0 2024-04-25T00:12:23.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 -- 172.21.15.83:0/3511992983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe081553e0 msgr2=0x7efe0806e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 --2- 172.21.15.83:0/3511992983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe081553e0 0x7efe0806e4c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efdf80097f0 tx=0x7efdf801c370 comp rx=0 tx=0).stop 2024-04-25T00:12:23.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 -- 172.21.15.83:0/3511992983 shutdown_connections 2024-04-25T00:12:23.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 --2- 172.21.15.83:0/3511992983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe081553e0 0x7efe0806e4c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 -- 172.21.15.83:0/3511992983 >> 172.21.15.83:0/3511992983 conn(0x7efe080b1830 msgr2=0x7efe080b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:23.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 -- 172.21.15.83:0/3511992983 shutdown_connections 2024-04-25T00:12:23.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 -- 172.21.15.83:0/3511992983 wait complete. 2024-04-25T00:12:23.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 Processor -- start 2024-04-25T00:12:23.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 -- start start 2024-04-25T00:12:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe0806ea00 0x7efe08147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe080b4a70 0x7efe0814af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe081553e0 0x7efe08157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe0806d530 con 0x7efe080b4a70 2024-04-25T00:12:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe0806d3b0 con 0x7efe081553e0 2024-04-25T00:12:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe0de00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efe0806d6b0 con 0x7efe0806ea00 2024-04-25T00:12:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efdffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe080b4a70 0x7efe0814af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efdffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe080b4a70 0x7efe0814af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34458/0 (socket says 172.21.15.83:34458) 2024-04-25T00:12:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efdffe00640 1 -- 172.21.15.83:0/3228513551 learned_addr learned my addr 172.21.15.83:0/3228513551 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe06a00640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe0806ea00 0x7efe08147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efe07400640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe081553e0 0x7efe08157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.446+0000 7efdffe00640 1 -- 172.21.15.83:0/3228513551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe0806ea00 msgr2=0x7efe08147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.450+0000 7efdffe00640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe0806ea00 0x7efe08147000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.450+0000 7efdffe00640 1 -- 172.21.15.83:0/3228513551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe081553e0 msgr2=0x7efe08157ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.450+0000 7efdffe00640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe081553e0 0x7efe08157ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.450+0000 7efdffe00640 1 -- 172.21.15.83:0/3228513551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0814b890 con 0x7efe080b4a70 2024-04-25T00:12:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.450+0000 7efe07400640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe081553e0 0x7efe08157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.450+0000 7efdffe00640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe080b4a70 0x7efe0814af40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c750 tx=0x7efdf000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.450+0000 7efdff400640 1 -- 172.21.15.83:0/3228513551 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0005ae0 con 0x7efe080b4a70 2024-04-25T00:12:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.450+0000 7efdff400640 1 -- 172.21.15.83:0/3228513551 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf0005c80 con 0x7efe080b4a70 2024-04-25T00:12:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.450+0000 7efdff400640 1 -- 172.21.15.83:0/3228513551 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf00072b0 con 0x7efe080b4a70 2024-04-25T00:12:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.450+0000 7efe0de00640 1 -- 172.21.15.83:0/3228513551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0814bb70 con 0x7efe080b4a70 2024-04-25T00:12:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.450+0000 7efe0de00640 1 -- 172.21.15.83:0/3228513551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe0814b480 con 0x7efe080b4a70 2024-04-25T00:12:23.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.450+0000 7efe0de00640 1 -- 172.21.15.83:0/3228513551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe080737f0 con 0x7efe080b4a70 2024-04-25T00:12:23.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.450+0000 7efdff400640 1 -- 172.21.15.83:0/3228513551 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efdf0007450 con 0x7efe080b4a70 2024-04-25T00:12:23.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.450+0000 7efdff400640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efde40678c0 0x7efde4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.454+0000 7efdff400640 1 -- 172.21.15.83:0/3228513551 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7efdf0085bf0 con 0x7efe080b4a70 2024-04-25T00:12:23.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.454+0000 7efe06a00640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efde40678c0 0x7efde4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.454+0000 7efe06a00640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efde40678c0 0x7efde4069d80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efdf80ad000 tx=0x7efdf80a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:23.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.458+0000 7efdff400640 1 -- 172.21.15.83:0/3228513551 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efdf00565a0 con 0x7efe080b4a70 2024-04-25T00:12:23.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.610+0000 7efe0de00640 1 -- 172.21.15.83:0/3228513551 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe0804f5f0 con 0x7efde40678c0 2024-04-25T00:12:23.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.614+0000 7efdff400640 1 -- 172.21.15.83:0/3228513551 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251890 (secure 0 0 0) 0x7efe0804f5f0 con 0x7efde40678c0 2024-04-25T00:12:23.620 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:23.621 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:12:23.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.618+0000 7efe0de00640 1 -- 172.21.15.83:0/3228513551 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efde40678c0 msgr2=0x7efde4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.618+0000 7efe0de00640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efde40678c0 0x7efde4069d80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efdf80ad000 tx=0x7efdf80a6000 comp rx=0 tx=0).stop 2024-04-25T00:12:23.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.618+0000 7efe0de00640 1 -- 172.21.15.83:0/3228513551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe080b4a70 msgr2=0x7efe0814af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.618+0000 7efe0de00640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe080b4a70 0x7efe0814af40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c750 tx=0x7efdf000cc10 comp rx=0 tx=0).stop 2024-04-25T00:12:23.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.618+0000 7efe0de00640 1 -- 172.21.15.83:0/3228513551 shutdown_connections 2024-04-25T00:12:23.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.618+0000 7efe0de00640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efde40678c0 0x7efde4069d80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.618+0000 7efe0de00640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe081553e0 0x7efe08157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.618+0000 7efe0de00640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe080b4a70 0x7efe0814af40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.618+0000 7efe0de00640 1 --2- 172.21.15.83:0/3228513551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe0806ea00 0x7efe08147000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.618+0000 7efe0de00640 1 -- 172.21.15.83:0/3228513551 >> 172.21.15.83:0/3228513551 conn(0x7efe080b1830 msgr2=0x7efe0814fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:23.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.618+0000 7efe0de00640 1 -- 172.21.15.83:0/3228513551 shutdown_connections 2024-04-25T00:12:23.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.618+0000 7efe0de00640 1 -- 172.21.15.83:0/3228513551 wait complete. 2024-04-25T00:12:23.639 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-04-25T00:12:22.459968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220584,"kb_used_data":4392,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754136,"statfs":{"total":773094113280,"available":772868235264,"internally_reserved":0,"allocated":4497408,"data_stored":2590152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.298861"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.204387+0000","last_change":"2024-04-25T00:12:19.198325+0000","last_active":"2024-04-25T00:12:19.204387+0000","last_peered":"2024-04-25T00:12:19.204387+0000","last_clean":"2024-04-25T00:12:19.204387+0000","last_became_active":"2024-04-25T00:12:19.198064+0000","last_became_peered":"2024-04-25T00:12:19.198064+0000","last_unstale":"2024-04-25T00:12:19.204387+0000","last_undegraded":"2024-04-25T00:12:19.204387+0000","last_fullsized":"2024-04-25T00:12:19.204387+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:30.398208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.205023+0000","last_change":"2024-04-25T00:12:19.198635+0000","last_active":"2024-04-25T00:12:19.205023+0000","last_peered":"2024-04-25T00:12:19.205023+0000","last_clean":"2024-04-25T00:12:19.205023+0000","last_became_active":"2024-04-25T00:12:19.198374+0000","last_became_peered":"2024-04-25T00:12:19.198374+0000","last_unstale":"2024-04-25T00:12:19.205023+0000","last_undegraded":"2024-04-25T00:12:19.205023+0000","last_fullsized":"2024-04-25T00:12:19.205023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:41.297808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.217606+0000","last_change":"2024-04-25T00:12:19.216865+0000","last_active":"2024-04-25T00:12:19.217606+0000","last_peered":"2024-04-25T00:12:19.217606+0000","last_clean":"2024-04-25T00:12:19.217606+0000","last_became_active":"2024-04-25T00:12:19.216557+0000","last_became_peered":"2024-04-25T00:12:19.216557+0000","last_unstale":"2024-04-25T00:12:19.217606+0000","last_undegraded":"2024-04-25T00:12:19.217606+0000","last_fullsized":"2024-04-25T00:12:19.217606+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:30.684139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.209148+0000","last_change":"2024-04-25T00:12:19.204017+0000","last_active":"2024-04-25T00:12:19.209148+0000","last_peered":"2024-04-25T00:12:19.209148+0000","last_clean":"2024-04-25T00:12:19.209148+0000","last_became_active":"2024-04-25T00:12:19.203770+0000","last_became_peered":"2024-04-25T00:12:19.203770+0000","last_unstale":"2024-04-25T00:12:19.209148+0000","last_undegraded":"2024-04-25T00:12:19.209148+0000","last_fullsized":"2024-04-25T00:12:19.209148+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:41.771920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.203624+0000","last_change":"2024-04-25T00:12:19.193889+0000","last_active":"2024-04-25T00:12:19.203624+0000","last_peered":"2024-04-25T00:12:19.203624+0000","last_clean":"2024-04-25T00:12:19.203624+0000","last_became_active":"2024-04-25T00:12:19.193690+0000","last_became_peered":"2024-04-25T00:12:19.193690+0000","last_unstale":"2024-04-25T00:12:19.203624+0000","last_undegraded":"2024-04-25T00:12:19.203624+0000","last_fullsized":"2024-04-25T00:12:19.203624+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:31.009303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.202778+0000","last_change":"2024-04-25T00:12:19.200943+0000","last_active":"2024-04-25T00:12:19.202778+0000","last_peered":"2024-04-25T00:12:19.202778+0000","last_clean":"2024-04-25T00:12:19.202778+0000","last_became_active":"2024-04-25T00:12:19.200683+0000","last_became_peered":"2024-04-25T00:12:19.200683+0000","last_unstale":"2024-04-25T00:12:19.202778+0000","last_undegraded":"2024-04-25T00:12:19.202778+0000","last_fullsized":"2024-04-25T00:12:19.202778+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:30.684139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.210170+0000","last_change":"2024-04-25T00:12:19.205772+0000","last_active":"2024-04-25T00:12:19.210170+0000","last_peered":"2024-04-25T00:12:19.210170+0000","last_clean":"2024-04-25T00:12:19.210170+0000","last_became_active":"2024-04-25T00:12:19.205315+0000","last_became_peered":"2024-04-25T00:12:19.205315+0000","last_unstale":"2024-04-25T00:12:19.210170+0000","last_undegraded":"2024-04-25T00:12:19.210170+0000","last_fullsized":"2024-04-25T00:12:19.210170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:17.550259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.209386+0000","last_change":"2024-04-25T00:12:19.206007+0000","last_active":"2024-04-25T00:12:19.209386+0000","last_peered":"2024-04-25T00:12:19.209386+0000","last_clean":"2024-04-25T00:12:19.209386+0000","last_became_active":"2024-04-25T00:12:19.205684+0000","last_became_peered":"2024-04-25T00:12:19.205684+0000","last_unstale":"2024-04-25T00:12:19.209386+0000","last_undegraded":"2024-04-25T00:12:19.209386+0000","last_fullsized":"2024-04-25T00:12:19.209386+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:01.841718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.211981+0000","last_change":"2024-04-25T00:12:19.206482+0000","last_active":"2024-04-25T00:12:19.211981+0000","last_peered":"2024-04-25T00:12:19.211981+0000","last_clean":"2024-04-25T00:12:19.211981+0000","last_became_active":"2024-04-25T00:12:19.206203+0000","last_became_peered":"2024-04-25T00:12:19.206203+0000","last_unstale":"2024-04-25T00:12:19.211981+0000","last_undegraded":"2024-04-25T00:12:19.211981+0000","last_fullsized":"2024-04-25T00:12:19.211981+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:54.046773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.220065+0000","last_change":"2024-04-25T00:12:19.216916+0000","last_active":"2024-04-25T00:12:19.220065+0000","last_peered":"2024-04-25T00:12:19.220065+0000","last_clean":"2024-04-25T00:12:19.220065+0000","last_became_active":"2024-04-25T00:12:19.216608+0000","last_became_peered":"2024-04-25T00:12:19.216608+0000","last_unstale":"2024-04-25T00:12:19.220065+0000","last_undegraded":"2024-04-25T00:12:19.220065+0000","last_fullsized":"2024-04-25T00:12:19.220065+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:59.322277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.219955+0000","last_change":"2024-04-25T00:12:19.214605+0000","last_active":"2024-04-25T00:12:19.219955+0000","last_peered":"2024-04-25T00:12:19.219955+0000","last_clean":"2024-04-25T00:12:19.219955+0000","last_became_active":"2024-04-25T00:12:19.214295+0000","last_became_peered":"2024-04-25T00:12:19.214295+0000","last_unstale":"2024-04-25T00:12:19.219955+0000","last_undegraded":"2024-04-25T00:12:19.219955+0000","last_fullsized":"2024-04-25T00:12:19.219955+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:30.398208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.200733+0000","last_change":"2024-04-25T00:12:19.195321+0000","last_active":"2024-04-25T00:12:19.200733+0000","last_peered":"2024-04-25T00:12:19.200733+0000","last_clean":"2024-04-25T00:12:19.200733+0000","last_became_active":"2024-04-25T00:12:19.195023+0000","last_became_peered":"2024-04-25T00:12:19.195023+0000","last_unstale":"2024-04-25T00:12:19.200733+0000","last_undegraded":"2024-04-25T00:12:19.200733+0000","last_fullsized":"2024-04-25T00:12:19.200733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:12.177125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.206284+0000","last_change":"2024-04-25T00:12:19.201761+0000","last_active":"2024-04-25T00:12:19.206284+0000","last_peered":"2024-04-25T00:12:19.206284+0000","last_clean":"2024-04-25T00:12:19.206284+0000","last_became_active":"2024-04-25T00:12:19.201481+0000","last_became_peered":"2024-04-25T00:12:19.201481+0000","last_unstale":"2024-04-25T00:12:19.206284+0000","last_undegraded":"2024-04-25T00:12:19.206284+0000","last_fullsized":"2024-04-25T00:12:19.206284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:30.541480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.207720+0000","last_change":"2024-04-25T00:12:19.199884+0000","last_active":"2024-04-25T00:12:19.207720+0000","last_peered":"2024-04-25T00:12:19.207720+0000","last_clean":"2024-04-25T00:12:19.207720+0000","last_became_active":"2024-04-25T00:12:19.199418+0000","last_became_peered":"2024-04-25T00:12:19.199418+0000","last_unstale":"2024-04-25T00:12:19.207720+0000","last_undegraded":"2024-04-25T00:12:19.207720+0000","last_fullsized":"2024-04-25T00:12:19.207720+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:45.148333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.210152+0000","last_change":"2024-04-25T00:12:19.205614+0000","last_active":"2024-04-25T00:12:19.210152+0000","last_peered":"2024-04-25T00:12:19.210152+0000","last_clean":"2024-04-25T00:12:19.210152+0000","last_became_active":"2024-04-25T00:12:19.205168+0000","last_became_peered":"2024-04-25T00:12:19.205168+0000","last_unstale":"2024-04-25T00:12:19.210152+0000","last_undegraded":"2024-04-25T00:12:19.210152+0000","last_fullsized":"2024-04-25T00:12:19.210152+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:52.162568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.215625+0000","last_change":"2024-04-25T00:12:19.211831+0000","last_active":"2024-04-25T00:12:19.215625+0000","last_peered":"2024-04-25T00:12:19.215625+0000","last_clean":"2024-04-25T00:12:19.215625+0000","last_became_active":"2024-04-25T00:12:19.211563+0000","last_became_peered":"2024-04-25T00:12:19.211563+0000","last_unstale":"2024-04-25T00:12:19.215625+0000","last_undegraded":"2024-04-25T00:12:19.215625+0000","last_fullsized":"2024-04-25T00:12:19.215625+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:52.162568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.214784+0000","last_change":"2024-04-25T00:12:19.209242+0000","last_active":"2024-04-25T00:12:19.214784+0000","last_peered":"2024-04-25T00:12:19.214784+0000","last_clean":"2024-04-25T00:12:19.214784+0000","last_became_active":"2024-04-25T00:12:19.208992+0000","last_became_peered":"2024-04-25T00:12:19.208992+0000","last_unstale":"2024-04-25T00:12:19.214784+0000","last_undegraded":"2024-04-25T00:12:19.214784+0000","last_fullsized":"2024-04-25T00:12:19.214784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:59.846695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.207002+0000","last_change":"2024-04-25T00:12:19.204643+0000","last_active":"2024-04-25T00:12:19.207002+0000","last_peered":"2024-04-25T00:12:19.207002+0000","last_clean":"2024-04-25T00:12:19.207002+0000","last_became_active":"2024-04-25T00:12:19.204373+0000","last_became_peered":"2024-04-25T00:12:19.204373+0000","last_unstale":"2024-04-25T00:12:19.207002+0000","last_undegraded":"2024-04-25T00:12:19.207002+0000","last_fullsized":"2024-04-25T00:12:19.207002+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:54.317722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.202767+0000","last_change":"2024-04-25T00:12:19.197380+0000","last_active":"2024-04-25T00:12:19.202767+0000","last_peered":"2024-04-25T00:12:19.202767+0000","last_clean":"2024-04-25T00:12:19.202767+0000","last_became_active":"2024-04-25T00:12:19.195729+0000","last_became_peered":"2024-04-25T00:12:19.195729+0000","last_unstale":"2024-04-25T00:12:19.202767+0000","last_undegraded":"2024-04-25T00:12:19.202767+0000","last_fullsized":"2024-04-25T00:12:19.202767+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:04.436136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.197799+0000","last_change":"2024-04-25T00:12:19.193388+0000","last_active":"2024-04-25T00:12:19.197799+0000","last_peered":"2024-04-25T00:12:19.197799+0000","last_clean":"2024-04-25T00:12:19.197799+0000","last_became_active":"2024-04-25T00:12:19.193178+0000","last_became_peered":"2024-04-25T00:12:19.193178+0000","last_unstale":"2024-04-25T00:12:19.197799+0000","last_undegraded":"2024-04-25T00:12:19.197799+0000","last_fullsized":"2024-04-25T00:12:19.197799+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:12.177125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.215469+0000","last_change":"2024-04-25T00:12:19.208286+0000","last_active":"2024-04-25T00:12:19.215469+0000","last_peered":"2024-04-25T00:12:19.215469+0000","last_clean":"2024-04-25T00:12:19.215469+0000","last_became_active":"2024-04-25T00:12:19.206423+0000","last_became_peered":"2024-04-25T00:12:19.206423+0000","last_unstale":"2024-04-25T00:12:19.215469+0000","last_undegraded":"2024-04-25T00:12:19.215469+0000","last_fullsized":"2024-04-25T00:12:19.215469+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:37:13.047475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.204037+0000","last_change":"2024-04-25T00:12:19.196257+0000","last_active":"2024-04-25T00:12:19.204037+0000","last_peered":"2024-04-25T00:12:19.204037+0000","last_clean":"2024-04-25T00:12:19.204037+0000","last_became_active":"2024-04-25T00:12:19.195953+0000","last_became_peered":"2024-04-25T00:12:19.195953+0000","last_unstale":"2024-04-25T00:12:19.204037+0000","last_undegraded":"2024-04-25T00:12:19.204037+0000","last_fullsized":"2024-04-25T00:12:19.204037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:39.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.201493+0000","last_change":"2024-04-25T00:12:19.192634+0000","last_active":"2024-04-25T00:12:19.201493+0000","last_peered":"2024-04-25T00:12:19.201493+0000","last_clean":"2024-04-25T00:12:19.201493+0000","last_became_active":"2024-04-25T00:12:19.192136+0000","last_became_peered":"2024-04-25T00:12:19.192136+0000","last_unstale":"2024-04-25T00:12:19.201493+0000","last_undegraded":"2024-04-25T00:12:19.201493+0000","last_fullsized":"2024-04-25T00:12:19.201493+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:55.625707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.203247+0000","last_change":"2024-04-25T00:12:19.201203+0000","last_active":"2024-04-25T00:12:19.203247+0000","last_peered":"2024-04-25T00:12:19.203247+0000","last_clean":"2024-04-25T00:12:19.203247+0000","last_became_active":"2024-04-25T00:12:19.200946+0000","last_became_peered":"2024-04-25T00:12:19.200946+0000","last_unstale":"2024-04-25T00:12:19.203247+0000","last_undegraded":"2024-04-25T00:12:19.203247+0000","last_fullsized":"2024-04-25T00:12:19.203247+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:08.524271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.197600+0000","last_change":"2024-04-25T00:12:19.193232+0000","last_active":"2024-04-25T00:12:19.197600+0000","last_peered":"2024-04-25T00:12:19.197600+0000","last_clean":"2024-04-25T00:12:19.197600+0000","last_became_active":"2024-04-25T00:12:19.192904+0000","last_became_peered":"2024-04-25T00:12:19.192904+0000","last_unstale":"2024-04-25T00:12:19.197600+0000","last_undegraded":"2024-04-25T00:12:19.197600+0000","last_fullsized":"2024-04-25T00:12:19.197600+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:39.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.215568+0000","last_change":"2024-04-25T00:12:19.214294+0000","last_active":"2024-04-25T00:12:19.215568+0000","last_peered":"2024-04-25T00:12:19.215568+0000","last_clean":"2024-04-25T00:12:19.215568+0000","last_became_active":"2024-04-25T00:12:19.214081+0000","last_became_peered":"2024-04-25T00:12:19.214081+0000","last_unstale":"2024-04-25T00:12:19.215568+0000","last_undegraded":"2024-04-25T00:12:19.215568+0000","last_fullsized":"2024-04-25T00:12:19.215568+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:12.784794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.187756+0000","last_change":"2024-04-25T00:12:19.181736+0000","last_active":"2024-04-25T00:12:19.187756+0000","last_peered":"2024-04-25T00:12:19.187756+0000","last_clean":"2024-04-25T00:12:19.187756+0000","last_became_active":"2024-04-25T00:12:19.181446+0000","last_became_peered":"2024-04-25T00:12:19.181446+0000","last_unstale":"2024-04-25T00:12:19.187756+0000","last_undegraded":"2024-04-25T00:12:19.187756+0000","last_fullsized":"2024-04-25T00:12:19.187756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:47.187867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.210239+0000","last_change":"2024-04-25T00:12:19.203769+0000","last_active":"2024-04-25T00:12:19.210239+0000","last_peered":"2024-04-25T00:12:19.210239+0000","last_clean":"2024-04-25T00:12:19.210239+0000","last_became_active":"2024-04-25T00:12:19.203324+0000","last_became_peered":"2024-04-25T00:12:19.203324+0000","last_unstale":"2024-04-25T00:12:19.210239+0000","last_undegraded":"2024-04-25T00:12:19.210239+0000","last_fullsized":"2024-04-25T00:12:19.210239+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:15.513443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.205431+0000","last_change":"2024-04-25T00:12:19.196097+0000","last_active":"2024-04-25T00:12:19.205431+0000","last_peered":"2024-04-25T00:12:19.205431+0000","last_clean":"2024-04-25T00:12:19.205431+0000","last_became_active":"2024-04-25T00:12:19.195829+0000","last_became_peered":"2024-04-25T00:12:19.195829+0000","last_unstale":"2024-04-25T00:12:19.205431+0000","last_undegraded":"2024-04-25T00:12:19.205431+0000","last_fullsized":"2024-04-25T00:12:19.205431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:45.148333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.206783+0000","last_change":"2024-04-25T00:12:19.199560+0000","last_active":"2024-04-25T00:12:19.206783+0000","last_peered":"2024-04-25T00:12:19.206783+0000","last_clean":"2024-04-25T00:12:19.206783+0000","last_became_active":"2024-04-25T00:12:19.199247+0000","last_became_peered":"2024-04-25T00:12:19.199247+0000","last_unstale":"2024-04-25T00:12:19.206783+0000","last_undegraded":"2024-04-25T00:12:19.206783+0000","last_fullsized":"2024-04-25T00:12:19.206783+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:42.766192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.208319+0000","last_change":"2024-04-25T00:12:19.197092+0000","last_active":"2024-04-25T00:12:19.208319+0000","last_peered":"2024-04-25T00:12:19.208319+0000","last_clean":"2024-04-25T00:12:19.208319+0000","last_became_active":"2024-04-25T00:12:19.196828+0000","last_became_peered":"2024-04-25T00:12:19.196828+0000","last_unstale":"2024-04-25T00:12:19.208319+0000","last_undegraded":"2024-04-25T00:12:19.208319+0000","last_fullsized":"2024-04-25T00:12:19.208319+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:30.541480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.199697+0000","last_change":"2024-04-25T00:12:19.192306+0000","last_active":"2024-04-25T00:12:19.199697+0000","last_peered":"2024-04-25T00:12:19.199697+0000","last_clean":"2024-04-25T00:12:19.199697+0000","last_became_active":"2024-04-25T00:12:19.192018+0000","last_became_peered":"2024-04-25T00:12:19.192018+0000","last_unstale":"2024-04-25T00:12:19.199697+0000","last_undegraded":"2024-04-25T00:12:19.199697+0000","last_fullsized":"2024-04-25T00:12:19.199697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:54.046773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.215786+0000","last_change":"2024-04-25T00:12:19.210776+0000","last_active":"2024-04-25T00:12:19.215786+0000","last_peered":"2024-04-25T00:12:19.215786+0000","last_clean":"2024-04-25T00:12:19.215786+0000","last_became_active":"2024-04-25T00:12:19.210489+0000","last_became_peered":"2024-04-25T00:12:19.210489+0000","last_unstale":"2024-04-25T00:12:19.215786+0000","last_undegraded":"2024-04-25T00:12:19.215786+0000","last_fullsized":"2024-04-25T00:12:19.215786+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:32.234020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.211602+0000","last_change":"2024-04-25T00:12:18.195226+0000","last_active":"2024-04-25T00:12:19.211602+0000","last_peered":"2024-04-25T00:12:19.211602+0000","last_clean":"2024-04-25T00:12:19.211602+0000","last_became_active":"2024-04-25T00:12:18.194964+0000","last_became_peered":"2024-04-25T00:12:18.194964+0000","last_unstale":"2024-04-25T00:12:19.211602+0000","last_undegraded":"2024-04-25T00:12:19.211602+0000","last_fullsized":"2024-04-25T00:12:19.211602+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:59.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.218209+0000","last_change":"2024-04-25T00:12:19.216346+0000","last_active":"2024-04-25T00:12:19.218209+0000","last_peered":"2024-04-25T00:12:19.218209+0000","last_clean":"2024-04-25T00:12:19.218209+0000","last_became_active":"2024-04-25T00:12:19.216086+0000","last_became_peered":"2024-04-25T00:12:19.216086+0000","last_unstale":"2024-04-25T00:12:19.218209+0000","last_undegraded":"2024-04-25T00:12:19.218209+0000","last_fullsized":"2024-04-25T00:12:19.218209+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:30.541480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.216349+0000","last_change":"2024-04-25T00:12:18.199602+0000","last_active":"2024-04-25T00:12:19.216349+0000","last_peered":"2024-04-25T00:12:19.216349+0000","last_clean":"2024-04-25T00:12:19.216349+0000","last_became_active":"2024-04-25T00:12:18.197254+0000","last_became_peered":"2024-04-25T00:12:18.197254+0000","last_unstale":"2024-04-25T00:12:19.216349+0000","last_undegraded":"2024-04-25T00:12:19.216349+0000","last_fullsized":"2024-04-25T00:12:19.216349+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:44.748128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.209584+0000","last_change":"2024-04-25T00:12:19.197752+0000","last_active":"2024-04-25T00:12:19.209584+0000","last_peered":"2024-04-25T00:12:19.209584+0000","last_clean":"2024-04-25T00:12:19.209584+0000","last_became_active":"2024-04-25T00:12:19.197336+0000","last_became_peered":"2024-04-25T00:12:19.197336+0000","last_unstale":"2024-04-25T00:12:19.209584+0000","last_undegraded":"2024-04-25T00:12:19.209584+0000","last_fullsized":"2024-04-25T00:12:19.209584+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:12.177125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.215658+0000","last_change":"2024-04-25T00:12:18.194323+0000","last_active":"2024-04-25T00:12:19.215658+0000","last_peered":"2024-04-25T00:12:19.215658+0000","last_clean":"2024-04-25T00:12:19.215658+0000","last_became_active":"2024-04-25T00:12:18.193587+0000","last_became_peered":"2024-04-25T00:12:18.193587+0000","last_unstale":"2024-04-25T00:12:19.215658+0000","last_undegraded":"2024-04-25T00:12:19.215658+0000","last_fullsized":"2024-04-25T00:12:19.215658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:53.311425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.199995+0000","last_change":"2024-04-25T00:12:19.186797+0000","last_active":"2024-04-25T00:12:19.199995+0000","last_peered":"2024-04-25T00:12:19.199995+0000","last_clean":"2024-04-25T00:12:19.199995+0000","last_became_active":"2024-04-25T00:12:19.186615+0000","last_became_peered":"2024-04-25T00:12:19.186615+0000","last_unstale":"2024-04-25T00:12:19.199995+0000","last_undegraded":"2024-04-25T00:12:19.199995+0000","last_fullsized":"2024-04-25T00:12:19.199995+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:17.550259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.209232+0000","last_change":"2024-04-25T00:12:18.197408+0000","last_active":"2024-04-25T00:12:19.209232+0000","last_peered":"2024-04-25T00:12:19.209232+0000","last_clean":"2024-04-25T00:12:19.209232+0000","last_became_active":"2024-04-25T00:12:18.197140+0000","last_became_peered":"2024-04-25T00:12:18.197140+0000","last_unstale":"2024-04-25T00:12:19.209232+0000","last_undegraded":"2024-04-25T00:12:19.209232+0000","last_fullsized":"2024-04-25T00:12:19.209232+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:44.748128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.200653+0000","last_change":"2024-04-25T00:12:19.194230+0000","last_active":"2024-04-25T00:12:19.200653+0000","last_peered":"2024-04-25T00:12:19.200653+0000","last_clean":"2024-04-25T00:12:19.200653+0000","last_became_active":"2024-04-25T00:12:19.194014+0000","last_became_peered":"2024-04-25T00:12:19.194014+0000","last_unstale":"2024-04-25T00:12:19.200653+0000","last_undegraded":"2024-04-25T00:12:19.200653+0000","last_fullsized":"2024-04-25T00:12:19.200653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:39.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.214347+0000","last_change":"2024-04-25T00:12:18.199830+0000","last_active":"2024-04-25T00:12:19.214347+0000","last_peered":"2024-04-25T00:12:19.214347+0000","last_clean":"2024-04-25T00:12:19.214347+0000","last_became_active":"2024-04-25T00:12:18.196815+0000","last_became_peered":"2024-04-25T00:12:18.196815+0000","last_unstale":"2024-04-25T00:12:19.214347+0000","last_undegraded":"2024-04-25T00:12:19.214347+0000","last_fullsized":"2024-04-25T00:12:19.214347+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:53.040476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.210127+0000","last_change":"2024-04-25T00:12:19.205500+0000","last_active":"2024-04-25T00:12:19.210127+0000","last_peered":"2024-04-25T00:12:19.210127+0000","last_clean":"2024-04-25T00:12:19.210127+0000","last_became_active":"2024-04-25T00:12:19.204988+0000","last_became_peered":"2024-04-25T00:12:19.204988+0000","last_unstale":"2024-04-25T00:12:19.210127+0000","last_undegraded":"2024-04-25T00:12:19.210127+0000","last_fullsized":"2024-04-25T00:12:19.210127+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:32.234020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.218749+0000","last_change":"2024-04-25T00:12:18.197636+0000","last_active":"2024-04-25T00:12:19.218749+0000","last_peered":"2024-04-25T00:12:19.218749+0000","last_clean":"2024-04-25T00:12:19.218749+0000","last_became_active":"2024-04-25T00:12:18.197386+0000","last_became_peered":"2024-04-25T00:12:18.197386+0000","last_unstale":"2024-04-25T00:12:19.218749+0000","last_undegraded":"2024-04-25T00:12:19.218749+0000","last_fullsized":"2024-04-25T00:12:19.218749+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:44.142036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.216059+0000","last_change":"2024-04-25T00:12:19.212366+0000","last_active":"2024-04-25T00:12:19.216059+0000","last_peered":"2024-04-25T00:12:19.216059+0000","last_clean":"2024-04-25T00:12:19.216059+0000","last_became_active":"2024-04-25T00:12:19.212101+0000","last_became_peered":"2024-04-25T00:12:19.212101+0000","last_unstale":"2024-04-25T00:12:19.216059+0000","last_undegraded":"2024-04-25T00:12:19.216059+0000","last_fullsized":"2024-04-25T00:12:19.216059+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:55.625707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.182436+0000","last_change":"2024-04-25T00:12:18.196175+0000","last_active":"2024-04-25T00:12:19.182436+0000","last_peered":"2024-04-25T00:12:19.182436+0000","last_clean":"2024-04-25T00:12:19.182436+0000","last_became_active":"2024-04-25T00:12:18.195845+0000","last_became_peered":"2024-04-25T00:12:18.195845+0000","last_unstale":"2024-04-25T00:12:19.182436+0000","last_undegraded":"2024-04-25T00:12:19.182436+0000","last_fullsized":"2024-04-25T00:12:19.182436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:41.759894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.205957+0000","last_change":"2024-04-25T00:12:19.199629+0000","last_active":"2024-04-25T00:12:19.205957+0000","last_peered":"2024-04-25T00:12:19.205957+0000","last_clean":"2024-04-25T00:12:19.205957+0000","last_became_active":"2024-04-25T00:12:19.199394+0000","last_became_peered":"2024-04-25T00:12:19.199394+0000","last_unstale":"2024-04-25T00:12:19.205957+0000","last_undegraded":"2024-04-25T00:12:19.205957+0000","last_fullsized":"2024-04-25T00:12:19.205957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:54.297255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.187567+0000","last_change":"2024-04-25T00:12:18.195857+0000","last_active":"2024-04-25T00:12:19.187567+0000","last_peered":"2024-04-25T00:12:19.187567+0000","last_clean":"2024-04-25T00:12:19.187567+0000","last_became_active":"2024-04-25T00:12:18.194730+0000","last_became_peered":"2024-04-25T00:12:18.194730+0000","last_unstale":"2024-04-25T00:12:19.187567+0000","last_undegraded":"2024-04-25T00:12:19.187567+0000","last_fullsized":"2024-04-25T00:12:19.187567+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:15.505830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.206778+0000","last_change":"2024-04-25T00:12:19.203206+0000","last_active":"2024-04-25T00:12:19.206778+0000","last_peered":"2024-04-25T00:12:19.206778+0000","last_clean":"2024-04-25T00:12:19.206778+0000","last_became_active":"2024-04-25T00:12:19.201288+0000","last_became_peered":"2024-04-25T00:12:19.201288+0000","last_unstale":"2024-04-25T00:12:19.206778+0000","last_undegraded":"2024-04-25T00:12:19.206778+0000","last_fullsized":"2024-04-25T00:12:19.206778+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:42.060401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.179910+0000","last_change":"2024-04-25T00:12:18.192334+0000","last_active":"2024-04-25T00:12:19.179910+0000","last_peered":"2024-04-25T00:12:19.179910+0000","last_clean":"2024-04-25T00:12:19.179910+0000","last_became_active":"2024-04-25T00:12:18.192051+0000","last_became_peered":"2024-04-25T00:12:18.192051+0000","last_unstale":"2024-04-25T00:12:19.179910+0000","last_undegraded":"2024-04-25T00:12:19.179910+0000","last_fullsized":"2024-04-25T00:12:19.179910+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:31.227722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.208746+0000","last_change":"2024-04-25T00:12:19.197804+0000","last_active":"2024-04-25T00:12:19.208746+0000","last_peered":"2024-04-25T00:12:19.208746+0000","last_clean":"2024-04-25T00:12:19.208746+0000","last_became_active":"2024-04-25T00:12:19.196950+0000","last_became_peered":"2024-04-25T00:12:19.196950+0000","last_unstale":"2024-04-25T00:12:19.208746+0000","last_undegraded":"2024-04-25T00:12:19.208746+0000","last_fullsized":"2024-04-25T00:12:19.208746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:26.110172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.190601+0000","last_change":"2024-04-25T00:12:18.199725+0000","last_active":"2024-04-25T00:12:19.190601+0000","last_peered":"2024-04-25T00:12:19.190601+0000","last_clean":"2024-04-25T00:12:19.190601+0000","last_became_active":"2024-04-25T00:12:18.196685+0000","last_became_peered":"2024-04-25T00:12:18.196685+0000","last_unstale":"2024-04-25T00:12:19.190601+0000","last_undegraded":"2024-04-25T00:12:19.190601+0000","last_fullsized":"2024-04-25T00:12:19.190601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:50.829524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.204957+0000","last_change":"2024-04-25T00:12:19.199143+0000","last_active":"2024-04-25T00:12:19.204957+0000","last_peered":"2024-04-25T00:12:19.204957+0000","last_clean":"2024-04-25T00:12:19.204957+0000","last_became_active":"2024-04-25T00:12:19.198850+0000","last_became_peered":"2024-04-25T00:12:19.198850+0000","last_unstale":"2024-04-25T00:12:19.204957+0000","last_undegraded":"2024-04-25T00:12:19.204957+0000","last_fullsized":"2024-04-25T00:12:19.204957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:45.148333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.220007+0000","last_change":"2024-04-25T00:12:18.197529+0000","last_active":"2024-04-25T00:12:19.220007+0000","last_peered":"2024-04-25T00:12:19.220007+0000","last_clean":"2024-04-25T00:12:19.220007+0000","last_became_active":"2024-04-25T00:12:18.197265+0000","last_became_peered":"2024-04-25T00:12:18.197265+0000","last_unstale":"2024-04-25T00:12:19.220007+0000","last_undegraded":"2024-04-25T00:12:19.220007+0000","last_fullsized":"2024-04-25T00:12:19.220007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:11.778497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.211043+0000","last_change":"2024-04-25T00:12:19.206496+0000","last_active":"2024-04-25T00:12:19.211043+0000","last_peered":"2024-04-25T00:12:19.211043+0000","last_clean":"2024-04-25T00:12:19.211043+0000","last_became_active":"2024-04-25T00:12:19.206233+0000","last_became_peered":"2024-04-25T00:12:19.206233+0000","last_unstale":"2024-04-25T00:12:19.211043+0000","last_undegraded":"2024-04-25T00:12:19.211043+0000","last_fullsized":"2024-04-25T00:12:19.211043+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:42.766192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.222844+0000","last_change":"2024-04-25T00:12:18.183408+0000","last_active":"2024-04-25T00:12:19.222844+0000","last_peered":"2024-04-25T00:12:19.222844+0000","last_clean":"2024-04-25T00:12:19.222844+0000","last_became_active":"2024-04-25T00:12:18.183090+0000","last_became_peered":"2024-04-25T00:12:18.183090+0000","last_unstale":"2024-04-25T00:12:19.222844+0000","last_undegraded":"2024-04-25T00:12:19.222844+0000","last_fullsized":"2024-04-25T00:12:19.222844+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:58.840397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.211965+0000","last_change":"2024-04-25T00:12:19.207118+0000","last_active":"2024-04-25T00:12:19.211965+0000","last_peered":"2024-04-25T00:12:19.211965+0000","last_clean":"2024-04-25T00:12:19.211965+0000","last_became_active":"2024-04-25T00:12:19.206483+0000","last_became_peered":"2024-04-25T00:12:19.206483+0000","last_unstale":"2024-04-25T00:12:19.211965+0000","last_undegraded":"2024-04-25T00:12:19.211965+0000","last_fullsized":"2024-04-25T00:12:19.211965+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:55.625707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.213299+0000","last_change":"2024-04-25T00:12:18.194759+0000","last_active":"2024-04-25T00:12:19.213299+0000","last_peered":"2024-04-25T00:12:19.213299+0000","last_clean":"2024-04-25T00:12:19.213299+0000","last_became_active":"2024-04-25T00:12:18.194483+0000","last_became_peered":"2024-04-25T00:12:18.194483+0000","last_unstale":"2024-04-25T00:12:19.213299+0000","last_undegraded":"2024-04-25T00:12:19.213299+0000","last_fullsized":"2024-04-25T00:12:19.213299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:51.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.204992+0000","last_change":"2024-04-25T00:12:19.198615+0000","last_active":"2024-04-25T00:12:19.204992+0000","last_peered":"2024-04-25T00:12:19.204992+0000","last_clean":"2024-04-25T00:12:19.204992+0000","last_became_active":"2024-04-25T00:12:19.198052+0000","last_became_peered":"2024-04-25T00:12:19.198052+0000","last_unstale":"2024-04-25T00:12:19.204992+0000","last_undegraded":"2024-04-25T00:12:19.204992+0000","last_fullsized":"2024-04-25T00:12:19.204992+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:59.322277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.181867+0000","last_change":"2024-04-25T00:12:18.200632+0000","last_active":"2024-04-25T00:12:19.181867+0000","last_peered":"2024-04-25T00:12:19.181867+0000","last_clean":"2024-04-25T00:12:19.181867+0000","last_became_active":"2024-04-25T00:12:18.200339+0000","last_became_peered":"2024-04-25T00:12:18.200339+0000","last_unstale":"2024-04-25T00:12:19.181867+0000","last_undegraded":"2024-04-25T00:12:19.181867+0000","last_fullsized":"2024-04-25T00:12:19.181867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:11.170827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.207239+0000","last_change":"2024-04-25T00:12:19.195117+0000","last_active":"2024-04-25T00:12:19.207239+0000","last_peered":"2024-04-25T00:12:19.207239+0000","last_clean":"2024-04-25T00:12:19.207239+0000","last_became_active":"2024-04-25T00:12:19.194759+0000","last_became_peered":"2024-04-25T00:12:19.194759+0000","last_unstale":"2024-04-25T00:12:19.207239+0000","last_undegraded":"2024-04-25T00:12:19.207239+0000","last_fullsized":"2024-04-25T00:12:19.207239+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:30.684139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.176894+0000","last_change":"2024-04-25T00:12:18.196345+0000","last_active":"2024-04-25T00:12:19.176894+0000","last_peered":"2024-04-25T00:12:19.176894+0000","last_clean":"2024-04-25T00:12:19.176894+0000","last_became_active":"2024-04-25T00:12:18.196026+0000","last_became_peered":"2024-04-25T00:12:18.196026+0000","last_unstale":"2024-04-25T00:12:19.176894+0000","last_undegraded":"2024-04-25T00:12:19.176894+0000","last_fullsized":"2024-04-25T00:12:19.176894+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:44.748128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.204440+0000","last_change":"2024-04-25T00:12:19.198292+0000","last_active":"2024-04-25T00:12:19.204440+0000","last_peered":"2024-04-25T00:12:19.204440+0000","last_clean":"2024-04-25T00:12:19.204440+0000","last_became_active":"2024-04-25T00:12:19.197980+0000","last_became_peered":"2024-04-25T00:12:19.197980+0000","last_unstale":"2024-04-25T00:12:19.204440+0000","last_undegraded":"2024-04-25T00:12:19.204440+0000","last_fullsized":"2024-04-25T00:12:19.204440+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:50.934891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.220633+0000","last_change":"2024-04-25T00:12:18.195596+0000","last_active":"2024-04-25T00:12:19.220633+0000","last_peered":"2024-04-25T00:12:19.220633+0000","last_clean":"2024-04-25T00:12:19.220633+0000","last_became_active":"2024-04-25T00:12:18.194339+0000","last_became_peered":"2024-04-25T00:12:18.194339+0000","last_unstale":"2024-04-25T00:12:19.220633+0000","last_undegraded":"2024-04-25T00:12:19.220633+0000","last_fullsized":"2024-04-25T00:12:19.220633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:53.040476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.200712+0000","last_change":"2024-04-25T00:12:19.187654+0000","last_active":"2024-04-25T00:12:19.200712+0000","last_peered":"2024-04-25T00:12:19.200712+0000","last_clean":"2024-04-25T00:12:19.200712+0000","last_became_active":"2024-04-25T00:12:19.187432+0000","last_became_peered":"2024-04-25T00:12:19.187432+0000","last_unstale":"2024-04-25T00:12:19.200712+0000","last_undegraded":"2024-04-25T00:12:19.200712+0000","last_fullsized":"2024-04-25T00:12:19.200712+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:37:13.047475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.219455+0000","last_change":"2024-04-25T00:12:18.200832+0000","last_active":"2024-04-25T00:12:19.219455+0000","last_peered":"2024-04-25T00:12:19.219455+0000","last_clean":"2024-04-25T00:12:19.219455+0000","last_became_active":"2024-04-25T00:12:18.200533+0000","last_became_peered":"2024-04-25T00:12:18.200533+0000","last_unstale":"2024-04-25T00:12:19.219455+0000","last_undegraded":"2024-04-25T00:12:19.219455+0000","last_fullsized":"2024-04-25T00:12:19.219455+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:54.619409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.215241+0000","last_change":"2024-04-25T00:12:19.203611+0000","last_active":"2024-04-25T00:12:19.215241+0000","last_peered":"2024-04-25T00:12:19.215241+0000","last_clean":"2024-04-25T00:12:19.215241+0000","last_became_active":"2024-04-25T00:12:19.203283+0000","last_became_peered":"2024-04-25T00:12:19.203283+0000","last_unstale":"2024-04-25T00:12:19.215241+0000","last_undegraded":"2024-04-25T00:12:19.215241+0000","last_fullsized":"2024-04-25T00:12:19.215241+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:26.110172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.220378+0000","last_change":"2024-04-25T00:12:18.199437+0000","last_active":"2024-04-25T00:12:19.220378+0000","last_peered":"2024-04-25T00:12:19.220378+0000","last_clean":"2024-04-25T00:12:19.220378+0000","last_became_active":"2024-04-25T00:12:18.199140+0000","last_became_peered":"2024-04-25T00:12:18.199140+0000","last_unstale":"2024-04-25T00:12:19.220378+0000","last_undegraded":"2024-04-25T00:12:19.220378+0000","last_fullsized":"2024-04-25T00:12:19.220378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:46.181569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.215975+0000","last_change":"2024-04-25T00:12:19.212030+0000","last_active":"2024-04-25T00:12:19.215975+0000","last_peered":"2024-04-25T00:12:19.215975+0000","last_clean":"2024-04-25T00:12:19.215975+0000","last_became_active":"2024-04-25T00:12:19.207986+0000","last_became_peered":"2024-04-25T00:12:19.207986+0000","last_unstale":"2024-04-25T00:12:19.215975+0000","last_undegraded":"2024-04-25T00:12:19.215975+0000","last_fullsized":"2024-04-25T00:12:19.215975+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:54.317722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.211293+0000","last_change":"2024-04-25T00:12:18.197059+0000","last_active":"2024-04-25T00:12:19.211293+0000","last_peered":"2024-04-25T00:12:19.211293+0000","last_clean":"2024-04-25T00:12:19.211293+0000","last_became_active":"2024-04-25T00:12:18.196783+0000","last_became_peered":"2024-04-25T00:12:18.196783+0000","last_unstale":"2024-04-25T00:12:19.211293+0000","last_undegraded":"2024-04-25T00:12:19.211293+0000","last_fullsized":"2024-04-25T00:12:19.211293+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:02:57.968512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.191460+0000","last_change":"2024-04-25T00:12:19.188997+0000","last_active":"2024-04-25T00:12:19.191460+0000","last_peered":"2024-04-25T00:12:19.191460+0000","last_clean":"2024-04-25T00:12:19.191460+0000","last_became_active":"2024-04-25T00:12:19.188631+0000","last_became_peered":"2024-04-25T00:12:19.188631+0000","last_unstale":"2024-04-25T00:12:19.191460+0000","last_undegraded":"2024-04-25T00:12:19.191460+0000","last_fullsized":"2024-04-25T00:12:19.191460+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:16.933678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.182143+0000","last_change":"2024-04-25T00:12:18.194583+0000","last_active":"2024-04-25T00:12:19.182143+0000","last_peered":"2024-04-25T00:12:19.182143+0000","last_clean":"2024-04-25T00:12:19.182143+0000","last_became_active":"2024-04-25T00:12:18.194237+0000","last_became_peered":"2024-04-25T00:12:18.194237+0000","last_unstale":"2024-04-25T00:12:19.182143+0000","last_undegraded":"2024-04-25T00:12:19.182143+0000","last_fullsized":"2024-04-25T00:12:19.182143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:44.142036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.211894+0000","last_change":"2024-04-25T00:12:19.203056+0000","last_active":"2024-04-25T00:12:19.211894+0000","last_peered":"2024-04-25T00:12:19.211894+0000","last_clean":"2024-04-25T00:12:19.211894+0000","last_became_active":"2024-04-25T00:12:19.202412+0000","last_became_peered":"2024-04-25T00:12:19.202412+0000","last_unstale":"2024-04-25T00:12:19.211894+0000","last_undegraded":"2024-04-25T00:12:19.211894+0000","last_fullsized":"2024-04-25T00:12:19.211894+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:15.513443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.206426+0000","last_change":"2024-04-25T00:12:18.195116+0000","last_active":"2024-04-25T00:12:19.206426+0000","last_peered":"2024-04-25T00:12:19.206426+0000","last_clean":"2024-04-25T00:12:19.206426+0000","last_became_active":"2024-04-25T00:12:18.194535+0000","last_became_peered":"2024-04-25T00:12:18.194535+0000","last_unstale":"2024-04-25T00:12:19.206426+0000","last_undegraded":"2024-04-25T00:12:19.206426+0000","last_fullsized":"2024-04-25T00:12:19.206426+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:23.278925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.210181+0000","last_change":"2024-04-25T00:12:19.203384+0000","last_active":"2024-04-25T00:12:19.210181+0000","last_peered":"2024-04-25T00:12:19.210181+0000","last_clean":"2024-04-25T00:12:19.210181+0000","last_became_active":"2024-04-25T00:12:19.203106+0000","last_became_peered":"2024-04-25T00:12:19.203106+0000","last_unstale":"2024-04-25T00:12:19.210181+0000","last_undegraded":"2024-04-25T00:12:19.210181+0000","last_fullsized":"2024-04-25T00:12:19.210181+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:42.766192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.176385+0000","last_change":"2024-04-25T00:12:18.197429+0000","last_active":"2024-04-25T00:12:19.176385+0000","last_peered":"2024-04-25T00:12:19.176385+0000","last_clean":"2024-04-25T00:12:19.176385+0000","last_became_active":"2024-04-25T00:12:18.197059+0000","last_became_peered":"2024-04-25T00:12:18.197059+0000","last_unstale":"2024-04-25T00:12:19.176385+0000","last_undegraded":"2024-04-25T00:12:19.176385+0000","last_fullsized":"2024-04-25T00:12:19.176385+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:34.858621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.209059+0000","last_change":"2024-04-25T00:12:19.204504+0000","last_active":"2024-04-25T00:12:19.209059+0000","last_peered":"2024-04-25T00:12:19.209059+0000","last_clean":"2024-04-25T00:12:19.209059+0000","last_became_active":"2024-04-25T00:12:19.204208+0000","last_became_peered":"2024-04-25T00:12:19.204208+0000","last_unstale":"2024-04-25T00:12:19.209059+0000","last_undegraded":"2024-04-25T00:12:19.209059+0000","last_fullsized":"2024-04-25T00:12:19.209059+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:59.846695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.181371+0000","last_change":"2024-04-25T00:12:18.192631+0000","last_active":"2024-04-25T00:12:19.181371+0000","last_peered":"2024-04-25T00:12:19.181371+0000","last_clean":"2024-04-25T00:12:19.181371+0000","last_became_active":"2024-04-25T00:12:18.192356+0000","last_became_peered":"2024-04-25T00:12:18.192356+0000","last_unstale":"2024-04-25T00:12:19.181371+0000","last_undegraded":"2024-04-25T00:12:19.181371+0000","last_fullsized":"2024-04-25T00:12:19.181371+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:41.759894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.206036+0000","last_change":"2024-04-25T00:12:19.199805+0000","last_active":"2024-04-25T00:12:19.206036+0000","last_peered":"2024-04-25T00:12:19.206036+0000","last_clean":"2024-04-25T00:12:19.206036+0000","last_became_active":"2024-04-25T00:12:19.199424+0000","last_became_peered":"2024-04-25T00:12:19.199424+0000","last_unstale":"2024-04-25T00:12:19.206036+0000","last_undegraded":"2024-04-25T00:12:19.206036+0000","last_fullsized":"2024-04-25T00:12:19.206036+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:22.932843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.176325+0000","last_change":"2024-04-25T00:12:18.191443+0000","last_active":"2024-04-25T00:12:19.176325+0000","last_peered":"2024-04-25T00:12:19.176325+0000","last_clean":"2024-04-25T00:12:19.176325+0000","last_became_active":"2024-04-25T00:12:18.191178+0000","last_became_peered":"2024-04-25T00:12:18.191178+0000","last_unstale":"2024-04-25T00:12:19.176325+0000","last_undegraded":"2024-04-25T00:12:19.176325+0000","last_fullsized":"2024-04-25T00:12:19.176325+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:53.804182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.212708+0000","last_change":"2024-04-25T00:12:19.207351+0000","last_active":"2024-04-25T00:12:19.212708+0000","last_peered":"2024-04-25T00:12:19.212708+0000","last_clean":"2024-04-25T00:12:19.212708+0000","last_became_active":"2024-04-25T00:12:19.207118+0000","last_became_peered":"2024-04-25T00:12:19.207118+0000","last_unstale":"2024-04-25T00:12:19.212708+0000","last_undegraded":"2024-04-25T00:12:19.212708+0000","last_fullsized":"2024-04-25T00:12:19.212708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:51.835822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.219564+0000","last_change":"2024-04-25T00:12:18.196809+0000","last_active":"2024-04-25T00:12:19.219564+0000","last_peered":"2024-04-25T00:12:19.219564+0000","last_clean":"2024-04-25T00:12:19.219564+0000","last_became_active":"2024-04-25T00:12:18.196483+0000","last_became_peered":"2024-04-25T00:12:18.196483+0000","last_unstale":"2024-04-25T00:12:19.219564+0000","last_undegraded":"2024-04-25T00:12:19.219564+0000","last_fullsized":"2024-04-25T00:12:19.219564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:29.677841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.206775+0000","last_change":"2024-04-25T00:12:19.195807+0000","last_active":"2024-04-25T00:12:19.206775+0000","last_peered":"2024-04-25T00:12:19.206775+0000","last_clean":"2024-04-25T00:12:19.206775+0000","last_became_active":"2024-04-25T00:12:19.195424+0000","last_became_peered":"2024-04-25T00:12:19.195424+0000","last_unstale":"2024-04-25T00:12:19.206775+0000","last_undegraded":"2024-04-25T00:12:19.206775+0000","last_fullsized":"2024-04-25T00:12:19.206775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:12.784794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.176113+0000","last_change":"2024-04-25T00:12:18.198414+0000","last_active":"2024-04-25T00:12:19.176113+0000","last_peered":"2024-04-25T00:12:19.176113+0000","last_clean":"2024-04-25T00:12:19.176113+0000","last_became_active":"2024-04-25T00:12:18.198120+0000","last_became_peered":"2024-04-25T00:12:18.198120+0000","last_unstale":"2024-04-25T00:12:19.176113+0000","last_undegraded":"2024-04-25T00:12:19.176113+0000","last_fullsized":"2024-04-25T00:12:19.176113+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:46.460272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.213192+0000","last_change":"2024-04-25T00:12:19.210555+0000","last_active":"2024-04-25T00:12:19.213192+0000","last_peered":"2024-04-25T00:12:19.213192+0000","last_clean":"2024-04-25T00:12:19.213192+0000","last_became_active":"2024-04-25T00:12:19.208885+0000","last_became_peered":"2024-04-25T00:12:19.208885+0000","last_unstale":"2024-04-25T00:12:19.213192+0000","last_undegraded":"2024-04-25T00:12:19.213192+0000","last_fullsized":"2024-04-25T00:12:19.213192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:38.281454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.189016+0000","last_change":"2024-04-25T00:12:18.197831+0000","last_active":"2024-04-25T00:12:19.189016+0000","last_peered":"2024-04-25T00:12:19.189016+0000","last_clean":"2024-04-25T00:12:19.189016+0000","last_became_active":"2024-04-25T00:12:18.197504+0000","last_became_peered":"2024-04-25T00:12:18.197504+0000","last_unstale":"2024-04-25T00:12:19.189016+0000","last_undegraded":"2024-04-25T00:12:19.189016+0000","last_fullsized":"2024-04-25T00:12:19.189016+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:59.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.211945+0000","last_change":"2024-04-25T00:12:19.205314+0000","last_active":"2024-04-25T00:12:19.211945+0000","last_peered":"2024-04-25T00:12:19.211945+0000","last_clean":"2024-04-25T00:12:19.211945+0000","last_became_active":"2024-04-25T00:12:19.205043+0000","last_became_peered":"2024-04-25T00:12:19.205043+0000","last_unstale":"2024-04-25T00:12:19.211945+0000","last_undegraded":"2024-04-25T00:12:19.211945+0000","last_fullsized":"2024-04-25T00:12:19.211945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:54.317722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.189637+0000","last_change":"2024-04-25T00:12:18.197556+0000","last_active":"2024-04-25T00:12:19.189637+0000","last_peered":"2024-04-25T00:12:19.189637+0000","last_clean":"2024-04-25T00:12:19.189637+0000","last_became_active":"2024-04-25T00:12:18.197268+0000","last_became_peered":"2024-04-25T00:12:18.197268+0000","last_unstale":"2024-04-25T00:12:19.189637+0000","last_undegraded":"2024-04-25T00:12:19.189637+0000","last_fullsized":"2024-04-25T00:12:19.189637+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:46.181569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.198533+0000","last_change":"2024-04-25T00:12:19.192310+0000","last_active":"2024-04-25T00:12:19.198533+0000","last_peered":"2024-04-25T00:12:19.198533+0000","last_clean":"2024-04-25T00:12:19.198533+0000","last_became_active":"2024-04-25T00:12:19.191888+0000","last_became_peered":"2024-04-25T00:12:19.191888+0000","last_unstale":"2024-04-25T00:12:19.198533+0000","last_undegraded":"2024-04-25T00:12:19.198533+0000","last_fullsized":"2024-04-25T00:12:19.198533+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:55.625707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.176533+0000","last_change":"2024-04-25T00:12:18.192181+0000","last_active":"2024-04-25T00:12:19.176533+0000","last_peered":"2024-04-25T00:12:19.176533+0000","last_clean":"2024-04-25T00:12:19.176533+0000","last_became_active":"2024-04-25T00:12:18.191908+0000","last_became_peered":"2024-04-25T00:12:18.191908+0000","last_unstale":"2024-04-25T00:12:19.176533+0000","last_undegraded":"2024-04-25T00:12:19.176533+0000","last_fullsized":"2024-04-25T00:12:19.176533+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:15.505830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.216038+0000","last_change":"2024-04-25T00:12:19.212224+0000","last_active":"2024-04-25T00:12:19.216038+0000","last_peered":"2024-04-25T00:12:19.216038+0000","last_clean":"2024-04-25T00:12:19.216038+0000","last_became_active":"2024-04-25T00:12:19.209567+0000","last_became_peered":"2024-04-25T00:12:19.209567+0000","last_unstale":"2024-04-25T00:12:19.216038+0000","last_undegraded":"2024-04-25T00:12:19.216038+0000","last_fullsized":"2024-04-25T00:12:19.216038+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:54.046773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.188807+0000","last_change":"2024-04-25T00:12:18.196698+0000","last_active":"2024-04-25T00:12:19.188807+0000","last_peered":"2024-04-25T00:12:19.188807+0000","last_clean":"2024-04-25T00:12:19.188807+0000","last_became_active":"2024-04-25T00:12:18.196413+0000","last_became_peered":"2024-04-25T00:12:18.196413+0000","last_unstale":"2024-04-25T00:12:19.188807+0000","last_undegraded":"2024-04-25T00:12:19.188807+0000","last_fullsized":"2024-04-25T00:12:19.188807+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:59.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.202792+0000","last_change":"2024-04-25T00:12:19.195164+0000","last_active":"2024-04-25T00:12:19.202792+0000","last_peered":"2024-04-25T00:12:19.202792+0000","last_clean":"2024-04-25T00:12:19.202792+0000","last_became_active":"2024-04-25T00:12:19.194892+0000","last_became_peered":"2024-04-25T00:12:19.194892+0000","last_unstale":"2024-04-25T00:12:19.202792+0000","last_undegraded":"2024-04-25T00:12:19.202792+0000","last_fullsized":"2024-04-25T00:12:19.202792+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:51.835822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.179458+0000","last_change":"2024-04-25T00:12:18.192155+0000","last_active":"2024-04-25T00:12:19.179458+0000","last_peered":"2024-04-25T00:12:19.179458+0000","last_clean":"2024-04-25T00:12:19.179458+0000","last_became_active":"2024-04-25T00:12:18.191865+0000","last_became_peered":"2024-04-25T00:12:18.191865+0000","last_unstale":"2024-04-25T00:12:19.179458+0000","last_undegraded":"2024-04-25T00:12:19.179458+0000","last_fullsized":"2024-04-25T00:12:19.179458+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:14.507145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.219861+0000","last_change":"2024-04-25T00:12:19.214000+0000","last_active":"2024-04-25T00:12:19.219861+0000","last_peered":"2024-04-25T00:12:19.219861+0000","last_clean":"2024-04-25T00:12:19.219861+0000","last_became_active":"2024-04-25T00:12:19.212643+0000","last_became_peered":"2024-04-25T00:12:19.212643+0000","last_unstale":"2024-04-25T00:12:19.219861+0000","last_undegraded":"2024-04-25T00:12:19.219861+0000","last_fullsized":"2024-04-25T00:12:19.219861+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:18.161354+0000","last_clean_scrub_stamp":"2024-04-25T00:12:18.161354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:50.934891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.182179+0000","last_change":"2024-04-25T00:12:18.194533+0000","last_active":"2024-04-25T00:12:19.182179+0000","last_peered":"2024-04-25T00:12:19.182179+0000","last_clean":"2024-04-25T00:12:19.182179+0000","last_became_active":"2024-04-25T00:12:18.194243+0000","last_became_peered":"2024-04-25T00:12:18.194243+0000","last_unstale":"2024-04-25T00:12:19.182179+0000","last_undegraded":"2024-04-25T00:12:19.182179+0000","last_fullsized":"2024-04-25T00:12:19.182179+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:17.155057+0000","last_clean_scrub_stamp":"2024-04-25T00:12:17.155057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:53.311425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":200,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T00:12:19.184505+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:12:19.184505+0000","last_peered":"2024-04-25T00:12:19.184505+0000","last_clean":"2024-04-25T00:12:19.184505+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:12:19.184505+0000","last_undegraded":"2024-04-25T00:12:19.184505+0000","last_fullsized":"2024-04-25T00:12:19.184505+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:12:23.643 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:12:23.643 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:12:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.822+0000 7fd2ba000640 1 Processor -- start 2024-04-25T00:12:23.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.822+0000 7fd2ba000640 1 -- start start 2024-04-25T00:12:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.822+0000 7fd2ba000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd2b406ebd0 0x7fd2b415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.822+0000 7fd2ba000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2b415c560 con 0x7fd2b41555c0 2024-04-25T00:12:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.822+0000 7fd2ba000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2b4072460 con 0x7fd2b41551e0 2024-04-25T00:12:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.822+0000 7fd2ba000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd2b415c7d0 con 0x7fd2b406ebd0 2024-04-25T00:12:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.822+0000 7fd2b2a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd2b41555c0 0x7fd2b406e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56976/0 (socket says 172.21.15.83:56976) 2024-04-25T00:12:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.822+0000 7fd2b2a00640 1 -- 172.21.15.83:0/2748822143 learned_addr learned my addr 172.21.15.83:0/2748822143 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.822+0000 7fd2b3e00640 1 --2- 172.21.15.83:0/2748822143 >> v2:172.21.15.86:3300/0 conn(0x7fd2b406ebd0 0x7fd2b415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.822+0000 7fd2b2000640 1 -- 172.21.15.83:0/2748822143 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531753974 0 0) 0x7fd2b415c560 con 0x7fd2b41555c0 2024-04-25T00:12:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.822+0000 7fd2b2000640 1 -- 172.21.15.83:0/2748822143 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd298005740 con 0x7fd2b41555c0 2024-04-25T00:12:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.822+0000 7fd2b3e00640 1 -- 172.21.15.83:0/2748822143 >> v1:172.21.15.86:6789/0 conn(0x7fd2b41551e0 legacy=0x7fd2b4071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.822+0000 7fd2b3e00640 1 -- 172.21.15.83:0/2748822143 >> v1:172.21.15.83:6789/0 conn(0x7fd2b41555c0 legacy=0x7fd2b406e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.822+0000 7fd2b3e00640 1 -- 172.21.15.83:0/2748822143 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b415cab0 con 0x7fd2b406ebd0 2024-04-25T00:12:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.826+0000 7fd2b3e00640 1 --2- 172.21.15.83:0/2748822143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b406ebd0 0x7fd2b415bea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0097f0 tx=0x7fd2ac01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e34ebc677f2f27b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.826+0000 7fd2b2000640 1 -- 172.21.15.83:0/2748822143 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0b0070 con 0x7fd2b406ebd0 2024-04-25T00:12:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.826+0000 7fd2b2000640 1 -- 172.21.15.83:0/2748822143 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac0ab600 con 0x7fd2b406ebd0 2024-04-25T00:12:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.826+0000 7fd2b2000640 1 -- 172.21.15.83:0/2748822143 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0ab9d0 con 0x7fd2b406ebd0 2024-04-25T00:12:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.826+0000 7fd2ba000640 1 -- 172.21.15.83:0/2748822143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b406ebd0 msgr2=0x7fd2b415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.826+0000 7fd2ba000640 1 --2- 172.21.15.83:0/2748822143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b406ebd0 0x7fd2b415bea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0097f0 tx=0x7fd2ac01c410 comp rx=0 tx=0).stop 2024-04-25T00:12:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.826+0000 7fd2ba000640 1 -- 172.21.15.83:0/2748822143 shutdown_connections 2024-04-25T00:12:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.826+0000 7fd2ba000640 1 --2- 172.21.15.83:0/2748822143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b406ebd0 0x7fd2b415bea0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.826+0000 7fd2ba000640 1 -- 172.21.15.83:0/2748822143 >> 172.21.15.83:0/2748822143 conn(0x7fd2b40b1830 msgr2=0x7fd2b40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.826+0000 7fd2ba000640 1 -- 172.21.15.83:0/2748822143 shutdown_connections 2024-04-25T00:12:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.826+0000 7fd2ba000640 1 -- 172.21.15.83:0/2748822143 wait complete. 2024-04-25T00:12:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2ba000640 1 Processor -- start 2024-04-25T00:12:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2ba000640 1 -- start start 2024-04-25T00:12:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2ba000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2b406ebd0 0x7fd2b40cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2ba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b41551e0 0x7fd2b4145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2ba000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd2b41555c0 0x7fd2b414f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2ba000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b406dfd0 con 0x7fd2b406ebd0 2024-04-25T00:12:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2ba000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b406de50 con 0x7fd2b41551e0 2024-04-25T00:12:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2ba000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b406e150 con 0x7fd2b41555c0 2024-04-25T00:12:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2b406ebd0 0x7fd2b40cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2b406ebd0 0x7fd2b40cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34470/0 (socket says 172.21.15.83:34470) 2024-04-25T00:12:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2b3400640 1 -- 172.21.15.83:0/3761405092 learned_addr learned my addr 172.21.15.83:0/3761405092 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2b2a00640 1 --2- 172.21.15.83:0/3761405092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b41551e0 0x7fd2b4145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2b3e00640 1 --2- 172.21.15.83:0/3761405092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd2b41555c0 0x7fd2b414f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2b3400640 1 -- 172.21.15.83:0/3761405092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd2b41555c0 msgr2=0x7fd2b414f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2b3400640 1 --2- 172.21.15.83:0/3761405092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd2b41555c0 0x7fd2b414f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2b3400640 1 -- 172.21.15.83:0/3761405092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b41551e0 msgr2=0x7fd2b4145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2b3400640 1 --2- 172.21.15.83:0/3761405092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b41551e0 0x7fd2b4145750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2b3400640 1 -- 172.21.15.83:0/3761405092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b414fa80 con 0x7fd2b406ebd0 2024-04-25T00:12:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2b3400640 1 --2- 172.21.15.83:0/3761405092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2b406ebd0 0x7fd2b40cdb30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400c820 tx=0x7fd2a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2ab400640 1 -- 172.21.15.83:0/3761405092 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4003b80 con 0x7fd2b406ebd0 2024-04-25T00:12:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2ab400640 1 -- 172.21.15.83:0/3761405092 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a4003d20 con 0x7fd2b406ebd0 2024-04-25T00:12:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2ab400640 1 -- 172.21.15.83:0/3761405092 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a400aa90 con 0x7fd2b406ebd0 2024-04-25T00:12:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2ba000640 1 -- 172.21.15.83:0/3761405092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b414fd60 con 0x7fd2b406ebd0 2024-04-25T00:12:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.830+0000 7fd2ba000640 1 -- 172.21.15.83:0/3761405092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b414f770 con 0x7fd2b406ebd0 2024-04-25T00:12:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.834+0000 7fd2ba000640 1 -- 172.21.15.83:0/3761405092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b404f510 con 0x7fd2b406ebd0 2024-04-25T00:12:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.834+0000 7fd2ab400640 1 -- 172.21.15.83:0/3761405092 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd2a400ac30 con 0x7fd2b406ebd0 2024-04-25T00:12:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.834+0000 7fd2ab400640 1 --2- 172.21.15.83:0/3761405092 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd2940678f0 0x7fd294069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.834+0000 7fd2ab400640 1 -- 172.21.15.83:0/3761405092 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fd2a4085d50 con 0x7fd2b406ebd0 2024-04-25T00:12:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.834+0000 7fd2b2a00640 1 --2- 172.21.15.83:0/3761405092 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd2940678f0 0x7fd294069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.834+0000 7fd2b2a00640 1 --2- 172.21.15.83:0/3761405092 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd2940678f0 0x7fd294069db0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4147ae0 tx=0x7fd2a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:23.838+0000 7fd2ab400640 1 -- 172.21.15.83:0/3761405092 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd2a40560a0 con 0x7fd2b406ebd0 2024-04-25T00:12:24.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.054+0000 7fd2ba000640 1 -- 172.21.15.83:0/3761405092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd2b4151af0 con 0x7fd2b406ebd0 2024-04-25T00:12:24.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.054+0000 7fd2ab400640 1 -- 172.21.15.83:0/3761405092 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7fd2a405a3a0 con 0x7fd2b406ebd0 2024-04-25T00:12:24.056 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-25T00:12:24.056 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylmOqooLBAApte+X6YkRmrpe12YdjYekg== 2024-04-25T00:12:24.056 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-25T00:12:24.057 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-25T00:12:24.057 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:12:24.057 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-25T00:12:24.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.058+0000 7fd2ba000640 1 -- 172.21.15.83:0/3761405092 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd2940678f0 msgr2=0x7fd294069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:24.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.058+0000 7fd2ba000640 1 --2- 172.21.15.83:0/3761405092 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd2940678f0 0x7fd294069db0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4147ae0 tx=0x7fd2a0009040 comp rx=0 tx=0).stop 2024-04-25T00:12:24.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.058+0000 7fd2ba000640 1 -- 172.21.15.83:0/3761405092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2b406ebd0 msgr2=0x7fd2b40cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:24.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.058+0000 7fd2ba000640 1 --2- 172.21.15.83:0/3761405092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2b406ebd0 0x7fd2b40cdb30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400c820 tx=0x7fd2a400cce0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.058+0000 7fd2ba000640 1 -- 172.21.15.83:0/3761405092 shutdown_connections 2024-04-25T00:12:24.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.058+0000 7fd2ba000640 1 --2- 172.21.15.83:0/3761405092 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd2940678f0 0x7fd294069db0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.058+0000 7fd2ba000640 1 --2- 172.21.15.83:0/3761405092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd2b41555c0 0x7fd2b414f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.058+0000 7fd2ba000640 1 --2- 172.21.15.83:0/3761405092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b41551e0 0x7fd2b4145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.058+0000 7fd2ba000640 1 --2- 172.21.15.83:0/3761405092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2b406ebd0 0x7fd2b40cdb30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.058+0000 7fd2ba000640 1 -- 172.21.15.83:0/3761405092 >> 172.21.15.83:0/3761405092 conn(0x7fd2b40b1830 msgr2=0x7fd2b4154850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.058+0000 7fd2ba000640 1 -- 172.21.15.83:0/3761405092 shutdown_connections 2024-04-25T00:12:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.058+0000 7fd2ba000640 1 -- 172.21.15.83:0/3761405092 wait complete. 2024-04-25T00:12:24.077 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:12:24.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.210+0000 7fd85d200640 1 Processor -- start 2024-04-25T00:12:24.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd85d200640 1 -- start start 2024-04-25T00:12:24.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd85d200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd8580b4360 0x7fd8580b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd85d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd8580b2b30 0x7fd8580b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd85d200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd85806ef80 con 0x7fd8581553e0 2024-04-25T00:12:24.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd85d200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd85815b510 con 0x7fd8580b4360 2024-04-25T00:12:24.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd85d200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd85815b7f0 con 0x7fd8580b2b30 2024-04-25T00:12:24.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd84fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd8580b4360 0x7fd8580b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd84fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd8580b4360 0x7fd8580b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34482/0 (socket says 172.21.15.83:34482) 2024-04-25T00:12:24.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd84fe00640 1 -- 172.21.15.83:0/2964208768 learned_addr learned my addr 172.21.15.83:0/2964208768 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:24.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd856800640 1 --2- 172.21.15.83:0/2964208768 >> v2:172.21.15.86:3300/0 conn(0x7fd8580b2b30 0x7fd8580b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd84fe00640 1 -- 172.21.15.83:0/2964208768 >> v2:172.21.15.86:3300/0 conn(0x7fd8580b2b30 msgr2=0x7fd8580b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:24.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd84fe00640 1 --2- 172.21.15.83:0/2964208768 >> v2:172.21.15.86:3300/0 conn(0x7fd8580b2b30 0x7fd8580b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd84fe00640 1 -- 172.21.15.83:0/2964208768 >> v1:172.21.15.86:6789/0 conn(0x7fd8581553e0 legacy=0x7fd85806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:24.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd84fe00640 1 -- 172.21.15.83:0/2964208768 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85815bad0 con 0x7fd8580b4360 2024-04-25T00:12:24.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.214+0000 7fd856800640 1 --2- 172.21.15.83:0/2964208768 >> v2:172.21.15.86:3300/0 conn(0x7fd8580b2b30 0x7fd8580b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:24.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.218+0000 7fd84fe00640 1 --2- 172.21.15.83:0/2964208768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8580b4360 0x7fd8580b47e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd858158fd0 tx=0x7fd84001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be310b77930e88f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:24.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.218+0000 7fd855400640 1 -- 172.21.15.83:0/2964208768 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8400b0070 con 0x7fd8580b4360 2024-04-25T00:12:24.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.218+0000 7fd855400640 1 -- 172.21.15.83:0/2964208768 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8400ab600 con 0x7fd8580b4360 2024-04-25T00:12:24.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.218+0000 7fd855400640 1 -- 172.21.15.83:0/2964208768 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8400ab9d0 con 0x7fd8580b4360 2024-04-25T00:12:24.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.218+0000 7fd85d200640 1 -- 172.21.15.83:0/2964208768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8580b4360 msgr2=0x7fd8580b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:24.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.218+0000 7fd85d200640 1 --2- 172.21.15.83:0/2964208768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8580b4360 0x7fd8580b47e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd858158fd0 tx=0x7fd84001c490 comp rx=0 tx=0).stop 2024-04-25T00:12:24.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.218+0000 7fd85d200640 1 -- 172.21.15.83:0/2964208768 shutdown_connections 2024-04-25T00:12:24.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.218+0000 7fd85d200640 1 --2- 172.21.15.83:0/2964208768 >> v2:172.21.15.86:3300/0 conn(0x7fd8580b2b30 0x7fd8580b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.218+0000 7fd85d200640 1 --2- 172.21.15.83:0/2964208768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8580b4360 0x7fd8580b47e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.218+0000 7fd85d200640 1 -- 172.21.15.83:0/2964208768 >> 172.21.15.83:0/2964208768 conn(0x7fd8580b1830 msgr2=0x7fd8580b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:24.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.218+0000 7fd85d200640 1 -- 172.21.15.83:0/2964208768 shutdown_connections 2024-04-25T00:12:24.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.218+0000 7fd85d200640 1 -- 172.21.15.83:0/2964208768 wait complete. 2024-04-25T00:12:24.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.218+0000 7fd85d200640 1 Processor -- start 2024-04-25T00:12:24.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.218+0000 7fd85d200640 1 -- start start 2024-04-25T00:12:24.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd85d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8580b2b30 0x7fd858151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd85d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8580b4360 0x7fd858156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd85d200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8581553e0 0x7fd85814ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd85d200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd85806ebd0 con 0x7fd8580b2b30 2024-04-25T00:12:24.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd85d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd858167d90 con 0x7fd8580b4360 2024-04-25T00:12:24.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd85d200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd8581568d0 con 0x7fd8581553e0 2024-04-25T00:12:24.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd855e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8580b2b30 0x7fd858151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd855e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8580b2b30 0x7fd858151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34492/0 (socket says 172.21.15.83:34492) 2024-04-25T00:12:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd855e00640 1 -- 172.21.15.83:0/4002700501 learned_addr learned my addr 172.21.15.83:0/4002700501 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd84fe00640 1 --2- 172.21.15.83:0/4002700501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8580b4360 0x7fd858156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd856800640 1 --2- 172.21.15.83:0/4002700501 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8581553e0 0x7fd85814ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd855e00640 1 -- 172.21.15.83:0/4002700501 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8581553e0 msgr2=0x7fd85814ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd855e00640 1 --2- 172.21.15.83:0/4002700501 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8581553e0 0x7fd85814ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd855e00640 1 -- 172.21.15.83:0/4002700501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8580b4360 msgr2=0x7fd858156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd855e00640 1 --2- 172.21.15.83:0/4002700501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8580b4360 0x7fd858156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd855e00640 1 -- 172.21.15.83:0/4002700501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd858156ba0 con 0x7fd8580b2b30 2024-04-25T00:12:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd855e00640 1 --2- 172.21.15.83:0/4002700501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8580b2b30 0x7fd858151f00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd84800c420 tx=0x7fd84800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd84e000640 1 -- 172.21.15.83:0/4002700501 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8480052d0 con 0x7fd8580b2b30 2024-04-25T00:12:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd84e000640 1 -- 172.21.15.83:0/4002700501 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd848005470 con 0x7fd8580b2b30 2024-04-25T00:12:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd84e000640 1 -- 172.21.15.83:0/4002700501 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848007260 con 0x7fd8580b2b30 2024-04-25T00:12:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd85d200640 1 -- 172.21.15.83:0/4002700501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd858156e80 con 0x7fd8580b2b30 2024-04-25T00:12:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd85d200640 1 -- 172.21.15.83:0/4002700501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd858157440 con 0x7fd8580b2b30 2024-04-25T00:12:24.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd85d200640 1 -- 172.21.15.83:0/4002700501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8580737f0 con 0x7fd8580b2b30 2024-04-25T00:12:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.222+0000 7fd84e000640 1 -- 172.21.15.83:0/4002700501 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd848007400 con 0x7fd8580b2b30 2024-04-25T00:12:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.226+0000 7fd84e000640 1 --2- 172.21.15.83:0/4002700501 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd8340678c0 0x7fd834069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.226+0000 7fd84e000640 1 -- 172.21.15.83:0/4002700501 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fd848085350 con 0x7fd8580b2b30 2024-04-25T00:12:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.226+0000 7fd84fe00640 1 --2- 172.21.15.83:0/4002700501 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd8340678c0 0x7fd834069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.226+0000 7fd84fe00640 1 --2- 172.21.15.83:0/4002700501 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd8340678c0 0x7fd834069d80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd8400ad000 tx=0x7fd840008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:24.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.230+0000 7fd84e000640 1 -- 172.21.15.83:0/4002700501 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd848055610 con 0x7fd8580b2b30 2024-04-25T00:12:24.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.414+0000 7fd85d200640 1 -- 172.21.15.83:0/4002700501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd85804f5f0 con 0x7fd8580b2b30 2024-04-25T00:12:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.414+0000 7fd84e000640 1 -- 172.21.15.83:0/4002700501 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7fd848059910 con 0x7fd8580b2b30 2024-04-25T00:12:24.416 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:24.416 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4277921554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":4277921554},{"type":"v1","addr":"172.21.15.86:6835","nonce":4277921554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/4145334103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":4145334103},{"type":"v1","addr":"172.21.15.86:6834","nonce":4145334103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3441801758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3441801758},{"type":"v1","addr":"172.21.15.83:6836","nonce":3441801758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:12:18.566138+0000","modified":"2024-04-25T00:12:21.572278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5328},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5328":{"gid":5328,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2704243724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2704243724},{"type":"v1","addr":"172.21.15.83:6837","nonce":2704243724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5328,"qdb_cluster":[5328]},"id":2}]} 2024-04-25T00:12:24.417 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 18 2024-04-25T00:12:24.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.418+0000 7fd85d200640 1 -- 172.21.15.83:0/4002700501 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd8340678c0 msgr2=0x7fd834069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:24.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.418+0000 7fd85d200640 1 --2- 172.21.15.83:0/4002700501 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd8340678c0 0x7fd834069d80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd8400ad000 tx=0x7fd840008040 comp rx=0 tx=0).stop 2024-04-25T00:12:24.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.418+0000 7fd85d200640 1 -- 172.21.15.83:0/4002700501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8580b2b30 msgr2=0x7fd858151f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:24.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.418+0000 7fd85d200640 1 --2- 172.21.15.83:0/4002700501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8580b2b30 0x7fd858151f00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd84800c420 tx=0x7fd84800c8e0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.418+0000 7fd85d200640 1 -- 172.21.15.83:0/4002700501 shutdown_connections 2024-04-25T00:12:24.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.418+0000 7fd85d200640 1 --2- 172.21.15.83:0/4002700501 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd8340678c0 0x7fd834069d80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.418+0000 7fd85d200640 1 --2- 172.21.15.83:0/4002700501 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8581553e0 0x7fd85814ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.418+0000 7fd85d200640 1 --2- 172.21.15.83:0/4002700501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8580b4360 0x7fd858156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.418+0000 7fd85d200640 1 --2- 172.21.15.83:0/4002700501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8580b2b30 0x7fd858151f00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.418+0000 7fd85d200640 1 -- 172.21.15.83:0/4002700501 >> 172.21.15.83:0/4002700501 conn(0x7fd8580b1830 msgr2=0x7fd85815d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:24.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.418+0000 7fd85d200640 1 -- 172.21.15.83:0/4002700501 shutdown_connections 2024-04-25T00:12:24.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.418+0000 7fd85d200640 1 -- 172.21.15.83:0/4002700501 wait complete. 2024-04-25T00:12:24.840 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:12:18.566138+0000', 'modified': '2024-04-25T00:12:21.572278+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5328}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5328': {'gid': 5328, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/2704243724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6835', 'nonce': 2704243724}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 2704243724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5328, 'qdb_cluster': [5328]} 2024-04-25T00:12:24.840 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:12:24.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:12:24.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.974+0000 7f7c5c400640 1 Processor -- start 2024-04-25T00:12:24.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.974+0000 7f7c5c400640 1 -- start start 2024-04-25T00:12:24.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c5c400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c540cb640 0x7f7c5406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c5c400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c540711a0 con 0x7f7c5406cfd0 2024-04-25T00:12:24.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c5c400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c5406e160 con 0x7f7c54070d10 2024-04-25T00:12:24.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c5c400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5406ea10 con 0x7f7c540cb640 2024-04-25T00:12:24.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c59a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7c5406cfd0 0x7f7c5406d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42592/0 (socket says 172.21.15.83:42592) 2024-04-25T00:12:24.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c59a00640 1 -- 172.21.15.83:0/2924048486 learned_addr learned my addr 172.21.15.83:0/2924048486 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:24.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c59000640 1 --2- 172.21.15.83:0/2924048486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c540cb640 0x7f7c5406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 -- 172.21.15.83:0/2924048486 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3346731149 0 0) 0x7f7c48001120 con 0x7f7c5406cfd0 2024-04-25T00:12:24.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 -- 172.21.15.83:0/2924048486 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c3c003530 con 0x7f7c5406cfd0 2024-04-25T00:12:24.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 -- 172.21.15.83:0/2924048486 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2842422545 0 0) 0x7f7c3c003530 con 0x7f7c5406cfd0 2024-04-25T00:12:24.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 -- 172.21.15.83:0/2924048486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c540cb640 msgr2=0x7f7c5406dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:24.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 --2- 172.21.15.83:0/2924048486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c540cb640 0x7f7c5406dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 -- 172.21.15.83:0/2924048486 >> v1:172.21.15.86:6789/0 conn(0x7f7c54070d10 legacy=0x7f7c54165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:24.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 -- 172.21.15.83:0/2924048486 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5406ecf0 con 0x7f7c5406cfd0 2024-04-25T00:12:24.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 -- 172.21.15.83:0/2924048486 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f7c48001850 con 0x7f7c5406cfd0 2024-04-25T00:12:24.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 -- 172.21.15.83:0/2924048486 >> v1:172.21.15.83:6789/0 conn(0x7f7c5406cfd0 legacy=0x7f7c5406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:24.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 --2- 172.21.15.83:0/2924048486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c3c0039f0 0x7f7c3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 --2- 172.21.15.83:0/2924048486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c3c0043a0 0x7f7c3c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 --2- 172.21.15.83:0/2924048486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c540cb640 0x7f7c3c0050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 -- 172.21.15.83:0/2924048486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5406ea10 con 0x7f7c3c0043a0 2024-04-25T00:12:24.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 -- 172.21.15.83:0/2924048486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c00b9d0 con 0x7f7c3c0039f0 2024-04-25T00:12:24.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c53e00640 1 -- 172.21.15.83:0/2924048486 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3c00bbd0 con 0x7f7c540cb640 2024-04-25T00:12:24.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c59000640 1 --2- 172.21.15.83:0/2924048486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c3c0043a0 0x7f7c3c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c59a00640 1 --2- 172.21.15.83:0/2924048486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c3c0039f0 0x7f7c3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c5a400640 1 --2- 172.21.15.83:0/2924048486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c540cb640 0x7f7c3c0050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c59000640 1 -- 172.21.15.83:0/2924048486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c540cb640 msgr2=0x7f7c3c0050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:24.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c59000640 1 --2- 172.21.15.83:0/2924048486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c540cb640 0x7f7c3c0050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c59000640 1 -- 172.21.15.83:0/2924048486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c3c0039f0 msgr2=0x7f7c3c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:24.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c59000640 1 --2- 172.21.15.83:0/2924048486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c3c0039f0 0x7f7c3c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.978+0000 7f7c59000640 1 -- 172.21.15.83:0/2924048486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3c003530 con 0x7f7c3c0043a0 2024-04-25T00:12:24.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.982+0000 7f7c59000640 1 --2- 172.21.15.83:0/2924048486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c3c0043a0 0x7f7c3c004870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009ac0 tx=0x7f7c4001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3579c0aec30258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:24.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.982+0000 7f7c53e00640 1 -- 172.21.15.83:0/2924048486 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c400b2070 con 0x7f7c3c0043a0 2024-04-25T00:12:24.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.982+0000 7f7c53e00640 1 -- 172.21.15.83:0/2924048486 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c400a5640 con 0x7f7c3c0043a0 2024-04-25T00:12:24.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.982+0000 7f7c53e00640 1 -- 172.21.15.83:0/2924048486 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40009690 con 0x7f7c3c0043a0 2024-04-25T00:12:24.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.982+0000 7f7c5c400640 1 -- 172.21.15.83:0/2924048486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c3c0043a0 msgr2=0x7f7c3c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:24.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.982+0000 7f7c5c400640 1 --2- 172.21.15.83:0/2924048486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c3c0043a0 0x7f7c3c004870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009ac0 tx=0x7f7c4001ca50 comp rx=0 tx=0).stop 2024-04-25T00:12:24.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.982+0000 7f7c5a400640 1 -- 172.21.15.83:0/2924048486 reap_dead start 2024-04-25T00:12:24.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.982+0000 7f7c5c400640 1 -- 172.21.15.83:0/2924048486 shutdown_connections 2024-04-25T00:12:24.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.982+0000 7f7c5c400640 1 -- 172.21.15.83:0/2924048486 >> 172.21.15.83:0/2924048486 conn(0x7f7c540cfa60 msgr2=0x7f7c540cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:24.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.982+0000 7f7c5c400640 1 -- 172.21.15.83:0/2924048486 shutdown_connections 2024-04-25T00:12:24.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.982+0000 7f7c5c400640 1 -- 172.21.15.83:0/2924048486 wait complete. 2024-04-25T00:12:24.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c5c400640 1 Processor -- start 2024-04-25T00:12:24.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c5c400640 1 -- start start 2024-04-25T00:12:24.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c5c400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c3c0043a0 0x7f7c54140330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c5c400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c541f4080 0x7f7c54159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c5c400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c5415a240 0x7f7c540ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c5c400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c541458e0 con 0x7f7c541f4080 2024-04-25T00:12:24.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c5c400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c54145760 con 0x7f7c5415a240 2024-04-25T00:12:24.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c5c400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5406dac0 con 0x7f7c3c0043a0 2024-04-25T00:12:24.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c59000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c541f4080 0x7f7c54159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c59000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c541f4080 0x7f7c54159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52656/0 (socket says 172.21.15.83:52656) 2024-04-25T00:12:24.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c59000640 1 -- 172.21.15.83:0/190464301 learned_addr learned my addr 172.21.15.83:0/190464301 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:24.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c59a00640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c3c0043a0 0x7f7c54140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c5a400640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c5415a240 0x7f7c540ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c59000640 1 -- 172.21.15.83:0/190464301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c3c0043a0 msgr2=0x7f7c54140330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:24.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c59000640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c3c0043a0 0x7f7c54140330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c59000640 1 -- 172.21.15.83:0/190464301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c5415a240 msgr2=0x7f7c540ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:24.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c59000640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c5415a240 0x7f7c540ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c59000640 1 -- 172.21.15.83:0/190464301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c54142ec0 con 0x7f7c541f4080 2024-04-25T00:12:24.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c5a400640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c5415a240 0x7f7c540ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:24.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c59000640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c541f4080 0x7f7c54159d00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7c4001c880 tx=0x7f7c3c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:24.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c51600640 1 -- 172.21.15.83:0/190464301 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c400c1070 con 0x7f7c541f4080 2024-04-25T00:12:24.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c51600640 1 -- 172.21.15.83:0/190464301 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c400b9660 con 0x7f7c541f4080 2024-04-25T00:12:24.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c5c400640 1 -- 172.21.15.83:0/190464301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c541431a0 con 0x7f7c541f4080 2024-04-25T00:12:24.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c51600640 1 -- 172.21.15.83:0/190464301 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c400b2040 con 0x7f7c541f4080 2024-04-25T00:12:24.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.986+0000 7f7c5c400640 1 -- 172.21.15.83:0/190464301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c54143760 con 0x7f7c541f4080 2024-04-25T00:12:24.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c51600640 1 -- 172.21.15.83:0/190464301 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7c400c4070 con 0x7f7c541f4080 2024-04-25T00:12:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c51600640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c340678c0 0x7f7c34069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c51600640 1 -- 172.21.15.83:0/190464301 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f7c4011fd30 con 0x7f7c541f4080 2024-04-25T00:12:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c59a00640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c340678c0 0x7f7c34069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5f8f9000 1 Processor -- start 2024-04-25T00:12:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5f8f9000 1 -- start start 2024-04-25T00:12:24.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c59a00640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c340678c0 0x7f7c34069d80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7c48001730 tx=0x7f7c48001120 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:24.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5f8f9000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c3ef0 0x563b0d1c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5f8f9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d1c4810 0x563b0d17a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5f8f9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d17abc0 0x563b0d17afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5f8f9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563b0cfc85e0 con 0x563b0d17abc0 2024-04-25T00:12:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5f8f9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b0cf4fc50 con 0x563b0d1c4810 2024-04-25T00:12:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5f8f9000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x563b0cfc45a0 con 0x563b0d1c3ef0 2024-04-25T00:12:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d17abc0 0x563b0d17afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d17abc0 0x563b0d17afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52670/0 (socket says 172.21.15.83:52670) 2024-04-25T00:12:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5a400640 1 -- 172.21.15.83:0/1355211781 learned_addr learned my addr 172.21.15.83:0/1355211781 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c59a00640 1 --2- 172.21.15.83:0/1355211781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c3ef0 0x563b0d1c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c59000640 1 --2- 172.21.15.83:0/1355211781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d1c4810 0x563b0d17a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5a400640 1 -- 172.21.15.83:0/1355211781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c3ef0 msgr2=0x563b0d1c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5a400640 1 --2- 172.21.15.83:0/1355211781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c3ef0 0x563b0d1c42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5a400640 1 -- 172.21.15.83:0/1355211781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d1c4810 msgr2=0x563b0d17a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:24.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5a400640 1 --2- 172.21.15.83:0/1355211781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d1c4810 0x563b0d17a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:24.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.990+0000 7f7c5a400640 1 -- 172.21.15.83:0/1355211781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b0cf4dd10 con 0x563b0d17abc0 2024-04-25T00:12:24.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5a400640 1 --2- 172.21.15.83:0/1355211781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d17abc0 0x563b0d17afa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7c540af630 tx=0x7f7c4c012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac183133202e0f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:24.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c46000640 1 -- 172.21.15.83:0/1355211781 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c01c070 con 0x563b0d17abc0 2024-04-25T00:12:24.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c46000640 1 -- 172.21.15.83:0/1355211781 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c4c012dd0 con 0x563b0d17abc0 2024-04-25T00:12:24.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c46000640 1 -- 172.21.15.83:0/1355211781 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c006d60 con 0x563b0d17abc0 2024-04-25T00:12:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1355211781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d17abc0 msgr2=0x563b0d17afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/1355211781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d17abc0 0x563b0d17afa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7c540af630 tx=0x7f7c4c012630 comp rx=0 tx=0).stop 2024-04-25T00:12:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1355211781 shutdown_connections 2024-04-25T00:12:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/1355211781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d17abc0 0x563b0d17afa0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/1355211781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d1c4810 0x563b0d17a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/1355211781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c3ef0 0x563b0d1c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1355211781 >> 172.21.15.83:0/1355211781 conn(0x563b0d176b70 msgr2=0x563b0d178f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1355211781 shutdown_connections 2024-04-25T00:12:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1355211781 wait complete. 2024-04-25T00:12:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 -- start start 2024-04-25T00:12:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 Processor -- start 2024-04-25T00:12:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d17abc0 0x563b0d1a8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d1c3ef0 0x563b0d1a9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c4810 0x563b0d1a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563b0cf82520 con 0x563b0d1c4810 2024-04-25T00:12:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b0d1769f0 con 0x563b0d1c3ef0 2024-04-25T00:12:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x563b0cf60dd0 con 0x563b0d17abc0 2024-04-25T00:12:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 10 client.0 _ll_register_callbacks cb 0x563b0d1655c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:12:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c4810 0x563b0d1a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c4810 0x563b0d1a99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52682/0 (socket says 172.21.15.83:52682) 2024-04-25T00:12:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5a400640 1 -- 172.21.15.83:0/1812964811 learned_addr learned my addr 172.21.15.83:0/1812964811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c59a00640 1 --2- 172.21.15.83:0/1812964811 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d17abc0 0x563b0d1a8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c59000640 1 --2- 172.21.15.83:0/1812964811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d1c3ef0 0x563b0d1a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5a400640 1 -- 172.21.15.83:0/1812964811 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d17abc0 msgr2=0x563b0d1a8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5a400640 1 --2- 172.21.15.83:0/1812964811 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d17abc0 0x563b0d1a8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5a400640 1 -- 172.21.15.83:0/1812964811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d1c3ef0 msgr2=0x563b0d1a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5a400640 1 --2- 172.21.15.83:0/1812964811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d1c3ef0 0x563b0d1a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5a400640 1 -- 172.21.15.83:0/1812964811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b0cf607b0 con 0x563b0d1c4810 2024-04-25T00:12:25.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5a400640 1 --2- 172.21.15.83:0/1812964811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c4810 0x563b0d1a99a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00b010 tx=0x7f7c4c0016b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc7cb01d8a41b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:25.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:12:25.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c3aa00640 1 -- 172.21.15.83:0/1812964811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c028060 con 0x563b0d1c4810 2024-04-25T00:12:25.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1812964811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c4c012dd0 con 0x563b0d1c4810 2024-04-25T00:12:25.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.994+0000 7f7c3aa00640 1 -- 172.21.15.83:0/1812964811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f7c4c005330 con 0x563b0d1c4810 2024-04-25T00:12:25.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c3aa00640 1 -- 172.21.15.83:0/1812964811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7c4c02e050 con 0x563b0d1c4810 2024-04-25T00:12:25.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 10 client.5496 fetch_fsmap learned FSMap version 18 2024-04-25T00:12:25.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1812964811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x563b0cf4fea0 con 0x563b0d1c4810 2024-04-25T00:12:25.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c3aa00640 1 -- 172.21.15.83:0/1812964811 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c4c012dd0 con 0x563b0d1c4810 2024-04-25T00:12:25.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c3aa00640 1 -- 172.21.15.83:0/1812964811 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f7c4c031020 con 0x563b0d1c4810 2024-04-25T00:12:25.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 10 client.5496 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T00:12:25.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 10 client.5496 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:12:25.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 20 client.5496 populate_metadata read hostname 'smithi083' 2024-04-25T00:12:25.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/1812964811 >> [v2:172.21.15.83:6835/2704243724,v1:172.21.15.83:6837/2704243724] conn(0x563b0d1d3f90 0x563b0d1d6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 4 client.5496 mds_command: new command op to 5328 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:12:25.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1812964811 --> [v2:172.21.15.83:6835/2704243724,v1:172.21.15.83:6837/2704243724] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b0cf63ab0 con 0x563b0d1d3f90 2024-04-25T00:12:25.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c59a00640 1 --2- 172.21.15.83:0/1812964811 >> [v2:172.21.15.83:6835/2704243724,v1:172.21.15.83:6837/2704243724] conn(0x563b0d1d3f90 0x563b0d1d6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c59a00640 1 --2- 172.21.15.83:0/1812964811 >> [v2:172.21.15.83:6835/2704243724,v1:172.21.15.83:6837/2704243724] conn(0x563b0d1d3f90 0x563b0d1d6380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d36828f669156418 server_cookie=26148a6184c08f0b in_seq=0 out_seq=0 2024-04-25T00:12:25.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c3aa00640 10 client.5496 ms_handle_connect on v2:172.21.15.83:6835/2704243724 2024-04-25T00:12:25.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c3aa00640 1 -- 172.21.15.83:0/1812964811 <== mds.0 v2:172.21.15.83:6835/2704243724 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b0cf63ab0 con 0x563b0d1d3f90 2024-04-25T00:12:25.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c3aa00640 10 client.5496 handle_command_reply: tid=0 2024-04-25T00:12:25.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 1 client.5496 shutdown 2024-04-25T00:12:25.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1812964811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c4810 msgr2=0x563b0d1a99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:25.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/1812964811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c4810 0x563b0d1a99a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00b010 tx=0x7f7c4c0016b0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1812964811 shutdown_connections 2024-04-25T00:12:25.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/1812964811 >> [v2:172.21.15.83:6835/2704243724,v1:172.21.15.83:6837/2704243724] conn(0x563b0d1d3f90 0x563b0d1d6380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/1812964811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c4810 0x563b0d1a99a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/1812964811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d1c3ef0 0x563b0d1a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c3aa00640 0 client.5496 ms_handle_reset on v2:172.21.15.83:6835/2704243724 2024-04-25T00:12:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/1812964811 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d17abc0 0x563b0d1a8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1812964811 >> 172.21.15.83:0/1812964811 conn(0x563b0d176b70 msgr2=0x563b0d1797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1812964811 shutdown_connections 2024-04-25T00:12:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1812964811 wait complete. 2024-04-25T00:12:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:24.998+0000 7f7c5f8f9000 20 client.5496 trim_cache size 0 max 16384 2024-04-25T00:12:25.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 Processor -- start 2024-04-25T00:12:25.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 -- start start 2024-04-25T00:12:25.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d17abc0 0x563b0d22f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c3ef0 0x563b0d22fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c4810 0x563b0d22ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c54200370 con 0x563b0d1c3ef0 2024-04-25T00:12:25.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c54142510 con 0x563b0d17abc0 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7c541ffd20 con 0x563b0d1c4810 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c59000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c3ef0 0x563b0d22fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c59000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c3ef0 0x563b0d22fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52690/0 (socket says 172.21.15.83:52690) 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c59000640 1 -- 172.21.15.83:0/1262418240 learned_addr learned my addr 172.21.15.83:0/1262418240 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5a400640 1 --2- 172.21.15.83:0/1262418240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c4810 0x563b0d22ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c59000640 1 -- 172.21.15.83:0/1262418240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c4810 msgr2=0x563b0d22ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c59000640 1 --2- 172.21.15.83:0/1262418240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c4810 0x563b0d22ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c59000640 1 -- 172.21.15.83:0/1262418240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d17abc0 msgr2=0x563b0d22f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c59a00640 1 --2- 172.21.15.83:0/1262418240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d17abc0 0x563b0d22f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c59000640 1 --2- 172.21.15.83:0/1262418240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d17abc0 0x563b0d22f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c59000640 1 -- 172.21.15.83:0/1262418240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b0cf50e80 con 0x563b0d1c3ef0 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5a400640 1 --2- 172.21.15.83:0/1262418240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c4810 0x563b0d22ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c59a00640 1 --2- 172.21.15.83:0/1262418240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d17abc0 0x563b0d22f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c59000640 1 --2- 172.21.15.83:0/1262418240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c3ef0 0x563b0d22fa40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000f210 tx=0x7f7c400a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da30de3213ddeba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:25.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c45600640 1 -- 172.21.15.83:0/1262418240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c400f4260 con 0x563b0d1c3ef0 2024-04-25T00:12:25.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c45600640 1 -- 172.21.15.83:0/1262418240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c400f8b20 con 0x563b0d1c3ef0 2024-04-25T00:12:25.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c45600640 1 -- 172.21.15.83:0/1262418240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4011e140 con 0x563b0d1c3ef0 2024-04-25T00:12:25.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1262418240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c3ef0 msgr2=0x563b0d22fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:25.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/1262418240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c3ef0 0x563b0d22fa40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000f210 tx=0x7f7c400a7040 comp rx=0 tx=0).stop 2024-04-25T00:12:25.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1262418240 shutdown_connections 2024-04-25T00:12:25.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/1262418240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c4810 0x563b0d22ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/1262418240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c3ef0 0x563b0d22fa40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/1262418240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d17abc0 0x563b0d22f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1262418240 >> 172.21.15.83:0/1262418240 conn(0x563b0d176b70 msgr2=0x563b0d1d14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:25.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1262418240 shutdown_connections 2024-04-25T00:12:25.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/1262418240 wait complete. 2024-04-25T00:12:25.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 -- start start 2024-04-25T00:12:25.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.018+0000 7f7c5f8f9000 1 Processor -- start 2024-04-25T00:12:25.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d17abc0 0x563b0d233dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c3ef0 0x563b0d234310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c4810 0x563b0d1d6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563b0cf7cc40 con 0x563b0d1c4810 2024-04-25T00:12:25.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b0d07ae80 con 0x563b0d17abc0 2024-04-25T00:12:25.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x563b0cf91810 con 0x563b0d1c3ef0 2024-04-25T00:12:25.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 10 client.0 _ll_register_callbacks cb 0x7f7c4c01a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:12:25.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c4810 0x563b0d1d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c4810 0x563b0d1d6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52706/0 (socket says 172.21.15.83:52706) 2024-04-25T00:12:25.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5a400640 1 -- 172.21.15.83:0/693837400 learned_addr learned my addr 172.21.15.83:0/693837400 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:25.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c59000640 1 --2- 172.21.15.83:0/693837400 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c3ef0 0x563b0d234310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c59a00640 1 --2- 172.21.15.83:0/693837400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d17abc0 0x563b0d233dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5a400640 1 -- 172.21.15.83:0/693837400 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c3ef0 msgr2=0x563b0d234310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:25.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5a400640 1 --2- 172.21.15.83:0/693837400 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c3ef0 0x563b0d234310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5a400640 1 -- 172.21.15.83:0/693837400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d17abc0 msgr2=0x563b0d233dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:25.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5a400640 1 --2- 172.21.15.83:0/693837400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d17abc0 0x563b0d233dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5a400640 1 -- 172.21.15.83:0/693837400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b0cf95760 con 0x563b0d1c4810 2024-04-25T00:12:25.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5a400640 1 --2- 172.21.15.83:0/693837400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c4810 0x563b0d1d6690 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00abf0 tx=0x7f7c4c00aa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d98bd2dd6f6f1f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:25.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:12:25.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c3aa00640 1 -- 172.21.15.83:0/693837400 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c01c070 con 0x563b0d1c4810 2024-04-25T00:12:25.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c3aa00640 1 -- 172.21.15.83:0/693837400 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f7c4c016480 con 0x563b0d1c4810 2024-04-25T00:12:25.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/693837400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c400f8b20 con 0x563b0d1c4810 2024-04-25T00:12:25.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c3aa00640 1 -- 172.21.15.83:0/693837400 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7c4c03f050 con 0x563b0d1c4810 2024-04-25T00:12:25.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 10 client.5502 fetch_fsmap learned FSMap version 18 2024-04-25T00:12:25.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/693837400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x563b0d0219e0 con 0x563b0d1c4810 2024-04-25T00:12:25.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c3aa00640 1 -- 172.21.15.83:0/693837400 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c400f8b20 con 0x563b0d1c4810 2024-04-25T00:12:25.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c3aa00640 1 -- 172.21.15.83:0/693837400 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f7c4c042020 con 0x563b0d1c4810 2024-04-25T00:12:25.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 10 client.5502 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T00:12:25.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 10 client.5502 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:12:25.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 20 client.5502 populate_metadata read hostname 'smithi083' 2024-04-25T00:12:25.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/693837400 >> [v2:172.21.15.83:6835/2704243724,v1:172.21.15.83:6837/2704243724] conn(0x563b0d1d6bd0 0x563b0d1d6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 4 client.5502 mds_command: new command op to 5328 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:12:25.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/693837400 --> [v2:172.21.15.83:6835/2704243724,v1:172.21.15.83:6837/2704243724] -- command(tid 0: {"prefix": "status"}) -- 0x563b0cfd5b20 con 0x563b0d1d6bd0 2024-04-25T00:12:25.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c59a00640 1 --2- 172.21.15.83:0/693837400 >> [v2:172.21.15.83:6835/2704243724,v1:172.21.15.83:6837/2704243724] conn(0x563b0d1d6bd0 0x563b0d1d6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c3aa00640 10 client.5502 ms_handle_connect on v2:172.21.15.83:6835/2704243724 2024-04-25T00:12:25.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c59a00640 1 --2- 172.21.15.83:0/693837400 >> [v2:172.21.15.83:6835/2704243724,v1:172.21.15.83:6837/2704243724] conn(0x563b0d1d6bd0 0x563b0d1d6fb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a04745684d20f9a8 server_cookie=8d93080e29a1dacf in_seq=0 out_seq=0 2024-04-25T00:12:25.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c3aa00640 1 -- 172.21.15.83:0/693837400 <== mds.0 v2:172.21.15.83:6835/2704243724 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x563b0cfd5b20 con 0x563b0d1d6bd0 2024-04-25T00:12:25.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c3aa00640 10 client.5502 handle_command_reply: tid=0 2024-04-25T00:12:25.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 client.5502 shutdown 2024-04-25T00:12:25.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/693837400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c4810 msgr2=0x563b0d1d6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:25.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/693837400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c4810 0x563b0d1d6690 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00abf0 tx=0x7f7c4c00aa40 comp rx=0 tx=0).stop 2024-04-25T00:12:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/693837400 shutdown_connections 2024-04-25T00:12:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/693837400 >> [v2:172.21.15.83:6835/2704243724,v1:172.21.15.83:6837/2704243724] conn(0x563b0d1d6bd0 0x563b0d1d6fb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/693837400 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563b0d1c3ef0 0x563b0d234310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/693837400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b0d1c4810 0x563b0d1d6690 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c3aa00640 0 client.5502 ms_handle_reset on v2:172.21.15.83:6835/2704243724 2024-04-25T00:12:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 --2- 172.21.15.83:0/693837400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b0d17abc0 0x563b0d233dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/693837400 >> 172.21.15.83:0/693837400 conn(0x563b0d176b70 msgr2=0x563b0d1c2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/693837400 shutdown_connections 2024-04-25T00:12:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.022+0000 7f7c5f8f9000 1 -- 172.21.15.83:0/693837400 wait complete. 2024-04-25T00:12:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.026+0000 7f7c5f8f9000 20 client.5502 trim_cache size 0 max 16384 2024-04-25T00:12:25.029 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:12:25.030 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "d58ae83a-ad72-4e18-824a-86e779979c35", 2024-04-25T00:12:25.030 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-25T00:12:25.030 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 5328, 2024-04-25T00:12:25.030 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T00:12:25.030 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T00:12:25.030 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T00:12:25.030 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.8441981009999999, 2024-04-25T00:12:25.030 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 18, 2024-04-25T00:12:25.030 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 38, 2024-04-25T00:12:25.030 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 38, 2024-04-25T00:12:25.030 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 12.89337259 2024-04-25T00:12:25.030 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:12:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.026+0000 7f7c5c400640 1 -- 172.21.15.83:0/190464301 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c340678c0 msgr2=0x7f7c34069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.026+0000 7f7c5c400640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c340678c0 0x7f7c34069d80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7c48001730 tx=0x7f7c48001120 comp rx=0 tx=0).stop 2024-04-25T00:12:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.026+0000 7f7c5c400640 1 -- 172.21.15.83:0/190464301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c541f4080 msgr2=0x7f7c54159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.026+0000 7f7c5c400640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c541f4080 0x7f7c54159d00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7c4001c880 tx=0x7f7c3c005840 comp rx=0 tx=0).stop 2024-04-25T00:12:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.026+0000 7f7c5c400640 1 -- 172.21.15.83:0/190464301 shutdown_connections 2024-04-25T00:12:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.026+0000 7f7c5c400640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c340678c0 0x7f7c34069d80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.026+0000 7f7c5c400640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c5415a240 0x7f7c540ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.026+0000 7f7c5c400640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c541f4080 0x7f7c54159d00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.026+0000 7f7c5c400640 1 --2- 172.21.15.83:0/190464301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c3c0043a0 0x7f7c54140330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.026+0000 7f7c5c400640 1 -- 172.21.15.83:0/190464301 >> 172.21.15.83:0/190464301 conn(0x7f7c540cfa60 msgr2=0x7f7c540b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.026+0000 7f7c5c400640 1 -- 172.21.15.83:0/190464301 shutdown_connections 2024-04-25T00:12:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.026+0000 7f7c5c400640 1 -- 172.21.15.83:0/190464301 wait complete. 2024-04-25T00:12:25.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:12:25.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.178+0000 7f8385600640 1 Processor -- start 2024-04-25T00:12:25.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.178+0000 7f8385600640 1 -- start start 2024-04-25T00:12:25.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.182+0000 7f8385600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83801599f0 0x7f83800cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.182+0000 7f8385600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f838015c340 0x7f838015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.182+0000 7f8385600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83800ccc60 con 0x7f838015ccf0 2024-04-25T00:12:25.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.182+0000 7f8385600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8380159dd0 con 0x7f83801599f0 2024-04-25T00:12:25.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.182+0000 7f8385600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f838006dba0 con 0x7f838015c340 2024-04-25T00:12:25.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.182+0000 7f837ea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83801599f0 0x7f83800cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.182+0000 7f837ea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83801599f0 0x7f83800cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52720/0 (socket says 172.21.15.83:52720) 2024-04-25T00:12:25.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.182+0000 7f837ea00640 1 -- 172.21.15.83:0/8813043 learned_addr learned my addr 172.21.15.83:0/8813043 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:25.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.182+0000 7f8376000640 1 --2- 172.21.15.83:0/8813043 >> v2:172.21.15.86:3300/0 conn(0x7f838015c340 0x7f838015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.182+0000 7f837ea00640 1 -- 172.21.15.83:0/8813043 >> v2:172.21.15.86:3300/0 conn(0x7f838015c340 msgr2=0x7f838015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:25.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.182+0000 7f837ea00640 1 --2- 172.21.15.83:0/8813043 >> v2:172.21.15.86:3300/0 conn(0x7f838015c340 0x7f838015c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.182+0000 7f837ea00640 1 -- 172.21.15.83:0/8813043 >> v1:172.21.15.86:6789/0 conn(0x7f838015ccf0 legacy=0x7f838015f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:25.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.182+0000 7f837ea00640 1 -- 172.21.15.83:0/8813043 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838006de80 con 0x7f83801599f0 2024-04-25T00:12:25.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.182+0000 7f8376000640 1 --2- 172.21.15.83:0/8813043 >> v2:172.21.15.86:3300/0 conn(0x7f838015c340 0x7f838015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:25.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.186+0000 7f837ea00640 1 --2- 172.21.15.83:0/8813043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83801599f0 0x7f83800cc510 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8370009870 tx=0x7f837001ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df865ac95d4ed084 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:25.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.186+0000 7f837e000640 1 -- 172.21.15.83:0/8813043 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83700b0070 con 0x7f83801599f0 2024-04-25T00:12:25.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.186+0000 7f837e000640 1 -- 172.21.15.83:0/8813043 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83700ab600 con 0x7f83801599f0 2024-04-25T00:12:25.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.186+0000 7f837e000640 1 -- 172.21.15.83:0/8813043 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83700ab9d0 con 0x7f83801599f0 2024-04-25T00:12:25.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.186+0000 7f8385600640 1 -- 172.21.15.83:0/8813043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83801599f0 msgr2=0x7f83800cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:25.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.186+0000 7f8385600640 1 --2- 172.21.15.83:0/8813043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83801599f0 0x7f83800cc510 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8370009870 tx=0x7f837001ccd0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.186+0000 7f8385600640 1 -- 172.21.15.83:0/8813043 shutdown_connections 2024-04-25T00:12:25.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.186+0000 7f8385600640 1 --2- 172.21.15.83:0/8813043 >> v2:172.21.15.86:3300/0 conn(0x7f838015c340 0x7f838015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.186+0000 7f8385600640 1 --2- 172.21.15.83:0/8813043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83801599f0 0x7f83800cc510 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.186+0000 7f8385600640 1 -- 172.21.15.83:0/8813043 >> 172.21.15.83:0/8813043 conn(0x7f83800b1830 msgr2=0x7f83800b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:25.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.186+0000 7f8385600640 1 -- 172.21.15.83:0/8813043 shutdown_connections 2024-04-25T00:12:25.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.186+0000 7f8385600640 1 -- 172.21.15.83:0/8813043 wait complete. 2024-04-25T00:12:25.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.186+0000 7f8385600640 1 Processor -- start 2024-04-25T00:12:25.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.186+0000 7f8385600640 1 -- start start 2024-04-25T00:12:25.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8385600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83801599f0 0x7f838013c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8385600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f838015c340 0x7f838015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8385600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f838015ccf0 0x7f838013bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8385600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83800cf870 con 0x7f83801599f0 2024-04-25T00:12:25.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8385600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83800cf6f0 con 0x7f838015c340 2024-04-25T00:12:25.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8385600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8380168cd0 con 0x7f838015ccf0 2024-04-25T00:12:25.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f837ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83801599f0 0x7f838013c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f837ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83801599f0 0x7f838013c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52722/0 (socket says 172.21.15.83:52722) 2024-04-25T00:12:25.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f837ea00640 1 -- 172.21.15.83:0/896589611 learned_addr learned my addr 172.21.15.83:0/896589611 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:25.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8376000640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f838015c340 0x7f838015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f837f400640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f838015ccf0 0x7f838013bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f837ea00640 1 -- 172.21.15.83:0/896589611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f838015ccf0 msgr2=0x7f838013bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:25.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f837ea00640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f838015ccf0 0x7f838013bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f837ea00640 1 -- 172.21.15.83:0/896589611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f838015c340 msgr2=0x7f838015b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:25.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f837ea00640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f838015c340 0x7f838015b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f837ea00640 1 -- 172.21.15.83:0/896589611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838013c9f0 con 0x7f83801599f0 2024-04-25T00:12:25.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8376000640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f838015c340 0x7f838015b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:25.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f837ea00640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83801599f0 0x7f838013c4b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f83700ae040 tx=0x7f837001aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:25.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8377400640 1 -- 172.21.15.83:0/896589611 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83700ba020 con 0x7f83801599f0 2024-04-25T00:12:25.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8377400640 1 -- 172.21.15.83:0/896589611 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83700b6620 con 0x7f83801599f0 2024-04-25T00:12:25.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8385600640 1 -- 172.21.15.83:0/896589611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f838015bd30 con 0x7f83801599f0 2024-04-25T00:12:25.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8377400640 1 -- 172.21.15.83:0/896589611 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83700b0040 con 0x7f83801599f0 2024-04-25T00:12:25.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8385600640 1 -- 172.21.15.83:0/896589611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f838015da10 con 0x7f83801599f0 2024-04-25T00:12:25.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8385600640 1 -- 172.21.15.83:0/896589611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83800737f0 con 0x7f83801599f0 2024-04-25T00:12:25.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.190+0000 7f8377400640 1 -- 172.21.15.83:0/896589611 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f83700bd020 con 0x7f83801599f0 2024-04-25T00:12:25.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.194+0000 7f8377400640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f835c0678c0 0x7f835c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:25.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.194+0000 7f8377400640 1 -- 172.21.15.83:0/896589611 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f83701200a0 con 0x7f83801599f0 2024-04-25T00:12:25.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.194+0000 7f8376000640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f835c0678c0 0x7f835c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:25.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.194+0000 7f8376000640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f835c0678c0 0x7f835c069d80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f83801557b0 tx=0x7f8368007ab0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:25.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.198+0000 7f8377400640 1 -- 172.21.15.83:0/896589611 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f83700c4030 con 0x7f83801599f0 2024-04-25T00:12:25.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.386+0000 7f8385600640 1 -- 172.21.15.83:0/896589611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f838004f5f0 con 0x7f83801599f0 2024-04-25T00:12:25.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.386+0000 7f8377400640 1 -- 172.21.15.83:0/896589611 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f83800ccc60 con 0x7f83801599f0 2024-04-25T00:12:25.388 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:12:25.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.390+0000 7f8385600640 1 -- 172.21.15.83:0/896589611 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f835c0678c0 msgr2=0x7f835c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:25.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.390+0000 7f8385600640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f835c0678c0 0x7f835c069d80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f83801557b0 tx=0x7f8368007ab0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.390+0000 7f8385600640 1 -- 172.21.15.83:0/896589611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83801599f0 msgr2=0x7f838013c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:25.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.390+0000 7f8385600640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83801599f0 0x7f838013c4b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f83700ae040 tx=0x7f837001aab0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.390+0000 7f8385600640 1 -- 172.21.15.83:0/896589611 shutdown_connections 2024-04-25T00:12:25.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.390+0000 7f8385600640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f835c0678c0 0x7f835c069d80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.390+0000 7f8385600640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f838015ccf0 0x7f838013bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.390+0000 7f8385600640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f838015c340 0x7f838015b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.390+0000 7f8385600640 1 --2- 172.21.15.83:0/896589611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83801599f0 0x7f838013c4b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:25.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.390+0000 7f8385600640 1 -- 172.21.15.83:0/896589611 >> 172.21.15.83:0/896589611 conn(0x7f83800b1830 msgr2=0x7f8380165c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:25.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.390+0000 7f8385600640 1 -- 172.21.15.83:0/896589611 shutdown_connections 2024-04-25T00:12:25.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25.390+0000 7f8385600640 1 -- 172.21.15.83:0/896589611 wait complete. 2024-04-25T00:12:25.409 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:12:25.409 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:12:25.409 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-25T00:12:25.409 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:12:25.409 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:12:25.409 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:12:25.409 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:12:25.410 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:12:25.410 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:12:25.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:25.410 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-25T00:12:25.419 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:12:25.419 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:12:25.419 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:12:25.419 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:25.419 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-25T00:12:25.419 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:25.419 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42389sec preferred_lft 42389sec 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:12:25.420 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:8d:d4:7d:b9 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:25.421 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:12:25.421 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:25.421 INFO:teuthology.orchestra.run.smithi083.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:12:25.421 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:25.421 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:12:25.421 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:25.421 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:12:25.421 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:25.421 INFO:teuthology.orchestra.run.smithi083.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:12:25.421 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:12:25.421 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:12:25.421 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:25.421 INFO:teuthology.orchestra.run.smithi083.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:12:25.421 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:12:25.422 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:12:25.422 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:25.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:25.423 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:12:25.486 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-25T00:12:25.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:25.488 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-25T00:12:25.544 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-25T00:12:25.544 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:12:25.544 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:12:25.545 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:12:25.545 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:12:25.545 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:12:25.545 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:12:25.545 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:12:25.545 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:12:25.545 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:12:25.545 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:12:25.545 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:12:25.545 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:12:25.545 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:12:25.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:12:25.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:12:25.546 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:12:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:12:25.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:12:25.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:25.677 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:12:25.686 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:12:25.686 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:12:25.686 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:12:25.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:25.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-25T00:12:25.749 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:12:25.749 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:12:25.796 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:12:25.797 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:12:25.797 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:12:25.844 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T00:12:25.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:25.845 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:12:25.932 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:12:25.932 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:12:25.946 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:12:25.946 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:12:25.946 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:12:25.946 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:12:25.946 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:12:25.950 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:12:25.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:25.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:12:25.971 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:12:26.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:12:26.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:12:26.074 INFO:teuthology.orchestra.run.smithi083.stdout:client5341 2024-04-25T00:12:26.075 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:12:26.075 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:26.129 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:12:26.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7b2600640 1 Processor -- start 2024-04-25T00:12:26.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7b2600640 1 -- start start 2024-04-25T00:12:26.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7b2600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd7ac1627e0 0x7fd7ac06c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:26.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7b2600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd7ac0b21f0 0x7fd7ac0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:26.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7b2600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7ac0aeb80 con 0x7fd7ac0c4c40 2024-04-25T00:12:26.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7b2600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd7ac06cb40 con 0x7fd7ac1627e0 2024-04-25T00:12:26.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7b2600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd7ac0aee60 con 0x7fd7ac0b21f0 2024-04-25T00:12:26.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7ab400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd7ac0c4c40 0x7fd7ac0c5020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42604/0 (socket says 172.21.15.83:42604) 2024-04-25T00:12:26.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7ab400640 1 -- 172.21.15.83:0/3024728049 learned_addr learned my addr 172.21.15.83:0/3024728049 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:26.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7aaa00640 1 --2- 172.21.15.83:0/3024728049 >> v2:172.21.15.83:3300/0 conn(0x7fd7ac1627e0 0x7fd7ac06c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:26.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7abe00640 1 --2- 172.21.15.83:0/3024728049 >> v2:172.21.15.86:3300/0 conn(0x7fd7ac0b21f0 0x7fd7ac0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:26.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7aa000640 1 -- 172.21.15.83:0/3024728049 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194596286 0 0) 0x7fd7ac0aeb80 con 0x7fd7ac0c4c40 2024-04-25T00:12:26.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7aa000640 1 -- 172.21.15.83:0/3024728049 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd790003530 con 0x7fd7ac0c4c40 2024-04-25T00:12:26.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7aaa00640 1 -- 172.21.15.83:0/3024728049 >> v2:172.21.15.86:3300/0 conn(0x7fd7ac0b21f0 msgr2=0x7fd7ac0af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:26.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7aaa00640 1 --2- 172.21.15.83:0/3024728049 >> v2:172.21.15.86:3300/0 conn(0x7fd7ac0b21f0 0x7fd7ac0af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7aaa00640 1 -- 172.21.15.83:0/3024728049 >> v1:172.21.15.83:6789/0 conn(0x7fd7ac0c4c40 legacy=0x7fd7ac0c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:26.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.314+0000 7fd7aaa00640 1 -- 172.21.15.83:0/3024728049 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ac0af140 con 0x7fd7ac1627e0 2024-04-25T00:12:26.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.318+0000 7fd7aaa00640 1 --2- 172.21.15.83:0/3024728049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7ac1627e0 0x7fd7ac06c2f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac072750 tx=0x7fd79801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84e4f40b933a3f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:26.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.318+0000 7fd7aa000640 1 -- 172.21.15.83:0/3024728049 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7980ab610 con 0x7fd7ac1627e0 2024-04-25T00:12:26.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.318+0000 7fd7aa000640 1 -- 172.21.15.83:0/3024728049 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7980b0070 con 0x7fd7ac1627e0 2024-04-25T00:12:26.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.318+0000 7fd7aa000640 1 -- 172.21.15.83:0/3024728049 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7980ab9d0 con 0x7fd7ac1627e0 2024-04-25T00:12:26.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.318+0000 7fd7b2600640 1 -- 172.21.15.83:0/3024728049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7ac1627e0 msgr2=0x7fd7ac06c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:26.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.318+0000 7fd7b2600640 1 --2- 172.21.15.83:0/3024728049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7ac1627e0 0x7fd7ac06c2f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac072750 tx=0x7fd79801c450 comp rx=0 tx=0).stop 2024-04-25T00:12:26.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.318+0000 7fd7b2600640 1 -- 172.21.15.83:0/3024728049 shutdown_connections 2024-04-25T00:12:26.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.318+0000 7fd7b2600640 1 --2- 172.21.15.83:0/3024728049 >> v2:172.21.15.86:3300/0 conn(0x7fd7ac0b21f0 0x7fd7ac0af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.318+0000 7fd7b2600640 1 --2- 172.21.15.83:0/3024728049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7ac1627e0 0x7fd7ac06c2f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.318+0000 7fd7b2600640 1 -- 172.21.15.83:0/3024728049 >> 172.21.15.83:0/3024728049 conn(0x7fd7ac0d1c70 msgr2=0x7fd7ac0d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:26.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.318+0000 7fd7b2600640 1 -- 172.21.15.83:0/3024728049 shutdown_connections 2024-04-25T00:12:26.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.318+0000 7fd7b2600640 1 -- 172.21.15.83:0/3024728049 wait complete. 2024-04-25T00:12:26.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.318+0000 7fd7b2600640 1 Processor -- start 2024-04-25T00:12:26.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.318+0000 7fd7b2600640 1 -- start start 2024-04-25T00:12:26.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7b2600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7ac0b21f0 0x7fd7ac1652a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:26.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7b2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7ac0c4c40 0x7fd7ac0c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:26.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7b2600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd7ac1627e0 0x7fd7ac0d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7b2600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac0b0810 con 0x7fd7ac0b21f0 2024-04-25T00:12:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7b2600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac0b0690 con 0x7fd7ac0c4c40 2024-04-25T00:12:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7b2600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd7ac0b0990 con 0x7fd7ac1627e0 2024-04-25T00:12:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7ab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7ac0b21f0 0x7fd7ac1652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7ab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7ac0b21f0 0x7fd7ac1652a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52740/0 (socket says 172.21.15.83:52740) 2024-04-25T00:12:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7ab400640 1 -- 172.21.15.83:0/4187024861 learned_addr learned my addr 172.21.15.83:0/4187024861 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7abe00640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd7ac1627e0 0x7fd7ac0d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7aaa00640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7ac0c4c40 0x7fd7ac0c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7ab400640 1 -- 172.21.15.83:0/4187024861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd7ac1627e0 msgr2=0x7fd7ac0d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7ab400640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd7ac1627e0 0x7fd7ac0d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7ab400640 1 -- 172.21.15.83:0/4187024861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7ac0c4c40 msgr2=0x7fd7ac0c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7ab400640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7ac0c4c40 0x7fd7ac0c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7ab400640 1 -- 172.21.15.83:0/4187024861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ac1657e0 con 0x7fd7ac0b21f0 2024-04-25T00:12:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7abe00640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd7ac1627e0 0x7fd7ac0d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:26.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7aaa00640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7ac0c4c40 0x7fd7ac0c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:26.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7ab400640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7ac0b21f0 0x7fd7ac1652a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00c3f0 tx=0x7fd79c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:26.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7a3400640 1 -- 172.21.15.83:0/4187024861 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c017070 con 0x7fd7ac0b21f0 2024-04-25T00:12:26.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7a3400640 1 -- 172.21.15.83:0/4187024861 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd79c005cf0 con 0x7fd7ac0b21f0 2024-04-25T00:12:26.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7b2600640 1 -- 172.21.15.83:0/4187024861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7ac0cc340 con 0x7fd7ac0b21f0 2024-04-25T00:12:26.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7a3400640 1 -- 172.21.15.83:0/4187024861 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0072f0 con 0x7fd7ac0b21f0 2024-04-25T00:12:26.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7b2600640 1 -- 172.21.15.83:0/4187024861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7ac0cc900 con 0x7fd7ac0b21f0 2024-04-25T00:12:26.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7b2600640 1 -- 172.21.15.83:0/4187024861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ac09fd60 con 0x7fd7ac0b21f0 2024-04-25T00:12:26.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.322+0000 7fd7a3400640 1 -- 172.21.15.83:0/4187024861 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd79c01a070 con 0x7fd7ac0b21f0 2024-04-25T00:12:26.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.326+0000 7fd7a3400640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd78c0678c0 0x7fd78c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:26.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.326+0000 7fd7a3400640 1 -- 172.21.15.83:0/4187024861 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fd79c085350 con 0x7fd7ac0b21f0 2024-04-25T00:12:26.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.326+0000 7fd7aaa00640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd78c0678c0 0x7fd78c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:26.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.326+0000 7fd7aaa00640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd78c0678c0 0x7fd78c069d80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd7980af000 tx=0x7fd798008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:26.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.330+0000 7fd7a3400640 1 -- 172.21.15.83:0/4187024861 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd79c055c60 con 0x7fd7ac0b21f0 2024-04-25T00:12:26.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.522+0000 7fd7b2600640 1 -- 172.21.15.83:0/4187024861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd7ac0cbf00 con 0x7fd7ac0b21f0 2024-04-25T00:12:26.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.522+0000 7fd7a3400640 1 -- 172.21.15.83:0/4187024861 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7fd79c055560 con 0x7fd7ac0b21f0 2024-04-25T00:12:26.522 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:12:26.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.522+0000 7fd7b2600640 1 -- 172.21.15.83:0/4187024861 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd78c0678c0 msgr2=0x7fd78c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:26.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.522+0000 7fd7b2600640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd78c0678c0 0x7fd78c069d80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd7980af000 tx=0x7fd798008040 comp rx=0 tx=0).stop 2024-04-25T00:12:26.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.522+0000 7fd7b2600640 1 -- 172.21.15.83:0/4187024861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7ac0b21f0 msgr2=0x7fd7ac1652a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:26.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.522+0000 7fd7b2600640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7ac0b21f0 0x7fd7ac1652a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00c3f0 tx=0x7fd79c00c8b0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.522+0000 7fd7b2600640 1 -- 172.21.15.83:0/4187024861 shutdown_connections 2024-04-25T00:12:26.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.522+0000 7fd7b2600640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd78c0678c0 0x7fd78c069d80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.522+0000 7fd7b2600640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd7ac1627e0 0x7fd7ac0d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.522+0000 7fd7b2600640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7ac0c4c40 0x7fd7ac0c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.522+0000 7fd7b2600640 1 --2- 172.21.15.83:0/4187024861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7ac0b21f0 0x7fd7ac1652a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.522+0000 7fd7b2600640 1 -- 172.21.15.83:0/4187024861 >> 172.21.15.83:0/4187024861 conn(0x7fd7ac0d1c70 msgr2=0x7fd7ac070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:26.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.522+0000 7fd7b2600640 1 -- 172.21.15.83:0/4187024861 shutdown_connections 2024-04-25T00:12:26.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:26.522+0000 7fd7b2600640 1 -- 172.21.15.83:0/4187024861 wait complete. 2024-04-25T00:12:26.544 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:12:26.544 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:12:26.544 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T00:12:26.544 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:12:26.544 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:12:26.544 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:12:26.545 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:12:26.545 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:12:26.545 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:12:26.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:26.545 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T00:12:26.555 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:12:26.556 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:12:26.556 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:12:26.556 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:26.556 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T00:12:26.556 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:26.556 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:26.556 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:26.556 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T00:12:26.556 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:26.556 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:26.557 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T00:12:26.557 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:26.557 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:26.557 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:12:26.557 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:26.557 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:12:26.557 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42382sec preferred_lft 42382sec 2024-04-25T00:12:26.557 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T00:12:26.557 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:26.557 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:12:26.557 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:35:63:ea:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:26.558 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:12:26.558 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:26.558 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:12:26.558 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:26.558 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:12:26.558 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:26.558 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:12:26.558 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:26.558 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:12:26.559 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:12:26.559 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:12:26.559 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:26.559 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:12:26.559 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:12:26.559 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:12:26.559 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:26.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:26.560 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:12:26.624 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T00:12:26.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:26.627 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T00:12:26.681 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T00:12:26.681 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:12:26.681 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:12:26.681 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:12:26.682 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:12:26.682 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:12:26.682 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:12:26.682 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:12:26.682 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:12:26.682 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:12:26.683 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:12:26.683 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:12:26.683 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:12:26.683 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:12:26.684 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:12:26.684 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:12:26.684 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:12:26.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:12:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:12:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:12:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:12:26.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:26.828 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T00:12:26.836 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:12:26.837 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:12:26.837 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:12:26.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:26.838 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T00:12:26.900 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:12:26.901 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:12:26.949 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:12:26.949 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:12:26.949 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:12:27.000 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T00:12:27.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:27.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:12:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:12:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:12:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:12:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:12:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:12:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with new device syntax: 1@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:12:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:12:27.107 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:12:27.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:27.107 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:12:27.137 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:12:27.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:12:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:12:27.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:12:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:12:27.245 INFO:teuthology.orchestra.run.smithi086.stdout:client5520 2024-04-25T00:12:27.246 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:12:27.246 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:27.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds ms_async_reap_threshold 1 2024-04-25T00:12:27.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4d2200640 1 Processor -- start 2024-04-25T00:12:27.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4d2200640 1 -- start start 2024-04-25T00:12:27.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4d2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa4cc15e270 0x7fa4cc0b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:27.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4d2200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa4cc071e50 0x7fa4cc0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:27.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4d2200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4cc06dcf0 con 0x7fa4cc15de90 2024-04-25T00:12:27.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4d2200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa4cc0b2ce0 con 0x7fa4cc071e50 2024-04-25T00:12:27.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4d2200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa4cc06dfd0 con 0x7fa4cc15e270 2024-04-25T00:12:27.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4cbe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa4cc071e50 0x7fa4cc0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:27.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4cbe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa4cc071e50 0x7fa4cc0b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52746/0 (socket says 172.21.15.83:52746) 2024-04-25T00:12:27.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4cbe00640 1 -- 172.21.15.83:0/3492504574 learned_addr learned my addr 172.21.15.83:0/3492504574 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:27.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4caa00640 1 --2- 172.21.15.83:0/3492504574 >> v2:172.21.15.86:3300/0 conn(0x7fa4cc15e270 0x7fa4cc0b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:27.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4cbe00640 1 -- 172.21.15.83:0/3492504574 >> v2:172.21.15.86:3300/0 conn(0x7fa4cc15e270 msgr2=0x7fa4cc0b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:27.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4cbe00640 1 --2- 172.21.15.83:0/3492504574 >> v2:172.21.15.86:3300/0 conn(0x7fa4cc15e270 0x7fa4cc0b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4cbe00640 1 -- 172.21.15.83:0/3492504574 >> v1:172.21.15.86:6789/0 conn(0x7fa4cc15de90 legacy=0x7fa4cc071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:27.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.450+0000 7fa4cbe00640 1 -- 172.21.15.83:0/3492504574 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc06e2b0 con 0x7fa4cc071e50 2024-04-25T00:12:27.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.454+0000 7fa4cbe00640 1 --2- 172.21.15.83:0/3492504574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4cc071e50 0x7fa4cc0b3990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0736c0 tx=0x7fa4c401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fad00a28f5fa88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:27.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.454+0000 7fa4ca000640 1 -- 172.21.15.83:0/3492504574 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40b0070 con 0x7fa4cc071e50 2024-04-25T00:12:27.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.454+0000 7fa4ca000640 1 -- 172.21.15.83:0/3492504574 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c40ab600 con 0x7fa4cc071e50 2024-04-25T00:12:27.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.454+0000 7fa4ca000640 1 -- 172.21.15.83:0/3492504574 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40ab9d0 con 0x7fa4cc071e50 2024-04-25T00:12:27.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.454+0000 7fa4d2200640 1 -- 172.21.15.83:0/3492504574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4cc071e50 msgr2=0x7fa4cc0b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:27.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.454+0000 7fa4d2200640 1 --2- 172.21.15.83:0/3492504574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4cc071e50 0x7fa4cc0b3990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0736c0 tx=0x7fa4c401c490 comp rx=0 tx=0).stop 2024-04-25T00:12:27.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.454+0000 7fa4d2200640 1 -- 172.21.15.83:0/3492504574 shutdown_connections 2024-04-25T00:12:27.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.454+0000 7fa4d2200640 1 --2- 172.21.15.83:0/3492504574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4cc071e50 0x7fa4cc0b3990 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.454+0000 7fa4d2200640 1 --2- 172.21.15.83:0/3492504574 >> v2:172.21.15.86:3300/0 conn(0x7fa4cc15e270 0x7fa4cc0b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.454+0000 7fa4d2200640 1 -- 172.21.15.83:0/3492504574 >> 172.21.15.83:0/3492504574 conn(0x7fa4cc0b1830 msgr2=0x7fa4cc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:27.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.454+0000 7fa4d2200640 1 -- 172.21.15.83:0/3492504574 shutdown_connections 2024-04-25T00:12:27.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.454+0000 7fa4d2200640 1 -- 172.21.15.83:0/3492504574 wait complete. 2024-04-25T00:12:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4d2200640 1 Processor -- start 2024-04-25T00:12:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4d2200640 1 -- start start 2024-04-25T00:12:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4d2200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4cc071e50 0x7fa4cc14eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4d2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4cc15de90 0x7fa4cc1596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:27.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4d2200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4cc15e270 0x7fa4cc150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:27.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4d2200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0b4050 con 0x7fa4cc071e50 2024-04-25T00:12:27.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4d2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0b3ed0 con 0x7fa4cc15de90 2024-04-25T00:12:27.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4d2200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc0b41d0 con 0x7fa4cc15e270 2024-04-25T00:12:27.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4cb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4cc071e50 0x7fa4cc14eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:27.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4cb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4cc071e50 0x7fa4cc14eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52756/0 (socket says 172.21.15.83:52756) 2024-04-25T00:12:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4cb400640 1 -- 172.21.15.83:0/3946287691 learned_addr learned my addr 172.21.15.83:0/3946287691 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4cbe00640 1 --2- 172.21.15.83:0/3946287691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4cc15e270 0x7fa4cc150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4caa00640 1 --2- 172.21.15.83:0/3946287691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4cc15de90 0x7fa4cc1596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4cb400640 1 -- 172.21.15.83:0/3946287691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4cc15e270 msgr2=0x7fa4cc150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4cb400640 1 --2- 172.21.15.83:0/3946287691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4cc15e270 0x7fa4cc150d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4cb400640 1 -- 172.21.15.83:0/3946287691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4cc15de90 msgr2=0x7fa4cc1596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4cb400640 1 --2- 172.21.15.83:0/3946287691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4cc15de90 0x7fa4cc1596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4cb400640 1 -- 172.21.15.83:0/3946287691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc158570 con 0x7fa4cc071e50 2024-04-25T00:12:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4cb400640 1 --2- 172.21.15.83:0/3946287691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4cc071e50 0x7fa4cc14eb80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00c790 tx=0x7fa4bc00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4c3400640 1 -- 172.21.15.83:0/3946287691 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc016020 con 0x7fa4cc071e50 2024-04-25T00:12:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4c3400640 1 -- 172.21.15.83:0/3946287691 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4bc006d60 con 0x7fa4cc071e50 2024-04-25T00:12:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4d2200640 1 -- 172.21.15.83:0/3946287691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4cc158bc0 con 0x7fa4cc071e50 2024-04-25T00:12:27.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4c3400640 1 -- 172.21.15.83:0/3946287691 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc003bc0 con 0x7fa4cc071e50 2024-04-25T00:12:27.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4d2200640 1 -- 172.21.15.83:0/3946287691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4cc14f0c0 con 0x7fa4cc071e50 2024-04-25T00:12:27.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.458+0000 7fa4d2200640 1 -- 172.21.15.83:0/3946287691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4cc158fb0 con 0x7fa4cc071e50 2024-04-25T00:12:27.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.462+0000 7fa4c3400640 1 -- 172.21.15.83:0/3946287691 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa4bc00a850 con 0x7fa4cc071e50 2024-04-25T00:12:27.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.462+0000 7fa4c3400640 1 --2- 172.21.15.83:0/3946287691 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4ac0678c0 0x7fa4ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:27.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.462+0000 7fa4c3400640 1 -- 172.21.15.83:0/3946287691 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fa4bc085a80 con 0x7fa4cc071e50 2024-04-25T00:12:27.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.462+0000 7fa4caa00640 1 --2- 172.21.15.83:0/3946287691 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4ac0678c0 0x7fa4ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:27.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.462+0000 7fa4caa00640 1 --2- 172.21.15.83:0/3946287691 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4ac0678c0 0x7fa4ac069d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc14f820 tx=0x7fa4b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:27.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.466+0000 7fa4c3400640 1 -- 172.21.15.83:0/3946287691 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa4bc087e20 con 0x7fa4cc071e50 2024-04-25T00:12:27.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.622+0000 7fa4d2200640 1 -- 172.21.15.83:0/3946287691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=ms_async_reap_threshold}] v 0) -- 0x7fa4cc1593c0 con 0x7fa4cc071e50 2024-04-25T00:12:27.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.626+0000 7fa4c3400640 1 -- 172.21.15.83:0/3946287691 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=ms_async_reap_threshold}]=0 v11)=0 v11) ==== 121+0+0 (secure 0 0 0) 0x7fa4cc1593c0 con 0x7fa4cc071e50 2024-04-25T00:12:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.626+0000 7fa4d2200640 1 -- 172.21.15.83:0/3946287691 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4ac0678c0 msgr2=0x7fa4ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.626+0000 7fa4d2200640 1 --2- 172.21.15.83:0/3946287691 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4ac0678c0 0x7fa4ac069d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc14f820 tx=0x7fa4b8008040 comp rx=0 tx=0).stop 2024-04-25T00:12:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.626+0000 7fa4d2200640 1 -- 172.21.15.83:0/3946287691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4cc071e50 msgr2=0x7fa4cc14eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.626+0000 7fa4d2200640 1 --2- 172.21.15.83:0/3946287691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4cc071e50 0x7fa4cc14eb80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00c790 tx=0x7fa4bc00cc50 comp rx=0 tx=0).stop 2024-04-25T00:12:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.626+0000 7fa4d2200640 1 -- 172.21.15.83:0/3946287691 shutdown_connections 2024-04-25T00:12:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.626+0000 7fa4d2200640 1 --2- 172.21.15.83:0/3946287691 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4ac0678c0 0x7fa4ac069d80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.626+0000 7fa4d2200640 1 --2- 172.21.15.83:0/3946287691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4cc15e270 0x7fa4cc150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.626+0000 7fa4d2200640 1 --2- 172.21.15.83:0/3946287691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4cc15de90 0x7fa4cc1596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.626+0000 7fa4d2200640 1 --2- 172.21.15.83:0/3946287691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4cc071e50 0x7fa4cc14eb80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.626+0000 7fa4d2200640 1 -- 172.21.15.83:0/3946287691 >> 172.21.15.83:0/3946287691 conn(0x7fa4cc0b1830 msgr2=0x7fa4cc0b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.626+0000 7fa4d2200640 1 -- 172.21.15.83:0/3946287691 shutdown_connections 2024-04-25T00:12:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.626+0000 7fa4d2200640 1 -- 172.21.15.83:0/3946287691 wait complete. 2024-04-25T00:12:27.647 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:12:27.647 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:27.655 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:12:27.656 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:27.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:27.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c6a200640 1 Processor -- start 2024-04-25T00:12:27.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c6a200640 1 -- start start 2024-04-25T00:12:27.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c6a200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c640b4360 0x7f4c640b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:27.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c6a200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4c640b2b30 0x7f4c640b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:27.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c6a200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c6406ef80 con 0x7f4c641553e0 2024-04-25T00:12:27.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c6a200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4c6415b510 con 0x7f4c640b2b30 2024-04-25T00:12:27.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c6a200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6415b7f0 con 0x7f4c640b4360 2024-04-25T00:12:27.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c63e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4c640b2b30 0x7f4c640b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:27.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c63e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4c640b2b30 0x7f4c640b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52762/0 (socket says 172.21.15.83:52762) 2024-04-25T00:12:27.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c63e00640 1 -- 172.21.15.83:0/2976029009 learned_addr learned my addr 172.21.15.83:0/2976029009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:27.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c5aa00640 1 --2- 172.21.15.83:0/2976029009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c640b4360 0x7f4c640b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c63e00640 1 -- 172.21.15.83:0/2976029009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c640b4360 msgr2=0x7f4c640b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c63e00640 1 --2- 172.21.15.83:0/2976029009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c640b4360 0x7f4c640b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c63e00640 1 -- 172.21.15.83:0/2976029009 >> v1:172.21.15.86:6789/0 conn(0x7f4c641553e0 legacy=0x7f4c6406e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c62a00640 1 -- 172.21.15.83:0/2976029009 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1358375188 0 0) 0x7f4c6406ef80 con 0x7f4c641553e0 2024-04-25T00:12:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.886+0000 7f4c63e00640 1 -- 172.21.15.83:0/2976029009 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6415bad0 con 0x7f4c640b2b30 2024-04-25T00:12:27.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.890+0000 7f4c63e00640 1 --2- 172.21.15.83:0/2976029009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c640b2b30 0x7f4c640b2fc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009870 tx=0x7f4c5c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf23e7a93ffc25f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:27.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.890+0000 7f4c62a00640 1 -- 172.21.15.83:0/2976029009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c0b0070 con 0x7f4c640b2b30 2024-04-25T00:12:27.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.890+0000 7f4c62a00640 1 -- 172.21.15.83:0/2976029009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c5c0ab600 con 0x7f4c640b2b30 2024-04-25T00:12:27.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.890+0000 7f4c62a00640 1 -- 172.21.15.83:0/2976029009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c0ab9d0 con 0x7f4c640b2b30 2024-04-25T00:12:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.890+0000 7f4c6a200640 1 -- 172.21.15.83:0/2976029009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c640b2b30 msgr2=0x7f4c640b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.890+0000 7f4c6a200640 1 --2- 172.21.15.83:0/2976029009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c640b2b30 0x7f4c640b2fc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009870 tx=0x7f4c5c01c4d0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.890+0000 7f4c6a200640 1 -- 172.21.15.83:0/2976029009 shutdown_connections 2024-04-25T00:12:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.890+0000 7f4c6a200640 1 --2- 172.21.15.83:0/2976029009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c640b2b30 0x7f4c640b2fc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.890+0000 7f4c6a200640 1 --2- 172.21.15.83:0/2976029009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c640b4360 0x7f4c640b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.890+0000 7f4c6a200640 1 -- 172.21.15.83:0/2976029009 >> 172.21.15.83:0/2976029009 conn(0x7f4c640b1830 msgr2=0x7f4c640b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.890+0000 7f4c6a200640 1 -- 172.21.15.83:0/2976029009 shutdown_connections 2024-04-25T00:12:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.890+0000 7f4c6a200640 1 -- 172.21.15.83:0/2976029009 wait complete. 2024-04-25T00:12:27.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c6a200640 1 Processor -- start 2024-04-25T00:12:27.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c6a200640 1 -- start start 2024-04-25T00:12:27.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c6a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c640b2b30 0x7f4c64144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c6a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c640b4360 0x7f4c64149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c6a200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c641553e0 0x7f4c6414a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c6a200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6406ebd0 con 0x7f4c640b4360 2024-04-25T00:12:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c6a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64167d90 con 0x7f4c640b2b30 2024-04-25T00:12:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c6a200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6414dfe0 con 0x7f4c641553e0 2024-04-25T00:12:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c5aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c640b4360 0x7f4c64149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c5aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c640b4360 0x7f4c64149ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52774/0 (socket says 172.21.15.83:52774) 2024-04-25T00:12:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c5aa00640 1 -- 172.21.15.83:0/4118816060 learned_addr learned my addr 172.21.15.83:0/4118816060 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c63400640 1 --2- 172.21.15.83:0/4118816060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c640b2b30 0x7f4c64144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c63e00640 1 --2- 172.21.15.83:0/4118816060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c641553e0 0x7f4c6414a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c5aa00640 1 -- 172.21.15.83:0/4118816060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c641553e0 msgr2=0x7f4c6414a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c5aa00640 1 --2- 172.21.15.83:0/4118816060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c641553e0 0x7f4c6414a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c5aa00640 1 -- 172.21.15.83:0/4118816060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c640b2b30 msgr2=0x7f4c64144ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:27.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c5aa00640 1 --2- 172.21.15.83:0/4118816060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c640b2b30 0x7f4c64144ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:27.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c5aa00640 1 -- 172.21.15.83:0/4118816060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6414e2c0 con 0x7f4c640b4360 2024-04-25T00:12:27.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c5aa00640 1 --2- 172.21.15.83:0/4118816060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c640b4360 0x7f4c64149ed0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00e870 tx=0x7f4c4c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:27.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c5be00640 1 -- 172.21.15.83:0/4118816060 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c017070 con 0x7f4c640b4360 2024-04-25T00:12:27.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c5be00640 1 -- 172.21.15.83:0/4118816060 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c4c00c940 con 0x7f4c640b4360 2024-04-25T00:12:27.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c5be00640 1 -- 172.21.15.83:0/4118816060 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00cd10 con 0x7f4c640b4360 2024-04-25T00:12:27.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c6a200640 1 -- 172.21.15.83:0/4118816060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c640cd630 con 0x7f4c640b4360 2024-04-25T00:12:27.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.894+0000 7f4c6a200640 1 -- 172.21.15.83:0/4118816060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c640cdbf0 con 0x7f4c640b4360 2024-04-25T00:12:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.898+0000 7f4c6a200640 1 -- 172.21.15.83:0/4118816060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c640737f0 con 0x7f4c640b4360 2024-04-25T00:12:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.898+0000 7f4c5be00640 1 -- 172.21.15.83:0/4118816060 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4c4c01a070 con 0x7f4c640b4360 2024-04-25T00:12:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.898+0000 7f4c5be00640 1 --2- 172.21.15.83:0/4118816060 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4c400678c0 0x7f4c40069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.898+0000 7f4c5be00640 1 -- 172.21.15.83:0/4118816060 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f4c4c0860d0 con 0x7f4c640b4360 2024-04-25T00:12:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.898+0000 7f4c63400640 1 --2- 172.21.15.83:0/4118816060 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4c400678c0 0x7f4c40069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.898+0000 7f4c63400640 1 --2- 172.21.15.83:0/4118816060 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4c400678c0 0x7f4c40069d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4c64165d50 tx=0x7f4c54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:27.902+0000 7f4c5be00640 1 -- 172.21.15.83:0/4118816060 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4c4c056390 con 0x7f4c640b4360 2024-04-25T00:12:28.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.058+0000 7f4c6a200640 1 -- 172.21.15.83:0/4118816060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c6404f5f0 con 0x7f4c640b4360 2024-04-25T00:12:28.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.058+0000 7f4c5be00640 1 -- 172.21.15.83:0/4118816060 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16858 (secure 0 0 0) 0x7f4c4c05a690 con 0x7f4c640b4360 2024-04-25T00:12:28.061 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:28.061 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:20.181255+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:17.079304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:17.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:28.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.062+0000 7f4c6a200640 1 -- 172.21.15.83:0/4118816060 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4c400678c0 msgr2=0x7f4c40069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.062+0000 7f4c6a200640 1 --2- 172.21.15.83:0/4118816060 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4c400678c0 0x7f4c40069d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4c64165d50 tx=0x7f4c54009040 comp rx=0 tx=0).stop 2024-04-25T00:12:28.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.062+0000 7f4c6a200640 1 -- 172.21.15.83:0/4118816060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c640b4360 msgr2=0x7f4c64149ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.062+0000 7f4c6a200640 1 --2- 172.21.15.83:0/4118816060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c640b4360 0x7f4c64149ed0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00e870 tx=0x7f4c4c00ed30 comp rx=0 tx=0).stop 2024-04-25T00:12:28.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.062+0000 7f4c6a200640 1 -- 172.21.15.83:0/4118816060 shutdown_connections 2024-04-25T00:12:28.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.062+0000 7f4c6a200640 1 --2- 172.21.15.83:0/4118816060 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4c400678c0 0x7f4c40069d80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.062+0000 7f4c6a200640 1 --2- 172.21.15.83:0/4118816060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c641553e0 0x7f4c6414a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.062+0000 7f4c6a200640 1 --2- 172.21.15.83:0/4118816060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c640b4360 0x7f4c64149ed0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.062+0000 7f4c6a200640 1 --2- 172.21.15.83:0/4118816060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c640b2b30 0x7f4c64144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.062+0000 7f4c6a200640 1 -- 172.21.15.83:0/4118816060 >> 172.21.15.83:0/4118816060 conn(0x7f4c640b1830 msgr2=0x7f4c6414eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:28.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.062+0000 7f4c6a200640 1 -- 172.21.15.83:0/4118816060 shutdown_connections 2024-04-25T00:12:28.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.062+0000 7f4c6a200640 1 -- 172.21.15.83:0/4118816060 wait complete. 2024-04-25T00:12:28.082 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:12:17.079304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:12:17.548692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:12:28.083 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:12:28.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.218+0000 7f6bb6200640 1 Processor -- start 2024-04-25T00:12:28.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.218+0000 7f6bb6200640 1 -- start start 2024-04-25T00:12:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.222+0000 7f6bb6200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6bb01551e0 0x7f6bb015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.222+0000 7f6bb6200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6bb006e4c0 0x7f6bb006e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.222+0000 7f6bb6200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bb015c560 con 0x7f6bb01555c0 2024-04-25T00:12:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.222+0000 7f6bb6200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f6bb0072460 con 0x7f6bb006e4c0 2024-04-25T00:12:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.222+0000 7f6bb6200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6bb015c7d0 con 0x7f6bb01551e0 2024-04-25T00:12:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.222+0000 7f6bafe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6bb006e4c0 0x7f6bb006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.222+0000 7f6bafe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6bb006e4c0 0x7f6bb006e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52776/0 (socket says 172.21.15.83:52776) 2024-04-25T00:12:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.222+0000 7f6bafe00640 1 -- 172.21.15.83:0/1623961273 learned_addr learned my addr 172.21.15.83:0/1623961273 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.222+0000 7f6baf400640 1 --2- 172.21.15.83:0/1623961273 >> v2:172.21.15.86:3300/0 conn(0x7f6bb01551e0 0x7f6bb015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.222+0000 7f6bafe00640 1 -- 172.21.15.83:0/1623961273 >> v2:172.21.15.86:3300/0 conn(0x7f6bb01551e0 msgr2=0x7f6bb015bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:28.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.222+0000 7f6bafe00640 1 --2- 172.21.15.83:0/1623961273 >> v2:172.21.15.86:3300/0 conn(0x7f6bb01551e0 0x7f6bb015bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.222+0000 7f6bafe00640 1 -- 172.21.15.83:0/1623961273 >> v1:172.21.15.86:6789/0 conn(0x7f6bb01555c0 legacy=0x7f6bb0071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.222+0000 7f6bafe00640 1 -- 172.21.15.83:0/1623961273 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb015cab0 con 0x7f6bb006e4c0 2024-04-25T00:12:28.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.226+0000 7f6bafe00640 1 --2- 172.21.15.83:0/1623961273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb006e4c0 0x7f6bb006e950 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800d150 tx=0x7f6ba801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1321687ba048c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.226+0000 7f6bae000640 1 -- 172.21.15.83:0/1623961273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800b2d0 con 0x7f6bb006e4c0 2024-04-25T00:12:28.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.226+0000 7f6bae000640 1 -- 172.21.15.83:0/1623961273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba800b470 con 0x7f6bb006e4c0 2024-04-25T00:12:28.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.226+0000 7f6bae000640 1 -- 172.21.15.83:0/1623961273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80a79e0 con 0x7f6bb006e4c0 2024-04-25T00:12:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.226+0000 7f6bb6200640 1 -- 172.21.15.83:0/1623961273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb006e4c0 msgr2=0x7f6bb006e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.226+0000 7f6bb6200640 1 --2- 172.21.15.83:0/1623961273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb006e4c0 0x7f6bb006e950 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800d150 tx=0x7f6ba801ebf0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.226+0000 7f6bb6200640 1 -- 172.21.15.83:0/1623961273 shutdown_connections 2024-04-25T00:12:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.226+0000 7f6bb6200640 1 --2- 172.21.15.83:0/1623961273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb006e4c0 0x7f6bb006e950 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.226+0000 7f6bb6200640 1 --2- 172.21.15.83:0/1623961273 >> v2:172.21.15.86:3300/0 conn(0x7f6bb01551e0 0x7f6bb015bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.226+0000 7f6bb6200640 1 -- 172.21.15.83:0/1623961273 >> 172.21.15.83:0/1623961273 conn(0x7f6bb00b1830 msgr2=0x7f6bb00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.226+0000 7f6bb6200640 1 -- 172.21.15.83:0/1623961273 shutdown_connections 2024-04-25T00:12:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.226+0000 7f6bb6200640 1 -- 172.21.15.83:0/1623961273 wait complete. 2024-04-25T00:12:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bb6200640 1 Processor -- start 2024-04-25T00:12:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bb6200640 1 -- start start 2024-04-25T00:12:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bb6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bb006e4c0 0x7f6bb00a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bb6200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bb01551e0 0x7f6bb00aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bb6200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb01555c0 0x7f6bb00ab300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bb6200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb006df50 con 0x7f6bb01555c0 2024-04-25T00:12:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bb6200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb006ddd0 con 0x7f6bb006e4c0 2024-04-25T00:12:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bb6200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb006e0d0 con 0x7f6bb01551e0 2024-04-25T00:12:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bafe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb01555c0 0x7f6bb00ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bafe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb01555c0 0x7f6bb00ab300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52782/0 (socket says 172.21.15.83:52782) 2024-04-25T00:12:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bafe00640 1 -- 172.21.15.83:0/754172412 learned_addr learned my addr 172.21.15.83:0/754172412 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6baf400640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bb006e4c0 0x7f6bb00a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6baea00640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bb01551e0 0x7f6bb00aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bafe00640 1 -- 172.21.15.83:0/754172412 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bb01551e0 msgr2=0x7f6bb00aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bafe00640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bb01551e0 0x7f6bb00aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bafe00640 1 -- 172.21.15.83:0/754172412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bb006e4c0 msgr2=0x7f6bb00a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bafe00640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bb006e4c0 0x7f6bb00a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bafe00640 1 -- 172.21.15.83:0/754172412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb01511e0 con 0x7f6bb01555c0 2024-04-25T00:12:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6baf400640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bb006e4c0 0x7f6bb00a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6baea00640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bb01551e0 0x7f6bb00aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bafe00640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb01555c0 0x7f6bb00ab300 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80b1040 tx=0x7f6ba801b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6ba7400640 1 -- 172.21.15.83:0/754172412 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800b720 con 0x7f6bb01555c0 2024-04-25T00:12:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6ba7400640 1 -- 172.21.15.83:0/754172412 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba80be070 con 0x7f6bb01555c0 2024-04-25T00:12:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6ba7400640 1 -- 172.21.15.83:0/754172412 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8009a50 con 0x7f6bb01555c0 2024-04-25T00:12:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bb6200640 1 -- 172.21.15.83:0/754172412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb00a9b50 con 0x7f6bb01555c0 2024-04-25T00:12:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bb6200640 1 -- 172.21.15.83:0/754172412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb00aa110 con 0x7f6bb01555c0 2024-04-25T00:12:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.230+0000 7f6bb6200640 1 -- 172.21.15.83:0/754172412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb004f510 con 0x7f6bb01555c0 2024-04-25T00:12:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.234+0000 7f6ba7400640 1 -- 172.21.15.83:0/754172412 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6ba80b6070 con 0x7f6bb01555c0 2024-04-25T00:12:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.234+0000 7f6ba7400640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6b900678c0 0x7f6b90069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.234+0000 7f6ba7400640 1 -- 172.21.15.83:0/754172412 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f6ba8120ec0 con 0x7f6bb01555c0 2024-04-25T00:12:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.234+0000 7f6baf400640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6b900678c0 0x7f6b90069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.234+0000 7f6baf400640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6b900678c0 0x7f6b90069d80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009ae0 tx=0x7f6ba0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.238+0000 7f6ba7400640 1 -- 172.21.15.83:0/754172412 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6ba80187e0 con 0x7f6bb01555c0 2024-04-25T00:12:28.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.394+0000 7f6bb6200640 1 -- 172.21.15.83:0/754172412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6bb006ce30 con 0x7f6bb01555c0 2024-04-25T00:12:28.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.394+0000 7f6ba7400640 1 -- 172.21.15.83:0/754172412 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f6ba80f1890 con 0x7f6bb01555c0 2024-04-25T00:12:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:12:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.398+0000 7f6bb6200640 1 -- 172.21.15.83:0/754172412 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6b900678c0 msgr2=0x7f6b90069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.398+0000 7f6bb6200640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6b900678c0 0x7f6b90069d80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009ae0 tx=0x7f6ba0008040 comp rx=0 tx=0).stop 2024-04-25T00:12:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.398+0000 7f6bb6200640 1 -- 172.21.15.83:0/754172412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb01555c0 msgr2=0x7f6bb00ab300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.398+0000 7f6bb6200640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb01555c0 0x7f6bb00ab300 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80b1040 tx=0x7f6ba801b990 comp rx=0 tx=0).stop 2024-04-25T00:12:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.398+0000 7f6bb6200640 1 -- 172.21.15.83:0/754172412 shutdown_connections 2024-04-25T00:12:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.398+0000 7f6bb6200640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6b900678c0 0x7f6b90069d80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.398+0000 7f6bb6200640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bb01551e0 0x7f6bb00aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.398+0000 7f6bb6200640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb01555c0 0x7f6bb00ab300 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.398+0000 7f6bb6200640 1 --2- 172.21.15.83:0/754172412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bb006e4c0 0x7f6bb00a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.398+0000 7f6bb6200640 1 -- 172.21.15.83:0/754172412 >> 172.21.15.83:0/754172412 conn(0x7f6bb00b1830 msgr2=0x7f6bb00af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.398+0000 7f6bb6200640 1 -- 172.21.15.83:0/754172412 shutdown_connections 2024-04-25T00:12:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.398+0000 7f6bb6200640 1 -- 172.21.15.83:0/754172412 wait complete. 2024-04-25T00:12:28.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:12:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.554+0000 7fdc5d000640 1 Processor -- start 2024-04-25T00:12:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.554+0000 7fdc5d000640 1 -- start start 2024-04-25T00:12:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc5d000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc5815df80 0x7fdc580ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc5d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdc580b4240 0x7fdc580c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc5d000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5806cce0 0x7fdc5806d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc5d000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fdc580c7ff0 con 0x7fdc5815df80 2024-04-25T00:12:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc5d000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdc5815e360 con 0x7fdc580b4240 2024-04-25T00:12:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc5d000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdc580c73e0 con 0x7fdc5806cce0 2024-04-25T00:12:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc57400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc5815df80 0x7fdc580ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc57400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc5815df80 0x7fdc580ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52786/0 (socket says 172.21.15.83:52786) 2024-04-25T00:12:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc57400640 1 -- 172.21.15.83:0/2437427979 learned_addr learned my addr 172.21.15.83:0/2437427979 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc4ea00640 1 --2- 172.21.15.83:0/2437427979 >> v2:172.21.15.86:3300/0 conn(0x7fdc580b4240 0x7fdc580c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc57e00640 1 --2- 172.21.15.83:0/2437427979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5806cce0 0x7fdc5806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc57400640 1 -- 172.21.15.83:0/2437427979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5806cce0 msgr2=0x7fdc5806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc57400640 1 --2- 172.21.15.83:0/2437427979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5806cce0 0x7fdc5806d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc57400640 1 -- 172.21.15.83:0/2437427979 >> v2:172.21.15.86:3300/0 conn(0x7fdc580b4240 msgr2=0x7fdc580c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc57400640 1 --2- 172.21.15.83:0/2437427979 >> v2:172.21.15.86:3300/0 conn(0x7fdc580b4240 0x7fdc580c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc57400640 1 -- 172.21.15.83:0/2437427979 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc580c76c0 con 0x7fdc5815df80 2024-04-25T00:12:28.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.558+0000 7fdc57400640 1 --2- 172.21.15.83:0/2437427979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5815df80 0x7fdc580ceda0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdc5809fd50 tx=0x7fdc4801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac27d3c6a4e488a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc56a00640 1 -- 172.21.15.83:0/2437427979 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc480a56d0 con 0x7fdc5815df80 2024-04-25T00:12:28.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc56a00640 1 -- 172.21.15.83:0/2437427979 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc480a5870 con 0x7fdc5815df80 2024-04-25T00:12:28.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc56a00640 1 -- 172.21.15.83:0/2437427979 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc480a5c40 con 0x7fdc5815df80 2024-04-25T00:12:28.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 -- 172.21.15.83:0/2437427979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5815df80 msgr2=0x7fdc580ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 --2- 172.21.15.83:0/2437427979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5815df80 0x7fdc580ceda0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdc5809fd50 tx=0x7fdc4801c960 comp rx=0 tx=0).stop 2024-04-25T00:12:28.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 -- 172.21.15.83:0/2437427979 shutdown_connections 2024-04-25T00:12:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 --2- 172.21.15.83:0/2437427979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5806cce0 0x7fdc5806d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 --2- 172.21.15.83:0/2437427979 >> v2:172.21.15.86:3300/0 conn(0x7fdc580b4240 0x7fdc580c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 --2- 172.21.15.83:0/2437427979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5815df80 0x7fdc580ceda0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 -- 172.21.15.83:0/2437427979 >> 172.21.15.83:0/2437427979 conn(0x7fdc580b3720 msgr2=0x7fdc580b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 -- 172.21.15.83:0/2437427979 shutdown_connections 2024-04-25T00:12:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 -- 172.21.15.83:0/2437427979 wait complete. 2024-04-25T00:12:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 Processor -- start 2024-04-25T00:12:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 -- start start 2024-04-25T00:12:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5806cce0 0x7fdc580b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc580b4240 0x7fdc5809f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5815df80 0x7fdc5814a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc580d0070 con 0x7fdc5806cce0 2024-04-25T00:12:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc580cfef0 con 0x7fdc5815df80 2024-04-25T00:12:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc5d000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdc580d01f0 con 0x7fdc580b4240 2024-04-25T00:12:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc57400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5806cce0 0x7fdc580b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc57400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5806cce0 0x7fdc580b6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52788/0 (socket says 172.21.15.83:52788) 2024-04-25T00:12:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc57400640 1 -- 172.21.15.83:0/1811253136 learned_addr learned my addr 172.21.15.83:0/1811253136 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc57e00640 1 --2- 172.21.15.83:0/1811253136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5815df80 0x7fdc5814a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.562+0000 7fdc4ea00640 1 --2- 172.21.15.83:0/1811253136 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc580b4240 0x7fdc5809f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc57400640 1 -- 172.21.15.83:0/1811253136 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc580b4240 msgr2=0x7fdc5809f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc57400640 1 --2- 172.21.15.83:0/1811253136 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc580b4240 0x7fdc5809f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc57400640 1 -- 172.21.15.83:0/1811253136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5815df80 msgr2=0x7fdc5814a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc57400640 1 --2- 172.21.15.83:0/1811253136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5815df80 0x7fdc5814a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc57400640 1 -- 172.21.15.83:0/1811253136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5814a650 con 0x7fdc5806cce0 2024-04-25T00:12:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc57400640 1 --2- 172.21.15.83:0/1811253136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5806cce0 0x7fdc580b6e50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdc480af040 tx=0x7fdc48019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc4fe00640 1 -- 172.21.15.83:0/1811253136 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc480bc020 con 0x7fdc5806cce0 2024-04-25T00:12:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc4fe00640 1 -- 172.21.15.83:0/1811253136 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc480b8690 con 0x7fdc5806cce0 2024-04-25T00:12:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc4fe00640 1 -- 172.21.15.83:0/1811253136 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48009510 con 0x7fdc5806cce0 2024-04-25T00:12:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc5d000640 1 -- 172.21.15.83:0/1811253136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5814a8f0 con 0x7fdc5806cce0 2024-04-25T00:12:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc5d000640 1 -- 172.21.15.83:0/1811253136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc5814aeb0 con 0x7fdc5806cce0 2024-04-25T00:12:28.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc5d000640 1 -- 172.21.15.83:0/1811253136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5804f510 con 0x7fdc5806cce0 2024-04-25T00:12:28.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc4fe00640 1 -- 172.21.15.83:0/1811253136 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdc480bf020 con 0x7fdc5806cce0 2024-04-25T00:12:28.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc4fe00640 1 --2- 172.21.15.83:0/1811253136 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdc340678c0 0x7fdc34069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc4fe00640 1 -- 172.21.15.83:0/1811253136 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fdc48120a00 con 0x7fdc5806cce0 2024-04-25T00:12:28.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.566+0000 7fdc4ea00640 1 --2- 172.21.15.83:0/1811253136 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdc340678c0 0x7fdc34069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.570+0000 7fdc4ea00640 1 --2- 172.21.15.83:0/1811253136 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdc340678c0 0x7fdc34069d80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdc580b7a50 tx=0x7fdc40008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.570+0000 7fdc4fe00640 1 -- 172.21.15.83:0/1811253136 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdc480c6020 con 0x7fdc5806cce0 2024-04-25T00:12:28.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.726+0000 7fdc5d000640 1 -- 172.21.15.83:0/1811253136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdc580b3ef0 con 0x7fdc5806cce0 2024-04-25T00:12:28.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.726+0000 7fdc4fe00640 1 -- 172.21.15.83:0/1811253136 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) ==== 104+0+11 (secure 0 0 0) 0x7fdc480165e0 con 0x7fdc5806cce0 2024-04-25T00:12:28.729 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:12:28.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.730+0000 7fdc5d000640 1 -- 172.21.15.83:0/1811253136 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdc340678c0 msgr2=0x7fdc34069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.730+0000 7fdc5d000640 1 --2- 172.21.15.83:0/1811253136 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdc340678c0 0x7fdc34069d80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdc580b7a50 tx=0x7fdc40008040 comp rx=0 tx=0).stop 2024-04-25T00:12:28.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.730+0000 7fdc5d000640 1 -- 172.21.15.83:0/1811253136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5806cce0 msgr2=0x7fdc580b6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.730+0000 7fdc5d000640 1 --2- 172.21.15.83:0/1811253136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5806cce0 0x7fdc580b6e50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdc480af040 tx=0x7fdc48019700 comp rx=0 tx=0).stop 2024-04-25T00:12:28.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.730+0000 7fdc5d000640 1 -- 172.21.15.83:0/1811253136 shutdown_connections 2024-04-25T00:12:28.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.730+0000 7fdc5d000640 1 --2- 172.21.15.83:0/1811253136 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdc340678c0 0x7fdc34069d80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.730+0000 7fdc5d000640 1 --2- 172.21.15.83:0/1811253136 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc580b4240 0x7fdc5809f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.730+0000 7fdc5d000640 1 --2- 172.21.15.83:0/1811253136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5815df80 0x7fdc5814a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.730+0000 7fdc5d000640 1 --2- 172.21.15.83:0/1811253136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5806cce0 0x7fdc580b6e50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.730+0000 7fdc5d000640 1 -- 172.21.15.83:0/1811253136 >> 172.21.15.83:0/1811253136 conn(0x7fdc580b3720 msgr2=0x7fdc580d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:28.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.730+0000 7fdc5d000640 1 -- 172.21.15.83:0/1811253136 shutdown_connections 2024-04-25T00:12:28.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.730+0000 7fdc5d000640 1 -- 172.21.15.83:0/1811253136 wait complete. 2024-04-25T00:12:28.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:12:28.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.886+0000 7f34ea400640 1 Processor -- start 2024-04-25T00:12:28.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.886+0000 7f34ea400640 1 -- start start 2024-04-25T00:12:28.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.886+0000 7f34ea400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f34e41551e0 0x7f34e415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.886+0000 7f34ea400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34e415c560 con 0x7f34e406e4c0 2024-04-25T00:12:28.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.886+0000 7f34ea400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34e4072460 con 0x7f34e41555c0 2024-04-25T00:12:28.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.886+0000 7f34ea400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f34e415c7d0 con 0x7f34e41551e0 2024-04-25T00:12:28.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.886+0000 7f34e3e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f34e406e4c0 0x7f34e415cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42610/0 (socket says 172.21.15.83:42610) 2024-04-25T00:12:28.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.886+0000 7f34e3e00640 1 -- 172.21.15.83:0/908835671 learned_addr learned my addr 172.21.15.83:0/908835671 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:28.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e3400640 1 --2- 172.21.15.83:0/908835671 >> v2:172.21.15.86:3300/0 conn(0x7f34e41551e0 0x7f34e415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 948860 0 0) 0x7f34e415c560 con 0x7f34e406e4c0 2024-04-25T00:12:28.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34c8003530 con 0x7f34e406e4c0 2024-04-25T00:12:28.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2679920901 0 0) 0x7f34e4072460 con 0x7f34e41555c0 2024-04-25T00:12:28.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34e415c560 con 0x7f34e41555c0 2024-04-25T00:12:28.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3523455721 0 0) 0x7f34c8003530 con 0x7f34e406e4c0 2024-04-25T00:12:28.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 >> v2:172.21.15.86:3300/0 conn(0x7f34e41551e0 msgr2=0x7f34e415bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 --2- 172.21.15.83:0/908835671 >> v2:172.21.15.86:3300/0 conn(0x7f34e41551e0 0x7f34e415bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 >> v1:172.21.15.86:6789/0 conn(0x7f34e41555c0 legacy=0x7f34e4071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e415cab0 con 0x7f34e406e4c0 2024-04-25T00:12:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f34dc001680 con 0x7f34e406e4c0 2024-04-25T00:12:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 >> v1:172.21.15.83:6789/0 conn(0x7f34e406e4c0 legacy=0x7f34e415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 --2- 172.21.15.83:0/908835671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34c8003c10 0x7f34c8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 --2- 172.21.15.83:0/908835671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34c8004810 0x7f34c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 --2- 172.21.15.83:0/908835671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34c80051d0 0x7f34c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34e415c7d0 con 0x7f34c8003c10 2024-04-25T00:12:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34c800c170 con 0x7f34c8004810 2024-04-25T00:12:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f34c800c370 con 0x7f34c80051d0 2024-04-25T00:12:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e3400640 1 --2- 172.21.15.83:0/908835671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34c8003c10 0x7f34c8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34daa00640 1 --2- 172.21.15.83:0/908835671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34c8004810 0x7f34c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e3e00640 1 --2- 172.21.15.83:0/908835671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34c80051d0 0x7f34c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e3400640 1 -- 172.21.15.83:0/908835671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34c80051d0 msgr2=0x7f34c800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e3400640 1 --2- 172.21.15.83:0/908835671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34c80051d0 0x7f34c800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e3400640 1 -- 172.21.15.83:0/908835671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34c8004810 msgr2=0x7f34c8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e3400640 1 --2- 172.21.15.83:0/908835671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34c8004810 0x7f34c8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e3400640 1 -- 172.21.15.83:0/908835671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c8003530 con 0x7f34c8003c10 2024-04-25T00:12:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.890+0000 7f34e3e00640 1 -- 172.21.15.83:0/908835671 reap_dead start 2024-04-25T00:12:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34e3400640 1 --2- 172.21.15.83:0/908835671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34c8003c10 0x7f34c8004010 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009fc0 tx=0x7f34d401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8dc9d225437cad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d40b2070 con 0x7f34c8003c10 2024-04-25T00:12:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d40a5b80 con 0x7f34c8003c10 2024-04-25T00:12:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34e2a00640 1 -- 172.21.15.83:0/908835671 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4009c10 con 0x7f34c8003c10 2024-04-25T00:12:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 -- 172.21.15.83:0/908835671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34c8003c10 msgr2=0x7f34c8004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 --2- 172.21.15.83:0/908835671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34c8003c10 0x7f34c8004010 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009fc0 tx=0x7f34d401cc80 comp rx=0 tx=0).stop 2024-04-25T00:12:28.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 -- 172.21.15.83:0/908835671 shutdown_connections 2024-04-25T00:12:28.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 --2- 172.21.15.83:0/908835671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34c8003c10 0x7f34c8004010 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 -- 172.21.15.83:0/908835671 >> 172.21.15.83:0/908835671 conn(0x7f34e40b1830 msgr2=0x7f34e40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:28.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 -- 172.21.15.83:0/908835671 shutdown_connections 2024-04-25T00:12:28.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 -- 172.21.15.83:0/908835671 wait complete. 2024-04-25T00:12:28.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 Processor -- start 2024-04-25T00:12:28.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 -- start start 2024-04-25T00:12:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34c8003c10 0x7f34e40cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34e4140700 0x7f34e40d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34e40b4110 0x7f34e40b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34e415ec80 con 0x7f34e4140700 2024-04-25T00:12:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34e415eb00 con 0x7f34c8003c10 2024-04-25T00:12:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34ea400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f34e415ee00 con 0x7f34e40b4110 2024-04-25T00:12:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34daa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34e4140700 0x7f34e40d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34daa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34e4140700 0x7f34e40d03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52810/0 (socket says 172.21.15.83:52810) 2024-04-25T00:12:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34daa00640 1 -- 172.21.15.83:0/1447864364 learned_addr learned my addr 172.21.15.83:0/1447864364 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.894+0000 7f34e3400640 1 --2- 172.21.15.83:0/1447864364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34c8003c10 0x7f34e40cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34e3e00640 1 --2- 172.21.15.83:0/1447864364 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34e40b4110 0x7f34e40b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34daa00640 1 -- 172.21.15.83:0/1447864364 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34e40b4110 msgr2=0x7f34e40b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34daa00640 1 --2- 172.21.15.83:0/1447864364 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34e40b4110 0x7f34e40b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34daa00640 1 -- 172.21.15.83:0/1447864364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34c8003c10 msgr2=0x7f34e40cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34daa00640 1 --2- 172.21.15.83:0/1447864364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34c8003c10 0x7f34e40cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34daa00640 1 -- 172.21.15.83:0/1447864364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e414d4e0 con 0x7f34e4140700 2024-04-25T00:12:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34daa00640 1 --2- 172.21.15.83:0/1447864364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34e4140700 0x7f34e40d03f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00e580 tx=0x7f34cc00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34dbe00640 1 -- 172.21.15.83:0/1447864364 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc019070 con 0x7f34e4140700 2024-04-25T00:12:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34dbe00640 1 -- 172.21.15.83:0/1447864364 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34cc005320 con 0x7f34e4140700 2024-04-25T00:12:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34dbe00640 1 -- 172.21.15.83:0/1447864364 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc007270 con 0x7f34e4140700 2024-04-25T00:12:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34ea400640 1 -- 172.21.15.83:0/1447864364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e414d7c0 con 0x7f34e4140700 2024-04-25T00:12:28.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34ea400640 1 -- 172.21.15.83:0/1447864364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e414dd80 con 0x7f34e4140700 2024-04-25T00:12:28.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34ea400640 1 -- 172.21.15.83:0/1447864364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e404f510 con 0x7f34e4140700 2024-04-25T00:12:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34dbe00640 1 -- 172.21.15.83:0/1447864364 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f34cc01c070 con 0x7f34e4140700 2024-04-25T00:12:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34dbe00640 1 --2- 172.21.15.83:0/1447864364 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34c00678f0 0x7f34c0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34dbe00640 1 -- 172.21.15.83:0/1447864364 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f34cc085470 con 0x7f34e4140700 2024-04-25T00:12:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.898+0000 7f34e3400640 1 --2- 172.21.15.83:0/1447864364 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34c00678f0 0x7f34c0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.902+0000 7f34e3400640 1 --2- 172.21.15.83:0/1447864364 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34c00678f0 0x7f34c0069db0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f34d40af000 tx=0x7f34d4004ac0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:28.902+0000 7f34dbe00640 1 -- 172.21.15.83:0/1447864364 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f34cc055740 con 0x7f34e4140700 2024-04-25T00:12:29.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.058+0000 7f34ea400640 1 -- 172.21.15.83:0/1447864364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f34e40b3ef0 con 0x7f34e4140700 2024-04-25T00:12:29.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.058+0000 7f34dbe00640 1 -- 172.21.15.83:0/1447864364 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) ==== 100+0+11 (secure 0 0 0) 0x7f34cc059a40 con 0x7f34e4140700 2024-04-25T00:12:29.059 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:12:29.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.058+0000 7f34ea400640 1 -- 172.21.15.83:0/1447864364 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34c00678f0 msgr2=0x7f34c0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.058+0000 7f34ea400640 1 --2- 172.21.15.83:0/1447864364 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34c00678f0 0x7f34c0069db0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f34d40af000 tx=0x7f34d4004ac0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.058+0000 7f34ea400640 1 -- 172.21.15.83:0/1447864364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34e4140700 msgr2=0x7f34e40d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.058+0000 7f34ea400640 1 --2- 172.21.15.83:0/1447864364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34e4140700 0x7f34e40d03f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00e580 tx=0x7f34cc00ea40 comp rx=0 tx=0).stop 2024-04-25T00:12:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.058+0000 7f34ea400640 1 -- 172.21.15.83:0/1447864364 shutdown_connections 2024-04-25T00:12:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.058+0000 7f34ea400640 1 --2- 172.21.15.83:0/1447864364 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34c00678f0 0x7f34c0069db0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.058+0000 7f34ea400640 1 --2- 172.21.15.83:0/1447864364 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34e40b4110 0x7f34e40b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.058+0000 7f34ea400640 1 --2- 172.21.15.83:0/1447864364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34e4140700 0x7f34e40d03f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.058+0000 7f34ea400640 1 --2- 172.21.15.83:0/1447864364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34c8003c10 0x7f34e40cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.058+0000 7f34ea400640 1 -- 172.21.15.83:0/1447864364 >> 172.21.15.83:0/1447864364 conn(0x7f34e40b1830 msgr2=0x7f34e4154490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.058+0000 7f34ea400640 1 -- 172.21.15.83:0/1447864364 shutdown_connections 2024-04-25T00:12:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.058+0000 7f34ea400640 1 -- 172.21.15.83:0/1447864364 wait complete. 2024-04-25T00:12:29.079 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:12:29.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c9800640 1 Processor -- start 2024-04-25T00:12:29.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c9800640 1 -- start start 2024-04-25T00:12:29.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c9800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66c406ee20 0x7f66c40cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:29.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c9800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f66c415f350 0x7f66c4071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:29.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c9800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66c40cb210 con 0x7f66c41446e0 2024-04-25T00:12:29.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c9800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f66c40715b0 con 0x7f66c415f350 2024-04-25T00:12:29.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c9800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f66c4144ac0 con 0x7f66c406ee20 2024-04-25T00:12:29.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c3400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f66c415f350 0x7f66c4071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:29.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c3400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f66c415f350 0x7f66c4071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52818/0 (socket says 172.21.15.83:52818) 2024-04-25T00:12:29.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c3400640 1 -- 172.21.15.83:0/390557081 learned_addr learned my addr 172.21.15.83:0/390557081 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:29.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c2000640 1 --2- 172.21.15.83:0/390557081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66c406ee20 0x7f66c40cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:29.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c3400640 1 -- 172.21.15.83:0/390557081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66c406ee20 msgr2=0x7f66c40cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:29.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c3400640 1 --2- 172.21.15.83:0/390557081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66c406ee20 0x7f66c40cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c3400640 1 -- 172.21.15.83:0/390557081 >> v1:172.21.15.86:6789/0 conn(0x7f66c41446e0 legacy=0x7f66c406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:29.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.214+0000 7f66c3400640 1 -- 172.21.15.83:0/390557081 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c40cb4f0 con 0x7f66c415f350 2024-04-25T00:12:29.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.218+0000 7f66c3400640 1 --2- 172.21.15.83:0/390557081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c415f350 0x7f66c4071fe0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009870 tx=0x7f66bc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c212e9e6c72be7b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:29.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.218+0000 7f66c1600640 1 -- 172.21.15.83:0/390557081 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0ab430 con 0x7f66c415f350 2024-04-25T00:12:29.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.218+0000 7f66c1600640 1 -- 172.21.15.83:0/390557081 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66bc0b0070 con 0x7f66c415f350 2024-04-25T00:12:29.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.218+0000 7f66c1600640 1 -- 172.21.15.83:0/390557081 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0ab7f0 con 0x7f66c415f350 2024-04-25T00:12:29.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.218+0000 7f66c9800640 1 -- 172.21.15.83:0/390557081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c415f350 msgr2=0x7f66c4071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:29.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.218+0000 7f66c9800640 1 --2- 172.21.15.83:0/390557081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c415f350 0x7f66c4071fe0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009870 tx=0x7f66bc01c560 comp rx=0 tx=0).stop 2024-04-25T00:12:29.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.218+0000 7f66c9800640 1 -- 172.21.15.83:0/390557081 shutdown_connections 2024-04-25T00:12:29.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.218+0000 7f66c9800640 1 --2- 172.21.15.83:0/390557081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c415f350 0x7f66c4071fe0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.218+0000 7f66c9800640 1 --2- 172.21.15.83:0/390557081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66c406ee20 0x7f66c40cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.218+0000 7f66c9800640 1 -- 172.21.15.83:0/390557081 >> 172.21.15.83:0/390557081 conn(0x7f66c40cfa60 msgr2=0x7f66c40cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:29.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.218+0000 7f66c9800640 1 -- 172.21.15.83:0/390557081 shutdown_connections 2024-04-25T00:12:29.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.218+0000 7f66c9800640 1 -- 172.21.15.83:0/390557081 wait complete. 2024-04-25T00:12:29.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c9800640 1 Processor -- start 2024-04-25T00:12:29.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c9800640 1 -- start start 2024-04-25T00:12:29.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c9800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66c406ee20 0x7f66c4142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:29.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c41446e0 0x7f66c40b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:29.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c415f350 0x7f66c413a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c9800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66c4072eb0 con 0x7f66c41446e0 2024-04-25T00:12:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c9800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66c4072d30 con 0x7f66c415f350 2024-04-25T00:12:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c9800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f66c4073030 con 0x7f66c406ee20 2024-04-25T00:12:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c41446e0 0x7f66c40b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c41446e0 0x7f66c40b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52824/0 (socket says 172.21.15.83:52824) 2024-04-25T00:12:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c2000640 1 -- 172.21.15.83:0/1409141534 learned_addr learned my addr 172.21.15.83:0/1409141534 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c2a00640 1 --2- 172.21.15.83:0/1409141534 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66c406ee20 0x7f66c4142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c3400640 1 --2- 172.21.15.83:0/1409141534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c415f350 0x7f66c413a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c2000640 1 -- 172.21.15.83:0/1409141534 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66c406ee20 msgr2=0x7f66c4142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c2000640 1 --2- 172.21.15.83:0/1409141534 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66c406ee20 0x7f66c4142740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c2000640 1 -- 172.21.15.83:0/1409141534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c415f350 msgr2=0x7f66c413a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c2000640 1 --2- 172.21.15.83:0/1409141534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c415f350 0x7f66c413a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c2000640 1 -- 172.21.15.83:0/1409141534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c415ed40 con 0x7f66c41446e0 2024-04-25T00:12:29.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c2000640 1 --2- 172.21.15.83:0/1409141534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c41446e0 0x7f66c40b4be0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f66b000d800 tx=0x7f66b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:29.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66baa00640 1 -- 172.21.15.83:0/1409141534 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b000b6c0 con 0x7f66c41446e0 2024-04-25T00:12:29.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66baa00640 1 -- 172.21.15.83:0/1409141534 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b0019070 con 0x7f66c41446e0 2024-04-25T00:12:29.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c9800640 1 -- 172.21.15.83:0/1409141534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c413acf0 con 0x7f66c41446e0 2024-04-25T00:12:29.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66baa00640 1 -- 172.21.15.83:0/1409141534 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b000baa0 con 0x7f66c41446e0 2024-04-25T00:12:29.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.222+0000 7f66c9800640 1 -- 172.21.15.83:0/1409141534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c4141c60 con 0x7f66c41446e0 2024-04-25T00:12:29.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.226+0000 7f66c9800640 1 -- 172.21.15.83:0/1409141534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c4068230 con 0x7f66c41446e0 2024-04-25T00:12:29.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.226+0000 7f66baa00640 1 -- 172.21.15.83:0/1409141534 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f66b000bc40 con 0x7f66c41446e0 2024-04-25T00:12:29.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.226+0000 7f66baa00640 1 --2- 172.21.15.83:0/1409141534 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66a40678c0 0x7f66a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:29.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.226+0000 7f66baa00640 1 -- 172.21.15.83:0/1409141534 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f66b00861d0 con 0x7f66c41446e0 2024-04-25T00:12:29.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.226+0000 7f66c2a00640 1 --2- 172.21.15.83:0/1409141534 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66a40678c0 0x7f66a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:29.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.226+0000 7f66c2a00640 1 --2- 172.21.15.83:0/1409141534 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66a40678c0 0x7f66a4069d80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f66c415ef60 tx=0x7f66b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:29.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.230+0000 7f66baa00640 1 -- 172.21.15.83:0/1409141534 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f66b0056490 con 0x7f66c41446e0 2024-04-25T00:12:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.402+0000 7f66c9800640 1 -- 172.21.15.83:0/1409141534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f66c40afe00 con 0x7f66c41446e0 2024-04-25T00:12:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.402+0000 7f66baa00640 1 -- 172.21.15.83:0/1409141534 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) ==== 80+0+0 (secure 0 0 0) 0x7f66b005a790 con 0x7f66c41446e0 2024-04-25T00:12:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:12:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.402+0000 7f66c9800640 1 -- 172.21.15.83:0/1409141534 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66a40678c0 msgr2=0x7f66a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.402+0000 7f66c9800640 1 --2- 172.21.15.83:0/1409141534 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66a40678c0 0x7f66a4069d80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f66c415ef60 tx=0x7f66b4009040 comp rx=0 tx=0).stop 2024-04-25T00:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.402+0000 7f66c9800640 1 -- 172.21.15.83:0/1409141534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c41446e0 msgr2=0x7f66c40b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.402+0000 7f66c9800640 1 --2- 172.21.15.83:0/1409141534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c41446e0 0x7f66c40b4be0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f66b000d800 tx=0x7f66b000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.402+0000 7f66c9800640 1 -- 172.21.15.83:0/1409141534 shutdown_connections 2024-04-25T00:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.402+0000 7f66c9800640 1 --2- 172.21.15.83:0/1409141534 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66a40678c0 0x7f66a4069d80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.402+0000 7f66c9800640 1 --2- 172.21.15.83:0/1409141534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c415f350 0x7f66c413a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.402+0000 7f66c9800640 1 --2- 172.21.15.83:0/1409141534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66c41446e0 0x7f66c40b4be0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.402+0000 7f66c9800640 1 --2- 172.21.15.83:0/1409141534 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66c406ee20 0x7f66c4142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.402+0000 7f66c9800640 1 -- 172.21.15.83:0/1409141534 >> 172.21.15.83:0/1409141534 conn(0x7f66c40cfa60 msgr2=0x7f66c406d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.402+0000 7f66c9800640 1 -- 172.21.15.83:0/1409141534 shutdown_connections 2024-04-25T00:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.402+0000 7f66c9800640 1 -- 172.21.15.83:0/1409141534 wait complete. 2024-04-25T00:12:29.424 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:12:29.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:29.424 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:12:29.451 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:12:29.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:29.451 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:12:29.501 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:12:29.501 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:29.511 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:12:29.511 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:29.561 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:29.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.694+0000 7fab9fe00640 1 Processor -- start 2024-04-25T00:12:29.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.694+0000 7fab9fe00640 1 -- start start 2024-04-25T00:12:29.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.698+0000 7fab9fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fab981599f0 0x7fab980cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:29.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.698+0000 7fab9fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab9815cec0 0x7fab980ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:29.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.698+0000 7fab9fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab980ccc60 con 0x7fab9815c340 2024-04-25T00:12:29.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.698+0000 7fab9fe00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fab98159dd0 con 0x7fab981599f0 2024-04-25T00:12:29.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.698+0000 7fab9fe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fab9806dba0 con 0x7fab9815cec0 2024-04-25T00:12:29.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.698+0000 7fab9d400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fab981599f0 0x7fab980cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:29.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.698+0000 7fab9d400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fab981599f0 0x7fab980cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52830/0 (socket says 172.21.15.83:52830) 2024-04-25T00:12:29.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.698+0000 7fab9d400640 1 -- 172.21.15.83:0/656203269 learned_addr learned my addr 172.21.15.83:0/656203269 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:29.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.698+0000 7fab9d400640 1 -- 172.21.15.83:0/656203269 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab9815cec0 msgr2=0x7fab980ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:29.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.698+0000 7fab9d400640 1 --2- 172.21.15.83:0/656203269 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab9815cec0 0x7fab980ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.698+0000 7fab9d400640 1 -- 172.21.15.83:0/656203269 >> v1:172.21.15.86:6789/0 conn(0x7fab9815c340 legacy=0x7fab9815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:29.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.698+0000 7fab9d400640 1 -- 172.21.15.83:0/656203269 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9806de80 con 0x7fab981599f0 2024-04-25T00:12:29.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.702+0000 7fab9d400640 1 --2- 172.21.15.83:0/656203269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab981599f0 0x7fab980cc510 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0098a0 tx=0x7fab8c01c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c512ad8b88fa65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:29.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.702+0000 7fab97e00640 1 -- 172.21.15.83:0/656203269 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0b0070 con 0x7fab981599f0 2024-04-25T00:12:29.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.702+0000 7fab97e00640 1 -- 172.21.15.83:0/656203269 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c0ab600 con 0x7fab981599f0 2024-04-25T00:12:29.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.702+0000 7fab97e00640 1 -- 172.21.15.83:0/656203269 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0ab9d0 con 0x7fab981599f0 2024-04-25T00:12:29.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.702+0000 7fab9fe00640 1 -- 172.21.15.83:0/656203269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab981599f0 msgr2=0x7fab980cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:29.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.702+0000 7fab9fe00640 1 --2- 172.21.15.83:0/656203269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab981599f0 0x7fab980cc510 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0098a0 tx=0x7fab8c01c290 comp rx=0 tx=0).stop 2024-04-25T00:12:29.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.702+0000 7fab9fe00640 1 -- 172.21.15.83:0/656203269 shutdown_connections 2024-04-25T00:12:29.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.702+0000 7fab9fe00640 1 --2- 172.21.15.83:0/656203269 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab9815cec0 0x7fab980ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.702+0000 7fab9fe00640 1 --2- 172.21.15.83:0/656203269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab981599f0 0x7fab980cc510 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.702+0000 7fab9fe00640 1 -- 172.21.15.83:0/656203269 >> 172.21.15.83:0/656203269 conn(0x7fab980b1830 msgr2=0x7fab980b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:29.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.702+0000 7fab9fe00640 1 -- 172.21.15.83:0/656203269 shutdown_connections 2024-04-25T00:12:29.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.702+0000 7fab9fe00640 1 -- 172.21.15.83:0/656203269 wait complete. 2024-04-25T00:12:29.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.702+0000 7fab9fe00640 1 Processor -- start 2024-04-25T00:12:29.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.702+0000 7fab9fe00640 1 -- start start 2024-04-25T00:12:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab981599f0 0x7fab9814c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab9815c340 0x7fab98164e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab9815cec0 0x7fab981653d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab980cf800 con 0x7fab9815cec0 2024-04-25T00:12:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab980cf680 con 0x7fab981599f0 2024-04-25T00:12:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9fe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fab98168cd0 con 0x7fab9815c340 2024-04-25T00:12:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab9815cec0 0x7fab981653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab9815cec0 0x7fab981653d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52842/0 (socket says 172.21.15.83:52842) 2024-04-25T00:12:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9de00640 1 -- 172.21.15.83:0/805007340 learned_addr learned my addr 172.21.15.83:0/805007340 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9ca00640 1 --2- 172.21.15.83:0/805007340 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab9815c340 0x7fab98164e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9d400640 1 --2- 172.21.15.83:0/805007340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab981599f0 0x7fab9814c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9de00640 1 -- 172.21.15.83:0/805007340 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab9815c340 msgr2=0x7fab98164e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9de00640 1 --2- 172.21.15.83:0/805007340 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab9815c340 0x7fab98164e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9de00640 1 -- 172.21.15.83:0/805007340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab981599f0 msgr2=0x7fab9814c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9de00640 1 --2- 172.21.15.83:0/805007340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab981599f0 0x7fab9814c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9de00640 1 -- 172.21.15.83:0/805007340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab98165a00 con 0x7fab9815cec0 2024-04-25T00:12:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9de00640 1 --2- 172.21.15.83:0/805007340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab9815cec0 0x7fab981653d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fab9000b970 tx=0x7fab9000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:29.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab95600640 1 -- 172.21.15.83:0/805007340 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90019070 con 0x7fab9815cec0 2024-04-25T00:12:29.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab95600640 1 -- 172.21.15.83:0/805007340 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9000cb60 con 0x7fab9815cec0 2024-04-25T00:12:29.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab95600640 1 -- 172.21.15.83:0/805007340 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90014440 con 0x7fab9815cec0 2024-04-25T00:12:29.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9fe00640 1 -- 172.21.15.83:0/805007340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab98165ce0 con 0x7fab9815cec0 2024-04-25T00:12:29.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9fe00640 1 -- 172.21.15.83:0/805007340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab981662a0 con 0x7fab9815cec0 2024-04-25T00:12:29.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab9fe00640 1 -- 172.21.15.83:0/805007340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab980737f0 con 0x7fab9815cec0 2024-04-25T00:12:29.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.706+0000 7fab95600640 1 -- 172.21.15.83:0/805007340 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fab9001c070 con 0x7fab9815cec0 2024-04-25T00:12:29.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.710+0000 7fab95600640 1 --2- 172.21.15.83:0/805007340 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab780678c0 0x7fab78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:29.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.710+0000 7fab95600640 1 -- 172.21.15.83:0/805007340 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fab90085b90 con 0x7fab9815cec0 2024-04-25T00:12:29.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.710+0000 7fab9d400640 1 --2- 172.21.15.83:0/805007340 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab780678c0 0x7fab78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:29.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.710+0000 7fab9d400640 1 --2- 172.21.15.83:0/805007340 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab780678c0 0x7fab78069d80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0ad000 tx=0x7fab8c0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:29.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.710+0000 7fab95600640 1 -- 172.21.15.83:0/805007340 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fab90055e50 con 0x7fab9815cec0 2024-04-25T00:12:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.866+0000 7fab9fe00640 1 -- 172.21.15.83:0/805007340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab9804f5f0 con 0x7fab9815cec0 2024-04-25T00:12:29.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.866+0000 7fab95600640 1 -- 172.21.15.83:0/805007340 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16858 (secure 0 0 0) 0x7fab9005a150 con 0x7fab9815cec0 2024-04-25T00:12:29.869 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:29.869 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:20.181255+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:17.079304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:17.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.870+0000 7fab9fe00640 1 -- 172.21.15.83:0/805007340 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab780678c0 msgr2=0x7fab78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.870+0000 7fab9fe00640 1 --2- 172.21.15.83:0/805007340 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab780678c0 0x7fab78069d80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0ad000 tx=0x7fab8c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:12:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.870+0000 7fab9fe00640 1 -- 172.21.15.83:0/805007340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab9815cec0 msgr2=0x7fab981653d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.870+0000 7fab9fe00640 1 --2- 172.21.15.83:0/805007340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab9815cec0 0x7fab981653d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fab9000b970 tx=0x7fab9000be30 comp rx=0 tx=0).stop 2024-04-25T00:12:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.870+0000 7fab9fe00640 1 -- 172.21.15.83:0/805007340 shutdown_connections 2024-04-25T00:12:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.870+0000 7fab9fe00640 1 --2- 172.21.15.83:0/805007340 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab780678c0 0x7fab78069d80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.870+0000 7fab9fe00640 1 --2- 172.21.15.83:0/805007340 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab9815c340 0x7fab98164e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.870+0000 7fab9fe00640 1 --2- 172.21.15.83:0/805007340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab9815cec0 0x7fab981653d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.870+0000 7fab9fe00640 1 --2- 172.21.15.83:0/805007340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab981599f0 0x7fab9814c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.870+0000 7fab9fe00640 1 -- 172.21.15.83:0/805007340 >> 172.21.15.83:0/805007340 conn(0x7fab980b1830 msgr2=0x7fab980b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.870+0000 7fab9fe00640 1 -- 172.21.15.83:0/805007340 shutdown_connections 2024-04-25T00:12:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:29.870+0000 7fab9fe00640 1 -- 172.21.15.83:0/805007340 wait complete. 2024-04-25T00:12:29.890 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:12:17.079304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:12:17.548692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:12:29.891 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:12:30.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72b2800640 1 Processor -- start 2024-04-25T00:12:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72b2800640 1 -- start start 2024-04-25T00:12:30.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72b2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f72ac15de90 0x7f72ac071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72b2800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72ac15e270 0x7f72ac071c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72b2800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72ac06dcf0 con 0x7f72ac06e840 2024-04-25T00:12:30.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72b2800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f72ac0b4460 con 0x7f72ac15de90 2024-04-25T00:12:30.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72b2800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f72ac0b2ce0 con 0x7f72ac15e270 2024-04-25T00:12:30.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72abe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f72ac06e840 0x7f72ac161bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42612/0 (socket says 172.21.15.83:42612) 2024-04-25T00:12:30.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72abe00640 1 -- 172.21.15.83:0/7833889 learned_addr learned my addr 172.21.15.83:0/7833889 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:30.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72ab400640 1 --2- 172.21.15.83:0/7833889 >> v2:172.21.15.86:3300/0 conn(0x7f72ac15de90 0x7f72ac071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72a2a00640 1 --2- 172.21.15.83:0/7833889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72ac15e270 0x7f72ac071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72aaa00640 1 -- 172.21.15.83:0/7833889 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 834421528 0 0) 0x7f72ac06dcf0 con 0x7f72ac06e840 2024-04-25T00:12:30.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72aaa00640 1 -- 172.21.15.83:0/7833889 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7290003530 con 0x7f72ac06e840 2024-04-25T00:12:30.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72ab400640 1 -- 172.21.15.83:0/7833889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72ac15e270 msgr2=0x7f72ac071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:30.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72ab400640 1 --2- 172.21.15.83:0/7833889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72ac15e270 0x7f72ac071c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72aaa00640 1 -- 172.21.15.83:0/7833889 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2604965979 0 0) 0x7f7290003530 con 0x7f72ac06e840 2024-04-25T00:12:30.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72ab400640 1 -- 172.21.15.83:0/7833889 >> v1:172.21.15.83:6789/0 conn(0x7f72ac06e840 legacy=0x7f72ac161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.030+0000 7f72ab400640 1 -- 172.21.15.83:0/7833889 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ac06dfd0 con 0x7f72ac15de90 2024-04-25T00:12:30.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.034+0000 7f72ab400640 1 --2- 172.21.15.83:0/7833889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ac15de90 0x7f72ac071740 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f729c0097c0 tx=0x7f729c01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77db3b2354a405f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:30.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.034+0000 7f72aaa00640 1 -- 172.21.15.83:0/7833889 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c0b0070 con 0x7f72ac15de90 2024-04-25T00:12:30.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.034+0000 7f72aaa00640 1 -- 172.21.15.83:0/7833889 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f729c0ab400 con 0x7f72ac15de90 2024-04-25T00:12:30.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.034+0000 7f72aaa00640 1 -- 172.21.15.83:0/7833889 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4001a00 con 0x7f72ac15de90 2024-04-25T00:12:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.034+0000 7f72b2800640 1 -- 172.21.15.83:0/7833889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ac15de90 msgr2=0x7f72ac071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.034+0000 7f72b2800640 1 --2- 172.21.15.83:0/7833889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ac15de90 0x7f72ac071740 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f729c0097c0 tx=0x7f729c01c330 comp rx=0 tx=0).stop 2024-04-25T00:12:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.034+0000 7f72b2800640 1 -- 172.21.15.83:0/7833889 shutdown_connections 2024-04-25T00:12:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.034+0000 7f72b2800640 1 --2- 172.21.15.83:0/7833889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72ac15e270 0x7f72ac071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.034+0000 7f72b2800640 1 --2- 172.21.15.83:0/7833889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ac15de90 0x7f72ac071740 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.034+0000 7f72b2800640 1 -- 172.21.15.83:0/7833889 >> 172.21.15.83:0/7833889 conn(0x7f72ac0b1830 msgr2=0x7f72ac0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.034+0000 7f72b2800640 1 -- 172.21.15.83:0/7833889 shutdown_connections 2024-04-25T00:12:30.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.034+0000 7f72b2800640 1 -- 172.21.15.83:0/7833889 wait complete. 2024-04-25T00:12:30.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72b2800640 1 Processor -- start 2024-04-25T00:12:30.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72b2800640 1 -- start start 2024-04-25T00:12:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72b2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72ac06e840 0x7f72ac15e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72b2800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72ac15de90 0x7f72ac14e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72b2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ac15e270 0x7f72ac14ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72b2800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac16ad60 con 0x7f72ac06e840 2024-04-25T00:12:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72b2800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac0ad860 con 0x7f72ac15e270 2024-04-25T00:12:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72b2800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f72ac16aee0 con 0x7f72ac15de90 2024-04-25T00:12:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72ab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72ac06e840 0x7f72ac15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72ab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72ac06e840 0x7f72ac15e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52858/0 (socket says 172.21.15.83:52858) 2024-04-25T00:12:30.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72ab400640 1 -- 172.21.15.83:0/1526161843 learned_addr learned my addr 172.21.15.83:0/1526161843 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:30.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72abe00640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ac15e270 0x7f72ac14ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72a2a00640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72ac15de90 0x7f72ac14e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72ab400640 1 -- 172.21.15.83:0/1526161843 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72ac15de90 msgr2=0x7f72ac14e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72ab400640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72ac15de90 0x7f72ac14e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72ab400640 1 -- 172.21.15.83:0/1526161843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ac15e270 msgr2=0x7f72ac14ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72ab400640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ac15e270 0x7f72ac14ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72ab400640 1 -- 172.21.15.83:0/1526161843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ac14ed90 con 0x7f72ac06e840 2024-04-25T00:12:30.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72abe00640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ac15e270 0x7f72ac14ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:30.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72ab400640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72ac06e840 0x7f72ac15e650 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f729c0ae040 tx=0x7f729c01ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:30.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72a3e00640 1 -- 172.21.15.83:0/1526161843 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c0b0050 con 0x7f72ac06e840 2024-04-25T00:12:30.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72a3e00640 1 -- 172.21.15.83:0/1526161843 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f729c0bc070 con 0x7f72ac06e840 2024-04-25T00:12:30.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72b2800640 1 -- 172.21.15.83:0/1526161843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ac150810 con 0x7f72ac06e840 2024-04-25T00:12:30.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72a3e00640 1 -- 172.21.15.83:0/1526161843 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c0b7690 con 0x7f72ac06e840 2024-04-25T00:12:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.038+0000 7f72b2800640 1 -- 172.21.15.83:0/1526161843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ac1632e0 con 0x7f72ac06e840 2024-04-25T00:12:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.042+0000 7f72b2800640 1 -- 172.21.15.83:0/1526161843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72ac04f510 con 0x7f72ac06e840 2024-04-25T00:12:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.042+0000 7f72a3e00640 1 -- 172.21.15.83:0/1526161843 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f729c0b7830 con 0x7f72ac06e840 2024-04-25T00:12:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.042+0000 7f72a3e00640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f72880678c0 0x7f7288069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.042+0000 7f72a3e00640 1 -- 172.21.15.83:0/1526161843 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f729c120450 con 0x7f72ac06e840 2024-04-25T00:12:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.042+0000 7f72a2a00640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f72880678c0 0x7f7288069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.042+0000 7f72a2a00640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f72880678c0 0x7f7288069d80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f72ac15d890 tx=0x7f7294008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:30.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.046+0000 7f72a3e00640 1 -- 172.21.15.83:0/1526161843 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f72ac150810 con 0x7f72ac06e840 2024-04-25T00:12:30.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.202+0000 7f72b2800640 1 -- 172.21.15.83:0/1526161843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f72ac156e60 con 0x7f72ac06e840 2024-04-25T00:12:30.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.202+0000 7f72a3e00640 1 -- 172.21.15.83:0/1526161843 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f729c015fb0 con 0x7f72ac06e840 2024-04-25T00:12:30.205 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:12:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.206+0000 7f72b2800640 1 -- 172.21.15.83:0/1526161843 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f72880678c0 msgr2=0x7f7288069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.206+0000 7f72b2800640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f72880678c0 0x7f7288069d80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f72ac15d890 tx=0x7f7294008040 comp rx=0 tx=0).stop 2024-04-25T00:12:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.206+0000 7f72b2800640 1 -- 172.21.15.83:0/1526161843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72ac06e840 msgr2=0x7f72ac15e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.206+0000 7f72b2800640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72ac06e840 0x7f72ac15e650 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f729c0ae040 tx=0x7f729c01ab70 comp rx=0 tx=0).stop 2024-04-25T00:12:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.206+0000 7f72b2800640 1 -- 172.21.15.83:0/1526161843 shutdown_connections 2024-04-25T00:12:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.206+0000 7f72b2800640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f72880678c0 0x7f7288069d80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.206+0000 7f72b2800640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72ac15de90 0x7f72ac14e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.206+0000 7f72b2800640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ac15e270 0x7f72ac14ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.206+0000 7f72b2800640 1 --2- 172.21.15.83:0/1526161843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72ac06e840 0x7f72ac15e650 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.206+0000 7f72b2800640 1 -- 172.21.15.83:0/1526161843 >> 172.21.15.83:0/1526161843 conn(0x7f72ac0b1830 msgr2=0x7f72ac0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:30.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.206+0000 7f72b2800640 1 -- 172.21.15.83:0/1526161843 shutdown_connections 2024-04-25T00:12:30.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.206+0000 7f72b2800640 1 -- 172.21.15.83:0/1526161843 wait complete. 2024-04-25T00:12:30.226 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:12:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.362+0000 7f8f5be00640 1 Processor -- start 2024-04-25T00:12:30.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.362+0000 7f8f5be00640 1 -- start start 2024-04-25T00:12:30.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.362+0000 7f8f5be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8f54159810 0x7f8f540cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.362+0000 7f8f5be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f5406e950 0x7f8f5415f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.362+0000 7f8f5be00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f540ce000 con 0x7f8f5415fe50 2024-04-25T00:12:30.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.362+0000 7f8f5be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8f5406e210 con 0x7f8f54159810 2024-04-25T00:12:30.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.362+0000 7f8f5be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8f5406da00 con 0x7f8f5406e950 2024-04-25T00:12:30.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.362+0000 7f8f59e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8f5415fe50 0x7f8f5415c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42628/0 (socket says 172.21.15.83:42628) 2024-04-25T00:12:30.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.362+0000 7f8f59e00640 1 -- 172.21.15.83:0/2895868379 learned_addr learned my addr 172.21.15.83:0/2895868379 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:30.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.366+0000 7f8f59400640 1 --2- 172.21.15.83:0/2895868379 >> v2:172.21.15.86:3300/0 conn(0x7f8f54159810 0x7f8f540cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.366+0000 7f8f58a00640 1 --2- 172.21.15.83:0/2895868379 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f5406e950 0x7f8f5415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.366+0000 7f8f53e00640 1 -- 172.21.15.83:0/2895868379 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4202429617 0 0) 0x7f8f540ce000 con 0x7f8f5415fe50 2024-04-25T00:12:30.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.366+0000 7f8f53e00640 1 -- 172.21.15.83:0/2895868379 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f3c003530 con 0x7f8f5415fe50 2024-04-25T00:12:30.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.366+0000 7f8f53e00640 1 -- 172.21.15.83:0/2895868379 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1517556751 0 0) 0x7f8f3c003530 con 0x7f8f5415fe50 2024-04-25T00:12:30.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.366+0000 7f8f58a00640 1 -- 172.21.15.83:0/2895868379 >> v2:172.21.15.86:3300/0 conn(0x7f8f54159810 msgr2=0x7f8f540cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:30.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.366+0000 7f8f58a00640 1 --2- 172.21.15.83:0/2895868379 >> v2:172.21.15.86:3300/0 conn(0x7f8f54159810 0x7f8f540cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.366+0000 7f8f58a00640 1 -- 172.21.15.83:0/2895868379 >> v1:172.21.15.83:6789/0 conn(0x7f8f5415fe50 legacy=0x7f8f5415c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.366+0000 7f8f58a00640 1 -- 172.21.15.83:0/2895868379 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5406eec0 con 0x7f8f5406e950 2024-04-25T00:12:30.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.366+0000 7f8f58a00640 1 --2- 172.21.15.83:0/2895868379 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f5406e950 0x7f8f5415f910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f400097c0 tx=0x7f8f4001c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0f690ed53722fdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:30.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.366+0000 7f8f53e00640 1 -- 172.21.15.83:0/2895868379 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f400b0070 con 0x7f8f5406e950 2024-04-25T00:12:30.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.366+0000 7f8f53e00640 1 -- 172.21.15.83:0/2895868379 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f400ab400 con 0x7f8f5406e950 2024-04-25T00:12:30.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f53e00640 1 -- 172.21.15.83:0/2895868379 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c001a00 con 0x7f8f5406e950 2024-04-25T00:12:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 -- 172.21.15.83:0/2895868379 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f5406e950 msgr2=0x7f8f5415f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 --2- 172.21.15.83:0/2895868379 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f5406e950 0x7f8f5415f910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f400097c0 tx=0x7f8f4001c330 comp rx=0 tx=0).stop 2024-04-25T00:12:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 -- 172.21.15.83:0/2895868379 shutdown_connections 2024-04-25T00:12:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 --2- 172.21.15.83:0/2895868379 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f5406e950 0x7f8f5415f910 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 --2- 172.21.15.83:0/2895868379 >> v2:172.21.15.86:3300/0 conn(0x7f8f54159810 0x7f8f540cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 -- 172.21.15.83:0/2895868379 >> 172.21.15.83:0/2895868379 conn(0x7f8f540b1830 msgr2=0x7f8f540b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 -- 172.21.15.83:0/2895868379 shutdown_connections 2024-04-25T00:12:30.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 -- 172.21.15.83:0/2895868379 wait complete. 2024-04-25T00:12:30.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 Processor -- start 2024-04-25T00:12:30.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 -- start start 2024-04-25T00:12:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5406e950 0x7f8f54166510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f54159810 0x7f8f540cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f5415fe50 0x7f8f540cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f54159d70 con 0x7f8f5415fe50 2024-04-25T00:12:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f54159bf0 con 0x7f8f5406e950 2024-04-25T00:12:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f5be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8f54070da0 con 0x7f8f54159810 2024-04-25T00:12:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f59e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f5415fe50 0x7f8f540cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f59e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f5415fe50 0x7f8f540cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52864/0 (socket says 172.21.15.83:52864) 2024-04-25T00:12:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f59e00640 1 -- 172.21.15.83:0/2656534029 learned_addr learned my addr 172.21.15.83:0/2656534029 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f58a00640 1 --2- 172.21.15.83:0/2656534029 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f54159810 0x7f8f540cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.370+0000 7f8f59400640 1 --2- 172.21.15.83:0/2656534029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5406e950 0x7f8f54166510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.374+0000 7f8f59e00640 1 -- 172.21.15.83:0/2656534029 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f54159810 msgr2=0x7f8f540cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.374+0000 7f8f59e00640 1 --2- 172.21.15.83:0/2656534029 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f54159810 0x7f8f540cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.374+0000 7f8f59e00640 1 -- 172.21.15.83:0/2656534029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5406e950 msgr2=0x7f8f54166510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.374+0000 7f8f59e00640 1 --2- 172.21.15.83:0/2656534029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5406e950 0x7f8f54166510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.374+0000 7f8f59e00640 1 -- 172.21.15.83:0/2656534029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f540afab0 con 0x7f8f5415fe50 2024-04-25T00:12:30.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.374+0000 7f8f59e00640 1 --2- 172.21.15.83:0/2656534029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f5415fe50 0x7f8f540cfd20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0018a0 tx=0x7f8f4c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:30.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.374+0000 7f8f51600640 1 -- 172.21.15.83:0/2656534029 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c012400 con 0x7f8f5415fe50 2024-04-25T00:12:30.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.374+0000 7f8f51600640 1 -- 172.21.15.83:0/2656534029 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4c0125c0 con 0x7f8f5415fe50 2024-04-25T00:12:30.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.374+0000 7f8f5be00640 1 -- 172.21.15.83:0/2656534029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f540afd90 con 0x7f8f5415fe50 2024-04-25T00:12:30.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.374+0000 7f8f51600640 1 -- 172.21.15.83:0/2656534029 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c0129b0 con 0x7f8f5415fe50 2024-04-25T00:12:30.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.374+0000 7f8f5be00640 1 -- 172.21.15.83:0/2656534029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5414f490 con 0x7f8f5415fe50 2024-04-25T00:12:30.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.374+0000 7f8f5be00640 1 -- 172.21.15.83:0/2656534029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f5404f510 con 0x7f8f5415fe50 2024-04-25T00:12:30.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.374+0000 7f8f51600640 1 -- 172.21.15.83:0/2656534029 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8f4c012b50 con 0x7f8f5415fe50 2024-04-25T00:12:30.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.374+0000 7f8f51600640 1 --2- 172.21.15.83:0/2656534029 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8f340678c0 0x7f8f34069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.378+0000 7f8f51600640 1 -- 172.21.15.83:0/2656534029 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f8f4c0865b0 con 0x7f8f5415fe50 2024-04-25T00:12:30.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.378+0000 7f8f59400640 1 --2- 172.21.15.83:0/2656534029 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8f340678c0 0x7f8f34069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.378+0000 7f8f59400640 1 --2- 172.21.15.83:0/2656534029 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8f340678c0 0x7f8f34069d80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8f540d0480 tx=0x7f8f48009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:30.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.382+0000 7f8f51600640 1 -- 172.21.15.83:0/2656534029 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8f4c056910 con 0x7f8f5415fe50 2024-04-25T00:12:30.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.534+0000 7f8f5be00640 1 -- 172.21.15.83:0/2656534029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8f540aff60 con 0x7f8f5415fe50 2024-04-25T00:12:30.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.534+0000 7f8f51600640 1 -- 172.21.15.83:0/2656534029 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) ==== 104+0+11 (secure 0 0 0) 0x7f8f4c05ac10 con 0x7f8f5415fe50 2024-04-25T00:12:30.537 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:12:30.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.538+0000 7f8f5be00640 1 -- 172.21.15.83:0/2656534029 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8f340678c0 msgr2=0x7f8f34069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.538+0000 7f8f5be00640 1 --2- 172.21.15.83:0/2656534029 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8f340678c0 0x7f8f34069d80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8f540d0480 tx=0x7f8f48009190 comp rx=0 tx=0).stop 2024-04-25T00:12:30.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.538+0000 7f8f5be00640 1 -- 172.21.15.83:0/2656534029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f5415fe50 msgr2=0x7f8f540cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.538+0000 7f8f5be00640 1 --2- 172.21.15.83:0/2656534029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f5415fe50 0x7f8f540cfd20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0018a0 tx=0x7f8f4c00a430 comp rx=0 tx=0).stop 2024-04-25T00:12:30.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.538+0000 7f8f5be00640 1 -- 172.21.15.83:0/2656534029 shutdown_connections 2024-04-25T00:12:30.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.538+0000 7f8f5be00640 1 --2- 172.21.15.83:0/2656534029 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8f340678c0 0x7f8f34069d80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.538+0000 7f8f5be00640 1 --2- 172.21.15.83:0/2656534029 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f54159810 0x7f8f540cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.538+0000 7f8f5be00640 1 --2- 172.21.15.83:0/2656534029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f5415fe50 0x7f8f540cfd20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.538+0000 7f8f5be00640 1 --2- 172.21.15.83:0/2656534029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f5406e950 0x7f8f54166510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.538+0000 7f8f5be00640 1 -- 172.21.15.83:0/2656534029 >> 172.21.15.83:0/2656534029 conn(0x7f8f540b1830 msgr2=0x7f8f5415e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:30.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.538+0000 7f8f5be00640 1 -- 172.21.15.83:0/2656534029 shutdown_connections 2024-04-25T00:12:30.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.538+0000 7f8f5be00640 1 -- 172.21.15.83:0/2656534029 wait complete. 2024-04-25T00:12:30.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:12:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.694+0000 7f0f3d800640 1 Processor -- start 2024-04-25T00:12:30.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.694+0000 7f0f3d800640 1 -- start start 2024-04-25T00:12:30.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.694+0000 7f0f3d800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f381551e0 0x7f0f3815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.694+0000 7f0f3d800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f3815c560 con 0x7f0f381555c0 2024-04-25T00:12:30.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.694+0000 7f0f3d800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f38072460 con 0x7f0f3806e4c0 2024-04-25T00:12:30.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.694+0000 7f0f3d800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0f3815c7d0 con 0x7f0f381551e0 2024-04-25T00:12:30.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.694+0000 7f0f36a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f381551e0 0x7f0f3815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.694+0000 7f0f2fe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0f381555c0 0x7f0f38071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42630/0 (socket says 172.21.15.83:42630) 2024-04-25T00:12:30.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.694+0000 7f0f2fe00640 1 -- 172.21.15.83:0/2414297971 learned_addr learned my addr 172.21.15.83:0/2414297971 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:30.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.694+0000 7f0f36a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f381551e0 0x7f0f3815bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52876/0 (socket says 172.21.15.83:52876) 2024-04-25T00:12:30.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.698+0000 7f0f36a00640 1 -- 172.21.15.83:0/2414297971 >> v1:172.21.15.86:6789/0 conn(0x7f0f3806e4c0 legacy=0x7f0f3815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:30.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.698+0000 7f0f36a00640 1 -- 172.21.15.83:0/2414297971 >> v1:172.21.15.83:6789/0 conn(0x7f0f381555c0 legacy=0x7f0f38071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.698+0000 7f0f36a00640 1 -- 172.21.15.83:0/2414297971 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3815cab0 con 0x7f0f381551e0 2024-04-25T00:12:30.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.698+0000 7f0f36000640 1 -- 172.21.15.83:0/2414297971 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1457909610 0 0) 0x7f0f3815c560 con 0x7f0f381555c0 2024-04-25T00:12:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.698+0000 7f0f36a00640 1 --2- 172.21.15.83:0/2414297971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f381551e0 0x7f0f3815bea0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0f280097f0 tx=0x7f0f2801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18114786cc991c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:30.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.698+0000 7f0f36000640 1 -- 172.21.15.83:0/2414297971 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f280b0070 con 0x7f0f381551e0 2024-04-25T00:12:30.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.698+0000 7f0f36000640 1 -- 172.21.15.83:0/2414297971 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f280ab600 con 0x7f0f381551e0 2024-04-25T00:12:30.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.698+0000 7f0f36000640 1 -- 172.21.15.83:0/2414297971 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f280ab9d0 con 0x7f0f381551e0 2024-04-25T00:12:30.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.698+0000 7f0f3d800640 1 -- 172.21.15.83:0/2414297971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f381551e0 msgr2=0x7f0f3815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.698+0000 7f0f3d800640 1 --2- 172.21.15.83:0/2414297971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f381551e0 0x7f0f3815bea0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0f280097f0 tx=0x7f0f2801c460 comp rx=0 tx=0).stop 2024-04-25T00:12:30.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f3d800640 1 -- 172.21.15.83:0/2414297971 shutdown_connections 2024-04-25T00:12:30.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f3d800640 1 --2- 172.21.15.83:0/2414297971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f381551e0 0x7f0f3815bea0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f3d800640 1 -- 172.21.15.83:0/2414297971 >> 172.21.15.83:0/2414297971 conn(0x7f0f380b1830 msgr2=0x7f0f380b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:30.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f3d800640 1 -- 172.21.15.83:0/2414297971 shutdown_connections 2024-04-25T00:12:30.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f3d800640 1 -- 172.21.15.83:0/2414297971 wait complete. 2024-04-25T00:12:30.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f3d800640 1 Processor -- start 2024-04-25T00:12:30.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f3d800640 1 -- start start 2024-04-25T00:12:30.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f3d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3806e4c0 0x7f0f380b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f3d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f381551e0 0x7f0f38154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f3d800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f381555c0 0x7f0f3814ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f3d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3815e2c0 con 0x7f0f381551e0 2024-04-25T00:12:30.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f3d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3815e140 con 0x7f0f3806e4c0 2024-04-25T00:12:30.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f3d800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0f38168ca0 con 0x7f0f381555c0 2024-04-25T00:12:30.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f2fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f381551e0 0x7f0f38154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f2fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f381551e0 0x7f0f38154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52880/0 (socket says 172.21.15.83:52880) 2024-04-25T00:12:30.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f2fe00640 1 -- 172.21.15.83:0/3594448424 learned_addr learned my addr 172.21.15.83:0/3594448424 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:30.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f36a00640 1 --2- 172.21.15.83:0/3594448424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3806e4c0 0x7f0f380b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f37400640 1 --2- 172.21.15.83:0/3594448424 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f381555c0 0x7f0f3814ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f2fe00640 1 -- 172.21.15.83:0/3594448424 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f381555c0 msgr2=0x7f0f3814ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f2fe00640 1 --2- 172.21.15.83:0/3594448424 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f381555c0 0x7f0f3814ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f2fe00640 1 -- 172.21.15.83:0/3594448424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3806e4c0 msgr2=0x7f0f380b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f2fe00640 1 --2- 172.21.15.83:0/3594448424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3806e4c0 0x7f0f380b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f2fe00640 1 -- 172.21.15.83:0/3594448424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f380b2d00 con 0x7f0f381551e0 2024-04-25T00:12:30.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.702+0000 7f0f2fe00640 1 --2- 172.21.15.83:0/3594448424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f381551e0 0x7f0f38154440 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000c450 tx=0x7f0f2000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:30.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.706+0000 7f0f2ea00640 1 -- 172.21.15.83:0/3594448424 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20018070 con 0x7f0f381551e0 2024-04-25T00:12:30.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.706+0000 7f0f2ea00640 1 -- 172.21.15.83:0/3594448424 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f200074e0 con 0x7f0f381551e0 2024-04-25T00:12:30.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.706+0000 7f0f2ea00640 1 -- 172.21.15.83:0/3594448424 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f200049f0 con 0x7f0f381551e0 2024-04-25T00:12:30.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.706+0000 7f0f3d800640 1 -- 172.21.15.83:0/3594448424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f380b2fe0 con 0x7f0f381551e0 2024-04-25T00:12:30.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.706+0000 7f0f3d800640 1 -- 172.21.15.83:0/3594448424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f380b35a0 con 0x7f0f381551e0 2024-04-25T00:12:30.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.706+0000 7f0f3d800640 1 -- 172.21.15.83:0/3594448424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f3804f510 con 0x7f0f381551e0 2024-04-25T00:12:30.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.706+0000 7f0f2ea00640 1 -- 172.21.15.83:0/3594448424 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0f20004b90 con 0x7f0f381551e0 2024-04-25T00:12:30.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.706+0000 7f0f2ea00640 1 --2- 172.21.15.83:0/3594448424 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0f140678c0 0x7f0f14069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.706+0000 7f0f2ea00640 1 -- 172.21.15.83:0/3594448424 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f0f200851a0 con 0x7f0f381551e0 2024-04-25T00:12:30.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.706+0000 7f0f36a00640 1 --2- 172.21.15.83:0/3594448424 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0f140678c0 0x7f0f14069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.706+0000 7f0f36a00640 1 --2- 172.21.15.83:0/3594448424 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0f140678c0 0x7f0f14069d80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f280ad000 tx=0x7f0f280a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:30.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.710+0000 7f0f2ea00640 1 -- 172.21.15.83:0/3594448424 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0f200553d0 con 0x7f0f381551e0 2024-04-25T00:12:30.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.866+0000 7f0f3d800640 1 -- 172.21.15.83:0/3594448424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0f3806ce30 con 0x7f0f381551e0 2024-04-25T00:12:30.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.870+0000 7f0f2ea00640 1 -- 172.21.15.83:0/3594448424 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) ==== 100+0+11 (secure 0 0 0) 0x7f0f200596d0 con 0x7f0f381551e0 2024-04-25T00:12:30.870 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:12:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.870+0000 7f0f3d800640 1 -- 172.21.15.83:0/3594448424 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0f140678c0 msgr2=0x7f0f14069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.870+0000 7f0f3d800640 1 --2- 172.21.15.83:0/3594448424 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0f140678c0 0x7f0f14069d80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f280ad000 tx=0x7f0f280a6000 comp rx=0 tx=0).stop 2024-04-25T00:12:30.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.870+0000 7f0f3d800640 1 -- 172.21.15.83:0/3594448424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f381551e0 msgr2=0x7f0f38154440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.870+0000 7f0f3d800640 1 --2- 172.21.15.83:0/3594448424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f381551e0 0x7f0f38154440 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000c450 tx=0x7f0f2000c910 comp rx=0 tx=0).stop 2024-04-25T00:12:30.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.870+0000 7f0f3d800640 1 -- 172.21.15.83:0/3594448424 shutdown_connections 2024-04-25T00:12:30.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.870+0000 7f0f3d800640 1 --2- 172.21.15.83:0/3594448424 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0f140678c0 0x7f0f14069d80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.870+0000 7f0f3d800640 1 --2- 172.21.15.83:0/3594448424 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f381555c0 0x7f0f3814ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.870+0000 7f0f3d800640 1 --2- 172.21.15.83:0/3594448424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f381551e0 0x7f0f38154440 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.870+0000 7f0f3d800640 1 --2- 172.21.15.83:0/3594448424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f3806e4c0 0x7f0f380b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.870+0000 7f0f3d800640 1 -- 172.21.15.83:0/3594448424 >> 172.21.15.83:0/3594448424 conn(0x7f0f380b1830 msgr2=0x7f0f3815ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:30.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.870+0000 7f0f3d800640 1 -- 172.21.15.83:0/3594448424 shutdown_connections 2024-04-25T00:12:30.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:30.870+0000 7f0f3d800640 1 -- 172.21.15.83:0/3594448424 wait complete. 2024-04-25T00:12:30.890 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:12:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.026+0000 7f7ae3200640 1 Processor -- start 2024-04-25T00:12:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.026+0000 7f7ae3200640 1 -- start start 2024-04-25T00:12:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.026+0000 7f7ae3200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae406b1e0 0x7f7ae40b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.026+0000 7f7ae3200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7ae415f6e0 0x7f7ae4071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.026+0000 7f7ae3200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ae40b0520 con 0x7f7ae40ce090 2024-04-25T00:12:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.026+0000 7f7ae3200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae415e6f0 con 0x7f7ae415f6e0 2024-04-25T00:12:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.026+0000 7f7ae3200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae415e9d0 con 0x7f7ae406b1e0 2024-04-25T00:12:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.030+0000 7f7ae1e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae406b1e0 0x7f7ae40b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.030+0000 7f7ae2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7ae415f6e0 0x7f7ae4071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.030+0000 7f7adbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7ae40ce090 0x7f7ae406b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:41738/0 (socket says 172.21.15.83:41738) 2024-04-25T00:12:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.030+0000 7f7adbe00640 1 -- 172.21.15.83:0/724844174 learned_addr learned my addr 172.21.15.83:0/724844174 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.030+0000 7f7ae2800640 1 -- 172.21.15.83:0/724844174 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae406b1e0 msgr2=0x7f7ae40b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.030+0000 7f7ae2800640 1 --2- 172.21.15.83:0/724844174 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae406b1e0 0x7f7ae40b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.030+0000 7f7ae2800640 1 -- 172.21.15.83:0/724844174 >> v1:172.21.15.86:6789/0 conn(0x7f7ae40ce090 legacy=0x7f7ae406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.030+0000 7f7ae2800640 1 -- 172.21.15.83:0/724844174 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae415ecb0 con 0x7f7ae415f6e0 2024-04-25T00:12:31.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.030+0000 7f7ae1e00640 1 --2- 172.21.15.83:0/724844174 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae406b1e0 0x7f7ae40b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:31.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.030+0000 7f7ae2800640 1 --2- 172.21.15.83:0/724844174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae415f6e0 0x7f7ae4071fe0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009d90 tx=0x7f7adc01c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8b908eef5119c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.030+0000 7f7ae1400640 1 -- 172.21.15.83:0/724844174 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0b2070 con 0x7f7ae415f6e0 2024-04-25T00:12:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.030+0000 7f7ae1400640 1 -- 172.21.15.83:0/724844174 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7adc0a5890 con 0x7f7ae415f6e0 2024-04-25T00:12:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.030+0000 7f7ae1400640 1 -- 172.21.15.83:0/724844174 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0a5c60 con 0x7f7ae415f6e0 2024-04-25T00:12:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 -- 172.21.15.83:0/724844174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae415f6e0 msgr2=0x7f7ae4071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 --2- 172.21.15.83:0/724844174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae415f6e0 0x7f7ae4071fe0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009d90 tx=0x7f7adc01c8c0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 -- 172.21.15.83:0/724844174 shutdown_connections 2024-04-25T00:12:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 --2- 172.21.15.83:0/724844174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae415f6e0 0x7f7ae4071fe0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 --2- 172.21.15.83:0/724844174 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae406b1e0 0x7f7ae40b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 -- 172.21.15.83:0/724844174 >> 172.21.15.83:0/724844174 conn(0x7f7ae40cd500 msgr2=0x7f7ae40cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 -- 172.21.15.83:0/724844174 shutdown_connections 2024-04-25T00:12:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 -- 172.21.15.83:0/724844174 wait complete. 2024-04-25T00:12:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 Processor -- start 2024-04-25T00:12:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 -- start start 2024-04-25T00:12:31.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae406b1e0 0x7f7ae414c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ae40ce090 0x7f7ae414dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae415f6e0 0x7f7ae406e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae416ae20 con 0x7f7ae40ce090 2024-04-25T00:12:31.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae416aca0 con 0x7f7ae415f6e0 2024-04-25T00:12:31.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae3200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae416afa0 con 0x7f7ae406b1e0 2024-04-25T00:12:31.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7adbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ae40ce090 0x7f7ae414dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae1e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae406b1e0 0x7f7ae414c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae1e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae406b1e0 0x7f7ae414c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:37776/0 (socket says 172.21.15.83:37776) 2024-04-25T00:12:31.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7adbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ae40ce090 0x7f7ae414dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52896/0 (socket says 172.21.15.83:52896) 2024-04-25T00:12:31.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae1e00640 1 -- 172.21.15.83:0/733109178 learned_addr learned my addr 172.21.15.83:0/733109178 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:31.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae2800640 1 --2- 172.21.15.83:0/733109178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae415f6e0 0x7f7ae406e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae1e00640 1 -- 172.21.15.83:0/733109178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae415f6e0 msgr2=0x7f7ae406e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae1e00640 1 --2- 172.21.15.83:0/733109178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae415f6e0 0x7f7ae406e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae1e00640 1 -- 172.21.15.83:0/733109178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ae40ce090 msgr2=0x7f7ae414dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae1e00640 1 --2- 172.21.15.83:0/733109178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ae40ce090 0x7f7ae414dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7ae1e00640 1 -- 172.21.15.83:0/733109178 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae406f360 con 0x7f7ae406b1e0 2024-04-25T00:12:31.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.034+0000 7f7adbe00640 1 --2- 172.21.15.83:0/733109178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ae40ce090 0x7f7ae414dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:31.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.038+0000 7f7ae1e00640 1 --2- 172.21.15.83:0/733109178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae406b1e0 0x7f7ae414c8e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400e980 tx=0x7f7ad400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:31.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.038+0000 7f7ada000640 1 -- 172.21.15.83:0/733109178 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4017070 con 0x7f7ae406b1e0 2024-04-25T00:12:31.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.038+0000 7f7ada000640 1 -- 172.21.15.83:0/733109178 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad400ca00 con 0x7f7ae406b1e0 2024-04-25T00:12:31.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.038+0000 7f7ada000640 1 -- 172.21.15.83:0/733109178 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad400cdd0 con 0x7f7ae406b1e0 2024-04-25T00:12:31.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.038+0000 7f7ae3200640 1 -- 172.21.15.83:0/733109178 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae414d2a0 con 0x7f7ae406b1e0 2024-04-25T00:12:31.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.038+0000 7f7ae3200640 1 -- 172.21.15.83:0/733109178 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae414d860 con 0x7f7ae406b1e0 2024-04-25T00:12:31.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.038+0000 7f7ae3200640 1 -- 172.21.15.83:0/733109178 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae404f510 con 0x7f7ae406b1e0 2024-04-25T00:12:31.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.038+0000 7f7ada000640 1 -- 172.21.15.83:0/733109178 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7ad401a070 con 0x7f7ae406b1e0 2024-04-25T00:12:31.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.038+0000 7f7ada000640 1 --2- 172.21.15.83:0/733109178 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ac00678c0 0x7f7ac0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.042+0000 7f7ada000640 1 -- 172.21.15.83:0/733109178 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f7ad4086340 con 0x7f7ae406b1e0 2024-04-25T00:12:31.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.042+0000 7f7adbe00640 1 --2- 172.21.15.83:0/733109178 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ac00678c0 0x7f7ac0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.042+0000 7f7adbe00640 1 --2- 172.21.15.83:0/733109178 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ac00678c0 0x7f7ac0069d80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7ae415cfa0 tx=0x7f7acc007e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:31.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.042+0000 7f7ada000640 1 -- 172.21.15.83:0/733109178 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7ad4056810 con 0x7f7ae406b1e0 2024-04-25T00:12:31.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.210+0000 7f7ae3200640 1 -- 172.21.15.83:0/733109178 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7ae40d07f0 con 0x7f7ae406b1e0 2024-04-25T00:12:31.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.214+0000 7f7ada000640 1 -- 172.21.15.83:0/733109178 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) ==== 80+0+0 (secure 0 0 0) 0x7f7ad405ab10 con 0x7f7ae406b1e0 2024-04-25T00:12:31.214 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:12:31.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.214+0000 7f7ae3200640 1 -- 172.21.15.83:0/733109178 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ac00678c0 msgr2=0x7f7ac0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.214+0000 7f7ae3200640 1 --2- 172.21.15.83:0/733109178 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ac00678c0 0x7f7ac0069d80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7ae415cfa0 tx=0x7f7acc007e90 comp rx=0 tx=0).stop 2024-04-25T00:12:31.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.214+0000 7f7ae3200640 1 -- 172.21.15.83:0/733109178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae406b1e0 msgr2=0x7f7ae414c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.214+0000 7f7ae3200640 1 --2- 172.21.15.83:0/733109178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae406b1e0 0x7f7ae414c8e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400e980 tx=0x7f7ad400ee40 comp rx=0 tx=0).stop 2024-04-25T00:12:31.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.214+0000 7f7ae3200640 1 -- 172.21.15.83:0/733109178 shutdown_connections 2024-04-25T00:12:31.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.214+0000 7f7ae3200640 1 --2- 172.21.15.83:0/733109178 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ac00678c0 0x7f7ac0069d80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.214+0000 7f7ae3200640 1 --2- 172.21.15.83:0/733109178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae415f6e0 0x7f7ae406e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.214+0000 7f7ae3200640 1 --2- 172.21.15.83:0/733109178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ae40ce090 0x7f7ae414dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.214+0000 7f7ae3200640 1 --2- 172.21.15.83:0/733109178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae406b1e0 0x7f7ae414c8e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.214+0000 7f7ae3200640 1 -- 172.21.15.83:0/733109178 >> 172.21.15.83:0/733109178 conn(0x7f7ae40cd500 msgr2=0x7f7ae4151f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:31.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.214+0000 7f7ae3200640 1 -- 172.21.15.83:0/733109178 shutdown_connections 2024-04-25T00:12:31.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.214+0000 7f7ae3200640 1 -- 172.21.15.83:0/733109178 wait complete. 2024-04-25T00:12:31.234 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T00:12:31.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:31.235 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T00:12:31.265 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:12:31.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:31.265 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:12:31.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:12:31.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc534200640 1 Processor -- start 2024-04-25T00:12:31.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc534200640 1 -- start start 2024-04-25T00:12:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc534200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc52c0b4360 0x7fc52c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc534200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc52c06ef80 con 0x7fc52c0b2b30 2024-04-25T00:12:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc534200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc52c15b510 con 0x7fc52c1553e0 2024-04-25T00:12:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc534200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc52c15b7f0 con 0x7fc52c0b4360 2024-04-25T00:12:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc532200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc52c0b2b30 0x7fc52c0b2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42638/0 (socket says 172.21.15.83:42638) 2024-04-25T00:12:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc532200640 1 -- 172.21.15.83:0/727058091 learned_addr learned my addr 172.21.15.83:0/727058091 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:31.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc530e00640 1 --2- 172.21.15.83:0/727058091 >> v2:172.21.15.86:3300/0 conn(0x7fc52c0b4360 0x7fc52c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 -- 172.21.15.83:0/727058091 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1282654553 0 0) 0x7fc52c06ef80 con 0x7fc52c0b2b30 2024-04-25T00:12:31.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 -- 172.21.15.83:0/727058091 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc514005740 con 0x7fc52c0b2b30 2024-04-25T00:12:31.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 -- 172.21.15.83:0/727058091 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1435410014 0 0) 0x7fc514005740 con 0x7fc52c0b2b30 2024-04-25T00:12:31.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 -- 172.21.15.83:0/727058091 >> v2:172.21.15.86:3300/0 conn(0x7fc52c0b4360 msgr2=0x7fc52c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:31.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 --2- 172.21.15.83:0/727058091 >> v2:172.21.15.86:3300/0 conn(0x7fc52c0b4360 0x7fc52c0b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 -- 172.21.15.83:0/727058091 >> v1:172.21.15.86:6789/0 conn(0x7fc52c1553e0 legacy=0x7fc52c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 -- 172.21.15.83:0/727058091 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c15bad0 con 0x7fc52c0b2b30 2024-04-25T00:12:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 -- 172.21.15.83:0/727058091 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fc524001a00 con 0x7fc52c0b2b30 2024-04-25T00:12:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 -- 172.21.15.83:0/727058091 >> v1:172.21.15.83:6789/0 conn(0x7fc52c0b2b30 legacy=0x7fc52c0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 --2- 172.21.15.83:0/727058091 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc514005bd0 0x7fc514005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 --2- 172.21.15.83:0/727058091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514006a20 0x7fc514006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 --2- 172.21.15.83:0/727058091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5140073e0 0x7fc51400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 -- 172.21.15.83:0/727058091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c15b7f0 con 0x7fc514006a20 2024-04-25T00:12:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 -- 172.21.15.83:0/727058091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc51400e380 con 0x7fc5140073e0 2024-04-25T00:12:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc52be00640 1 -- 172.21.15.83:0/727058091 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc51400e580 con 0x7fc514005bd0 2024-04-25T00:12:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc530e00640 1 --2- 172.21.15.83:0/727058091 >> v2:172.21.15.86:3300/0 conn(0x7fc52c0b4360 0x7fc52c0b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc530e00640 1 --2- 172.21.15.83:0/727058091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514006a20 0x7fc514006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc531800640 1 --2- 172.21.15.83:0/727058091 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc514005bd0 0x7fc514005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc532200640 1 --2- 172.21.15.83:0/727058091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5140073e0 0x7fc51400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc530e00640 1 -- 172.21.15.83:0/727058091 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc514005bd0 msgr2=0x7fc514005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:31.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc530e00640 1 --2- 172.21.15.83:0/727058091 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc514005bd0 0x7fc514005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc530e00640 1 -- 172.21.15.83:0/727058091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5140073e0 msgr2=0x7fc51400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:31.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc530e00640 1 --2- 172.21.15.83:0/727058091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5140073e0 0x7fc51400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc530e00640 1 -- 172.21.15.83:0/727058091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc514005740 con 0x7fc514006a20 2024-04-25T00:12:31.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.450+0000 7fc532200640 1 -- 172.21.15.83:0/727058091 reap_dead start 2024-04-25T00:12:31.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.454+0000 7fc530e00640 1 --2- 172.21.15.83:0/727058091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514006a20 0x7fc514006ea0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc518009c90 tx=0x7fc51801cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfd44ccf16e9edf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:31.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.454+0000 7fc52be00640 1 -- 172.21.15.83:0/727058091 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518009860 con 0x7fc514006a20 2024-04-25T00:12:31.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.454+0000 7fc52be00640 1 -- 172.21.15.83:0/727058091 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc518004020 con 0x7fc514006a20 2024-04-25T00:12:31.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.454+0000 7fc52be00640 1 -- 172.21.15.83:0/727058091 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518004450 con 0x7fc514006a20 2024-04-25T00:12:31.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.454+0000 7fc534200640 1 -- 172.21.15.83:0/727058091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514006a20 msgr2=0x7fc514006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.454+0000 7fc534200640 1 --2- 172.21.15.83:0/727058091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514006a20 0x7fc514006ea0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc518009c90 tx=0x7fc51801cbf0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.454+0000 7fc534200640 1 -- 172.21.15.83:0/727058091 shutdown_connections 2024-04-25T00:12:31.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.454+0000 7fc534200640 1 --2- 172.21.15.83:0/727058091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514006a20 0x7fc514006ea0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.454+0000 7fc534200640 1 -- 172.21.15.83:0/727058091 >> 172.21.15.83:0/727058091 conn(0x7fc52c0b1830 msgr2=0x7fc52c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:31.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.454+0000 7fc534200640 1 -- 172.21.15.83:0/727058091 shutdown_connections 2024-04-25T00:12:31.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.454+0000 7fc534200640 1 -- 172.21.15.83:0/727058091 wait complete. 2024-04-25T00:12:31.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc534200640 1 Processor -- start 2024-04-25T00:12:31.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc534200640 1 -- start start 2024-04-25T00:12:31.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc534200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc514006a20 0x7fc52c145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc534200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc52c1f2a00 0x7fc52c150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc534200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc52c071be0 0x7fc52c072070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc534200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c0b4cd0 con 0x7fc52c071be0 2024-04-25T00:12:31.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc534200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c0b4b50 con 0x7fc514006a20 2024-04-25T00:12:31.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc534200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc52c0b3720 con 0x7fc52c1f2a00 2024-04-25T00:12:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc532200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc52c071be0 0x7fc52c072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc532200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc52c071be0 0x7fc52c072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52922/0 (socket says 172.21.15.83:52922) 2024-04-25T00:12:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc532200640 1 -- 172.21.15.83:0/3850228261 learned_addr learned my addr 172.21.15.83:0/3850228261 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc532200640 1 -- 172.21.15.83:0/3850228261 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc52c1f2a00 msgr2=0x7fc52c150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc532200640 1 --2- 172.21.15.83:0/3850228261 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc52c1f2a00 0x7fc52c150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc532200640 1 -- 172.21.15.83:0/3850228261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc514006a20 msgr2=0x7fc52c145580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc532200640 1 --2- 172.21.15.83:0/3850228261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc514006a20 0x7fc52c145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc532200640 1 -- 172.21.15.83:0/3850228261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c158800 con 0x7fc52c071be0 2024-04-25T00:12:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc532200640 1 --2- 172.21.15.83:0/3850228261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc52c071be0 0x7fc52c072070 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc524000f90 tx=0x7fc524007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc529600640 1 -- 172.21.15.83:0/3850228261 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52400ac20 con 0x7fc52c071be0 2024-04-25T00:12:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc529600640 1 -- 172.21.15.83:0/3850228261 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc524019070 con 0x7fc52c071be0 2024-04-25T00:12:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc529600640 1 -- 172.21.15.83:0/3850228261 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5240133f0 con 0x7fc52c071be0 2024-04-25T00:12:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc534200640 1 -- 172.21.15.83:0/3850228261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52c1f2f00 con 0x7fc52c071be0 2024-04-25T00:12:31.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.458+0000 7fc534200640 1 -- 172.21.15.83:0/3850228261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52c1f34c0 con 0x7fc52c071be0 2024-04-25T00:12:31.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.462+0000 7fc534200640 1 -- 172.21.15.83:0/3850228261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52c0737f0 con 0x7fc52c071be0 2024-04-25T00:12:31.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.462+0000 7fc529600640 1 -- 172.21.15.83:0/3850228261 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc524013590 con 0x7fc52c071be0 2024-04-25T00:12:31.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.462+0000 7fc529600640 1 --2- 172.21.15.83:0/3850228261 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc50c0678f0 0x7fc50c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.462+0000 7fc529600640 1 -- 172.21.15.83:0/3850228261 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fc524086430 con 0x7fc52c071be0 2024-04-25T00:12:31.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.462+0000 7fc531800640 1 --2- 172.21.15.83:0/3850228261 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc50c0678f0 0x7fc50c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.462+0000 7fc531800640 1 --2- 172.21.15.83:0/3850228261 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc50c0678f0 0x7fc50c069db0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc52c069f90 tx=0x7fc520009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.466+0000 7fc529600640 1 -- 172.21.15.83:0/3850228261 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc52c158800 con 0x7fc52c071be0 2024-04-25T00:12:31.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.654+0000 7fc534200640 1 -- 172.21.15.83:0/3850228261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc52c04f5f0 con 0x7fc52c071be0 2024-04-25T00:12:31.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.654+0000 7fc529600640 1 -- 172.21.15.83:0/3850228261 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7fc524056d40 con 0x7fc52c071be0 2024-04-25T00:12:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:31.655 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4277921554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":4277921554},{"type":"v1","addr":"172.21.15.86:6835","nonce":4277921554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/4145334103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":4145334103},{"type":"v1","addr":"172.21.15.86:6834","nonce":4145334103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3441801758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3441801758},{"type":"v1","addr":"172.21.15.83:6836","nonce":3441801758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:12:18.566138+0000","modified":"2024-04-25T00:12:21.572278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5328},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5328":{"gid":5328,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2704243724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2704243724},{"type":"v1","addr":"172.21.15.83:6837","nonce":2704243724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5328,"qdb_cluster":[5328]},"id":2}]} 2024-04-25T00:12:31.656 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 18 2024-04-25T00:12:31.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.654+0000 7fc534200640 1 -- 172.21.15.83:0/3850228261 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc50c0678f0 msgr2=0x7fc50c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.654+0000 7fc534200640 1 --2- 172.21.15.83:0/3850228261 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc50c0678f0 0x7fc50c069db0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc52c069f90 tx=0x7fc520009040 comp rx=0 tx=0).stop 2024-04-25T00:12:31.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.654+0000 7fc534200640 1 -- 172.21.15.83:0/3850228261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc52c071be0 msgr2=0x7fc52c072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.654+0000 7fc534200640 1 --2- 172.21.15.83:0/3850228261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc52c071be0 0x7fc52c072070 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc524000f90 tx=0x7fc524007bd0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.654+0000 7fc534200640 1 -- 172.21.15.83:0/3850228261 shutdown_connections 2024-04-25T00:12:31.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.654+0000 7fc534200640 1 --2- 172.21.15.83:0/3850228261 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc50c0678f0 0x7fc50c069db0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.654+0000 7fc534200640 1 --2- 172.21.15.83:0/3850228261 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc52c1f2a00 0x7fc52c150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.654+0000 7fc534200640 1 --2- 172.21.15.83:0/3850228261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc52c071be0 0x7fc52c072070 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.654+0000 7fc534200640 1 --2- 172.21.15.83:0/3850228261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc514006a20 0x7fc52c145580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.654+0000 7fc534200640 1 -- 172.21.15.83:0/3850228261 >> 172.21.15.83:0/3850228261 conn(0x7fc52c0b1830 msgr2=0x7fc52c15d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:31.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.658+0000 7fc534200640 1 -- 172.21.15.83:0/3850228261 shutdown_connections 2024-04-25T00:12:31.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.658+0000 7fc534200640 1 -- 172.21.15.83:0/3850228261 wait complete. 2024-04-25T00:12:31.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-04-25T00:12:31.833 INFO:teuthology.orchestra.run.smithi083.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":40,"msgr_send_messages":35,"msgr_recv_bytes":11582,"msgr_send_bytes":22139,"msgr_created_connections":8,"msgr_active_connections":2,"msgr_running_total_time":0.018626877,"msgr_running_send_time":0.001174094,"msgr_running_recv_time":0.013911809,"msgr_running_fast_dispatch_time":0.000770293,"msgr_send_messages_queue_lat":{"avgcount":34,"sum":0.305885810,"avgtime":0.008996641},"msgr_handle_ack_lat":{"avgcount":15,"sum":0.000002352,"avgtime":0.000000156},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":11,"msgr_send_messages":17,"msgr_recv_bytes":2872,"msgr_send_bytes":97726,"msgr_created_connections":6,"msgr_active_connections":4,"msgr_running_total_time":0.013824467,"msgr_running_send_time":0.000848100,"msgr_running_recv_time":0.012516215,"msgr_running_fast_dispatch_time":0.000242373,"msgr_send_messages_queue_lat":{"avgcount":15,"sum":0.141006166,"avgtime":0.009400411},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":93072},"AsyncMessenger::Worker-2":{"msgr_recv_messages":97,"msgr_send_messages":88,"msgr_recv_bytes":122214,"msgr_send_bytes":52668,"msgr_created_connections":7,"msgr_active_connections":3,"msgr_running_total_time":0.013975821,"msgr_running_send_time":0.004048058,"msgr_running_recv_time":0.015280988,"msgr_running_fast_dispatch_time":0.000490142,"msgr_send_messages_queue_lat":{"avgcount":88,"sum":0.060158405,"avgtime":0.000683618},"msgr_handle_ack_lat":{"avgcount":59,"sum":0.000009912,"avgtime":0.000000168},"msgr_recv_encrypted_bytes":110432,"msgr_send_encrypted_bytes":11408},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":52,"sum":0.003638717,"avgtime":0.000069975}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000008570,"avgtime":0.000004285}},"mds":{"request":29,"reply":29,"reply_latency":{"avgcount":29,"sum":0.004220923,"avgtime":0.000145549},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":2,"subtrees":2,"traverse":55,"traverse_hit":53,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":8,"evex":0,"evtrm":0,"ev":8,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197029,"rdpos":0,"jlat":{"avgcount":5,"sum":0.011027354,"avgtime":0.002205470},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":1,"cap+":2,"cap-":1,"rss":46520,"heap":231732},"mds_server":{"dispatch_client_request":29,"dispatch_server_request":0,"handle_client_request":29,"handle_client_session":6,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":26,"sum":0.003675048,"avgtime":0.000141348},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":2,"sum":0.000270239,"avgtime":0.000135119},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000275636,"avgtime":0.000275636},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":27,"average_load":13,"avg_session_uptime":0,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":233843,"buffer_anon_items":93,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":53576,"mds_co_items":218,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":43,"op_send_bytes":13845,"op_resend":35,"op_reply":43,"op_latency":{"avgcount":43,"sum":1.051979981,"avgtime":0.024464650},"op_inflight":0,"oplen_avg":{"avgcount":43,"sum":67},"op":43,"op_r":0,"op_w":43,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":38,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":143,"get_sum":119514,"get_or_fail_fail":0,"get_or_fail_success":143,"take":0,"take_sum":0,"put":143,"put_sum":119514,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":13845,"put":43,"put_sum":13845,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":43,"put":43,"put_sum":43,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55d89c54c780":{"val":0,"max":3758096384,"get_started":0,"get":8,"get_sum":2725,"get_or_fail_fail":0,"get_or_fail_success":8,"take":0,"take_sum":0,"put":5,"put_sum":2725,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-04-25T00:12:31.835 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 15, "avgtime": 1.56e-07, "sum": 2.352e-06 }, "msgr_recv_bytes": 11582, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 40, "msgr_running_fast_dispatch_time": 0.000770293, "msgr_running_recv_time": 0.013911809, "msgr_running_send_time": 0.001174094, "msgr_running_total_time": 0.018626877, "msgr_send_bytes": 22139, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 35, "msgr_send_messages_queue_lat": { "avgcount": 34, "avgtime": 0.008996641, "sum": 0.30588581 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 2872, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 11, "msgr_running_fast_dispatch_time": 0.000242373, "msgr_running_recv_time": 0.012516215, "msgr_running_send_time": 0.0008481, "msgr_running_total_time": 0.013824467, "msgr_send_bytes": 97726, "msgr_send_encrypted_bytes": 93072, "msgr_send_messages": 17, "msgr_send_messages_queue_lat": { "avgcount": 15, "avgtime": 0.009400411, "sum": 0.141006166 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 59, "avgtime": 1.68e-07, "sum": 9.912e-06 }, "msgr_recv_bytes": 122214, "msgr_recv_encrypted_bytes": 110432, "msgr_recv_messages": 97, "msgr_running_fast_dispatch_time": 0.000490142, "msgr_running_recv_time": 0.015280988, "msgr_running_send_time": 0.004048058, "msgr_running_total_time": 0.013975821, "msgr_send_bytes": 52668, "msgr_send_encrypted_bytes": 11408, "msgr_send_messages": 88, "msgr_send_messages_queue_lat": { "avgcount": 88, "avgtime": 0.000683618, "sum": 0.060158405 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 52, "avgtime": 6.9975e-05, "sum": 0.003638717 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 4.285e-06, "sum": 8.57e-06 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 29, "reply_latency": { "avgcount": 29, "avgtime": 0.000145549, "sum": 0.004220923 }, "request": 29, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 55, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 53, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 8, "evadd": 8, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.00220547, "sum": 0.011027354 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197029 }, "mds_mem": { "cap": 1, "cap+": 2, "cap-": 1, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 231732, "ino": 13, "ino+": 13, "ino-": 0, "rss": 46520 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 29, "dispatch_server_request": 0, "handle_client_request": 29, "handle_client_session": 6, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 26, "avgtime": 0.000141348, "sum": 0.003675048 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 2, "avgtime": 0.000135119, "sum": 0.000270239 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000275636, "sum": 0.000275636 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 13, "avg_session_uptime": 0, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 27 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 233843, "buffer_anon_items": 93, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 53576, "mds_co_items": 218, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 38, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 43, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 43, "avgtime": 0.02446465, "sum": 1.051979981 }, "op_pg": 0, "op_r": 0, "op_reply": 43, "op_resend": 35, "op_rmw": 0, "op_send": 43, "op_send_bytes": 13845, "op_w": 43, "oplen_avg": { "avgcount": 43, "sum": 67 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 143, "get_or_fail_fail": 0, "get_or_fail_success": 143, "get_started": 0, "get_sum": 119514, "max": 104857600, "put": 143, "put_sum": 119514, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 43, "put_sum": 13845, "take": 43, "take_sum": 13845, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 43, "put_sum": 43, "take": 43, "take_sum": 43, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55d89c54c780": { "get": 8, "get_or_fail_fail": 0, "get_or_fail_success": 8, "get_started": 0, "get_sum": 2725, "max": 3758096384, "put": 5, "put_sum": 2725, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-04-25T00:12:31.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:12:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.978+0000 7f903a800640 1 Processor -- start 2024-04-25T00:12:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.978+0000 7f903a800640 1 -- start start 2024-04-25T00:12:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.982+0000 7f903a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f903415dee0 0x7f90340b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.982+0000 7f903a800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90340aee90 con 0x7f9034071cd0 2024-04-25T00:12:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.982+0000 7f903a800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90340af170 con 0x7f903415e2c0 2024-04-25T00:12:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.982+0000 7f903a800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f90340af7c0 con 0x7f903415dee0 2024-04-25T00:12:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.982+0000 7f9033e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9034071cd0 0x7f903406d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42650/0 (socket says 172.21.15.83:42650) 2024-04-25T00:12:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.982+0000 7f9033e00640 1 -- 172.21.15.83:0/2312741567 learned_addr learned my addr 172.21.15.83:0/2312741567 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.982+0000 7f9033400640 1 --2- 172.21.15.83:0/2312741567 >> v2:172.21.15.86:3300/0 conn(0x7f903415dee0 0x7f90340b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.982+0000 7f9032a00640 1 -- 172.21.15.83:0/2312741567 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3739283279 0 0) 0x7f90340aee90 con 0x7f9034071cd0 2024-04-25T00:12:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.982+0000 7f9032a00640 1 -- 172.21.15.83:0/2312741567 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9018003530 con 0x7f9034071cd0 2024-04-25T00:12:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.982+0000 7f9033400640 1 -- 172.21.15.83:0/2312741567 >> v1:172.21.15.86:6789/0 conn(0x7f903415e2c0 legacy=0x7f90340715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.982+0000 7f9033400640 1 -- 172.21.15.83:0/2312741567 >> v1:172.21.15.83:6789/0 conn(0x7f9034071cd0 legacy=0x7f903406d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.982+0000 7f9033400640 1 -- 172.21.15.83:0/2312741567 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90340af450 con 0x7f903415dee0 2024-04-25T00:12:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.982+0000 7f9033400640 1 --2- 172.21.15.83:0/2312741567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f903415dee0 0x7f90340b44b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f90240097c0 tx=0x7f902401c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8a21a849270e680 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f9032a00640 1 -- 172.21.15.83:0/2312741567 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90240b0070 con 0x7f903415dee0 2024-04-25T00:12:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f9032a00640 1 -- 172.21.15.83:0/2312741567 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90240ab600 con 0x7f903415dee0 2024-04-25T00:12:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f9032a00640 1 -- 172.21.15.83:0/2312741567 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90240ab9d0 con 0x7f903415dee0 2024-04-25T00:12:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 -- 172.21.15.83:0/2312741567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f903415dee0 msgr2=0x7f90340b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 --2- 172.21.15.83:0/2312741567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f903415dee0 0x7f90340b44b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f90240097c0 tx=0x7f902401c380 comp rx=0 tx=0).stop 2024-04-25T00:12:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 -- 172.21.15.83:0/2312741567 shutdown_connections 2024-04-25T00:12:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 --2- 172.21.15.83:0/2312741567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f903415dee0 0x7f90340b44b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 -- 172.21.15.83:0/2312741567 >> 172.21.15.83:0/2312741567 conn(0x7f903406b1e0 msgr2=0x7f903406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 -- 172.21.15.83:0/2312741567 shutdown_connections 2024-04-25T00:12:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 -- 172.21.15.83:0/2312741567 wait complete. 2024-04-25T00:12:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 Processor -- start 2024-04-25T00:12:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 -- start start 2024-04-25T00:12:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9034071cd0 0x7f9034142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f903415dee0 0x7f90341636e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9034154ad0 0x7f9034157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f903416c140 con 0x7f9034071cd0 2024-04-25T00:12:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f903416bfc0 con 0x7f903415dee0 2024-04-25T00:12:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f903a800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f903416c2c0 con 0x7f9034154ad0 2024-04-25T00:12:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f9033400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9034071cd0 0x7f9034142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f9033400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9034071cd0 0x7f9034142c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52934/0 (socket says 172.21.15.83:52934) 2024-04-25T00:12:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f9033400640 1 -- 172.21.15.83:0/3430913876 learned_addr learned my addr 172.21.15.83:0/3430913876 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f9033400640 1 -- 172.21.15.83:0/3430913876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9034154ad0 msgr2=0x7f9034157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.986+0000 7f902aa00640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f903415dee0 0x7f90341636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f9033e00640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9034154ad0 0x7f9034157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f9033400640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9034154ad0 0x7f9034157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f9033400640 1 -- 172.21.15.83:0/3430913876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f903415dee0 msgr2=0x7f90341636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f9033400640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f903415dee0 0x7f90341636e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f9033400640 1 -- 172.21.15.83:0/3430913876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9034155040 con 0x7f9034071cd0 2024-04-25T00:12:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f902aa00640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f903415dee0 0x7f90341636e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f9033e00640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9034154ad0 0x7f9034157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f9033400640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9034071cd0 0x7f9034142c50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f90240ae040 tx=0x7f902401abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f902be00640 1 -- 172.21.15.83:0/3430913876 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90240b0050 con 0x7f9034071cd0 2024-04-25T00:12:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f902be00640 1 -- 172.21.15.83:0/3430913876 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90240bc070 con 0x7f9034071cd0 2024-04-25T00:12:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f902be00640 1 -- 172.21.15.83:0/3430913876 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90240b5870 con 0x7f9034071cd0 2024-04-25T00:12:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f903a800640 1 -- 172.21.15.83:0/3430913876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9034155320 con 0x7f9034071cd0 2024-04-25T00:12:31.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f903a800640 1 -- 172.21.15.83:0/3430913876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90341558e0 con 0x7f9034071cd0 2024-04-25T00:12:31.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f903a800640 1 -- 172.21.15.83:0/3430913876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9034073000 con 0x7f9034071cd0 2024-04-25T00:12:31.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f902be00640 1 -- 172.21.15.83:0/3430913876 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f90240b5a10 con 0x7f9034071cd0 2024-04-25T00:12:31.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f902be00640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f90100678c0 0x7f9010069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f902be00640 1 -- 172.21.15.83:0/3430913876 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f9024120620 con 0x7f9034071cd0 2024-04-25T00:12:31.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.990+0000 7f902aa00640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f90100678c0 0x7f9010069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.994+0000 7f902aa00640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f90100678c0 0x7f9010069d80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f90341fa6f0 tx=0x7f901c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:31.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:31.994+0000 7f902be00640 1 -- 172.21.15.83:0/3430913876 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9024016090 con 0x7f9034071cd0 2024-04-25T00:12:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32.190+0000 7f903a800640 1 -- 172.21.15.83:0/3430913876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f90340adba0 con 0x7f9034071cd0 2024-04-25T00:12:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32.190+0000 7f902be00640 1 -- 172.21.15.83:0/3430913876 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f90240f10e0 con 0x7f9034071cd0 2024-04-25T00:12:32.193 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:12:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32.194+0000 7f903a800640 1 -- 172.21.15.83:0/3430913876 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f90100678c0 msgr2=0x7f9010069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32.194+0000 7f903a800640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f90100678c0 0x7f9010069d80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f90341fa6f0 tx=0x7f901c009040 comp rx=0 tx=0).stop 2024-04-25T00:12:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32.194+0000 7f903a800640 1 -- 172.21.15.83:0/3430913876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9034071cd0 msgr2=0x7f9034142c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32.194+0000 7f903a800640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9034071cd0 0x7f9034142c50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f90240ae040 tx=0x7f902401abc0 comp rx=0 tx=0).stop 2024-04-25T00:12:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32.194+0000 7f903a800640 1 -- 172.21.15.83:0/3430913876 shutdown_connections 2024-04-25T00:12:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32.194+0000 7f903a800640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f90100678c0 0x7f9010069d80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32.194+0000 7f903a800640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9034154ad0 0x7f9034157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32.194+0000 7f903a800640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f903415dee0 0x7f90341636e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32.194+0000 7f903a800640 1 --2- 172.21.15.83:0/3430913876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9034071cd0 0x7f9034142c50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32.194+0000 7f903a800640 1 -- 172.21.15.83:0/3430913876 >> 172.21.15.83:0/3430913876 conn(0x7f903406b1e0 msgr2=0x7f9034160670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32.194+0000 7f903a800640 1 -- 172.21.15.83:0/3430913876 shutdown_connections 2024-04-25T00:12:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32.194+0000 7f903a800640 1 -- 172.21.15.83:0/3430913876 wait complete. 2024-04-25T00:12:32.213 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:12:32.213 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:12:32.213 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-25T00:12:32.213 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:12:32.213 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:12:32.214 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:12:32.214 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:12:32.214 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:12:32.214 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:12:32.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:32.214 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-25T00:12:32.223 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:12:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:12:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:12:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-25T00:12:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:32.223 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-25T00:12:32.224 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-25T00:12:32.224 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:32.224 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:12:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:12:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42382sec preferred_lft 42382sec 2024-04-25T00:12:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-25T00:12:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:32.225 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:12:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:8d:d4:7d:b9 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:12:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:32.225 INFO:teuthology.orchestra.run.smithi083.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:12:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:12:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:12:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:32.227 INFO:teuthology.orchestra.run.smithi083.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:12:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:12:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:12:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:32.227 INFO:teuthology.orchestra.run.smithi083.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:12:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:12:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:12:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:32.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:32.227 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:12:32.296 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-25T00:12:32.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:32.298 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-25T00:12:32.372 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-25T00:12:32.372 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:12:32.372 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:12:32.372 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:12:32.372 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:12:32.372 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:12:32.372 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:12:32.372 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:12:32.372 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:12:32.372 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:12:32.372 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:12:32.373 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:12:32.373 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:12:32.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:12:32.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:12:32.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:12:32.373 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:12:32.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:12:32.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:12:32.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:32.516 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:12:32.586 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:12:32.586 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:12:32.587 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:12:32.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:32.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-25T00:12:32.655 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:12:32.655 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:12:32.711 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:12:32.711 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:12:32.712 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:12:32.762 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T00:12:32.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:32.762 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:12:32.857 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:12:32.857 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:12:32.870 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:12:32.870 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:12:32.870 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:12:32.870 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:12:32.870 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:12:32.874 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:12:32.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:32.874 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:12:32.953 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:12:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:12:33.058 INFO:teuthology.orchestra.run.smithi083.stdout:client5601 2024-04-25T00:12:33.059 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:12:33.059 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:33.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-04-25T00:12:33.330 INFO:teuthology.orchestra.run.smithi083.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":41,"msgr_send_messages":36,"msgr_recv_bytes":11828,"msgr_send_bytes":22715,"msgr_created_connections":8,"msgr_active_connections":2,"msgr_running_total_time":0.018818331,"msgr_running_send_time":0.001214000,"msgr_running_recv_time":0.013930022,"msgr_running_fast_dispatch_time":0.000791459,"msgr_send_messages_queue_lat":{"avgcount":35,"sum":0.305909134,"avgtime":0.008740260},"msgr_handle_ack_lat":{"avgcount":15,"sum":0.000002352,"avgtime":0.000000156},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":11,"msgr_send_messages":18,"msgr_recv_bytes":2872,"msgr_send_bytes":99534,"msgr_created_connections":6,"msgr_active_connections":4,"msgr_running_total_time":0.013892617,"msgr_running_send_time":0.000909072,"msgr_running_recv_time":0.012516215,"msgr_running_fast_dispatch_time":0.000242373,"msgr_send_messages_queue_lat":{"avgcount":16,"sum":0.141040409,"avgtime":0.008815025},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":94880},"AsyncMessenger::Worker-2":{"msgr_recv_messages":102,"msgr_send_messages":93,"msgr_recv_bytes":123653,"msgr_send_bytes":54907,"msgr_created_connections":8,"msgr_active_connections":4,"msgr_running_total_time":0.015094384,"msgr_running_send_time":0.004491169,"msgr_running_recv_time":0.016006445,"msgr_running_fast_dispatch_time":0.000490142,"msgr_send_messages_queue_lat":{"avgcount":93,"sum":0.060252630,"avgtime":0.000647877},"msgr_handle_ack_lat":{"avgcount":68,"sum":0.000011475,"avgtime":0.000000168},"msgr_recv_encrypted_bytes":110432,"msgr_send_encrypted_bytes":11408},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.003681255,"avgtime":0.000068171}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000008570,"avgtime":0.000004285}},"mds":{"request":33,"reply":33,"reply_latency":{"avgcount":33,"sum":0.004725398,"avgtime":0.000143193},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":62,"traverse_hit":59,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":9,"evex":0,"evtrm":0,"ev":9,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197287,"rdpos":0,"jlat":{"avgcount":6,"sum":0.013419311,"avgtime":0.002236551},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":1,"cap+":3,"cap-":2,"rss":46904,"heap":231732},"mds_server":{"dispatch_client_request":33,"dispatch_server_request":0,"handle_client_request":33,"handle_client_session":7,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":29,"sum":0.004019769,"avgtime":0.000138612},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":3,"sum":0.000429993,"avgtime":0.000143331},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000275636,"avgtime":0.000275636},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":3,"session_remove":2,"sessions_open":1,"sessions_stale":0,"total_load":30,"average_load":30,"avg_session_uptime":0,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":233053,"buffer_anon_items":83,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":52856,"mds_co_items":216,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":44,"op_send_bytes":14103,"op_resend":35,"op_reply":44,"op_latency":{"avgcount":44,"sum":1.051979981,"avgtime":0.023908635},"op_inflight":0,"oplen_avg":{"avgcount":44,"sum":68},"op":44,"op_r":0,"op_w":44,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":6,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":38,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":149,"get_sum":120659,"get_or_fail_fail":0,"get_or_fail_success":149,"take":0,"take_sum":0,"put":149,"put_sum":120659,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":14103,"put":44,"put_sum":14103,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":44,"put":44,"put_sum":44,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55d89c54c780":{"val":0,"max":3758096384,"get_started":0,"get":9,"get_sum":2983,"get_or_fail_fail":0,"get_or_fail_success":9,"take":0,"take_sum":0,"put":6,"put_sum":2983,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-04-25T00:12:33.332 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 15, "avgtime": 1.56e-07, "sum": 2.352e-06 }, "msgr_recv_bytes": 11828, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 41, "msgr_running_fast_dispatch_time": 0.000791459, "msgr_running_recv_time": 0.013930022, "msgr_running_send_time": 0.001214, "msgr_running_total_time": 0.018818331, "msgr_send_bytes": 22715, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 36, "msgr_send_messages_queue_lat": { "avgcount": 35, "avgtime": 0.00874026, "sum": 0.305909134 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 2872, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 11, "msgr_running_fast_dispatch_time": 0.000242373, "msgr_running_recv_time": 0.012516215, "msgr_running_send_time": 0.000909072, "msgr_running_total_time": 0.013892617, "msgr_send_bytes": 99534, "msgr_send_encrypted_bytes": 94880, "msgr_send_messages": 18, "msgr_send_messages_queue_lat": { "avgcount": 16, "avgtime": 0.008815025, "sum": 0.141040409 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 68, "avgtime": 1.68e-07, "sum": 1.1475e-05 }, "msgr_recv_bytes": 123653, "msgr_recv_encrypted_bytes": 110432, "msgr_recv_messages": 102, "msgr_running_fast_dispatch_time": 0.000490142, "msgr_running_recv_time": 0.016006445, "msgr_running_send_time": 0.004491169, "msgr_running_total_time": 0.015094384, "msgr_send_bytes": 54907, "msgr_send_encrypted_bytes": 11408, "msgr_send_messages": 93, "msgr_send_messages_queue_lat": { "avgcount": 93, "avgtime": 0.000647877, "sum": 0.06025263 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 6.8171e-05, "sum": 0.003681255 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 4.285e-06, "sum": 8.57e-06 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 33, "reply_latency": { "avgcount": 33, "avgtime": 0.000143193, "sum": 0.004725398 }, "request": 33, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 62, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 59, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 9, "evadd": 9, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 6, "avgtime": 0.002236551, "sum": 0.013419311 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197287 }, "mds_mem": { "cap": 1, "cap+": 3, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 231732, "ino": 13, "ino+": 13, "ino-": 0, "rss": 46904 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 33, "dispatch_server_request": 0, "handle_client_request": 33, "handle_client_session": 7, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 29, "avgtime": 0.000138612, "sum": 0.004019769 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 3, "avgtime": 0.000143331, "sum": 0.000429993 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000275636, "sum": 0.000275636 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 30, "avg_session_uptime": 0, "mdthresh_evicted": 0, "session_add": 3, "session_count": 1, "session_remove": 2, "sessions_open": 1, "sessions_stale": 0, "total_load": 30 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 233053, "buffer_anon_items": 83, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 52856, "mds_co_items": 216, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 38, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 44, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 44, "avgtime": 0.023908635, "sum": 1.051979981 }, "op_pg": 0, "op_r": 0, "op_reply": 44, "op_resend": 35, "op_rmw": 0, "op_send": 44, "op_send_bytes": 14103, "op_w": 44, "oplen_avg": { "avgcount": 44, "sum": 68 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 6, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 149, "get_or_fail_fail": 0, "get_or_fail_success": 149, "get_started": 0, "get_sum": 120659, "max": 104857600, "put": 149, "put_sum": 120659, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 44, "put_sum": 14103, "take": 44, "take_sum": 14103, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 44, "put_sum": 44, "take": 44, "take_sum": 44, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55d89c54c780": { "get": 9, "get_or_fail_fail": 0, "get_or_fail_success": 9, "get_started": 0, "get_sum": 2983, "max": 3758096384, "put": 6, "put_sum": 2983, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-04-25T00:12:33.332 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:12:33.332 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:33.346 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:12:33.346 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:33.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cf9e00640 1 Processor -- start 2024-04-25T00:12:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cf9e00640 1 -- start start 2024-04-25T00:12:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cf9e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9cf40b4360 0x7f9cf40b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cf9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9cf40b2b30 0x7f9cf40b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cf9e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cf406ef80 con 0x7f9cf41553e0 2024-04-25T00:12:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cf9e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9cf415b510 con 0x7f9cf40b4360 2024-04-25T00:12:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cf9e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9cf415b7f0 con 0x7f9cf40b2b30 2024-04-25T00:12:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cebe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9cf40b4360 0x7f9cf40b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cebe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9cf40b4360 0x7f9cf40b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52948/0 (socket says 172.21.15.83:52948) 2024-04-25T00:12:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cebe00640 1 -- 172.21.15.83:0/658077528 learned_addr learned my addr 172.21.15.83:0/658077528 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cf3400640 1 --2- 172.21.15.83:0/658077528 >> v2:172.21.15.86:3300/0 conn(0x7f9cf40b2b30 0x7f9cf40b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cebe00640 1 -- 172.21.15.83:0/658077528 >> v2:172.21.15.86:3300/0 conn(0x7f9cf40b2b30 msgr2=0x7f9cf40b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:33.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cebe00640 1 --2- 172.21.15.83:0/658077528 >> v2:172.21.15.86:3300/0 conn(0x7f9cf40b2b30 0x7f9cf40b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cebe00640 1 -- 172.21.15.83:0/658077528 >> v1:172.21.15.86:6789/0 conn(0x7f9cf41553e0 legacy=0x7f9cf406e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:33.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.586+0000 7f9cebe00640 1 -- 172.21.15.83:0/658077528 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf415bad0 con 0x7f9cf40b4360 2024-04-25T00:12:33.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.590+0000 7f9cebe00640 1 --2- 172.21.15.83:0/658077528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf40b4360 0x7f9cf40b47e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4158fd0 tx=0x7f9cdc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ec0e796da5663e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.590+0000 7f9cf2000640 1 -- 172.21.15.83:0/658077528 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0b0070 con 0x7f9cf40b4360 2024-04-25T00:12:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.590+0000 7f9cf2000640 1 -- 172.21.15.83:0/658077528 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cdc0ab600 con 0x7f9cf40b4360 2024-04-25T00:12:33.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.590+0000 7f9cf2000640 1 -- 172.21.15.83:0/658077528 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0ab9d0 con 0x7f9cf40b4360 2024-04-25T00:12:33.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.590+0000 7f9cf9e00640 1 -- 172.21.15.83:0/658077528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf40b4360 msgr2=0x7f9cf40b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:33.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.590+0000 7f9cf9e00640 1 --2- 172.21.15.83:0/658077528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf40b4360 0x7f9cf40b47e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4158fd0 tx=0x7f9cdc01c490 comp rx=0 tx=0).stop 2024-04-25T00:12:33.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.590+0000 7f9cf9e00640 1 -- 172.21.15.83:0/658077528 shutdown_connections 2024-04-25T00:12:33.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.590+0000 7f9cf9e00640 1 --2- 172.21.15.83:0/658077528 >> v2:172.21.15.86:3300/0 conn(0x7f9cf40b2b30 0x7f9cf40b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.590+0000 7f9cf9e00640 1 --2- 172.21.15.83:0/658077528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf40b4360 0x7f9cf40b47e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.590+0000 7f9cf9e00640 1 -- 172.21.15.83:0/658077528 >> 172.21.15.83:0/658077528 conn(0x7f9cf40b1830 msgr2=0x7f9cf40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:33.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.590+0000 7f9cf9e00640 1 -- 172.21.15.83:0/658077528 shutdown_connections 2024-04-25T00:12:33.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.590+0000 7f9cf9e00640 1 -- 172.21.15.83:0/658077528 wait complete. 2024-04-25T00:12:33.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cf9e00640 1 Processor -- start 2024-04-25T00:12:33.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cf9e00640 1 -- start start 2024-04-25T00:12:33.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cf9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf40b2b30 0x7f9cf4154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cf9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf40b4360 0x7f9cf414f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cf9e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9cf41553e0 0x7f9cf4150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cf9e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf406ebd0 con 0x7f9cf40b4360 2024-04-25T00:12:33.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cf9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf4165d80 con 0x7f9cf40b2b30 2024-04-25T00:12:33.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cf9e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf41507b0 con 0x7f9cf41553e0 2024-04-25T00:12:33.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf40b4360 0x7f9cf414f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf40b4360 0x7f9cf414f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52950/0 (socket says 172.21.15.83:52950) 2024-04-25T00:12:33.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cebe00640 1 -- 172.21.15.83:0/659137979 learned_addr learned my addr 172.21.15.83:0/659137979 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:33.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cf3400640 1 --2- 172.21.15.83:0/659137979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9cf41553e0 0x7f9cf4150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cf2a00640 1 --2- 172.21.15.83:0/659137979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf40b2b30 0x7f9cf4154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cebe00640 1 -- 172.21.15.83:0/659137979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9cf41553e0 msgr2=0x7f9cf4150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:33.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cebe00640 1 --2- 172.21.15.83:0/659137979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9cf41553e0 0x7f9cf4150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cebe00640 1 -- 172.21.15.83:0/659137979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf40b2b30 msgr2=0x7f9cf4154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:33.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cebe00640 1 --2- 172.21.15.83:0/659137979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf40b2b30 0x7f9cf4154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cebe00640 1 -- 172.21.15.83:0/659137979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf4157680 con 0x7f9cf40b4360 2024-04-25T00:12:33.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cebe00640 1 --2- 172.21.15.83:0/659137979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf40b4360 0x7f9cf414f0a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc01ab40 tx=0x7f9cdc019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:33.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9ceb400640 1 -- 172.21.15.83:0/659137979 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0b0040 con 0x7f9cf40b4360 2024-04-25T00:12:33.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9ceb400640 1 -- 172.21.15.83:0/659137979 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cdc0b4420 con 0x7f9cf40b4360 2024-04-25T00:12:33.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9ceb400640 1 -- 172.21.15.83:0/659137979 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0b4810 con 0x7f9cf40b4360 2024-04-25T00:12:33.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cf9e00640 1 -- 172.21.15.83:0/659137979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf4157960 con 0x7f9cf40b4360 2024-04-25T00:12:33.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.594+0000 7f9cf9e00640 1 -- 172.21.15.83:0/659137979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf4157f20 con 0x7f9cf40b4360 2024-04-25T00:12:33.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.598+0000 7f9cf9e00640 1 -- 172.21.15.83:0/659137979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf40737f0 con 0x7f9cf40b4360 2024-04-25T00:12:33.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.598+0000 7f9ceb400640 1 -- 172.21.15.83:0/659137979 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9cdc0b49b0 con 0x7f9cf40b4360 2024-04-25T00:12:33.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.598+0000 7f9ceb400640 1 --2- 172.21.15.83:0/659137979 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9cd00678c0 0x7f9cd0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.598+0000 7f9ceb400640 1 -- 172.21.15.83:0/659137979 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f9cdc11fa60 con 0x7f9cf40b4360 2024-04-25T00:12:33.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.598+0000 7f9cf2a00640 1 --2- 172.21.15.83:0/659137979 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9cd00678c0 0x7f9cd0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.598+0000 7f9cf2a00640 1 --2- 172.21.15.83:0/659137979 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9cd00678c0 0x7f9cd0069d80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40015b0 tx=0x7f9ce4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:33.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.602+0000 7f9ceb400640 1 -- 172.21.15.83:0/659137979 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9cdc016110 con 0x7f9cf40b4360 2024-04-25T00:12:33.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.774+0000 7f9cf9e00640 1 -- 172.21.15.83:0/659137979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cf41580f0 con 0x7f9cf40b4360 2024-04-25T00:12:33.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.774+0000 7f9ceb400640 1 -- 172.21.15.83:0/659137979 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16858 (secure 0 0 0) 0x7f9cdc0f0400 con 0x7f9cf40b4360 2024-04-25T00:12:33.776 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:33.776 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:20.181255+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:17.079304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:17.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:33.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.778+0000 7f9cf9e00640 1 -- 172.21.15.83:0/659137979 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9cd00678c0 msgr2=0x7f9cd0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:33.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.778+0000 7f9cf9e00640 1 --2- 172.21.15.83:0/659137979 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9cd00678c0 0x7f9cd0069d80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40015b0 tx=0x7f9ce4009040 comp rx=0 tx=0).stop 2024-04-25T00:12:33.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.778+0000 7f9cf9e00640 1 -- 172.21.15.83:0/659137979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf40b4360 msgr2=0x7f9cf414f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:33.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.778+0000 7f9cf9e00640 1 --2- 172.21.15.83:0/659137979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf40b4360 0x7f9cf414f0a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc01ab40 tx=0x7f9cdc019230 comp rx=0 tx=0).stop 2024-04-25T00:12:33.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.778+0000 7f9cf9e00640 1 -- 172.21.15.83:0/659137979 shutdown_connections 2024-04-25T00:12:33.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.778+0000 7f9cf9e00640 1 --2- 172.21.15.83:0/659137979 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9cd00678c0 0x7f9cd0069d80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.778+0000 7f9cf9e00640 1 --2- 172.21.15.83:0/659137979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9cf41553e0 0x7f9cf4150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.778+0000 7f9cf9e00640 1 --2- 172.21.15.83:0/659137979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf40b4360 0x7f9cf414f0a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.778+0000 7f9cf9e00640 1 --2- 172.21.15.83:0/659137979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cf40b2b30 0x7f9cf4154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.778+0000 7f9cf9e00640 1 -- 172.21.15.83:0/659137979 >> 172.21.15.83:0/659137979 conn(0x7f9cf40b1830 msgr2=0x7f9cf415c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:33.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.778+0000 7f9cf9e00640 1 -- 172.21.15.83:0/659137979 shutdown_connections 2024-04-25T00:12:33.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.778+0000 7f9cf9e00640 1 -- 172.21.15.83:0/659137979 wait complete. 2024-04-25T00:12:33.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:12:17.079304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:12:17.548692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:12:33.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:12:33.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.934+0000 7f1893200640 1 Processor -- start 2024-04-25T00:12:33.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.934+0000 7f1893200640 1 -- start start 2024-04-25T00:12:33.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1893200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f188c15e180 0x7f188c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1893200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f188c0cea30 0x7f188c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1893200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f188c0b2ff0 con 0x7f188c0b2850 2024-04-25T00:12:33.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1893200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f188c0b11e0 con 0x7f188c15e180 2024-04-25T00:12:33.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1893200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f188c0b2da0 con 0x7f188c0cea30 2024-04-25T00:12:33.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1891e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f188c15e180 0x7f188c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1891400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f188c0b2850 0x7f188c0ce320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42666/0 (socket says 172.21.15.83:42666) 2024-04-25T00:12:33.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1891400640 1 -- 172.21.15.83:0/2739097936 learned_addr learned my addr 172.21.15.83:0/2739097936 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:33.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1891e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f188c15e180 0x7f188c0b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52966/0 (socket says 172.21.15.83:52966) 2024-04-25T00:12:33.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1891e00640 1 -- 172.21.15.83:0/2739097936 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f188c0cea30 msgr2=0x7f188c0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:33.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1892800640 1 --2- 172.21.15.83:0/2739097936 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f188c0cea30 0x7f188c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1890a00640 1 -- 172.21.15.83:0/2739097936 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1458133923 0 0) 0x7f188c0b2ff0 con 0x7f188c0b2850 2024-04-25T00:12:33.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1891e00640 1 --2- 172.21.15.83:0/2739097936 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f188c0cea30 0x7f188c0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1891e00640 1 -- 172.21.15.83:0/2739097936 >> v1:172.21.15.83:6789/0 conn(0x7f188c0b2850 legacy=0x7f188c0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1891e00640 1 -- 172.21.15.83:0/2739097936 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c0b32d0 con 0x7f188c15e180 2024-04-25T00:12:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1892800640 1 --2- 172.21.15.83:0/2739097936 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f188c0cea30 0x7f188c0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.938+0000 7f1891e00640 1 --2- 172.21.15.83:0/2739097936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f188c15e180 0x7f188c0b40d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f18800097c0 tx=0x7f188001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843d854434fca4d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1890a00640 1 -- 172.21.15.83:0/2739097936 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18800b0070 con 0x7f188c15e180 2024-04-25T00:12:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1890a00640 1 -- 172.21.15.83:0/2739097936 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18800ab600 con 0x7f188c15e180 2024-04-25T00:12:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1890a00640 1 -- 172.21.15.83:0/2739097936 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18800ab9d0 con 0x7f188c15e180 2024-04-25T00:12:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 -- 172.21.15.83:0/2739097936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f188c15e180 msgr2=0x7f188c0b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 --2- 172.21.15.83:0/2739097936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f188c15e180 0x7f188c0b40d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f18800097c0 tx=0x7f188001c380 comp rx=0 tx=0).stop 2024-04-25T00:12:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 -- 172.21.15.83:0/2739097936 shutdown_connections 2024-04-25T00:12:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 --2- 172.21.15.83:0/2739097936 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f188c0cea30 0x7f188c0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 --2- 172.21.15.83:0/2739097936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f188c15e180 0x7f188c0b40d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 -- 172.21.15.83:0/2739097936 >> 172.21.15.83:0/2739097936 conn(0x7f188c0b35b0 msgr2=0x7f188c0b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 -- 172.21.15.83:0/2739097936 shutdown_connections 2024-04-25T00:12:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 -- 172.21.15.83:0/2739097936 wait complete. 2024-04-25T00:12:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 Processor -- start 2024-04-25T00:12:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 -- start start 2024-04-25T00:12:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f188c0b2850 0x7f188c0c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188c0cea30 0x7f188c158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f188c15e180 0x7f188c14dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f188c15e6e0 con 0x7f188c15e180 2024-04-25T00:12:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f188c15e560 con 0x7f188c0cea30 2024-04-25T00:12:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1893200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f188c0d0b10 con 0x7f188c0b2850 2024-04-25T00:12:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1892800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f188c15e180 0x7f188c14dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1892800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f188c15e180 0x7f188c14dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52968/0 (socket says 172.21.15.83:52968) 2024-04-25T00:12:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1892800640 1 -- 172.21.15.83:0/404187642 learned_addr learned my addr 172.21.15.83:0/404187642 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1891e00640 1 --2- 172.21.15.83:0/404187642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f188c0b2850 0x7f188c0c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1891400640 1 --2- 172.21.15.83:0/404187642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188c0cea30 0x7f188c158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1892800640 1 -- 172.21.15.83:0/404187642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f188c0b2850 msgr2=0x7f188c0c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1892800640 1 --2- 172.21.15.83:0/404187642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f188c0b2850 0x7f188c0c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1892800640 1 -- 172.21.15.83:0/404187642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188c0cea30 msgr2=0x7f188c158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1892800640 1 --2- 172.21.15.83:0/404187642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188c0cea30 0x7f188c158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.942+0000 7f1892800640 1 -- 172.21.15.83:0/404187642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c1594d0 con 0x7f188c15e180 2024-04-25T00:12:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.946+0000 7f1892800640 1 --2- 172.21.15.83:0/404187642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f188c15e180 0x7f188c14dba0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f188800cef0 tx=0x7f188800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.946+0000 7f1886000640 1 -- 172.21.15.83:0/404187642 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188801a070 con 0x7f188c15e180 2024-04-25T00:12:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.946+0000 7f1886000640 1 -- 172.21.15.83:0/404187642 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f188800d860 con 0x7f188c15e180 2024-04-25T00:12:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.946+0000 7f1886000640 1 -- 172.21.15.83:0/404187642 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188800dc50 con 0x7f188c15e180 2024-04-25T00:12:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.946+0000 7f1893200640 1 -- 172.21.15.83:0/404187642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188c0ca570 con 0x7f188c15e180 2024-04-25T00:12:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.946+0000 7f1893200640 1 -- 172.21.15.83:0/404187642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188c0cab30 con 0x7f188c15e180 2024-04-25T00:12:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.946+0000 7f1893200640 1 -- 172.21.15.83:0/404187642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f188c04f510 con 0x7f188c15e180 2024-04-25T00:12:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.946+0000 7f1886000640 1 -- 172.21.15.83:0/404187642 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f188801d070 con 0x7f188c15e180 2024-04-25T00:12:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.946+0000 7f1886000640 1 --2- 172.21.15.83:0/404187642 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f18700678c0 0x7f1870069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.946+0000 7f1886000640 1 -- 172.21.15.83:0/404187642 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1888086670 con 0x7f188c15e180 2024-04-25T00:12:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.946+0000 7f1891e00640 1 --2- 172.21.15.83:0/404187642 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f18700678c0 0x7f1870069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.950+0000 7f1891e00640 1 --2- 172.21.15.83:0/404187642 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f18700678c0 0x7f1870069d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f18800ad000 tx=0x7f18800a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:33.950+0000 7f1886000640 1 -- 172.21.15.83:0/404187642 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f18880569f0 con 0x7f188c15e180 2024-04-25T00:12:34.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:12:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.106+0000 7f1893200640 1 -- 172.21.15.83:0/404187642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f188c06bf50 con 0x7f188c15e180 2024-04-25T00:12:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.106+0000 7f1886000640 1 -- 172.21.15.83:0/404187642 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f188805acf0 con 0x7f188c15e180 2024-04-25T00:12:34.113 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:12:34.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.110+0000 7f1893200640 1 -- 172.21.15.83:0/404187642 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f18700678c0 msgr2=0x7f1870069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.110+0000 7f1893200640 1 --2- 172.21.15.83:0/404187642 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f18700678c0 0x7f1870069d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f18800ad000 tx=0x7f18800a6000 comp rx=0 tx=0).stop 2024-04-25T00:12:34.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.110+0000 7f1893200640 1 -- 172.21.15.83:0/404187642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f188c15e180 msgr2=0x7f188c14dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.110+0000 7f1893200640 1 --2- 172.21.15.83:0/404187642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f188c15e180 0x7f188c14dba0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f188800cef0 tx=0x7f188800a710 comp rx=0 tx=0).stop 2024-04-25T00:12:34.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.110+0000 7f1893200640 1 -- 172.21.15.83:0/404187642 shutdown_connections 2024-04-25T00:12:34.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.110+0000 7f1893200640 1 --2- 172.21.15.83:0/404187642 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f18700678c0 0x7f1870069d80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.110+0000 7f1893200640 1 --2- 172.21.15.83:0/404187642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f188c15e180 0x7f188c14dba0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.110+0000 7f1893200640 1 --2- 172.21.15.83:0/404187642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188c0cea30 0x7f188c158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.110+0000 7f1893200640 1 --2- 172.21.15.83:0/404187642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f188c0b2850 0x7f188c0c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.110+0000 7f1893200640 1 -- 172.21.15.83:0/404187642 >> 172.21.15.83:0/404187642 conn(0x7f188c0b35b0 msgr2=0x7f188c156c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:34.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.110+0000 7f1893200640 1 -- 172.21.15.83:0/404187642 shutdown_connections 2024-04-25T00:12:34.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.110+0000 7f1893200640 1 -- 172.21.15.83:0/404187642 wait complete. 2024-04-25T00:12:34.118 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:12:34.131 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:12:34.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.314+0000 7fe7c8a00640 1 Processor -- start 2024-04-25T00:12:34.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.314+0000 7fe7c8a00640 1 -- start start 2024-04-25T00:12:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c8a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe7c4155310 0x7fe7c406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c8a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c40b3db0 0x7fe7c415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c8a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7c40719f0 con 0x7fe7c415e790 2024-04-25T00:12:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c8a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe7c415f610 con 0x7fe7c4155310 2024-04-25T00:12:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c8a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c4071cd0 con 0x7fe7c40b3db0 2024-04-25T00:12:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c3400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe7c4155310 0x7fe7c406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c3400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe7c4155310 0x7fe7c406cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44562/0 (socket says 172.21.15.83:44562) 2024-04-25T00:12:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c2a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe7c415e790 0x7fe7c415b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57216/0 (socket says 172.21.15.83:57216) 2024-04-25T00:12:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c3400640 1 -- 172.21.15.83:0/2009379779 learned_addr learned my addr 172.21.15.83:0/2009379779 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c3400640 1 -- 172.21.15.83:0/2009379779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c40b3db0 msgr2=0x7fe7c415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c3e00640 1 --2- 172.21.15.83:0/2009379779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c40b3db0 0x7fe7c415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c3400640 1 --2- 172.21.15.83:0/2009379779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c40b3db0 0x7fe7c415ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c3400640 1 -- 172.21.15.83:0/2009379779 >> v1:172.21.15.83:6789/0 conn(0x7fe7c415e790 legacy=0x7fe7c415b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c2000640 1 -- 172.21.15.83:0/2009379779 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1838560203 0 0) 0x7fe7c40719f0 con 0x7fe7c415e790 2024-04-25T00:12:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c3400640 1 -- 172.21.15.83:0/2009379779 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c4071fb0 con 0x7fe7c4155310 2024-04-25T00:12:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.318+0000 7fe7c3e00640 1 --2- 172.21.15.83:0/2009379779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c40b3db0 0x7fe7c415ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c3400640 1 --2- 172.21.15.83:0/2009379779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c4155310 0x7fe7c406cc60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009e50 tx=0x7fe7b401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b3da8e3ad46e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c2000640 1 -- 172.21.15.83:0/2009379779 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40a5660 con 0x7fe7c4155310 2024-04-25T00:12:34.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c2000640 1 -- 172.21.15.83:0/2009379779 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b40a5800 con 0x7fe7c4155310 2024-04-25T00:12:34.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c2000640 1 -- 172.21.15.83:0/2009379779 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40a5bd0 con 0x7fe7c4155310 2024-04-25T00:12:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 -- 172.21.15.83:0/2009379779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c4155310 msgr2=0x7fe7c406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 --2- 172.21.15.83:0/2009379779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c4155310 0x7fe7c406cc60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009e50 tx=0x7fe7b401c9a0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 -- 172.21.15.83:0/2009379779 shutdown_connections 2024-04-25T00:12:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 --2- 172.21.15.83:0/2009379779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c40b3db0 0x7fe7c415ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 --2- 172.21.15.83:0/2009379779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c4155310 0x7fe7c406cc60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 -- 172.21.15.83:0/2009379779 >> 172.21.15.83:0/2009379779 conn(0x7fe7c40cf980 msgr2=0x7fe7c40cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 -- 172.21.15.83:0/2009379779 shutdown_connections 2024-04-25T00:12:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 -- 172.21.15.83:0/2009379779 wait complete. 2024-04-25T00:12:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 Processor -- start 2024-04-25T00:12:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 -- start start 2024-04-25T00:12:34.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c40b3db0 0x7fe7c40cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4155310 0x7fe7c40b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c415e790 0x7fe7c414bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c4071740 con 0x7fe7c40b3db0 2024-04-25T00:12:34.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c40715c0 con 0x7fe7c4155310 2024-04-25T00:12:34.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c8a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c4167b30 con 0x7fe7c415e790 2024-04-25T00:12:34.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c40b3db0 0x7fe7c40cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c40b3db0 0x7fe7c40cedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44568/0 (socket says 172.21.15.83:44568) 2024-04-25T00:12:34.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.322+0000 7fe7c3400640 1 -- 172.21.15.83:0/1599886388 learned_addr learned my addr 172.21.15.83:0/1599886388 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:34.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7c3e00640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c415e790 0x7fe7c414bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7c2a00640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4155310 0x7fe7c40b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7c3400640 1 -- 172.21.15.83:0/1599886388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c415e790 msgr2=0x7fe7c414bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7c3400640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c415e790 0x7fe7c414bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7c3400640 1 -- 172.21.15.83:0/1599886388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4155310 msgr2=0x7fe7c40b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7c3400640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4155310 0x7fe7c40b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7c3400640 1 -- 172.21.15.83:0/1599886388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c415d2e0 con 0x7fe7c40b3db0 2024-04-25T00:12:34.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7c3e00640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c415e790 0x7fe7c414bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:34.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7c3400640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c40b3db0 0x7fe7c40cedd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe7b401a5d0 tx=0x7fe7b4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:34.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7bb400640 1 -- 172.21.15.83:0/1599886388 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40bc020 con 0x7fe7c40b3db0 2024-04-25T00:12:34.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7bb400640 1 -- 172.21.15.83:0/1599886388 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b40b8670 con 0x7fe7c40b3db0 2024-04-25T00:12:34.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7bb400640 1 -- 172.21.15.83:0/1599886388 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40b8a40 con 0x7fe7c40b3db0 2024-04-25T00:12:34.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7c8a00640 1 -- 172.21.15.83:0/1599886388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c40cd940 con 0x7fe7c40b3db0 2024-04-25T00:12:34.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7c8a00640 1 -- 172.21.15.83:0/1599886388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c40cdf00 con 0x7fe7c40b3db0 2024-04-25T00:12:34.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7c8a00640 1 -- 172.21.15.83:0/1599886388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c40cd770 con 0x7fe7c40b3db0 2024-04-25T00:12:34.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7bb400640 1 -- 172.21.15.83:0/1599886388 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe7b40bf020 con 0x7fe7c40b3db0 2024-04-25T00:12:34.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.326+0000 7fe7bb400640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7a40678c0 0x7fe7a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.330+0000 7fe7bb400640 1 -- 172.21.15.83:0/1599886388 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe7b4120de0 con 0x7fe7c40b3db0 2024-04-25T00:12:34.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.330+0000 7fe7c2a00640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7a40678c0 0x7fe7a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.330+0000 7fe7c2a00640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7a40678c0 0x7fe7a4069d80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe7c41fa2b0 tx=0x7fe7b0007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:34.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.334+0000 7fe7bb400640 1 -- 172.21.15.83:0/1599886388 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe7b40c6030 con 0x7fe7c40b3db0 2024-04-25T00:12:34.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.490+0000 7fe7c8a00640 1 -- 172.21.15.83:0/1599886388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe7c404f510 con 0x7fe7c40b3db0 2024-04-25T00:12:34.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.490+0000 7fe7bb400640 1 -- 172.21.15.83:0/1599886388 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) ==== 104+0+11 (secure 0 0 0) 0x7fe7b4016620 con 0x7fe7c40b3db0 2024-04-25T00:12:34.490 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:12:34.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.490+0000 7fe7c8a00640 1 -- 172.21.15.83:0/1599886388 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7a40678c0 msgr2=0x7fe7a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.490+0000 7fe7c8a00640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7a40678c0 0x7fe7a4069d80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe7c41fa2b0 tx=0x7fe7b0007df0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.490+0000 7fe7c8a00640 1 -- 172.21.15.83:0/1599886388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c40b3db0 msgr2=0x7fe7c40cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.490+0000 7fe7c8a00640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c40b3db0 0x7fe7c40cedd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe7b401a5d0 tx=0x7fe7b4019740 comp rx=0 tx=0).stop 2024-04-25T00:12:34.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.490+0000 7fe7c8a00640 1 -- 172.21.15.83:0/1599886388 shutdown_connections 2024-04-25T00:12:34.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.490+0000 7fe7c8a00640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7a40678c0 0x7fe7a4069d80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.490+0000 7fe7c8a00640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c415e790 0x7fe7c414bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.490+0000 7fe7c8a00640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c4155310 0x7fe7c40b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.490+0000 7fe7c8a00640 1 --2- 172.21.15.83:0/1599886388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c40b3db0 0x7fe7c40cedd0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.490+0000 7fe7c8a00640 1 -- 172.21.15.83:0/1599886388 >> 172.21.15.83:0/1599886388 conn(0x7fe7c40cf980 msgr2=0x7fe7c415dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:34.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.490+0000 7fe7c8a00640 1 -- 172.21.15.83:0/1599886388 shutdown_connections 2024-04-25T00:12:34.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.490+0000 7fe7c8a00640 1 -- 172.21.15.83:0/1599886388 wait complete. 2024-04-25T00:12:34.511 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:12:34.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.650+0000 7face7200640 1 Processor -- start 2024-04-25T00:12:34.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.650+0000 7face7200640 1 -- start start 2024-04-25T00:12:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.650+0000 7face7200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7face815e1b0 0x7face80cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.650+0000 7face7200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7face806f0b0 0x7face80ad830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.650+0000 7face7200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face80ad050 con 0x7face80d0230 2024-04-25T00:12:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.650+0000 7face7200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7face80d09b0 con 0x7face806f0b0 2024-04-25T00:12:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.650+0000 7face7200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7face80d0c90 con 0x7face815e1b0 2024-04-25T00:12:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.650+0000 7facdd400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7face80d0230 0x7face806e9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57228/0 (socket says 172.21.15.83:57228) 2024-04-25T00:12:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.650+0000 7facdd400640 1 -- 172.21.15.83:0/556198617 learned_addr learned my addr 172.21.15.83:0/556198617 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.650+0000 7face6800640 1 --2- 172.21.15.83:0/556198617 >> v2:172.21.15.83:3300/0 conn(0x7face806f0b0 0x7face80ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.650+0000 7face5e00640 1 --2- 172.21.15.83:0/556198617 >> v2:172.21.15.86:3300/0 conn(0x7face815e1b0 0x7face80cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.650+0000 7face5400640 1 -- 172.21.15.83:0/556198617 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 216341819 0 0) 0x7face80ad050 con 0x7face80d0230 2024-04-25T00:12:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face6800640 1 -- 172.21.15.83:0/556198617 >> v2:172.21.15.86:3300/0 conn(0x7face815e1b0 msgr2=0x7face80cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face6800640 1 --2- 172.21.15.83:0/556198617 >> v2:172.21.15.86:3300/0 conn(0x7face815e1b0 0x7face80cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face6800640 1 -- 172.21.15.83:0/556198617 >> v1:172.21.15.83:6789/0 conn(0x7face80d0230 legacy=0x7face806e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face6800640 1 -- 172.21.15.83:0/556198617 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face80d0f70 con 0x7face806f0b0 2024-04-25T00:12:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face6800640 1 --2- 172.21.15.83:0/556198617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face806f0b0 0x7face80ad830 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7face8072210 tx=0x7face001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bae0b5e90220974 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face5400640 1 -- 172.21.15.83:0/556198617 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face00b1070 con 0x7face806f0b0 2024-04-25T00:12:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face5400640 1 -- 172.21.15.83:0/556198617 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face00a8050 con 0x7face806f0b0 2024-04-25T00:12:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face5400640 1 -- 172.21.15.83:0/556198617 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face00ac860 con 0x7face806f0b0 2024-04-25T00:12:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face7200640 1 -- 172.21.15.83:0/556198617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face806f0b0 msgr2=0x7face80ad830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face7200640 1 --2- 172.21.15.83:0/556198617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face806f0b0 0x7face80ad830 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7face8072210 tx=0x7face001c3c0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face7200640 1 -- 172.21.15.83:0/556198617 shutdown_connections 2024-04-25T00:12:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face7200640 1 --2- 172.21.15.83:0/556198617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face806f0b0 0x7face80ad830 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face7200640 1 --2- 172.21.15.83:0/556198617 >> v2:172.21.15.86:3300/0 conn(0x7face815e1b0 0x7face80cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face7200640 1 -- 172.21.15.83:0/556198617 >> 172.21.15.83:0/556198617 conn(0x7face8072820 msgr2=0x7face8072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face7200640 1 -- 172.21.15.83:0/556198617 shutdown_connections 2024-04-25T00:12:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.654+0000 7face7200640 1 -- 172.21.15.83:0/556198617 wait complete. 2024-04-25T00:12:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face7200640 1 Processor -- start 2024-04-25T00:12:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face7200640 1 -- start start 2024-04-25T00:12:34.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face806f0b0 0x7face80cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face7200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7face80d0230 0x7face80ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face815e1b0 0x7face81fa090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7face80ae700 con 0x7face815e1b0 2024-04-25T00:12:34.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7face80ae580 con 0x7face806f0b0 2024-04-25T00:12:34.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face7200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7face80ae880 con 0x7face80d0230 2024-04-25T00:12:34.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face815e1b0 0x7face81fa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face815e1b0 0x7face81fa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44592/0 (socket says 172.21.15.83:44592) 2024-04-25T00:12:34.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face6800640 1 -- 172.21.15.83:0/863866083 learned_addr learned my addr 172.21.15.83:0/863866083 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face5e00640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face806f0b0 0x7face80cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7facdd400640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7face80d0230 0x7face80ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face6800640 1 -- 172.21.15.83:0/863866083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7face80d0230 msgr2=0x7face80ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face6800640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7face80d0230 0x7face80ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face6800640 1 -- 172.21.15.83:0/863866083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face806f0b0 msgr2=0x7face80cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face6800640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face806f0b0 0x7face80cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face6800640 1 -- 172.21.15.83:0/863866083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face80cb3a0 con 0x7face815e1b0 2024-04-25T00:12:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face5e00640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face806f0b0 0x7face80cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face6800640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face815e1b0 0x7face81fa090 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7face00af040 tx=0x7face0019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7facdea00640 1 -- 172.21.15.83:0/863866083 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face00b1040 con 0x7face815e1b0 2024-04-25T00:12:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7facdea00640 1 -- 172.21.15.83:0/863866083 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face00bc070 con 0x7face815e1b0 2024-04-25T00:12:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7facdea00640 1 -- 172.21.15.83:0/863866083 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face00b5410 con 0x7face815e1b0 2024-04-25T00:12:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.658+0000 7face7200640 1 -- 172.21.15.83:0/863866083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face80cb680 con 0x7face815e1b0 2024-04-25T00:12:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.662+0000 7face7200640 1 -- 172.21.15.83:0/863866083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face80cbc40 con 0x7face815e1b0 2024-04-25T00:12:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.662+0000 7face7200640 1 -- 172.21.15.83:0/863866083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face804f510 con 0x7face815e1b0 2024-04-25T00:12:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.662+0000 7facdea00640 1 -- 172.21.15.83:0/863866083 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7face00b55b0 con 0x7face815e1b0 2024-04-25T00:12:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.662+0000 7facdea00640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7facc40678c0 0x7facc4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.662+0000 7facdea00640 1 -- 172.21.15.83:0/863866083 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7face011fce0 con 0x7face815e1b0 2024-04-25T00:12:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.662+0000 7face5e00640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7facc40678c0 0x7facc4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.662+0000 7face5e00640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7facc40678c0 0x7facc4069d80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7face806dac0 tx=0x7facd800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:34.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.666+0000 7facdea00640 1 -- 172.21.15.83:0/863866083 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7face0016070 con 0x7face815e1b0 2024-04-25T00:12:34.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.822+0000 7face7200640 1 -- 172.21.15.83:0/863866083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7face80af4d0 con 0x7face815e1b0 2024-04-25T00:12:34.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.822+0000 7facdea00640 1 -- 172.21.15.83:0/863866083 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) ==== 100+0+11 (secure 0 0 0) 0x7face00f05f0 con 0x7face815e1b0 2024-04-25T00:12:34.823 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:12:34.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.822+0000 7face7200640 1 -- 172.21.15.83:0/863866083 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7facc40678c0 msgr2=0x7facc4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.822+0000 7face7200640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7facc40678c0 0x7facc4069d80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7face806dac0 tx=0x7facd800b040 comp rx=0 tx=0).stop 2024-04-25T00:12:34.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.822+0000 7face7200640 1 -- 172.21.15.83:0/863866083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face815e1b0 msgr2=0x7face81fa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.822+0000 7face7200640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face815e1b0 0x7face81fa090 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7face00af040 tx=0x7face0019160 comp rx=0 tx=0).stop 2024-04-25T00:12:34.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.822+0000 7face7200640 1 -- 172.21.15.83:0/863866083 shutdown_connections 2024-04-25T00:12:34.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.822+0000 7face7200640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7facc40678c0 0x7facc4069d80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.822+0000 7face7200640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7face80d0230 0x7face80ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.822+0000 7face7200640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7face815e1b0 0x7face81fa090 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.822+0000 7face7200640 1 --2- 172.21.15.83:0/863866083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face806f0b0 0x7face80cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.822+0000 7face7200640 1 -- 172.21.15.83:0/863866083 >> 172.21.15.83:0/863866083 conn(0x7face8072820 msgr2=0x7face80b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:34.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.822+0000 7face7200640 1 -- 172.21.15.83:0/863866083 shutdown_connections 2024-04-25T00:12:34.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.822+0000 7face7200640 1 -- 172.21.15.83:0/863866083 wait complete. 2024-04-25T00:12:34.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:12:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbda1200640 1 Processor -- start 2024-04-25T00:12:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbda1200640 1 -- start start 2024-04-25T00:12:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbda1200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd9c0cb640 0x7fbd9c06dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbda1200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd9c0711a0 con 0x7fbd9c06cfd0 2024-04-25T00:12:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbda1200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd9c06e160 con 0x7fbd9c070d10 2024-04-25T00:12:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbda1200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c06ea10 con 0x7fbd9c0cb640 2024-04-25T00:12:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd99e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbd9c06cfd0 0x7fbd9c06d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57232/0 (socket says 172.21.15.83:57232) 2024-04-25T00:12:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd99e00640 1 -- 172.21.15.83:0/2321472423 learned_addr learned my addr 172.21.15.83:0/2321472423 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd99400640 1 --2- 172.21.15.83:0/2321472423 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd9c0cb640 0x7fbd9c06dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 -- 172.21.15.83:0/2321472423 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1524306406 0 0) 0x7fbd8c0014a0 con 0x7fbd9c06cfd0 2024-04-25T00:12:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 -- 172.21.15.83:0/2321472423 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd80003530 con 0x7fbd9c06cfd0 2024-04-25T00:12:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 -- 172.21.15.83:0/2321472423 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 608638052 0 0) 0x7fbd80003530 con 0x7fbd9c06cfd0 2024-04-25T00:12:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 -- 172.21.15.83:0/2321472423 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd9c0cb640 msgr2=0x7fbd9c06dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 --2- 172.21.15.83:0/2321472423 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd9c0cb640 0x7fbd9c06dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 -- 172.21.15.83:0/2321472423 >> v1:172.21.15.86:6789/0 conn(0x7fbd9c070d10 legacy=0x7fbd9c165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 -- 172.21.15.83:0/2321472423 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c06ecf0 con 0x7fbd9c06cfd0 2024-04-25T00:12:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 -- 172.21.15.83:0/2321472423 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fbd8c001bd0 con 0x7fbd9c06cfd0 2024-04-25T00:12:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 -- 172.21.15.83:0/2321472423 >> v1:172.21.15.83:6789/0 conn(0x7fbd9c06cfd0 legacy=0x7fbd9c06d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 --2- 172.21.15.83:0/2321472423 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd9c0cb640 0x7fbd800039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 --2- 172.21.15.83:0/2321472423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd80003f30 0x7fbd80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 --2- 172.21.15.83:0/2321472423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd80004d50 0x7fbd800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 -- 172.21.15.83:0/2321472423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c06ea10 con 0x7fbd80003f30 2024-04-25T00:12:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 -- 172.21.15.83:0/2321472423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8000ba30 con 0x7fbd80004d50 2024-04-25T00:12:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd98a00640 1 -- 172.21.15.83:0/2321472423 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbd8000bc30 con 0x7fbd9c0cb640 2024-04-25T00:12:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd99400640 1 --2- 172.21.15.83:0/2321472423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd80003f30 0x7fbd80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd99e00640 1 --2- 172.21.15.83:0/2321472423 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd9c0cb640 0x7fbd800039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd9a800640 1 --2- 172.21.15.83:0/2321472423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd80004d50 0x7fbd800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd99400640 1 -- 172.21.15.83:0/2321472423 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd9c0cb640 msgr2=0x7fbd800039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd99400640 1 --2- 172.21.15.83:0/2321472423 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd9c0cb640 0x7fbd800039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd99400640 1 -- 172.21.15.83:0/2321472423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd80004d50 msgr2=0x7fbd800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd99400640 1 --2- 172.21.15.83:0/2321472423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd80004d50 0x7fbd800051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.986+0000 7fbd99400640 1 -- 172.21.15.83:0/2321472423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd80003530 con 0x7fbd80003f30 2024-04-25T00:12:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.990+0000 7fbd99400640 1 --2- 172.21.15.83:0/2321472423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd80003f30 0x7fbd80004810 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009ac0 tx=0x7fbd8801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1165c3fc1ab6b9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.990+0000 7fbd98a00640 1 -- 172.21.15.83:0/2321472423 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd880b2070 con 0x7fbd80003f30 2024-04-25T00:12:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.990+0000 7fbd98a00640 1 -- 172.21.15.83:0/2321472423 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd880a5580 con 0x7fbd80003f30 2024-04-25T00:12:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.990+0000 7fbd98a00640 1 -- 172.21.15.83:0/2321472423 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88009690 con 0x7fbd80003f30 2024-04-25T00:12:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.990+0000 7fbda1200640 1 -- 172.21.15.83:0/2321472423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd80003f30 msgr2=0x7fbd80004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.990+0000 7fbda1200640 1 --2- 172.21.15.83:0/2321472423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd80003f30 0x7fbd80004810 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009ac0 tx=0x7fbd8801c990 comp rx=0 tx=0).stop 2024-04-25T00:12:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.990+0000 7fbd9a800640 1 -- 172.21.15.83:0/2321472423 reap_dead start 2024-04-25T00:12:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.990+0000 7fbda1200640 1 -- 172.21.15.83:0/2321472423 shutdown_connections 2024-04-25T00:12:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.990+0000 7fbda1200640 1 -- 172.21.15.83:0/2321472423 >> 172.21.15.83:0/2321472423 conn(0x7fbd9c0cfa60 msgr2=0x7fbd9c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.990+0000 7fbda1200640 1 -- 172.21.15.83:0/2321472423 shutdown_connections 2024-04-25T00:12:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.990+0000 7fbda1200640 1 -- 172.21.15.83:0/2321472423 wait complete. 2024-04-25T00:12:34.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbda1200640 1 Processor -- start 2024-04-25T00:12:34.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbda1200640 1 -- start start 2024-04-25T00:12:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbda1200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd80003f30 0x7fbd9c0b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbda1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd9c13f4c0 0x7fbd9c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbda1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c1570a0 0x7fbd9c157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbda1200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1458e0 con 0x7fbd9c13f4c0 2024-04-25T00:12:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbda1200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c145760 con 0x7fbd9c1570a0 2024-04-25T00:12:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbda1200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c06dac0 con 0x7fbd80003f30 2024-04-25T00:12:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbd99400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd9c13f4c0 0x7fbd9c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbd99400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd9c13f4c0 0x7fbd9c0b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44604/0 (socket says 172.21.15.83:44604) 2024-04-25T00:12:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbd99400640 1 -- 172.21.15.83:0/1891897662 learned_addr learned my addr 172.21.15.83:0/1891897662 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbd99e00640 1 --2- 172.21.15.83:0/1891897662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd80003f30 0x7fbd9c0b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbd9a800640 1 --2- 172.21.15.83:0/1891897662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c1570a0 0x7fbd9c157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbd99400640 1 -- 172.21.15.83:0/1891897662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd80003f30 msgr2=0x7fbd9c0b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbd99400640 1 --2- 172.21.15.83:0/1891897662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd80003f30 0x7fbd9c0b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbd99400640 1 -- 172.21.15.83:0/1891897662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c1570a0 msgr2=0x7fbd9c157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbd99400640 1 --2- 172.21.15.83:0/1891897662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c1570a0 0x7fbd9c157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbd99400640 1 -- 172.21.15.83:0/1891897662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c0b40a0 con 0x7fbd9c13f4c0 2024-04-25T00:12:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbd99400640 1 --2- 172.21.15.83:0/1891897662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd9c13f4c0 0x7fbd9c0b4e50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbd8801c7c0 tx=0x7fbd80005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbd92000640 1 -- 172.21.15.83:0/1891897662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd880bc070 con 0x7fbd9c13f4c0 2024-04-25T00:12:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbd92000640 1 -- 172.21.15.83:0/1891897662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd880b7460 con 0x7fbd9c13f4c0 2024-04-25T00:12:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbd92000640 1 -- 172.21.15.83:0/1891897662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd880b2040 con 0x7fbd9c13f4c0 2024-04-25T00:12:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbda1200640 1 -- 172.21.15.83:0/1891897662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c0b3000 con 0x7fbd9c13f4c0 2024-04-25T00:12:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbda1200640 1 -- 172.21.15.83:0/1891897662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c0b35c0 con 0x7fbd9c13f4c0 2024-04-25T00:12:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.994+0000 7fbda1200640 1 -- 172.21.15.83:0/1891897662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd9c068230 con 0x7fbd9c13f4c0 2024-04-25T00:12:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.998+0000 7fbd92000640 1 -- 172.21.15.83:0/1891897662 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbd880b5070 con 0x7fbd9c13f4c0 2024-04-25T00:12:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.998+0000 7fbd92000640 1 --2- 172.21.15.83:0/1891897662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd7c0678f0 0x7fbd7c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.998+0000 7fbd92000640 1 -- 172.21.15.83:0/1891897662 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fbd8811ffe0 con 0x7fbd9c13f4c0 2024-04-25T00:12:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.998+0000 7fbd99e00640 1 --2- 172.21.15.83:0/1891897662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd7c0678f0 0x7fbd7c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:34.998+0000 7fbd99e00640 1 --2- 172.21.15.83:0/1891897662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd7c0678f0 0x7fbd7c069db0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0013e0 tx=0x7fbd8c001550 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.002+0000 7fbd92000640 1 -- 172.21.15.83:0/1891897662 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbd880c6020 con 0x7fbd9c13f4c0 2024-04-25T00:12:35.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.170+0000 7fbda1200640 1 -- 172.21.15.83:0/1891897662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbd9c06e160 con 0x7fbd9c13f4c0 2024-04-25T00:12:35.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.170+0000 7fbd92000640 1 -- 172.21.15.83:0/1891897662 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) ==== 80+0+0 (secure 0 0 0) 0x7fbd880166a0 con 0x7fbd9c13f4c0 2024-04-25T00:12:35.170 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:12:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.170+0000 7fbda1200640 1 -- 172.21.15.83:0/1891897662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd7c0678f0 msgr2=0x7fbd7c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.170+0000 7fbda1200640 1 --2- 172.21.15.83:0/1891897662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd7c0678f0 0x7fbd7c069db0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0013e0 tx=0x7fbd8c001550 comp rx=0 tx=0).stop 2024-04-25T00:12:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.170+0000 7fbda1200640 1 -- 172.21.15.83:0/1891897662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd9c13f4c0 msgr2=0x7fbd9c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.170+0000 7fbda1200640 1 --2- 172.21.15.83:0/1891897662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd9c13f4c0 0x7fbd9c0b4e50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbd8801c7c0 tx=0x7fbd80005840 comp rx=0 tx=0).stop 2024-04-25T00:12:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.170+0000 7fbda1200640 1 -- 172.21.15.83:0/1891897662 shutdown_connections 2024-04-25T00:12:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.170+0000 7fbda1200640 1 --2- 172.21.15.83:0/1891897662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd7c0678f0 0x7fbd7c069db0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.170+0000 7fbda1200640 1 --2- 172.21.15.83:0/1891897662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c1570a0 0x7fbd9c157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.170+0000 7fbda1200640 1 --2- 172.21.15.83:0/1891897662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd9c13f4c0 0x7fbd9c0b4e50 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.170+0000 7fbda1200640 1 --2- 172.21.15.83:0/1891897662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd80003f30 0x7fbd9c0b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.170+0000 7fbda1200640 1 -- 172.21.15.83:0/1891897662 >> 172.21.15.83:0/1891897662 conn(0x7fbd9c0cfa60 msgr2=0x7fbd9c06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.170+0000 7fbda1200640 1 -- 172.21.15.83:0/1891897662 shutdown_connections 2024-04-25T00:12:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.170+0000 7fbda1200640 1 -- 172.21.15.83:0/1891897662 wait complete. 2024-04-25T00:12:35.193 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:12:35.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:35.193 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:12:35.220 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:12:35.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:35.221 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:12:35.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-04-25T00:12:35.480 INFO:teuthology.orchestra.run.smithi083.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":43,"msgr_send_messages":38,"msgr_recv_bytes":12320,"msgr_send_bytes":23582,"msgr_created_connections":8,"msgr_active_connections":2,"msgr_running_total_time":0.018985469,"msgr_running_send_time":0.001271937,"msgr_running_recv_time":0.013964868,"msgr_running_fast_dispatch_time":0.000840483,"msgr_send_messages_queue_lat":{"avgcount":37,"sum":0.305951234,"avgtime":0.008268952},"msgr_handle_ack_lat":{"avgcount":15,"sum":0.000002352,"avgtime":0.000000156},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":11,"msgr_send_messages":18,"msgr_recv_bytes":2872,"msgr_send_bytes":99534,"msgr_created_connections":6,"msgr_active_connections":4,"msgr_running_total_time":0.013892617,"msgr_running_send_time":0.000909072,"msgr_running_recv_time":0.012516215,"msgr_running_fast_dispatch_time":0.000242373,"msgr_send_messages_queue_lat":{"avgcount":16,"sum":0.141040409,"avgtime":0.008815025},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":94880},"AsyncMessenger::Worker-2":{"msgr_recv_messages":106,"msgr_send_messages":95,"msgr_recv_bytes":124579,"msgr_send_bytes":55665,"msgr_created_connections":8,"msgr_active_connections":3,"msgr_running_total_time":0.015741559,"msgr_running_send_time":0.004792203,"msgr_running_recv_time":0.016176227,"msgr_running_fast_dispatch_time":0.000551383,"msgr_send_messages_queue_lat":{"avgcount":95,"sum":0.060360453,"avgtime":0.000635373},"msgr_handle_ack_lat":{"avgcount":71,"sum":0.000011800,"avgtime":0.000000166},"msgr_recv_encrypted_bytes":110704,"msgr_send_encrypted_bytes":12048},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":57,"sum":0.003751085,"avgtime":0.000065808}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000008570,"avgtime":0.000004285}},"mds":{"request":33,"reply":33,"reply_latency":{"avgcount":33,"sum":0.004725398,"avgtime":0.000143193},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":62,"traverse_hit":59,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":10,"evex":0,"evtrm":0,"ev":10,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197428,"rdpos":0,"jlat":{"avgcount":7,"sum":0.015578881,"avgtime":0.002225554},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":1,"cap+":3,"cap-":2,"rss":46904,"heap":231732},"mds_server":{"dispatch_client_request":33,"dispatch_server_request":0,"handle_client_request":33,"handle_client_session":9,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":29,"sum":0.004019769,"avgtime":0.000138612},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":3,"sum":0.000429993,"avgtime":0.000143331},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000275636,"avgtime":0.000275636},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":3,"session_remove":3,"sessions_open":0,"sessions_stale":0,"total_load":30,"average_load":30,"avg_session_uptime":0,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":233053,"buffer_anon_items":83,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":52856,"mds_co_items":216,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":46,"op_send_bytes":14334,"op_resend":35,"op_reply":46,"op_latency":{"avgcount":46,"sum":1.051979981,"avgtime":0.022869130},"op_inflight":0,"oplen_avg":{"avgcount":46,"sum":70},"op":46,"op_r":0,"op_w":46,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":7,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":38,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":155,"get_sum":121485,"get_or_fail_fail":0,"get_or_fail_success":155,"take":0,"take_sum":0,"put":155,"put_sum":121485,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":14334,"put":46,"put_sum":14334,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":46,"put":46,"put_sum":46,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55d89c54c780":{"val":0,"max":3758096384,"get_started":0,"get":10,"get_sum":3124,"get_or_fail_fail":0,"get_or_fail_success":10,"take":0,"take_sum":0,"put":7,"put_sum":3124,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-04-25T00:12:35.482 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 15, "avgtime": 1.56e-07, "sum": 2.352e-06 }, "msgr_recv_bytes": 12320, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 43, "msgr_running_fast_dispatch_time": 0.000840483, "msgr_running_recv_time": 0.013964868, "msgr_running_send_time": 0.001271937, "msgr_running_total_time": 0.018985469, "msgr_send_bytes": 23582, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 38, "msgr_send_messages_queue_lat": { "avgcount": 37, "avgtime": 0.008268952, "sum": 0.305951234 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 2872, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 11, "msgr_running_fast_dispatch_time": 0.000242373, "msgr_running_recv_time": 0.012516215, "msgr_running_send_time": 0.000909072, "msgr_running_total_time": 0.013892617, "msgr_send_bytes": 99534, "msgr_send_encrypted_bytes": 94880, "msgr_send_messages": 18, "msgr_send_messages_queue_lat": { "avgcount": 16, "avgtime": 0.008815025, "sum": 0.141040409 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 71, "avgtime": 1.66e-07, "sum": 1.18e-05 }, "msgr_recv_bytes": 124579, "msgr_recv_encrypted_bytes": 110704, "msgr_recv_messages": 106, "msgr_running_fast_dispatch_time": 0.000551383, "msgr_running_recv_time": 0.016176227, "msgr_running_send_time": 0.004792203, "msgr_running_total_time": 0.015741559, "msgr_send_bytes": 55665, "msgr_send_encrypted_bytes": 12048, "msgr_send_messages": 95, "msgr_send_messages_queue_lat": { "avgcount": 95, "avgtime": 0.000635373, "sum": 0.060360453 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 57, "avgtime": 6.5808e-05, "sum": 0.003751085 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 4.285e-06, "sum": 8.57e-06 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 33, "reply_latency": { "avgcount": 33, "avgtime": 0.000143193, "sum": 0.004725398 }, "request": 33, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 62, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 59, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 10, "evadd": 10, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 7, "avgtime": 0.002225554, "sum": 0.015578881 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197428 }, "mds_mem": { "cap": 1, "cap+": 3, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 231732, "ino": 13, "ino+": 13, "ino-": 0, "rss": 46904 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 33, "dispatch_server_request": 0, "handle_client_request": 33, "handle_client_session": 9, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 29, "avgtime": 0.000138612, "sum": 0.004019769 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 3, "avgtime": 0.000143331, "sum": 0.000429993 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000275636, "sum": 0.000275636 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 30, "avg_session_uptime": 0, "mdthresh_evicted": 0, "session_add": 3, "session_count": 0, "session_remove": 3, "sessions_open": 0, "sessions_stale": 0, "total_load": 30 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 233053, "buffer_anon_items": 83, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 52856, "mds_co_items": 216, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 38, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 46, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 46, "avgtime": 0.02286913, "sum": 1.051979981 }, "op_pg": 0, "op_r": 0, "op_reply": 46, "op_resend": 35, "op_rmw": 0, "op_send": 46, "op_send_bytes": 14334, "op_w": 46, "oplen_avg": { "avgcount": 46, "sum": 70 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 7, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 155, "get_or_fail_fail": 0, "get_or_fail_success": 155, "get_started": 0, "get_sum": 121485, "max": 104857600, "put": 155, "put_sum": 121485, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 46, "put_sum": 14334, "take": 46, "take_sum": 14334, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 46, "put_sum": 46, "take": 46, "take_sum": 46, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55d89c54c780": { "get": 10, "get_or_fail_fail": 0, "get_or_fail_success": 10, "get_started": 0, "get_sum": 3124, "max": 3758096384, "put": 7, "put_sum": 3124, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-04-25T00:12:35.482 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T00:12:35.482 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:12:35.482 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:12:35.588 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:12:35.677 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:12:35.677 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:12:35.677 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:12:35.677 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:35.685 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:12:35.686 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:12:35.686 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:12:35.686 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:35.695 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:12:35.695 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:12:35.695 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:12:35.695 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:35.736 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:12:35.737 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:12:35.737 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:12:35.737 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:35.749 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:12:35.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:12:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.910+0000 7f6a28800640 1 Processor -- start 2024-04-25T00:12:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.910+0000 7f6a28800640 1 -- start start 2024-04-25T00:12:35.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a28800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a201553e0 0x7f6a2006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:35.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a28800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a2006ef80 con 0x7f6a2006ea00 2024-04-25T00:12:35.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a28800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a2015b510 con 0x7f6a200b4a70 2024-04-25T00:12:35.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a28800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2015b7f0 con 0x7f6a201553e0 2024-04-25T00:12:35.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a25400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6a2006ea00 0x7f6a200b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57234/0 (socket says 172.21.15.83:57234) 2024-04-25T00:12:35.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a25400640 1 -- 172.21.15.83:0/3076151388 learned_addr learned my addr 172.21.15.83:0/3076151388 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:35.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 -- 172.21.15.83:0/3076151388 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1930859581 0 0) 0x7f6a2006ef80 con 0x7f6a2006ea00 2024-04-25T00:12:35.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 -- 172.21.15.83:0/3076151388 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a08003530 con 0x7f6a2006ea00 2024-04-25T00:12:35.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a25e00640 1 --2- 172.21.15.83:0/3076151388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a201553e0 0x7f6a2006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:35.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 -- 172.21.15.83:0/3076151388 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1881638499 0 0) 0x7f6a08003530 con 0x7f6a2006ea00 2024-04-25T00:12:35.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 -- 172.21.15.83:0/3076151388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a201553e0 msgr2=0x7f6a2006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 --2- 172.21.15.83:0/3076151388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a201553e0 0x7f6a2006e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 -- 172.21.15.83:0/3076151388 >> v1:172.21.15.86:6789/0 conn(0x7f6a200b4a70 legacy=0x7f6a200b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 -- 172.21.15.83:0/3076151388 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2015bad0 con 0x7f6a2006ea00 2024-04-25T00:12:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 -- 172.21.15.83:0/3076151388 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f6a10001680 con 0x7f6a2006ea00 2024-04-25T00:12:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 -- 172.21.15.83:0/3076151388 >> v1:172.21.15.83:6789/0 conn(0x7f6a2006ea00 legacy=0x7f6a200b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 --2- 172.21.15.83:0/3076151388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a080039f0 0x7f6a08003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 --2- 172.21.15.83:0/3076151388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a201553e0 0x7f6a08004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 --2- 172.21.15.83:0/3076151388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a08004d50 0x7f6a080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 -- 172.21.15.83:0/3076151388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2015b7f0 con 0x7f6a08004d50 2024-04-25T00:12:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 -- 172.21.15.83:0/3076151388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0800bbb0 con 0x7f6a080039f0 2024-04-25T00:12:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a24a00640 1 -- 172.21.15.83:0/3076151388 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0800bdb0 con 0x7f6a201553e0 2024-04-25T00:12:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a25e00640 1 --2- 172.21.15.83:0/3076151388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a080039f0 0x7f6a08003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a26800640 1 --2- 172.21.15.83:0/3076151388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a08004d50 0x7f6a080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a25400640 1 --2- 172.21.15.83:0/3076151388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a201553e0 0x7f6a08004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a26800640 1 -- 172.21.15.83:0/3076151388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a201553e0 msgr2=0x7f6a08004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a26800640 1 --2- 172.21.15.83:0/3076151388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a201553e0 0x7f6a08004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a26800640 1 -- 172.21.15.83:0/3076151388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a080039f0 msgr2=0x7f6a08003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a26800640 1 --2- 172.21.15.83:0/3076151388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a080039f0 0x7f6a08003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.914+0000 7f6a26800640 1 -- 172.21.15.83:0/3076151388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a08003530 con 0x7f6a08004d50 2024-04-25T00:12:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.918+0000 7f6a26800640 1 --2- 172.21.15.83:0/3076151388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a08004d50 0x7f6a080051d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c001e70 tx=0x7f6a1c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c567349f5fc5edef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.918+0000 7f6a24a00640 1 -- 172.21.15.83:0/3076151388 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c001a90 con 0x7f6a08004d50 2024-04-25T00:12:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.918+0000 7f6a24a00640 1 -- 172.21.15.83:0/3076151388 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a1c003b80 con 0x7f6a08004d50 2024-04-25T00:12:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.918+0000 7f6a24a00640 1 -- 172.21.15.83:0/3076151388 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c005080 con 0x7f6a08004d50 2024-04-25T00:12:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.918+0000 7f6a28800640 1 -- 172.21.15.83:0/3076151388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a08004d50 msgr2=0x7f6a080051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.918+0000 7f6a28800640 1 --2- 172.21.15.83:0/3076151388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a08004d50 0x7f6a080051d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c001e70 tx=0x7f6a1c01c780 comp rx=0 tx=0).stop 2024-04-25T00:12:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.918+0000 7f6a26800640 1 -- 172.21.15.83:0/3076151388 reap_dead start 2024-04-25T00:12:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.918+0000 7f6a28800640 1 -- 172.21.15.83:0/3076151388 shutdown_connections 2024-04-25T00:12:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.918+0000 7f6a28800640 1 -- 172.21.15.83:0/3076151388 >> 172.21.15.83:0/3076151388 conn(0x7f6a200b1830 msgr2=0x7f6a200b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.918+0000 7f6a28800640 1 -- 172.21.15.83:0/3076151388 shutdown_connections 2024-04-25T00:12:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.918+0000 7f6a28800640 1 -- 172.21.15.83:0/3076151388 wait complete. 2024-04-25T00:12:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.918+0000 7f6a28800640 1 Processor -- start 2024-04-25T00:12:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.918+0000 7f6a28800640 1 -- start start 2024-04-25T00:12:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a28800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a08004d50 0x7f6a200ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a28800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a20149110 0x7f6a20149590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a28800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a2015a050 0x7f6a2015a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a28800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a201557c0 con 0x7f6a2015a050 2024-04-25T00:12:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a28800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a20070f30 con 0x7f6a08004d50 2024-04-25T00:12:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a28800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6a20155940 con 0x7f6a20149110 2024-04-25T00:12:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a26800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a2015a050 0x7f6a2015a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a26800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a2015a050 0x7f6a2015a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44618/0 (socket says 172.21.15.83:44618) 2024-04-25T00:12:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a26800640 1 -- 172.21.15.83:0/2875008434 learned_addr learned my addr 172.21.15.83:0/2875008434 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a25400640 1 --2- 172.21.15.83:0/2875008434 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a20149110 0x7f6a20149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a25e00640 1 --2- 172.21.15.83:0/2875008434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a08004d50 0x7f6a200ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a26800640 1 -- 172.21.15.83:0/2875008434 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a20149110 msgr2=0x7f6a20149590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a26800640 1 --2- 172.21.15.83:0/2875008434 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a20149110 0x7f6a20149590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a26800640 1 -- 172.21.15.83:0/2875008434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a08004d50 msgr2=0x7f6a200ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:35.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a26800640 1 --2- 172.21.15.83:0/2875008434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a08004d50 0x7f6a200ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:35.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a26800640 1 -- 172.21.15.83:0/2875008434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a20149ad0 con 0x7f6a2015a050 2024-04-25T00:12:35.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a26800640 1 --2- 172.21.15.83:0/2875008434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a2015a050 0x7f6a2015a4d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c098080 tx=0x7f6a08005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:35.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a1a000640 1 -- 172.21.15.83:0/2875008434 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0bc070 con 0x7f6a2015a050 2024-04-25T00:12:35.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a1a000640 1 -- 172.21.15.83:0/2875008434 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a1c005610 con 0x7f6a2015a050 2024-04-25T00:12:35.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a1a000640 1 -- 172.21.15.83:0/2875008434 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c001a90 con 0x7f6a2015a050 2024-04-25T00:12:35.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a28800640 1 -- 172.21.15.83:0/2875008434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a200c9ec0 con 0x7f6a2015a050 2024-04-25T00:12:35.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a28800640 1 -- 172.21.15.83:0/2875008434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a200c93e0 con 0x7f6a2015a050 2024-04-25T00:12:35.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a28800640 1 -- 172.21.15.83:0/2875008434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a200737f0 con 0x7f6a2015a050 2024-04-25T00:12:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.922+0000 7f6a1a000640 1 -- 172.21.15.83:0/2875008434 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6a1c0b1040 con 0x7f6a2015a050 2024-04-25T00:12:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.926+0000 7f6a1a000640 1 --2- 172.21.15.83:0/2875008434 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6a040678c0 0x7f6a04069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.926+0000 7f6a1a000640 1 -- 172.21.15.83:0/2875008434 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f6a1c11f8f0 con 0x7f6a2015a050 2024-04-25T00:12:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.926+0000 7f6a25e00640 1 --2- 172.21.15.83:0/2875008434 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6a040678c0 0x7f6a04069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.926+0000 7f6a25e00640 1 --2- 172.21.15.83:0/2875008434 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6a040678c0 0x7f6a04069d80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6a20149e00 tx=0x7f6a14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:35.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:35.930+0000 7f6a1a000640 1 -- 172.21.15.83:0/2875008434 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6a1c0ad020 con 0x7f6a2015a050 2024-04-25T00:12:36.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.114+0000 7f6a28800640 1 -- 172.21.15.83:0/2875008434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a2004f5f0 con 0x7f6a2015a050 2024-04-25T00:12:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.114+0000 7f6a1a000640 1 -- 172.21.15.83:0/2875008434 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7f6a1c016490 con 0x7f6a2015a050 2024-04-25T00:12:36.117 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:36.117 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4277921554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":4277921554},{"type":"v1","addr":"172.21.15.86:6835","nonce":4277921554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/4145334103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":4145334103},{"type":"v1","addr":"172.21.15.86:6834","nonce":4145334103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3441801758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3441801758},{"type":"v1","addr":"172.21.15.83:6836","nonce":3441801758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:12:18.566138+0000","modified":"2024-04-25T00:12:21.572278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5328},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5328":{"gid":5328,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2704243724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2704243724},{"type":"v1","addr":"172.21.15.83:6837","nonce":2704243724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5328,"qdb_cluster":[5328]},"id":2}]} 2024-04-25T00:12:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 18 2024-04-25T00:12:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.118+0000 7f6a28800640 1 -- 172.21.15.83:0/2875008434 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6a040678c0 msgr2=0x7f6a04069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.118+0000 7f6a28800640 1 --2- 172.21.15.83:0/2875008434 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6a040678c0 0x7f6a04069d80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6a20149e00 tx=0x7f6a14008040 comp rx=0 tx=0).stop 2024-04-25T00:12:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.118+0000 7f6a28800640 1 -- 172.21.15.83:0/2875008434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a2015a050 msgr2=0x7f6a2015a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.118+0000 7f6a28800640 1 --2- 172.21.15.83:0/2875008434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a2015a050 0x7f6a2015a4d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c098080 tx=0x7f6a08005840 comp rx=0 tx=0).stop 2024-04-25T00:12:36.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.118+0000 7f6a28800640 1 -- 172.21.15.83:0/2875008434 shutdown_connections 2024-04-25T00:12:36.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.118+0000 7f6a28800640 1 --2- 172.21.15.83:0/2875008434 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6a040678c0 0x7f6a04069d80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.118+0000 7f6a28800640 1 --2- 172.21.15.83:0/2875008434 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a20149110 0x7f6a20149590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.118+0000 7f6a28800640 1 --2- 172.21.15.83:0/2875008434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a2015a050 0x7f6a2015a4d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.118+0000 7f6a28800640 1 --2- 172.21.15.83:0/2875008434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a08004d50 0x7f6a200ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.118+0000 7f6a28800640 1 -- 172.21.15.83:0/2875008434 >> 172.21.15.83:0/2875008434 conn(0x7f6a200b1830 msgr2=0x7f6a2015c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:36.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.118+0000 7f6a28800640 1 -- 172.21.15.83:0/2875008434 shutdown_connections 2024-04-25T00:12:36.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.118+0000 7f6a28800640 1 -- 172.21.15.83:0/2875008434 wait complete. 2024-04-25T00:12:36.138 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d63200640 1 Processor -- start 2024-04-25T00:12:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d63200640 1 -- start start 2024-04-25T00:12:36.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d63200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d5c15bf70 0x7f0d5c0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d63200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d5c0cfd80 0x7f0d5c0d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d63200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0d5c0d0770 0x7f0d5c0b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d63200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0d5c0d0bc0 con 0x7f0d5c0cfd80 2024-04-25T00:12:36.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d63200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0d5c0cc9b0 con 0x7f0d5c0d0770 2024-04-25T00:12:36.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d63200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5c0ccc90 con 0x7f0d5c15bf70 2024-04-25T00:12:36.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d53400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d5c0cfd80 0x7f0d5c0d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d53400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d5c0cfd80 0x7f0d5c0d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44632/0 (socket says 172.21.15.83:44632) 2024-04-25T00:12:36.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d53400640 1 -- 172.21.15.83:0/4182742736 learned_addr learned my addr 172.21.15.83:0/4182742736 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:36.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d61200640 1 --2- 172.21.15.83:0/4182742736 >> v2:172.21.15.86:3300/0 conn(0x7f0d5c0d0770 0x7f0d5c0b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d5be00640 1 --2- 172.21.15.83:0/4182742736 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d5c15bf70 0x7f0d5c0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d53400640 1 -- 172.21.15.83:0/4182742736 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d5c15bf70 msgr2=0x7f0d5c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:36.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d53400640 1 --2- 172.21.15.83:0/4182742736 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d5c15bf70 0x7f0d5c0cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d53400640 1 -- 172.21.15.83:0/4182742736 >> v2:172.21.15.86:3300/0 conn(0x7f0d5c0d0770 msgr2=0x7f0d5c0b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:36.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d53400640 1 --2- 172.21.15.83:0/4182742736 >> v2:172.21.15.86:3300/0 conn(0x7f0d5c0d0770 0x7f0d5c0b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d53400640 1 -- 172.21.15.83:0/4182742736 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c0ccf70 con 0x7f0d5c0cfd80 2024-04-25T00:12:36.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.274+0000 7f0d5be00640 1 --2- 172.21.15.83:0/4182742736 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d5c15bf70 0x7f0d5c0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:36.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.278+0000 7f0d53400640 1 --2- 172.21.15.83:0/4182742736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5c0cfd80 0x7f0d5c0d0230 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c072b70 tx=0x7f0d4401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6641952c9e4c957f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:36.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.278+0000 7f0d5b400640 1 -- 172.21.15.83:0/4182742736 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440aa610 con 0x7f0d5c0cfd80 2024-04-25T00:12:36.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.278+0000 7f0d5b400640 1 -- 172.21.15.83:0/4182742736 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d440af070 con 0x7f0d5c0cfd80 2024-04-25T00:12:36.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.278+0000 7f0d5b400640 1 -- 172.21.15.83:0/4182742736 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440aa9d0 con 0x7f0d5c0cfd80 2024-04-25T00:12:36.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.278+0000 7f0d63200640 1 -- 172.21.15.83:0/4182742736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5c0cfd80 msgr2=0x7f0d5c0d0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.278+0000 7f0d63200640 1 --2- 172.21.15.83:0/4182742736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5c0cfd80 0x7f0d5c0d0230 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c072b70 tx=0x7f0d4401c4f0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.278+0000 7f0d63200640 1 -- 172.21.15.83:0/4182742736 shutdown_connections 2024-04-25T00:12:36.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.278+0000 7f0d63200640 1 --2- 172.21.15.83:0/4182742736 >> v2:172.21.15.86:3300/0 conn(0x7f0d5c0d0770 0x7f0d5c0b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.278+0000 7f0d63200640 1 --2- 172.21.15.83:0/4182742736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5c0cfd80 0x7f0d5c0d0230 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.278+0000 7f0d63200640 1 --2- 172.21.15.83:0/4182742736 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d5c15bf70 0x7f0d5c0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.278+0000 7f0d63200640 1 -- 172.21.15.83:0/4182742736 >> 172.21.15.83:0/4182742736 conn(0x7f0d5c0d1c70 msgr2=0x7f0d5c0d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:36.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.278+0000 7f0d63200640 1 -- 172.21.15.83:0/4182742736 shutdown_connections 2024-04-25T00:12:36.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.278+0000 7f0d63200640 1 -- 172.21.15.83:0/4182742736 wait complete. 2024-04-25T00:12:36.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d63200640 1 Processor -- start 2024-04-25T00:12:36.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d63200640 1 -- start start 2024-04-25T00:12:36.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d63200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c0cfd80 0x7f0d5c154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d63200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d5c0d0770 0x7f0d5c155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d63200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5c15bf70 0x7f0d5c15a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d63200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c0cda50 con 0x7f0d5c15bf70 2024-04-25T00:12:36.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d63200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c15c350 con 0x7f0d5c0cfd80 2024-04-25T00:12:36.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d63200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5c158fa0 con 0x7f0d5c0d0770 2024-04-25T00:12:36.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d61200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5c15bf70 0x7f0d5c15a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d61200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5c15bf70 0x7f0d5c15a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44640/0 (socket says 172.21.15.83:44640) 2024-04-25T00:12:36.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d61200640 1 -- 172.21.15.83:0/3541999896 learned_addr learned my addr 172.21.15.83:0/3541999896 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:36.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d5be00640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c0cfd80 0x7f0d5c154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d61200640 1 -- 172.21.15.83:0/3541999896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d5c0d0770 msgr2=0x7f0d5c155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d53400640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d5c0d0770 0x7f0d5c155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d61200640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d5c0d0770 0x7f0d5c155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d61200640 1 -- 172.21.15.83:0/3541999896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c0cfd80 msgr2=0x7f0d5c154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d61200640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c0cfd80 0x7f0d5c154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d61200640 1 -- 172.21.15.83:0/3541999896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c159280 con 0x7f0d5c15bf70 2024-04-25T00:12:36.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d5be00640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c0cfd80 0x7f0d5c154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:36.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d53400640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d5c0d0770 0x7f0d5c155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:36.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d61200640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5c15bf70 0x7f0d5c15a040 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400eed0 tx=0x7f0d5400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:36.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d58c00640 1 -- 172.21.15.83:0/3541999896 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54018020 con 0x7f0d5c15bf70 2024-04-25T00:12:36.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d58c00640 1 -- 172.21.15.83:0/3541999896 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d5400bd20 con 0x7f0d5c15bf70 2024-04-25T00:12:36.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d63200640 1 -- 172.21.15.83:0/3541999896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5c159560 con 0x7f0d5c15bf70 2024-04-25T00:12:36.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d58c00640 1 -- 172.21.15.83:0/3541999896 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54014640 con 0x7f0d5c15bf70 2024-04-25T00:12:36.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d63200640 1 -- 172.21.15.83:0/3541999896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5c159b20 con 0x7f0d5c15bf70 2024-04-25T00:12:36.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.282+0000 7f0d63200640 1 -- 172.21.15.83:0/3541999896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d5c09fd60 con 0x7f0d5c15bf70 2024-04-25T00:12:36.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.286+0000 7f0d58c00640 1 -- 172.21.15.83:0/3541999896 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0d5401b020 con 0x7f0d5c15bf70 2024-04-25T00:12:36.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.286+0000 7f0d58c00640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0d380678c0 0x7f0d38069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.286+0000 7f0d58c00640 1 -- 172.21.15.83:0/3541999896 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f0d540864d0 con 0x7f0d5c15bf70 2024-04-25T00:12:36.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.286+0000 7f0d5be00640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0d380678c0 0x7f0d38069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.286+0000 7f0d5be00640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0d380678c0 0x7f0d38069d80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009980 tx=0x7f0d4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:36.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.290+0000 7f0d58c00640 1 -- 172.21.15.83:0/3541999896 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0d54022070 con 0x7f0d5c15bf70 2024-04-25T00:12:36.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.450+0000 7f0d63200640 1 -- 172.21.15.83:0/3541999896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d5c04f5f0 con 0x7f0d5c15bf70 2024-04-25T00:12:36.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.450+0000 7f0d58c00640 1 -- 172.21.15.83:0/3541999896 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16858 (secure 0 0 0) 0x7f0d54056de0 con 0x7f0d5c15bf70 2024-04-25T00:12:36.452 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:36.452 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:20.181255+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:17.079304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:17.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:36.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.450+0000 7f0d63200640 1 -- 172.21.15.83:0/3541999896 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0d380678c0 msgr2=0x7f0d38069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.450+0000 7f0d63200640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0d380678c0 0x7f0d38069d80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009980 tx=0x7f0d4c008040 comp rx=0 tx=0).stop 2024-04-25T00:12:36.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.450+0000 7f0d63200640 1 -- 172.21.15.83:0/3541999896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5c15bf70 msgr2=0x7f0d5c15a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.450+0000 7f0d63200640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5c15bf70 0x7f0d5c15a040 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400eed0 tx=0x7f0d5400b480 comp rx=0 tx=0).stop 2024-04-25T00:12:36.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.450+0000 7f0d63200640 1 -- 172.21.15.83:0/3541999896 shutdown_connections 2024-04-25T00:12:36.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.450+0000 7f0d63200640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0d380678c0 0x7f0d38069d80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.450+0000 7f0d63200640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d5c0d0770 0x7f0d5c155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.450+0000 7f0d63200640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d5c15bf70 0x7f0d5c15a040 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.450+0000 7f0d63200640 1 --2- 172.21.15.83:0/3541999896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d5c0cfd80 0x7f0d5c154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.450+0000 7f0d63200640 1 -- 172.21.15.83:0/3541999896 >> 172.21.15.83:0/3541999896 conn(0x7f0d5c0d1c70 msgr2=0x7f0d5c06dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:36.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.450+0000 7f0d63200640 1 -- 172.21.15.83:0/3541999896 shutdown_connections 2024-04-25T00:12:36.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.450+0000 7f0d63200640 1 -- 172.21.15.83:0/3541999896 wait complete. 2024-04-25T00:12:36.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:12:17.079304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:12:17.548692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:12:36.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:12:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.610+0000 7f1cbb400640 1 Processor -- start 2024-04-25T00:12:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.610+0000 7f1cbb400640 1 -- start start 2024-04-25T00:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.610+0000 7f1cbb400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1cb41551e0 0x7f1cb415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.610+0000 7f1cbb400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cb415c560 con 0x7f1cb406e4c0 2024-04-25T00:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.614+0000 7f1cbb400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cb4072460 con 0x7f1cb41555c0 2024-04-25T00:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.614+0000 7f1cbb400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1cb415c7d0 con 0x7f1cb41551e0 2024-04-25T00:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.614+0000 7f1cb9400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1cb406e4c0 0x7f1cb415cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57250/0 (socket says 172.21.15.83:57250) 2024-04-25T00:12:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.614+0000 7f1cb8a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1cb41551e0 0x7f1cb415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.614+0000 7f1cb9400640 1 -- 172.21.15.83:0/1755561633 learned_addr learned my addr 172.21.15.83:0/1755561633 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.614+0000 7f1cb8a00640 1 -- 172.21.15.83:0/1755561633 >> v1:172.21.15.86:6789/0 conn(0x7f1cb41555c0 legacy=0x7f1cb4071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.614+0000 7f1cb8a00640 1 -- 172.21.15.83:0/1755561633 >> v1:172.21.15.83:6789/0 conn(0x7f1cb406e4c0 legacy=0x7f1cb415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.614+0000 7f1cb8a00640 1 -- 172.21.15.83:0/1755561633 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb415cab0 con 0x7f1cb41551e0 2024-04-25T00:12:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.614+0000 7f1cb8a00640 1 --2- 172.21.15.83:0/1755561633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb41551e0 0x7f1cb415bea0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40097f0 tx=0x7f1ca401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfa62dc6afa28e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.614+0000 7f1cb3e00640 1 -- 172.21.15.83:0/1755561633 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40b0070 con 0x7f1cb41551e0 2024-04-25T00:12:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.614+0000 7f1cb3e00640 1 -- 172.21.15.83:0/1755561633 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca40ab600 con 0x7f1cb41551e0 2024-04-25T00:12:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.614+0000 7f1cb3e00640 1 -- 172.21.15.83:0/1755561633 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40ab9d0 con 0x7f1cb41551e0 2024-04-25T00:12:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 -- 172.21.15.83:0/1755561633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb41551e0 msgr2=0x7f1cb415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 --2- 172.21.15.83:0/1755561633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb41551e0 0x7f1cb415bea0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40097f0 tx=0x7f1ca401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 -- 172.21.15.83:0/1755561633 shutdown_connections 2024-04-25T00:12:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 --2- 172.21.15.83:0/1755561633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb41551e0 0x7f1cb415bea0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 -- 172.21.15.83:0/1755561633 >> 172.21.15.83:0/1755561633 conn(0x7f1cb40b1830 msgr2=0x7f1cb40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 -- 172.21.15.83:0/1755561633 shutdown_connections 2024-04-25T00:12:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 -- 172.21.15.83:0/1755561633 wait complete. 2024-04-25T00:12:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 Processor -- start 2024-04-25T00:12:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 -- start start 2024-04-25T00:12:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb406e4c0 0x7f1cb4144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb41551e0 0x7f1cb4142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cb41555c0 0x7f1cb4148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb415e300 con 0x7f1cb406e4c0 2024-04-25T00:12:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb415e180 con 0x7f1cb41551e0 2024-04-25T00:12:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cbb400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb406d3b0 con 0x7f1cb41555c0 2024-04-25T00:12:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cb8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb406e4c0 0x7f1cb4144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cb8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb406e4c0 0x7f1cb4144640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44666/0 (socket says 172.21.15.83:44666) 2024-04-25T00:12:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cb8a00640 1 -- 172.21.15.83:0/1723509697 learned_addr learned my addr 172.21.15.83:0/1723509697 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cabe00640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb41551e0 0x7f1cb4142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cb9400640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cb41555c0 0x7f1cb4148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cb8a00640 1 -- 172.21.15.83:0/1723509697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cb41555c0 msgr2=0x7f1cb4148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cb8a00640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cb41555c0 0x7f1cb4148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cb8a00640 1 -- 172.21.15.83:0/1723509697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb41551e0 msgr2=0x7f1cb4142e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cb8a00640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb41551e0 0x7f1cb4142e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cb8a00640 1 -- 172.21.15.83:0/1723509697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb4143880 con 0x7f1cb406e4c0 2024-04-25T00:12:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.618+0000 7f1cabe00640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb41551e0 0x7f1cb4142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.622+0000 7f1cb8a00640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb406e4c0 0x7f1cb4144640 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40ae040 tx=0x7f1ca401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.622+0000 7f1cb1600640 1 -- 172.21.15.83:0/1723509697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40b0050 con 0x7f1cb406e4c0 2024-04-25T00:12:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.622+0000 7f1cb1600640 1 -- 172.21.15.83:0/1723509697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca40bc070 con 0x7f1cb406e4c0 2024-04-25T00:12:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.622+0000 7f1cb1600640 1 -- 172.21.15.83:0/1723509697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40b5a70 con 0x7f1cb406e4c0 2024-04-25T00:12:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.622+0000 7f1cbb400640 1 -- 172.21.15.83:0/1723509697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb4143b60 con 0x7f1cb406e4c0 2024-04-25T00:12:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.622+0000 7f1cbb400640 1 -- 172.21.15.83:0/1723509697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb41433c0 con 0x7f1cb406e4c0 2024-04-25T00:12:36.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.622+0000 7f1cbb400640 1 -- 172.21.15.83:0/1723509697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb404f510 con 0x7f1cb406e4c0 2024-04-25T00:12:36.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.622+0000 7f1cb1600640 1 -- 172.21.15.83:0/1723509697 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1ca40ab480 con 0x7f1cb406e4c0 2024-04-25T00:12:36.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.622+0000 7f1cb1600640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1c900678f0 0x7f1c90069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.622+0000 7f1cb1600640 1 -- 172.21.15.83:0/1723509697 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1ca41206f0 con 0x7f1cb406e4c0 2024-04-25T00:12:36.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.622+0000 7f1cabe00640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1c900678f0 0x7f1c90069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.626+0000 7f1cabe00640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1c900678f0 0x7f1c90069db0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1cb415c2e0 tx=0x7f1c9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:36.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.626+0000 7f1cb1600640 1 -- 172.21.15.83:0/1723509697 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1ca4016090 con 0x7f1cb406e4c0 2024-04-25T00:12:36.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.782+0000 7f1cbb400640 1 -- 172.21.15.83:0/1723509697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1cb406ce30 con 0x7f1cb406e4c0 2024-04-25T00:12:36.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.782+0000 7f1cb1600640 1 -- 172.21.15.83:0/1723509697 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f1ca40f11b0 con 0x7f1cb406e4c0 2024-04-25T00:12:36.784 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:12:36.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.786+0000 7f1cbb400640 1 -- 172.21.15.83:0/1723509697 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1c900678f0 msgr2=0x7f1c90069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.786+0000 7f1cbb400640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1c900678f0 0x7f1c90069db0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1cb415c2e0 tx=0x7f1c9c009040 comp rx=0 tx=0).stop 2024-04-25T00:12:36.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.786+0000 7f1cbb400640 1 -- 172.21.15.83:0/1723509697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb406e4c0 msgr2=0x7f1cb4144640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.786+0000 7f1cbb400640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb406e4c0 0x7f1cb4144640 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40ae040 tx=0x7f1ca401ace0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.786+0000 7f1cbb400640 1 -- 172.21.15.83:0/1723509697 shutdown_connections 2024-04-25T00:12:36.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.786+0000 7f1cbb400640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1c900678f0 0x7f1c90069db0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.786+0000 7f1cbb400640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cb41555c0 0x7f1cb4148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.786+0000 7f1cbb400640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cb41551e0 0x7f1cb4142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.786+0000 7f1cbb400640 1 --2- 172.21.15.83:0/1723509697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb406e4c0 0x7f1cb4144640 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.786+0000 7f1cbb400640 1 -- 172.21.15.83:0/1723509697 >> 172.21.15.83:0/1723509697 conn(0x7f1cb40b1830 msgr2=0x7f1cb41496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:36.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.786+0000 7f1cbb400640 1 -- 172.21.15.83:0/1723509697 shutdown_connections 2024-04-25T00:12:36.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.786+0000 7f1cbb400640 1 -- 172.21.15.83:0/1723509697 wait complete. 2024-04-25T00:12:36.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:12:36.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f69800640 1 Processor -- start 2024-04-25T00:12:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f69800640 1 -- start start 2024-04-25T00:12:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f69800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6406ebd0 0x7f5f6415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f69800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f6415c560 con 0x7f5f641551e0 2024-04-25T00:12:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f69800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f64072460 con 0x7f5f641555c0 2024-04-25T00:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f69800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6415c7d0 con 0x7f5f6406ebd0 2024-04-25T00:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f62a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5f641551e0 0x7f5f64071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57252/0 (socket says 172.21.15.83:57252) 2024-04-25T00:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f62a00640 1 -- 172.21.15.83:0/159326973 learned_addr learned my addr 172.21.15.83:0/159326973 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f63400640 1 --2- 172.21.15.83:0/159326973 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6406ebd0 0x7f5f6415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1232515903 0 0) 0x7f5f6415c560 con 0x7f5f641551e0 2024-04-25T00:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f48003530 con 0x7f5f641551e0 2024-04-25T00:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 539623792 0 0) 0x7f5f64072460 con 0x7f5f641555c0 2024-04-25T00:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f6415c560 con 0x7f5f641555c0 2024-04-25T00:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4048527204 0 0) 0x7f5f48003530 con 0x7f5f641551e0 2024-04-25T00:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6406ebd0 msgr2=0x7f5f6415bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 --2- 172.21.15.83:0/159326973 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6406ebd0 0x7f5f6415bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 >> v1:172.21.15.86:6789/0 conn(0x7f5f641555c0 legacy=0x7f5f6406e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6415cab0 con 0x7f5f641551e0 2024-04-25T00:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f5f54001680 con 0x7f5f641551e0 2024-04-25T00:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f63400640 1 --2- 172.21.15.83:0/159326973 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6406ebd0 0x7f5f6415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 >> v1:172.21.15.83:6789/0 conn(0x7f5f641551e0 legacy=0x7f5f64071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 --2- 172.21.15.83:0/159326973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f48003bf0 0x7f5f48003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 --2- 172.21.15.83:0/159326973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f48004810 0x7f5f48004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 --2- 172.21.15.83:0/159326973 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6406ebd0 0x7f5f480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6415c7d0 con 0x7f5f48003bf0 2024-04-25T00:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4800bb00 con 0x7f5f48004810 2024-04-25T00:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5f4800df10 con 0x7f5f6406ebd0 2024-04-25T00:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f62a00640 1 --2- 172.21.15.83:0/159326973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f48003bf0 0x7f5f48003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f63400640 1 --2- 172.21.15.83:0/159326973 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6406ebd0 0x7f5f480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f62000640 1 --2- 172.21.15.83:0/159326973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f48004810 0x7f5f48004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f62a00640 1 -- 172.21.15.83:0/159326973 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6406ebd0 msgr2=0x7f5f480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:36.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f62a00640 1 --2- 172.21.15.83:0/159326973 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6406ebd0 0x7f5f480051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f62a00640 1 -- 172.21.15.83:0/159326973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f48004810 msgr2=0x7f5f48004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:36.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f62a00640 1 --2- 172.21.15.83:0/159326973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f48004810 0x7f5f48004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f62a00640 1 -- 172.21.15.83:0/159326973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f48003530 con 0x7f5f48003bf0 2024-04-25T00:12:36.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.946+0000 7f5f62000640 1 --2- 172.21.15.83:0/159326973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f48004810 0x7f5f48004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:36.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.950+0000 7f5f62a00640 1 --2- 172.21.15.83:0/159326973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f48003bf0 0x7f5f48003ff0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400c110 tx=0x7f5f5401cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b25db2d181bbc48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:36.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.950+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54007c10 con 0x7f5f48003bf0 2024-04-25T00:12:36.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.950+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f540ac070 con 0x7f5f48003bf0 2024-04-25T00:12:36.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.950+0000 7f5f61600640 1 -- 172.21.15.83:0/159326973 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54005d20 con 0x7f5f48003bf0 2024-04-25T00:12:36.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.950+0000 7f5f69800640 1 -- 172.21.15.83:0/159326973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f48003bf0 msgr2=0x7f5f48003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.950+0000 7f5f69800640 1 --2- 172.21.15.83:0/159326973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f48003bf0 0x7f5f48003ff0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400c110 tx=0x7f5f5401cb60 comp rx=0 tx=0).stop 2024-04-25T00:12:36.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.950+0000 7f5f63400640 1 -- 172.21.15.83:0/159326973 reap_dead start 2024-04-25T00:12:36.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.950+0000 7f5f69800640 1 -- 172.21.15.83:0/159326973 shutdown_connections 2024-04-25T00:12:36.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.950+0000 7f5f69800640 1 -- 172.21.15.83:0/159326973 >> 172.21.15.83:0/159326973 conn(0x7f5f640b1830 msgr2=0x7f5f640b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:36.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.950+0000 7f5f69800640 1 -- 172.21.15.83:0/159326973 shutdown_connections 2024-04-25T00:12:36.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.950+0000 7f5f69800640 1 -- 172.21.15.83:0/159326973 wait complete. 2024-04-25T00:12:36.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f69800640 1 Processor -- start 2024-04-25T00:12:36.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f69800640 1 -- start start 2024-04-25T00:12:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f69800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f48003bf0 0x7f5f64163500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f69800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f640cd550 0x7f5f640cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f69800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6414f7b0 0x7f5f6414fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f69800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6406df90 con 0x7f5f640cd550 2024-04-25T00:12:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f69800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6406de10 con 0x7f5f48003bf0 2024-04-25T00:12:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f69800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6406e110 con 0x7f5f6414f7b0 2024-04-25T00:12:36.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f62000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f640cd550 0x7f5f640cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f62000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f640cd550 0x7f5f640cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44680/0 (socket says 172.21.15.83:44680) 2024-04-25T00:12:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f62000640 1 -- 172.21.15.83:0/2174153337 learned_addr learned my addr 172.21.15.83:0/2174153337 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f63400640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6414f7b0 0x7f5f6414fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f62a00640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f48003bf0 0x7f5f64163500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f62000640 1 -- 172.21.15.83:0/2174153337 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6414f7b0 msgr2=0x7f5f6414fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f62000640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6414f7b0 0x7f5f6414fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f62000640 1 -- 172.21.15.83:0/2174153337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f48003bf0 msgr2=0x7f5f64163500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f62000640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f48003bf0 0x7f5f64163500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f62000640 1 -- 172.21.15.83:0/2174153337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f64150550 con 0x7f5f640cd550 2024-04-25T00:12:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f62a00640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f48003bf0 0x7f5f64163500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f62000640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f640cd550 0x7f5f640cd9d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000e550 tx=0x7f5f5000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f5aa00640 1 -- 172.21.15.83:0/2174153337 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f500052e0 con 0x7f5f640cd550 2024-04-25T00:12:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f5aa00640 1 -- 172.21.15.83:0/2174153337 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f50007020 con 0x7f5f640cd550 2024-04-25T00:12:36.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f5aa00640 1 -- 172.21.15.83:0/2174153337 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50005480 con 0x7f5f640cd550 2024-04-25T00:12:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f69800640 1 -- 172.21.15.83:0/2174153337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f64150830 con 0x7f5f640cd550 2024-04-25T00:12:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.954+0000 7f5f69800640 1 -- 172.21.15.83:0/2174153337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f64150df0 con 0x7f5f640cd550 2024-04-25T00:12:36.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.958+0000 7f5f69800640 1 -- 172.21.15.83:0/2174153337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f6404f510 con 0x7f5f640cd550 2024-04-25T00:12:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.958+0000 7f5f5aa00640 1 -- 172.21.15.83:0/2174153337 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5f5001a020 con 0x7f5f640cd550 2024-04-25T00:12:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.958+0000 7f5f5aa00640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f440678c0 0x7f5f44069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.958+0000 7f5f5aa00640 1 -- 172.21.15.83:0/2174153337 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f5f50085490 con 0x7f5f640cd550 2024-04-25T00:12:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.958+0000 7f5f62a00640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f440678c0 0x7f5f44069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.958+0000 7f5f62a00640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f440678c0 0x7f5f44069d80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5f540ab000 tx=0x7f5f540a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:36.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:36.962+0000 7f5f5aa00640 1 -- 172.21.15.83:0/2174153337 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5f50055ed0 con 0x7f5f640cd550 2024-04-25T00:12:37.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.118+0000 7f5f69800640 1 -- 172.21.15.83:0/2174153337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5f6406ce30 con 0x7f5f640cd550 2024-04-25T00:12:37.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.118+0000 7f5f5aa00640 1 -- 172.21.15.83:0/2174153337 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) ==== 104+0+11 (secure 0 0 0) 0x7f5f500557d0 con 0x7f5f640cd550 2024-04-25T00:12:37.118 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:12:37.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.118+0000 7f5f69800640 1 -- 172.21.15.83:0/2174153337 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f440678c0 msgr2=0x7f5f44069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.118+0000 7f5f69800640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f440678c0 0x7f5f44069d80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5f540ab000 tx=0x7f5f540a7000 comp rx=0 tx=0).stop 2024-04-25T00:12:37.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.118+0000 7f5f69800640 1 -- 172.21.15.83:0/2174153337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f640cd550 msgr2=0x7f5f640cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.118+0000 7f5f69800640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f640cd550 0x7f5f640cd9d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000e550 tx=0x7f5f5000ea10 comp rx=0 tx=0).stop 2024-04-25T00:12:37.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.118+0000 7f5f69800640 1 -- 172.21.15.83:0/2174153337 shutdown_connections 2024-04-25T00:12:37.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.118+0000 7f5f69800640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f440678c0 0x7f5f44069d80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.118+0000 7f5f69800640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6414f7b0 0x7f5f6414fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.118+0000 7f5f69800640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f640cd550 0x7f5f640cd9d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.118+0000 7f5f69800640 1 --2- 172.21.15.83:0/2174153337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f48003bf0 0x7f5f64163500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.118+0000 7f5f69800640 1 -- 172.21.15.83:0/2174153337 >> 172.21.15.83:0/2174153337 conn(0x7f5f640b1830 msgr2=0x7f5f640d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.118+0000 7f5f69800640 1 -- 172.21.15.83:0/2174153337 shutdown_connections 2024-04-25T00:12:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.118+0000 7f5f69800640 1 -- 172.21.15.83:0/2174153337 wait complete. 2024-04-25T00:12:37.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:12:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc057000640 1 Processor -- start 2024-04-25T00:12:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc057000640 1 -- start start 2024-04-25T00:12:37.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc057000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc05006ebd0 0x7fc05015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc057000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc05015c560 con 0x7fc0501555c0 2024-04-25T00:12:37.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc057000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc050072460 con 0x7fc0501551e0 2024-04-25T00:12:37.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc057000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc05015c7d0 con 0x7fc05006ebd0 2024-04-25T00:12:37.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04f400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc0501555c0 0x7fc05006e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57260/0 (socket says 172.21.15.83:57260) 2024-04-25T00:12:37.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04f400640 1 -- 172.21.15.83:0/3731060068 learned_addr learned my addr 172.21.15.83:0/3731060068 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:37.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc055000640 1 --2- 172.21.15.83:0/3731060068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc05006ebd0 0x7fc05015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3092180491 0 0) 0x7fc05015c560 con 0x7fc0501555c0 2024-04-25T00:12:37.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc034005740 con 0x7fc0501555c0 2024-04-25T00:12:37.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2749870293 0 0) 0x7fc050072460 con 0x7fc0501551e0 2024-04-25T00:12:37.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc05015c560 con 0x7fc0501551e0 2024-04-25T00:12:37.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3307854893 0 0) 0x7fc034005740 con 0x7fc0501555c0 2024-04-25T00:12:37.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc05006ebd0 msgr2=0x7fc05015bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:37.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 --2- 172.21.15.83:0/3731060068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc05006ebd0 0x7fc05015bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 >> v1:172.21.15.86:6789/0 conn(0x7fc0501551e0 legacy=0x7fc050071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05015cab0 con 0x7fc0501555c0 2024-04-25T00:12:37.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fc03c001680 con 0x7fc0501555c0 2024-04-25T00:12:37.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 >> v1:172.21.15.83:6789/0 conn(0x7fc0501555c0 legacy=0x7fc05006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 --2- 172.21.15.83:0/3731060068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc034005e00 0x7fc034006200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 --2- 172.21.15.83:0/3731060068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034006a20 0x7fc034006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 --2- 172.21.15.83:0/3731060068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc05006ebd0 0x7fc034007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc05015c7d0 con 0x7fc034006a20 2024-04-25T00:12:37.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc03400dd40 con 0x7fc034005e00 2024-04-25T00:12:37.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.278+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc03400df40 con 0x7fc05006ebd0 2024-04-25T00:12:37.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.282+0000 7fc04f400640 1 --2- 172.21.15.83:0/3731060068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034006a20 0x7fc034006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.282+0000 7fc04fe00640 1 --2- 172.21.15.83:0/3731060068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc034005e00 0x7fc034006200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.282+0000 7fc055000640 1 --2- 172.21.15.83:0/3731060068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc05006ebd0 0x7fc034007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.282+0000 7fc04f400640 1 -- 172.21.15.83:0/3731060068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc05006ebd0 msgr2=0x7fc034007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:37.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.282+0000 7fc04f400640 1 --2- 172.21.15.83:0/3731060068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc05006ebd0 0x7fc034007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.282+0000 7fc04f400640 1 -- 172.21.15.83:0/3731060068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc034005e00 msgr2=0x7fc034006200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:37.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.282+0000 7fc04f400640 1 --2- 172.21.15.83:0/3731060068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc034005e00 0x7fc034006200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.282+0000 7fc04f400640 1 -- 172.21.15.83:0/3731060068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc034005740 con 0x7fc034006a20 2024-04-25T00:12:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.282+0000 7fc04f400640 1 --2- 172.21.15.83:0/3731060068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034006a20 0x7fc034006ea0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc03c007fa0 tx=0x7fc03c01cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5304e789838b71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.282+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c007bf0 con 0x7fc034006a20 2024-04-25T00:12:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.282+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03c0a9070 con 0x7fc034006a20 2024-04-25T00:12:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.282+0000 7fc04ea00640 1 -- 172.21.15.83:0/3731060068 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c005d40 con 0x7fc034006a20 2024-04-25T00:12:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.282+0000 7fc057000640 1 -- 172.21.15.83:0/3731060068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034006a20 msgr2=0x7fc034006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.282+0000 7fc057000640 1 --2- 172.21.15.83:0/3731060068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034006a20 0x7fc034006ea0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc03c007fa0 tx=0x7fc03c01cb00 comp rx=0 tx=0).stop 2024-04-25T00:12:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc055000640 1 -- 172.21.15.83:0/3731060068 reap_dead start 2024-04-25T00:12:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc057000640 1 -- 172.21.15.83:0/3731060068 shutdown_connections 2024-04-25T00:12:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc057000640 1 -- 172.21.15.83:0/3731060068 >> 172.21.15.83:0/3731060068 conn(0x7fc0500b1830 msgr2=0x7fc0500b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc057000640 1 -- 172.21.15.83:0/3731060068 shutdown_connections 2024-04-25T00:12:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc057000640 1 -- 172.21.15.83:0/3731060068 wait complete. 2024-04-25T00:12:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc057000640 1 Processor -- start 2024-04-25T00:12:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc057000640 1 -- start start 2024-04-25T00:12:37.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc057000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034006a20 0x7fc05013ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc057000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc05013f590 0x7fc050141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc057000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc050150200 0x7fc0500b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc057000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc05015e160 con 0x7fc034006a20 2024-04-25T00:12:37.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc057000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc05015dfe0 con 0x7fc050150200 2024-04-25T00:12:37.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc057000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc05015e2e0 con 0x7fc05013f590 2024-04-25T00:12:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034006a20 0x7fc05013ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034006a20 0x7fc05013ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44692/0 (socket says 172.21.15.83:44692) 2024-04-25T00:12:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc04fe00640 1 -- 172.21.15.83:0/1595362850 learned_addr learned my addr 172.21.15.83:0/1595362850 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc04f400640 1 --2- 172.21.15.83:0/1595362850 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc05013f590 0x7fc050141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc055000640 1 --2- 172.21.15.83:0/1595362850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc050150200 0x7fc0500b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc04fe00640 1 -- 172.21.15.83:0/1595362850 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc05013f590 msgr2=0x7fc050141fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc04fe00640 1 --2- 172.21.15.83:0/1595362850 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc05013f590 0x7fc050141fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc04fe00640 1 -- 172.21.15.83:0/1595362850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc050150200 msgr2=0x7fc0500b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc04fe00640 1 --2- 172.21.15.83:0/1595362850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc050150200 0x7fc0500b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc04fe00640 1 -- 172.21.15.83:0/1595362850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05013f270 con 0x7fc034006a20 2024-04-25T00:12:37.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.286+0000 7fc04fe00640 1 --2- 172.21.15.83:0/1595362850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034006a20 0x7fc05013ed30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc040012800 tx=0x7fc040012cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:37.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.290+0000 7fc047e00640 1 -- 172.21.15.83:0/1595362850 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040010bd0 con 0x7fc034006a20 2024-04-25T00:12:37.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.290+0000 7fc047e00640 1 -- 172.21.15.83:0/1595362850 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc04001b070 con 0x7fc034006a20 2024-04-25T00:12:37.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.290+0000 7fc047e00640 1 -- 172.21.15.83:0/1595362850 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040016480 con 0x7fc034006a20 2024-04-25T00:12:37.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.290+0000 7fc057000640 1 -- 172.21.15.83:0/1595362850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc050142520 con 0x7fc034006a20 2024-04-25T00:12:37.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.290+0000 7fc057000640 1 -- 172.21.15.83:0/1595362850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc050152570 con 0x7fc034006a20 2024-04-25T00:12:37.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.290+0000 7fc057000640 1 -- 172.21.15.83:0/1595362850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc05004f510 con 0x7fc034006a20 2024-04-25T00:12:37.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.290+0000 7fc047e00640 1 -- 172.21.15.83:0/1595362850 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc040016620 con 0x7fc034006a20 2024-04-25T00:12:37.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.290+0000 7fc047e00640 1 --2- 172.21.15.83:0/1595362850 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc0300678c0 0x7fc030069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.290+0000 7fc047e00640 1 -- 172.21.15.83:0/1595362850 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fc040089480 con 0x7fc034006a20 2024-04-25T00:12:37.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.290+0000 7fc04f400640 1 --2- 172.21.15.83:0/1595362850 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc0300678c0 0x7fc030069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.294+0000 7fc04f400640 1 --2- 172.21.15.83:0/1595362850 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc0300678c0 0x7fc030069d80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0ab000 tx=0x7fc03c0b1040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:37.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.294+0000 7fc047e00640 1 -- 172.21.15.83:0/1595362850 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc0400597d0 con 0x7fc034006a20 2024-04-25T00:12:37.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.450+0000 7fc057000640 1 -- 172.21.15.83:0/1595362850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc05006ce30 con 0x7fc034006a20 2024-04-25T00:12:37.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.450+0000 7fc047e00640 1 -- 172.21.15.83:0/1595362850 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) ==== 100+0+11 (secure 0 0 0) 0x7fc04005dad0 con 0x7fc034006a20 2024-04-25T00:12:37.452 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:12:37.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.454+0000 7fc057000640 1 -- 172.21.15.83:0/1595362850 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc0300678c0 msgr2=0x7fc030069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.454+0000 7fc057000640 1 --2- 172.21.15.83:0/1595362850 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc0300678c0 0x7fc030069d80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0ab000 tx=0x7fc03c0b1040 comp rx=0 tx=0).stop 2024-04-25T00:12:37.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.454+0000 7fc057000640 1 -- 172.21.15.83:0/1595362850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034006a20 msgr2=0x7fc05013ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.454+0000 7fc057000640 1 --2- 172.21.15.83:0/1595362850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034006a20 0x7fc05013ed30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc040012800 tx=0x7fc040012cc0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.454+0000 7fc057000640 1 -- 172.21.15.83:0/1595362850 shutdown_connections 2024-04-25T00:12:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.454+0000 7fc057000640 1 --2- 172.21.15.83:0/1595362850 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc0300678c0 0x7fc030069d80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.454+0000 7fc057000640 1 --2- 172.21.15.83:0/1595362850 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc05013f590 0x7fc050141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.454+0000 7fc057000640 1 --2- 172.21.15.83:0/1595362850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc050150200 0x7fc0500b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.454+0000 7fc057000640 1 --2- 172.21.15.83:0/1595362850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034006a20 0x7fc05013ed30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.454+0000 7fc057000640 1 -- 172.21.15.83:0/1595362850 >> 172.21.15.83:0/1595362850 conn(0x7fc0500b1830 msgr2=0x7fc05006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.454+0000 7fc057000640 1 -- 172.21.15.83:0/1595362850 shutdown_connections 2024-04-25T00:12:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.454+0000 7fc057000640 1 -- 172.21.15.83:0/1595362850 wait complete. 2024-04-25T00:12:37.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:12:37.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f9600a00640 1 Processor -- start 2024-04-25T00:12:37.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f9600a00640 1 -- start start 2024-04-25T00:12:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f9600a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f95fc140340 0x7f95fc06e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f9600a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95fc06eac0 0x7f95fc15b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f9600a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95fc141280 con 0x7f95fc06e1a0 2024-04-25T00:12:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f9600a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f95fc1407c0 con 0x7f95fc140340 2024-04-25T00:12:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f9600a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f95fc15b400 con 0x7f95fc06eac0 2024-04-25T00:12:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f95f9400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f95fc140340 0x7f95fc06e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f95f9400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f95fc140340 0x7f95fc06e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44704/0 (socket says 172.21.15.83:44704) 2024-04-25T00:12:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f95f9400640 1 -- 172.21.15.83:0/664155377 learned_addr learned my addr 172.21.15.83:0/664155377 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f95f9400640 1 -- 172.21.15.83:0/664155377 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95fc06eac0 msgr2=0x7f95fc15b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:37.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f95fa800640 1 --2- 172.21.15.83:0/664155377 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95fc06eac0 0x7f95fc15b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f95f9400640 1 --2- 172.21.15.83:0/664155377 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95fc06eac0 0x7f95fc15b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f95f9400640 1 -- 172.21.15.83:0/664155377 >> v1:172.21.15.86:6789/0 conn(0x7f95fc06e1a0 legacy=0x7f95fc15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:37.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f95f9400640 1 -- 172.21.15.83:0/664155377 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc1409e0 con 0x7f95fc140340 2024-04-25T00:12:37.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.610+0000 7f95fa800640 1 --2- 172.21.15.83:0/664155377 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95fc06eac0 0x7f95fc15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:37.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.614+0000 7f95f9400640 1 --2- 172.21.15.83:0/664155377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95fc140340 0x7f95fc06e580 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f95e800bb10 tx=0x7f95e801ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c94e7c695a643e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:37.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.614+0000 7f95f8a00640 1 -- 172.21.15.83:0/664155377 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80b4070 con 0x7f95fc140340 2024-04-25T00:12:37.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.614+0000 7f95f8a00640 1 -- 172.21.15.83:0/664155377 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e80a7960 con 0x7f95fc140340 2024-04-25T00:12:37.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.614+0000 7f95f8a00640 1 -- 172.21.15.83:0/664155377 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80a7d30 con 0x7f95fc140340 2024-04-25T00:12:37.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.614+0000 7f9600a00640 1 -- 172.21.15.83:0/664155377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95fc140340 msgr2=0x7f95fc06e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.614+0000 7f9600a00640 1 --2- 172.21.15.83:0/664155377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95fc140340 0x7f95fc06e580 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f95e800bb10 tx=0x7f95e801ead0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.614+0000 7f9600a00640 1 -- 172.21.15.83:0/664155377 shutdown_connections 2024-04-25T00:12:37.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.614+0000 7f9600a00640 1 --2- 172.21.15.83:0/664155377 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95fc06eac0 0x7f95fc15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.614+0000 7f9600a00640 1 --2- 172.21.15.83:0/664155377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95fc140340 0x7f95fc06e580 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.614+0000 7f9600a00640 1 -- 172.21.15.83:0/664155377 >> 172.21.15.83:0/664155377 conn(0x7f95fc0b2840 msgr2=0x7f95fc0b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:37.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.614+0000 7f9600a00640 1 -- 172.21.15.83:0/664155377 shutdown_connections 2024-04-25T00:12:37.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.614+0000 7f9600a00640 1 -- 172.21.15.83:0/664155377 wait complete. 2024-04-25T00:12:37.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.614+0000 7f9600a00640 1 Processor -- start 2024-04-25T00:12:37.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.614+0000 7f9600a00640 1 -- start start 2024-04-25T00:12:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f9600a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95fc06e1a0 0x7f95fc138fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f9600a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95fc06eac0 0x7f95fc0ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f9600a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95fc140340 0x7f95fc0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f9600a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc0ccca0 con 0x7f95fc06e1a0 2024-04-25T00:12:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f9600a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc0ccb20 con 0x7f95fc140340 2024-04-25T00:12:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f9600a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f95fc0cce20 con 0x7f95fc06eac0 2024-04-25T00:12:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95f9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95fc06e1a0 0x7f95fc138fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95f9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95fc06e1a0 0x7f95fc138fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44706/0 (socket says 172.21.15.83:44706) 2024-04-25T00:12:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95f9e00640 1 -- 172.21.15.83:0/2296938938 learned_addr learned my addr 172.21.15.83:0/2296938938 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95f9400640 1 --2- 172.21.15.83:0/2296938938 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95fc06eac0 0x7f95fc0ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95fa800640 1 --2- 172.21.15.83:0/2296938938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95fc140340 0x7f95fc0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95f9e00640 1 -- 172.21.15.83:0/2296938938 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95fc06eac0 msgr2=0x7f95fc0ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95f9e00640 1 --2- 172.21.15.83:0/2296938938 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95fc06eac0 0x7f95fc0ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95f9e00640 1 -- 172.21.15.83:0/2296938938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95fc140340 msgr2=0x7f95fc0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95f9e00640 1 --2- 172.21.15.83:0/2296938938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95fc140340 0x7f95fc0ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95f9e00640 1 -- 172.21.15.83:0/2296938938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc0cf270 con 0x7f95fc06e1a0 2024-04-25T00:12:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95f9e00640 1 --2- 172.21.15.83:0/2296938938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95fc06e1a0 0x7f95fc138fd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c450 tx=0x7f95ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95f2000640 1 -- 172.21.15.83:0/2296938938 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec012830 con 0x7f95fc06e1a0 2024-04-25T00:12:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95f2000640 1 -- 172.21.15.83:0/2296938938 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95ec017070 con 0x7f95fc06e1a0 2024-04-25T00:12:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95f2000640 1 -- 172.21.15.83:0/2296938938 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec012c30 con 0x7f95fc06e1a0 2024-04-25T00:12:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f9600a00640 1 -- 172.21.15.83:0/2296938938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95fc0cecb0 con 0x7f95fc06e1a0 2024-04-25T00:12:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f9600a00640 1 -- 172.21.15.83:0/2296938938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95fc136430 con 0x7f95fc06e1a0 2024-04-25T00:12:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f9600a00640 1 -- 172.21.15.83:0/2296938938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95fc073790 con 0x7f95fc06e1a0 2024-04-25T00:12:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.618+0000 7f95f2000640 1 -- 172.21.15.83:0/2296938938 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f95ec01c460 con 0x7f95fc06e1a0 2024-04-25T00:12:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.622+0000 7f95f2000640 1 --2- 172.21.15.83:0/2296938938 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95dc0678f0 0x7f95dc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.622+0000 7f95f2000640 1 -- 172.21.15.83:0/2296938938 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f95ec085c50 con 0x7f95fc06e1a0 2024-04-25T00:12:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.622+0000 7f95f9400640 1 --2- 172.21.15.83:0/2296938938 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95dc0678f0 0x7f95dc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.622+0000 7f95f9400640 1 --2- 172.21.15.83:0/2296938938 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95dc0678f0 0x7f95dc069db0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95e80b1000 tx=0x7f95e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:37.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.626+0000 7f95f2000640 1 -- 172.21.15.83:0/2296938938 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f95fc0cf270 con 0x7f95fc06e1a0 2024-04-25T00:12:37.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.822+0000 7f9600a00640 1 -- 172.21.15.83:0/2296938938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95fc0cdc50 con 0x7f95fc06e1a0 2024-04-25T00:12:37.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.822+0000 7f95f2000640 1 -- 172.21.15.83:0/2296938938 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7f95ec056560 con 0x7f95fc06e1a0 2024-04-25T00:12:37.822 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:37.823 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4277921554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":4277921554},{"type":"v1","addr":"172.21.15.86:6835","nonce":4277921554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/4145334103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":4145334103},{"type":"v1","addr":"172.21.15.86:6834","nonce":4145334103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3441801758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3441801758},{"type":"v1","addr":"172.21.15.83:6836","nonce":3441801758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:12:18.566138+0000","modified":"2024-04-25T00:12:21.572278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5328},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5328":{"gid":5328,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2704243724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2704243724},{"type":"v1","addr":"172.21.15.83:6837","nonce":2704243724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5328,"qdb_cluster":[5328]},"id":2}]} 2024-04-25T00:12:37.823 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 18 2024-04-25T00:12:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.822+0000 7f9600a00640 1 -- 172.21.15.83:0/2296938938 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95dc0678f0 msgr2=0x7f95dc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.822+0000 7f9600a00640 1 --2- 172.21.15.83:0/2296938938 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95dc0678f0 0x7f95dc069db0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95e80b1000 tx=0x7f95e8008040 comp rx=0 tx=0).stop 2024-04-25T00:12:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.822+0000 7f9600a00640 1 -- 172.21.15.83:0/2296938938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95fc06e1a0 msgr2=0x7f95fc138fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.822+0000 7f9600a00640 1 --2- 172.21.15.83:0/2296938938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95fc06e1a0 0x7f95fc138fd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c450 tx=0x7f95ec00c910 comp rx=0 tx=0).stop 2024-04-25T00:12:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.822+0000 7f9600a00640 1 -- 172.21.15.83:0/2296938938 shutdown_connections 2024-04-25T00:12:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.822+0000 7f9600a00640 1 --2- 172.21.15.83:0/2296938938 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95dc0678f0 0x7f95dc069db0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.822+0000 7f9600a00640 1 --2- 172.21.15.83:0/2296938938 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95fc06eac0 0x7f95fc0ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.822+0000 7f9600a00640 1 --2- 172.21.15.83:0/2296938938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95fc140340 0x7f95fc0ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.822+0000 7f9600a00640 1 --2- 172.21.15.83:0/2296938938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95fc06e1a0 0x7f95fc138fd0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.822+0000 7f9600a00640 1 -- 172.21.15.83:0/2296938938 >> 172.21.15.83:0/2296938938 conn(0x7f95fc0b2840 msgr2=0x7f95fc151950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.822+0000 7f9600a00640 1 -- 172.21.15.83:0/2296938938 shutdown_connections 2024-04-25T00:12:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.822+0000 7f9600a00640 1 -- 172.21.15.83:0/2296938938 wait complete. 2024-04-25T00:12:37.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:37.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d9400640 1 Processor -- start 2024-04-25T00:12:37.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d9400640 1 -- start start 2024-04-25T00:12:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d9400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff8d41599f0 0x7ff8d40cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d9400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8d415c340 0x7ff8d415c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff8d415ccc0 0x7ff8d415f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d9400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff8d4159dd0 con 0x7ff8d41599f0 2024-04-25T00:12:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d9400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff8d40ccc60 con 0x7ff8d415ccc0 2024-04-25T00:12:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d9400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff8d406dba0 con 0x7ff8d415c340 2024-04-25T00:12:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d2800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff8d41599f0 0x7ff8d40cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d2800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff8d41599f0 0x7ff8d40cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44716/0 (socket says 172.21.15.83:44716) 2024-04-25T00:12:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d2800640 1 -- 172.21.15.83:0/4134848925 learned_addr learned my addr 172.21.15.83:0/4134848925 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8c9e00640 1 --2- 172.21.15.83:0/4134848925 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8d415c340 0x7ff8d415c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d3200640 1 --2- 172.21.15.83:0/4134848925 >> v2:172.21.15.86:3300/0 conn(0x7ff8d415ccc0 0x7ff8d415f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d2800640 1 -- 172.21.15.83:0/4134848925 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8d415c340 msgr2=0x7ff8d415c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d2800640 1 --2- 172.21.15.83:0/4134848925 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8d415c340 0x7ff8d415c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d2800640 1 -- 172.21.15.83:0/4134848925 >> v2:172.21.15.86:3300/0 conn(0x7ff8d415ccc0 msgr2=0x7ff8d415f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d2800640 1 --2- 172.21.15.83:0/4134848925 >> v2:172.21.15.86:3300/0 conn(0x7ff8d415ccc0 0x7ff8d415f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.978+0000 7ff8d2800640 1 -- 172.21.15.83:0/4134848925 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d406de80 con 0x7ff8d41599f0 2024-04-25T00:12:37.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.982+0000 7ff8d2800640 1 --2- 172.21.15.83:0/4134848925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d41599f0 0x7ff8d40cc510 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009870 tx=0x7ff8c401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27e4c22b97bb5fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:37.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.982+0000 7ff8d1e00640 1 -- 172.21.15.83:0/4134848925 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c40b2070 con 0x7ff8d41599f0 2024-04-25T00:12:37.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.982+0000 7ff8d1e00640 1 -- 172.21.15.83:0/4134848925 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8c40a9070 con 0x7ff8d41599f0 2024-04-25T00:12:37.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.982+0000 7ff8d1e00640 1 -- 172.21.15.83:0/4134848925 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c40ad880 con 0x7ff8d41599f0 2024-04-25T00:12:37.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.982+0000 7ff8d9400640 1 -- 172.21.15.83:0/4134848925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d41599f0 msgr2=0x7ff8d40cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.982+0000 7ff8d9400640 1 --2- 172.21.15.83:0/4134848925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d41599f0 0x7ff8d40cc510 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009870 tx=0x7ff8c401c490 comp rx=0 tx=0).stop 2024-04-25T00:12:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.982+0000 7ff8d9400640 1 -- 172.21.15.83:0/4134848925 shutdown_connections 2024-04-25T00:12:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.982+0000 7ff8d9400640 1 --2- 172.21.15.83:0/4134848925 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8d415c340 0x7ff8d415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.982+0000 7ff8d9400640 1 --2- 172.21.15.83:0/4134848925 >> v2:172.21.15.86:3300/0 conn(0x7ff8d415ccc0 0x7ff8d415f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.982+0000 7ff8d9400640 1 --2- 172.21.15.83:0/4134848925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d41599f0 0x7ff8d40cc510 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.982+0000 7ff8d9400640 1 -- 172.21.15.83:0/4134848925 >> 172.21.15.83:0/4134848925 conn(0x7ff8d40b1830 msgr2=0x7ff8d40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.982+0000 7ff8d9400640 1 -- 172.21.15.83:0/4134848925 shutdown_connections 2024-04-25T00:12:37.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.982+0000 7ff8d9400640 1 -- 172.21.15.83:0/4134848925 wait complete. 2024-04-25T00:12:37.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8d9400640 1 Processor -- start 2024-04-25T00:12:37.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8d9400640 1 -- start start 2024-04-25T00:12:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8d9400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8d41599f0 0x7ff8d4152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8d9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d415c340 0x7ff8d415b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8d9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d415ccc0 0x7ff8d415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8d9400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d40ce860 con 0x7ff8d415c340 2024-04-25T00:12:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8d9400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d40ce6e0 con 0x7ff8d415ccc0 2024-04-25T00:12:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8d9400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff8d40ce9e0 con 0x7ff8d41599f0 2024-04-25T00:12:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8c9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d415c340 0x7ff8d415b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8c9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d415c340 0x7ff8d415b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44728/0 (socket says 172.21.15.83:44728) 2024-04-25T00:12:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8c9e00640 1 -- 172.21.15.83:0/1961027372 learned_addr learned my addr 172.21.15.83:0/1961027372 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8d2800640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8d41599f0 0x7ff8d4152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8d3200640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d415ccc0 0x7ff8d415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8c9e00640 1 -- 172.21.15.83:0/1961027372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8d41599f0 msgr2=0x7ff8d4152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8c9e00640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8d41599f0 0x7ff8d4152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8c9e00640 1 -- 172.21.15.83:0/1961027372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d415ccc0 msgr2=0x7ff8d415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8c9e00640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d415ccc0 0x7ff8d415b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8c9e00640 1 -- 172.21.15.83:0/1961027372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d415bce0 con 0x7ff8d415c340 2024-04-25T00:12:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8d3200640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d415ccc0 0x7ff8d415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8c9e00640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d415c340 0x7ff8d415b020 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00d800 tx=0x7ff8bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8cb400640 1 -- 172.21.15.83:0/1961027372 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc018020 con 0x7ff8d415c340 2024-04-25T00:12:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8cb400640 1 -- 172.21.15.83:0/1961027372 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8bc0096c0 con 0x7ff8d415c340 2024-04-25T00:12:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8cb400640 1 -- 172.21.15.83:0/1961027372 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc009ab0 con 0x7ff8d415c340 2024-04-25T00:12:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8d9400640 1 -- 172.21.15.83:0/1961027372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d415d730 con 0x7ff8d415c340 2024-04-25T00:12:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.986+0000 7ff8d9400640 1 -- 172.21.15.83:0/1961027372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d4152f80 con 0x7ff8d415c340 2024-04-25T00:12:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.990+0000 7ff8d9400640 1 -- 172.21.15.83:0/1961027372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8d40737f0 con 0x7ff8d415c340 2024-04-25T00:12:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.990+0000 7ff8cb400640 1 -- 172.21.15.83:0/1961027372 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff8bc01b020 con 0x7ff8d415c340 2024-04-25T00:12:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.990+0000 7ff8cb400640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff8b00678c0 0x7ff8b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.990+0000 7ff8cb400640 1 -- 172.21.15.83:0/1961027372 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7ff8bc0863b0 con 0x7ff8d415c340 2024-04-25T00:12:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.990+0000 7ff8d2800640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff8b00678c0 0x7ff8b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.990+0000 7ff8d2800640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff8b00678c0 0x7ff8b0069d80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40af000 tx=0x7ff8c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:37.994+0000 7ff8cb400640 1 -- 172.21.15.83:0/1961027372 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff8bc022020 con 0x7ff8d415c340 2024-04-25T00:12:38.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.150+0000 7ff8d9400640 1 -- 172.21.15.83:0/1961027372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8d404f5f0 con 0x7ff8d415c340 2024-04-25T00:12:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.154+0000 7ff8cb400640 1 -- 172.21.15.83:0/1961027372 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16858 (secure 0 0 0) 0x7ff8bc056d50 con 0x7ff8d415c340 2024-04-25T00:12:38.154 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:38.154 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:20.181255+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:17.079304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:17.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:38.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.154+0000 7ff8d9400640 1 -- 172.21.15.83:0/1961027372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff8b00678c0 msgr2=0x7ff8b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.154+0000 7ff8d9400640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff8b00678c0 0x7ff8b0069d80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40af000 tx=0x7ff8c4008040 comp rx=0 tx=0).stop 2024-04-25T00:12:38.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.154+0000 7ff8d9400640 1 -- 172.21.15.83:0/1961027372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d415c340 msgr2=0x7ff8d415b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.154+0000 7ff8d9400640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d415c340 0x7ff8d415b020 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00d800 tx=0x7ff8bc00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.154+0000 7ff8d9400640 1 -- 172.21.15.83:0/1961027372 shutdown_connections 2024-04-25T00:12:38.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.154+0000 7ff8d9400640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff8b00678c0 0x7ff8b0069d80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.154+0000 7ff8d9400640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d415ccc0 0x7ff8d415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.154+0000 7ff8d9400640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d415c340 0x7ff8d415b020 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.154+0000 7ff8d9400640 1 --2- 172.21.15.83:0/1961027372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8d41599f0 0x7ff8d4152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.154+0000 7ff8d9400640 1 -- 172.21.15.83:0/1961027372 >> 172.21.15.83:0/1961027372 conn(0x7ff8d40b1830 msgr2=0x7ff8d41659c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:38.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.154+0000 7ff8d9400640 1 -- 172.21.15.83:0/1961027372 shutdown_connections 2024-04-25T00:12:38.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.154+0000 7ff8d9400640 1 -- 172.21.15.83:0/1961027372 wait complete. 2024-04-25T00:12:38.175 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:12:38.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:12:38.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.310+0000 7fe57a800640 1 Processor -- start 2024-04-25T00:12:38.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.310+0000 7fe57a800640 1 -- start start 2024-04-25T00:12:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.314+0000 7fe57a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe5741599f0 0x7fe5740cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.314+0000 7fe57a800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe57415c340 0x7fe57415c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.314+0000 7fe57a800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5740ccc60 con 0x7fe57415ccf0 2024-04-25T00:12:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.314+0000 7fe57a800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe574159dd0 con 0x7fe57415c340 2024-04-25T00:12:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.314+0000 7fe57a800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe57406dba0 con 0x7fe5741599f0 2024-04-25T00:12:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.314+0000 7fe573e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe57415ccf0 0x7fe57415f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57262/0 (socket says 172.21.15.83:57262) 2024-04-25T00:12:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.314+0000 7fe573e00640 1 -- 172.21.15.83:0/2239158532 learned_addr learned my addr 172.21.15.83:0/2239158532 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.314+0000 7fe572a00640 1 --2- 172.21.15.83:0/2239158532 >> v2:172.21.15.83:3300/0 conn(0x7fe57415c340 0x7fe57415c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.314+0000 7fe573400640 1 --2- 172.21.15.83:0/2239158532 >> v2:172.21.15.86:3300/0 conn(0x7fe5741599f0 0x7fe5740cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:38.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.314+0000 7fe572a00640 1 -- 172.21.15.83:0/2239158532 >> v2:172.21.15.86:3300/0 conn(0x7fe5741599f0 msgr2=0x7fe5740cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:38.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.314+0000 7fe572a00640 1 --2- 172.21.15.83:0/2239158532 >> v2:172.21.15.86:3300/0 conn(0x7fe5741599f0 0x7fe5740cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.314+0000 7fe572a00640 1 -- 172.21.15.83:0/2239158532 >> v1:172.21.15.83:6789/0 conn(0x7fe57415ccf0 legacy=0x7fe57415f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.314+0000 7fe572000640 1 -- 172.21.15.83:0/2239158532 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 133891906 0 0) 0x7fe56c0014a0 con 0x7fe57415ccf0 2024-04-25T00:12:38.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.314+0000 7fe572a00640 1 -- 172.21.15.83:0/2239158532 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57406de80 con 0x7fe57415c340 2024-04-25T00:12:38.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe572a00640 1 --2- 172.21.15.83:0/2239158532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe57415c340 0x7fe57415c7b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe5600097c0 tx=0x7fe56001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7519dcb4156ba864 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:38.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe572000640 1 -- 172.21.15.83:0/2239158532 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5600b0070 con 0x7fe57415c340 2024-04-25T00:12:38.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe572000640 1 -- 172.21.15.83:0/2239158532 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5600ab600 con 0x7fe57415c340 2024-04-25T00:12:38.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe572000640 1 -- 172.21.15.83:0/2239158532 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5600ab9d0 con 0x7fe57415c340 2024-04-25T00:12:38.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 -- 172.21.15.83:0/2239158532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe57415c340 msgr2=0x7fe57415c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 --2- 172.21.15.83:0/2239158532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe57415c340 0x7fe57415c7b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe5600097c0 tx=0x7fe56001c3d0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 -- 172.21.15.83:0/2239158532 shutdown_connections 2024-04-25T00:12:38.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 --2- 172.21.15.83:0/2239158532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe57415c340 0x7fe57415c7b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 --2- 172.21.15.83:0/2239158532 >> v2:172.21.15.86:3300/0 conn(0x7fe5741599f0 0x7fe5740cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 -- 172.21.15.83:0/2239158532 >> 172.21.15.83:0/2239158532 conn(0x7fe5740b1830 msgr2=0x7fe5740b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:38.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 -- 172.21.15.83:0/2239158532 shutdown_connections 2024-04-25T00:12:38.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 -- 172.21.15.83:0/2239158532 wait complete. 2024-04-25T00:12:38.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 Processor -- start 2024-04-25T00:12:38.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 -- start start 2024-04-25T00:12:38.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5741599f0 0x7fe57413c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:38.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe57415c340 0x7fe57413e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:38.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe57415ccf0 0x7fe5740b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:38.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe5740cf870 con 0x7fe57415ccf0 2024-04-25T00:12:38.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5740cf6f0 con 0x7fe5741599f0 2024-04-25T00:12:38.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.318+0000 7fe57a800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe574168cd0 con 0x7fe57415c340 2024-04-25T00:12:38.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe573e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe57415ccf0 0x7fe5740b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:38.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe573e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe57415ccf0 0x7fe5740b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44740/0 (socket says 172.21.15.83:44740) 2024-04-25T00:12:38.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe573e00640 1 -- 172.21.15.83:0/3138685651 learned_addr learned my addr 172.21.15.83:0/3138685651 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:38.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe573e00640 1 -- 172.21.15.83:0/3138685651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe57415c340 msgr2=0x7fe57413e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe573400640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5741599f0 0x7fe57413c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:38.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe572a00640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe57415c340 0x7fe57413e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:38.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe573e00640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe57415c340 0x7fe57413e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe573e00640 1 -- 172.21.15.83:0/3138685651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5741599f0 msgr2=0x7fe57413c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe573e00640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5741599f0 0x7fe57413c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe573e00640 1 -- 172.21.15.83:0/3138685651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe574068440 con 0x7fe57415ccf0 2024-04-25T00:12:38.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe573400640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5741599f0 0x7fe57413c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:38.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe572a00640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe57415c340 0x7fe57413e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:38.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe573e00640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe57415ccf0 0x7fe5740b10d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00c700 tx=0x7fe56c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:38.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe56b400640 1 -- 172.21.15.83:0/3138685651 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c018070 con 0x7fe57415ccf0 2024-04-25T00:12:38.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe56b400640 1 -- 172.21.15.83:0/3138685651 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56c00ce10 con 0x7fe57415ccf0 2024-04-25T00:12:38.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe56b400640 1 -- 172.21.15.83:0/3138685651 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00a880 con 0x7fe57415ccf0 2024-04-25T00:12:38.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe57a800640 1 -- 172.21.15.83:0/3138685651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57413e7c0 con 0x7fe57415ccf0 2024-04-25T00:12:38.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe57a800640 1 -- 172.21.15.83:0/3138685651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57413ed80 con 0x7fe57415ccf0 2024-04-25T00:12:38.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe57a800640 1 -- 172.21.15.83:0/3138685651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5740737f0 con 0x7fe57415ccf0 2024-04-25T00:12:38.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe56b400640 1 -- 172.21.15.83:0/3138685651 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe56c01b070 con 0x7fe57415ccf0 2024-04-25T00:12:38.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe56b400640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe5540678c0 0x7fe554069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:38.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.322+0000 7fe56b400640 1 -- 172.21.15.83:0/3138685651 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe56c085510 con 0x7fe57415ccf0 2024-04-25T00:12:38.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.326+0000 7fe573400640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe5540678c0 0x7fe554069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:38.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.326+0000 7fe573400640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe5540678c0 0x7fe554069d80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe564009ae0 tx=0x7fe564008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:38.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.326+0000 7fe56b400640 1 -- 172.21.15.83:0/3138685651 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe5740ccc60 con 0x7fe57415ccf0 2024-04-25T00:12:38.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.518+0000 7fe57a800640 1 -- 172.21.15.83:0/3138685651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe57404f5f0 con 0x7fe57415ccf0 2024-04-25T00:12:38.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.518+0000 7fe56b400640 1 -- 172.21.15.83:0/3138685651 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7fe56c055d90 con 0x7fe57415ccf0 2024-04-25T00:12:38.519 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:38.519 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4277921554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":4277921554},{"type":"v1","addr":"172.21.15.86:6835","nonce":4277921554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/4145334103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":4145334103},{"type":"v1","addr":"172.21.15.86:6834","nonce":4145334103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3441801758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3441801758},{"type":"v1","addr":"172.21.15.83:6836","nonce":3441801758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:12:18.566138+0000","modified":"2024-04-25T00:12:21.572278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5328},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5328":{"gid":5328,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2704243724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2704243724},{"type":"v1","addr":"172.21.15.83:6837","nonce":2704243724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5328,"qdb_cluster":[5328]},"id":2}]} 2024-04-25T00:12:38.519 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 18 2024-04-25T00:12:38.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.518+0000 7fe57a800640 1 -- 172.21.15.83:0/3138685651 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe5540678c0 msgr2=0x7fe554069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.518+0000 7fe57a800640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe5540678c0 0x7fe554069d80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe564009ae0 tx=0x7fe564008040 comp rx=0 tx=0).stop 2024-04-25T00:12:38.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.518+0000 7fe57a800640 1 -- 172.21.15.83:0/3138685651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe57415ccf0 msgr2=0x7fe5740b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.518+0000 7fe57a800640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe57415ccf0 0x7fe5740b10d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00c700 tx=0x7fe56c00cbc0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.518+0000 7fe57a800640 1 -- 172.21.15.83:0/3138685651 shutdown_connections 2024-04-25T00:12:38.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.518+0000 7fe57a800640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe5540678c0 0x7fe554069d80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.518+0000 7fe57a800640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe57415c340 0x7fe57413e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.518+0000 7fe57a800640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe57415ccf0 0x7fe5740b10d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.518+0000 7fe57a800640 1 --2- 172.21.15.83:0/3138685651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5741599f0 0x7fe57413c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.518+0000 7fe57a800640 1 -- 172.21.15.83:0/3138685651 >> 172.21.15.83:0/3138685651 conn(0x7fe5740b1830 msgr2=0x7fe57406e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:38.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.518+0000 7fe57a800640 1 -- 172.21.15.83:0/3138685651 shutdown_connections 2024-04-25T00:12:38.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.518+0000 7fe57a800640 1 -- 172.21.15.83:0/3138685651 wait complete. 2024-04-25T00:12:38.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:12:38.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbcd8200640 1 Processor -- start 2024-04-25T00:12:38.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbcd8200640 1 -- start start 2024-04-25T00:12:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbcd8200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcd0071dc0 0x7fbcd006d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbcd8200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd006e0c0 con 0x7fbcd015df00 2024-04-25T00:12:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbcd8200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd00ad990 con 0x7fbcd006e7b0 2024-04-25T00:12:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbcd8200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd006f150 con 0x7fbcd0071dc0 2024-04-25T00:12:38.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbcd5800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbcd015df00 0x7fbcd00716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57272/0 (socket says 172.21.15.83:57272) 2024-04-25T00:12:38.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbcd5800640 1 -- 172.21.15.83:0/1334293744 learned_addr learned my addr 172.21.15.83:0/1334293744 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:38.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbcd4e00640 1 --2- 172.21.15.83:0/1334293744 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcd0071dc0 0x7fbcd006d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:38.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbccfe00640 1 -- 172.21.15.83:0/1334293744 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 58018086 0 0) 0x7fbcd006e0c0 con 0x7fbcd015df00 2024-04-25T00:12:38.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbccfe00640 1 -- 172.21.15.83:0/1334293744 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcb8003530 con 0x7fbcd015df00 2024-04-25T00:12:38.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbccfe00640 1 -- 172.21.15.83:0/1334293744 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1486344588 0 0) 0x7fbcd00ad990 con 0x7fbcd006e7b0 2024-04-25T00:12:38.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbccfe00640 1 -- 172.21.15.83:0/1334293744 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcd006e0c0 con 0x7fbcd006e7b0 2024-04-25T00:12:38.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbccfe00640 1 -- 172.21.15.83:0/1334293744 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2578815200 0 0) 0x7fbcb8003530 con 0x7fbcd015df00 2024-04-25T00:12:38.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbcd4e00640 1 -- 172.21.15.83:0/1334293744 >> v1:172.21.15.86:6789/0 conn(0x7fbcd006e7b0 legacy=0x7fbcd00ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbcd4e00640 1 -- 172.21.15.83:0/1334293744 >> v1:172.21.15.83:6789/0 conn(0x7fbcd015df00 legacy=0x7fbcd00716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.674+0000 7fbcd4e00640 1 -- 172.21.15.83:0/1334293744 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd00adc70 con 0x7fbcd0071dc0 2024-04-25T00:12:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.678+0000 7fbcd4e00640 1 --2- 172.21.15.83:0/1334293744 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcd0071dc0 0x7fbcd006d970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0097f0 tx=0x7fbcbc01c2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8c2eecf8890f75f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.678+0000 7fbccfe00640 1 -- 172.21.15.83:0/1334293744 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0b0070 con 0x7fbcd0071dc0 2024-04-25T00:12:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.678+0000 7fbccfe00640 1 -- 172.21.15.83:0/1334293744 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcbc0ab400 con 0x7fbcd0071dc0 2024-04-25T00:12:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.678+0000 7fbccfe00640 1 -- 172.21.15.83:0/1334293744 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4001680 con 0x7fbcd0071dc0 2024-04-25T00:12:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.678+0000 7fbcd8200640 1 -- 172.21.15.83:0/1334293744 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcd0071dc0 msgr2=0x7fbcd006d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.678+0000 7fbcd8200640 1 --2- 172.21.15.83:0/1334293744 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcd0071dc0 0x7fbcd006d970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0097f0 tx=0x7fbcbc01c2e0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.678+0000 7fbcd8200640 1 -- 172.21.15.83:0/1334293744 shutdown_connections 2024-04-25T00:12:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.678+0000 7fbcd8200640 1 --2- 172.21.15.83:0/1334293744 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcd0071dc0 0x7fbcd006d970 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.678+0000 7fbcd8200640 1 -- 172.21.15.83:0/1334293744 >> 172.21.15.83:0/1334293744 conn(0x7fbcd00b1830 msgr2=0x7fbcd00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.678+0000 7fbcd8200640 1 -- 172.21.15.83:0/1334293744 shutdown_connections 2024-04-25T00:12:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.678+0000 7fbcd8200640 1 -- 172.21.15.83:0/1334293744 wait complete. 2024-04-25T00:12:38.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd8200640 1 Processor -- start 2024-04-25T00:12:38.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd8200640 1 -- start start 2024-04-25T00:12:38.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd8200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd006e7b0 0x7fbcd0160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:38.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd8200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcd0071dc0 0x7fbcd014d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:38.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd8200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcd015df00 0x7fbcd014d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:38.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd8200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd016c010 con 0x7fbcd006e7b0 2024-04-25T00:12:38.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd8200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd016be90 con 0x7fbcd0071dc0 2024-04-25T00:12:38.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd8200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd016c190 con 0x7fbcd015df00 2024-04-25T00:12:38.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd006e7b0 0x7fbcd0160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:38.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd006e7b0 0x7fbcd0160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44750/0 (socket says 172.21.15.83:44750) 2024-04-25T00:12:38.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd5800640 1 -- 172.21.15.83:0/3778417324 learned_addr learned my addr 172.21.15.83:0/3778417324 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:38.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd6200640 1 --2- 172.21.15.83:0/3778417324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcd015df00 0x7fbcd014d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:38.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd4e00640 1 --2- 172.21.15.83:0/3778417324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcd0071dc0 0x7fbcd014d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:38.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd5800640 1 -- 172.21.15.83:0/3778417324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcd015df00 msgr2=0x7fbcd014d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd5800640 1 --2- 172.21.15.83:0/3778417324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcd015df00 0x7fbcd014d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd5800640 1 -- 172.21.15.83:0/3778417324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcd0071dc0 msgr2=0x7fbcd014d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd5800640 1 --2- 172.21.15.83:0/3778417324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcd0071dc0 0x7fbcd014d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd5800640 1 -- 172.21.15.83:0/3778417324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd0151520 con 0x7fbcd006e7b0 2024-04-25T00:12:38.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd5800640 1 --2- 172.21.15.83:0/3778417324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd006e7b0 0x7fbcd0160ad0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400c780 tx=0x7fbcc400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:38.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbccd600640 1 -- 172.21.15.83:0/3778417324 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4017070 con 0x7fbcd006e7b0 2024-04-25T00:12:38.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbccd600640 1 -- 172.21.15.83:0/3778417324 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc4005b00 con 0x7fbcd006e7b0 2024-04-25T00:12:38.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd8200640 1 -- 172.21.15.83:0/3778417324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd0151800 con 0x7fbcd006e7b0 2024-04-25T00:12:38.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbccd600640 1 -- 172.21.15.83:0/3778417324 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4007070 con 0x7fbcd006e7b0 2024-04-25T00:12:38.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd8200640 1 -- 172.21.15.83:0/3778417324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd0161010 con 0x7fbcd006e7b0 2024-04-25T00:12:38.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.682+0000 7fbcd8200640 1 -- 172.21.15.83:0/3778417324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd00737f0 con 0x7fbcd006e7b0 2024-04-25T00:12:38.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.686+0000 7fbccd600640 1 -- 172.21.15.83:0/3778417324 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbcc401a070 con 0x7fbcd006e7b0 2024-04-25T00:12:38.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.686+0000 7fbccd600640 1 --2- 172.21.15.83:0/3778417324 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbcb00678c0 0x7fbcb0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:38.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.686+0000 7fbccd600640 1 -- 172.21.15.83:0/3778417324 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fbcc40854f0 con 0x7fbcd006e7b0 2024-04-25T00:12:38.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.686+0000 7fbcd4e00640 1 --2- 172.21.15.83:0/3778417324 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbcb00678c0 0x7fbcb0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:38.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.686+0000 7fbcd4e00640 1 --2- 172.21.15.83:0/3778417324 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbcb00678c0 0x7fbcb0069d80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0ad000 tx=0x7fbcbc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:38.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.690+0000 7fbccd600640 1 -- 172.21.15.83:0/3778417324 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbcc4055d70 con 0x7fbcd006e7b0 2024-04-25T00:12:38.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.878+0000 7fbcd8200640 1 -- 172.21.15.83:0/3778417324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcd004f5f0 con 0x7fbcd006e7b0 2024-04-25T00:12:38.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.878+0000 7fbccd600640 1 -- 172.21.15.83:0/3778417324 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7fbcc4055670 con 0x7fbcd006e7b0 2024-04-25T00:12:38.881 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:38.881 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4277921554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":4277921554},{"type":"v1","addr":"172.21.15.86:6835","nonce":4277921554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/4145334103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":4145334103},{"type":"v1","addr":"172.21.15.86:6834","nonce":4145334103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3441801758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3441801758},{"type":"v1","addr":"172.21.15.83:6836","nonce":3441801758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:12:18.566138+0000","modified":"2024-04-25T00:12:21.572278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5328},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5328":{"gid":5328,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2704243724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2704243724},{"type":"v1","addr":"172.21.15.83:6837","nonce":2704243724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5328,"qdb_cluster":[5328]},"id":2}]} 2024-04-25T00:12:38.882 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 18 2024-04-25T00:12:38.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.882+0000 7fbcd8200640 1 -- 172.21.15.83:0/3778417324 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbcb00678c0 msgr2=0x7fbcb0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.882+0000 7fbcd8200640 1 --2- 172.21.15.83:0/3778417324 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbcb00678c0 0x7fbcb0069d80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0ad000 tx=0x7fbcbc008040 comp rx=0 tx=0).stop 2024-04-25T00:12:38.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.882+0000 7fbcd8200640 1 -- 172.21.15.83:0/3778417324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd006e7b0 msgr2=0x7fbcd0160ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:38.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.882+0000 7fbcd8200640 1 --2- 172.21.15.83:0/3778417324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd006e7b0 0x7fbcd0160ad0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400c780 tx=0x7fbcc400cc40 comp rx=0 tx=0).stop 2024-04-25T00:12:38.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.882+0000 7fbcd8200640 1 -- 172.21.15.83:0/3778417324 shutdown_connections 2024-04-25T00:12:38.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.882+0000 7fbcd8200640 1 --2- 172.21.15.83:0/3778417324 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbcb00678c0 0x7fbcb0069d80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.882+0000 7fbcd8200640 1 --2- 172.21.15.83:0/3778417324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcd015df00 0x7fbcd014d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.882+0000 7fbcd8200640 1 --2- 172.21.15.83:0/3778417324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcd0071dc0 0x7fbcd014d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.882+0000 7fbcd8200640 1 --2- 172.21.15.83:0/3778417324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd006e7b0 0x7fbcd0160ad0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:38.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.882+0000 7fbcd8200640 1 -- 172.21.15.83:0/3778417324 >> 172.21.15.83:0/3778417324 conn(0x7fbcd00b1830 msgr2=0x7fbcd00710d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:38.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.882+0000 7fbcd8200640 1 -- 172.21.15.83:0/3778417324 shutdown_connections 2024-04-25T00:12:38.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:38.882+0000 7fbcd8200640 1 -- 172.21.15.83:0/3778417324 wait complete. 2024-04-25T00:12:38.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:39.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9951200640 1 Processor -- start 2024-04-25T00:12:39.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9951200640 1 -- start start 2024-04-25T00:12:39.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9951200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f994c15df40 0x7f994c0b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9951200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f994c15e320 0x7f994c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9951200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f994c06fee0 con 0x7f994c06fa50 2024-04-25T00:12:39.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9951200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f994c0704a0 con 0x7f994c15df40 2024-04-25T00:12:39.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9951200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f994c0701c0 con 0x7f994c15e320 2024-04-25T00:12:39.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9949e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f994c15df40 0x7f994c0b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9941400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f994c06fa50 0x7f994c0cc2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57276/0 (socket says 172.21.15.83:57276) 2024-04-25T00:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9941400640 1 -- 172.21.15.83:0/1043321093 learned_addr learned my addr 172.21.15.83:0/1043321093 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9949e00640 1 -- 172.21.15.83:0/1043321093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f994c15e320 msgr2=0x7f994c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9949e00640 1 --2- 172.21.15.83:0/1043321093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f994c15e320 0x7f994c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9949e00640 1 -- 172.21.15.83:0/1043321093 >> v1:172.21.15.83:6789/0 conn(0x7f994c06fa50 legacy=0x7f994c0cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9949e00640 1 -- 172.21.15.83:0/1043321093 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c0d00a0 con 0x7f994c15df40 2024-04-25T00:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.034+0000 7f9949400640 1 -- 172.21.15.83:0/1043321093 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3644553902 0 0) 0x7f994c06fee0 con 0x7f994c06fa50 2024-04-25T00:12:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.038+0000 7f9949e00640 1 --2- 172.21.15.83:0/1043321093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f994c15df40 0x7f994c0b4300 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f993c0097c0 tx=0x7f993c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b8f75f3c0dd3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.038+0000 7f9949400640 1 -- 172.21.15.83:0/1043321093 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c0b0070 con 0x7f994c15df40 2024-04-25T00:12:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.038+0000 7f9949400640 1 -- 172.21.15.83:0/1043321093 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993c0ab600 con 0x7f994c15df40 2024-04-25T00:12:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.038+0000 7f9949400640 1 -- 172.21.15.83:0/1043321093 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c0ab9d0 con 0x7f994c15df40 2024-04-25T00:12:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.038+0000 7f9951200640 1 -- 172.21.15.83:0/1043321093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f994c15df40 msgr2=0x7f994c0b4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.038+0000 7f9951200640 1 --2- 172.21.15.83:0/1043321093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f994c15df40 0x7f994c0b4300 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f993c0097c0 tx=0x7f993c01c410 comp rx=0 tx=0).stop 2024-04-25T00:12:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.038+0000 7f9951200640 1 -- 172.21.15.83:0/1043321093 shutdown_connections 2024-04-25T00:12:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.038+0000 7f9951200640 1 --2- 172.21.15.83:0/1043321093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f994c15e320 0x7f994c0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.038+0000 7f9951200640 1 --2- 172.21.15.83:0/1043321093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f994c15df40 0x7f994c0b4300 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.038+0000 7f9951200640 1 -- 172.21.15.83:0/1043321093 >> 172.21.15.83:0/1043321093 conn(0x7f994c0b37e0 msgr2=0x7f994c0b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.038+0000 7f9951200640 1 -- 172.21.15.83:0/1043321093 shutdown_connections 2024-04-25T00:12:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.038+0000 7f9951200640 1 -- 172.21.15.83:0/1043321093 wait complete. 2024-04-25T00:12:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9951200640 1 Processor -- start 2024-04-25T00:12:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9951200640 1 -- start start 2024-04-25T00:12:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9951200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f994c15df40 0x7f994c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9951200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f994c15e320 0x7f994c06ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9951200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f994c150e10 0x7f994c156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9951200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f994c16b1a0 con 0x7f994c15e320 2024-04-25T00:12:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9951200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f994c16b020 con 0x7f994c15df40 2024-04-25T00:12:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9951200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f994c16b320 con 0x7f994c150e10 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9941400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f994c15e320 0x7f994c06ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9941400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f994c15e320 0x7f994c06ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44760/0 (socket says 172.21.15.83:44760) 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9941400640 1 -- 172.21.15.83:0/3878935383 learned_addr learned my addr 172.21.15.83:0/3878935383 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9949e00640 1 --2- 172.21.15.83:0/3878935383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f994c15df40 0x7f994c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f994a800640 1 --2- 172.21.15.83:0/3878935383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f994c150e10 0x7f994c156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9941400640 1 -- 172.21.15.83:0/3878935383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f994c150e10 msgr2=0x7f994c156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9941400640 1 --2- 172.21.15.83:0/3878935383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f994c150e10 0x7f994c156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9941400640 1 -- 172.21.15.83:0/3878935383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f994c15df40 msgr2=0x7f994c15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9941400640 1 --2- 172.21.15.83:0/3878935383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f994c15df40 0x7f994c15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9941400640 1 -- 172.21.15.83:0/3878935383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c15bce0 con 0x7f994c15e320 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9941400640 1 --2- 172.21.15.83:0/3878935383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f994c15e320 0x7f994c06ddb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f993400c480 tx=0x7f993400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9942a00640 1 -- 172.21.15.83:0/3878935383 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934018070 con 0x7f994c15e320 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9942a00640 1 -- 172.21.15.83:0/3878935383 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9934007500 con 0x7f994c15e320 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9951200640 1 -- 172.21.15.83:0/3878935383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994c1558d0 con 0x7f994c15e320 2024-04-25T00:12:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9942a00640 1 -- 172.21.15.83:0/3878935383 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934004a30 con 0x7f994c15e320 2024-04-25T00:12:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9951200640 1 -- 172.21.15.83:0/3878935383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994c155e90 con 0x7f994c15e320 2024-04-25T00:12:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.042+0000 7f9951200640 1 -- 172.21.15.83:0/3878935383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f994c09fe40 con 0x7f994c15e320 2024-04-25T00:12:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.046+0000 7f9942a00640 1 -- 172.21.15.83:0/3878935383 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f993401b070 con 0x7f994c15e320 2024-04-25T00:12:39.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.046+0000 7f9942a00640 1 --2- 172.21.15.83:0/3878935383 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99280678f0 0x7f9928069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.046+0000 7f9942a00640 1 -- 172.21.15.83:0/3878935383 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f9934084ff0 con 0x7f994c15e320 2024-04-25T00:12:39.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.046+0000 7f9949e00640 1 --2- 172.21.15.83:0/3878935383 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99280678f0 0x7f9928069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.046+0000 7f9949e00640 1 --2- 172.21.15.83:0/3878935383 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99280678f0 0x7f9928069db0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f993c0ad000 tx=0x7f993c0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:39.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.050+0000 7f9942a00640 1 -- 172.21.15.83:0/3878935383 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9934055380 con 0x7f994c15e320 2024-04-25T00:12:39.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.210+0000 7f9951200640 1 -- 172.21.15.83:0/3878935383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f994c073280 con 0x7f994c15e320 2024-04-25T00:12:39.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.210+0000 7f9942a00640 1 -- 172.21.15.83:0/3878935383 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16858 (secure 0 0 0) 0x7f9934059680 con 0x7f994c15e320 2024-04-25T00:12:39.211 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:39.211 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:20.181255+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:17.079304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:17.548692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.210+0000 7f9951200640 1 -- 172.21.15.83:0/3878935383 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99280678f0 msgr2=0x7f9928069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.210+0000 7f9951200640 1 --2- 172.21.15.83:0/3878935383 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99280678f0 0x7f9928069db0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f993c0ad000 tx=0x7f993c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:12:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.210+0000 7f9951200640 1 -- 172.21.15.83:0/3878935383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f994c15e320 msgr2=0x7f994c06ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.210+0000 7f9951200640 1 --2- 172.21.15.83:0/3878935383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f994c15e320 0x7f994c06ddb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f993400c480 tx=0x7f993400c940 comp rx=0 tx=0).stop 2024-04-25T00:12:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.210+0000 7f9951200640 1 -- 172.21.15.83:0/3878935383 shutdown_connections 2024-04-25T00:12:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.210+0000 7f9951200640 1 --2- 172.21.15.83:0/3878935383 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99280678f0 0x7f9928069db0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.210+0000 7f9951200640 1 --2- 172.21.15.83:0/3878935383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f994c150e10 0x7f994c156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.210+0000 7f9951200640 1 --2- 172.21.15.83:0/3878935383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f994c15e320 0x7f994c06ddb0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.210+0000 7f9951200640 1 --2- 172.21.15.83:0/3878935383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f994c15df40 0x7f994c15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.210+0000 7f9951200640 1 -- 172.21.15.83:0/3878935383 >> 172.21.15.83:0/3878935383 conn(0x7f994c0b37e0 msgr2=0x7f994c0ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.210+0000 7f9951200640 1 -- 172.21.15.83:0/3878935383 shutdown_connections 2024-04-25T00:12:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.210+0000 7f9951200640 1 -- 172.21.15.83:0/3878935383 wait complete. 2024-04-25T00:12:39.233 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:12:39.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.370+0000 7f97a2c00640 1 Processor -- start 2024-04-25T00:12:39.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.370+0000 7f97a2c00640 1 -- start start 2024-04-25T00:12:39.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.370+0000 7f97a2c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f979c1446e0 0x7f979c0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.370+0000 7f97a2c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f979c06ee20 0x7f979c071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.370+0000 7f97a2c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f979c0cb210 con 0x7f979c15f350 2024-04-25T00:12:39.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.370+0000 7f97a2c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f979c0715b0 con 0x7f979c06ee20 2024-04-25T00:12:39.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.370+0000 7f97a2c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f979c144ac0 con 0x7f979c1446e0 2024-04-25T00:12:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.374+0000 7f979be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f979c1446e0 0x7f979c0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.374+0000 7f97a0c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f979c06ee20 0x7f979c071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.374+0000 7f979be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f979c1446e0 0x7f979c0cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:56392/0 (socket says 172.21.15.83:56392) 2024-04-25T00:12:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.374+0000 7f979be00640 1 -- 172.21.15.83:0/2023740383 learned_addr learned my addr 172.21.15.83:0/2023740383 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.374+0000 7f979be00640 1 -- 172.21.15.83:0/2023740383 >> v2:172.21.15.86:3300/0 conn(0x7f979c06ee20 msgr2=0x7f979c071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.374+0000 7f979be00640 1 --2- 172.21.15.83:0/2023740383 >> v2:172.21.15.86:3300/0 conn(0x7f979c06ee20 0x7f979c071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.374+0000 7f979be00640 1 -- 172.21.15.83:0/2023740383 >> v1:172.21.15.86:6789/0 conn(0x7f979c15f350 legacy=0x7f979c06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.374+0000 7f979be00640 1 -- 172.21.15.83:0/2023740383 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979c0cb4f0 con 0x7f979c1446e0 2024-04-25T00:12:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.374+0000 7f979b400640 1 -- 172.21.15.83:0/2023740383 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2612799891 0 0) 0x7f979c0cb210 con 0x7f979c15f350 2024-04-25T00:12:39.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.374+0000 7f979be00640 1 --2- 172.21.15.83:0/2023740383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f979c1446e0 0x7f979c0cab50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f978c009870 tx=0x7f978c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b41677fd8dfcf34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:39.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f979b400640 1 -- 172.21.15.83:0/2023740383 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c0a7070 con 0x7f979c1446e0 2024-04-25T00:12:39.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f979b400640 1 -- 172.21.15.83:0/2023740383 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f978c0b0070 con 0x7f979c1446e0 2024-04-25T00:12:39.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f979b400640 1 -- 172.21.15.83:0/2023740383 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c0ab890 con 0x7f979c1446e0 2024-04-25T00:12:39.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 -- 172.21.15.83:0/2023740383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f979c1446e0 msgr2=0x7f979c0cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 --2- 172.21.15.83:0/2023740383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f979c1446e0 0x7f979c0cab50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f978c009870 tx=0x7f978c01c560 comp rx=0 tx=0).stop 2024-04-25T00:12:39.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 -- 172.21.15.83:0/2023740383 shutdown_connections 2024-04-25T00:12:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 --2- 172.21.15.83:0/2023740383 >> v2:172.21.15.86:3300/0 conn(0x7f979c06ee20 0x7f979c071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 --2- 172.21.15.83:0/2023740383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f979c1446e0 0x7f979c0cab50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 -- 172.21.15.83:0/2023740383 >> 172.21.15.83:0/2023740383 conn(0x7f979c0cfa60 msgr2=0x7f979c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 -- 172.21.15.83:0/2023740383 shutdown_connections 2024-04-25T00:12:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 -- 172.21.15.83:0/2023740383 wait complete. 2024-04-25T00:12:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 Processor -- start 2024-04-25T00:12:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 -- start start 2024-04-25T00:12:39.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c06ee20 0x7f979c135d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f979c1446e0 0x7f979c13ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f979c15f350 0x7f979c13f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f979c072eb0 con 0x7f979c06ee20 2024-04-25T00:12:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f979c072d30 con 0x7f979c15f350 2024-04-25T00:12:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a2c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f979c073030 con 0x7f979c1446e0 2024-04-25T00:12:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f979be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c06ee20 0x7f979c135d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f979be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c06ee20 0x7f979c135d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44774/0 (socket says 172.21.15.83:44774) 2024-04-25T00:12:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f979be00640 1 -- 172.21.15.83:0/1245028820 learned_addr learned my addr 172.21.15.83:0/1245028820 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f97a0c00640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f979c15f350 0x7f979c13f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f9793400640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f979c1446e0 0x7f979c13ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f979be00640 1 -- 172.21.15.83:0/1245028820 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f979c1446e0 msgr2=0x7f979c13ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.378+0000 7f979be00640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f979c1446e0 0x7f979c13ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f979be00640 1 -- 172.21.15.83:0/1245028820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f979c15f350 msgr2=0x7f979c13f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f979be00640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f979c15f350 0x7f979c13f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f979be00640 1 -- 172.21.15.83:0/1245028820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979c1377e0 con 0x7f979c06ee20 2024-04-25T00:12:39.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f97a0c00640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f979c15f350 0x7f979c13f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:39.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f979be00640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c06ee20 0x7f979c135d60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f978c01a190 tx=0x7f978c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:39.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f9798c00640 1 -- 172.21.15.83:0/1245028820 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c0ba070 con 0x7f979c06ee20 2024-04-25T00:12:39.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f9798c00640 1 -- 172.21.15.83:0/1245028820 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f978c004220 con 0x7f979c06ee20 2024-04-25T00:12:39.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f97a2c00640 1 -- 172.21.15.83:0/1245028820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979c137ac0 con 0x7f979c06ee20 2024-04-25T00:12:39.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f9798c00640 1 -- 172.21.15.83:0/1245028820 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c0ab940 con 0x7f979c06ee20 2024-04-25T00:12:39.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f97a2c00640 1 -- 172.21.15.83:0/1245028820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f979c138080 con 0x7f979c06ee20 2024-04-25T00:12:39.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f97a2c00640 1 -- 172.21.15.83:0/1245028820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f979c068230 con 0x7f979c06ee20 2024-04-25T00:12:39.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f9798c00640 1 -- 172.21.15.83:0/1245028820 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f978c0abae0 con 0x7f979c06ee20 2024-04-25T00:12:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f9798c00640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97780678c0 0x7f9778069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f9798c00640 1 -- 172.21.15.83:0/1245028820 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f978c11fb60 con 0x7f979c06ee20 2024-04-25T00:12:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.382+0000 7f9793400640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97780678c0 0x7f9778069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.386+0000 7f9793400640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97780678c0 0x7f9778069d80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f979c0ce620 tx=0x7f9784009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:39.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.386+0000 7f9798c00640 1 -- 172.21.15.83:0/1245028820 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f978c0c5030 con 0x7f979c06ee20 2024-04-25T00:12:39.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.578+0000 7f97a2c00640 1 -- 172.21.15.83:0/1245028820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f979c06dea0 con 0x7f979c06ee20 2024-04-25T00:12:39.579 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-25T00:12:39.578+0000 7efeeee00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:12:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.578+0000 7f9798c00640 1 -- 172.21.15.83:0/1245028820 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f978c016150 con 0x7f979c06ee20 2024-04-25T00:12:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:12:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.582+0000 7f97a2c00640 1 -- 172.21.15.83:0/1245028820 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97780678c0 msgr2=0x7f9778069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.582+0000 7f97a2c00640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97780678c0 0x7f9778069d80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f979c0ce620 tx=0x7f9784009040 comp rx=0 tx=0).stop 2024-04-25T00:12:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.582+0000 7f97a2c00640 1 -- 172.21.15.83:0/1245028820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c06ee20 msgr2=0x7f979c135d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.582+0000 7f97a2c00640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c06ee20 0x7f979c135d60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f978c01a190 tx=0x7f978c01ada0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.582+0000 7f97a2c00640 1 -- 172.21.15.83:0/1245028820 shutdown_connections 2024-04-25T00:12:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.582+0000 7f97a2c00640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97780678c0 0x7f9778069d80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.582+0000 7f97a2c00640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f979c1446e0 0x7f979c13ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.582+0000 7f97a2c00640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f979c15f350 0x7f979c13f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.582+0000 7f97a2c00640 1 --2- 172.21.15.83:0/1245028820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c06ee20 0x7f979c135d60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.582+0000 7f97a2c00640 1 -- 172.21.15.83:0/1245028820 >> 172.21.15.83:0/1245028820 conn(0x7f979c0cfa60 msgr2=0x7f979c06e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.582+0000 7f97a2c00640 1 -- 172.21.15.83:0/1245028820 shutdown_connections 2024-04-25T00:12:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.582+0000 7f97a2c00640 1 -- 172.21.15.83:0/1245028820 wait complete. 2024-04-25T00:12:39.590 INFO:tasks.ceph.mds.a.smithi083.stderr: -2575> 2024-04-25T00:12:22.982+0000 7fc2de800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:12:39.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:12:39.638 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-25T00:12:39.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b7600640 1 Processor -- start 2024-04-25T00:12:39.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b7600640 1 -- start start 2024-04-25T00:12:39.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b7600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2b0159810 0x7ff2b00cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b7600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2b0160020 0x7ff2b00ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b7600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b00ce000 con 0x7ff2b006e950 2024-04-25T00:12:39.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b7600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b006e210 con 0x7ff2b0159810 2024-04-25T00:12:39.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b7600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b006da00 con 0x7ff2b0160020 2024-04-25T00:12:39.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2a7e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff2b006e950 0x7ff2b015f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57286/0 (socket says 172.21.15.83:57286) 2024-04-25T00:12:39.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2a7e00640 1 -- 172.21.15.83:0/644733688 learned_addr learned my addr 172.21.15.83:0/644733688 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:39.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b4c00640 1 --2- 172.21.15.83:0/644733688 >> v2:172.21.15.86:3300/0 conn(0x7ff2b0159810 0x7ff2b00cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 -- 172.21.15.83:0/644733688 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1332813416 0 0) 0x7ff2b00ce000 con 0x7ff2b006e950 2024-04-25T00:12:39.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b5600640 1 --2- 172.21.15.83:0/644733688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2b0160020 0x7ff2b00ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 -- 172.21.15.83:0/644733688 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff294003530 con 0x7ff2b006e950 2024-04-25T00:12:39.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 -- 172.21.15.83:0/644733688 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1206994662 0 0) 0x7ff294003530 con 0x7ff2b006e950 2024-04-25T00:12:39.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 -- 172.21.15.83:0/644733688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2b0160020 msgr2=0x7ff2b00ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:39.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 --2- 172.21.15.83:0/644733688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2b0160020 0x7ff2b00ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 -- 172.21.15.83:0/644733688 >> v2:172.21.15.86:3300/0 conn(0x7ff2b0159810 msgr2=0x7ff2b00cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:39.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 --2- 172.21.15.83:0/644733688 >> v2:172.21.15.86:3300/0 conn(0x7ff2b0159810 0x7ff2b00cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 -- 172.21.15.83:0/644733688 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b006eec0 con 0x7ff2b006e950 2024-04-25T00:12:39.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 -- 172.21.15.83:0/644733688 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7ff2980016a0 con 0x7ff2b006e950 2024-04-25T00:12:39.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 -- 172.21.15.83:0/644733688 >> v1:172.21.15.83:6789/0 conn(0x7ff2b006e950 legacy=0x7ff2b015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 --2- 172.21.15.83:0/644733688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff294003a30 0x7ff294003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 --2- 172.21.15.83:0/644733688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2b0160020 0x7ff294004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 --2- 172.21.15.83:0/644733688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff294004d50 0x7ff2940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 -- 172.21.15.83:0/644733688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b006e210 con 0x7ff294004d50 2024-04-25T00:12:39.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 -- 172.21.15.83:0/644733688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b006da00 con 0x7ff294003a30 2024-04-25T00:12:39.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2afe00640 1 -- 172.21.15.83:0/644733688 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff29400ba50 con 0x7ff2b0160020 2024-04-25T00:12:39.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b5600640 1 --2- 172.21.15.83:0/644733688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff294004d50 0x7ff2940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b4c00640 1 --2- 172.21.15.83:0/644733688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff294003a30 0x7ff294003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2a7e00640 1 --2- 172.21.15.83:0/644733688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2b0160020 0x7ff294004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b5600640 1 -- 172.21.15.83:0/644733688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2b0160020 msgr2=0x7ff294004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:39.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b5600640 1 --2- 172.21.15.83:0/644733688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2b0160020 0x7ff294004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b5600640 1 -- 172.21.15.83:0/644733688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff294003a30 msgr2=0x7ff294003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:39.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b5600640 1 --2- 172.21.15.83:0/644733688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff294003a30 0x7ff294003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.746+0000 7ff2b5600640 1 -- 172.21.15.83:0/644733688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff294003530 con 0x7ff294004d50 2024-04-25T00:12:39.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.750+0000 7ff2b5600640 1 --2- 172.21.15.83:0/644733688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff294004d50 0x7ff2940051d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00737c0 tx=0x7ff2a801cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21da32a9f1000784 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.750+0000 7ff2afe00640 1 -- 172.21.15.83:0/644733688 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80b2070 con 0x7ff294004d50 2024-04-25T00:12:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.750+0000 7ff2afe00640 1 -- 172.21.15.83:0/644733688 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a80a5cd0 con 0x7ff294004d50 2024-04-25T00:12:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.750+0000 7ff2afe00640 1 -- 172.21.15.83:0/644733688 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8009bf0 con 0x7ff294004d50 2024-04-25T00:12:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.750+0000 7ff2b7600640 1 -- 172.21.15.83:0/644733688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff294004d50 msgr2=0x7ff2940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.750+0000 7ff2b7600640 1 --2- 172.21.15.83:0/644733688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff294004d50 0x7ff2940051d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00737c0 tx=0x7ff2a801cc00 comp rx=0 tx=0).stop 2024-04-25T00:12:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.750+0000 7ff2b5600640 1 -- 172.21.15.83:0/644733688 reap_dead start 2024-04-25T00:12:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.750+0000 7ff2b7600640 1 -- 172.21.15.83:0/644733688 shutdown_connections 2024-04-25T00:12:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.750+0000 7ff2b7600640 1 -- 172.21.15.83:0/644733688 >> 172.21.15.83:0/644733688 conn(0x7ff2b00b1830 msgr2=0x7ff2b00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.750+0000 7ff2b7600640 1 -- 172.21.15.83:0/644733688 shutdown_connections 2024-04-25T00:12:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.750+0000 7ff2b7600640 1 -- 172.21.15.83:0/644733688 wait complete. 2024-04-25T00:12:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b7600640 1 Processor -- start 2024-04-25T00:12:39.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b7600640 1 -- start start 2024-04-25T00:12:39.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b7600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff294004d50 0x7ff2b014d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b7600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2b00cea60 0x7ff2b00ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b7600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b00b3e20 0x7ff2b00b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b7600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b00cc9e0 con 0x7ff2b00b3e20 2024-04-25T00:12:39.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b7600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b006df30 con 0x7ff294004d50 2024-04-25T00:12:39.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b7600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b0159bf0 con 0x7ff2b00cea60 2024-04-25T00:12:39.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b00b3e20 0x7ff2b00b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b00b3e20 0x7ff2b00b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44802/0 (socket says 172.21.15.83:44802) 2024-04-25T00:12:39.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b5600640 1 -- 172.21.15.83:0/3307170740 learned_addr learned my addr 172.21.15.83:0/3307170740 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:39.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b4c00640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff294004d50 0x7ff2b014d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2a7e00640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2b00cea60 0x7ff2b00ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b5600640 1 -- 172.21.15.83:0/3307170740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2b00cea60 msgr2=0x7ff2b00ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b5600640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2b00cea60 0x7ff2b00ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b5600640 1 -- 172.21.15.83:0/3307170740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff294004d50 msgr2=0x7ff2b014d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b5600640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff294004d50 0x7ff2b014d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b5600640 1 -- 172.21.15.83:0/3307170740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b00b4c70 con 0x7ff2b00b3e20 2024-04-25T00:12:39.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2a7e00640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2b00cea60 0x7ff2b00ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:39.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b5600640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b00b3e20 0x7ff2b00b42a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80b0040 tx=0x7ff2a8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:39.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2ad600640 1 -- 172.21.15.83:0/3307170740 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80b2040 con 0x7ff2b00b3e20 2024-04-25T00:12:39.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2ad600640 1 -- 172.21.15.83:0/3307170740 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a80a5ad0 con 0x7ff2b00b3e20 2024-04-25T00:12:39.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2ad600640 1 -- 172.21.15.83:0/3307170740 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8009bf0 con 0x7ff2b00b3e20 2024-04-25T00:12:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b7600640 1 -- 172.21.15.83:0/3307170740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b01f4e70 con 0x7ff2b00b3e20 2024-04-25T00:12:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.754+0000 7ff2b7600640 1 -- 172.21.15.83:0/3307170740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b01f4390 con 0x7ff2b00b3e20 2024-04-25T00:12:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.758+0000 7ff2b7600640 1 -- 172.21.15.83:0/3307170740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b004f510 con 0x7ff2b00b3e20 2024-04-25T00:12:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.758+0000 7ff2ad600640 1 -- 172.21.15.83:0/3307170740 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff2a80c0020 con 0x7ff2b00b3e20 2024-04-25T00:12:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.758+0000 7ff2ad600640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff28c0678f0 0x7ff28c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.758+0000 7ff2ad600640 1 -- 172.21.15.83:0/3307170740 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7ff2a81204f0 con 0x7ff2b00b3e20 2024-04-25T00:12:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.758+0000 7ff2b4c00640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff28c0678f0 0x7ff28c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.758+0000 7ff2b4c00640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff28c0678f0 0x7ff28c069db0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff2b014e050 tx=0x7ff2a0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:39.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.762+0000 7ff2ad600640 1 -- 172.21.15.83:0/3307170740 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff2a80c7020 con 0x7ff2b00b3e20 2024-04-25T00:12:39.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:39.954+0000 7ff2b7600640 1 -- 172.21.15.83:0/3307170740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff2b0151ac0 con 0x7ff2b00b3e20 2024-04-25T00:12:40.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.582+0000 7ff2ad600640 1 -- 172.21.15.83:0/3307170740 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7ff2a8016910 con 0x7ff2b00b3e20 2024-04-25T00:12:40.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.586+0000 7ff2b7600640 1 -- 172.21.15.83:0/3307170740 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff28c0678f0 msgr2=0x7ff28c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:40.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.586+0000 7ff2b7600640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff28c0678f0 0x7ff28c069db0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff2b014e050 tx=0x7ff2a0009190 comp rx=0 tx=0).stop 2024-04-25T00:12:40.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.586+0000 7ff2b7600640 1 -- 172.21.15.83:0/3307170740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b00b3e20 msgr2=0x7ff2b00b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:40.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.586+0000 7ff2b7600640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b00b3e20 0x7ff2b00b42a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80b0040 tx=0x7ff2a8009750 comp rx=0 tx=0).stop 2024-04-25T00:12:40.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.586+0000 7ff2b7600640 1 -- 172.21.15.83:0/3307170740 shutdown_connections 2024-04-25T00:12:40.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.586+0000 7ff2b7600640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff28c0678f0 0x7ff28c069db0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:40.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.586+0000 7ff2b7600640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2b00cea60 0x7ff2b00ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:40.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.586+0000 7ff2b7600640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b00b3e20 0x7ff2b00b42a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:40.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.586+0000 7ff2b7600640 1 --2- 172.21.15.83:0/3307170740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff294004d50 0x7ff2b014d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:40.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.586+0000 7ff2b7600640 1 -- 172.21.15.83:0/3307170740 >> 172.21.15.83:0/3307170740 conn(0x7ff2b00b1830 msgr2=0x7ff2b015c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:40.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.586+0000 7ff2b7600640 1 -- 172.21.15.83:0/3307170740 shutdown_connections 2024-04-25T00:12:40.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.586+0000 7ff2b7600640 1 -- 172.21.15.83:0/3307170740 wait complete. 2024-04-25T00:12:40.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:12:40.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f510e800640 1 Processor -- start 2024-04-25T00:12:40.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f510e800640 1 -- start start 2024-04-25T00:12:40.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f510e800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ce080 0x7f51080cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:40.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f510e800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f51080d0280 0x7f51080b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:40.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f510e800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51080ce610 con 0x7f51081455d0 2024-04-25T00:12:40.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f510e800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f51080d06f0 con 0x7f51080d0280 2024-04-25T00:12:40.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f510e800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f51080ce8f0 con 0x7f51080ce080 2024-04-25T00:12:40.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f5107400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f51081455d0 0x7f510806d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57298/0 (socket says 172.21.15.83:57298) 2024-04-25T00:12:40.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f5107e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f51080d0280 0x7f51080b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:40.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f5107400640 1 -- 172.21.15.83:0/3172143010 learned_addr learned my addr 172.21.15.83:0/3172143010 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:40.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f5106a00640 1 --2- 172.21.15.83:0/3172143010 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ce080 0x7f51080cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:40.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f5106000640 1 -- 172.21.15.83:0/3172143010 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3243993578 0 0) 0x7f51080ce610 con 0x7f51081455d0 2024-04-25T00:12:40.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f5106000640 1 -- 172.21.15.83:0/3172143010 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50ec003530 con 0x7f51081455d0 2024-04-25T00:12:40.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f5107e00640 1 -- 172.21.15.83:0/3172143010 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ce080 msgr2=0x7f51080cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:40.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f5107e00640 1 --2- 172.21.15.83:0/3172143010 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ce080 0x7f51080cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:40.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f5107e00640 1 -- 172.21.15.83:0/3172143010 >> v1:172.21.15.83:6789/0 conn(0x7f51081455d0 legacy=0x7f510806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:40.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f5107e00640 1 -- 172.21.15.83:0/3172143010 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51080cebd0 con 0x7f51080d0280 2024-04-25T00:12:40.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.790+0000 7f5106a00640 1 --2- 172.21.15.83:0/3172143010 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ce080 0x7f51080cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.794+0000 7f5107e00640 1 --2- 172.21.15.83:0/3172143010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51080d0280 0x7f51080b2b30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f51000097c0 tx=0x7f510001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d133ac83eda547 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.794+0000 7f5106000640 1 -- 172.21.15.83:0/3172143010 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51000aa600 con 0x7f51080d0280 2024-04-25T00:12:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.794+0000 7f5106000640 1 -- 172.21.15.83:0/3172143010 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51000af070 con 0x7f51080d0280 2024-04-25T00:12:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.794+0000 7f5106000640 1 -- 172.21.15.83:0/3172143010 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51000aa9c0 con 0x7f51080d0280 2024-04-25T00:12:40.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.794+0000 7f510e800640 1 -- 172.21.15.83:0/3172143010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51080d0280 msgr2=0x7f51080b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:40.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.794+0000 7f510e800640 1 --2- 172.21.15.83:0/3172143010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51080d0280 0x7f51080b2b30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f51000097c0 tx=0x7f510001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:12:40.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.794+0000 7f510e800640 1 -- 172.21.15.83:0/3172143010 shutdown_connections 2024-04-25T00:12:40.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.794+0000 7f510e800640 1 --2- 172.21.15.83:0/3172143010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51080d0280 0x7f51080b2b30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:40.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.794+0000 7f510e800640 1 --2- 172.21.15.83:0/3172143010 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ce080 0x7f51080cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:40.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.794+0000 7f510e800640 1 -- 172.21.15.83:0/3172143010 >> 172.21.15.83:0/3172143010 conn(0x7f510806b3b0 msgr2=0x7f510806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:40.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.794+0000 7f510e800640 1 -- 172.21.15.83:0/3172143010 shutdown_connections 2024-04-25T00:12:40.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.794+0000 7f510e800640 1 -- 172.21.15.83:0/3172143010 wait complete. 2024-04-25T00:12:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.794+0000 7f510e800640 1 Processor -- start 2024-04-25T00:12:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.794+0000 7f510e800640 1 -- start start 2024-04-25T00:12:40.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f510e800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ce080 0x7f510813ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:40.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f510e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51080d0280 0x7f510813c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:40.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f510e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51081455d0 0x7f51081f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:40.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f510e800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51080b3a00 con 0x7f51080d0280 2024-04-25T00:12:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f510e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51080b3880 con 0x7f51081455d0 2024-04-25T00:12:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f510e800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f51080b3b80 con 0x7f51080ce080 2024-04-25T00:12:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f5106a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51080d0280 0x7f510813c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f5106a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51080d0280 0x7f510813c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44822/0 (socket says 172.21.15.83:44822) 2024-04-25T00:12:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f5106a00640 1 -- 172.21.15.83:0/1210577962 learned_addr learned my addr 172.21.15.83:0/1210577962 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f5107400640 1 --2- 172.21.15.83:0/1210577962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ce080 0x7f510813ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f5107e00640 1 --2- 172.21.15.83:0/1210577962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51081455d0 0x7f51081f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f5106a00640 1 -- 172.21.15.83:0/1210577962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ce080 msgr2=0x7f510813ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f5106a00640 1 --2- 172.21.15.83:0/1210577962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ce080 0x7f510813ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f5106a00640 1 -- 172.21.15.83:0/1210577962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51081455d0 msgr2=0x7f51081f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f5106a00640 1 --2- 172.21.15.83:0/1210577962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51081455d0 0x7f51081f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f5106a00640 1 -- 172.21.15.83:0/1210577962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510813b3a0 con 0x7f51080d0280 2024-04-25T00:12:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f5106a00640 1 --2- 172.21.15.83:0/1210577962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51080d0280 0x7f510813c0d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f50f400e810 tx=0x7f50f400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f50ff400640 1 -- 172.21.15.83:0/1210577962 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4017070 con 0x7f51080d0280 2024-04-25T00:12:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f50ff400640 1 -- 172.21.15.83:0/1210577962 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f400c8d0 con 0x7f51080d0280 2024-04-25T00:12:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f50ff400640 1 -- 172.21.15.83:0/1210577962 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f400f040 con 0x7f51080d0280 2024-04-25T00:12:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f510e800640 1 -- 172.21.15.83:0/1210577962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510813ca70 con 0x7f51080d0280 2024-04-25T00:12:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f510e800640 1 -- 172.21.15.83:0/1210577962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510813d030 con 0x7f51080d0280 2024-04-25T00:12:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f510e800640 1 -- 172.21.15.83:0/1210577962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51080736f0 con 0x7f51080d0280 2024-04-25T00:12:40.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.798+0000 7f50ff400640 1 -- 172.21.15.83:0/1210577962 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f50f401a070 con 0x7f51080d0280 2024-04-25T00:12:40.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.802+0000 7f50ff400640 1 --2- 172.21.15.83:0/1210577962 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50e80678c0 0x7f50e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:40.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.802+0000 7f50ff400640 1 -- 172.21.15.83:0/1210577962 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f50f4085b50 con 0x7f51080d0280 2024-04-25T00:12:40.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.802+0000 7f5107400640 1 --2- 172.21.15.83:0/1210577962 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50e80678c0 0x7f50e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:40.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.802+0000 7f5107400640 1 --2- 172.21.15.83:0/1210577962 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50e80678c0 0x7f50e8069d80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f50f8005c10 tx=0x7f50f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:40.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.806+0000 7f50ff400640 1 -- 172.21.15.83:0/1210577962 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f50f4056410 con 0x7f51080d0280 2024-04-25T00:12:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:40.962+0000 7f510e800640 1 -- 172.21.15.83:0/1210577962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f510804f5f0 con 0x7f51080d0280 2024-04-25T00:12:41.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.585+0000 7f50ff400640 1 -- 172.21.15.83:0/1210577962 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) ==== 181+0+0 (secure 0 0 0) 0x7f50f4055d10 con 0x7f51080d0280 2024-04-25T00:12:41.586 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:12:41.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.585+0000 7f510e800640 1 -- 172.21.15.83:0/1210577962 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50e80678c0 msgr2=0x7f50e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.585+0000 7f510e800640 1 --2- 172.21.15.83:0/1210577962 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50e80678c0 0x7f50e8069d80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f50f8005c10 tx=0x7f50f8009040 comp rx=0 tx=0).stop 2024-04-25T00:12:41.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.585+0000 7f510e800640 1 -- 172.21.15.83:0/1210577962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51080d0280 msgr2=0x7f510813c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.585+0000 7f510e800640 1 --2- 172.21.15.83:0/1210577962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51080d0280 0x7f510813c0d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f50f400e810 tx=0x7f50f400ecd0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.585+0000 7f510e800640 1 -- 172.21.15.83:0/1210577962 shutdown_connections 2024-04-25T00:12:41.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.585+0000 7f510e800640 1 --2- 172.21.15.83:0/1210577962 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50e80678c0 0x7f50e8069d80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.585+0000 7f510e800640 1 --2- 172.21.15.83:0/1210577962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51081455d0 0x7f51081f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.585+0000 7f510e800640 1 --2- 172.21.15.83:0/1210577962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51080d0280 0x7f510813c0d0 secure :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f50f400e810 tx=0x7f50f400ecd0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.585+0000 7f510e800640 1 --2- 172.21.15.83:0/1210577962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ce080 0x7f510813ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.585+0000 7f510e800640 1 -- 172.21.15.83:0/1210577962 >> 172.21.15.83:0/1210577962 conn(0x7f510806b3b0 msgr2=0x7f51080726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:41.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.589+0000 7f510e800640 1 -- 172.21.15.83:0/1210577962 shutdown_connections 2024-04-25T00:12:41.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.589+0000 7f510e800640 1 -- 172.21.15.83:0/1210577962 wait complete. 2024-04-25T00:12:41.613 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:12:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.753+0000 7f267f400640 1 Processor -- start 2024-04-25T00:12:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.753+0000 7f267f400640 1 -- start start 2024-04-25T00:12:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.753+0000 7f267f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26780ce080 0x7f26780cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.753+0000 7f267f400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26780ce610 con 0x7f26781455d0 2024-04-25T00:12:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.753+0000 7f267f400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26780ce8f0 con 0x7f26780d0280 2024-04-25T00:12:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.753+0000 7f267f400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f26780d06d0 con 0x7f26780ce080 2024-04-25T00:12:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.753+0000 7f267ca00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f26781455d0 0x7f267806d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57302/0 (socket says 172.21.15.83:57302) 2024-04-25T00:12:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.753+0000 7f267ca00640 1 -- 172.21.15.83:0/2445487009 learned_addr learned my addr 172.21.15.83:0/2445487009 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.753+0000 7f266fe00640 1 --2- 172.21.15.83:0/2445487009 >> v2:172.21.15.86:3300/0 conn(0x7f26780ce080 0x7f26780cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.753+0000 7f2677e00640 1 -- 172.21.15.83:0/2445487009 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1098943480 0 0) 0x7f26780ce610 con 0x7f26781455d0 2024-04-25T00:12:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f2677e00640 1 -- 172.21.15.83:0/2445487009 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f265c003530 con 0x7f26781455d0 2024-04-25T00:12:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f2677e00640 1 -- 172.21.15.83:0/2445487009 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 130400322 0 0) 0x7f265c003530 con 0x7f26781455d0 2024-04-25T00:12:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f2677e00640 1 -- 172.21.15.83:0/2445487009 >> v2:172.21.15.86:3300/0 conn(0x7f26780ce080 msgr2=0x7f26780cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f2677e00640 1 --2- 172.21.15.83:0/2445487009 >> v2:172.21.15.86:3300/0 conn(0x7f26780ce080 0x7f26780cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f2677e00640 1 -- 172.21.15.83:0/2445487009 >> v1:172.21.15.86:6789/0 conn(0x7f26780d0280 legacy=0x7f26780b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f2677e00640 1 -- 172.21.15.83:0/2445487009 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26780cebd0 con 0x7f26781455d0 2024-04-25T00:12:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f2677e00640 1 -- 172.21.15.83:0/2445487009 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f2668001680 con 0x7f26781455d0 2024-04-25T00:12:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f2677e00640 1 -- 172.21.15.83:0/2445487009 >> v1:172.21.15.83:6789/0 conn(0x7f26781455d0 legacy=0x7f267806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f2677e00640 1 --2- 172.21.15.83:0/2445487009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265c003a10 0x7f265c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f2677e00640 1 --2- 172.21.15.83:0/2445487009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265c004810 0x7f265c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f2677e00640 1 --2- 172.21.15.83:0/2445487009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f265c0051d0 0x7f265c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f2677e00640 1 -- 172.21.15.83:0/2445487009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26780d06d0 con 0x7f265c004810 2024-04-25T00:12:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f2677e00640 1 -- 172.21.15.83:0/2445487009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f265c00c0d0 con 0x7f265c003a10 2024-04-25T00:12:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f2677e00640 1 -- 172.21.15.83:0/2445487009 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f265c00c2d0 con 0x7f265c0051d0 2024-04-25T00:12:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f266fe00640 1 --2- 172.21.15.83:0/2445487009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265c004810 0x7f265c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f267ca00640 1 --2- 172.21.15.83:0/2445487009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265c003a10 0x7f265c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f267d400640 1 --2- 172.21.15.83:0/2445487009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f265c0051d0 0x7f265c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f266fe00640 1 -- 172.21.15.83:0/2445487009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f265c0051d0 msgr2=0x7f265c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f266fe00640 1 --2- 172.21.15.83:0/2445487009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f265c0051d0 0x7f265c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f266fe00640 1 -- 172.21.15.83:0/2445487009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265c003a10 msgr2=0x7f265c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f266fe00640 1 --2- 172.21.15.83:0/2445487009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265c003a10 0x7f265c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f266fe00640 1 -- 172.21.15.83:0/2445487009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c003530 con 0x7f265c004810 2024-04-25T00:12:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f267d400640 1 -- 172.21.15.83:0/2445487009 reap_dead start 2024-04-25T00:12:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.757+0000 7f266fe00640 1 --2- 172.21.15.83:0/2445487009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265c004810 0x7f265c004c90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2660009ae0 tx=0x7f266001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b6f8cde6370934 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f2677e00640 1 -- 172.21.15.83:0/2445487009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26600b2070 con 0x7f265c004810 2024-04-25T00:12:41.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f2677e00640 1 -- 172.21.15.83:0/2445487009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26600a5550 con 0x7f265c004810 2024-04-25T00:12:41.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f2677e00640 1 -- 172.21.15.83:0/2445487009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26600096b0 con 0x7f265c004810 2024-04-25T00:12:41.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 -- 172.21.15.83:0/2445487009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265c004810 msgr2=0x7f265c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 --2- 172.21.15.83:0/2445487009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265c004810 0x7f265c004c90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2660009ae0 tx=0x7f266001ca70 comp rx=0 tx=0).stop 2024-04-25T00:12:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 -- 172.21.15.83:0/2445487009 shutdown_connections 2024-04-25T00:12:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 --2- 172.21.15.83:0/2445487009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265c004810 0x7f265c004c90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 -- 172.21.15.83:0/2445487009 >> 172.21.15.83:0/2445487009 conn(0x7f267806b3b0 msgr2=0x7f267806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 -- 172.21.15.83:0/2445487009 shutdown_connections 2024-04-25T00:12:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 -- 172.21.15.83:0/2445487009 wait complete. 2024-04-25T00:12:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 Processor -- start 2024-04-25T00:12:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 -- start start 2024-04-25T00:12:41.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f265c004810 0x7f26781546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2678157940 0x7f26781e7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2678071e10 0x7f2678072290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26781459b0 con 0x7f2678157940 2024-04-25T00:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26780cf190 con 0x7f2678071e10 2024-04-25T00:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267f400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2678145b30 con 0x7f265c004810 2024-04-25T00:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f266fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2678157940 0x7f26781e7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f266fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2678157940 0x7f26781e7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44842/0 (socket says 172.21.15.83:44842) 2024-04-25T00:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f266fe00640 1 -- 172.21.15.83:0/3920405759 learned_addr learned my addr 172.21.15.83:0/3920405759 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267d400640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2678071e10 0x7f2678072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.761+0000 7f267ca00640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f265c004810 0x7f26781546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f266fe00640 1 -- 172.21.15.83:0/3920405759 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f265c004810 msgr2=0x7f26781546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f266fe00640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f265c004810 0x7f26781546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f266fe00640 1 -- 172.21.15.83:0/3920405759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2678071e10 msgr2=0x7f2678072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f266fe00640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2678071e10 0x7f2678072290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f266fe00640 1 -- 172.21.15.83:0/3920405759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26781e8110 con 0x7f2678157940 2024-04-25T00:12:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f267ca00640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f265c004810 0x7f26781546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f267d400640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2678071e10 0x7f2678072290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f266fe00640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2678157940 0x7f26781e7800 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f26600b0040 tx=0x7f2660019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f2675600640 1 -- 172.21.15.83:0/3920405759 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26600b2040 con 0x7f2678157940 2024-04-25T00:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f2675600640 1 -- 172.21.15.83:0/3920405759 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26600a5e20 con 0x7f2678157940 2024-04-25T00:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f2675600640 1 -- 172.21.15.83:0/3920405759 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660004260 con 0x7f2678157940 2024-04-25T00:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f267f400640 1 -- 172.21.15.83:0/3920405759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26781e83f0 con 0x7f2678157940 2024-04-25T00:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f267f400640 1 -- 172.21.15.83:0/3920405759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26781e89b0 con 0x7f2678157940 2024-04-25T00:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f267f400640 1 -- 172.21.15.83:0/3920405759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26780736f0 con 0x7f2678157940 2024-04-25T00:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f2675600640 1 -- 172.21.15.83:0/3920405759 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f26600bf060 con 0x7f2678157940 2024-04-25T00:12:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.765+0000 7f2675600640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f26540678c0 0x7f2654069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.769+0000 7f2675600640 1 -- 172.21.15.83:0/3920405759 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5749+0+0 (secure 0 0 0) 0x7f266011f790 con 0x7f2678157940 2024-04-25T00:12:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.769+0000 7f267ca00640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f26540678c0 0x7f2654069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.769+0000 7f267ca00640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f26540678c0 0x7f2654069d80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2668007c00 tx=0x7f2668009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:41.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.773+0000 7f2675600640 1 -- 172.21.15.83:0/3920405759 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2660016780 con 0x7f2678157940 2024-04-25T00:12:41.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:41.929+0000 7f267f400640 1 -- 172.21.15.83:0/3920405759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f267804f5f0 con 0x7f2678157940 2024-04-25T00:12:42.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.585+0000 7f2675600640 1 -- 172.21.15.83:0/3920405759 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) ==== 169+0+0 (secure 0 0 0) 0x7f26600f0200 con 0x7f2678157940 2024-04-25T00:12:42.588 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-25T00:12:42.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.589+0000 7f267f400640 1 -- 172.21.15.83:0/3920405759 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f26540678c0 msgr2=0x7f2654069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:42.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.589+0000 7f267f400640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f26540678c0 0x7f2654069d80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2668007c00 tx=0x7f2668009040 comp rx=0 tx=0).stop 2024-04-25T00:12:42.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.589+0000 7f267f400640 1 -- 172.21.15.83:0/3920405759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2678157940 msgr2=0x7f26781e7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:42.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.589+0000 7f267f400640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2678157940 0x7f26781e7800 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f26600b0040 tx=0x7f2660019810 comp rx=0 tx=0).stop 2024-04-25T00:12:42.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.589+0000 7f267f400640 1 -- 172.21.15.83:0/3920405759 shutdown_connections 2024-04-25T00:12:42.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.589+0000 7f267f400640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f26540678c0 0x7f2654069d80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.589+0000 7f267f400640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2678071e10 0x7f2678072290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.589+0000 7f267f400640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2678157940 0x7f26781e7800 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.589+0000 7f267f400640 1 --2- 172.21.15.83:0/3920405759 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f265c004810 0x7f26781546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.589+0000 7f267f400640 1 -- 172.21.15.83:0/3920405759 >> 172.21.15.83:0/3920405759 conn(0x7f267806b3b0 msgr2=0x7f267815a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:42.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.589+0000 7f267f400640 1 -- 172.21.15.83:0/3920405759 shutdown_connections 2024-04-25T00:12:42.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.589+0000 7f267f400640 1 -- 172.21.15.83:0/3920405759 wait complete. 2024-04-25T00:12:42.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:12:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.753+0000 7fc954800640 1 Processor -- start 2024-04-25T00:12:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc954800640 1 -- start start 2024-04-25T00:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc954800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc94c15a330 0x7fc94c0b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc954800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94c15a710 0x7fc94c070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc954800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc94c06ddc0 0x7fc94c06e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc954800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc94c0cc690 con 0x7fc94c15a330 2024-04-25T00:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc954800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc94c071850 con 0x7fc94c06ddc0 2024-04-25T00:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc954800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc94c0b41b0 con 0x7fc94c15a710 2024-04-25T00:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc951e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc94c15a330 0x7fc94c0b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc951e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc94c15a330 0x7fc94c0b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44854/0 (socket says 172.21.15.83:44854) 2024-04-25T00:12:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc951e00640 1 -- 172.21.15.83:0/304560094 learned_addr learned my addr 172.21.15.83:0/304560094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc951e00640 1 -- 172.21.15.83:0/304560094 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94c15a710 msgr2=0x7fc94c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc951e00640 1 --2- 172.21.15.83:0/304560094 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94c15a710 0x7fc94c070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc951e00640 1 -- 172.21.15.83:0/304560094 >> v2:172.21.15.86:3300/0 conn(0x7fc94c06ddc0 msgr2=0x7fc94c06e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc951e00640 1 --2- 172.21.15.83:0/304560094 >> v2:172.21.15.86:3300/0 conn(0x7fc94c06ddc0 0x7fc94c06e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.757+0000 7fc951e00640 1 -- 172.21.15.83:0/304560094 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c071ac0 con 0x7fc94c15a330 2024-04-25T00:12:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc951e00640 1 --2- 172.21.15.83:0/304560094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c15a330 0x7fc94c0b4a40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0af750 tx=0x7fc94001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98f98c57eb2fb4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc950a00640 1 -- 172.21.15.83:0/304560094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9400b5070 con 0x7fc94c15a330 2024-04-25T00:12:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc950a00640 1 -- 172.21.15.83:0/304560094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9400a89d0 con 0x7fc94c15a330 2024-04-25T00:12:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc950a00640 1 -- 172.21.15.83:0/304560094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9400a8da0 con 0x7fc94c15a330 2024-04-25T00:12:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc954800640 1 -- 172.21.15.83:0/304560094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c15a330 msgr2=0x7fc94c0b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc954800640 1 --2- 172.21.15.83:0/304560094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c15a330 0x7fc94c0b4a40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0af750 tx=0x7fc94001ed10 comp rx=0 tx=0).stop 2024-04-25T00:12:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc954800640 1 -- 172.21.15.83:0/304560094 shutdown_connections 2024-04-25T00:12:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc954800640 1 --2- 172.21.15.83:0/304560094 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94c15a710 0x7fc94c070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc954800640 1 --2- 172.21.15.83:0/304560094 >> v2:172.21.15.86:3300/0 conn(0x7fc94c06ddc0 0x7fc94c06e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc954800640 1 --2- 172.21.15.83:0/304560094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c15a330 0x7fc94c0b4a40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc954800640 1 -- 172.21.15.83:0/304560094 >> 172.21.15.83:0/304560094 conn(0x7fc94c0b1830 msgr2=0x7fc94c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:42.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc954800640 1 -- 172.21.15.83:0/304560094 shutdown_connections 2024-04-25T00:12:42.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc954800640 1 -- 172.21.15.83:0/304560094 wait complete. 2024-04-25T00:12:42.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc954800640 1 Processor -- start 2024-04-25T00:12:42.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.761+0000 7fc954800640 1 -- start start 2024-04-25T00:12:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc954800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c06ddc0 0x7fc94c0ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc954800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94c15a330 0x7fc94c0c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc954800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94c14f9f0 0x7fc94c0c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc954800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c06e750 con 0x7fc94c06ddc0 2024-04-25T00:12:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc954800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c0b4490 con 0x7fc94c14f9f0 2024-04-25T00:12:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc954800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc94c06e8d0 con 0x7fc94c15a330 2024-04-25T00:12:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc951e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c06ddc0 0x7fc94c0ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc951400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94c15a330 0x7fc94c0c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc951e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c06ddc0 0x7fc94c0ab6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44868/0 (socket says 172.21.15.83:44868) 2024-04-25T00:12:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc951400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94c15a330 0x7fc94c0c5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:56492/0 (socket says 172.21.15.83:56492) 2024-04-25T00:12:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc952800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94c14f9f0 0x7fc94c0c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc951e00640 1 -- 172.21.15.83:0/379230579 learned_addr learned my addr 172.21.15.83:0/379230579 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc951e00640 1 -- 172.21.15.83:0/379230579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94c15a330 msgr2=0x7fc94c0c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc951e00640 1 --2- 172.21.15.83:0/379230579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94c15a330 0x7fc94c0c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc951e00640 1 -- 172.21.15.83:0/379230579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94c14f9f0 msgr2=0x7fc94c0c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc951e00640 1 --2- 172.21.15.83:0/379230579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94c14f9f0 0x7fc94c0c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc951e00640 1 -- 172.21.15.83:0/379230579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c0abd30 con 0x7fc94c06ddc0 2024-04-25T00:12:42.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc951e00640 1 --2- 172.21.15.83:0/379230579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c06ddc0 0x7fc94c0ab6b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc9400b3040 tx=0x7fc94001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:42.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc946000640 1 -- 172.21.15.83:0/379230579 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9400c0070 con 0x7fc94c06ddc0 2024-04-25T00:12:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc946000640 1 -- 172.21.15.83:0/379230579 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9400b08d0 con 0x7fc94c06ddc0 2024-04-25T00:12:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc954800640 1 -- 172.21.15.83:0/379230579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94c0ac010 con 0x7fc94c06ddc0 2024-04-25T00:12:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc946000640 1 -- 172.21.15.83:0/379230579 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9400b5040 con 0x7fc94c06ddc0 2024-04-25T00:12:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc954800640 1 -- 172.21.15.83:0/379230579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94c0ac5d0 con 0x7fc94c06ddc0 2024-04-25T00:12:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc954800640 1 -- 172.21.15.83:0/379230579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc94c04f510 con 0x7fc94c06ddc0 2024-04-25T00:12:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.765+0000 7fc946000640 1 -- 172.21.15.83:0/379230579 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc9400c3070 con 0x7fc94c06ddc0 2024-04-25T00:12:42.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.769+0000 7fc946000640 1 --2- 172.21.15.83:0/379230579 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc9300678c0 0x7fc930069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:42.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.769+0000 7fc946000640 1 -- 172.21.15.83:0/379230579 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7fc940122a40 con 0x7fc94c06ddc0 2024-04-25T00:12:42.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.769+0000 7fc951400640 1 --2- 172.21.15.83:0/379230579 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc9300678c0 0x7fc930069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:42.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.769+0000 7fc951400640 1 --2- 172.21.15.83:0/379230579 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc9300678c0 0x7fc930069d80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc94c069490 tx=0x7fc93c009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:42.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.773+0000 7fc946000640 1 -- 172.21.15.83:0/379230579 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc940018870 con 0x7fc94c06ddc0 2024-04-25T00:12:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.925+0000 7fc954800640 1 -- 172.21.15.83:0/379230579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc94c0cfd10 con 0x7fc94c06ddc0 2024-04-25T00:12:42.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.929+0000 7fc946000640 1 -- 172.21.15.83:0/379230579 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v12) ==== 104+0+0 (secure 0 0 0) 0x7fc9400f3000 con 0x7fc94c06ddc0 2024-04-25T00:12:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.933+0000 7fc954800640 1 -- 172.21.15.83:0/379230579 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc9300678c0 msgr2=0x7fc930069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.933+0000 7fc954800640 1 --2- 172.21.15.83:0/379230579 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc9300678c0 0x7fc930069d80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc94c069490 tx=0x7fc93c009110 comp rx=0 tx=0).stop 2024-04-25T00:12:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.933+0000 7fc954800640 1 -- 172.21.15.83:0/379230579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c06ddc0 msgr2=0x7fc94c0ab6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.933+0000 7fc954800640 1 --2- 172.21.15.83:0/379230579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c06ddc0 0x7fc94c0ab6b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc9400b3040 tx=0x7fc94001bab0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.933+0000 7fc954800640 1 -- 172.21.15.83:0/379230579 shutdown_connections 2024-04-25T00:12:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.933+0000 7fc954800640 1 --2- 172.21.15.83:0/379230579 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc9300678c0 0x7fc930069d80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.933+0000 7fc954800640 1 --2- 172.21.15.83:0/379230579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94c15a330 0x7fc94c0c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.933+0000 7fc954800640 1 --2- 172.21.15.83:0/379230579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94c14f9f0 0x7fc94c0c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.933+0000 7fc954800640 1 --2- 172.21.15.83:0/379230579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c06ddc0 0x7fc94c0ab6b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.933+0000 7fc954800640 1 -- 172.21.15.83:0/379230579 >> 172.21.15.83:0/379230579 conn(0x7fc94c0b1830 msgr2=0x7fc94c06d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:42.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.933+0000 7fc954800640 1 -- 172.21.15.83:0/379230579 shutdown_connections 2024-04-25T00:12:42.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:42.933+0000 7fc954800640 1 -- 172.21.15.83:0/379230579 wait complete. 2024-04-25T00:12:42.954 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds ms_async_reap_threshold 2024-04-25T00:12:43.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.089+0000 7fd9bdc00640 1 Processor -- start 2024-04-25T00:12:43.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.089+0000 7fd9bdc00640 1 -- start start 2024-04-25T00:12:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.089+0000 7fd9bdc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b81551e0 0x7fd9b815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.089+0000 7fd9bdc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd9b806e4c0 0x7fd9b806e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.089+0000 7fd9bdc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9b815c560 con 0x7fd9b81555c0 2024-04-25T00:12:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.089+0000 7fd9bdc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd9b806ee70 con 0x7fd9b806e4c0 2024-04-25T00:12:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.089+0000 7fd9bdc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b8072460 con 0x7fd9b81551e0 2024-04-25T00:12:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.093+0000 7fd9b7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd9b806e4c0 0x7fd9b806e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.093+0000 7fd9b6a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b81551e0 0x7fd9b815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.093+0000 7fd9ae000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd9b81555c0 0x7fd9b8071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:35886/0 (socket says 172.21.15.83:35886) 2024-04-25T00:12:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.093+0000 7fd9ae000640 1 -- 172.21.15.83:0/2778499446 learned_addr learned my addr 172.21.15.83:0/2778499446 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.093+0000 7fd9b6000640 1 -- 172.21.15.83:0/2778499446 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3937735951 0 0) 0x7fd9b815c560 con 0x7fd9b81555c0 2024-04-25T00:12:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.093+0000 7fd9b6000640 1 -- 172.21.15.83:0/2778499446 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd99c003530 con 0x7fd9b81555c0 2024-04-25T00:12:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.093+0000 7fd9b7400640 1 -- 172.21.15.83:0/2778499446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b81551e0 msgr2=0x7fd9b815bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.093+0000 7fd9b7400640 1 --2- 172.21.15.83:0/2778499446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b81551e0 0x7fd9b815bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.093+0000 7fd9b7400640 1 -- 172.21.15.83:0/2778499446 >> v1:172.21.15.86:6789/0 conn(0x7fd9b81555c0 legacy=0x7fd9b8071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.093+0000 7fd9b7400640 1 -- 172.21.15.83:0/2778499446 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b815c7d0 con 0x7fd9b806e4c0 2024-04-25T00:12:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.093+0000 7fd9b7400640 1 --2- 172.21.15.83:0/2778499446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b806e4c0 0x7fd9b806e930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009d90 tx=0x7fd9b001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf59b589b7d6e95c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9b6000640 1 -- 172.21.15.83:0/2778499446 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b00b2070 con 0x7fd9b806e4c0 2024-04-25T00:12:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9b6000640 1 -- 172.21.15.83:0/2778499446 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b00a5940 con 0x7fd9b806e4c0 2024-04-25T00:12:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9b6000640 1 -- 172.21.15.83:0/2778499446 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b00a5d10 con 0x7fd9b806e4c0 2024-04-25T00:12:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 -- 172.21.15.83:0/2778499446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b806e4c0 msgr2=0x7fd9b806e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 --2- 172.21.15.83:0/2778499446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b806e4c0 0x7fd9b806e930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009d90 tx=0x7fd9b001c970 comp rx=0 tx=0).stop 2024-04-25T00:12:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 -- 172.21.15.83:0/2778499446 shutdown_connections 2024-04-25T00:12:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 --2- 172.21.15.83:0/2778499446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b806e4c0 0x7fd9b806e930 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 --2- 172.21.15.83:0/2778499446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b81551e0 0x7fd9b815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 -- 172.21.15.83:0/2778499446 >> 172.21.15.83:0/2778499446 conn(0x7fd9b80b1830 msgr2=0x7fd9b80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:43.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 -- 172.21.15.83:0/2778499446 shutdown_connections 2024-04-25T00:12:43.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 -- 172.21.15.83:0/2778499446 wait complete. 2024-04-25T00:12:43.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 Processor -- start 2024-04-25T00:12:43.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 -- start start 2024-04-25T00:12:43.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b806e4c0 0x7fd9b81423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:43.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9b81551e0 0x7fd9b8152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:43.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b81555c0 0x7fd9b81535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:43.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b815db70 con 0x7fd9b81551e0 2024-04-25T00:12:43.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b815d9f0 con 0x7fd9b806e4c0 2024-04-25T00:12:43.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9bdc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b815dcf0 con 0x7fd9b81555c0 2024-04-25T00:12:43.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9ae000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9b81551e0 0x7fd9b8152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:43.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9ae000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9b81551e0 0x7fd9b8152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44874/0 (socket says 172.21.15.83:44874) 2024-04-25T00:12:43.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9ae000640 1 -- 172.21.15.83:0/3821385935 learned_addr learned my addr 172.21.15.83:0/3821385935 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:43.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9b6a00640 1 --2- 172.21.15.83:0/3821385935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b806e4c0 0x7fd9b81423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:43.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.097+0000 7fd9b7400640 1 --2- 172.21.15.83:0/3821385935 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b81555c0 0x7fd9b81535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:43.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9ae000640 1 -- 172.21.15.83:0/3821385935 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b81555c0 msgr2=0x7fd9b81535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:43.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9ae000640 1 --2- 172.21.15.83:0/3821385935 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b81555c0 0x7fd9b81535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:43.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9ae000640 1 -- 172.21.15.83:0/3821385935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b806e4c0 msgr2=0x7fd9b81423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:43.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9ae000640 1 --2- 172.21.15.83:0/3821385935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b806e4c0 0x7fd9b81423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:43.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9ae000640 1 -- 172.21.15.83:0/3821385935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b80cd1d0 con 0x7fd9b81551e0 2024-04-25T00:12:43.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9ae000640 1 --2- 172.21.15.83:0/3821385935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9b81551e0 0x7fd9b8152c50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000c3f0 tx=0x7fd9a000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:43.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9af400640 1 -- 172.21.15.83:0/3821385935 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0017070 con 0x7fd9b81551e0 2024-04-25T00:12:43.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9af400640 1 -- 172.21.15.83:0/3821385935 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a0005cf0 con 0x7fd9b81551e0 2024-04-25T00:12:43.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9af400640 1 -- 172.21.15.83:0/3821385935 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00072f0 con 0x7fd9b81551e0 2024-04-25T00:12:43.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9bdc00640 1 -- 172.21.15.83:0/3821385935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b80cd4b0 con 0x7fd9b81551e0 2024-04-25T00:12:43.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9bdc00640 1 -- 172.21.15.83:0/3821385935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b80cda70 con 0x7fd9b81551e0 2024-04-25T00:12:43.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9bdc00640 1 -- 172.21.15.83:0/3821385935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b804f510 con 0x7fd9b81551e0 2024-04-25T00:12:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9af400640 1 -- 172.21.15.83:0/3821385935 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd9a001a070 con 0x7fd9b81551e0 2024-04-25T00:12:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9af400640 1 --2- 172.21.15.83:0/3821385935 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd9940678c0 0x7fd994069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9af400640 1 -- 172.21.15.83:0/3821385935 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7fd9a0084f30 con 0x7fd9b81551e0 2024-04-25T00:12:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.101+0000 7fd9b6a00640 1 --2- 172.21.15.83:0/3821385935 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd9940678c0 0x7fd994069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.105+0000 7fd9b6a00640 1 --2- 172.21.15.83:0/3821385935 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd9940678c0 0x7fd994069d80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009b60 tx=0x7fd9a8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.105+0000 7fd9af400640 1 -- 172.21.15.83:0/3821385935 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd9a00554f0 con 0x7fd9b81551e0 2024-04-25T00:12:43.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.269+0000 7fd9bdc00640 1 -- 172.21.15.83:0/3821385935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"} v 0) -- 0x7fd9b806ce30 con 0x7fd9b81551e0 2024-04-25T00:12:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.273+0000 7fd9af400640 1 -- 172.21.15.83:0/3821385935 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"}]=0 v13) ==== 106+0+0 (secure 0 0 0) 0x7fd9a00597f0 con 0x7fd9b81551e0 2024-04-25T00:12:43.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.277+0000 7fd9bdc00640 1 -- 172.21.15.83:0/3821385935 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd9940678c0 msgr2=0x7fd994069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:43.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.277+0000 7fd9bdc00640 1 --2- 172.21.15.83:0/3821385935 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd9940678c0 0x7fd994069d80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009b60 tx=0x7fd9a8009190 comp rx=0 tx=0).stop 2024-04-25T00:12:43.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.277+0000 7fd9bdc00640 1 -- 172.21.15.83:0/3821385935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9b81551e0 msgr2=0x7fd9b8152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:43.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.277+0000 7fd9bdc00640 1 --2- 172.21.15.83:0/3821385935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9b81551e0 0x7fd9b8152c50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000c3f0 tx=0x7fd9a000c8b0 comp rx=0 tx=0).stop 2024-04-25T00:12:43.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.277+0000 7fd9bdc00640 1 -- 172.21.15.83:0/3821385935 shutdown_connections 2024-04-25T00:12:43.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.277+0000 7fd9bdc00640 1 --2- 172.21.15.83:0/3821385935 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd9940678c0 0x7fd994069d80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:43.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.277+0000 7fd9bdc00640 1 --2- 172.21.15.83:0/3821385935 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b81555c0 0x7fd9b81535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:43.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.277+0000 7fd9bdc00640 1 --2- 172.21.15.83:0/3821385935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9b81551e0 0x7fd9b8152c50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:43.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.277+0000 7fd9bdc00640 1 --2- 172.21.15.83:0/3821385935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b806e4c0 0x7fd9b81423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:43.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.277+0000 7fd9bdc00640 1 -- 172.21.15.83:0/3821385935 >> 172.21.15.83:0/3821385935 conn(0x7fd9b80b1830 msgr2=0x7fd9b815a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:43.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.277+0000 7fd9bdc00640 1 -- 172.21.15.83:0/3821385935 shutdown_connections 2024-04-25T00:12:43.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.277+0000 7fd9bdc00640 1 -- 172.21.15.83:0/3821385935 wait complete. 2024-04-25T00:12:43.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close' 2024-04-25T00:12:43.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.437+0000 7f0cfec00640 1 Processor -- start 2024-04-25T00:12:43.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.437+0000 7f0cfec00640 1 -- start start 2024-04-25T00:12:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.441+0000 7f0cfec00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0cf80b4360 0x7f0cf80b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.441+0000 7f0cfec00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cf806ef80 con 0x7f0cf80b2b30 2024-04-25T00:12:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.441+0000 7f0cfec00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cf815b510 con 0x7f0cf81553e0 2024-04-25T00:12:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.441+0000 7f0cfec00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0cf815b7f0 con 0x7f0cf80b4360 2024-04-25T00:12:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.441+0000 7f0cef400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0cf80b4360 0x7f0cf80b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.441+0000 7f0cfcc00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0cf80b2b30 0x7f0cf80b2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57310/0 (socket says 172.21.15.83:57310) 2024-04-25T00:12:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.441+0000 7f0cfcc00640 1 -- 172.21.15.83:0/595648089 learned_addr learned my addr 172.21.15.83:0/595648089 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.441+0000 7f0cef400640 1 -- 172.21.15.83:0/595648089 >> v1:172.21.15.86:6789/0 conn(0x7f0cf81553e0 legacy=0x7f0cf806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.441+0000 7f0cef400640 1 -- 172.21.15.83:0/595648089 >> v1:172.21.15.83:6789/0 conn(0x7f0cf80b2b30 legacy=0x7f0cf80b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.441+0000 7f0cef400640 1 -- 172.21.15.83:0/595648089 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf815bad0 con 0x7f0cf80b4360 2024-04-25T00:12:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.441+0000 7f0cf7400640 1 -- 172.21.15.83:0/595648089 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1006372042 0 0) 0x7f0cf806ef80 con 0x7f0cf80b2b30 2024-04-25T00:12:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cef400640 1 --2- 172.21.15.83:0/595648089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf80b4360 0x7f0cf80b47e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00097c0 tx=0x7f0ce001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa72e8b3124d98c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:43.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cf7400640 1 -- 172.21.15.83:0/595648089 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00b0070 con 0x7f0cf80b4360 2024-04-25T00:12:43.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cf7400640 1 -- 172.21.15.83:0/595648089 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce00ab600 con 0x7f0cf80b4360 2024-04-25T00:12:43.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cf7400640 1 -- 172.21.15.83:0/595648089 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00ab9d0 con 0x7f0cf80b4360 2024-04-25T00:12:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 -- 172.21.15.83:0/595648089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf80b4360 msgr2=0x7f0cf80b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 --2- 172.21.15.83:0/595648089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf80b4360 0x7f0cf80b47e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00097c0 tx=0x7f0ce001c410 comp rx=0 tx=0).stop 2024-04-25T00:12:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 -- 172.21.15.83:0/595648089 shutdown_connections 2024-04-25T00:12:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 --2- 172.21.15.83:0/595648089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf80b4360 0x7f0cf80b47e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 -- 172.21.15.83:0/595648089 >> 172.21.15.83:0/595648089 conn(0x7f0cf80b1830 msgr2=0x7f0cf80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 -- 172.21.15.83:0/595648089 shutdown_connections 2024-04-25T00:12:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 -- 172.21.15.83:0/595648089 wait complete. 2024-04-25T00:12:43.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 Processor -- start 2024-04-25T00:12:43.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 -- start start 2024-04-25T00:12:43.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf80b2b30 0x7f0cf80ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:43.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cf80b4360 0x7f0cf813e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0cf81553e0 0x7f0cf80c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8155940 con 0x7f0cf80b2b30 2024-04-25T00:12:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81557c0 con 0x7f0cf80b4360 2024-04-25T00:12:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.445+0000 7f0cfec00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf806ebd0 con 0x7f0cf81553e0 2024-04-25T00:12:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cf7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf80b2b30 0x7f0cf80ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cf7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf80b2b30 0x7f0cf80ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44886/0 (socket says 172.21.15.83:44886) 2024-04-25T00:12:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cf7e00640 1 -- 172.21.15.83:0/1963795569 learned_addr learned my addr 172.21.15.83:0/1963795569 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cef400640 1 --2- 172.21.15.83:0/1963795569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cf80b4360 0x7f0cf813e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cfcc00640 1 --2- 172.21.15.83:0/1963795569 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0cf81553e0 0x7f0cf80c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cf7e00640 1 -- 172.21.15.83:0/1963795569 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0cf81553e0 msgr2=0x7f0cf80c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cf7e00640 1 --2- 172.21.15.83:0/1963795569 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0cf81553e0 0x7f0cf80c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cf7e00640 1 -- 172.21.15.83:0/1963795569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cf80b4360 msgr2=0x7f0cf813e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cf7e00640 1 --2- 172.21.15.83:0/1963795569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cf80b4360 0x7f0cf813e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cf7e00640 1 -- 172.21.15.83:0/1963795569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf80c55e0 con 0x7f0cf80b2b30 2024-04-25T00:12:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cf7e00640 1 --2- 172.21.15.83:0/1963795569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf80b2b30 0x7f0cf80ceea0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800c500 tx=0x7f0ce800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cf4c00640 1 -- 172.21.15.83:0/1963795569 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8016020 con 0x7f0cf80b2b30 2024-04-25T00:12:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cf4c00640 1 -- 172.21.15.83:0/1963795569 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce8005220 con 0x7f0cf80b2b30 2024-04-25T00:12:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cf4c00640 1 -- 172.21.15.83:0/1963795569 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8007030 con 0x7f0cf80b2b30 2024-04-25T00:12:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cfec00640 1 -- 172.21.15.83:0/1963795569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf80c58c0 con 0x7f0cf80b2b30 2024-04-25T00:12:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cfec00640 1 -- 172.21.15.83:0/1963795569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf8150ef0 con 0x7f0cf80b2b30 2024-04-25T00:12:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cfec00640 1 -- 172.21.15.83:0/1963795569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf81511e0 con 0x7f0cf80b2b30 2024-04-25T00:12:43.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.449+0000 7f0cf4c00640 1 -- 172.21.15.83:0/1963795569 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0ce80071f0 con 0x7f0cf80b2b30 2024-04-25T00:12:43.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.453+0000 7f0cf4c00640 1 --2- 172.21.15.83:0/1963795569 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0cd40678c0 0x7f0cd4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:43.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.453+0000 7f0cf4c00640 1 -- 172.21.15.83:0/1963795569 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7f0ce8085520 con 0x7f0cf80b2b30 2024-04-25T00:12:43.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.453+0000 7f0cef400640 1 --2- 172.21.15.83:0/1963795569 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0cd40678c0 0x7f0cd4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:43.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.453+0000 7f0cef400640 1 --2- 172.21.15.83:0/1963795569 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0cd40678c0 0x7f0cd4069d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00ad000 tx=0x7f0ce00a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:43.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.457+0000 7f0cf4c00640 1 -- 172.21.15.83:0/1963795569 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0ce8020020 con 0x7f0cf80b2b30 2024-04-25T00:12:43.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:43.665+0000 7f0cfec00640 1 -- 172.21.15.83:0/1963795569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]} v 0) -- 0x7f0cf80737f0 con 0x7f0cf80b2b30 2024-04-25T00:12:44.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.593+0000 7f0cf4c00640 1 -- 172.21.15.83:0/1963795569 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]}]=0 v121) ==== 144+0+0 (secure 0 0 0) 0x7f0ce8056140 con 0x7f0cf80b2b30 2024-04-25T00:12:44.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.597+0000 7f0cfec00640 1 -- 172.21.15.83:0/1963795569 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0cd40678c0 msgr2=0x7f0cd4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.597+0000 7f0cfec00640 1 --2- 172.21.15.83:0/1963795569 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0cd40678c0 0x7f0cd4069d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00ad000 tx=0x7f0ce00a6000 comp rx=0 tx=0).stop 2024-04-25T00:12:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.597+0000 7f0cfec00640 1 -- 172.21.15.83:0/1963795569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf80b2b30 msgr2=0x7f0cf80ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.597+0000 7f0cfec00640 1 --2- 172.21.15.83:0/1963795569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf80b2b30 0x7f0cf80ceea0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800c500 tx=0x7f0ce800c9c0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.597+0000 7f0cfec00640 1 -- 172.21.15.83:0/1963795569 shutdown_connections 2024-04-25T00:12:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.597+0000 7f0cfec00640 1 --2- 172.21.15.83:0/1963795569 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0cd40678c0 0x7f0cd4069d80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.597+0000 7f0cfec00640 1 --2- 172.21.15.83:0/1963795569 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0cf81553e0 0x7f0cf80c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.597+0000 7f0cfec00640 1 --2- 172.21.15.83:0/1963795569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cf80b4360 0x7f0cf813e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.597+0000 7f0cfec00640 1 --2- 172.21.15.83:0/1963795569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf80b2b30 0x7f0cf80ceea0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.597+0000 7f0cfec00640 1 -- 172.21.15.83:0/1963795569 >> 172.21.15.83:0/1963795569 conn(0x7f0cf80b1830 msgr2=0x7f0cf815c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.597+0000 7f0cfec00640 1 -- 172.21.15.83:0/1963795569 shutdown_connections 2024-04-25T00:12:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.597+0000 7f0cfec00640 1 -- 172.21.15.83:0/1963795569 wait complete. 2024-04-25T00:12:44.622 INFO:tasks.cephfs_test_runner:That when a client unmounts, the thread count on the MDS goes back ... ok 2024-04-25T00:12:44.622 INFO:tasks.cephfs_test_runner:Starting test: test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) Check that mds evicts blocklisted client 2024-04-25T00:12:44.622 INFO:tasks.cephfs_test_runner:test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-04-25T00:12:44.623 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:44.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f4800640 1 Processor -- start 2024-04-25T00:12:44.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f4800640 1 -- start start 2024-04-25T00:12:44.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f4800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f63ec06fc10 0x7f63ec0cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:44.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f4800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f63ec1452c0 0x7f63ec0cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:44.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f4800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63ec0d1850 con 0x7f63ec06b1e0 2024-04-25T00:12:44.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f4800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f63ec0d1b30 con 0x7f63ec06fc10 2024-04-25T00:12:44.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f4800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f63ec0cd710 con 0x7f63ec1452c0 2024-04-25T00:12:44.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f1e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f63ec06b1e0 0x7f63ec06b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55802/0 (socket says 172.21.15.83:55802) 2024-04-25T00:12:44.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f1e00640 1 -- 172.21.15.83:0/3914412127 learned_addr learned my addr 172.21.15.83:0/3914412127 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:44.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f1400640 1 --2- 172.21.15.83:0/3914412127 >> v2:172.21.15.83:3300/0 conn(0x7f63ec06fc10 0x7f63ec0cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:44.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f2800640 1 --2- 172.21.15.83:0/3914412127 >> v2:172.21.15.86:3300/0 conn(0x7f63ec1452c0 0x7f63ec0cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:44.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f1400640 1 -- 172.21.15.83:0/3914412127 >> v2:172.21.15.86:3300/0 conn(0x7f63ec1452c0 msgr2=0x7f63ec0cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:44.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f1400640 1 --2- 172.21.15.83:0/3914412127 >> v2:172.21.15.86:3300/0 conn(0x7f63ec1452c0 0x7f63ec0cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f1400640 1 -- 172.21.15.83:0/3914412127 >> v1:172.21.15.83:6789/0 conn(0x7f63ec06b1e0 legacy=0x7f63ec06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f0a00640 1 -- 172.21.15.83:0/3914412127 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3236078878 0 0) 0x7f63ec0d1850 con 0x7f63ec06b1e0 2024-04-25T00:12:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.761+0000 7f63f1400640 1 -- 172.21.15.83:0/3914412127 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec0cd9f0 con 0x7f63ec06fc10 2024-04-25T00:12:44.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.765+0000 7f63f1400640 1 --2- 172.21.15.83:0/3914412127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec06fc10 0x7f63ec0cf260 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f63ec071dd0 tx=0x7f63dc01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e716720548ed85e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:44.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.765+0000 7f63f0a00640 1 -- 172.21.15.83:0/3914412127 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0b1070 con 0x7f63ec06fc10 2024-04-25T00:12:44.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.765+0000 7f63f0a00640 1 -- 172.21.15.83:0/3914412127 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63dc0a47f0 con 0x7f63ec06fc10 2024-04-25T00:12:44.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.765+0000 7f63f0a00640 1 -- 172.21.15.83:0/3914412127 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0a4bc0 con 0x7f63ec06fc10 2024-04-25T00:12:44.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.765+0000 7f63f4800640 1 -- 172.21.15.83:0/3914412127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec06fc10 msgr2=0x7f63ec0cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:44.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.765+0000 7f63f4800640 1 --2- 172.21.15.83:0/3914412127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec06fc10 0x7f63ec0cf260 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f63ec071dd0 tx=0x7f63dc01c040 comp rx=0 tx=0).stop 2024-04-25T00:12:44.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.765+0000 7f63f4800640 1 -- 172.21.15.83:0/3914412127 shutdown_connections 2024-04-25T00:12:44.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.765+0000 7f63f4800640 1 --2- 172.21.15.83:0/3914412127 >> v2:172.21.15.86:3300/0 conn(0x7f63ec1452c0 0x7f63ec0cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.765+0000 7f63f4800640 1 --2- 172.21.15.83:0/3914412127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec06fc10 0x7f63ec0cf260 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.765+0000 7f63f4800640 1 -- 172.21.15.83:0/3914412127 >> 172.21.15.83:0/3914412127 conn(0x7f63ec0d0b90 msgr2=0x7f63ec0d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:44.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.765+0000 7f63f4800640 1 -- 172.21.15.83:0/3914412127 shutdown_connections 2024-04-25T00:12:44.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.765+0000 7f63f4800640 1 -- 172.21.15.83:0/3914412127 wait complete. 2024-04-25T00:12:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f4800640 1 Processor -- start 2024-04-25T00:12:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f4800640 1 -- start start 2024-04-25T00:12:44.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f4800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec06b1e0 0x7f63ec0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:44.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f4800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63ec06fc10 0x7f63ec164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:44.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f4800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63ec1452c0 0x7f63ec0b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:44.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f4800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec169ad0 con 0x7f63ec06b1e0 2024-04-25T00:12:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f4800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec169950 con 0x7f63ec06fc10 2024-04-25T00:12:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f4800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f63ec169c50 con 0x7f63ec1452c0 2024-04-25T00:12:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec06b1e0 0x7f63ec0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec06b1e0 0x7f63ec0b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54744/0 (socket says 172.21.15.83:54744) 2024-04-25T00:12:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f1e00640 1 -- 172.21.15.83:0/245504166 learned_addr learned my addr 172.21.15.83:0/245504166 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f1e00640 1 -- 172.21.15.83:0/245504166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63ec1452c0 msgr2=0x7f63ec0b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f1e00640 1 --2- 172.21.15.83:0/245504166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63ec1452c0 0x7f63ec0b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f1e00640 1 -- 172.21.15.83:0/245504166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63ec06fc10 msgr2=0x7f63ec164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f1400640 1 --2- 172.21.15.83:0/245504166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63ec06fc10 0x7f63ec164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f1e00640 1 --2- 172.21.15.83:0/245504166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63ec06fc10 0x7f63ec164dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f1e00640 1 -- 172.21.15.83:0/245504166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec15daa0 con 0x7f63ec06b1e0 2024-04-25T00:12:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f1400640 1 --2- 172.21.15.83:0/245504166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63ec06fc10 0x7f63ec164dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f1e00640 1 --2- 172.21.15.83:0/245504166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec06b1e0 0x7f63ec0b1080 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f63e000c400 tx=0x7f63e000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63e6000640 1 -- 172.21.15.83:0/245504166 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e00074e0 con 0x7f63ec06b1e0 2024-04-25T00:12:44.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63e6000640 1 -- 172.21.15.83:0/245504166 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e00047f0 con 0x7f63ec06b1e0 2024-04-25T00:12:44.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63e6000640 1 -- 172.21.15.83:0/245504166 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0004be0 con 0x7f63ec06b1e0 2024-04-25T00:12:44.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f4800640 1 -- 172.21.15.83:0/245504166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ec15dd80 con 0x7f63ec06b1e0 2024-04-25T00:12:44.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.769+0000 7f63f4800640 1 -- 172.21.15.83:0/245504166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ec0b4580 con 0x7f63ec06b1e0 2024-04-25T00:12:44.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.773+0000 7f63f4800640 1 -- 172.21.15.83:0/245504166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63ec0737d0 con 0x7f63ec06b1e0 2024-04-25T00:12:44.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.773+0000 7f63e6000640 1 -- 172.21.15.83:0/245504166 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f63e0011070 con 0x7f63ec06b1e0 2024-04-25T00:12:44.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.773+0000 7f63e6000640 1 --2- 172.21.15.83:0/245504166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63d00678c0 0x7f63d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:44.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.773+0000 7f63e6000640 1 -- 172.21.15.83:0/245504166 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7f63e0085730 con 0x7f63ec06b1e0 2024-04-25T00:12:44.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.773+0000 7f63f1400640 1 --2- 172.21.15.83:0/245504166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63d00678c0 0x7f63d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.773+0000 7f63f1400640 1 --2- 172.21.15.83:0/245504166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63d00678c0 0x7f63d0069d80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0ae000 tx=0x7f63dc0a6070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:44.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.777+0000 7f63e6000640 1 -- 172.21.15.83:0/245504166 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f63e000e280 con 0x7f63ec06b1e0 2024-04-25T00:12:44.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.933+0000 7f63f4800640 1 -- 172.21.15.83:0/245504166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63ec04f5f0 con 0x7f63ec06b1e0 2024-04-25T00:12:44.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.933+0000 7f63e6000640 1 -- 172.21.15.83:0/245504166 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+13209 (secure 0 0 0) 0x7f63e00563d0 con 0x7f63ec06b1e0 2024-04-25T00:12:44.937 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:44.937 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":41,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:42.586946+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/2704243724":"2024-04-26T00:12:39.580645+0000","172.21.15.83:6835/2704243724":"2024-04-26T00:12:39.580645+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.937+0000 7f63f4800640 1 -- 172.21.15.83:0/245504166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63d00678c0 msgr2=0x7f63d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.937+0000 7f63f4800640 1 --2- 172.21.15.83:0/245504166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63d00678c0 0x7f63d0069d80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0ae000 tx=0x7f63dc0a6070 comp rx=0 tx=0).stop 2024-04-25T00:12:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.937+0000 7f63f4800640 1 -- 172.21.15.83:0/245504166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec06b1e0 msgr2=0x7f63ec0b1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.937+0000 7f63f4800640 1 --2- 172.21.15.83:0/245504166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec06b1e0 0x7f63ec0b1080 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f63e000c400 tx=0x7f63e000c8c0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.937+0000 7f63f4800640 1 -- 172.21.15.83:0/245504166 shutdown_connections 2024-04-25T00:12:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.937+0000 7f63f4800640 1 --2- 172.21.15.83:0/245504166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63d00678c0 0x7f63d0069d80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.937+0000 7f63f4800640 1 --2- 172.21.15.83:0/245504166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63ec1452c0 0x7f63ec0b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.937+0000 7f63f4800640 1 --2- 172.21.15.83:0/245504166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63ec06fc10 0x7f63ec164dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.937+0000 7f63f4800640 1 --2- 172.21.15.83:0/245504166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec06b1e0 0x7f63ec0b1080 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:44.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.937+0000 7f63f4800640 1 -- 172.21.15.83:0/245504166 >> 172.21.15.83:0/245504166 conn(0x7f63ec0d0b90 msgr2=0x7f63ec1423c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:44.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.937+0000 7f63f4800640 1 -- 172.21.15.83:0/245504166 shutdown_connections 2024-04-25T00:12:44.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:44.937+0000 7f63f4800640 1 -- 172.21.15.83:0/245504166 wait complete. 2024-04-25T00:12:44.958 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:12:44.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:12:45.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.093+0000 7efc57000640 1 Processor -- start 2024-04-25T00:12:45.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.093+0000 7efc57000640 1 -- start start 2024-04-25T00:12:45.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.097+0000 7efc57000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc501555c0 0x7efc5015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.097+0000 7efc57000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efc5006e4c0 0x7efc5006e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.097+0000 7efc57000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc5015c560 con 0x7efc501551e0 2024-04-25T00:12:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.097+0000 7efc57000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7efc5006ee70 con 0x7efc5006e4c0 2024-04-25T00:12:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.097+0000 7efc57000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efc50072460 con 0x7efc501555c0 2024-04-25T00:12:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.097+0000 7efc4fe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efc501551e0 0x7efc50071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55818/0 (socket says 172.21.15.83:55818) 2024-04-25T00:12:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.097+0000 7efc55000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efc5006e4c0 0x7efc5006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.097+0000 7efc4fe00640 1 -- 172.21.15.83:0/2038928965 learned_addr learned my addr 172.21.15.83:0/2038928965 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.097+0000 7efc47400640 1 --2- 172.21.15.83:0/2038928965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc501555c0 0x7efc5015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.097+0000 7efc55000640 1 -- 172.21.15.83:0/2038928965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc501555c0 msgr2=0x7efc5015bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.097+0000 7efc55000640 1 --2- 172.21.15.83:0/2038928965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc501555c0 0x7efc5015bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.097+0000 7efc55000640 1 -- 172.21.15.83:0/2038928965 >> v1:172.21.15.83:6789/0 conn(0x7efc501551e0 legacy=0x7efc50071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.097+0000 7efc55000640 1 -- 172.21.15.83:0/2038928965 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5015c7d0 con 0x7efc5006e4c0 2024-04-25T00:12:45.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.097+0000 7efc4f400640 1 -- 172.21.15.83:0/2038928965 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2839721395 0 0) 0x7efc5015c560 con 0x7efc501551e0 2024-04-25T00:12:45.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc55000640 1 --2- 172.21.15.83:0/2038928965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5006e4c0 0x7efc5006e930 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efc480097c0 tx=0x7efc4801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db48c6e0a0d21176 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:45.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc4f400640 1 -- 172.21.15.83:0/2038928965 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc480b1070 con 0x7efc5006e4c0 2024-04-25T00:12:45.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc4f400640 1 -- 172.21.15.83:0/2038928965 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc480ac410 con 0x7efc5006e4c0 2024-04-25T00:12:45.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc4f400640 1 -- 172.21.15.83:0/2038928965 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc480ac7e0 con 0x7efc5006e4c0 2024-04-25T00:12:45.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc57000640 1 -- 172.21.15.83:0/2038928965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5006e4c0 msgr2=0x7efc5006e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:45.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc57000640 1 --2- 172.21.15.83:0/2038928965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5006e4c0 0x7efc5006e930 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efc480097c0 tx=0x7efc4801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc57000640 1 -- 172.21.15.83:0/2038928965 shutdown_connections 2024-04-25T00:12:45.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc57000640 1 --2- 172.21.15.83:0/2038928965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5006e4c0 0x7efc5006e930 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc57000640 1 --2- 172.21.15.83:0/2038928965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc501555c0 0x7efc5015bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc57000640 1 -- 172.21.15.83:0/2038928965 >> 172.21.15.83:0/2038928965 conn(0x7efc500b1830 msgr2=0x7efc500b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:45.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc57000640 1 -- 172.21.15.83:0/2038928965 shutdown_connections 2024-04-25T00:12:45.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc57000640 1 -- 172.21.15.83:0/2038928965 wait complete. 2024-04-25T00:12:45.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc57000640 1 Processor -- start 2024-04-25T00:12:45.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc57000640 1 -- start start 2024-04-25T00:12:45.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc57000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc5006e4c0 0x7efc5014d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:45.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.101+0000 7efc57000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc501551e0 0x7efc500af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:45.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc57000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc501555c0 0x7efc500afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:45.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc57000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc5015db70 con 0x7efc501551e0 2024-04-25T00:12:45.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc57000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc5015d9f0 con 0x7efc501555c0 2024-04-25T00:12:45.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc57000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efc5015dcf0 con 0x7efc5006e4c0 2024-04-25T00:12:45.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc47400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc501551e0 0x7efc500af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:45.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc47400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc501551e0 0x7efc500af780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54764/0 (socket says 172.21.15.83:54764) 2024-04-25T00:12:45.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc47400640 1 -- 172.21.15.83:0/1926655742 learned_addr learned my addr 172.21.15.83:0/1926655742 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:45.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc4fe00640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc5006e4c0 0x7efc5014d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:45.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc55000640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc501555c0 0x7efc500afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:45.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc47400640 1 -- 172.21.15.83:0/1926655742 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc5006e4c0 msgr2=0x7efc5014d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:45.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc47400640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc5006e4c0 0x7efc5014d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc47400640 1 -- 172.21.15.83:0/1926655742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc501555c0 msgr2=0x7efc500afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:45.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc47400640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc501555c0 0x7efc500afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc47400640 1 -- 172.21.15.83:0/1926655742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc501575a0 con 0x7efc501551e0 2024-04-25T00:12:45.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc55000640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc501555c0 0x7efc500afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:45.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc47400640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc501551e0 0x7efc500af780 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efc3800d5f0 tx=0x7efc3800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:45.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc4cc00640 1 -- 172.21.15.83:0/1926655742 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc380125f0 con 0x7efc501551e0 2024-04-25T00:12:45.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc4cc00640 1 -- 172.21.15.83:0/1926655742 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc38019070 con 0x7efc501551e0 2024-04-25T00:12:45.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc57000640 1 -- 172.21.15.83:0/1926655742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc50157880 con 0x7efc501551e0 2024-04-25T00:12:45.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc4cc00640 1 -- 172.21.15.83:0/1926655742 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc380129d0 con 0x7efc501551e0 2024-04-25T00:12:45.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc57000640 1 -- 172.21.15.83:0/1926655742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5014db60 con 0x7efc501551e0 2024-04-25T00:12:45.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc57000640 1 -- 172.21.15.83:0/1926655742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc5004f510 con 0x7efc501551e0 2024-04-25T00:12:45.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.105+0000 7efc4cc00640 1 -- 172.21.15.83:0/1926655742 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efc38012b70 con 0x7efc501551e0 2024-04-25T00:12:45.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.109+0000 7efc4cc00640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc2c0678f0 0x7efc2c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:45.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.109+0000 7efc4cc00640 1 -- 172.21.15.83:0/1926655742 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7efc38085f00 con 0x7efc501551e0 2024-04-25T00:12:45.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.109+0000 7efc4fe00640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc2c0678f0 0x7efc2c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:45.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.109+0000 7efc4fe00640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc2c0678f0 0x7efc2c069db0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efc400012f0 tx=0x7efc40001770 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:45.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.113+0000 7efc4cc00640 1 -- 172.21.15.83:0/1926655742 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efc38056ba0 con 0x7efc501551e0 2024-04-25T00:12:45.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.269+0000 7efc57000640 1 -- 172.21.15.83:0/1926655742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efc500cc670 con 0x7efc501551e0 2024-04-25T00:12:45.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.269+0000 7efc4cc00640 1 -- 172.21.15.83:0/1926655742 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7efc380564a0 con 0x7efc501551e0 2024-04-25T00:12:45.270 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:12:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.269+0000 7efc57000640 1 -- 172.21.15.83:0/1926655742 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc2c0678f0 msgr2=0x7efc2c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.269+0000 7efc57000640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc2c0678f0 0x7efc2c069db0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efc400012f0 tx=0x7efc40001770 comp rx=0 tx=0).stop 2024-04-25T00:12:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.269+0000 7efc57000640 1 -- 172.21.15.83:0/1926655742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc501551e0 msgr2=0x7efc500af780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.269+0000 7efc57000640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc501551e0 0x7efc500af780 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efc3800d5f0 tx=0x7efc3800dab0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.269+0000 7efc57000640 1 -- 172.21.15.83:0/1926655742 shutdown_connections 2024-04-25T00:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.269+0000 7efc57000640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc2c0678f0 0x7efc2c069db0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.269+0000 7efc57000640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc501555c0 0x7efc500afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.269+0000 7efc57000640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc501551e0 0x7efc500af780 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.269+0000 7efc57000640 1 --2- 172.21.15.83:0/1926655742 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc5006e4c0 0x7efc5014d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.269+0000 7efc57000640 1 -- 172.21.15.83:0/1926655742 >> 172.21.15.83:0/1926655742 conn(0x7efc500b1830 msgr2=0x7efc50152d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.269+0000 7efc57000640 1 -- 172.21.15.83:0/1926655742 shutdown_connections 2024-04-25T00:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.269+0000 7efc57000640 1 -- 172.21.15.83:0/1926655742 wait complete. 2024-04-25T00:12:45.291 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted' 2024-04-25T00:12:45.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfd0600640 1 Processor -- start 2024-04-25T00:12:45.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfd0600640 1 -- start start 2024-04-25T00:12:45.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfd0600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcfc80c4c40 0x7fcfc80c5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:45.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfd0600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcfc81627e0 0x7fcfc806c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:45.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfd0600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcfc80b21f0 0x7fcfc80b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:45.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfd0600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc806cb40 con 0x7fcfc81627e0 2024-04-25T00:12:45.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfd0600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc80c5560 con 0x7fcfc80c4c40 2024-04-25T00:12:45.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfd0600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc80aeb80 con 0x7fcfc80b21f0 2024-04-25T00:12:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfcd200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcfc81627e0 0x7fcfc806c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfcd200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcfc81627e0 0x7fcfc806c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54778/0 (socket says 172.21.15.83:54778) 2024-04-25T00:12:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfcd200640 1 -- 172.21.15.83:0/644186830 learned_addr learned my addr 172.21.15.83:0/644186830 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfcdc00640 1 --2- 172.21.15.83:0/644186830 >> v2:172.21.15.86:3300/0 conn(0x7fcfc80c4c40 0x7fcfc80c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfce600640 1 --2- 172.21.15.83:0/644186830 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcfc80b21f0 0x7fcfc80b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfcd200640 1 -- 172.21.15.83:0/644186830 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcfc80b21f0 msgr2=0x7fcfc80b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfcd200640 1 --2- 172.21.15.83:0/644186830 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcfc80b21f0 0x7fcfc80b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfcd200640 1 -- 172.21.15.83:0/644186830 >> v2:172.21.15.86:3300/0 conn(0x7fcfc80c4c40 msgr2=0x7fcfc80c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfcd200640 1 --2- 172.21.15.83:0/644186830 >> v2:172.21.15.86:3300/0 conn(0x7fcfc80c4c40 0x7fcfc80c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.425+0000 7fcfcd200640 1 -- 172.21.15.83:0/644186830 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc80aee60 con 0x7fcfc81627e0 2024-04-25T00:12:45.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.429+0000 7fcfcd200640 1 --2- 172.21.15.83:0/644186830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc81627e0 0x7fcfc806c2f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80b3730 tx=0x7fcfb401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb5b4c82809ef18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:45.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.429+0000 7fcfc7e00640 1 -- 172.21.15.83:0/644186830 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40b0070 con 0x7fcfc81627e0 2024-04-25T00:12:45.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.429+0000 7fcfc7e00640 1 -- 172.21.15.83:0/644186830 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb40ab600 con 0x7fcfc81627e0 2024-04-25T00:12:45.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.429+0000 7fcfc7e00640 1 -- 172.21.15.83:0/644186830 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40ab9d0 con 0x7fcfc81627e0 2024-04-25T00:12:45.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.429+0000 7fcfd0600640 1 -- 172.21.15.83:0/644186830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc81627e0 msgr2=0x7fcfc806c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:45.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.429+0000 7fcfd0600640 1 --2- 172.21.15.83:0/644186830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc81627e0 0x7fcfc806c2f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80b3730 tx=0x7fcfb401c230 comp rx=0 tx=0).stop 2024-04-25T00:12:45.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.429+0000 7fcfd0600640 1 -- 172.21.15.83:0/644186830 shutdown_connections 2024-04-25T00:12:45.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.429+0000 7fcfd0600640 1 --2- 172.21.15.83:0/644186830 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcfc80b21f0 0x7fcfc80b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.429+0000 7fcfd0600640 1 --2- 172.21.15.83:0/644186830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc81627e0 0x7fcfc806c2f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.429+0000 7fcfd0600640 1 --2- 172.21.15.83:0/644186830 >> v2:172.21.15.86:3300/0 conn(0x7fcfc80c4c40 0x7fcfc80c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.429+0000 7fcfd0600640 1 -- 172.21.15.83:0/644186830 >> 172.21.15.83:0/644186830 conn(0x7fcfc80d1c70 msgr2=0x7fcfc80d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:45.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.429+0000 7fcfd0600640 1 -- 172.21.15.83:0/644186830 shutdown_connections 2024-04-25T00:12:45.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.429+0000 7fcfd0600640 1 -- 172.21.15.83:0/644186830 wait complete. 2024-04-25T00:12:45.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfd0600640 1 Processor -- start 2024-04-25T00:12:45.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfd0600640 1 -- start start 2024-04-25T00:12:45.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfd0600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcfc80b21f0 0x7fcfc80c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:45.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfd0600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc80c4c40 0x7fcfc80c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:45.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfd0600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfc81627e0 0x7fcfc8155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:45.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfd0600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc80af940 con 0x7fcfc80c4c40 2024-04-25T00:12:45.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfd0600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc80cf840 con 0x7fcfc81627e0 2024-04-25T00:12:45.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfd0600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc80afac0 con 0x7fcfc80b21f0 2024-04-25T00:12:45.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfcd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc80c4c40 0x7fcfc80c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:45.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfcd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc80c4c40 0x7fcfc80c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54788/0 (socket says 172.21.15.83:54788) 2024-04-25T00:12:45.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfcd200640 1 -- 172.21.15.83:0/935227414 learned_addr learned my addr 172.21.15.83:0/935227414 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:45.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfcdc00640 1 --2- 172.21.15.83:0/935227414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcfc80b21f0 0x7fcfc80c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:45.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfce600640 1 --2- 172.21.15.83:0/935227414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfc81627e0 0x7fcfc8155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:45.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfcd200640 1 -- 172.21.15.83:0/935227414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcfc80b21f0 msgr2=0x7fcfc80c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:45.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfcd200640 1 --2- 172.21.15.83:0/935227414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcfc80b21f0 0x7fcfc80c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfcd200640 1 -- 172.21.15.83:0/935227414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfc81627e0 msgr2=0x7fcfc8155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:45.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfcd200640 1 --2- 172.21.15.83:0/935227414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfc81627e0 0x7fcfc8155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:45.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfcd200640 1 -- 172.21.15.83:0/935227414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc80ca640 con 0x7fcfc80c4c40 2024-04-25T00:12:45.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfcd200640 1 --2- 172.21.15.83:0/935227414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc80c4c40 0x7fcfc80c6150 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40ae040 tx=0x7fcfb4018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:45.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfc5600640 1 -- 172.21.15.83:0/935227414 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40b0040 con 0x7fcfc80c4c40 2024-04-25T00:12:45.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfc5600640 1 -- 172.21.15.83:0/935227414 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb40b4620 con 0x7fcfc80c4c40 2024-04-25T00:12:45.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfc5600640 1 -- 172.21.15.83:0/935227414 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40b4a10 con 0x7fcfc80c4c40 2024-04-25T00:12:45.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfd0600640 1 -- 172.21.15.83:0/935227414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc80ca920 con 0x7fcfc80c4c40 2024-04-25T00:12:45.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfd0600640 1 -- 172.21.15.83:0/935227414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc80caee0 con 0x7fcfc80c4c40 2024-04-25T00:12:45.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.433+0000 7fcfd0600640 1 -- 172.21.15.83:0/935227414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc809fd60 con 0x7fcfc80c4c40 2024-04-25T00:12:45.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.437+0000 7fcfc5600640 1 -- 172.21.15.83:0/935227414 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcfb40ab4c0 con 0x7fcfc80c4c40 2024-04-25T00:12:45.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.437+0000 7fcfc5600640 1 --2- 172.21.15.83:0/935227414 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcfa80678c0 0x7fcfa8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:45.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.437+0000 7fcfc5600640 1 -- 172.21.15.83:0/935227414 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7fcfb411f6a0 con 0x7fcfc80c4c40 2024-04-25T00:12:45.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.437+0000 7fcfcdc00640 1 --2- 172.21.15.83:0/935227414 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcfa80678c0 0x7fcfa8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:45.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.437+0000 7fcfcdc00640 1 --2- 172.21.15.83:0/935227414 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcfa80678c0 0x7fcfa8069d80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc007a60 tx=0x7fcfbc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:45.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.441+0000 7fcfc5600640 1 -- 172.21.15.83:0/935227414 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcfb4016110 con 0x7fcfc80c4c40 2024-04-25T00:12:45.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:45.653+0000 7fcfd0600640 1 -- 172.21.15.83:0/935227414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]} v 0) -- 0x7fcfc80cbf00 con 0x7fcfc80c4c40 2024-04-25T00:12:46.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.601+0000 7fcfc5600640 1 -- 172.21.15.83:0/935227414 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]}]=0 v123) ==== 156+0+0 (secure 0 0 0) 0x7fcfb40efcf0 con 0x7fcfc80c4c40 2024-04-25T00:12:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.601+0000 7fcfd0600640 1 -- 172.21.15.83:0/935227414 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcfa80678c0 msgr2=0x7fcfa8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.601+0000 7fcfd0600640 1 --2- 172.21.15.83:0/935227414 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcfa80678c0 0x7fcfa8069d80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc007a60 tx=0x7fcfbc008040 comp rx=0 tx=0).stop 2024-04-25T00:12:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.601+0000 7fcfd0600640 1 -- 172.21.15.83:0/935227414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc80c4c40 msgr2=0x7fcfc80c6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.601+0000 7fcfd0600640 1 --2- 172.21.15.83:0/935227414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc80c4c40 0x7fcfc80c6150 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40ae040 tx=0x7fcfb4018fd0 comp rx=0 tx=0).stop 2024-04-25T00:12:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.601+0000 7fcfd0600640 1 -- 172.21.15.83:0/935227414 shutdown_connections 2024-04-25T00:12:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.601+0000 7fcfd0600640 1 --2- 172.21.15.83:0/935227414 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcfa80678c0 0x7fcfa8069d80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.601+0000 7fcfd0600640 1 --2- 172.21.15.83:0/935227414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfc81627e0 0x7fcfc8155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.601+0000 7fcfd0600640 1 --2- 172.21.15.83:0/935227414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc80c4c40 0x7fcfc80c6150 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.601+0000 7fcfd0600640 1 --2- 172.21.15.83:0/935227414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcfc80b21f0 0x7fcfc80c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.601+0000 7fcfd0600640 1 -- 172.21.15.83:0/935227414 >> 172.21.15.83:0/935227414 conn(0x7fcfc80d1c70 msgr2=0x7fcfc80afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.601+0000 7fcfd0600640 1 -- 172.21.15.83:0/935227414 shutdown_connections 2024-04-25T00:12:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.601+0000 7fcfd0600640 1 -- 172.21.15.83:0/935227414 wait complete. 2024-04-25T00:12:46.623 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:12:46.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8f2200640 1 Processor -- start 2024-04-25T00:12:46.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8f2200640 1 -- start start 2024-04-25T00:12:46.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8f2200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8ec15c160 0x7fe8ec0d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:46.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8f2200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe8ec0cd7e0 0x7fe8ec0d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:46.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8f2200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8ec15c540 con 0x7fe8ec0d0280 2024-04-25T00:12:46.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8f2200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe8ec0cf660 con 0x7fe8ec0cd7e0 2024-04-25T00:12:46.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8f2200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe8ec0d0c20 con 0x7fe8ec15c160 2024-04-25T00:12:46.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8eaa00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe8ec0d0280 0x7fe8ec0b1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55834/0 (socket says 172.21.15.83:55834) 2024-04-25T00:12:46.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8eaa00640 1 -- 172.21.15.83:0/4045588866 learned_addr learned my addr 172.21.15.83:0/4045588866 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:46.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8ebe00640 1 --2- 172.21.15.83:0/4045588866 >> v2:172.21.15.83:3300/0 conn(0x7fe8ec0cd7e0 0x7fe8ec0d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:46.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8eb400640 1 --2- 172.21.15.83:0/4045588866 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8ec15c160 0x7fe8ec0d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:46.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8ea000640 1 -- 172.21.15.83:0/4045588866 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4110143452 0 0) 0x7fe8ec15c540 con 0x7fe8ec0d0280 2024-04-25T00:12:46.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8ea000640 1 -- 172.21.15.83:0/4045588866 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8d0003530 con 0x7fe8ec0d0280 2024-04-25T00:12:46.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8ebe00640 1 -- 172.21.15.83:0/4045588866 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8ec15c160 msgr2=0x7fe8ec0d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:46.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8ebe00640 1 --2- 172.21.15.83:0/4045588866 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8ec15c160 0x7fe8ec0d27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:46.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8ebe00640 1 -- 172.21.15.83:0/4045588866 >> v1:172.21.15.83:6789/0 conn(0x7fe8ec0d0280 legacy=0x7fe8ec0b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:46.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.761+0000 7fe8ebe00640 1 -- 172.21.15.83:0/4045588866 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ec0cfa30 con 0x7fe8ec0cd7e0 2024-04-25T00:12:46.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.765+0000 7fe8ebe00640 1 --2- 172.21.15.83:0/4045588866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0cd7e0 0x7fe8ec0d06e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009d90 tx=0x7fe8e401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0ce50ceaf34df0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:46.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.765+0000 7fe8ea000640 1 -- 172.21.15.83:0/4045588866 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e40b2070 con 0x7fe8ec0cd7e0 2024-04-25T00:12:46.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.765+0000 7fe8ea000640 1 -- 172.21.15.83:0/4045588866 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e40a5920 con 0x7fe8ec0cd7e0 2024-04-25T00:12:46.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.765+0000 7fe8ea000640 1 -- 172.21.15.83:0/4045588866 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e40a5cf0 con 0x7fe8ec0cd7e0 2024-04-25T00:12:46.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.765+0000 7fe8f2200640 1 -- 172.21.15.83:0/4045588866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0cd7e0 msgr2=0x7fe8ec0d06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:46.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.765+0000 7fe8f2200640 1 --2- 172.21.15.83:0/4045588866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0cd7e0 0x7fe8ec0d06e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009d90 tx=0x7fe8e401c970 comp rx=0 tx=0).stop 2024-04-25T00:12:46.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.765+0000 7fe8f2200640 1 -- 172.21.15.83:0/4045588866 shutdown_connections 2024-04-25T00:12:46.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.765+0000 7fe8f2200640 1 --2- 172.21.15.83:0/4045588866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0cd7e0 0x7fe8ec0d06e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:46.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.765+0000 7fe8f2200640 1 --2- 172.21.15.83:0/4045588866 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8ec15c160 0x7fe8ec0d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:46.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.765+0000 7fe8f2200640 1 -- 172.21.15.83:0/4045588866 >> 172.21.15.83:0/4045588866 conn(0x7fe8ec0d1cc0 msgr2=0x7fe8ec0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:46.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.765+0000 7fe8f2200640 1 -- 172.21.15.83:0/4045588866 shutdown_connections 2024-04-25T00:12:46.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.765+0000 7fe8f2200640 1 -- 172.21.15.83:0/4045588866 wait complete. 2024-04-25T00:12:46.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.765+0000 7fe8f2200640 1 Processor -- start 2024-04-25T00:12:46.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8f2200640 1 -- start start 2024-04-25T00:12:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8f2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8ec0cd7e0 0x7fe8ec0c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8f2200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0d0280 0x7fe8ec0ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8f2200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8ec15c160 0x7fe8ec0cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8f2200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec0b2de0 con 0x7fe8ec0d0280 2024-04-25T00:12:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8f2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec0b19a0 con 0x7fe8ec0cd7e0 2024-04-25T00:12:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8f2200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe8ec0b2f60 con 0x7fe8ec15c160 2024-04-25T00:12:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8eaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0d0280 0x7fe8ec0ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8eaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0d0280 0x7fe8ec0ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54810/0 (socket says 172.21.15.83:54810) 2024-04-25T00:12:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8eaa00640 1 -- 172.21.15.83:0/1015510555 learned_addr learned my addr 172.21.15.83:0/1015510555 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8eb400640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8ec0cd7e0 0x7fe8ec0c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8ebe00640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8ec15c160 0x7fe8ec0cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8eaa00640 1 -- 172.21.15.83:0/1015510555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8ec15c160 msgr2=0x7fe8ec0cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8eaa00640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8ec15c160 0x7fe8ec0cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8eaa00640 1 -- 172.21.15.83:0/1015510555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8ec0cd7e0 msgr2=0x7fe8ec0c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8eaa00640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8ec0cd7e0 0x7fe8ec0c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8eaa00640 1 -- 172.21.15.83:0/1015510555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ec0731b0 con 0x7fe8ec0d0280 2024-04-25T00:12:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8eb400640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8ec0cd7e0 0x7fe8ec0c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8eaa00640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0d0280 0x7fe8ec0ca6b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800c450 tx=0x7fe8d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8e3400640 1 -- 172.21.15.83:0/1015510555 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8017070 con 0x7fe8ec0d0280 2024-04-25T00:12:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8e3400640 1 -- 172.21.15.83:0/1015510555 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d8005cf0 con 0x7fe8ec0d0280 2024-04-25T00:12:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8e3400640 1 -- 172.21.15.83:0/1015510555 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8007290 con 0x7fe8ec0d0280 2024-04-25T00:12:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8f2200640 1 -- 172.21.15.83:0/1015510555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ec0c72e0 con 0x7fe8ec0d0280 2024-04-25T00:12:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8f2200640 1 -- 172.21.15.83:0/1015510555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ec0c6520 con 0x7fe8ec0d0280 2024-04-25T00:12:46.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.769+0000 7fe8f2200640 1 -- 172.21.15.83:0/1015510555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8ec04f510 con 0x7fe8ec0d0280 2024-04-25T00:12:46.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.773+0000 7fe8e3400640 1 -- 172.21.15.83:0/1015510555 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe8d801a070 con 0x7fe8ec0d0280 2024-04-25T00:12:46.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.773+0000 7fe8e3400640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe8cc0678c0 0x7fe8cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:46.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.773+0000 7fe8e3400640 1 -- 172.21.15.83:0/1015510555 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7fe8d8084fa0 con 0x7fe8ec0d0280 2024-04-25T00:12:46.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.773+0000 7fe8eb400640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe8cc0678c0 0x7fe8cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:46.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.773+0000 7fe8eb400640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe8cc0678c0 0x7fe8cc069d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0d1ae0 tx=0x7fe8dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:46.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.777+0000 7fe8e3400640 1 -- 172.21.15.83:0/1015510555 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe8ec0c72e0 con 0x7fe8ec0d0280 2024-04-25T00:12:46.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.933+0000 7fe8f2200640 1 -- 172.21.15.83:0/1015510555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe8ec0b0f30 con 0x7fe8ec0d0280 2024-04-25T00:12:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.937+0000 7fe8e3400640 1 -- 172.21.15.83:0/1015510555 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fe8d8055bb0 con 0x7fe8ec0d0280 2024-04-25T00:12:46.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.937+0000 7fe8f2200640 1 -- 172.21.15.83:0/1015510555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe8cc0678c0 msgr2=0x7fe8cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:46.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.937+0000 7fe8f2200640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe8cc0678c0 0x7fe8cc069d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0d1ae0 tx=0x7fe8dc008040 comp rx=0 tx=0).stop 2024-04-25T00:12:46.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.937+0000 7fe8f2200640 1 -- 172.21.15.83:0/1015510555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0d0280 msgr2=0x7fe8ec0ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:46.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.937+0000 7fe8f2200640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0d0280 0x7fe8ec0ca6b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800c450 tx=0x7fe8d800c910 comp rx=0 tx=0).stop 2024-04-25T00:12:46.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.937+0000 7fe8f2200640 1 -- 172.21.15.83:0/1015510555 shutdown_connections 2024-04-25T00:12:46.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.937+0000 7fe8f2200640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe8cc0678c0 0x7fe8cc069d80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.937+0000 7fe8f2200640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8ec15c160 0x7fe8ec0cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.937+0000 7fe8f2200640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0d0280 0x7fe8ec0ca6b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.937+0000 7fe8f2200640 1 --2- 172.21.15.83:0/1015510555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8ec0cd7e0 0x7fe8ec0c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.937+0000 7fe8f2200640 1 -- 172.21.15.83:0/1015510555 >> 172.21.15.83:0/1015510555 conn(0x7fe8ec0d1cc0 msgr2=0x7fe8ec0b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.937+0000 7fe8f2200640 1 -- 172.21.15.83:0/1015510555 shutdown_connections 2024-04-25T00:12:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:46.937+0000 7fe8f2200640 1 -- 172.21.15.83:0/1015510555 wait complete. 2024-04-25T00:12:46.959 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:12:46.960 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:12:47.004 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:12:47.100 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:12:47.100 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:12:47.100 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:47.109 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:12:47.109 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:12:47.109 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:47.118 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:12:47.118 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:12:47.118 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:47.161 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:12:47.161 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:12:47.161 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:47.170 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:12:47.170 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:12:47.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:12:47.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.337+0000 7fa887c00640 1 Processor -- start 2024-04-25T00:12:47.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.337+0000 7fa887c00640 1 -- start start 2024-04-25T00:12:47.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.337+0000 7fa887c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880164cb0 0x7fa8801670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.337+0000 7fa887c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8800ccc60 con 0x7fa8801599f0 2024-04-25T00:12:47.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.337+0000 7fa887c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa88006dba0 con 0x7fa8800cc510 2024-04-25T00:12:47.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.337+0000 7fa887c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa880159dd0 con 0x7fa880164cb0 2024-04-25T00:12:47.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.337+0000 7fa885200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa8801599f0 0x7fa88015c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55848/0 (socket says 172.21.15.83:55848) 2024-04-25T00:12:47.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.337+0000 7fa885200640 1 -- 172.21.15.83:0/278064305 learned_addr learned my addr 172.21.15.83:0/278064305 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:47.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.337+0000 7fa885c00640 1 --2- 172.21.15.83:0/278064305 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880164cb0 0x7fa8801670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.337+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 446027943 0 0) 0x7fa870001120 con 0x7fa8801599f0 2024-04-25T00:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.337+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa864003530 con 0x7fa8801599f0 2024-04-25T00:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 714508015 0 0) 0x7fa88006dba0 con 0x7fa8800cc510 2024-04-25T00:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa870001120 con 0x7fa8800cc510 2024-04-25T00:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 338883746 0 0) 0x7fa864003530 con 0x7fa8801599f0 2024-04-25T00:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880164cb0 msgr2=0x7fa8801670a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 --2- 172.21.15.83:0/278064305 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880164cb0 0x7fa8801670a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 >> v1:172.21.15.86:6789/0 conn(0x7fa8800cc510 legacy=0x7fa88015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88006de80 con 0x7fa8801599f0 2024-04-25T00:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fa8700017d0 con 0x7fa8801599f0 2024-04-25T00:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 >> v1:172.21.15.83:6789/0 conn(0x7fa8801599f0 legacy=0x7fa88015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 --2- 172.21.15.83:0/278064305 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880164cb0 0x7fa864003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 --2- 172.21.15.83:0/278064305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864004130 0x7fa864004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 --2- 172.21.15.83:0/278064305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa864004d50 0x7fa8640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa86400bb60 con 0x7fa864004d50 2024-04-25T00:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa86400bd10 con 0x7fa864004130 2024-04-25T00:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa86400bf10 con 0x7fa880164cb0 2024-04-25T00:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa885c00640 1 --2- 172.21.15.83:0/278064305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa864004d50 0x7fa8640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa885200640 1 --2- 172.21.15.83:0/278064305 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880164cb0 0x7fa864003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa877e00640 1 --2- 172.21.15.83:0/278064305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864004130 0x7fa864004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa885c00640 1 -- 172.21.15.83:0/278064305 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880164cb0 msgr2=0x7fa864003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa885c00640 1 --2- 172.21.15.83:0/278064305 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880164cb0 0x7fa864003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa885c00640 1 -- 172.21.15.83:0/278064305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864004130 msgr2=0x7fa864004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa885c00640 1 --2- 172.21.15.83:0/278064305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864004130 0x7fa864004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa885c00640 1 -- 172.21.15.83:0/278064305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa864003530 con 0x7fa864004d50 2024-04-25T00:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.341+0000 7fa885200640 1 --2- 172.21.15.83:0/278064305 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880164cb0 0x7fa864003b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa885c00640 1 --2- 172.21.15.83:0/278064305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa864004d50 0x7fa8640051d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa878009f70 tx=0x7fa87801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1dc4f5fa68a8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8780b2070 con 0x7fa864004d50 2024-04-25T00:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8780a5cf0 con 0x7fa864004d50 2024-04-25T00:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa87fe00640 1 -- 172.21.15.83:0/278064305 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878009bf0 con 0x7fa864004d50 2024-04-25T00:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa887c00640 1 -- 172.21.15.83:0/278064305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa864004d50 msgr2=0x7fa8640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa887c00640 1 --2- 172.21.15.83:0/278064305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa864004d50 0x7fa8640051d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa878009f70 tx=0x7fa87801cc20 comp rx=0 tx=0).stop 2024-04-25T00:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa885c00640 1 -- 172.21.15.83:0/278064305 reap_dead start 2024-04-25T00:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa887c00640 1 -- 172.21.15.83:0/278064305 shutdown_connections 2024-04-25T00:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa887c00640 1 -- 172.21.15.83:0/278064305 >> 172.21.15.83:0/278064305 conn(0x7fa8800b1830 msgr2=0x7fa8800b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa887c00640 1 -- 172.21.15.83:0/278064305 shutdown_connections 2024-04-25T00:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa887c00640 1 -- 172.21.15.83:0/278064305 wait complete. 2024-04-25T00:12:47.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa887c00640 1 Processor -- start 2024-04-25T00:12:47.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa887c00640 1 -- start start 2024-04-25T00:12:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa887c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864004d50 0x7fa8800ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa887c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880146590 0x7fa880154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa887c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880152ec0 0x7fa8800b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa887c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa88015d160 con 0x7fa880152ec0 2024-04-25T00:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa887c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8800cc980 con 0x7fa864004d50 2024-04-25T00:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.345+0000 7fa887c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa8800ce0c0 con 0x7fa880146590 2024-04-25T00:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa885c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880152ec0 0x7fa8800b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa885c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880152ec0 0x7fa8800b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54814/0 (socket says 172.21.15.83:54814) 2024-04-25T00:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa885c00640 1 -- 172.21.15.83:0/2034942192 learned_addr learned my addr 172.21.15.83:0/2034942192 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa877e00640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880146590 0x7fa880154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa885200640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864004d50 0x7fa8800ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa885c00640 1 -- 172.21.15.83:0/2034942192 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880146590 msgr2=0x7fa880154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa885c00640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880146590 0x7fa880154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa885c00640 1 -- 172.21.15.83:0/2034942192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864004d50 msgr2=0x7fa8800ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa885c00640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864004d50 0x7fa8800ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa885c00640 1 -- 172.21.15.83:0/2034942192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa880146a10 con 0x7fa880152ec0 2024-04-25T00:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa877e00640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880146590 0x7fa880154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa885200640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864004d50 0x7fa8800ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa885c00640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880152ec0 0x7fa8800b3880 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa8780b0040 tx=0x7fa878009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa87d600640 1 -- 172.21.15.83:0/2034942192 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8780b2040 con 0x7fa880152ec0 2024-04-25T00:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa87d600640 1 -- 172.21.15.83:0/2034942192 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8780a5af0 con 0x7fa880152ec0 2024-04-25T00:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa87d600640 1 -- 172.21.15.83:0/2034942192 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878009bf0 con 0x7fa880152ec0 2024-04-25T00:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa887c00640 1 -- 172.21.15.83:0/2034942192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa880154fb0 con 0x7fa880152ec0 2024-04-25T00:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa887c00640 1 -- 172.21.15.83:0/2034942192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8800d0590 con 0x7fa880152ec0 2024-04-25T00:12:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa887c00640 1 -- 172.21.15.83:0/2034942192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8800737f0 con 0x7fa880152ec0 2024-04-25T00:12:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa87d600640 1 -- 172.21.15.83:0/2034942192 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa8780c0030 con 0x7fa880152ec0 2024-04-25T00:12:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.349+0000 7fa87d600640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa85c0678c0 0x7fa85c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.353+0000 7fa87d600640 1 -- 172.21.15.83:0/2034942192 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7fa878120180 con 0x7fa880152ec0 2024-04-25T00:12:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.353+0000 7fa885200640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa85c0678c0 0x7fa85c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.353+0000 7fa885200640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa85c0678c0 0x7fa85c069d80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa870000c10 tx=0x7fa870009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:47.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.357+0000 7fa87d600640 1 -- 172.21.15.83:0/2034942192 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa8780c7060 con 0x7fa880152ec0 2024-04-25T00:12:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.541+0000 7fa887c00640 1 -- 172.21.15.83:0/2034942192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8800cbb10 con 0x7fa880152ec0 2024-04-25T00:12:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.541+0000 7fa87d600640 1 -- 172.21.15.83:0/2034942192 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3863 (secure 0 0 0) 0x7fa878016930 con 0x7fa880152ec0 2024-04-25T00:12:47.546 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:47.547 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4277921554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":4277921554},{"type":"v1","addr":"172.21.15.86:6835","nonce":4277921554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/4145334103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":4145334103},{"type":"v1","addr":"172.21.15.86:6834","nonce":4145334103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3441801758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3441801758},{"type":"v1","addr":"172.21.15.83:6836","nonce":3441801758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5718,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/89611369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":89611369},{"type":"v1","addr":"172.21.15.83:6837","nonce":89611369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-04-25T00:12:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 20 2024-04-25T00:12:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.545+0000 7fa887c00640 1 -- 172.21.15.83:0/2034942192 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa85c0678c0 msgr2=0x7fa85c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.545+0000 7fa887c00640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa85c0678c0 0x7fa85c069d80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa870000c10 tx=0x7fa870009040 comp rx=0 tx=0).stop 2024-04-25T00:12:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.545+0000 7fa887c00640 1 -- 172.21.15.83:0/2034942192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880152ec0 msgr2=0x7fa8800b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.545+0000 7fa887c00640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880152ec0 0x7fa8800b3880 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa8780b0040 tx=0x7fa878009750 comp rx=0 tx=0).stop 2024-04-25T00:12:47.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.545+0000 7fa887c00640 1 -- 172.21.15.83:0/2034942192 shutdown_connections 2024-04-25T00:12:47.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.545+0000 7fa887c00640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa85c0678c0 0x7fa85c069d80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.545+0000 7fa887c00640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa880146590 0x7fa880154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.545+0000 7fa887c00640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880152ec0 0x7fa8800b3880 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.545+0000 7fa887c00640 1 --2- 172.21.15.83:0/2034942192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa864004d50 0x7fa8800ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.545+0000 7fa887c00640 1 -- 172.21.15.83:0/2034942192 >> 172.21.15.83:0/2034942192 conn(0x7fa8800b1830 msgr2=0x7fa8800b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:47.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.545+0000 7fa887c00640 1 -- 172.21.15.83:0/2034942192 shutdown_connections 2024-04-25T00:12:47.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.545+0000 7fa887c00640 1 -- 172.21.15.83:0/2034942192 wait complete. 2024-04-25T00:12:47.566 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:12:47.566 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:12:47.567 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:12:47.567 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:12:47.567 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:12:47.567 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:12:47.567 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:12:47.567 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:12:47.568 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:12:47.568 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:12:47.568 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:12:47.568 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:12:47.568 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:12:47.568 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:12:47.569 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:12:47.569 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:12:47.583 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:12:47.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:12:47.584 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:12:47.585 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:12:47.586 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:12:47.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:12:47.588 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:12:47.588 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:12:47.632 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:12:47.632 DEBUG:teuthology.parallel:result is None 2024-04-25T00:12:47.633 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:12:47.633 DEBUG:teuthology.parallel:result is None 2024-04-25T00:12:47.634 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:12:47.634 DEBUG:teuthology.parallel:result is None 2024-04-25T00:12:47.635 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:12:47.635 DEBUG:teuthology.parallel:result is None 2024-04-25T00:12:47.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:12:47.718 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-25T00:12:47.719 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-25T00:12:47.721 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2024-04-25T00:12:47.722 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2024-04-25T00:12:47.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.777+0000 7f85b3a00640 1 Processor -- start 2024-04-25T00:12:47.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.777+0000 7f85b3a00640 1 -- start start 2024-04-25T00:12:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.777+0000 7f85b3a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85ac15bf50 0x7f85ac0cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.777+0000 7f85b3a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85ac0cf3d0 con 0x7f85ac0b28a0 2024-04-25T00:12:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.777+0000 7f85b3a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85ac0cfc00 con 0x7f85ac0b1100 2024-04-25T00:12:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.777+0000 7f85b3a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f85ac0d0c90 con 0x7f85ac15bf50 2024-04-25T00:12:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.781+0000 7f85b1000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85ac15bf50 0x7f85ac0cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.781+0000 7f85b1a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f85ac0b28a0 0x7f85ac0cecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55858/0 (socket says 172.21.15.83:55858) 2024-04-25T00:12:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.781+0000 7f85b1a00640 1 -- 172.21.15.83:0/2520729902 learned_addr learned my addr 172.21.15.83:0/2520729902 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.781+0000 7f85b1000640 1 -- 172.21.15.83:0/2520729902 >> v1:172.21.15.86:6789/0 conn(0x7f85ac0b1100 legacy=0x7f85ac0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.781+0000 7f85b1000640 1 -- 172.21.15.83:0/2520729902 >> v1:172.21.15.83:6789/0 conn(0x7f85ac0b28a0 legacy=0x7f85ac0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.781+0000 7f85b1000640 1 -- 172.21.15.83:0/2520729902 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ac0cd310 con 0x7f85ac15bf50 2024-04-25T00:12:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.781+0000 7f85abe00640 1 -- 172.21.15.83:0/2520729902 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 927625237 0 0) 0x7f85ac0cf3d0 con 0x7f85ac0b28a0 2024-04-25T00:12:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.781+0000 7f85b1000640 1 --2- 172.21.15.83:0/2520729902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac15bf50 0x7f85ac0cfe50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f859c0097c0 tx=0x7f859c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85021f2f8894a635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:47.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.781+0000 7f85abe00640 1 -- 172.21.15.83:0/2520729902 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0b0070 con 0x7f85ac15bf50 2024-04-25T00:12:47.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.781+0000 7f85abe00640 1 -- 172.21.15.83:0/2520729902 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f859c0ab600 con 0x7f85ac15bf50 2024-04-25T00:12:47.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.781+0000 7f85abe00640 1 -- 172.21.15.83:0/2520729902 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0ab9d0 con 0x7f85ac15bf50 2024-04-25T00:12:47.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 -- 172.21.15.83:0/2520729902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac15bf50 msgr2=0x7f85ac0cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 --2- 172.21.15.83:0/2520729902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac15bf50 0x7f85ac0cfe50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f859c0097c0 tx=0x7f859c01c410 comp rx=0 tx=0).stop 2024-04-25T00:12:47.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 -- 172.21.15.83:0/2520729902 shutdown_connections 2024-04-25T00:12:47.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 --2- 172.21.15.83:0/2520729902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac15bf50 0x7f85ac0cfe50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 -- 172.21.15.83:0/2520729902 >> 172.21.15.83:0/2520729902 conn(0x7f85ac0d1c50 msgr2=0x7f85ac0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:47.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 -- 172.21.15.83:0/2520729902 shutdown_connections 2024-04-25T00:12:47.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 -- 172.21.15.83:0/2520729902 wait complete. 2024-04-25T00:12:47.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 Processor -- start 2024-04-25T00:12:47.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 -- start start 2024-04-25T00:12:47.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85ac0b1100 0x7f85ac0af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ac0b28a0 0x7f85ac152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac15bf50 0x7f85ac155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac15c330 con 0x7f85ac15bf50 2024-04-25T00:12:47.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac0d07a0 con 0x7f85ac0b28a0 2024-04-25T00:12:47.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b3a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f85ac15c4b0 con 0x7f85ac0b1100 2024-04-25T00:12:47.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac15bf50 0x7f85ac155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac15bf50 0x7f85ac155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54864/0 (socket says 172.21.15.83:54864) 2024-04-25T00:12:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b1a00640 1 -- 172.21.15.83:0/501936934 learned_addr learned my addr 172.21.15.83:0/501936934 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85a3e00640 1 --2- 172.21.15.83:0/501936934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ac0b28a0 0x7f85ac152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b1000640 1 --2- 172.21.15.83:0/501936934 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85ac0b1100 0x7f85ac0af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b1a00640 1 -- 172.21.15.83:0/501936934 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85ac0b1100 msgr2=0x7f85ac0af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b1a00640 1 --2- 172.21.15.83:0/501936934 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85ac0b1100 0x7f85ac0af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b1a00640 1 -- 172.21.15.83:0/501936934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ac0b28a0 msgr2=0x7f85ac152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b1a00640 1 --2- 172.21.15.83:0/501936934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ac0b28a0 0x7f85ac152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.785+0000 7f85b1a00640 1 -- 172.21.15.83:0/501936934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ac1527b0 con 0x7f85ac15bf50 2024-04-25T00:12:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.789+0000 7f85b1a00640 1 --2- 172.21.15.83:0/501936934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac15bf50 0x7f85ac155eb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f85a400c780 tx=0x7f85a400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.789+0000 7f85a9600640 1 -- 172.21.15.83:0/501936934 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4018070 con 0x7f85ac15bf50 2024-04-25T00:12:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.789+0000 7f85a9600640 1 -- 172.21.15.83:0/501936934 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a400a640 con 0x7f85ac15bf50 2024-04-25T00:12:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.789+0000 7f85b3a00640 1 -- 172.21.15.83:0/501936934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ac15ef70 con 0x7f85ac15bf50 2024-04-25T00:12:47.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.789+0000 7f85a9600640 1 -- 172.21.15.83:0/501936934 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a400aa70 con 0x7f85ac15bf50 2024-04-25T00:12:47.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.789+0000 7f85b3a00640 1 -- 172.21.15.83:0/501936934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ac1553d0 con 0x7f85ac15bf50 2024-04-25T00:12:47.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T00:12:47.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.789+0000 7f85b3a00640 1 -- 172.21.15.83:0/501936934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85ac068230 con 0x7f85ac15bf50 2024-04-25T00:12:47.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.789+0000 7f85a9600640 1 -- 172.21.15.83:0/501936934 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f85a401b070 con 0x7f85ac15bf50 2024-04-25T00:12:47.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.789+0000 7f85a9600640 1 --2- 172.21.15.83:0/501936934 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85880678c0 0x7f8588069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.789+0000 7f85a9600640 1 -- 172.21.15.83:0/501936934 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7f85a4085100 con 0x7f85ac15bf50 2024-04-25T00:12:47.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.789+0000 7f85b1000640 1 --2- 172.21.15.83:0/501936934 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85880678c0 0x7f8588069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.793+0000 7f85b1000640 1 --2- 172.21.15.83:0/501936934 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85880678c0 0x7f8588069d80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f859c0ad000 tx=0x7f859c0a4f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:47.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.793+0000 7f85a9600640 1 -- 172.21.15.83:0/501936934 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f85a4055750 con 0x7f85ac15bf50 2024-04-25T00:12:47.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:47.961+0000 7f85b3a00640 1 -- 172.21.15.83:0/501936934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f85ac074850 con 0x7f85ac15bf50 2024-04-25T00:12:48.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.613+0000 7f85a9600640 1 -- 172.21.15.83:0/501936934 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) ==== 97+0+0 (secure 0 0 0) 0x7f85a4059a50 con 0x7f85ac15bf50 2024-04-25T00:12:48.617 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-25T00:12:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.617+0000 7f85b3a00640 1 -- 172.21.15.83:0/501936934 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85880678c0 msgr2=0x7f8588069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.617+0000 7f85b3a00640 1 --2- 172.21.15.83:0/501936934 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85880678c0 0x7f8588069d80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f859c0ad000 tx=0x7f859c0a4f90 comp rx=0 tx=0).stop 2024-04-25T00:12:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.617+0000 7f85b3a00640 1 -- 172.21.15.83:0/501936934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac15bf50 msgr2=0x7f85ac155eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.617+0000 7f85b3a00640 1 --2- 172.21.15.83:0/501936934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac15bf50 0x7f85ac155eb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f85a400c780 tx=0x7f85a400cc40 comp rx=0 tx=0).stop 2024-04-25T00:12:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.617+0000 7f85b3a00640 1 -- 172.21.15.83:0/501936934 shutdown_connections 2024-04-25T00:12:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.617+0000 7f85b3a00640 1 --2- 172.21.15.83:0/501936934 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85880678c0 0x7f8588069d80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.617+0000 7f85b3a00640 1 --2- 172.21.15.83:0/501936934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac15bf50 0x7f85ac155eb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.617+0000 7f85b3a00640 1 --2- 172.21.15.83:0/501936934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ac0b28a0 0x7f85ac152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.617+0000 7f85b3a00640 1 --2- 172.21.15.83:0/501936934 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85ac0b1100 0x7f85ac0af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.617+0000 7f85b3a00640 1 -- 172.21.15.83:0/501936934 >> 172.21.15.83:0/501936934 conn(0x7f85ac0d1c50 msgr2=0x7f85ac06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.617+0000 7f85b3a00640 1 -- 172.21.15.83:0/501936934 shutdown_connections 2024-04-25T00:12:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.617+0000 7f85b3a00640 1 -- 172.21.15.83:0/501936934 wait complete. 2024-04-25T00:12:48.644 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:12:48.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.777+0000 7fc32da00640 1 Processor -- start 2024-04-25T00:12:48.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.777+0000 7fc32da00640 1 -- start start 2024-04-25T00:12:48.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.781+0000 7fc32da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc3280b4360 0x7fc3280b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:48.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.781+0000 7fc32da00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3280b2b30 0x7fc3280b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:48.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.781+0000 7fc32da00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc32806ef80 con 0x7fc3281553e0 2024-04-25T00:12:48.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.781+0000 7fc32da00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc32815b510 con 0x7fc3280b4360 2024-04-25T00:12:48.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.781+0000 7fc32da00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc32815b7f0 con 0x7fc3280b2b30 2024-04-25T00:12:48.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.781+0000 7fc31fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc3280b4360 0x7fc3280b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:48.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.781+0000 7fc327400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3280b2b30 0x7fc3280b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:48.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.781+0000 7fc31fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc3280b4360 0x7fc3280b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:40674/0 (socket says 172.21.15.83:40674) 2024-04-25T00:12:48.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.781+0000 7fc31fe00640 1 -- 172.21.15.83:0/2145086631 learned_addr learned my addr 172.21.15.83:0/2145086631 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:48.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.781+0000 7fc31fe00640 1 -- 172.21.15.83:0/2145086631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3280b2b30 msgr2=0x7fc3280b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:48.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.781+0000 7fc31fe00640 1 --2- 172.21.15.83:0/2145086631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3280b2b30 0x7fc3280b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:48.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.781+0000 7fc31fe00640 1 -- 172.21.15.83:0/2145086631 >> v1:172.21.15.86:6789/0 conn(0x7fc3281553e0 legacy=0x7fc32806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:48.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.781+0000 7fc31fe00640 1 -- 172.21.15.83:0/2145086631 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32815bad0 con 0x7fc3280b4360 2024-04-25T00:12:48.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.785+0000 7fc31fe00640 1 --2- 172.21.15.83:0/2145086631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3280b4360 0x7fc3280b47e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc310009870 tx=0x7fc31001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b78df176b1b39c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.785+0000 7fc326000640 1 -- 172.21.15.83:0/2145086631 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3100b0070 con 0x7fc3280b4360 2024-04-25T00:12:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.785+0000 7fc326000640 1 -- 172.21.15.83:0/2145086631 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3100ab600 con 0x7fc3280b4360 2024-04-25T00:12:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.785+0000 7fc326000640 1 -- 172.21.15.83:0/2145086631 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3100ab9d0 con 0x7fc3280b4360 2024-04-25T00:12:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.785+0000 7fc32da00640 1 -- 172.21.15.83:0/2145086631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3280b4360 msgr2=0x7fc3280b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:48.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.785+0000 7fc32da00640 1 --2- 172.21.15.83:0/2145086631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3280b4360 0x7fc3280b47e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc310009870 tx=0x7fc31001c4d0 comp rx=0 tx=0).stop 2024-04-25T00:12:48.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.785+0000 7fc32da00640 1 -- 172.21.15.83:0/2145086631 shutdown_connections 2024-04-25T00:12:48.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.785+0000 7fc32da00640 1 --2- 172.21.15.83:0/2145086631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3280b2b30 0x7fc3280b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:48.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.785+0000 7fc32da00640 1 --2- 172.21.15.83:0/2145086631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3280b4360 0x7fc3280b47e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:48.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.785+0000 7fc32da00640 1 -- 172.21.15.83:0/2145086631 >> 172.21.15.83:0/2145086631 conn(0x7fc3280b1830 msgr2=0x7fc3280b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:48.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.785+0000 7fc32da00640 1 -- 172.21.15.83:0/2145086631 shutdown_connections 2024-04-25T00:12:48.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.785+0000 7fc32da00640 1 -- 172.21.15.83:0/2145086631 wait complete. 2024-04-25T00:12:48.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.785+0000 7fc32da00640 1 Processor -- start 2024-04-25T00:12:48.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.785+0000 7fc32da00640 1 -- start start 2024-04-25T00:12:48.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc32da00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3280b2b30 0x7fc32814be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:48.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc32da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3280b4360 0x7fc32806daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:48.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc32da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3281553e0 0x7fc32806e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:48.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc32da00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc32806ebd0 con 0x7fc3281553e0 2024-04-25T00:12:48.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc32da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc328167d90 con 0x7fc3280b4360 2024-04-25T00:12:48.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc32da00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc3280ce9a0 con 0x7fc3280b2b30 2024-04-25T00:12:48.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc327400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3281553e0 0x7fc32806e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc31fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3280b4360 0x7fc32806daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc327400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3281553e0 0x7fc32806e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54870/0 (socket says 172.21.15.83:54870) 2024-04-25T00:12:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc327400640 1 -- 172.21.15.83:0/2951799682 learned_addr learned my addr 172.21.15.83:0/2951799682 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc326a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3280b2b30 0x7fc32814be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc31fe00640 1 -- 172.21.15.83:0/2951799682 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3280b2b30 msgr2=0x7fc32814be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc31fe00640 1 --2- 172.21.15.83:0/2951799682 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3280b2b30 0x7fc32814be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc31fe00640 1 -- 172.21.15.83:0/2951799682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3281553e0 msgr2=0x7fc32806e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc31fe00640 1 --2- 172.21.15.83:0/2951799682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3281553e0 0x7fc32806e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc31fe00640 1 -- 172.21.15.83:0/2951799682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3280cee30 con 0x7fc3280b4360 2024-04-25T00:12:48.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc327400640 1 --2- 172.21.15.83:0/2951799682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3281553e0 0x7fc32806e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:48.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc31fe00640 1 --2- 172.21.15.83:0/2951799682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3280b4360 0x7fc32806daf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc328068be0 tx=0x7fc310019270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:48.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc31ea00640 1 -- 172.21.15.83:0/2951799682 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3100b0040 con 0x7fc3280b4360 2024-04-25T00:12:48.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc31ea00640 1 -- 172.21.15.83:0/2951799682 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3100b4420 con 0x7fc3280b4360 2024-04-25T00:12:48.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc31ea00640 1 -- 172.21.15.83:0/2951799682 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3100b4830 con 0x7fc3280b4360 2024-04-25T00:12:48.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc32da00640 1 -- 172.21.15.83:0/2951799682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3280ce0b0 con 0x7fc3280b4360 2024-04-25T00:12:48.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc32da00640 1 -- 172.21.15.83:0/2951799682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3280ce350 con 0x7fc3280b4360 2024-04-25T00:12:48.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.789+0000 7fc32da00640 1 -- 172.21.15.83:0/2951799682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3280737f0 con 0x7fc3280b4360 2024-04-25T00:12:48.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.793+0000 7fc31ea00640 1 -- 172.21.15.83:0/2951799682 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc3100b49d0 con 0x7fc3280b4360 2024-04-25T00:12:48.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.793+0000 7fc31ea00640 1 --2- 172.21.15.83:0/2951799682 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc3040678c0 0x7fc304069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:48.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.793+0000 7fc31ea00640 1 -- 172.21.15.83:0/2951799682 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5249+0+0 (secure 0 0 0) 0x7fc31011f470 con 0x7fc3280b4360 2024-04-25T00:12:48.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.793+0000 7fc326a00640 1 --2- 172.21.15.83:0/2951799682 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc3040678c0 0x7fc304069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:48.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.793+0000 7fc326a00640 1 --2- 172.21.15.83:0/2951799682 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc3040678c0 0x7fc304069d80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc3280cf080 tx=0x7fc318009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:48.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:48.797+0000 7fc31ea00640 1 -- 172.21.15.83:0/2951799682 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc310016150 con 0x7fc3280b4360 2024-04-25T00:12:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.005+0000 7fc32da00640 1 -- 172.21.15.83:0/2951799682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc32804f5f0 con 0x7fc3280b4360 2024-04-25T00:12:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.005+0000 7fc31ea00640 1 -- 172.21.15.83:0/2951799682 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7fc3100efa80 con 0x7fc3280b4360 2024-04-25T00:12:49.008 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:49.009 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:12:49.009 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-25T00:12:49.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.009 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-25T00:12:49.009 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmytlRFRAA3vHLqK3vHSB+8+TZHAuj7Q==", 2024-04-25T00:12:49.010 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.010 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:49.010 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:12:49.010 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:12:49.011 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.011 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.011 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-25T00:12:49.011 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm088MIRAApJvvePXyFoO8JnVErd57zQ==", 2024-04-25T00:12:49.011 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.011 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:49.011 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:12:49.011 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:12:49.012 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.012 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.012 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-25T00:12:49.012 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmCqZXHBAAPy5JGbEU11i4gc3gFtbGIQ==", 2024-04-25T00:12:49.012 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.012 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:49.012 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:12:49.012 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:12:49.013 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.013 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.013 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-25T00:12:49.013 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmiJZGKBAA5CBwH/cA+LOAj+inS/Hx2g==", 2024-04-25T00:12:49.013 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.013 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:49.013 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:12:49.014 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:12:49.014 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.014 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.014 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-25T00:12:49.014 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBhnylmMOiqLRAAdqrCTz4KfghoJ7cZc+b4qg==", 2024-04-25T00:12:49.014 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.014 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:49.014 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:49.015 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.015 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.015 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-25T00:12:49.015 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBknylmlp8OOBAAl+y4vBrCzDyV/ne9qICQ+w==", 2024-04-25T00:12:49.015 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.015 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:49.015 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:49.015 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.015 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.016 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-25T00:12:49.016 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBonylm85MOBhAA3ciY1UT/Qg4Pe72OA0IvnA==", 2024-04-25T00:12:49.016 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.016 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:49.016 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:49.016 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.016 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.016 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-25T00:12:49.016 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBrnylmnAbDEhAAkCQXjbiKINiftX71HBKvQA==", 2024-04-25T00:12:49.017 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.017 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:49.017 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:49.017 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.017 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.017 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-25T00:12:49.017 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBxnylmxhy2KxAAs6qjxZA+Fj5umHU/4FVqGw==", 2024-04-25T00:12:49.017 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.017 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:49.017 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:49.017 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.018 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.018 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-25T00:12:49.018 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB0nylmGfSlNRAA45tuuh+5ISOMlhaFdCE5rQ==", 2024-04-25T00:12:49.018 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.018 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:49.018 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:49.018 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.018 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.019 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-25T00:12:49.019 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB4nylmouyBARAAz4s4S1BSiDU9YxQ/Vj0d0A==", 2024-04-25T00:12:49.019 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.019 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:49.019 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:49.019 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.019 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.019 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-25T00:12:49.019 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB7nylmtIh0CxAApWJLvC8Zlt298VJ55gPrqw==", 2024-04-25T00:12:49.019 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.020 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:12:49.020 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:12:49.020 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.020 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.020 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-25T00:12:49.020 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmOqooLBAApte+X6YkRmrpe12YdjYekg==", 2024-04-25T00:12:49.020 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.020 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:49.020 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-04-25T00:12:49.020 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:12:49.021 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.021 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-25T00:12:49.021 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmDH93NRAASQzJNC3nI0NeU8cv951eBA==", 2024-04-25T00:12:49.021 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.021 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:49.021 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:12:49.021 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:12:49.021 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:12:49.021 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.022 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-25T00:12:49.022 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmHw3+MRAAciXRmexPxmrf7U2jKwpZQQ==", 2024-04-25T00:12:49.022 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.022 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:49.022 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:12:49.022 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:12:49.022 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:12:49.022 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.023 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-25T00:12:49.023 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBfnylmIsMCABAAu+yzPRupFFqA/yH4GSZ+8w==", 2024-04-25T00:12:49.023 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.023 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:12:49.023 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:12:49.023 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:12:49.023 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:12:49.023 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.024 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-25T00:12:49.024 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdnylmZ7SKNRAArVHYbh7fBiOUNRTenIwT1w==", 2024-04-25T00:12:49.024 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.024 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:12:49.024 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:12:49.024 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:12:49.024 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.024 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.024 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-25T00:12:49.025 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm6TMqERAAiPL81HRFrlz4ku20HemS4w==", 2024-04-25T00:12:49.025 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.025 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:12:49.025 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-25T00:12:49.025 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.025 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:12:49.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:49.025 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-25T00:12:49.026 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm9VRpDRAA5XUtYwjiWBpF1ObJ/Y5g0w==", 2024-04-25T00:12:49.026 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:12:49.026 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:12:49.026 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-25T00:12:49.026 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:12:49.026 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.026 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:49.026 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T00:12:49.026 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:12:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.009+0000 7fc32da00640 1 -- 172.21.15.83:0/2951799682 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc3040678c0 msgr2=0x7fc304069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.009+0000 7fc32da00640 1 --2- 172.21.15.83:0/2951799682 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc3040678c0 0x7fc304069d80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc3280cf080 tx=0x7fc318009040 comp rx=0 tx=0).stop 2024-04-25T00:12:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.009+0000 7fc32da00640 1 -- 172.21.15.83:0/2951799682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3280b4360 msgr2=0x7fc32806daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.009+0000 7fc32da00640 1 --2- 172.21.15.83:0/2951799682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3280b4360 0x7fc32806daf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc328068be0 tx=0x7fc310019270 comp rx=0 tx=0).stop 2024-04-25T00:12:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.009+0000 7fc32da00640 1 -- 172.21.15.83:0/2951799682 shutdown_connections 2024-04-25T00:12:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.009+0000 7fc32da00640 1 --2- 172.21.15.83:0/2951799682 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc3040678c0 0x7fc304069d80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.009+0000 7fc32da00640 1 --2- 172.21.15.83:0/2951799682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3281553e0 0x7fc32806e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.009+0000 7fc32da00640 1 --2- 172.21.15.83:0/2951799682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3280b4360 0x7fc32806daf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.009+0000 7fc32da00640 1 --2- 172.21.15.83:0/2951799682 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3280b2b30 0x7fc32814be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.009+0000 7fc32da00640 1 -- 172.21.15.83:0/2951799682 >> 172.21.15.83:0/2951799682 conn(0x7fc3280b1830 msgr2=0x7fc328152fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:49.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.009+0000 7fc32da00640 1 -- 172.21.15.83:0/2951799682 shutdown_connections 2024-04-25T00:12:49.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.009+0000 7fc32da00640 1 -- 172.21.15.83:0/2951799682 wait complete. 2024-04-25T00:12:49.030 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3cdc00640 1 Processor -- start 2024-04-25T00:12:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3cdc00640 1 -- start start 2024-04-25T00:12:49.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3cdc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa3c815a080 0x7fa3c806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3cdc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3c80b4350 0x7fa3c8070da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3cdc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3c815a790 con 0x7fa3c806dc30 2024-04-25T00:12:49.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3cdc00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa3c806ce70 con 0x7fa3c815a080 2024-04-25T00:12:49.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3cdc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c815b510 con 0x7fa3c80b4350 2024-04-25T00:12:49.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3c6a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa3c815a080 0x7fa3c806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3c6a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa3c815a080 0x7fa3c806d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54874/0 (socket says 172.21.15.83:54874) 2024-04-25T00:12:49.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3c6a00640 1 -- 172.21.15.83:0/100161031 learned_addr learned my addr 172.21.15.83:0/100161031 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:49.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3c7400640 1 --2- 172.21.15.83:0/100161031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3c80b4350 0x7fa3c8070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3c6a00640 1 -- 172.21.15.83:0/100161031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3c80b4350 msgr2=0x7fa3c8070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:49.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3c6a00640 1 --2- 172.21.15.83:0/100161031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3c80b4350 0x7fa3c8070da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3c6a00640 1 -- 172.21.15.83:0/100161031 >> v1:172.21.15.86:6789/0 conn(0x7fa3c806dc30 legacy=0x7fa3c80b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.165+0000 7fa3c6a00640 1 -- 172.21.15.83:0/100161031 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c806e2b0 con 0x7fa3c815a080 2024-04-25T00:12:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.169+0000 7fa3c6a00640 1 --2- 172.21.15.83:0/100161031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c815a080 0x7fa3c806d6f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009870 tx=0x7fa3b801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1537920d94758c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.169+0000 7fa3c5600640 1 -- 172.21.15.83:0/100161031 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80b0070 con 0x7fa3c815a080 2024-04-25T00:12:49.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.169+0000 7fa3c5600640 1 -- 172.21.15.83:0/100161031 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b80ab600 con 0x7fa3c815a080 2024-04-25T00:12:49.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.169+0000 7fa3c5600640 1 -- 172.21.15.83:0/100161031 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80ab9d0 con 0x7fa3c815a080 2024-04-25T00:12:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.169+0000 7fa3cdc00640 1 -- 172.21.15.83:0/100161031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c815a080 msgr2=0x7fa3c806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.169+0000 7fa3cdc00640 1 --2- 172.21.15.83:0/100161031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c815a080 0x7fa3c806d6f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009870 tx=0x7fa3b801c4d0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.169+0000 7fa3cdc00640 1 -- 172.21.15.83:0/100161031 shutdown_connections 2024-04-25T00:12:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.169+0000 7fa3cdc00640 1 --2- 172.21.15.83:0/100161031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3c80b4350 0x7fa3c8070da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.169+0000 7fa3cdc00640 1 --2- 172.21.15.83:0/100161031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c815a080 0x7fa3c806d6f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.169+0000 7fa3cdc00640 1 -- 172.21.15.83:0/100161031 >> 172.21.15.83:0/100161031 conn(0x7fa3c80cfaa0 msgr2=0x7fa3c80cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.169+0000 7fa3cdc00640 1 -- 172.21.15.83:0/100161031 shutdown_connections 2024-04-25T00:12:49.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.169+0000 7fa3cdc00640 1 -- 172.21.15.83:0/100161031 wait complete. 2024-04-25T00:12:49.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3cdc00640 1 Processor -- start 2024-04-25T00:12:49.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3cdc00640 1 -- start start 2024-04-25T00:12:49.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3cdc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3c806dc30 0x7fa3c814b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3cdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c80b4350 0x7fa3c814aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3cdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c815a080 0x7fa3c8151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3cdc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8071af0 con 0x7fa3c815a080 2024-04-25T00:12:49.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3cdc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c815a460 con 0x7fa3c80b4350 2024-04-25T00:12:49.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3cdc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c8071c70 con 0x7fa3c806dc30 2024-04-25T00:12:49.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3c7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c815a080 0x7fa3c8151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3c7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c815a080 0x7fa3c8151dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54878/0 (socket says 172.21.15.83:54878) 2024-04-25T00:12:49.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3c6a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3c806dc30 0x7fa3c814b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3c7400640 1 -- 172.21.15.83:0/1168421142 learned_addr learned my addr 172.21.15.83:0/1168421142 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:49.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3c6000640 1 --2- 172.21.15.83:0/1168421142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c80b4350 0x7fa3c814aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3c7400640 1 -- 172.21.15.83:0/1168421142 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3c806dc30 msgr2=0x7fa3c814b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3c7400640 1 --2- 172.21.15.83:0/1168421142 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3c806dc30 0x7fa3c814b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3c7400640 1 -- 172.21.15.83:0/1168421142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c80b4350 msgr2=0x7fa3c814aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3c7400640 1 --2- 172.21.15.83:0/1168421142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c80b4350 0x7fa3c814aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3c7400640 1 -- 172.21.15.83:0/1168421142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c81f76f0 con 0x7fa3c815a080 2024-04-25T00:12:49.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3c6a00640 1 --2- 172.21.15.83:0/1168421142 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3c806dc30 0x7fa3c814b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:49.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3c7400640 1 --2- 172.21.15.83:0/1168421142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c815a080 0x7fa3c8151dc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0007e30 tx=0x7fa3c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:49.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3bea00640 1 -- 172.21.15.83:0/1168421142 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0014620 con 0x7fa3c815a080 2024-04-25T00:12:49.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3bea00640 1 -- 172.21.15.83:0/1168421142 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c0019070 con 0x7fa3c815a080 2024-04-25T00:12:49.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3bea00640 1 -- 172.21.15.83:0/1168421142 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0014a00 con 0x7fa3c815a080 2024-04-25T00:12:49.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3cdc00640 1 -- 172.21.15.83:0/1168421142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c81f79d0 con 0x7fa3c815a080 2024-04-25T00:12:49.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3cdc00640 1 -- 172.21.15.83:0/1168421142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c80cdcc0 con 0x7fa3c815a080 2024-04-25T00:12:49.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.173+0000 7fa3cdc00640 1 -- 172.21.15.83:0/1168421142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c80737f0 con 0x7fa3c815a080 2024-04-25T00:12:49.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.177+0000 7fa3bea00640 1 -- 172.21.15.83:0/1168421142 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa3c0014ba0 con 0x7fa3c815a080 2024-04-25T00:12:49.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.177+0000 7fa3bea00640 1 --2- 172.21.15.83:0/1168421142 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3a80678c0 0x7fa3a8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.177+0000 7fa3bea00640 1 -- 172.21.15.83:0/1168421142 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5249+0+0 (secure 0 0 0) 0x7fa3c0086210 con 0x7fa3c815a080 2024-04-25T00:12:49.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.177+0000 7fa3c6a00640 1 --2- 172.21.15.83:0/1168421142 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3a80678c0 0x7fa3a8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.177+0000 7fa3c6a00640 1 --2- 172.21.15.83:0/1168421142 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3a80678c0 0x7fa3a8069d80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80ad000 tx=0x7fa3b80a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:49.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.181+0000 7fa3bea00640 1 -- 172.21.15.83:0/1168421142 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa3c00568b0 con 0x7fa3c815a080 2024-04-25T00:12:49.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.341+0000 7fa3cdc00640 1 -- 172.21.15.83:0/1168421142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3c804f5f0 con 0x7fa3c815a080 2024-04-25T00:12:49.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.341+0000 7fa3bea00640 1 -- 172.21.15.83:0/1168421142 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+13080 (secure 0 0 0) 0x7fa3c005abb0 con 0x7fa3c815a080 2024-04-25T00:12:49.343 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:49.344 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":45,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:48.629371+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.341+0000 7fa3cdc00640 1 -- 172.21.15.83:0/1168421142 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3a80678c0 msgr2=0x7fa3a8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.341+0000 7fa3cdc00640 1 --2- 172.21.15.83:0/1168421142 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3a80678c0 0x7fa3a8069d80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80ad000 tx=0x7fa3b80a6000 comp rx=0 tx=0).stop 2024-04-25T00:12:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.345+0000 7fa3cdc00640 1 -- 172.21.15.83:0/1168421142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c815a080 msgr2=0x7fa3c8151dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.345+0000 7fa3cdc00640 1 --2- 172.21.15.83:0/1168421142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c815a080 0x7fa3c8151dc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0007e30 tx=0x7fa3c000e4b0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.345+0000 7fa3cdc00640 1 -- 172.21.15.83:0/1168421142 shutdown_connections 2024-04-25T00:12:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.345+0000 7fa3cdc00640 1 --2- 172.21.15.83:0/1168421142 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3a80678c0 0x7fa3a8069d80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.345+0000 7fa3cdc00640 1 --2- 172.21.15.83:0/1168421142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c815a080 0x7fa3c8151dc0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.345+0000 7fa3cdc00640 1 --2- 172.21.15.83:0/1168421142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c80b4350 0x7fa3c814aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.345+0000 7fa3cdc00640 1 --2- 172.21.15.83:0/1168421142 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3c806dc30 0x7fa3c814b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.345+0000 7fa3cdc00640 1 -- 172.21.15.83:0/1168421142 >> 172.21.15.83:0/1168421142 conn(0x7fa3c80cfaa0 msgr2=0x7fa3c80b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.345+0000 7fa3cdc00640 1 -- 172.21.15.83:0/1168421142 shutdown_connections 2024-04-25T00:12:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.345+0000 7fa3cdc00640 1 -- 172.21.15.83:0/1168421142 wait complete. 2024-04-25T00:12:49.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:12:49.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:12:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.501+0000 7f2e63c00640 1 Processor -- start 2024-04-25T00:12:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.501+0000 7f2e63c00640 1 -- start start 2024-04-25T00:12:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.501+0000 7f2e63c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c1555c0 0x7f2e5c15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.501+0000 7f2e63c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e5c06e4c0 0x7f2e5c06e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.501+0000 7f2e63c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e5c15c560 con 0x7f2e5c1551e0 2024-04-25T00:12:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.501+0000 7f2e63c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2e5c06ee70 con 0x7f2e5c06e4c0 2024-04-25T00:12:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.501+0000 7f2e63c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2e5c072460 con 0x7f2e5c1555c0 2024-04-25T00:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.501+0000 7f2e61c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e5c06e4c0 0x7f2e5c06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.501+0000 7f2e61200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2e5c1551e0 0x7f2e5c071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55870/0 (socket says 172.21.15.83:55870) 2024-04-25T00:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.501+0000 7f2e61200640 1 -- 172.21.15.83:0/1864383921 learned_addr learned my addr 172.21.15.83:0/1864383921 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e53e00640 1 --2- 172.21.15.83:0/1864383921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c1555c0 0x7f2e5c15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e61c00640 1 -- 172.21.15.83:0/1864383921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c1555c0 msgr2=0x7f2e5c15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e61c00640 1 --2- 172.21.15.83:0/1864383921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c1555c0 0x7f2e5c15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e61c00640 1 -- 172.21.15.83:0/1864383921 >> v1:172.21.15.83:6789/0 conn(0x7f2e5c1551e0 legacy=0x7f2e5c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e61c00640 1 -- 172.21.15.83:0/1864383921 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c15c7d0 con 0x7f2e5c06e4c0 2024-04-25T00:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e5be00640 1 -- 172.21.15.83:0/1864383921 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3600057321 0 0) 0x7f2e5c15c560 con 0x7f2e5c1551e0 2024-04-25T00:12:49.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e61c00640 1 --2- 172.21.15.83:0/1864383921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c06e4c0 0x7f2e5c06e930 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009d90 tx=0x7f2e5401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62e305f0c1e04a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:49.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e5be00640 1 -- 172.21.15.83:0/1864383921 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e540b2070 con 0x7f2e5c06e4c0 2024-04-25T00:12:49.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e5be00640 1 -- 172.21.15.83:0/1864383921 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e540a5940 con 0x7f2e5c06e4c0 2024-04-25T00:12:49.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e5be00640 1 -- 172.21.15.83:0/1864383921 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e540a5d10 con 0x7f2e5c06e4c0 2024-04-25T00:12:49.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e63c00640 1 -- 172.21.15.83:0/1864383921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c06e4c0 msgr2=0x7f2e5c06e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e63c00640 1 --2- 172.21.15.83:0/1864383921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c06e4c0 0x7f2e5c06e930 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009d90 tx=0x7f2e5401c970 comp rx=0 tx=0).stop 2024-04-25T00:12:49.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e63c00640 1 -- 172.21.15.83:0/1864383921 shutdown_connections 2024-04-25T00:12:49.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e63c00640 1 --2- 172.21.15.83:0/1864383921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c06e4c0 0x7f2e5c06e930 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e63c00640 1 --2- 172.21.15.83:0/1864383921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c1555c0 0x7f2e5c15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e63c00640 1 -- 172.21.15.83:0/1864383921 >> 172.21.15.83:0/1864383921 conn(0x7f2e5c0b1830 msgr2=0x7f2e5c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:49.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.505+0000 7f2e63c00640 1 -- 172.21.15.83:0/1864383921 shutdown_connections 2024-04-25T00:12:49.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e63c00640 1 -- 172.21.15.83:0/1864383921 wait complete. 2024-04-25T00:12:49.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e63c00640 1 Processor -- start 2024-04-25T00:12:49.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e63c00640 1 -- start start 2024-04-25T00:12:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e63c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e5c06e4c0 0x7f2e5c0ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e63c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c1551e0 0x7f2e5c14f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e63c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c1555c0 0x7f2e5c147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e63c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c15db70 con 0x7f2e5c1555c0 2024-04-25T00:12:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e63c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c15d9f0 con 0x7f2e5c06e4c0 2024-04-25T00:12:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e63c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2e5c15dcf0 con 0x7f2e5c1551e0 2024-04-25T00:12:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e61c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c1555c0 0x7f2e5c147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e61c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c1555c0 0x7f2e5c147100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54902/0 (socket says 172.21.15.83:54902) 2024-04-25T00:12:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e61c00640 1 -- 172.21.15.83:0/1006173907 learned_addr learned my addr 172.21.15.83:0/1006173907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e61200640 1 --2- 172.21.15.83:0/1006173907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e5c06e4c0 0x7f2e5c0ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e53e00640 1 --2- 172.21.15.83:0/1006173907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c1551e0 0x7f2e5c14f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e61c00640 1 -- 172.21.15.83:0/1006173907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c1551e0 msgr2=0x7f2e5c14f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e61c00640 1 --2- 172.21.15.83:0/1006173907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c1551e0 0x7f2e5c14f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e61c00640 1 -- 172.21.15.83:0/1006173907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e5c06e4c0 msgr2=0x7f2e5c0ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e61c00640 1 --2- 172.21.15.83:0/1006173907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e5c06e4c0 0x7f2e5c0ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e61c00640 1 -- 172.21.15.83:0/1006173907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c14fd50 con 0x7f2e5c1555c0 2024-04-25T00:12:49.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e61c00640 1 --2- 172.21.15.83:0/1006173907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c1555c0 0x7f2e5c147100 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2e540b0040 tx=0x7f2e54019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:49.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e59600640 1 -- 172.21.15.83:0/1006173907 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e540b2050 con 0x7f2e5c1555c0 2024-04-25T00:12:49.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e59600640 1 -- 172.21.15.83:0/1006173907 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e540b3040 con 0x7f2e5c1555c0 2024-04-25T00:12:49.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e59600640 1 -- 172.21.15.83:0/1006173907 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e540ad870 con 0x7f2e5c1555c0 2024-04-25T00:12:49.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.509+0000 7f2e63c00640 1 -- 172.21.15.83:0/1006173907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5c150030 con 0x7f2e5c1555c0 2024-04-25T00:12:49.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.513+0000 7f2e63c00640 1 -- 172.21.15.83:0/1006173907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5c1505f0 con 0x7f2e5c1555c0 2024-04-25T00:12:49.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.513+0000 7f2e63c00640 1 -- 172.21.15.83:0/1006173907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e5c04f510 con 0x7f2e5c1555c0 2024-04-25T00:12:49.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.513+0000 7f2e59600640 1 -- 172.21.15.83:0/1006173907 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2e540c0020 con 0x7f2e5c1555c0 2024-04-25T00:12:49.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.513+0000 7f2e59600640 1 --2- 172.21.15.83:0/1006173907 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e380678c0 0x7f2e38069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.513+0000 7f2e59600640 1 -- 172.21.15.83:0/1006173907 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5249+0+0 (secure 0 0 0) 0x7f2e54120480 con 0x7f2e5c1555c0 2024-04-25T00:12:49.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.513+0000 7f2e61200640 1 --2- 172.21.15.83:0/1006173907 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e380678c0 0x7f2e38069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.513+0000 7f2e61200640 1 --2- 172.21.15.83:0/1006173907 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e380678c0 0x7f2e38069d80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c007fd0 tx=0x7f2e4c007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:49.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.517+0000 7f2e59600640 1 -- 172.21.15.83:0/1006173907 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2e540c7020 con 0x7f2e5c1555c0 2024-04-25T00:12:49.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.677+0000 7f2e63c00640 1 -- 172.21.15.83:0/1006173907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2e5c06ce30 con 0x7f2e5c1555c0 2024-04-25T00:12:49.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.677+0000 7f2e59600640 1 -- 172.21.15.83:0/1006173907 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7f2e54016560 con 0x7f2e5c1555c0 2024-04-25T00:12:49.683 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:12:49.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.681+0000 7f2e63c00640 1 -- 172.21.15.83:0/1006173907 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e380678c0 msgr2=0x7f2e38069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.681+0000 7f2e63c00640 1 --2- 172.21.15.83:0/1006173907 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e380678c0 0x7f2e38069d80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c007fd0 tx=0x7f2e4c007d60 comp rx=0 tx=0).stop 2024-04-25T00:12:49.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.681+0000 7f2e63c00640 1 -- 172.21.15.83:0/1006173907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c1555c0 msgr2=0x7f2e5c147100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.681+0000 7f2e63c00640 1 --2- 172.21.15.83:0/1006173907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c1555c0 0x7f2e5c147100 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2e540b0040 tx=0x7f2e54019710 comp rx=0 tx=0).stop 2024-04-25T00:12:49.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.681+0000 7f2e63c00640 1 -- 172.21.15.83:0/1006173907 shutdown_connections 2024-04-25T00:12:49.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.681+0000 7f2e63c00640 1 --2- 172.21.15.83:0/1006173907 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e380678c0 0x7f2e38069d80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.681+0000 7f2e63c00640 1 --2- 172.21.15.83:0/1006173907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c1551e0 0x7f2e5c14f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.681+0000 7f2e63c00640 1 --2- 172.21.15.83:0/1006173907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c1555c0 0x7f2e5c147100 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.681+0000 7f2e63c00640 1 --2- 172.21.15.83:0/1006173907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e5c06e4c0 0x7f2e5c0ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.681+0000 7f2e63c00640 1 -- 172.21.15.83:0/1006173907 >> 172.21.15.83:0/1006173907 conn(0x7f2e5c0b1830 msgr2=0x7f2e5c15f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:49.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.681+0000 7f2e63c00640 1 -- 172.21.15.83:0/1006173907 shutdown_connections 2024-04-25T00:12:49.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.681+0000 7f2e63c00640 1 -- 172.21.15.83:0/1006173907 wait complete. 2024-04-25T00:12:49.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:12:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.833+0000 7f8a8f000640 1 Processor -- start 2024-04-25T00:12:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.833+0000 7f8a8f000640 1 -- start start 2024-04-25T00:12:49.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.837+0000 7f8a8f000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8a8806c2f0 0x7f8a880d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.837+0000 7f8a8f000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8a880d0e60 0x7f8a8815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.837+0000 7f8a8f000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a880cc470 con 0x7f8a8806f9e0 2024-04-25T00:12:49.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.837+0000 7f8a8f000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8a880ccda0 con 0x7f8a880d0e60 2024-04-25T00:12:49.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.837+0000 7f8a8f000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8a880cc750 con 0x7f8a8806c2f0 2024-04-25T00:12:49.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.837+0000 7f8a8d000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8a880d0e60 0x7f8a8815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.837+0000 7f8a8d000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8a880d0e60 0x7f8a8815d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54918/0 (socket says 172.21.15.83:54918) 2024-04-25T00:12:49.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.837+0000 7f8a8d000640 1 -- 172.21.15.83:0/3979202576 learned_addr learned my addr 172.21.15.83:0/3979202576 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:49.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.837+0000 7f8a87400640 1 --2- 172.21.15.83:0/3979202576 >> v2:172.21.15.86:3300/0 conn(0x7f8a8806c2f0 0x7f8a880d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.837+0000 7f8a8d000640 1 -- 172.21.15.83:0/3979202576 >> v2:172.21.15.86:3300/0 conn(0x7f8a8806c2f0 msgr2=0x7f8a880d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:49.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.837+0000 7f8a8d000640 1 --2- 172.21.15.83:0/3979202576 >> v2:172.21.15.86:3300/0 conn(0x7f8a8806c2f0 0x7f8a880d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.837+0000 7f8a8d000640 1 -- 172.21.15.83:0/3979202576 >> v1:172.21.15.86:6789/0 conn(0x7f8a8806f9e0 legacy=0x7f8a8806fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:49.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.837+0000 7f8a8d000640 1 -- 172.21.15.83:0/3979202576 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a880cca30 con 0x7f8a880d0e60 2024-04-25T00:12:49.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.837+0000 7f8a87400640 1 --2- 172.21.15.83:0/3979202576 >> v2:172.21.15.86:3300/0 conn(0x7f8a8806c2f0 0x7f8a880d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:49.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8d000640 1 --2- 172.21.15.83:0/3979202576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a880d0e60 0x7f8a8815d8f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8a88071bd0 tx=0x7f8a8001f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4231b7cd936e9466 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:49.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a86a00640 1 -- 172.21.15.83:0/3979202576 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a800b4070 con 0x7f8a880d0e60 2024-04-25T00:12:49.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a86a00640 1 -- 172.21.15.83:0/3979202576 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a800af400 con 0x7f8a880d0e60 2024-04-25T00:12:49.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a86a00640 1 -- 172.21.15.83:0/3979202576 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a800af7d0 con 0x7f8a880d0e60 2024-04-25T00:12:49.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 -- 172.21.15.83:0/3979202576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a880d0e60 msgr2=0x7f8a8815d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 --2- 172.21.15.83:0/3979202576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a880d0e60 0x7f8a8815d8f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8a88071bd0 tx=0x7f8a8001f1a0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 -- 172.21.15.83:0/3979202576 shutdown_connections 2024-04-25T00:12:49.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 --2- 172.21.15.83:0/3979202576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a880d0e60 0x7f8a8815d8f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 --2- 172.21.15.83:0/3979202576 >> v2:172.21.15.86:3300/0 conn(0x7f8a8806c2f0 0x7f8a880d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 -- 172.21.15.83:0/3979202576 >> 172.21.15.83:0/3979202576 conn(0x7f8a880d1c80 msgr2=0x7f8a880d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:49.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 -- 172.21.15.83:0/3979202576 shutdown_connections 2024-04-25T00:12:49.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 -- 172.21.15.83:0/3979202576 wait complete. 2024-04-25T00:12:49.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 Processor -- start 2024-04-25T00:12:49.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 -- start start 2024-04-25T00:12:49.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a8806c2f0 0x7f8a88151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a8806f9e0 0x7f8a880c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a880d0e60 0x7f8a88157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8815de30 con 0x7f8a880d0e60 2024-04-25T00:12:49.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8806f540 con 0x7f8a8806c2f0 2024-04-25T00:12:49.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8f000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8a8815dfb0 con 0x7f8a8806f9e0 2024-04-25T00:12:49.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a880d0e60 0x7f8a88157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a880d0e60 0x7f8a88157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54932/0 (socket says 172.21.15.83:54932) 2024-04-25T00:12:49.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a8d000640 1 -- 172.21.15.83:0/2686438104 learned_addr learned my addr 172.21.15.83:0/2686438104 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:49.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a87400640 1 --2- 172.21.15.83:0/2686438104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a8806f9e0 0x7f8a880c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.841+0000 7f8a87e00640 1 --2- 172.21.15.83:0/2686438104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a8806c2f0 0x7f8a88151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a8d000640 1 -- 172.21.15.83:0/2686438104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a8806f9e0 msgr2=0x7f8a880c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a8d000640 1 --2- 172.21.15.83:0/2686438104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a8806f9e0 0x7f8a880c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a8d000640 1 -- 172.21.15.83:0/2686438104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a8806c2f0 msgr2=0x7f8a88151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a8d000640 1 --2- 172.21.15.83:0/2686438104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a8806c2f0 0x7f8a88151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a8d000640 1 -- 172.21.15.83:0/2686438104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a880c6990 con 0x7f8a880d0e60 2024-04-25T00:12:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a8d000640 1 --2- 172.21.15.83:0/2686438104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a880d0e60 0x7f8a88157220 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8a800b2040 tx=0x7f8a8001bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a7fe00640 1 -- 172.21.15.83:0/2686438104 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a800b4040 con 0x7f8a880d0e60 2024-04-25T00:12:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a7fe00640 1 -- 172.21.15.83:0/2686438104 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a800c0070 con 0x7f8a880d0e60 2024-04-25T00:12:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a7fe00640 1 -- 172.21.15.83:0/2686438104 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80009510 con 0x7f8a880d0e60 2024-04-25T00:12:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a8f000640 1 -- 172.21.15.83:0/2686438104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a880c6c70 con 0x7f8a880d0e60 2024-04-25T00:12:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a8f000640 1 -- 172.21.15.83:0/2686438104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a880c7230 con 0x7f8a880d0e60 2024-04-25T00:12:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a8f000640 1 -- 172.21.15.83:0/2686438104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a8809fdd0 con 0x7f8a880d0e60 2024-04-25T00:12:49.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a7fe00640 1 -- 172.21.15.83:0/2686438104 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8a800c5460 con 0x7f8a880d0e60 2024-04-25T00:12:49.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a7fe00640 1 --2- 172.21.15.83:0/2686438104 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8a640678c0 0x7f8a64069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a7fe00640 1 -- 172.21.15.83:0/2686438104 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5249+0+0 (secure 0 0 0) 0x7f8a80123eb0 con 0x7f8a880d0e60 2024-04-25T00:12:49.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.845+0000 7f8a87e00640 1 --2- 172.21.15.83:0/2686438104 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8a640678c0 0x7f8a64069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.849+0000 7f8a87e00640 1 --2- 172.21.15.83:0/2686438104 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8a640678c0 0x7f8a64069d80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8a70004e30 tx=0x7f8a70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:49.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:49.849+0000 7f8a7fe00640 1 -- 172.21.15.83:0/2686438104 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8a80018d00 con 0x7f8a880d0e60 2024-04-25T00:12:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.009+0000 7f8a8f000640 1 -- 172.21.15.83:0/2686438104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8a880702f0 con 0x7f8a880d0e60 2024-04-25T00:12:50.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.009+0000 7f8a7fe00640 1 -- 172.21.15.83:0/2686438104 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v45) ==== 84+0+65 (secure 0 0 0) 0x7f8a800f4b10 con 0x7f8a880d0e60 2024-04-25T00:12:50.010 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:50.010 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-25T00:12:50.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:12:50.010 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-25T00:12:50.011 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-25T00:12:50.011 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:12:50.011 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-25T00:12:50.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.009+0000 7f8a8f000640 1 -- 172.21.15.83:0/2686438104 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8a640678c0 msgr2=0x7f8a64069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:50.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.009+0000 7f8a8f000640 1 --2- 172.21.15.83:0/2686438104 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8a640678c0 0x7f8a64069d80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8a70004e30 tx=0x7f8a70009040 comp rx=0 tx=0).stop 2024-04-25T00:12:50.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.009+0000 7f8a8f000640 1 -- 172.21.15.83:0/2686438104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a880d0e60 msgr2=0x7f8a88157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:50.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.009+0000 7f8a8f000640 1 --2- 172.21.15.83:0/2686438104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a880d0e60 0x7f8a88157220 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8a800b2040 tx=0x7f8a8001bf40 comp rx=0 tx=0).stop 2024-04-25T00:12:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.009+0000 7f8a8f000640 1 -- 172.21.15.83:0/2686438104 shutdown_connections 2024-04-25T00:12:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.009+0000 7f8a8f000640 1 --2- 172.21.15.83:0/2686438104 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8a640678c0 0x7f8a64069d80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.009+0000 7f8a8f000640 1 --2- 172.21.15.83:0/2686438104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a8806f9e0 0x7f8a880c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.009+0000 7f8a8f000640 1 --2- 172.21.15.83:0/2686438104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a880d0e60 0x7f8a88157220 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.009+0000 7f8a8f000640 1 --2- 172.21.15.83:0/2686438104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a8806c2f0 0x7f8a88151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.009+0000 7f8a8f000640 1 -- 172.21.15.83:0/2686438104 >> 172.21.15.83:0/2686438104 conn(0x7f8a880d1c80 msgr2=0x7f8a880b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.009+0000 7f8a8f000640 1 -- 172.21.15.83:0/2686438104 shutdown_connections 2024-04-25T00:12:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.009+0000 7f8a8f000640 1 -- 172.21.15.83:0/2686438104 wait complete. 2024-04-25T00:12:50.032 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:12:50.032 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:12:50.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.169+0000 7f2c1d400640 1 Processor -- start 2024-04-25T00:12:50.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.169+0000 7f2c1d400640 1 -- start start 2024-04-25T00:12:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.173+0000 7f2c1d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c1806cd00 0x7f2c180cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.173+0000 7f2c1d400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2c180cc8e0 0x7f2c1815e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.173+0000 7f2c1d400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c180ccd70 con 0x7f2c1815e430 2024-04-25T00:12:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.173+0000 7f2c1d400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2c180c8c70 con 0x7f2c1806cd00 2024-04-25T00:12:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.173+0000 7f2c1d400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2c180d1200 con 0x7f2c180cc8e0 2024-04-25T00:12:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.173+0000 7f2c0fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c1806cd00 0x7f2c180cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.173+0000 7f2c16800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2c1815e430 0x7f2c180d0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:52144/0 (socket says 172.21.15.83:52144) 2024-04-25T00:12:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.173+0000 7f2c16800640 1 -- 172.21.15.83:0/4285021085 learned_addr learned my addr 172.21.15.83:0/4285021085 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.173+0000 7f2c17200640 1 --2- 172.21.15.83:0/4285021085 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2c180cc8e0 0x7f2c1815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:50.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.173+0000 7f2c0fe00640 1 -- 172.21.15.83:0/4285021085 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2c180cc8e0 msgr2=0x7f2c1815e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:50.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.173+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/4285021085 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2c180cc8e0 0x7f2c1815e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.173+0000 7f2c0fe00640 1 -- 172.21.15.83:0/4285021085 >> v1:172.21.15.86:6789/0 conn(0x7f2c1815e430 legacy=0x7f2c180d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:50.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.173+0000 7f2c0fe00640 1 -- 172.21.15.83:0/4285021085 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1806d190 con 0x7f2c1806cd00 2024-04-25T00:12:50.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.173+0000 7f2c17200640 1 --2- 172.21.15.83:0/4285021085 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2c180cc8e0 0x7f2c1815e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:50.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.177+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/4285021085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c1806cd00 0x7f2c180cc3a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2c000097c0 tx=0x7f2c0001c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df0b1c0528a4a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:50.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.177+0000 7f2c15e00640 1 -- 172.21.15.83:0/4285021085 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c000b0070 con 0x7f2c1806cd00 2024-04-25T00:12:50.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.177+0000 7f2c15e00640 1 -- 172.21.15.83:0/4285021085 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c000ab600 con 0x7f2c1806cd00 2024-04-25T00:12:50.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.177+0000 7f2c15e00640 1 -- 172.21.15.83:0/4285021085 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c000ab9d0 con 0x7f2c1806cd00 2024-04-25T00:12:50.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.177+0000 7f2c1d400640 1 -- 172.21.15.83:0/4285021085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c1806cd00 msgr2=0x7f2c180cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:50.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.177+0000 7f2c1d400640 1 --2- 172.21.15.83:0/4285021085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c1806cd00 0x7f2c180cc3a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2c000097c0 tx=0x7f2c0001c300 comp rx=0 tx=0).stop 2024-04-25T00:12:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.177+0000 7f2c1d400640 1 -- 172.21.15.83:0/4285021085 shutdown_connections 2024-04-25T00:12:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.177+0000 7f2c1d400640 1 --2- 172.21.15.83:0/4285021085 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2c180cc8e0 0x7f2c1815e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.177+0000 7f2c1d400640 1 --2- 172.21.15.83:0/4285021085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c1806cd00 0x7f2c180cc3a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.177+0000 7f2c1d400640 1 -- 172.21.15.83:0/4285021085 >> 172.21.15.83:0/4285021085 conn(0x7f2c180d1d50 msgr2=0x7f2c180d2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.177+0000 7f2c1d400640 1 -- 172.21.15.83:0/4285021085 shutdown_connections 2024-04-25T00:12:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.177+0000 7f2c1d400640 1 -- 172.21.15.83:0/4285021085 wait complete. 2024-04-25T00:12:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.177+0000 7f2c1d400640 1 Processor -- start 2024-04-25T00:12:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.177+0000 7f2c1d400640 1 -- start start 2024-04-25T00:12:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c1d400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2c180cc8e0 0x7f2c181482d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c1d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c1815e430 0x7f2c1814c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c1d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c18158cb0 0x7f2c1814c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c1d400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c180d1b90 con 0x7f2c1815e430 2024-04-25T00:12:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c1d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c180d1a10 con 0x7f2c18158cb0 2024-04-25T00:12:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c1d400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2c1806d470 con 0x7f2c180cc8e0 2024-04-25T00:12:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c16800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2c180cc8e0 0x7f2c181482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c1815e430 0x7f2c1814c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c16800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2c180cc8e0 0x7f2c181482d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:32790/0 (socket says 172.21.15.83:32790) 2024-04-25T00:12:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c16800640 1 -- 172.21.15.83:0/1119944024 learned_addr learned my addr 172.21.15.83:0/1119944024 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c17200640 1 --2- 172.21.15.83:0/1119944024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c18158cb0 0x7f2c1814c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c16800640 1 -- 172.21.15.83:0/1119944024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c18158cb0 msgr2=0x7f2c1814c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c16800640 1 --2- 172.21.15.83:0/1119944024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c18158cb0 0x7f2c1814c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c16800640 1 -- 172.21.15.83:0/1119944024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c1815e430 msgr2=0x7f2c1814c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c16800640 1 --2- 172.21.15.83:0/1119944024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c1815e430 0x7f2c1814c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c16800640 1 -- 172.21.15.83:0/1119944024 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c18152e80 con 0x7f2c180cc8e0 2024-04-25T00:12:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/1119944024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c1815e430 0x7f2c1814c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c16800640 1 --2- 172.21.15.83:0/1119944024 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2c180cc8e0 0x7f2c181482d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800c750 tx=0x7f2c0800cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c0ea00640 1 -- 172.21.15.83:0/1119944024 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08017070 con 0x7f2c180cc8e0 2024-04-25T00:12:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c0ea00640 1 -- 172.21.15.83:0/1119944024 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c08005a60 con 0x7f2c180cc8e0 2024-04-25T00:12:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c0ea00640 1 -- 172.21.15.83:0/1119944024 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08007070 con 0x7f2c180cc8e0 2024-04-25T00:12:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c1d400640 1 -- 172.21.15.83:0/1119944024 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1815a440 con 0x7f2c180cc8e0 2024-04-25T00:12:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c1d400640 1 -- 172.21.15.83:0/1119944024 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c1815aa00 con 0x7f2c180cc8e0 2024-04-25T00:12:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c1d400640 1 -- 172.21.15.83:0/1119944024 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1809fdf0 con 0x7f2c180cc8e0 2024-04-25T00:12:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.181+0000 7f2c0ea00640 1 -- 172.21.15.83:0/1119944024 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2c0801a070 con 0x7f2c180cc8e0 2024-04-25T00:12:50.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.185+0000 7f2c0ea00640 1 --2- 172.21.15.83:0/1119944024 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2bf40678c0 0x7f2bf4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:50.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.185+0000 7f2c0ea00640 1 -- 172.21.15.83:0/1119944024 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(45..45 src has 1..45) ==== 5249+0+0 (secure 0 0 0) 0x7f2c08084f50 con 0x7f2c180cc8e0 2024-04-25T00:12:50.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.185+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/1119944024 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2bf40678c0 0x7f2bf4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:50.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.185+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/1119944024 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2bf40678c0 0x7f2bf4069d80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2c000ad000 tx=0x7f2c000a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.189+0000 7f2c0ea00640 1 -- 172.21.15.83:0/1119944024 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2c08055560 con 0x7f2c180cc8e0 2024-04-25T00:12:50.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.345+0000 7f2c1d400640 1 -- 172.21.15.83:0/1119944024 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2c180cf6f0 con 0x7f2c180cc8e0 2024-04-25T00:12:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.629+0000 7f2c0ea00640 1 -- 172.21.15.83:0/1119944024 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v46) ==== 138+0+0 (secure 0 0 0) 0x7f2c08059860 con 0x7f2c180cc8e0 2024-04-25T00:12:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-25T00:12:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.633+0000 7f2c1d400640 1 -- 172.21.15.83:0/1119944024 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2bf40678c0 msgr2=0x7f2bf4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.633+0000 7f2c1d400640 1 --2- 172.21.15.83:0/1119944024 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2bf40678c0 0x7f2bf4069d80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2c000ad000 tx=0x7f2c000a6000 comp rx=0 tx=0).stop 2024-04-25T00:12:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.633+0000 7f2c1d400640 1 -- 172.21.15.83:0/1119944024 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2c180cc8e0 msgr2=0x7f2c181482d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.633+0000 7f2c1d400640 1 --2- 172.21.15.83:0/1119944024 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2c180cc8e0 0x7f2c181482d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800c750 tx=0x7f2c0800cb50 comp rx=0 tx=0).stop 2024-04-25T00:12:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.633+0000 7f2c1d400640 1 -- 172.21.15.83:0/1119944024 shutdown_connections 2024-04-25T00:12:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.633+0000 7f2c1d400640 1 --2- 172.21.15.83:0/1119944024 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2bf40678c0 0x7f2bf4069d80 secure :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2c000ad000 tx=0x7f2c000a6000 comp rx=0 tx=0).stop 2024-04-25T00:12:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.633+0000 7f2c1d400640 1 --2- 172.21.15.83:0/1119944024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c18158cb0 0x7f2c1814c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.633+0000 7f2c1d400640 1 --2- 172.21.15.83:0/1119944024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c1815e430 0x7f2c1814c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.633+0000 7f2c1d400640 1 --2- 172.21.15.83:0/1119944024 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2c180cc8e0 0x7f2c181482d0 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800c750 tx=0x7f2c0800cb50 comp rx=0 tx=0).stop 2024-04-25T00:12:50.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.633+0000 7f2c1d400640 1 -- 172.21.15.83:0/1119944024 >> 172.21.15.83:0/1119944024 conn(0x7f2c180d1d50 msgr2=0x7f2c1806fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:50.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.633+0000 7f2c1d400640 1 -- 172.21.15.83:0/1119944024 shutdown_connections 2024-04-25T00:12:50.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.633+0000 7f2c1d400640 1 -- 172.21.15.83:0/1119944024 wait complete. 2024-04-25T00:12:50.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:12:50.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.797+0000 7f121dc00640 1 Processor -- start 2024-04-25T00:12:50.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.797+0000 7f121dc00640 1 -- start start 2024-04-25T00:12:50.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.801+0000 7f121dc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1218159810 0x7f12180cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:50.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.801+0000 7f121dc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f121806e950 0x7f121815f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:50.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.801+0000 7f121dc00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12180ce000 con 0x7f121815fe50 2024-04-25T00:12:50.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.801+0000 7f121dc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f121806e210 con 0x7f121806e950 2024-04-25T00:12:50.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.801+0000 7f121dc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f121806da00 con 0x7f1218159810 2024-04-25T00:12:50.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.801+0000 7f1217400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f121815fe50 0x7f121815c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55886/0 (socket says 172.21.15.83:55886) 2024-04-25T00:12:50.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.801+0000 7f1217400640 1 -- 172.21.15.83:0/3139256497 learned_addr learned my addr 172.21.15.83:0/3139256497 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:50.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.801+0000 7f120e000640 1 --2- 172.21.15.83:0/3139256497 >> v2:172.21.15.86:3300/0 conn(0x7f121806e950 0x7f121815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:50.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.801+0000 7f1216a00640 1 --2- 172.21.15.83:0/3139256497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1218159810 0x7f12180cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:50.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.801+0000 7f120e000640 1 -- 172.21.15.83:0/3139256497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1218159810 msgr2=0x7f12180cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:50.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.801+0000 7f120e000640 1 --2- 172.21.15.83:0/3139256497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1218159810 0x7f12180cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.801+0000 7f120e000640 1 -- 172.21.15.83:0/3139256497 >> v1:172.21.15.83:6789/0 conn(0x7f121815fe50 legacy=0x7f121815c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:50.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.801+0000 7f120e000640 1 -- 172.21.15.83:0/3139256497 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121806eec0 con 0x7f121806e950 2024-04-25T00:12:50.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.801+0000 7f1216000640 1 -- 172.21.15.83:0/3139256497 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4292984524 0 0) 0x7f12180ce000 con 0x7f121815fe50 2024-04-25T00:12:50.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f120e000640 1 --2- 172.21.15.83:0/3139256497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121806e950 0x7f121815f910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f12000097c0 tx=0x7f120001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a170e03b6d61b4d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:50.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f1216000640 1 -- 172.21.15.83:0/3139256497 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12000b0070 con 0x7f121806e950 2024-04-25T00:12:50.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f1216000640 1 -- 172.21.15.83:0/3139256497 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12000ab600 con 0x7f121806e950 2024-04-25T00:12:50.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f1216000640 1 -- 172.21.15.83:0/3139256497 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12000ab9d0 con 0x7f121806e950 2024-04-25T00:12:50.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 -- 172.21.15.83:0/3139256497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121806e950 msgr2=0x7f121815f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:50.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 --2- 172.21.15.83:0/3139256497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121806e950 0x7f121815f910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f12000097c0 tx=0x7f120001c410 comp rx=0 tx=0).stop 2024-04-25T00:12:50.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 -- 172.21.15.83:0/3139256497 shutdown_connections 2024-04-25T00:12:50.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 --2- 172.21.15.83:0/3139256497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121806e950 0x7f121815f910 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 --2- 172.21.15.83:0/3139256497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1218159810 0x7f12180cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 -- 172.21.15.83:0/3139256497 >> 172.21.15.83:0/3139256497 conn(0x7f12180b1830 msgr2=0x7f12180b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:50.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 -- 172.21.15.83:0/3139256497 shutdown_connections 2024-04-25T00:12:50.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 -- 172.21.15.83:0/3139256497 wait complete. 2024-04-25T00:12:50.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 Processor -- start 2024-04-25T00:12:50.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 -- start start 2024-04-25T00:12:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121806e950 0x7f121814c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1218159810 0x7f12180b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f121815fe50 0x7f12180b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1218159d70 con 0x7f1218159810 2024-04-25T00:12:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1218159bf0 con 0x7f121806e950 2024-04-25T00:12:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.805+0000 7f121dc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1218070da0 con 0x7f121815fe50 2024-04-25T00:12:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f120e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1218159810 0x7f12180b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f1216a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121806e950 0x7f121814c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f1216a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121806e950 0x7f121814c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:40760/0 (socket says 172.21.15.83:40760) 2024-04-25T00:12:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f1216a00640 1 -- 172.21.15.83:0/1672570232 learned_addr learned my addr 172.21.15.83:0/1672570232 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f1217400640 1 --2- 172.21.15.83:0/1672570232 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f121815fe50 0x7f12180b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f1216a00640 1 -- 172.21.15.83:0/1672570232 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f121815fe50 msgr2=0x7f12180b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:50.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f1216a00640 1 --2- 172.21.15.83:0/1672570232 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f121815fe50 0x7f12180b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f1216a00640 1 -- 172.21.15.83:0/1672570232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1218159810 msgr2=0x7f12180b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:50.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f1216a00640 1 --2- 172.21.15.83:0/1672570232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1218159810 0x7f12180b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:50.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f1216a00640 1 -- 172.21.15.83:0/1672570232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121814ccb0 con 0x7f121806e950 2024-04-25T00:12:50.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f120e000640 1 --2- 172.21.15.83:0/1672570232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1218159810 0x7f12180b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:50.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f1216a00640 1 --2- 172.21.15.83:0/1672570232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121806e950 0x7f121814c2c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f120800e930 tx=0x7f120800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:50.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f120f400640 1 -- 172.21.15.83:0/1672570232 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208017070 con 0x7f121806e950 2024-04-25T00:12:50.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f120f400640 1 -- 172.21.15.83:0/1672570232 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f120800ca00 con 0x7f121806e950 2024-04-25T00:12:50.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f121dc00640 1 -- 172.21.15.83:0/1672570232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12180af920 con 0x7f121806e950 2024-04-25T00:12:50.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f120f400640 1 -- 172.21.15.83:0/1672570232 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120800cdb0 con 0x7f121806e950 2024-04-25T00:12:50.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f121dc00640 1 -- 172.21.15.83:0/1672570232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12180afea0 con 0x7f121806e950 2024-04-25T00:12:50.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f121dc00640 1 -- 172.21.15.83:0/1672570232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f121804f510 con 0x7f121806e950 2024-04-25T00:12:50.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f120f400640 1 -- 172.21.15.83:0/1672570232 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f120801a070 con 0x7f121806e950 2024-04-25T00:12:50.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f120f400640 1 --2- 172.21.15.83:0/1672570232 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f11f40678c0 0x7f11f4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:50.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.809+0000 7f120f400640 1 -- 172.21.15.83:0/1672570232 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5619+0+0 (secure 0 0 0) 0x7f1208085c10 con 0x7f121806e950 2024-04-25T00:12:50.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.813+0000 7f120e000640 1 --2- 172.21.15.83:0/1672570232 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f11f40678c0 0x7f11f4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:50.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.813+0000 7f120e000640 1 --2- 172.21.15.83:0/1672570232 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f11f40678c0 0x7f11f4069d80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f12000ad000 tx=0x7f1200008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:50.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.813+0000 7f120f400640 1 -- 172.21.15.83:0/1672570232 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f121814ccb0 con 0x7f121806e950 2024-04-25T00:12:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:50.973+0000 7f121dc00640 1 -- 172.21.15.83:0/1672570232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f12180d04e0 con 0x7f121806e950 2024-04-25T00:12:51.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.633+0000 7f120f400640 1 -- 172.21.15.83:0/1672570232 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v47) ==== 170+0+0 (secure 0 0 0) 0x7f1208056200 con 0x7f121806e950 2024-04-25T00:12:51.636 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-25T00:12:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.637+0000 7f121dc00640 1 -- 172.21.15.83:0/1672570232 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f11f40678c0 msgr2=0x7f11f4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.637+0000 7f121dc00640 1 --2- 172.21.15.83:0/1672570232 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f11f40678c0 0x7f11f4069d80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f12000ad000 tx=0x7f1200008040 comp rx=0 tx=0).stop 2024-04-25T00:12:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.637+0000 7f121dc00640 1 -- 172.21.15.83:0/1672570232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121806e950 msgr2=0x7f121814c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.637+0000 7f121dc00640 1 --2- 172.21.15.83:0/1672570232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121806e950 0x7f121814c2c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f120800e930 tx=0x7f120800edf0 comp rx=0 tx=0).stop 2024-04-25T00:12:51.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.645+0000 7f121dc00640 1 -- 172.21.15.83:0/1672570232 shutdown_connections 2024-04-25T00:12:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.645+0000 7f121dc00640 1 --2- 172.21.15.83:0/1672570232 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f11f40678c0 0x7f11f4069d80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.645+0000 7f121dc00640 1 --2- 172.21.15.83:0/1672570232 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f121815fe50 0x7f12180b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.645+0000 7f121dc00640 1 --2- 172.21.15.83:0/1672570232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1218159810 0x7f12180b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.645+0000 7f121dc00640 1 --2- 172.21.15.83:0/1672570232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121806e950 0x7f121814c2c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.645+0000 7f121dc00640 1 -- 172.21.15.83:0/1672570232 >> 172.21.15.83:0/1672570232 conn(0x7f12180b1830 msgr2=0x7f1218164e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:51.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.645+0000 7f121dc00640 1 -- 172.21.15.83:0/1672570232 shutdown_connections 2024-04-25T00:12:51.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.645+0000 7f121dc00640 1 -- 172.21.15.83:0/1672570232 wait complete. 2024-04-25T00:12:51.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:12:51.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae12400640 1 Processor -- start 2024-04-25T00:12:51.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae12400640 1 -- start start 2024-04-25T00:12:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae12400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fae0c06e950 0x7fae0c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae12400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae0c15fe50 0x7fae0c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae12400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae0c0ce000 con 0x7fae0c159810 2024-04-25T00:12:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae12400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fae0c06e210 con 0x7fae0c15fe50 2024-04-25T00:12:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae12400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fae0c1602a0 con 0x7fae0c06e950 2024-04-25T00:12:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae0be00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae0c15fe50 0x7fae0c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae0be00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae0c15fe50 0x7fae0c0ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54970/0 (socket says 172.21.15.83:54970) 2024-04-25T00:12:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae0be00640 1 -- 172.21.15.83:0/3277861612 learned_addr learned my addr 172.21.15.83:0/3277861612 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae0aa00640 1 --2- 172.21.15.83:0/3277861612 >> v2:172.21.15.86:3300/0 conn(0x7fae0c06e950 0x7fae0c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:51.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae0be00640 1 -- 172.21.15.83:0/3277861612 >> v2:172.21.15.86:3300/0 conn(0x7fae0c06e950 msgr2=0x7fae0c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:51.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae0be00640 1 --2- 172.21.15.83:0/3277861612 >> v2:172.21.15.86:3300/0 conn(0x7fae0c06e950 0x7fae0c15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:51.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae0be00640 1 -- 172.21.15.83:0/3277861612 >> v1:172.21.15.86:6789/0 conn(0x7fae0c159810 legacy=0x7fae0c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:51.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.817+0000 7fae0be00640 1 -- 172.21.15.83:0/3277861612 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c06eec0 con 0x7fae0c15fe50 2024-04-25T00:12:51.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.821+0000 7fae0be00640 1 --2- 172.21.15.83:0/3277861612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c15fe50 0x7fae0c0ce4a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0709a0 tx=0x7fae0401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d9517db386725d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:51.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.821+0000 7fae0a000640 1 -- 172.21.15.83:0/3277861612 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae040b2070 con 0x7fae0c15fe50 2024-04-25T00:12:51.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.821+0000 7fae0a000640 1 -- 172.21.15.83:0/3277861612 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae040ad680 con 0x7fae0c15fe50 2024-04-25T00:12:51.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.821+0000 7fae0a000640 1 -- 172.21.15.83:0/3277861612 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae040ada50 con 0x7fae0c15fe50 2024-04-25T00:12:51.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.821+0000 7fae12400640 1 -- 172.21.15.83:0/3277861612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c15fe50 msgr2=0x7fae0c0ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:51.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.821+0000 7fae12400640 1 --2- 172.21.15.83:0/3277861612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c15fe50 0x7fae0c0ce4a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0709a0 tx=0x7fae0401c490 comp rx=0 tx=0).stop 2024-04-25T00:12:51.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.821+0000 7fae12400640 1 -- 172.21.15.83:0/3277861612 shutdown_connections 2024-04-25T00:12:51.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.821+0000 7fae12400640 1 --2- 172.21.15.83:0/3277861612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c15fe50 0x7fae0c0ce4a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:51.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.821+0000 7fae12400640 1 --2- 172.21.15.83:0/3277861612 >> v2:172.21.15.86:3300/0 conn(0x7fae0c06e950 0x7fae0c15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:51.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.821+0000 7fae12400640 1 -- 172.21.15.83:0/3277861612 >> 172.21.15.83:0/3277861612 conn(0x7fae0c0b1830 msgr2=0x7fae0c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:51.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.821+0000 7fae12400640 1 -- 172.21.15.83:0/3277861612 shutdown_connections 2024-04-25T00:12:51.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.821+0000 7fae12400640 1 -- 172.21.15.83:0/3277861612 wait complete. 2024-04-25T00:12:51.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae12400640 1 Processor -- start 2024-04-25T00:12:51.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae12400640 1 -- start start 2024-04-25T00:12:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae12400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae0c06e950 0x7fae0c157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae12400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae0c159810 0x7fae0c1420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae12400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c15fe50 0x7fae0c14b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae12400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c070f20 con 0x7fae0c15fe50 2024-04-25T00:12:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae12400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c070da0 con 0x7fae0c06e950 2024-04-25T00:12:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae12400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fae0c168cb0 con 0x7fae0c159810 2024-04-25T00:12:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae0be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c15fe50 0x7fae0c14b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae0b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae0c06e950 0x7fae0c157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae0be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c15fe50 0x7fae0c14b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54982/0 (socket says 172.21.15.83:54982) 2024-04-25T00:12:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae0be00640 1 -- 172.21.15.83:0/2702541886 learned_addr learned my addr 172.21.15.83:0/2702541886 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae0aa00640 1 --2- 172.21.15.83:0/2702541886 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae0c159810 0x7fae0c1420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae0be00640 1 -- 172.21.15.83:0/2702541886 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae0c159810 msgr2=0x7fae0c1420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae0be00640 1 --2- 172.21.15.83:0/2702541886 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae0c159810 0x7fae0c1420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae0be00640 1 -- 172.21.15.83:0/2702541886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae0c06e950 msgr2=0x7fae0c157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae0be00640 1 --2- 172.21.15.83:0/2702541886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae0c06e950 0x7fae0c157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae0be00640 1 -- 172.21.15.83:0/2702541886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c14bc40 con 0x7fae0c15fe50 2024-04-25T00:12:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae0be00640 1 --2- 172.21.15.83:0/2702541886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c15fe50 0x7fae0c14b2f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fae0401c4d0 tx=0x7fae0401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae03400640 1 -- 172.21.15.83:0/2702541886 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae040b2040 con 0x7fae0c15fe50 2024-04-25T00:12:51.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae03400640 1 -- 172.21.15.83:0/2702541886 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae040bd070 con 0x7fae0c15fe50 2024-04-25T00:12:51.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae12400640 1 -- 172.21.15.83:0/2702541886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0c14bf20 con 0x7fae0c15fe50 2024-04-25T00:12:51.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae03400640 1 -- 172.21.15.83:0/2702541886 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae040add90 con 0x7fae0c15fe50 2024-04-25T00:12:51.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae12400640 1 -- 172.21.15.83:0/2702541886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0c0afa60 con 0x7fae0c15fe50 2024-04-25T00:12:51.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.825+0000 7fae12400640 1 -- 172.21.15.83:0/2702541886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0c04f510 con 0x7fae0c15fe50 2024-04-25T00:12:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.829+0000 7fae03400640 1 -- 172.21.15.83:0/2702541886 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fae040c2460 con 0x7fae0c15fe50 2024-04-25T00:12:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.829+0000 7fae03400640 1 --2- 172.21.15.83:0/2702541886 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fadec0678c0 0x7fadec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:51.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.829+0000 7fae03400640 1 -- 172.21.15.83:0/2702541886 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6001+0+0 (secure 0 0 0) 0x7fae04120970 con 0x7fae0c15fe50 2024-04-25T00:12:51.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.829+0000 7fae0b400640 1 --2- 172.21.15.83:0/2702541886 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fadec0678c0 0x7fadec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.829+0000 7fae0b400640 1 --2- 172.21.15.83:0/2702541886 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fadec0678c0 0x7fadec069d80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0012f0 tx=0x7fadfc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:51.833+0000 7fae03400640 1 -- 172.21.15.83:0/2702541886 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fae04016110 con 0x7fae0c15fe50 2024-04-25T00:12:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.021+0000 7fae12400640 1 -- 172.21.15.83:0/2702541886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fae0c072c90 con 0x7fae0c15fe50 2024-04-25T00:12:52.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.649+0000 7fae03400640 1 -- 172.21.15.83:0/2702541886 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v23) ==== 172+0+0 (secure 0 0 0) 0x7fae040f0c80 con 0x7fae0c15fe50 2024-04-25T00:12:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T00:12:52.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.657+0000 7fae12400640 1 -- 172.21.15.83:0/2702541886 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fadec0678c0 msgr2=0x7fadec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:52.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.657+0000 7fae12400640 1 --2- 172.21.15.83:0/2702541886 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fadec0678c0 0x7fadec069d80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0012f0 tx=0x7fadfc009040 comp rx=0 tx=0).stop 2024-04-25T00:12:52.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.657+0000 7fae12400640 1 -- 172.21.15.83:0/2702541886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c15fe50 msgr2=0x7fae0c14b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.657+0000 7fae12400640 1 --2- 172.21.15.83:0/2702541886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c15fe50 0x7fae0c14b2f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fae0401c4d0 tx=0x7fae0401acd0 comp rx=0 tx=0).stop 2024-04-25T00:12:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.657+0000 7fae12400640 1 -- 172.21.15.83:0/2702541886 shutdown_connections 2024-04-25T00:12:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.657+0000 7fae12400640 1 --2- 172.21.15.83:0/2702541886 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fadec0678c0 0x7fadec069d80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.657+0000 7fae12400640 1 --2- 172.21.15.83:0/2702541886 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae0c159810 0x7fae0c1420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.657+0000 7fae12400640 1 --2- 172.21.15.83:0/2702541886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae0c15fe50 0x7fae0c14b2f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.657+0000 7fae12400640 1 --2- 172.21.15.83:0/2702541886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae0c06e950 0x7fae0c157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.657+0000 7fae12400640 1 -- 172.21.15.83:0/2702541886 >> 172.21.15.83:0/2702541886 conn(0x7fae0c0b1830 msgr2=0x7fae0c1501a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:52.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.661+0000 7fae12400640 1 -- 172.21.15.83:0/2702541886 shutdown_connections 2024-04-25T00:12:52.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.665+0000 7fae12400640 1 -- 172.21.15.83:0/2702541886 wait complete. 2024-04-25T00:12:52.703 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:52.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.833+0000 7fc8fa600640 1 Processor -- start 2024-04-25T00:12:52.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.833+0000 7fc8fa600640 1 -- start start 2024-04-25T00:12:52.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.837+0000 7fc8fa600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8f40b4360 0x7fc8f40b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:52.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.837+0000 7fc8fa600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8f40b2b30 0x7fc8f40b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:52.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.837+0000 7fc8fa600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8f406ef80 con 0x7fc8f41553e0 2024-04-25T00:12:52.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.837+0000 7fc8fa600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc8f415b510 con 0x7fc8f40b2b30 2024-04-25T00:12:52.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.837+0000 7fc8fa600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f415b7f0 con 0x7fc8f40b4360 2024-04-25T00:12:52.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.837+0000 7fc8f3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8f40b2b30 0x7fc8f40b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:52.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.837+0000 7fc8eaa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8f40b4360 0x7fc8f40b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:52.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.837+0000 7fc8f3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc8f41553e0 0x7fc8f406e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:52170/0 (socket says 172.21.15.83:52170) 2024-04-25T00:12:52.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.837+0000 7fc8f3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8f40b2b30 0x7fc8f40b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:40792/0 (socket says 172.21.15.83:40792) 2024-04-25T00:12:52.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.837+0000 7fc8f3400640 1 -- 172.21.15.83:0/255729888 learned_addr learned my addr 172.21.15.83:0/255729888 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:52.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.837+0000 7fc8eaa00640 1 -- 172.21.15.83:0/255729888 >> v2:172.21.15.86:3300/0 conn(0x7fc8f40b2b30 msgr2=0x7fc8f40b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:52.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.837+0000 7fc8eaa00640 1 --2- 172.21.15.83:0/255729888 >> v2:172.21.15.86:3300/0 conn(0x7fc8f40b2b30 0x7fc8f40b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:52.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.837+0000 7fc8eaa00640 1 -- 172.21.15.83:0/255729888 >> v1:172.21.15.86:6789/0 conn(0x7fc8f41553e0 legacy=0x7fc8f406e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:52.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.837+0000 7fc8eaa00640 1 -- 172.21.15.83:0/255729888 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f415bad0 con 0x7fc8f40b4360 2024-04-25T00:12:52.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.841+0000 7fc8eaa00640 1 --2- 172.21.15.83:0/255729888 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8f40b4360 0x7fc8f40b47e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0097c0 tx=0x7fc8dc01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=778b31e448c61c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:52.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.841+0000 7fc8f2a00640 1 -- 172.21.15.83:0/255729888 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0b0070 con 0x7fc8f40b4360 2024-04-25T00:12:52.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.841+0000 7fc8f2a00640 1 -- 172.21.15.83:0/255729888 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8dc0ab600 con 0x7fc8f40b4360 2024-04-25T00:12:52.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.841+0000 7fc8f2a00640 1 -- 172.21.15.83:0/255729888 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0a4e60 con 0x7fc8f40b4360 2024-04-25T00:12:52.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.841+0000 7fc8fa600640 1 -- 172.21.15.83:0/255729888 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8f40b4360 msgr2=0x7fc8f40b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:52.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.841+0000 7fc8fa600640 1 --2- 172.21.15.83:0/255729888 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8f40b4360 0x7fc8f40b47e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0097c0 tx=0x7fc8dc01c410 comp rx=0 tx=0).stop 2024-04-25T00:12:52.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.841+0000 7fc8fa600640 1 -- 172.21.15.83:0/255729888 shutdown_connections 2024-04-25T00:12:52.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.841+0000 7fc8fa600640 1 --2- 172.21.15.83:0/255729888 >> v2:172.21.15.86:3300/0 conn(0x7fc8f40b2b30 0x7fc8f40b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:52.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.841+0000 7fc8fa600640 1 --2- 172.21.15.83:0/255729888 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8f40b4360 0x7fc8f40b47e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:52.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.841+0000 7fc8fa600640 1 -- 172.21.15.83:0/255729888 >> 172.21.15.83:0/255729888 conn(0x7fc8f40b1830 msgr2=0x7fc8f40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:52.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.841+0000 7fc8fa600640 1 -- 172.21.15.83:0/255729888 shutdown_connections 2024-04-25T00:12:52.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.841+0000 7fc8fa600640 1 -- 172.21.15.83:0/255729888 wait complete. 2024-04-25T00:12:52.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.841+0000 7fc8fa600640 1 Processor -- start 2024-04-25T00:12:52.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.841+0000 7fc8fa600640 1 -- start start 2024-04-25T00:12:52.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8fa600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40b2b30 0x7fc8f41542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:52.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8fa600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8f40b4360 0x7fc8f40c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:52.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8fa600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8f41553e0 0x7fc8f415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8fa600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f406ebd0 con 0x7fc8f40b2b30 2024-04-25T00:12:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8fa600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4167d90 con 0x7fc8f40b4360 2024-04-25T00:12:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8fa600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f40cb4d0 con 0x7fc8f41553e0 2024-04-25T00:12:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8f3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40b2b30 0x7fc8f41542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8f3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40b2b30 0x7fc8f41542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54998/0 (socket says 172.21.15.83:54998) 2024-04-25T00:12:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8f3400640 1 -- 172.21.15.83:0/59075902 learned_addr learned my addr 172.21.15.83:0/59075902 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8eaa00640 1 --2- 172.21.15.83:0/59075902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8f40b4360 0x7fc8f40c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8f3e00640 1 --2- 172.21.15.83:0/59075902 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8f41553e0 0x7fc8f415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:52.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8f3400640 1 -- 172.21.15.83:0/59075902 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8f41553e0 msgr2=0x7fc8f415d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:52.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8f3400640 1 --2- 172.21.15.83:0/59075902 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8f41553e0 0x7fc8f415d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:52.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8f3400640 1 -- 172.21.15.83:0/59075902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8f40b4360 msgr2=0x7fc8f40c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:52.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8f3400640 1 --2- 172.21.15.83:0/59075902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8f40b4360 0x7fc8f40c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:52.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8f3400640 1 -- 172.21.15.83:0/59075902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f40cdd20 con 0x7fc8f40b2b30 2024-04-25T00:12:52.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8f3400640 1 --2- 172.21.15.83:0/59075902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40b2b30 0x7fc8f41542d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c450 tx=0x7fc8e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:52.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8ebe00640 1 -- 172.21.15.83:0/59075902 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4017070 con 0x7fc8f40b2b30 2024-04-25T00:12:52.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8ebe00640 1 -- 172.21.15.83:0/59075902 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e4005cf0 con 0x7fc8f40b2b30 2024-04-25T00:12:52.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8fa600640 1 -- 172.21.15.83:0/59075902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f40ce000 con 0x7fc8f40b2b30 2024-04-25T00:12:52.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8ebe00640 1 -- 172.21.15.83:0/59075902 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4007290 con 0x7fc8f40b2b30 2024-04-25T00:12:52.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8fa600640 1 -- 172.21.15.83:0/59075902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f40cd440 con 0x7fc8f40b2b30 2024-04-25T00:12:52.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8fa600640 1 -- 172.21.15.83:0/59075902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f40737f0 con 0x7fc8f40b2b30 2024-04-25T00:12:52.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.845+0000 7fc8ebe00640 1 -- 172.21.15.83:0/59075902 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc8e401a070 con 0x7fc8f40b2b30 2024-04-25T00:12:52.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.849+0000 7fc8ebe00640 1 --2- 172.21.15.83:0/59075902 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8d00678c0 0x7fc8d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:52.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.849+0000 7fc8ebe00640 1 -- 172.21.15.83:0/59075902 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7fc8e4085100 con 0x7fc8f40b2b30 2024-04-25T00:12:52.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.849+0000 7fc8eaa00640 1 --2- 172.21.15.83:0/59075902 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8d00678c0 0x7fc8d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:52.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.849+0000 7fc8eaa00640 1 --2- 172.21.15.83:0/59075902 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8d00678c0 0x7fc8d0069d80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0ad000 tx=0x7fc8dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:52.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:52.853+0000 7fc8ebe00640 1 -- 172.21.15.83:0/59075902 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc8e40553c0 con 0x7fc8f40b2b30 2024-04-25T00:12:53.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.009+0000 7fc8fa600640 1 -- 172.21.15.83:0/59075902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8f404f5f0 con 0x7fc8f40b2b30 2024-04-25T00:12:53.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.009+0000 7fc8ebe00640 1 -- 172.21.15.83:0/59075902 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16871 (secure 0 0 0) 0x7fc8e40596c0 con 0x7fc8f40b2b30 2024-04-25T00:12:53.013 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:53.013 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":48,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:52.025814+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:50.394645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:51.018842+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:53.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.013+0000 7fc8fa600640 1 -- 172.21.15.83:0/59075902 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8d00678c0 msgr2=0x7fc8d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:53.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.013+0000 7fc8fa600640 1 --2- 172.21.15.83:0/59075902 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8d00678c0 0x7fc8d0069d80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0ad000 tx=0x7fc8dc008040 comp rx=0 tx=0).stop 2024-04-25T00:12:53.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.013+0000 7fc8fa600640 1 -- 172.21.15.83:0/59075902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40b2b30 msgr2=0x7fc8f41542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:53.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.013+0000 7fc8fa600640 1 --2- 172.21.15.83:0/59075902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40b2b30 0x7fc8f41542d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c450 tx=0x7fc8e400c910 comp rx=0 tx=0).stop 2024-04-25T00:12:53.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.013+0000 7fc8fa600640 1 -- 172.21.15.83:0/59075902 shutdown_connections 2024-04-25T00:12:53.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.013+0000 7fc8fa600640 1 --2- 172.21.15.83:0/59075902 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8d00678c0 0x7fc8d0069d80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.013+0000 7fc8fa600640 1 --2- 172.21.15.83:0/59075902 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8f41553e0 0x7fc8f415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.013+0000 7fc8fa600640 1 --2- 172.21.15.83:0/59075902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8f40b4360 0x7fc8f40c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.013+0000 7fc8fa600640 1 --2- 172.21.15.83:0/59075902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40b2b30 0x7fc8f41542d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.013+0000 7fc8fa600640 1 -- 172.21.15.83:0/59075902 >> 172.21.15.83:0/59075902 conn(0x7fc8f40b1830 msgr2=0x7fc8f40b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:53.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.013+0000 7fc8fa600640 1 -- 172.21.15.83:0/59075902 shutdown_connections 2024-04-25T00:12:53.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.013+0000 7fc8fa600640 1 -- 172.21.15.83:0/59075902 wait complete. 2024-04-25T00:12:53.034 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:53.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.165+0000 7fcb9e400640 1 Processor -- start 2024-04-25T00:12:53.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.165+0000 7fcb9e400640 1 -- start start 2024-04-25T00:12:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.169+0000 7fcb9e400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb980b4360 0x7fcb980b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.169+0000 7fcb9e400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb980b2b30 0x7fcb980b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.169+0000 7fcb9e400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb9806ef80 con 0x7fcb981553e0 2024-04-25T00:12:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.169+0000 7fcb9e400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fcb9815b510 con 0x7fcb980b4360 2024-04-25T00:12:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.169+0000 7fcb9e400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9815b7f0 con 0x7fcb980b2b30 2024-04-25T00:12:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.169+0000 7fcb8fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb980b4360 0x7fcb980b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.169+0000 7fcb8fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb980b4360 0x7fcb980b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55008/0 (socket says 172.21.15.83:55008) 2024-04-25T00:12:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.169+0000 7fcb8fe00640 1 -- 172.21.15.83:0/3329325144 learned_addr learned my addr 172.21.15.83:0/3329325144 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.169+0000 7fcb97e00640 1 --2- 172.21.15.83:0/3329325144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb980b2b30 0x7fcb980b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.169+0000 7fcb8fe00640 1 -- 172.21.15.83:0/3329325144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb980b2b30 msgr2=0x7fcb980b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.169+0000 7fcb8fe00640 1 --2- 172.21.15.83:0/3329325144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb980b2b30 0x7fcb980b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.169+0000 7fcb8fe00640 1 -- 172.21.15.83:0/3329325144 >> v1:172.21.15.86:6789/0 conn(0x7fcb981553e0 legacy=0x7fcb9806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.169+0000 7fcb8fe00640 1 -- 172.21.15.83:0/3329325144 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9815bad0 con 0x7fcb980b4360 2024-04-25T00:12:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.169+0000 7fcb97e00640 1 --2- 172.21.15.83:0/3329325144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb980b2b30 0x7fcb980b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:53.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.173+0000 7fcb8fe00640 1 --2- 172.21.15.83:0/3329325144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb980b4360 0x7fcb980b47e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009870 tx=0x7fcb8001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b1514309d8188e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:53.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.173+0000 7fcb96a00640 1 -- 172.21.15.83:0/3329325144 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb800b0070 con 0x7fcb980b4360 2024-04-25T00:12:53.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.173+0000 7fcb96a00640 1 -- 172.21.15.83:0/3329325144 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb800ab600 con 0x7fcb980b4360 2024-04-25T00:12:53.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.173+0000 7fcb96a00640 1 -- 172.21.15.83:0/3329325144 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb800ab9d0 con 0x7fcb980b4360 2024-04-25T00:12:53.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.173+0000 7fcb9e400640 1 -- 172.21.15.83:0/3329325144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb980b4360 msgr2=0x7fcb980b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:53.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.173+0000 7fcb9e400640 1 --2- 172.21.15.83:0/3329325144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb980b4360 0x7fcb980b47e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009870 tx=0x7fcb8001c4d0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.173+0000 7fcb9e400640 1 -- 172.21.15.83:0/3329325144 shutdown_connections 2024-04-25T00:12:53.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.173+0000 7fcb9e400640 1 --2- 172.21.15.83:0/3329325144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb980b2b30 0x7fcb980b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.173+0000 7fcb9e400640 1 --2- 172.21.15.83:0/3329325144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb980b4360 0x7fcb980b47e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.173+0000 7fcb9e400640 1 -- 172.21.15.83:0/3329325144 >> 172.21.15.83:0/3329325144 conn(0x7fcb980b1830 msgr2=0x7fcb980b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:53.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.173+0000 7fcb9e400640 1 -- 172.21.15.83:0/3329325144 shutdown_connections 2024-04-25T00:12:53.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.173+0000 7fcb9e400640 1 -- 172.21.15.83:0/3329325144 wait complete. 2024-04-25T00:12:53.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.173+0000 7fcb9e400640 1 Processor -- start 2024-04-25T00:12:53.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.173+0000 7fcb9e400640 1 -- start start 2024-04-25T00:12:53.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb9e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb980b2b30 0x7fcb9814c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb9e400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb980b4360 0x7fcb981f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb9e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb981553e0 0x7fcb98162c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb9e400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9806ebd0 con 0x7fcb980b2b30 2024-04-25T00:12:53.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb9e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb98167d90 con 0x7fcb981553e0 2024-04-25T00:12:53.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb9e400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9814cb50 con 0x7fcb980b4360 2024-04-25T00:12:53.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb97400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb980b2b30 0x7fcb9814c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb97400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb980b2b30 0x7fcb9814c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55016/0 (socket says 172.21.15.83:55016) 2024-04-25T00:12:53.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb97400640 1 -- 172.21.15.83:0/3844136533 learned_addr learned my addr 172.21.15.83:0/3844136533 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:53.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb8fe00640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb980b4360 0x7fcb981f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb97e00640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb981553e0 0x7fcb98162c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb97400640 1 -- 172.21.15.83:0/3844136533 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb980b4360 msgr2=0x7fcb981f6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:53.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb97400640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb980b4360 0x7fcb981f6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb97400640 1 -- 172.21.15.83:0/3844136533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb981553e0 msgr2=0x7fcb98162c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:53.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb97400640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb981553e0 0x7fcb98162c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb97400640 1 -- 172.21.15.83:0/3844136533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb98165830 con 0x7fcb980b2b30 2024-04-25T00:12:53.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb8fe00640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb980b4360 0x7fcb981f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:53.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb97400640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb980b2b30 0x7fcb9814c610 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800c500 tx=0x7fcb8800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:53.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb8f400640 1 -- 172.21.15.83:0/3844136533 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88016020 con 0x7fcb980b2b30 2024-04-25T00:12:53.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb8f400640 1 -- 172.21.15.83:0/3844136533 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb88005220 con 0x7fcb980b2b30 2024-04-25T00:12:53.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb8f400640 1 -- 172.21.15.83:0/3844136533 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88007030 con 0x7fcb980b2b30 2024-04-25T00:12:53.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb9e400640 1 -- 172.21.15.83:0/3844136533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb981590c0 con 0x7fcb980b2b30 2024-04-25T00:12:53.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb9e400640 1 -- 172.21.15.83:0/3844136533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb98165b10 con 0x7fcb980b2b30 2024-04-25T00:12:53.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb9e400640 1 -- 172.21.15.83:0/3844136533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb981594b0 con 0x7fcb980b2b30 2024-04-25T00:12:53.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.177+0000 7fcb8f400640 1 -- 172.21.15.83:0/3844136533 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcb880071f0 con 0x7fcb980b2b30 2024-04-25T00:12:53.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.181+0000 7fcb8f400640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb740678c0 0x7fcb74069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.181+0000 7fcb8f400640 1 -- 172.21.15.83:0/3844136533 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7fcb880858e0 con 0x7fcb980b2b30 2024-04-25T00:12:53.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.181+0000 7fcb8fe00640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb740678c0 0x7fcb74069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.181+0000 7fcb8fe00640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb740678c0 0x7fcb74069d80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcb800ad000 tx=0x7fcb80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:53.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.185+0000 7fcb8f400640 1 -- 172.21.15.83:0/3844136533 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcb88020020 con 0x7fcb980b2b30 2024-04-25T00:12:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.341+0000 7fcb9e400640 1 -- 172.21.15.83:0/3844136533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb980737f0 con 0x7fcb980b2b30 2024-04-25T00:12:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.341+0000 7fcb8f400640 1 -- 172.21.15.83:0/3844136533 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16871 (secure 0 0 0) 0x7fcb88056200 con 0x7fcb980b2b30 2024-04-25T00:12:53.344 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:53.345 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":48,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:52.025814+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:50.394645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:51.018842+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.345+0000 7fcb9e400640 1 -- 172.21.15.83:0/3844136533 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb740678c0 msgr2=0x7fcb74069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.345+0000 7fcb9e400640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb740678c0 0x7fcb74069d80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcb800ad000 tx=0x7fcb80008040 comp rx=0 tx=0).stop 2024-04-25T00:12:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.345+0000 7fcb9e400640 1 -- 172.21.15.83:0/3844136533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb980b2b30 msgr2=0x7fcb9814c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.345+0000 7fcb9e400640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb980b2b30 0x7fcb9814c610 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800c500 tx=0x7fcb8800c9c0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.345+0000 7fcb9e400640 1 -- 172.21.15.83:0/3844136533 shutdown_connections 2024-04-25T00:12:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.345+0000 7fcb9e400640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb740678c0 0x7fcb74069d80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.345+0000 7fcb9e400640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb980b4360 0x7fcb981f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.345+0000 7fcb9e400640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb981553e0 0x7fcb98162c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.345+0000 7fcb9e400640 1 --2- 172.21.15.83:0/3844136533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb980b2b30 0x7fcb9814c610 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.345+0000 7fcb9e400640 1 -- 172.21.15.83:0/3844136533 >> 172.21.15.83:0/3844136533 conn(0x7fcb980b1830 msgr2=0x7fcb980b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:53.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.345+0000 7fcb9e400640 1 -- 172.21.15.83:0/3844136533 shutdown_connections 2024-04-25T00:12:53.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.345+0000 7fcb9e400640 1 -- 172.21.15.83:0/3844136533 wait complete. 2024-04-25T00:12:53.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:12:53.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.501+0000 7f9741000640 1 Processor -- start 2024-04-25T00:12:53.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.501+0000 7f9741000640 1 -- start start 2024-04-25T00:12:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9741000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c1555c0 0x7f973c15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9741000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f973c06e4c0 0x7f973c06e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9741000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f973c15c560 con 0x7f973c1551e0 2024-04-25T00:12:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9741000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f973c06ee70 con 0x7f973c06e4c0 2024-04-25T00:12:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9741000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f973c072460 con 0x7f973c1555c0 2024-04-25T00:12:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f973c1551e0 0x7f973c071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55898/0 (socket says 172.21.15.83:55898) 2024-04-25T00:12:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739e00640 1 -- 172.21.15.83:0/3053726316 learned_addr learned my addr 172.21.15.83:0/3053726316 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:53.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f973a800640 1 --2- 172.21.15.83:0/3053726316 >> v2:172.21.15.86:3300/0 conn(0x7f973c06e4c0 0x7f973c06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9731400640 1 --2- 172.21.15.83:0/3053726316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c1555c0 0x7f973c15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 -- 172.21.15.83:0/3053726316 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2397899978 0 0) 0x7f973c15c560 con 0x7f973c1551e0 2024-04-25T00:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 -- 172.21.15.83:0/3053726316 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9720003530 con 0x7f973c1551e0 2024-04-25T00:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 -- 172.21.15.83:0/3053726316 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3835935506 0 0) 0x7f9720003530 con 0x7f973c1551e0 2024-04-25T00:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 -- 172.21.15.83:0/3053726316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c1555c0 msgr2=0x7f973c15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 --2- 172.21.15.83:0/3053726316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c1555c0 0x7f973c15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 -- 172.21.15.83:0/3053726316 >> v2:172.21.15.86:3300/0 conn(0x7f973c06e4c0 msgr2=0x7f973c06e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 --2- 172.21.15.83:0/3053726316 >> v2:172.21.15.86:3300/0 conn(0x7f973c06e4c0 0x7f973c06e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 -- 172.21.15.83:0/3053726316 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973c15c7d0 con 0x7f973c1551e0 2024-04-25T00:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 -- 172.21.15.83:0/3053726316 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f972c001a00 con 0x7f973c1551e0 2024-04-25T00:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f973a800640 1 --2- 172.21.15.83:0/3053726316 >> v2:172.21.15.86:3300/0 conn(0x7f973c06e4c0 0x7f973c06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 -- 172.21.15.83:0/3053726316 >> v1:172.21.15.83:6789/0 conn(0x7f973c1551e0 legacy=0x7f973c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 --2- 172.21.15.83:0/3053726316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9720003a30 0x7f9720003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 --2- 172.21.15.83:0/3053726316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c1555c0 0x7f9720004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 --2- 172.21.15.83:0/3053726316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720004d50 0x7f97200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 -- 172.21.15.83:0/3053726316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f973c06ee70 con 0x7f9720003a30 2024-04-25T00:12:53.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 -- 172.21.15.83:0/3053726316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f973c072460 con 0x7f9720004d50 2024-04-25T00:12:53.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739400640 1 -- 172.21.15.83:0/3053726316 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f972000dbe0 con 0x7f973c1555c0 2024-04-25T00:12:53.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739e00640 1 --2- 172.21.15.83:0/3053726316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9720003a30 0x7f9720003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9731400640 1 --2- 172.21.15.83:0/3053726316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c1555c0 0x7f9720004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f973a800640 1 --2- 172.21.15.83:0/3053726316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720004d50 0x7f97200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739e00640 1 -- 172.21.15.83:0/3053726316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c1555c0 msgr2=0x7f9720004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:53.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739e00640 1 --2- 172.21.15.83:0/3053726316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c1555c0 0x7f9720004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739e00640 1 -- 172.21.15.83:0/3053726316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720004d50 msgr2=0x7f97200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:53.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739e00640 1 --2- 172.21.15.83:0/3053726316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720004d50 0x7f97200051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f9739e00640 1 -- 172.21.15.83:0/3053726316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9720003530 con 0x7f9720003a30 2024-04-25T00:12:53.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.505+0000 7f973a800640 1 --2- 172.21.15.83:0/3053726316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720004d50 0x7f97200051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:53.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.509+0000 7f9739e00640 1 --2- 172.21.15.83:0/3053726316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9720003a30 0x7f9720003e50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f972c00a7a0 tx=0x7f972c01d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ba88e693f9d9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:53.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.509+0000 7f9739400640 1 -- 172.21.15.83:0/3053726316 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c00a3f0 con 0x7f9720003a30 2024-04-25T00:12:53.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.509+0000 7f9739400640 1 -- 172.21.15.83:0/3053726316 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972c0a9020 con 0x7f9720003a30 2024-04-25T00:12:53.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.509+0000 7f9739400640 1 -- 172.21.15.83:0/3053726316 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c0070d0 con 0x7f9720003a30 2024-04-25T00:12:53.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.509+0000 7f9741000640 1 -- 172.21.15.83:0/3053726316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9720003a30 msgr2=0x7f9720003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:53.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.509+0000 7f9741000640 1 --2- 172.21.15.83:0/3053726316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9720003a30 0x7f9720003e50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f972c00a7a0 tx=0x7f972c01d2e0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.509+0000 7f973a800640 1 -- 172.21.15.83:0/3053726316 reap_dead start 2024-04-25T00:12:53.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.509+0000 7f9741000640 1 -- 172.21.15.83:0/3053726316 shutdown_connections 2024-04-25T00:12:53.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.509+0000 7f9741000640 1 -- 172.21.15.83:0/3053726316 >> 172.21.15.83:0/3053726316 conn(0x7f973c0b1830 msgr2=0x7f973c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:53.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.509+0000 7f9741000640 1 -- 172.21.15.83:0/3053726316 shutdown_connections 2024-04-25T00:12:53.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.509+0000 7f9741000640 1 -- 172.21.15.83:0/3053726316 wait complete. 2024-04-25T00:12:53.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.509+0000 7f9741000640 1 Processor -- start 2024-04-25T00:12:53.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9741000640 1 -- start start 2024-04-25T00:12:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9741000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9720003a30 0x7f973c15f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9741000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c14fd90 0x7f973c1427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9741000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973c13f660 0x7f973c0b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9741000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f973c06df50 con 0x7f9720003a30 2024-04-25T00:12:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9741000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f973c06ddd0 con 0x7f973c13f660 2024-04-25T00:12:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9741000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f973c06e0d0 con 0x7f973c14fd90 2024-04-25T00:12:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9739e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9720003a30 0x7f973c15f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9739e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9720003a30 0x7f973c15f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55036/0 (socket says 172.21.15.83:55036) 2024-04-25T00:12:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9739e00640 1 -- 172.21.15.83:0/1256220623 learned_addr learned my addr 172.21.15.83:0/1256220623 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9731400640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c14fd90 0x7f973c1427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f973a800640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973c13f660 0x7f973c0b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9739e00640 1 -- 172.21.15.83:0/1256220623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c14fd90 msgr2=0x7f973c1427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9739e00640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c14fd90 0x7f973c1427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9739e00640 1 -- 172.21.15.83:0/1256220623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973c13f660 msgr2=0x7f973c0b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:53.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9739e00640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973c13f660 0x7f973c0b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:53.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9739e00640 1 -- 172.21.15.83:0/1256220623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973c140140 con 0x7f9720003a30 2024-04-25T00:12:53.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f973a800640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973c13f660 0x7f973c0b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:53.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9739e00640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9720003a30 0x7f973c15f6f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f972c0ab040 tx=0x7f9720005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:53.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9732a00640 1 -- 172.21.15.83:0/1256220623 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c0bc070 con 0x7f9720003a30 2024-04-25T00:12:53.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9732a00640 1 -- 172.21.15.83:0/1256220623 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972c001940 con 0x7f9720003a30 2024-04-25T00:12:53.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9732a00640 1 -- 172.21.15.83:0/1256220623 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c0072f0 con 0x7f9720003a30 2024-04-25T00:12:53.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9741000640 1 -- 172.21.15.83:0/1256220623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973c140420 con 0x7f9720003a30 2024-04-25T00:12:53.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9741000640 1 -- 172.21.15.83:0/1256220623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f973c1409e0 con 0x7f9720003a30 2024-04-25T00:12:53.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.513+0000 7f9741000640 1 -- 172.21.15.83:0/1256220623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f973c04f510 con 0x7f9720003a30 2024-04-25T00:12:53.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.517+0000 7f9732a00640 1 -- 172.21.15.83:0/1256220623 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f972c007490 con 0x7f9720003a30 2024-04-25T00:12:53.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.517+0000 7f9732a00640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97180678c0 0x7f9718069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:53.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.517+0000 7f9732a00640 1 -- 172.21.15.83:0/1256220623 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f972c11fcf0 con 0x7f9720003a30 2024-04-25T00:12:53.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.517+0000 7f9731400640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97180678c0 0x7f9718069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:53.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.517+0000 7f9731400640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97180678c0 0x7f9718069d80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f973c0b49a0 tx=0x7f9724008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:53.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.521+0000 7f9732a00640 1 -- 172.21.15.83:0/1256220623 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f972c0b4070 con 0x7f9720003a30 2024-04-25T00:12:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:53.717+0000 7f9741000640 1 -- 172.21.15.83:0/1256220623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f973c06ce30 con 0x7f9720003a30 2024-04-25T00:12:54.505 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T00:12:54.627 INFO:tasks.ceph.mds.d.smithi086.stderr:2024-04-25T00:12:54.624+0000 7f6f97a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:12:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.669+0000 7f9732a00640 1 -- 172.21.15.83:0/1256220623 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f972c016ff0 con 0x7f9720003a30 2024-04-25T00:12:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.673+0000 7f9741000640 1 -- 172.21.15.83:0/1256220623 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97180678c0 msgr2=0x7f9718069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.673+0000 7f9741000640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97180678c0 0x7f9718069d80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f973c0b49a0 tx=0x7f9724008040 comp rx=0 tx=0).stop 2024-04-25T00:12:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.673+0000 7f9741000640 1 -- 172.21.15.83:0/1256220623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9720003a30 msgr2=0x7f973c15f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.673+0000 7f9741000640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9720003a30 0x7f973c15f6f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f972c0ab040 tx=0x7f9720005840 comp rx=0 tx=0).stop 2024-04-25T00:12:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.673+0000 7f9741000640 1 -- 172.21.15.83:0/1256220623 shutdown_connections 2024-04-25T00:12:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.673+0000 7f9741000640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97180678c0 0x7f9718069d80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.673+0000 7f9741000640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c14fd90 0x7f973c1427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.673+0000 7f9741000640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973c13f660 0x7f973c0b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.673+0000 7f9741000640 1 --2- 172.21.15.83:0/1256220623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9720003a30 0x7f973c15f6f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.673+0000 7f9741000640 1 -- 172.21.15.83:0/1256220623 >> 172.21.15.83:0/1256220623 conn(0x7f973c0b1830 msgr2=0x7f973c153310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.673+0000 7f9741000640 1 -- 172.21.15.83:0/1256220623 shutdown_connections 2024-04-25T00:12:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.673+0000 7f9741000640 1 -- 172.21.15.83:0/1256220623 wait complete. 2024-04-25T00:12:54.698 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:12:54.698 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:12:54.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.837+0000 7f667a000640 1 Processor -- start 2024-04-25T00:12:54.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.837+0000 7f667a000640 1 -- start start 2024-04-25T00:12:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.837+0000 7f667a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f667406e950 0x7f667415f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.837+0000 7f667a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f667415fe50 0x7f66740ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.837+0000 7f667a000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66740ce000 con 0x7f6674159810 2024-04-25T00:12:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.837+0000 7f667a000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f667406e210 con 0x7f667406e950 2024-04-25T00:12:54.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.837+0000 7f667a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f667406da00 con 0x7f667415fe50 2024-04-25T00:12:54.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.837+0000 7f666aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f667406e950 0x7f667415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.837+0000 7f6673e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f667415fe50 0x7f66740ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.837+0000 7f6673400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6674159810 0x7f66740cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:47542/0 (socket says 172.21.15.83:47542) 2024-04-25T00:12:54.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.837+0000 7f6673400640 1 -- 172.21.15.83:0/3536743548 learned_addr learned my addr 172.21.15.83:0/3536743548 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:54.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.841+0000 7f6672a00640 1 -- 172.21.15.83:0/3536743548 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3510204447 0 0) 0x7f66740ce000 con 0x7f6674159810 2024-04-25T00:12:54.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.841+0000 7f6672a00640 1 -- 172.21.15.83:0/3536743548 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6658003530 con 0x7f6674159810 2024-04-25T00:12:54.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.841+0000 7f666aa00640 1 -- 172.21.15.83:0/3536743548 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f667415fe50 msgr2=0x7f66740ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:54.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.841+0000 7f666aa00640 1 --2- 172.21.15.83:0/3536743548 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f667415fe50 0x7f66740ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.841+0000 7f666aa00640 1 -- 172.21.15.83:0/3536743548 >> v1:172.21.15.86:6789/0 conn(0x7f6674159810 legacy=0x7f66740cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.841+0000 7f666aa00640 1 -- 172.21.15.83:0/3536743548 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667406eec0 con 0x7f667406e950 2024-04-25T00:12:54.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.841+0000 7f6673e00640 1 --2- 172.21.15.83:0/3536743548 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f667415fe50 0x7f66740ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:54.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.841+0000 7f666aa00640 1 --2- 172.21.15.83:0/3536743548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667406e950 0x7f667415f910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f667406db80 tx=0x7f665c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9f667a01845ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:54.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.841+0000 7f6672a00640 1 -- 172.21.15.83:0/3536743548 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0b0070 con 0x7f667406e950 2024-04-25T00:12:54.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.841+0000 7f6672a00640 1 -- 172.21.15.83:0/3536743548 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665c0ab600 con 0x7f667406e950 2024-04-25T00:12:54.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.841+0000 7f6672a00640 1 -- 172.21.15.83:0/3536743548 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0ab9d0 con 0x7f667406e950 2024-04-25T00:12:54.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.841+0000 7f667a000640 1 -- 172.21.15.83:0/3536743548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667406e950 msgr2=0x7f667415f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.841+0000 7f667a000640 1 --2- 172.21.15.83:0/3536743548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667406e950 0x7f667415f910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f667406db80 tx=0x7f665c01c410 comp rx=0 tx=0).stop 2024-04-25T00:12:54.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f667a000640 1 -- 172.21.15.83:0/3536743548 shutdown_connections 2024-04-25T00:12:54.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f667a000640 1 --2- 172.21.15.83:0/3536743548 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f667415fe50 0x7f66740ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f667a000640 1 --2- 172.21.15.83:0/3536743548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667406e950 0x7f667415f910 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f667a000640 1 -- 172.21.15.83:0/3536743548 >> 172.21.15.83:0/3536743548 conn(0x7f66740b1830 msgr2=0x7f66740b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:54.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f667a000640 1 -- 172.21.15.83:0/3536743548 shutdown_connections 2024-04-25T00:12:54.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f667a000640 1 -- 172.21.15.83:0/3536743548 wait complete. 2024-04-25T00:12:54.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f667a000640 1 Processor -- start 2024-04-25T00:12:54.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f667a000640 1 -- start start 2024-04-25T00:12:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f667a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f667406e950 0x7f667414fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f667a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6674159810 0x7f6674153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f667a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667415fe50 0x7f66741521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f667a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6674070f20 con 0x7f6674159810 2024-04-25T00:12:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f667a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6674070da0 con 0x7f667415fe50 2024-04-25T00:12:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f667a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6674168cb0 con 0x7f667406e950 2024-04-25T00:12:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f666aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6674159810 0x7f6674153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f666aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6674159810 0x7f6674153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56012/0 (socket says 172.21.15.83:56012) 2024-04-25T00:12:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f666aa00640 1 -- 172.21.15.83:0/2234416144 learned_addr learned my addr 172.21.15.83:0/2234416144 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f6673400640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f667406e950 0x7f667414fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f6673e00640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667415fe50 0x7f66741521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f666aa00640 1 -- 172.21.15.83:0/2234416144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f667406e950 msgr2=0x7f667414fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f666aa00640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f667406e950 0x7f667414fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f666aa00640 1 -- 172.21.15.83:0/2234416144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667415fe50 msgr2=0x7f66741521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f666aa00640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667415fe50 0x7f66741521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f666aa00640 1 -- 172.21.15.83:0/2234416144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66741500b0 con 0x7f6674159810 2024-04-25T00:12:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f6673e00640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667415fe50 0x7f66741521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.845+0000 7f6673400640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f667406e950 0x7f667414fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.849+0000 7f666aa00640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6674159810 0x7f6674153c00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f665c0ae040 tx=0x7f665c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.849+0000 7f666be00640 1 -- 172.21.15.83:0/2234416144 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0b8020 con 0x7f6674159810 2024-04-25T00:12:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.849+0000 7f666be00640 1 -- 172.21.15.83:0/2234416144 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665c0b4410 con 0x7f6674159810 2024-04-25T00:12:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.849+0000 7f667a000640 1 -- 172.21.15.83:0/2234416144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6674154140 con 0x7f6674159810 2024-04-25T00:12:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.849+0000 7f666be00640 1 -- 172.21.15.83:0/2234416144 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0b0040 con 0x7f6674159810 2024-04-25T00:12:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.849+0000 7f667a000640 1 -- 172.21.15.83:0/2234416144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6674152e40 con 0x7f6674159810 2024-04-25T00:12:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.849+0000 7f667a000640 1 -- 172.21.15.83:0/2234416144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f667404f510 con 0x7f6674159810 2024-04-25T00:12:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.849+0000 7f666be00640 1 -- 172.21.15.83:0/2234416144 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f665c0bb020 con 0x7f6674159810 2024-04-25T00:12:54.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.849+0000 7f666be00640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66500678f0 0x7f6650069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.849+0000 7f666be00640 1 -- 172.21.15.83:0/2234416144 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f665c11f8e0 con 0x7f6674159810 2024-04-25T00:12:54.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.849+0000 7f6673400640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66500678f0 0x7f6650069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.853+0000 7f6673400640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66500678f0 0x7f6650069db0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6664005890 tx=0x7f6664009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:54.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:54.853+0000 7f666be00640 1 -- 172.21.15.83:0/2234416144 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f665c0c2030 con 0x7f6674159810 2024-04-25T00:12:55.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.041+0000 7f667a000640 1 -- 172.21.15.83:0/2234416144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f667415f3e0 con 0x7f6674159810 2024-04-25T00:12:55.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.669+0000 7f666be00640 1 -- 172.21.15.83:0/2234416144 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7f665c016090 con 0x7f6674159810 2024-04-25T00:12:55.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.673+0000 7f667a000640 1 -- 172.21.15.83:0/2234416144 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66500678f0 msgr2=0x7f6650069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:55.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.673+0000 7f667a000640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66500678f0 0x7f6650069db0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6664005890 tx=0x7f6664009040 comp rx=0 tx=0).stop 2024-04-25T00:12:55.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.673+0000 7f667a000640 1 -- 172.21.15.83:0/2234416144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6674159810 msgr2=0x7f6674153c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:55.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.673+0000 7f667a000640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6674159810 0x7f6674153c00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f665c0ae040 tx=0x7f665c0191b0 comp rx=0 tx=0).stop 2024-04-25T00:12:55.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.673+0000 7f667a000640 1 -- 172.21.15.83:0/2234416144 shutdown_connections 2024-04-25T00:12:55.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.673+0000 7f667a000640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f66500678f0 0x7f6650069db0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:55.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.673+0000 7f667a000640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667415fe50 0x7f66741521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:55.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.673+0000 7f667a000640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6674159810 0x7f6674153c00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:55.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.673+0000 7f667a000640 1 --2- 172.21.15.83:0/2234416144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f667406e950 0x7f667414fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:55.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.673+0000 7f667a000640 1 -- 172.21.15.83:0/2234416144 >> 172.21.15.83:0/2234416144 conn(0x7f66740b1830 msgr2=0x7f66740cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:55.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.673+0000 7f667a000640 1 -- 172.21.15.83:0/2234416144 shutdown_connections 2024-04-25T00:12:55.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.673+0000 7f667a000640 1 -- 172.21.15.83:0/2234416144 wait complete. 2024-04-25T00:12:55.694 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:12:55.695 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:12:55.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8288400640 1 Processor -- start 2024-04-25T00:12:55.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8288400640 1 -- start start 2024-04-25T00:12:55.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8288400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82801599f0 0x7f82800cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:55.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8288400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f828015c340 0x7f828015c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:55.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8288400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82800ccc60 con 0x7f828015ccc0 2024-04-25T00:12:55.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8288400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8280159dd0 con 0x7f828015c340 2024-04-25T00:12:55.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8288400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f828006dba0 con 0x7f82801599f0 2024-04-25T00:12:55.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8285000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f828015c340 0x7f828015c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:55.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8285000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f828015c340 0x7f828015c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56022/0 (socket says 172.21.15.83:56022) 2024-04-25T00:12:55.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8285000640 1 -- 172.21.15.83:0/1431306455 learned_addr learned my addr 172.21.15.83:0/1431306455 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:55.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8285a00640 1 --2- 172.21.15.83:0/1431306455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82801599f0 0x7f82800cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:55.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8285000640 1 -- 172.21.15.83:0/1431306455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82801599f0 msgr2=0x7f82800cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:55.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8285000640 1 --2- 172.21.15.83:0/1431306455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82801599f0 0x7f82800cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:55.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8285000640 1 -- 172.21.15.83:0/1431306455 >> v1:172.21.15.86:6789/0 conn(0x7f828015ccc0 legacy=0x7f828015f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:55.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.829+0000 7f8285000640 1 -- 172.21.15.83:0/1431306455 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828006de80 con 0x7f828015c340 2024-04-25T00:12:55.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.833+0000 7f8285000640 1 --2- 172.21.15.83:0/1431306455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828015c340 0x7f828015c780 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f826c0098a0 tx=0x7f826c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4e12bf182b3095 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:55.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.833+0000 7f827fe00640 1 -- 172.21.15.83:0/1431306455 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0b1070 con 0x7f828015c340 2024-04-25T00:12:55.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.833+0000 7f827fe00640 1 -- 172.21.15.83:0/1431306455 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826c0ac620 con 0x7f828015c340 2024-04-25T00:12:55.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.833+0000 7f827fe00640 1 -- 172.21.15.83:0/1431306455 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0ac9f0 con 0x7f828015c340 2024-04-25T00:12:55.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.833+0000 7f8288400640 1 -- 172.21.15.83:0/1431306455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828015c340 msgr2=0x7f828015c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:55.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.833+0000 7f8288400640 1 --2- 172.21.15.83:0/1431306455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828015c340 0x7f828015c780 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f826c0098a0 tx=0x7f826c01c4b0 comp rx=0 tx=0).stop 2024-04-25T00:12:55.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.833+0000 7f8288400640 1 -- 172.21.15.83:0/1431306455 shutdown_connections 2024-04-25T00:12:55.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.833+0000 7f8288400640 1 --2- 172.21.15.83:0/1431306455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828015c340 0x7f828015c780 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:55.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.833+0000 7f8288400640 1 --2- 172.21.15.83:0/1431306455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82801599f0 0x7f82800cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:55.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.833+0000 7f8288400640 1 -- 172.21.15.83:0/1431306455 >> 172.21.15.83:0/1431306455 conn(0x7f82800b1830 msgr2=0x7f82800b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:55.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.833+0000 7f8288400640 1 -- 172.21.15.83:0/1431306455 shutdown_connections 2024-04-25T00:12:55.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.833+0000 7f8288400640 1 -- 172.21.15.83:0/1431306455 wait complete. 2024-04-25T00:12:55.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8288400640 1 Processor -- start 2024-04-25T00:12:55.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8288400640 1 -- start start 2024-04-25T00:12:55.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8288400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82801599f0 0x7f828015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:55.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8288400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828015c340 0x7f82801f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:55.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8288400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f828015ccc0 0x7f8280149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:55.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8288400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82800cf870 con 0x7f828015c340 2024-04-25T00:12:55.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8288400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82800cf6f0 con 0x7f82801599f0 2024-04-25T00:12:55.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8288400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8280168cd0 con 0x7f828015ccc0 2024-04-25T00:12:55.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8285000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828015c340 0x7f82801f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:55.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8285000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828015c340 0x7f82801f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56032/0 (socket says 172.21.15.83:56032) 2024-04-25T00:12:55.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8285000640 1 -- 172.21.15.83:0/3139221613 learned_addr learned my addr 172.21.15.83:0/3139221613 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:55.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8285a00640 1 --2- 172.21.15.83:0/3139221613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82801599f0 0x7f828015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:55.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8286400640 1 --2- 172.21.15.83:0/3139221613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f828015ccc0 0x7f8280149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:55.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8285000640 1 -- 172.21.15.83:0/3139221613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f828015ccc0 msgr2=0x7f8280149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:55.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8285000640 1 --2- 172.21.15.83:0/3139221613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f828015ccc0 0x7f8280149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8285000640 1 -- 172.21.15.83:0/3139221613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82801599f0 msgr2=0x7f828015b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8285000640 1 --2- 172.21.15.83:0/3139221613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82801599f0 0x7f828015b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8285000640 1 -- 172.21.15.83:0/3139221613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828014b370 con 0x7f828015c340 2024-04-25T00:12:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8285000640 1 --2- 172.21.15.83:0/3139221613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828015c340 0x7f82801f5dc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f826c01a0e0 tx=0x7f826c019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f827d600640 1 -- 172.21.15.83:0/3139221613 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0b1040 con 0x7f828015c340 2024-04-25T00:12:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f827d600640 1 -- 172.21.15.83:0/3139221613 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826c0ba070 con 0x7f828015c340 2024-04-25T00:12:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f827d600640 1 -- 172.21.15.83:0/3139221613 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0b5660 con 0x7f828015c340 2024-04-25T00:12:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8288400640 1 -- 172.21.15.83:0/3139221613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82801f6300 con 0x7f828015c340 2024-04-25T00:12:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8288400640 1 -- 172.21.15.83:0/3139221613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82801f55c0 con 0x7f828015c340 2024-04-25T00:12:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.837+0000 7f8288400640 1 -- 172.21.15.83:0/3139221613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82801f53b0 con 0x7f828015c340 2024-04-25T00:12:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.841+0000 7f827d600640 1 -- 172.21.15.83:0/3139221613 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f826c0b5800 con 0x7f828015c340 2024-04-25T00:12:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.841+0000 7f827d600640 1 --2- 172.21.15.83:0/3139221613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f82600678c0 0x7f8260069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.841+0000 7f827d600640 1 -- 172.21.15.83:0/3139221613 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f826c11fbe0 con 0x7f828015c340 2024-04-25T00:12:55.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.841+0000 7f8285a00640 1 --2- 172.21.15.83:0/3139221613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f82600678c0 0x7f8260069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:55.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.841+0000 7f8285a00640 1 --2- 172.21.15.83:0/3139221613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f82600678c0 0x7f8260069d80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8274006aa0 tx=0x7f8274008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:55.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:55.845+0000 7f827d600640 1 -- 172.21.15.83:0/3139221613 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f826c016130 con 0x7f828015c340 2024-04-25T00:12:56.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.029+0000 7f8288400640 1 -- 172.21.15.83:0/3139221613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82800737f0 con 0x7f828015c340 2024-04-25T00:12:56.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.029+0000 7f827d600640 1 -- 172.21.15.83:0/3139221613 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5181 (secure 0 0 0) 0x7f826c0f04f0 con 0x7f828015c340 2024-04-25T00:12:56.034 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:56.034 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2761747029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2761747029},{"type":"v1","addr":"172.21.15.83:6837","nonce":2761747029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2989939837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2989939837},{"type":"v1","addr":"172.21.15.86:6835","nonce":2989939837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2291906415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2291906415},{"type":"v1","addr":"172.21.15.83:6836","nonce":2291906415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:12:52.026135+0000","modified":"2024-04-25T00:12:55.045476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5814":{"gid":5814,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6834/1690820142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1690820142},{"type":"v1","addr":"172.21.15.86:6834","nonce":1690820142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5814,"qdb_cluster":[5814]},"id":3}]} 2024-04-25T00:12:56.035 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-04-25T00:12:56.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.033+0000 7f8288400640 1 -- 172.21.15.83:0/3139221613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f82600678c0 msgr2=0x7f8260069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.033+0000 7f8288400640 1 --2- 172.21.15.83:0/3139221613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f82600678c0 0x7f8260069d80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8274006aa0 tx=0x7f8274008040 comp rx=0 tx=0).stop 2024-04-25T00:12:56.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.033+0000 7f8288400640 1 -- 172.21.15.83:0/3139221613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828015c340 msgr2=0x7f82801f5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.033+0000 7f8288400640 1 --2- 172.21.15.83:0/3139221613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828015c340 0x7f82801f5dc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f826c01a0e0 tx=0x7f826c019250 comp rx=0 tx=0).stop 2024-04-25T00:12:56.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.033+0000 7f8288400640 1 -- 172.21.15.83:0/3139221613 shutdown_connections 2024-04-25T00:12:56.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.033+0000 7f8288400640 1 --2- 172.21.15.83:0/3139221613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f82600678c0 0x7f8260069d80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.033+0000 7f8288400640 1 --2- 172.21.15.83:0/3139221613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f828015ccc0 0x7f8280149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.033+0000 7f8288400640 1 --2- 172.21.15.83:0/3139221613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828015c340 0x7f82801f5dc0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.033+0000 7f8288400640 1 --2- 172.21.15.83:0/3139221613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82801599f0 0x7f828015b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.033+0000 7f8288400640 1 -- 172.21.15.83:0/3139221613 >> 172.21.15.83:0/3139221613 conn(0x7f82800b1830 msgr2=0x7f82800b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:56.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.033+0000 7f8288400640 1 -- 172.21.15.83:0/3139221613 shutdown_connections 2024-04-25T00:12:56.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.033+0000 7f8288400640 1 -- 172.21.15.83:0/3139221613 wait complete. 2024-04-25T00:12:56.054 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:12:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.185+0000 7f4ec5a00640 1 Processor -- start 2024-04-25T00:12:56.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.185+0000 7f4ec5a00640 1 -- start start 2024-04-25T00:12:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4ec5a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ec00cc510 0x7f4ec015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4ec5a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ec015cf90 0x7f4ec00ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4ec5a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec00ccc60 con 0x7f4ec01599f0 2024-04-25T00:12:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4ec5a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec0159dd0 con 0x7f4ec015cf90 2024-04-25T00:12:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4ec5a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec006dba0 con 0x7f4ec00cc510 2024-04-25T00:12:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4ebf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ec015cf90 0x7f4ec00ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4ebf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ec015cf90 0x7f4ec00ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:36530/0 (socket says 172.21.15.83:36530) 2024-04-25T00:12:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4ebf400640 1 -- 172.21.15.83:0/1743848593 learned_addr learned my addr 172.21.15.83:0/1743848593 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4ebea00640 1 --1- 172.21.15.83:0/1743848593 >> v1:172.21.15.86:6789/0 conn(0x7f4ec01599f0 0x7f4ec015c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:47564/0 (socket says 172.21.15.83:47564) 2024-04-25T00:12:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4eb6000640 1 --2- 172.21.15.83:0/1743848593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ec00cc510 0x7f4ec015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4ebf400640 1 -- 172.21.15.83:0/1743848593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ec00cc510 msgr2=0x7f4ec015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4ebf400640 1 --2- 172.21.15.83:0/1743848593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ec00cc510 0x7f4ec015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4ebf400640 1 -- 172.21.15.83:0/1743848593 >> v1:172.21.15.86:6789/0 conn(0x7f4ec01599f0 legacy=0x7f4ec015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4ebf400640 1 -- 172.21.15.83:0/1743848593 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec006de80 con 0x7f4ec015cf90 2024-04-25T00:12:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.189+0000 7f4ebe000640 1 -- 172.21.15.83:0/1743848593 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2435180749 0 0) 0x7f4eb0001120 con 0x7f4ec01599f0 2024-04-25T00:12:56.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ebf400640 1 --2- 172.21.15.83:0/1743848593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ec015cf90 0x7f4ec00ce6e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009e50 tx=0x7f4eb801c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb6878b68214aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ebe000640 1 -- 172.21.15.83:0/1743848593 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb80a56b0 con 0x7f4ec015cf90 2024-04-25T00:12:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ebe000640 1 -- 172.21.15.83:0/1743848593 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb80a5850 con 0x7f4ec015cf90 2024-04-25T00:12:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ebe000640 1 -- 172.21.15.83:0/1743848593 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb80a5c20 con 0x7f4ec015cf90 2024-04-25T00:12:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1743848593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ec015cf90 msgr2=0x7f4ec00ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 --2- 172.21.15.83:0/1743848593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ec015cf90 0x7f4ec00ce6e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009e50 tx=0x7f4eb801c9a0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1743848593 shutdown_connections 2024-04-25T00:12:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 --2- 172.21.15.83:0/1743848593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ec015cf90 0x7f4ec00ce6e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 --2- 172.21.15.83:0/1743848593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ec00cc510 0x7f4ec015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1743848593 >> 172.21.15.83:0/1743848593 conn(0x7f4ec00b1830 msgr2=0x7f4ec00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1743848593 shutdown_connections 2024-04-25T00:12:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1743848593 wait complete. 2024-04-25T00:12:56.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 Processor -- start 2024-04-25T00:12:56.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 -- start start 2024-04-25T00:12:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ec00cc510 0x7f4ec014b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec01599f0 0x7f4ec01572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ec015cf90 0x7f4ec0155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec00cf5b0 con 0x7f4ec01599f0 2024-04-25T00:12:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec00cf430 con 0x7f4ec015cf90 2024-04-25T00:12:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.193+0000 7f4ec5a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec00cf730 con 0x7f4ec00cc510 2024-04-25T00:12:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4eb6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec01599f0 0x7f4ec01572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4eb6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec01599f0 0x7f4ec01572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56046/0 (socket says 172.21.15.83:56046) 2024-04-25T00:12:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4eb6000640 1 -- 172.21.15.83:0/1372329093 learned_addr learned my addr 172.21.15.83:0/1372329093 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4ebea00640 1 --2- 172.21.15.83:0/1372329093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ec00cc510 0x7f4ec014b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4ebf400640 1 --2- 172.21.15.83:0/1372329093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ec015cf90 0x7f4ec0155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4eb6000640 1 -- 172.21.15.83:0/1372329093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ec00cc510 msgr2=0x7f4ec014b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4eb6000640 1 --2- 172.21.15.83:0/1372329093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ec00cc510 0x7f4ec014b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4eb6000640 1 -- 172.21.15.83:0/1372329093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ec015cf90 msgr2=0x7f4ec0155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4eb6000640 1 --2- 172.21.15.83:0/1372329093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ec015cf90 0x7f4ec0155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4eb6000640 1 -- 172.21.15.83:0/1372329093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec015d730 con 0x7f4ec01599f0 2024-04-25T00:12:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4eb6000640 1 --2- 172.21.15.83:0/1372329093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec01599f0 0x7f4ec01572e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800e870 tx=0x7f4ea800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:56.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4eb7400640 1 -- 172.21.15.83:0/1372329093 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8017070 con 0x7f4ec01599f0 2024-04-25T00:12:56.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4eb7400640 1 -- 172.21.15.83:0/1372329093 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea800c940 con 0x7f4ec01599f0 2024-04-25T00:12:56.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4eb7400640 1 -- 172.21.15.83:0/1372329093 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea800cd10 con 0x7f4ec01599f0 2024-04-25T00:12:56.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1372329093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec015da10 con 0x7f4ec01599f0 2024-04-25T00:12:56.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1372329093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec015dfd0 con 0x7f4ec01599f0 2024-04-25T00:12:56.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1372329093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec00737f0 con 0x7f4ec01599f0 2024-04-25T00:12:56.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.197+0000 7f4eb7400640 1 -- 172.21.15.83:0/1372329093 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4ea801a070 con 0x7f4ec01599f0 2024-04-25T00:12:56.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.201+0000 7f4eb7400640 1 --2- 172.21.15.83:0/1372329093 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4e9c0678f0 0x7f4e9c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.201+0000 7f4eb7400640 1 -- 172.21.15.83:0/1372329093 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f4ea8085fe0 con 0x7f4ec01599f0 2024-04-25T00:12:56.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.201+0000 7f4ebea00640 1 --2- 172.21.15.83:0/1372329093 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4e9c0678f0 0x7f4e9c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.201+0000 7f4ebea00640 1 --2- 172.21.15.83:0/1372329093 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4e9c0678f0 0x7f4e9c069db0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0000f70 tx=0x7f4eb0007b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:56.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.205+0000 7f4eb7400640 1 -- 172.21.15.83:0/1372329093 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4ea8056390 con 0x7f4ec01599f0 2024-04-25T00:12:56.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.361+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1372329093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ec004f5f0 con 0x7f4ec01599f0 2024-04-25T00:12:56.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.361+0000 7f4eb7400640 1 -- 172.21.15.83:0/1372329093 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16858 (secure 0 0 0) 0x7f4ea805a690 con 0x7f4ec01599f0 2024-04-25T00:12:56.364 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:56.364 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":49,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:53.651846+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:50.394645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:51.018842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:12:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.365+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1372329093 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4e9c0678f0 msgr2=0x7f4e9c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.365+0000 7f4ec5a00640 1 --2- 172.21.15.83:0/1372329093 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4e9c0678f0 0x7f4e9c069db0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0000f70 tx=0x7f4eb0007b60 comp rx=0 tx=0).stop 2024-04-25T00:12:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.365+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1372329093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec01599f0 msgr2=0x7f4ec01572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.365+0000 7f4ec5a00640 1 --2- 172.21.15.83:0/1372329093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec01599f0 0x7f4ec01572e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800e870 tx=0x7f4ea800ed30 comp rx=0 tx=0).stop 2024-04-25T00:12:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.365+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1372329093 shutdown_connections 2024-04-25T00:12:56.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.365+0000 7f4ec5a00640 1 --2- 172.21.15.83:0/1372329093 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4e9c0678f0 0x7f4e9c069db0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.365+0000 7f4ec5a00640 1 --2- 172.21.15.83:0/1372329093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ec015cf90 0x7f4ec0155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.365+0000 7f4ec5a00640 1 --2- 172.21.15.83:0/1372329093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec01599f0 0x7f4ec01572e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.365+0000 7f4ec5a00640 1 --2- 172.21.15.83:0/1372329093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ec00cc510 0x7f4ec014b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.365+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1372329093 >> 172.21.15.83:0/1372329093 conn(0x7f4ec00b1830 msgr2=0x7f4ec0070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:56.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.365+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1372329093 shutdown_connections 2024-04-25T00:12:56.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.365+0000 7f4ec5a00640 1 -- 172.21.15.83:0/1372329093 wait complete. 2024-04-25T00:12:56.385 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:12:56.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:12:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.521+0000 7faa9d000640 1 Processor -- start 2024-04-25T00:12:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.521+0000 7faa9d000640 1 -- start start 2024-04-25T00:12:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.521+0000 7faa9d000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa98160040 0x7faa980cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.521+0000 7faa9d000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa9815c340 con 0x7faa980b07d0 2024-04-25T00:12:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.521+0000 7faa9d000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa980cc280 con 0x7faa980b18c0 2024-04-25T00:12:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.521+0000 7faa9d000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7faa980cc560 con 0x7faa98160040 2024-04-25T00:12:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.521+0000 7faa97400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7faa980b07d0 0x7faa9815cae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42244/0 (socket says 172.21.15.83:42244) 2024-04-25T00:12:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.521+0000 7faa97400640 1 -- 172.21.15.83:0/4034209073 learned_addr learned my addr 172.21.15.83:0/4034209073 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.521+0000 7faa97e00640 1 --2- 172.21.15.83:0/4034209073 >> v2:172.21.15.83:3300/0 conn(0x7faa98160040 0x7faa980cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.521+0000 7faa96000640 1 -- 172.21.15.83:0/4034209073 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3730225180 0 0) 0x7faa88001120 con 0x7faa980b07d0 2024-04-25T00:12:56.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.521+0000 7faa96000640 1 -- 172.21.15.83:0/4034209073 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa7c003530 con 0x7faa980b07d0 2024-04-25T00:12:56.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.521+0000 7faa97e00640 1 -- 172.21.15.83:0/4034209073 >> v1:172.21.15.86:6789/0 conn(0x7faa980b18c0 legacy=0x7faa9815f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.521+0000 7faa97e00640 1 -- 172.21.15.83:0/4034209073 >> v1:172.21.15.83:6789/0 conn(0x7faa980b07d0 legacy=0x7faa9815cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.521+0000 7faa97e00640 1 -- 172.21.15.83:0/4034209073 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa980cc840 con 0x7faa98160040 2024-04-25T00:12:56.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.525+0000 7faa97e00640 1 --2- 172.21.15.83:0/4034209073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa98160040 0x7faa980cbc80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7faa90009d90 tx=0x7faa9001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e47ad6d6e789175 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:56.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.525+0000 7faa96000640 1 -- 172.21.15.83:0/4034209073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa900b2070 con 0x7faa98160040 2024-04-25T00:12:56.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.525+0000 7faa96000640 1 -- 172.21.15.83:0/4034209073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa900a5a20 con 0x7faa98160040 2024-04-25T00:12:56.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.525+0000 7faa96000640 1 -- 172.21.15.83:0/4034209073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa900a5df0 con 0x7faa98160040 2024-04-25T00:12:56.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.525+0000 7faa9d000640 1 -- 172.21.15.83:0/4034209073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa98160040 msgr2=0x7faa980cbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.525+0000 7faa9d000640 1 --2- 172.21.15.83:0/4034209073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa98160040 0x7faa980cbc80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7faa90009d90 tx=0x7faa9001c940 comp rx=0 tx=0).stop 2024-04-25T00:12:56.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.525+0000 7faa9d000640 1 -- 172.21.15.83:0/4034209073 shutdown_connections 2024-04-25T00:12:56.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.525+0000 7faa9d000640 1 --2- 172.21.15.83:0/4034209073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa98160040 0x7faa980cbc80 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.525+0000 7faa9d000640 1 -- 172.21.15.83:0/4034209073 >> 172.21.15.83:0/4034209073 conn(0x7faa980d0210 msgr2=0x7faa980d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:56.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.525+0000 7faa9d000640 1 -- 172.21.15.83:0/4034209073 shutdown_connections 2024-04-25T00:12:56.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.525+0000 7faa9d000640 1 -- 172.21.15.83:0/4034209073 wait complete. 2024-04-25T00:12:56.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.525+0000 7faa9d000640 1 Processor -- start 2024-04-25T00:12:56.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.525+0000 7faa9d000640 1 -- start start 2024-04-25T00:12:56.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa9d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa980b07d0 0x7faa980999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa9d000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa980b18c0 0x7faa980af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa9d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa980af580 0x7faa980b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa9d000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa9816a070 con 0x7faa980b07d0 2024-04-25T00:12:56.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa9d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa98169ef0 con 0x7faa980af580 2024-04-25T00:12:56.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa9d000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7faa9816a1f0 con 0x7faa980b18c0 2024-04-25T00:12:56.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa97400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa980b07d0 0x7faa980999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa97400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa980b07d0 0x7faa980999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56076/0 (socket says 172.21.15.83:56076) 2024-04-25T00:12:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa97400640 1 -- 172.21.15.83:0/4238723733 learned_addr learned my addr 172.21.15.83:0/4238723733 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa97e00640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa980af580 0x7faa980b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa96a00640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa980b18c0 0x7faa980af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa97400640 1 -- 172.21.15.83:0/4238723733 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa980b18c0 msgr2=0x7faa980af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa97400640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa980b18c0 0x7faa980af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa97400640 1 -- 172.21.15.83:0/4238723733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa980af580 msgr2=0x7faa980b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa97400640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa980af580 0x7faa980b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa97400640 1 -- 172.21.15.83:0/4238723733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa980afb90 con 0x7faa980b07d0 2024-04-25T00:12:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa96a00640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa980b18c0 0x7faa980af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa97e00640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa980af580 0x7faa980b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa97400640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa980b07d0 0x7faa980999c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7faa8800c760 tx=0x7faa8800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa8f400640 1 -- 172.21.15.83:0/4238723733 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88017070 con 0x7faa980b07d0 2024-04-25T00:12:56.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa8f400640 1 -- 172.21.15.83:0/4238723733 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa88005ac0 con 0x7faa980b07d0 2024-04-25T00:12:56.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa8f400640 1 -- 172.21.15.83:0/4238723733 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88007070 con 0x7faa980b07d0 2024-04-25T00:12:56.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa9d000640 1 -- 172.21.15.83:0/4238723733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa980cdf80 con 0x7faa980b07d0 2024-04-25T00:12:56.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa9d000640 1 -- 172.21.15.83:0/4238723733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa980b5160 con 0x7faa980b07d0 2024-04-25T00:12:56.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa9d000640 1 -- 172.21.15.83:0/4238723733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9804f510 con 0x7faa980b07d0 2024-04-25T00:12:56.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.529+0000 7faa8f400640 1 -- 172.21.15.83:0/4238723733 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faa8801a070 con 0x7faa980b07d0 2024-04-25T00:12:56.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.533+0000 7faa8f400640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faa780678c0 0x7faa78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.533+0000 7faa8f400640 1 -- 172.21.15.83:0/4238723733 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7faa88085350 con 0x7faa980b07d0 2024-04-25T00:12:56.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.533+0000 7faa96a00640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faa780678c0 0x7faa78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.533+0000 7faa96a00640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faa780678c0 0x7faa78069d80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faa981534a0 tx=0x7faa84009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:56.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.537+0000 7faa8f400640 1 -- 172.21.15.83:0/4238723733 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faa9815c340 con 0x7faa980b07d0 2024-04-25T00:12:56.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.693+0000 7faa9d000640 1 -- 172.21.15.83:0/4238723733 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa980b6190 con 0x7faa780678c0 2024-04-25T00:12:56.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.697+0000 7faa8f400640 1 -- 172.21.15.83:0/4238723733 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251934 (secure 0 0 0) 0x7faa980b6190 con 0x7faa780678c0 2024-04-25T00:12:56.704 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:56.705 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:12:56.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.701+0000 7faa9d000640 1 -- 172.21.15.83:0/4238723733 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faa780678c0 msgr2=0x7faa78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.701+0000 7faa9d000640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faa780678c0 0x7faa78069d80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faa981534a0 tx=0x7faa84009040 comp rx=0 tx=0).stop 2024-04-25T00:12:56.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.701+0000 7faa9d000640 1 -- 172.21.15.83:0/4238723733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa980b07d0 msgr2=0x7faa980999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.701+0000 7faa9d000640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa980b07d0 0x7faa980999c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7faa8800c760 tx=0x7faa8800cc20 comp rx=0 tx=0).stop 2024-04-25T00:12:56.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.701+0000 7faa9d000640 1 -- 172.21.15.83:0/4238723733 shutdown_connections 2024-04-25T00:12:56.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.701+0000 7faa9d000640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faa780678c0 0x7faa78069d80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.701+0000 7faa9d000640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa980b18c0 0x7faa980af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.701+0000 7faa9d000640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa980af580 0x7faa980b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.701+0000 7faa9d000640 1 --2- 172.21.15.83:0/4238723733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa980b07d0 0x7faa980999c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.701+0000 7faa9d000640 1 -- 172.21.15.83:0/4238723733 >> 172.21.15.83:0/4238723733 conn(0x7faa980d0210 msgr2=0x7faa9815f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:56.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.701+0000 7faa9d000640 1 -- 172.21.15.83:0/4238723733 shutdown_connections 2024-04-25T00:12:56.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.701+0000 7faa9d000640 1 -- 172.21.15.83:0/4238723733 wait complete. 2024-04-25T00:12:56.724 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-04-25T00:12:56.466164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226036,"kb_used_data":5708,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748684,"statfs":{"total":773094113280,"available":772862652416,"internally_reserved":0,"allocated":5844992,"data_stored":3212480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.835076"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.669780+0000","last_change":"2024-04-25T00:12:52.663528+0000","last_active":"2024-04-25T00:12:52.669780+0000","last_peered":"2024-04-25T00:12:52.669780+0000","last_clean":"2024-04-25T00:12:52.669780+0000","last_became_active":"2024-04-25T00:12:52.663254+0000","last_became_peered":"2024-04-25T00:12:52.663254+0000","last_unstale":"2024-04-25T00:12:52.669780+0000","last_undegraded":"2024-04-25T00:12:52.669780+0000","last_fullsized":"2024-04-25T00:12:52.669780+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:34.570444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.670023+0000","last_change":"2024-04-25T00:12:52.668871+0000","last_active":"2024-04-25T00:12:53.670023+0000","last_peered":"2024-04-25T00:12:53.670023+0000","last_clean":"2024-04-25T00:12:53.670023+0000","last_became_active":"2024-04-25T00:12:52.667659+0000","last_became_peered":"2024-04-25T00:12:52.667659+0000","last_unstale":"2024-04-25T00:12:53.670023+0000","last_undegraded":"2024-04-25T00:12:53.670023+0000","last_fullsized":"2024-04-25T00:12:53.670023+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:12.909892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656795+0000","last_change":"2024-04-25T00:12:52.667397+0000","last_active":"2024-04-25T00:12:53.656795+0000","last_peered":"2024-04-25T00:12:53.656795+0000","last_clean":"2024-04-25T00:12:53.656795+0000","last_became_active":"2024-04-25T00:12:52.661782+0000","last_became_peered":"2024-04-25T00:12:52.661782+0000","last_unstale":"2024-04-25T00:12:53.656795+0000","last_undegraded":"2024-04-25T00:12:53.656795+0000","last_fullsized":"2024-04-25T00:12:53.656795+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:01.498756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669725+0000","last_change":"2024-04-25T00:12:52.678156+0000","last_active":"2024-04-25T00:12:53.669725+0000","last_peered":"2024-04-25T00:12:53.669725+0000","last_clean":"2024-04-25T00:12:53.669725+0000","last_became_active":"2024-04-25T00:12:52.677825+0000","last_became_peered":"2024-04-25T00:12:52.677825+0000","last_unstale":"2024-04-25T00:12:53.669725+0000","last_undegraded":"2024-04-25T00:12:53.669725+0000","last_fullsized":"2024-04-25T00:12:53.669725+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:48.214697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.675582+0000","last_change":"2024-04-25T00:12:52.672099+0000","last_active":"2024-04-25T00:12:52.675582+0000","last_peered":"2024-04-25T00:12:52.675582+0000","last_clean":"2024-04-25T00:12:52.675582+0000","last_became_active":"2024-04-25T00:12:52.671658+0000","last_became_peered":"2024-04-25T00:12:52.671658+0000","last_unstale":"2024-04-25T00:12:52.675582+0000","last_undegraded":"2024-04-25T00:12:52.675582+0000","last_fullsized":"2024-04-25T00:12:52.675582+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:40.554694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.672046+0000","last_change":"2024-04-25T00:12:52.668232+0000","last_active":"2024-04-25T00:12:52.672046+0000","last_peered":"2024-04-25T00:12:52.672046+0000","last_clean":"2024-04-25T00:12:52.672046+0000","last_became_active":"2024-04-25T00:12:52.667983+0000","last_became_peered":"2024-04-25T00:12:52.667983+0000","last_unstale":"2024-04-25T00:12:52.672046+0000","last_undegraded":"2024-04-25T00:12:52.672046+0000","last_fullsized":"2024-04-25T00:12:52.672046+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:05.558235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.655919+0000","last_change":"2024-04-25T00:12:52.666008+0000","last_active":"2024-04-25T00:12:53.655919+0000","last_peered":"2024-04-25T00:12:53.655919+0000","last_clean":"2024-04-25T00:12:53.655919+0000","last_became_active":"2024-04-25T00:12:52.665732+0000","last_became_peered":"2024-04-25T00:12:52.665732+0000","last_unstale":"2024-04-25T00:12:53.655919+0000","last_undegraded":"2024-04-25T00:12:53.655919+0000","last_fullsized":"2024-04-25T00:12:53.655919+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:42.072677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326299+0000","last_change":"2024-04-25T00:12:52.666739+0000","last_active":"2024-04-25T00:12:54.326299+0000","last_peered":"2024-04-25T00:12:54.326299+0000","last_clean":"2024-04-25T00:12:54.326299+0000","last_became_active":"2024-04-25T00:12:52.666476+0000","last_became_peered":"2024-04-25T00:12:52.666476+0000","last_unstale":"2024-04-25T00:12:54.326299+0000","last_undegraded":"2024-04-25T00:12:54.326299+0000","last_fullsized":"2024-04-25T00:12:54.326299+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:41.077845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326682+0000","last_change":"2024-04-25T00:12:52.665887+0000","last_active":"2024-04-25T00:12:54.326682+0000","last_peered":"2024-04-25T00:12:54.326682+0000","last_clean":"2024-04-25T00:12:54.326682+0000","last_became_active":"2024-04-25T00:12:52.665579+0000","last_became_peered":"2024-04-25T00:12:52.665579+0000","last_unstale":"2024-04-25T00:12:54.326682+0000","last_undegraded":"2024-04-25T00:12:54.326682+0000","last_fullsized":"2024-04-25T00:12:54.326682+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:46.257055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.669946+0000","last_change":"2024-04-25T00:12:52.666663+0000","last_active":"2024-04-25T00:12:52.669946+0000","last_peered":"2024-04-25T00:12:52.669946+0000","last_clean":"2024-04-25T00:12:52.669946+0000","last_became_active":"2024-04-25T00:12:52.666391+0000","last_became_peered":"2024-04-25T00:12:52.666391+0000","last_unstale":"2024-04-25T00:12:52.669946+0000","last_undegraded":"2024-04-25T00:12:52.669946+0000","last_fullsized":"2024-04-25T00:12:52.669946+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:10.726792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.657270+0000","last_change":"2024-04-25T00:12:52.669222+0000","last_active":"2024-04-25T00:12:53.657270+0000","last_peered":"2024-04-25T00:12:53.657270+0000","last_clean":"2024-04-25T00:12:53.657270+0000","last_became_active":"2024-04-25T00:12:52.668951+0000","last_became_peered":"2024-04-25T00:12:52.668951+0000","last_unstale":"2024-04-25T00:12:53.657270+0000","last_undegraded":"2024-04-25T00:12:53.657270+0000","last_fullsized":"2024-04-25T00:12:53.657270+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:03.865061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326645+0000","last_change":"2024-04-25T00:12:52.668528+0000","last_active":"2024-04-25T00:12:54.326645+0000","last_peered":"2024-04-25T00:12:54.326645+0000","last_clean":"2024-04-25T00:12:54.326645+0000","last_became_active":"2024-04-25T00:12:52.668260+0000","last_became_peered":"2024-04-25T00:12:52.668260+0000","last_unstale":"2024-04-25T00:12:54.326645+0000","last_undegraded":"2024-04-25T00:12:54.326645+0000","last_fullsized":"2024-04-25T00:12:54.326645+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:20.491683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.670209+0000","last_change":"2024-04-25T00:12:52.666556+0000","last_active":"2024-04-25T00:12:52.670209+0000","last_peered":"2024-04-25T00:12:52.670209+0000","last_clean":"2024-04-25T00:12:52.670209+0000","last_became_active":"2024-04-25T00:12:52.666368+0000","last_became_peered":"2024-04-25T00:12:52.666368+0000","last_unstale":"2024-04-25T00:12:52.670209+0000","last_undegraded":"2024-04-25T00:12:52.670209+0000","last_fullsized":"2024-04-25T00:12:52.670209+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:25.378561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.655987+0000","last_change":"2024-04-25T00:12:52.660788+0000","last_active":"2024-04-25T00:12:53.655987+0000","last_peered":"2024-04-25T00:12:53.655987+0000","last_clean":"2024-04-25T00:12:53.655987+0000","last_became_active":"2024-04-25T00:12:52.660433+0000","last_became_peered":"2024-04-25T00:12:52.660433+0000","last_unstale":"2024-04-25T00:12:53.655987+0000","last_undegraded":"2024-04-25T00:12:53.655987+0000","last_fullsized":"2024-04-25T00:12:53.655987+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:08.409969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669448+0000","last_change":"2024-04-25T00:12:52.669101+0000","last_active":"2024-04-25T00:12:53.669448+0000","last_peered":"2024-04-25T00:12:53.669448+0000","last_clean":"2024-04-25T00:12:53.669448+0000","last_became_active":"2024-04-25T00:12:52.668678+0000","last_became_peered":"2024-04-25T00:12:52.668678+0000","last_unstale":"2024-04-25T00:12:53.669448+0000","last_undegraded":"2024-04-25T00:12:53.669448+0000","last_fullsized":"2024-04-25T00:12:53.669448+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:42.072677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.667742+0000","last_change":"2024-04-25T00:12:52.658199+0000","last_active":"2024-04-25T00:12:52.667742+0000","last_peered":"2024-04-25T00:12:52.667742+0000","last_clean":"2024-04-25T00:12:52.667742+0000","last_became_active":"2024-04-25T00:12:52.657719+0000","last_became_peered":"2024-04-25T00:12:52.657719+0000","last_unstale":"2024-04-25T00:12:52.667742+0000","last_undegraded":"2024-04-25T00:12:52.667742+0000","last_fullsized":"2024-04-25T00:12:52.667742+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:02.685857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.657085+0000","last_change":"2024-04-25T00:12:52.673521+0000","last_active":"2024-04-25T00:12:53.657085+0000","last_peered":"2024-04-25T00:12:53.657085+0000","last_clean":"2024-04-25T00:12:53.657085+0000","last_became_active":"2024-04-25T00:12:52.673287+0000","last_became_peered":"2024-04-25T00:12:52.673287+0000","last_unstale":"2024-04-25T00:12:53.657085+0000","last_undegraded":"2024-04-25T00:12:53.657085+0000","last_fullsized":"2024-04-25T00:12:53.657085+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:53.473371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.670097+0000","last_change":"2024-04-25T00:12:52.672451+0000","last_active":"2024-04-25T00:12:53.670097+0000","last_peered":"2024-04-25T00:12:53.670097+0000","last_clean":"2024-04-25T00:12:53.670097+0000","last_became_active":"2024-04-25T00:12:52.672205+0000","last_became_peered":"2024-04-25T00:12:52.672205+0000","last_unstale":"2024-04-25T00:12:53.670097+0000","last_undegraded":"2024-04-25T00:12:53.670097+0000","last_fullsized":"2024-04-25T00:12:53.670097+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:01.498756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656926+0000","last_change":"2024-04-25T00:12:52.666728+0000","last_active":"2024-04-25T00:12:53.656926+0000","last_peered":"2024-04-25T00:12:53.656926+0000","last_clean":"2024-04-25T00:12:53.656926+0000","last_became_active":"2024-04-25T00:12:52.666299+0000","last_became_peered":"2024-04-25T00:12:52.666299+0000","last_unstale":"2024-04-25T00:12:53.656926+0000","last_undegraded":"2024-04-25T00:12:53.656926+0000","last_fullsized":"2024-04-25T00:12:53.656926+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:55.805741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659476+0000","last_change":"2024-04-25T00:12:52.660419+0000","last_active":"2024-04-25T00:12:53.659476+0000","last_peered":"2024-04-25T00:12:53.659476+0000","last_clean":"2024-04-25T00:12:53.659476+0000","last_became_active":"2024-04-25T00:12:52.660088+0000","last_became_peered":"2024-04-25T00:12:52.660088+0000","last_unstale":"2024-04-25T00:12:53.659476+0000","last_undegraded":"2024-04-25T00:12:53.659476+0000","last_fullsized":"2024-04-25T00:12:53.659476+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:35.629702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659221+0000","last_change":"2024-04-25T00:12:52.668961+0000","last_active":"2024-04-25T00:12:53.659221+0000","last_peered":"2024-04-25T00:12:53.659221+0000","last_clean":"2024-04-25T00:12:53.659221+0000","last_became_active":"2024-04-25T00:12:52.668711+0000","last_became_peered":"2024-04-25T00:12:52.668711+0000","last_unstale":"2024-04-25T00:12:53.659221+0000","last_undegraded":"2024-04-25T00:12:53.659221+0000","last_fullsized":"2024-04-25T00:12:53.659221+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:39.232682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326230+0000","last_change":"2024-04-25T00:12:52.666501+0000","last_active":"2024-04-25T00:12:54.326230+0000","last_peered":"2024-04-25T00:12:54.326230+0000","last_clean":"2024-04-25T00:12:54.326230+0000","last_became_active":"2024-04-25T00:12:52.666262+0000","last_became_peered":"2024-04-25T00:12:52.666262+0000","last_unstale":"2024-04-25T00:12:54.326230+0000","last_undegraded":"2024-04-25T00:12:54.326230+0000","last_fullsized":"2024-04-25T00:12:54.326230+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:07.020427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.657103+0000","last_change":"2024-04-25T00:12:52.670022+0000","last_active":"2024-04-25T00:12:53.657103+0000","last_peered":"2024-04-25T00:12:53.657103+0000","last_clean":"2024-04-25T00:12:53.657103+0000","last_became_active":"2024-04-25T00:12:52.668266+0000","last_became_peered":"2024-04-25T00:12:52.668266+0000","last_unstale":"2024-04-25T00:12:53.657103+0000","last_undegraded":"2024-04-25T00:12:53.657103+0000","last_fullsized":"2024-04-25T00:12:53.657103+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:24.401745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.655815+0000","last_change":"2024-04-25T00:12:52.657444+0000","last_active":"2024-04-25T00:12:53.655815+0000","last_peered":"2024-04-25T00:12:53.655815+0000","last_clean":"2024-04-25T00:12:53.655815+0000","last_became_active":"2024-04-25T00:12:52.657177+0000","last_became_peered":"2024-04-25T00:12:52.657177+0000","last_unstale":"2024-04-25T00:12:53.655815+0000","last_undegraded":"2024-04-25T00:12:53.655815+0000","last_fullsized":"2024-04-25T00:12:53.655815+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:05.076681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656848+0000","last_change":"2024-04-25T00:12:52.670638+0000","last_active":"2024-04-25T00:12:53.656848+0000","last_peered":"2024-04-25T00:12:53.656848+0000","last_clean":"2024-04-25T00:12:53.656848+0000","last_became_active":"2024-04-25T00:12:52.670372+0000","last_became_peered":"2024-04-25T00:12:52.670372+0000","last_unstale":"2024-04-25T00:12:53.656848+0000","last_undegraded":"2024-04-25T00:12:53.656848+0000","last_fullsized":"2024-04-25T00:12:53.656848+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.083420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326388+0000","last_change":"2024-04-25T00:12:52.663251+0000","last_active":"2024-04-25T00:12:54.326388+0000","last_peered":"2024-04-25T00:12:54.326388+0000","last_clean":"2024-04-25T00:12:54.326388+0000","last_became_active":"2024-04-25T00:12:52.662779+0000","last_became_peered":"2024-04-25T00:12:52.662779+0000","last_unstale":"2024-04-25T00:12:54.326388+0000","last_undegraded":"2024-04-25T00:12:54.326388+0000","last_fullsized":"2024-04-25T00:12:54.326388+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:12.858553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.672014+0000","last_change":"2024-04-25T00:12:52.668345+0000","last_active":"2024-04-25T00:12:52.672014+0000","last_peered":"2024-04-25T00:12:52.672014+0000","last_clean":"2024-04-25T00:12:52.672014+0000","last_became_active":"2024-04-25T00:12:52.667886+0000","last_became_peered":"2024-04-25T00:12:52.667886+0000","last_unstale":"2024-04-25T00:12:52.672014+0000","last_undegraded":"2024-04-25T00:12:52.672014+0000","last_fullsized":"2024-04-25T00:12:52.672014+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:02.842914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659562+0000","last_change":"2024-04-25T00:12:52.657048+0000","last_active":"2024-04-25T00:12:53.659562+0000","last_peered":"2024-04-25T00:12:53.659562+0000","last_clean":"2024-04-25T00:12:53.659562+0000","last_became_active":"2024-04-25T00:12:52.656755+0000","last_became_peered":"2024-04-25T00:12:52.656755+0000","last_unstale":"2024-04-25T00:12:53.659562+0000","last_undegraded":"2024-04-25T00:12:53.659562+0000","last_fullsized":"2024-04-25T00:12:53.659562+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:54.818422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656053+0000","last_change":"2024-04-25T00:12:52.664729+0000","last_active":"2024-04-25T00:12:53.656053+0000","last_peered":"2024-04-25T00:12:53.656053+0000","last_clean":"2024-04-25T00:12:53.656053+0000","last_became_active":"2024-04-25T00:12:52.659880+0000","last_became_peered":"2024-04-25T00:12:52.659880+0000","last_unstale":"2024-04-25T00:12:53.656053+0000","last_undegraded":"2024-04-25T00:12:53.656053+0000","last_fullsized":"2024-04-25T00:12:53.656053+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:48.214697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669566+0000","last_change":"2024-04-25T00:12:52.682054+0000","last_active":"2024-04-25T00:12:53.669566+0000","last_peered":"2024-04-25T00:12:53.669566+0000","last_clean":"2024-04-25T00:12:53.669566+0000","last_became_active":"2024-04-25T00:12:52.681764+0000","last_became_peered":"2024-04-25T00:12:52.681764+0000","last_unstale":"2024-04-25T00:12:53.669566+0000","last_undegraded":"2024-04-25T00:12:53.669566+0000","last_fullsized":"2024-04-25T00:12:53.669566+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:08.409969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656061+0000","last_change":"2024-04-25T00:12:52.661360+0000","last_active":"2024-04-25T00:12:53.656061+0000","last_peered":"2024-04-25T00:12:53.656061+0000","last_clean":"2024-04-25T00:12:53.656061+0000","last_became_active":"2024-04-25T00:12:52.661087+0000","last_became_peered":"2024-04-25T00:12:52.661087+0000","last_unstale":"2024-04-25T00:12:53.656061+0000","last_undegraded":"2024-04-25T00:12:53.656061+0000","last_fullsized":"2024-04-25T00:12:53.656061+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:50.441126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.680881+0000","last_change":"2024-04-25T00:12:52.675427+0000","last_active":"2024-04-25T00:12:52.680881+0000","last_peered":"2024-04-25T00:12:52.680881+0000","last_clean":"2024-04-25T00:12:52.680881+0000","last_became_active":"2024-04-25T00:12:52.675137+0000","last_became_peered":"2024-04-25T00:12:52.675137+0000","last_unstale":"2024-04-25T00:12:52.680881+0000","last_undegraded":"2024-04-25T00:12:52.680881+0000","last_fullsized":"2024-04-25T00:12:52.680881+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:37.301188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669650+0000","last_change":"2024-04-25T00:12:52.676969+0000","last_active":"2024-04-25T00:12:53.669650+0000","last_peered":"2024-04-25T00:12:53.669650+0000","last_clean":"2024-04-25T00:12:53.669650+0000","last_became_active":"2024-04-25T00:12:52.676714+0000","last_became_peered":"2024-04-25T00:12:52.676714+0000","last_unstale":"2024-04-25T00:12:53.669650+0000","last_undegraded":"2024-04-25T00:12:53.669650+0000","last_fullsized":"2024-04-25T00:12:53.669650+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:25.196761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"48'4","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659518+0000","last_change":"2024-04-25T00:12:51.652099+0000","last_active":"2024-04-25T00:12:53.659518+0000","last_peered":"2024-04-25T00:12:53.659518+0000","last_clean":"2024-04-25T00:12:53.659518+0000","last_became_active":"2024-04-25T00:12:51.650949+0000","last_became_peered":"2024-04-25T00:12:51.650949+0000","last_unstale":"2024-04-25T00:12:53.659518+0000","last_undegraded":"2024-04-25T00:12:53.659518+0000","last_fullsized":"2024-04-25T00:12:53.659518+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:10.741559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.675741+0000","last_change":"2024-04-25T00:12:52.670482+0000","last_active":"2024-04-25T00:12:52.675741+0000","last_peered":"2024-04-25T00:12:52.675741+0000","last_clean":"2024-04-25T00:12:52.675741+0000","last_became_active":"2024-04-25T00:12:52.670152+0000","last_became_peered":"2024-04-25T00:12:52.670152+0000","last_unstale":"2024-04-25T00:12:52.675741+0000","last_undegraded":"2024-04-25T00:12:52.675741+0000","last_fullsized":"2024-04-25T00:12:52.675741+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:56.140286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.679193+0000","last_change":"2024-04-25T00:12:51.661434+0000","last_active":"2024-04-25T00:12:52.679193+0000","last_peered":"2024-04-25T00:12:52.679193+0000","last_clean":"2024-04-25T00:12:52.679193+0000","last_became_active":"2024-04-25T00:12:51.661181+0000","last_became_peered":"2024-04-25T00:12:51.661181+0000","last_unstale":"2024-04-25T00:12:52.679193+0000","last_undegraded":"2024-04-25T00:12:52.679193+0000","last_fullsized":"2024-04-25T00:12:52.679193+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:32.652596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659205+0000","last_change":"2024-04-25T00:12:52.668960+0000","last_active":"2024-04-25T00:12:53.659205+0000","last_peered":"2024-04-25T00:12:53.659205+0000","last_clean":"2024-04-25T00:12:53.659205+0000","last_became_active":"2024-04-25T00:12:52.668615+0000","last_became_peered":"2024-04-25T00:12:52.668615+0000","last_unstale":"2024-04-25T00:12:53.659205+0000","last_undegraded":"2024-04-25T00:12:53.659205+0000","last_fullsized":"2024-04-25T00:12:53.659205+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:23.249577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.674828+0000","last_change":"2024-04-25T00:12:51.648958+0000","last_active":"2024-04-25T00:12:52.674828+0000","last_peered":"2024-04-25T00:12:52.674828+0000","last_clean":"2024-04-25T00:12:52.674828+0000","last_became_active":"2024-04-25T00:12:51.648688+0000","last_became_peered":"2024-04-25T00:12:51.648688+0000","last_unstale":"2024-04-25T00:12:52.674828+0000","last_undegraded":"2024-04-25T00:12:52.674828+0000","last_fullsized":"2024-04-25T00:12:52.674828+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:53:38.334610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326214+0000","last_change":"2024-04-25T00:12:52.660917+0000","last_active":"2024-04-25T00:12:54.326214+0000","last_peered":"2024-04-25T00:12:54.326214+0000","last_clean":"2024-04-25T00:12:54.326214+0000","last_became_active":"2024-04-25T00:12:52.660632+0000","last_became_peered":"2024-04-25T00:12:52.660632+0000","last_unstale":"2024-04-25T00:12:54.326214+0000","last_undegraded":"2024-04-25T00:12:54.326214+0000","last_fullsized":"2024-04-25T00:12:54.326214+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:53.473371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.685391+0000","last_change":"2024-04-25T00:12:51.654965+0000","last_active":"2024-04-25T00:12:52.685391+0000","last_peered":"2024-04-25T00:12:52.685391+0000","last_clean":"2024-04-25T00:12:52.685391+0000","last_became_active":"2024-04-25T00:12:51.654724+0000","last_became_peered":"2024-04-25T00:12:51.654724+0000","last_unstale":"2024-04-25T00:12:52.685391+0000","last_undegraded":"2024-04-25T00:12:52.685391+0000","last_fullsized":"2024-04-25T00:12:52.685391+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:16.308294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.676239+0000","last_change":"2024-04-25T00:12:52.674588+0000","last_active":"2024-04-25T00:12:52.676239+0000","last_peered":"2024-04-25T00:12:52.676239+0000","last_clean":"2024-04-25T00:12:52.676239+0000","last_became_active":"2024-04-25T00:12:52.674333+0000","last_became_peered":"2024-04-25T00:12:52.674333+0000","last_unstale":"2024-04-25T00:12:52.676239+0000","last_undegraded":"2024-04-25T00:12:52.676239+0000","last_fullsized":"2024-04-25T00:12:52.676239+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:02.685857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656227+0000","last_change":"2024-04-25T00:12:51.659654+0000","last_active":"2024-04-25T00:12:53.656227+0000","last_peered":"2024-04-25T00:12:53.656227+0000","last_clean":"2024-04-25T00:12:53.656227+0000","last_became_active":"2024-04-25T00:12:51.659433+0000","last_became_peered":"2024-04-25T00:12:51.659433+0000","last_unstale":"2024-04-25T00:12:53.656227+0000","last_undegraded":"2024-04-25T00:12:53.656227+0000","last_fullsized":"2024-04-25T00:12:53.656227+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:01.679108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669881+0000","last_change":"2024-04-25T00:12:52.680204+0000","last_active":"2024-04-25T00:12:53.669881+0000","last_peered":"2024-04-25T00:12:53.669881+0000","last_clean":"2024-04-25T00:12:53.669881+0000","last_became_active":"2024-04-25T00:12:52.679933+0000","last_became_peered":"2024-04-25T00:12:52.679933+0000","last_unstale":"2024-04-25T00:12:53.669881+0000","last_undegraded":"2024-04-25T00:12:53.669881+0000","last_fullsized":"2024-04-25T00:12:53.669881+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:50.441126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.677893+0000","last_change":"2024-04-25T00:12:51.651147+0000","last_active":"2024-04-25T00:12:52.677893+0000","last_peered":"2024-04-25T00:12:52.677893+0000","last_clean":"2024-04-25T00:12:52.677893+0000","last_became_active":"2024-04-25T00:12:51.650857+0000","last_became_peered":"2024-04-25T00:12:51.650857+0000","last_unstale":"2024-04-25T00:12:52.677893+0000","last_undegraded":"2024-04-25T00:12:52.677893+0000","last_fullsized":"2024-04-25T00:12:52.677893+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:32.652596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326568+0000","last_change":"2024-04-25T00:12:52.663050+0000","last_active":"2024-04-25T00:12:54.326568+0000","last_peered":"2024-04-25T00:12:54.326568+0000","last_clean":"2024-04-25T00:12:54.326568+0000","last_became_active":"2024-04-25T00:12:52.662487+0000","last_became_peered":"2024-04-25T00:12:52.662487+0000","last_unstale":"2024-04-25T00:12:54.326568+0000","last_undegraded":"2024-04-25T00:12:54.326568+0000","last_fullsized":"2024-04-25T00:12:54.326568+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:16.048766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659080+0000","last_change":"2024-04-25T00:12:51.649636+0000","last_active":"2024-04-25T00:12:53.659080+0000","last_peered":"2024-04-25T00:12:53.659080+0000","last_clean":"2024-04-25T00:12:53.659080+0000","last_became_active":"2024-04-25T00:12:51.649332+0000","last_became_peered":"2024-04-25T00:12:51.649332+0000","last_unstale":"2024-04-25T00:12:53.659080+0000","last_undegraded":"2024-04-25T00:12:53.659080+0000","last_fullsized":"2024-04-25T00:12:53.659080+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:56.763479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.670025+0000","last_change":"2024-04-25T00:12:52.679726+0000","last_active":"2024-04-25T00:12:53.670025+0000","last_peered":"2024-04-25T00:12:53.670025+0000","last_clean":"2024-04-25T00:12:53.670025+0000","last_became_active":"2024-04-25T00:12:52.679456+0000","last_became_peered":"2024-04-25T00:12:52.679456+0000","last_unstale":"2024-04-25T00:12:53.670025+0000","last_undegraded":"2024-04-25T00:12:53.670025+0000","last_fullsized":"2024-04-25T00:12:53.670025+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:37.141163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.655801+0000","last_change":"2024-04-25T00:12:51.653108+0000","last_active":"2024-04-25T00:12:53.655801+0000","last_peered":"2024-04-25T00:12:53.655801+0000","last_clean":"2024-04-25T00:12:53.655801+0000","last_became_active":"2024-04-25T00:12:51.652852+0000","last_became_peered":"2024-04-25T00:12:51.652852+0000","last_unstale":"2024-04-25T00:12:53.655801+0000","last_undegraded":"2024-04-25T00:12:53.655801+0000","last_fullsized":"2024-04-25T00:12:53.655801+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:33.563695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.681038+0000","last_change":"2024-04-25T00:12:52.678919+0000","last_active":"2024-04-25T00:12:52.681038+0000","last_peered":"2024-04-25T00:12:52.681038+0000","last_clean":"2024-04-25T00:12:52.681038+0000","last_became_active":"2024-04-25T00:12:52.678611+0000","last_became_peered":"2024-04-25T00:12:52.678611+0000","last_unstale":"2024-04-25T00:12:52.681038+0000","last_undegraded":"2024-04-25T00:12:52.681038+0000","last_fullsized":"2024-04-25T00:12:52.681038+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:34.570444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.670056+0000","last_change":"2024-04-25T00:12:51.647688+0000","last_active":"2024-04-25T00:12:53.670056+0000","last_peered":"2024-04-25T00:12:53.670056+0000","last_clean":"2024-04-25T00:12:53.670056+0000","last_became_active":"2024-04-25T00:12:51.647381+0000","last_became_peered":"2024-04-25T00:12:51.647381+0000","last_unstale":"2024-04-25T00:12:53.670056+0000","last_undegraded":"2024-04-25T00:12:53.670056+0000","last_fullsized":"2024-04-25T00:12:53.670056+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:53.811673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.675729+0000","last_change":"2024-04-25T00:12:52.668194+0000","last_active":"2024-04-25T00:12:52.675729+0000","last_peered":"2024-04-25T00:12:52.675729+0000","last_clean":"2024-04-25T00:12:52.675729+0000","last_became_active":"2024-04-25T00:12:52.666724+0000","last_became_peered":"2024-04-25T00:12:52.666724+0000","last_unstale":"2024-04-25T00:12:52.675729+0000","last_undegraded":"2024-04-25T00:12:52.675729+0000","last_fullsized":"2024-04-25T00:12:52.675729+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:28.819655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.654798+0000","last_change":"2024-04-25T00:12:51.660958+0000","last_active":"2024-04-25T00:12:52.654798+0000","last_peered":"2024-04-25T00:12:52.654798+0000","last_clean":"2024-04-25T00:12:52.654798+0000","last_became_active":"2024-04-25T00:12:51.660706+0000","last_became_peered":"2024-04-25T00:12:51.660706+0000","last_unstale":"2024-04-25T00:12:52.654798+0000","last_undegraded":"2024-04-25T00:12:52.654798+0000","last_fullsized":"2024-04-25T00:12:52.654798+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:56.763479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656040+0000","last_change":"2024-04-25T00:12:52.662098+0000","last_active":"2024-04-25T00:12:53.656040+0000","last_peered":"2024-04-25T00:12:53.656040+0000","last_clean":"2024-04-25T00:12:53.656040+0000","last_became_active":"2024-04-25T00:12:52.661655+0000","last_became_peered":"2024-04-25T00:12:52.661655+0000","last_unstale":"2024-04-25T00:12:53.656040+0000","last_undegraded":"2024-04-25T00:12:53.656040+0000","last_fullsized":"2024-04-25T00:12:53.656040+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:25.196761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669950+0000","last_change":"2024-04-25T00:12:51.654881+0000","last_active":"2024-04-25T00:12:53.669950+0000","last_peered":"2024-04-25T00:12:53.669950+0000","last_clean":"2024-04-25T00:12:53.669950+0000","last_became_active":"2024-04-25T00:12:51.654585+0000","last_became_peered":"2024-04-25T00:12:51.654585+0000","last_unstale":"2024-04-25T00:12:53.669950+0000","last_undegraded":"2024-04-25T00:12:53.669950+0000","last_fullsized":"2024-04-25T00:12:53.669950+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:34.622954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.675561+0000","last_change":"2024-04-25T00:12:52.671369+0000","last_active":"2024-04-25T00:12:52.675561+0000","last_peered":"2024-04-25T00:12:52.675561+0000","last_clean":"2024-04-25T00:12:52.675561+0000","last_became_active":"2024-04-25T00:12:52.667621+0000","last_became_peered":"2024-04-25T00:12:52.667621+0000","last_unstale":"2024-04-25T00:12:52.675561+0000","last_undegraded":"2024-04-25T00:12:52.675561+0000","last_fullsized":"2024-04-25T00:12:52.675561+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:53:39.341359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326624+0000","last_change":"2024-04-25T00:12:51.649041+0000","last_active":"2024-04-25T00:12:54.326624+0000","last_peered":"2024-04-25T00:12:54.326624+0000","last_clean":"2024-04-25T00:12:54.326624+0000","last_became_active":"2024-04-25T00:12:51.648787+0000","last_became_peered":"2024-04-25T00:12:51.648787+0000","last_unstale":"2024-04-25T00:12:54.326624+0000","last_undegraded":"2024-04-25T00:12:54.326624+0000","last_fullsized":"2024-04-25T00:12:54.326624+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:41.065928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.676017+0000","last_change":"2024-04-25T00:12:52.670705+0000","last_active":"2024-04-25T00:12:52.676017+0000","last_peered":"2024-04-25T00:12:52.676017+0000","last_clean":"2024-04-25T00:12:52.676017+0000","last_became_active":"2024-04-25T00:12:52.670390+0000","last_became_peered":"2024-04-25T00:12:52.670390+0000","last_unstale":"2024-04-25T00:12:52.676017+0000","last_undegraded":"2024-04-25T00:12:52.676017+0000","last_fullsized":"2024-04-25T00:12:52.676017+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:01.593192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656226+0000","last_change":"2024-04-25T00:12:51.659075+0000","last_active":"2024-04-25T00:12:53.656226+0000","last_peered":"2024-04-25T00:12:53.656226+0000","last_clean":"2024-04-25T00:12:53.656226+0000","last_became_active":"2024-04-25T00:12:51.656316+0000","last_became_peered":"2024-04-25T00:12:51.656316+0000","last_unstale":"2024-04-25T00:12:53.656226+0000","last_undegraded":"2024-04-25T00:12:53.656226+0000","last_fullsized":"2024-04-25T00:12:53.656226+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:28.666067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.657062+0000","last_change":"2024-04-25T00:12:52.666531+0000","last_active":"2024-04-25T00:12:53.657062+0000","last_peered":"2024-04-25T00:12:53.657062+0000","last_clean":"2024-04-25T00:12:53.657062+0000","last_became_active":"2024-04-25T00:12:52.665946+0000","last_became_peered":"2024-04-25T00:12:52.665946+0000","last_unstale":"2024-04-25T00:12:53.657062+0000","last_undegraded":"2024-04-25T00:12:53.657062+0000","last_fullsized":"2024-04-25T00:12:53.657062+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:39.182004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659247+0000","last_change":"2024-04-25T00:12:51.656091+0000","last_active":"2024-04-25T00:12:53.659247+0000","last_peered":"2024-04-25T00:12:53.659247+0000","last_clean":"2024-04-25T00:12:53.659247+0000","last_became_active":"2024-04-25T00:12:51.655836+0000","last_became_peered":"2024-04-25T00:12:51.655836+0000","last_unstale":"2024-04-25T00:12:53.659247+0000","last_undegraded":"2024-04-25T00:12:53.659247+0000","last_fullsized":"2024-04-25T00:12:53.659247+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:01.679108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326428+0000","last_change":"2024-04-25T00:12:52.669389+0000","last_active":"2024-04-25T00:12:54.326428+0000","last_peered":"2024-04-25T00:12:54.326428+0000","last_clean":"2024-04-25T00:12:54.326428+0000","last_became_active":"2024-04-25T00:12:52.669065+0000","last_became_peered":"2024-04-25T00:12:52.669065+0000","last_unstale":"2024-04-25T00:12:54.326428+0000","last_undegraded":"2024-04-25T00:12:54.326428+0000","last_fullsized":"2024-04-25T00:12:54.326428+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:54.804868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669434+0000","last_change":"2024-04-25T00:12:51.648853+0000","last_active":"2024-04-25T00:12:53.669434+0000","last_peered":"2024-04-25T00:12:53.669434+0000","last_clean":"2024-04-25T00:12:53.669434+0000","last_became_active":"2024-04-25T00:12:51.648568+0000","last_became_peered":"2024-04-25T00:12:51.648568+0000","last_unstale":"2024-04-25T00:12:53.669434+0000","last_undegraded":"2024-04-25T00:12:53.669434+0000","last_fullsized":"2024-04-25T00:12:53.669434+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:16.308294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656730+0000","last_change":"2024-04-25T00:12:52.672115+0000","last_active":"2024-04-25T00:12:53.656730+0000","last_peered":"2024-04-25T00:12:53.656730+0000","last_clean":"2024-04-25T00:12:53.656730+0000","last_became_active":"2024-04-25T00:12:52.671673+0000","last_became_peered":"2024-04-25T00:12:52.671673+0000","last_unstale":"2024-04-25T00:12:53.656730+0000","last_undegraded":"2024-04-25T00:12:53.656730+0000","last_fullsized":"2024-04-25T00:12:53.656730+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:37.141163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":22,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326450+0000","last_change":"2024-04-25T00:12:51.656996+0000","last_active":"2024-04-25T00:12:54.326450+0000","last_peered":"2024-04-25T00:12:54.326450+0000","last_clean":"2024-04-25T00:12:54.326450+0000","last_became_active":"2024-04-25T00:12:51.656554+0000","last_became_peered":"2024-04-25T00:12:51.656554+0000","last_unstale":"2024-04-25T00:12:54.326450+0000","last_undegraded":"2024-04-25T00:12:54.326450+0000","last_fullsized":"2024-04-25T00:12:54.326450+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:36.294440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326499+0000","last_change":"2024-04-25T00:12:52.674490+0000","last_active":"2024-04-25T00:12:54.326499+0000","last_peered":"2024-04-25T00:12:54.326499+0000","last_clean":"2024-04-25T00:12:54.326499+0000","last_became_active":"2024-04-25T00:12:52.674198+0000","last_became_peered":"2024-04-25T00:12:52.674198+0000","last_unstale":"2024-04-25T00:12:54.326499+0000","last_undegraded":"2024-04-25T00:12:54.326499+0000","last_fullsized":"2024-04-25T00:12:54.326499+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:36.631117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656954+0000","last_change":"2024-04-25T00:12:51.661693+0000","last_active":"2024-04-25T00:12:53.656954+0000","last_peered":"2024-04-25T00:12:53.656954+0000","last_clean":"2024-04-25T00:12:53.656954+0000","last_became_active":"2024-04-25T00:12:51.661409+0000","last_became_peered":"2024-04-25T00:12:51.661409+0000","last_unstale":"2024-04-25T00:12:53.656954+0000","last_undegraded":"2024-04-25T00:12:53.656954+0000","last_fullsized":"2024-04-25T00:12:53.656954+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:04.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.674836+0000","last_change":"2024-04-25T00:12:52.671048+0000","last_active":"2024-04-25T00:12:52.674836+0000","last_peered":"2024-04-25T00:12:52.674836+0000","last_clean":"2024-04-25T00:12:52.674836+0000","last_became_active":"2024-04-25T00:12:52.669534+0000","last_became_peered":"2024-04-25T00:12:52.669534+0000","last_unstale":"2024-04-25T00:12:52.674836+0000","last_undegraded":"2024-04-25T00:12:52.674836+0000","last_fullsized":"2024-04-25T00:12:52.674836+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:39.232682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.657408+0000","last_change":"2024-04-25T00:12:51.652635+0000","last_active":"2024-04-25T00:12:53.657408+0000","last_peered":"2024-04-25T00:12:53.657408+0000","last_clean":"2024-04-25T00:12:53.657408+0000","last_became_active":"2024-04-25T00:12:51.652328+0000","last_became_peered":"2024-04-25T00:12:51.652328+0000","last_unstale":"2024-04-25T00:12:53.657408+0000","last_undegraded":"2024-04-25T00:12:53.657408+0000","last_fullsized":"2024-04-25T00:12:53.657408+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:24.622672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.680601+0000","last_change":"2024-04-25T00:12:52.678059+0000","last_active":"2024-04-25T00:12:52.680601+0000","last_peered":"2024-04-25T00:12:52.680601+0000","last_clean":"2024-04-25T00:12:52.680601+0000","last_became_active":"2024-04-25T00:12:52.677557+0000","last_became_peered":"2024-04-25T00:12:52.677557+0000","last_unstale":"2024-04-25T00:12:52.680601+0000","last_undegraded":"2024-04-25T00:12:52.680601+0000","last_fullsized":"2024-04-25T00:12:52.680601+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:00.773724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659437+0000","last_change":"2024-04-25T00:12:51.651367+0000","last_active":"2024-04-25T00:12:53.659437+0000","last_peered":"2024-04-25T00:12:53.659437+0000","last_clean":"2024-04-25T00:12:53.659437+0000","last_became_active":"2024-04-25T00:12:51.651107+0000","last_became_peered":"2024-04-25T00:12:51.651107+0000","last_unstale":"2024-04-25T00:12:53.659437+0000","last_undegraded":"2024-04-25T00:12:53.659437+0000","last_fullsized":"2024-04-25T00:12:53.659437+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:53:38.334610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.669625+0000","last_change":"2024-04-25T00:12:52.664917+0000","last_active":"2024-04-25T00:12:52.669625+0000","last_peered":"2024-04-25T00:12:52.669625+0000","last_clean":"2024-04-25T00:12:52.669625+0000","last_became_active":"2024-04-25T00:12:52.663589+0000","last_became_peered":"2024-04-25T00:12:52.663589+0000","last_unstale":"2024-04-25T00:12:52.669625+0000","last_undegraded":"2024-04-25T00:12:52.669625+0000","last_fullsized":"2024-04-25T00:12:52.669625+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:29.672816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659577+0000","last_change":"2024-04-25T00:12:51.654187+0000","last_active":"2024-04-25T00:12:53.659577+0000","last_peered":"2024-04-25T00:12:53.659577+0000","last_clean":"2024-04-25T00:12:53.659577+0000","last_became_active":"2024-04-25T00:12:51.653868+0000","last_became_peered":"2024-04-25T00:12:51.653868+0000","last_unstale":"2024-04-25T00:12:53.659577+0000","last_undegraded":"2024-04-25T00:12:53.659577+0000","last_fullsized":"2024-04-25T00:12:53.659577+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:39.547945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.679536+0000","last_change":"2024-04-25T00:12:52.670746+0000","last_active":"2024-04-25T00:12:52.679536+0000","last_peered":"2024-04-25T00:12:52.679536+0000","last_clean":"2024-04-25T00:12:52.679536+0000","last_became_active":"2024-04-25T00:12:52.670432+0000","last_became_peered":"2024-04-25T00:12:52.670432+0000","last_unstale":"2024-04-25T00:12:52.679536+0000","last_undegraded":"2024-04-25T00:12:52.679536+0000","last_fullsized":"2024-04-25T00:12:52.679536+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:28.940781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.678967+0000","last_change":"2024-04-25T00:12:51.650824+0000","last_active":"2024-04-25T00:12:52.678967+0000","last_peered":"2024-04-25T00:12:52.678967+0000","last_clean":"2024-04-25T00:12:52.678967+0000","last_became_active":"2024-04-25T00:12:51.650486+0000","last_became_peered":"2024-04-25T00:12:51.650486+0000","last_unstale":"2024-04-25T00:12:52.678967+0000","last_undegraded":"2024-04-25T00:12:52.678967+0000","last_fullsized":"2024-04-25T00:12:52.678967+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:39.547945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659496+0000","last_change":"2024-04-25T00:12:52.664793+0000","last_active":"2024-04-25T00:12:53.659496+0000","last_peered":"2024-04-25T00:12:53.659496+0000","last_clean":"2024-04-25T00:12:53.659496+0000","last_became_active":"2024-04-25T00:12:52.664225+0000","last_became_peered":"2024-04-25T00:12:52.664225+0000","last_unstale":"2024-04-25T00:12:53.659496+0000","last_undegraded":"2024-04-25T00:12:53.659496+0000","last_fullsized":"2024-04-25T00:12:53.659496+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:47.181480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326104+0000","last_change":"2024-04-25T00:12:51.655623+0000","last_active":"2024-04-25T00:12:54.326104+0000","last_peered":"2024-04-25T00:12:54.326104+0000","last_clean":"2024-04-25T00:12:54.326104+0000","last_became_active":"2024-04-25T00:12:51.655310+0000","last_became_peered":"2024-04-25T00:12:51.655310+0000","last_unstale":"2024-04-25T00:12:54.326104+0000","last_undegraded":"2024-04-25T00:12:54.326104+0000","last_fullsized":"2024-04-25T00:12:54.326104+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:23.722877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656903+0000","last_change":"2024-04-25T00:12:52.660660+0000","last_active":"2024-04-25T00:12:53.656903+0000","last_peered":"2024-04-25T00:12:53.656903+0000","last_clean":"2024-04-25T00:12:53.656903+0000","last_became_active":"2024-04-25T00:12:52.660296+0000","last_became_peered":"2024-04-25T00:12:52.660296+0000","last_unstale":"2024-04-25T00:12:53.656903+0000","last_undegraded":"2024-04-25T00:12:53.656903+0000","last_fullsized":"2024-04-25T00:12:53.656903+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:39.543364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656150+0000","last_change":"2024-04-25T00:12:51.658951+0000","last_active":"2024-04-25T00:12:53.656150+0000","last_peered":"2024-04-25T00:12:53.656150+0000","last_clean":"2024-04-25T00:12:53.656150+0000","last_became_active":"2024-04-25T00:12:51.658654+0000","last_became_peered":"2024-04-25T00:12:51.658654+0000","last_unstale":"2024-04-25T00:12:53.656150+0000","last_undegraded":"2024-04-25T00:12:53.656150+0000","last_fullsized":"2024-04-25T00:12:53.656150+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:24.371812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.673870+0000","last_change":"2024-04-25T00:12:52.669989+0000","last_active":"2024-04-25T00:12:52.673870+0000","last_peered":"2024-04-25T00:12:52.673870+0000","last_clean":"2024-04-25T00:12:52.673870+0000","last_became_active":"2024-04-25T00:12:52.669707+0000","last_became_peered":"2024-04-25T00:12:52.669707+0000","last_unstale":"2024-04-25T00:12:52.673870+0000","last_undegraded":"2024-04-25T00:12:52.673870+0000","last_fullsized":"2024-04-25T00:12:52.673870+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:23.249577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656441+0000","last_change":"2024-04-25T00:12:51.652957+0000","last_active":"2024-04-25T00:12:53.656441+0000","last_peered":"2024-04-25T00:12:53.656441+0000","last_clean":"2024-04-25T00:12:53.656441+0000","last_became_active":"2024-04-25T00:12:51.652648+0000","last_became_peered":"2024-04-25T00:12:51.652648+0000","last_unstale":"2024-04-25T00:12:53.656441+0000","last_undegraded":"2024-04-25T00:12:53.656441+0000","last_fullsized":"2024-04-25T00:12:53.656441+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:04.694125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.680902+0000","last_change":"2024-04-25T00:12:52.675957+0000","last_active":"2024-04-25T00:12:52.680902+0000","last_peered":"2024-04-25T00:12:52.680902+0000","last_clean":"2024-04-25T00:12:52.680902+0000","last_became_active":"2024-04-25T00:12:52.675456+0000","last_became_peered":"2024-04-25T00:12:52.675456+0000","last_unstale":"2024-04-25T00:12:52.680902+0000","last_undegraded":"2024-04-25T00:12:52.680902+0000","last_fullsized":"2024-04-25T00:12:52.680902+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:24.729625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659638+0000","last_change":"2024-04-25T00:12:51.656066+0000","last_active":"2024-04-25T00:12:53.659638+0000","last_peered":"2024-04-25T00:12:53.659638+0000","last_clean":"2024-04-25T00:12:53.659638+0000","last_became_active":"2024-04-25T00:12:51.655792+0000","last_became_peered":"2024-04-25T00:12:51.655792+0000","last_unstale":"2024-04-25T00:12:53.659638+0000","last_undegraded":"2024-04-25T00:12:53.659638+0000","last_fullsized":"2024-04-25T00:12:53.659638+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:32.652596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.669255+0000","last_change":"2024-04-25T00:12:52.658546+0000","last_active":"2024-04-25T00:12:52.669255+0000","last_peered":"2024-04-25T00:12:52.669255+0000","last_clean":"2024-04-25T00:12:52.669255+0000","last_became_active":"2024-04-25T00:12:52.658256+0000","last_became_peered":"2024-04-25T00:12:52.658256+0000","last_unstale":"2024-04-25T00:12:52.669255+0000","last_undegraded":"2024-04-25T00:12:52.669255+0000","last_fullsized":"2024-04-25T00:12:52.669255+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:00.773724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.670034+0000","last_change":"2024-04-25T00:12:51.650127+0000","last_active":"2024-04-25T00:12:53.670034+0000","last_peered":"2024-04-25T00:12:53.670034+0000","last_clean":"2024-04-25T00:12:53.670034+0000","last_became_active":"2024-04-25T00:12:51.649817+0000","last_became_peered":"2024-04-25T00:12:51.649817+0000","last_unstale":"2024-04-25T00:12:53.670034+0000","last_undegraded":"2024-04-25T00:12:53.670034+0000","last_fullsized":"2024-04-25T00:12:53.670034+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:46.174731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326265+0000","last_change":"2024-04-25T00:12:52.669622+0000","last_active":"2024-04-25T00:12:54.326265+0000","last_peered":"2024-04-25T00:12:54.326265+0000","last_clean":"2024-04-25T00:12:54.326265+0000","last_became_active":"2024-04-25T00:12:52.669327+0000","last_became_peered":"2024-04-25T00:12:52.669327+0000","last_unstale":"2024-04-25T00:12:54.326265+0000","last_undegraded":"2024-04-25T00:12:54.326265+0000","last_fullsized":"2024-04-25T00:12:54.326265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:10.803393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656460+0000","last_change":"2024-04-25T00:12:51.652620+0000","last_active":"2024-04-25T00:12:53.656460+0000","last_peered":"2024-04-25T00:12:53.656460+0000","last_clean":"2024-04-25T00:12:53.656460+0000","last_became_active":"2024-04-25T00:12:51.652283+0000","last_became_peered":"2024-04-25T00:12:51.652283+0000","last_unstale":"2024-04-25T00:12:53.656460+0000","last_undegraded":"2024-04-25T00:12:53.656460+0000","last_fullsized":"2024-04-25T00:12:53.656460+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:59.766975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656402+0000","last_change":"2024-04-25T00:12:52.668754+0000","last_active":"2024-04-25T00:12:53.656402+0000","last_peered":"2024-04-25T00:12:53.656402+0000","last_clean":"2024-04-25T00:12:53.656402+0000","last_became_active":"2024-04-25T00:12:52.668453+0000","last_became_peered":"2024-04-25T00:12:52.668453+0000","last_unstale":"2024-04-25T00:12:53.656402+0000","last_undegraded":"2024-04-25T00:12:53.656402+0000","last_fullsized":"2024-04-25T00:12:53.656402+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:12.909892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326340+0000","last_change":"2024-04-25T00:12:51.656484+0000","last_active":"2024-04-25T00:12:54.326340+0000","last_peered":"2024-04-25T00:12:54.326340+0000","last_clean":"2024-04-25T00:12:54.326340+0000","last_became_active":"2024-04-25T00:12:51.656076+0000","last_became_peered":"2024-04-25T00:12:51.656076+0000","last_unstale":"2024-04-25T00:12:54.326340+0000","last_undegraded":"2024-04-25T00:12:54.326340+0000","last_fullsized":"2024-04-25T00:12:54.326340+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:27.934032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.675761+0000","last_change":"2024-04-25T00:12:52.669713+0000","last_active":"2024-04-25T00:12:52.675761+0000","last_peered":"2024-04-25T00:12:52.675761+0000","last_clean":"2024-04-25T00:12:52.675761+0000","last_became_active":"2024-04-25T00:12:52.667872+0000","last_became_peered":"2024-04-25T00:12:52.667872+0000","last_unstale":"2024-04-25T00:12:52.675761+0000","last_undegraded":"2024-04-25T00:12:52.675761+0000","last_fullsized":"2024-04-25T00:12:52.675761+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:16.703927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326208+0000","last_change":"2024-04-25T00:12:51.658109+0000","last_active":"2024-04-25T00:12:54.326208+0000","last_peered":"2024-04-25T00:12:54.326208+0000","last_clean":"2024-04-25T00:12:54.326208+0000","last_became_active":"2024-04-25T00:12:51.656811+0000","last_became_peered":"2024-04-25T00:12:51.656811+0000","last_unstale":"2024-04-25T00:12:54.326208+0000","last_undegraded":"2024-04-25T00:12:54.326208+0000","last_fullsized":"2024-04-25T00:12:54.326208+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:04.069932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656828+0000","last_change":"2024-04-25T00:12:52.660936+0000","last_active":"2024-04-25T00:12:53.656828+0000","last_peered":"2024-04-25T00:12:53.656828+0000","last_clean":"2024-04-25T00:12:53.656828+0000","last_became_active":"2024-04-25T00:12:52.660605+0000","last_became_peered":"2024-04-25T00:12:52.660605+0000","last_unstale":"2024-04-25T00:12:53.656828+0000","last_undegraded":"2024-04-25T00:12:53.656828+0000","last_fullsized":"2024-04-25T00:12:53.656828+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:22.055408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.653254+0000","last_change":"2024-04-25T00:12:51.660978+0000","last_active":"2024-04-25T00:12:52.653254+0000","last_peered":"2024-04-25T00:12:52.653254+0000","last_clean":"2024-04-25T00:12:52.653254+0000","last_became_active":"2024-04-25T00:12:51.660727+0000","last_became_peered":"2024-04-25T00:12:51.660727+0000","last_unstale":"2024-04-25T00:12:52.653254+0000","last_undegraded":"2024-04-25T00:12:52.653254+0000","last_fullsized":"2024-04-25T00:12:52.653254+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:10.741559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326730+0000","last_change":"2024-04-25T00:12:52.664358+0000","last_active":"2024-04-25T00:12:54.326730+0000","last_peered":"2024-04-25T00:12:54.326730+0000","last_clean":"2024-04-25T00:12:54.326730+0000","last_became_active":"2024-04-25T00:12:52.664021+0000","last_became_peered":"2024-04-25T00:12:52.664021+0000","last_unstale":"2024-04-25T00:12:54.326730+0000","last_undegraded":"2024-04-25T00:12:54.326730+0000","last_fullsized":"2024-04-25T00:12:54.326730+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:53.240194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.657675+0000","last_change":"2024-04-25T00:12:51.651914+0000","last_active":"2024-04-25T00:12:52.657675+0000","last_peered":"2024-04-25T00:12:52.657675+0000","last_clean":"2024-04-25T00:12:52.657675+0000","last_became_active":"2024-04-25T00:12:51.651689+0000","last_became_peered":"2024-04-25T00:12:51.651689+0000","last_unstale":"2024-04-25T00:12:52.657675+0000","last_undegraded":"2024-04-25T00:12:52.657675+0000","last_fullsized":"2024-04-25T00:12:52.657675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:00.247417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659307+0000","last_change":"2024-04-25T00:12:52.671225+0000","last_active":"2024-04-25T00:12:53.659307+0000","last_peered":"2024-04-25T00:12:53.659307+0000","last_clean":"2024-04-25T00:12:53.659307+0000","last_became_active":"2024-04-25T00:12:52.670874+0000","last_became_peered":"2024-04-25T00:12:52.670874+0000","last_unstale":"2024-04-25T00:12:53.659307+0000","last_undegraded":"2024-04-25T00:12:53.659307+0000","last_fullsized":"2024-04-25T00:12:53.659307+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:57.466333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.657020+0000","last_change":"2024-04-25T00:12:51.656724+0000","last_active":"2024-04-25T00:12:53.657020+0000","last_peered":"2024-04-25T00:12:53.657020+0000","last_clean":"2024-04-25T00:12:53.657020+0000","last_became_active":"2024-04-25T00:12:51.656419+0000","last_became_peered":"2024-04-25T00:12:51.656419+0000","last_unstale":"2024-04-25T00:12:53.657020+0000","last_undegraded":"2024-04-25T00:12:53.657020+0000","last_fullsized":"2024-04-25T00:12:53.657020+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:09.720043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":222,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669244+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:12:53.669244+0000","last_peered":"2024-04-25T00:12:53.669244+0000","last_clean":"2024-04-25T00:12:53.669244+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:12:53.669244+0000","last_undegraded":"2024-04-25T00:12:53.669244+0000","last_fullsized":"2024-04-25T00:12:53.669244+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1183744,"data_stored":848040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":251508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":250856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":251438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":252490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":259160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":251016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1183744,"data_stored":847972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:12:56.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:12:56.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.897+0000 7f2355c00640 1 Processor -- start 2024-04-25T00:12:56.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.897+0000 7f2355c00640 1 -- start start 2024-04-25T00:12:56.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.897+0000 7f2355c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f23500d1930 0x7f23500ced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.901+0000 7f2355c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2350161cb0 con 0x7f23500b0d20 2024-04-25T00:12:56.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.901+0000 7f2355c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23500d0970 con 0x7f23501618d0 2024-04-25T00:12:56.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.901+0000 7f2355c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f23500d11d0 con 0x7f23500d1930 2024-04-25T00:12:56.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.901+0000 7f234f400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f23500b0d20 0x7f23500cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42254/0 (socket says 172.21.15.83:42254) 2024-04-25T00:12:56.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.901+0000 7f234f400640 1 -- 172.21.15.83:0/3101548768 learned_addr learned my addr 172.21.15.83:0/3101548768 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:56.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.901+0000 7f2346000640 1 --2- 172.21.15.83:0/3101548768 >> v2:172.21.15.86:3300/0 conn(0x7f23500d1930 0x7f23500ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.901+0000 7f234e000640 1 -- 172.21.15.83:0/3101548768 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2871683409 0 0) 0x7f2350161cb0 con 0x7f23500b0d20 2024-04-25T00:12:56.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.901+0000 7f234e000640 1 -- 172.21.15.83:0/3101548768 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2334003530 con 0x7f23500b0d20 2024-04-25T00:12:56.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.901+0000 7f2346000640 1 -- 172.21.15.83:0/3101548768 >> v1:172.21.15.86:6789/0 conn(0x7f23501618d0 legacy=0x7f23500ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.901+0000 7f234e000640 1 -- 172.21.15.83:0/3101548768 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3372062742 0 0) 0x7f2334003530 con 0x7f23500b0d20 2024-04-25T00:12:56.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.901+0000 7f2346000640 1 -- 172.21.15.83:0/3101548768 >> v1:172.21.15.83:6789/0 conn(0x7f23500b0d20 legacy=0x7f23500cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.901+0000 7f2346000640 1 -- 172.21.15.83:0/3101548768 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23500d0c10 con 0x7f23500d1930 2024-04-25T00:12:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.901+0000 7f2346000640 1 --2- 172.21.15.83:0/3101548768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23500d1930 0x7f23500ced00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f23380097c0 tx=0x7f233801cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=412183c87a20b937 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f234e000640 1 -- 172.21.15.83:0/3101548768 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23380b0070 con 0x7f23500d1930 2024-04-25T00:12:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f234e000640 1 -- 172.21.15.83:0/3101548768 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23380ab400 con 0x7f23500d1930 2024-04-25T00:12:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f234e000640 1 -- 172.21.15.83:0/3101548768 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348001680 con 0x7f23500d1930 2024-04-25T00:12:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 -- 172.21.15.83:0/3101548768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23500d1930 msgr2=0x7f23500ced00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 --2- 172.21.15.83:0/3101548768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23500d1930 0x7f23500ced00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f23380097c0 tx=0x7f233801cca0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 -- 172.21.15.83:0/3101548768 shutdown_connections 2024-04-25T00:12:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 --2- 172.21.15.83:0/3101548768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23500d1930 0x7f23500ced00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 -- 172.21.15.83:0/3101548768 >> 172.21.15.83:0/3101548768 conn(0x7f23500d1d70 msgr2=0x7f23500d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 -- 172.21.15.83:0/3101548768 shutdown_connections 2024-04-25T00:12:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 -- 172.21.15.83:0/3101548768 wait complete. 2024-04-25T00:12:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 Processor -- start 2024-04-25T00:12:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 -- start start 2024-04-25T00:12:56.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23500b0d20 0x7f2350150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23500d1930 0x7f235015d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23501618d0 0x7f23501f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23500cd7d0 con 0x7f23500d1930 2024-04-25T00:12:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23500cd650 con 0x7f23501618d0 2024-04-25T00:12:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2355c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f23500cd950 con 0x7f23500b0d20 2024-04-25T00:12:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2346000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23500d1930 0x7f235015d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2346000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23500d1930 0x7f235015d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56078/0 (socket says 172.21.15.83:56078) 2024-04-25T00:12:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.905+0000 7f2346000640 1 -- 172.21.15.83:0/1844712317 learned_addr learned my addr 172.21.15.83:0/1844712317 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f234f400640 1 --2- 172.21.15.83:0/1844712317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23501618d0 0x7f23501f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f234ea00640 1 --2- 172.21.15.83:0/1844712317 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23500b0d20 0x7f2350150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2346000640 1 -- 172.21.15.83:0/1844712317 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23500b0d20 msgr2=0x7f2350150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2346000640 1 --2- 172.21.15.83:0/1844712317 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23500b0d20 0x7f2350150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2346000640 1 -- 172.21.15.83:0/1844712317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23501618d0 msgr2=0x7f23501f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2346000640 1 --2- 172.21.15.83:0/1844712317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23501618d0 0x7f23501f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2346000640 1 -- 172.21.15.83:0/1844712317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2350154ea0 con 0x7f23500d1930 2024-04-25T00:12:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2346000640 1 --2- 172.21.15.83:0/1844712317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23500d1930 0x7f235015d4d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f233801a8f0 tx=0x7f2338018fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2347400640 1 -- 172.21.15.83:0/1844712317 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23380b0040 con 0x7f23500d1930 2024-04-25T00:12:56.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2347400640 1 -- 172.21.15.83:0/1844712317 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23380bb070 con 0x7f23500d1930 2024-04-25T00:12:56.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2355c00640 1 -- 172.21.15.83:0/1844712317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2350155180 con 0x7f23500d1930 2024-04-25T00:12:56.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2347400640 1 -- 172.21.15.83:0/1844712317 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23380b56c0 con 0x7f23500d1930 2024-04-25T00:12:56.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2355c00640 1 -- 172.21.15.83:0/1844712317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2350155740 con 0x7f23500d1930 2024-04-25T00:12:56.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2355c00640 1 -- 172.21.15.83:0/1844712317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f235004f5f0 con 0x7f23500d1930 2024-04-25T00:12:56.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2347400640 1 -- 172.21.15.83:0/1844712317 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f23380b3070 con 0x7f23500d1930 2024-04-25T00:12:56.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2347400640 1 --2- 172.21.15.83:0/1844712317 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f232c0678c0 0x7f232c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:56.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f2347400640 1 -- 172.21.15.83:0/1844712317 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f233811f9c0 con 0x7f23500d1930 2024-04-25T00:12:56.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.909+0000 7f234ea00640 1 --2- 172.21.15.83:0/1844712317 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f232c0678c0 0x7f232c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:56.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.913+0000 7f234ea00640 1 --2- 172.21.15.83:0/1844712317 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f232c0678c0 0x7f232c069d80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2340001e90 tx=0x7f2340009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:56.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:56.913+0000 7f2347400640 1 -- 172.21.15.83:0/1844712317 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2338015fb0 con 0x7f23500d1930 2024-04-25T00:12:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.069+0000 7f2355c00640 1 -- 172.21.15.83:0/1844712317 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2350153280 con 0x7f232c0678c0 2024-04-25T00:12:57.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.073+0000 7f2347400640 1 -- 172.21.15.83:0/1844712317 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251934 (secure 0 0 0) 0x7f2350153280 con 0x7f232c0678c0 2024-04-25T00:12:57.078 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:57.080 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:12:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.077+0000 7f2355c00640 1 -- 172.21.15.83:0/1844712317 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f232c0678c0 msgr2=0x7f232c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.077+0000 7f2355c00640 1 --2- 172.21.15.83:0/1844712317 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f232c0678c0 0x7f232c069d80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2340001e90 tx=0x7f2340009040 comp rx=0 tx=0).stop 2024-04-25T00:12:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.077+0000 7f2355c00640 1 -- 172.21.15.83:0/1844712317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23500d1930 msgr2=0x7f235015d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.077+0000 7f2355c00640 1 --2- 172.21.15.83:0/1844712317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23500d1930 0x7f235015d4d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f233801a8f0 tx=0x7f2338018fe0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.077+0000 7f2355c00640 1 -- 172.21.15.83:0/1844712317 shutdown_connections 2024-04-25T00:12:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.077+0000 7f2355c00640 1 --2- 172.21.15.83:0/1844712317 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f232c0678c0 0x7f232c069d80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.077+0000 7f2355c00640 1 --2- 172.21.15.83:0/1844712317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23501618d0 0x7f23501f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.077+0000 7f2355c00640 1 --2- 172.21.15.83:0/1844712317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23500d1930 0x7f235015d4d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.077+0000 7f2355c00640 1 --2- 172.21.15.83:0/1844712317 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23500b0d20 0x7f2350150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.077+0000 7f2355c00640 1 -- 172.21.15.83:0/1844712317 >> 172.21.15.83:0/1844712317 conn(0x7f23500d1d70 msgr2=0x7f23500d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.077+0000 7f2355c00640 1 -- 172.21.15.83:0/1844712317 shutdown_connections 2024-04-25T00:12:57.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.077+0000 7f2355c00640 1 -- 172.21.15.83:0/1844712317 wait complete. 2024-04-25T00:12:57.099 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-04-25T00:12:56.466164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226036,"kb_used_data":5708,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748684,"statfs":{"total":773094113280,"available":772862652416,"internally_reserved":0,"allocated":5844992,"data_stored":3212480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.835076"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.669780+0000","last_change":"2024-04-25T00:12:52.663528+0000","last_active":"2024-04-25T00:12:52.669780+0000","last_peered":"2024-04-25T00:12:52.669780+0000","last_clean":"2024-04-25T00:12:52.669780+0000","last_became_active":"2024-04-25T00:12:52.663254+0000","last_became_peered":"2024-04-25T00:12:52.663254+0000","last_unstale":"2024-04-25T00:12:52.669780+0000","last_undegraded":"2024-04-25T00:12:52.669780+0000","last_fullsized":"2024-04-25T00:12:52.669780+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:34.570444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.670023+0000","last_change":"2024-04-25T00:12:52.668871+0000","last_active":"2024-04-25T00:12:53.670023+0000","last_peered":"2024-04-25T00:12:53.670023+0000","last_clean":"2024-04-25T00:12:53.670023+0000","last_became_active":"2024-04-25T00:12:52.667659+0000","last_became_peered":"2024-04-25T00:12:52.667659+0000","last_unstale":"2024-04-25T00:12:53.670023+0000","last_undegraded":"2024-04-25T00:12:53.670023+0000","last_fullsized":"2024-04-25T00:12:53.670023+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:12.909892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656795+0000","last_change":"2024-04-25T00:12:52.667397+0000","last_active":"2024-04-25T00:12:53.656795+0000","last_peered":"2024-04-25T00:12:53.656795+0000","last_clean":"2024-04-25T00:12:53.656795+0000","last_became_active":"2024-04-25T00:12:52.661782+0000","last_became_peered":"2024-04-25T00:12:52.661782+0000","last_unstale":"2024-04-25T00:12:53.656795+0000","last_undegraded":"2024-04-25T00:12:53.656795+0000","last_fullsized":"2024-04-25T00:12:53.656795+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:01.498756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669725+0000","last_change":"2024-04-25T00:12:52.678156+0000","last_active":"2024-04-25T00:12:53.669725+0000","last_peered":"2024-04-25T00:12:53.669725+0000","last_clean":"2024-04-25T00:12:53.669725+0000","last_became_active":"2024-04-25T00:12:52.677825+0000","last_became_peered":"2024-04-25T00:12:52.677825+0000","last_unstale":"2024-04-25T00:12:53.669725+0000","last_undegraded":"2024-04-25T00:12:53.669725+0000","last_fullsized":"2024-04-25T00:12:53.669725+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:48.214697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.675582+0000","last_change":"2024-04-25T00:12:52.672099+0000","last_active":"2024-04-25T00:12:52.675582+0000","last_peered":"2024-04-25T00:12:52.675582+0000","last_clean":"2024-04-25T00:12:52.675582+0000","last_became_active":"2024-04-25T00:12:52.671658+0000","last_became_peered":"2024-04-25T00:12:52.671658+0000","last_unstale":"2024-04-25T00:12:52.675582+0000","last_undegraded":"2024-04-25T00:12:52.675582+0000","last_fullsized":"2024-04-25T00:12:52.675582+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:40.554694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.672046+0000","last_change":"2024-04-25T00:12:52.668232+0000","last_active":"2024-04-25T00:12:52.672046+0000","last_peered":"2024-04-25T00:12:52.672046+0000","last_clean":"2024-04-25T00:12:52.672046+0000","last_became_active":"2024-04-25T00:12:52.667983+0000","last_became_peered":"2024-04-25T00:12:52.667983+0000","last_unstale":"2024-04-25T00:12:52.672046+0000","last_undegraded":"2024-04-25T00:12:52.672046+0000","last_fullsized":"2024-04-25T00:12:52.672046+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:05.558235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.655919+0000","last_change":"2024-04-25T00:12:52.666008+0000","last_active":"2024-04-25T00:12:53.655919+0000","last_peered":"2024-04-25T00:12:53.655919+0000","last_clean":"2024-04-25T00:12:53.655919+0000","last_became_active":"2024-04-25T00:12:52.665732+0000","last_became_peered":"2024-04-25T00:12:52.665732+0000","last_unstale":"2024-04-25T00:12:53.655919+0000","last_undegraded":"2024-04-25T00:12:53.655919+0000","last_fullsized":"2024-04-25T00:12:53.655919+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:42.072677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326299+0000","last_change":"2024-04-25T00:12:52.666739+0000","last_active":"2024-04-25T00:12:54.326299+0000","last_peered":"2024-04-25T00:12:54.326299+0000","last_clean":"2024-04-25T00:12:54.326299+0000","last_became_active":"2024-04-25T00:12:52.666476+0000","last_became_peered":"2024-04-25T00:12:52.666476+0000","last_unstale":"2024-04-25T00:12:54.326299+0000","last_undegraded":"2024-04-25T00:12:54.326299+0000","last_fullsized":"2024-04-25T00:12:54.326299+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:41.077845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326682+0000","last_change":"2024-04-25T00:12:52.665887+0000","last_active":"2024-04-25T00:12:54.326682+0000","last_peered":"2024-04-25T00:12:54.326682+0000","last_clean":"2024-04-25T00:12:54.326682+0000","last_became_active":"2024-04-25T00:12:52.665579+0000","last_became_peered":"2024-04-25T00:12:52.665579+0000","last_unstale":"2024-04-25T00:12:54.326682+0000","last_undegraded":"2024-04-25T00:12:54.326682+0000","last_fullsized":"2024-04-25T00:12:54.326682+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:46.257055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.669946+0000","last_change":"2024-04-25T00:12:52.666663+0000","last_active":"2024-04-25T00:12:52.669946+0000","last_peered":"2024-04-25T00:12:52.669946+0000","last_clean":"2024-04-25T00:12:52.669946+0000","last_became_active":"2024-04-25T00:12:52.666391+0000","last_became_peered":"2024-04-25T00:12:52.666391+0000","last_unstale":"2024-04-25T00:12:52.669946+0000","last_undegraded":"2024-04-25T00:12:52.669946+0000","last_fullsized":"2024-04-25T00:12:52.669946+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:10.726792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.657270+0000","last_change":"2024-04-25T00:12:52.669222+0000","last_active":"2024-04-25T00:12:53.657270+0000","last_peered":"2024-04-25T00:12:53.657270+0000","last_clean":"2024-04-25T00:12:53.657270+0000","last_became_active":"2024-04-25T00:12:52.668951+0000","last_became_peered":"2024-04-25T00:12:52.668951+0000","last_unstale":"2024-04-25T00:12:53.657270+0000","last_undegraded":"2024-04-25T00:12:53.657270+0000","last_fullsized":"2024-04-25T00:12:53.657270+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:03.865061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326645+0000","last_change":"2024-04-25T00:12:52.668528+0000","last_active":"2024-04-25T00:12:54.326645+0000","last_peered":"2024-04-25T00:12:54.326645+0000","last_clean":"2024-04-25T00:12:54.326645+0000","last_became_active":"2024-04-25T00:12:52.668260+0000","last_became_peered":"2024-04-25T00:12:52.668260+0000","last_unstale":"2024-04-25T00:12:54.326645+0000","last_undegraded":"2024-04-25T00:12:54.326645+0000","last_fullsized":"2024-04-25T00:12:54.326645+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:20.491683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.670209+0000","last_change":"2024-04-25T00:12:52.666556+0000","last_active":"2024-04-25T00:12:52.670209+0000","last_peered":"2024-04-25T00:12:52.670209+0000","last_clean":"2024-04-25T00:12:52.670209+0000","last_became_active":"2024-04-25T00:12:52.666368+0000","last_became_peered":"2024-04-25T00:12:52.666368+0000","last_unstale":"2024-04-25T00:12:52.670209+0000","last_undegraded":"2024-04-25T00:12:52.670209+0000","last_fullsized":"2024-04-25T00:12:52.670209+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:25.378561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.655987+0000","last_change":"2024-04-25T00:12:52.660788+0000","last_active":"2024-04-25T00:12:53.655987+0000","last_peered":"2024-04-25T00:12:53.655987+0000","last_clean":"2024-04-25T00:12:53.655987+0000","last_became_active":"2024-04-25T00:12:52.660433+0000","last_became_peered":"2024-04-25T00:12:52.660433+0000","last_unstale":"2024-04-25T00:12:53.655987+0000","last_undegraded":"2024-04-25T00:12:53.655987+0000","last_fullsized":"2024-04-25T00:12:53.655987+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:08.409969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669448+0000","last_change":"2024-04-25T00:12:52.669101+0000","last_active":"2024-04-25T00:12:53.669448+0000","last_peered":"2024-04-25T00:12:53.669448+0000","last_clean":"2024-04-25T00:12:53.669448+0000","last_became_active":"2024-04-25T00:12:52.668678+0000","last_became_peered":"2024-04-25T00:12:52.668678+0000","last_unstale":"2024-04-25T00:12:53.669448+0000","last_undegraded":"2024-04-25T00:12:53.669448+0000","last_fullsized":"2024-04-25T00:12:53.669448+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:42.072677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.667742+0000","last_change":"2024-04-25T00:12:52.658199+0000","last_active":"2024-04-25T00:12:52.667742+0000","last_peered":"2024-04-25T00:12:52.667742+0000","last_clean":"2024-04-25T00:12:52.667742+0000","last_became_active":"2024-04-25T00:12:52.657719+0000","last_became_peered":"2024-04-25T00:12:52.657719+0000","last_unstale":"2024-04-25T00:12:52.667742+0000","last_undegraded":"2024-04-25T00:12:52.667742+0000","last_fullsized":"2024-04-25T00:12:52.667742+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:02.685857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.657085+0000","last_change":"2024-04-25T00:12:52.673521+0000","last_active":"2024-04-25T00:12:53.657085+0000","last_peered":"2024-04-25T00:12:53.657085+0000","last_clean":"2024-04-25T00:12:53.657085+0000","last_became_active":"2024-04-25T00:12:52.673287+0000","last_became_peered":"2024-04-25T00:12:52.673287+0000","last_unstale":"2024-04-25T00:12:53.657085+0000","last_undegraded":"2024-04-25T00:12:53.657085+0000","last_fullsized":"2024-04-25T00:12:53.657085+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:53.473371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.670097+0000","last_change":"2024-04-25T00:12:52.672451+0000","last_active":"2024-04-25T00:12:53.670097+0000","last_peered":"2024-04-25T00:12:53.670097+0000","last_clean":"2024-04-25T00:12:53.670097+0000","last_became_active":"2024-04-25T00:12:52.672205+0000","last_became_peered":"2024-04-25T00:12:52.672205+0000","last_unstale":"2024-04-25T00:12:53.670097+0000","last_undegraded":"2024-04-25T00:12:53.670097+0000","last_fullsized":"2024-04-25T00:12:53.670097+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:01.498756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656926+0000","last_change":"2024-04-25T00:12:52.666728+0000","last_active":"2024-04-25T00:12:53.656926+0000","last_peered":"2024-04-25T00:12:53.656926+0000","last_clean":"2024-04-25T00:12:53.656926+0000","last_became_active":"2024-04-25T00:12:52.666299+0000","last_became_peered":"2024-04-25T00:12:52.666299+0000","last_unstale":"2024-04-25T00:12:53.656926+0000","last_undegraded":"2024-04-25T00:12:53.656926+0000","last_fullsized":"2024-04-25T00:12:53.656926+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:55.805741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659476+0000","last_change":"2024-04-25T00:12:52.660419+0000","last_active":"2024-04-25T00:12:53.659476+0000","last_peered":"2024-04-25T00:12:53.659476+0000","last_clean":"2024-04-25T00:12:53.659476+0000","last_became_active":"2024-04-25T00:12:52.660088+0000","last_became_peered":"2024-04-25T00:12:52.660088+0000","last_unstale":"2024-04-25T00:12:53.659476+0000","last_undegraded":"2024-04-25T00:12:53.659476+0000","last_fullsized":"2024-04-25T00:12:53.659476+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:35.629702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659221+0000","last_change":"2024-04-25T00:12:52.668961+0000","last_active":"2024-04-25T00:12:53.659221+0000","last_peered":"2024-04-25T00:12:53.659221+0000","last_clean":"2024-04-25T00:12:53.659221+0000","last_became_active":"2024-04-25T00:12:52.668711+0000","last_became_peered":"2024-04-25T00:12:52.668711+0000","last_unstale":"2024-04-25T00:12:53.659221+0000","last_undegraded":"2024-04-25T00:12:53.659221+0000","last_fullsized":"2024-04-25T00:12:53.659221+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:39.232682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326230+0000","last_change":"2024-04-25T00:12:52.666501+0000","last_active":"2024-04-25T00:12:54.326230+0000","last_peered":"2024-04-25T00:12:54.326230+0000","last_clean":"2024-04-25T00:12:54.326230+0000","last_became_active":"2024-04-25T00:12:52.666262+0000","last_became_peered":"2024-04-25T00:12:52.666262+0000","last_unstale":"2024-04-25T00:12:54.326230+0000","last_undegraded":"2024-04-25T00:12:54.326230+0000","last_fullsized":"2024-04-25T00:12:54.326230+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:07.020427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.657103+0000","last_change":"2024-04-25T00:12:52.670022+0000","last_active":"2024-04-25T00:12:53.657103+0000","last_peered":"2024-04-25T00:12:53.657103+0000","last_clean":"2024-04-25T00:12:53.657103+0000","last_became_active":"2024-04-25T00:12:52.668266+0000","last_became_peered":"2024-04-25T00:12:52.668266+0000","last_unstale":"2024-04-25T00:12:53.657103+0000","last_undegraded":"2024-04-25T00:12:53.657103+0000","last_fullsized":"2024-04-25T00:12:53.657103+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:24.401745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.655815+0000","last_change":"2024-04-25T00:12:52.657444+0000","last_active":"2024-04-25T00:12:53.655815+0000","last_peered":"2024-04-25T00:12:53.655815+0000","last_clean":"2024-04-25T00:12:53.655815+0000","last_became_active":"2024-04-25T00:12:52.657177+0000","last_became_peered":"2024-04-25T00:12:52.657177+0000","last_unstale":"2024-04-25T00:12:53.655815+0000","last_undegraded":"2024-04-25T00:12:53.655815+0000","last_fullsized":"2024-04-25T00:12:53.655815+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:05.076681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656848+0000","last_change":"2024-04-25T00:12:52.670638+0000","last_active":"2024-04-25T00:12:53.656848+0000","last_peered":"2024-04-25T00:12:53.656848+0000","last_clean":"2024-04-25T00:12:53.656848+0000","last_became_active":"2024-04-25T00:12:52.670372+0000","last_became_peered":"2024-04-25T00:12:52.670372+0000","last_unstale":"2024-04-25T00:12:53.656848+0000","last_undegraded":"2024-04-25T00:12:53.656848+0000","last_fullsized":"2024-04-25T00:12:53.656848+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.083420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326388+0000","last_change":"2024-04-25T00:12:52.663251+0000","last_active":"2024-04-25T00:12:54.326388+0000","last_peered":"2024-04-25T00:12:54.326388+0000","last_clean":"2024-04-25T00:12:54.326388+0000","last_became_active":"2024-04-25T00:12:52.662779+0000","last_became_peered":"2024-04-25T00:12:52.662779+0000","last_unstale":"2024-04-25T00:12:54.326388+0000","last_undegraded":"2024-04-25T00:12:54.326388+0000","last_fullsized":"2024-04-25T00:12:54.326388+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:12.858553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.672014+0000","last_change":"2024-04-25T00:12:52.668345+0000","last_active":"2024-04-25T00:12:52.672014+0000","last_peered":"2024-04-25T00:12:52.672014+0000","last_clean":"2024-04-25T00:12:52.672014+0000","last_became_active":"2024-04-25T00:12:52.667886+0000","last_became_peered":"2024-04-25T00:12:52.667886+0000","last_unstale":"2024-04-25T00:12:52.672014+0000","last_undegraded":"2024-04-25T00:12:52.672014+0000","last_fullsized":"2024-04-25T00:12:52.672014+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:02.842914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659562+0000","last_change":"2024-04-25T00:12:52.657048+0000","last_active":"2024-04-25T00:12:53.659562+0000","last_peered":"2024-04-25T00:12:53.659562+0000","last_clean":"2024-04-25T00:12:53.659562+0000","last_became_active":"2024-04-25T00:12:52.656755+0000","last_became_peered":"2024-04-25T00:12:52.656755+0000","last_unstale":"2024-04-25T00:12:53.659562+0000","last_undegraded":"2024-04-25T00:12:53.659562+0000","last_fullsized":"2024-04-25T00:12:53.659562+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:54.818422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656053+0000","last_change":"2024-04-25T00:12:52.664729+0000","last_active":"2024-04-25T00:12:53.656053+0000","last_peered":"2024-04-25T00:12:53.656053+0000","last_clean":"2024-04-25T00:12:53.656053+0000","last_became_active":"2024-04-25T00:12:52.659880+0000","last_became_peered":"2024-04-25T00:12:52.659880+0000","last_unstale":"2024-04-25T00:12:53.656053+0000","last_undegraded":"2024-04-25T00:12:53.656053+0000","last_fullsized":"2024-04-25T00:12:53.656053+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:48.214697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669566+0000","last_change":"2024-04-25T00:12:52.682054+0000","last_active":"2024-04-25T00:12:53.669566+0000","last_peered":"2024-04-25T00:12:53.669566+0000","last_clean":"2024-04-25T00:12:53.669566+0000","last_became_active":"2024-04-25T00:12:52.681764+0000","last_became_peered":"2024-04-25T00:12:52.681764+0000","last_unstale":"2024-04-25T00:12:53.669566+0000","last_undegraded":"2024-04-25T00:12:53.669566+0000","last_fullsized":"2024-04-25T00:12:53.669566+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:08.409969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656061+0000","last_change":"2024-04-25T00:12:52.661360+0000","last_active":"2024-04-25T00:12:53.656061+0000","last_peered":"2024-04-25T00:12:53.656061+0000","last_clean":"2024-04-25T00:12:53.656061+0000","last_became_active":"2024-04-25T00:12:52.661087+0000","last_became_peered":"2024-04-25T00:12:52.661087+0000","last_unstale":"2024-04-25T00:12:53.656061+0000","last_undegraded":"2024-04-25T00:12:53.656061+0000","last_fullsized":"2024-04-25T00:12:53.656061+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:50.441126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.680881+0000","last_change":"2024-04-25T00:12:52.675427+0000","last_active":"2024-04-25T00:12:52.680881+0000","last_peered":"2024-04-25T00:12:52.680881+0000","last_clean":"2024-04-25T00:12:52.680881+0000","last_became_active":"2024-04-25T00:12:52.675137+0000","last_became_peered":"2024-04-25T00:12:52.675137+0000","last_unstale":"2024-04-25T00:12:52.680881+0000","last_undegraded":"2024-04-25T00:12:52.680881+0000","last_fullsized":"2024-04-25T00:12:52.680881+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:37.301188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669650+0000","last_change":"2024-04-25T00:12:52.676969+0000","last_active":"2024-04-25T00:12:53.669650+0000","last_peered":"2024-04-25T00:12:53.669650+0000","last_clean":"2024-04-25T00:12:53.669650+0000","last_became_active":"2024-04-25T00:12:52.676714+0000","last_became_peered":"2024-04-25T00:12:52.676714+0000","last_unstale":"2024-04-25T00:12:53.669650+0000","last_undegraded":"2024-04-25T00:12:53.669650+0000","last_fullsized":"2024-04-25T00:12:53.669650+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:25.196761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"48'4","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659518+0000","last_change":"2024-04-25T00:12:51.652099+0000","last_active":"2024-04-25T00:12:53.659518+0000","last_peered":"2024-04-25T00:12:53.659518+0000","last_clean":"2024-04-25T00:12:53.659518+0000","last_became_active":"2024-04-25T00:12:51.650949+0000","last_became_peered":"2024-04-25T00:12:51.650949+0000","last_unstale":"2024-04-25T00:12:53.659518+0000","last_undegraded":"2024-04-25T00:12:53.659518+0000","last_fullsized":"2024-04-25T00:12:53.659518+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:10.741559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.675741+0000","last_change":"2024-04-25T00:12:52.670482+0000","last_active":"2024-04-25T00:12:52.675741+0000","last_peered":"2024-04-25T00:12:52.675741+0000","last_clean":"2024-04-25T00:12:52.675741+0000","last_became_active":"2024-04-25T00:12:52.670152+0000","last_became_peered":"2024-04-25T00:12:52.670152+0000","last_unstale":"2024-04-25T00:12:52.675741+0000","last_undegraded":"2024-04-25T00:12:52.675741+0000","last_fullsized":"2024-04-25T00:12:52.675741+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:56.140286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.679193+0000","last_change":"2024-04-25T00:12:51.661434+0000","last_active":"2024-04-25T00:12:52.679193+0000","last_peered":"2024-04-25T00:12:52.679193+0000","last_clean":"2024-04-25T00:12:52.679193+0000","last_became_active":"2024-04-25T00:12:51.661181+0000","last_became_peered":"2024-04-25T00:12:51.661181+0000","last_unstale":"2024-04-25T00:12:52.679193+0000","last_undegraded":"2024-04-25T00:12:52.679193+0000","last_fullsized":"2024-04-25T00:12:52.679193+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:32.652596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659205+0000","last_change":"2024-04-25T00:12:52.668960+0000","last_active":"2024-04-25T00:12:53.659205+0000","last_peered":"2024-04-25T00:12:53.659205+0000","last_clean":"2024-04-25T00:12:53.659205+0000","last_became_active":"2024-04-25T00:12:52.668615+0000","last_became_peered":"2024-04-25T00:12:52.668615+0000","last_unstale":"2024-04-25T00:12:53.659205+0000","last_undegraded":"2024-04-25T00:12:53.659205+0000","last_fullsized":"2024-04-25T00:12:53.659205+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:23.249577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.674828+0000","last_change":"2024-04-25T00:12:51.648958+0000","last_active":"2024-04-25T00:12:52.674828+0000","last_peered":"2024-04-25T00:12:52.674828+0000","last_clean":"2024-04-25T00:12:52.674828+0000","last_became_active":"2024-04-25T00:12:51.648688+0000","last_became_peered":"2024-04-25T00:12:51.648688+0000","last_unstale":"2024-04-25T00:12:52.674828+0000","last_undegraded":"2024-04-25T00:12:52.674828+0000","last_fullsized":"2024-04-25T00:12:52.674828+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:53:38.334610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326214+0000","last_change":"2024-04-25T00:12:52.660917+0000","last_active":"2024-04-25T00:12:54.326214+0000","last_peered":"2024-04-25T00:12:54.326214+0000","last_clean":"2024-04-25T00:12:54.326214+0000","last_became_active":"2024-04-25T00:12:52.660632+0000","last_became_peered":"2024-04-25T00:12:52.660632+0000","last_unstale":"2024-04-25T00:12:54.326214+0000","last_undegraded":"2024-04-25T00:12:54.326214+0000","last_fullsized":"2024-04-25T00:12:54.326214+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:53.473371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.685391+0000","last_change":"2024-04-25T00:12:51.654965+0000","last_active":"2024-04-25T00:12:52.685391+0000","last_peered":"2024-04-25T00:12:52.685391+0000","last_clean":"2024-04-25T00:12:52.685391+0000","last_became_active":"2024-04-25T00:12:51.654724+0000","last_became_peered":"2024-04-25T00:12:51.654724+0000","last_unstale":"2024-04-25T00:12:52.685391+0000","last_undegraded":"2024-04-25T00:12:52.685391+0000","last_fullsized":"2024-04-25T00:12:52.685391+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:16.308294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.676239+0000","last_change":"2024-04-25T00:12:52.674588+0000","last_active":"2024-04-25T00:12:52.676239+0000","last_peered":"2024-04-25T00:12:52.676239+0000","last_clean":"2024-04-25T00:12:52.676239+0000","last_became_active":"2024-04-25T00:12:52.674333+0000","last_became_peered":"2024-04-25T00:12:52.674333+0000","last_unstale":"2024-04-25T00:12:52.676239+0000","last_undegraded":"2024-04-25T00:12:52.676239+0000","last_fullsized":"2024-04-25T00:12:52.676239+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:02.685857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656227+0000","last_change":"2024-04-25T00:12:51.659654+0000","last_active":"2024-04-25T00:12:53.656227+0000","last_peered":"2024-04-25T00:12:53.656227+0000","last_clean":"2024-04-25T00:12:53.656227+0000","last_became_active":"2024-04-25T00:12:51.659433+0000","last_became_peered":"2024-04-25T00:12:51.659433+0000","last_unstale":"2024-04-25T00:12:53.656227+0000","last_undegraded":"2024-04-25T00:12:53.656227+0000","last_fullsized":"2024-04-25T00:12:53.656227+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:01.679108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669881+0000","last_change":"2024-04-25T00:12:52.680204+0000","last_active":"2024-04-25T00:12:53.669881+0000","last_peered":"2024-04-25T00:12:53.669881+0000","last_clean":"2024-04-25T00:12:53.669881+0000","last_became_active":"2024-04-25T00:12:52.679933+0000","last_became_peered":"2024-04-25T00:12:52.679933+0000","last_unstale":"2024-04-25T00:12:53.669881+0000","last_undegraded":"2024-04-25T00:12:53.669881+0000","last_fullsized":"2024-04-25T00:12:53.669881+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:50.441126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.677893+0000","last_change":"2024-04-25T00:12:51.651147+0000","last_active":"2024-04-25T00:12:52.677893+0000","last_peered":"2024-04-25T00:12:52.677893+0000","last_clean":"2024-04-25T00:12:52.677893+0000","last_became_active":"2024-04-25T00:12:51.650857+0000","last_became_peered":"2024-04-25T00:12:51.650857+0000","last_unstale":"2024-04-25T00:12:52.677893+0000","last_undegraded":"2024-04-25T00:12:52.677893+0000","last_fullsized":"2024-04-25T00:12:52.677893+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:32.652596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326568+0000","last_change":"2024-04-25T00:12:52.663050+0000","last_active":"2024-04-25T00:12:54.326568+0000","last_peered":"2024-04-25T00:12:54.326568+0000","last_clean":"2024-04-25T00:12:54.326568+0000","last_became_active":"2024-04-25T00:12:52.662487+0000","last_became_peered":"2024-04-25T00:12:52.662487+0000","last_unstale":"2024-04-25T00:12:54.326568+0000","last_undegraded":"2024-04-25T00:12:54.326568+0000","last_fullsized":"2024-04-25T00:12:54.326568+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:16.048766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659080+0000","last_change":"2024-04-25T00:12:51.649636+0000","last_active":"2024-04-25T00:12:53.659080+0000","last_peered":"2024-04-25T00:12:53.659080+0000","last_clean":"2024-04-25T00:12:53.659080+0000","last_became_active":"2024-04-25T00:12:51.649332+0000","last_became_peered":"2024-04-25T00:12:51.649332+0000","last_unstale":"2024-04-25T00:12:53.659080+0000","last_undegraded":"2024-04-25T00:12:53.659080+0000","last_fullsized":"2024-04-25T00:12:53.659080+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:56.763479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.670025+0000","last_change":"2024-04-25T00:12:52.679726+0000","last_active":"2024-04-25T00:12:53.670025+0000","last_peered":"2024-04-25T00:12:53.670025+0000","last_clean":"2024-04-25T00:12:53.670025+0000","last_became_active":"2024-04-25T00:12:52.679456+0000","last_became_peered":"2024-04-25T00:12:52.679456+0000","last_unstale":"2024-04-25T00:12:53.670025+0000","last_undegraded":"2024-04-25T00:12:53.670025+0000","last_fullsized":"2024-04-25T00:12:53.670025+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:37.141163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.655801+0000","last_change":"2024-04-25T00:12:51.653108+0000","last_active":"2024-04-25T00:12:53.655801+0000","last_peered":"2024-04-25T00:12:53.655801+0000","last_clean":"2024-04-25T00:12:53.655801+0000","last_became_active":"2024-04-25T00:12:51.652852+0000","last_became_peered":"2024-04-25T00:12:51.652852+0000","last_unstale":"2024-04-25T00:12:53.655801+0000","last_undegraded":"2024-04-25T00:12:53.655801+0000","last_fullsized":"2024-04-25T00:12:53.655801+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:33.563695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.681038+0000","last_change":"2024-04-25T00:12:52.678919+0000","last_active":"2024-04-25T00:12:52.681038+0000","last_peered":"2024-04-25T00:12:52.681038+0000","last_clean":"2024-04-25T00:12:52.681038+0000","last_became_active":"2024-04-25T00:12:52.678611+0000","last_became_peered":"2024-04-25T00:12:52.678611+0000","last_unstale":"2024-04-25T00:12:52.681038+0000","last_undegraded":"2024-04-25T00:12:52.681038+0000","last_fullsized":"2024-04-25T00:12:52.681038+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:34.570444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.670056+0000","last_change":"2024-04-25T00:12:51.647688+0000","last_active":"2024-04-25T00:12:53.670056+0000","last_peered":"2024-04-25T00:12:53.670056+0000","last_clean":"2024-04-25T00:12:53.670056+0000","last_became_active":"2024-04-25T00:12:51.647381+0000","last_became_peered":"2024-04-25T00:12:51.647381+0000","last_unstale":"2024-04-25T00:12:53.670056+0000","last_undegraded":"2024-04-25T00:12:53.670056+0000","last_fullsized":"2024-04-25T00:12:53.670056+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:53.811673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.675729+0000","last_change":"2024-04-25T00:12:52.668194+0000","last_active":"2024-04-25T00:12:52.675729+0000","last_peered":"2024-04-25T00:12:52.675729+0000","last_clean":"2024-04-25T00:12:52.675729+0000","last_became_active":"2024-04-25T00:12:52.666724+0000","last_became_peered":"2024-04-25T00:12:52.666724+0000","last_unstale":"2024-04-25T00:12:52.675729+0000","last_undegraded":"2024-04-25T00:12:52.675729+0000","last_fullsized":"2024-04-25T00:12:52.675729+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:28.819655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.654798+0000","last_change":"2024-04-25T00:12:51.660958+0000","last_active":"2024-04-25T00:12:52.654798+0000","last_peered":"2024-04-25T00:12:52.654798+0000","last_clean":"2024-04-25T00:12:52.654798+0000","last_became_active":"2024-04-25T00:12:51.660706+0000","last_became_peered":"2024-04-25T00:12:51.660706+0000","last_unstale":"2024-04-25T00:12:52.654798+0000","last_undegraded":"2024-04-25T00:12:52.654798+0000","last_fullsized":"2024-04-25T00:12:52.654798+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:56.763479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656040+0000","last_change":"2024-04-25T00:12:52.662098+0000","last_active":"2024-04-25T00:12:53.656040+0000","last_peered":"2024-04-25T00:12:53.656040+0000","last_clean":"2024-04-25T00:12:53.656040+0000","last_became_active":"2024-04-25T00:12:52.661655+0000","last_became_peered":"2024-04-25T00:12:52.661655+0000","last_unstale":"2024-04-25T00:12:53.656040+0000","last_undegraded":"2024-04-25T00:12:53.656040+0000","last_fullsized":"2024-04-25T00:12:53.656040+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:25.196761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669950+0000","last_change":"2024-04-25T00:12:51.654881+0000","last_active":"2024-04-25T00:12:53.669950+0000","last_peered":"2024-04-25T00:12:53.669950+0000","last_clean":"2024-04-25T00:12:53.669950+0000","last_became_active":"2024-04-25T00:12:51.654585+0000","last_became_peered":"2024-04-25T00:12:51.654585+0000","last_unstale":"2024-04-25T00:12:53.669950+0000","last_undegraded":"2024-04-25T00:12:53.669950+0000","last_fullsized":"2024-04-25T00:12:53.669950+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:34.622954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.675561+0000","last_change":"2024-04-25T00:12:52.671369+0000","last_active":"2024-04-25T00:12:52.675561+0000","last_peered":"2024-04-25T00:12:52.675561+0000","last_clean":"2024-04-25T00:12:52.675561+0000","last_became_active":"2024-04-25T00:12:52.667621+0000","last_became_peered":"2024-04-25T00:12:52.667621+0000","last_unstale":"2024-04-25T00:12:52.675561+0000","last_undegraded":"2024-04-25T00:12:52.675561+0000","last_fullsized":"2024-04-25T00:12:52.675561+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:53:39.341359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326624+0000","last_change":"2024-04-25T00:12:51.649041+0000","last_active":"2024-04-25T00:12:54.326624+0000","last_peered":"2024-04-25T00:12:54.326624+0000","last_clean":"2024-04-25T00:12:54.326624+0000","last_became_active":"2024-04-25T00:12:51.648787+0000","last_became_peered":"2024-04-25T00:12:51.648787+0000","last_unstale":"2024-04-25T00:12:54.326624+0000","last_undegraded":"2024-04-25T00:12:54.326624+0000","last_fullsized":"2024-04-25T00:12:54.326624+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:41.065928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.676017+0000","last_change":"2024-04-25T00:12:52.670705+0000","last_active":"2024-04-25T00:12:52.676017+0000","last_peered":"2024-04-25T00:12:52.676017+0000","last_clean":"2024-04-25T00:12:52.676017+0000","last_became_active":"2024-04-25T00:12:52.670390+0000","last_became_peered":"2024-04-25T00:12:52.670390+0000","last_unstale":"2024-04-25T00:12:52.676017+0000","last_undegraded":"2024-04-25T00:12:52.676017+0000","last_fullsized":"2024-04-25T00:12:52.676017+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:01.593192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656226+0000","last_change":"2024-04-25T00:12:51.659075+0000","last_active":"2024-04-25T00:12:53.656226+0000","last_peered":"2024-04-25T00:12:53.656226+0000","last_clean":"2024-04-25T00:12:53.656226+0000","last_became_active":"2024-04-25T00:12:51.656316+0000","last_became_peered":"2024-04-25T00:12:51.656316+0000","last_unstale":"2024-04-25T00:12:53.656226+0000","last_undegraded":"2024-04-25T00:12:53.656226+0000","last_fullsized":"2024-04-25T00:12:53.656226+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:28.666067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.657062+0000","last_change":"2024-04-25T00:12:52.666531+0000","last_active":"2024-04-25T00:12:53.657062+0000","last_peered":"2024-04-25T00:12:53.657062+0000","last_clean":"2024-04-25T00:12:53.657062+0000","last_became_active":"2024-04-25T00:12:52.665946+0000","last_became_peered":"2024-04-25T00:12:52.665946+0000","last_unstale":"2024-04-25T00:12:53.657062+0000","last_undegraded":"2024-04-25T00:12:53.657062+0000","last_fullsized":"2024-04-25T00:12:53.657062+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:39.182004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659247+0000","last_change":"2024-04-25T00:12:51.656091+0000","last_active":"2024-04-25T00:12:53.659247+0000","last_peered":"2024-04-25T00:12:53.659247+0000","last_clean":"2024-04-25T00:12:53.659247+0000","last_became_active":"2024-04-25T00:12:51.655836+0000","last_became_peered":"2024-04-25T00:12:51.655836+0000","last_unstale":"2024-04-25T00:12:53.659247+0000","last_undegraded":"2024-04-25T00:12:53.659247+0000","last_fullsized":"2024-04-25T00:12:53.659247+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:01.679108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326428+0000","last_change":"2024-04-25T00:12:52.669389+0000","last_active":"2024-04-25T00:12:54.326428+0000","last_peered":"2024-04-25T00:12:54.326428+0000","last_clean":"2024-04-25T00:12:54.326428+0000","last_became_active":"2024-04-25T00:12:52.669065+0000","last_became_peered":"2024-04-25T00:12:52.669065+0000","last_unstale":"2024-04-25T00:12:54.326428+0000","last_undegraded":"2024-04-25T00:12:54.326428+0000","last_fullsized":"2024-04-25T00:12:54.326428+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:54.804868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669434+0000","last_change":"2024-04-25T00:12:51.648853+0000","last_active":"2024-04-25T00:12:53.669434+0000","last_peered":"2024-04-25T00:12:53.669434+0000","last_clean":"2024-04-25T00:12:53.669434+0000","last_became_active":"2024-04-25T00:12:51.648568+0000","last_became_peered":"2024-04-25T00:12:51.648568+0000","last_unstale":"2024-04-25T00:12:53.669434+0000","last_undegraded":"2024-04-25T00:12:53.669434+0000","last_fullsized":"2024-04-25T00:12:53.669434+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:16.308294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656730+0000","last_change":"2024-04-25T00:12:52.672115+0000","last_active":"2024-04-25T00:12:53.656730+0000","last_peered":"2024-04-25T00:12:53.656730+0000","last_clean":"2024-04-25T00:12:53.656730+0000","last_became_active":"2024-04-25T00:12:52.671673+0000","last_became_peered":"2024-04-25T00:12:52.671673+0000","last_unstale":"2024-04-25T00:12:53.656730+0000","last_undegraded":"2024-04-25T00:12:53.656730+0000","last_fullsized":"2024-04-25T00:12:53.656730+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:37.141163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":22,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326450+0000","last_change":"2024-04-25T00:12:51.656996+0000","last_active":"2024-04-25T00:12:54.326450+0000","last_peered":"2024-04-25T00:12:54.326450+0000","last_clean":"2024-04-25T00:12:54.326450+0000","last_became_active":"2024-04-25T00:12:51.656554+0000","last_became_peered":"2024-04-25T00:12:51.656554+0000","last_unstale":"2024-04-25T00:12:54.326450+0000","last_undegraded":"2024-04-25T00:12:54.326450+0000","last_fullsized":"2024-04-25T00:12:54.326450+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:36.294440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326499+0000","last_change":"2024-04-25T00:12:52.674490+0000","last_active":"2024-04-25T00:12:54.326499+0000","last_peered":"2024-04-25T00:12:54.326499+0000","last_clean":"2024-04-25T00:12:54.326499+0000","last_became_active":"2024-04-25T00:12:52.674198+0000","last_became_peered":"2024-04-25T00:12:52.674198+0000","last_unstale":"2024-04-25T00:12:54.326499+0000","last_undegraded":"2024-04-25T00:12:54.326499+0000","last_fullsized":"2024-04-25T00:12:54.326499+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:36.631117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656954+0000","last_change":"2024-04-25T00:12:51.661693+0000","last_active":"2024-04-25T00:12:53.656954+0000","last_peered":"2024-04-25T00:12:53.656954+0000","last_clean":"2024-04-25T00:12:53.656954+0000","last_became_active":"2024-04-25T00:12:51.661409+0000","last_became_peered":"2024-04-25T00:12:51.661409+0000","last_unstale":"2024-04-25T00:12:53.656954+0000","last_undegraded":"2024-04-25T00:12:53.656954+0000","last_fullsized":"2024-04-25T00:12:53.656954+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:04.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.674836+0000","last_change":"2024-04-25T00:12:52.671048+0000","last_active":"2024-04-25T00:12:52.674836+0000","last_peered":"2024-04-25T00:12:52.674836+0000","last_clean":"2024-04-25T00:12:52.674836+0000","last_became_active":"2024-04-25T00:12:52.669534+0000","last_became_peered":"2024-04-25T00:12:52.669534+0000","last_unstale":"2024-04-25T00:12:52.674836+0000","last_undegraded":"2024-04-25T00:12:52.674836+0000","last_fullsized":"2024-04-25T00:12:52.674836+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:39.232682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.657408+0000","last_change":"2024-04-25T00:12:51.652635+0000","last_active":"2024-04-25T00:12:53.657408+0000","last_peered":"2024-04-25T00:12:53.657408+0000","last_clean":"2024-04-25T00:12:53.657408+0000","last_became_active":"2024-04-25T00:12:51.652328+0000","last_became_peered":"2024-04-25T00:12:51.652328+0000","last_unstale":"2024-04-25T00:12:53.657408+0000","last_undegraded":"2024-04-25T00:12:53.657408+0000","last_fullsized":"2024-04-25T00:12:53.657408+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:24.622672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.680601+0000","last_change":"2024-04-25T00:12:52.678059+0000","last_active":"2024-04-25T00:12:52.680601+0000","last_peered":"2024-04-25T00:12:52.680601+0000","last_clean":"2024-04-25T00:12:52.680601+0000","last_became_active":"2024-04-25T00:12:52.677557+0000","last_became_peered":"2024-04-25T00:12:52.677557+0000","last_unstale":"2024-04-25T00:12:52.680601+0000","last_undegraded":"2024-04-25T00:12:52.680601+0000","last_fullsized":"2024-04-25T00:12:52.680601+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:00.773724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659437+0000","last_change":"2024-04-25T00:12:51.651367+0000","last_active":"2024-04-25T00:12:53.659437+0000","last_peered":"2024-04-25T00:12:53.659437+0000","last_clean":"2024-04-25T00:12:53.659437+0000","last_became_active":"2024-04-25T00:12:51.651107+0000","last_became_peered":"2024-04-25T00:12:51.651107+0000","last_unstale":"2024-04-25T00:12:53.659437+0000","last_undegraded":"2024-04-25T00:12:53.659437+0000","last_fullsized":"2024-04-25T00:12:53.659437+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:53:38.334610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.669625+0000","last_change":"2024-04-25T00:12:52.664917+0000","last_active":"2024-04-25T00:12:52.669625+0000","last_peered":"2024-04-25T00:12:52.669625+0000","last_clean":"2024-04-25T00:12:52.669625+0000","last_became_active":"2024-04-25T00:12:52.663589+0000","last_became_peered":"2024-04-25T00:12:52.663589+0000","last_unstale":"2024-04-25T00:12:52.669625+0000","last_undegraded":"2024-04-25T00:12:52.669625+0000","last_fullsized":"2024-04-25T00:12:52.669625+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:29.672816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659577+0000","last_change":"2024-04-25T00:12:51.654187+0000","last_active":"2024-04-25T00:12:53.659577+0000","last_peered":"2024-04-25T00:12:53.659577+0000","last_clean":"2024-04-25T00:12:53.659577+0000","last_became_active":"2024-04-25T00:12:51.653868+0000","last_became_peered":"2024-04-25T00:12:51.653868+0000","last_unstale":"2024-04-25T00:12:53.659577+0000","last_undegraded":"2024-04-25T00:12:53.659577+0000","last_fullsized":"2024-04-25T00:12:53.659577+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:39.547945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.679536+0000","last_change":"2024-04-25T00:12:52.670746+0000","last_active":"2024-04-25T00:12:52.679536+0000","last_peered":"2024-04-25T00:12:52.679536+0000","last_clean":"2024-04-25T00:12:52.679536+0000","last_became_active":"2024-04-25T00:12:52.670432+0000","last_became_peered":"2024-04-25T00:12:52.670432+0000","last_unstale":"2024-04-25T00:12:52.679536+0000","last_undegraded":"2024-04-25T00:12:52.679536+0000","last_fullsized":"2024-04-25T00:12:52.679536+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:28.940781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.678967+0000","last_change":"2024-04-25T00:12:51.650824+0000","last_active":"2024-04-25T00:12:52.678967+0000","last_peered":"2024-04-25T00:12:52.678967+0000","last_clean":"2024-04-25T00:12:52.678967+0000","last_became_active":"2024-04-25T00:12:51.650486+0000","last_became_peered":"2024-04-25T00:12:51.650486+0000","last_unstale":"2024-04-25T00:12:52.678967+0000","last_undegraded":"2024-04-25T00:12:52.678967+0000","last_fullsized":"2024-04-25T00:12:52.678967+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:39.547945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659496+0000","last_change":"2024-04-25T00:12:52.664793+0000","last_active":"2024-04-25T00:12:53.659496+0000","last_peered":"2024-04-25T00:12:53.659496+0000","last_clean":"2024-04-25T00:12:53.659496+0000","last_became_active":"2024-04-25T00:12:52.664225+0000","last_became_peered":"2024-04-25T00:12:52.664225+0000","last_unstale":"2024-04-25T00:12:53.659496+0000","last_undegraded":"2024-04-25T00:12:53.659496+0000","last_fullsized":"2024-04-25T00:12:53.659496+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:47.181480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326104+0000","last_change":"2024-04-25T00:12:51.655623+0000","last_active":"2024-04-25T00:12:54.326104+0000","last_peered":"2024-04-25T00:12:54.326104+0000","last_clean":"2024-04-25T00:12:54.326104+0000","last_became_active":"2024-04-25T00:12:51.655310+0000","last_became_peered":"2024-04-25T00:12:51.655310+0000","last_unstale":"2024-04-25T00:12:54.326104+0000","last_undegraded":"2024-04-25T00:12:54.326104+0000","last_fullsized":"2024-04-25T00:12:54.326104+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:23.722877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656903+0000","last_change":"2024-04-25T00:12:52.660660+0000","last_active":"2024-04-25T00:12:53.656903+0000","last_peered":"2024-04-25T00:12:53.656903+0000","last_clean":"2024-04-25T00:12:53.656903+0000","last_became_active":"2024-04-25T00:12:52.660296+0000","last_became_peered":"2024-04-25T00:12:52.660296+0000","last_unstale":"2024-04-25T00:12:53.656903+0000","last_undegraded":"2024-04-25T00:12:53.656903+0000","last_fullsized":"2024-04-25T00:12:53.656903+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:39.543364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656150+0000","last_change":"2024-04-25T00:12:51.658951+0000","last_active":"2024-04-25T00:12:53.656150+0000","last_peered":"2024-04-25T00:12:53.656150+0000","last_clean":"2024-04-25T00:12:53.656150+0000","last_became_active":"2024-04-25T00:12:51.658654+0000","last_became_peered":"2024-04-25T00:12:51.658654+0000","last_unstale":"2024-04-25T00:12:53.656150+0000","last_undegraded":"2024-04-25T00:12:53.656150+0000","last_fullsized":"2024-04-25T00:12:53.656150+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:24.371812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.673870+0000","last_change":"2024-04-25T00:12:52.669989+0000","last_active":"2024-04-25T00:12:52.673870+0000","last_peered":"2024-04-25T00:12:52.673870+0000","last_clean":"2024-04-25T00:12:52.673870+0000","last_became_active":"2024-04-25T00:12:52.669707+0000","last_became_peered":"2024-04-25T00:12:52.669707+0000","last_unstale":"2024-04-25T00:12:52.673870+0000","last_undegraded":"2024-04-25T00:12:52.673870+0000","last_fullsized":"2024-04-25T00:12:52.673870+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:23.249577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656441+0000","last_change":"2024-04-25T00:12:51.652957+0000","last_active":"2024-04-25T00:12:53.656441+0000","last_peered":"2024-04-25T00:12:53.656441+0000","last_clean":"2024-04-25T00:12:53.656441+0000","last_became_active":"2024-04-25T00:12:51.652648+0000","last_became_peered":"2024-04-25T00:12:51.652648+0000","last_unstale":"2024-04-25T00:12:53.656441+0000","last_undegraded":"2024-04-25T00:12:53.656441+0000","last_fullsized":"2024-04-25T00:12:53.656441+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:04.694125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.680902+0000","last_change":"2024-04-25T00:12:52.675957+0000","last_active":"2024-04-25T00:12:52.680902+0000","last_peered":"2024-04-25T00:12:52.680902+0000","last_clean":"2024-04-25T00:12:52.680902+0000","last_became_active":"2024-04-25T00:12:52.675456+0000","last_became_peered":"2024-04-25T00:12:52.675456+0000","last_unstale":"2024-04-25T00:12:52.680902+0000","last_undegraded":"2024-04-25T00:12:52.680902+0000","last_fullsized":"2024-04-25T00:12:52.680902+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:24.729625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659638+0000","last_change":"2024-04-25T00:12:51.656066+0000","last_active":"2024-04-25T00:12:53.659638+0000","last_peered":"2024-04-25T00:12:53.659638+0000","last_clean":"2024-04-25T00:12:53.659638+0000","last_became_active":"2024-04-25T00:12:51.655792+0000","last_became_peered":"2024-04-25T00:12:51.655792+0000","last_unstale":"2024-04-25T00:12:53.659638+0000","last_undegraded":"2024-04-25T00:12:53.659638+0000","last_fullsized":"2024-04-25T00:12:53.659638+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:32.652596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.669255+0000","last_change":"2024-04-25T00:12:52.658546+0000","last_active":"2024-04-25T00:12:52.669255+0000","last_peered":"2024-04-25T00:12:52.669255+0000","last_clean":"2024-04-25T00:12:52.669255+0000","last_became_active":"2024-04-25T00:12:52.658256+0000","last_became_peered":"2024-04-25T00:12:52.658256+0000","last_unstale":"2024-04-25T00:12:52.669255+0000","last_undegraded":"2024-04-25T00:12:52.669255+0000","last_fullsized":"2024-04-25T00:12:52.669255+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:00.773724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.670034+0000","last_change":"2024-04-25T00:12:51.650127+0000","last_active":"2024-04-25T00:12:53.670034+0000","last_peered":"2024-04-25T00:12:53.670034+0000","last_clean":"2024-04-25T00:12:53.670034+0000","last_became_active":"2024-04-25T00:12:51.649817+0000","last_became_peered":"2024-04-25T00:12:51.649817+0000","last_unstale":"2024-04-25T00:12:53.670034+0000","last_undegraded":"2024-04-25T00:12:53.670034+0000","last_fullsized":"2024-04-25T00:12:53.670034+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:46.174731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326265+0000","last_change":"2024-04-25T00:12:52.669622+0000","last_active":"2024-04-25T00:12:54.326265+0000","last_peered":"2024-04-25T00:12:54.326265+0000","last_clean":"2024-04-25T00:12:54.326265+0000","last_became_active":"2024-04-25T00:12:52.669327+0000","last_became_peered":"2024-04-25T00:12:52.669327+0000","last_unstale":"2024-04-25T00:12:54.326265+0000","last_undegraded":"2024-04-25T00:12:54.326265+0000","last_fullsized":"2024-04-25T00:12:54.326265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:10.803393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656460+0000","last_change":"2024-04-25T00:12:51.652620+0000","last_active":"2024-04-25T00:12:53.656460+0000","last_peered":"2024-04-25T00:12:53.656460+0000","last_clean":"2024-04-25T00:12:53.656460+0000","last_became_active":"2024-04-25T00:12:51.652283+0000","last_became_peered":"2024-04-25T00:12:51.652283+0000","last_unstale":"2024-04-25T00:12:53.656460+0000","last_undegraded":"2024-04-25T00:12:53.656460+0000","last_fullsized":"2024-04-25T00:12:53.656460+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:59.766975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656402+0000","last_change":"2024-04-25T00:12:52.668754+0000","last_active":"2024-04-25T00:12:53.656402+0000","last_peered":"2024-04-25T00:12:53.656402+0000","last_clean":"2024-04-25T00:12:53.656402+0000","last_became_active":"2024-04-25T00:12:52.668453+0000","last_became_peered":"2024-04-25T00:12:52.668453+0000","last_unstale":"2024-04-25T00:12:53.656402+0000","last_undegraded":"2024-04-25T00:12:53.656402+0000","last_fullsized":"2024-04-25T00:12:53.656402+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:12.909892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326340+0000","last_change":"2024-04-25T00:12:51.656484+0000","last_active":"2024-04-25T00:12:54.326340+0000","last_peered":"2024-04-25T00:12:54.326340+0000","last_clean":"2024-04-25T00:12:54.326340+0000","last_became_active":"2024-04-25T00:12:51.656076+0000","last_became_peered":"2024-04-25T00:12:51.656076+0000","last_unstale":"2024-04-25T00:12:54.326340+0000","last_undegraded":"2024-04-25T00:12:54.326340+0000","last_fullsized":"2024-04-25T00:12:54.326340+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:27.934032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.675761+0000","last_change":"2024-04-25T00:12:52.669713+0000","last_active":"2024-04-25T00:12:52.675761+0000","last_peered":"2024-04-25T00:12:52.675761+0000","last_clean":"2024-04-25T00:12:52.675761+0000","last_became_active":"2024-04-25T00:12:52.667872+0000","last_became_peered":"2024-04-25T00:12:52.667872+0000","last_unstale":"2024-04-25T00:12:52.675761+0000","last_undegraded":"2024-04-25T00:12:52.675761+0000","last_fullsized":"2024-04-25T00:12:52.675761+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:16.703927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326208+0000","last_change":"2024-04-25T00:12:51.658109+0000","last_active":"2024-04-25T00:12:54.326208+0000","last_peered":"2024-04-25T00:12:54.326208+0000","last_clean":"2024-04-25T00:12:54.326208+0000","last_became_active":"2024-04-25T00:12:51.656811+0000","last_became_peered":"2024-04-25T00:12:51.656811+0000","last_unstale":"2024-04-25T00:12:54.326208+0000","last_undegraded":"2024-04-25T00:12:54.326208+0000","last_fullsized":"2024-04-25T00:12:54.326208+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:04.069932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.656828+0000","last_change":"2024-04-25T00:12:52.660936+0000","last_active":"2024-04-25T00:12:53.656828+0000","last_peered":"2024-04-25T00:12:53.656828+0000","last_clean":"2024-04-25T00:12:53.656828+0000","last_became_active":"2024-04-25T00:12:52.660605+0000","last_became_peered":"2024-04-25T00:12:52.660605+0000","last_unstale":"2024-04-25T00:12:53.656828+0000","last_undegraded":"2024-04-25T00:12:53.656828+0000","last_fullsized":"2024-04-25T00:12:53.656828+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:22.055408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.653254+0000","last_change":"2024-04-25T00:12:51.660978+0000","last_active":"2024-04-25T00:12:52.653254+0000","last_peered":"2024-04-25T00:12:52.653254+0000","last_clean":"2024-04-25T00:12:52.653254+0000","last_became_active":"2024-04-25T00:12:51.660727+0000","last_became_peered":"2024-04-25T00:12:51.660727+0000","last_unstale":"2024-04-25T00:12:52.653254+0000","last_undegraded":"2024-04-25T00:12:52.653254+0000","last_fullsized":"2024-04-25T00:12:52.653254+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:10.741559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:54.326730+0000","last_change":"2024-04-25T00:12:52.664358+0000","last_active":"2024-04-25T00:12:54.326730+0000","last_peered":"2024-04-25T00:12:54.326730+0000","last_clean":"2024-04-25T00:12:54.326730+0000","last_became_active":"2024-04-25T00:12:52.664021+0000","last_became_peered":"2024-04-25T00:12:52.664021+0000","last_unstale":"2024-04-25T00:12:54.326730+0000","last_undegraded":"2024-04-25T00:12:54.326730+0000","last_fullsized":"2024-04-25T00:12:54.326730+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:53.240194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T00:12:52.657675+0000","last_change":"2024-04-25T00:12:51.651914+0000","last_active":"2024-04-25T00:12:52.657675+0000","last_peered":"2024-04-25T00:12:52.657675+0000","last_clean":"2024-04-25T00:12:52.657675+0000","last_became_active":"2024-04-25T00:12:51.651689+0000","last_became_peered":"2024-04-25T00:12:51.651689+0000","last_unstale":"2024-04-25T00:12:52.657675+0000","last_undegraded":"2024-04-25T00:12:52.657675+0000","last_fullsized":"2024-04-25T00:12:52.657675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:00.247417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.659307+0000","last_change":"2024-04-25T00:12:52.671225+0000","last_active":"2024-04-25T00:12:53.659307+0000","last_peered":"2024-04-25T00:12:53.659307+0000","last_clean":"2024-04-25T00:12:53.659307+0000","last_became_active":"2024-04-25T00:12:52.670874+0000","last_became_peered":"2024-04-25T00:12:52.670874+0000","last_unstale":"2024-04-25T00:12:53.659307+0000","last_undegraded":"2024-04-25T00:12:53.659307+0000","last_fullsized":"2024-04-25T00:12:53.659307+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:51.628208+0000","last_clean_scrub_stamp":"2024-04-25T00:12:51.628208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:57.466333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.657020+0000","last_change":"2024-04-25T00:12:51.656724+0000","last_active":"2024-04-25T00:12:53.657020+0000","last_peered":"2024-04-25T00:12:53.657020+0000","last_clean":"2024-04-25T00:12:53.657020+0000","last_became_active":"2024-04-25T00:12:51.656419+0000","last_became_peered":"2024-04-25T00:12:51.656419+0000","last_unstale":"2024-04-25T00:12:53.657020+0000","last_undegraded":"2024-04-25T00:12:53.657020+0000","last_fullsized":"2024-04-25T00:12:53.657020+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:12:50.621459+0000","last_clean_scrub_stamp":"2024-04-25T00:12:50.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:09.720043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":222,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T00:12:53.669244+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:12:53.669244+0000","last_peered":"2024-04-25T00:12:53.669244+0000","last_clean":"2024-04-25T00:12:53.669244+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:12:53.669244+0000","last_undegraded":"2024-04-25T00:12:53.669244+0000","last_fullsized":"2024-04-25T00:12:53.669244+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1183744,"data_stored":848040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":251508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":250856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":251438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":252490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":259160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":251016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1183744,"data_stored":847972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:12:57.103 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:12:57.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:12:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f63ada00640 1 Processor -- start 2024-04-25T00:12:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f63ada00640 1 -- start start 2024-04-25T00:12:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f63ada00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f63a81555c0 0x7f63a815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f63ada00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a815c560 con 0x7f63a806e4c0 2024-04-25T00:12:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f63ada00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a8072460 con 0x7f63a81551e0 2024-04-25T00:12:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f63ada00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f63a815c7d0 con 0x7f63a81555c0 2024-04-25T00:12:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f63a7400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f63a806e4c0 0x7f63a815cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42258/0 (socket says 172.21.15.83:42258) 2024-04-25T00:12:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f63a7400640 1 -- 172.21.15.83:0/2761341663 learned_addr learned my addr 172.21.15.83:0/2761341663 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f639e000640 1 --2- 172.21.15.83:0/2761341663 >> v2:172.21.15.86:3300/0 conn(0x7f63a81555c0 0x7f63a815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f63a6000640 1 -- 172.21.15.83:0/2761341663 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 149673594 0 0) 0x7f63a815c560 con 0x7f63a806e4c0 2024-04-25T00:12:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f63a6000640 1 -- 172.21.15.83:0/2761341663 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f638c003530 con 0x7f63a806e4c0 2024-04-25T00:12:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f63a6000640 1 -- 172.21.15.83:0/2761341663 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3477259857 0 0) 0x7f638c003530 con 0x7f63a806e4c0 2024-04-25T00:12:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f639e000640 1 -- 172.21.15.83:0/2761341663 >> v1:172.21.15.86:6789/0 conn(0x7f63a81551e0 legacy=0x7f63a8071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f639e000640 1 -- 172.21.15.83:0/2761341663 >> v1:172.21.15.83:6789/0 conn(0x7f63a806e4c0 legacy=0x7f63a815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.281+0000 7f639e000640 1 -- 172.21.15.83:0/2761341663 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a815cab0 con 0x7f63a81555c0 2024-04-25T00:12:57.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.285+0000 7f639e000640 1 --2- 172.21.15.83:0/2761341663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63a81555c0 0x7f63a815bea0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f63900097f0 tx=0x7f639001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7b4a94eece01da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:57.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.285+0000 7f63a6000640 1 -- 172.21.15.83:0/2761341663 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63900b0070 con 0x7f63a81555c0 2024-04-25T00:12:57.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.285+0000 7f63a6000640 1 -- 172.21.15.83:0/2761341663 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63900ab400 con 0x7f63a81555c0 2024-04-25T00:12:57.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.285+0000 7f63a6000640 1 -- 172.21.15.83:0/2761341663 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0001680 con 0x7f63a81555c0 2024-04-25T00:12:57.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.285+0000 7f63ada00640 1 -- 172.21.15.83:0/2761341663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63a81555c0 msgr2=0x7f63a815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.285+0000 7f63ada00640 1 --2- 172.21.15.83:0/2761341663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63a81555c0 0x7f63a815bea0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f63900097f0 tx=0x7f639001c3c0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.285+0000 7f63ada00640 1 -- 172.21.15.83:0/2761341663 shutdown_connections 2024-04-25T00:12:57.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.285+0000 7f63ada00640 1 --2- 172.21.15.83:0/2761341663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63a81555c0 0x7f63a815bea0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.285+0000 7f63ada00640 1 -- 172.21.15.83:0/2761341663 >> 172.21.15.83:0/2761341663 conn(0x7f63a80b1830 msgr2=0x7f63a80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:57.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.285+0000 7f63ada00640 1 -- 172.21.15.83:0/2761341663 shutdown_connections 2024-04-25T00:12:57.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.285+0000 7f63ada00640 1 -- 172.21.15.83:0/2761341663 wait complete. 2024-04-25T00:12:57.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63ada00640 1 Processor -- start 2024-04-25T00:12:57.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63ada00640 1 -- start start 2024-04-25T00:12:57.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63ada00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a806e4c0 0x7f63a814f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:57.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63ada00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63a81551e0 0x7f63a81468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:57.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63ada00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63a81555c0 0x7f63a8147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:57.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63ada00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63a815e280 con 0x7f63a806e4c0 2024-04-25T00:12:57.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63ada00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63a815e100 con 0x7f63a81555c0 2024-04-25T00:12:57.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63ada00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f63a8168ca0 con 0x7f63a81551e0 2024-04-25T00:12:57.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63a6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a806e4c0 0x7f63a814f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63a6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a806e4c0 0x7f63a814f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56084/0 (socket says 172.21.15.83:56084) 2024-04-25T00:12:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63a6a00640 1 -- 172.21.15.83:0/3639701100 learned_addr learned my addr 172.21.15.83:0/3639701100 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f639e000640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63a81551e0 0x7f63a81468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63a7400640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63a81555c0 0x7f63a8147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63a6a00640 1 -- 172.21.15.83:0/3639701100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63a81551e0 msgr2=0x7f63a81468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63a6a00640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63a81551e0 0x7f63a81468d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63a6a00640 1 -- 172.21.15.83:0/3639701100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63a81555c0 msgr2=0x7f63a8147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63a6a00640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63a81555c0 0x7f63a8147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63a6a00640 1 -- 172.21.15.83:0/3639701100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a814f6b0 con 0x7f63a806e4c0 2024-04-25T00:12:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63a7400640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63a81555c0 0x7f63a8147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63a6a00640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a806e4c0 0x7f63a814f0d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f639800c7f0 tx=0x7f639800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f639f400640 1 -- 172.21.15.83:0/3639701100 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398003b80 con 0x7f63a806e4c0 2024-04-25T00:12:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f639f400640 1 -- 172.21.15.83:0/3639701100 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6398003d20 con 0x7f63a806e4c0 2024-04-25T00:12:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f639f400640 1 -- 172.21.15.83:0/3639701100 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639800aa90 con 0x7f63a806e4c0 2024-04-25T00:12:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63ada00640 1 -- 172.21.15.83:0/3639701100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a814f990 con 0x7f63a806e4c0 2024-04-25T00:12:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63ada00640 1 -- 172.21.15.83:0/3639701100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a80b3af0 con 0x7f63a806e4c0 2024-04-25T00:12:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.289+0000 7f63ada00640 1 -- 172.21.15.83:0/3639701100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a804f510 con 0x7f63a806e4c0 2024-04-25T00:12:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.293+0000 7f639f400640 1 -- 172.21.15.83:0/3639701100 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f639800ac30 con 0x7f63a806e4c0 2024-04-25T00:12:57.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.293+0000 7f639f400640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63840678c0 0x7f6384069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:57.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.293+0000 7f639f400640 1 -- 172.21.15.83:0/3639701100 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f6398085e50 con 0x7f63a806e4c0 2024-04-25T00:12:57.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.293+0000 7f639e000640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63840678c0 0x7f6384069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:57.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.293+0000 7f639e000640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63840678c0 0x7f6384069d80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f63900ad000 tx=0x7f6390008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:57.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.297+0000 7f639f400640 1 -- 172.21.15.83:0/3639701100 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6398056760 con 0x7f63a806e4c0 2024-04-25T00:12:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.509+0000 7f63ada00640 1 -- 172.21.15.83:0/3639701100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f63a806ce30 con 0x7f63a806e4c0 2024-04-25T00:12:57.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.513+0000 7f639f400640 1 -- 172.21.15.83:0/3639701100 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f6398056060 con 0x7f63a806e4c0 2024-04-25T00:12:57.514 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-25T00:12:57.514 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylmOqooLBAApte+X6YkRmrpe12YdjYekg== 2024-04-25T00:12:57.514 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-25T00:12:57.514 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-25T00:12:57.514 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:12:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-25T00:12:57.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.513+0000 7f63ada00640 1 -- 172.21.15.83:0/3639701100 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63840678c0 msgr2=0x7f6384069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.513+0000 7f63ada00640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63840678c0 0x7f6384069d80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f63900ad000 tx=0x7f6390008040 comp rx=0 tx=0).stop 2024-04-25T00:12:57.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.513+0000 7f63ada00640 1 -- 172.21.15.83:0/3639701100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a806e4c0 msgr2=0x7f63a814f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.513+0000 7f63ada00640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a806e4c0 0x7f63a814f0d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f639800c7f0 tx=0x7f639800ccb0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.513+0000 7f63ada00640 1 -- 172.21.15.83:0/3639701100 shutdown_connections 2024-04-25T00:12:57.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.513+0000 7f63ada00640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63840678c0 0x7f6384069d80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.513+0000 7f63ada00640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63a81551e0 0x7f63a81468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.513+0000 7f63ada00640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63a81555c0 0x7f63a8147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.513+0000 7f63ada00640 1 --2- 172.21.15.83:0/3639701100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a806e4c0 0x7f63a814f0d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.513+0000 7f63ada00640 1 -- 172.21.15.83:0/3639701100 >> 172.21.15.83:0/3639701100 conn(0x7f63a80b1830 msgr2=0x7f63a81540c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:57.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.513+0000 7f63ada00640 1 -- 172.21.15.83:0/3639701100 shutdown_connections 2024-04-25T00:12:57.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.513+0000 7f63ada00640 1 -- 172.21.15.83:0/3639701100 wait complete. 2024-04-25T00:12:57.535 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:12:57.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.665+0000 7f22ab200640 1 Processor -- start 2024-04-25T00:12:57.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.665+0000 7f22ab200640 1 -- start start 2024-04-25T00:12:57.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.669+0000 7f22ab200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22a40cc510 0x7f22a415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:57.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.669+0000 7f22ab200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f22a415cf90 0x7f22a40ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:57.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.669+0000 7f22ab200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22a40ccc60 con 0x7f22a41599f0 2024-04-25T00:12:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.669+0000 7f22ab200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f22a4159dd0 con 0x7f22a40cc510 2024-04-25T00:12:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.669+0000 7f22ab200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f22a406dba0 con 0x7f22a415cf90 2024-04-25T00:12:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.669+0000 7f22a3400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22a40cc510 0x7f22a415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.669+0000 7f22a3400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22a40cc510 0x7f22a415ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56086/0 (socket says 172.21.15.83:56086) 2024-04-25T00:12:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.669+0000 7f22a3400640 1 -- 172.21.15.83:0/3214184025 learned_addr learned my addr 172.21.15.83:0/3214184025 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.669+0000 7f22a9200640 1 --2- 172.21.15.83:0/3214184025 >> v2:172.21.15.86:3300/0 conn(0x7f22a415cf90 0x7f22a40ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.669+0000 7f22a3400640 1 -- 172.21.15.83:0/3214184025 >> v2:172.21.15.86:3300/0 conn(0x7f22a415cf90 msgr2=0x7f22a40ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.669+0000 7f22a3400640 1 --2- 172.21.15.83:0/3214184025 >> v2:172.21.15.86:3300/0 conn(0x7f22a415cf90 0x7f22a40ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.669+0000 7f22a3400640 1 -- 172.21.15.83:0/3214184025 >> v1:172.21.15.86:6789/0 conn(0x7f22a41599f0 legacy=0x7f22a415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.669+0000 7f22a3400640 1 -- 172.21.15.83:0/3214184025 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a406de80 con 0x7f22a40cc510 2024-04-25T00:12:57.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22a3400640 1 --2- 172.21.15.83:0/3214184025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a40cc510 0x7f22a415ca50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2290009870 tx=0x7f229001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599fbc56a591efd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:57.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22a2a00640 1 -- 172.21.15.83:0/3214184025 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22900b2070 con 0x7f22a40cc510 2024-04-25T00:12:57.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22a2a00640 1 -- 172.21.15.83:0/3214184025 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22900ad680 con 0x7f22a40cc510 2024-04-25T00:12:57.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22a2a00640 1 -- 172.21.15.83:0/3214184025 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22900ada50 con 0x7f22a40cc510 2024-04-25T00:12:57.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 -- 172.21.15.83:0/3214184025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a40cc510 msgr2=0x7f22a415ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 --2- 172.21.15.83:0/3214184025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a40cc510 0x7f22a415ca50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2290009870 tx=0x7f229001c490 comp rx=0 tx=0).stop 2024-04-25T00:12:57.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 -- 172.21.15.83:0/3214184025 shutdown_connections 2024-04-25T00:12:57.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 --2- 172.21.15.83:0/3214184025 >> v2:172.21.15.86:3300/0 conn(0x7f22a415cf90 0x7f22a40ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 --2- 172.21.15.83:0/3214184025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a40cc510 0x7f22a415ca50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 -- 172.21.15.83:0/3214184025 >> 172.21.15.83:0/3214184025 conn(0x7f22a40b1830 msgr2=0x7f22a40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:57.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 -- 172.21.15.83:0/3214184025 shutdown_connections 2024-04-25T00:12:57.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 -- 172.21.15.83:0/3214184025 wait complete. 2024-04-25T00:12:57.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 Processor -- start 2024-04-25T00:12:57.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 -- start start 2024-04-25T00:12:57.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a40cc510 0x7f22a414ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:57.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a41599f0 0x7f22a4152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:57.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a415cf90 0x7f22a4153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:57.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22a40cf5b0 con 0x7f22a41599f0 2024-04-25T00:12:57.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22a40cf430 con 0x7f22a415cf90 2024-04-25T00:12:57.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.673+0000 7f22ab200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f22a40cf730 con 0x7f22a40cc510 2024-04-25T00:12:57.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f22a3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a41599f0 0x7f22a4152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:57.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f22a3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a41599f0 0x7f22a4152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56094/0 (socket says 172.21.15.83:56094) 2024-04-25T00:12:57.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f22a3400640 1 -- 172.21.15.83:0/1582673701 learned_addr learned my addr 172.21.15.83:0/1582673701 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:57.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f22a9200640 1 --2- 172.21.15.83:0/1582673701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a415cf90 0x7f22a4153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:57.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f22a3e00640 1 --2- 172.21.15.83:0/1582673701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a40cc510 0x7f22a414ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:57.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f22a3400640 1 -- 172.21.15.83:0/1582673701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a40cc510 msgr2=0x7f22a414ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f22a3400640 1 --2- 172.21.15.83:0/1582673701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a40cc510 0x7f22a414ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f22a3400640 1 -- 172.21.15.83:0/1582673701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a415cf90 msgr2=0x7f22a4153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f22a3400640 1 --2- 172.21.15.83:0/1582673701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a415cf90 0x7f22a4153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f22a3400640 1 -- 172.21.15.83:0/1582673701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a415e2a0 con 0x7f22a41599f0 2024-04-25T00:12:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f22a3400640 1 --2- 172.21.15.83:0/1582673701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a41599f0 0x7f22a4152ed0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f229001a0c0 tx=0x7f229001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f229be00640 1 -- 172.21.15.83:0/1582673701 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22900bd070 con 0x7f22a41599f0 2024-04-25T00:12:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f229be00640 1 -- 172.21.15.83:0/1582673701 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22900adbf0 con 0x7f22a41599f0 2024-04-25T00:12:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f22ab200640 1 -- 172.21.15.83:0/1582673701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a415e580 con 0x7f22a41599f0 2024-04-25T00:12:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f229be00640 1 -- 172.21.15.83:0/1582673701 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22900b2040 con 0x7f22a41599f0 2024-04-25T00:12:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f22ab200640 1 -- 172.21.15.83:0/1582673701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a415eb40 con 0x7f22a41599f0 2024-04-25T00:12:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f22ab200640 1 -- 172.21.15.83:0/1582673701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a40737f0 con 0x7f22a41599f0 2024-04-25T00:12:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.677+0000 7f229be00640 1 -- 172.21.15.83:0/1582673701 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f22900c0070 con 0x7f22a41599f0 2024-04-25T00:12:57.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.681+0000 7f229be00640 1 --2- 172.21.15.83:0/1582673701 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22840678f0 0x7f2284069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:57.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.681+0000 7f229be00640 1 -- 172.21.15.83:0/1582673701 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f229011f350 con 0x7f22a41599f0 2024-04-25T00:12:57.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.681+0000 7f22a3e00640 1 --2- 172.21.15.83:0/1582673701 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22840678f0 0x7f2284069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:57.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.681+0000 7f22a3e00640 1 --2- 172.21.15.83:0/1582673701 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22840678f0 0x7f2284069db0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2294001d50 tx=0x7f2294009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.685+0000 7f229be00640 1 -- 172.21.15.83:0/1582673701 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2290016110 con 0x7f22a41599f0 2024-04-25T00:12:57.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.873+0000 7f22ab200640 1 -- 172.21.15.83:0/1582673701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22a404f5f0 con 0x7f22a41599f0 2024-04-25T00:12:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.873+0000 7f229be00640 1 -- 172.21.15.83:0/1582673701 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5181 (secure 0 0 0) 0x7f22900efc60 con 0x7f22a41599f0 2024-04-25T00:12:57.883 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:12:57.883 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2761747029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2761747029},{"type":"v1","addr":"172.21.15.83:6837","nonce":2761747029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2989939837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2989939837},{"type":"v1","addr":"172.21.15.86:6835","nonce":2989939837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2291906415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2291906415},{"type":"v1","addr":"172.21.15.83:6836","nonce":2291906415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:12:52.026135+0000","modified":"2024-04-25T00:12:55.045476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5814":{"gid":5814,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6834/1690820142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1690820142},{"type":"v1","addr":"172.21.15.86:6834","nonce":1690820142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5814,"qdb_cluster":[5814]},"id":3}]} 2024-04-25T00:12:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-04-25T00:12:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.873+0000 7f22ab200640 1 -- 172.21.15.83:0/1582673701 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22840678f0 msgr2=0x7f2284069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.873+0000 7f22ab200640 1 --2- 172.21.15.83:0/1582673701 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22840678f0 0x7f2284069db0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2294001d50 tx=0x7f2294009040 comp rx=0 tx=0).stop 2024-04-25T00:12:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.873+0000 7f22ab200640 1 -- 172.21.15.83:0/1582673701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a41599f0 msgr2=0x7f22a4152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.873+0000 7f22ab200640 1 --2- 172.21.15.83:0/1582673701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a41599f0 0x7f22a4152ed0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f229001a0c0 tx=0x7f229001acd0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.873+0000 7f22ab200640 1 -- 172.21.15.83:0/1582673701 shutdown_connections 2024-04-25T00:12:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.873+0000 7f22ab200640 1 --2- 172.21.15.83:0/1582673701 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22840678f0 0x7f2284069db0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.873+0000 7f22ab200640 1 --2- 172.21.15.83:0/1582673701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a415cf90 0x7f22a4153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.873+0000 7f22ab200640 1 --2- 172.21.15.83:0/1582673701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a41599f0 0x7f22a4152ed0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.873+0000 7f22ab200640 1 --2- 172.21.15.83:0/1582673701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a40cc510 0x7f22a414ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:57.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.873+0000 7f22ab200640 1 -- 172.21.15.83:0/1582673701 >> 172.21.15.83:0/1582673701 conn(0x7f22a40b1830 msgr2=0x7f22a4165720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:57.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.873+0000 7f22ab200640 1 -- 172.21.15.83:0/1582673701 shutdown_connections 2024-04-25T00:12:57.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:57.873+0000 7f22ab200640 1 -- 172.21.15.83:0/1582673701 wait complete. 2024-04-25T00:12:58.298 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:12:52.026135+0000', 'modified': '2024-04-25T00:12:55.045476+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5814}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5814': {'gid': 5814, 'name': 'd', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6834/1690820142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6832', 'nonce': 1690820142}, {'type': 'v1', 'addr': '172.21.15.86:6834', 'nonce': 1690820142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5814, 'qdb_cluster': [5814]} 2024-04-25T00:12:58.298 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:12:58.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T00:12:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4105e00640 1 Processor -- start 2024-04-25T00:12:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4105e00640 1 -- start start 2024-04-25T00:12:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4105e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f410015e110 0x7f41000ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4105e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f410006fdc0 0x7f41000ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4105e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41000d0ad0 con 0x7f410015e4f0 2024-04-25T00:12:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4105e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f41000d0db0 con 0x7f410006fdc0 2024-04-25T00:12:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4105e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f41000cc3b0 con 0x7f410015e110 2024-04-25T00:12:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4105400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f410006fdc0 0x7f41000ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4105400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f410006fdc0 0x7f41000ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56096/0 (socket says 172.21.15.83:56096) 2024-04-25T00:12:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4105400640 1 -- 172.21.15.83:0/1778538576 learned_addr learned my addr 172.21.15.83:0/1778538576 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4104a00640 1 --2- 172.21.15.83:0/1778538576 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f410015e110 0x7f41000ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4105400640 1 -- 172.21.15.83:0/1778538576 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f410015e110 msgr2=0x7f41000ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4105400640 1 --2- 172.21.15.83:0/1778538576 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f410015e110 0x7f41000ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4105400640 1 -- 172.21.15.83:0/1778538576 >> v1:172.21.15.86:6789/0 conn(0x7f410015e4f0 legacy=0x7f41000ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.433+0000 7f4105400640 1 -- 172.21.15.83:0/1778538576 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41000d0760 con 0x7f410006fdc0 2024-04-25T00:12:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.437+0000 7f4105400640 1 --2- 172.21.15.83:0/1778538576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410006fdc0 0x7f41000ccbb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009e50 tx=0x7f40f801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a056c807c1745a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.437+0000 7f40ffe00640 1 -- 172.21.15.83:0/1778538576 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f80a5660 con 0x7f410006fdc0 2024-04-25T00:12:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.437+0000 7f40ffe00640 1 -- 172.21.15.83:0/1778538576 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f80a5800 con 0x7f410006fdc0 2024-04-25T00:12:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.437+0000 7f40ffe00640 1 -- 172.21.15.83:0/1778538576 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f80a5bd0 con 0x7f410006fdc0 2024-04-25T00:12:58.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.437+0000 7f4105e00640 1 -- 172.21.15.83:0/1778538576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410006fdc0 msgr2=0x7f41000ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.437+0000 7f4105e00640 1 --2- 172.21.15.83:0/1778538576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410006fdc0 0x7f41000ccbb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009e50 tx=0x7f40f801c9a0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.437+0000 7f4105e00640 1 -- 172.21.15.83:0/1778538576 shutdown_connections 2024-04-25T00:12:58.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.437+0000 7f4105e00640 1 --2- 172.21.15.83:0/1778538576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410006fdc0 0x7f41000ccbb0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.437+0000 7f4105e00640 1 --2- 172.21.15.83:0/1778538576 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f410015e110 0x7f41000ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.437+0000 7f4105e00640 1 -- 172.21.15.83:0/1778538576 >> 172.21.15.83:0/1778538576 conn(0x7f41000d1bc0 msgr2=0x7f41000d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:58.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.437+0000 7f4105e00640 1 -- 172.21.15.83:0/1778538576 shutdown_connections 2024-04-25T00:12:58.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.437+0000 7f4105e00640 1 -- 172.21.15.83:0/1778538576 wait complete. 2024-04-25T00:12:58.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105e00640 1 Processor -- start 2024-04-25T00:12:58.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105e00640 1 -- start start 2024-04-25T00:12:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f410006fdc0 0x7f410015b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f410015e110 0x7f4100155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410015e4f0 0x7f4100155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41000cdcd0 con 0x7f410015e4f0 2024-04-25T00:12:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41000cdb50 con 0x7f410006fdc0 2024-04-25T00:12:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f41000cde50 con 0x7f410015e110 2024-04-25T00:12:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410015e4f0 0x7f4100155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410015e4f0 0x7f4100155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56098/0 (socket says 172.21.15.83:56098) 2024-04-25T00:12:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105400640 1 -- 172.21.15.83:0/2145701821 learned_addr learned my addr 172.21.15.83:0/2145701821 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105400640 1 -- 172.21.15.83:0/2145701821 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f410015e110 msgr2=0x7f4100155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105400640 1 --2- 172.21.15.83:0/2145701821 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f410015e110 0x7f4100155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105400640 1 -- 172.21.15.83:0/2145701821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f410006fdc0 msgr2=0x7f410015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105400640 1 --2- 172.21.15.83:0/2145701821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f410006fdc0 0x7f410015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105400640 1 -- 172.21.15.83:0/2145701821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410014cc40 con 0x7f410015e4f0 2024-04-25T00:12:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105400640 1 --2- 172.21.15.83:0/2145701821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410015e4f0 0x7f4100155a50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f40f801a5d0 tx=0x7f40f8019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f40fd600640 1 -- 172.21.15.83:0/2145701821 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f80bc020 con 0x7f410015e4f0 2024-04-25T00:12:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f40fd600640 1 -- 172.21.15.83:0/2145701821 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f80b8880 con 0x7f410015e4f0 2024-04-25T00:12:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f40fd600640 1 -- 172.21.15.83:0/2145701821 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f80a5660 con 0x7f410015e4f0 2024-04-25T00:12:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105e00640 1 -- 172.21.15.83:0/2145701821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f410014cf20 con 0x7f410015e4f0 2024-04-25T00:12:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.441+0000 7f4105e00640 1 -- 172.21.15.83:0/2145701821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41000c70b0 con 0x7f410015e4f0 2024-04-25T00:12:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f40fd600640 1 -- 172.21.15.83:0/2145701821 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f40f80bf020 con 0x7f410015e4f0 2024-04-25T00:12:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f40fd600640 1 --2- 172.21.15.83:0/2145701821 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f40dc0678c0 0x7f40dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f40fd600640 1 -- 172.21.15.83:0/2145701821 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f40f8120f80 con 0x7f410015e4f0 2024-04-25T00:12:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f4104a00640 1 --2- 172.21.15.83:0/2145701821 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f40dc0678c0 0x7f40dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f410a841000 1 Processor -- start 2024-04-25T00:12:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f410a841000 1 -- start start 2024-04-25T00:12:58.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f4104a00640 1 --2- 172.21.15.83:0/2145701821 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f40dc0678c0 0x7f40dc069d80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f40f00098f0 tx=0x7f40f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f410a841000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4230ef0 0x55ebc42312d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f410a841000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4231810 0x55ebc41e7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f410a841000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc41e7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f410a841000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ebc40355e0 con 0x55ebc41e7bc0 2024-04-25T00:12:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f410a841000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ebc3fbcc50 con 0x55ebc4230ef0 2024-04-25T00:12:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f410a841000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55ebc40315a0 con 0x55ebc4231810 2024-04-25T00:12:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f4105400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc41e7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f4105400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc41e7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56108/0 (socket says 172.21.15.83:56108) 2024-04-25T00:12:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f4105400640 1 -- 172.21.15.83:0/2270335702 learned_addr learned my addr 172.21.15.83:0/2270335702 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f4104a00640 1 --2- 172.21.15.83:0/2270335702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4230ef0 0x55ebc42312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f40f7e00640 1 --2- 172.21.15.83:0/2270335702 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4231810 0x55ebc41e7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f4105400640 1 -- 172.21.15.83:0/2270335702 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4231810 msgr2=0x55ebc41e7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f4105400640 1 --2- 172.21.15.83:0/2270335702 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4231810 0x55ebc41e7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f4105400640 1 -- 172.21.15.83:0/2270335702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4230ef0 msgr2=0x55ebc42312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f4105400640 1 --2- 172.21.15.83:0/2270335702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4230ef0 0x55ebc42312d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f4105400640 1 -- 172.21.15.83:0/2270335702 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebc3fbad10 con 0x55ebc41e7bc0 2024-04-25T00:12:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f4105400640 1 --2- 172.21.15.83:0/2270335702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc41e7fa0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f40f80072a0 tx=0x7f40f8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9245e5da0cd56099 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.445+0000 7f40f5600640 1 -- 172.21.15.83:0/2270335702 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f81091c0 con 0x55ebc41e7bc0 2024-04-25T00:12:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f40f5600640 1 -- 172.21.15.83:0/2270335702 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f80f5550 con 0x55ebc41e7bc0 2024-04-25T00:12:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f40f5600640 1 -- 172.21.15.83:0/2270335702 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f80f9d80 con 0x55ebc41e7bc0 2024-04-25T00:12:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 -- 172.21.15.83:0/2270335702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 msgr2=0x55ebc41e7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 --2- 172.21.15.83:0/2270335702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc41e7fa0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f40f80072a0 tx=0x7f40f8009510 comp rx=0 tx=0).stop 2024-04-25T00:12:58.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 -- 172.21.15.83:0/2270335702 shutdown_connections 2024-04-25T00:12:58.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 --2- 172.21.15.83:0/2270335702 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4231810 0x55ebc41e7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 --2- 172.21.15.83:0/2270335702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc41e7fa0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 --2- 172.21.15.83:0/2270335702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4230ef0 0x55ebc42312d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 -- 172.21.15.83:0/2270335702 >> 172.21.15.83:0/2270335702 conn(0x55ebc41e3b70 msgr2=0x55ebc41e5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:58.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 -- 172.21.15.83:0/2270335702 shutdown_connections 2024-04-25T00:12:58.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 -- 172.21.15.83:0/2270335702 wait complete. 2024-04-25T00:12:58.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 -- start start 2024-04-25T00:12:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 Processor -- start 2024-04-25T00:12:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc4215f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4230ef0 0x55ebc4216460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 0x55ebc42169a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ebc40355e0 con 0x55ebc41e7bc0 2024-04-25T00:12:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ebc3fef520 con 0x55ebc4231810 2024-04-25T00:12:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55ebc3ff9670 con 0x55ebc4230ef0 2024-04-25T00:12:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 10 client.0 _ll_register_callbacks cb 0x55ebc41d25c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:12:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f4104a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc4215f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f4104a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc4215f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56118/0 (socket says 172.21.15.83:56118) 2024-04-25T00:12:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f4104a00640 1 -- 172.21.15.83:0/47507566 learned_addr learned my addr 172.21.15.83:0/47507566 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f40f7e00640 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4230ef0 0x55ebc4216460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f4105400640 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 0x55ebc42169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f4104a00640 1 -- 172.21.15.83:0/47507566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4230ef0 msgr2=0x55ebc4216460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f4104a00640 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4230ef0 0x55ebc4216460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f4104a00640 1 -- 172.21.15.83:0/47507566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 msgr2=0x55ebc42169a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f4104a00640 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 0x55ebc42169a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f4104a00640 1 -- 172.21.15.83:0/47507566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebc3fcd7b0 con 0x55ebc41e7bc0 2024-04-25T00:12:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f40f7e00640 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4230ef0 0x55ebc4216460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f4105400640 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 0x55ebc42169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f4104a00640 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc4215f20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f40f000dfd0 tx=0x7f40f00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddfda75ef810fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f40ee000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:12:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f40ee000640 1 -- 172.21.15.83:0/47507566 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0024020 con 0x55ebc41e7bc0 2024-04-25T00:12:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f410a841000 1 -- 172.21.15.83:0/47507566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40f80f5550 con 0x55ebc41e7bc0 2024-04-25T00:12:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.449+0000 7f40ee000640 1 -- 172.21.15.83:0/47507566 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f40f000f040 con 0x55ebc41e7bc0 2024-04-25T00:12:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f40ee000640 1 -- 172.21.15.83:0/47507566 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f40f002e050 con 0x55ebc41e7bc0 2024-04-25T00:12:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 10 client.5931 fetch_fsmap learned FSMap version 27 2024-04-25T00:12:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 1 -- 172.21.15.83:0/47507566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55ebc3fbcea0 con 0x55ebc41e7bc0 2024-04-25T00:12:58.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f40ee000640 1 -- 172.21.15.83:0/47507566 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f80f5550 con 0x55ebc41e7bc0 2024-04-25T00:12:58.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f40ee000640 1 -- 172.21.15.83:0/47507566 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7f40f0031020 con 0x55ebc41e7bc0 2024-04-25T00:12:58.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 10 client.5931 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:12:58.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 10 client.5931 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T00:12:58.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 20 client.5931 populate_metadata read hostname 'smithi083' 2024-04-25T00:12:58.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.86:6832/1690820142,v1:172.21.15.86:6834/1690820142] conn(0x55ebc4240f90 0x55ebc4243380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 4 client.5931 mds_command: new command op to 5814 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:12:58.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 1 -- 172.21.15.83:0/47507566 --> [v2:172.21.15.86:6832/1690820142,v1:172.21.15.86:6834/1690820142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebc3fd0ab0 con 0x55ebc4240f90 2024-04-25T00:12:58.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f40f7e00640 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.86:6832/1690820142,v1:172.21.15.86:6834/1690820142] conn(0x55ebc4240f90 0x55ebc4243380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f40f7e00640 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.86:6832/1690820142,v1:172.21.15.86:6834/1690820142] conn(0x55ebc4240f90 0x55ebc4243380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c8c069007d74d36 server_cookie=205fa640cdb9b4a6 in_seq=0 out_seq=0 2024-04-25T00:12:58.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f40ee000640 10 client.5931 ms_handle_connect on v2:172.21.15.86:6832/1690820142 2024-04-25T00:12:58.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f40ee000640 1 -- 172.21.15.83:0/47507566 <== mds.0 v2:172.21.15.86:6832/1690820142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ebc3fd0ab0 con 0x55ebc4240f90 2024-04-25T00:12:58.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f40ee000640 10 client.5931 handle_command_reply: tid=0 2024-04-25T00:12:58.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 1 client.5931 shutdown 2024-04-25T00:12:58.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 1 -- 172.21.15.83:0/47507566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 msgr2=0x55ebc4215f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc4215f20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f40f000dfd0 tx=0x7f40f00046d0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 1 -- 172.21.15.83:0/47507566 shutdown_connections 2024-04-25T00:12:58.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.86:6832/1690820142,v1:172.21.15.86:6834/1690820142] conn(0x55ebc4240f90 0x55ebc4243380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4230ef0 0x55ebc4216460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 0x55ebc42169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f40ee000640 0 client.5931 ms_handle_reset on v2:172.21.15.86:6832/1690820142 2024-04-25T00:12:58.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 1 --2- 172.21.15.83:0/47507566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc4215f20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 1 -- 172.21.15.83:0/47507566 >> 172.21.15.83:0/47507566 conn(0x55ebc41e3b70 msgr2=0x55ebc41e67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:58.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 1 -- 172.21.15.83:0/47507566 shutdown_connections 2024-04-25T00:12:58.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 1 -- 172.21.15.83:0/47507566 wait complete. 2024-04-25T00:12:58.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.453+0000 7f410a841000 20 client.5931 trim_cache size 0 max 16384 2024-04-25T00:12:58.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f410a841000 1 Processor -- start 2024-04-25T00:12:58.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f410a841000 1 -- start start 2024-04-25T00:12:58.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f410a841000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc41e7bc0 0x55ebc429c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f410a841000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc4230ef0 0x55ebc429ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f410a841000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 0x55ebc429cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f410a841000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41002027b0 con 0x55ebc4230ef0 2024-04-25T00:12:58.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f410a841000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4100158980 con 0x55ebc4231810 2024-04-25T00:12:58.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f410a841000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f410014c7f0 con 0x55ebc41e7bc0 2024-04-25T00:12:58.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc4230ef0 0x55ebc429ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc4230ef0 0x55ebc429ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56120/0 (socket says 172.21.15.83:56120) 2024-04-25T00:12:58.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f40f7e00640 1 -- 172.21.15.83:0/4157754256 learned_addr learned my addr 172.21.15.83:0/4157754256 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:58.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f40f7e00640 1 -- 172.21.15.83:0/4157754256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc41e7bc0 msgr2=0x55ebc429c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:58.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f40f7e00640 1 --2- 172.21.15.83:0/4157754256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc41e7bc0 0x55ebc429c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f40f7e00640 1 -- 172.21.15.83:0/4157754256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 msgr2=0x55ebc429cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:58.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f40f7e00640 1 --2- 172.21.15.83:0/4157754256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 0x55ebc429cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f40f7e00640 1 -- 172.21.15.83:0/4157754256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebc3fbde80 con 0x55ebc4230ef0 2024-04-25T00:12:58.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f40f7e00640 1 --2- 172.21.15.83:0/4157754256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc4230ef0 0x55ebc429ca40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f40e8007fd0 tx=0x7f40e8007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903154f7ecbfd723 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:58.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f40f4c00640 1 -- 172.21.15.83:0/4157754256 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8019570 con 0x55ebc4230ef0 2024-04-25T00:12:58.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f40f4c00640 1 -- 172.21.15.83:0/4157754256 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e8019710 con 0x55ebc4230ef0 2024-04-25T00:12:58.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f40f4c00640 1 -- 172.21.15.83:0/4157754256 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8007030 con 0x55ebc4230ef0 2024-04-25T00:12:58.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f410a841000 1 -- 172.21.15.83:0/4157754256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc4230ef0 msgr2=0x55ebc429ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.473+0000 7f410a841000 1 --2- 172.21.15.83:0/4157754256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc4230ef0 0x55ebc429ca40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f40e8007fd0 tx=0x7f40e8007d90 comp rx=0 tx=0).stop 2024-04-25T00:12:58.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 -- 172.21.15.83:0/4157754256 shutdown_connections 2024-04-25T00:12:58.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 --2- 172.21.15.83:0/4157754256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 0x55ebc429cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 --2- 172.21.15.83:0/4157754256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc4230ef0 0x55ebc429ca40 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 --2- 172.21.15.83:0/4157754256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc41e7bc0 0x55ebc429c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 -- 172.21.15.83:0/4157754256 >> 172.21.15.83:0/4157754256 conn(0x55ebc41e3b70 msgr2=0x55ebc423e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 -- 172.21.15.83:0/4157754256 shutdown_connections 2024-04-25T00:12:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 -- 172.21.15.83:0/4157754256 wait complete. 2024-04-25T00:12:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 -- start start 2024-04-25T00:12:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 Processor -- start 2024-04-25T00:12:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc42a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4230ef0 0x55ebc42a1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 0x55ebc42a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ebc3fbd640 con 0x55ebc41e7bc0 2024-04-25T00:12:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ebc3ffe810 con 0x55ebc4231810 2024-04-25T00:12:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55ebc3ffe3c0 con 0x55ebc4230ef0 2024-04-25T00:12:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 10 client.0 _ll_register_callbacks cb 0x7f40f00090e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:12:58.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f4104a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc42a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f4104a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc42a0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56136/0 (socket says 172.21.15.83:56136) 2024-04-25T00:12:58.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f4104a00640 1 -- 172.21.15.83:0/4082349495 learned_addr learned my addr 172.21.15.83:0/4082349495 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:58.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f40f7e00640 1 --2- 172.21.15.83:0/4082349495 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4230ef0 0x55ebc42a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f4105400640 1 --2- 172.21.15.83:0/4082349495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 0x55ebc42a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f4104a00640 1 -- 172.21.15.83:0/4082349495 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4230ef0 msgr2=0x55ebc42a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:58.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f4104a00640 1 --2- 172.21.15.83:0/4082349495 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4230ef0 0x55ebc42a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f4104a00640 1 -- 172.21.15.83:0/4082349495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 msgr2=0x55ebc42a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:58.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f4104a00640 1 --2- 172.21.15.83:0/4082349495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 0x55ebc42a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f4104a00640 1 -- 172.21.15.83:0/4082349495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebc4002760 con 0x55ebc41e7bc0 2024-04-25T00:12:58.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f40f7e00640 1 --2- 172.21.15.83:0/4082349495 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4230ef0 0x55ebc42a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:58.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f4104a00640 1 --2- 172.21.15.83:0/4082349495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc42a0f50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009530 tx=0x7f40f0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5949dc0c9e50b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:58.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f40ee000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:12:58.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f40ee000640 1 -- 172.21.15.83:0/4082349495 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0024040 con 0x55ebc41e7bc0 2024-04-25T00:12:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f40ee000640 1 -- 172.21.15.83:0/4082349495 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f40f0020650 con 0x55ebc41e7bc0 2024-04-25T00:12:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 -- 172.21.15.83:0/4082349495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e8019710 con 0x55ebc41e7bc0 2024-04-25T00:12:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f40ee000640 1 -- 172.21.15.83:0/4082349495 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f40f003e050 con 0x55ebc41e7bc0 2024-04-25T00:12:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 10 client.5937 fetch_fsmap learned FSMap version 27 2024-04-25T00:12:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 -- 172.21.15.83:0/4082349495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55ebc408e9e0 con 0x55ebc41e7bc0 2024-04-25T00:12:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f40ee000640 1 -- 172.21.15.83:0/4082349495 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e8019710 con 0x55ebc41e7bc0 2024-04-25T00:12:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f40ee000640 1 -- 172.21.15.83:0/4082349495 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7f40f0041020 con 0x55ebc41e7bc0 2024-04-25T00:12:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 10 client.5937 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:12:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 10 client.5937 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T00:12:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 20 client.5937 populate_metadata read hostname 'smithi083' 2024-04-25T00:12:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 --2- 172.21.15.83:0/4082349495 >> [v2:172.21.15.86:6832/1690820142,v1:172.21.15.86:6834/1690820142] conn(0x55ebc429de10 0x55ebc429e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 4 client.5937 mds_command: new command op to 5814 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:12:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.477+0000 7f410a841000 1 -- 172.21.15.83:0/4082349495 --> [v2:172.21.15.86:6832/1690820142,v1:172.21.15.86:6834/1690820142] -- command(tid 0: {"prefix": "status"}) -- 0x55ebc4042b20 con 0x55ebc429de10 2024-04-25T00:12:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f40f7e00640 1 --2- 172.21.15.83:0/4082349495 >> [v2:172.21.15.86:6832/1690820142,v1:172.21.15.86:6834/1690820142] conn(0x55ebc429de10 0x55ebc429e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f40f7e00640 1 --2- 172.21.15.83:0/4082349495 >> [v2:172.21.15.86:6832/1690820142,v1:172.21.15.86:6834/1690820142] conn(0x55ebc429de10 0x55ebc429e1f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ba66c2a277561cd server_cookie=68b9ffe8f3b9e805 in_seq=0 out_seq=0 2024-04-25T00:12:58.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f40ee000640 10 client.5937 ms_handle_connect on v2:172.21.15.86:6832/1690820142 2024-04-25T00:12:58.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f40ee000640 1 -- 172.21.15.83:0/4082349495 <== mds.0 v2:172.21.15.86:6832/1690820142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55ebc4042b20 con 0x55ebc429de10 2024-04-25T00:12:58.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f40ee000640 10 client.5937 handle_command_reply: tid=0 2024-04-25T00:12:58.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f410a841000 1 client.5937 shutdown 2024-04-25T00:12:58.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f410a841000 1 -- 172.21.15.83:0/4082349495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 msgr2=0x55ebc42a0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f410a841000 1 --2- 172.21.15.83:0/4082349495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc42a0f50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009530 tx=0x7f40f0012000 comp rx=0 tx=0).stop 2024-04-25T00:12:58.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f410a841000 1 -- 172.21.15.83:0/4082349495 shutdown_connections 2024-04-25T00:12:58.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f410a841000 1 --2- 172.21.15.83:0/4082349495 >> [v2:172.21.15.86:6832/1690820142,v1:172.21.15.86:6834/1690820142] conn(0x55ebc429de10 0x55ebc429e1f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f410a841000 1 --2- 172.21.15.83:0/4082349495 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ebc4230ef0 0x55ebc42a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f410a841000 1 --2- 172.21.15.83:0/4082349495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebc4231810 0x55ebc42a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f410a841000 1 --2- 172.21.15.83:0/4082349495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebc41e7bc0 0x55ebc42a0f50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f40ee000640 0 client.5937 ms_handle_reset on v2:172.21.15.86:6832/1690820142 2024-04-25T00:12:58.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f410a841000 1 -- 172.21.15.83:0/4082349495 >> 172.21.15.83:0/4082349495 conn(0x55ebc41e3b70 msgr2=0x55ebc422f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:58.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f410a841000 1 -- 172.21.15.83:0/4082349495 shutdown_connections 2024-04-25T00:12:58.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f410a841000 1 -- 172.21.15.83:0/4082349495 wait complete. 2024-04-25T00:12:58.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f410a841000 20 client.5937 trim_cache size 0 max 16384 2024-04-25T00:12:58.487 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:12:58.487 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "d58ae83a-ad72-4e18-824a-86e779979c35", 2024-04-25T00:12:58.487 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-25T00:12:58.487 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 5814, 2024-04-25T00:12:58.487 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T00:12:58.488 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T00:12:58.488 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T00:12:58.488 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.8300864800000003, 2024-04-25T00:12:58.488 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 27, 2024-04-25T00:12:58.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 49, 2024-04-25T00:12:58.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 49, 2024-04-25T00:12:58.488 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 10.757052304 2024-04-25T00:12:58.488 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:12:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f4105e00640 1 -- 172.21.15.83:0/2145701821 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f40dc0678c0 msgr2=0x7f40dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f4105e00640 1 --2- 172.21.15.83:0/2145701821 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f40dc0678c0 0x7f40dc069d80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f40f00098f0 tx=0x7f40f0008040 comp rx=0 tx=0).stop 2024-04-25T00:12:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f4105e00640 1 -- 172.21.15.83:0/2145701821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410015e4f0 msgr2=0x7f4100155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f4105e00640 1 --2- 172.21.15.83:0/2145701821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410015e4f0 0x7f4100155a50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f40f801a5d0 tx=0x7f40f8019740 comp rx=0 tx=0).stop 2024-04-25T00:12:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f4105e00640 1 -- 172.21.15.83:0/2145701821 shutdown_connections 2024-04-25T00:12:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f4105e00640 1 --2- 172.21.15.83:0/2145701821 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f40dc0678c0 0x7f40dc069d80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f4105e00640 1 --2- 172.21.15.83:0/2145701821 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f410015e110 0x7f4100155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f4105e00640 1 --2- 172.21.15.83:0/2145701821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410015e4f0 0x7f4100155a50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f4105e00640 1 --2- 172.21.15.83:0/2145701821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f410006fdc0 0x7f410015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f4105e00640 1 -- 172.21.15.83:0/2145701821 >> 172.21.15.83:0/2145701821 conn(0x7f41000d1bc0 msgr2=0x7f41000d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:58.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f4105e00640 1 -- 172.21.15.83:0/2145701821 shutdown_connections 2024-04-25T00:12:58.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.481+0000 7f4105e00640 1 -- 172.21.15.83:0/2145701821 wait complete. 2024-04-25T00:12:58.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:12:58.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.633+0000 7fbcaa600640 1 Processor -- start 2024-04-25T00:12:58.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.637+0000 7fbcaa600640 1 -- start start 2024-04-25T00:12:58.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.637+0000 7fbcaa600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbca4144470 0x7fbca415f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.637+0000 7fbcaa600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbca40b3f40 con 0x7fbca40b34f0 2024-04-25T00:12:58.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.637+0000 7fbcaa600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbca40b4220 con 0x7fbca40d01e0 2024-04-25T00:12:58.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.637+0000 7fbcaa600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbca4144850 con 0x7fbca4144470 2024-04-25T00:12:58.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.637+0000 7fbca9200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbca4144470 0x7fbca415f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.637+0000 7fbca3e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbca40b34f0 0x7fbca40cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42272/0 (socket says 172.21.15.83:42272) 2024-04-25T00:12:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.637+0000 7fbca3e00640 1 -- 172.21.15.83:0/1435453290 learned_addr learned my addr 172.21.15.83:0/1435453290 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:12:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.637+0000 7fbca9200640 1 -- 172.21.15.83:0/1435453290 >> v1:172.21.15.86:6789/0 conn(0x7fbca40d01e0 legacy=0x7fbca40ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.637+0000 7fbca9200640 1 -- 172.21.15.83:0/1435453290 >> v1:172.21.15.83:6789/0 conn(0x7fbca40b34f0 legacy=0x7fbca40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.637+0000 7fbca9200640 1 -- 172.21.15.83:0/1435453290 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca415e5e0 con 0x7fbca4144470 2024-04-25T00:12:58.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.637+0000 7fbca3400640 1 -- 172.21.15.83:0/1435453290 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2913016934 0 0) 0x7fbca40b3f40 con 0x7fbca40b34f0 2024-04-25T00:12:58.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.641+0000 7fbca9200640 1 --2- 172.21.15.83:0/1435453290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca4144470 0x7fbca415f0c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbc980097c0 tx=0x7fbc9801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb1a27ca78595a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.641+0000 7fbca3400640 1 -- 172.21.15.83:0/1435453290 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc980b0070 con 0x7fbca4144470 2024-04-25T00:12:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.641+0000 7fbca3400640 1 -- 172.21.15.83:0/1435453290 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc980ab600 con 0x7fbca4144470 2024-04-25T00:12:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.641+0000 7fbca3400640 1 -- 172.21.15.83:0/1435453290 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc980ab9d0 con 0x7fbca4144470 2024-04-25T00:12:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.641+0000 7fbcaa600640 1 -- 172.21.15.83:0/1435453290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca4144470 msgr2=0x7fbca415f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.641+0000 7fbcaa600640 1 --2- 172.21.15.83:0/1435453290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca4144470 0x7fbca415f0c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbc980097c0 tx=0x7fbc9801c3e0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.641+0000 7fbcaa600640 1 -- 172.21.15.83:0/1435453290 shutdown_connections 2024-04-25T00:12:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.641+0000 7fbcaa600640 1 --2- 172.21.15.83:0/1435453290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca4144470 0x7fbca415f0c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.641+0000 7fbcaa600640 1 -- 172.21.15.83:0/1435453290 >> 172.21.15.83:0/1435453290 conn(0x7fbca406b3b0 msgr2=0x7fbca406b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.641+0000 7fbcaa600640 1 -- 172.21.15.83:0/1435453290 shutdown_connections 2024-04-25T00:12:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.641+0000 7fbcaa600640 1 -- 172.21.15.83:0/1435453290 wait complete. 2024-04-25T00:12:58.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.641+0000 7fbcaa600640 1 Processor -- start 2024-04-25T00:12:58.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.641+0000 7fbcaa600640 1 -- start start 2024-04-25T00:12:58.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbcaa600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca40b34f0 0x7fbca40a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbcaa600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbca40d01e0 0x7fbca40a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbcaa600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca4144470 0x7fbca4140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbcaa600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbca4168870 con 0x7fbca4144470 2024-04-25T00:12:58.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbcaa600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbca41686f0 con 0x7fbca40d01e0 2024-04-25T00:12:58.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbcaa600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbca41689f0 con 0x7fbca40b34f0 2024-04-25T00:12:58.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca4144470 0x7fbca4140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca4144470 0x7fbca4140320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56156/0 (socket says 172.21.15.83:56156) 2024-04-25T00:12:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca9c00640 1 -- 172.21.15.83:0/3612013648 learned_addr learned my addr 172.21.15.83:0/3612013648 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca3e00640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbca40d01e0 0x7fbca40a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca9200640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca40b34f0 0x7fbca40a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca9c00640 1 -- 172.21.15.83:0/3612013648 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca40b34f0 msgr2=0x7fbca40a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca9c00640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca40b34f0 0x7fbca40a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca9c00640 1 -- 172.21.15.83:0/3612013648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbca40d01e0 msgr2=0x7fbca40a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca9c00640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbca40d01e0 0x7fbca40a9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca9c00640 1 -- 172.21.15.83:0/3612013648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca4162060 con 0x7fbca4144470 2024-04-25T00:12:58.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca3e00640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbca40d01e0 0x7fbca40a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:58.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca9c00640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca4144470 0x7fbca4140320 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00c7a0 tx=0x7fbc9c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:58.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca0c00640 1 -- 172.21.15.83:0/3612013648 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c003b80 con 0x7fbca4144470 2024-04-25T00:12:58.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca0c00640 1 -- 172.21.15.83:0/3612013648 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc9c003d20 con 0x7fbca4144470 2024-04-25T00:12:58.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca0c00640 1 -- 172.21.15.83:0/3612013648 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c00aa90 con 0x7fbca4144470 2024-04-25T00:12:58.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbcaa600640 1 -- 172.21.15.83:0/3612013648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca4162d80 con 0x7fbca4144470 2024-04-25T00:12:58.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbcaa600640 1 -- 172.21.15.83:0/3612013648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca4140860 con 0x7fbca4144470 2024-04-25T00:12:58.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbcaa600640 1 -- 172.21.15.83:0/3612013648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca404f510 con 0x7fbca4144470 2024-04-25T00:12:58.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.645+0000 7fbca0c00640 1 -- 172.21.15.83:0/3612013648 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbc9c00ac30 con 0x7fbca4144470 2024-04-25T00:12:58.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.649+0000 7fbca0c00640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc840678c0 0x7fbc84069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.649+0000 7fbca0c00640 1 -- 172.21.15.83:0/3612013648 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fbc9c085c90 con 0x7fbca4144470 2024-04-25T00:12:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.649+0000 7fbca9200640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc840678c0 0x7fbc84069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.649+0000 7fbca9200640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc840678c0 0x7fbc84069d80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbc980ad000 tx=0x7fbc980a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:58.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.653+0000 7fbca0c00640 1 -- 172.21.15.83:0/3612013648 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbc9c056630 con 0x7fbca4144470 2024-04-25T00:12:58.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.845+0000 7fbcaa600640 1 -- 172.21.15.83:0/3612013648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbca406ceb0 con 0x7fbca4144470 2024-04-25T00:12:58.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.845+0000 7fbca0c00640 1 -- 172.21.15.83:0/3612013648 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7fbc9c055f30 con 0x7fbca4144470 2024-04-25T00:12:58.848 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:12:58.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.849+0000 7fbcaa600640 1 -- 172.21.15.83:0/3612013648 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc840678c0 msgr2=0x7fbc84069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.849+0000 7fbcaa600640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc840678c0 0x7fbc84069d80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbc980ad000 tx=0x7fbc980a6000 comp rx=0 tx=0).stop 2024-04-25T00:12:58.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.849+0000 7fbcaa600640 1 -- 172.21.15.83:0/3612013648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca4144470 msgr2=0x7fbca4140320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.849+0000 7fbcaa600640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca4144470 0x7fbca4140320 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00c7a0 tx=0x7fbc9c00cc60 comp rx=0 tx=0).stop 2024-04-25T00:12:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.849+0000 7fbcaa600640 1 -- 172.21.15.83:0/3612013648 shutdown_connections 2024-04-25T00:12:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.849+0000 7fbcaa600640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc840678c0 0x7fbc84069d80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.849+0000 7fbcaa600640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca4144470 0x7fbca4140320 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.849+0000 7fbcaa600640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbca40d01e0 0x7fbca40a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.849+0000 7fbcaa600640 1 --2- 172.21.15.83:0/3612013648 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca40b34f0 0x7fbca40a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.849+0000 7fbcaa600640 1 -- 172.21.15.83:0/3612013648 >> 172.21.15.83:0/3612013648 conn(0x7fbca406b3b0 msgr2=0x7fbca415d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.849+0000 7fbcaa600640 1 -- 172.21.15.83:0/3612013648 shutdown_connections 2024-04-25T00:12:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:58.849+0000 7fbcaa600640 1 -- 172.21.15.83:0/3612013648 wait complete. 2024-04-25T00:12:58.869 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:12:58.869 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:12:58.869 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-25T00:12:58.869 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:12:58.869 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:12:58.869 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:12:58.869 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:12:58.870 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:12:58.870 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:12:58.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:58.870 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-25T00:12:58.879 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:12:58.879 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:12:58.879 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:12:58.879 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:58.879 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-25T00:12:58.879 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:58.879 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:58.879 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:58.879 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-25T00:12:58.879 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:58.880 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:58.880 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-25T00:12:58.880 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:58.880 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:58.880 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:12:58.880 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:58.880 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:12:58.880 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42355sec preferred_lft 42355sec 2024-04-25T00:12:58.880 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-25T00:12:58.880 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:58.880 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:12:58.880 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:8d:d4:7d:b9 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:58.880 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:12:58.880 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:58.881 INFO:teuthology.orchestra.run.smithi083.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:12:58.881 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:58.881 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:12:58.881 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:58.881 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:12:58.881 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:58.881 INFO:teuthology.orchestra.run.smithi083.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:12:58.881 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:12:58.881 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:12:58.881 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:58.881 INFO:teuthology.orchestra.run.smithi083.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:12:58.881 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:12:58.881 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:12:58.881 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:58.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:58.882 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:12:58.942 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-25T00:12:58.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:58.944 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-25T00:12:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-25T00:12:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:12:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:12:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:12:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:12:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:12:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:12:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:12:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:12:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:12:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:12:58.997 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:12:58.997 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:12:58.997 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:12:58.997 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:12:58.997 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:12:58.997 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:12:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:12:59.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:12:59.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:59.134 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:12:59.143 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:12:59.143 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:12:59.143 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:12:59.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:59.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-25T00:12:59.205 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:12:59.205 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:12:59.257 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:12:59.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:12:59.257 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:12:59.308 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T00:12:59.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:59.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:12:59.396 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:12:59.396 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:12:59.408 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:12:59.408 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:12:59.408 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:12:59.409 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:12:59.409 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:12:59.412 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:12:59.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:59.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:12:59.433 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:12:59.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:12:59.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:12:59.548 INFO:teuthology.orchestra.run.smithi083.stdout:client5949 2024-04-25T00:12:59.549 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:12:59.549 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:12:59.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:12:59.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f8401200640 1 Processor -- start 2024-04-25T00:12:59.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f8401200640 1 -- start start 2024-04-25T00:12:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f8401200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83fc1599f0 0x7f83fc0cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f8401200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f83fc15c340 0x7f83fc15c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f8401200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83fc0ccc60 con 0x7f83fc15ccf0 2024-04-25T00:12:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f8401200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f83fc159dd0 con 0x7f83fc1599f0 2024-04-25T00:12:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f8401200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f83fc06dba0 con 0x7f83fc15c340 2024-04-25T00:12:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f83f9e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83fc1599f0 0x7f83fc0cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f83f9e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83fc1599f0 0x7f83fc0cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56168/0 (socket says 172.21.15.83:56168) 2024-04-25T00:12:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f83f9e00640 1 -- 172.21.15.83:0/3115562269 learned_addr learned my addr 172.21.15.83:0/3115562269 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f83f1400640 1 --2- 172.21.15.83:0/3115562269 >> v2:172.21.15.86:3300/0 conn(0x7f83fc15c340 0x7f83fc15c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f83f9e00640 1 -- 172.21.15.83:0/3115562269 >> v2:172.21.15.86:3300/0 conn(0x7f83fc15c340 msgr2=0x7f83fc15c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f83f9e00640 1 --2- 172.21.15.83:0/3115562269 >> v2:172.21.15.86:3300/0 conn(0x7f83fc15c340 0x7f83fc15c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f83f9e00640 1 -- 172.21.15.83:0/3115562269 >> v1:172.21.15.86:6789/0 conn(0x7f83fc15ccf0 legacy=0x7f83fc15f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f83f9e00640 1 -- 172.21.15.83:0/3115562269 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc06de80 con 0x7f83fc1599f0 2024-04-25T00:12:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.741+0000 7f83f1400640 1 --2- 172.21.15.83:0/3115562269 >> v2:172.21.15.86:3300/0 conn(0x7f83fc15c340 0x7f83fc15c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:12:59.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.745+0000 7f83f9e00640 1 --2- 172.21.15.83:0/3115562269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83fc1599f0 0x7f83fc0cc510 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009870 tx=0x7f83ec01ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6232fd7bfa7566d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:59.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.745+0000 7f83f9400640 1 -- 172.21.15.83:0/3115562269 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0b0070 con 0x7f83fc1599f0 2024-04-25T00:12:59.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.745+0000 7f83f9400640 1 -- 172.21.15.83:0/3115562269 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ec0ab600 con 0x7f83fc1599f0 2024-04-25T00:12:59.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.745+0000 7f83f9400640 1 -- 172.21.15.83:0/3115562269 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0ab9d0 con 0x7f83fc1599f0 2024-04-25T00:12:59.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.745+0000 7f8401200640 1 -- 172.21.15.83:0/3115562269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83fc1599f0 msgr2=0x7f83fc0cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:59.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.745+0000 7f8401200640 1 --2- 172.21.15.83:0/3115562269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83fc1599f0 0x7f83fc0cc510 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009870 tx=0x7f83ec01ccd0 comp rx=0 tx=0).stop 2024-04-25T00:12:59.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.745+0000 7f8401200640 1 -- 172.21.15.83:0/3115562269 shutdown_connections 2024-04-25T00:12:59.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.745+0000 7f8401200640 1 --2- 172.21.15.83:0/3115562269 >> v2:172.21.15.86:3300/0 conn(0x7f83fc15c340 0x7f83fc15c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:59.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.745+0000 7f8401200640 1 --2- 172.21.15.83:0/3115562269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83fc1599f0 0x7f83fc0cc510 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:59.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.745+0000 7f8401200640 1 -- 172.21.15.83:0/3115562269 >> 172.21.15.83:0/3115562269 conn(0x7f83fc0b1830 msgr2=0x7f83fc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:59.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.745+0000 7f8401200640 1 -- 172.21.15.83:0/3115562269 shutdown_connections 2024-04-25T00:12:59.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.745+0000 7f8401200640 1 -- 172.21.15.83:0/3115562269 wait complete. 2024-04-25T00:12:59.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f8401200640 1 Processor -- start 2024-04-25T00:12:59.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f8401200640 1 -- start start 2024-04-25T00:12:59.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f8401200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83fc1599f0 0x7f83fc15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:59.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f8401200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83fc15c340 0x7f83fc151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:59.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f8401200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc15ccf0 0x7f83fc151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:59.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f8401200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc0cf870 con 0x7f83fc15c340 2024-04-25T00:12:59.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f8401200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc0cf6f0 con 0x7f83fc15ccf0 2024-04-25T00:12:59.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f8401200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f83fc168cd0 con 0x7f83fc1599f0 2024-04-25T00:12:59.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f83f1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83fc15c340 0x7f83fc151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:59.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f83f1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83fc15c340 0x7f83fc151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56174/0 (socket says 172.21.15.83:56174) 2024-04-25T00:12:59.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f83f1400640 1 -- 172.21.15.83:0/1760115865 learned_addr learned my addr 172.21.15.83:0/1760115865 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:12:59.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f83f9e00640 1 --2- 172.21.15.83:0/1760115865 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83fc1599f0 0x7f83fc15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:59.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f83fa800640 1 --2- 172.21.15.83:0/1760115865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc15ccf0 0x7f83fc151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:59.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f83f1400640 1 -- 172.21.15.83:0/1760115865 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83fc1599f0 msgr2=0x7f83fc15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:59.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f83f1400640 1 --2- 172.21.15.83:0/1760115865 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83fc1599f0 0x7f83fc15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:59.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f83f1400640 1 -- 172.21.15.83:0/1760115865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc15ccf0 msgr2=0x7f83fc151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:59.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f83f1400640 1 --2- 172.21.15.83:0/1760115865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc15ccf0 0x7f83fc151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:59.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f83f1400640 1 -- 172.21.15.83:0/1760115865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc165600 con 0x7f83fc15c340 2024-04-25T00:12:59.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f83f1400640 1 --2- 172.21.15.83:0/1760115865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83fc15c340 0x7f83fc151010 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f83e400c7c0 tx=0x7f83e400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:59.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f83f2a00640 1 -- 172.21.15.83:0/1760115865 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4007a30 con 0x7f83fc15c340 2024-04-25T00:12:59.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f83f2a00640 1 -- 172.21.15.83:0/1760115865 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e4007bd0 con 0x7f83fc15c340 2024-04-25T00:12:59.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f8401200640 1 -- 172.21.15.83:0/1760115865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83fc1658e0 con 0x7f83fc15c340 2024-04-25T00:12:59.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f83f2a00640 1 -- 172.21.15.83:0/1760115865 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e40113f0 con 0x7f83fc15c340 2024-04-25T00:12:59.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f8401200640 1 -- 172.21.15.83:0/1760115865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83fc165ea0 con 0x7f83fc15c340 2024-04-25T00:12:59.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.749+0000 7f8401200640 1 -- 172.21.15.83:0/1760115865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83fc0737f0 con 0x7f83fc15c340 2024-04-25T00:12:59.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.753+0000 7f83f2a00640 1 -- 172.21.15.83:0/1760115865 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f83e4011590 con 0x7f83fc15c340 2024-04-25T00:12:59.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.753+0000 7f83f2a00640 1 --2- 172.21.15.83:0/1760115865 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f83d80678f0 0x7f83d8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:59.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.753+0000 7f83f2a00640 1 -- 172.21.15.83:0/1760115865 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f83e4086150 con 0x7f83fc15c340 2024-04-25T00:12:59.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.753+0000 7f83f9e00640 1 --2- 172.21.15.83:0/1760115865 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f83d80678f0 0x7f83d8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:59.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.753+0000 7f83f9e00640 1 --2- 172.21.15.83:0/1760115865 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f83d80678f0 0x7f83d8069db0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0ad000 tx=0x7f83ec004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:59.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.757+0000 7f83f2a00640 1 -- 172.21.15.83:0/1760115865 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f83e4015020 con 0x7f83fc15c340 2024-04-25T00:12:59.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.945+0000 7f8401200640 1 -- 172.21.15.83:0/1760115865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f83fc04f5f0 con 0x7f83fc15c340 2024-04-25T00:12:59.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.945+0000 7f83f2a00640 1 -- 172.21.15.83:0/1760115865 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f83e40569d0 con 0x7f83fc15c340 2024-04-25T00:12:59.948 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:12:59.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.949+0000 7f8401200640 1 -- 172.21.15.83:0/1760115865 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f83d80678f0 msgr2=0x7f83d8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:59.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.949+0000 7f8401200640 1 --2- 172.21.15.83:0/1760115865 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f83d80678f0 0x7f83d8069db0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0ad000 tx=0x7f83ec004740 comp rx=0 tx=0).stop 2024-04-25T00:12:59.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.949+0000 7f8401200640 1 -- 172.21.15.83:0/1760115865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83fc15c340 msgr2=0x7f83fc151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:59.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.949+0000 7f8401200640 1 --2- 172.21.15.83:0/1760115865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83fc15c340 0x7f83fc151010 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f83e400c7c0 tx=0x7f83e400cc80 comp rx=0 tx=0).stop 2024-04-25T00:12:59.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.949+0000 7f8401200640 1 -- 172.21.15.83:0/1760115865 shutdown_connections 2024-04-25T00:12:59.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.949+0000 7f8401200640 1 --2- 172.21.15.83:0/1760115865 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f83d80678f0 0x7f83d8069db0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:59.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.949+0000 7f8401200640 1 --2- 172.21.15.83:0/1760115865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc15ccf0 0x7f83fc151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:59.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.949+0000 7f8401200640 1 --2- 172.21.15.83:0/1760115865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83fc15c340 0x7f83fc151010 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:59.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.949+0000 7f8401200640 1 --2- 172.21.15.83:0/1760115865 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83fc1599f0 0x7f83fc15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:59.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.949+0000 7f8401200640 1 -- 172.21.15.83:0/1760115865 >> 172.21.15.83:0/1760115865 conn(0x7f83fc0b1830 msgr2=0x7f83fc14c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:59.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.949+0000 7f8401200640 1 -- 172.21.15.83:0/1760115865 shutdown_connections 2024-04-25T00:12:59.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:12:59.949+0000 7f8401200640 1 -- 172.21.15.83:0/1760115865 wait complete. 2024-04-25T00:12:59.969 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:12:59.970 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:12:59.970 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T00:12:59.970 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:12:59.970 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:12:59.970 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:12:59.970 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:12:59.970 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:12:59.970 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:12:59.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:59.970 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T00:12:59.981 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:12:59.981 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:12:59.982 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:12:59.982 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:59.982 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T00:12:59.982 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:59.982 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:59.982 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:59.982 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T00:12:59.982 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:59.983 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:59.983 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T00:12:59.983 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:12:59.983 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:59.983 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:12:59.983 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:59.983 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:12:59.983 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42348sec preferred_lft 42348sec 2024-04-25T00:12:59.983 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T00:12:59.983 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:59.983 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:12:59.983 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:35:63:ea:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:59.984 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:12:59.984 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:59.984 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:12:59.984 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:12:59.984 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:12:59.984 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:59.984 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:12:59.984 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:59.984 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:12:59.984 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:12:59.984 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:12:59.984 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:59.985 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:12:59.985 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:12:59.985 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:12:59.985 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:12:59.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:12:59.985 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:13:00.044 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T00:13:00.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:00.047 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T00:13:00.102 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T00:13:00.102 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:13:00.102 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:13:00.102 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:13:00.102 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:13:00.102 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:13:00.102 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:13:00.102 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:13:00.103 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:13:00.103 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:13:00.103 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:13:00.103 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:13:00.103 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:13:00.103 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:13:00.103 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:13:00.103 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:13:00.103 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:13:00.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:13:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:13:00.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:13:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:13:00.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:00.248 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T00:13:00.256 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:13:00.256 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:13:00.257 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:13:00.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:00.257 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T00:13:00.317 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:13:00.317 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:13:00.369 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:13:00.369 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:13:00.369 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:13:00.420 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T00:13:00.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:00.421 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:13:00.510 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:13:00.511 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:13:00.523 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:13:00.523 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:13:00.523 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:13:00.523 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with new device syntax: 1@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:13:00.523 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:13:00.528 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:13:00.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:00.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:13:00.548 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:13:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:13:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:13:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:13:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:13:00.645 INFO:teuthology.orchestra.run.smithi086.stdout:client5728 2024-04-25T00:13:00.645 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:13:00.645 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:00.654 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:13:00.654 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:13:00.740 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:13:00.804 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:13:00.805 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:13:00.805 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:13:00.805 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:00.813 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:13:00.813 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:00.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:01.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.049+0000 7f4886c00640 1 Processor -- start 2024-04-25T00:13:01.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4886c00640 1 -- start start 2024-04-25T00:13:01.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4886c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48800b3a40 0x7f4880071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4886c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f488015e5c0 con 0x7f488006fc00 2024-04-25T00:13:01.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4886c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48800b3ec0 con 0x7f488006b1e0 2024-04-25T00:13:01.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4886c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f488006b7a0 con 0x7f48800b3a40 2024-04-25T00:13:01.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4885800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f488006fc00 0x7f488006ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42286/0 (socket says 172.21.15.83:42286) 2024-04-25T00:13:01.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4885800640 1 -- 172.21.15.83:0/207357577 learned_addr learned my addr 172.21.15.83:0/207357577 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:01.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4886200640 1 --2- 172.21.15.83:0/207357577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48800b3a40 0x7f4880071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3796146037 0 0) 0x7f488015e5c0 con 0x7f488006fc00 2024-04-25T00:13:01.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4868003530 con 0x7f488006fc00 2024-04-25T00:13:01.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4145682354 0 0) 0x7f48800b3ec0 con 0x7f488006b1e0 2024-04-25T00:13:01.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f488015e5c0 con 0x7f488006b1e0 2024-04-25T00:13:01.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1902909221 0 0) 0x7f4868003530 con 0x7f488006fc00 2024-04-25T00:13:01.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48800b3a40 msgr2=0x7f4880071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 --2- 172.21.15.83:0/207357577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48800b3a40 0x7f4880071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 >> v1:172.21.15.86:6789/0 conn(0x7f488006b1e0 legacy=0x7f488015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488006ba80 con 0x7f488006fc00 2024-04-25T00:13:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f4874001680 con 0x7f488006fc00 2024-04-25T00:13:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 >> v1:172.21.15.83:6789/0 conn(0x7f488006fc00 legacy=0x7f488006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 --2- 172.21.15.83:0/207357577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4868003b80 0x7f4868003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 --2- 172.21.15.83:0/207357577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4868004810 0x7f4868004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 --2- 172.21.15.83:0/207357577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48680051d0 0x7f486800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f488006b7a0 con 0x7f4868004810 2024-04-25T00:13:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f486800bfc0 con 0x7f48680051d0 2024-04-25T00:13:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f486800c1c0 con 0x7f4868003b80 2024-04-25T00:13:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4884e00640 1 --2- 172.21.15.83:0/207357577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4868004810 0x7f4868004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4885800640 1 --2- 172.21.15.83:0/207357577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4868003b80 0x7f4868003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4886200640 1 --2- 172.21.15.83:0/207357577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48680051d0 0x7f486800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4884e00640 1 -- 172.21.15.83:0/207357577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4868003b80 msgr2=0x7f4868003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4884e00640 1 --2- 172.21.15.83:0/207357577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4868003b80 0x7f4868003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4884e00640 1 -- 172.21.15.83:0/207357577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48680051d0 msgr2=0x7f486800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4884e00640 1 --2- 172.21.15.83:0/207357577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48680051d0 0x7f486800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.053+0000 7f4884e00640 1 -- 172.21.15.83:0/207357577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4868003530 con 0x7f4868004810 2024-04-25T00:13:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.057+0000 7f4884e00640 1 --2- 172.21.15.83:0/207357577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4868004810 0x7f4868004c90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f48800746e0 tx=0x7f486c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e366bbcb4dd4cb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:01.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.057+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c001520 con 0x7f4868004810 2024-04-25T00:13:01.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.057+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486c0052f0 con 0x7f4868004810 2024-04-25T00:13:01.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.057+0000 7f487fe00640 1 -- 172.21.15.83:0/207357577 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c0a8930 con 0x7f4868004810 2024-04-25T00:13:01.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.057+0000 7f4886c00640 1 -- 172.21.15.83:0/207357577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4868004810 msgr2=0x7f4868004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.057+0000 7f4886c00640 1 --2- 172.21.15.83:0/207357577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4868004810 0x7f4868004c90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f48800746e0 tx=0x7f486c01ecd0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.057+0000 7f4886200640 1 -- 172.21.15.83:0/207357577 reap_dead start 2024-04-25T00:13:01.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.057+0000 7f4886c00640 1 -- 172.21.15.83:0/207357577 shutdown_connections 2024-04-25T00:13:01.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.057+0000 7f4886c00640 1 -- 172.21.15.83:0/207357577 >> 172.21.15.83:0/207357577 conn(0x7f48800d0a70 msgr2=0x7f48800d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:01.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.057+0000 7f4886c00640 1 -- 172.21.15.83:0/207357577 shutdown_connections 2024-04-25T00:13:01.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.057+0000 7f4886c00640 1 -- 172.21.15.83:0/207357577 wait complete. 2024-04-25T00:13:01.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886c00640 1 Processor -- start 2024-04-25T00:13:01.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886c00640 1 -- start start 2024-04-25T00:13:01.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4868004810 0x7f48801f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488014cf20 0x7f488015bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c430 0x7f48800b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4880071c50 con 0x7f488015c430 2024-04-25T00:13:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f488006bd60 con 0x7f488014cf20 2024-04-25T00:13:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4880071dd0 con 0x7f4868004810 2024-04-25T00:13:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c430 0x7f48800b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c430 0x7f48800b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56178/0 (socket says 172.21.15.83:56178) 2024-04-25T00:13:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886200640 1 -- 172.21.15.83:0/2983363070 learned_addr learned my addr 172.21.15.83:0/2983363070 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4884e00640 1 --2- 172.21.15.83:0/2983363070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488014cf20 0x7f488015bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4885800640 1 --2- 172.21.15.83:0/2983363070 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4868004810 0x7f48801f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886200640 1 -- 172.21.15.83:0/2983363070 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4868004810 msgr2=0x7f48801f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886200640 1 --2- 172.21.15.83:0/2983363070 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4868004810 0x7f48801f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886200640 1 -- 172.21.15.83:0/2983363070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488014cf20 msgr2=0x7f488015bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886200640 1 --2- 172.21.15.83:0/2983363070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488014cf20 0x7f488015bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886200640 1 -- 172.21.15.83:0/2983363070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488015c8d0 con 0x7f488015c430 2024-04-25T00:13:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886200640 1 --2- 172.21.15.83:0/2983363070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c430 0x7f48800b1b50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f48800cd4e0 tx=0x7f487800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f487d600640 1 -- 172.21.15.83:0/2983363070 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878011e50 con 0x7f488015c430 2024-04-25T00:13:01.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f487d600640 1 -- 172.21.15.83:0/2983363070 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f487801c070 con 0x7f488015c430 2024-04-25T00:13:01.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f487d600640 1 -- 172.21.15.83:0/2983363070 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878003be0 con 0x7f488015c430 2024-04-25T00:13:01.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886c00640 1 -- 172.21.15.83:0/2983363070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488015cbb0 con 0x7f488015c430 2024-04-25T00:13:01.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886c00640 1 -- 172.21.15.83:0/2983363070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4880146d80 con 0x7f488015c430 2024-04-25T00:13:01.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.061+0000 7f4886c00640 1 -- 172.21.15.83:0/2983363070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f488004f510 con 0x7f488015c430 2024-04-25T00:13:01.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.065+0000 7f487d600640 1 -- 172.21.15.83:0/2983363070 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4878004030 con 0x7f488015c430 2024-04-25T00:13:01.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.065+0000 7f487d600640 1 --2- 172.21.15.83:0/2983363070 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f48600678f0 0x7f4860069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.065+0000 7f487d600640 1 -- 172.21.15.83:0/2983363070 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f4878086560 con 0x7f488015c430 2024-04-25T00:13:01.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.065+0000 7f4885800640 1 --2- 172.21.15.83:0/2983363070 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f48600678f0 0x7f4860069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.065+0000 7f4885800640 1 --2- 172.21.15.83:0/2983363070 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f48600678f0 0x7f4860069db0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4874000c10 tx=0x7f4874001b40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:01.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.069+0000 7f487d600640 1 -- 172.21.15.83:0/2983363070 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4878010040 con 0x7f488015c430 2024-04-25T00:13:01.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.225+0000 7f4886c00640 1 -- 172.21.15.83:0/2983363070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48800b3ec0 con 0x7f488015c430 2024-04-25T00:13:01.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.225+0000 7f487d600640 1 -- 172.21.15.83:0/2983363070 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16858 (secure 0 0 0) 0x7f4878056820 con 0x7f488015c430 2024-04-25T00:13:01.228 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:01.228 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":49,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:53.651846+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:50.394645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:51.018842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:01.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.229+0000 7f4886c00640 1 -- 172.21.15.83:0/2983363070 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f48600678f0 msgr2=0x7f4860069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.229+0000 7f4886c00640 1 --2- 172.21.15.83:0/2983363070 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f48600678f0 0x7f4860069db0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4874000c10 tx=0x7f4874001b40 comp rx=0 tx=0).stop 2024-04-25T00:13:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.229+0000 7f4886c00640 1 -- 172.21.15.83:0/2983363070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c430 msgr2=0x7f48800b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.229+0000 7f4886c00640 1 --2- 172.21.15.83:0/2983363070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c430 0x7f48800b1b50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f48800cd4e0 tx=0x7f487800f580 comp rx=0 tx=0).stop 2024-04-25T00:13:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.229+0000 7f4886c00640 1 -- 172.21.15.83:0/2983363070 shutdown_connections 2024-04-25T00:13:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.229+0000 7f4886c00640 1 --2- 172.21.15.83:0/2983363070 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f48600678f0 0x7f4860069db0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.229+0000 7f4886c00640 1 --2- 172.21.15.83:0/2983363070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f488015c430 0x7f48800b1b50 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.229+0000 7f4886c00640 1 --2- 172.21.15.83:0/2983363070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488014cf20 0x7f488015bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.229+0000 7f4886c00640 1 --2- 172.21.15.83:0/2983363070 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4868004810 0x7f48801f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.229+0000 7f4886c00640 1 -- 172.21.15.83:0/2983363070 >> 172.21.15.83:0/2983363070 conn(0x7f48800d0a70 msgr2=0x7f48800d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:01.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.229+0000 7f4886c00640 1 -- 172.21.15.83:0/2983363070 shutdown_connections 2024-04-25T00:13:01.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.229+0000 7f4886c00640 1 -- 172.21.15.83:0/2983363070 wait complete. 2024-04-25T00:13:01.250 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:12:50.394645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:12:51.018842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:13:01.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:13:01.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.385+0000 7f17b5600640 1 Processor -- start 2024-04-25T00:13:01.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.385+0000 7f17b5600640 1 -- start start 2024-04-25T00:13:01.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.389+0000 7f17b5600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f17b01555c0 0x7f17b015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.389+0000 7f17b5600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17b015c560 con 0x7f17b006e4c0 2024-04-25T00:13:01.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.389+0000 7f17b5600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17b0072460 con 0x7f17b01551e0 2024-04-25T00:13:01.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.389+0000 7f17b5600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f17b015c7d0 con 0x7f17b01555c0 2024-04-25T00:13:01.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.389+0000 7f17a6000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f17b01555c0 0x7f17b015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.389+0000 7f17a6000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f17b01555c0 0x7f17b015bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56188/0 (socket says 172.21.15.83:56188) 2024-04-25T00:13:01.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.389+0000 7f17a6000640 1 -- 172.21.15.83:0/864767933 learned_addr learned my addr 172.21.15.83:0/864767933 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:01.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.389+0000 7f17a6000640 1 -- 172.21.15.83:0/864767933 >> v1:172.21.15.86:6789/0 conn(0x7f17b01551e0 legacy=0x7f17b0071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:01.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.389+0000 7f17a6000640 1 -- 172.21.15.83:0/864767933 >> v1:172.21.15.83:6789/0 conn(0x7f17b006e4c0 legacy=0x7f17b015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.389+0000 7f17a6000640 1 -- 172.21.15.83:0/864767933 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b015cab0 con 0x7f17b01555c0 2024-04-25T00:13:01.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.393+0000 7f17a6000640 1 --2- 172.21.15.83:0/864767933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b01555c0 0x7f17b015bea0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f17980098a0 tx=0x7f179801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186e68fe8fa4efc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:01.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.393+0000 7f17ae000640 1 -- 172.21.15.83:0/864767933 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17980b0070 con 0x7f17b01555c0 2024-04-25T00:13:01.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.393+0000 7f17ae000640 1 -- 172.21.15.83:0/864767933 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17980ab600 con 0x7f17b01555c0 2024-04-25T00:13:01.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.393+0000 7f17ae000640 1 -- 172.21.15.83:0/864767933 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17980ab9d0 con 0x7f17b01555c0 2024-04-25T00:13:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.393+0000 7f17b5600640 1 -- 172.21.15.83:0/864767933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b01555c0 msgr2=0x7f17b015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.393+0000 7f17b5600640 1 --2- 172.21.15.83:0/864767933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b01555c0 0x7f17b015bea0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f17980098a0 tx=0x7f179801c390 comp rx=0 tx=0).stop 2024-04-25T00:13:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.393+0000 7f17b5600640 1 -- 172.21.15.83:0/864767933 shutdown_connections 2024-04-25T00:13:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.393+0000 7f17b5600640 1 --2- 172.21.15.83:0/864767933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b01555c0 0x7f17b015bea0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.393+0000 7f17b5600640 1 -- 172.21.15.83:0/864767933 >> 172.21.15.83:0/864767933 conn(0x7f17b00b1830 msgr2=0x7f17b00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.393+0000 7f17b5600640 1 -- 172.21.15.83:0/864767933 shutdown_connections 2024-04-25T00:13:01.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.393+0000 7f17b5600640 1 -- 172.21.15.83:0/864767933 wait complete. 2024-04-25T00:13:01.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.393+0000 7f17b5600640 1 Processor -- start 2024-04-25T00:13:01.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.393+0000 7f17b5600640 1 -- start start 2024-04-25T00:13:01.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17b5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b006e4c0 0x7f17b01486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17b5600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17b01551e0 0x7f17b00afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17b5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b01555c0 0x7f17b0159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17b5600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17b006ead0 con 0x7f17b01555c0 2024-04-25T00:13:01.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17b5600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17b006e950 con 0x7f17b006e4c0 2024-04-25T00:13:01.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17b5600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f17b006ec50 con 0x7f17b01551e0 2024-04-25T00:13:01.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17af400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b01555c0 0x7f17b0159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17af400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b01555c0 0x7f17b0159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56196/0 (socket says 172.21.15.83:56196) 2024-04-25T00:13:01.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17af400640 1 -- 172.21.15.83:0/2420616219 learned_addr learned my addr 172.21.15.83:0/2420616219 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:01.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17aea00640 1 --2- 172.21.15.83:0/2420616219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b006e4c0 0x7f17b01486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17a6000640 1 --2- 172.21.15.83:0/2420616219 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17b01551e0 0x7f17b00afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17af400640 1 -- 172.21.15.83:0/2420616219 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17b01551e0 msgr2=0x7f17b00afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17af400640 1 --2- 172.21.15.83:0/2420616219 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17b01551e0 0x7f17b00afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17af400640 1 -- 172.21.15.83:0/2420616219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b006e4c0 msgr2=0x7f17b01486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17af400640 1 --2- 172.21.15.83:0/2420616219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b006e4c0 0x7f17b01486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17af400640 1 -- 172.21.15.83:0/2420616219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b01f48a0 con 0x7f17b01555c0 2024-04-25T00:13:01.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17af400640 1 --2- 172.21.15.83:0/2420616219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b01555c0 0x7f17b0159dd0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f17a800c3c0 tx=0x7f17a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:01.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17a7400640 1 -- 172.21.15.83:0/2420616219 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8003980 con 0x7f17b01555c0 2024-04-25T00:13:01.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17a7400640 1 -- 172.21.15.83:0/2420616219 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17a8003b20 con 0x7f17b01555c0 2024-04-25T00:13:01.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17a7400640 1 -- 172.21.15.83:0/2420616219 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8004020 con 0x7f17b01555c0 2024-04-25T00:13:01.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17b5600640 1 -- 172.21.15.83:0/2420616219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b01f4b80 con 0x7f17b01555c0 2024-04-25T00:13:01.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17b5600640 1 -- 172.21.15.83:0/2420616219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b01f5140 con 0x7f17b01555c0 2024-04-25T00:13:01.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17b5600640 1 -- 172.21.15.83:0/2420616219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b004f510 con 0x7f17b01555c0 2024-04-25T00:13:01.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.397+0000 7f17a7400640 1 -- 172.21.15.83:0/2420616219 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f17a801b020 con 0x7f17b01555c0 2024-04-25T00:13:01.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.401+0000 7f17a7400640 1 --2- 172.21.15.83:0/2420616219 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f178c0678c0 0x7f178c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.401+0000 7f17a7400640 1 -- 172.21.15.83:0/2420616219 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f17a8088160 con 0x7f17b01555c0 2024-04-25T00:13:01.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.401+0000 7f17aea00640 1 --2- 172.21.15.83:0/2420616219 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f178c0678c0 0x7f178c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.401+0000 7f17aea00640 1 --2- 172.21.15.83:0/2420616219 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f178c0678c0 0x7f178c069d80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f17b0154ce0 tx=0x7f17a0007e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:01.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.405+0000 7f17a7400640 1 -- 172.21.15.83:0/2420616219 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f17a8058530 con 0x7f17b01555c0 2024-04-25T00:13:01.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.569+0000 7f17b5600640 1 -- 172.21.15.83:0/2420616219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f17b006ce30 con 0x7f17b01555c0 2024-04-25T00:13:01.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.569+0000 7f17a7400640 1 -- 172.21.15.83:0/2420616219 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) ==== 93+0+10 (secure 0 0 0) 0x7f17a805c830 con 0x7f17b01555c0 2024-04-25T00:13:01.571 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:13:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.569+0000 7f17b5600640 1 -- 172.21.15.83:0/2420616219 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f178c0678c0 msgr2=0x7f178c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.569+0000 7f17b5600640 1 --2- 172.21.15.83:0/2420616219 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f178c0678c0 0x7f178c069d80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f17b0154ce0 tx=0x7f17a0007e40 comp rx=0 tx=0).stop 2024-04-25T00:13:01.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.569+0000 7f17b5600640 1 -- 172.21.15.83:0/2420616219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b01555c0 msgr2=0x7f17b0159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.569+0000 7f17b5600640 1 --2- 172.21.15.83:0/2420616219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b01555c0 0x7f17b0159dd0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f17a800c3c0 tx=0x7f17a800c880 comp rx=0 tx=0).stop 2024-04-25T00:13:01.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.569+0000 7f17b5600640 1 -- 172.21.15.83:0/2420616219 shutdown_connections 2024-04-25T00:13:01.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.569+0000 7f17b5600640 1 --2- 172.21.15.83:0/2420616219 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f178c0678c0 0x7f178c069d80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.569+0000 7f17b5600640 1 --2- 172.21.15.83:0/2420616219 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17b01551e0 0x7f17b00afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.569+0000 7f17b5600640 1 --2- 172.21.15.83:0/2420616219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b01555c0 0x7f17b0159dd0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.569+0000 7f17b5600640 1 --2- 172.21.15.83:0/2420616219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b006e4c0 0x7f17b01486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.569+0000 7f17b5600640 1 -- 172.21.15.83:0/2420616219 >> 172.21.15.83:0/2420616219 conn(0x7f17b00b1830 msgr2=0x7f17b00b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:01.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.569+0000 7f17b5600640 1 -- 172.21.15.83:0/2420616219 shutdown_connections 2024-04-25T00:13:01.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.569+0000 7f17b5600640 1 -- 172.21.15.83:0/2420616219 wait complete. 2024-04-25T00:13:01.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:13:01.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccdaa00640 1 Processor -- start 2024-04-25T00:13:01.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccdaa00640 1 -- start start 2024-04-25T00:13:01.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccdaa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccd4159810 0x7fccd40cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccdaa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fccd4160020 0x7fccd40ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccdaa00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccd40ce000 con 0x7fccd406e950 2024-04-25T00:13:01.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccdaa00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fccd406e210 con 0x7fccd4160020 2024-04-25T00:13:01.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccdaa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fccd406da00 con 0x7fccd4159810 2024-04-25T00:13:01.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccd8a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fccd4160020 0x7fccd40ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccd3400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fccd406e950 0x7fccd415f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42304/0 (socket says 172.21.15.83:42304) 2024-04-25T00:13:01.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccd8a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fccd4160020 0x7fccd40ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56204/0 (socket says 172.21.15.83:56204) 2024-04-25T00:13:01.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccd3400640 1 -- 172.21.15.83:0/3996226656 learned_addr learned my addr 172.21.15.83:0/3996226656 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:01.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccd3e00640 1 --2- 172.21.15.83:0/3996226656 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccd4159810 0x7fccd40cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccd8a00640 1 -- 172.21.15.83:0/3996226656 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccd4159810 msgr2=0x7fccd40cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:01.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccd8a00640 1 --2- 172.21.15.83:0/3996226656 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccd4159810 0x7fccd40cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccd8a00640 1 -- 172.21.15.83:0/3996226656 >> v1:172.21.15.83:6789/0 conn(0x7fccd406e950 legacy=0x7fccd415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccd2a00640 1 -- 172.21.15.83:0/3996226656 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3864527542 0 0) 0x7fccd40ce000 con 0x7fccd406e950 2024-04-25T00:13:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.729+0000 7fccd8a00640 1 -- 172.21.15.83:0/3996226656 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd406eec0 con 0x7fccd4160020 2024-04-25T00:13:01.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.733+0000 7fccd8a00640 1 --2- 172.21.15.83:0/3996226656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd4160020 0x7fccd40ce4a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fccd406db80 tx=0x7fcccc01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4195fa9d349a1241 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:01.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.733+0000 7fccd2a00640 1 -- 172.21.15.83:0/3996226656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0b2070 con 0x7fccd4160020 2024-04-25T00:13:01.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.733+0000 7fccd2a00640 1 -- 172.21.15.83:0/3996226656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcccc0a5950 con 0x7fccd4160020 2024-04-25T00:13:01.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.733+0000 7fccd2a00640 1 -- 172.21.15.83:0/3996226656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0a5d20 con 0x7fccd4160020 2024-04-25T00:13:01.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.733+0000 7fccdaa00640 1 -- 172.21.15.83:0/3996226656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd4160020 msgr2=0x7fccd40ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.733+0000 7fccdaa00640 1 --2- 172.21.15.83:0/3996226656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd4160020 0x7fccd40ce4a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fccd406db80 tx=0x7fcccc01c890 comp rx=0 tx=0).stop 2024-04-25T00:13:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.733+0000 7fccdaa00640 1 -- 172.21.15.83:0/3996226656 shutdown_connections 2024-04-25T00:13:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.733+0000 7fccdaa00640 1 --2- 172.21.15.83:0/3996226656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd4160020 0x7fccd40ce4a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.733+0000 7fccdaa00640 1 --2- 172.21.15.83:0/3996226656 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccd4159810 0x7fccd40cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.733+0000 7fccdaa00640 1 -- 172.21.15.83:0/3996226656 >> 172.21.15.83:0/3996226656 conn(0x7fccd40b1830 msgr2=0x7fccd40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.733+0000 7fccdaa00640 1 -- 172.21.15.83:0/3996226656 shutdown_connections 2024-04-25T00:13:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.733+0000 7fccdaa00640 1 -- 172.21.15.83:0/3996226656 wait complete. 2024-04-25T00:13:01.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccdaa00640 1 Processor -- start 2024-04-25T00:13:01.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccdaa00640 1 -- start start 2024-04-25T00:13:01.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccdaa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccd406e950 0x7fccd40a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccdaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccd4159810 0x7fccd40aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccdaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd4160020 0x7fccd414b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccdaa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccd4159d70 con 0x7fccd4160020 2024-04-25T00:13:01.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccdaa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fccd4159bf0 con 0x7fccd4159810 2024-04-25T00:13:01.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccdaa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fccd4070da0 con 0x7fccd406e950 2024-04-25T00:13:01.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccd8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd4160020 0x7fccd414b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccd8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd4160020 0x7fccd414b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56212/0 (socket says 172.21.15.83:56212) 2024-04-25T00:13:01.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccd8a00640 1 -- 172.21.15.83:0/4185099725 learned_addr learned my addr 172.21.15.83:0/4185099725 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:01.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccd3400640 1 --2- 172.21.15.83:0/4185099725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccd4159810 0x7fccd40aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccd3e00640 1 --2- 172.21.15.83:0/4185099725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccd406e950 0x7fccd40a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccd8a00640 1 -- 172.21.15.83:0/4185099725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccd406e950 msgr2=0x7fccd40a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccd8a00640 1 --2- 172.21.15.83:0/4185099725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccd406e950 0x7fccd40a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccd8a00640 1 -- 172.21.15.83:0/4185099725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccd4159810 msgr2=0x7fccd40aaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccd8a00640 1 --2- 172.21.15.83:0/4185099725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccd4159810 0x7fccd40aaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccd8a00640 1 -- 172.21.15.83:0/4185099725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd40a9a30 con 0x7fccd4160020 2024-04-25T00:13:01.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccd8a00640 1 --2- 172.21.15.83:0/4185099725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd4160020 0x7fccd414b590 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcccc01af40 tx=0x7fcccc019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:01.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fcccbe00640 1 -- 172.21.15.83:0/4185099725 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0bd020 con 0x7fccd4160020 2024-04-25T00:13:01.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fcccbe00640 1 -- 172.21.15.83:0/4185099725 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcccc0b8670 con 0x7fccd4160020 2024-04-25T00:13:01.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fcccbe00640 1 -- 172.21.15.83:0/4185099725 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0b2040 con 0x7fccd4160020 2024-04-25T00:13:01.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccdaa00640 1 -- 172.21.15.83:0/4185099725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd40a9d10 con 0x7fccd4160020 2024-04-25T00:13:01.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccdaa00640 1 -- 172.21.15.83:0/4185099725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd40aa2d0 con 0x7fccd4160020 2024-04-25T00:13:01.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.737+0000 7fccdaa00640 1 -- 172.21.15.83:0/4185099725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd404f510 con 0x7fccd4160020 2024-04-25T00:13:01.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.741+0000 7fcccbe00640 1 -- 172.21.15.83:0/4185099725 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcccc0c0020 con 0x7fccd4160020 2024-04-25T00:13:01.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.741+0000 7fcccbe00640 1 --2- 172.21.15.83:0/4185099725 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccb40678c0 0x7fccb4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:01.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.741+0000 7fcccbe00640 1 -- 172.21.15.83:0/4185099725 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fcccc120d90 con 0x7fccd4160020 2024-04-25T00:13:01.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.741+0000 7fccd3e00640 1 --2- 172.21.15.83:0/4185099725 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccb40678c0 0x7fccb4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:01.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.741+0000 7fccd3e00640 1 --2- 172.21.15.83:0/4185099725 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccb40678c0 0x7fccb4069d80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009b60 tx=0x7fccc4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:01.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.745+0000 7fcccbe00640 1 -- 172.21.15.83:0/4185099725 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcccc0c7030 con 0x7fccd4160020 2024-04-25T00:13:01.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.901+0000 7fccdaa00640 1 -- 172.21.15.83:0/4185099725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fccd40cd300 con 0x7fccd4160020 2024-04-25T00:13:01.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.901+0000 7fcccbe00640 1 -- 172.21.15.83:0/4185099725 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) ==== 104+0+11 (secure 0 0 0) 0x7fcccc016540 con 0x7fccd4160020 2024-04-25T00:13:01.903 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:13:01.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.901+0000 7fccdaa00640 1 -- 172.21.15.83:0/4185099725 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccb40678c0 msgr2=0x7fccb4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.901+0000 7fccdaa00640 1 --2- 172.21.15.83:0/4185099725 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccb40678c0 0x7fccb4069d80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009b60 tx=0x7fccc4009190 comp rx=0 tx=0).stop 2024-04-25T00:13:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.901+0000 7fccdaa00640 1 -- 172.21.15.83:0/4185099725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd4160020 msgr2=0x7fccd414b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.901+0000 7fccdaa00640 1 --2- 172.21.15.83:0/4185099725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd4160020 0x7fccd414b590 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcccc01af40 tx=0x7fcccc019630 comp rx=0 tx=0).stop 2024-04-25T00:13:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.901+0000 7fccdaa00640 1 -- 172.21.15.83:0/4185099725 shutdown_connections 2024-04-25T00:13:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.901+0000 7fccdaa00640 1 --2- 172.21.15.83:0/4185099725 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccb40678c0 0x7fccb4069d80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.901+0000 7fccdaa00640 1 --2- 172.21.15.83:0/4185099725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd4160020 0x7fccd414b590 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.901+0000 7fccdaa00640 1 --2- 172.21.15.83:0/4185099725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccd4159810 0x7fccd40aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.901+0000 7fccdaa00640 1 --2- 172.21.15.83:0/4185099725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccd406e950 0x7fccd40a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.901+0000 7fccdaa00640 1 -- 172.21.15.83:0/4185099725 >> 172.21.15.83:0/4185099725 conn(0x7fccd40b1830 msgr2=0x7fccd415ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.901+0000 7fccdaa00640 1 -- 172.21.15.83:0/4185099725 shutdown_connections 2024-04-25T00:13:01.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:01.901+0000 7fccdaa00640 1 -- 172.21.15.83:0/4185099725 wait complete. 2024-04-25T00:13:01.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:13:02.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.057+0000 7f9d63200640 1 Processor -- start 2024-04-25T00:13:02.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.057+0000 7f9d63200640 1 -- start start 2024-04-25T00:13:02.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.057+0000 7f9d63200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d5c159810 0x7f9d5c0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.057+0000 7f9d63200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d5c160020 0x7f9d5c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.057+0000 7f9d63200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d5c0ce000 con 0x7f9d5c06e950 2024-04-25T00:13:02.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.057+0000 7f9d63200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9d5c06e210 con 0x7f9d5c160020 2024-04-25T00:13:02.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.057+0000 7f9d63200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9d5c06da00 con 0x7f9d5c159810 2024-04-25T00:13:02.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.057+0000 7f9d61200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d5c160020 0x7f9d5c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.057+0000 7f9d61200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d5c160020 0x7f9d5c0ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56220/0 (socket says 172.21.15.83:56220) 2024-04-25T00:13:02.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.057+0000 7f9d61200640 1 -- 172.21.15.83:0/690450737 learned_addr learned my addr 172.21.15.83:0/690450737 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:02.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.057+0000 7f9d5b400640 1 --1- 172.21.15.83:0/690450737 >> v1:172.21.15.83:6789/0 conn(0x7f9d5c06e950 0x7f9d5c15f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42310/0 (socket says 172.21.15.83:42310) 2024-04-25T00:13:02.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.061+0000 7f9d5be00640 1 --2- 172.21.15.83:0/690450737 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d5c159810 0x7f9d5c0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.061+0000 7f9d61200640 1 -- 172.21.15.83:0/690450737 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d5c159810 msgr2=0x7f9d5c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.061+0000 7f9d61200640 1 --2- 172.21.15.83:0/690450737 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d5c159810 0x7f9d5c0cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.061+0000 7f9d61200640 1 -- 172.21.15.83:0/690450737 >> v1:172.21.15.83:6789/0 conn(0x7f9d5c06e950 legacy=0x7f9d5c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.061+0000 7f9d61200640 1 -- 172.21.15.83:0/690450737 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5c06eec0 con 0x7f9d5c160020 2024-04-25T00:13:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.061+0000 7f9d5be00640 1 --2- 172.21.15.83:0/690450737 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d5c159810 0x7f9d5c0cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:02.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.061+0000 7f9d61200640 1 --2- 172.21.15.83:0/690450737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d5c160020 0x7f9d5c0ce4a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c06db80 tx=0x7f9d5401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0b66ecdaf8bf9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:02.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.061+0000 7f9d5aa00640 1 -- 172.21.15.83:0/690450737 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d540b0070 con 0x7f9d5c160020 2024-04-25T00:13:02.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.061+0000 7f9d5aa00640 1 -- 172.21.15.83:0/690450737 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d540ab600 con 0x7f9d5c160020 2024-04-25T00:13:02.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.061+0000 7f9d5aa00640 1 -- 172.21.15.83:0/690450737 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d540ab9d0 con 0x7f9d5c160020 2024-04-25T00:13:02.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.061+0000 7f9d63200640 1 -- 172.21.15.83:0/690450737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d5c160020 msgr2=0x7f9d5c0ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.061+0000 7f9d63200640 1 --2- 172.21.15.83:0/690450737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d5c160020 0x7f9d5c0ce4a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c06db80 tx=0x7f9d5401c310 comp rx=0 tx=0).stop 2024-04-25T00:13:02.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d63200640 1 -- 172.21.15.83:0/690450737 shutdown_connections 2024-04-25T00:13:02.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d63200640 1 --2- 172.21.15.83:0/690450737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d5c160020 0x7f9d5c0ce4a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d63200640 1 --2- 172.21.15.83:0/690450737 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d5c159810 0x7f9d5c0cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d63200640 1 -- 172.21.15.83:0/690450737 >> 172.21.15.83:0/690450737 conn(0x7f9d5c0b1830 msgr2=0x7f9d5c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:02.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d63200640 1 -- 172.21.15.83:0/690450737 shutdown_connections 2024-04-25T00:13:02.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d63200640 1 -- 172.21.15.83:0/690450737 wait complete. 2024-04-25T00:13:02.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d63200640 1 Processor -- start 2024-04-25T00:13:02.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d63200640 1 -- start start 2024-04-25T00:13:02.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d63200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d5c06e950 0x7f9d5c14c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d63200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d5c159810 0x7f9d5c14d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d63200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d5c160020 0x7f9d5c0b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d63200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c159d70 con 0x7f9d5c160020 2024-04-25T00:13:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d63200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c159bf0 con 0x7f9d5c06e950 2024-04-25T00:13:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d63200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9d5c070da0 con 0x7f9d5c159810 2024-04-25T00:13:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d61200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d5c160020 0x7f9d5c0b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d61200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d5c160020 0x7f9d5c0b2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56232/0 (socket says 172.21.15.83:56232) 2024-04-25T00:13:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d61200640 1 -- 172.21.15.83:0/2221850861 learned_addr learned my addr 172.21.15.83:0/2221850861 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d5be00640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d5c06e950 0x7f9d5c14c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d5b400640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d5c159810 0x7f9d5c14d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d61200640 1 -- 172.21.15.83:0/2221850861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d5c159810 msgr2=0x7f9d5c14d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d61200640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d5c159810 0x7f9d5c14d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d61200640 1 -- 172.21.15.83:0/2221850861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d5c06e950 msgr2=0x7f9d5c14c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d61200640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d5c06e950 0x7f9d5c14c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d61200640 1 -- 172.21.15.83:0/2221850861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5c14dcc0 con 0x7f9d5c160020 2024-04-25T00:13:02.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d5be00640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d5c06e950 0x7f9d5c14c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:02.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d5b400640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d5c159810 0x7f9d5c14d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:02.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d61200640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d5c160020 0x7f9d5c0b2c30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009870 tx=0x7f9d540190b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:02.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d53e00640 1 -- 172.21.15.83:0/2221850861 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d540b0040 con 0x7f9d5c160020 2024-04-25T00:13:02.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.065+0000 7f9d53e00640 1 -- 172.21.15.83:0/2221850861 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d540b4420 con 0x7f9d5c160020 2024-04-25T00:13:02.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.069+0000 7f9d63200640 1 -- 172.21.15.83:0/2221850861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5c14dfa0 con 0x7f9d5c160020 2024-04-25T00:13:02.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.069+0000 7f9d53e00640 1 -- 172.21.15.83:0/2221850861 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d540b47d0 con 0x7f9d5c160020 2024-04-25T00:13:02.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.069+0000 7f9d63200640 1 -- 172.21.15.83:0/2221850861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d5c0b3170 con 0x7f9d5c160020 2024-04-25T00:13:02.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.069+0000 7f9d63200640 1 -- 172.21.15.83:0/2221850861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d5c04f510 con 0x7f9d5c160020 2024-04-25T00:13:02.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.069+0000 7f9d53e00640 1 -- 172.21.15.83:0/2221850861 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9d540b4970 con 0x7f9d5c160020 2024-04-25T00:13:02.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.069+0000 7f9d53e00640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9d3c0678c0 0x7f9d3c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.069+0000 7f9d53e00640 1 -- 172.21.15.83:0/2221850861 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f9d5411fb30 con 0x7f9d5c160020 2024-04-25T00:13:02.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.069+0000 7f9d5be00640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9d3c0678c0 0x7f9d3c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.069+0000 7f9d5be00640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9d3c0678c0 0x7f9d3c069d80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0094c0 tx=0x7f9d4c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:02.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.073+0000 7f9d53e00640 1 -- 172.21.15.83:0/2221850861 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9d54016090 con 0x7f9d5c160020 2024-04-25T00:13:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.225+0000 7f9d63200640 1 -- 172.21.15.83:0/2221850861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9d5c0727b0 con 0x7f9d5c160020 2024-04-25T00:13:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.229+0000 7f9d53e00640 1 -- 172.21.15.83:0/2221850861 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) ==== 100+0+11 (secure 0 0 0) 0x7f9d540f04d0 con 0x7f9d5c160020 2024-04-25T00:13:02.230 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:13:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.229+0000 7f9d63200640 1 -- 172.21.15.83:0/2221850861 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9d3c0678c0 msgr2=0x7f9d3c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.229+0000 7f9d63200640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9d3c0678c0 0x7f9d3c069d80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0094c0 tx=0x7f9d4c00b040 comp rx=0 tx=0).stop 2024-04-25T00:13:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.229+0000 7f9d63200640 1 -- 172.21.15.83:0/2221850861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d5c160020 msgr2=0x7f9d5c0b2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.229+0000 7f9d63200640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d5c160020 0x7f9d5c0b2c30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009870 tx=0x7f9d540190b0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.229+0000 7f9d63200640 1 -- 172.21.15.83:0/2221850861 shutdown_connections 2024-04-25T00:13:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.229+0000 7f9d63200640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9d3c0678c0 0x7f9d3c069d80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.229+0000 7f9d63200640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d5c159810 0x7f9d5c14d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.229+0000 7f9d63200640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d5c160020 0x7f9d5c0b2c30 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.229+0000 7f9d63200640 1 --2- 172.21.15.83:0/2221850861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d5c06e950 0x7f9d5c14c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.229+0000 7f9d63200640 1 -- 172.21.15.83:0/2221850861 >> 172.21.15.83:0/2221850861 conn(0x7f9d5c0b1830 msgr2=0x7f9d5c0d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:02.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.229+0000 7f9d63200640 1 -- 172.21.15.83:0/2221850861 shutdown_connections 2024-04-25T00:13:02.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.229+0000 7f9d63200640 1 -- 172.21.15.83:0/2221850861 wait complete. 2024-04-25T00:13:02.251 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:13:02.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f320c000640 1 Processor -- start 2024-04-25T00:13:02.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f320c000640 1 -- start start 2024-04-25T00:13:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f320c000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f320415c0e0 0x7f32040d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f320c000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32040d03f0 0x7f3204071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f320c000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f320406bd90 con 0x7f3204160780 2024-04-25T00:13:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f320c000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f320406c070 con 0x7f320415c0e0 2024-04-25T00:13:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f320c000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f320415fca0 con 0x7f32040d03f0 2024-04-25T00:13:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f3208c00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3204160780 0x7f32040cfce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42316/0 (socket says 172.21.15.83:42316) 2024-04-25T00:13:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f3208c00640 1 -- 172.21.15.83:0/4000220022 learned_addr learned my addr 172.21.15.83:0/4000220022 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f3209600640 1 --2- 172.21.15.83:0/4000220022 >> v2:172.21.15.86:3300/0 conn(0x7f320415c0e0 0x7f32040d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f320a000640 1 --2- 172.21.15.83:0/4000220022 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32040d03f0 0x7f3204071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f3203e00640 1 -- 172.21.15.83:0/4000220022 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2280917484 0 0) 0x7f320406bd90 con 0x7f3204160780 2024-04-25T00:13:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f3203e00640 1 -- 172.21.15.83:0/4000220022 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31ec003530 con 0x7f3204160780 2024-04-25T00:13:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f3209600640 1 -- 172.21.15.83:0/4000220022 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32040d03f0 msgr2=0x7f3204071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f3209600640 1 --2- 172.21.15.83:0/4000220022 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32040d03f0 0x7f3204071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f3209600640 1 -- 172.21.15.83:0/4000220022 >> v1:172.21.15.83:6789/0 conn(0x7f3204160780 legacy=0x7f32040cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f3203e00640 1 -- 172.21.15.83:0/4000220022 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2415697336 0 0) 0x7f31ec003530 con 0x7f3204160780 2024-04-25T00:13:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.385+0000 7f3209600640 1 -- 172.21.15.83:0/4000220022 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320415ff80 con 0x7f320415c0e0 2024-04-25T00:13:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.389+0000 7f3209600640 1 --2- 172.21.15.83:0/4000220022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f320415c0e0 0x7f32040d16e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f31f80097c0 tx=0x7f31f801c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc9d23878e9dfe0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.389+0000 7f3203e00640 1 -- 172.21.15.83:0/4000220022 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f80b0070 con 0x7f320415c0e0 2024-04-25T00:13:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.389+0000 7f3203e00640 1 -- 172.21.15.83:0/4000220022 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f80ab400 con 0x7f320415c0e0 2024-04-25T00:13:02.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.389+0000 7f3203e00640 1 -- 172.21.15.83:0/4000220022 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0001680 con 0x7f320415c0e0 2024-04-25T00:13:02.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.389+0000 7f320c000640 1 -- 172.21.15.83:0/4000220022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f320415c0e0 msgr2=0x7f32040d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.389+0000 7f320c000640 1 --2- 172.21.15.83:0/4000220022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f320415c0e0 0x7f32040d16e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f31f80097c0 tx=0x7f31f801c390 comp rx=0 tx=0).stop 2024-04-25T00:13:02.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.389+0000 7f320c000640 1 -- 172.21.15.83:0/4000220022 shutdown_connections 2024-04-25T00:13:02.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.389+0000 7f320c000640 1 --2- 172.21.15.83:0/4000220022 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32040d03f0 0x7f3204071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.389+0000 7f320c000640 1 --2- 172.21.15.83:0/4000220022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f320415c0e0 0x7f32040d16e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.389+0000 7f320c000640 1 -- 172.21.15.83:0/4000220022 >> 172.21.15.83:0/4000220022 conn(0x7f32040d0bc0 msgr2=0x7f32040d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:02.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.389+0000 7f320c000640 1 -- 172.21.15.83:0/4000220022 shutdown_connections 2024-04-25T00:13:02.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.389+0000 7f320c000640 1 -- 172.21.15.83:0/4000220022 wait complete. 2024-04-25T00:13:02.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320c000640 1 Processor -- start 2024-04-25T00:13:02.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320c000640 1 -- start start 2024-04-25T00:13:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32040d03f0 0x7f32040cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320c000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f320415c0e0 0x7f32041496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320c000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204160780 0x7f3204149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320c000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3204169020 con 0x7f3204160780 2024-04-25T00:13:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3204168ea0 con 0x7f32040d03f0 2024-04-25T00:13:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320c000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f32041691a0 con 0x7f320415c0e0 2024-04-25T00:13:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204160780 0x7f3204149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204160780 0x7f3204149be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56242/0 (socket says 172.21.15.83:56242) 2024-04-25T00:13:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320a000640 1 -- 172.21.15.83:0/365669706 learned_addr learned my addr 172.21.15.83:0/365669706 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f3208c00640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f320415c0e0 0x7f32041496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f3209600640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32040d03f0 0x7f32040cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320a000640 1 -- 172.21.15.83:0/365669706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f320415c0e0 msgr2=0x7f32041496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320a000640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f320415c0e0 0x7f32041496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320a000640 1 -- 172.21.15.83:0/365669706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32040d03f0 msgr2=0x7f32040cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320a000640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32040d03f0 0x7f32040cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320a000640 1 -- 172.21.15.83:0/365669706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320414b1b0 con 0x7f3204160780 2024-04-25T00:13:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f3208c00640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f320415c0e0 0x7f32041496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320a000640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204160780 0x7f3204149be0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00ec10 tx=0x7f31fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f3201600640 1 -- 172.21.15.83:0/365669706 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc018020 con 0x7f3204160780 2024-04-25T00:13:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f3201600640 1 -- 172.21.15.83:0/365669706 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31fc00cd40 con 0x7f3204160780 2024-04-25T00:13:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f3201600640 1 -- 172.21.15.83:0/365669706 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0146c0 con 0x7f3204160780 2024-04-25T00:13:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320c000640 1 -- 172.21.15.83:0/365669706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320414b490 con 0x7f3204160780 2024-04-25T00:13:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.393+0000 7f320c000640 1 -- 172.21.15.83:0/365669706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f320414ba50 con 0x7f3204160780 2024-04-25T00:13:02.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.397+0000 7f320c000640 1 -- 172.21.15.83:0/365669706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3204068230 con 0x7f3204160780 2024-04-25T00:13:02.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.397+0000 7f3201600640 1 -- 172.21.15.83:0/365669706 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f31fc01b020 con 0x7f3204160780 2024-04-25T00:13:02.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.397+0000 7f3201600640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f31e40678c0 0x7f31e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.397+0000 7f3201600640 1 -- 172.21.15.83:0/365669706 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f31fc086b40 con 0x7f3204160780 2024-04-25T00:13:02.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.397+0000 7f3209600640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f31e40678c0 0x7f31e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.397+0000 7f3209600640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f31e40678c0 0x7f31e4069d80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f31f80ad000 tx=0x7f31f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.401+0000 7f3201600640 1 -- 172.21.15.83:0/365669706 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f31fc08a030 con 0x7f3204160780 2024-04-25T00:13:02.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.569+0000 7f320c000640 1 -- 172.21.15.83:0/365669706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f320415f950 con 0x7f3204160780 2024-04-25T00:13:02.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.569+0000 7f3201600640 1 -- 172.21.15.83:0/365669706 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) ==== 80+0+0 (secure 0 0 0) 0x7f31fc022030 con 0x7f3204160780 2024-04-25T00:13:02.572 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:13:02.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.573+0000 7f320c000640 1 -- 172.21.15.83:0/365669706 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f31e40678c0 msgr2=0x7f31e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.573+0000 7f320c000640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f31e40678c0 0x7f31e4069d80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f31f80ad000 tx=0x7f31f8008040 comp rx=0 tx=0).stop 2024-04-25T00:13:02.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.573+0000 7f320c000640 1 -- 172.21.15.83:0/365669706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204160780 msgr2=0x7f3204149be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.573+0000 7f320c000640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204160780 0x7f3204149be0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00ec10 tx=0x7f31fc00c490 comp rx=0 tx=0).stop 2024-04-25T00:13:02.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.573+0000 7f320c000640 1 -- 172.21.15.83:0/365669706 shutdown_connections 2024-04-25T00:13:02.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.573+0000 7f320c000640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f31e40678c0 0x7f31e4069d80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.573+0000 7f320c000640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f320415c0e0 0x7f32041496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.573+0000 7f320c000640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204160780 0x7f3204149be0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.573+0000 7f320c000640 1 --2- 172.21.15.83:0/365669706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32040d03f0 0x7f32040cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.573+0000 7f320c000640 1 -- 172.21.15.83:0/365669706 >> 172.21.15.83:0/365669706 conn(0x7f32040d0bc0 msgr2=0x7f32040ceac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:02.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.573+0000 7f320c000640 1 -- 172.21.15.83:0/365669706 shutdown_connections 2024-04-25T00:13:02.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.573+0000 7f320c000640 1 -- 172.21.15.83:0/365669706 wait complete. 2024-04-25T00:13:02.593 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:13:02.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:02.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:13:02.620 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:13:02.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:02.620 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:13:02.673 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:13:02.674 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:13:02.674 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:02.683 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:13:02.683 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:02.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:02.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe7e00640 1 Processor -- start 2024-04-25T00:13:02.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe7e00640 1 -- start start 2024-04-25T00:13:02.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fabe80b18c0 0x7fabe80cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe7e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabe8074740 0x7fabe815f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe7e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabe815c340 con 0x7fabe80b07d0 2024-04-25T00:13:02.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe7e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fabe80cc280 con 0x7fabe80b18c0 2024-04-25T00:13:02.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe7e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fabe80cc560 con 0x7fabe8074740 2024-04-25T00:13:02.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe7400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabe8074740 0x7fabe815f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe6a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fabe80b07d0 0x7fabe815cae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:47650/0 (socket says 172.21.15.83:47650) 2024-04-25T00:13:02.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe6a00640 1 -- 172.21.15.83:0/2227940731 learned_addr learned my addr 172.21.15.83:0/2227940731 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:02.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe6000640 1 --2- 172.21.15.83:0/2227940731 >> v2:172.21.15.86:3300/0 conn(0x7fabe80b18c0 0x7fabe80cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe5600640 1 -- 172.21.15.83:0/2227940731 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1448276792 0 0) 0x7fabd8001120 con 0x7fabe80b07d0 2024-04-25T00:13:02.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe5600640 1 -- 172.21.15.83:0/2227940731 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabcc003530 con 0x7fabe80b07d0 2024-04-25T00:13:02.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe7400640 1 -- 172.21.15.83:0/2227940731 >> v2:172.21.15.86:3300/0 conn(0x7fabe80b18c0 msgr2=0x7fabe80cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:02.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe7400640 1 --2- 172.21.15.83:0/2227940731 >> v2:172.21.15.86:3300/0 conn(0x7fabe80b18c0 0x7fabe80cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe7400640 1 -- 172.21.15.83:0/2227940731 >> v1:172.21.15.86:6789/0 conn(0x7fabe80b07d0 legacy=0x7fabe815cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe7400640 1 -- 172.21.15.83:0/2227940731 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe80cc840 con 0x7fabe8074740 2024-04-25T00:13:02.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.869+0000 7fabe6000640 1 --2- 172.21.15.83:0/2227940731 >> v2:172.21.15.86:3300/0 conn(0x7fabe80b18c0 0x7fabe80cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:02.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.873+0000 7fabe7400640 1 --2- 172.21.15.83:0/2227940731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabe8074740 0x7fabe815f930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009d90 tx=0x7fabe001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8883a7e1d4f94341 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:02.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.873+0000 7fabe5600640 1 -- 172.21.15.83:0/2227940731 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe00b2070 con 0x7fabe8074740 2024-04-25T00:13:02.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.873+0000 7fabe5600640 1 -- 172.21.15.83:0/2227940731 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe00a5910 con 0x7fabe8074740 2024-04-25T00:13:02.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.873+0000 7fabe5600640 1 -- 172.21.15.83:0/2227940731 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe00a5ce0 con 0x7fabe8074740 2024-04-25T00:13:02.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.873+0000 7fabe7e00640 1 -- 172.21.15.83:0/2227940731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabe8074740 msgr2=0x7fabe815f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.873+0000 7fabe7e00640 1 --2- 172.21.15.83:0/2227940731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabe8074740 0x7fabe815f930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009d90 tx=0x7fabe001c940 comp rx=0 tx=0).stop 2024-04-25T00:13:02.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.873+0000 7fabe7e00640 1 -- 172.21.15.83:0/2227940731 shutdown_connections 2024-04-25T00:13:02.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.873+0000 7fabe7e00640 1 --2- 172.21.15.83:0/2227940731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabe8074740 0x7fabe815f930 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.873+0000 7fabe7e00640 1 --2- 172.21.15.83:0/2227940731 >> v2:172.21.15.86:3300/0 conn(0x7fabe80b18c0 0x7fabe80cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.873+0000 7fabe7e00640 1 -- 172.21.15.83:0/2227940731 >> 172.21.15.83:0/2227940731 conn(0x7fabe80d0210 msgr2=0x7fabe80d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:02.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.873+0000 7fabe7e00640 1 -- 172.21.15.83:0/2227940731 shutdown_connections 2024-04-25T00:13:02.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.873+0000 7fabe7e00640 1 -- 172.21.15.83:0/2227940731 wait complete. 2024-04-25T00:13:02.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7e00640 1 Processor -- start 2024-04-25T00:13:02.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7e00640 1 -- start start 2024-04-25T00:13:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabe8074740 0x7fabe80998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80b07d0 0x7fabe80b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabe80b18c0 0x7fabe80b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8169060 con 0x7fabe80b18c0 2024-04-25T00:13:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8168ee0 con 0x7fabe80b07d0 2024-04-25T00:13:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fabe81691e0 con 0x7fabe8074740 2024-04-25T00:13:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabe80b18c0 0x7fabe80b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabe80b18c0 0x7fabe80b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56258/0 (socket says 172.21.15.83:56258) 2024-04-25T00:13:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7400640 1 -- 172.21.15.83:0/2936645691 learned_addr learned my addr 172.21.15.83:0/2936645691 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe6a00640 1 --2- 172.21.15.83:0/2936645691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabe8074740 0x7fabe80998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe6000640 1 --2- 172.21.15.83:0/2936645691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80b07d0 0x7fabe80b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7400640 1 -- 172.21.15.83:0/2936645691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabe8074740 msgr2=0x7fabe80998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7400640 1 --2- 172.21.15.83:0/2936645691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabe8074740 0x7fabe80998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7400640 1 -- 172.21.15.83:0/2936645691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80b07d0 msgr2=0x7fabe80b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7400640 1 --2- 172.21.15.83:0/2936645691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80b07d0 0x7fabe80b3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7400640 1 -- 172.21.15.83:0/2936645691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe814f7c0 con 0x7fabe80b18c0 2024-04-25T00:13:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7400640 1 --2- 172.21.15.83:0/2936645691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabe80b18c0 0x7fabe80b3cb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fabe00b0040 tx=0x7fabe00196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabdea00640 1 -- 172.21.15.83:0/2936645691 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe00b2040 con 0x7fabe80b18c0 2024-04-25T00:13:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabdea00640 1 -- 172.21.15.83:0/2936645691 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe00bf070 con 0x7fabe80b18c0 2024-04-25T00:13:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7e00640 1 -- 172.21.15.83:0/2936645691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe814faa0 con 0x7fabe80b18c0 2024-04-25T00:13:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabdea00640 1 -- 172.21.15.83:0/2936645691 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe00b8890 con 0x7fabe80b18c0 2024-04-25T00:13:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7e00640 1 -- 172.21.15.83:0/2936645691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe80ce0c0 con 0x7fabe80b18c0 2024-04-25T00:13:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.877+0000 7fabe7e00640 1 -- 172.21.15.83:0/2936645691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe814fc70 con 0x7fabe80b18c0 2024-04-25T00:13:02.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.881+0000 7fabdea00640 1 -- 172.21.15.83:0/2936645691 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fabe00a5790 con 0x7fabe80b18c0 2024-04-25T00:13:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.881+0000 7fabdea00640 1 --2- 172.21.15.83:0/2936645691 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fabc80678c0 0x7fabc8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.881+0000 7fabdea00640 1 -- 172.21.15.83:0/2936645691 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fabe0120f20 con 0x7fabe80b18c0 2024-04-25T00:13:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.881+0000 7fabe6a00640 1 --2- 172.21.15.83:0/2936645691 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fabc80678c0 0x7fabc8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.881+0000 7fabe6a00640 1 --2- 172.21.15.83:0/2936645691 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fabc80678c0 0x7fabc8069d80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fabd8005890 tx=0x7fabd8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:02.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:02.885+0000 7fabdea00640 1 -- 172.21.15.83:0/2936645691 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fabe0016560 con 0x7fabe80b18c0 2024-04-25T00:13:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.041+0000 7fabe7e00640 1 -- 172.21.15.83:0/2936645691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabe804f510 con 0x7fabe80b18c0 2024-04-25T00:13:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.041+0000 7fabdea00640 1 -- 172.21.15.83:0/2936645691 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16858 (secure 0 0 0) 0x7fabe00f1300 con 0x7fabe80b18c0 2024-04-25T00:13:03.045 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:03.045 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":49,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:53.651846+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:50.394645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:51.018842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.045+0000 7fabe7e00640 1 -- 172.21.15.83:0/2936645691 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fabc80678c0 msgr2=0x7fabc8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.045+0000 7fabe7e00640 1 --2- 172.21.15.83:0/2936645691 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fabc80678c0 0x7fabc8069d80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fabd8005890 tx=0x7fabd8009040 comp rx=0 tx=0).stop 2024-04-25T00:13:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.045+0000 7fabe7e00640 1 -- 172.21.15.83:0/2936645691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabe80b18c0 msgr2=0x7fabe80b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.045+0000 7fabe7e00640 1 --2- 172.21.15.83:0/2936645691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabe80b18c0 0x7fabe80b3cb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fabe00b0040 tx=0x7fabe00196e0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.045+0000 7fabe7e00640 1 -- 172.21.15.83:0/2936645691 shutdown_connections 2024-04-25T00:13:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.045+0000 7fabe7e00640 1 --2- 172.21.15.83:0/2936645691 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fabc80678c0 0x7fabc8069d80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.045+0000 7fabe7e00640 1 --2- 172.21.15.83:0/2936645691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabe80b18c0 0x7fabe80b3cb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.045+0000 7fabe7e00640 1 --2- 172.21.15.83:0/2936645691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80b07d0 0x7fabe80b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.045+0000 7fabe7e00640 1 --2- 172.21.15.83:0/2936645691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabe8074740 0x7fabe80998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.045+0000 7fabe7e00640 1 -- 172.21.15.83:0/2936645691 >> 172.21.15.83:0/2936645691 conn(0x7fabe80d0210 msgr2=0x7fabe815ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.045+0000 7fabe7e00640 1 -- 172.21.15.83:0/2936645691 shutdown_connections 2024-04-25T00:13:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.045+0000 7fabe7e00640 1 -- 172.21.15.83:0/2936645691 wait complete. 2024-04-25T00:13:03.066 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:12:50.394645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:12:51.018842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:13:03.066 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:13:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd8000640 1 Processor -- start 2024-04-25T00:13:03.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd8000640 1 -- start start 2024-04-25T00:13:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd8000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bd006e950 0x7f8bd015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd8000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8bd015fe50 0x7f8bd00ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd8000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bd00ce000 con 0x7f8bd0159810 2024-04-25T00:13:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd8000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8bd006e210 con 0x7f8bd006e950 2024-04-25T00:13:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd8000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8bd01602a0 con 0x7f8bd015fe50 2024-04-25T00:13:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd4c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bd006e950 0x7f8bd015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd4c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bd006e950 0x7f8bd015f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56262/0 (socket says 172.21.15.83:56262) 2024-04-25T00:13:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd4c00640 1 -- 172.21.15.83:0/2876243087 learned_addr learned my addr 172.21.15.83:0/2876243087 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd6000640 1 --2- 172.21.15.83:0/2876243087 >> v2:172.21.15.86:3300/0 conn(0x7f8bd015fe50 0x7f8bd00ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd4c00640 1 -- 172.21.15.83:0/2876243087 >> v2:172.21.15.86:3300/0 conn(0x7f8bd015fe50 msgr2=0x7f8bd00ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd4c00640 1 --2- 172.21.15.83:0/2876243087 >> v2:172.21.15.86:3300/0 conn(0x7f8bd015fe50 0x7f8bd00ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd4c00640 1 -- 172.21.15.83:0/2876243087 >> v1:172.21.15.86:6789/0 conn(0x7f8bd0159810 legacy=0x7f8bd00cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.217+0000 7f8bd4c00640 1 -- 172.21.15.83:0/2876243087 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd006eec0 con 0x7f8bd006e950 2024-04-25T00:13:03.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.221+0000 7f8bd4c00640 1 --2- 172.21.15.83:0/2876243087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd006e950 0x7f8bd015f910 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00af750 tx=0x7f8bbc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75e9e52ec620cde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:03.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.221+0000 7f8bcfe00640 1 -- 172.21.15.83:0/2876243087 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0b2070 con 0x7f8bd006e950 2024-04-25T00:13:03.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.221+0000 7f8bcfe00640 1 -- 172.21.15.83:0/2876243087 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bbc0ad680 con 0x7f8bd006e950 2024-04-25T00:13:03.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.221+0000 7f8bcfe00640 1 -- 172.21.15.83:0/2876243087 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0ada50 con 0x7f8bd006e950 2024-04-25T00:13:03.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.221+0000 7f8bd8000640 1 -- 172.21.15.83:0/2876243087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd006e950 msgr2=0x7f8bd015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.221+0000 7f8bd8000640 1 --2- 172.21.15.83:0/2876243087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd006e950 0x7f8bd015f910 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00af750 tx=0x7f8bbc01c490 comp rx=0 tx=0).stop 2024-04-25T00:13:03.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.221+0000 7f8bd8000640 1 -- 172.21.15.83:0/2876243087 shutdown_connections 2024-04-25T00:13:03.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.221+0000 7f8bd8000640 1 --2- 172.21.15.83:0/2876243087 >> v2:172.21.15.86:3300/0 conn(0x7f8bd015fe50 0x7f8bd00ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.221+0000 7f8bd8000640 1 --2- 172.21.15.83:0/2876243087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd006e950 0x7f8bd015f910 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.221+0000 7f8bd8000640 1 -- 172.21.15.83:0/2876243087 >> 172.21.15.83:0/2876243087 conn(0x7f8bd00b1830 msgr2=0x7f8bd00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:03.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.221+0000 7f8bd8000640 1 -- 172.21.15.83:0/2876243087 shutdown_connections 2024-04-25T00:13:03.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.221+0000 7f8bd8000640 1 -- 172.21.15.83:0/2876243087 wait complete. 2024-04-25T00:13:03.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd8000640 1 Processor -- start 2024-04-25T00:13:03.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd8000640 1 -- start start 2024-04-25T00:13:03.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd8000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd006e950 0x7f8bd013e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd8000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd0159810 0x7f8bd0153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd8000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bd015fe50 0x7f8bd00afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd8000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd0070f20 con 0x7f8bd0159810 2024-04-25T00:13:03.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd8000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd0070da0 con 0x7f8bd006e950 2024-04-25T00:13:03.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd8000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd0168cb0 con 0x7f8bd015fe50 2024-04-25T00:13:03.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd0159810 0x7f8bd0153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd0159810 0x7f8bd0153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56270/0 (socket says 172.21.15.83:56270) 2024-04-25T00:13:03.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd4c00640 1 -- 172.21.15.83:0/2522508878 learned_addr learned my addr 172.21.15.83:0/2522508878 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:03.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd5600640 1 --2- 172.21.15.83:0/2522508878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd006e950 0x7f8bd013e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd6000640 1 --2- 172.21.15.83:0/2522508878 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bd015fe50 0x7f8bd00afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd4c00640 1 -- 172.21.15.83:0/2522508878 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bd015fe50 msgr2=0x7f8bd00afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd4c00640 1 --2- 172.21.15.83:0/2522508878 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bd015fe50 0x7f8bd00afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd4c00640 1 -- 172.21.15.83:0/2522508878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd006e950 msgr2=0x7f8bd013e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd4c00640 1 --2- 172.21.15.83:0/2522508878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd006e950 0x7f8bd013e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd4c00640 1 -- 172.21.15.83:0/2522508878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd0155e60 con 0x7f8bd0159810 2024-04-25T00:13:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd4c00640 1 --2- 172.21.15.83:0/2522508878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd0159810 0x7f8bd0153580 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc01a0c0 tx=0x7f8bbc01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bcd600640 1 -- 172.21.15.83:0/2522508878 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0b2040 con 0x7f8bd0159810 2024-04-25T00:13:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bcd600640 1 -- 172.21.15.83:0/2522508878 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bbc0ad9b0 con 0x7f8bd0159810 2024-04-25T00:13:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bcd600640 1 -- 172.21.15.83:0/2522508878 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0add80 con 0x7f8bd0159810 2024-04-25T00:13:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd8000640 1 -- 172.21.15.83:0/2522508878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd0156140 con 0x7f8bd0159810 2024-04-25T00:13:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.225+0000 7f8bd8000640 1 -- 172.21.15.83:0/2522508878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd0156700 con 0x7f8bd0159810 2024-04-25T00:13:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.229+0000 7f8bd8000640 1 -- 172.21.15.83:0/2522508878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd004f510 con 0x7f8bd0159810 2024-04-25T00:13:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.229+0000 7f8bcd600640 1 -- 172.21.15.83:0/2522508878 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8bbc0bf030 con 0x7f8bd0159810 2024-04-25T00:13:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.229+0000 7f8bcd600640 1 --2- 172.21.15.83:0/2522508878 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8bb00678f0 0x7f8bb0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.229+0000 7f8bcd600640 1 -- 172.21.15.83:0/2522508878 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f8bbc11f230 con 0x7f8bd0159810 2024-04-25T00:13:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.229+0000 7f8bd5600640 1 --2- 172.21.15.83:0/2522508878 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8bb00678f0 0x7f8bb0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.229+0000 7f8bd5600640 1 --2- 172.21.15.83:0/2522508878 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8bb00678f0 0x7f8bb0069db0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4007fd0 tx=0x7f8bc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:03.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.233+0000 7f8bcd600640 1 -- 172.21.15.83:0/2522508878 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8bbc016110 con 0x7f8bd0159810 2024-04-25T00:13:03.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.389+0000 7f8bd8000640 1 -- 172.21.15.83:0/2522508878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8bd0072c90 con 0x7f8bd0159810 2024-04-25T00:13:03.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.389+0000 7f8bcd600640 1 -- 172.21.15.83:0/2522508878 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) ==== 93+0+10 (secure 0 0 0) 0x7f8bbc0ef580 con 0x7f8bd0159810 2024-04-25T00:13:03.391 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:13:03.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.389+0000 7f8bd8000640 1 -- 172.21.15.83:0/2522508878 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8bb00678f0 msgr2=0x7f8bb0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.389+0000 7f8bd8000640 1 --2- 172.21.15.83:0/2522508878 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8bb00678f0 0x7f8bb0069db0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4007fd0 tx=0x7f8bc4009040 comp rx=0 tx=0).stop 2024-04-25T00:13:03.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.389+0000 7f8bd8000640 1 -- 172.21.15.83:0/2522508878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd0159810 msgr2=0x7f8bd0153580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.389+0000 7f8bd8000640 1 --2- 172.21.15.83:0/2522508878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd0159810 0x7f8bd0153580 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc01a0c0 tx=0x7f8bbc01acd0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.389+0000 7f8bd8000640 1 -- 172.21.15.83:0/2522508878 shutdown_connections 2024-04-25T00:13:03.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.389+0000 7f8bd8000640 1 --2- 172.21.15.83:0/2522508878 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8bb00678f0 0x7f8bb0069db0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.389+0000 7f8bd8000640 1 --2- 172.21.15.83:0/2522508878 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bd015fe50 0x7f8bd00afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.389+0000 7f8bd8000640 1 --2- 172.21.15.83:0/2522508878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd0159810 0x7f8bd0153580 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.389+0000 7f8bd8000640 1 --2- 172.21.15.83:0/2522508878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd006e950 0x7f8bd013e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.389+0000 7f8bd8000640 1 -- 172.21.15.83:0/2522508878 >> 172.21.15.83:0/2522508878 conn(0x7f8bd00b1830 msgr2=0x7f8bd00ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:03.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.389+0000 7f8bd8000640 1 -- 172.21.15.83:0/2522508878 shutdown_connections 2024-04-25T00:13:03.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.389+0000 7f8bd8000640 1 -- 172.21.15.83:0/2522508878 wait complete. 2024-04-25T00:13:03.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:13:03.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.549+0000 7f2f29200640 1 Processor -- start 2024-04-25T00:13:03.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.549+0000 7f2f29200640 1 -- start start 2024-04-25T00:13:03.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.549+0000 7f2f29200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2f241551e0 0x7f2f2415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.549+0000 7f2f29200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f2415c560 con 0x7f2f2406e4c0 2024-04-25T00:13:03.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.549+0000 7f2f29200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f24072460 con 0x7f2f241555c0 2024-04-25T00:13:03.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.549+0000 7f2f29200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2f2415c7d0 con 0x7f2f241551e0 2024-04-25T00:13:03.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.549+0000 7f2f22800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2f2406e4c0 0x7f2f2415cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42318/0 (socket says 172.21.15.83:42318) 2024-04-25T00:13:03.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.549+0000 7f2f22800640 1 -- 172.21.15.83:0/2233090288 learned_addr learned my addr 172.21.15.83:0/2233090288 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:03.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.549+0000 7f2f21e00640 1 --2- 172.21.15.83:0/2233090288 >> v2:172.21.15.86:3300/0 conn(0x7f2f241551e0 0x7f2f2415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.549+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1431868419 0 0) 0x7f2f2415c560 con 0x7f2f2406e4c0 2024-04-25T00:13:03.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.549+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f08003530 con 0x7f2f2406e4c0 2024-04-25T00:13:03.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.549+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2409042723 0 0) 0x7f2f24072460 con 0x7f2f241555c0 2024-04-25T00:13:03.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.549+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f2415c560 con 0x7f2f241555c0 2024-04-25T00:13:03.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1835863290 0 0) 0x7f2f08003530 con 0x7f2f2406e4c0 2024-04-25T00:13:03.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 >> v2:172.21.15.86:3300/0 conn(0x7f2f241551e0 msgr2=0x7f2f2415bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:03.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f21400640 1 --2- 172.21.15.83:0/2233090288 >> v2:172.21.15.86:3300/0 conn(0x7f2f241551e0 0x7f2f2415bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 >> v1:172.21.15.86:6789/0 conn(0x7f2f241555c0 legacy=0x7f2f24071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2415cab0 con 0x7f2f2406e4c0 2024-04-25T00:13:03.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f2f1c001680 con 0x7f2f2406e4c0 2024-04-25T00:13:03.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 >> v1:172.21.15.83:6789/0 conn(0x7f2f2406e4c0 legacy=0x7f2f2415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f21400640 1 --2- 172.21.15.83:0/2233090288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f08003bc0 0x7f2f08003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f21400640 1 --2- 172.21.15.83:0/2233090288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f08004810 0x7f2f08004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f21400640 1 --2- 172.21.15.83:0/2233090288 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f080051d0 0x7f2f0800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2415c7d0 con 0x7f2f08004810 2024-04-25T00:13:03.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0800e2e0 con 0x7f2f08003bc0 2024-04-25T00:13:03.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0800e4e0 con 0x7f2f080051d0 2024-04-25T00:13:03.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f19400640 1 --2- 172.21.15.83:0/2233090288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f08004810 0x7f2f08004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f21e00640 1 --2- 172.21.15.83:0/2233090288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f08003bc0 0x7f2f08003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f22800640 1 --2- 172.21.15.83:0/2233090288 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f080051d0 0x7f2f0800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f19400640 1 -- 172.21.15.83:0/2233090288 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f080051d0 msgr2=0x7f2f0800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:03.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f19400640 1 --2- 172.21.15.83:0/2233090288 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f080051d0 0x7f2f0800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f19400640 1 -- 172.21.15.83:0/2233090288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f08003bc0 msgr2=0x7f2f08003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:03.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f19400640 1 --2- 172.21.15.83:0/2233090288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f08003bc0 0x7f2f08003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f19400640 1 -- 172.21.15.83:0/2233090288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f08003530 con 0x7f2f08004810 2024-04-25T00:13:03.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f22800640 1 --2- 172.21.15.83:0/2233090288 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f080051d0 0x7f2f0800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:03.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f22800640 1 -- 172.21.15.83:0/2233090288 reap_dead start 2024-04-25T00:13:03.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.553+0000 7f2f19400640 1 --2- 172.21.15.83:0/2233090288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f08004810 0x7f2f08004c90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2f240af770 tx=0x7f2f0c01ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bb9f4b14c2d96e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:03.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c001520 con 0x7f2f08004810 2024-04-25T00:13:03.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0c0052f0 con 0x7f2f08004810 2024-04-25T00:13:03.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f21400640 1 -- 172.21.15.83:0/2233090288 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c0a8910 con 0x7f2f08004810 2024-04-25T00:13:03.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 -- 172.21.15.83:0/2233090288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f08004810 msgr2=0x7f2f08004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 --2- 172.21.15.83:0/2233090288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f08004810 0x7f2f08004c90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2f240af770 tx=0x7f2f0c01ecb0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 -- 172.21.15.83:0/2233090288 shutdown_connections 2024-04-25T00:13:03.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 --2- 172.21.15.83:0/2233090288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f08004810 0x7f2f08004c90 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 -- 172.21.15.83:0/2233090288 >> 172.21.15.83:0/2233090288 conn(0x7f2f240b1830 msgr2=0x7f2f240b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:03.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 -- 172.21.15.83:0/2233090288 shutdown_connections 2024-04-25T00:13:03.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 -- 172.21.15.83:0/2233090288 wait complete. 2024-04-25T00:13:03.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 Processor -- start 2024-04-25T00:13:03.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 -- start start 2024-04-25T00:13:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f08004810 0x7f2f2414b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f240b0240 0x7f2f2413bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f24152560 0x7f2f240cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2415c020 con 0x7f2f08004810 2024-04-25T00:13:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2415bea0 con 0x7f2f24152560 2024-04-25T00:13:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f29200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2f2415c1a0 con 0x7f2f240b0240 2024-04-25T00:13:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f08004810 0x7f2f2414b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f08004810 0x7f2f2414b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56276/0 (socket says 172.21.15.83:56276) 2024-04-25T00:13:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f21e00640 1 -- 172.21.15.83:0/1235202500 learned_addr learned my addr 172.21.15.83:0/1235202500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f19400640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f240b0240 0x7f2f2413bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f22800640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f24152560 0x7f2f240cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f21e00640 1 -- 172.21.15.83:0/1235202500 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f240b0240 msgr2=0x7f2f2413bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f21e00640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f240b0240 0x7f2f2413bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f21e00640 1 -- 172.21.15.83:0/1235202500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f24152560 msgr2=0x7f2f240cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f21e00640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f24152560 0x7f2f240cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f21e00640 1 -- 172.21.15.83:0/1235202500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f24152e70 con 0x7f2f08004810 2024-04-25T00:13:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.557+0000 7f2f22800640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f24152560 0x7f2f240cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.561+0000 7f2f21e00640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f08004810 0x7f2f2414b490 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2f14011e60 tx=0x7f2f1400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.561+0000 7f2f1aa00640 1 -- 172.21.15.83:0/1235202500 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14015630 con 0x7f2f08004810 2024-04-25T00:13:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.561+0000 7f2f1aa00640 1 -- 172.21.15.83:0/1235202500 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f140157d0 con 0x7f2f08004810 2024-04-25T00:13:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.561+0000 7f2f1aa00640 1 -- 172.21.15.83:0/1235202500 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14015ba0 con 0x7f2f08004810 2024-04-25T00:13:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.561+0000 7f2f29200640 1 -- 172.21.15.83:0/1235202500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f240af760 con 0x7f2f08004810 2024-04-25T00:13:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.561+0000 7f2f29200640 1 -- 172.21.15.83:0/1235202500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f240b06c0 con 0x7f2f08004810 2024-04-25T00:13:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.561+0000 7f2f29200640 1 -- 172.21.15.83:0/1235202500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f2404f510 con 0x7f2f08004810 2024-04-25T00:13:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.561+0000 7f2f1aa00640 1 -- 172.21.15.83:0/1235202500 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2f1401d400 con 0x7f2f08004810 2024-04-25T00:13:03.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.561+0000 7f2f1aa00640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2f000678c0 0x7f2f00069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.561+0000 7f2f1aa00640 1 -- 172.21.15.83:0/1235202500 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f2f14086fe0 con 0x7f2f08004810 2024-04-25T00:13:03.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.561+0000 7f2f19400640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2f000678c0 0x7f2f00069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.565+0000 7f2f19400640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2f000678c0 0x7f2f00069d80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c0b0000 tx=0x7f2f0c0aa070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.565+0000 7f2f1aa00640 1 -- 172.21.15.83:0/1235202500 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2f14019030 con 0x7f2f08004810 2024-04-25T00:13:03.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.721+0000 7f2f29200640 1 -- 172.21.15.83:0/1235202500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2f240cbb10 con 0x7f2f08004810 2024-04-25T00:13:03.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.721+0000 7f2f1aa00640 1 -- 172.21.15.83:0/1235202500 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) ==== 104+0+11 (secure 0 0 0) 0x7f2f14010040 con 0x7f2f08004810 2024-04-25T00:13:03.725 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:13:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.725+0000 7f2f29200640 1 -- 172.21.15.83:0/1235202500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2f000678c0 msgr2=0x7f2f00069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.725+0000 7f2f29200640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2f000678c0 0x7f2f00069d80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c0b0000 tx=0x7f2f0c0aa070 comp rx=0 tx=0).stop 2024-04-25T00:13:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.725+0000 7f2f29200640 1 -- 172.21.15.83:0/1235202500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f08004810 msgr2=0x7f2f2414b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.725+0000 7f2f29200640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f08004810 0x7f2f2414b490 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2f14011e60 tx=0x7f2f1400f580 comp rx=0 tx=0).stop 2024-04-25T00:13:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.725+0000 7f2f29200640 1 -- 172.21.15.83:0/1235202500 shutdown_connections 2024-04-25T00:13:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.725+0000 7f2f29200640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2f000678c0 0x7f2f00069d80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.725+0000 7f2f29200640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f240b0240 0x7f2f2413bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.725+0000 7f2f29200640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f24152560 0x7f2f240cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.725+0000 7f2f29200640 1 --2- 172.21.15.83:0/1235202500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f08004810 0x7f2f2414b490 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.725+0000 7f2f29200640 1 -- 172.21.15.83:0/1235202500 >> 172.21.15.83:0/1235202500 conn(0x7f2f240b1830 msgr2=0x7f2f2406e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.725+0000 7f2f29200640 1 -- 172.21.15.83:0/1235202500 shutdown_connections 2024-04-25T00:13:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.725+0000 7f2f29200640 1 -- 172.21.15.83:0/1235202500 wait complete. 2024-04-25T00:13:03.747 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:13:03.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.881+0000 7efcb4600640 1 Processor -- start 2024-04-25T00:13:03.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.881+0000 7efcb4600640 1 -- start start 2024-04-25T00:13:03.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcb4600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efcac06e950 0x7efcac15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcb4600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efcac15fe50 0x7efcac0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcb4600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcac0ce000 con 0x7efcac159810 2024-04-25T00:13:03.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcb4600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7efcac06e210 con 0x7efcac15fe50 2024-04-25T00:13:03.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcb4600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efcac06da00 con 0x7efcac06e950 2024-04-25T00:13:03.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcb1c00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efcac159810 0x7efcac0cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42326/0 (socket says 172.21.15.83:42326) 2024-04-25T00:13:03.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcb1c00640 1 -- 172.21.15.83:0/1218570646 learned_addr learned my addr 172.21.15.83:0/1218570646 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:03.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcb2600640 1 --2- 172.21.15.83:0/1218570646 >> v2:172.21.15.83:3300/0 conn(0x7efcac15fe50 0x7efcac0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcb1200640 1 --2- 172.21.15.83:0/1218570646 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efcac06e950 0x7efcac15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcabe00640 1 -- 172.21.15.83:0/1218570646 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3715496634 0 0) 0x7efcac0ce000 con 0x7efcac159810 2024-04-25T00:13:03.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcabe00640 1 -- 172.21.15.83:0/1218570646 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc94003530 con 0x7efcac159810 2024-04-25T00:13:03.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcb2600640 1 -- 172.21.15.83:0/1218570646 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efcac06e950 msgr2=0x7efcac15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:03.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcb2600640 1 --2- 172.21.15.83:0/1218570646 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efcac06e950 0x7efcac15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcb2600640 1 -- 172.21.15.83:0/1218570646 >> v1:172.21.15.83:6789/0 conn(0x7efcac159810 legacy=0x7efcac0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.885+0000 7efcb2600640 1 -- 172.21.15.83:0/1218570646 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcac06eec0 con 0x7efcac15fe50 2024-04-25T00:13:03.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.889+0000 7efcb2600640 1 --2- 172.21.15.83:0/1218570646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcac15fe50 0x7efcac0ce4a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efca4009d90 tx=0x7efca401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba7f945b31ea752 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:03.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.889+0000 7efcabe00640 1 -- 172.21.15.83:0/1218570646 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca40b2070 con 0x7efcac15fe50 2024-04-25T00:13:03.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.889+0000 7efcabe00640 1 -- 172.21.15.83:0/1218570646 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca40a5890 con 0x7efcac15fe50 2024-04-25T00:13:03.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.889+0000 7efcabe00640 1 -- 172.21.15.83:0/1218570646 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca40a5c60 con 0x7efcac15fe50 2024-04-25T00:13:03.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.889+0000 7efcb4600640 1 -- 172.21.15.83:0/1218570646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcac15fe50 msgr2=0x7efcac0ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.889+0000 7efcb4600640 1 --2- 172.21.15.83:0/1218570646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcac15fe50 0x7efcac0ce4a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efca4009d90 tx=0x7efca401c8e0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.889+0000 7efcb4600640 1 -- 172.21.15.83:0/1218570646 shutdown_connections 2024-04-25T00:13:03.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.889+0000 7efcb4600640 1 --2- 172.21.15.83:0/1218570646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcac15fe50 0x7efcac0ce4a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.889+0000 7efcb4600640 1 --2- 172.21.15.83:0/1218570646 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efcac06e950 0x7efcac15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.889+0000 7efcb4600640 1 -- 172.21.15.83:0/1218570646 >> 172.21.15.83:0/1218570646 conn(0x7efcac0b1830 msgr2=0x7efcac0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:03.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.889+0000 7efcb4600640 1 -- 172.21.15.83:0/1218570646 shutdown_connections 2024-04-25T00:13:03.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.889+0000 7efcb4600640 1 -- 172.21.15.83:0/1218570646 wait complete. 2024-04-25T00:13:03.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.889+0000 7efcb4600640 1 Processor -- start 2024-04-25T00:13:03.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.889+0000 7efcb4600640 1 -- start start 2024-04-25T00:13:03.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcac06e950 0x7efcac142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb4600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcac159810 0x7efcac1556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb4600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efcac15fe50 0x7efcac155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb4600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efcac070f20 con 0x7efcac159810 2024-04-25T00:13:03.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efcac070da0 con 0x7efcac06e950 2024-04-25T00:13:03.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb4600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efcac168cb0 con 0x7efcac15fe50 2024-04-25T00:13:03.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcac159810 0x7efcac1556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcac159810 0x7efcac1556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56280/0 (socket says 172.21.15.83:56280) 2024-04-25T00:13:03.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb1200640 1 -- 172.21.15.83:0/1643620835 learned_addr learned my addr 172.21.15.83:0/1643620835 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:03.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb2600640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efcac15fe50 0x7efcac155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb1c00640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcac06e950 0x7efcac142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb1200640 1 -- 172.21.15.83:0/1643620835 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efcac15fe50 msgr2=0x7efcac155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb1200640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efcac15fe50 0x7efcac155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb1200640 1 -- 172.21.15.83:0/1643620835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcac06e950 msgr2=0x7efcac142570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:03.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb1200640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcac06e950 0x7efcac142570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:03.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb1200640 1 -- 172.21.15.83:0/1643620835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcac0b2f60 con 0x7efcac159810 2024-04-25T00:13:03.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb2600640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efcac15fe50 0x7efcac155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:03.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb1200640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcac159810 0x7efcac1556b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7efc9800e8a0 tx=0x7efc9800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:03.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efca9600640 1 -- 172.21.15.83:0/1643620835 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98017070 con 0x7efcac159810 2024-04-25T00:13:03.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efca9600640 1 -- 172.21.15.83:0/1643620835 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc9800c940 con 0x7efcac159810 2024-04-25T00:13:03.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb4600640 1 -- 172.21.15.83:0/1643620835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcac165260 con 0x7efcac159810 2024-04-25T00:13:03.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efca9600640 1 -- 172.21.15.83:0/1643620835 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9800cd10 con 0x7efcac159810 2024-04-25T00:13:03.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb4600640 1 -- 172.21.15.83:0/1643620835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcac151c40 con 0x7efcac159810 2024-04-25T00:13:03.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efcb4600640 1 -- 172.21.15.83:0/1643620835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcac04f510 con 0x7efcac159810 2024-04-25T00:13:03.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.893+0000 7efca9600640 1 -- 172.21.15.83:0/1643620835 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efc9801a070 con 0x7efcac159810 2024-04-25T00:13:03.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.897+0000 7efca9600640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc8c0678c0 0x7efc8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:03.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.897+0000 7efca9600640 1 -- 172.21.15.83:0/1643620835 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7efc98085de0 con 0x7efcac159810 2024-04-25T00:13:03.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.897+0000 7efcb1c00640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc8c0678c0 0x7efc8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:03.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.897+0000 7efcb1c00640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc8c0678c0 0x7efc8c069d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efca0005890 tx=0x7efca0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:03.901+0000 7efca9600640 1 -- 172.21.15.83:0/1643620835 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efcac0b2f60 con 0x7efcac159810 2024-04-25T00:13:04.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.053+0000 7efcb4600640 1 -- 172.21.15.83:0/1643620835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efcac159020 con 0x7efcac159810 2024-04-25T00:13:04.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.053+0000 7efca9600640 1 -- 172.21.15.83:0/1643620835 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) ==== 100+0+11 (secure 0 0 0) 0x7efc980566f0 con 0x7efcac159810 2024-04-25T00:13:04.055 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:13:04.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.053+0000 7efcb4600640 1 -- 172.21.15.83:0/1643620835 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc8c0678c0 msgr2=0x7efc8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:04.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.053+0000 7efcb4600640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc8c0678c0 0x7efc8c069d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efca0005890 tx=0x7efca0009040 comp rx=0 tx=0).stop 2024-04-25T00:13:04.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.053+0000 7efcb4600640 1 -- 172.21.15.83:0/1643620835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcac159810 msgr2=0x7efcac1556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:04.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.053+0000 7efcb4600640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcac159810 0x7efcac1556b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7efc9800e8a0 tx=0x7efc9800ed60 comp rx=0 tx=0).stop 2024-04-25T00:13:04.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.053+0000 7efcb4600640 1 -- 172.21.15.83:0/1643620835 shutdown_connections 2024-04-25T00:13:04.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.053+0000 7efcb4600640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc8c0678c0 0x7efc8c069d80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.053+0000 7efcb4600640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efcac15fe50 0x7efcac155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.053+0000 7efcb4600640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcac159810 0x7efcac1556b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.053+0000 7efcb4600640 1 --2- 172.21.15.83:0/1643620835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcac06e950 0x7efcac142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.053+0000 7efcb4600640 1 -- 172.21.15.83:0/1643620835 >> 172.21.15.83:0/1643620835 conn(0x7efcac0b1830 msgr2=0x7efcac0710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:04.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.053+0000 7efcb4600640 1 -- 172.21.15.83:0/1643620835 shutdown_connections 2024-04-25T00:13:04.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.053+0000 7efcb4600640 1 -- 172.21.15.83:0/1643620835 wait complete. 2024-04-25T00:13:04.077 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:13:04.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:13:04.189 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:13:04.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f803b600640 1 Processor -- start 2024-04-25T00:13:04.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f803b600640 1 -- start start 2024-04-25T00:13:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f803b600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f80341446e0 0x7f80340cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f803b600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f803415f350 0x7f803406e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f803b600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8034072ff0 con 0x7f803406ec50 2024-04-25T00:13:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f803b600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f80340715b0 con 0x7f80341446e0 2024-04-25T00:13:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f803b600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8034144ac0 con 0x7f803415f350 2024-04-25T00:13:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f8038c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f80341446e0 0x7f80340cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f8038c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f80341446e0 0x7f80340cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56294/0 (socket says 172.21.15.83:56294) 2024-04-25T00:13:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f8038c00640 1 -- 172.21.15.83:0/3909931457 learned_addr learned my addr 172.21.15.83:0/3909931457 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:04.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f8033e00640 1 --2- 172.21.15.83:0/3909931457 >> v2:172.21.15.86:3300/0 conn(0x7f803415f350 0x7f803406e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f8038c00640 1 -- 172.21.15.83:0/3909931457 >> v2:172.21.15.86:3300/0 conn(0x7f803415f350 msgr2=0x7f803406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:04.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f8038c00640 1 --2- 172.21.15.83:0/3909931457 >> v2:172.21.15.86:3300/0 conn(0x7f803415f350 0x7f803406e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f8038c00640 1 -- 172.21.15.83:0/3909931457 >> v1:172.21.15.86:6789/0 conn(0x7f803406ec50 legacy=0x7f803415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:04.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f8038c00640 1 -- 172.21.15.83:0/3909931457 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80340cb210 con 0x7f80341446e0 2024-04-25T00:13:04.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.213+0000 7f8033e00640 1 --2- 172.21.15.83:0/3909931457 >> v2:172.21.15.86:3300/0 conn(0x7f803415f350 0x7f803406e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.217+0000 7f8038c00640 1 --2- 172.21.15.83:0/3909931457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80341446e0 0x7f80340cab50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8034068a60 tx=0x7f802801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2633296738fa575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.217+0000 7f8033400640 1 -- 172.21.15.83:0/3909931457 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80280b0070 con 0x7f80341446e0 2024-04-25T00:13:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.217+0000 7f8033400640 1 -- 172.21.15.83:0/3909931457 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80280ab600 con 0x7f80341446e0 2024-04-25T00:13:04.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.217+0000 7f8033400640 1 -- 172.21.15.83:0/3909931457 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80280ab9d0 con 0x7f80341446e0 2024-04-25T00:13:04.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.217+0000 7f803b600640 1 -- 172.21.15.83:0/3909931457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80341446e0 msgr2=0x7f80340cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:04.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.217+0000 7f803b600640 1 --2- 172.21.15.83:0/3909931457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80341446e0 0x7f80340cab50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8034068a60 tx=0x7f802801c4e0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.217+0000 7f803b600640 1 -- 172.21.15.83:0/3909931457 shutdown_connections 2024-04-25T00:13:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.217+0000 7f803b600640 1 --2- 172.21.15.83:0/3909931457 >> v2:172.21.15.86:3300/0 conn(0x7f803415f350 0x7f803406e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.217+0000 7f803b600640 1 --2- 172.21.15.83:0/3909931457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80341446e0 0x7f80340cab50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.217+0000 7f803b600640 1 -- 172.21.15.83:0/3909931457 >> 172.21.15.83:0/3909931457 conn(0x7f80340cfa60 msgr2=0x7f80340cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.217+0000 7f803b600640 1 -- 172.21.15.83:0/3909931457 shutdown_connections 2024-04-25T00:13:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.217+0000 7f803b600640 1 -- 172.21.15.83:0/3909931457 wait complete. 2024-04-25T00:13:04.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f803b600640 1 Processor -- start 2024-04-25T00:13:04.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f803b600640 1 -- start start 2024-04-25T00:13:04.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f803b600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f803406ec50 0x7f80340b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f803b600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80341446e0 0x7f80340b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f803b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f803415f350 0x7f803415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f803b600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8034167a90 con 0x7f80341446e0 2024-04-25T00:13:04.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f803b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f803415f760 con 0x7f803415f350 2024-04-25T00:13:04.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f803b600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8034167c10 con 0x7f803406ec50 2024-04-25T00:13:04.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f8033e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80341446e0 0x7f80340b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f8033e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80341446e0 0x7f80340b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56306/0 (socket says 172.21.15.83:56306) 2024-04-25T00:13:04.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f8033e00640 1 -- 172.21.15.83:0/2276686444 learned_addr learned my addr 172.21.15.83:0/2276686444 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:04.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f8038c00640 1 --2- 172.21.15.83:0/2276686444 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f803406ec50 0x7f80340b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f8039600640 1 --2- 172.21.15.83:0/2276686444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f803415f350 0x7f803415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f8033e00640 1 -- 172.21.15.83:0/2276686444 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f803406ec50 msgr2=0x7f80340b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:04.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f8033e00640 1 --2- 172.21.15.83:0/2276686444 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f803406ec50 0x7f80340b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f8033e00640 1 -- 172.21.15.83:0/2276686444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f803415f350 msgr2=0x7f803415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:04.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f8033e00640 1 --2- 172.21.15.83:0/2276686444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f803415f350 0x7f803415a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f8033e00640 1 -- 172.21.15.83:0/2276686444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80340b4aa0 con 0x7f80341446e0 2024-04-25T00:13:04.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f8033e00640 1 --2- 172.21.15.83:0/2276686444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80341446e0 0x7f80340b30e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f802000c7f0 tx=0x7f802000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:04.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f8030c00640 1 -- 172.21.15.83:0/2276686444 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020007a30 con 0x7f80341446e0 2024-04-25T00:13:04.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f8030c00640 1 -- 172.21.15.83:0/2276686444 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8020007bd0 con 0x7f80341446e0 2024-04-25T00:13:04.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f803b600640 1 -- 172.21.15.83:0/2276686444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80340b3620 con 0x7f80341446e0 2024-04-25T00:13:04.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f8030c00640 1 -- 172.21.15.83:0/2276686444 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80200113f0 con 0x7f80341446e0 2024-04-25T00:13:04.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.221+0000 7f803b600640 1 -- 172.21.15.83:0/2276686444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80340b4d40 con 0x7f80341446e0 2024-04-25T00:13:04.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.225+0000 7f803b600640 1 -- 172.21.15.83:0/2276686444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8034068230 con 0x7f80341446e0 2024-04-25T00:13:04.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.225+0000 7f8030c00640 1 -- 172.21.15.83:0/2276686444 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8020011590 con 0x7f80341446e0 2024-04-25T00:13:04.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.225+0000 7f8030c00640 1 --2- 172.21.15.83:0/2276686444 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80140678c0 0x7f8014069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.225+0000 7f8030c00640 1 -- 172.21.15.83:0/2276686444 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f8020085f70 con 0x7f80341446e0 2024-04-25T00:13:04.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.225+0000 7f8038c00640 1 --2- 172.21.15.83:0/2276686444 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80140678c0 0x7f8014069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.225+0000 7f8038c00640 1 --2- 172.21.15.83:0/2276686444 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80140678c0 0x7f8014069d80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f80280ad000 tx=0x7f8028008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:04.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.229+0000 7f8030c00640 1 -- 172.21.15.83:0/2276686444 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8020015020 con 0x7f80341446e0 2024-04-25T00:13:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.397+0000 7f803b600640 1 -- 172.21.15.83:0/2276686444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8034140ec0 con 0x7f80341446e0 2024-04-25T00:13:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.397+0000 7f8030c00640 1 -- 172.21.15.83:0/2276686444 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) ==== 80+0+0 (secure 0 0 0) 0x7f80200562d0 con 0x7f80341446e0 2024-04-25T00:13:04.401 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:13:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.401+0000 7f803b600640 1 -- 172.21.15.83:0/2276686444 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80140678c0 msgr2=0x7f8014069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.401+0000 7f803b600640 1 --2- 172.21.15.83:0/2276686444 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80140678c0 0x7f8014069d80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f80280ad000 tx=0x7f8028008040 comp rx=0 tx=0).stop 2024-04-25T00:13:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.401+0000 7f803b600640 1 -- 172.21.15.83:0/2276686444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80341446e0 msgr2=0x7f80340b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.401+0000 7f803b600640 1 --2- 172.21.15.83:0/2276686444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80341446e0 0x7f80340b30e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f802000c7f0 tx=0x7f802000ccb0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.401+0000 7f803b600640 1 -- 172.21.15.83:0/2276686444 shutdown_connections 2024-04-25T00:13:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.401+0000 7f803b600640 1 --2- 172.21.15.83:0/2276686444 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80140678c0 0x7f8014069d80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.401+0000 7f803b600640 1 --2- 172.21.15.83:0/2276686444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f803415f350 0x7f803415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.401+0000 7f803b600640 1 --2- 172.21.15.83:0/2276686444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80341446e0 0x7f80340b30e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.401+0000 7f803b600640 1 --2- 172.21.15.83:0/2276686444 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f803406ec50 0x7f80340b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.401+0000 7f803b600640 1 -- 172.21.15.83:0/2276686444 >> 172.21.15.83:0/2276686444 conn(0x7f80340cfa60 msgr2=0x7f8034143900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.401+0000 7f803b600640 1 -- 172.21.15.83:0/2276686444 shutdown_connections 2024-04-25T00:13:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.401+0000 7f803b600640 1 -- 172.21.15.83:0/2276686444 wait complete. 2024-04-25T00:13:04.422 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T00:13:04.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:04.423 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T00:13:04.450 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:13:04.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:04.450 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:13:04.503 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:13:04.503 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:13:04.503 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:04.512 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:13:04.512 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:13:04.513 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:13:04.513 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:04.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:13:04.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.689+0000 7fe151200640 1 Processor -- start 2024-04-25T00:13:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.689+0000 7fe151200640 1 -- start start 2024-04-25T00:13:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe151200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe14c1553e0 0x7fe14c06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe151200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe14c0b4a70 0x7fe14c0b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe151200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe14c06ef80 con 0x7fe14c06ea00 2024-04-25T00:13:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe151200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe14c15b510 con 0x7fe14c0b4a70 2024-04-25T00:13:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe151200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe14c15b7f0 con 0x7fe14c1553e0 2024-04-25T00:13:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe141400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe14c06ea00 0x7fe14c0b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33282/0 (socket says 172.21.15.83:33282) 2024-04-25T00:13:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe141400640 1 -- 172.21.15.83:0/3489134895 learned_addr learned my addr 172.21.15.83:0/3489134895 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe14a800640 1 --2- 172.21.15.83:0/3489134895 >> v2:172.21.15.86:3300/0 conn(0x7fe14c0b4a70 0x7fe14c0b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149e00640 1 --2- 172.21.15.83:0/3489134895 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe14c1553e0 0x7fe14c06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 -- 172.21.15.83:0/3489134895 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3468213931 0 0) 0x7fe14c06ef80 con 0x7fe14c06ea00 2024-04-25T00:13:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 -- 172.21.15.83:0/3489134895 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe130003530 con 0x7fe14c06ea00 2024-04-25T00:13:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 -- 172.21.15.83:0/3489134895 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 259323310 0 0) 0x7fe130003530 con 0x7fe14c06ea00 2024-04-25T00:13:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 -- 172.21.15.83:0/3489134895 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe14c1553e0 msgr2=0x7fe14c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 --2- 172.21.15.83:0/3489134895 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe14c1553e0 0x7fe14c06e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 -- 172.21.15.83:0/3489134895 >> v2:172.21.15.86:3300/0 conn(0x7fe14c0b4a70 msgr2=0x7fe14c0b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 --2- 172.21.15.83:0/3489134895 >> v2:172.21.15.86:3300/0 conn(0x7fe14c0b4a70 0x7fe14c0b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 -- 172.21.15.83:0/3489134895 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14c15bad0 con 0x7fe14c06ea00 2024-04-25T00:13:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 -- 172.21.15.83:0/3489134895 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fe134001680 con 0x7fe14c06ea00 2024-04-25T00:13:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 -- 172.21.15.83:0/3489134895 >> v1:172.21.15.83:6789/0 conn(0x7fe14c06ea00 legacy=0x7fe14c0b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 --2- 172.21.15.83:0/3489134895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1300039c0 0x7fe130003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 --2- 172.21.15.83:0/3489134895 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe14c1553e0 0x7fe130004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 --2- 172.21.15.83:0/3489134895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe130004d50 0x7fe1300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 -- 172.21.15.83:0/3489134895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c15b510 con 0x7fe1300039c0 2024-04-25T00:13:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 -- 172.21.15.83:0/3489134895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c15b7f0 con 0x7fe130004d50 2024-04-25T00:13:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149400640 1 -- 172.21.15.83:0/3489134895 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe13000dc10 con 0x7fe14c1553e0 2024-04-25T00:13:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149e00640 1 --2- 172.21.15.83:0/3489134895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1300039c0 0x7fe130003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe141400640 1 --2- 172.21.15.83:0/3489134895 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe14c1553e0 0x7fe130004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe14a800640 1 --2- 172.21.15.83:0/3489134895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe130004d50 0x7fe1300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149e00640 1 -- 172.21.15.83:0/3489134895 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe14c1553e0 msgr2=0x7fe130004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149e00640 1 --2- 172.21.15.83:0/3489134895 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe14c1553e0 0x7fe130004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149e00640 1 -- 172.21.15.83:0/3489134895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe130004d50 msgr2=0x7fe1300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149e00640 1 --2- 172.21.15.83:0/3489134895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe130004d50 0x7fe1300051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.693+0000 7fe149e00640 1 -- 172.21.15.83:0/3489134895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe130003530 con 0x7fe1300039c0 2024-04-25T00:13:04.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.697+0000 7fe149e00640 1 --2- 172.21.15.83:0/3489134895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1300039c0 0x7fe130003de0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009fa0 tx=0x7fe13c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd92dd1f339e7d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:04.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.697+0000 7fe149400640 1 -- 172.21.15.83:0/3489134895 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0b2070 con 0x7fe1300039c0 2024-04-25T00:13:04.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.697+0000 7fe149400640 1 -- 172.21.15.83:0/3489134895 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe13c0a5c00 con 0x7fe1300039c0 2024-04-25T00:13:04.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.697+0000 7fe149400640 1 -- 172.21.15.83:0/3489134895 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c009bf0 con 0x7fe1300039c0 2024-04-25T00:13:04.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.697+0000 7fe151200640 1 -- 172.21.15.83:0/3489134895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1300039c0 msgr2=0x7fe130003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.697+0000 7fe151200640 1 --2- 172.21.15.83:0/3489134895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1300039c0 0x7fe130003de0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009fa0 tx=0x7fe13c01cc40 comp rx=0 tx=0).stop 2024-04-25T00:13:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.697+0000 7fe14a800640 1 -- 172.21.15.83:0/3489134895 reap_dead start 2024-04-25T00:13:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.697+0000 7fe151200640 1 -- 172.21.15.83:0/3489134895 shutdown_connections 2024-04-25T00:13:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.697+0000 7fe151200640 1 -- 172.21.15.83:0/3489134895 >> 172.21.15.83:0/3489134895 conn(0x7fe14c0b1830 msgr2=0x7fe14c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.697+0000 7fe151200640 1 -- 172.21.15.83:0/3489134895 shutdown_connections 2024-04-25T00:13:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.697+0000 7fe151200640 1 -- 172.21.15.83:0/3489134895 wait complete. 2024-04-25T00:13:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.697+0000 7fe151200640 1 Processor -- start 2024-04-25T00:13:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.697+0000 7fe151200640 1 -- start start 2024-04-25T00:13:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe151200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1300039c0 0x7fe14c15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe151200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe14c1559d0 0x7fe14c148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe151200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe14c155e50 0x7fe14c149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe151200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c0b3a00 con 0x7fe1300039c0 2024-04-25T00:13:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe151200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c0b3880 con 0x7fe14c1559d0 2024-04-25T00:13:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe151200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe14c0b3b80 con 0x7fe14c155e50 2024-04-25T00:13:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe149e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1300039c0 0x7fe14c15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe149e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1300039c0 0x7fe14c15a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60468/0 (socket says 172.21.15.83:60468) 2024-04-25T00:13:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe149e00640 1 -- 172.21.15.83:0/1796327500 learned_addr learned my addr 172.21.15.83:0/1796327500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe14a800640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe14c155e50 0x7fe14c149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe141400640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe14c1559d0 0x7fe14c148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe149e00640 1 -- 172.21.15.83:0/1796327500 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe14c155e50 msgr2=0x7fe14c149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe149e00640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe14c155e50 0x7fe14c149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe149e00640 1 -- 172.21.15.83:0/1796327500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe14c1559d0 msgr2=0x7fe14c148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe149e00640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe14c1559d0 0x7fe14c148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe149e00640 1 -- 172.21.15.83:0/1796327500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14c157990 con 0x7fe1300039c0 2024-04-25T00:13:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe141400640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe14c1559d0 0x7fe14c148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe149e00640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1300039c0 0x7fe14c15a2d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe13c01ca70 tx=0x7fe130005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe142a00640 1 -- 172.21.15.83:0/1796327500 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0be070 con 0x7fe1300039c0 2024-04-25T00:13:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe142a00640 1 -- 172.21.15.83:0/1796327500 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe13c0b3040 con 0x7fe1300039c0 2024-04-25T00:13:04.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe142a00640 1 -- 172.21.15.83:0/1796327500 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0b2050 con 0x7fe1300039c0 2024-04-25T00:13:04.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe151200640 1 -- 172.21.15.83:0/1796327500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14c157c70 con 0x7fe1300039c0 2024-04-25T00:13:04.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe151200640 1 -- 172.21.15.83:0/1796327500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe14c158230 con 0x7fe1300039c0 2024-04-25T00:13:04.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.701+0000 7fe151200640 1 -- 172.21.15.83:0/1796327500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe14c0737f0 con 0x7fe1300039c0 2024-04-25T00:13:04.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.705+0000 7fe142a00640 1 -- 172.21.15.83:0/1796327500 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe13c0c1070 con 0x7fe1300039c0 2024-04-25T00:13:04.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.705+0000 7fe142a00640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe1280678f0 0x7fe128069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.705+0000 7fe142a00640 1 -- 172.21.15.83:0/1796327500 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fe13c120400 con 0x7fe1300039c0 2024-04-25T00:13:04.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.705+0000 7fe141400640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe1280678f0 0x7fe128069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.705+0000 7fe141400640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe1280678f0 0x7fe128069db0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0cf5b0 tx=0x7fe134009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:04.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.709+0000 7fe142a00640 1 -- 172.21.15.83:0/1796327500 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe13c016950 con 0x7fe1300039c0 2024-04-25T00:13:04.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.901+0000 7fe151200640 1 -- 172.21.15.83:0/1796327500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe14c04f5f0 con 0x7fe1300039c0 2024-04-25T00:13:04.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.901+0000 7fe142a00640 1 -- 172.21.15.83:0/1796327500 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5181 (secure 0 0 0) 0x7fe13c0f06c0 con 0x7fe1300039c0 2024-04-25T00:13:04.902 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:04.902 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2761747029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2761747029},{"type":"v1","addr":"172.21.15.83:6837","nonce":2761747029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2989939837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2989939837},{"type":"v1","addr":"172.21.15.86:6835","nonce":2989939837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2291906415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2291906415},{"type":"v1","addr":"172.21.15.83:6836","nonce":2291906415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:12:52.026135+0000","modified":"2024-04-25T00:12:55.045476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5814":{"gid":5814,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6834/1690820142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1690820142},{"type":"v1","addr":"172.21.15.86:6834","nonce":1690820142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5814,"qdb_cluster":[5814]},"id":3}]} 2024-04-25T00:13:04.903 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-04-25T00:13:04.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.901+0000 7fe151200640 1 -- 172.21.15.83:0/1796327500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe1280678f0 msgr2=0x7fe128069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:04.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.901+0000 7fe151200640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe1280678f0 0x7fe128069db0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0cf5b0 tx=0x7fe134009040 comp rx=0 tx=0).stop 2024-04-25T00:13:04.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.901+0000 7fe151200640 1 -- 172.21.15.83:0/1796327500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1300039c0 msgr2=0x7fe14c15a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:04.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.901+0000 7fe151200640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1300039c0 0x7fe14c15a2d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe13c01ca70 tx=0x7fe130005840 comp rx=0 tx=0).stop 2024-04-25T00:13:04.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.901+0000 7fe151200640 1 -- 172.21.15.83:0/1796327500 shutdown_connections 2024-04-25T00:13:04.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.901+0000 7fe151200640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe1280678f0 0x7fe128069db0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.901+0000 7fe151200640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe14c155e50 0x7fe14c149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.901+0000 7fe151200640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe14c1559d0 0x7fe14c148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.901+0000 7fe151200640 1 --2- 172.21.15.83:0/1796327500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1300039c0 0x7fe14c15a2d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.901+0000 7fe151200640 1 -- 172.21.15.83:0/1796327500 >> 172.21.15.83:0/1796327500 conn(0x7fe14c0b1830 msgr2=0x7fe14c152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:04.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.901+0000 7fe151200640 1 -- 172.21.15.83:0/1796327500 shutdown_connections 2024-04-25T00:13:04.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:04.901+0000 7fe151200640 1 -- 172.21.15.83:0/1796327500 wait complete. 2024-04-25T00:13:04.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:05.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.057+0000 7f3423000640 1 Processor -- start 2024-04-25T00:13:05.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.057+0000 7f3423000640 1 -- start start 2024-04-25T00:13:05.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.057+0000 7f3423000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f341c0b47e0 0x7f341c0b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.057+0000 7f3423000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f341c0b4c70 con 0x7f341c0b4360 2024-04-25T00:13:05.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.057+0000 7f3423000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f341c06ef80 con 0x7f341c1553e0 2024-04-25T00:13:05.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.057+0000 7f3423000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f341c15b510 con 0x7f341c0b47e0 2024-04-25T00:13:05.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.057+0000 7f3421000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f341c0b47e0 0x7f341c0b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.057+0000 7f341b400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f341c0b4360 0x7f341c0b2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33294/0 (socket says 172.21.15.83:33294) 2024-04-25T00:13:05.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.057+0000 7f341b400640 1 -- 172.21.15.83:0/1334234040 learned_addr learned my addr 172.21.15.83:0/1334234040 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:05.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.057+0000 7f3421000640 1 -- 172.21.15.83:0/1334234040 >> v1:172.21.15.86:6789/0 conn(0x7f341c1553e0 legacy=0x7f341c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.057+0000 7f3421000640 1 -- 172.21.15.83:0/1334234040 >> v1:172.21.15.83:6789/0 conn(0x7f341c0b4360 legacy=0x7f341c0b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.057+0000 7f3421000640 1 -- 172.21.15.83:0/1334234040 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341c15b7f0 con 0x7f341c0b47e0 2024-04-25T00:13:05.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.057+0000 7f341aa00640 1 -- 172.21.15.83:0/1334234040 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2389394079 0 0) 0x7f341c0b4c70 con 0x7f341c0b4360 2024-04-25T00:13:05.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.061+0000 7f3421000640 1 --2- 172.21.15.83:0/1334234040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f341c0b47e0 0x7f341c0b3240 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f34140097c0 tx=0x7f341401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef74b814f1f3bcd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:05.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.061+0000 7f341aa00640 1 -- 172.21.15.83:0/1334234040 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34140b0070 con 0x7f341c0b47e0 2024-04-25T00:13:05.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.061+0000 7f341aa00640 1 -- 172.21.15.83:0/1334234040 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34140ab600 con 0x7f341c0b47e0 2024-04-25T00:13:05.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.061+0000 7f341aa00640 1 -- 172.21.15.83:0/1334234040 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34140ab9d0 con 0x7f341c0b47e0 2024-04-25T00:13:05.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.061+0000 7f3423000640 1 -- 172.21.15.83:0/1334234040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f341c0b47e0 msgr2=0x7f341c0b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.061+0000 7f3423000640 1 --2- 172.21.15.83:0/1334234040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f341c0b47e0 0x7f341c0b3240 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f34140097c0 tx=0x7f341401c410 comp rx=0 tx=0).stop 2024-04-25T00:13:05.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.061+0000 7f3423000640 1 -- 172.21.15.83:0/1334234040 shutdown_connections 2024-04-25T00:13:05.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.061+0000 7f3423000640 1 --2- 172.21.15.83:0/1334234040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f341c0b47e0 0x7f341c0b3240 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.061+0000 7f3423000640 1 -- 172.21.15.83:0/1334234040 >> 172.21.15.83:0/1334234040 conn(0x7f341c0b1830 msgr2=0x7f341c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:05.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.061+0000 7f3423000640 1 -- 172.21.15.83:0/1334234040 shutdown_connections 2024-04-25T00:13:05.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.061+0000 7f3423000640 1 -- 172.21.15.83:0/1334234040 wait complete. 2024-04-25T00:13:05.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3423000640 1 Processor -- start 2024-04-25T00:13:05.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3423000640 1 -- start start 2024-04-25T00:13:05.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3423000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f341c0b4360 0x7f341c159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3423000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f341c0b47e0 0x7f341c14f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3423000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f341c1553e0 0x7f341c163f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3423000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f341c155940 con 0x7f341c1553e0 2024-04-25T00:13:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3423000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f341c1557c0 con 0x7f341c0b47e0 2024-04-25T00:13:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3423000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f341c06ebd0 con 0x7f341c0b4360 2024-04-25T00:13:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3421000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f341c1553e0 0x7f341c163f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3421000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f341c1553e0 0x7f341c163f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60490/0 (socket says 172.21.15.83:60490) 2024-04-25T00:13:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3421000640 1 -- 172.21.15.83:0/2360742402 learned_addr learned my addr 172.21.15.83:0/2360742402 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f341b400640 1 --2- 172.21.15.83:0/2360742402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f341c0b47e0 0x7f341c14f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f341be00640 1 --2- 172.21.15.83:0/2360742402 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f341c0b4360 0x7f341c159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3421000640 1 -- 172.21.15.83:0/2360742402 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f341c0b4360 msgr2=0x7f341c159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3421000640 1 --2- 172.21.15.83:0/2360742402 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f341c0b4360 0x7f341c159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3421000640 1 -- 172.21.15.83:0/2360742402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f341c0b47e0 msgr2=0x7f341c14f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3421000640 1 --2- 172.21.15.83:0/2360742402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f341c0b47e0 0x7f341c14f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3421000640 1 -- 172.21.15.83:0/2360742402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341c0ce340 con 0x7f341c1553e0 2024-04-25T00:13:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3421000640 1 --2- 172.21.15.83:0/2360742402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f341c1553e0 0x7f341c163f30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f34140ae040 tx=0x7f341401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3413e00640 1 -- 172.21.15.83:0/2360742402 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34140b0050 con 0x7f341c1553e0 2024-04-25T00:13:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3413e00640 1 -- 172.21.15.83:0/2360742402 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34140bc070 con 0x7f341c1553e0 2024-04-25T00:13:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3423000640 1 -- 172.21.15.83:0/2360742402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341c0ce620 con 0x7f341c1553e0 2024-04-25T00:13:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3413e00640 1 -- 172.21.15.83:0/2360742402 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34140b5830 con 0x7f341c1553e0 2024-04-25T00:13:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.065+0000 7f3423000640 1 -- 172.21.15.83:0/2360742402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f341c0cebe0 con 0x7f341c1553e0 2024-04-25T00:13:05.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.069+0000 7f3423000640 1 -- 172.21.15.83:0/2360742402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f341c0737f0 con 0x7f341c1553e0 2024-04-25T00:13:05.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.069+0000 7f3413e00640 1 -- 172.21.15.83:0/2360742402 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f34140b59d0 con 0x7f341c1553e0 2024-04-25T00:13:05.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.069+0000 7f3413e00640 1 --2- 172.21.15.83:0/2360742402 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f33fc0678c0 0x7f33fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.069+0000 7f3413e00640 1 -- 172.21.15.83:0/2360742402 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f3414120770 con 0x7f341c1553e0 2024-04-25T00:13:05.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.069+0000 7f341be00640 1 --2- 172.21.15.83:0/2360742402 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f33fc0678c0 0x7f33fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.069+0000 7f341be00640 1 --2- 172.21.15.83:0/2360742402 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f33fc0678c0 0x7f33fc069d80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f340c0012f0 tx=0x7f340c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:05.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.073+0000 7f3413e00640 1 -- 172.21.15.83:0/2360742402 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3414016090 con 0x7f341c1553e0 2024-04-25T00:13:05.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.229+0000 7f3423000640 1 -- 172.21.15.83:0/2360742402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f341c04f5f0 con 0x7f341c1553e0 2024-04-25T00:13:05.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.229+0000 7f3413e00640 1 -- 172.21.15.83:0/2360742402 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16858 (secure 0 0 0) 0x7f34140f1080 con 0x7f341c1553e0 2024-04-25T00:13:05.232 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:05.233 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":49,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:53.651846+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:50.394645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:51.018842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:05.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.233+0000 7f3423000640 1 -- 172.21.15.83:0/2360742402 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f33fc0678c0 msgr2=0x7f33fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.233+0000 7f3423000640 1 --2- 172.21.15.83:0/2360742402 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f33fc0678c0 0x7f33fc069d80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f340c0012f0 tx=0x7f340c009040 comp rx=0 tx=0).stop 2024-04-25T00:13:05.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.233+0000 7f3423000640 1 -- 172.21.15.83:0/2360742402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f341c1553e0 msgr2=0x7f341c163f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.233+0000 7f3423000640 1 --2- 172.21.15.83:0/2360742402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f341c1553e0 0x7f341c163f30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f34140ae040 tx=0x7f341401ac50 comp rx=0 tx=0).stop 2024-04-25T00:13:05.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.233+0000 7f3423000640 1 -- 172.21.15.83:0/2360742402 shutdown_connections 2024-04-25T00:13:05.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.233+0000 7f3423000640 1 --2- 172.21.15.83:0/2360742402 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f33fc0678c0 0x7f33fc069d80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.233+0000 7f3423000640 1 --2- 172.21.15.83:0/2360742402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f341c1553e0 0x7f341c163f30 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.233+0000 7f3423000640 1 --2- 172.21.15.83:0/2360742402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f341c0b47e0 0x7f341c14f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.233+0000 7f3423000640 1 --2- 172.21.15.83:0/2360742402 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f341c0b4360 0x7f341c159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.233+0000 7f3423000640 1 -- 172.21.15.83:0/2360742402 >> 172.21.15.83:0/2360742402 conn(0x7f341c0b1830 msgr2=0x7f341c0cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:05.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.233+0000 7f3423000640 1 -- 172.21.15.83:0/2360742402 shutdown_connections 2024-04-25T00:13:05.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.233+0000 7f3423000640 1 -- 172.21.15.83:0/2360742402 wait complete. 2024-04-25T00:13:05.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:12:50.394645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:12:51.018842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:13:05.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:13:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.389+0000 7f28fca00640 1 Processor -- start 2024-04-25T00:13:05.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.389+0000 7f28fca00640 1 -- start start 2024-04-25T00:13:05.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.393+0000 7f28fca00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f28f8155310 0x7f28f806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.393+0000 7f28fca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28f815e790 0x7f28f815ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.393+0000 7f28fca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28f80719f0 con 0x7f28f815b1a0 2024-04-25T00:13:05.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.393+0000 7f28fca00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f28f815f610 con 0x7f28f8155310 2024-04-25T00:13:05.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.393+0000 7f28fca00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f28f8071cd0 con 0x7f28f815e790 2024-04-25T00:13:05.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.393+0000 7f28f6800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f28f8155310 0x7f28f806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.393+0000 7f28f6800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f28f8155310 0x7f28f806cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60492/0 (socket says 172.21.15.83:60492) 2024-04-25T00:13:05.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.393+0000 7f28f6800640 1 -- 172.21.15.83:0/4018090972 learned_addr learned my addr 172.21.15.83:0/4018090972 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:05.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.393+0000 7f28f6800640 1 -- 172.21.15.83:0/4018090972 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28f815e790 msgr2=0x7f28f815ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:05.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.393+0000 7f28f6800640 1 --2- 172.21.15.83:0/4018090972 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28f815e790 0x7f28f815ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.393+0000 7f28f6800640 1 -- 172.21.15.83:0/4018090972 >> v1:172.21.15.86:6789/0 conn(0x7f28f815b1a0 legacy=0x7f28f815b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:05.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.393+0000 7f28f6800640 1 -- 172.21.15.83:0/4018090972 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f8071fb0 con 0x7f28f8155310 2024-04-25T00:13:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.393+0000 7f28f6800640 1 --2- 172.21.15.83:0/4018090972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f8155310 0x7f28f806cc60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f28e800d150 tx=0x7f28e801eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f689b62fbc1ebab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.393+0000 7f28f5e00640 1 -- 172.21.15.83:0/4018090972 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e800b2d0 con 0x7f28f8155310 2024-04-25T00:13:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28f5e00640 1 -- 172.21.15.83:0/4018090972 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e800b470 con 0x7f28f8155310 2024-04-25T00:13:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28f5e00640 1 -- 172.21.15.83:0/4018090972 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e80a7910 con 0x7f28f8155310 2024-04-25T00:13:05.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 -- 172.21.15.83:0/4018090972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f8155310 msgr2=0x7f28f806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 --2- 172.21.15.83:0/4018090972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f8155310 0x7f28f806cc60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f28e800d150 tx=0x7f28e801eb60 comp rx=0 tx=0).stop 2024-04-25T00:13:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 -- 172.21.15.83:0/4018090972 shutdown_connections 2024-04-25T00:13:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 --2- 172.21.15.83:0/4018090972 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28f815e790 0x7f28f815ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 --2- 172.21.15.83:0/4018090972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f8155310 0x7f28f806cc60 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 -- 172.21.15.83:0/4018090972 >> 172.21.15.83:0/4018090972 conn(0x7f28f80cf980 msgr2=0x7f28f80cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 -- 172.21.15.83:0/4018090972 shutdown_connections 2024-04-25T00:13:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 -- 172.21.15.83:0/4018090972 wait complete. 2024-04-25T00:13:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 Processor -- start 2024-04-25T00:13:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 -- start start 2024-04-25T00:13:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28f8155310 0x7f28f8149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28f815b1a0 0x7f28f8155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f815e790 0x7f28f814e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28f8071740 con 0x7f28f815e790 2024-04-25T00:13:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28f80715c0 con 0x7f28f815b1a0 2024-04-25T00:13:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28fca00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f28f806dd00 con 0x7f28f8155310 2024-04-25T00:13:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28f7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f815e790 0x7f28f814e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28f7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f815e790 0x7f28f814e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60496/0 (socket says 172.21.15.83:60496) 2024-04-25T00:13:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28f7200640 1 -- 172.21.15.83:0/3188139085 learned_addr learned my addr 172.21.15.83:0/3188139085 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28f6800640 1 --2- 172.21.15.83:0/3188139085 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28f8155310 0x7f28f8149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.397+0000 7f28ede00640 1 --2- 172.21.15.83:0/3188139085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28f815b1a0 0x7f28f8155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28f7200640 1 -- 172.21.15.83:0/3188139085 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28f8155310 msgr2=0x7f28f8149a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28f7200640 1 --2- 172.21.15.83:0/3188139085 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28f8155310 0x7f28f8149a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28f7200640 1 -- 172.21.15.83:0/3188139085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28f815b1a0 msgr2=0x7f28f8155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28f7200640 1 --2- 172.21.15.83:0/3188139085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28f815b1a0 0x7f28f8155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28f7200640 1 -- 172.21.15.83:0/3188139085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f8152fa0 con 0x7f28f815e790 2024-04-25T00:13:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28f7200640 1 --2- 172.21.15.83:0/3188139085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f815e790 0x7f28f814e550 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f28f000c440 tx=0x7f28f000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28ef400640 1 -- 172.21.15.83:0/3188139085 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0017070 con 0x7f28f815e790 2024-04-25T00:13:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28ef400640 1 -- 172.21.15.83:0/3188139085 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f0003600 con 0x7f28f815e790 2024-04-25T00:13:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28fca00640 1 -- 172.21.15.83:0/3188139085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f8153280 con 0x7f28f815e790 2024-04-25T00:13:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28ef400640 1 -- 172.21.15.83:0/3188139085 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f00039f0 con 0x7f28f815e790 2024-04-25T00:13:05.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28fca00640 1 -- 172.21.15.83:0/3188139085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f8153840 con 0x7f28f815e790 2024-04-25T00:13:05.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28fca00640 1 -- 172.21.15.83:0/3188139085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f804f510 con 0x7f28f815e790 2024-04-25T00:13:05.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28ef400640 1 -- 172.21.15.83:0/3188139085 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f28f001a070 con 0x7f28f815e790 2024-04-25T00:13:05.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28ef400640 1 --2- 172.21.15.83:0/3188139085 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28d40678c0 0x7f28d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.401+0000 7f28ef400640 1 -- 172.21.15.83:0/3188139085 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f28f0085070 con 0x7f28f815e790 2024-04-25T00:13:05.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.405+0000 7f28f6800640 1 --2- 172.21.15.83:0/3188139085 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28d40678c0 0x7f28d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.405+0000 7f28f6800640 1 --2- 172.21.15.83:0/3188139085 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28d40678c0 0x7f28d4069d80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f28e80b0000 tx=0x7f28e80a9070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:05.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.409+0000 7f28ef400640 1 -- 172.21.15.83:0/3188139085 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f28f0055480 con 0x7f28f815e790 2024-04-25T00:13:05.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.561+0000 7f28fca00640 1 -- 172.21.15.83:0/3188139085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28f814aaa0 con 0x7f28f815e790 2024-04-25T00:13:05.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.565+0000 7f28ef400640 1 -- 172.21.15.83:0/3188139085 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) ==== 93+0+10 (secure 0 0 0) 0x7f28f0059780 con 0x7f28f815e790 2024-04-25T00:13:05.566 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:13:05.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.565+0000 7f28fca00640 1 -- 172.21.15.83:0/3188139085 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28d40678c0 msgr2=0x7f28d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.565+0000 7f28fca00640 1 --2- 172.21.15.83:0/3188139085 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28d40678c0 0x7f28d4069d80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f28e80b0000 tx=0x7f28e80a9070 comp rx=0 tx=0).stop 2024-04-25T00:13:05.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.565+0000 7f28fca00640 1 -- 172.21.15.83:0/3188139085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f815e790 msgr2=0x7f28f814e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.565+0000 7f28fca00640 1 --2- 172.21.15.83:0/3188139085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f815e790 0x7f28f814e550 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f28f000c440 tx=0x7f28f000c900 comp rx=0 tx=0).stop 2024-04-25T00:13:05.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.565+0000 7f28fca00640 1 -- 172.21.15.83:0/3188139085 shutdown_connections 2024-04-25T00:13:05.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.565+0000 7f28fca00640 1 --2- 172.21.15.83:0/3188139085 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f28d40678c0 0x7f28d4069d80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.565+0000 7f28fca00640 1 --2- 172.21.15.83:0/3188139085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f815e790 0x7f28f814e550 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.565+0000 7f28fca00640 1 --2- 172.21.15.83:0/3188139085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28f815b1a0 0x7f28f8155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.565+0000 7f28fca00640 1 --2- 172.21.15.83:0/3188139085 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28f8155310 0x7f28f8149a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.565+0000 7f28fca00640 1 -- 172.21.15.83:0/3188139085 >> 172.21.15.83:0/3188139085 conn(0x7f28f80cf980 msgr2=0x7f28f815b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:05.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.565+0000 7f28fca00640 1 -- 172.21.15.83:0/3188139085 shutdown_connections 2024-04-25T00:13:05.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.565+0000 7f28fca00640 1 -- 172.21.15.83:0/3188139085 wait complete. 2024-04-25T00:13:05.586 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:13:05.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.721+0000 7ff54aa00640 1 Processor -- start 2024-04-25T00:13:05.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.721+0000 7ff54aa00640 1 -- start start 2024-04-25T00:13:05.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.725+0000 7ff54aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5441555c0 0x7ff54415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.725+0000 7ff54aa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff54406e4c0 0x7ff54406e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.725+0000 7ff54aa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff54415c560 con 0x7ff5441551e0 2024-04-25T00:13:05.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.725+0000 7ff54aa00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff544072460 con 0x7ff54406e4c0 2024-04-25T00:13:05.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.725+0000 7ff54aa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff54415c7d0 con 0x7ff5441555c0 2024-04-25T00:13:05.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.725+0000 7ff548a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff54406e4c0 0x7ff54406e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.725+0000 7ff548a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff54406e4c0 0x7ff54406e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60512/0 (socket says 172.21.15.83:60512) 2024-04-25T00:13:05.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.725+0000 7ff548a00640 1 -- 172.21.15.83:0/3596049749 learned_addr learned my addr 172.21.15.83:0/3596049749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:05.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.725+0000 7ff53b400640 1 --2- 172.21.15.83:0/3596049749 >> v2:172.21.15.86:3300/0 conn(0x7ff5441555c0 0x7ff54415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.725+0000 7ff548a00640 1 -- 172.21.15.83:0/3596049749 >> v2:172.21.15.86:3300/0 conn(0x7ff5441555c0 msgr2=0x7ff54415bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:05.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.725+0000 7ff548a00640 1 --2- 172.21.15.83:0/3596049749 >> v2:172.21.15.86:3300/0 conn(0x7ff5441555c0 0x7ff54415bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.725+0000 7ff543400640 1 -- 172.21.15.83:0/3596049749 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2345026752 0 0) 0x7ff54415c560 con 0x7ff5441551e0 2024-04-25T00:13:05.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.725+0000 7ff548a00640 1 -- 172.21.15.83:0/3596049749 >> v1:172.21.15.86:6789/0 conn(0x7ff5441551e0 legacy=0x7ff544071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.725+0000 7ff548a00640 1 -- 172.21.15.83:0/3596049749 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54415cab0 con 0x7ff54406e4c0 2024-04-25T00:13:05.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.729+0000 7ff548a00640 1 --2- 172.21.15.83:0/3596049749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54406e4c0 0x7ff54406e950 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009e50 tx=0x7ff53c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9300e14df18e6ce1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:05.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.729+0000 7ff543400640 1 -- 172.21.15.83:0/3596049749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0a56c0 con 0x7ff54406e4c0 2024-04-25T00:13:05.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.729+0000 7ff543400640 1 -- 172.21.15.83:0/3596049749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff53c0a5860 con 0x7ff54406e4c0 2024-04-25T00:13:05.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.729+0000 7ff543400640 1 -- 172.21.15.83:0/3596049749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0a5c30 con 0x7ff54406e4c0 2024-04-25T00:13:05.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.729+0000 7ff54aa00640 1 -- 172.21.15.83:0/3596049749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54406e4c0 msgr2=0x7ff54406e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.729+0000 7ff54aa00640 1 --2- 172.21.15.83:0/3596049749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54406e4c0 0x7ff54406e950 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009e50 tx=0x7ff53c01c9f0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.729+0000 7ff54aa00640 1 -- 172.21.15.83:0/3596049749 shutdown_connections 2024-04-25T00:13:05.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.729+0000 7ff54aa00640 1 --2- 172.21.15.83:0/3596049749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54406e4c0 0x7ff54406e950 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.729+0000 7ff54aa00640 1 --2- 172.21.15.83:0/3596049749 >> v2:172.21.15.86:3300/0 conn(0x7ff5441555c0 0x7ff54415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.729+0000 7ff54aa00640 1 -- 172.21.15.83:0/3596049749 >> 172.21.15.83:0/3596049749 conn(0x7ff5440b1830 msgr2=0x7ff5440b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:05.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.729+0000 7ff54aa00640 1 -- 172.21.15.83:0/3596049749 shutdown_connections 2024-04-25T00:13:05.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.729+0000 7ff54aa00640 1 -- 172.21.15.83:0/3596049749 wait complete. 2024-04-25T00:13:05.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.729+0000 7ff54aa00640 1 Processor -- start 2024-04-25T00:13:05.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.729+0000 7ff54aa00640 1 -- start start 2024-04-25T00:13:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff54aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54406e4c0 0x7ff5440cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff54aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5441551e0 0x7ff5441f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff54aa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5441555c0 0x7ff54415a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff54aa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff54415db70 con 0x7ff54406e4c0 2024-04-25T00:13:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff54aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff54415d9f0 con 0x7ff5441551e0 2024-04-25T00:13:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff54aa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff54415dcf0 con 0x7ff5441555c0 2024-04-25T00:13:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff543e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54406e4c0 0x7ff5440cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff543e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54406e4c0 0x7ff5440cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60518/0 (socket says 172.21.15.83:60518) 2024-04-25T00:13:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff543e00640 1 -- 172.21.15.83:0/4151678835 learned_addr learned my addr 172.21.15.83:0/4151678835 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff53b400640 1 --2- 172.21.15.83:0/4151678835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5441551e0 0x7ff5441f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff548a00640 1 --2- 172.21.15.83:0/4151678835 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5441555c0 0x7ff54415a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff543e00640 1 -- 172.21.15.83:0/4151678835 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5441555c0 msgr2=0x7ff54415a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff543e00640 1 --2- 172.21.15.83:0/4151678835 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5441555c0 0x7ff54415a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff543e00640 1 -- 172.21.15.83:0/4151678835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5441551e0 msgr2=0x7ff5441f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff543e00640 1 --2- 172.21.15.83:0/4151678835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5441551e0 0x7ff5441f6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff543e00640 1 -- 172.21.15.83:0/4151678835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5440cd4f0 con 0x7ff54406e4c0 2024-04-25T00:13:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff543e00640 1 --2- 172.21.15.83:0/4151678835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54406e4c0 0x7ff5440cea40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff53400c4b0 tx=0x7ff53400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff540c00640 1 -- 172.21.15.83:0/4151678835 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff534005480 con 0x7ff54406e4c0 2024-04-25T00:13:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff540c00640 1 -- 172.21.15.83:0/4151678835 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff534003600 con 0x7ff54406e4c0 2024-04-25T00:13:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff540c00640 1 -- 172.21.15.83:0/4151678835 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5340039f0 con 0x7ff54406e4c0 2024-04-25T00:13:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff54aa00640 1 -- 172.21.15.83:0/4151678835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5440cd7d0 con 0x7ff54406e4c0 2024-04-25T00:13:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff54aa00640 1 -- 172.21.15.83:0/4151678835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5440cdd90 con 0x7ff54406e4c0 2024-04-25T00:13:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff54aa00640 1 -- 172.21.15.83:0/4151678835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff54404f510 con 0x7ff54406e4c0 2024-04-25T00:13:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.733+0000 7ff540c00640 1 -- 172.21.15.83:0/4151678835 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff534003b90 con 0x7ff54406e4c0 2024-04-25T00:13:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.737+0000 7ff540c00640 1 --2- 172.21.15.83:0/4151678835 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5200678c0 0x7ff520069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.737+0000 7ff540c00640 1 -- 172.21.15.83:0/4151678835 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7ff534085a00 con 0x7ff54406e4c0 2024-04-25T00:13:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.737+0000 7ff53b400640 1 --2- 172.21.15.83:0/4151678835 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5200678c0 0x7ff520069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.737+0000 7ff53b400640 1 --2- 172.21.15.83:0/4151678835 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5200678c0 0x7ff520069d80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff5441509b0 tx=0x7ff52c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:05.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.741+0000 7ff540c00640 1 -- 172.21.15.83:0/4151678835 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff534055cc0 con 0x7ff54406e4c0 2024-04-25T00:13:05.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.893+0000 7ff54aa00640 1 -- 172.21.15.83:0/4151678835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff54414ef10 con 0x7ff54406e4c0 2024-04-25T00:13:05.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.897+0000 7ff540c00640 1 -- 172.21.15.83:0/4151678835 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) ==== 104+0+11 (secure 0 0 0) 0x7ff534059fc0 con 0x7ff54406e4c0 2024-04-25T00:13:05.898 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:13:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.897+0000 7ff54aa00640 1 -- 172.21.15.83:0/4151678835 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5200678c0 msgr2=0x7ff520069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.897+0000 7ff54aa00640 1 --2- 172.21.15.83:0/4151678835 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5200678c0 0x7ff520069d80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff5441509b0 tx=0x7ff52c009190 comp rx=0 tx=0).stop 2024-04-25T00:13:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.897+0000 7ff54aa00640 1 -- 172.21.15.83:0/4151678835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54406e4c0 msgr2=0x7ff5440cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.897+0000 7ff54aa00640 1 --2- 172.21.15.83:0/4151678835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54406e4c0 0x7ff5440cea40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff53400c4b0 tx=0x7ff53400c970 comp rx=0 tx=0).stop 2024-04-25T00:13:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.897+0000 7ff54aa00640 1 -- 172.21.15.83:0/4151678835 shutdown_connections 2024-04-25T00:13:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.897+0000 7ff54aa00640 1 --2- 172.21.15.83:0/4151678835 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5200678c0 0x7ff520069d80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.897+0000 7ff54aa00640 1 --2- 172.21.15.83:0/4151678835 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5441555c0 0x7ff54415a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.897+0000 7ff54aa00640 1 --2- 172.21.15.83:0/4151678835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5441551e0 0x7ff5441f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.897+0000 7ff54aa00640 1 --2- 172.21.15.83:0/4151678835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54406e4c0 0x7ff5440cea40 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.897+0000 7ff54aa00640 1 -- 172.21.15.83:0/4151678835 >> 172.21.15.83:0/4151678835 conn(0x7ff5440b1830 msgr2=0x7ff54415e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.897+0000 7ff54aa00640 1 -- 172.21.15.83:0/4151678835 shutdown_connections 2024-04-25T00:13:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:05.897+0000 7ff54aa00640 1 -- 172.21.15.83:0/4151678835 wait complete. 2024-04-25T00:13:05.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:13:06.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.053+0000 7f8f8ca00640 1 Processor -- start 2024-04-25T00:13:06.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.053+0000 7f8f8ca00640 1 -- start start 2024-04-25T00:13:06.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f8ca00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8f8815ba60 0x7f8f8806b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f8ca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f8806b720 0x7f8f880cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f8ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8f880cfb30 0x7f8f88161a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f8ca00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8f8806bb80 con 0x7f8f8815ba60 2024-04-25T00:13:06.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f8ca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8f8815be40 con 0x7f8f880cfb30 2024-04-25T00:13:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f8ca00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8f8815e440 con 0x7f8f8806b720 2024-04-25T00:13:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f86800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8f8815ba60 0x7f8f8806b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f86800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8f8815ba60 0x7f8f8806b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60534/0 (socket says 172.21.15.83:60534) 2024-04-25T00:13:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f86800640 1 -- 172.21.15.83:0/3105379131 learned_addr learned my addr 172.21.15.83:0/3105379131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f87200640 1 --2- 172.21.15.83:0/3105379131 >> v2:172.21.15.86:3300/0 conn(0x7f8f880cfb30 0x7f8f88161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f7de00640 1 --2- 172.21.15.83:0/3105379131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f8806b720 0x7f8f880cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f86800640 1 -- 172.21.15.83:0/3105379131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f8806b720 msgr2=0x7f8f880cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f86800640 1 --2- 172.21.15.83:0/3105379131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f8806b720 0x7f8f880cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f86800640 1 -- 172.21.15.83:0/3105379131 >> v2:172.21.15.86:3300/0 conn(0x7f8f880cfb30 msgr2=0x7f8f88161a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f86800640 1 --2- 172.21.15.83:0/3105379131 >> v2:172.21.15.86:3300/0 conn(0x7f8f880cfb30 0x7f8f88161a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f86800640 1 -- 172.21.15.83:0/3105379131 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8815e700 con 0x7f8f8815ba60 2024-04-25T00:13:06.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.057+0000 7f8f86800640 1 --2- 172.21.15.83:0/3105379131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f8815ba60 0x7f8f8806b1e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009870 tx=0x7f8f7801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5aa2ff7da0ea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f85e00640 1 -- 172.21.15.83:0/3105379131 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f780b0070 con 0x7f8f8815ba60 2024-04-25T00:13:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f85e00640 1 -- 172.21.15.83:0/3105379131 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f780ab600 con 0x7f8f8815ba60 2024-04-25T00:13:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f85e00640 1 -- 172.21.15.83:0/3105379131 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f780ab9d0 con 0x7f8f8815ba60 2024-04-25T00:13:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3105379131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f8815ba60 msgr2=0x7f8f8806b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 --2- 172.21.15.83:0/3105379131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f8815ba60 0x7f8f8806b1e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009870 tx=0x7f8f7801c260 comp rx=0 tx=0).stop 2024-04-25T00:13:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3105379131 shutdown_connections 2024-04-25T00:13:06.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 --2- 172.21.15.83:0/3105379131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f8806b720 0x7f8f880cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 --2- 172.21.15.83:0/3105379131 >> v2:172.21.15.86:3300/0 conn(0x7f8f880cfb30 0x7f8f88161a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 --2- 172.21.15.83:0/3105379131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f8815ba60 0x7f8f8806b1e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3105379131 >> 172.21.15.83:0/3105379131 conn(0x7f8f880d0a90 msgr2=0x7f8f880d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:06.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3105379131 shutdown_connections 2024-04-25T00:13:06.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3105379131 wait complete. 2024-04-25T00:13:06.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 Processor -- start 2024-04-25T00:13:06.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 -- start start 2024-04-25T00:13:06.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f8806b720 0x7f8f88140510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f880cfb30 0x7f8f8806e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f8815ba60 0x7f8f8815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f880cbfc0 con 0x7f8f880cfb30 2024-04-25T00:13:06.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f880cbe40 con 0x7f8f8815ba60 2024-04-25T00:13:06.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f8ca00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8f880cff40 con 0x7f8f8806b720 2024-04-25T00:13:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f7de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f880cfb30 0x7f8f8806e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f7de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f880cfb30 0x7f8f8806e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60550/0 (socket says 172.21.15.83:60550) 2024-04-25T00:13:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f7de00640 1 -- 172.21.15.83:0/3853801160 learned_addr learned my addr 172.21.15.83:0/3853801160 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f86800640 1 --2- 172.21.15.83:0/3853801160 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f8806b720 0x7f8f88140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.061+0000 7f8f87200640 1 --2- 172.21.15.83:0/3853801160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f8815ba60 0x7f8f8815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f7de00640 1 -- 172.21.15.83:0/3853801160 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f8806b720 msgr2=0x7f8f88140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f7de00640 1 --2- 172.21.15.83:0/3853801160 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f8806b720 0x7f8f88140510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f7de00640 1 -- 172.21.15.83:0/3853801160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f8815ba60 msgr2=0x7f8f8815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f7de00640 1 --2- 172.21.15.83:0/3853801160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f8815ba60 0x7f8f8815f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f7de00640 1 -- 172.21.15.83:0/3853801160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8806ebe0 con 0x7f8f880cfb30 2024-04-25T00:13:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f7de00640 1 --2- 172.21.15.83:0/3853801160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f880cfb30 0x7f8f8806e6a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000e870 tx=0x7f8f7000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f7f400640 1 -- 172.21.15.83:0/3853801160 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70017070 con 0x7f8f880cfb30 2024-04-25T00:13:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f7f400640 1 -- 172.21.15.83:0/3853801160 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f7000ee60 con 0x7f8f880cfb30 2024-04-25T00:13:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3853801160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8806ee60 con 0x7f8f880cfb30 2024-04-25T00:13:06.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f7f400640 1 -- 172.21.15.83:0/3853801160 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7000cb00 con 0x7f8f880cfb30 2024-04-25T00:13:06.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3853801160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f88156e80 con 0x7f8f880cfb30 2024-04-25T00:13:06.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3853801160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f8804f510 con 0x7f8f880cfb30 2024-04-25T00:13:06.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f7f400640 1 -- 172.21.15.83:0/3853801160 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8f7001a070 con 0x7f8f880cfb30 2024-04-25T00:13:06.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f7f400640 1 --2- 172.21.15.83:0/3853801160 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8f640678c0 0x7f8f64069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f7f400640 1 -- 172.21.15.83:0/3853801160 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f8f70085d00 con 0x7f8f880cfb30 2024-04-25T00:13:06.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.065+0000 7f8f86800640 1 --2- 172.21.15.83:0/3853801160 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8f640678c0 0x7f8f64069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.069+0000 7f8f86800640 1 --2- 172.21.15.83:0/3853801160 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8f640678c0 0x7f8f64069d80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8f780ad000 tx=0x7f8f780a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:06.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.069+0000 7f8f7f400640 1 -- 172.21.15.83:0/3853801160 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8f8806ebe0 con 0x7f8f880cfb30 2024-04-25T00:13:06.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.225+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3853801160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8f880d1850 con 0x7f8f880cfb30 2024-04-25T00:13:06.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.225+0000 7f8f7f400640 1 -- 172.21.15.83:0/3853801160 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) ==== 100+0+11 (secure 0 0 0) 0x7f8f70055f30 con 0x7f8f880cfb30 2024-04-25T00:13:06.227 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:13:06.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.225+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3853801160 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8f640678c0 msgr2=0x7f8f64069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.225+0000 7f8f8ca00640 1 --2- 172.21.15.83:0/3853801160 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8f640678c0 0x7f8f64069d80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8f780ad000 tx=0x7f8f780a6000 comp rx=0 tx=0).stop 2024-04-25T00:13:06.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.225+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3853801160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f880cfb30 msgr2=0x7f8f8806e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.225+0000 7f8f8ca00640 1 --2- 172.21.15.83:0/3853801160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f880cfb30 0x7f8f8806e6a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000e870 tx=0x7f8f7000ed30 comp rx=0 tx=0).stop 2024-04-25T00:13:06.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.229+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3853801160 shutdown_connections 2024-04-25T00:13:06.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.229+0000 7f8f8ca00640 1 --2- 172.21.15.83:0/3853801160 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8f640678c0 0x7f8f64069d80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.229+0000 7f8f8ca00640 1 --2- 172.21.15.83:0/3853801160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f8815ba60 0x7f8f8815f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.229+0000 7f8f8ca00640 1 --2- 172.21.15.83:0/3853801160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f880cfb30 0x7f8f8806e6a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.229+0000 7f8f8ca00640 1 --2- 172.21.15.83:0/3853801160 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8f8806b720 0x7f8f88140510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.229+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3853801160 >> 172.21.15.83:0/3853801160 conn(0x7f8f880d0a90 msgr2=0x7f8f880cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:06.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.229+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3853801160 shutdown_connections 2024-04-25T00:13:06.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.229+0000 7f8f8ca00640 1 -- 172.21.15.83:0/3853801160 wait complete. 2024-04-25T00:13:06.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:06.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.381+0000 7fccb6800640 1 Processor -- start 2024-04-25T00:13:06.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.381+0000 7fccb6800640 1 -- start start 2024-04-25T00:13:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.385+0000 7fccb6800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccb01581d0 0x7fccb00b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.385+0000 7fccb6800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccb00b4c30 con 0x7fccb015e6b0 2024-04-25T00:13:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.385+0000 7fccb6800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccb00b3440 con 0x7fccb00cf3c0 2024-04-25T00:13:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.385+0000 7fccb6800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fccb00b3720 con 0x7fccb01581d0 2024-04-25T00:13:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.385+0000 7fccafe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fccb015e6b0 0x7fccb006d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33310/0 (socket says 172.21.15.83:33310) 2024-04-25T00:13:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.385+0000 7fccafe00640 1 -- 172.21.15.83:0/855046455 learned_addr learned my addr 172.21.15.83:0/855046455 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.385+0000 7fccaf400640 1 --2- 172.21.15.83:0/855046455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccb01581d0 0x7fccb00b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.385+0000 7fccaf400640 1 -- 172.21.15.83:0/855046455 >> v1:172.21.15.86:6789/0 conn(0x7fccb00cf3c0 legacy=0x7fccb015dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.385+0000 7fccaf400640 1 -- 172.21.15.83:0/855046455 >> v1:172.21.15.83:6789/0 conn(0x7fccb015e6b0 legacy=0x7fccb006d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.385+0000 7fccae000640 1 -- 172.21.15.83:0/855046455 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604160364 0 0) 0x7fccb00b4c30 con 0x7fccb015e6b0 2024-04-25T00:13:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.385+0000 7fccaf400640 1 -- 172.21.15.83:0/855046455 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb00b3a00 con 0x7fccb01581d0 2024-04-25T00:13:06.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccaf400640 1 --2- 172.21.15.83:0/855046455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccb01581d0 0x7fccb00b3f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcc980097f0 tx=0x7fcc9801d5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e47aa4fecde8bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:06.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccae000640 1 -- 172.21.15.83:0/855046455 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc980aa660 con 0x7fccb01581d0 2024-04-25T00:13:06.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccae000640 1 -- 172.21.15.83:0/855046455 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc980aa800 con 0x7fccb01581d0 2024-04-25T00:13:06.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccae000640 1 -- 172.21.15.83:0/855046455 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc980aabd0 con 0x7fccb01581d0 2024-04-25T00:13:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 -- 172.21.15.83:0/855046455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccb01581d0 msgr2=0x7fccb00b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 --2- 172.21.15.83:0/855046455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccb01581d0 0x7fccb00b3f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcc980097f0 tx=0x7fcc9801d5f0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 -- 172.21.15.83:0/855046455 shutdown_connections 2024-04-25T00:13:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 --2- 172.21.15.83:0/855046455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccb01581d0 0x7fccb00b3f20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 -- 172.21.15.83:0/855046455 >> 172.21.15.83:0/855046455 conn(0x7fccb00b1830 msgr2=0x7fccb00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 -- 172.21.15.83:0/855046455 shutdown_connections 2024-04-25T00:13:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 -- 172.21.15.83:0/855046455 wait complete. 2024-04-25T00:13:06.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 Processor -- start 2024-04-25T00:13:06.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 -- start start 2024-04-25T00:13:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccb00cf3c0 0x7fccb014c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb01581d0 0x7fccb01510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccb015e6b0 0x7fccb014e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccb00b4980 con 0x7fccb01581d0 2024-04-25T00:13:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fccb00b4800 con 0x7fccb00cf3c0 2024-04-25T00:13:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.389+0000 7fccb6800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fccb0168350 con 0x7fccb015e6b0 2024-04-25T00:13:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fccaea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb01581d0 0x7fccb01510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fccaea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb01581d0 0x7fccb01510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60562/0 (socket says 172.21.15.83:60562) 2024-04-25T00:13:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fccaea00640 1 -- 172.21.15.83:0/642369905 learned_addr learned my addr 172.21.15.83:0/642369905 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fccafe00640 1 --2- 172.21.15.83:0/642369905 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccb015e6b0 0x7fccb014e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fccaf400640 1 --2- 172.21.15.83:0/642369905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccb00cf3c0 0x7fccb014c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fccaea00640 1 -- 172.21.15.83:0/642369905 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccb015e6b0 msgr2=0x7fccb014e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fccaea00640 1 --2- 172.21.15.83:0/642369905 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccb015e6b0 0x7fccb014e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fccaea00640 1 -- 172.21.15.83:0/642369905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccb00cf3c0 msgr2=0x7fccb014c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fccaea00640 1 --2- 172.21.15.83:0/642369905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccb00cf3c0 0x7fccb014c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fccaea00640 1 -- 172.21.15.83:0/642369905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb014f9b0 con 0x7fccb01581d0 2024-04-25T00:13:06.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fccaea00640 1 --2- 172.21.15.83:0/642369905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb01581d0 0x7fccb01510a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcca000c500 tx=0x7fcca000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:06.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fcca7400640 1 -- 172.21.15.83:0/642369905 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0016020 con 0x7fccb01581d0 2024-04-25T00:13:06.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fcca7400640 1 -- 172.21.15.83:0/642369905 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca0005220 con 0x7fccb01581d0 2024-04-25T00:13:06.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fcca7400640 1 -- 172.21.15.83:0/642369905 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0007030 con 0x7fccb01581d0 2024-04-25T00:13:06.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fccb6800640 1 -- 172.21.15.83:0/642369905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb014fc90 con 0x7fccb01581d0 2024-04-25T00:13:06.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fccb6800640 1 -- 172.21.15.83:0/642369905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb0150250 con 0x7fccb01581d0 2024-04-25T00:13:06.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fccb6800640 1 -- 172.21.15.83:0/642369905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb00737f0 con 0x7fccb01581d0 2024-04-25T00:13:06.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fcca7400640 1 -- 172.21.15.83:0/642369905 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcca00071f0 con 0x7fccb01581d0 2024-04-25T00:13:06.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fcca7400640 1 --2- 172.21.15.83:0/642369905 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcc8c0678c0 0x7fcc8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.393+0000 7fcca7400640 1 -- 172.21.15.83:0/642369905 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fcca0085770 con 0x7fccb01581d0 2024-04-25T00:13:06.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.397+0000 7fccaf400640 1 --2- 172.21.15.83:0/642369905 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcc8c0678c0 0x7fcc8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.397+0000 7fccaf400640 1 --2- 172.21.15.83:0/642369905 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcc8c0678c0 0x7fcc8c069d80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcc980af000 tx=0x7fcc980b4040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:06.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.401+0000 7fcca7400640 1 -- 172.21.15.83:0/642369905 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcca0020020 con 0x7fccb01581d0 2024-04-25T00:13:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.585+0000 7fccb6800640 1 -- 172.21.15.83:0/642369905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccb004f5f0 con 0x7fccb01581d0 2024-04-25T00:13:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.589+0000 7fcca7400640 1 -- 172.21.15.83:0/642369905 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5181 (secure 0 0 0) 0x7fcca0055a40 con 0x7fccb01581d0 2024-04-25T00:13:06.590 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:06.590 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2761747029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2761747029},{"type":"v1","addr":"172.21.15.83:6837","nonce":2761747029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2989939837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2989939837},{"type":"v1","addr":"172.21.15.86:6835","nonce":2989939837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2291906415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2291906415},{"type":"v1","addr":"172.21.15.83:6836","nonce":2291906415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:12:52.026135+0000","modified":"2024-04-25T00:12:55.045476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5814":{"gid":5814,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6834/1690820142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1690820142},{"type":"v1","addr":"172.21.15.86:6834","nonce":1690820142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5814,"qdb_cluster":[5814]},"id":3}]} 2024-04-25T00:13:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-04-25T00:13:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.589+0000 7fccb6800640 1 -- 172.21.15.83:0/642369905 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcc8c0678c0 msgr2=0x7fcc8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.589+0000 7fccb6800640 1 --2- 172.21.15.83:0/642369905 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcc8c0678c0 0x7fcc8c069d80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcc980af000 tx=0x7fcc980b4040 comp rx=0 tx=0).stop 2024-04-25T00:13:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.589+0000 7fccb6800640 1 -- 172.21.15.83:0/642369905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb01581d0 msgr2=0x7fccb01510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.589+0000 7fccb6800640 1 --2- 172.21.15.83:0/642369905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb01581d0 0x7fccb01510a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcca000c500 tx=0x7fcca000c9c0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.589+0000 7fccb6800640 1 -- 172.21.15.83:0/642369905 shutdown_connections 2024-04-25T00:13:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.589+0000 7fccb6800640 1 --2- 172.21.15.83:0/642369905 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcc8c0678c0 0x7fcc8c069d80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.589+0000 7fccb6800640 1 --2- 172.21.15.83:0/642369905 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccb015e6b0 0x7fccb014e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.589+0000 7fccb6800640 1 --2- 172.21.15.83:0/642369905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb01581d0 0x7fccb01510a0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.589+0000 7fccb6800640 1 --2- 172.21.15.83:0/642369905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccb00cf3c0 0x7fccb014c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.589+0000 7fccb6800640 1 -- 172.21.15.83:0/642369905 >> 172.21.15.83:0/642369905 conn(0x7fccb00b1830 msgr2=0x7fccb00b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.589+0000 7fccb6800640 1 -- 172.21.15.83:0/642369905 shutdown_connections 2024-04-25T00:13:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.589+0000 7fccb6800640 1 -- 172.21.15.83:0/642369905 wait complete. 2024-04-25T00:13:06.611 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:06.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb17e00640 1 Processor -- start 2024-04-25T00:13:06.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb17e00640 1 -- start start 2024-04-25T00:13:06.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb17e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbb101599f0 0x7fbb100cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb17e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbb1015cec0 0x7fbb100ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb17e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb100ccc60 con 0x7fbb1015c340 2024-04-25T00:13:06.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb17e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbb10159dd0 con 0x7fbb101599f0 2024-04-25T00:13:06.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb17e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbb1006dba0 con 0x7fbb1015cec0 2024-04-25T00:13:06.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb15400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbb101599f0 0x7fbb100cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb15400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbb101599f0 0x7fbb100cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60572/0 (socket says 172.21.15.83:60572) 2024-04-25T00:13:06.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb15400640 1 -- 172.21.15.83:0/235190030 learned_addr learned my addr 172.21.15.83:0/235190030 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:06.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb15e00640 1 --2- 172.21.15.83:0/235190030 >> v2:172.21.15.86:3300/0 conn(0x7fbb1015cec0 0x7fbb100ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb15400640 1 -- 172.21.15.83:0/235190030 >> v2:172.21.15.86:3300/0 conn(0x7fbb1015cec0 msgr2=0x7fbb100ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:06.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb15400640 1 --2- 172.21.15.83:0/235190030 >> v2:172.21.15.86:3300/0 conn(0x7fbb1015cec0 0x7fbb100ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb15400640 1 -- 172.21.15.83:0/235190030 >> v1:172.21.15.86:6789/0 conn(0x7fbb1015c340 legacy=0x7fbb1015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:06.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb15400640 1 -- 172.21.15.83:0/235190030 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1006de80 con 0x7fbb101599f0 2024-04-25T00:13:06.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.745+0000 7fbb15e00640 1 --2- 172.21.15.83:0/235190030 >> v2:172.21.15.86:3300/0 conn(0x7fbb1015cec0 0x7fbb100ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:06.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.749+0000 7fbb15400640 1 --2- 172.21.15.83:0/235190030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb101599f0 0x7fbb100cc510 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009870 tx=0x7fbb0401c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d83c7d1f781e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.749+0000 7fbb0fe00640 1 -- 172.21.15.83:0/235190030 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb040b0070 con 0x7fbb101599f0 2024-04-25T00:13:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.749+0000 7fbb0fe00640 1 -- 172.21.15.83:0/235190030 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb040ab600 con 0x7fbb101599f0 2024-04-25T00:13:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.749+0000 7fbb0fe00640 1 -- 172.21.15.83:0/235190030 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb040ab9d0 con 0x7fbb101599f0 2024-04-25T00:13:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.749+0000 7fbb17e00640 1 -- 172.21.15.83:0/235190030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb101599f0 msgr2=0x7fbb100cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.749+0000 7fbb17e00640 1 --2- 172.21.15.83:0/235190030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb101599f0 0x7fbb100cc510 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009870 tx=0x7fbb0401c270 comp rx=0 tx=0).stop 2024-04-25T00:13:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.749+0000 7fbb17e00640 1 -- 172.21.15.83:0/235190030 shutdown_connections 2024-04-25T00:13:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.749+0000 7fbb17e00640 1 --2- 172.21.15.83:0/235190030 >> v2:172.21.15.86:3300/0 conn(0x7fbb1015cec0 0x7fbb100ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.749+0000 7fbb17e00640 1 --2- 172.21.15.83:0/235190030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb101599f0 0x7fbb100cc510 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.749+0000 7fbb17e00640 1 -- 172.21.15.83:0/235190030 >> 172.21.15.83:0/235190030 conn(0x7fbb100b1830 msgr2=0x7fbb100b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.749+0000 7fbb17e00640 1 -- 172.21.15.83:0/235190030 shutdown_connections 2024-04-25T00:13:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.749+0000 7fbb17e00640 1 -- 172.21.15.83:0/235190030 wait complete. 2024-04-25T00:13:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb17e00640 1 Processor -- start 2024-04-25T00:13:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb17e00640 1 -- start start 2024-04-25T00:13:06.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb17e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb101599f0 0x7fbb101f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb17e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1015c340 0x7fbb101f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb17e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb1015cec0 0x7fbb10152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb17e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb100cf800 con 0x7fbb101599f0 2024-04-25T00:13:06.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb17e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb100cf680 con 0x7fbb1015c340 2024-04-25T00:13:06.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb17e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbb10168cd0 con 0x7fbb1015cec0 2024-04-25T00:13:06.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb15400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb101599f0 0x7fbb101f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb15400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb101599f0 0x7fbb101f7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60588/0 (socket says 172.21.15.83:60588) 2024-04-25T00:13:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb15400640 1 -- 172.21.15.83:0/4045856807 learned_addr learned my addr 172.21.15.83:0/4045856807 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb14a00640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1015c340 0x7fbb101f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb15e00640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb1015cec0 0x7fbb10152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb15400640 1 -- 172.21.15.83:0/4045856807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb1015cec0 msgr2=0x7fbb10152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb15400640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb1015cec0 0x7fbb10152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb15400640 1 -- 172.21.15.83:0/4045856807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1015c340 msgr2=0x7fbb101f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb15400640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1015c340 0x7fbb101f5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb15400640 1 -- 172.21.15.83:0/4045856807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb101f5ff0 con 0x7fbb101599f0 2024-04-25T00:13:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb14a00640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1015c340 0x7fbb101f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb15400640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb101599f0 0x7fbb101f7ff0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbb040ae040 tx=0x7fbb04019010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb0d600640 1 -- 172.21.15.83:0/4045856807 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb040b0040 con 0x7fbb101599f0 2024-04-25T00:13:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb0d600640 1 -- 172.21.15.83:0/4045856807 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb040b9070 con 0x7fbb101599f0 2024-04-25T00:13:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb0d600640 1 -- 172.21.15.83:0/4045856807 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb040b4880 con 0x7fbb101599f0 2024-04-25T00:13:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb17e00640 1 -- 172.21.15.83:0/4045856807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb101545d0 con 0x7fbb101599f0 2024-04-25T00:13:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.753+0000 7fbb17e00640 1 -- 172.21.15.83:0/4045856807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb10154b50 con 0x7fbb101599f0 2024-04-25T00:13:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.757+0000 7fbb17e00640 1 -- 172.21.15.83:0/4045856807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb100737f0 con 0x7fbb101599f0 2024-04-25T00:13:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.757+0000 7fbb0d600640 1 -- 172.21.15.83:0/4045856807 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbb040ab4c0 con 0x7fbb101599f0 2024-04-25T00:13:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.757+0000 7fbb0d600640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbaf00678c0 0x7fbaf0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.757+0000 7fbb0d600640 1 -- 172.21.15.83:0/4045856807 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fbb0411fb90 con 0x7fbb101599f0 2024-04-25T00:13:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.757+0000 7fbb14a00640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbaf00678c0 0x7fbaf0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.757+0000 7fbb14a00640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbaf00678c0 0x7fbaf0069d80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbb10153870 tx=0x7fbafc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.761+0000 7fbb0d600640 1 -- 172.21.15.83:0/4045856807 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbb101f5ff0 con 0x7fbb101599f0 2024-04-25T00:13:06.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.917+0000 7fbb17e00640 1 -- 172.21.15.83:0/4045856807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb10072c40 con 0x7fbb101599f0 2024-04-25T00:13:06.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.917+0000 7fbb0d600640 1 -- 172.21.15.83:0/4045856807 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16858 (secure 0 0 0) 0x7fbb100ccc60 con 0x7fbb101599f0 2024-04-25T00:13:06.920 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:06.920 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":49,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:53.651846+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:50.394645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:51.018842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:06.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.917+0000 7fbb17e00640 1 -- 172.21.15.83:0/4045856807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbaf00678c0 msgr2=0x7fbaf0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.917+0000 7fbb17e00640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbaf00678c0 0x7fbaf0069d80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbb10153870 tx=0x7fbafc009040 comp rx=0 tx=0).stop 2024-04-25T00:13:06.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.917+0000 7fbb17e00640 1 -- 172.21.15.83:0/4045856807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb101599f0 msgr2=0x7fbb101f7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:06.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.917+0000 7fbb17e00640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb101599f0 0x7fbb101f7ff0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbb040ae040 tx=0x7fbb04019010 comp rx=0 tx=0).stop 2024-04-25T00:13:06.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.917+0000 7fbb17e00640 1 -- 172.21.15.83:0/4045856807 shutdown_connections 2024-04-25T00:13:06.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.917+0000 7fbb17e00640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbaf00678c0 0x7fbaf0069d80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.917+0000 7fbb17e00640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb1015cec0 0x7fbb10152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.917+0000 7fbb17e00640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1015c340 0x7fbb101f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.917+0000 7fbb17e00640 1 --2- 172.21.15.83:0/4045856807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb101599f0 0x7fbb101f7ff0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:06.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.917+0000 7fbb17e00640 1 -- 172.21.15.83:0/4045856807 >> 172.21.15.83:0/4045856807 conn(0x7fbb100b1830 msgr2=0x7fbb1015b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:06.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.921+0000 7fbb17e00640 1 -- 172.21.15.83:0/4045856807 shutdown_connections 2024-04-25T00:13:06.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:06.921+0000 7fbb17e00640 1 -- 172.21.15.83:0/4045856807 wait complete. 2024-04-25T00:13:06.941 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:13:06.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:07.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.073+0000 7f7710a00640 1 Processor -- start 2024-04-25T00:13:07.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.073+0000 7f7710a00640 1 -- start start 2024-04-25T00:13:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.077+0000 7f7710a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c1581d0 0x7f770c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.077+0000 7f7710a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f770c0cf3c0 0x7f770c15dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.077+0000 7f7710a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f770c0b4c30 con 0x7f770c15e4e0 2024-04-25T00:13:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.077+0000 7f7710a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f770c15e930 con 0x7f770c0cf3c0 2024-04-25T00:13:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.077+0000 7f7710a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f770c0b3440 con 0x7f770c1581d0 2024-04-25T00:13:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.077+0000 7f7709400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f770c0cf3c0 0x7f770c15dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.077+0000 7f770a800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f770c15e4e0 0x7f770c15a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33322/0 (socket says 172.21.15.83:33322) 2024-04-25T00:13:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.077+0000 7f770a800640 1 -- 172.21.15.83:0/1197909925 learned_addr learned my addr 172.21.15.83:0/1197909925 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.077+0000 7f7709e00640 1 --2- 172.21.15.83:0/1197909925 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c1581d0 0x7f770c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.077+0000 7f7709400640 1 -- 172.21.15.83:0/1197909925 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c1581d0 msgr2=0x7f770c0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.077+0000 7f7709400640 1 --2- 172.21.15.83:0/1197909925 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c1581d0 0x7f770c0b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.077+0000 7f7709400640 1 -- 172.21.15.83:0/1197909925 >> v1:172.21.15.83:6789/0 conn(0x7f770c15e4e0 legacy=0x7f770c15a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.077+0000 7f7708a00640 1 -- 172.21.15.83:0/1197909925 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2357886520 0 0) 0x7f770c0b4c30 con 0x7f770c15e4e0 2024-04-25T00:13:07.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.077+0000 7f7709400640 1 -- 172.21.15.83:0/1197909925 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c0b3720 con 0x7f770c0cf3c0 2024-04-25T00:13:07.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.081+0000 7f7709400640 1 --2- 172.21.15.83:0/1197909925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c0cf3c0 0x7f770c15dfa0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f76f80097c0 tx=0x7f76f801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bd7599dbf44211 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.081+0000 7f7708a00640 1 -- 172.21.15.83:0/1197909925 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f80b1070 con 0x7f770c0cf3c0 2024-04-25T00:13:07.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.081+0000 7f7708a00640 1 -- 172.21.15.83:0/1197909925 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f770c15b880 con 0x7f770c0cf3c0 2024-04-25T00:13:07.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.081+0000 7f7708a00640 1 -- 172.21.15.83:0/1197909925 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f80ac630 con 0x7f770c0cf3c0 2024-04-25T00:13:07.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.081+0000 7f7710a00640 1 -- 172.21.15.83:0/1197909925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c0cf3c0 msgr2=0x7f770c15dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.081+0000 7f7710a00640 1 --2- 172.21.15.83:0/1197909925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c0cf3c0 0x7f770c15dfa0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f76f80097c0 tx=0x7f76f801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.081+0000 7f7710a00640 1 -- 172.21.15.83:0/1197909925 shutdown_connections 2024-04-25T00:13:07.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.081+0000 7f7710a00640 1 --2- 172.21.15.83:0/1197909925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c0cf3c0 0x7f770c15dfa0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.081+0000 7f7710a00640 1 --2- 172.21.15.83:0/1197909925 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c1581d0 0x7f770c0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.081+0000 7f7710a00640 1 -- 172.21.15.83:0/1197909925 >> 172.21.15.83:0/1197909925 conn(0x7f770c0b1830 msgr2=0x7f770c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:07.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.081+0000 7f7710a00640 1 -- 172.21.15.83:0/1197909925 shutdown_connections 2024-04-25T00:13:07.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.081+0000 7f7710a00640 1 -- 172.21.15.83:0/1197909925 wait complete. 2024-04-25T00:13:07.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.081+0000 7f7710a00640 1 Processor -- start 2024-04-25T00:13:07.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.081+0000 7f7710a00640 1 -- start start 2024-04-25T00:13:07.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7710a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c0cf3c0 0x7f770c14c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7710a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f770c1581d0 0x7f770c1561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7710a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c15e4e0 0x7f770c1566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7710a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f770c0b4800 con 0x7f770c15e4e0 2024-04-25T00:13:07.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7710a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f770c15ed00 con 0x7f770c1581d0 2024-04-25T00:13:07.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7710a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f770c0b4980 con 0x7f770c0cf3c0 2024-04-25T00:13:07.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f770a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c15e4e0 0x7f770c1566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f770a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c15e4e0 0x7f770c1566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60610/0 (socket says 172.21.15.83:60610) 2024-04-25T00:13:07.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7709e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c0cf3c0 0x7f770c14c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f770a800640 1 -- 172.21.15.83:0/2329263343 learned_addr learned my addr 172.21.15.83:0/2329263343 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:07.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7709400640 1 --2- 172.21.15.83:0/2329263343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f770c1581d0 0x7f770c1561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f770a800640 1 -- 172.21.15.83:0/2329263343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c0cf3c0 msgr2=0x7f770c14c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f770a800640 1 --2- 172.21.15.83:0/2329263343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c0cf3c0 0x7f770c14c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f770a800640 1 -- 172.21.15.83:0/2329263343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f770c1581d0 msgr2=0x7f770c1561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f770a800640 1 --2- 172.21.15.83:0/2329263343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f770c1581d0 0x7f770c1561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f770a800640 1 -- 172.21.15.83:0/2329263343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c0ae310 con 0x7f770c15e4e0 2024-04-25T00:13:07.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f770a800640 1 --2- 172.21.15.83:0/2329263343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c15e4e0 0x7f770c1566e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f770400c3e0 tx=0x7f770400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7702000640 1 -- 172.21.15.83:0/2329263343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77040074e0 con 0x7f770c15e4e0 2024-04-25T00:13:07.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7702000640 1 -- 172.21.15.83:0/2329263343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77040047f0 con 0x7f770c15e4e0 2024-04-25T00:13:07.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7702000640 1 -- 172.21.15.83:0/2329263343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704004be0 con 0x7f770c15e4e0 2024-04-25T00:13:07.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7710a00640 1 -- 172.21.15.83:0/2329263343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770c0ae580 con 0x7f770c15e4e0 2024-04-25T00:13:07.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7710a00640 1 -- 172.21.15.83:0/2329263343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f770c0aeb40 con 0x7f770c15e4e0 2024-04-25T00:13:07.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7710a00640 1 -- 172.21.15.83:0/2329263343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770c0737f0 con 0x7f770c15e4e0 2024-04-25T00:13:07.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.085+0000 7f7702000640 1 -- 172.21.15.83:0/2329263343 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7704011070 con 0x7f770c15e4e0 2024-04-25T00:13:07.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.089+0000 7f7702000640 1 --2- 172.21.15.83:0/2329263343 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76ec0678c0 0x7f76ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.089+0000 7f7702000640 1 -- 172.21.15.83:0/2329263343 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f7704085910 con 0x7f770c15e4e0 2024-04-25T00:13:07.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.089+0000 7f7709e00640 1 --2- 172.21.15.83:0/2329263343 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76ec0678c0 0x7f76ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.089+0000 7f7709e00640 1 --2- 172.21.15.83:0/2329263343 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76ec0678c0 0x7f76ec069d80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f76fc003b90 tx=0x7f76fc0098d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.093+0000 7f7702000640 1 -- 172.21.15.83:0/2329263343 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7704055bd0 con 0x7f770c15e4e0 2024-04-25T00:13:07.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.277+0000 7f7710a00640 1 -- 172.21.15.83:0/2329263343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f770c04f5f0 con 0x7f770c15e4e0 2024-04-25T00:13:07.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.277+0000 7f7702000640 1 -- 172.21.15.83:0/2329263343 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5181 (secure 0 0 0) 0x7f7704059ed0 con 0x7f770c15e4e0 2024-04-25T00:13:07.281 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:07.281 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2761747029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2761747029},{"type":"v1","addr":"172.21.15.83:6837","nonce":2761747029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2989939837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2989939837},{"type":"v1","addr":"172.21.15.86:6835","nonce":2989939837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2291906415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2291906415},{"type":"v1","addr":"172.21.15.83:6836","nonce":2291906415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:12:52.026135+0000","modified":"2024-04-25T00:12:55.045476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5814":{"gid":5814,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6834/1690820142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1690820142},{"type":"v1","addr":"172.21.15.86:6834","nonce":1690820142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5814,"qdb_cluster":[5814]},"id":3}]} 2024-04-25T00:13:07.281 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-04-25T00:13:07.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.281+0000 7f7710a00640 1 -- 172.21.15.83:0/2329263343 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76ec0678c0 msgr2=0x7f76ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.281+0000 7f7710a00640 1 --2- 172.21.15.83:0/2329263343 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76ec0678c0 0x7f76ec069d80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f76fc003b90 tx=0x7f76fc0098d0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.281+0000 7f7710a00640 1 -- 172.21.15.83:0/2329263343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c15e4e0 msgr2=0x7f770c1566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.281+0000 7f7710a00640 1 --2- 172.21.15.83:0/2329263343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c15e4e0 0x7f770c1566e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f770400c3e0 tx=0x7f770400c8a0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.281+0000 7f7710a00640 1 -- 172.21.15.83:0/2329263343 shutdown_connections 2024-04-25T00:13:07.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.281+0000 7f7710a00640 1 --2- 172.21.15.83:0/2329263343 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76ec0678c0 0x7f76ec069d80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.281+0000 7f7710a00640 1 --2- 172.21.15.83:0/2329263343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c15e4e0 0x7f770c1566e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.281+0000 7f7710a00640 1 --2- 172.21.15.83:0/2329263343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f770c1581d0 0x7f770c1561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.281+0000 7f7710a00640 1 --2- 172.21.15.83:0/2329263343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c0cf3c0 0x7f770c14c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.281+0000 7f7710a00640 1 -- 172.21.15.83:0/2329263343 >> 172.21.15.83:0/2329263343 conn(0x7f770c0b1830 msgr2=0x7f770c15cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:07.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.281+0000 7f7710a00640 1 -- 172.21.15.83:0/2329263343 shutdown_connections 2024-04-25T00:13:07.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.281+0000 7f7710a00640 1 -- 172.21.15.83:0/2329263343 wait complete. 2024-04-25T00:13:07.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:07.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.433+0000 7fcb43600640 1 Processor -- start 2024-04-25T00:13:07.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.433+0000 7fcb43600640 1 -- start start 2024-04-25T00:13:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb43600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb3c1581d0 0x7fcb3c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb43600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb3c15e6b0 0x7fcb3c06cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb43600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb3c0b4c30 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb43600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fcb3c0b3440 con 0x7fcb3c1581d0 2024-04-25T00:13:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb43600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3c0b3720 con 0x7fcb3c15e6b0 2024-04-25T00:13:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb33e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcb3c0cf3c0 0x7fcb3c15dfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33328/0 (socket says 172.21.15.83:33328) 2024-04-25T00:13:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb33e00640 1 -- 172.21.15.83:0/3954045080 learned_addr learned my addr 172.21.15.83:0/3954045080 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb40c00640 1 --2- 172.21.15.83:0/3954045080 >> v2:172.21.15.83:3300/0 conn(0x7fcb3c1581d0 0x7fcb3c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb41600640 1 --2- 172.21.15.83:0/3954045080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb3c15e6b0 0x7fcb3c06cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb3be00640 1 -- 172.21.15.83:0/3954045080 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2201920116 0 0) 0x7fcb3c0b4c30 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb3be00640 1 -- 172.21.15.83:0/3954045080 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb20003530 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb40c00640 1 -- 172.21.15.83:0/3954045080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb3c15e6b0 msgr2=0x7fcb3c06cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb40c00640 1 --2- 172.21.15.83:0/3954045080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb3c15e6b0 0x7fcb3c06cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb40c00640 1 -- 172.21.15.83:0/3954045080 >> v1:172.21.15.83:6789/0 conn(0x7fcb3c0cf3c0 legacy=0x7fcb3c15dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.437+0000 7fcb40c00640 1 -- 172.21.15.83:0/3954045080 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3c0b3a00 con 0x7fcb3c1581d0 2024-04-25T00:13:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.441+0000 7fcb40c00640 1 --2- 172.21.15.83:0/3954045080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c1581d0 0x7fcb3c0b3f20 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009d90 tx=0x7fcb2c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5c6f9b5fd83790 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.441+0000 7fcb3be00640 1 -- 172.21.15.83:0/3954045080 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c0b2070 con 0x7fcb3c1581d0 2024-04-25T00:13:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.441+0000 7fcb3be00640 1 -- 172.21.15.83:0/3954045080 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb2c0a5a50 con 0x7fcb3c1581d0 2024-04-25T00:13:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.441+0000 7fcb3be00640 1 -- 172.21.15.83:0/3954045080 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c0a5e20 con 0x7fcb3c1581d0 2024-04-25T00:13:07.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.441+0000 7fcb43600640 1 -- 172.21.15.83:0/3954045080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c1581d0 msgr2=0x7fcb3c0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.441+0000 7fcb43600640 1 --2- 172.21.15.83:0/3954045080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c1581d0 0x7fcb3c0b3f20 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009d90 tx=0x7fcb2c01c970 comp rx=0 tx=0).stop 2024-04-25T00:13:07.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.441+0000 7fcb43600640 1 -- 172.21.15.83:0/3954045080 shutdown_connections 2024-04-25T00:13:07.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.441+0000 7fcb43600640 1 --2- 172.21.15.83:0/3954045080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb3c15e6b0 0x7fcb3c06cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.441+0000 7fcb43600640 1 --2- 172.21.15.83:0/3954045080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c1581d0 0x7fcb3c0b3f20 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.441+0000 7fcb43600640 1 -- 172.21.15.83:0/3954045080 >> 172.21.15.83:0/3954045080 conn(0x7fcb3c0b1830 msgr2=0x7fcb3c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:07.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.441+0000 7fcb43600640 1 -- 172.21.15.83:0/3954045080 shutdown_connections 2024-04-25T00:13:07.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.441+0000 7fcb43600640 1 -- 172.21.15.83:0/3954045080 wait complete. 2024-04-25T00:13:07.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.441+0000 7fcb43600640 1 Processor -- start 2024-04-25T00:13:07.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.441+0000 7fcb43600640 1 -- start start 2024-04-25T00:13:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb43600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c0cf3c0 0x7fcb3c0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb43600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb3c1581d0 0x7fcb3c1f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb43600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb3c15e6b0 0x7fcb3c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb43600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c06d9f0 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb43600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c06d870 con 0x7fcb3c1581d0 2024-04-25T00:13:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb43600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3c06db70 con 0x7fcb3c15e6b0 2024-04-25T00:13:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb40c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c0cf3c0 0x7fcb3c0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb40c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c0cf3c0 0x7fcb3c0ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60630/0 (socket says 172.21.15.83:60630) 2024-04-25T00:13:07.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb40c00640 1 -- 172.21.15.83:0/672175481 learned_addr learned my addr 172.21.15.83:0/672175481 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:07.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb41600640 1 --2- 172.21.15.83:0/672175481 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb3c15e6b0 0x7fcb3c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb33e00640 1 --2- 172.21.15.83:0/672175481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb3c1581d0 0x7fcb3c1f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb40c00640 1 -- 172.21.15.83:0/672175481 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb3c15e6b0 msgr2=0x7fcb3c15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb40c00640 1 --2- 172.21.15.83:0/672175481 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb3c15e6b0 0x7fcb3c15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb40c00640 1 -- 172.21.15.83:0/672175481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb3c1581d0 msgr2=0x7fcb3c1f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb40c00640 1 --2- 172.21.15.83:0/672175481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb3c1581d0 0x7fcb3c1f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb40c00640 1 -- 172.21.15.83:0/672175481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3c0ce750 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb40c00640 1 --2- 172.21.15.83:0/672175481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c0cf3c0 0x7fcb3c0ce210 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c01a5a0 tx=0x7fcb2c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb39600640 1 -- 172.21.15.83:0/672175481 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c0b2050 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb39600640 1 -- 172.21.15.83:0/672175481 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb2c0be070 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb39600640 1 -- 172.21.15.83:0/672175481 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c009450 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb43600640 1 -- 172.21.15.83:0/672175481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3c1444f0 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb43600640 1 -- 172.21.15.83:0/672175481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3c144ab0 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb43600640 1 -- 172.21.15.83:0/672175481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb3c0737f0 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.445+0000 7fcb39600640 1 -- 172.21.15.83:0/672175481 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcb2c0c3460 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.449+0000 7fcb39600640 1 --2- 172.21.15.83:0/672175481 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb180678c0 0x7fcb18069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.449+0000 7fcb39600640 1 -- 172.21.15.83:0/672175481 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fcb2c1212c0 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.449+0000 7fcb33e00640 1 --2- 172.21.15.83:0/672175481 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb180678c0 0x7fcb18069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.449+0000 7fcb33e00640 1 --2- 172.21.15.83:0/672175481 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb180678c0 0x7fcb18069d80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcb24007fd0 tx=0x7fcb24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.453+0000 7fcb39600640 1 -- 172.21.15.83:0/672175481 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcb3c0ce750 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.641+0000 7fcb43600640 1 -- 172.21.15.83:0/672175481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb3c04f5f0 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.641+0000 7fcb39600640 1 -- 172.21.15.83:0/672175481 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5181 (secure 0 0 0) 0x7fcb2c016560 con 0x7fcb3c0cf3c0 2024-04-25T00:13:07.644 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:07.644 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2761747029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2761747029},{"type":"v1","addr":"172.21.15.83:6837","nonce":2761747029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2989939837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2989939837},{"type":"v1","addr":"172.21.15.86:6835","nonce":2989939837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2291906415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2291906415},{"type":"v1","addr":"172.21.15.83:6836","nonce":2291906415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:12:52.026135+0000","modified":"2024-04-25T00:12:55.045476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5814":{"gid":5814,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6834/1690820142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1690820142},{"type":"v1","addr":"172.21.15.86:6834","nonce":1690820142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5814,"qdb_cluster":[5814]},"id":3}]} 2024-04-25T00:13:07.645 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-04-25T00:13:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.641+0000 7fcb43600640 1 -- 172.21.15.83:0/672175481 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb180678c0 msgr2=0x7fcb18069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.641+0000 7fcb43600640 1 --2- 172.21.15.83:0/672175481 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb180678c0 0x7fcb18069d80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcb24007fd0 tx=0x7fcb24009040 comp rx=0 tx=0).stop 2024-04-25T00:13:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.641+0000 7fcb43600640 1 -- 172.21.15.83:0/672175481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c0cf3c0 msgr2=0x7fcb3c0ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.641+0000 7fcb43600640 1 --2- 172.21.15.83:0/672175481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c0cf3c0 0x7fcb3c0ce210 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c01a5a0 tx=0x7fcb2c019710 comp rx=0 tx=0).stop 2024-04-25T00:13:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.641+0000 7fcb43600640 1 -- 172.21.15.83:0/672175481 shutdown_connections 2024-04-25T00:13:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.641+0000 7fcb43600640 1 --2- 172.21.15.83:0/672175481 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb180678c0 0x7fcb18069d80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.641+0000 7fcb43600640 1 --2- 172.21.15.83:0/672175481 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb3c15e6b0 0x7fcb3c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.641+0000 7fcb43600640 1 --2- 172.21.15.83:0/672175481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb3c1581d0 0x7fcb3c1f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.641+0000 7fcb43600640 1 --2- 172.21.15.83:0/672175481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c0cf3c0 0x7fcb3c0ce210 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.641+0000 7fcb43600640 1 -- 172.21.15.83:0/672175481 >> 172.21.15.83:0/672175481 conn(0x7fcb3c0b1830 msgr2=0x7fcb3c0ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.641+0000 7fcb43600640 1 -- 172.21.15.83:0/672175481 shutdown_connections 2024-04-25T00:13:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.641+0000 7fcb43600640 1 -- 172.21.15.83:0/672175481 wait complete. 2024-04-25T00:13:07.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:07.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedcaa00640 1 Processor -- start 2024-04-25T00:13:07.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedcaa00640 1 -- start start 2024-04-25T00:13:07.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedcaa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedc40cc510 0x7fedc415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedcaa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fedc415cf90 0x7fedc40ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedcaa00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedc40ccc60 con 0x7fedc41599f0 2024-04-25T00:13:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedcaa00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fedc4159dd0 con 0x7fedc415cf90 2024-04-25T00:13:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedcaa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fedc406dba0 con 0x7fedc40cc510 2024-04-25T00:13:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedc3e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fedc41599f0 0x7fedc415c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33336/0 (socket says 172.21.15.83:33336) 2024-04-25T00:13:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedc3e00640 1 -- 172.21.15.83:0/2674058083 learned_addr learned my addr 172.21.15.83:0/2674058083 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedc8a00640 1 --2- 172.21.15.83:0/2674058083 >> v2:172.21.15.83:3300/0 conn(0x7fedc415cf90 0x7fedc40ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedc3400640 1 --2- 172.21.15.83:0/2674058083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedc40cc510 0x7fedc415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedc2a00640 1 -- 172.21.15.83:0/2674058083 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 158800495 0 0) 0x7fedb4001120 con 0x7fedc41599f0 2024-04-25T00:13:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedc2a00640 1 -- 172.21.15.83:0/2674058083 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feda8003530 con 0x7fedc41599f0 2024-04-25T00:13:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedc8a00640 1 -- 172.21.15.83:0/2674058083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedc40cc510 msgr2=0x7fedc415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedc8a00640 1 --2- 172.21.15.83:0/2674058083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedc40cc510 0x7fedc415ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedc8a00640 1 -- 172.21.15.83:0/2674058083 >> v1:172.21.15.83:6789/0 conn(0x7fedc41599f0 legacy=0x7fedc415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedc8a00640 1 -- 172.21.15.83:0/2674058083 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc406de80 con 0x7fedc415cf90 2024-04-25T00:13:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.801+0000 7fedc3400640 1 --2- 172.21.15.83:0/2674058083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedc40cc510 0x7fedc415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:07.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.805+0000 7fedc8a00640 1 --2- 172.21.15.83:0/2674058083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc415cf90 0x7fedc40ce6e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009d90 tx=0x7fedbc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672b06d4d6f068fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.805+0000 7fedc2a00640 1 -- 172.21.15.83:0/2674058083 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0b2070 con 0x7fedc415cf90 2024-04-25T00:13:07.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.805+0000 7fedc2a00640 1 -- 172.21.15.83:0/2674058083 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedbc0a57f0 con 0x7fedc415cf90 2024-04-25T00:13:07.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.805+0000 7fedc2a00640 1 -- 172.21.15.83:0/2674058083 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0a5bc0 con 0x7fedc415cf90 2024-04-25T00:13:07.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.805+0000 7fedcaa00640 1 -- 172.21.15.83:0/2674058083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc415cf90 msgr2=0x7fedc40ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.805+0000 7fedcaa00640 1 --2- 172.21.15.83:0/2674058083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc415cf90 0x7fedc40ce6e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009d90 tx=0x7fedbc01c8e0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.805+0000 7fedcaa00640 1 -- 172.21.15.83:0/2674058083 shutdown_connections 2024-04-25T00:13:07.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.805+0000 7fedcaa00640 1 --2- 172.21.15.83:0/2674058083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc415cf90 0x7fedc40ce6e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.805+0000 7fedcaa00640 1 --2- 172.21.15.83:0/2674058083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedc40cc510 0x7fedc415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.805+0000 7fedcaa00640 1 -- 172.21.15.83:0/2674058083 >> 172.21.15.83:0/2674058083 conn(0x7fedc40b1830 msgr2=0x7fedc40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:07.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.805+0000 7fedcaa00640 1 -- 172.21.15.83:0/2674058083 shutdown_connections 2024-04-25T00:13:07.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.805+0000 7fedcaa00640 1 -- 172.21.15.83:0/2674058083 wait complete. 2024-04-25T00:13:07.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedcaa00640 1 Processor -- start 2024-04-25T00:13:07.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedcaa00640 1 -- start start 2024-04-25T00:13:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedcaa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedc40cc510 0x7fedc415eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedcaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedc41599f0 0x7fedc414b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedcaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc415cf90 0x7fedc4150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedcaa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fedc40cf5b0 con 0x7fedc415cf90 2024-04-25T00:13:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedcaa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fedc40cf430 con 0x7fedc41599f0 2024-04-25T00:13:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedcaa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fedc40cf730 con 0x7fedc40cc510 2024-04-25T00:13:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedc8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc415cf90 0x7fedc4150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedc8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc415cf90 0x7fedc4150410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60646/0 (socket says 172.21.15.83:60646) 2024-04-25T00:13:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedc8a00640 1 -- 172.21.15.83:0/3999124069 learned_addr learned my addr 172.21.15.83:0/3999124069 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedc3400640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedc41599f0 0x7fedc414b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedc3e00640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedc40cc510 0x7fedc415eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedc8a00640 1 -- 172.21.15.83:0/3999124069 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedc40cc510 msgr2=0x7fedc415eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedc8a00640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedc40cc510 0x7fedc415eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedc8a00640 1 -- 172.21.15.83:0/3999124069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedc41599f0 msgr2=0x7fedc414b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedc8a00640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedc41599f0 0x7fedc414b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedc8a00640 1 -- 172.21.15.83:0/3999124069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc415da50 con 0x7fedc415cf90 2024-04-25T00:13:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedc3400640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedc41599f0 0x7fedc414b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedc8a00640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc415cf90 0x7fedc4150410 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fedbc01af90 tx=0x7fedbc019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedbbe00640 1 -- 172.21.15.83:0/3999124069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0b2050 con 0x7fedc415cf90 2024-04-25T00:13:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedbbe00640 1 -- 172.21.15.83:0/3999124069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedbc0be070 con 0x7fedc415cf90 2024-04-25T00:13:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedbbe00640 1 -- 172.21.15.83:0/3999124069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0ad850 con 0x7fedc415cf90 2024-04-25T00:13:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedcaa00640 1 -- 172.21.15.83:0/3999124069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc415dd30 con 0x7fedc415cf90 2024-04-25T00:13:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedcaa00640 1 -- 172.21.15.83:0/3999124069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc415e2f0 con 0x7fedc415cf90 2024-04-25T00:13:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.809+0000 7fedcaa00640 1 -- 172.21.15.83:0/3999124069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc40737f0 con 0x7fedc415cf90 2024-04-25T00:13:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.813+0000 7fedbbe00640 1 -- 172.21.15.83:0/3999124069 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fedbc0ad9f0 con 0x7fedc415cf90 2024-04-25T00:13:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.813+0000 7fedbbe00640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feda40678c0 0x7feda4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.813+0000 7fedbbe00640 1 -- 172.21.15.83:0/3999124069 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fedbc120fa0 con 0x7fedc415cf90 2024-04-25T00:13:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.813+0000 7fedc3e00640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feda40678c0 0x7feda4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.813+0000 7fedc3e00640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feda40678c0 0x7feda4069d80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fedb4005890 tx=0x7fedb4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.817+0000 7fedbbe00640 1 -- 172.21.15.83:0/3999124069 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fedbc016560 con 0x7fedc415cf90 2024-04-25T00:13:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.973+0000 7fedcaa00640 1 -- 172.21.15.83:0/3999124069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedc404f5f0 con 0x7fedc415cf90 2024-04-25T00:13:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.977+0000 7fedbbe00640 1 -- 172.21.15.83:0/3999124069 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16858 (secure 0 0 0) 0x7fedbc0f1940 con 0x7fedc415cf90 2024-04-25T00:13:07.978 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:07.978 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":49,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:12:53.651846+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:12:50.394645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T00:12:51.018842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.977+0000 7fedcaa00640 1 -- 172.21.15.83:0/3999124069 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feda40678c0 msgr2=0x7feda4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.977+0000 7fedcaa00640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feda40678c0 0x7feda4069d80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fedb4005890 tx=0x7fedb4009040 comp rx=0 tx=0).stop 2024-04-25T00:13:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.977+0000 7fedcaa00640 1 -- 172.21.15.83:0/3999124069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc415cf90 msgr2=0x7fedc4150410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.977+0000 7fedcaa00640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc415cf90 0x7fedc4150410 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fedbc01af90 tx=0x7fedbc019680 comp rx=0 tx=0).stop 2024-04-25T00:13:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.977+0000 7fedcaa00640 1 -- 172.21.15.83:0/3999124069 shutdown_connections 2024-04-25T00:13:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.977+0000 7fedcaa00640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feda40678c0 0x7feda4069d80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.977+0000 7fedcaa00640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc415cf90 0x7fedc4150410 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.977+0000 7fedcaa00640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedc41599f0 0x7fedc414b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.977+0000 7fedcaa00640 1 --2- 172.21.15.83:0/3999124069 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedc40cc510 0x7fedc415eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.977+0000 7fedcaa00640 1 -- 172.21.15.83:0/3999124069 >> 172.21.15.83:0/3999124069 conn(0x7fedc40b1830 msgr2=0x7fedc406e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.977+0000 7fedcaa00640 1 -- 172.21.15.83:0/3999124069 shutdown_connections 2024-04-25T00:13:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:07.977+0000 7fedcaa00640 1 -- 172.21.15.83:0/3999124069 wait complete. 2024-04-25T00:13:07.999 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:13:08.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff595600640 1 Processor -- start 2024-04-25T00:13:08.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff595600640 1 -- start start 2024-04-25T00:13:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff595600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff59006dac0 0x7ff590070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff595600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff59006e160 con 0x7ff59006cfd0 2024-04-25T00:13:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff595600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff59006ea10 con 0x7ff5900cb640 2024-04-25T00:13:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff595600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff59006ecf0 con 0x7ff59006dac0 2024-04-25T00:13:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58ea00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff59006cfd0 0x7ff59006d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33340/0 (socket says 172.21.15.83:33340) 2024-04-25T00:13:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58ea00640 1 -- 172.21.15.83:0/3880235324 learned_addr learned my addr 172.21.15.83:0/3880235324 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58f400640 1 --2- 172.21.15.83:0/3880235324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff59006dac0 0x7ff590070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1307500451 0 0) 0x7ff59006e160 con 0x7ff59006cfd0 2024-04-25T00:13:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff574003530 con 0x7ff59006cfd0 2024-04-25T00:13:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4235813143 0 0) 0x7ff59006ea10 con 0x7ff5900cb640 2024-04-25T00:13:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff59006e160 con 0x7ff5900cb640 2024-04-25T00:13:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3794746500 0 0) 0x7ff574003530 con 0x7ff59006cfd0 2024-04-25T00:13:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff59006dac0 msgr2=0x7ff590070d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 --2- 172.21.15.83:0/3880235324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff59006dac0 0x7ff590070d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 >> v1:172.21.15.86:6789/0 conn(0x7ff5900cb640 legacy=0x7ff590163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59006efd0 con 0x7ff59006cfd0 2024-04-25T00:13:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7ff580001680 con 0x7ff59006cfd0 2024-04-25T00:13:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58f400640 1 --2- 172.21.15.83:0/3880235324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff59006dac0 0x7ff590070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 >> v1:172.21.15.83:6789/0 conn(0x7ff59006cfd0 legacy=0x7ff59006d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 --2- 172.21.15.83:0/3880235324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff574003b80 0x7ff574003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 --2- 172.21.15.83:0/3880235324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff574004810 0x7ff574004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 --2- 172.21.15.83:0/3880235324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff59006dac0 0x7ff5740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff59006ecf0 con 0x7ff574004810 2024-04-25T00:13:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff57400ba00 con 0x7ff574003b80 2024-04-25T00:13:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff57400bc00 con 0x7ff59006dac0 2024-04-25T00:13:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff586000640 1 --2- 172.21.15.83:0/3880235324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff574004810 0x7ff574004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58f400640 1 --2- 172.21.15.83:0/3880235324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff59006dac0 0x7ff5740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.133+0000 7ff58ea00640 1 --2- 172.21.15.83:0/3880235324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff574003b80 0x7ff574003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff586000640 1 -- 172.21.15.83:0/3880235324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff59006dac0 msgr2=0x7ff5740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff586000640 1 --2- 172.21.15.83:0/3880235324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff59006dac0 0x7ff5740051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff586000640 1 -- 172.21.15.83:0/3880235324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff574003b80 msgr2=0x7ff574003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff586000640 1 --2- 172.21.15.83:0/3880235324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff574003b80 0x7ff574003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff586000640 1 -- 172.21.15.83:0/3880235324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff574003530 con 0x7ff574004810 2024-04-25T00:13:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff586000640 1 --2- 172.21.15.83:0/3880235324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff574004810 0x7ff574004c90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff578007fd0 tx=0x7ff57801ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbd5f7e6c9bb42c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578001520 con 0x7ff574004810 2024-04-25T00:13:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5780052f0 con 0x7ff574004810 2024-04-25T00:13:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff58e000640 1 -- 172.21.15.83:0/3880235324 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5780a8910 con 0x7ff574004810 2024-04-25T00:13:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff595600640 1 -- 172.21.15.83:0/3880235324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff574004810 msgr2=0x7ff574004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff595600640 1 --2- 172.21.15.83:0/3880235324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff574004810 0x7ff574004c90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff578007fd0 tx=0x7ff57801ecb0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff58f400640 1 -- 172.21.15.83:0/3880235324 reap_dead start 2024-04-25T00:13:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff595600640 1 -- 172.21.15.83:0/3880235324 shutdown_connections 2024-04-25T00:13:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff595600640 1 -- 172.21.15.83:0/3880235324 >> 172.21.15.83:0/3880235324 conn(0x7ff5900cfa60 msgr2=0x7ff5900cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff595600640 1 -- 172.21.15.83:0/3880235324 shutdown_connections 2024-04-25T00:13:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.137+0000 7ff595600640 1 -- 172.21.15.83:0/3880235324 wait complete. 2024-04-25T00:13:08.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff595600640 1 Processor -- start 2024-04-25T00:13:08.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff595600640 1 -- start start 2024-04-25T00:13:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff595600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff574004810 0x7ff59013be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff595600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff59015b120 0x7ff59013d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff595600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5901f1ce0 0x7ff5900b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff595600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff590070e90 con 0x7ff59015b120 2024-04-25T00:13:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff595600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff590070d10 con 0x7ff5901f1ce0 2024-04-25T00:13:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff595600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff590071010 con 0x7ff574004810 2024-04-25T00:13:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff586000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff59015b120 0x7ff59013d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff586000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff59015b120 0x7ff59013d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60658/0 (socket says 172.21.15.83:60658) 2024-04-25T00:13:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff586000640 1 -- 172.21.15.83:0/4019897420 learned_addr learned my addr 172.21.15.83:0/4019897420 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff58f400640 1 --2- 172.21.15.83:0/4019897420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5901f1ce0 0x7ff5900b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff58ea00640 1 --2- 172.21.15.83:0/4019897420 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff574004810 0x7ff59013be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:08.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff586000640 1 -- 172.21.15.83:0/4019897420 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff574004810 msgr2=0x7ff59013be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:08.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff586000640 1 --2- 172.21.15.83:0/4019897420 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff574004810 0x7ff59013be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff586000640 1 -- 172.21.15.83:0/4019897420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5901f1ce0 msgr2=0x7ff5900b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:08.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff586000640 1 --2- 172.21.15.83:0/4019897420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5901f1ce0 0x7ff5900b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff586000640 1 -- 172.21.15.83:0/4019897420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59013e030 con 0x7ff59015b120 2024-04-25T00:13:08.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff586000640 1 --2- 172.21.15.83:0/4019897420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff59015b120 0x7ff59013d6e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff578005040 tx=0x7ff574005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:08.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.141+0000 7ff587400640 1 -- 172.21.15.83:0/4019897420 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5780b5040 con 0x7ff59015b120 2024-04-25T00:13:08.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.145+0000 7ff587400640 1 -- 172.21.15.83:0/4019897420 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5780c0070 con 0x7ff59015b120 2024-04-25T00:13:08.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.145+0000 7ff595600640 1 -- 172.21.15.83:0/4019897420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff59013e310 con 0x7ff59015b120 2024-04-25T00:13:08.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.145+0000 7ff587400640 1 -- 172.21.15.83:0/4019897420 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5780054a0 con 0x7ff59015b120 2024-04-25T00:13:08.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.145+0000 7ff595600640 1 -- 172.21.15.83:0/4019897420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff59013dc20 con 0x7ff59015b120 2024-04-25T00:13:08.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.145+0000 7ff595600640 1 -- 172.21.15.83:0/4019897420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff590068230 con 0x7ff59015b120 2024-04-25T00:13:08.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.145+0000 7ff587400640 1 -- 172.21.15.83:0/4019897420 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff578003600 con 0x7ff59015b120 2024-04-25T00:13:08.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.145+0000 7ff587400640 1 --2- 172.21.15.83:0/4019897420 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff56c0678f0 0x7ff56c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:08.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.145+0000 7ff587400640 1 -- 172.21.15.83:0/4019897420 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7ff578123a40 con 0x7ff59015b120 2024-04-25T00:13:08.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.145+0000 7ff58ea00640 1 --2- 172.21.15.83:0/4019897420 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff56c0678f0 0x7ff56c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:08.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.145+0000 7ff58ea00640 1 --2- 172.21.15.83:0/4019897420 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff56c0678f0 0x7ff56c069db0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff580001060 tx=0x7ff580001e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:08.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.149+0000 7ff587400640 1 -- 172.21.15.83:0/4019897420 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff5780189c0 con 0x7ff59015b120 2024-04-25T00:13:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.337+0000 7ff595600640 1 -- 172.21.15.83:0/4019897420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff59015d940 con 0x7ff59015b120 2024-04-25T00:13:08.340 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-25T00:13:08.337+0000 7efeeee00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:13:08.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.341+0000 7ff587400640 1 -- 172.21.15.83:0/4019897420 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7ff5780f3e20 con 0x7ff59015b120 2024-04-25T00:13:08.342 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:13:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.341+0000 7ff595600640 1 -- 172.21.15.83:0/4019897420 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff56c0678f0 msgr2=0x7ff56c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.341+0000 7ff595600640 1 --2- 172.21.15.83:0/4019897420 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff56c0678f0 0x7ff56c069db0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff580001060 tx=0x7ff580001e50 comp rx=0 tx=0).stop 2024-04-25T00:13:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.341+0000 7ff595600640 1 -- 172.21.15.83:0/4019897420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff59015b120 msgr2=0x7ff59013d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.341+0000 7ff595600640 1 --2- 172.21.15.83:0/4019897420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff59015b120 0x7ff59013d6e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff578005040 tx=0x7ff574005840 comp rx=0 tx=0).stop 2024-04-25T00:13:08.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.341+0000 7ff595600640 1 -- 172.21.15.83:0/4019897420 shutdown_connections 2024-04-25T00:13:08.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.341+0000 7ff595600640 1 --2- 172.21.15.83:0/4019897420 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff56c0678f0 0x7ff56c069db0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.341+0000 7ff595600640 1 --2- 172.21.15.83:0/4019897420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5901f1ce0 0x7ff5900b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.341+0000 7ff595600640 1 --2- 172.21.15.83:0/4019897420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff59015b120 0x7ff59013d6e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.341+0000 7ff595600640 1 --2- 172.21.15.83:0/4019897420 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff574004810 0x7ff59013be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.341+0000 7ff595600640 1 -- 172.21.15.83:0/4019897420 >> 172.21.15.83:0/4019897420 conn(0x7ff5900cfa60 msgr2=0x7ff59015cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:08.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.341+0000 7ff595600640 1 -- 172.21.15.83:0/4019897420 shutdown_connections 2024-04-25T00:13:08.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.341+0000 7ff595600640 1 -- 172.21.15.83:0/4019897420 wait complete. 2024-04-25T00:13:08.351 INFO:tasks.ceph.mds.d.smithi086.stderr: -2257> 2024-04-25T00:12:54.624+0000 7f6f97a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:13:08.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:13:08.398 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2024-04-25T00:13:08.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.497+0000 7f99dca00640 1 Processor -- start 2024-04-25T00:13:08.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.497+0000 7f99dca00640 1 -- start start 2024-04-25T00:13:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99dca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99d8159810 0x7f99d80cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99dca00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f99d8160020 0x7f99d80ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99dca00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99d80ce000 con 0x7f99d806e950 2024-04-25T00:13:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99dca00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f99d806e210 con 0x7f99d8160020 2024-04-25T00:13:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99dca00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f99d806da00 con 0x7f99d8159810 2024-04-25T00:13:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99d5400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f99d806e950 0x7f99d815f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33346/0 (socket says 172.21.15.83:33346) 2024-04-25T00:13:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99d5400640 1 -- 172.21.15.83:0/4216049149 learned_addr learned my addr 172.21.15.83:0/4216049149 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99d6800640 1 --2- 172.21.15.83:0/4216049149 >> v2:172.21.15.83:3300/0 conn(0x7f99d8160020 0x7f99d80ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99d5e00640 1 --2- 172.21.15.83:0/4216049149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99d8159810 0x7f99d80cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99d6800640 1 -- 172.21.15.83:0/4216049149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99d8159810 msgr2=0x7f99d80cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99d6800640 1 --2- 172.21.15.83:0/4216049149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99d8159810 0x7f99d80cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99d6800640 1 -- 172.21.15.83:0/4216049149 >> v1:172.21.15.83:6789/0 conn(0x7f99d806e950 legacy=0x7f99d815f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99d6800640 1 -- 172.21.15.83:0/4216049149 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d806eec0 con 0x7f99d8160020 2024-04-25T00:13:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99d4a00640 1 -- 172.21.15.83:0/4216049149 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1220504037 0 0) 0x7f99d80ce000 con 0x7f99d806e950 2024-04-25T00:13:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.501+0000 7f99d5e00640 1 --2- 172.21.15.83:0/4216049149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99d8159810 0x7f99d80cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.505+0000 7f99d6800640 1 --2- 172.21.15.83:0/4216049149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d8160020 0x7f99d80ce4a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f99d806db80 tx=0x7f99d001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824f0c6d1e37afb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.505+0000 7f99d4a00640 1 -- 172.21.15.83:0/4216049149 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d00b2070 con 0x7f99d8160020 2024-04-25T00:13:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.505+0000 7f99d4a00640 1 -- 172.21.15.83:0/4216049149 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d00a5950 con 0x7f99d8160020 2024-04-25T00:13:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.505+0000 7f99d4a00640 1 -- 172.21.15.83:0/4216049149 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d00a5d20 con 0x7f99d8160020 2024-04-25T00:13:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.505+0000 7f99dca00640 1 -- 172.21.15.83:0/4216049149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d8160020 msgr2=0x7f99d80ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.505+0000 7f99dca00640 1 --2- 172.21.15.83:0/4216049149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d8160020 0x7f99d80ce4a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f99d806db80 tx=0x7f99d001c890 comp rx=0 tx=0).stop 2024-04-25T00:13:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.505+0000 7f99dca00640 1 -- 172.21.15.83:0/4216049149 shutdown_connections 2024-04-25T00:13:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.505+0000 7f99dca00640 1 --2- 172.21.15.83:0/4216049149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d8160020 0x7f99d80ce4a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.505+0000 7f99dca00640 1 --2- 172.21.15.83:0/4216049149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99d8159810 0x7f99d80cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.505+0000 7f99dca00640 1 -- 172.21.15.83:0/4216049149 >> 172.21.15.83:0/4216049149 conn(0x7f99d80b1830 msgr2=0x7f99d80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.505+0000 7f99dca00640 1 -- 172.21.15.83:0/4216049149 shutdown_connections 2024-04-25T00:13:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.505+0000 7f99dca00640 1 -- 172.21.15.83:0/4216049149 wait complete. 2024-04-25T00:13:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.505+0000 7f99dca00640 1 Processor -- start 2024-04-25T00:13:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.505+0000 7f99dca00640 1 -- start start 2024-04-25T00:13:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99dca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d806e950 0x7f99d81403b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99dca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99d8159810 0x7f99d815d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99dca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d8160020 0x7f99d815de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99dca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99d8159d70 con 0x7f99d8160020 2024-04-25T00:13:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99dca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99d8159bf0 con 0x7f99d806e950 2024-04-25T00:13:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99dca00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f99d8070da0 con 0x7f99d8159810 2024-04-25T00:13:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d8160020 0x7f99d815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d8160020 0x7f99d815de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60676/0 (socket says 172.21.15.83:60676) 2024-04-25T00:13:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99d6800640 1 -- 172.21.15.83:0/1965266324 learned_addr learned my addr 172.21.15.83:0/1965266324 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99d5e00640 1 --2- 172.21.15.83:0/1965266324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d806e950 0x7f99d81403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99d5400640 1 --2- 172.21.15.83:0/1965266324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99d8159810 0x7f99d815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99d6800640 1 -- 172.21.15.83:0/1965266324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99d8159810 msgr2=0x7f99d815d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99d6800640 1 --2- 172.21.15.83:0/1965266324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99d8159810 0x7f99d815d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99d6800640 1 -- 172.21.15.83:0/1965266324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d806e950 msgr2=0x7f99d81403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99d6800640 1 --2- 172.21.15.83:0/1965266324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d806e950 0x7f99d81403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99d6800640 1 -- 172.21.15.83:0/1965266324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d81f8ae0 con 0x7f99d8160020 2024-04-25T00:13:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99d6800640 1 --2- 172.21.15.83:0/1965266324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d8160020 0x7f99d815de60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f99d001af40 tx=0x7f99d0019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99ce000640 1 -- 172.21.15.83:0/1965266324 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d00b2050 con 0x7f99d8160020 2024-04-25T00:13:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99ce000640 1 -- 172.21.15.83:0/1965266324 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d00be070 con 0x7f99d8160020 2024-04-25T00:13:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99dca00640 1 -- 172.21.15.83:0/1965266324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d81f8dc0 con 0x7f99d8160020 2024-04-25T00:13:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99ce000640 1 -- 172.21.15.83:0/1965266324 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d00adaf0 con 0x7f99d8160020 2024-04-25T00:13:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99dca00640 1 -- 172.21.15.83:0/1965266324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d81f9380 con 0x7f99d8160020 2024-04-25T00:13:08.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99dca00640 1 -- 172.21.15.83:0/1965266324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d804f510 con 0x7f99d8160020 2024-04-25T00:13:08.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.509+0000 7f99ce000640 1 -- 172.21.15.83:0/1965266324 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f99d00adc90 con 0x7f99d8160020 2024-04-25T00:13:08.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.513+0000 7f99ce000640 1 --2- 172.21.15.83:0/1965266324 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99b80678c0 0x7f99b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:08.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.513+0000 7f99ce000640 1 -- 172.21.15.83:0/1965266324 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6187+0+0 (secure 0 0 0) 0x7f99d0121290 con 0x7f99d8160020 2024-04-25T00:13:08.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.513+0000 7f99d5e00640 1 --2- 172.21.15.83:0/1965266324 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99b80678c0 0x7f99b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:08.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.513+0000 7f99d5e00640 1 --2- 172.21.15.83:0/1965266324 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99b80678c0 0x7f99b8069d80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f99c80098b0 tx=0x7f99c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.517+0000 7f99ce000640 1 -- 172.21.15.83:0/1965266324 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f99d0016540 con 0x7f99d8160020 2024-04-25T00:13:08.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:08.709+0000 7f99dca00640 1 -- 172.21.15.83:0/1965266324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f99d815bd60 con 0x7f99d8160020 2024-04-25T00:13:09.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.345+0000 7f99ce000640 1 -- 172.21.15.83:0/1965266324 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f99d00ab340 con 0x7f99d8160020 2024-04-25T00:13:09.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.345+0000 7f99dca00640 1 -- 172.21.15.83:0/1965266324 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99b80678c0 msgr2=0x7f99b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:09.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.345+0000 7f99dca00640 1 --2- 172.21.15.83:0/1965266324 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99b80678c0 0x7f99b8069d80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f99c80098b0 tx=0x7f99c8008040 comp rx=0 tx=0).stop 2024-04-25T00:13:09.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.345+0000 7f99dca00640 1 -- 172.21.15.83:0/1965266324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d8160020 msgr2=0x7f99d815de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:09.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.345+0000 7f99dca00640 1 --2- 172.21.15.83:0/1965266324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d8160020 0x7f99d815de60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f99d001af40 tx=0x7f99d0019630 comp rx=0 tx=0).stop 2024-04-25T00:13:09.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.345+0000 7f99dca00640 1 -- 172.21.15.83:0/1965266324 shutdown_connections 2024-04-25T00:13:09.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.345+0000 7f99dca00640 1 --2- 172.21.15.83:0/1965266324 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99b80678c0 0x7f99b8069d80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:09.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.345+0000 7f99dca00640 1 --2- 172.21.15.83:0/1965266324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99d8159810 0x7f99d815d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:09.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.349+0000 7f99dca00640 1 --2- 172.21.15.83:0/1965266324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d8160020 0x7f99d815de60 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:09.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.349+0000 7f99dca00640 1 --2- 172.21.15.83:0/1965266324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d806e950 0x7f99d81403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:09.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.349+0000 7f99dca00640 1 -- 172.21.15.83:0/1965266324 >> 172.21.15.83:0/1965266324 conn(0x7f99d80b1830 msgr2=0x7f99d80710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:09.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.349+0000 7f99dca00640 1 -- 172.21.15.83:0/1965266324 shutdown_connections 2024-04-25T00:13:09.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.349+0000 7f99dca00640 1 -- 172.21.15.83:0/1965266324 wait complete. 2024-04-25T00:13:09.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:13:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.505+0000 7f4b63600640 1 Processor -- start 2024-04-25T00:13:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.505+0000 7f4b63600640 1 -- start start 2024-04-25T00:13:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b63600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4b5c1455d0 0x7f4b5c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b63600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b5c0d0450 0x7f4b5c0b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b63600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5c0ce610 con 0x7f4b5c0ce080 2024-04-25T00:13:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b63600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5c0ce8f0 con 0x7f4b5c1455d0 2024-04-25T00:13:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b63600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5c0cebd0 con 0x7f4b5c0d0450 2024-04-25T00:13:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b61600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b5c0d0450 0x7f4b5c0b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b61600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b5c0d0450 0x7f4b5c0b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:43698/0 (socket says 172.21.15.83:43698) 2024-04-25T00:13:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b5be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4b5c0ce080 0x7f4b5c0cfd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:43910/0 (socket says 172.21.15.83:43910) 2024-04-25T00:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b61600640 1 -- 172.21.15.83:0/2865655535 learned_addr learned my addr 172.21.15.83:0/2865655535 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b60c00640 1 --2- 172.21.15.83:0/2865655535 >> v2:172.21.15.86:3300/0 conn(0x7f4b5c1455d0 0x7f4b5c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b61600640 1 -- 172.21.15.83:0/2865655535 >> v2:172.21.15.86:3300/0 conn(0x7f4b5c1455d0 msgr2=0x7f4b5c06d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b5b400640 1 -- 172.21.15.83:0/2865655535 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3079065996 0 0) 0x7f4b5c0ce610 con 0x7f4b5c0ce080 2024-04-25T00:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b61600640 1 --2- 172.21.15.83:0/2865655535 >> v2:172.21.15.86:3300/0 conn(0x7f4b5c1455d0 0x7f4b5c06d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b61600640 1 -- 172.21.15.83:0/2865655535 >> v1:172.21.15.86:6789/0 conn(0x7f4b5c0ce080 legacy=0x7f4b5c0cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.509+0000 7f4b61600640 1 -- 172.21.15.83:0/2865655535 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c0ceeb0 con 0x7f4b5c0d0450 2024-04-25T00:13:09.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.513+0000 7f4b61600640 1 --2- 172.21.15.83:0/2865655535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b5c0d0450 0x7f4b5c0b2b30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009e50 tx=0x7f4b5401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=659f3c43dbdfa67e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:09.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.513+0000 7f4b5b400640 1 -- 172.21.15.83:0/2865655535 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b540a5860 con 0x7f4b5c0d0450 2024-04-25T00:13:09.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.513+0000 7f4b5b400640 1 -- 172.21.15.83:0/2865655535 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b540a5a00 con 0x7f4b5c0d0450 2024-04-25T00:13:09.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.513+0000 7f4b5b400640 1 -- 172.21.15.83:0/2865655535 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b540a5dd0 con 0x7f4b5c0d0450 2024-04-25T00:13:09.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.513+0000 7f4b63600640 1 -- 172.21.15.83:0/2865655535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b5c0d0450 msgr2=0x7f4b5c0b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:09.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.513+0000 7f4b63600640 1 --2- 172.21.15.83:0/2865655535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b5c0d0450 0x7f4b5c0b2b30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009e50 tx=0x7f4b5401ca30 comp rx=0 tx=0).stop 2024-04-25T00:13:09.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.513+0000 7f4b63600640 1 -- 172.21.15.83:0/2865655535 shutdown_connections 2024-04-25T00:13:09.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.513+0000 7f4b63600640 1 --2- 172.21.15.83:0/2865655535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b5c0d0450 0x7f4b5c0b2b30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:09.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.513+0000 7f4b63600640 1 --2- 172.21.15.83:0/2865655535 >> v2:172.21.15.86:3300/0 conn(0x7f4b5c1455d0 0x7f4b5c06d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:09.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.513+0000 7f4b63600640 1 -- 172.21.15.83:0/2865655535 >> 172.21.15.83:0/2865655535 conn(0x7f4b5c06b3b0 msgr2=0x7f4b5c06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:09.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.513+0000 7f4b63600640 1 -- 172.21.15.83:0/2865655535 shutdown_connections 2024-04-25T00:13:09.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.513+0000 7f4b63600640 1 -- 172.21.15.83:0/2865655535 wait complete. 2024-04-25T00:13:09.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.513+0000 7f4b63600640 1 Processor -- start 2024-04-25T00:13:09.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.513+0000 7f4b63600640 1 -- start start 2024-04-25T00:13:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b63600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5c0ce080 0x7f4b5c134a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b63600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b5c0d0450 0x7f4b5c154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b63600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c1455d0 0x7f4b5c156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b63600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c0b3a00 con 0x7f4b5c1455d0 2024-04-25T00:13:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b63600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c0b3880 con 0x7f4b5c0ce080 2024-04-25T00:13:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b63600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5c0b3b80 con 0x7f4b5c0d0450 2024-04-25T00:13:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b61600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c1455d0 0x7f4b5c156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b61600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c1455d0 0x7f4b5c156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60692/0 (socket says 172.21.15.83:60692) 2024-04-25T00:13:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b61600640 1 -- 172.21.15.83:0/1051355347 learned_addr learned my addr 172.21.15.83:0/1051355347 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b60c00640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5c0ce080 0x7f4b5c134a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b5be00640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b5c0d0450 0x7f4b5c154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b61600640 1 -- 172.21.15.83:0/1051355347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b5c0d0450 msgr2=0x7f4b5c154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b61600640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b5c0d0450 0x7f4b5c154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b61600640 1 -- 172.21.15.83:0/1051355347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5c0ce080 msgr2=0x7f4b5c134a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b61600640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5c0ce080 0x7f4b5c134a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b61600640 1 -- 172.21.15.83:0/1051355347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c134fc0 con 0x7f4b5c1455d0 2024-04-25T00:13:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b5be00640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b5c0d0450 0x7f4b5c154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b61600640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c1455d0 0x7f4b5c156520 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4b540af040 tx=0x7f4b540197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b58c00640 1 -- 172.21.15.83:0/1051355347 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b540b8670 con 0x7f4b5c1455d0 2024-04-25T00:13:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b58c00640 1 -- 172.21.15.83:0/1051355347 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b540be070 con 0x7f4b5c1455d0 2024-04-25T00:13:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b58c00640 1 -- 172.21.15.83:0/1051355347 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b540b8a50 con 0x7f4b5c1455d0 2024-04-25T00:13:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b63600640 1 -- 172.21.15.83:0/1051355347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c135260 con 0x7f4b5c1455d0 2024-04-25T00:13:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b63600640 1 -- 172.21.15.83:0/1051355347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c135820 con 0x7f4b5c1455d0 2024-04-25T00:13:09.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.517+0000 7f4b63600640 1 -- 172.21.15.83:0/1051355347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5c1362e0 con 0x7f4b5c1455d0 2024-04-25T00:13:09.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.521+0000 7f4b58c00640 1 -- 172.21.15.83:0/1051355347 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4b540b8bf0 con 0x7f4b5c1455d0 2024-04-25T00:13:09.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.521+0000 7f4b58c00640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b3c0678c0 0x7f4b3c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:09.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.521+0000 7f4b58c00640 1 -- 172.21.15.83:0/1051355347 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6187+0+0 (secure 0 0 0) 0x7f4b54120fd0 con 0x7f4b5c1455d0 2024-04-25T00:13:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.521+0000 7f4b60c00640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b3c0678c0 0x7f4b3c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.521+0000 7f4b60c00640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b3c0678c0 0x7f4b3c069d80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009ac0 tx=0x7f4b50009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.525+0000 7f4b58c00640 1 -- 172.21.15.83:0/1051355347 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4b5c1362e0 con 0x7f4b5c1455d0 2024-04-25T00:13:09.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:09.681+0000 7f4b63600640 1 -- 172.21.15.83:0/1051355347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4b5c0736f0 con 0x7f4b5c1455d0 2024-04-25T00:13:10.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.345+0000 7f4b58c00640 1 -- 172.21.15.83:0/1051355347 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) ==== 181+0+0 (secure 0 0 0) 0x7f4b54016620 con 0x7f4b5c1455d0 2024-04-25T00:13:10.350 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:13:10.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.353+0000 7f4b63600640 1 -- 172.21.15.83:0/1051355347 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b3c0678c0 msgr2=0x7f4b3c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:10.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.353+0000 7f4b63600640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b3c0678c0 0x7f4b3c069d80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009ac0 tx=0x7f4b50009190 comp rx=0 tx=0).stop 2024-04-25T00:13:10.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.353+0000 7f4b63600640 1 -- 172.21.15.83:0/1051355347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c1455d0 msgr2=0x7f4b5c156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.353+0000 7f4b63600640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c1455d0 0x7f4b5c156520 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4b540af040 tx=0x7f4b540197d0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.353+0000 7f4b63600640 1 -- 172.21.15.83:0/1051355347 shutdown_connections 2024-04-25T00:13:10.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.353+0000 7f4b63600640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b3c0678c0 0x7f4b3c069d80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.353+0000 7f4b63600640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b5c0d0450 0x7f4b5c154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.353+0000 7f4b63600640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c1455d0 0x7f4b5c156520 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.353+0000 7f4b63600640 1 --2- 172.21.15.83:0/1051355347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b5c0ce080 0x7f4b5c134a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.353+0000 7f4b63600640 1 -- 172.21.15.83:0/1051355347 >> 172.21.15.83:0/1051355347 conn(0x7f4b5c06b3b0 msgr2=0x7f4b5c0b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:10.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.353+0000 7f4b63600640 1 -- 172.21.15.83:0/1051355347 shutdown_connections 2024-04-25T00:13:10.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.353+0000 7f4b63600640 1 -- 172.21.15.83:0/1051355347 wait complete. 2024-04-25T00:13:10.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:13:10.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920f200640 1 Processor -- start 2024-04-25T00:13:10.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920f200640 1 -- start start 2024-04-25T00:13:10.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92081445e0 0x7f92080b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:10.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9208072120 0x7f920815bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:10.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920f200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f920815c1a0 0x7f92080cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:10.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920f200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f920815b7d0 con 0x7f920815c1a0 2024-04-25T00:13:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f920815b480 con 0x7f9208072120 2024-04-25T00:13:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f92081449c0 con 0x7f92081445e0 2024-04-25T00:13:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920d200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f920815c1a0 0x7f92080cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f91ff400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9208072120 0x7f920815bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920d200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f920815c1a0 0x7f92080cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60698/0 (socket says 172.21.15.83:60698) 2024-04-25T00:13:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920d200640 1 -- 172.21.15.83:0/1057164715 learned_addr learned my addr 172.21.15.83:0/1057164715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f9207e00640 1 --2- 172.21.15.83:0/1057164715 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92081445e0 0x7f92080b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920d200640 1 -- 172.21.15.83:0/1057164715 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92081445e0 msgr2=0x7f92080b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920d200640 1 --2- 172.21.15.83:0/1057164715 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92081445e0 0x7f92080b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920d200640 1 -- 172.21.15.83:0/1057164715 >> v2:172.21.15.86:3300/0 conn(0x7f9208072120 msgr2=0x7f920815bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920d200640 1 --2- 172.21.15.83:0/1057164715 >> v2:172.21.15.86:3300/0 conn(0x7f9208072120 0x7f920815bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f920d200640 1 -- 172.21.15.83:0/1057164715 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9208072910 con 0x7f920815c1a0 2024-04-25T00:13:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.521+0000 7f91ff400640 1 --2- 172.21.15.83:0/1057164715 >> v2:172.21.15.86:3300/0 conn(0x7f9208072120 0x7f920815bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:10.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.525+0000 7f920d200640 1 --2- 172.21.15.83:0/1057164715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f920815c1a0 0x7f92080cfd40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9208144bd0 tx=0x7f920001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd37fc88d884468 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:10.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.525+0000 7f9207400640 1 -- 172.21.15.83:0/1057164715 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92000b0070 con 0x7f920815c1a0 2024-04-25T00:13:10.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.525+0000 7f9207400640 1 -- 172.21.15.83:0/1057164715 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92000ab600 con 0x7f920815c1a0 2024-04-25T00:13:10.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.525+0000 7f9207400640 1 -- 172.21.15.83:0/1057164715 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92000ab9d0 con 0x7f920815c1a0 2024-04-25T00:13:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.525+0000 7f920f200640 1 -- 172.21.15.83:0/1057164715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f920815c1a0 msgr2=0x7f92080cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.525+0000 7f920f200640 1 --2- 172.21.15.83:0/1057164715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f920815c1a0 0x7f92080cfd40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9208144bd0 tx=0x7f920001c470 comp rx=0 tx=0).stop 2024-04-25T00:13:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.525+0000 7f920f200640 1 -- 172.21.15.83:0/1057164715 shutdown_connections 2024-04-25T00:13:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.525+0000 7f920f200640 1 --2- 172.21.15.83:0/1057164715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f920815c1a0 0x7f92080cfd40 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.525+0000 7f920f200640 1 --2- 172.21.15.83:0/1057164715 >> v2:172.21.15.86:3300/0 conn(0x7f9208072120 0x7f920815bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.525+0000 7f920f200640 1 --2- 172.21.15.83:0/1057164715 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92081445e0 0x7f92080b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.525+0000 7f920f200640 1 -- 172.21.15.83:0/1057164715 >> 172.21.15.83:0/1057164715 conn(0x7f920806b3b0 msgr2=0x7f920806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.525+0000 7f920f200640 1 -- 172.21.15.83:0/1057164715 shutdown_connections 2024-04-25T00:13:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.525+0000 7f920f200640 1 -- 172.21.15.83:0/1057164715 wait complete. 2024-04-25T00:13:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f920f200640 1 Processor -- start 2024-04-25T00:13:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f920f200640 1 -- start start 2024-04-25T00:13:10.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f920f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9208072120 0x7f920813a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f920f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92081445e0 0x7f92081584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f920f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f920813c290 0x7f92080cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f920f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9208072680 con 0x7f9208072120 2024-04-25T00:13:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f920f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9208072500 con 0x7f920813c290 2024-04-25T00:13:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f920f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f920806df30 con 0x7f92081445e0 2024-04-25T00:13:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f9207e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9208072120 0x7f920813a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f9207e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9208072120 0x7f920813a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60712/0 (socket says 172.21.15.83:60712) 2024-04-25T00:13:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f9207e00640 1 -- 172.21.15.83:0/1081407893 learned_addr learned my addr 172.21.15.83:0/1081407893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f91ff400640 1 --2- 172.21.15.83:0/1081407893 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92081445e0 0x7f92081584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f920d200640 1 --2- 172.21.15.83:0/1081407893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f920813c290 0x7f92080cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f9207e00640 1 -- 172.21.15.83:0/1081407893 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92081445e0 msgr2=0x7f92081584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f9207e00640 1 --2- 172.21.15.83:0/1081407893 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92081445e0 0x7f92081584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f9207e00640 1 -- 172.21.15.83:0/1081407893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f920813c290 msgr2=0x7f92080cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f9207e00640 1 --2- 172.21.15.83:0/1081407893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f920813c290 0x7f92080cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f9207e00640 1 -- 172.21.15.83:0/1081407893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920813ac90 con 0x7f9208072120 2024-04-25T00:13:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f9207e00640 1 --2- 172.21.15.83:0/1081407893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9208072120 0x7f920813a520 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f91f800efc0 tx=0x7f91f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f9204c00640 1 -- 172.21.15.83:0/1081407893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8018020 con 0x7f9208072120 2024-04-25T00:13:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f9204c00640 1 -- 172.21.15.83:0/1081407893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f8014480 con 0x7f9208072120 2024-04-25T00:13:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f9204c00640 1 -- 172.21.15.83:0/1081407893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8014870 con 0x7f9208072120 2024-04-25T00:13:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f920f200640 1 -- 172.21.15.83:0/1081407893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920813af70 con 0x7f9208072120 2024-04-25T00:13:10.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.529+0000 7f920f200640 1 -- 172.21.15.83:0/1081407893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920813b530 con 0x7f9208072120 2024-04-25T00:13:10.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.533+0000 7f920f200640 1 -- 172.21.15.83:0/1081407893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92080736f0 con 0x7f9208072120 2024-04-25T00:13:10.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.533+0000 7f9204c00640 1 -- 172.21.15.83:0/1081407893 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f91f801b020 con 0x7f9208072120 2024-04-25T00:13:10.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.533+0000 7f9204c00640 1 --2- 172.21.15.83:0/1081407893 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f91e40678c0 0x7f91e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:10.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.533+0000 7f9204c00640 1 -- 172.21.15.83:0/1081407893 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5749+0+0 (secure 0 0 0) 0x7f91f8086bb0 con 0x7f9208072120 2024-04-25T00:13:10.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.533+0000 7f91ff400640 1 --2- 172.21.15.83:0/1081407893 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f91e40678c0 0x7f91e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:10.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.533+0000 7f91ff400640 1 --2- 172.21.15.83:0/1081407893 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f91e40678c0 0x7f91e4069d80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009580 tx=0x7f91f0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:10.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.537+0000 7f9204c00640 1 -- 172.21.15.83:0/1081407893 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f91f8022020 con 0x7f9208072120 2024-04-25T00:13:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:10.701+0000 7f920f200640 1 -- 172.21.15.83:0/1081407893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f920804f5f0 con 0x7f9208072120 2024-04-25T00:13:11.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.353+0000 7f9204c00640 1 -- 172.21.15.83:0/1081407893 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) ==== 169+0+0 (secure 0 0 0) 0x7f91f8010040 con 0x7f9208072120 2024-04-25T00:13:11.355 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-25T00:13:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.357+0000 7f920f200640 1 -- 172.21.15.83:0/1081407893 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f91e40678c0 msgr2=0x7f91e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.357+0000 7f920f200640 1 --2- 172.21.15.83:0/1081407893 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f91e40678c0 0x7f91e4069d80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009580 tx=0x7f91f0009190 comp rx=0 tx=0).stop 2024-04-25T00:13:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.357+0000 7f920f200640 1 -- 172.21.15.83:0/1081407893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9208072120 msgr2=0x7f920813a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:11.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.357+0000 7f920f200640 1 --2- 172.21.15.83:0/1081407893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9208072120 0x7f920813a520 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f91f800efc0 tx=0x7f91f800c490 comp rx=0 tx=0).stop 2024-04-25T00:13:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.357+0000 7f920f200640 1 -- 172.21.15.83:0/1081407893 shutdown_connections 2024-04-25T00:13:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.357+0000 7f920f200640 1 --2- 172.21.15.83:0/1081407893 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f91e40678c0 0x7f91e4069d80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.357+0000 7f920f200640 1 --2- 172.21.15.83:0/1081407893 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92081445e0 0x7f92081584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.357+0000 7f920f200640 1 --2- 172.21.15.83:0/1081407893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f920813c290 0x7f92080cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.357+0000 7f920f200640 1 --2- 172.21.15.83:0/1081407893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9208072120 0x7f920813a520 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.357+0000 7f920f200640 1 -- 172.21.15.83:0/1081407893 >> 172.21.15.83:0/1081407893 conn(0x7f920806b3b0 msgr2=0x7f9208141600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:11.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.357+0000 7f920f200640 1 -- 172.21.15.83:0/1081407893 shutdown_connections 2024-04-25T00:13:11.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.357+0000 7f920f200640 1 -- 172.21.15.83:0/1081407893 wait complete. 2024-04-25T00:13:11.381 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:13:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb703200640 1 Processor -- start 2024-04-25T00:13:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb703200640 1 -- start start 2024-04-25T00:13:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb703200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb70415e790 0x7fb70415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb703200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7040b2da0 0x7fb7040b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb703200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7040719f0 con 0x7fb704155310 2024-04-25T00:13:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb703200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb70415f610 con 0x7fb70415e790 2024-04-25T00:13:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb703200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb704071cd0 con 0x7fb7040b2da0 2024-04-25T00:13:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb701400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb70415e790 0x7fb70415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb701400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb70415e790 0x7fb70415ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60726/0 (socket says 172.21.15.83:60726) 2024-04-25T00:13:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb701400640 1 -- 172.21.15.83:0/285069188 learned_addr learned my addr 172.21.15.83:0/285069188 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb702800640 1 --2- 172.21.15.83:0/285069188 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7040b2da0 0x7fb7040b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb701400640 1 -- 172.21.15.83:0/285069188 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7040b2da0 msgr2=0x7fb7040b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb701400640 1 --2- 172.21.15.83:0/285069188 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7040b2da0 0x7fb7040b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb701400640 1 -- 172.21.15.83:0/285069188 >> v1:172.21.15.86:6789/0 conn(0x7fb704155310 legacy=0x7fb70406cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.529+0000 7fb701400640 1 -- 172.21.15.83:0/285069188 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb704071fb0 con 0x7fb70415e790 2024-04-25T00:13:11.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.533+0000 7fb701400640 1 --2- 172.21.15.83:0/285069188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb70415e790 0x7fb70415ec40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009870 tx=0x7fb6f001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da28af25e17fa4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:11.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.533+0000 7fb700a00640 1 -- 172.21.15.83:0/285069188 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00b0070 con 0x7fb70415e790 2024-04-25T00:13:11.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.533+0000 7fb700a00640 1 -- 172.21.15.83:0/285069188 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f00ab600 con 0x7fb70415e790 2024-04-25T00:13:11.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.533+0000 7fb700a00640 1 -- 172.21.15.83:0/285069188 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00ab9d0 con 0x7fb70415e790 2024-04-25T00:13:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.533+0000 7fb703200640 1 -- 172.21.15.83:0/285069188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb70415e790 msgr2=0x7fb70415ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.533+0000 7fb703200640 1 --2- 172.21.15.83:0/285069188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb70415e790 0x7fb70415ec40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009870 tx=0x7fb6f001c4d0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.533+0000 7fb703200640 1 -- 172.21.15.83:0/285069188 shutdown_connections 2024-04-25T00:13:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.533+0000 7fb703200640 1 --2- 172.21.15.83:0/285069188 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7040b2da0 0x7fb7040b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.533+0000 7fb703200640 1 --2- 172.21.15.83:0/285069188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb70415e790 0x7fb70415ec40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.533+0000 7fb703200640 1 -- 172.21.15.83:0/285069188 >> 172.21.15.83:0/285069188 conn(0x7fb7040cf980 msgr2=0x7fb7040cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.533+0000 7fb703200640 1 -- 172.21.15.83:0/285069188 shutdown_connections 2024-04-25T00:13:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.533+0000 7fb703200640 1 -- 172.21.15.83:0/285069188 wait complete. 2024-04-25T00:13:11.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb703200640 1 Processor -- start 2024-04-25T00:13:11.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb703200640 1 -- start start 2024-04-25T00:13:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb703200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7040b2da0 0x7fb7041538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb703200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704155310 0x7fb704168240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb703200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb70415e790 0x7fb7040ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb703200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb704165b20 con 0x7fb7040b2da0 2024-04-25T00:13:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb703200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7040717d0 con 0x7fb704155310 2024-04-25T00:13:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb703200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb704165ca0 con 0x7fb70415e790 2024-04-25T00:13:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb701e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7040b2da0 0x7fb7041538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb701e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7040b2da0 0x7fb7041538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60728/0 (socket says 172.21.15.83:60728) 2024-04-25T00:13:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb701e00640 1 -- 172.21.15.83:0/2963099084 learned_addr learned my addr 172.21.15.83:0/2963099084 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb702800640 1 --2- 172.21.15.83:0/2963099084 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb70415e790 0x7fb7040ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb701400640 1 --2- 172.21.15.83:0/2963099084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704155310 0x7fb704168240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb701e00640 1 -- 172.21.15.83:0/2963099084 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb70415e790 msgr2=0x7fb7040ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb701e00640 1 --2- 172.21.15.83:0/2963099084 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb70415e790 0x7fb7040ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb701e00640 1 -- 172.21.15.83:0/2963099084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704155310 msgr2=0x7fb704168240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb701e00640 1 --2- 172.21.15.83:0/2963099084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704155310 0x7fb704168240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb701e00640 1 -- 172.21.15.83:0/2963099084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb704153e80 con 0x7fb7040b2da0 2024-04-25T00:13:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb701e00640 1 --2- 172.21.15.83:0/2963099084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7040b2da0 0x7fb7041538a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400c420 tx=0x7fb6f400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb6fa000640 1 -- 172.21.15.83:0/2963099084 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f40052d0 con 0x7fb7040b2da0 2024-04-25T00:13:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb6fa000640 1 -- 172.21.15.83:0/2963099084 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f4005470 con 0x7fb7040b2da0 2024-04-25T00:13:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb6fa000640 1 -- 172.21.15.83:0/2963099084 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4007260 con 0x7fb7040b2da0 2024-04-25T00:13:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb703200640 1 -- 172.21.15.83:0/2963099084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7041464a0 con 0x7fb7040b2da0 2024-04-25T00:13:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb703200640 1 -- 172.21.15.83:0/2963099084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb704146a60 con 0x7fb7040b2da0 2024-04-25T00:13:11.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.537+0000 7fb703200640 1 -- 172.21.15.83:0/2963099084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70404f510 con 0x7fb7040b2da0 2024-04-25T00:13:11.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.541+0000 7fb6fa000640 1 -- 172.21.15.83:0/2963099084 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb6f4007400 con 0x7fb7040b2da0 2024-04-25T00:13:11.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.541+0000 7fb6fa000640 1 --2- 172.21.15.83:0/2963099084 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6e40678c0 0x7fb6e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:11.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.541+0000 7fb6fa000640 1 -- 172.21.15.83:0/2963099084 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7fb6f4084ff0 con 0x7fb7040b2da0 2024-04-25T00:13:11.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.541+0000 7fb701400640 1 --2- 172.21.15.83:0/2963099084 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6e40678c0 0x7fb6e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:11.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.541+0000 7fb701400640 1 --2- 172.21.15.83:0/2963099084 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6e40678c0 0x7fb6e4069d80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00ad000 tx=0x7fb6f00a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.545+0000 7fb6fa000640 1 -- 172.21.15.83:0/2963099084 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb6f40555d0 con 0x7fb7040b2da0 2024-04-25T00:13:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.701+0000 7fb703200640 1 -- 172.21.15.83:0/2963099084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb7040d0740 con 0x7fb7040b2da0 2024-04-25T00:13:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.705+0000 7fb6fa000640 1 -- 172.21.15.83:0/2963099084 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7fb6f40598d0 con 0x7fb7040b2da0 2024-04-25T00:13:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.705+0000 7fb703200640 1 -- 172.21.15.83:0/2963099084 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6e40678c0 msgr2=0x7fb6e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.705+0000 7fb703200640 1 --2- 172.21.15.83:0/2963099084 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6e40678c0 0x7fb6e4069d80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00ad000 tx=0x7fb6f00a6000 comp rx=0 tx=0).stop 2024-04-25T00:13:11.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.705+0000 7fb703200640 1 -- 172.21.15.83:0/2963099084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7040b2da0 msgr2=0x7fb7041538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:11.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.705+0000 7fb703200640 1 --2- 172.21.15.83:0/2963099084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7040b2da0 0x7fb7041538a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400c420 tx=0x7fb6f400c8e0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.705+0000 7fb703200640 1 -- 172.21.15.83:0/2963099084 shutdown_connections 2024-04-25T00:13:11.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.705+0000 7fb703200640 1 --2- 172.21.15.83:0/2963099084 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6e40678c0 0x7fb6e4069d80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.705+0000 7fb703200640 1 --2- 172.21.15.83:0/2963099084 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb70415e790 0x7fb7040ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.705+0000 7fb703200640 1 --2- 172.21.15.83:0/2963099084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704155310 0x7fb704168240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.705+0000 7fb703200640 1 --2- 172.21.15.83:0/2963099084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7040b2da0 0x7fb7041538a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.705+0000 7fb703200640 1 -- 172.21.15.83:0/2963099084 >> 172.21.15.83:0/2963099084 conn(0x7fb7040cf980 msgr2=0x7fb70415b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:11.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.705+0000 7fb703200640 1 -- 172.21.15.83:0/2963099084 shutdown_connections 2024-04-25T00:13:11.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.709+0000 7fb703200640 1 -- 172.21.15.83:0/2963099084 wait complete. 2024-04-25T00:13:11.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted' 2024-04-25T00:13:11.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.861+0000 7f8379200640 1 Processor -- start 2024-04-25T00:13:11.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.861+0000 7f8379200640 1 -- start start 2024-04-25T00:13:11.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.861+0000 7f8379200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83741553e0 0x7f837406e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:11.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.861+0000 7f8379200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f837406ef80 con 0x7f83740b4a70 2024-04-25T00:13:11.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.861+0000 7f8379200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f837415b510 con 0x7f837406ea00 2024-04-25T00:13:11.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.861+0000 7f8379200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f837415b7f0 con 0x7f83741553e0 2024-04-25T00:13:11.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.861+0000 7f8371e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83741553e0 0x7f837406e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:11.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.861+0000 7f8371e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83741553e0 0x7f837406e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60740/0 (socket says 172.21.15.83:60740) 2024-04-25T00:13:11.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.861+0000 7f8372800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f83740b4a70 0x7f83740b2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33358/0 (socket says 172.21.15.83:33358) 2024-04-25T00:13:11.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.861+0000 7f8371e00640 1 -- 172.21.15.83:0/259797135 learned_addr learned my addr 172.21.15.83:0/259797135 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:11.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.861+0000 7f8371e00640 1 -- 172.21.15.83:0/259797135 >> v1:172.21.15.86:6789/0 conn(0x7f837406ea00 legacy=0x7f83740b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:11.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.861+0000 7f8371e00640 1 -- 172.21.15.83:0/259797135 >> v1:172.21.15.83:6789/0 conn(0x7f83740b4a70 legacy=0x7f83740b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:11.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.861+0000 7f8371e00640 1 -- 172.21.15.83:0/259797135 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837415bad0 con 0x7f83741553e0 2024-04-25T00:13:11.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.861+0000 7f8371400640 1 -- 172.21.15.83:0/259797135 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318388651 0 0) 0x7f837406ef80 con 0x7f83740b4a70 2024-04-25T00:13:11.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.865+0000 7f8371e00640 1 --2- 172.21.15.83:0/259797135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83741553e0 0x7f837406e4c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8364009e50 tx=0x7f836401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0fcb676fa920b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:11.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.865+0000 7f8371400640 1 -- 172.21.15.83:0/259797135 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83640a5770 con 0x7f83741553e0 2024-04-25T00:13:11.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.865+0000 7f8371400640 1 -- 172.21.15.83:0/259797135 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83640a5910 con 0x7f83741553e0 2024-04-25T00:13:11.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.865+0000 7f8371400640 1 -- 172.21.15.83:0/259797135 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83640a5ce0 con 0x7f83741553e0 2024-04-25T00:13:11.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.865+0000 7f8379200640 1 -- 172.21.15.83:0/259797135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83741553e0 msgr2=0x7f837406e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:11.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.865+0000 7f8379200640 1 --2- 172.21.15.83:0/259797135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83741553e0 0x7f837406e4c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8364009e50 tx=0x7f836401c9a0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.865+0000 7f8379200640 1 -- 172.21.15.83:0/259797135 shutdown_connections 2024-04-25T00:13:11.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.865+0000 7f8379200640 1 --2- 172.21.15.83:0/259797135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83741553e0 0x7f837406e4c0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.865+0000 7f8379200640 1 -- 172.21.15.83:0/259797135 >> 172.21.15.83:0/259797135 conn(0x7f83740b1830 msgr2=0x7f83740b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:11.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.865+0000 7f8379200640 1 -- 172.21.15.83:0/259797135 shutdown_connections 2024-04-25T00:13:11.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.865+0000 7f8379200640 1 -- 172.21.15.83:0/259797135 wait complete. 2024-04-25T00:13:11.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8379200640 1 Processor -- start 2024-04-25T00:13:11.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8379200640 1 -- start start 2024-04-25T00:13:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8379200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f837406ea00 0x7f837414d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8379200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83740b4a70 0x7f83741564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8379200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83741553e0 0x7f8374071be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8379200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8374168f20 con 0x7f83741553e0 2024-04-25T00:13:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8379200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8374168da0 con 0x7f83740b4a70 2024-04-25T00:13:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8379200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f83741690a0 con 0x7f837406ea00 2024-04-25T00:13:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8372800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83741553e0 0x7f8374071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8372800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83741553e0 0x7f8374071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60756/0 (socket says 172.21.15.83:60756) 2024-04-25T00:13:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8372800640 1 -- 172.21.15.83:0/303689025 learned_addr learned my addr 172.21.15.83:0/303689025 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8371e00640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f837406ea00 0x7f837414d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:11.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8369400640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83740b4a70 0x7f83741564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:11.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8372800640 1 -- 172.21.15.83:0/303689025 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f837406ea00 msgr2=0x7f837414d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:11.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8372800640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f837406ea00 0x7f837414d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8372800640 1 -- 172.21.15.83:0/303689025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83740b4a70 msgr2=0x7f83741564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:11.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8372800640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83740b4a70 0x7f83741564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:11.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8372800640 1 -- 172.21.15.83:0/303689025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83741521f0 con 0x7f83741553e0 2024-04-25T00:13:11.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8369400640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83740b4a70 0x7f83741564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:11.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8372800640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83741553e0 0x7f8374071be0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f836c00c3e0 tx=0x7f836c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:11.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f836aa00640 1 -- 172.21.15.83:0/303689025 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0074e0 con 0x7f83741553e0 2024-04-25T00:13:11.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f836aa00640 1 -- 172.21.15.83:0/303689025 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f836c0047f0 con 0x7f83741553e0 2024-04-25T00:13:11.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f836aa00640 1 -- 172.21.15.83:0/303689025 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c007680 con 0x7f83741553e0 2024-04-25T00:13:11.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8379200640 1 -- 172.21.15.83:0/303689025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837414dbd0 con 0x7f83741553e0 2024-04-25T00:13:11.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.869+0000 7f8379200640 1 -- 172.21.15.83:0/303689025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8374157100 con 0x7f83741553e0 2024-04-25T00:13:11.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.873+0000 7f8379200640 1 -- 172.21.15.83:0/303689025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83741579f0 con 0x7f83741553e0 2024-04-25T00:13:11.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.873+0000 7f836aa00640 1 -- 172.21.15.83:0/303689025 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f836c011070 con 0x7f83741553e0 2024-04-25T00:13:11.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.873+0000 7f836aa00640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f83500678f0 0x7f8350069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:11.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.873+0000 7f836aa00640 1 -- 172.21.15.83:0/303689025 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f836c085630 con 0x7f83741553e0 2024-04-25T00:13:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.873+0000 7f8371e00640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f83500678f0 0x7f8350069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.873+0000 7f8371e00640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f83500678f0 0x7f8350069db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f83640ae000 tx=0x7f83640a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:11.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:11.877+0000 7f836aa00640 1 -- 172.21.15.83:0/303689025 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f836c0562d0 con 0x7f83741553e0 2024-04-25T00:13:12.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.085+0000 7f8379200640 1 -- 172.21.15.83:0/303689025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]} v 0) -- 0x7f83740737f0 con 0x7f83741553e0 2024-04-25T00:13:12.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.353+0000 7f836aa00640 1 -- 172.21.15.83:0/303689025 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]}]=0 v148) ==== 153+0+0 (secure 0 0 0) 0x7f836c055bd0 con 0x7f83741553e0 2024-04-25T00:13:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.357+0000 7f8379200640 1 -- 172.21.15.83:0/303689025 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f83500678f0 msgr2=0x7f8350069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.357+0000 7f8379200640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f83500678f0 0x7f8350069db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f83640ae000 tx=0x7f83640a7070 comp rx=0 tx=0).stop 2024-04-25T00:13:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.357+0000 7f8379200640 1 -- 172.21.15.83:0/303689025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83741553e0 msgr2=0x7f8374071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.357+0000 7f8379200640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83741553e0 0x7f8374071be0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f836c00c3e0 tx=0x7f836c00c7e0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.357+0000 7f8379200640 1 -- 172.21.15.83:0/303689025 shutdown_connections 2024-04-25T00:13:12.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.357+0000 7f8379200640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f83500678f0 0x7f8350069db0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.357+0000 7f8379200640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83741553e0 0x7f8374071be0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.357+0000 7f8379200640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83740b4a70 0x7f83741564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.357+0000 7f8379200640 1 --2- 172.21.15.83:0/303689025 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f837406ea00 0x7f837414d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.357+0000 7f8379200640 1 -- 172.21.15.83:0/303689025 >> 172.21.15.83:0/303689025 conn(0x7f83740b1830 msgr2=0x7f83741544f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:12.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.357+0000 7f8379200640 1 -- 172.21.15.83:0/303689025 shutdown_connections 2024-04-25T00:13:12.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.357+0000 7f8379200640 1 -- 172.21.15.83:0/303689025 wait complete. 2024-04-25T00:13:12.379 INFO:tasks.cephfs_test_runner:Check that mds evicts blocklisted client ... skipped 'Requires FUSE client to use mds_cluster.is_addr_blocklisted()' 2024-04-25T00:13:12.379 INFO:tasks.cephfs_test_runner:Starting test: test_session_reject (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-04-25T00:13:12.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:12.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.513+0000 7f7cbf400640 1 Processor -- start 2024-04-25T00:13:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.513+0000 7f7cbf400640 1 -- start start 2024-04-25T00:13:12.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.513+0000 7f7cbf400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7cb81553e0 0x7f7cb806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:12.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.513+0000 7f7cbf400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cb806ef80 con 0x7f7cb806ea00 2024-04-25T00:13:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.513+0000 7f7cbf400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cb815b510 con 0x7f7cb80b4a70 2024-04-25T00:13:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.513+0000 7f7cbf400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7cb815b7f0 con 0x7f7cb81553e0 2024-04-25T00:13:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.513+0000 7f7cbca00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7cb81553e0 0x7f7cb806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.513+0000 7f7cbca00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7cb81553e0 0x7f7cb806e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60760/0 (socket says 172.21.15.83:60760) 2024-04-25T00:13:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.513+0000 7f7cb7e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7cb806ea00 0x7f7cb80b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33374/0 (socket says 172.21.15.83:33374) 2024-04-25T00:13:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.513+0000 7f7cbca00640 1 -- 172.21.15.83:0/17330838 learned_addr learned my addr 172.21.15.83:0/17330838 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cbca00640 1 -- 172.21.15.83:0/17330838 >> v1:172.21.15.86:6789/0 conn(0x7f7cb80b4a70 legacy=0x7f7cb80b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cbca00640 1 -- 172.21.15.83:0/17330838 >> v1:172.21.15.83:6789/0 conn(0x7f7cb806ea00 legacy=0x7f7cb80b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cbca00640 1 -- 172.21.15.83:0/17330838 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb815bad0 con 0x7f7cb81553e0 2024-04-25T00:13:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cb7400640 1 -- 172.21.15.83:0/17330838 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3881559057 0 0) 0x7f7cb806ef80 con 0x7f7cb806ea00 2024-04-25T00:13:12.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cbca00640 1 --2- 172.21.15.83:0/17330838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb81553e0 0x7f7cb806e4c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009e50 tx=0x7f7ca801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d8d2ab707b608e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cb7400640 1 -- 172.21.15.83:0/17330838 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80a5770 con 0x7f7cb81553e0 2024-04-25T00:13:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cb7400640 1 -- 172.21.15.83:0/17330838 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca80a5910 con 0x7f7cb81553e0 2024-04-25T00:13:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cb7400640 1 -- 172.21.15.83:0/17330838 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80a5ce0 con 0x7f7cb81553e0 2024-04-25T00:13:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cbf400640 1 -- 172.21.15.83:0/17330838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb81553e0 msgr2=0x7f7cb806e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cbf400640 1 --2- 172.21.15.83:0/17330838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb81553e0 0x7f7cb806e4c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009e50 tx=0x7f7ca801c9a0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cbf400640 1 -- 172.21.15.83:0/17330838 shutdown_connections 2024-04-25T00:13:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cbf400640 1 --2- 172.21.15.83:0/17330838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb81553e0 0x7f7cb806e4c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cbf400640 1 -- 172.21.15.83:0/17330838 >> 172.21.15.83:0/17330838 conn(0x7f7cb80b1830 msgr2=0x7f7cb80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:12.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cbf400640 1 -- 172.21.15.83:0/17330838 shutdown_connections 2024-04-25T00:13:12.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.517+0000 7f7cbf400640 1 -- 172.21.15.83:0/17330838 wait complete. 2024-04-25T00:13:12.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cbf400640 1 Processor -- start 2024-04-25T00:13:12.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cbf400640 1 -- start start 2024-04-25T00:13:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cbf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cb806ea00 0x7f7cb814abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cbf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb80b4a70 0x7f7cb814b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cbf400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7cb81553e0 0x7f7cb8141e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cbf400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb806d530 con 0x7f7cb80b4a70 2024-04-25T00:13:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cbf400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb806d3b0 con 0x7f7cb806ea00 2024-04-25T00:13:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cbf400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb806d6b0 con 0x7f7cb81553e0 2024-04-25T00:13:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cb7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb80b4a70 0x7f7cb814b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cb7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb80b4a70 0x7f7cb814b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60768/0 (socket says 172.21.15.83:60768) 2024-04-25T00:13:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cb7e00640 1 -- 172.21.15.83:0/763699886 learned_addr learned my addr 172.21.15.83:0/763699886 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cbca00640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cb806ea00 0x7f7cb814abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cbd400640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7cb81553e0 0x7f7cb8141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cb7e00640 1 -- 172.21.15.83:0/763699886 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7cb81553e0 msgr2=0x7f7cb8141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cb7e00640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7cb81553e0 0x7f7cb8141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cb7e00640 1 -- 172.21.15.83:0/763699886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cb806ea00 msgr2=0x7f7cb814abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cb7e00640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cb806ea00 0x7f7cb814abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cb7e00640 1 -- 172.21.15.83:0/763699886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb8159400 con 0x7f7cb80b4a70 2024-04-25T00:13:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cbd400640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7cb81553e0 0x7f7cb8141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cb7e00640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb80b4a70 0x7f7cb814b0f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400c4d0 tx=0x7f7ca400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cb4c00640 1 -- 172.21.15.83:0/763699886 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4018070 con 0x7f7cb80b4a70 2024-04-25T00:13:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cb4c00640 1 -- 172.21.15.83:0/763699886 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca40074e0 con 0x7f7cb80b4a70 2024-04-25T00:13:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cb4c00640 1 -- 172.21.15.83:0/763699886 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4004a30 con 0x7f7cb80b4a70 2024-04-25T00:13:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.521+0000 7f7cbf400640 1 -- 172.21.15.83:0/763699886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb81596e0 con 0x7f7cb80b4a70 2024-04-25T00:13:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.525+0000 7f7cbf400640 1 -- 172.21.15.83:0/763699886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb8159ca0 con 0x7f7cb80b4a70 2024-04-25T00:13:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.525+0000 7f7cbf400640 1 -- 172.21.15.83:0/763699886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb813dda0 con 0x7f7cb80b4a70 2024-04-25T00:13:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.525+0000 7f7cb4c00640 1 -- 172.21.15.83:0/763699886 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7ca4004bd0 con 0x7f7cb80b4a70 2024-04-25T00:13:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.525+0000 7f7cb4c00640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c980678c0 0x7f7c98069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.525+0000 7f7cb4c00640 1 -- 172.21.15.83:0/763699886 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f7ca4084d80 con 0x7f7cb80b4a70 2024-04-25T00:13:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.525+0000 7f7cbca00640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c980678c0 0x7f7c98069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.525+0000 7f7cbca00640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c980678c0 0x7f7c98069d80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80ae000 tx=0x7f7ca80a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:12.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.529+0000 7f7cb4c00640 1 -- 172.21.15.83:0/763699886 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7cb813dda0 con 0x7f7cb80b4a70 2024-04-25T00:13:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.689+0000 7f7cbf400640 1 -- 172.21.15.83:0/763699886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7cb80737f0 con 0x7f7cb80b4a70 2024-04-25T00:13:12.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.689+0000 7f7cb4c00640 1 -- 172.21.15.83:0/763699886 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+13209 (secure 0 0 0) 0x7f7ca4055470 con 0x7f7cb80b4a70 2024-04-25T00:13:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:12.691 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":52,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:11.351175+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6834/1690820142":"2024-04-26T00:13:08.341131+0000","172.21.15.86:6832/1690820142":"2024-04-26T00:13:08.341131+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.689+0000 7f7cbf400640 1 -- 172.21.15.83:0/763699886 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c980678c0 msgr2=0x7f7c98069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.689+0000 7f7cbf400640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c980678c0 0x7f7c98069d80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80ae000 tx=0x7f7ca80a7070 comp rx=0 tx=0).stop 2024-04-25T00:13:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.689+0000 7f7cbf400640 1 -- 172.21.15.83:0/763699886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb80b4a70 msgr2=0x7f7cb814b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.689+0000 7f7cbf400640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb80b4a70 0x7f7cb814b0f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400c4d0 tx=0x7f7ca400c990 comp rx=0 tx=0).stop 2024-04-25T00:13:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.689+0000 7f7cbf400640 1 -- 172.21.15.83:0/763699886 shutdown_connections 2024-04-25T00:13:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.689+0000 7f7cbf400640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c980678c0 0x7f7c98069d80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.689+0000 7f7cbf400640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7cb81553e0 0x7f7cb8141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.689+0000 7f7cbf400640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb80b4a70 0x7f7cb814b0f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.689+0000 7f7cbf400640 1 --2- 172.21.15.83:0/763699886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cb806ea00 0x7f7cb814abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.689+0000 7f7cbf400640 1 -- 172.21.15.83:0/763699886 >> 172.21.15.83:0/763699886 conn(0x7f7cb80b1830 msgr2=0x7f7cb80cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.689+0000 7f7cbf400640 1 -- 172.21.15.83:0/763699886 shutdown_connections 2024-04-25T00:13:12.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.689+0000 7f7cbf400640 1 -- 172.21.15.83:0/763699886 wait complete. 2024-04-25T00:13:12.712 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:13:12.712 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:13:12.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1d7600640 1 Processor -- start 2024-04-25T00:13:12.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1d7600640 1 -- start start 2024-04-25T00:13:12.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1d7600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd1d01555c0 0x7fd1d015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:12.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1d7600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1d006e4c0 0x7fd1d006e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:12.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1d7600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1d015c560 con 0x7fd1d01551e0 2024-04-25T00:13:12.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1d7600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d0072460 con 0x7fd1d01555c0 2024-04-25T00:13:12.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1d7600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d015c7d0 con 0x7fd1d006e4c0 2024-04-25T00:13:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1cfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd1d01555c0 0x7fd1d015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1d4c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd1d01551e0 0x7fd1d0071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:43948/0 (socket says 172.21.15.83:43948) 2024-04-25T00:13:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1d4c00640 1 -- 172.21.15.83:0/1393151171 learned_addr learned my addr 172.21.15.83:0/1393151171 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1d5600640 1 --2- 172.21.15.83:0/1393151171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1d006e4c0 0x7fd1d006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1cf400640 1 -- 172.21.15.83:0/1393151171 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2172213908 0 0) 0x7fd1d015c560 con 0x7fd1d01551e0 2024-04-25T00:13:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1cf400640 1 -- 172.21.15.83:0/1393151171 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1b8003530 con 0x7fd1d01551e0 2024-04-25T00:13:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1cfe00640 1 -- 172.21.15.83:0/1393151171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1d006e4c0 msgr2=0x7fd1d006e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1cfe00640 1 --2- 172.21.15.83:0/1393151171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1d006e4c0 0x7fd1d006e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1cfe00640 1 -- 172.21.15.83:0/1393151171 >> v1:172.21.15.86:6789/0 conn(0x7fd1d01551e0 legacy=0x7fd1d0071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1cfe00640 1 -- 172.21.15.83:0/1393151171 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d015cab0 con 0x7fd1d01555c0 2024-04-25T00:13:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.849+0000 7fd1d5600640 1 --2- 172.21.15.83:0/1393151171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1d006e4c0 0x7fd1d006e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:12.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.853+0000 7fd1cfe00640 1 --2- 172.21.15.83:0/1393151171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1d01555c0 0x7fd1d015bea0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0097f0 tx=0x7fd1bc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c94f0c8c2448a373 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:12.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.853+0000 7fd1cf400640 1 -- 172.21.15.83:0/1393151171 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0b0070 con 0x7fd1d01555c0 2024-04-25T00:13:12.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.853+0000 7fd1cf400640 1 -- 172.21.15.83:0/1393151171 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1bc0ab600 con 0x7fd1d01555c0 2024-04-25T00:13:12.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.853+0000 7fd1cf400640 1 -- 172.21.15.83:0/1393151171 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0ab9d0 con 0x7fd1d01555c0 2024-04-25T00:13:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.853+0000 7fd1d7600640 1 -- 172.21.15.83:0/1393151171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1d01555c0 msgr2=0x7fd1d015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.853+0000 7fd1d7600640 1 --2- 172.21.15.83:0/1393151171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1d01555c0 0x7fd1d015bea0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0097f0 tx=0x7fd1bc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.853+0000 7fd1d7600640 1 -- 172.21.15.83:0/1393151171 shutdown_connections 2024-04-25T00:13:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.853+0000 7fd1d7600640 1 --2- 172.21.15.83:0/1393151171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1d006e4c0 0x7fd1d006e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.853+0000 7fd1d7600640 1 --2- 172.21.15.83:0/1393151171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1d01555c0 0x7fd1d015bea0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.853+0000 7fd1d7600640 1 -- 172.21.15.83:0/1393151171 >> 172.21.15.83:0/1393151171 conn(0x7fd1d00b1830 msgr2=0x7fd1d00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.853+0000 7fd1d7600640 1 -- 172.21.15.83:0/1393151171 shutdown_connections 2024-04-25T00:13:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.853+0000 7fd1d7600640 1 -- 172.21.15.83:0/1393151171 wait complete. 2024-04-25T00:13:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d7600640 1 Processor -- start 2024-04-25T00:13:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d7600640 1 -- start start 2024-04-25T00:13:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d7600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d006e4c0 0x7fd1d00b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d7600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1d01551e0 0x7fd1d0145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d7600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1d01555c0 0x7fd1d01461f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d7600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d015dbb0 con 0x7fd1d006e4c0 2024-04-25T00:13:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d7600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d015da30 con 0x7fd1d01551e0 2024-04-25T00:13:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d7600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d015dd30 con 0x7fd1d01555c0 2024-04-25T00:13:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d006e4c0 0x7fd1d00b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d006e4c0 0x7fd1d00b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60772/0 (socket says 172.21.15.83:60772) 2024-04-25T00:13:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d4c00640 1 -- 172.21.15.83:0/1876448906 learned_addr learned my addr 172.21.15.83:0/1876448906 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1cfe00640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1d01551e0 0x7fd1d0145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d5600640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1d01555c0 0x7fd1d01461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d4c00640 1 -- 172.21.15.83:0/1876448906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1d01555c0 msgr2=0x7fd1d01461f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d4c00640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1d01555c0 0x7fd1d01461f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d4c00640 1 -- 172.21.15.83:0/1876448906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1d01551e0 msgr2=0x7fd1d0145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d4c00640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1d01551e0 0x7fd1d0145cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d4c00640 1 -- 172.21.15.83:0/1876448906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d00b30f0 con 0x7fd1d006e4c0 2024-04-25T00:13:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d5600640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1d01555c0 0x7fd1d01461f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d4c00640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d006e4c0 0x7fd1d00b4190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400c450 tx=0x7fd1c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1ccc00640 1 -- 172.21.15.83:0/1876448906 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4017070 con 0x7fd1d006e4c0 2024-04-25T00:13:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1ccc00640 1 -- 172.21.15.83:0/1876448906 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c4005cf0 con 0x7fd1d006e4c0 2024-04-25T00:13:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1ccc00640 1 -- 172.21.15.83:0/1876448906 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4007290 con 0x7fd1d006e4c0 2024-04-25T00:13:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d7600640 1 -- 172.21.15.83:0/1876448906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d00b33d0 con 0x7fd1d006e4c0 2024-04-25T00:13:12.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d7600640 1 -- 172.21.15.83:0/1876448906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d00b3990 con 0x7fd1d006e4c0 2024-04-25T00:13:12.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.857+0000 7fd1d7600640 1 -- 172.21.15.83:0/1876448906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d004f510 con 0x7fd1d006e4c0 2024-04-25T00:13:12.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.861+0000 7fd1ccc00640 1 -- 172.21.15.83:0/1876448906 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd1c401a070 con 0x7fd1d006e4c0 2024-04-25T00:13:12.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.861+0000 7fd1ccc00640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd1b00678c0 0x7fd1b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:12.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.861+0000 7fd1ccc00640 1 -- 172.21.15.83:0/1876448906 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7fd1c4084ee0 con 0x7fd1d006e4c0 2024-04-25T00:13:12.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.861+0000 7fd1cfe00640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd1b00678c0 0x7fd1b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:12.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.861+0000 7fd1cfe00640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd1b00678c0 0x7fd1b0069d80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0ad000 tx=0x7fd1bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:12.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:12.865+0000 7fd1ccc00640 1 -- 172.21.15.83:0/1876448906 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd1c4055b80 con 0x7fd1d006e4c0 2024-04-25T00:13:13.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.017+0000 7fd1d7600640 1 -- 172.21.15.83:0/1876448906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd1d006ce30 con 0x7fd1d006e4c0 2024-04-25T00:13:13.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.017+0000 7fd1ccc00640 1 -- 172.21.15.83:0/1876448906 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7fd1c4055480 con 0x7fd1d006e4c0 2024-04-25T00:13:13.021 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:13:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.021+0000 7fd1d7600640 1 -- 172.21.15.83:0/1876448906 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd1b00678c0 msgr2=0x7fd1b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.021+0000 7fd1d7600640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd1b00678c0 0x7fd1b0069d80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0ad000 tx=0x7fd1bc008040 comp rx=0 tx=0).stop 2024-04-25T00:13:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.021+0000 7fd1d7600640 1 -- 172.21.15.83:0/1876448906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d006e4c0 msgr2=0x7fd1d00b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.021+0000 7fd1d7600640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d006e4c0 0x7fd1d00b4190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400c450 tx=0x7fd1c400c910 comp rx=0 tx=0).stop 2024-04-25T00:13:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.021+0000 7fd1d7600640 1 -- 172.21.15.83:0/1876448906 shutdown_connections 2024-04-25T00:13:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.021+0000 7fd1d7600640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd1b00678c0 0x7fd1b0069d80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.021+0000 7fd1d7600640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1d01555c0 0x7fd1d01461f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.021+0000 7fd1d7600640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1d01551e0 0x7fd1d0145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.021+0000 7fd1d7600640 1 --2- 172.21.15.83:0/1876448906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d006e4c0 0x7fd1d00b4190 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.021+0000 7fd1d7600640 1 -- 172.21.15.83:0/1876448906 >> 172.21.15.83:0/1876448906 conn(0x7fd1d00b1830 msgr2=0x7fd1d00afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.021+0000 7fd1d7600640 1 -- 172.21.15.83:0/1876448906 shutdown_connections 2024-04-25T00:13:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.021+0000 7fd1d7600640 1 -- 172.21.15.83:0/1876448906 wait complete. 2024-04-25T00:13:13.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject' 2024-04-25T00:13:13.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.173+0000 7fab79a00640 1 Processor -- start 2024-04-25T00:13:13.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.173+0000 7fab79a00640 1 -- start start 2024-04-25T00:13:13.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.173+0000 7fab79a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab74164cb0 0x7fab741670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:13.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.173+0000 7fab79a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab740ccc60 con 0x7fab740cc510 2024-04-25T00:13:13.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.173+0000 7fab79a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab7406dba0 con 0x7fab741599f0 2024-04-25T00:13:13.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.173+0000 7fab79a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fab74159dd0 con 0x7fab74164cb0 2024-04-25T00:13:13.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.173+0000 7fab6a000640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fab740cc510 0x7fab7415ca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33378/0 (socket says 172.21.15.83:33378) 2024-04-25T00:13:13.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.173+0000 7fab6a000640 1 -- 172.21.15.83:0/789270532 learned_addr learned my addr 172.21.15.83:0/789270532 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:13.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.173+0000 7fab73400640 1 --2- 172.21.15.83:0/789270532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab74164cb0 0x7fab741670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:13.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.177+0000 7fab72000640 1 -- 172.21.15.83:0/789270532 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2993837128 0 0) 0x7fab5c001120 con 0x7fab740cc510 2024-04-25T00:13:13.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.177+0000 7fab72000640 1 -- 172.21.15.83:0/789270532 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab58003530 con 0x7fab740cc510 2024-04-25T00:13:13.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.177+0000 7fab72000640 1 -- 172.21.15.83:0/789270532 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1151778514 0 0) 0x7fab7406dba0 con 0x7fab741599f0 2024-04-25T00:13:13.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.177+0000 7fab72000640 1 -- 172.21.15.83:0/789270532 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab5c001120 con 0x7fab741599f0 2024-04-25T00:13:13.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.177+0000 7fab73400640 1 -- 172.21.15.83:0/789270532 >> v1:172.21.15.86:6789/0 conn(0x7fab741599f0 legacy=0x7fab7415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:13.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.177+0000 7fab73400640 1 -- 172.21.15.83:0/789270532 >> v1:172.21.15.83:6789/0 conn(0x7fab740cc510 legacy=0x7fab7415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:13.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.177+0000 7fab73400640 1 -- 172.21.15.83:0/789270532 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7406de80 con 0x7fab74164cb0 2024-04-25T00:13:13.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.177+0000 7fab73400640 1 --2- 172.21.15.83:0/789270532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab74164cb0 0x7fab741670a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009df0 tx=0x7fab6c01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37d2209a296d38b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:13.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.177+0000 7fab72000640 1 -- 172.21.15.83:0/789270532 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0b2070 con 0x7fab74164cb0 2024-04-25T00:13:13.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.177+0000 7fab72000640 1 -- 172.21.15.83:0/789270532 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6c0a58d0 con 0x7fab74164cb0 2024-04-25T00:13:13.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.177+0000 7fab72000640 1 -- 172.21.15.83:0/789270532 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0a5ca0 con 0x7fab74164cb0 2024-04-25T00:13:13.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 -- 172.21.15.83:0/789270532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab74164cb0 msgr2=0x7fab741670a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:13.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 --2- 172.21.15.83:0/789270532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab74164cb0 0x7fab741670a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009df0 tx=0x7fab6c01c900 comp rx=0 tx=0).stop 2024-04-25T00:13:13.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 -- 172.21.15.83:0/789270532 shutdown_connections 2024-04-25T00:13:13.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 --2- 172.21.15.83:0/789270532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab74164cb0 0x7fab741670a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:13.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 -- 172.21.15.83:0/789270532 >> 172.21.15.83:0/789270532 conn(0x7fab740b1830 msgr2=0x7fab740b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:13.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 -- 172.21.15.83:0/789270532 shutdown_connections 2024-04-25T00:13:13.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 -- 172.21.15.83:0/789270532 wait complete. 2024-04-25T00:13:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 Processor -- start 2024-04-25T00:13:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 -- start start 2024-04-25T00:13:13.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab740cc510 0x7fab7414f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:13.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab741599f0 0x7fab74150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:13.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab74164cb0 0x7fab7415e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:13.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab740ce240 con 0x7fab74164cb0 2024-04-25T00:13:13.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab740ce0c0 con 0x7fab740cc510 2024-04-25T00:13:13.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab79a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fab740cf6f0 con 0x7fab741599f0 2024-04-25T00:13:13.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab73400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab74164cb0 0x7fab7415e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:13.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab73400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab74164cb0 0x7fab7415e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60784/0 (socket says 172.21.15.83:60784) 2024-04-25T00:13:13.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab73400640 1 -- 172.21.15.83:0/1879692366 learned_addr learned my addr 172.21.15.83:0/1879692366 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:13.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab72a00640 1 --2- 172.21.15.83:0/1879692366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab740cc510 0x7fab7414f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:13.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab6a000640 1 --2- 172.21.15.83:0/1879692366 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab741599f0 0x7fab74150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:13.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab73400640 1 -- 172.21.15.83:0/1879692366 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab741599f0 msgr2=0x7fab74150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:13.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab73400640 1 --2- 172.21.15.83:0/1879692366 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab741599f0 0x7fab74150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:13.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab73400640 1 -- 172.21.15.83:0/1879692366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab740cc510 msgr2=0x7fab7414f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:13.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab73400640 1 --2- 172.21.15.83:0/1879692366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab740cc510 0x7fab7414f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:13.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab73400640 1 -- 172.21.15.83:0/1879692366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7415b300 con 0x7fab74164cb0 2024-04-25T00:13:13.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab73400640 1 --2- 172.21.15.83:0/1879692366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab74164cb0 0x7fab7415e4f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009df0 tx=0x7fab6c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:13.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.181+0000 7fab6b400640 1 -- 172.21.15.83:0/1879692366 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0b2050 con 0x7fab74164cb0 2024-04-25T00:13:13.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.185+0000 7fab6b400640 1 -- 172.21.15.83:0/1879692366 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6c0b3040 con 0x7fab74164cb0 2024-04-25T00:13:13.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.185+0000 7fab79a00640 1 -- 172.21.15.83:0/1879692366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7415d730 con 0x7fab74164cb0 2024-04-25T00:13:13.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.185+0000 7fab6b400640 1 -- 172.21.15.83:0/1879692366 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0ad650 con 0x7fab74164cb0 2024-04-25T00:13:13.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.185+0000 7fab79a00640 1 -- 172.21.15.83:0/1879692366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7415dcf0 con 0x7fab74164cb0 2024-04-25T00:13:13.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.185+0000 7fab79a00640 1 -- 172.21.15.83:0/1879692366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab740737f0 con 0x7fab74164cb0 2024-04-25T00:13:13.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.185+0000 7fab6b400640 1 -- 172.21.15.83:0/1879692366 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fab6c0c0020 con 0x7fab74164cb0 2024-04-25T00:13:13.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.185+0000 7fab6b400640 1 --2- 172.21.15.83:0/1879692366 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab500678c0 0x7fab50069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:13.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.185+0000 7fab6b400640 1 -- 172.21.15.83:0/1879692366 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7fab6c120380 con 0x7fab74164cb0 2024-04-25T00:13:13.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.185+0000 7fab72a00640 1 --2- 172.21.15.83:0/1879692366 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab500678c0 0x7fab50069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:13.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.185+0000 7fab72a00640 1 --2- 172.21.15.83:0/1879692366 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab500678c0 0x7fab50069d80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fab64007fd0 tx=0x7fab64009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:13.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.189+0000 7fab6b400640 1 -- 172.21.15.83:0/1879692366 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fab6c016580 con 0x7fab74164cb0 2024-04-25T00:13:13.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:13.401+0000 7fab79a00640 1 -- 172.21.15.83:0/1879692366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]} v 0) -- 0x7fab7415f370 con 0x7fab74164cb0 2024-04-25T00:13:14.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.361+0000 7fab6b400640 1 -- 172.21.15.83:0/1879692366 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]}]=0 v150) ==== 145+0+0 (secure 0 0 0) 0x7fab6c0f0f90 con 0x7fab74164cb0 2024-04-25T00:13:14.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.365+0000 7fab79a00640 1 -- 172.21.15.83:0/1879692366 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab500678c0 msgr2=0x7fab50069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:14.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.365+0000 7fab79a00640 1 --2- 172.21.15.83:0/1879692366 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab500678c0 0x7fab50069d80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fab64007fd0 tx=0x7fab64009040 comp rx=0 tx=0).stop 2024-04-25T00:13:14.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.365+0000 7fab79a00640 1 -- 172.21.15.83:0/1879692366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab74164cb0 msgr2=0x7fab7415e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:14.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.365+0000 7fab79a00640 1 --2- 172.21.15.83:0/1879692366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab74164cb0 0x7fab7415e4f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009df0 tx=0x7fab6c0196a0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.365+0000 7fab79a00640 1 -- 172.21.15.83:0/1879692366 shutdown_connections 2024-04-25T00:13:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.365+0000 7fab79a00640 1 --2- 172.21.15.83:0/1879692366 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fab500678c0 0x7fab50069d80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.365+0000 7fab79a00640 1 --2- 172.21.15.83:0/1879692366 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab741599f0 0x7fab74150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.365+0000 7fab79a00640 1 --2- 172.21.15.83:0/1879692366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab74164cb0 0x7fab7415e4f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.365+0000 7fab79a00640 1 --2- 172.21.15.83:0/1879692366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab740cc510 0x7fab7414f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.365+0000 7fab79a00640 1 -- 172.21.15.83:0/1879692366 >> 172.21.15.83:0/1879692366 conn(0x7fab740b1830 msgr2=0x7fab7415c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.365+0000 7fab79a00640 1 -- 172.21.15.83:0/1879692366 shutdown_connections 2024-04-25T00:13:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.365+0000 7fab79a00640 1 -- 172.21.15.83:0/1879692366 wait complete. 2024-04-25T00:13:14.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:13:14.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f6095c00640 1 Processor -- start 2024-04-25T00:13:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f6095c00640 1 -- start start 2024-04-25T00:13:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f6095c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6090159810 0x7f60900cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f6095c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60900ce000 con 0x7f6090160020 2024-04-25T00:13:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f6095c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f609006eec0 con 0x7f609006e950 2024-04-25T00:13:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f6095c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f609006e210 con 0x7f6090159810 2024-04-25T00:13:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f608f400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6090160020 0x7f60900ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49422/0 (socket says 172.21.15.83:49422) 2024-04-25T00:13:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f608f400640 1 -- 172.21.15.83:0/3990363809 learned_addr learned my addr 172.21.15.83:0/3990363809 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f608ea00640 1 --2- 172.21.15.83:0/3990363809 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6090159810 0x7f60900cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f608d600640 1 -- 172.21.15.83:0/3990363809 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2296032595 0 0) 0x7f60900ce000 con 0x7f6090160020 2024-04-25T00:13:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f608d600640 1 -- 172.21.15.83:0/3990363809 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6074003530 con 0x7f6090160020 2024-04-25T00:13:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f608d600640 1 -- 172.21.15.83:0/3990363809 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2562990044 0 0) 0x7f609006eec0 con 0x7f609006e950 2024-04-25T00:13:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f608d600640 1 -- 172.21.15.83:0/3990363809 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60900ce000 con 0x7f609006e950 2024-04-25T00:13:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f608ea00640 1 -- 172.21.15.83:0/3990363809 >> v1:172.21.15.86:6789/0 conn(0x7f609006e950 legacy=0x7f609015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f608ea00640 1 -- 172.21.15.83:0/3990363809 >> v1:172.21.15.83:6789/0 conn(0x7f6090160020 legacy=0x7f60900ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.529+0000 7f608ea00640 1 -- 172.21.15.83:0/3990363809 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609006dc50 con 0x7f6090159810 2024-04-25T00:13:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.533+0000 7f608ea00640 1 --2- 172.21.15.83:0/3990363809 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6090159810 0x7f60900cc4a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6080009820 tx=0x7f608001c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=129fd5702c6c5721 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.533+0000 7f608d600640 1 -- 172.21.15.83:0/3990363809 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60800b0070 con 0x7f6090159810 2024-04-25T00:13:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.533+0000 7f608d600640 1 -- 172.21.15.83:0/3990363809 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60800ab600 con 0x7f6090159810 2024-04-25T00:13:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.533+0000 7f608d600640 1 -- 172.21.15.83:0/3990363809 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60800ab9b0 con 0x7f6090159810 2024-04-25T00:13:14.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.533+0000 7f6095c00640 1 -- 172.21.15.83:0/3990363809 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6090159810 msgr2=0x7f60900cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:14.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.533+0000 7f6095c00640 1 --2- 172.21.15.83:0/3990363809 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6090159810 0x7f60900cc4a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6080009820 tx=0x7f608001c430 comp rx=0 tx=0).stop 2024-04-25T00:13:14.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.533+0000 7f6095c00640 1 -- 172.21.15.83:0/3990363809 shutdown_connections 2024-04-25T00:13:14.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.533+0000 7f6095c00640 1 --2- 172.21.15.83:0/3990363809 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6090159810 0x7f60900cc4a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.533+0000 7f6095c00640 1 -- 172.21.15.83:0/3990363809 >> 172.21.15.83:0/3990363809 conn(0x7f60900b1830 msgr2=0x7f60900b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:14.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.533+0000 7f6095c00640 1 -- 172.21.15.83:0/3990363809 shutdown_connections 2024-04-25T00:13:14.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.533+0000 7f6095c00640 1 -- 172.21.15.83:0/3990363809 wait complete. 2024-04-25T00:13:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f6095c00640 1 Processor -- start 2024-04-25T00:13:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f6095c00640 1 -- start start 2024-04-25T00:13:14.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f6095c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f609006e950 0x7f60901500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:14.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f6095c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6090159810 0x7f6090153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:14.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f6095c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6090160020 0x7f60900b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:14.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f6095c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6090070f20 con 0x7f6090160020 2024-04-25T00:13:14.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f6095c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6090070da0 con 0x7f6090159810 2024-04-25T00:13:14.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f6095c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6090169cc0 con 0x7f609006e950 2024-04-25T00:13:14.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f608f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6090160020 0x7f60900b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:14.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f608f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6090160020 0x7f60900b02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55116/0 (socket says 172.21.15.83:55116) 2024-04-25T00:13:14.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f608f400640 1 -- 172.21.15.83:0/661487376 learned_addr learned my addr 172.21.15.83:0/661487376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:14.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f608ea00640 1 --2- 172.21.15.83:0/661487376 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f609006e950 0x7f60901500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:14.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f608e000640 1 --2- 172.21.15.83:0/661487376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6090159810 0x7f6090153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:14.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f608f400640 1 -- 172.21.15.83:0/661487376 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f609006e950 msgr2=0x7f60901500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f608f400640 1 --2- 172.21.15.83:0/661487376 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f609006e950 0x7f60901500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f608f400640 1 -- 172.21.15.83:0/661487376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6090159810 msgr2=0x7f6090153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f608f400640 1 --2- 172.21.15.83:0/661487376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6090159810 0x7f6090153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f608f400640 1 -- 172.21.15.83:0/661487376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6090159ea0 con 0x7f6090160020 2024-04-25T00:13:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f608f400640 1 --2- 172.21.15.83:0/661487376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6090160020 0x7f60900b02f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f608800cbc0 tx=0x7f608800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f6086a00640 1 -- 172.21.15.83:0/661487376 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088017070 con 0x7f6090160020 2024-04-25T00:13:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f6086a00640 1 -- 172.21.15.83:0/661487376 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f608800ace0 con 0x7f6090160020 2024-04-25T00:13:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f6086a00640 1 -- 172.21.15.83:0/661487376 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088012600 con 0x7f6090160020 2024-04-25T00:13:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f6095c00640 1 -- 172.21.15.83:0/661487376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60900aec30 con 0x7f6090160020 2024-04-25T00:13:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f6095c00640 1 -- 172.21.15.83:0/661487376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60900af1f0 con 0x7f6090160020 2024-04-25T00:13:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.537+0000 7f6095c00640 1 -- 172.21.15.83:0/661487376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f609004f510 con 0x7f6090160020 2024-04-25T00:13:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.541+0000 7f6086a00640 1 -- 172.21.15.83:0/661487376 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f608801a070 con 0x7f6090160020 2024-04-25T00:13:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.541+0000 7f6086a00640 1 --2- 172.21.15.83:0/661487376 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f60700678f0 0x7f6070069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.541+0000 7f6086a00640 1 -- 172.21.15.83:0/661487376 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f6088085800 con 0x7f6090160020 2024-04-25T00:13:14.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.541+0000 7f608ea00640 1 --2- 172.21.15.83:0/661487376 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f60700678f0 0x7f6070069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:14.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.541+0000 7f608ea00640 1 --2- 172.21.15.83:0/661487376 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f60700678f0 0x7f6070069db0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f60800097f0 tx=0x7f6080008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:14.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.545+0000 7f6086a00640 1 -- 172.21.15.83:0/661487376 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6088055dc0 con 0x7f6090160020 2024-04-25T00:13:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.701+0000 7f6095c00640 1 -- 172.21.15.83:0/661487376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f609015a070 con 0x7f6090160020 2024-04-25T00:13:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.705+0000 7f6086a00640 1 -- 172.21.15.83:0/661487376 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f608805a0c0 con 0x7f6090160020 2024-04-25T00:13:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.709+0000 7f6095c00640 1 -- 172.21.15.83:0/661487376 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f60700678f0 msgr2=0x7f6070069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.709+0000 7f6095c00640 1 --2- 172.21.15.83:0/661487376 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f60700678f0 0x7f6070069db0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f60800097f0 tx=0x7f6080008040 comp rx=0 tx=0).stop 2024-04-25T00:13:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.709+0000 7f6095c00640 1 -- 172.21.15.83:0/661487376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6090160020 msgr2=0x7f60900b02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.709+0000 7f6095c00640 1 --2- 172.21.15.83:0/661487376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6090160020 0x7f60900b02f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f608800cbc0 tx=0x7f608800a430 comp rx=0 tx=0).stop 2024-04-25T00:13:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.709+0000 7f6095c00640 1 -- 172.21.15.83:0/661487376 shutdown_connections 2024-04-25T00:13:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.709+0000 7f6095c00640 1 --2- 172.21.15.83:0/661487376 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f60700678f0 0x7f6070069db0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.709+0000 7f6095c00640 1 --2- 172.21.15.83:0/661487376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6090160020 0x7f60900b02f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.709+0000 7f6095c00640 1 --2- 172.21.15.83:0/661487376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6090159810 0x7f6090153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.709+0000 7f6095c00640 1 --2- 172.21.15.83:0/661487376 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f609006e950 0x7f60901500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.709+0000 7f6095c00640 1 -- 172.21.15.83:0/661487376 >> 172.21.15.83:0/661487376 conn(0x7f60900b1830 msgr2=0x7f609015d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.709+0000 7f6095c00640 1 -- 172.21.15.83:0/661487376 shutdown_connections 2024-04-25T00:13:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:14.709+0000 7f6095c00640 1 -- 172.21.15.83:0/661487376 wait complete. 2024-04-25T00:13:14.729 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:13:14.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:13:14.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:13:14.928 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:13:14.928 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:13:14.929 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:14.937 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:13:14.937 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:13:14.937 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:14.947 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:13:14.947 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:13:14.947 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:14.989 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:13:14.989 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:13:14.989 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:14.998 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:13:14.998 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:13:14.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:15.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.165+0000 7f2e9c000640 1 Processor -- start 2024-04-25T00:13:15.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.165+0000 7f2e9c000640 1 -- start start 2024-04-25T00:13:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.165+0000 7f2e9c000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e941599f0 0x7f2e940cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.165+0000 7f2e9c000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e9415c340 0x7f2e9415c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.165+0000 7f2e9c000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e9415ccc0 0x7f2e9415f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.165+0000 7f2e9c000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2e94159dd0 con 0x7f2e9415ccc0 2024-04-25T00:13:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.165+0000 7f2e9c000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2e940ccc60 con 0x7f2e941599f0 2024-04-25T00:13:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.165+0000 7f2e9c000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2e9406dba0 con 0x7f2e9415c340 2024-04-25T00:13:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.165+0000 7f2e9a000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e9415ccc0 0x7f2e9415f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.165+0000 7f2e9a000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e9415ccc0 0x7f2e9415f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55122/0 (socket says 172.21.15.83:55122) 2024-04-25T00:13:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.165+0000 7f2e9a000640 1 -- 172.21.15.83:0/803691411 learned_addr learned my addr 172.21.15.83:0/803691411 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.165+0000 7f2e99600640 1 --2- 172.21.15.83:0/803691411 >> v2:172.21.15.86:3300/0 conn(0x7f2e941599f0 0x7f2e940cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.165+0000 7f2e98c00640 1 --2- 172.21.15.83:0/803691411 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e9415c340 0x7f2e9415c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9a000640 1 -- 172.21.15.83:0/803691411 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e9415c340 msgr2=0x7f2e9415c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9a000640 1 --2- 172.21.15.83:0/803691411 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e9415c340 0x7f2e9415c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9a000640 1 -- 172.21.15.83:0/803691411 >> v2:172.21.15.86:3300/0 conn(0x7f2e941599f0 msgr2=0x7f2e940cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9a000640 1 --2- 172.21.15.83:0/803691411 >> v2:172.21.15.86:3300/0 conn(0x7f2e941599f0 0x7f2e940cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9a000640 1 -- 172.21.15.83:0/803691411 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9406de80 con 0x7f2e9415ccc0 2024-04-25T00:13:15.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9a000640 1 --2- 172.21.15.83:0/803691411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e9415ccc0 0x7f2e9415f930 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009e50 tx=0x7f2e8c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45efdc7b752a3c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:15.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e93e00640 1 -- 172.21.15.83:0/803691411 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0a5690 con 0x7f2e9415ccc0 2024-04-25T00:13:15.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e93e00640 1 -- 172.21.15.83:0/803691411 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e8c0a5830 con 0x7f2e9415ccc0 2024-04-25T00:13:15.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e93e00640 1 -- 172.21.15.83:0/803691411 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0a5c00 con 0x7f2e9415ccc0 2024-04-25T00:13:15.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9c000640 1 -- 172.21.15.83:0/803691411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e9415ccc0 msgr2=0x7f2e9415f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:15.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9c000640 1 --2- 172.21.15.83:0/803691411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e9415ccc0 0x7f2e9415f930 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009e50 tx=0x7f2e8c01c960 comp rx=0 tx=0).stop 2024-04-25T00:13:15.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9c000640 1 -- 172.21.15.83:0/803691411 shutdown_connections 2024-04-25T00:13:15.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9c000640 1 --2- 172.21.15.83:0/803691411 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e9415c340 0x7f2e9415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9c000640 1 --2- 172.21.15.83:0/803691411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e9415ccc0 0x7f2e9415f930 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9c000640 1 --2- 172.21.15.83:0/803691411 >> v2:172.21.15.86:3300/0 conn(0x7f2e941599f0 0x7f2e940cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9c000640 1 -- 172.21.15.83:0/803691411 >> 172.21.15.83:0/803691411 conn(0x7f2e940b1830 msgr2=0x7f2e940b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:15.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9c000640 1 -- 172.21.15.83:0/803691411 shutdown_connections 2024-04-25T00:13:15.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.169+0000 7f2e9c000640 1 -- 172.21.15.83:0/803691411 wait complete. 2024-04-25T00:13:15.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e9c000640 1 Processor -- start 2024-04-25T00:13:15.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e9c000640 1 -- start start 2024-04-25T00:13:15.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e9c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e941599f0 0x7f2e941435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:15.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e9c000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e9415c340 0x7f2e94140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:15.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e9c000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e9415ccc0 0x7f2e94155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:15.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e9c000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e940ce860 con 0x7f2e9415c340 2024-04-25T00:13:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e9c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e940ce6e0 con 0x7f2e941599f0 2024-04-25T00:13:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e9c000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2e940ce9e0 con 0x7f2e9415ccc0 2024-04-25T00:13:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e98c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e9415c340 0x7f2e94140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e98c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e9415c340 0x7f2e94140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55128/0 (socket says 172.21.15.83:55128) 2024-04-25T00:13:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e98c00640 1 -- 172.21.15.83:0/2239450125 learned_addr learned my addr 172.21.15.83:0/2239450125 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e99600640 1 --2- 172.21.15.83:0/2239450125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e941599f0 0x7f2e941435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e9a000640 1 --2- 172.21.15.83:0/2239450125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e9415ccc0 0x7f2e94155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e98c00640 1 -- 172.21.15.83:0/2239450125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e9415ccc0 msgr2=0x7f2e94155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e98c00640 1 --2- 172.21.15.83:0/2239450125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e9415ccc0 0x7f2e94155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e98c00640 1 -- 172.21.15.83:0/2239450125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e941599f0 msgr2=0x7f2e941435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e98c00640 1 --2- 172.21.15.83:0/2239450125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e941599f0 0x7f2e941435b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e98c00640 1 -- 172.21.15.83:0/2239450125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e94143af0 con 0x7f2e9415c340 2024-04-25T00:13:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e98c00640 1 --2- 172.21.15.83:0/2239450125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e9415c340 0x7f2e94140290 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000d800 tx=0x7f2e8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.173+0000 7f2e91600640 1 -- 172.21.15.83:0/2239450125 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000b6c0 con 0x7f2e9415c340 2024-04-25T00:13:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.177+0000 7f2e91600640 1 -- 172.21.15.83:0/2239450125 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e80019070 con 0x7f2e9415c340 2024-04-25T00:13:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.177+0000 7f2e9c000640 1 -- 172.21.15.83:0/2239450125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e94140f00 con 0x7f2e9415c340 2024-04-25T00:13:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.177+0000 7f2e91600640 1 -- 172.21.15.83:0/2239450125 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000bae0 con 0x7f2e9415c340 2024-04-25T00:13:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.177+0000 7f2e9c000640 1 -- 172.21.15.83:0/2239450125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e941414c0 con 0x7f2e9415c340 2024-04-25T00:13:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.177+0000 7f2e9c000640 1 -- 172.21.15.83:0/2239450125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e940737f0 con 0x7f2e9415c340 2024-04-25T00:13:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.177+0000 7f2e91600640 1 -- 172.21.15.83:0/2239450125 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2e8000bc80 con 0x7f2e9415c340 2024-04-25T00:13:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.177+0000 7f2e91600640 1 --2- 172.21.15.83:0/2239450125 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e740678c0 0x7f2e74069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.177+0000 7f2e91600640 1 -- 172.21.15.83:0/2239450125 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f2e80085fc0 con 0x7f2e9415c340 2024-04-25T00:13:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.177+0000 7f2e99600640 1 --2- 172.21.15.83:0/2239450125 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e740678c0 0x7f2e74069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:15.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.177+0000 7f2e99600640 1 --2- 172.21.15.83:0/2239450125 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e740678c0 0x7f2e74069d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2e880045f0 tx=0x7f2e88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.181+0000 7f2e91600640 1 -- 172.21.15.83:0/2239450125 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2e80056580 con 0x7f2e9415c340 2024-04-25T00:13:15.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.369+0000 7f2e9c000640 1 -- 172.21.15.83:0/2239450125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e94072c40 con 0x7f2e9415c340 2024-04-25T00:13:15.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.373+0000 7f2e91600640 1 -- 172.21.15.83:0/2239450125 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3863 (secure 0 0 0) 0x7f2e8005a880 con 0x7f2e9415c340 2024-04-25T00:13:15.376 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:15.376 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2761747029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2761747029},{"type":"v1","addr":"172.21.15.83:6837","nonce":2761747029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2989939837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2989939837},{"type":"v1","addr":"172.21.15.86:6835","nonce":2989939837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2291906415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2291906415},{"type":"v1","addr":"172.21.15.83:6836","nonce":2291906415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/70646880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":70646880},{"type":"v1","addr":"172.21.15.86:6834","nonce":70646880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-04-25T00:13:15.377 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-04-25T00:13:15.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.377+0000 7f2e9c000640 1 -- 172.21.15.83:0/2239450125 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e740678c0 msgr2=0x7f2e74069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:15.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.377+0000 7f2e9c000640 1 --2- 172.21.15.83:0/2239450125 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e740678c0 0x7f2e74069d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2e880045f0 tx=0x7f2e88008040 comp rx=0 tx=0).stop 2024-04-25T00:13:15.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.377+0000 7f2e9c000640 1 -- 172.21.15.83:0/2239450125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e9415c340 msgr2=0x7f2e94140290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:15.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.377+0000 7f2e9c000640 1 --2- 172.21.15.83:0/2239450125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e9415c340 0x7f2e94140290 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000d800 tx=0x7f2e8000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.377+0000 7f2e9c000640 1 -- 172.21.15.83:0/2239450125 shutdown_connections 2024-04-25T00:13:15.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.377+0000 7f2e9c000640 1 --2- 172.21.15.83:0/2239450125 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e740678c0 0x7f2e74069d80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.377+0000 7f2e9c000640 1 --2- 172.21.15.83:0/2239450125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e9415ccc0 0x7f2e94155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.377+0000 7f2e9c000640 1 --2- 172.21.15.83:0/2239450125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e9415c340 0x7f2e94140290 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.377+0000 7f2e9c000640 1 --2- 172.21.15.83:0/2239450125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e941599f0 0x7f2e941435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.377+0000 7f2e9c000640 1 -- 172.21.15.83:0/2239450125 >> 172.21.15.83:0/2239450125 conn(0x7f2e940b1830 msgr2=0x7f2e9406e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:15.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.377+0000 7f2e9c000640 1 -- 172.21.15.83:0/2239450125 shutdown_connections 2024-04-25T00:13:15.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.377+0000 7f2e9c000640 1 -- 172.21.15.83:0/2239450125 wait complete. 2024-04-25T00:13:15.398 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:13:15.398 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:13:15.398 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:13:15.399 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:13:15.399 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:13:15.399 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:13:15.399 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:13:15.399 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:13:15.399 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:13:15.399 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:13:15.400 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:13:15.400 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:13:15.400 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:13:15.400 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:13:15.400 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:13:15.400 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:13:15.411 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:13:15.411 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:13:15.412 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:13:15.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:13:15.414 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:13:15.414 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:13:15.416 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:13:15.416 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:13:15.460 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:13:15.460 DEBUG:teuthology.parallel:result is None 2024-04-25T00:13:15.461 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:13:15.461 DEBUG:teuthology.parallel:result is None 2024-04-25T00:13:15.463 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:13:15.464 DEBUG:teuthology.parallel:result is None 2024-04-25T00:13:15.466 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:13:15.466 DEBUG:teuthology.parallel:result is None 2024-04-25T00:13:15.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:13:15.546 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-25T00:13:15.547 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-25T00:13:15.553 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2024-04-25T00:13:15.554 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2024-04-25T00:13:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff932e00640 1 Processor -- start 2024-04-25T00:13:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff932e00640 1 -- start start 2024-04-25T00:13:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff932e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff92c1446e0 0x7ff92c0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff932e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff92c06ee20 0x7ff92c071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff932e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff92c0cb210 con 0x7ff92c15f350 2024-04-25T00:13:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff932e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff92c0715b0 con 0x7ff92c06ee20 2024-04-25T00:13:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff932e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff92c144ac0 con 0x7ff92c1446e0 2024-04-25T00:13:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff930e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff92c06ee20 0x7ff92c071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff930e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff92c06ee20 0x7ff92c071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55174/0 (socket says 172.21.15.83:55174) 2024-04-25T00:13:15.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff930e00640 1 -- 172.21.15.83:0/3940823928 learned_addr learned my addr 172.21.15.83:0/3940823928 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:15.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff92be00640 1 --2- 172.21.15.83:0/3940823928 >> v2:172.21.15.86:3300/0 conn(0x7ff92c1446e0 0x7ff92c0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:15.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff930e00640 1 -- 172.21.15.83:0/3940823928 >> v2:172.21.15.86:3300/0 conn(0x7ff92c1446e0 msgr2=0x7ff92c0cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:15.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff930e00640 1 --2- 172.21.15.83:0/3940823928 >> v2:172.21.15.86:3300/0 conn(0x7ff92c1446e0 0x7ff92c0cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff930e00640 1 -- 172.21.15.83:0/3940823928 >> v1:172.21.15.86:6789/0 conn(0x7ff92c15f350 legacy=0x7ff92c06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:15.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff930e00640 1 -- 172.21.15.83:0/3940823928 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92c0cb4f0 con 0x7ff92c06ee20 2024-04-25T00:13:15.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.609+0000 7ff92be00640 1 --2- 172.21.15.83:0/3940823928 >> v2:172.21.15.86:3300/0 conn(0x7ff92c1446e0 0x7ff92c0cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.613+0000 7ff930e00640 1 --2- 172.21.15.83:0/3940823928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c06ee20 0x7ff92c071fe0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff92c04fb60 tx=0x7ff92401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87351b284241a7cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.613+0000 7ff92aa00640 1 -- 172.21.15.83:0/3940823928 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9240a5740 con 0x7ff92c06ee20 2024-04-25T00:13:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.613+0000 7ff92aa00640 1 -- 172.21.15.83:0/3940823928 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9240a58e0 con 0x7ff92c06ee20 2024-04-25T00:13:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.613+0000 7ff92aa00640 1 -- 172.21.15.83:0/3940823928 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9240a5cb0 con 0x7ff92c06ee20 2024-04-25T00:13:15.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.613+0000 7ff932e00640 1 -- 172.21.15.83:0/3940823928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c06ee20 msgr2=0x7ff92c071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:15.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.613+0000 7ff932e00640 1 --2- 172.21.15.83:0/3940823928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c06ee20 0x7ff92c071fe0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff92c04fb60 tx=0x7ff92401c9b0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.613+0000 7ff932e00640 1 -- 172.21.15.83:0/3940823928 shutdown_connections 2024-04-25T00:13:15.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.613+0000 7ff932e00640 1 --2- 172.21.15.83:0/3940823928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c06ee20 0x7ff92c071fe0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.613+0000 7ff932e00640 1 --2- 172.21.15.83:0/3940823928 >> v2:172.21.15.86:3300/0 conn(0x7ff92c1446e0 0x7ff92c0cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.613+0000 7ff932e00640 1 -- 172.21.15.83:0/3940823928 >> 172.21.15.83:0/3940823928 conn(0x7ff92c0cfa60 msgr2=0x7ff92c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:15.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.613+0000 7ff932e00640 1 -- 172.21.15.83:0/3940823928 shutdown_connections 2024-04-25T00:13:15.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.613+0000 7ff932e00640 1 -- 172.21.15.83:0/3940823928 wait complete. 2024-04-25T00:13:15.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.613+0000 7ff932e00640 1 Processor -- start 2024-04-25T00:13:15.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.613+0000 7ff932e00640 1 -- start start 2024-04-25T00:13:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff932e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff92c06ee20 0x7ff92c15c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff932e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff92c1446e0 0x7ff92c1405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff932e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c15f350 0x7ff92c15b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff932e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c072eb0 con 0x7ff92c15f350 2024-04-25T00:13:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff932e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c072d30 con 0x7ff92c1446e0 2024-04-25T00:13:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff932e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff92c073030 con 0x7ff92c06ee20 2024-04-25T00:13:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff930e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c15f350 0x7ff92c15b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff930e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c15f350 0x7ff92c15b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55186/0 (socket says 172.21.15.83:55186) 2024-04-25T00:13:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff930e00640 1 -- 172.21.15.83:0/1113852271 learned_addr learned my addr 172.21.15.83:0/1113852271 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff92b400640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff92c1446e0 0x7ff92c1405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff92be00640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff92c06ee20 0x7ff92c15c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff930e00640 1 -- 172.21.15.83:0/1113852271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff92c06ee20 msgr2=0x7ff92c15c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff930e00640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff92c06ee20 0x7ff92c15c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff930e00640 1 -- 172.21.15.83:0/1113852271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff92c1446e0 msgr2=0x7ff92c1405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff930e00640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff92c1446e0 0x7ff92c1405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff930e00640 1 -- 172.21.15.83:0/1113852271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92c13f820 con 0x7ff92c15f350 2024-04-25T00:13:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff92b400640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff92c1446e0 0x7ff92c1405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff92be00640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff92c06ee20 0x7ff92c15c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff930e00640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c15f350 0x7ff92c15b7a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff92401b060 tx=0x7ff924019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:15.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff923e00640 1 -- 172.21.15.83:0/1113852271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9240bc030 con 0x7ff92c15f350 2024-04-25T00:13:15.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff923e00640 1 -- 172.21.15.83:0/1113852271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9240b8470 con 0x7ff92c15f350 2024-04-25T00:13:15.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff923e00640 1 -- 172.21.15.83:0/1113852271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9240b8860 con 0x7ff92c15f350 2024-04-25T00:13:15.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff932e00640 1 -- 172.21.15.83:0/1113852271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92c13fb00 con 0x7ff92c15f350 2024-04-25T00:13:15.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff932e00640 1 -- 172.21.15.83:0/1113852271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92c140b20 con 0x7ff92c15f350 2024-04-25T00:13:15.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff932e00640 1 -- 172.21.15.83:0/1113852271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff92c068230 con 0x7ff92c15f350 2024-04-25T00:13:15.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.617+0000 7ff923e00640 1 -- 172.21.15.83:0/1113852271 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff9240bf070 con 0x7ff92c15f350 2024-04-25T00:13:15.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.621+0000 7ff923e00640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff90c0678c0 0x7ff90c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:15.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.621+0000 7ff923e00640 1 -- 172.21.15.83:0/1113852271 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7ff924120aa0 con 0x7ff92c15f350 2024-04-25T00:13:15.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.621+0000 7ff92be00640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff90c0678c0 0x7ff90c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:15.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.621+0000 7ff92be00640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff90c0678c0 0x7ff90c069d80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0ce620 tx=0x7ff91c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:15.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.625+0000 7ff923e00640 1 -- 172.21.15.83:0/1113852271 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff9240c6070 con 0x7ff92c15f350 2024-04-25T00:13:15.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:15.789+0000 7ff932e00640 1 -- 172.21.15.83:0/1113852271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff92c06e610 con 0x7ff92c15f350 2024-04-25T00:13:16.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.389+0000 7ff923e00640 1 -- 172.21.15.83:0/1113852271 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) ==== 97+0+0 (secure 0 0 0) 0x7ff9240165a0 con 0x7ff92c15f350 2024-04-25T00:13:16.390 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-25T00:13:16.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.389+0000 7ff932e00640 1 -- 172.21.15.83:0/1113852271 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff90c0678c0 msgr2=0x7ff90c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.389+0000 7ff932e00640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff90c0678c0 0x7ff90c069d80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0ce620 tx=0x7ff91c008040 comp rx=0 tx=0).stop 2024-04-25T00:13:16.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.389+0000 7ff932e00640 1 -- 172.21.15.83:0/1113852271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c15f350 msgr2=0x7ff92c15b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.389+0000 7ff932e00640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c15f350 0x7ff92c15b7a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff92401b060 tx=0x7ff924019750 comp rx=0 tx=0).stop 2024-04-25T00:13:16.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.389+0000 7ff932e00640 1 -- 172.21.15.83:0/1113852271 shutdown_connections 2024-04-25T00:13:16.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.389+0000 7ff932e00640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff90c0678c0 0x7ff90c069d80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.389+0000 7ff932e00640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c15f350 0x7ff92c15b7a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.389+0000 7ff932e00640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff92c1446e0 0x7ff92c1405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.389+0000 7ff932e00640 1 --2- 172.21.15.83:0/1113852271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff92c06ee20 0x7ff92c15c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.389+0000 7ff932e00640 1 -- 172.21.15.83:0/1113852271 >> 172.21.15.83:0/1113852271 conn(0x7ff92c0cfa60 msgr2=0x7ff92c06d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:16.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.389+0000 7ff932e00640 1 -- 172.21.15.83:0/1113852271 shutdown_connections 2024-04-25T00:13:16.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.389+0000 7ff932e00640 1 -- 172.21.15.83:0/1113852271 wait complete. 2024-04-25T00:13:16.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:13:16.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.549+0000 7f22e7200640 1 Processor -- start 2024-04-25T00:13:16.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.549+0000 7f22e7200640 1 -- start start 2024-04-25T00:13:16.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.549+0000 7f22e7200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22e01599f0 0x7f22e00cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.549+0000 7f22e7200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f22e015cec0 0x7f22e00ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.549+0000 7f22e7200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22e00ccc60 con 0x7f22e015c340 2024-04-25T00:13:16.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.549+0000 7f22e7200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f22e0159dd0 con 0x7f22e01599f0 2024-04-25T00:13:16.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.549+0000 7f22e7200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f22e006dba0 con 0x7f22e015cec0 2024-04-25T00:13:16.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.549+0000 7f22d7400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f22e015c340 0x7f22e015c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49444/0 (socket says 172.21.15.83:49444) 2024-04-25T00:13:16.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.549+0000 7f22d7400640 1 -- 172.21.15.83:0/2354423525 learned_addr learned my addr 172.21.15.83:0/2354423525 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:16.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.549+0000 7f22dfe00640 1 --2- 172.21.15.83:0/2354423525 >> v2:172.21.15.83:3300/0 conn(0x7f22e01599f0 0x7f22e00cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22e5200640 1 --2- 172.21.15.83:0/2354423525 >> v2:172.21.15.86:3300/0 conn(0x7f22e015cec0 0x7f22e00ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22dfe00640 1 -- 172.21.15.83:0/2354423525 >> v2:172.21.15.86:3300/0 conn(0x7f22e015cec0 msgr2=0x7f22e00ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:16.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22dfe00640 1 --2- 172.21.15.83:0/2354423525 >> v2:172.21.15.86:3300/0 conn(0x7f22e015cec0 0x7f22e00ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22dfe00640 1 -- 172.21.15.83:0/2354423525 >> v1:172.21.15.83:6789/0 conn(0x7f22e015c340 legacy=0x7f22e015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22dfe00640 1 -- 172.21.15.83:0/2354423525 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e006de80 con 0x7f22e01599f0 2024-04-25T00:13:16.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22df400640 1 -- 172.21.15.83:0/2354423525 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1813455310 0 0) 0x7f22c8001120 con 0x7f22e015c340 2024-04-25T00:13:16.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22dfe00640 1 --2- 172.21.15.83:0/2354423525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e01599f0 0x7f22e00cc510 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009d90 tx=0x7f22d001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8633204cf4511275 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:16.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22df400640 1 -- 172.21.15.83:0/2354423525 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d00b2070 con 0x7f22e01599f0 2024-04-25T00:13:16.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22df400640 1 -- 172.21.15.83:0/2354423525 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d00a5760 con 0x7f22e01599f0 2024-04-25T00:13:16.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22df400640 1 -- 172.21.15.83:0/2354423525 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d00a5b30 con 0x7f22e01599f0 2024-04-25T00:13:16.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22e7200640 1 -- 172.21.15.83:0/2354423525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e01599f0 msgr2=0x7f22e00cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22e7200640 1 --2- 172.21.15.83:0/2354423525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e01599f0 0x7f22e00cc510 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009d90 tx=0x7f22d001c8a0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22e7200640 1 -- 172.21.15.83:0/2354423525 shutdown_connections 2024-04-25T00:13:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22e7200640 1 --2- 172.21.15.83:0/2354423525 >> v2:172.21.15.86:3300/0 conn(0x7f22e015cec0 0x7f22e00ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22e7200640 1 --2- 172.21.15.83:0/2354423525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e01599f0 0x7f22e00cc510 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.553+0000 7f22e7200640 1 -- 172.21.15.83:0/2354423525 >> 172.21.15.83:0/2354423525 conn(0x7f22e00b1830 msgr2=0x7f22e00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22e7200640 1 -- 172.21.15.83:0/2354423525 shutdown_connections 2024-04-25T00:13:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22e7200640 1 -- 172.21.15.83:0/2354423525 wait complete. 2024-04-25T00:13:16.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22e7200640 1 Processor -- start 2024-04-25T00:13:16.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22e7200640 1 -- start start 2024-04-25T00:13:16.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22e7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e01599f0 0x7f22e0150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22e7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22e015c340 0x7f22e01fbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22e7200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22e015cec0 0x7f22e015e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22e7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22e00cf7c0 con 0x7f22e01599f0 2024-04-25T00:13:16.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22e7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22e00cf640 con 0x7f22e015c340 2024-04-25T00:13:16.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22e7200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f22e0168cd0 con 0x7f22e015cec0 2024-04-25T00:13:16.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e01599f0 0x7f22e0150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e01599f0 0x7f22e0150450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55198/0 (socket says 172.21.15.83:55198) 2024-04-25T00:13:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22dfe00640 1 -- 172.21.15.83:0/2461764222 learned_addr learned my addr 172.21.15.83:0/2461764222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22d7400640 1 --2- 172.21.15.83:0/2461764222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22e015c340 0x7f22e01fbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22e5200640 1 --2- 172.21.15.83:0/2461764222 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22e015cec0 0x7f22e015e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22dfe00640 1 -- 172.21.15.83:0/2461764222 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22e015cec0 msgr2=0x7f22e015e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22dfe00640 1 --2- 172.21.15.83:0/2461764222 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22e015cec0 0x7f22e015e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22dfe00640 1 -- 172.21.15.83:0/2461764222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22e015c340 msgr2=0x7f22e01fbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22dfe00640 1 --2- 172.21.15.83:0/2461764222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22e015c340 0x7f22e01fbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22dfe00640 1 -- 172.21.15.83:0/2461764222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e0150dc0 con 0x7f22e01599f0 2024-04-25T00:13:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22dfe00640 1 --2- 172.21.15.83:0/2461764222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e01599f0 0x7f22e0150450 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009d90 tx=0x7f22d0019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22dcc00640 1 -- 172.21.15.83:0/2461764222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d00bd030 con 0x7f22e01599f0 2024-04-25T00:13:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22dcc00640 1 -- 172.21.15.83:0/2461764222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d00a5560 con 0x7f22e01599f0 2024-04-25T00:13:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22dcc00640 1 -- 172.21.15.83:0/2461764222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d00b2040 con 0x7f22e01599f0 2024-04-25T00:13:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.557+0000 7f22e7200640 1 -- 172.21.15.83:0/2461764222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e00b2dd0 con 0x7f22e01599f0 2024-04-25T00:13:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.561+0000 7f22e7200640 1 -- 172.21.15.83:0/2461764222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e00b3390 con 0x7f22e01599f0 2024-04-25T00:13:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.561+0000 7f22e7200640 1 -- 172.21.15.83:0/2461764222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22e00737f0 con 0x7f22e01599f0 2024-04-25T00:13:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.561+0000 7f22dcc00640 1 -- 172.21.15.83:0/2461764222 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f22d00c0070 con 0x7f22e01599f0 2024-04-25T00:13:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.561+0000 7f22dcc00640 1 --2- 172.21.15.83:0/2461764222 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22bc0678f0 0x7f22bc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.561+0000 7f22dcc00640 1 -- 172.21.15.83:0/2461764222 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5249+0+0 (secure 0 0 0) 0x7f22d01203a0 con 0x7f22e01599f0 2024-04-25T00:13:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.561+0000 7f22d7400640 1 --2- 172.21.15.83:0/2461764222 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22bc0678f0 0x7f22bc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.561+0000 7f22d7400640 1 --2- 172.21.15.83:0/2461764222 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22bc0678f0 0x7f22bc069db0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f22c8007fd0 tx=0x7f22c8007a00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.565+0000 7f22dcc00640 1 -- 172.21.15.83:0/2461764222 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f22d0016520 con 0x7f22e01599f0 2024-04-25T00:13:16.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.773+0000 7f22e7200640 1 -- 172.21.15.83:0/2461764222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f22e00b3560 con 0x7f22e01599f0 2024-04-25T00:13:16.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.773+0000 7f22dcc00640 1 -- 172.21.15.83:0/2461764222 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f22d00b6370 con 0x7f22e01599f0 2024-04-25T00:13:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:16.777 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:13:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-25T00:13:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-25T00:13:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmytlRFRAA3vHLqK3vHSB+8+TZHAuj7Q==", 2024-04-25T00:13:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:13:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:13:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-25T00:13:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm088MIRAApJvvePXyFoO8JnVErd57zQ==", 2024-04-25T00:13:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:13:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:13:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-25T00:13:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmCqZXHBAAPy5JGbEU11i4gc3gFtbGIQ==", 2024-04-25T00:13:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:13:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:13:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-25T00:13:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmiJZGKBAA5CBwH/cA+LOAj+inS/Hx2g==", 2024-04-25T00:13:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:13:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:13:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-25T00:13:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBhnylmMOiqLRAAdqrCTz4KfghoJ7cZc+b4qg==", 2024-04-25T00:13:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-25T00:13:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBknylmlp8OOBAAl+y4vBrCzDyV/ne9qICQ+w==", 2024-04-25T00:13:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-25T00:13:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBonylm85MOBhAA3ciY1UT/Qg4Pe72OA0IvnA==", 2024-04-25T00:13:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-25T00:13:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBrnylmnAbDEhAAkCQXjbiKINiftX71HBKvQA==", 2024-04-25T00:13:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBxnylmxhy2KxAAs6qjxZA+Fj5umHU/4FVqGw==", 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB0nylmGfSlNRAA45tuuh+5ISOMlhaFdCE5rQ==", 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-25T00:13:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB4nylmouyBARAAz4s4S1BSiDU9YxQ/Vj0d0A==", 2024-04-25T00:13:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB7nylmtIh0CxAApWJLvC8Zlt298VJ55gPrqw==", 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmOqooLBAApte+X6YkRmrpe12YdjYekg==", 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-25T00:13:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmDH93NRAASQzJNC3nI0NeU8cv951eBA==", 2024-04-25T00:13:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:13:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:13:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:13:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-25T00:13:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmHw3+MRAAciXRmexPxmrf7U2jKwpZQQ==", 2024-04-25T00:13:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:13:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:13:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:13:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-25T00:13:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBfnylmIsMCABAAu+yzPRupFFqA/yH4GSZ+8w==", 2024-04-25T00:13:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:13:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:13:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdnylmZ7SKNRAArVHYbh7fBiOUNRTenIwT1w==", 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-25T00:13:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm6TMqERAAiPL81HRFrlz4ku20HemS4w==", 2024-04-25T00:13:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:13:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-25T00:13:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-25T00:13:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm9VRpDRAA5XUtYwjiWBpF1ObJ/Y5g0w==", 2024-04-25T00:13:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:13:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-25T00:13:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T00:13:16.790 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:13:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.777+0000 7f22e7200640 1 -- 172.21.15.83:0/2461764222 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22bc0678f0 msgr2=0x7f22bc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.777+0000 7f22e7200640 1 --2- 172.21.15.83:0/2461764222 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22bc0678f0 0x7f22bc069db0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f22c8007fd0 tx=0x7f22c8007a00 comp rx=0 tx=0).stop 2024-04-25T00:13:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.777+0000 7f22e7200640 1 -- 172.21.15.83:0/2461764222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e01599f0 msgr2=0x7f22e0150450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.777+0000 7f22e7200640 1 --2- 172.21.15.83:0/2461764222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e01599f0 0x7f22e0150450 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009d90 tx=0x7f22d0019640 comp rx=0 tx=0).stop 2024-04-25T00:13:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.777+0000 7f22e7200640 1 -- 172.21.15.83:0/2461764222 shutdown_connections 2024-04-25T00:13:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.777+0000 7f22e7200640 1 --2- 172.21.15.83:0/2461764222 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22bc0678f0 0x7f22bc069db0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.777+0000 7f22e7200640 1 --2- 172.21.15.83:0/2461764222 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22e015cec0 0x7f22e015e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.777+0000 7f22e7200640 1 --2- 172.21.15.83:0/2461764222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22e015c340 0x7f22e01fbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.777+0000 7f22e7200640 1 --2- 172.21.15.83:0/2461764222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e01599f0 0x7f22e0150450 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.777+0000 7f22e7200640 1 -- 172.21.15.83:0/2461764222 >> 172.21.15.83:0/2461764222 conn(0x7f22e00b1830 msgr2=0x7f22e014c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.777+0000 7f22e7200640 1 -- 172.21.15.83:0/2461764222 shutdown_connections 2024-04-25T00:13:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.777+0000 7f22e7200640 1 -- 172.21.15.83:0/2461764222 wait complete. 2024-04-25T00:13:16.799 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:16.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff086800640 1 Processor -- start 2024-04-25T00:13:16.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff086800640 1 -- start start 2024-04-25T00:13:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff086800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff080145970 0x7ff0800726f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff086800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff080072c30 0x7ff08006b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff086800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0800c6160 con 0x7ff08006b720 2024-04-25T00:13:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff086800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff080145d50 con 0x7ff080072c30 2024-04-25T00:13:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff086800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff0800c6d70 con 0x7ff080145970 2024-04-25T00:13:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff076a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff080072c30 0x7ff08006b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff07fe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff08006b720 0x7ff08015fb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49458/0 (socket says 172.21.15.83:49458) 2024-04-25T00:13:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff07fe00640 1 -- 172.21.15.83:0/1362434312 learned_addr learned my addr 172.21.15.83:0/1362434312 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff07f400640 1 --2- 172.21.15.83:0/1362434312 >> v2:172.21.15.86:3300/0 conn(0x7ff080145970 0x7ff0800726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff076a00640 1 -- 172.21.15.83:0/1362434312 >> v2:172.21.15.86:3300/0 conn(0x7ff080145970 msgr2=0x7ff0800726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:16.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff076a00640 1 --2- 172.21.15.83:0/1362434312 >> v2:172.21.15.86:3300/0 conn(0x7ff080145970 0x7ff0800726f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff076a00640 1 -- 172.21.15.83:0/1362434312 >> v1:172.21.15.83:6789/0 conn(0x7ff08006b720 legacy=0x7ff08015fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff076a00640 1 -- 172.21.15.83:0/1362434312 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0800c6440 con 0x7ff080072c30 2024-04-25T00:13:16.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff07f400640 1 --2- 172.21.15.83:0/1362434312 >> v2:172.21.15.86:3300/0 conn(0x7ff080145970 0x7ff0800726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:16.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.933+0000 7ff07ea00640 1 -- 172.21.15.83:0/1362434312 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 887250839 0 0) 0x7ff0800c6160 con 0x7ff08006b720 2024-04-25T00:13:16.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.937+0000 7ff076a00640 1 --2- 172.21.15.83:0/1362434312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff080072c30 0x7ff08006b1e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff080067960 tx=0x7ff06801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d17006aee06eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:16.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.937+0000 7ff07ea00640 1 -- 172.21.15.83:0/1362434312 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0680b0070 con 0x7ff080072c30 2024-04-25T00:13:16.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.937+0000 7ff07ea00640 1 -- 172.21.15.83:0/1362434312 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0680ab600 con 0x7ff080072c30 2024-04-25T00:13:16.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.937+0000 7ff07ea00640 1 -- 172.21.15.83:0/1362434312 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0680ab9d0 con 0x7ff080072c30 2024-04-25T00:13:16.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.937+0000 7ff086800640 1 -- 172.21.15.83:0/1362434312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff080072c30 msgr2=0x7ff08006b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.937+0000 7ff086800640 1 --2- 172.21.15.83:0/1362434312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff080072c30 0x7ff08006b1e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff080067960 tx=0x7ff06801c480 comp rx=0 tx=0).stop 2024-04-25T00:13:16.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.937+0000 7ff086800640 1 -- 172.21.15.83:0/1362434312 shutdown_connections 2024-04-25T00:13:16.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.937+0000 7ff086800640 1 --2- 172.21.15.83:0/1362434312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff080072c30 0x7ff08006b1e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.937+0000 7ff086800640 1 --2- 172.21.15.83:0/1362434312 >> v2:172.21.15.86:3300/0 conn(0x7ff080145970 0x7ff0800726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.937+0000 7ff086800640 1 -- 172.21.15.83:0/1362434312 >> 172.21.15.83:0/1362434312 conn(0x7ff0800cd670 msgr2=0x7ff0800cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:16.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.937+0000 7ff086800640 1 -- 172.21.15.83:0/1362434312 shutdown_connections 2024-04-25T00:13:16.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.937+0000 7ff086800640 1 -- 172.21.15.83:0/1362434312 wait complete. 2024-04-25T00:13:16.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff086800640 1 Processor -- start 2024-04-25T00:13:16.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff086800640 1 -- start start 2024-04-25T00:13:16.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff086800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff08006b720 0x7ff08015ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff086800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff080072c30 0x7ff0800b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff086800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff080145970 0x7ff08015dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff086800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff08006bb70 con 0x7ff080145970 2024-04-25T00:13:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff086800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0800c73f0 con 0x7ff08006b720 2024-04-25T00:13:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff086800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff080168e70 con 0x7ff080072c30 2024-04-25T00:13:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff07fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff080145970 0x7ff08015dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff07fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff080145970 0x7ff08015dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55202/0 (socket says 172.21.15.83:55202) 2024-04-25T00:13:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff07fe00640 1 -- 172.21.15.83:0/779936019 learned_addr learned my addr 172.21.15.83:0/779936019 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff07f400640 1 --2- 172.21.15.83:0/779936019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff08006b720 0x7ff08015ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff076a00640 1 --2- 172.21.15.83:0/779936019 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff080072c30 0x7ff0800b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff07fe00640 1 -- 172.21.15.83:0/779936019 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff080072c30 msgr2=0x7ff0800b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff07fe00640 1 --2- 172.21.15.83:0/779936019 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff080072c30 0x7ff0800b62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff07fe00640 1 -- 172.21.15.83:0/779936019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff08006b720 msgr2=0x7ff08015ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff07fe00640 1 --2- 172.21.15.83:0/779936019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff08006b720 0x7ff08015ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff07fe00640 1 -- 172.21.15.83:0/779936019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0800b5b20 con 0x7ff080145970 2024-04-25T00:13:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff07fe00640 1 --2- 172.21.15.83:0/779936019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff080145970 0x7ff08015dbe0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff07800c7d0 tx=0x7ff07800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff077e00640 1 -- 172.21.15.83:0/779936019 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078018070 con 0x7ff080145970 2024-04-25T00:13:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff077e00640 1 -- 172.21.15.83:0/779936019 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07800a640 con 0x7ff080145970 2024-04-25T00:13:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff077e00640 1 -- 172.21.15.83:0/779936019 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07800aa70 con 0x7ff080145970 2024-04-25T00:13:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff086800640 1 -- 172.21.15.83:0/779936019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff080144f20 con 0x7ff080145970 2024-04-25T00:13:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff086800640 1 -- 172.21.15.83:0/779936019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0800b5840 con 0x7ff080145970 2024-04-25T00:13:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.941+0000 7ff086800640 1 -- 172.21.15.83:0/779936019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff080073790 con 0x7ff080145970 2024-04-25T00:13:16.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.945+0000 7ff077e00640 1 -- 172.21.15.83:0/779936019 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff07800ac10 con 0x7ff080145970 2024-04-25T00:13:16.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.945+0000 7ff077e00640 1 --2- 172.21.15.83:0/779936019 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff05c0678f0 0x7ff05c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.945+0000 7ff077e00640 1 -- 172.21.15.83:0/779936019 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5249+0+0 (secure 0 0 0) 0x7ff078084f10 con 0x7ff080145970 2024-04-25T00:13:16.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.945+0000 7ff07f400640 1 --2- 172.21.15.83:0/779936019 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff05c0678f0 0x7ff05c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.945+0000 7ff07f400640 1 --2- 172.21.15.83:0/779936019 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff05c0678f0 0x7ff05c069db0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff0700045f0 tx=0x7ff070008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:16.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:16.949+0000 7ff077e00640 1 -- 172.21.15.83:0/779936019 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff0780555b0 con 0x7ff080145970 2024-04-25T00:13:17.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.105+0000 7ff086800640 1 -- 172.21.15.83:0/779936019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff08006e9c0 con 0x7ff080145970 2024-04-25T00:13:17.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.105+0000 7ff077e00640 1 -- 172.21.15.83:0/779936019 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+13080 (secure 0 0 0) 0x7ff0780598b0 con 0x7ff080145970 2024-04-25T00:13:17.107 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:17.107 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":56,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:16.404944+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.105+0000 7ff086800640 1 -- 172.21.15.83:0/779936019 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff05c0678f0 msgr2=0x7ff05c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.105+0000 7ff086800640 1 --2- 172.21.15.83:0/779936019 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff05c0678f0 0x7ff05c069db0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff0700045f0 tx=0x7ff070008040 comp rx=0 tx=0).stop 2024-04-25T00:13:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.105+0000 7ff086800640 1 -- 172.21.15.83:0/779936019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff080145970 msgr2=0x7ff08015dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.105+0000 7ff086800640 1 --2- 172.21.15.83:0/779936019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff080145970 0x7ff08015dbe0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff07800c7d0 tx=0x7ff07800cc90 comp rx=0 tx=0).stop 2024-04-25T00:13:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.105+0000 7ff086800640 1 -- 172.21.15.83:0/779936019 shutdown_connections 2024-04-25T00:13:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.105+0000 7ff086800640 1 --2- 172.21.15.83:0/779936019 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff05c0678f0 0x7ff05c069db0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.105+0000 7ff086800640 1 --2- 172.21.15.83:0/779936019 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff080072c30 0x7ff0800b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.105+0000 7ff086800640 1 --2- 172.21.15.83:0/779936019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff080145970 0x7ff08015dbe0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.105+0000 7ff086800640 1 --2- 172.21.15.83:0/779936019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff08006b720 0x7ff08015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.105+0000 7ff086800640 1 -- 172.21.15.83:0/779936019 >> 172.21.15.83:0/779936019 conn(0x7ff0800cd670 msgr2=0x7ff08006eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.105+0000 7ff086800640 1 -- 172.21.15.83:0/779936019 shutdown_connections 2024-04-25T00:13:17.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.105+0000 7ff086800640 1 -- 172.21.15.83:0/779936019 wait complete. 2024-04-25T00:13:17.128 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:13:17.129 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:13:17.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.265+0000 7ff58da00640 1 Processor -- start 2024-04-25T00:13:17.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.265+0000 7ff58da00640 1 -- start start 2024-04-25T00:13:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff58da00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5881551e0 0x7ff58815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff58da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff58806e4c0 0x7ff58806e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff58da00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff58815c560 con 0x7ff5881555c0 2024-04-25T00:13:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff58da00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff58806ee70 con 0x7ff58806e4c0 2024-04-25T00:13:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff58da00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff588072460 con 0x7ff5881551e0 2024-04-25T00:13:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff587400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff58806e4c0 0x7ff58806e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff586a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5881551e0 0x7ff58815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff57e000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff5881555c0 0x7ff588071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:42562/0 (socket says 172.21.15.83:42562) 2024-04-25T00:13:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff587400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff58806e4c0 0x7ff58806e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:33190/0 (socket says 172.21.15.83:33190) 2024-04-25T00:13:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff57e000640 1 -- 172.21.15.83:0/3435219404 learned_addr learned my addr 172.21.15.83:0/3435219404 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff586a00640 1 -- 172.21.15.83:0/3435219404 >> v2:172.21.15.86:3300/0 conn(0x7ff58806e4c0 msgr2=0x7ff58806e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff586a00640 1 --2- 172.21.15.83:0/3435219404 >> v2:172.21.15.86:3300/0 conn(0x7ff58806e4c0 0x7ff58806e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff586000640 1 -- 172.21.15.83:0/3435219404 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1071054056 0 0) 0x7ff58815c560 con 0x7ff5881555c0 2024-04-25T00:13:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff586a00640 1 -- 172.21.15.83:0/3435219404 >> v1:172.21.15.86:6789/0 conn(0x7ff5881555c0 legacy=0x7ff588071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff586a00640 1 -- 172.21.15.83:0/3435219404 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58815c7d0 con 0x7ff5881551e0 2024-04-25T00:13:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.269+0000 7ff587400640 1 --2- 172.21.15.83:0/3435219404 >> v2:172.21.15.86:3300/0 conn(0x7ff58806e4c0 0x7ff58806e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:17.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff586a00640 1 --2- 172.21.15.83:0/3435219404 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5881551e0 0x7ff58815bea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff5780097c0 tx=0x7ff57801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7fa32e6166121fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:17.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff586000640 1 -- 172.21.15.83:0/3435219404 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5780b1070 con 0x7ff5881551e0 2024-04-25T00:13:17.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff586000640 1 -- 172.21.15.83:0/3435219404 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5780ac410 con 0x7ff5881551e0 2024-04-25T00:13:17.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff586000640 1 -- 172.21.15.83:0/3435219404 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5780ac7e0 con 0x7ff5881551e0 2024-04-25T00:13:17.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 -- 172.21.15.83:0/3435219404 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5881551e0 msgr2=0x7ff58815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 --2- 172.21.15.83:0/3435219404 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5881551e0 0x7ff58815bea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff5780097c0 tx=0x7ff57801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 -- 172.21.15.83:0/3435219404 shutdown_connections 2024-04-25T00:13:17.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 --2- 172.21.15.83:0/3435219404 >> v2:172.21.15.86:3300/0 conn(0x7ff58806e4c0 0x7ff58806e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 --2- 172.21.15.83:0/3435219404 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5881551e0 0x7ff58815bea0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 -- 172.21.15.83:0/3435219404 >> 172.21.15.83:0/3435219404 conn(0x7ff5880b1830 msgr2=0x7ff5880b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:17.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 -- 172.21.15.83:0/3435219404 shutdown_connections 2024-04-25T00:13:17.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 -- 172.21.15.83:0/3435219404 wait complete. 2024-04-25T00:13:17.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 Processor -- start 2024-04-25T00:13:17.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 -- start start 2024-04-25T00:13:17.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff58806e4c0 0x7ff5881509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5881551e0 0x7ff5880b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5881555c0 0x7ff5881429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff58815db70 con 0x7ff5881551e0 2024-04-25T00:13:17.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff58815d9f0 con 0x7ff58806e4c0 2024-04-25T00:13:17.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.273+0000 7ff58da00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff58815dcf0 con 0x7ff5881555c0 2024-04-25T00:13:17.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff57e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5881551e0 0x7ff5880b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff586a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff58806e4c0 0x7ff5881509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff57e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5881551e0 0x7ff5880b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55218/0 (socket says 172.21.15.83:55218) 2024-04-25T00:13:17.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff57e000640 1 -- 172.21.15.83:0/554891467 learned_addr learned my addr 172.21.15.83:0/554891467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:17.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff587400640 1 --2- 172.21.15.83:0/554891467 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5881555c0 0x7ff5881429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff57e000640 1 -- 172.21.15.83:0/554891467 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5881555c0 msgr2=0x7ff5881429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff57e000640 1 --2- 172.21.15.83:0/554891467 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5881555c0 0x7ff5881429c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff57e000640 1 -- 172.21.15.83:0/554891467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff58806e4c0 msgr2=0x7ff5881509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff57e000640 1 --2- 172.21.15.83:0/554891467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff58806e4c0 0x7ff5881509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff57e000640 1 -- 172.21.15.83:0/554891467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5880b40e0 con 0x7ff5881551e0 2024-04-25T00:13:17.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff586a00640 1 --2- 172.21.15.83:0/554891467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff58806e4c0 0x7ff5881509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:17.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff57e000640 1 --2- 172.21.15.83:0/554891467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5881551e0 0x7ff5880b3970 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c4b0 tx=0x7ff57000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:17.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff57f400640 1 -- 172.21.15.83:0/554891467 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5700074e0 con 0x7ff5881551e0 2024-04-25T00:13:17.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff57f400640 1 -- 172.21.15.83:0/554891467 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5700047f0 con 0x7ff5881551e0 2024-04-25T00:13:17.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff57f400640 1 -- 172.21.15.83:0/554891467 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570004be0 con 0x7ff5881551e0 2024-04-25T00:13:17.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff58da00640 1 -- 172.21.15.83:0/554891467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5880b43c0 con 0x7ff5881551e0 2024-04-25T00:13:17.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff58da00640 1 -- 172.21.15.83:0/554891467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff588142f00 con 0x7ff5881551e0 2024-04-25T00:13:17.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff58da00640 1 -- 172.21.15.83:0/554891467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58804f510 con 0x7ff5881551e0 2024-04-25T00:13:17.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.277+0000 7ff57f400640 1 -- 172.21.15.83:0/554891467 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff570011070 con 0x7ff5881551e0 2024-04-25T00:13:17.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.281+0000 7ff57f400640 1 --2- 172.21.15.83:0/554891467 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5640678c0 0x7ff564069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.281+0000 7ff57f400640 1 -- 172.21.15.83:0/554891467 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5249+0+0 (secure 0 0 0) 0x7ff570085660 con 0x7ff5881551e0 2024-04-25T00:13:17.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.281+0000 7ff586a00640 1 --2- 172.21.15.83:0/554891467 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5640678c0 0x7ff564069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.281+0000 7ff586a00640 1 --2- 172.21.15.83:0/554891467 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5640678c0 0x7ff564069d80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff5780ae000 tx=0x7ff578008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:17.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.285+0000 7ff57f400640 1 -- 172.21.15.83:0/554891467 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff5700562c0 con 0x7ff5881551e0 2024-04-25T00:13:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.445+0000 7ff58da00640 1 -- 172.21.15.83:0/554891467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff5880cc140 con 0x7ff5881551e0 2024-04-25T00:13:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.445+0000 7ff57f400640 1 -- 172.21.15.83:0/554891467 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) ==== 93+0+10 (secure 0 0 0) 0x7ff570055bc0 con 0x7ff5881551e0 2024-04-25T00:13:17.447 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:13:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.445+0000 7ff58da00640 1 -- 172.21.15.83:0/554891467 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5640678c0 msgr2=0x7ff564069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.445+0000 7ff58da00640 1 --2- 172.21.15.83:0/554891467 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5640678c0 0x7ff564069d80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff5780ae000 tx=0x7ff578008040 comp rx=0 tx=0).stop 2024-04-25T00:13:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.445+0000 7ff58da00640 1 -- 172.21.15.83:0/554891467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5881551e0 msgr2=0x7ff5880b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.445+0000 7ff58da00640 1 --2- 172.21.15.83:0/554891467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5881551e0 0x7ff5880b3970 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c4b0 tx=0x7ff57000c8b0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.445+0000 7ff58da00640 1 -- 172.21.15.83:0/554891467 shutdown_connections 2024-04-25T00:13:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.445+0000 7ff58da00640 1 --2- 172.21.15.83:0/554891467 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5640678c0 0x7ff564069d80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.445+0000 7ff58da00640 1 --2- 172.21.15.83:0/554891467 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5881555c0 0x7ff5881429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.445+0000 7ff58da00640 1 --2- 172.21.15.83:0/554891467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5881551e0 0x7ff5880b3970 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.445+0000 7ff58da00640 1 --2- 172.21.15.83:0/554891467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff58806e4c0 0x7ff5881509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.445+0000 7ff58da00640 1 -- 172.21.15.83:0/554891467 >> 172.21.15.83:0/554891467 conn(0x7ff5880b1830 msgr2=0x7ff5880713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.445+0000 7ff58da00640 1 -- 172.21.15.83:0/554891467 shutdown_connections 2024-04-25T00:13:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.445+0000 7ff58da00640 1 -- 172.21.15.83:0/554891467 wait complete. 2024-04-25T00:13:17.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:13:17.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.601+0000 7f9ff6e00640 1 Processor -- start 2024-04-25T00:13:17.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.601+0000 7f9ff6e00640 1 -- start start 2024-04-25T00:13:17.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9ff6e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ff01553e0 0x7f9ff006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9ff6e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ff00b4a70 0x7f9ff00b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9ff6e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff006ef80 con 0x7f9ff006ea00 2024-04-25T00:13:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9ff6e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff015b510 con 0x7f9ff00b4a70 2024-04-25T00:13:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9ff6e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff015b7f0 con 0x7f9ff01553e0 2024-04-25T00:13:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9fef400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9ff006ea00 0x7f9ff00b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49474/0 (socket says 172.21.15.83:49474) 2024-04-25T00:13:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9fef400640 1 -- 172.21.15.83:0/2823950479 learned_addr learned my addr 172.21.15.83:0/2823950479 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9ff4e00640 1 --2- 172.21.15.83:0/2823950479 >> v2:172.21.15.86:3300/0 conn(0x7f9ff00b4a70 0x7f9ff00b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9fefe00640 1 --2- 172.21.15.83:0/2823950479 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ff01553e0 0x7f9ff006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9feea00640 1 -- 172.21.15.83:0/2823950479 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3892244686 0 0) 0x7f9ff006ef80 con 0x7f9ff006ea00 2024-04-25T00:13:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9feea00640 1 -- 172.21.15.83:0/2823950479 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fd4003530 con 0x7f9ff006ea00 2024-04-25T00:13:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9ff4e00640 1 -- 172.21.15.83:0/2823950479 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ff01553e0 msgr2=0x7f9ff006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9ff4e00640 1 --2- 172.21.15.83:0/2823950479 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ff01553e0 0x7f9ff006e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9ff4e00640 1 -- 172.21.15.83:0/2823950479 >> v1:172.21.15.83:6789/0 conn(0x7f9ff006ea00 legacy=0x7f9ff00b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.605+0000 7f9ff4e00640 1 -- 172.21.15.83:0/2823950479 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff015bad0 con 0x7f9ff00b4a70 2024-04-25T00:13:17.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.609+0000 7f9ff4e00640 1 --2- 172.21.15.83:0/2823950479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff00b4a70 0x7f9ff00b2b30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009d90 tx=0x7f9fe801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22f678c2bb0b16e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:17.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.609+0000 7f9feea00640 1 -- 172.21.15.83:0/2823950479 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80b2070 con 0x7f9ff00b4a70 2024-04-25T00:13:17.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.609+0000 7f9feea00640 1 -- 172.21.15.83:0/2823950479 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe80a58b0 con 0x7f9ff00b4a70 2024-04-25T00:13:17.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.609+0000 7f9feea00640 1 -- 172.21.15.83:0/2823950479 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80a5c80 con 0x7f9ff00b4a70 2024-04-25T00:13:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.609+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2823950479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff00b4a70 msgr2=0x7f9ff00b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.609+0000 7f9ff6e00640 1 --2- 172.21.15.83:0/2823950479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff00b4a70 0x7f9ff00b2b30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009d90 tx=0x7f9fe801c8e0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.609+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2823950479 shutdown_connections 2024-04-25T00:13:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.609+0000 7f9ff6e00640 1 --2- 172.21.15.83:0/2823950479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff00b4a70 0x7f9ff00b2b30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.609+0000 7f9ff6e00640 1 --2- 172.21.15.83:0/2823950479 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ff01553e0 0x7f9ff006e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.609+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2823950479 >> 172.21.15.83:0/2823950479 conn(0x7f9ff00b1830 msgr2=0x7f9ff00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.609+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2823950479 shutdown_connections 2024-04-25T00:13:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.609+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2823950479 wait complete. 2024-04-25T00:13:17.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.609+0000 7f9ff6e00640 1 Processor -- start 2024-04-25T00:13:17.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.609+0000 7f9ff6e00640 1 -- start start 2024-04-25T00:13:17.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9ff6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff006ea00 0x7f9ff0154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9ff6e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff00b4a70 0x7f9ff014fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9ff6e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ff01553e0 0x7f9ff0150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9ff6e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff00b3a00 con 0x7f9ff00b4a70 2024-04-25T00:13:17.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9ff6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff00b3880 con 0x7f9ff006ea00 2024-04-25T00:13:17.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9ff6e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff00b3b80 con 0x7f9ff01553e0 2024-04-25T00:13:17.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9fef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff00b4a70 0x7f9ff014fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9fef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff00b4a70 0x7f9ff014fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55222/0 (socket says 172.21.15.83:55222) 2024-04-25T00:13:17.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9fef400640 1 -- 172.21.15.83:0/2750491588 learned_addr learned my addr 172.21.15.83:0/2750491588 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:17.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9fefe00640 1 --2- 172.21.15.83:0/2750491588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff006ea00 0x7f9ff0154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9ff4e00640 1 --2- 172.21.15.83:0/2750491588 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ff01553e0 0x7f9ff0150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9fef400640 1 -- 172.21.15.83:0/2750491588 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ff01553e0 msgr2=0x7f9ff0150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9fef400640 1 --2- 172.21.15.83:0/2750491588 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ff01553e0 0x7f9ff0150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9fef400640 1 -- 172.21.15.83:0/2750491588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff006ea00 msgr2=0x7f9ff0154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9fef400640 1 --2- 172.21.15.83:0/2750491588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff006ea00 0x7f9ff0154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9fef400640 1 -- 172.21.15.83:0/2750491588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff014c3b0 con 0x7f9ff00b4a70 2024-04-25T00:13:17.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9fef400640 1 --2- 172.21.15.83:0/2750491588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff00b4a70 0x7f9ff014fff0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c3f0 tx=0x7f9fdc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:17.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9fe7e00640 1 -- 172.21.15.83:0/2750491588 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc017070 con 0x7f9ff00b4a70 2024-04-25T00:13:17.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9fe7e00640 1 -- 172.21.15.83:0/2750491588 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc005cf0 con 0x7f9ff00b4a70 2024-04-25T00:13:17.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9fe7e00640 1 -- 172.21.15.83:0/2750491588 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0072f0 con 0x7f9ff00b4a70 2024-04-25T00:13:17.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2750491588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff014c930 con 0x7f9ff00b4a70 2024-04-25T00:13:17.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2750491588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff014ac70 con 0x7f9ff00b4a70 2024-04-25T00:13:17.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2750491588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff00737f0 con 0x7f9ff00b4a70 2024-04-25T00:13:17.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.613+0000 7f9fe7e00640 1 -- 172.21.15.83:0/2750491588 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9fdc01a070 con 0x7f9ff00b4a70 2024-04-25T00:13:17.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.617+0000 7f9fe7e00640 1 --2- 172.21.15.83:0/2750491588 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9fd00678c0 0x7f9fd0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.617+0000 7f9fe7e00640 1 -- 172.21.15.83:0/2750491588 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5249+0+0 (secure 0 0 0) 0x7f9fdc084eb0 con 0x7f9ff00b4a70 2024-04-25T00:13:17.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.617+0000 7f9fefe00640 1 --2- 172.21.15.83:0/2750491588 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9fd00678c0 0x7f9fd0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.617+0000 7f9fefe00640 1 --2- 172.21.15.83:0/2750491588 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9fd00678c0 0x7f9fd0069d80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0150c30 tx=0x7f9fe0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:17.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.621+0000 7f9fe7e00640 1 -- 172.21.15.83:0/2750491588 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9fdc055560 con 0x7f9ff00b4a70 2024-04-25T00:13:17.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.777+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2750491588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9ff004f5f0 con 0x7f9ff00b4a70 2024-04-25T00:13:17.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.777+0000 7f9fe7e00640 1 -- 172.21.15.83:0/2750491588 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) ==== 84+0+65 (secure 0 0 0) 0x7f9fdc059860 con 0x7f9ff00b4a70 2024-04-25T00:13:17.781 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:17.782 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-25T00:13:17.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:17.782 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-25T00:13:17.782 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-25T00:13:17.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:17.783 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-25T00:13:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.781+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2750491588 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9fd00678c0 msgr2=0x7f9fd0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.781+0000 7f9ff6e00640 1 --2- 172.21.15.83:0/2750491588 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9fd00678c0 0x7f9fd0069d80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0150c30 tx=0x7f9fe0009190 comp rx=0 tx=0).stop 2024-04-25T00:13:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.781+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2750491588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff00b4a70 msgr2=0x7f9ff014fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.781+0000 7f9ff6e00640 1 --2- 172.21.15.83:0/2750491588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff00b4a70 0x7f9ff014fff0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c3f0 tx=0x7f9fdc00c8b0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.781+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2750491588 shutdown_connections 2024-04-25T00:13:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.781+0000 7f9ff6e00640 1 --2- 172.21.15.83:0/2750491588 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9fd00678c0 0x7f9fd0069d80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.781+0000 7f9ff6e00640 1 --2- 172.21.15.83:0/2750491588 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ff01553e0 0x7f9ff0150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.781+0000 7f9ff6e00640 1 --2- 172.21.15.83:0/2750491588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff00b4a70 0x7f9ff014fff0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.781+0000 7f9ff6e00640 1 --2- 172.21.15.83:0/2750491588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ff006ea00 0x7f9ff0154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.781+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2750491588 >> 172.21.15.83:0/2750491588 conn(0x7f9ff00b1830 msgr2=0x7f9ff015ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.781+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2750491588 shutdown_connections 2024-04-25T00:13:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.781+0000 7f9ff6e00640 1 -- 172.21.15.83:0/2750491588 wait complete. 2024-04-25T00:13:17.802 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:13:17.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:13:17.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7feff7c00640 1 Processor -- start 2024-04-25T00:13:17.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7feff7c00640 1 -- start start 2024-04-25T00:13:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7feff7c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feff00d00d0 0x7feff00d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7feff7c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feff015f6e0 0x7feff00b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7feff7c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feff00d09f0 con 0x7feff015fb40 2024-04-25T00:13:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7feff7c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7feff0158a30 con 0x7feff00d00d0 2024-04-25T00:13:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7feff7c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7feff015f280 con 0x7feff015f6e0 2024-04-25T00:13:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7feff5200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feff00d00d0 0x7feff00d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7feff5200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feff00d00d0 0x7feff00d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55232/0 (socket says 172.21.15.83:55232) 2024-04-25T00:13:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7feff5200640 1 -- 172.21.15.83:0/1540826584 learned_addr learned my addr 172.21.15.83:0/1540826584 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7feff5200640 1 -- 172.21.15.83:0/1540826584 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feff015f6e0 msgr2=0x7feff00b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7fefefe00640 1 --2- 172.21.15.83:0/1540826584 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feff015f6e0 0x7feff00b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7feff5200640 1 --2- 172.21.15.83:0/1540826584 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feff015f6e0 0x7feff00b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7feff5200640 1 -- 172.21.15.83:0/1540826584 >> v1:172.21.15.86:6789/0 conn(0x7feff015fb40 legacy=0x7feff015e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7feff5200640 1 -- 172.21.15.83:0/1540826584 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff00b3530 con 0x7feff00d00d0 2024-04-25T00:13:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.941+0000 7fefefe00640 1 --2- 172.21.15.83:0/1540826584 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feff015f6e0 0x7feff00b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:17.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.945+0000 7feff5200640 1 --2- 172.21.15.83:0/1540826584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00d00d0 0x7feff00d04b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009870 tx=0x7fefe001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7969d59b40e8163c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:17.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.945+0000 7fefef400640 1 -- 172.21.15.83:0/1540826584 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe00b1070 con 0x7feff00d00d0 2024-04-25T00:13:17.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.945+0000 7fefef400640 1 -- 172.21.15.83:0/1540826584 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe00ac620 con 0x7feff00d00d0 2024-04-25T00:13:17.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.945+0000 7fefef400640 1 -- 172.21.15.83:0/1540826584 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe00ac9f0 con 0x7feff00d00d0 2024-04-25T00:13:17.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.945+0000 7feff7c00640 1 -- 172.21.15.83:0/1540826584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00d00d0 msgr2=0x7feff00d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.945+0000 7feff7c00640 1 --2- 172.21.15.83:0/1540826584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00d00d0 0x7feff00d04b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009870 tx=0x7fefe001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.945+0000 7feff7c00640 1 -- 172.21.15.83:0/1540826584 shutdown_connections 2024-04-25T00:13:17.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.945+0000 7feff7c00640 1 --2- 172.21.15.83:0/1540826584 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feff015f6e0 0x7feff00b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.945+0000 7feff7c00640 1 --2- 172.21.15.83:0/1540826584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00d00d0 0x7feff00d04b0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.945+0000 7feff7c00640 1 -- 172.21.15.83:0/1540826584 >> 172.21.15.83:0/1540826584 conn(0x7feff006b1e0 msgr2=0x7feff006b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:17.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.945+0000 7feff7c00640 1 -- 172.21.15.83:0/1540826584 shutdown_connections 2024-04-25T00:13:17.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.945+0000 7feff7c00640 1 -- 172.21.15.83:0/1540826584 wait complete. 2024-04-25T00:13:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff7c00640 1 Processor -- start 2024-04-25T00:13:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff7c00640 1 -- start start 2024-04-25T00:13:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff7c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feff00d00d0 0x7feff0139650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff7c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff015f6e0 0x7feff00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff7c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff015fb40 0x7feff00b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff7c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feff006ff60 con 0x7feff015fb40 2024-04-25T00:13:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff7c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feff006fde0 con 0x7feff015f6e0 2024-04-25T00:13:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff7c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7feff016b370 con 0x7feff00d00d0 2024-04-25T00:13:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff015fb40 0x7feff00b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff015fb40 0x7feff00b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55244/0 (socket says 172.21.15.83:55244) 2024-04-25T00:13:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff5c00640 1 -- 172.21.15.83:0/1349629252 learned_addr learned my addr 172.21.15.83:0/1349629252 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff5200640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feff00d00d0 0x7feff0139650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7fefefe00640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff015f6e0 0x7feff00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff5c00640 1 -- 172.21.15.83:0/1349629252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feff00d00d0 msgr2=0x7feff0139650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff5c00640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feff00d00d0 0x7feff0139650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff5c00640 1 -- 172.21.15.83:0/1349629252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff015f6e0 msgr2=0x7feff00b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff5c00640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff015f6e0 0x7feff00b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff5c00640 1 -- 172.21.15.83:0/1349629252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff00cfb70 con 0x7feff015fb40 2024-04-25T00:13:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7fefefe00640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff015f6e0 0x7feff00b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff5c00640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff015fb40 0x7feff00b25d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fefe800c7b0 tx=0x7fefe800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7fefecc00640 1 -- 172.21.15.83:0/1349629252 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8004b70 con 0x7feff015fb40 2024-04-25T00:13:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7fefecc00640 1 -- 172.21.15.83:0/1349629252 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe8004d10 con 0x7feff015fb40 2024-04-25T00:13:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff7c00640 1 -- 172.21.15.83:0/1349629252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff00cedb0 con 0x7feff015fb40 2024-04-25T00:13:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7fefecc00640 1 -- 172.21.15.83:0/1349629252 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe800aa90 con 0x7feff015fb40 2024-04-25T00:13:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.949+0000 7feff7c00640 1 -- 172.21.15.83:0/1349629252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff00cf370 con 0x7feff015fb40 2024-04-25T00:13:17.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.953+0000 7feff7c00640 1 -- 172.21.15.83:0/1349629252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff0072fa0 con 0x7feff015fb40 2024-04-25T00:13:17.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.953+0000 7fefecc00640 1 -- 172.21.15.83:0/1349629252 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fefe800ac30 con 0x7feff015fb40 2024-04-25T00:13:17.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.953+0000 7fefecc00640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fefd00678c0 0x7fefd0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.953+0000 7fefecc00640 1 -- 172.21.15.83:0/1349629252 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5249+0+0 (secure 0 0 0) 0x7fefe8085d40 con 0x7feff015fb40 2024-04-25T00:13:17.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.953+0000 7feff5200640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fefd00678c0 0x7fefd0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.953+0000 7feff5200640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fefd00678c0 0x7fefd0069d80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fefe00ae000 tx=0x7fefe00a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:17.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:17.957+0000 7fefecc00640 1 -- 172.21.15.83:0/1349629252 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fefe8020070 con 0x7feff015fb40 2024-04-25T00:13:18.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.113+0000 7feff7c00640 1 -- 172.21.15.83:0/1349629252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7feff004f5f0 con 0x7feff015fb40 2024-04-25T00:13:18.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.409+0000 7fefecc00640 1 -- 172.21.15.83:0/1349629252 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) ==== 138+0+0 (secure 0 0 0) 0x7fefe80569a0 con 0x7feff015fb40 2024-04-25T00:13:18.411 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-25T00:13:18.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.409+0000 7feff7c00640 1 -- 172.21.15.83:0/1349629252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fefd00678c0 msgr2=0x7fefd0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:18.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.409+0000 7feff7c00640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fefd00678c0 0x7fefd0069d80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fefe00ae000 tx=0x7fefe00a7000 comp rx=0 tx=0).stop 2024-04-25T00:13:18.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.409+0000 7feff7c00640 1 -- 172.21.15.83:0/1349629252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff015fb40 msgr2=0x7feff00b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:18.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.409+0000 7feff7c00640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff015fb40 0x7feff00b25d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fefe800c7b0 tx=0x7fefe800cc70 comp rx=0 tx=0).stop 2024-04-25T00:13:18.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.413+0000 7feff7c00640 1 -- 172.21.15.83:0/1349629252 shutdown_connections 2024-04-25T00:13:18.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.413+0000 7feff7c00640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fefd00678c0 0x7fefd0069d80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:18.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.413+0000 7feff7c00640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff015fb40 0x7feff00b25d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:18.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.413+0000 7feff7c00640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff015f6e0 0x7feff00b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:18.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.413+0000 7feff7c00640 1 --2- 172.21.15.83:0/1349629252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feff00d00d0 0x7feff0139650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:18.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.413+0000 7feff7c00640 1 -- 172.21.15.83:0/1349629252 >> 172.21.15.83:0/1349629252 conn(0x7feff006b1e0 msgr2=0x7feff015e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:18.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.413+0000 7feff7c00640 1 -- 172.21.15.83:0/1349629252 shutdown_connections 2024-04-25T00:13:18.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.413+0000 7feff7c00640 1 -- 172.21.15.83:0/1349629252 wait complete. 2024-04-25T00:13:18.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:13:18.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.577+0000 7f84f8200640 1 Processor -- start 2024-04-25T00:13:18.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.577+0000 7f84f8200640 1 -- start start 2024-04-25T00:13:18.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84f8200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f84f01551e0 0x7f84f015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:18.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84f8200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84f006e4c0 0x7f84f006e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:18.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84f8200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84f015c560 con 0x7f84f01555c0 2024-04-25T00:13:18.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84f8200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f84f0072460 con 0x7f84f01551e0 2024-04-25T00:13:18.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84f8200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f84f015c7d0 con 0x7f84f006e4c0 2024-04-25T00:13:18.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84f5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f84f01551e0 0x7f84f015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:18.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84f6200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84f006e4c0 0x7f84f006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:18.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84f5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f84f01551e0 0x7f84f015bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:33250/0 (socket says 172.21.15.83:33250) 2024-04-25T00:13:18.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84f4e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f84f01555c0 0x7f84f0071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:42590/0 (socket says 172.21.15.83:42590) 2024-04-25T00:13:18.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84f5800640 1 -- 172.21.15.83:0/183400971 learned_addr learned my addr 172.21.15.83:0/183400971 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:18.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84f5800640 1 -- 172.21.15.83:0/183400971 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84f006e4c0 msgr2=0x7f84f006e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:18.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84f5800640 1 --2- 172.21.15.83:0/183400971 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84f006e4c0 0x7f84f006e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:18.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84f5800640 1 -- 172.21.15.83:0/183400971 >> v1:172.21.15.86:6789/0 conn(0x7f84f01555c0 legacy=0x7f84f0071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:18.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84f5800640 1 -- 172.21.15.83:0/183400971 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f015cab0 con 0x7f84f01551e0 2024-04-25T00:13:18.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.581+0000 7f84efe00640 1 -- 172.21.15.83:0/183400971 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3962344293 0 0) 0x7f84f015c560 con 0x7f84f01555c0 2024-04-25T00:13:18.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.585+0000 7f84f5800640 1 --2- 172.21.15.83:0/183400971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f01551e0 0x7f84f015bea0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f84e40098a0 tx=0x7f84e401c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a8125593f72fbce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.585+0000 7f84efe00640 1 -- 172.21.15.83:0/183400971 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e40b0070 con 0x7f84f01551e0 2024-04-25T00:13:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.585+0000 7f84efe00640 1 -- 172.21.15.83:0/183400971 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e40ab600 con 0x7f84f01551e0 2024-04-25T00:13:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.585+0000 7f84efe00640 1 -- 172.21.15.83:0/183400971 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e40ab9d0 con 0x7f84f01551e0 2024-04-25T00:13:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.585+0000 7f84f8200640 1 -- 172.21.15.83:0/183400971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f01551e0 msgr2=0x7f84f015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.585+0000 7f84f8200640 1 --2- 172.21.15.83:0/183400971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f01551e0 0x7f84f015bea0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f84e40098a0 tx=0x7f84e401c540 comp rx=0 tx=0).stop 2024-04-25T00:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.585+0000 7f84f8200640 1 -- 172.21.15.83:0/183400971 shutdown_connections 2024-04-25T00:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.585+0000 7f84f8200640 1 --2- 172.21.15.83:0/183400971 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84f006e4c0 0x7f84f006e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.585+0000 7f84f8200640 1 --2- 172.21.15.83:0/183400971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f01551e0 0x7f84f015bea0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.585+0000 7f84f8200640 1 -- 172.21.15.83:0/183400971 >> 172.21.15.83:0/183400971 conn(0x7f84f00b1830 msgr2=0x7f84f00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.585+0000 7f84f8200640 1 -- 172.21.15.83:0/183400971 shutdown_connections 2024-04-25T00:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.585+0000 7f84f8200640 1 -- 172.21.15.83:0/183400971 wait complete. 2024-04-25T00:13:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.585+0000 7f84f8200640 1 Processor -- start 2024-04-25T00:13:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.585+0000 7f84f8200640 1 -- start start 2024-04-25T00:13:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f8200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84f006e4c0 0x7f84f0145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f8200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f01551e0 0x7f84f0153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f8200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84f01555c0 0x7f84f0146820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f8200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84f015dbb0 con 0x7f84f01555c0 2024-04-25T00:13:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f8200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84f015da30 con 0x7f84f01551e0 2024-04-25T00:13:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f8200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f84f015dd30 con 0x7f84f006e4c0 2024-04-25T00:13:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f6200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84f01555c0 0x7f84f0146820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f6200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84f01555c0 0x7f84f0146820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55256/0 (socket says 172.21.15.83:55256) 2024-04-25T00:13:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f6200640 1 -- 172.21.15.83:0/2110738166 learned_addr learned my addr 172.21.15.83:0/2110738166 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f4e00640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f01551e0 0x7f84f0153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f5800640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84f006e4c0 0x7f84f0145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f6200640 1 -- 172.21.15.83:0/2110738166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84f006e4c0 msgr2=0x7f84f0145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f6200640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84f006e4c0 0x7f84f0145630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f6200640 1 -- 172.21.15.83:0/2110738166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f01551e0 msgr2=0x7f84f0153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f6200640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f01551e0 0x7f84f0153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f6200640 1 -- 172.21.15.83:0/2110738166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f01525b0 con 0x7f84f01555c0 2024-04-25T00:13:18.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f5800640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84f006e4c0 0x7f84f0145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:18.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f6200640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84f01555c0 0x7f84f0146820 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f84e800efc0 tx=0x7f84e800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:18.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84ed600640 1 -- 172.21.15.83:0/2110738166 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8018020 con 0x7f84f01555c0 2024-04-25T00:13:18.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84ed600640 1 -- 172.21.15.83:0/2110738166 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e8014480 con 0x7f84f01555c0 2024-04-25T00:13:18.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f8200640 1 -- 172.21.15.83:0/2110738166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f0152890 con 0x7f84f01555c0 2024-04-25T00:13:18.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84ed600640 1 -- 172.21.15.83:0/2110738166 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8014870 con 0x7f84f01555c0 2024-04-25T00:13:18.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f8200640 1 -- 172.21.15.83:0/2110738166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f0152e50 con 0x7f84f01555c0 2024-04-25T00:13:18.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.589+0000 7f84f8200640 1 -- 172.21.15.83:0/2110738166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f004f510 con 0x7f84f01555c0 2024-04-25T00:13:18.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.593+0000 7f84ed600640 1 -- 172.21.15.83:0/2110738166 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f84e801b020 con 0x7f84f01555c0 2024-04-25T00:13:18.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.593+0000 7f84ed600640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f84d00678c0 0x7f84d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:18.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.593+0000 7f84ed600640 1 -- 172.21.15.83:0/2110738166 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5619+0+0 (secure 0 0 0) 0x7f84e8086820 con 0x7f84f01555c0 2024-04-25T00:13:18.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.593+0000 7f84f5800640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f84d00678c0 0x7f84d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:18.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.593+0000 7f84f5800640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f84d00678c0 0x7f84d0069d80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009870 tx=0x7f84e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:18.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.597+0000 7f84ed600640 1 -- 172.21.15.83:0/2110738166 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f84e8022070 con 0x7f84f01555c0 2024-04-25T00:13:18.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:18.753+0000 7f84f8200640 1 -- 172.21.15.83:0/2110738166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f84f006ce30 con 0x7f84f01555c0 2024-04-25T00:13:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.413+0000 7f84ed600640 1 -- 172.21.15.83:0/2110738166 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) ==== 170+0+0 (secure 0 0 0) 0x7f84e8010040 con 0x7f84f01555c0 2024-04-25T00:13:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-25T00:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.413+0000 7f84f8200640 1 -- 172.21.15.83:0/2110738166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f84d00678c0 msgr2=0x7f84d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.413+0000 7f84f8200640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f84d00678c0 0x7f84d0069d80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009870 tx=0x7f84e4008040 comp rx=0 tx=0).stop 2024-04-25T00:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.413+0000 7f84f8200640 1 -- 172.21.15.83:0/2110738166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84f01555c0 msgr2=0x7f84f0146820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.413+0000 7f84f8200640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84f01555c0 0x7f84f0146820 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f84e800efc0 tx=0x7f84e800c490 comp rx=0 tx=0).stop 2024-04-25T00:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.413+0000 7f84f8200640 1 -- 172.21.15.83:0/2110738166 shutdown_connections 2024-04-25T00:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.413+0000 7f84f8200640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f84d00678c0 0x7f84d0069d80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.413+0000 7f84f8200640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84f01555c0 0x7f84f0146820 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.413+0000 7f84f8200640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f01551e0 0x7f84f0153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.413+0000 7f84f8200640 1 --2- 172.21.15.83:0/2110738166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84f006e4c0 0x7f84f0145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.413+0000 7f84f8200640 1 -- 172.21.15.83:0/2110738166 >> 172.21.15.83:0/2110738166 conn(0x7f84f00b1830 msgr2=0x7f84f0151790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.417+0000 7f84f8200640 1 -- 172.21.15.83:0/2110738166 shutdown_connections 2024-04-25T00:13:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.417+0000 7f84f8200640 1 -- 172.21.15.83:0/2110738166 wait complete. 2024-04-25T00:13:19.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:13:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.593+0000 7fa555000640 1 Processor -- start 2024-04-25T00:13:19.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.593+0000 7fa555000640 1 -- start start 2024-04-25T00:13:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.597+0000 7fa555000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa550159810 0x7fa5500cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.597+0000 7fa555000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa55006e950 0x7fa55015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.597+0000 7fa555000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5500ce000 con 0x7fa55015fe50 2024-04-25T00:13:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.597+0000 7fa555000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa55006e210 con 0x7fa55006e950 2024-04-25T00:13:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.597+0000 7fa555000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa55006da00 con 0x7fa550159810 2024-04-25T00:13:19.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.597+0000 7fa54de00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa550159810 0x7fa5500cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:19.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.597+0000 7fa54de00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa550159810 0x7fa5500cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:39098/0 (socket says 172.21.15.83:39098) 2024-04-25T00:13:19.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.597+0000 7fa545400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa55006e950 0x7fa55015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:19.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.597+0000 7fa54de00640 1 -- 172.21.15.83:0/480259801 learned_addr learned my addr 172.21.15.83:0/480259801 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:19.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.597+0000 7fa545400640 1 -- 172.21.15.83:0/480259801 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa550159810 msgr2=0x7fa5500cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:19.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.597+0000 7fa545400640 1 --2- 172.21.15.83:0/480259801 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa550159810 0x7fa5500cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.597+0000 7fa545400640 1 -- 172.21.15.83:0/480259801 >> v1:172.21.15.86:6789/0 conn(0x7fa55015fe50 legacy=0x7fa55015c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:19.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.597+0000 7fa545400640 1 -- 172.21.15.83:0/480259801 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55006eec0 con 0x7fa55006e950 2024-04-25T00:13:19.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.597+0000 7fa54de00640 1 --2- 172.21.15.83:0/480259801 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa550159810 0x7fa5500cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:19.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.601+0000 7fa545400640 1 --2- 172.21.15.83:0/480259801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55006e950 0x7fa55015f910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5380097c0 tx=0x7fa53801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d69c41307e8a1625 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:19.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.601+0000 7fa54d400640 1 -- 172.21.15.83:0/480259801 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5380b0070 con 0x7fa55006e950 2024-04-25T00:13:19.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.601+0000 7fa54d400640 1 -- 172.21.15.83:0/480259801 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5380ab600 con 0x7fa55006e950 2024-04-25T00:13:19.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.601+0000 7fa54d400640 1 -- 172.21.15.83:0/480259801 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5380ab9d0 con 0x7fa55006e950 2024-04-25T00:13:19.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.601+0000 7fa555000640 1 -- 172.21.15.83:0/480259801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55006e950 msgr2=0x7fa55015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:19.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.601+0000 7fa555000640 1 --2- 172.21.15.83:0/480259801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55006e950 0x7fa55015f910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5380097c0 tx=0x7fa53801c410 comp rx=0 tx=0).stop 2024-04-25T00:13:19.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.601+0000 7fa555000640 1 -- 172.21.15.83:0/480259801 shutdown_connections 2024-04-25T00:13:19.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.601+0000 7fa555000640 1 --2- 172.21.15.83:0/480259801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55006e950 0x7fa55015f910 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.601+0000 7fa555000640 1 --2- 172.21.15.83:0/480259801 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa550159810 0x7fa5500cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.601+0000 7fa555000640 1 -- 172.21.15.83:0/480259801 >> 172.21.15.83:0/480259801 conn(0x7fa5500b1830 msgr2=0x7fa5500b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:19.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.601+0000 7fa555000640 1 -- 172.21.15.83:0/480259801 shutdown_connections 2024-04-25T00:13:19.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.601+0000 7fa555000640 1 -- 172.21.15.83:0/480259801 wait complete. 2024-04-25T00:13:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.601+0000 7fa555000640 1 Processor -- start 2024-04-25T00:13:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.601+0000 7fa555000640 1 -- start start 2024-04-25T00:13:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa555000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa55006e950 0x7fa55014f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa555000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa550159810 0x7fa550150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa555000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55015fe50 0x7fa5500710d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa555000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa550159d70 con 0x7fa55006e950 2024-04-25T00:13:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa555000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa550159bf0 con 0x7fa55015fe50 2024-04-25T00:13:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa555000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa550070da0 con 0x7fa550159810 2024-04-25T00:13:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa54de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa55006e950 0x7fa55014f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa54de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa55006e950 0x7fa55014f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55272/0 (socket says 172.21.15.83:55272) 2024-04-25T00:13:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa54de00640 1 -- 172.21.15.83:0/2348172511 learned_addr learned my addr 172.21.15.83:0/2348172511 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:19.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa54e800640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55015fe50 0x7fa5500710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa545400640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa550159810 0x7fa550150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa54de00640 1 -- 172.21.15.83:0/2348172511 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa550159810 msgr2=0x7fa550150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa54de00640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa550159810 0x7fa550150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa54de00640 1 -- 172.21.15.83:0/2348172511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55015fe50 msgr2=0x7fa5500710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa54de00640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55015fe50 0x7fa5500710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa54de00640 1 -- 172.21.15.83:0/2348172511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5501554e0 con 0x7fa55006e950 2024-04-25T00:13:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa545400640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa550159810 0x7fa550150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa54e800640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55015fe50 0x7fa5500710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa54de00640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa55006e950 0x7fa55014f4a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa54000efc0 tx=0x7fa54000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa546a00640 1 -- 172.21.15.83:0/2348172511 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54000ce60 con 0x7fa55006e950 2024-04-25T00:13:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa546a00640 1 -- 172.21.15.83:0/2348172511 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa540019070 con 0x7fa55006e950 2024-04-25T00:13:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa555000640 1 -- 172.21.15.83:0/2348172511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa550151590 con 0x7fa55006e950 2024-04-25T00:13:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa546a00640 1 -- 172.21.15.83:0/2348172511 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540014680 con 0x7fa55006e950 2024-04-25T00:13:19.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa555000640 1 -- 172.21.15.83:0/2348172511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa550151b50 con 0x7fa55006e950 2024-04-25T00:13:19.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa555000640 1 -- 172.21.15.83:0/2348172511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa55004f510 con 0x7fa55006e950 2024-04-25T00:13:19.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.605+0000 7fa546a00640 1 -- 172.21.15.83:0/2348172511 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa540012050 con 0x7fa55006e950 2024-04-25T00:13:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.609+0000 7fa546a00640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa52c0678c0 0x7fa52c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.609+0000 7fa546a00640 1 -- 172.21.15.83:0/2348172511 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6001+0+0 (secure 0 0 0) 0x7fa540086ee0 con 0x7fa55006e950 2024-04-25T00:13:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.609+0000 7fa545400640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa52c0678c0 0x7fa52c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.609+0000 7fa545400640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa52c0678c0 0x7fa52c069d80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa5380ad000 tx=0x7fa538008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.613+0000 7fa546a00640 1 -- 172.21.15.83:0/2348172511 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa540010040 con 0x7fa55006e950 2024-04-25T00:13:19.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:19.801+0000 7fa555000640 1 -- 172.21.15.83:0/2348172511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa550072c90 con 0x7fa55006e950 2024-04-25T00:13:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.425+0000 7fa546a00640 1 -- 172.21.15.83:0/2348172511 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v32) ==== 173+0+0 (secure 0 0 0) 0x7fa540057840 con 0x7fa55006e950 2024-04-25T00:13:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T00:13:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.429+0000 7fa555000640 1 -- 172.21.15.83:0/2348172511 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa52c0678c0 msgr2=0x7fa52c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.429+0000 7fa555000640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa52c0678c0 0x7fa52c069d80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa5380ad000 tx=0x7fa538008040 comp rx=0 tx=0).stop 2024-04-25T00:13:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.429+0000 7fa555000640 1 -- 172.21.15.83:0/2348172511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa55006e950 msgr2=0x7fa55014f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.429+0000 7fa555000640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa55006e950 0x7fa55014f4a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa54000efc0 tx=0x7fa54000c490 comp rx=0 tx=0).stop 2024-04-25T00:13:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.429+0000 7fa555000640 1 -- 172.21.15.83:0/2348172511 shutdown_connections 2024-04-25T00:13:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.429+0000 7fa555000640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa52c0678c0 0x7fa52c069d80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.429+0000 7fa555000640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa550159810 0x7fa550150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.429+0000 7fa555000640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55015fe50 0x7fa5500710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.429+0000 7fa555000640 1 --2- 172.21.15.83:0/2348172511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa55006e950 0x7fa55014f4a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.429+0000 7fa555000640 1 -- 172.21.15.83:0/2348172511 >> 172.21.15.83:0/2348172511 conn(0x7fa5500b1830 msgr2=0x7fa55015deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.429+0000 7fa555000640 1 -- 172.21.15.83:0/2348172511 shutdown_connections 2024-04-25T00:13:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.429+0000 7fa555000640 1 -- 172.21.15.83:0/2348172511 wait complete. 2024-04-25T00:13:20.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:20.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13e6200640 1 Processor -- start 2024-04-25T00:13:20.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13e6200640 1 -- start start 2024-04-25T00:13:20.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13e6200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f13e015e140 0x7f13e00d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13e6200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f13e00d0df0 0x7f13e00d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13e6200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13e00d28f0 0x7f13e00ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13e6200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f13e00d4c20 con 0x7f13e00d0df0 2024-04-25T00:13:20.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13e6200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f13e00c7920 con 0x7f13e015e140 2024-04-25T00:13:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13e6200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f13e00d2d80 con 0x7f13e00d28f0 2024-04-25T00:13:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13d7e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f13e00d0df0 0x7f13e00d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13d7e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f13e00d0df0 0x7f13e00d23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55280/0 (socket says 172.21.15.83:55280) 2024-04-25T00:13:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13d7e00640 1 -- 172.21.15.83:0/2917215120 learned_addr learned my addr 172.21.15.83:0/2917215120 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13e5800640 1 --2- 172.21.15.83:0/2917215120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13e00d28f0 0x7f13e00ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13e4e00640 1 --2- 172.21.15.83:0/2917215120 >> v2:172.21.15.86:3300/0 conn(0x7f13e015e140 0x7f13e00d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13d7e00640 1 -- 172.21.15.83:0/2917215120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13e00d28f0 msgr2=0x7f13e00ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13d7e00640 1 --2- 172.21.15.83:0/2917215120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13e00d28f0 0x7f13e00ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13d7e00640 1 -- 172.21.15.83:0/2917215120 >> v2:172.21.15.86:3300/0 conn(0x7f13e015e140 msgr2=0x7f13e00d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13d7e00640 1 --2- 172.21.15.83:0/2917215120 >> v2:172.21.15.86:3300/0 conn(0x7f13e015e140 0x7f13e00d1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13d7e00640 1 -- 172.21.15.83:0/2917215120 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e00d12d0 con 0x7f13e00d0df0 2024-04-25T00:13:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13e5800640 1 --2- 172.21.15.83:0/2917215120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13e00d28f0 0x7f13e00ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.621+0000 7f13e4e00640 1 --2- 172.21.15.83:0/2917215120 >> v2:172.21.15.86:3300/0 conn(0x7f13e015e140 0x7f13e00d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:20.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13d7e00640 1 --2- 172.21.15.83:0/2917215120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e00d0df0 0x7f13e00d23b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f13e008c630 tx=0x7f13c801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4960712390656a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:20.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13dfe00640 1 -- 172.21.15.83:0/2917215120 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c80ab620 con 0x7f13e00d0df0 2024-04-25T00:13:20.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13dfe00640 1 -- 172.21.15.83:0/2917215120 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c80b0070 con 0x7f13e00d0df0 2024-04-25T00:13:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13dfe00640 1 -- 172.21.15.83:0/2917215120 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c80ab9e0 con 0x7f13e00d0df0 2024-04-25T00:13:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13e6200640 1 -- 172.21.15.83:0/2917215120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e00d0df0 msgr2=0x7f13e00d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13e6200640 1 --2- 172.21.15.83:0/2917215120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e00d0df0 0x7f13e00d23b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f13e008c630 tx=0x7f13c801c460 comp rx=0 tx=0).stop 2024-04-25T00:13:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13e6200640 1 -- 172.21.15.83:0/2917215120 shutdown_connections 2024-04-25T00:13:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13e6200640 1 --2- 172.21.15.83:0/2917215120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13e00d28f0 0x7f13e00ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13e6200640 1 --2- 172.21.15.83:0/2917215120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e00d0df0 0x7f13e00d23b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13e6200640 1 --2- 172.21.15.83:0/2917215120 >> v2:172.21.15.86:3300/0 conn(0x7f13e015e140 0x7f13e00d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13e6200640 1 -- 172.21.15.83:0/2917215120 >> 172.21.15.83:0/2917215120 conn(0x7f13e00d3dd0 msgr2=0x7f13e00d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13e6200640 1 -- 172.21.15.83:0/2917215120 shutdown_connections 2024-04-25T00:13:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13e6200640 1 -- 172.21.15.83:0/2917215120 wait complete. 2024-04-25T00:13:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13e6200640 1 Processor -- start 2024-04-25T00:13:20.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.625+0000 7f13e6200640 1 -- start start 2024-04-25T00:13:20.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13e6200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13e00d0df0 0x7f13e01561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13e6200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e00d28f0 0x7f13e0153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13e6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e015e140 0x7f13e01fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13e6200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13e015e520 con 0x7f13e00d28f0 2024-04-25T00:13:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13e6200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13e00d20f0 con 0x7f13e015e140 2024-04-25T00:13:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13e6200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f13e00c8370 con 0x7f13e00d0df0 2024-04-25T00:13:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13d7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e00d28f0 0x7f13e0153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13d7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e00d28f0 0x7f13e0153400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55296/0 (socket says 172.21.15.83:55296) 2024-04-25T00:13:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13d7e00640 1 -- 172.21.15.83:0/2991458806 learned_addr learned my addr 172.21.15.83:0/2991458806 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13e4e00640 1 --2- 172.21.15.83:0/2991458806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13e00d0df0 0x7f13e01561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13e5800640 1 --2- 172.21.15.83:0/2991458806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e015e140 0x7f13e01fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13d7e00640 1 -- 172.21.15.83:0/2991458806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13e00d0df0 msgr2=0x7f13e01561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13d7e00640 1 --2- 172.21.15.83:0/2991458806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13e00d0df0 0x7f13e01561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13d7e00640 1 -- 172.21.15.83:0/2991458806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e015e140 msgr2=0x7f13e01fa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13d7e00640 1 --2- 172.21.15.83:0/2991458806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e015e140 0x7f13e01fa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13d7e00640 1 -- 172.21.15.83:0/2991458806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e00c9d60 con 0x7f13e00d28f0 2024-04-25T00:13:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13d7e00640 1 --2- 172.21.15.83:0/2991458806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e00d28f0 0x7f13e0153400 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f13c801ab10 tx=0x7f13c801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13dd600640 1 -- 172.21.15.83:0/2991458806 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c80abdb0 con 0x7f13e00d28f0 2024-04-25T00:13:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13dd600640 1 -- 172.21.15.83:0/2991458806 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c80be3f0 con 0x7f13e00d28f0 2024-04-25T00:13:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13dd600640 1 -- 172.21.15.83:0/2991458806 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c80be7a0 con 0x7f13e00d28f0 2024-04-25T00:13:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13e6200640 1 -- 172.21.15.83:0/2991458806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e00d2720 con 0x7f13e00d28f0 2024-04-25T00:13:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13e6200640 1 -- 172.21.15.83:0/2991458806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e01541a0 con 0x7f13e00d28f0 2024-04-25T00:13:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13e6200640 1 -- 172.21.15.83:0/2991458806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e004f510 con 0x7f13e00d28f0 2024-04-25T00:13:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.629+0000 7f13dd600640 1 -- 172.21.15.83:0/2991458806 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f13c80c7440 con 0x7f13e00d28f0 2024-04-25T00:13:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.633+0000 7f13dd600640 1 --2- 172.21.15.83:0/2991458806 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f13bc0678c0 0x7f13bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.633+0000 7f13dd600640 1 -- 172.21.15.83:0/2991458806 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6101+0+0 (secure 0 0 0) 0x7f13c8120320 con 0x7f13e00d28f0 2024-04-25T00:13:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.633+0000 7f13e4e00640 1 --2- 172.21.15.83:0/2991458806 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f13bc0678c0 0x7f13bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.633+0000 7f13e4e00640 1 --2- 172.21.15.83:0/2991458806 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f13bc0678c0 0x7f13bc069d80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f13d00045f0 tx=0x7f13d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.637+0000 7f13dd600640 1 -- 172.21.15.83:0/2991458806 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f13c8016110 con 0x7f13e00d28f0 2024-04-25T00:13:20.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.793+0000 7f13e6200640 1 -- 172.21.15.83:0/2991458806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13e00c8cb0 con 0x7f13e00d28f0 2024-04-25T00:13:20.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.793+0000 7f13dd600640 1 -- 172.21.15.83:0/2991458806 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16813 (secure 0 0 0) 0x7f13c80f0670 con 0x7f13e00d28f0 2024-04-25T00:13:20.796 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:20.796 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":59,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:19.804989+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:18.160239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:18.800529+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:20.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.797+0000 7f13e6200640 1 -- 172.21.15.83:0/2991458806 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f13bc0678c0 msgr2=0x7f13bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.797+0000 7f13e6200640 1 --2- 172.21.15.83:0/2991458806 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f13bc0678c0 0x7f13bc069d80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f13d00045f0 tx=0x7f13d0008040 comp rx=0 tx=0).stop 2024-04-25T00:13:20.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.797+0000 7f13e6200640 1 -- 172.21.15.83:0/2991458806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e00d28f0 msgr2=0x7f13e0153400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.797+0000 7f13e6200640 1 --2- 172.21.15.83:0/2991458806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e00d28f0 0x7f13e0153400 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f13c801ab10 tx=0x7f13c801aca0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.797+0000 7f13e6200640 1 -- 172.21.15.83:0/2991458806 shutdown_connections 2024-04-25T00:13:20.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.797+0000 7f13e6200640 1 --2- 172.21.15.83:0/2991458806 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f13bc0678c0 0x7f13bc069d80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.797+0000 7f13e6200640 1 --2- 172.21.15.83:0/2991458806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e015e140 0x7f13e01fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.797+0000 7f13e6200640 1 --2- 172.21.15.83:0/2991458806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e00d28f0 0x7f13e0153400 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.797+0000 7f13e6200640 1 --2- 172.21.15.83:0/2991458806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13e00d0df0 0x7f13e01561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.797+0000 7f13e6200640 1 -- 172.21.15.83:0/2991458806 >> 172.21.15.83:0/2991458806 conn(0x7f13e00d3dd0 msgr2=0x7f13e00d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:20.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.797+0000 7f13e6200640 1 -- 172.21.15.83:0/2991458806 shutdown_connections 2024-04-25T00:13:20.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.797+0000 7f13e6200640 1 -- 172.21.15.83:0/2991458806 wait complete. 2024-04-25T00:13:20.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:20.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.949+0000 7faefe000640 1 Processor -- start 2024-04-25T00:13:20.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.949+0000 7faefe000640 1 -- start start 2024-04-25T00:13:20.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.953+0000 7faefe000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef80b4360 0x7faef80b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.953+0000 7faefe000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faef806ef80 con 0x7faef80b2b30 2024-04-25T00:13:20.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.953+0000 7faefe000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faef815b510 con 0x7faef81553e0 2024-04-25T00:13:20.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.953+0000 7faefe000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7faef815b7f0 con 0x7faef80b4360 2024-04-25T00:13:20.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.953+0000 7faef7e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7faef80b2b30 0x7faef80b2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49490/0 (socket says 172.21.15.83:49490) 2024-04-25T00:13:20.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.953+0000 7faef7e00640 1 -- 172.21.15.83:0/2876700475 learned_addr learned my addr 172.21.15.83:0/2876700475 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:20.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.953+0000 7faef6a00640 1 --2- 172.21.15.83:0/2876700475 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef80b4360 0x7faef80b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.953+0000 7faef6000640 1 -- 172.21.15.83:0/2876700475 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2505847794 0 0) 0x7faef806ef80 con 0x7faef80b2b30 2024-04-25T00:13:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.953+0000 7faef6000640 1 -- 172.21.15.83:0/2876700475 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faedc005740 con 0x7faef80b2b30 2024-04-25T00:13:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.953+0000 7faef6a00640 1 -- 172.21.15.83:0/2876700475 >> v1:172.21.15.86:6789/0 conn(0x7faef81553e0 legacy=0x7faef806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.953+0000 7faef6000640 1 -- 172.21.15.83:0/2876700475 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1838671970 0 0) 0x7faef815b510 con 0x7faef81553e0 2024-04-25T00:13:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.953+0000 7faef6a00640 1 -- 172.21.15.83:0/2876700475 >> v1:172.21.15.83:6789/0 conn(0x7faef80b2b30 legacy=0x7faef80b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.953+0000 7faef6a00640 1 -- 172.21.15.83:0/2876700475 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef815bad0 con 0x7faef80b4360 2024-04-25T00:13:20.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faef6a00640 1 --2- 172.21.15.83:0/2876700475 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef80b4360 0x7faef80b47e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faee40097f0 tx=0x7faee401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7932ff46219ca43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:20.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faef6000640 1 -- 172.21.15.83:0/2876700475 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee40b0070 con 0x7faef80b4360 2024-04-25T00:13:20.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faef6000640 1 -- 172.21.15.83:0/2876700475 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee40ab600 con 0x7faef80b4360 2024-04-25T00:13:20.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faef6000640 1 -- 172.21.15.83:0/2876700475 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee40ab9d0 con 0x7faef80b4360 2024-04-25T00:13:20.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 -- 172.21.15.83:0/2876700475 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef80b4360 msgr2=0x7faef80b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 --2- 172.21.15.83:0/2876700475 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef80b4360 0x7faef80b47e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faee40097f0 tx=0x7faee401c450 comp rx=0 tx=0).stop 2024-04-25T00:13:20.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 -- 172.21.15.83:0/2876700475 shutdown_connections 2024-04-25T00:13:20.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 --2- 172.21.15.83:0/2876700475 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef80b4360 0x7faef80b47e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 -- 172.21.15.83:0/2876700475 >> 172.21.15.83:0/2876700475 conn(0x7faef80b1830 msgr2=0x7faef80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:20.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 -- 172.21.15.83:0/2876700475 shutdown_connections 2024-04-25T00:13:20.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 -- 172.21.15.83:0/2876700475 wait complete. 2024-04-25T00:13:20.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 Processor -- start 2024-04-25T00:13:20.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 -- start start 2024-04-25T00:13:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef80b2b30 0x7faef814c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef80b4360 0x7faef81f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef81553e0 0x7faef8150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faef8155940 con 0x7faef81553e0 2024-04-25T00:13:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faef81557c0 con 0x7faef80b2b30 2024-04-25T00:13:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.957+0000 7faefe000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7faef806ebd0 con 0x7faef80b4360 2024-04-25T00:13:20.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faef6a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef80b4360 0x7faef81f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faef7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef81553e0 0x7faef8150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faef7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef80b2b30 0x7faef814c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faef6a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef80b4360 0x7faef81f6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:39134/0 (socket says 172.21.15.83:39134) 2024-04-25T00:13:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faef6a00640 1 -- 172.21.15.83:0/3682777501 learned_addr learned my addr 172.21.15.83:0/3682777501 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faef6a00640 1 -- 172.21.15.83:0/3682777501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef80b2b30 msgr2=0x7faef814c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faef6a00640 1 --2- 172.21.15.83:0/3682777501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef80b2b30 0x7faef814c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faef6a00640 1 -- 172.21.15.83:0/3682777501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef81553e0 msgr2=0x7faef8150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faef6a00640 1 --2- 172.21.15.83:0/3682777501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef81553e0 0x7faef8150710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faef6a00640 1 -- 172.21.15.83:0/3682777501 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef81f70a0 con 0x7faef80b4360 2024-04-25T00:13:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faef7e00640 1 --2- 172.21.15.83:0/3682777501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef81553e0 0x7faef8150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faef6a00640 1 --2- 172.21.15.83:0/3682777501 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef80b4360 0x7faef81f6170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faee40ae040 tx=0x7faee401ac90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faeef400640 1 -- 172.21.15.83:0/3682777501 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee40bb020 con 0x7faef80b4360 2024-04-25T00:13:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faeef400640 1 -- 172.21.15.83:0/3682777501 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee40b1040 con 0x7faef80b4360 2024-04-25T00:13:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faeef400640 1 -- 172.21.15.83:0/3682777501 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee40b0050 con 0x7faef80b4360 2024-04-25T00:13:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faefe000640 1 -- 172.21.15.83:0/3682777501 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef81f6800 con 0x7faef80b4360 2024-04-25T00:13:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faefe000640 1 -- 172.21.15.83:0/3682777501 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faef81f6dc0 con 0x7faef80b4360 2024-04-25T00:13:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faefe000640 1 -- 172.21.15.83:0/3682777501 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef80737f0 con 0x7faef80b4360 2024-04-25T00:13:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.961+0000 7faeef400640 1 -- 172.21.15.83:0/3682777501 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faee40be070 con 0x7faef80b4360 2024-04-25T00:13:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.965+0000 7faeef400640 1 --2- 172.21.15.83:0/3682777501 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faed80678c0 0x7faed8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.965+0000 7faeef400640 1 -- 172.21.15.83:0/3682777501 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(59..59 src has 1..59) ==== 6101+0+0 (secure 0 0 0) 0x7faee411f3a0 con 0x7faef80b4360 2024-04-25T00:13:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.965+0000 7faef7400640 1 --2- 172.21.15.83:0/3682777501 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faed80678c0 0x7faed8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.965+0000 7faef7400640 1 --2- 172.21.15.83:0/3682777501 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faed80678c0 0x7faed8069d80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faee80012f0 tx=0x7faee8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:20.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:20.969+0000 7faeef400640 1 -- 172.21.15.83:0/3682777501 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faef81f70a0 con 0x7faef80b4360 2024-04-25T00:13:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.125+0000 7faefe000640 1 -- 172.21.15.83:0/3682777501 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faef804f5f0 con 0x7faef80b4360 2024-04-25T00:13:21.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.129+0000 7faeef400640 1 -- 172.21.15.83:0/3682777501 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16813 (secure 0 0 0) 0x7faee40160d0 con 0x7faef80b4360 2024-04-25T00:13:21.130 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:21.130 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":59,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:19.804989+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:18.160239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:18.800529+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.129+0000 7faefe000640 1 -- 172.21.15.83:0/3682777501 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faed80678c0 msgr2=0x7faed8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.129+0000 7faefe000640 1 --2- 172.21.15.83:0/3682777501 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faed80678c0 0x7faed8069d80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faee80012f0 tx=0x7faee8009040 comp rx=0 tx=0).stop 2024-04-25T00:13:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.129+0000 7faefe000640 1 -- 172.21.15.83:0/3682777501 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef80b4360 msgr2=0x7faef81f6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.129+0000 7faefe000640 1 --2- 172.21.15.83:0/3682777501 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef80b4360 0x7faef81f6170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faee40ae040 tx=0x7faee401ac90 comp rx=0 tx=0).stop 2024-04-25T00:13:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.129+0000 7faefe000640 1 -- 172.21.15.83:0/3682777501 shutdown_connections 2024-04-25T00:13:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.129+0000 7faefe000640 1 --2- 172.21.15.83:0/3682777501 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faed80678c0 0x7faed8069d80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.129+0000 7faefe000640 1 --2- 172.21.15.83:0/3682777501 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef80b4360 0x7faef81f6170 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.129+0000 7faefe000640 1 --2- 172.21.15.83:0/3682777501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef81553e0 0x7faef8150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.129+0000 7faefe000640 1 --2- 172.21.15.83:0/3682777501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef80b2b30 0x7faef814c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.129+0000 7faefe000640 1 -- 172.21.15.83:0/3682777501 >> 172.21.15.83:0/3682777501 conn(0x7faef80b1830 msgr2=0x7faef806e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.129+0000 7faefe000640 1 -- 172.21.15.83:0/3682777501 shutdown_connections 2024-04-25T00:13:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.129+0000 7faefe000640 1 -- 172.21.15.83:0/3682777501 wait complete. 2024-04-25T00:13:21.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:13:21.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f99a2e00640 1 Processor -- start 2024-04-25T00:13:21.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f99a2e00640 1 -- start start 2024-04-25T00:13:21.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f99a2e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f999c159810 0x7f999c0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:21.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f99a2e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f999c160020 0x7f999c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:21.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f99a2e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f999c0ce000 con 0x7f999c06e950 2024-04-25T00:13:21.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f99a2e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f999c06e210 con 0x7f999c160020 2024-04-25T00:13:21.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f99a2e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f999c06da00 con 0x7f999c159810 2024-04-25T00:13:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f99a0e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f999c160020 0x7f999c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f999b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f999c06e950 0x7f999c15f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:42626/0 (socket says 172.21.15.83:42626) 2024-04-25T00:13:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f999b400640 1 -- 172.21.15.83:0/247777081 learned_addr learned my addr 172.21.15.83:0/247777081 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f999be00640 1 --2- 172.21.15.83:0/247777081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f999c159810 0x7f999c0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f999aa00640 1 -- 172.21.15.83:0/247777081 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 519817798 0 0) 0x7f999c0ce000 con 0x7f999c06e950 2024-04-25T00:13:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f999aa00640 1 -- 172.21.15.83:0/247777081 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9980003530 con 0x7f999c06e950 2024-04-25T00:13:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f99a0e00640 1 -- 172.21.15.83:0/247777081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f999c159810 msgr2=0x7f999c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f99a0e00640 1 --2- 172.21.15.83:0/247777081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f999c159810 0x7f999c0cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f99a0e00640 1 -- 172.21.15.83:0/247777081 >> v1:172.21.15.86:6789/0 conn(0x7f999c06e950 legacy=0x7f999c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f99a0e00640 1 -- 172.21.15.83:0/247777081 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c06eec0 con 0x7f999c160020 2024-04-25T00:13:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.289+0000 7f999be00640 1 --2- 172.21.15.83:0/247777081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f999c159810 0x7f999c0cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.293+0000 7f99a0e00640 1 --2- 172.21.15.83:0/247777081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f999c160020 0x7f999c0ce4a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f999c06db80 tx=0x7f999401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b094ff0b499dc76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.293+0000 7f999aa00640 1 -- 172.21.15.83:0/247777081 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99940b2070 con 0x7f999c160020 2024-04-25T00:13:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.293+0000 7f999aa00640 1 -- 172.21.15.83:0/247777081 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99940a5950 con 0x7f999c160020 2024-04-25T00:13:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.293+0000 7f999aa00640 1 -- 172.21.15.83:0/247777081 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99940a5d20 con 0x7f999c160020 2024-04-25T00:13:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.293+0000 7f99a2e00640 1 -- 172.21.15.83:0/247777081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f999c160020 msgr2=0x7f999c0ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.293+0000 7f99a2e00640 1 --2- 172.21.15.83:0/247777081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f999c160020 0x7f999c0ce4a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f999c06db80 tx=0x7f999401c890 comp rx=0 tx=0).stop 2024-04-25T00:13:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.293+0000 7f99a2e00640 1 -- 172.21.15.83:0/247777081 shutdown_connections 2024-04-25T00:13:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.293+0000 7f99a2e00640 1 --2- 172.21.15.83:0/247777081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f999c160020 0x7f999c0ce4a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.293+0000 7f99a2e00640 1 --2- 172.21.15.83:0/247777081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f999c159810 0x7f999c0cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.293+0000 7f99a2e00640 1 -- 172.21.15.83:0/247777081 >> 172.21.15.83:0/247777081 conn(0x7f999c0b1830 msgr2=0x7f999c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.293+0000 7f99a2e00640 1 -- 172.21.15.83:0/247777081 shutdown_connections 2024-04-25T00:13:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.293+0000 7f99a2e00640 1 -- 172.21.15.83:0/247777081 wait complete. 2024-04-25T00:13:21.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f99a2e00640 1 Processor -- start 2024-04-25T00:13:21.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f99a2e00640 1 -- start start 2024-04-25T00:13:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f99a2e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f999c06e950 0x7f999c1486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f99a2e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f999c159810 0x7f999c154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f99a2e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f999c160020 0x7f999c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f99a2e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f999c159d70 con 0x7f999c06e950 2024-04-25T00:13:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f99a2e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f999c159bf0 con 0x7f999c160020 2024-04-25T00:13:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f99a2e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f999c070da0 con 0x7f999c159810 2024-04-25T00:13:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f999be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f999c06e950 0x7f999c1486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f999be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f999c06e950 0x7f999c1486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55320/0 (socket says 172.21.15.83:55320) 2024-04-25T00:13:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f999be00640 1 -- 172.21.15.83:0/3107768679 learned_addr learned my addr 172.21.15.83:0/3107768679 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f99a0e00640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f999c160020 0x7f999c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f999b400640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f999c159810 0x7f999c154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f999be00640 1 -- 172.21.15.83:0/3107768679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f999c159810 msgr2=0x7f999c154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f999be00640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f999c159810 0x7f999c154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f999be00640 1 -- 172.21.15.83:0/3107768679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f999c160020 msgr2=0x7f999c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f999be00640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f999c160020 0x7f999c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f999be00640 1 -- 172.21.15.83:0/3107768679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c154630 con 0x7f999c06e950 2024-04-25T00:13:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f999b400640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f999c159810 0x7f999c154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f999be00640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f999c06e950 0x7f999c1486f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f998c00ec10 tx=0x7f998c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f9993e00640 1 -- 172.21.15.83:0/3107768679 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c017070 con 0x7f999c06e950 2024-04-25T00:13:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f9993e00640 1 -- 172.21.15.83:0/3107768679 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f998c00cc30 con 0x7f999c06e950 2024-04-25T00:13:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f9993e00640 1 -- 172.21.15.83:0/3107768679 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c013470 con 0x7f999c06e950 2024-04-25T00:13:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f99a2e00640 1 -- 172.21.15.83:0/3107768679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c147c10 con 0x7f999c06e950 2024-04-25T00:13:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.297+0000 7f99a2e00640 1 -- 172.21.15.83:0/3107768679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c148c30 con 0x7f999c06e950 2024-04-25T00:13:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.301+0000 7f99a2e00640 1 -- 172.21.15.83:0/3107768679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f999c04f510 con 0x7f999c06e950 2024-04-25T00:13:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.301+0000 7f9993e00640 1 -- 172.21.15.83:0/3107768679 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f998c01a070 con 0x7f999c06e950 2024-04-25T00:13:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.301+0000 7f9993e00640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f997c0678c0 0x7f997c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.301+0000 7f9993e00640 1 -- 172.21.15.83:0/3107768679 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6101+0+0 (secure 0 0 0) 0x7f998c086200 con 0x7f999c06e950 2024-04-25T00:13:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.301+0000 7f999b400640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f997c0678c0 0x7f997c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.301+0000 7f999b400640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f997c0678c0 0x7f997c069d80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f999c160830 tx=0x7f9988009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.305+0000 7f9993e00640 1 -- 172.21.15.83:0/3107768679 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f998c056440 con 0x7f999c06e950 2024-04-25T00:13:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:21.497+0000 7f99a2e00640 1 -- 172.21.15.83:0/3107768679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f999c153110 con 0x7f999c06e950 2024-04-25T00:13:22.395 INFO:tasks.ceph.mds.d.smithi086.stderr:2024-04-25T00:13:22.392+0000 7f47c0800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:13:22.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.445+0000 7f9993e00640 1 -- 172.21.15.83:0/3107768679 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) ==== 118+0+0 (secure 0 0 0) 0x7f998c05a740 con 0x7f999c06e950 2024-04-25T00:13:22.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.449+0000 7f99a2e00640 1 -- 172.21.15.83:0/3107768679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f997c0678c0 msgr2=0x7f997c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:22.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.449+0000 7f99a2e00640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f997c0678c0 0x7f997c069d80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f999c160830 tx=0x7f9988009040 comp rx=0 tx=0).stop 2024-04-25T00:13:22.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.449+0000 7f99a2e00640 1 -- 172.21.15.83:0/3107768679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f999c06e950 msgr2=0x7f999c1486f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:22.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.449+0000 7f99a2e00640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f999c06e950 0x7f999c1486f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f998c00ec10 tx=0x7f998c00c520 comp rx=0 tx=0).stop 2024-04-25T00:13:22.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.449+0000 7f99a2e00640 1 -- 172.21.15.83:0/3107768679 shutdown_connections 2024-04-25T00:13:22.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.449+0000 7f99a2e00640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f997c0678c0 0x7f997c069d80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.449+0000 7f99a2e00640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f999c159810 0x7f999c154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.449+0000 7f99a2e00640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f999c160020 0x7f999c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.449+0000 7f99a2e00640 1 --2- 172.21.15.83:0/3107768679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f999c06e950 0x7f999c1486f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.449+0000 7f99a2e00640 1 -- 172.21.15.83:0/3107768679 >> 172.21.15.83:0/3107768679 conn(0x7f999c0b1830 msgr2=0x7f999c15d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:22.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.449+0000 7f99a2e00640 1 -- 172.21.15.83:0/3107768679 shutdown_connections 2024-04-25T00:13:22.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.449+0000 7f99a2e00640 1 -- 172.21.15.83:0/3107768679 wait complete. 2024-04-25T00:13:22.473 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:13:22.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:13:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.613+0000 7f9fa3200640 1 Processor -- start 2024-04-25T00:13:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.613+0000 7f9fa3200640 1 -- start start 2024-04-25T00:13:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.613+0000 7f9fa3200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9f9c159810 0x7f9f9c0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.613+0000 7f9fa3200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f9c06e950 0x7f9f9c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.613+0000 7f9fa3200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f9c0ce000 con 0x7f9f9c15fe50 2024-04-25T00:13:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.613+0000 7f9fa3200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9f9c06e210 con 0x7f9f9c06e950 2024-04-25T00:13:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.613+0000 7f9fa3200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9f9c1602a0 con 0x7f9f9c159810 2024-04-25T00:13:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.617+0000 7f9f93400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f9c06e950 0x7f9f9c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.617+0000 7f9f93400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f9c06e950 0x7f9f9c15f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55322/0 (socket says 172.21.15.83:55322) 2024-04-25T00:13:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.617+0000 7f9f93400640 1 -- 172.21.15.83:0/2317156360 learned_addr learned my addr 172.21.15.83:0/2317156360 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.617+0000 7f9f93400640 1 -- 172.21.15.83:0/2317156360 >> v2:172.21.15.86:3300/0 conn(0x7f9f9c159810 msgr2=0x7f9f9c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.617+0000 7f9f93400640 1 --2- 172.21.15.83:0/2317156360 >> v2:172.21.15.86:3300/0 conn(0x7f9f9c159810 0x7f9f9c0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.617+0000 7f9f93400640 1 -- 172.21.15.83:0/2317156360 >> v1:172.21.15.86:6789/0 conn(0x7f9f9c15fe50 legacy=0x7f9f9c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:22.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.617+0000 7f9f93400640 1 -- 172.21.15.83:0/2317156360 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9c06eec0 con 0x7f9f9c06e950 2024-04-25T00:13:22.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.617+0000 7f9f93400640 1 --2- 172.21.15.83:0/2317156360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f9c06e950 0x7f9f9c15f910 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0709a0 tx=0x7f9f8401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b8a2d076b27b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:22.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.617+0000 7f9f9b400640 1 -- 172.21.15.83:0/2317156360 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f840b2070 con 0x7f9f9c06e950 2024-04-25T00:13:22.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.617+0000 7f9f9b400640 1 -- 172.21.15.83:0/2317156360 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f840a9070 con 0x7f9f9c06e950 2024-04-25T00:13:22.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.617+0000 7f9f9b400640 1 -- 172.21.15.83:0/2317156360 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f840ad880 con 0x7f9f9c06e950 2024-04-25T00:13:22.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 -- 172.21.15.83:0/2317156360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f9c06e950 msgr2=0x7f9f9c15f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:22.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 --2- 172.21.15.83:0/2317156360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f9c06e950 0x7f9f9c15f910 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0709a0 tx=0x7f9f8401c450 comp rx=0 tx=0).stop 2024-04-25T00:13:22.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 -- 172.21.15.83:0/2317156360 shutdown_connections 2024-04-25T00:13:22.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 --2- 172.21.15.83:0/2317156360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f9c06e950 0x7f9f9c15f910 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 --2- 172.21.15.83:0/2317156360 >> v2:172.21.15.86:3300/0 conn(0x7f9f9c159810 0x7f9f9c0cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 -- 172.21.15.83:0/2317156360 >> 172.21.15.83:0/2317156360 conn(0x7f9f9c0b1830 msgr2=0x7f9f9c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:22.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 -- 172.21.15.83:0/2317156360 shutdown_connections 2024-04-25T00:13:22.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 -- 172.21.15.83:0/2317156360 wait complete. 2024-04-25T00:13:22.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 Processor -- start 2024-04-25T00:13:22.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 -- start start 2024-04-25T00:13:22.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f9c06e950 0x7f9f9c157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:22.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f9c159810 0x7f9f9c152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:22.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f9c15fe50 0x7f9f9c151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:22.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c159d70 con 0x7f9f9c06e950 2024-04-25T00:13:22.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c159bf0 con 0x7f9f9c15fe50 2024-04-25T00:13:22.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9c070da0 con 0x7f9f9c159810 2024-04-25T00:13:22.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9f9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f9c06e950 0x7f9f9c157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:22.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9f9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f9c06e950 0x7f9f9c157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55332/0 (socket says 172.21.15.83:55332) 2024-04-25T00:13:22.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9f9be00640 1 -- 172.21.15.83:0/1470668000 learned_addr learned my addr 172.21.15.83:0/1470668000 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:22.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9f93400640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f9c159810 0x7f9f9c152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:22.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa1200640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f9c15fe50 0x7f9f9c151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:22.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9f9be00640 1 -- 172.21.15.83:0/1470668000 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f9c159810 msgr2=0x7f9f9c152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:22.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9f9be00640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f9c159810 0x7f9f9c152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9f9be00640 1 -- 172.21.15.83:0/1470668000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f9c15fe50 msgr2=0x7f9f9c151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:22.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9f9be00640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f9c15fe50 0x7f9f9c151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9f9be00640 1 -- 172.21.15.83:0/1470668000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9c157b10 con 0x7f9f9c06e950 2024-04-25T00:13:22.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9f93400640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f9c159810 0x7f9f9c152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:22.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa1200640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f9c15fe50 0x7f9f9c151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:22.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9f9be00640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f9c06e950 0x7f9f9c157430 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00ba50 tx=0x7f9f8c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:22.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9f98c00640 1 -- 172.21.15.83:0/1470668000 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c019070 con 0x7f9f9c06e950 2024-04-25T00:13:22.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9f98c00640 1 -- 172.21.15.83:0/1470668000 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8c00cc00 con 0x7f9f9c06e950 2024-04-25T00:13:22.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9f98c00640 1 -- 172.21.15.83:0/1470668000 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c014440 con 0x7f9f9c06e950 2024-04-25T00:13:22.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.621+0000 7f9fa3200640 1 -- 172.21.15.83:0/1470668000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f9c150c50 con 0x7f9f9c06e950 2024-04-25T00:13:22.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.625+0000 7f9fa3200640 1 -- 172.21.15.83:0/1470668000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f9c151210 con 0x7f9f9c06e950 2024-04-25T00:13:22.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.625+0000 7f9fa3200640 1 -- 172.21.15.83:0/1470668000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f9c04f510 con 0x7f9f9c06e950 2024-04-25T00:13:22.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.625+0000 7f9f98c00640 1 -- 172.21.15.83:0/1470668000 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9f8c01c070 con 0x7f9f9c06e950 2024-04-25T00:13:22.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.625+0000 7f9f98c00640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9f780678c0 0x7f9f78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:22.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.625+0000 7f9f98c00640 1 -- 172.21.15.83:0/1470668000 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f9f8c085be0 con 0x7f9f9c06e950 2024-04-25T00:13:22.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.625+0000 7f9f93400640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9f780678c0 0x7f9f78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:22.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.625+0000 7f9f93400640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9f780678c0 0x7f9f78069d80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9f840af000 tx=0x7f9f84008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:22.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.629+0000 7f9f98c00640 1 -- 172.21.15.83:0/1470668000 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9f8c056580 con 0x7f9f9c06e950 2024-04-25T00:13:22.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:22.829+0000 7f9fa3200640 1 -- 172.21.15.83:0/1470668000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9f9c072c90 con 0x7f9f9c06e950 2024-04-25T00:13:23.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.449+0000 7f9f98c00640 1 -- 172.21.15.83:0/1470668000 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7f9f8c055e80 con 0x7f9f9c06e950 2024-04-25T00:13:23.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.453+0000 7f9fa3200640 1 -- 172.21.15.83:0/1470668000 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9f780678c0 msgr2=0x7f9f78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.453+0000 7f9fa3200640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9f780678c0 0x7f9f78069d80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9f840af000 tx=0x7f9f84008040 comp rx=0 tx=0).stop 2024-04-25T00:13:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.453+0000 7f9fa3200640 1 -- 172.21.15.83:0/1470668000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f9c06e950 msgr2=0x7f9f9c157430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.453+0000 7f9fa3200640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f9c06e950 0x7f9f9c157430 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00ba50 tx=0x7f9f8c00bf10 comp rx=0 tx=0).stop 2024-04-25T00:13:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.453+0000 7f9fa3200640 1 -- 172.21.15.83:0/1470668000 shutdown_connections 2024-04-25T00:13:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.453+0000 7f9fa3200640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9f780678c0 0x7f9f78069d80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.453+0000 7f9fa3200640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f9c159810 0x7f9f9c152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.453+0000 7f9fa3200640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f9c15fe50 0x7f9f9c151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.453+0000 7f9fa3200640 1 --2- 172.21.15.83:0/1470668000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f9c06e950 0x7f9f9c157430 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.453+0000 7f9fa3200640 1 -- 172.21.15.83:0/1470668000 >> 172.21.15.83:0/1470668000 conn(0x7f9f9c0b1830 msgr2=0x7f9f9c15dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.453+0000 7f9fa3200640 1 -- 172.21.15.83:0/1470668000 shutdown_connections 2024-04-25T00:13:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.453+0000 7f9fa3200640 1 -- 172.21.15.83:0/1470668000 wait complete. 2024-04-25T00:13:23.477 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:13:23.477 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:23.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f3432e00640 1 Processor -- start 2024-04-25T00:13:23.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f3432e00640 1 -- start start 2024-04-25T00:13:23.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f3432e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f342c0b3a50 0x7f342c15f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:23.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f3432e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f342c1627c0 con 0x7f342c161be0 2024-04-25T00:13:23.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f3432e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f342c162aa0 con 0x7f342c0d0940 2024-04-25T00:13:23.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f3432e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f342c0b5760 con 0x7f342c0b3a50 2024-04-25T00:13:23.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f3430e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f342c161be0 0x7f342c06d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49502/0 (socket says 172.21.15.83:49502) 2024-04-25T00:13:23.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f3430e00640 1 -- 172.21.15.83:0/158425672 learned_addr learned my addr 172.21.15.83:0/158425672 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342b400640 1 --2- 172.21.15.83:0/158425672 >> v2:172.21.15.86:3300/0 conn(0x7f342c0b3a50 0x7f342c15f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 -- 172.21.15.83:0/158425672 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3262839306 0 0) 0x7f342c1627c0 con 0x7f342c161be0 2024-04-25T00:13:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 -- 172.21.15.83:0/158425672 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3410003530 con 0x7f342c161be0 2024-04-25T00:13:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 -- 172.21.15.83:0/158425672 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2780844398 0 0) 0x7f3410003530 con 0x7f342c161be0 2024-04-25T00:13:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 -- 172.21.15.83:0/158425672 >> v2:172.21.15.86:3300/0 conn(0x7f342c0b3a50 msgr2=0x7f342c15f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 --2- 172.21.15.83:0/158425672 >> v2:172.21.15.86:3300/0 conn(0x7f342c0b3a50 0x7f342c15f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 -- 172.21.15.83:0/158425672 >> v1:172.21.15.86:6789/0 conn(0x7f342c0d0940 legacy=0x7f342c0b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 -- 172.21.15.83:0/158425672 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c15e700 con 0x7f342c161be0 2024-04-25T00:13:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 -- 172.21.15.83:0/158425672 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f3424001680 con 0x7f342c161be0 2024-04-25T00:13:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 -- 172.21.15.83:0/158425672 >> v1:172.21.15.83:6789/0 conn(0x7f342c161be0 legacy=0x7f342c06d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 --2- 172.21.15.83:0/158425672 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34100039c0 0x7f3410003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 --2- 172.21.15.83:0/158425672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34100043c0 0x7f3410004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 --2- 172.21.15.83:0/158425672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3410005090 0x7f341000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 -- 172.21.15.83:0/158425672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f341000bfc0 con 0x7f34100043c0 2024-04-25T00:13:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 -- 172.21.15.83:0/158425672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f341000c170 con 0x7f3410005090 2024-04-25T00:13:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342aa00640 1 -- 172.21.15.83:0/158425672 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f341000c370 con 0x7f34100039c0 2024-04-25T00:13:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342b400640 1 --2- 172.21.15.83:0/158425672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34100043c0 0x7f3410004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f3430e00640 1 --2- 172.21.15.83:0/158425672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3410005090 0x7f341000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.613+0000 7f342be00640 1 --2- 172.21.15.83:0/158425672 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34100039c0 0x7f3410003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f342b400640 1 -- 172.21.15.83:0/158425672 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34100039c0 msgr2=0x7f3410003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f342b400640 1 --2- 172.21.15.83:0/158425672 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34100039c0 0x7f3410003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f342b400640 1 -- 172.21.15.83:0/158425672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3410005090 msgr2=0x7f341000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f342b400640 1 --2- 172.21.15.83:0/158425672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3410005090 0x7f341000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f342b400640 1 -- 172.21.15.83:0/158425672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3410003530 con 0x7f34100043c0 2024-04-25T00:13:23.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f3430e00640 1 -- 172.21.15.83:0/158425672 reap_dead start 2024-04-25T00:13:23.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f342b400640 1 --2- 172.21.15.83:0/158425672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34100043c0 0x7f3410004890 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3418009ac0 tx=0x7f341801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b085af19ef41adc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:23.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f342aa00640 1 -- 172.21.15.83:0/158425672 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34180b2070 con 0x7f34100043c0 2024-04-25T00:13:23.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f342aa00640 1 -- 172.21.15.83:0/158425672 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34180a55c0 con 0x7f34100043c0 2024-04-25T00:13:23.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f342aa00640 1 -- 172.21.15.83:0/158425672 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418009690 con 0x7f34100043c0 2024-04-25T00:13:23.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f3432e00640 1 -- 172.21.15.83:0/158425672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34100043c0 msgr2=0x7f3410004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f3432e00640 1 --2- 172.21.15.83:0/158425672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34100043c0 0x7f3410004890 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3418009ac0 tx=0x7f341801ca50 comp rx=0 tx=0).stop 2024-04-25T00:13:23.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f3432e00640 1 -- 172.21.15.83:0/158425672 shutdown_connections 2024-04-25T00:13:23.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f3432e00640 1 --2- 172.21.15.83:0/158425672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34100043c0 0x7f3410004890 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.617+0000 7f3432e00640 1 -- 172.21.15.83:0/158425672 >> 172.21.15.83:0/158425672 conn(0x7f342c067120 msgr2=0x7f342c06b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:23.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3432e00640 1 -- 172.21.15.83:0/158425672 shutdown_connections 2024-04-25T00:13:23.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3432e00640 1 -- 172.21.15.83:0/158425672 wait complete. 2024-04-25T00:13:23.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3432e00640 1 Processor -- start 2024-04-25T00:13:23.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3432e00640 1 -- start start 2024-04-25T00:13:23.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3432e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34100043c0 0x7f342c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:23.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3432e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f342c15daf0 0x7f342c13eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:23.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3432e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f342c13c8a0 0x7f342c13f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:23.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3432e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f342c15f1e0 con 0x7f34100043c0 2024-04-25T00:13:23.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3432e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f342c15ecc0 con 0x7f342c13c8a0 2024-04-25T00:13:23.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3432e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f342c15f360 con 0x7f342c15daf0 2024-04-25T00:13:23.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f342be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34100043c0 0x7f342c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:23.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f342be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34100043c0 0x7f342c0b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55362/0 (socket says 172.21.15.83:55362) 2024-04-25T00:13:23.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f342be00640 1 -- 172.21.15.83:0/241925783 learned_addr learned my addr 172.21.15.83:0/241925783 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:23.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f342b400640 1 --2- 172.21.15.83:0/241925783 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f342c15daf0 0x7f342c13eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:23.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3430e00640 1 --2- 172.21.15.83:0/241925783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f342c13c8a0 0x7f342c13f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:23.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f342be00640 1 -- 172.21.15.83:0/241925783 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f342c15daf0 msgr2=0x7f342c13eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f342be00640 1 --2- 172.21.15.83:0/241925783 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f342c15daf0 0x7f342c13eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f342be00640 1 -- 172.21.15.83:0/241925783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f342c13c8a0 msgr2=0x7f342c13f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f342be00640 1 --2- 172.21.15.83:0/241925783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f342c13c8a0 0x7f342c13f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f342be00640 1 -- 172.21.15.83:0/241925783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c1f7ff0 con 0x7f34100043c0 2024-04-25T00:13:23.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f342be00640 1 --2- 172.21.15.83:0/241925783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34100043c0 0x7f342c0b15c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f341c012ab0 tx=0x7f341c012f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:23.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3423e00640 1 -- 172.21.15.83:0/241925783 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c010c70 con 0x7f34100043c0 2024-04-25T00:13:23.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3423e00640 1 -- 172.21.15.83:0/241925783 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341c01b070 con 0x7f34100043c0 2024-04-25T00:13:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3423e00640 1 -- 172.21.15.83:0/241925783 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c0166c0 con 0x7f34100043c0 2024-04-25T00:13:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.621+0000 7f3432e00640 1 -- 172.21.15.83:0/241925783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c1f82d0 con 0x7f34100043c0 2024-04-25T00:13:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.625+0000 7f3432e00640 1 -- 172.21.15.83:0/241925783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1f8890 con 0x7f34100043c0 2024-04-25T00:13:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.625+0000 7f3432e00640 1 -- 172.21.15.83:0/241925783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342c1f8a90 con 0x7f34100043c0 2024-04-25T00:13:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.625+0000 7f3423e00640 1 -- 172.21.15.83:0/241925783 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f341c016860 con 0x7f34100043c0 2024-04-25T00:13:23.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.625+0000 7f3423e00640 1 --2- 172.21.15.83:0/241925783 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f340c0678f0 0x7f340c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:23.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.625+0000 7f3423e00640 1 -- 172.21.15.83:0/241925783 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f341c089690 con 0x7f34100043c0 2024-04-25T00:13:23.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.625+0000 7f342b400640 1 --2- 172.21.15.83:0/241925783 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f340c0678f0 0x7f340c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:23.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.625+0000 7f342b400640 1 --2- 172.21.15.83:0/241925783 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f340c0678f0 0x7f340c069db0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f34180af000 tx=0x7f34180046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:23.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.629+0000 7f3423e00640 1 -- 172.21.15.83:0/241925783 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f342c1f8a90 con 0x7f34100043c0 2024-04-25T00:13:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.821+0000 7f3432e00640 1 -- 172.21.15.83:0/241925783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f342c072f00 con 0x7f34100043c0 2024-04-25T00:13:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.821+0000 7f3423e00640 1 -- 172.21.15.83:0/241925783 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5176 (secure 0 0 0) 0x7f341c059fa0 con 0x7f34100043c0 2024-04-25T00:13:23.823 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:23.823 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3667389928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":3667389928},{"type":"v1","addr":"172.21.15.86:6835","nonce":3667389928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5923,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2174516900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2174516900},{"type":"v1","addr":"172.21.15.83:6837","nonce":2174516900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2877383406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2877383406},{"type":"v1","addr":"172.21.15.83:6836","nonce":2877383406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:13:19.805315+0000","modified":"2024-04-25T00:13:22.834474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6834/52944842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":52944842},{"type":"v1","addr":"172.21.15.86:6834","nonce":52944842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-04-25T00:13:23.823 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 35 2024-04-25T00:13:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.821+0000 7f3432e00640 1 -- 172.21.15.83:0/241925783 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f340c0678f0 msgr2=0x7f340c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.821+0000 7f3432e00640 1 --2- 172.21.15.83:0/241925783 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f340c0678f0 0x7f340c069db0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f34180af000 tx=0x7f34180046d0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.821+0000 7f3432e00640 1 -- 172.21.15.83:0/241925783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34100043c0 msgr2=0x7f342c0b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.821+0000 7f3432e00640 1 --2- 172.21.15.83:0/241925783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34100043c0 0x7f342c0b15c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f341c012ab0 tx=0x7f341c012f70 comp rx=0 tx=0).stop 2024-04-25T00:13:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.821+0000 7f3432e00640 1 -- 172.21.15.83:0/241925783 shutdown_connections 2024-04-25T00:13:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.821+0000 7f3432e00640 1 --2- 172.21.15.83:0/241925783 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f340c0678f0 0x7f340c069db0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.821+0000 7f3432e00640 1 --2- 172.21.15.83:0/241925783 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f342c15daf0 0x7f342c13eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.821+0000 7f3432e00640 1 --2- 172.21.15.83:0/241925783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f342c13c8a0 0x7f342c13f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.821+0000 7f3432e00640 1 --2- 172.21.15.83:0/241925783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34100043c0 0x7f342c0b15c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.821+0000 7f3432e00640 1 -- 172.21.15.83:0/241925783 >> 172.21.15.83:0/241925783 conn(0x7f342c067120 msgr2=0x7f342c0b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.821+0000 7f3432e00640 1 -- 172.21.15.83:0/241925783 shutdown_connections 2024-04-25T00:13:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.821+0000 7f3432e00640 1 -- 172.21.15.83:0/241925783 wait complete. 2024-04-25T00:13:23.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:23.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f555fa00640 1 Processor -- start 2024-04-25T00:13:23.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f555fa00640 1 -- start start 2024-04-25T00:13:23.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f555fa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f555806c500 0x7f555815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:23.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f555fa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f555815de20 0x7f555815e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:23.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f555fa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55580b1cc0 con 0x7f5558161dc0 2024-04-25T00:13:23.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f555fa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f55580ae9e0 con 0x7f555815de20 2024-04-25T00:13:23.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f555fa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f555806cd00 con 0x7f555806c500 2024-04-25T00:13:23.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f555d000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5558161dc0 0x7f55581621a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:42650/0 (socket says 172.21.15.83:42650) 2024-04-25T00:13:23.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f555d000640 1 -- 172.21.15.83:0/3858052250 learned_addr learned my addr 172.21.15.83:0/3858052250 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:23.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f555da00640 1 --2- 172.21.15.83:0/3858052250 >> v2:172.21.15.86:3300/0 conn(0x7f555815de20 0x7f555815e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:23.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f5557e00640 1 --2- 172.21.15.83:0/3858052250 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f555806c500 0x7f555815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:23.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f555da00640 1 -- 172.21.15.83:0/3858052250 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f555806c500 msgr2=0x7f555815d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:23.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f555da00640 1 --2- 172.21.15.83:0/3858052250 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f555806c500 0x7f555815d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f555da00640 1 -- 172.21.15.83:0/3858052250 >> v1:172.21.15.86:6789/0 conn(0x7f5558161dc0 legacy=0x7f55581621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.977+0000 7f555da00640 1 -- 172.21.15.83:0/3858052250 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555806cfe0 con 0x7f555815de20 2024-04-25T00:13:23.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.981+0000 7f555da00640 1 --2- 172.21.15.83:0/3858052250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555815de20 0x7f555815e2d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f55500097c0 tx=0x7f555001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f66772b0a99b890d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:23.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.981+0000 7f5557400640 1 -- 172.21.15.83:0/3858052250 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55500b2070 con 0x7f555815de20 2024-04-25T00:13:23.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.981+0000 7f5557400640 1 -- 172.21.15.83:0/3858052250 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55500ad640 con 0x7f555815de20 2024-04-25T00:13:23.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.981+0000 7f5557400640 1 -- 172.21.15.83:0/3858052250 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55500ada10 con 0x7f555815de20 2024-04-25T00:13:23.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.981+0000 7f555fa00640 1 -- 172.21.15.83:0/3858052250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555815de20 msgr2=0x7f555815e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.981+0000 7f555fa00640 1 --2- 172.21.15.83:0/3858052250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555815de20 0x7f555815e2d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f55500097c0 tx=0x7f555001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.981+0000 7f555fa00640 1 -- 172.21.15.83:0/3858052250 shutdown_connections 2024-04-25T00:13:23.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.981+0000 7f555fa00640 1 --2- 172.21.15.83:0/3858052250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555815de20 0x7f555815e2d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.981+0000 7f555fa00640 1 --2- 172.21.15.83:0/3858052250 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f555806c500 0x7f555815d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.981+0000 7f555fa00640 1 -- 172.21.15.83:0/3858052250 >> 172.21.15.83:0/3858052250 conn(0x7f55580d1c70 msgr2=0x7f55580d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:23.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.981+0000 7f555fa00640 1 -- 172.21.15.83:0/3858052250 shutdown_connections 2024-04-25T00:13:23.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.981+0000 7f555fa00640 1 -- 172.21.15.83:0/3858052250 wait complete. 2024-04-25T00:13:23.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555fa00640 1 Processor -- start 2024-04-25T00:13:23.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555fa00640 1 -- start start 2024-04-25T00:13:23.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555806c500 0x7f55581607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:23.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555fa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f555815de20 0x7f5558147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:23.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555fa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5558161dc0 0x7f5558146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:23.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555fa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55580d28e0 con 0x7f5558161dc0 2024-04-25T00:13:23.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55580d2760 con 0x7f555806c500 2024-04-25T00:13:23.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555fa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f55580d0750 con 0x7f555815de20 2024-04-25T00:13:23.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5558161dc0 0x7f5558146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:23.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5558161dc0 0x7f5558146fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55370/0 (socket says 172.21.15.83:55370) 2024-04-25T00:13:23.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555da00640 1 -- 172.21.15.83:0/2827887488 learned_addr learned my addr 172.21.15.83:0/2827887488 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:23.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f5557e00640 1 --2- 172.21.15.83:0/2827887488 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f555815de20 0x7f5558147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:23.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555d000640 1 --2- 172.21.15.83:0/2827887488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555806c500 0x7f55581607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:23.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555da00640 1 -- 172.21.15.83:0/2827887488 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f555815de20 msgr2=0x7f5558147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555da00640 1 --2- 172.21.15.83:0/2827887488 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f555815de20 0x7f5558147830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555da00640 1 -- 172.21.15.83:0/2827887488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555806c500 msgr2=0x7f55581607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555da00640 1 --2- 172.21.15.83:0/2827887488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555806c500 0x7f55581607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555da00640 1 -- 172.21.15.83:0/2827887488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5558147520 con 0x7f5558161dc0 2024-04-25T00:13:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555da00640 1 --2- 172.21.15.83:0/2827887488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5558161dc0 0x7f5558146fe0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f555001a0d0 tx=0x7f5550019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f5554c00640 1 -- 172.21.15.83:0/2827887488 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55500b2040 con 0x7f5558161dc0 2024-04-25T00:13:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f5554c00640 1 -- 172.21.15.83:0/2827887488 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55500bc070 con 0x7f5558161dc0 2024-04-25T00:13:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555fa00640 1 -- 172.21.15.83:0/2827887488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5558147fa0 con 0x7f5558161dc0 2024-04-25T00:13:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f5554c00640 1 -- 172.21.15.83:0/2827887488 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550006e20 con 0x7f5558161dc0 2024-04-25T00:13:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555fa00640 1 -- 172.21.15.83:0/2827887488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5558148560 con 0x7f5558161dc0 2024-04-25T00:13:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.985+0000 7f555fa00640 1 -- 172.21.15.83:0/2827887488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f555809fd60 con 0x7f5558161dc0 2024-04-25T00:13:23.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.989+0000 7f5554c00640 1 -- 172.21.15.83:0/2827887488 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f55500b6630 con 0x7f5558161dc0 2024-04-25T00:13:23.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.989+0000 7f5554c00640 1 --2- 172.21.15.83:0/2827887488 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f55380678c0 0x7f5538069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:23.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.989+0000 7f5554c00640 1 -- 172.21.15.83:0/2827887488 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f555011fd00 con 0x7f5558161dc0 2024-04-25T00:13:23.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.989+0000 7f555d000640 1 --2- 172.21.15.83:0/2827887488 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f55380678c0 0x7f5538069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:23.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.989+0000 7f555d000640 1 --2- 172.21.15.83:0/2827887488 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f55380678c0 0x7f5538069d80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f55581fa440 tx=0x7f55480019a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:23.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:23.993+0000 7f5554c00640 1 -- 172.21.15.83:0/2827887488 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5550016090 con 0x7f5558161dc0 2024-04-25T00:13:24.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.149+0000 7f555fa00640 1 -- 172.21.15.83:0/2827887488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f555804f5f0 con 0x7f5558161dc0 2024-04-25T00:13:24.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.149+0000 7f5554c00640 1 -- 172.21.15.83:0/2827887488 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16800 (secure 0 0 0) 0x7f55500effc0 con 0x7f5558161dc0 2024-04-25T00:13:24.153 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:24.153 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":60,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:21.438568+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:18.160239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:18.800529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:24.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.153+0000 7f555fa00640 1 -- 172.21.15.83:0/2827887488 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f55380678c0 msgr2=0x7f5538069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.153+0000 7f555fa00640 1 --2- 172.21.15.83:0/2827887488 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f55380678c0 0x7f5538069d80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f55581fa440 tx=0x7f55480019a0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.153+0000 7f555fa00640 1 -- 172.21.15.83:0/2827887488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5558161dc0 msgr2=0x7f5558146fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.153+0000 7f555fa00640 1 --2- 172.21.15.83:0/2827887488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5558161dc0 0x7f5558146fe0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f555001a0d0 tx=0x7f5550019240 comp rx=0 tx=0).stop 2024-04-25T00:13:24.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.153+0000 7f555fa00640 1 -- 172.21.15.83:0/2827887488 shutdown_connections 2024-04-25T00:13:24.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.153+0000 7f555fa00640 1 --2- 172.21.15.83:0/2827887488 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f55380678c0 0x7f5538069d80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.153+0000 7f555fa00640 1 --2- 172.21.15.83:0/2827887488 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f555815de20 0x7f5558147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.153+0000 7f555fa00640 1 --2- 172.21.15.83:0/2827887488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5558161dc0 0x7f5558146fe0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.153+0000 7f555fa00640 1 --2- 172.21.15.83:0/2827887488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555806c500 0x7f55581607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.153+0000 7f555fa00640 1 -- 172.21.15.83:0/2827887488 >> 172.21.15.83:0/2827887488 conn(0x7f55580d1c70 msgr2=0x7f5558169130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:24.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.153+0000 7f555fa00640 1 -- 172.21.15.83:0/2827887488 shutdown_connections 2024-04-25T00:13:24.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.153+0000 7f555fa00640 1 -- 172.21.15.83:0/2827887488 wait complete. 2024-04-25T00:13:24.174 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:13:24.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:13:24.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa544e00640 1 Processor -- start 2024-04-25T00:13:24.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa544e00640 1 -- start start 2024-04-25T00:13:24.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa544e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5401615a0 0x7fa540072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa544e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa540074840 0x7fa5400af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa544e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5400d0000 con 0x7fa5400afdc0 2024-04-25T00:13:24.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa544e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa5400b0220 con 0x7fa540074840 2024-04-25T00:13:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa544e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa5400d10a0 con 0x7fa5401615a0 2024-04-25T00:13:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa53fe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa5400afdc0 0x7fa540161d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54864/0 (socket says 172.21.15.83:54864) 2024-04-25T00:13:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa53fe00640 1 -- 172.21.15.83:0/1058721864 learned_addr learned my addr 172.21.15.83:0/1058721864 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa53ea00640 1 --2- 172.21.15.83:0/1058721864 >> v2:172.21.15.83:3300/0 conn(0x7fa540074840 0x7fa5400af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa53f400640 1 --2- 172.21.15.83:0/1058721864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5401615a0 0x7fa540072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa53ea00640 1 -- 172.21.15.83:0/1058721864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5401615a0 msgr2=0x7fa540072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa53ea00640 1 --2- 172.21.15.83:0/1058721864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5401615a0 0x7fa540072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa53ea00640 1 -- 172.21.15.83:0/1058721864 >> v1:172.21.15.83:6789/0 conn(0x7fa5400afdc0 legacy=0x7fa540161d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa53e000640 1 -- 172.21.15.83:0/1058721864 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 763367358 0 0) 0x7fa5400d0000 con 0x7fa5400afdc0 2024-04-25T00:13:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.309+0000 7fa53ea00640 1 -- 172.21.15.83:0/1058721864 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5400d02e0 con 0x7fa540074840 2024-04-25T00:13:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.313+0000 7fa53ea00640 1 --2- 172.21.15.83:0/1058721864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa540074840 0x7fa5400af880 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0097c0 tx=0x7fa52c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a0393aff9a62df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.313+0000 7fa53e000640 1 -- 172.21.15.83:0/1058721864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0b2070 con 0x7fa540074840 2024-04-25T00:13:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.313+0000 7fa53e000640 1 -- 172.21.15.83:0/1058721864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52c0ad640 con 0x7fa540074840 2024-04-25T00:13:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.313+0000 7fa53e000640 1 -- 172.21.15.83:0/1058721864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0ada10 con 0x7fa540074840 2024-04-25T00:13:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.313+0000 7fa544e00640 1 -- 172.21.15.83:0/1058721864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa540074840 msgr2=0x7fa5400af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.313+0000 7fa544e00640 1 --2- 172.21.15.83:0/1058721864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa540074840 0x7fa5400af880 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0097c0 tx=0x7fa52c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.313+0000 7fa544e00640 1 -- 172.21.15.83:0/1058721864 shutdown_connections 2024-04-25T00:13:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.313+0000 7fa544e00640 1 --2- 172.21.15.83:0/1058721864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa540074840 0x7fa5400af880 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.313+0000 7fa544e00640 1 --2- 172.21.15.83:0/1058721864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5401615a0 0x7fa540072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.313+0000 7fa544e00640 1 -- 172.21.15.83:0/1058721864 >> 172.21.15.83:0/1058721864 conn(0x7fa5400b5850 msgr2=0x7fa540072820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.313+0000 7fa544e00640 1 -- 172.21.15.83:0/1058721864 shutdown_connections 2024-04-25T00:13:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.313+0000 7fa544e00640 1 -- 172.21.15.83:0/1058721864 wait complete. 2024-04-25T00:13:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.313+0000 7fa544e00640 1 Processor -- start 2024-04-25T00:13:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.313+0000 7fa544e00640 1 -- start start 2024-04-25T00:13:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa544e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa540074840 0x7fa540158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa544e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5400afdc0 0x7fa5400ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa544e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5401615a0 0x7fa54013bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa544e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa540161b00 con 0x7fa5401615a0 2024-04-25T00:13:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa544e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa540161980 con 0x7fa5400afdc0 2024-04-25T00:13:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa544e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa5400cf830 con 0x7fa540074840 2024-04-25T00:13:24.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa53fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5401615a0 0x7fa54013bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa53fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5401615a0 0x7fa54013bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56768/0 (socket says 172.21.15.83:56768) 2024-04-25T00:13:24.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa53fe00640 1 -- 172.21.15.83:0/1254499047 learned_addr learned my addr 172.21.15.83:0/1254499047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:24.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa53f400640 1 --2- 172.21.15.83:0/1254499047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa540074840 0x7fa540158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa53ea00640 1 --2- 172.21.15.83:0/1254499047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5400afdc0 0x7fa5400ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa53fe00640 1 -- 172.21.15.83:0/1254499047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa540074840 msgr2=0x7fa540158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa53fe00640 1 --2- 172.21.15.83:0/1254499047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa540074840 0x7fa540158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa53fe00640 1 -- 172.21.15.83:0/1254499047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5400afdc0 msgr2=0x7fa5400ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa53fe00640 1 --2- 172.21.15.83:0/1254499047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5400afdc0 0x7fa5400ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa53fe00640 1 -- 172.21.15.83:0/1254499047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa540158720 con 0x7fa5401615a0 2024-04-25T00:13:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa53fe00640 1 --2- 172.21.15.83:0/1254499047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5401615a0 0x7fa54013bef0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa53800c7a0 tx=0x7fa53800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa537400640 1 -- 172.21.15.83:0/1254499047 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538018070 con 0x7fa5401615a0 2024-04-25T00:13:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa537400640 1 -- 172.21.15.83:0/1254499047 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa53800a640 con 0x7fa5401615a0 2024-04-25T00:13:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa537400640 1 -- 172.21.15.83:0/1254499047 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53800aab0 con 0x7fa5401615a0 2024-04-25T00:13:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa544e00640 1 -- 172.21.15.83:0/1254499047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa540158a00 con 0x7fa5401615a0 2024-04-25T00:13:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa544e00640 1 -- 172.21.15.83:0/1254499047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5400b35a0 con 0x7fa5401615a0 2024-04-25T00:13:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.317+0000 7fa544e00640 1 -- 172.21.15.83:0/1254499047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa54004f510 con 0x7fa5401615a0 2024-04-25T00:13:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.321+0000 7fa537400640 1 -- 172.21.15.83:0/1254499047 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa53800ac50 con 0x7fa5401615a0 2024-04-25T00:13:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.321+0000 7fa537400640 1 --2- 172.21.15.83:0/1254499047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5200678c0 0x7fa520069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.321+0000 7fa537400640 1 -- 172.21.15.83:0/1254499047 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fa538085410 con 0x7fa5401615a0 2024-04-25T00:13:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.321+0000 7fa53f400640 1 --2- 172.21.15.83:0/1254499047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5200678c0 0x7fa520069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.321+0000 7fa53f400640 1 --2- 172.21.15.83:0/1254499047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5200678c0 0x7fa520069d80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa5300079c0 tx=0x7fa530008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.325+0000 7fa537400640 1 -- 172.21.15.83:0/1254499047 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa538055760 con 0x7fa5401615a0 2024-04-25T00:13:24.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.481+0000 7fa544e00640 1 -- 172.21.15.83:0/1254499047 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa54006cd60 con 0x7fa5200678c0 2024-04-25T00:13:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.485+0000 7fa537400640 1 -- 172.21.15.83:0/1254499047 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246710 (secure 0 0 0) 0x7fa54006cd60 con 0x7fa5200678c0 2024-04-25T00:13:24.491 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:24.493 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:13:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.489+0000 7fa544e00640 1 -- 172.21.15.83:0/1254499047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5200678c0 msgr2=0x7fa520069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.489+0000 7fa544e00640 1 --2- 172.21.15.83:0/1254499047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5200678c0 0x7fa520069d80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa5300079c0 tx=0x7fa530008040 comp rx=0 tx=0).stop 2024-04-25T00:13:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.489+0000 7fa544e00640 1 -- 172.21.15.83:0/1254499047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5401615a0 msgr2=0x7fa54013bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.489+0000 7fa544e00640 1 --2- 172.21.15.83:0/1254499047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5401615a0 0x7fa54013bef0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa53800c7a0 tx=0x7fa53800cc60 comp rx=0 tx=0).stop 2024-04-25T00:13:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.489+0000 7fa544e00640 1 -- 172.21.15.83:0/1254499047 shutdown_connections 2024-04-25T00:13:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.489+0000 7fa544e00640 1 --2- 172.21.15.83:0/1254499047 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5200678c0 0x7fa520069d80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.489+0000 7fa544e00640 1 --2- 172.21.15.83:0/1254499047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5401615a0 0x7fa54013bef0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.489+0000 7fa544e00640 1 --2- 172.21.15.83:0/1254499047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5400afdc0 0x7fa5400ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.489+0000 7fa544e00640 1 --2- 172.21.15.83:0/1254499047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa540074840 0x7fa540158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.489+0000 7fa544e00640 1 -- 172.21.15.83:0/1254499047 >> 172.21.15.83:0/1254499047 conn(0x7fa5400b5850 msgr2=0x7fa5400ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.489+0000 7fa544e00640 1 -- 172.21.15.83:0/1254499047 shutdown_connections 2024-04-25T00:13:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.489+0000 7fa544e00640 1 -- 172.21.15.83:0/1254499047 wait complete. 2024-04-25T00:13:24.510 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-04-25T00:13:24.470989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226940,"kb_used_data":6560,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747780,"statfs":{"total":773094113280,"available":772861726720,"internally_reserved":0,"allocated":6717440,"data_stored":3630326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000984"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427611+0000","last_change":"2024-04-25T00:13:19.427611+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427611+0000","last_undegraded":"2024-04-25T00:13:19.427611+0000","last_fullsized":"2024-04-25T00:13:19.427611+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427995+0000","last_change":"2024-04-25T00:13:19.427995+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427995+0000","last_undegraded":"2024-04-25T00:13:19.427995+0000","last_fullsized":"2024-04-25T00:13:19.427995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420276+0000","last_change":"2024-04-25T00:13:19.420276+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420276+0000","last_undegraded":"2024-04-25T00:13:19.420276+0000","last_fullsized":"2024-04-25T00:13:19.420276+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428639+0000","last_change":"2024-04-25T00:13:19.428639+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428639+0000","last_undegraded":"2024-04-25T00:13:19.428639+0000","last_fullsized":"2024-04-25T00:13:19.428639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420673+0000","last_change":"2024-04-25T00:13:19.420673+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420673+0000","last_undegraded":"2024-04-25T00:13:19.420673+0000","last_fullsized":"2024-04-25T00:13:19.420673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422946+0000","last_change":"2024-04-25T00:13:19.422946+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422946+0000","last_undegraded":"2024-04-25T00:13:19.422946+0000","last_fullsized":"2024-04-25T00:13:19.422946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431248+0000","last_change":"2024-04-25T00:13:19.431248+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431248+0000","last_undegraded":"2024-04-25T00:13:19.431248+0000","last_fullsized":"2024-04-25T00:13:19.431248+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.429195+0000","last_change":"2024-04-25T00:13:19.429195+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.429195+0000","last_undegraded":"2024-04-25T00:13:19.429195+0000","last_fullsized":"2024-04-25T00:13:19.429195+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.421312+0000","last_change":"2024-04-25T00:13:19.421312+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.421312+0000","last_undegraded":"2024-04-25T00:13:19.421312+0000","last_fullsized":"2024-04-25T00:13:19.421312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423950+0000","last_change":"2024-04-25T00:13:19.423950+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423950+0000","last_undegraded":"2024-04-25T00:13:19.423950+0000","last_fullsized":"2024-04-25T00:13:19.423950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427090+0000","last_change":"2024-04-25T00:13:19.427090+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427090+0000","last_undegraded":"2024-04-25T00:13:19.427090+0000","last_fullsized":"2024-04-25T00:13:19.427090+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431410+0000","last_change":"2024-04-25T00:13:19.431410+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431410+0000","last_undegraded":"2024-04-25T00:13:19.431410+0000","last_fullsized":"2024-04-25T00:13:19.431410+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431226+0000","last_change":"2024-04-25T00:13:19.431226+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431226+0000","last_undegraded":"2024-04-25T00:13:19.431226+0000","last_fullsized":"2024-04-25T00:13:19.431226+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419593+0000","last_change":"2024-04-25T00:13:19.419593+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419593+0000","last_undegraded":"2024-04-25T00:13:19.419593+0000","last_fullsized":"2024-04-25T00:13:19.419593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419376+0000","last_change":"2024-04-25T00:13:19.419376+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419376+0000","last_undegraded":"2024-04-25T00:13:19.419376+0000","last_fullsized":"2024-04-25T00:13:19.419376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431142+0000","last_change":"2024-04-25T00:13:19.431142+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431142+0000","last_undegraded":"2024-04-25T00:13:19.431142+0000","last_fullsized":"2024-04-25T00:13:19.431142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427186+0000","last_change":"2024-04-25T00:13:19.427186+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427186+0000","last_undegraded":"2024-04-25T00:13:19.427186+0000","last_fullsized":"2024-04-25T00:13:19.427186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428152+0000","last_change":"2024-04-25T00:13:19.428152+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428152+0000","last_undegraded":"2024-04-25T00:13:19.428152+0000","last_fullsized":"2024-04-25T00:13:19.428152+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420621+0000","last_change":"2024-04-25T00:13:19.420621+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420621+0000","last_undegraded":"2024-04-25T00:13:19.420621+0000","last_fullsized":"2024-04-25T00:13:19.420621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423212+0000","last_change":"2024-04-25T00:13:19.423212+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423212+0000","last_undegraded":"2024-04-25T00:13:19.423212+0000","last_fullsized":"2024-04-25T00:13:19.423212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430777+0000","last_change":"2024-04-25T00:13:19.430777+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430777+0000","last_undegraded":"2024-04-25T00:13:19.430777+0000","last_fullsized":"2024-04-25T00:13:19.430777+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422795+0000","last_change":"2024-04-25T00:13:19.422795+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422795+0000","last_undegraded":"2024-04-25T00:13:19.422795+0000","last_fullsized":"2024-04-25T00:13:19.422795+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422637+0000","last_change":"2024-04-25T00:13:19.422637+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422637+0000","last_undegraded":"2024-04-25T00:13:19.422637+0000","last_fullsized":"2024-04-25T00:13:19.422637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423888+0000","last_change":"2024-04-25T00:13:19.423888+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423888+0000","last_undegraded":"2024-04-25T00:13:19.423888+0000","last_fullsized":"2024-04-25T00:13:19.423888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.429151+0000","last_change":"2024-04-25T00:13:19.429151+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.429151+0000","last_undegraded":"2024-04-25T00:13:19.429151+0000","last_fullsized":"2024-04-25T00:13:19.429151+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428403+0000","last_change":"2024-04-25T00:13:19.428403+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428403+0000","last_undegraded":"2024-04-25T00:13:19.428403+0000","last_fullsized":"2024-04-25T00:13:19.428403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426616+0000","last_change":"2024-04-25T00:13:19.426616+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426616+0000","last_undegraded":"2024-04-25T00:13:19.426616+0000","last_fullsized":"2024-04-25T00:13:19.426616+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426605+0000","last_change":"2024-04-25T00:13:19.426605+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426605+0000","last_undegraded":"2024-04-25T00:13:19.426605+0000","last_fullsized":"2024-04-25T00:13:19.426605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420098+0000","last_change":"2024-04-25T00:13:19.420098+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420098+0000","last_undegraded":"2024-04-25T00:13:19.420098+0000","last_fullsized":"2024-04-25T00:13:19.420098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431639+0000","last_change":"2024-04-25T00:13:19.431639+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431639+0000","last_undegraded":"2024-04-25T00:13:19.431639+0000","last_fullsized":"2024-04-25T00:13:19.431639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.429845+0000","last_change":"2024-04-25T00:13:19.429845+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.429845+0000","last_undegraded":"2024-04-25T00:13:19.429845+0000","last_fullsized":"2024-04-25T00:13:19.429845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423891+0000","last_change":"2024-04-25T00:13:19.423891+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423891+0000","last_undegraded":"2024-04-25T00:13:19.423891+0000","last_fullsized":"2024-04-25T00:13:19.423891+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419810+0000","last_change":"2024-04-25T00:13:19.419810+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419810+0000","last_undegraded":"2024-04-25T00:13:19.419810+0000","last_fullsized":"2024-04-25T00:13:19.419810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.448008+0000","last_change":"2024-04-25T00:13:19.445153+0000","last_active":"2024-04-25T00:13:19.448008+0000","last_peered":"2024-04-25T00:13:19.448008+0000","last_clean":"2024-04-25T00:13:19.448008+0000","last_became_active":"2024-04-25T00:13:19.444845+0000","last_became_peered":"2024-04-25T00:13:19.444845+0000","last_unstale":"2024-04-25T00:13:19.448008+0000","last_undegraded":"2024-04-25T00:13:19.448008+0000","last_fullsized":"2024-04-25T00:13:19.448008+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:03.923113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426198+0000","last_change":"2024-04-25T00:13:19.426198+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426198+0000","last_undegraded":"2024-04-25T00:13:19.426198+0000","last_fullsized":"2024-04-25T00:13:19.426198+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.441989+0000","last_change":"2024-04-25T00:13:19.439199+0000","last_active":"2024-04-25T00:13:19.441989+0000","last_peered":"2024-04-25T00:13:19.441989+0000","last_clean":"2024-04-25T00:13:19.441989+0000","last_became_active":"2024-04-25T00:13:19.438895+0000","last_became_peered":"2024-04-25T00:13:19.438895+0000","last_unstale":"2024-04-25T00:13:19.441989+0000","last_undegraded":"2024-04-25T00:13:19.441989+0000","last_fullsized":"2024-04-25T00:13:19.441989+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:20.022144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428957+0000","last_change":"2024-04-25T00:13:19.428957+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428957+0000","last_undegraded":"2024-04-25T00:13:19.428957+0000","last_fullsized":"2024-04-25T00:13:19.428957+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447654+0000","last_change":"2024-04-25T00:13:19.442726+0000","last_active":"2024-04-25T00:13:19.447654+0000","last_peered":"2024-04-25T00:13:19.447654+0000","last_clean":"2024-04-25T00:13:19.447654+0000","last_became_active":"2024-04-25T00:13:19.442433+0000","last_became_peered":"2024-04-25T00:13:19.442433+0000","last_unstale":"2024-04-25T00:13:19.447654+0000","last_undegraded":"2024-04-25T00:13:19.447654+0000","last_fullsized":"2024-04-25T00:13:19.447654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:12.349352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423306+0000","last_change":"2024-04-25T00:13:19.423306+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423306+0000","last_undegraded":"2024-04-25T00:13:19.423306+0000","last_fullsized":"2024-04-25T00:13:19.423306+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.439924+0000","last_change":"2024-04-25T00:13:19.438204+0000","last_active":"2024-04-25T00:13:19.439924+0000","last_peered":"2024-04-25T00:13:19.439924+0000","last_clean":"2024-04-25T00:13:19.439924+0000","last_became_active":"2024-04-25T00:13:19.437845+0000","last_became_peered":"2024-04-25T00:13:19.437845+0000","last_unstale":"2024-04-25T00:13:19.439924+0000","last_undegraded":"2024-04-25T00:13:19.439924+0000","last_fullsized":"2024-04-25T00:13:19.439924+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:47.273633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430288+0000","last_change":"2024-04-25T00:13:19.430288+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430288+0000","last_undegraded":"2024-04-25T00:13:19.430288+0000","last_fullsized":"2024-04-25T00:13:19.430288+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.435286+0000","last_change":"2024-04-25T00:13:19.431189+0000","last_active":"2024-04-25T00:13:19.435286+0000","last_peered":"2024-04-25T00:13:19.435286+0000","last_clean":"2024-04-25T00:13:19.435286+0000","last_became_active":"2024-04-25T00:13:19.430933+0000","last_became_peered":"2024-04-25T00:13:19.430933+0000","last_unstale":"2024-04-25T00:13:19.435286+0000","last_undegraded":"2024-04-25T00:13:19.435286+0000","last_fullsized":"2024-04-25T00:13:19.435286+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:50.255023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.425319+0000","last_change":"2024-04-25T00:13:19.425319+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.425319+0000","last_undegraded":"2024-04-25T00:13:19.425319+0000","last_fullsized":"2024-04-25T00:13:19.425319+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442691+0000","last_change":"2024-04-25T00:13:19.441394+0000","last_active":"2024-04-25T00:13:19.442691+0000","last_peered":"2024-04-25T00:13:19.442691+0000","last_clean":"2024-04-25T00:13:19.442691+0000","last_became_active":"2024-04-25T00:13:19.438688+0000","last_became_peered":"2024-04-25T00:13:19.438688+0000","last_unstale":"2024-04-25T00:13:19.442691+0000","last_undegraded":"2024-04-25T00:13:19.442691+0000","last_fullsized":"2024-04-25T00:13:19.442691+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:28.553754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420922+0000","last_change":"2024-04-25T00:13:19.420922+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420922+0000","last_undegraded":"2024-04-25T00:13:19.420922+0000","last_fullsized":"2024-04-25T00:13:19.420922+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.436636+0000","last_change":"2024-04-25T00:13:19.433617+0000","last_active":"2024-04-25T00:13:19.436636+0000","last_peered":"2024-04-25T00:13:19.436636+0000","last_clean":"2024-04-25T00:13:19.436636+0000","last_became_active":"2024-04-25T00:13:19.433063+0000","last_became_peered":"2024-04-25T00:13:19.433063+0000","last_unstale":"2024-04-25T00:13:19.436636+0000","last_undegraded":"2024-04-25T00:13:19.436636+0000","last_fullsized":"2024-04-25T00:13:19.436636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:18.683206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423602+0000","last_change":"2024-04-25T00:13:19.423602+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423602+0000","last_undegraded":"2024-04-25T00:13:19.423602+0000","last_fullsized":"2024-04-25T00:13:19.423602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442465+0000","last_change":"2024-04-25T00:13:19.431998+0000","last_active":"2024-04-25T00:13:19.442465+0000","last_peered":"2024-04-25T00:13:19.442465+0000","last_clean":"2024-04-25T00:13:19.442465+0000","last_became_active":"2024-04-25T00:13:19.431728+0000","last_became_peered":"2024-04-25T00:13:19.431728+0000","last_unstale":"2024-04-25T00:13:19.442465+0000","last_undegraded":"2024-04-25T00:13:19.442465+0000","last_fullsized":"2024-04-25T00:13:19.442465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:04.083138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426189+0000","last_change":"2024-04-25T00:13:19.426189+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426189+0000","last_undegraded":"2024-04-25T00:13:19.426189+0000","last_fullsized":"2024-04-25T00:13:19.426189+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446032+0000","last_change":"2024-04-25T00:13:19.440887+0000","last_active":"2024-04-25T00:13:19.446032+0000","last_peered":"2024-04-25T00:13:19.446032+0000","last_clean":"2024-04-25T00:13:19.446032+0000","last_became_active":"2024-04-25T00:13:19.437895+0000","last_became_peered":"2024-04-25T00:13:19.437895+0000","last_unstale":"2024-04-25T00:13:19.446032+0000","last_undegraded":"2024-04-25T00:13:19.446032+0000","last_fullsized":"2024-04-25T00:13:19.446032+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:47.273633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422117+0000","last_change":"2024-04-25T00:13:19.422117+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422117+0000","last_undegraded":"2024-04-25T00:13:19.422117+0000","last_fullsized":"2024-04-25T00:13:19.422117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.443114+0000","last_change":"2024-04-25T00:13:19.437705+0000","last_active":"2024-04-25T00:13:19.443114+0000","last_peered":"2024-04-25T00:13:19.443114+0000","last_clean":"2024-04-25T00:13:19.443114+0000","last_became_active":"2024-04-25T00:13:19.437390+0000","last_became_peered":"2024-04-25T00:13:19.437390+0000","last_unstale":"2024-04-25T00:13:19.443114+0000","last_undegraded":"2024-04-25T00:13:19.443114+0000","last_fullsized":"2024-04-25T00:13:19.443114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:44.096993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.421441+0000","last_change":"2024-04-25T00:13:19.421441+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.421441+0000","last_undegraded":"2024-04-25T00:13:19.421441+0000","last_fullsized":"2024-04-25T00:13:19.421441+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442484+0000","last_change":"2024-04-25T00:13:19.438746+0000","last_active":"2024-04-25T00:13:19.442484+0000","last_peered":"2024-04-25T00:13:19.442484+0000","last_clean":"2024-04-25T00:13:19.442484+0000","last_became_active":"2024-04-25T00:13:19.437749+0000","last_became_peered":"2024-04-25T00:13:19.437749+0000","last_unstale":"2024-04-25T00:13:19.442484+0000","last_undegraded":"2024-04-25T00:13:19.442484+0000","last_fullsized":"2024-04-25T00:13:19.442484+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:39.640503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426025+0000","last_change":"2024-04-25T00:13:19.426025+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426025+0000","last_undegraded":"2024-04-25T00:13:19.426025+0000","last_fullsized":"2024-04-25T00:13:19.426025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.448034+0000","last_change":"2024-04-25T00:13:19.445212+0000","last_active":"2024-04-25T00:13:19.448034+0000","last_peered":"2024-04-25T00:13:19.448034+0000","last_clean":"2024-04-25T00:13:19.448034+0000","last_became_active":"2024-04-25T00:13:19.444953+0000","last_became_peered":"2024-04-25T00:13:19.444953+0000","last_unstale":"2024-04-25T00:13:19.448034+0000","last_undegraded":"2024-04-25T00:13:19.448034+0000","last_fullsized":"2024-04-25T00:13:19.448034+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:28.280706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426254+0000","last_change":"2024-04-25T00:13:19.426254+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426254+0000","last_undegraded":"2024-04-25T00:13:19.426254+0000","last_fullsized":"2024-04-25T00:13:19.426254+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445679+0000","last_change":"2024-04-25T00:13:19.439464+0000","last_active":"2024-04-25T00:13:19.445679+0000","last_peered":"2024-04-25T00:13:19.445679+0000","last_clean":"2024-04-25T00:13:19.445679+0000","last_became_active":"2024-04-25T00:13:19.439143+0000","last_became_peered":"2024-04-25T00:13:19.439143+0000","last_unstale":"2024-04-25T00:13:19.445679+0000","last_undegraded":"2024-04-25T00:13:19.445679+0000","last_fullsized":"2024-04-25T00:13:19.445679+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:03.413067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.435804+0000","last_change":"2024-04-25T00:13:19.435804+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.435804+0000","last_undegraded":"2024-04-25T00:13:19.435804+0000","last_fullsized":"2024-04-25T00:13:19.435804+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446509+0000","last_change":"2024-04-25T00:13:19.443972+0000","last_active":"2024-04-25T00:13:19.446509+0000","last_peered":"2024-04-25T00:13:19.446509+0000","last_clean":"2024-04-25T00:13:19.446509+0000","last_became_active":"2024-04-25T00:13:19.443634+0000","last_became_peered":"2024-04-25T00:13:19.443634+0000","last_unstale":"2024-04-25T00:13:19.446509+0000","last_undegraded":"2024-04-25T00:13:19.446509+0000","last_fullsized":"2024-04-25T00:13:19.446509+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:03.413067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420470+0000","last_change":"2024-04-25T00:13:19.420470+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420470+0000","last_undegraded":"2024-04-25T00:13:19.420470+0000","last_fullsized":"2024-04-25T00:13:19.420470+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.436847+0000","last_change":"2024-04-25T00:13:19.430607+0000","last_active":"2024-04-25T00:13:19.436847+0000","last_peered":"2024-04-25T00:13:19.436847+0000","last_clean":"2024-04-25T00:13:19.436847+0000","last_became_active":"2024-04-25T00:13:19.430240+0000","last_became_peered":"2024-04-25T00:13:19.430240+0000","last_unstale":"2024-04-25T00:13:19.436847+0000","last_undegraded":"2024-04-25T00:13:19.436847+0000","last_fullsized":"2024-04-25T00:13:19.436847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:07.859795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428661+0000","last_change":"2024-04-25T00:13:19.428661+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428661+0000","last_undegraded":"2024-04-25T00:13:19.428661+0000","last_fullsized":"2024-04-25T00:13:19.428661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.443661+0000","last_change":"2024-04-25T00:13:19.438379+0000","last_active":"2024-04-25T00:13:19.443661+0000","last_peered":"2024-04-25T00:13:19.443661+0000","last_clean":"2024-04-25T00:13:19.443661+0000","last_became_active":"2024-04-25T00:13:19.438093+0000","last_became_peered":"2024-04-25T00:13:19.438093+0000","last_unstale":"2024-04-25T00:13:19.443661+0000","last_undegraded":"2024-04-25T00:13:19.443661+0000","last_fullsized":"2024-04-25T00:13:19.443661+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:28.036116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.424260+0000","last_change":"2024-04-25T00:13:19.424260+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.424260+0000","last_undegraded":"2024-04-25T00:13:19.424260+0000","last_fullsized":"2024-04-25T00:13:19.424260+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445046+0000","last_change":"2024-04-25T00:13:19.440461+0000","last_active":"2024-04-25T00:13:19.445046+0000","last_peered":"2024-04-25T00:13:19.445046+0000","last_clean":"2024-04-25T00:13:19.445046+0000","last_became_active":"2024-04-25T00:13:19.440197+0000","last_became_peered":"2024-04-25T00:13:19.440197+0000","last_unstale":"2024-04-25T00:13:19.445046+0000","last_undegraded":"2024-04-25T00:13:19.445046+0000","last_fullsized":"2024-04-25T00:13:19.445046+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:37.585343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420102+0000","last_change":"2024-04-25T00:13:19.420102+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420102+0000","last_undegraded":"2024-04-25T00:13:19.420102+0000","last_fullsized":"2024-04-25T00:13:19.420102+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442434+0000","last_change":"2024-04-25T00:13:19.438945+0000","last_active":"2024-04-25T00:13:19.442434+0000","last_peered":"2024-04-25T00:13:19.442434+0000","last_clean":"2024-04-25T00:13:19.442434+0000","last_became_active":"2024-04-25T00:13:19.438052+0000","last_became_peered":"2024-04-25T00:13:19.438052+0000","last_unstale":"2024-04-25T00:13:19.442434+0000","last_undegraded":"2024-04-25T00:13:19.442434+0000","last_fullsized":"2024-04-25T00:13:19.442434+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:21.586818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427651+0000","last_change":"2024-04-25T00:13:19.427651+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427651+0000","last_undegraded":"2024-04-25T00:13:19.427651+0000","last_fullsized":"2024-04-25T00:13:19.427651+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445826+0000","last_change":"2024-04-25T00:13:19.440459+0000","last_active":"2024-04-25T00:13:19.445826+0000","last_peered":"2024-04-25T00:13:19.445826+0000","last_clean":"2024-04-25T00:13:19.445826+0000","last_became_active":"2024-04-25T00:13:19.438442+0000","last_became_peered":"2024-04-25T00:13:19.438442+0000","last_unstale":"2024-04-25T00:13:19.445826+0000","last_undegraded":"2024-04-25T00:13:19.445826+0000","last_fullsized":"2024-04-25T00:13:19.445826+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:42.830716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420129+0000","last_change":"2024-04-25T00:13:19.420129+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420129+0000","last_undegraded":"2024-04-25T00:13:19.420129+0000","last_fullsized":"2024-04-25T00:13:19.420129+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442276+0000","last_change":"2024-04-25T00:13:19.440785+0000","last_active":"2024-04-25T00:13:19.442276+0000","last_peered":"2024-04-25T00:13:19.442276+0000","last_clean":"2024-04-25T00:13:19.442276+0000","last_became_active":"2024-04-25T00:13:19.440512+0000","last_became_peered":"2024-04-25T00:13:19.440512+0000","last_unstale":"2024-04-25T00:13:19.442276+0000","last_undegraded":"2024-04-25T00:13:19.442276+0000","last_fullsized":"2024-04-25T00:13:19.442276+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:35.767979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423501+0000","last_change":"2024-04-25T00:13:19.423501+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423501+0000","last_undegraded":"2024-04-25T00:13:19.423501+0000","last_fullsized":"2024-04-25T00:13:19.423501+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447676+0000","last_change":"2024-04-25T00:13:19.443427+0000","last_active":"2024-04-25T00:13:19.447676+0000","last_peered":"2024-04-25T00:13:19.447676+0000","last_clean":"2024-04-25T00:13:19.447676+0000","last_became_active":"2024-04-25T00:13:19.443147+0000","last_became_peered":"2024-04-25T00:13:19.443147+0000","last_unstale":"2024-04-25T00:13:19.447676+0000","last_undegraded":"2024-04-25T00:13:19.447676+0000","last_fullsized":"2024-04-25T00:13:19.447676+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:17.551103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.435341+0000","last_change":"2024-04-25T00:13:19.435341+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.435341+0000","last_undegraded":"2024-04-25T00:13:19.435341+0000","last_fullsized":"2024-04-25T00:13:19.435341+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445110+0000","last_change":"2024-04-25T00:13:19.441302+0000","last_active":"2024-04-25T00:13:19.445110+0000","last_peered":"2024-04-25T00:13:19.445110+0000","last_clean":"2024-04-25T00:13:19.445110+0000","last_became_active":"2024-04-25T00:13:19.440951+0000","last_became_peered":"2024-04-25T00:13:19.440951+0000","last_unstale":"2024-04-25T00:13:19.445110+0000","last_undegraded":"2024-04-25T00:13:19.445110+0000","last_fullsized":"2024-04-25T00:13:19.445110+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:07.859795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419439+0000","last_change":"2024-04-25T00:13:19.419439+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419439+0000","last_undegraded":"2024-04-25T00:13:19.419439+0000","last_fullsized":"2024-04-25T00:13:19.419439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.441866+0000","last_change":"2024-04-25T00:13:19.438811+0000","last_active":"2024-04-25T00:13:19.441866+0000","last_peered":"2024-04-25T00:13:19.441866+0000","last_clean":"2024-04-25T00:13:19.441866+0000","last_became_active":"2024-04-25T00:13:19.438527+0000","last_became_peered":"2024-04-25T00:13:19.438527+0000","last_unstale":"2024-04-25T00:13:19.441866+0000","last_undegraded":"2024-04-25T00:13:19.441866+0000","last_fullsized":"2024-04-25T00:13:19.441866+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:13.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.424634+0000","last_change":"2024-04-25T00:13:19.424634+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.424634+0000","last_undegraded":"2024-04-25T00:13:19.424634+0000","last_fullsized":"2024-04-25T00:13:19.424634+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446400+0000","last_change":"2024-04-25T00:13:19.442162+0000","last_active":"2024-04-25T00:13:19.446400+0000","last_peered":"2024-04-25T00:13:19.446400+0000","last_clean":"2024-04-25T00:13:19.446400+0000","last_became_active":"2024-04-25T00:13:19.441874+0000","last_became_peered":"2024-04-25T00:13:19.441874+0000","last_unstale":"2024-04-25T00:13:19.446400+0000","last_undegraded":"2024-04-25T00:13:19.446400+0000","last_fullsized":"2024-04-25T00:13:19.446400+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:39.640503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430355+0000","last_change":"2024-04-25T00:13:19.430355+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430355+0000","last_undegraded":"2024-04-25T00:13:19.430355+0000","last_fullsized":"2024-04-25T00:13:19.430355+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446460+0000","last_change":"2024-04-25T00:13:19.443257+0000","last_active":"2024-04-25T00:13:19.446460+0000","last_peered":"2024-04-25T00:13:19.446460+0000","last_clean":"2024-04-25T00:13:19.446460+0000","last_became_active":"2024-04-25T00:13:19.442853+0000","last_became_peered":"2024-04-25T00:13:19.442853+0000","last_unstale":"2024-04-25T00:13:19.446460+0000","last_undegraded":"2024-04-25T00:13:19.446460+0000","last_fullsized":"2024-04-25T00:13:19.446460+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:21.586818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427928+0000","last_change":"2024-04-25T00:13:19.427928+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427928+0000","last_undegraded":"2024-04-25T00:13:19.427928+0000","last_fullsized":"2024-04-25T00:13:19.427928+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447682+0000","last_change":"2024-04-25T00:13:19.444471+0000","last_active":"2024-04-25T00:13:19.447682+0000","last_peered":"2024-04-25T00:13:19.447682+0000","last_clean":"2024-04-25T00:13:19.447682+0000","last_became_active":"2024-04-25T00:13:19.444197+0000","last_became_peered":"2024-04-25T00:13:19.444197+0000","last_unstale":"2024-04-25T00:13:19.447682+0000","last_undegraded":"2024-04-25T00:13:19.447682+0000","last_fullsized":"2024-04-25T00:13:19.447682+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:29.987108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419511+0000","last_change":"2024-04-25T00:13:19.419511+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419511+0000","last_undegraded":"2024-04-25T00:13:19.419511+0000","last_fullsized":"2024-04-25T00:13:19.419511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447962+0000","last_change":"2024-04-25T00:13:19.445103+0000","last_active":"2024-04-25T00:13:19.447962+0000","last_peered":"2024-04-25T00:13:19.447962+0000","last_clean":"2024-04-25T00:13:19.447962+0000","last_became_active":"2024-04-25T00:13:19.444652+0000","last_became_peered":"2024-04-25T00:13:19.444652+0000","last_unstale":"2024-04-25T00:13:19.447962+0000","last_undegraded":"2024-04-25T00:13:19.447962+0000","last_fullsized":"2024-04-25T00:13:19.447962+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:39.691842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419832+0000","last_change":"2024-04-25T00:13:19.419832+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419832+0000","last_undegraded":"2024-04-25T00:13:19.419832+0000","last_fullsized":"2024-04-25T00:13:19.419832+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.448638+0000","last_change":"2024-04-25T00:13:19.442646+0000","last_active":"2024-04-25T00:13:19.448638+0000","last_peered":"2024-04-25T00:13:19.448638+0000","last_clean":"2024-04-25T00:13:19.448638+0000","last_became_active":"2024-04-25T00:13:19.438848+0000","last_became_peered":"2024-04-25T00:13:19.438848+0000","last_unstale":"2024-04-25T00:13:19.448638+0000","last_undegraded":"2024-04-25T00:13:19.448638+0000","last_fullsized":"2024-04-25T00:13:19.448638+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:13.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.425579+0000","last_change":"2024-04-25T00:13:19.425579+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.425579+0000","last_undegraded":"2024-04-25T00:13:19.425579+0000","last_fullsized":"2024-04-25T00:13:19.425579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.439322+0000","last_change":"2024-04-25T00:13:19.436516+0000","last_active":"2024-04-25T00:13:19.439322+0000","last_peered":"2024-04-25T00:13:19.439322+0000","last_clean":"2024-04-25T00:13:19.439322+0000","last_became_active":"2024-04-25T00:13:19.435936+0000","last_became_peered":"2024-04-25T00:13:19.435936+0000","last_unstale":"2024-04-25T00:13:19.439322+0000","last_undegraded":"2024-04-25T00:13:19.439322+0000","last_fullsized":"2024-04-25T00:13:19.439322+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:57.960615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.424954+0000","last_change":"2024-04-25T00:13:19.424954+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.424954+0000","last_undegraded":"2024-04-25T00:13:19.424954+0000","last_fullsized":"2024-04-25T00:13:19.424954+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.429368+0000","last_change":"2024-04-25T00:13:19.427399+0000","last_active":"2024-04-25T00:13:19.429368+0000","last_peered":"2024-04-25T00:13:19.429368+0000","last_clean":"2024-04-25T00:13:19.429368+0000","last_became_active":"2024-04-25T00:13:19.427145+0000","last_became_peered":"2024-04-25T00:13:19.427145+0000","last_unstale":"2024-04-25T00:13:19.429368+0000","last_undegraded":"2024-04-25T00:13:19.429368+0000","last_fullsized":"2024-04-25T00:13:19.429368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:19.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430930+0000","last_change":"2024-04-25T00:13:19.430930+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430930+0000","last_undegraded":"2024-04-25T00:13:19.430930+0000","last_fullsized":"2024-04-25T00:13:19.430930+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.441056+0000","last_change":"2024-04-25T00:13:19.438635+0000","last_active":"2024-04-25T00:13:19.441056+0000","last_peered":"2024-04-25T00:13:19.441056+0000","last_clean":"2024-04-25T00:13:19.441056+0000","last_became_active":"2024-04-25T00:13:19.438328+0000","last_became_peered":"2024-04-25T00:13:19.438328+0000","last_unstale":"2024-04-25T00:13:19.441056+0000","last_undegraded":"2024-04-25T00:13:19.441056+0000","last_fullsized":"2024-04-25T00:13:19.441056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:17.223076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419237+0000","last_change":"2024-04-25T00:13:19.419237+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419237+0000","last_undegraded":"2024-04-25T00:13:19.419237+0000","last_fullsized":"2024-04-25T00:13:19.419237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.433512+0000","last_change":"2024-04-25T00:13:19.430683+0000","last_active":"2024-04-25T00:13:19.433512+0000","last_peered":"2024-04-25T00:13:19.433512+0000","last_clean":"2024-04-25T00:13:19.433512+0000","last_became_active":"2024-04-25T00:13:19.430464+0000","last_became_peered":"2024-04-25T00:13:19.430464+0000","last_unstale":"2024-04-25T00:13:19.433512+0000","last_undegraded":"2024-04-25T00:13:19.433512+0000","last_fullsized":"2024-04-25T00:13:19.433512+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:36.678931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":239,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.424193+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:13:19.424193+0000","last_peered":"2024-04-25T00:13:19.424193+0000","last_clean":"2024-04-25T00:13:19.424193+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:13:19.424193+0000","last_undegraded":"2024-04-25T00:13:19.424193+0000","last_fullsized":"2024-04-25T00:13:19.424193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1294336,"data_stored":901561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":700416,"data_stored":311193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1282048,"data_stored":894902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:13:24.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:13:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.685+0000 7f606d400640 1 Processor -- start 2024-04-25T00:13:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.685+0000 7f606d400640 1 -- start start 2024-04-25T00:13:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f606d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f60680cc510 0x7f606815ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f606d400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f606815cf90 0x7f60680ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f606d400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60680ccc60 con 0x7f60681599f0 2024-04-25T00:13:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f606d400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f6068159dd0 con 0x7f606815cf90 2024-04-25T00:13:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f606d400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f606806dba0 con 0x7f60680cc510 2024-04-25T00:13:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f6067200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f606815cf90 0x7f60680ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f6067200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f606815cf90 0x7f60680ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56782/0 (socket says 172.21.15.83:56782) 2024-04-25T00:13:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f6067200640 1 -- 172.21.15.83:0/3902334891 learned_addr learned my addr 172.21.15.83:0/3902334891 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f6065e00640 1 --2- 172.21.15.83:0/3902334891 >> v2:172.21.15.86:3300/0 conn(0x7f60680cc510 0x7f606815ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f6067200640 1 -- 172.21.15.83:0/3902334891 >> v2:172.21.15.86:3300/0 conn(0x7f60680cc510 msgr2=0x7f606815ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f6067200640 1 --2- 172.21.15.83:0/3902334891 >> v2:172.21.15.86:3300/0 conn(0x7f60680cc510 0x7f606815ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f6067200640 1 -- 172.21.15.83:0/3902334891 >> v1:172.21.15.86:6789/0 conn(0x7f60681599f0 legacy=0x7f606815c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f6067200640 1 -- 172.21.15.83:0/3902334891 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606806de80 con 0x7f606815cf90 2024-04-25T00:13:24.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f6067200640 1 --2- 172.21.15.83:0/3902334891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f606815cf90 0x7f60680ce6e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6060009e50 tx=0x7f606001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e752fbbcd4d1f967 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f6065400640 1 -- 172.21.15.83:0/3902334891 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600a56a0 con 0x7f606815cf90 2024-04-25T00:13:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f6065400640 1 -- 172.21.15.83:0/3902334891 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60600a5840 con 0x7f606815cf90 2024-04-25T00:13:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.689+0000 7f6065400640 1 -- 172.21.15.83:0/3902334891 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600a5c10 con 0x7f606815cf90 2024-04-25T00:13:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 -- 172.21.15.83:0/3902334891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f606815cf90 msgr2=0x7f60680ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 --2- 172.21.15.83:0/3902334891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f606815cf90 0x7f60680ce6e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6060009e50 tx=0x7f606001c960 comp rx=0 tx=0).stop 2024-04-25T00:13:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 -- 172.21.15.83:0/3902334891 shutdown_connections 2024-04-25T00:13:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 --2- 172.21.15.83:0/3902334891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f606815cf90 0x7f60680ce6e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 --2- 172.21.15.83:0/3902334891 >> v2:172.21.15.86:3300/0 conn(0x7f60680cc510 0x7f606815ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 -- 172.21.15.83:0/3902334891 >> 172.21.15.83:0/3902334891 conn(0x7f60680b1830 msgr2=0x7f60680b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 -- 172.21.15.83:0/3902334891 shutdown_connections 2024-04-25T00:13:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 -- 172.21.15.83:0/3902334891 wait complete. 2024-04-25T00:13:24.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 Processor -- start 2024-04-25T00:13:24.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 -- start start 2024-04-25T00:13:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60680cc510 0x7f606814f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60681599f0 0x7f606815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f606815cf90 0x7f6068155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f60680cf5b0 con 0x7f60680cc510 2024-04-25T00:13:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60680cf430 con 0x7f60681599f0 2024-04-25T00:13:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f606d400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f60680cf730 con 0x7f606815cf90 2024-04-25T00:13:24.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f6066800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60680cc510 0x7f606814f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f6066800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60680cc510 0x7f606814f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56786/0 (socket says 172.21.15.83:56786) 2024-04-25T00:13:24.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f6066800640 1 -- 172.21.15.83:0/725286017 learned_addr learned my addr 172.21.15.83:0/725286017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:24.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f6067200640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f606815cf90 0x7f6068155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f6065e00640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60681599f0 0x7f606815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f6066800640 1 -- 172.21.15.83:0/725286017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f606815cf90 msgr2=0x7f6068155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f6066800640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f606815cf90 0x7f6068155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f6066800640 1 -- 172.21.15.83:0/725286017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60681599f0 msgr2=0x7f606815c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f6066800640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60681599f0 0x7f606815c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f6066800640 1 -- 172.21.15.83:0/725286017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606814f6a0 con 0x7f60680cc510 2024-04-25T00:13:24.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.693+0000 7f6065e00640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60681599f0 0x7f606815c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:24.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.697+0000 7f6066800640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60680cc510 0x7f606814f160 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f605800c420 tx=0x7f605800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:24.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.697+0000 7f605ea00640 1 -- 172.21.15.83:0/725286017 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60580052d0 con 0x7f60680cc510 2024-04-25T00:13:24.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.697+0000 7f605ea00640 1 -- 172.21.15.83:0/725286017 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6058005470 con 0x7f60680cc510 2024-04-25T00:13:24.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.697+0000 7f605ea00640 1 -- 172.21.15.83:0/725286017 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058007260 con 0x7f60680cc510 2024-04-25T00:13:24.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.697+0000 7f606d400640 1 -- 172.21.15.83:0/725286017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6068150360 con 0x7f60680cc510 2024-04-25T00:13:24.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.697+0000 7f606d400640 1 -- 172.21.15.83:0/725286017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6068157940 con 0x7f60680cc510 2024-04-25T00:13:24.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.697+0000 7f606d400640 1 -- 172.21.15.83:0/725286017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60680737f0 con 0x7f60680cc510 2024-04-25T00:13:24.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.697+0000 7f605ea00640 1 -- 172.21.15.83:0/725286017 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6058007400 con 0x7f60680cc510 2024-04-25T00:13:24.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.697+0000 7f605ea00640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f60480678c0 0x7f6048069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.697+0000 7f605ea00640 1 -- 172.21.15.83:0/725286017 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f6058085350 con 0x7f60680cc510 2024-04-25T00:13:24.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.697+0000 7f6065e00640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f60480678c0 0x7f6048069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.701+0000 7f6065e00640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f60480678c0 0x7f6048069d80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f606806e990 tx=0x7f6054008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:24.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.701+0000 7f605ea00640 1 -- 172.21.15.83:0/725286017 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f606814f6a0 con 0x7f60680cc510 2024-04-25T00:13:24.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.853+0000 7f606d400640 1 -- 172.21.15.83:0/725286017 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f606804f5f0 con 0x7f60480678c0 2024-04-25T00:13:24.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.861+0000 7f605ea00640 1 -- 172.21.15.83:0/725286017 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246710 (secure 0 0 0) 0x7f606804f5f0 con 0x7f60480678c0 2024-04-25T00:13:24.864 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:24.866 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:13:24.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.865+0000 7f606d400640 1 -- 172.21.15.83:0/725286017 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f60480678c0 msgr2=0x7f6048069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.865+0000 7f606d400640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f60480678c0 0x7f6048069d80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f606806e990 tx=0x7f6054008040 comp rx=0 tx=0).stop 2024-04-25T00:13:24.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.865+0000 7f606d400640 1 -- 172.21.15.83:0/725286017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60680cc510 msgr2=0x7f606814f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.865+0000 7f606d400640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60680cc510 0x7f606814f160 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f605800c420 tx=0x7f605800c8e0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.865+0000 7f606d400640 1 -- 172.21.15.83:0/725286017 shutdown_connections 2024-04-25T00:13:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.865+0000 7f606d400640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f60480678c0 0x7f6048069d80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.865+0000 7f606d400640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f606815cf90 0x7f6068155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.865+0000 7f606d400640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60681599f0 0x7f606815c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.865+0000 7f606d400640 1 --2- 172.21.15.83:0/725286017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60680cc510 0x7f606814f160 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.865+0000 7f606d400640 1 -- 172.21.15.83:0/725286017 >> 172.21.15.83:0/725286017 conn(0x7f60680b1830 msgr2=0x7f60680b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.865+0000 7f606d400640 1 -- 172.21.15.83:0/725286017 shutdown_connections 2024-04-25T00:13:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:24.865+0000 7f606d400640 1 -- 172.21.15.83:0/725286017 wait complete. 2024-04-25T00:13:24.884 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-04-25T00:13:24.470989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226940,"kb_used_data":6560,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747780,"statfs":{"total":773094113280,"available":772861726720,"internally_reserved":0,"allocated":6717440,"data_stored":3630326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000984"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427611+0000","last_change":"2024-04-25T00:13:19.427611+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427611+0000","last_undegraded":"2024-04-25T00:13:19.427611+0000","last_fullsized":"2024-04-25T00:13:19.427611+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427995+0000","last_change":"2024-04-25T00:13:19.427995+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427995+0000","last_undegraded":"2024-04-25T00:13:19.427995+0000","last_fullsized":"2024-04-25T00:13:19.427995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420276+0000","last_change":"2024-04-25T00:13:19.420276+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420276+0000","last_undegraded":"2024-04-25T00:13:19.420276+0000","last_fullsized":"2024-04-25T00:13:19.420276+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428639+0000","last_change":"2024-04-25T00:13:19.428639+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428639+0000","last_undegraded":"2024-04-25T00:13:19.428639+0000","last_fullsized":"2024-04-25T00:13:19.428639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420673+0000","last_change":"2024-04-25T00:13:19.420673+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420673+0000","last_undegraded":"2024-04-25T00:13:19.420673+0000","last_fullsized":"2024-04-25T00:13:19.420673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422946+0000","last_change":"2024-04-25T00:13:19.422946+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422946+0000","last_undegraded":"2024-04-25T00:13:19.422946+0000","last_fullsized":"2024-04-25T00:13:19.422946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431248+0000","last_change":"2024-04-25T00:13:19.431248+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431248+0000","last_undegraded":"2024-04-25T00:13:19.431248+0000","last_fullsized":"2024-04-25T00:13:19.431248+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.429195+0000","last_change":"2024-04-25T00:13:19.429195+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.429195+0000","last_undegraded":"2024-04-25T00:13:19.429195+0000","last_fullsized":"2024-04-25T00:13:19.429195+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.421312+0000","last_change":"2024-04-25T00:13:19.421312+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.421312+0000","last_undegraded":"2024-04-25T00:13:19.421312+0000","last_fullsized":"2024-04-25T00:13:19.421312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423950+0000","last_change":"2024-04-25T00:13:19.423950+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423950+0000","last_undegraded":"2024-04-25T00:13:19.423950+0000","last_fullsized":"2024-04-25T00:13:19.423950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427090+0000","last_change":"2024-04-25T00:13:19.427090+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427090+0000","last_undegraded":"2024-04-25T00:13:19.427090+0000","last_fullsized":"2024-04-25T00:13:19.427090+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431410+0000","last_change":"2024-04-25T00:13:19.431410+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431410+0000","last_undegraded":"2024-04-25T00:13:19.431410+0000","last_fullsized":"2024-04-25T00:13:19.431410+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431226+0000","last_change":"2024-04-25T00:13:19.431226+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431226+0000","last_undegraded":"2024-04-25T00:13:19.431226+0000","last_fullsized":"2024-04-25T00:13:19.431226+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419593+0000","last_change":"2024-04-25T00:13:19.419593+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419593+0000","last_undegraded":"2024-04-25T00:13:19.419593+0000","last_fullsized":"2024-04-25T00:13:19.419593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419376+0000","last_change":"2024-04-25T00:13:19.419376+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419376+0000","last_undegraded":"2024-04-25T00:13:19.419376+0000","last_fullsized":"2024-04-25T00:13:19.419376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431142+0000","last_change":"2024-04-25T00:13:19.431142+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431142+0000","last_undegraded":"2024-04-25T00:13:19.431142+0000","last_fullsized":"2024-04-25T00:13:19.431142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427186+0000","last_change":"2024-04-25T00:13:19.427186+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427186+0000","last_undegraded":"2024-04-25T00:13:19.427186+0000","last_fullsized":"2024-04-25T00:13:19.427186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428152+0000","last_change":"2024-04-25T00:13:19.428152+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428152+0000","last_undegraded":"2024-04-25T00:13:19.428152+0000","last_fullsized":"2024-04-25T00:13:19.428152+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420621+0000","last_change":"2024-04-25T00:13:19.420621+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420621+0000","last_undegraded":"2024-04-25T00:13:19.420621+0000","last_fullsized":"2024-04-25T00:13:19.420621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423212+0000","last_change":"2024-04-25T00:13:19.423212+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423212+0000","last_undegraded":"2024-04-25T00:13:19.423212+0000","last_fullsized":"2024-04-25T00:13:19.423212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430777+0000","last_change":"2024-04-25T00:13:19.430777+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430777+0000","last_undegraded":"2024-04-25T00:13:19.430777+0000","last_fullsized":"2024-04-25T00:13:19.430777+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422795+0000","last_change":"2024-04-25T00:13:19.422795+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422795+0000","last_undegraded":"2024-04-25T00:13:19.422795+0000","last_fullsized":"2024-04-25T00:13:19.422795+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422637+0000","last_change":"2024-04-25T00:13:19.422637+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422637+0000","last_undegraded":"2024-04-25T00:13:19.422637+0000","last_fullsized":"2024-04-25T00:13:19.422637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423888+0000","last_change":"2024-04-25T00:13:19.423888+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423888+0000","last_undegraded":"2024-04-25T00:13:19.423888+0000","last_fullsized":"2024-04-25T00:13:19.423888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.429151+0000","last_change":"2024-04-25T00:13:19.429151+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.429151+0000","last_undegraded":"2024-04-25T00:13:19.429151+0000","last_fullsized":"2024-04-25T00:13:19.429151+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428403+0000","last_change":"2024-04-25T00:13:19.428403+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428403+0000","last_undegraded":"2024-04-25T00:13:19.428403+0000","last_fullsized":"2024-04-25T00:13:19.428403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426616+0000","last_change":"2024-04-25T00:13:19.426616+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426616+0000","last_undegraded":"2024-04-25T00:13:19.426616+0000","last_fullsized":"2024-04-25T00:13:19.426616+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426605+0000","last_change":"2024-04-25T00:13:19.426605+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426605+0000","last_undegraded":"2024-04-25T00:13:19.426605+0000","last_fullsized":"2024-04-25T00:13:19.426605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420098+0000","last_change":"2024-04-25T00:13:19.420098+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420098+0000","last_undegraded":"2024-04-25T00:13:19.420098+0000","last_fullsized":"2024-04-25T00:13:19.420098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431639+0000","last_change":"2024-04-25T00:13:19.431639+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431639+0000","last_undegraded":"2024-04-25T00:13:19.431639+0000","last_fullsized":"2024-04-25T00:13:19.431639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.429845+0000","last_change":"2024-04-25T00:13:19.429845+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.429845+0000","last_undegraded":"2024-04-25T00:13:19.429845+0000","last_fullsized":"2024-04-25T00:13:19.429845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423891+0000","last_change":"2024-04-25T00:13:19.423891+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423891+0000","last_undegraded":"2024-04-25T00:13:19.423891+0000","last_fullsized":"2024-04-25T00:13:19.423891+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419810+0000","last_change":"2024-04-25T00:13:19.419810+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419810+0000","last_undegraded":"2024-04-25T00:13:19.419810+0000","last_fullsized":"2024-04-25T00:13:19.419810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.448008+0000","last_change":"2024-04-25T00:13:19.445153+0000","last_active":"2024-04-25T00:13:19.448008+0000","last_peered":"2024-04-25T00:13:19.448008+0000","last_clean":"2024-04-25T00:13:19.448008+0000","last_became_active":"2024-04-25T00:13:19.444845+0000","last_became_peered":"2024-04-25T00:13:19.444845+0000","last_unstale":"2024-04-25T00:13:19.448008+0000","last_undegraded":"2024-04-25T00:13:19.448008+0000","last_fullsized":"2024-04-25T00:13:19.448008+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:03.923113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426198+0000","last_change":"2024-04-25T00:13:19.426198+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426198+0000","last_undegraded":"2024-04-25T00:13:19.426198+0000","last_fullsized":"2024-04-25T00:13:19.426198+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.441989+0000","last_change":"2024-04-25T00:13:19.439199+0000","last_active":"2024-04-25T00:13:19.441989+0000","last_peered":"2024-04-25T00:13:19.441989+0000","last_clean":"2024-04-25T00:13:19.441989+0000","last_became_active":"2024-04-25T00:13:19.438895+0000","last_became_peered":"2024-04-25T00:13:19.438895+0000","last_unstale":"2024-04-25T00:13:19.441989+0000","last_undegraded":"2024-04-25T00:13:19.441989+0000","last_fullsized":"2024-04-25T00:13:19.441989+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:20.022144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428957+0000","last_change":"2024-04-25T00:13:19.428957+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428957+0000","last_undegraded":"2024-04-25T00:13:19.428957+0000","last_fullsized":"2024-04-25T00:13:19.428957+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447654+0000","last_change":"2024-04-25T00:13:19.442726+0000","last_active":"2024-04-25T00:13:19.447654+0000","last_peered":"2024-04-25T00:13:19.447654+0000","last_clean":"2024-04-25T00:13:19.447654+0000","last_became_active":"2024-04-25T00:13:19.442433+0000","last_became_peered":"2024-04-25T00:13:19.442433+0000","last_unstale":"2024-04-25T00:13:19.447654+0000","last_undegraded":"2024-04-25T00:13:19.447654+0000","last_fullsized":"2024-04-25T00:13:19.447654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:12.349352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423306+0000","last_change":"2024-04-25T00:13:19.423306+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423306+0000","last_undegraded":"2024-04-25T00:13:19.423306+0000","last_fullsized":"2024-04-25T00:13:19.423306+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.439924+0000","last_change":"2024-04-25T00:13:19.438204+0000","last_active":"2024-04-25T00:13:19.439924+0000","last_peered":"2024-04-25T00:13:19.439924+0000","last_clean":"2024-04-25T00:13:19.439924+0000","last_became_active":"2024-04-25T00:13:19.437845+0000","last_became_peered":"2024-04-25T00:13:19.437845+0000","last_unstale":"2024-04-25T00:13:19.439924+0000","last_undegraded":"2024-04-25T00:13:19.439924+0000","last_fullsized":"2024-04-25T00:13:19.439924+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:47.273633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430288+0000","last_change":"2024-04-25T00:13:19.430288+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430288+0000","last_undegraded":"2024-04-25T00:13:19.430288+0000","last_fullsized":"2024-04-25T00:13:19.430288+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.435286+0000","last_change":"2024-04-25T00:13:19.431189+0000","last_active":"2024-04-25T00:13:19.435286+0000","last_peered":"2024-04-25T00:13:19.435286+0000","last_clean":"2024-04-25T00:13:19.435286+0000","last_became_active":"2024-04-25T00:13:19.430933+0000","last_became_peered":"2024-04-25T00:13:19.430933+0000","last_unstale":"2024-04-25T00:13:19.435286+0000","last_undegraded":"2024-04-25T00:13:19.435286+0000","last_fullsized":"2024-04-25T00:13:19.435286+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:50.255023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.425319+0000","last_change":"2024-04-25T00:13:19.425319+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.425319+0000","last_undegraded":"2024-04-25T00:13:19.425319+0000","last_fullsized":"2024-04-25T00:13:19.425319+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442691+0000","last_change":"2024-04-25T00:13:19.441394+0000","last_active":"2024-04-25T00:13:19.442691+0000","last_peered":"2024-04-25T00:13:19.442691+0000","last_clean":"2024-04-25T00:13:19.442691+0000","last_became_active":"2024-04-25T00:13:19.438688+0000","last_became_peered":"2024-04-25T00:13:19.438688+0000","last_unstale":"2024-04-25T00:13:19.442691+0000","last_undegraded":"2024-04-25T00:13:19.442691+0000","last_fullsized":"2024-04-25T00:13:19.442691+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:28.553754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420922+0000","last_change":"2024-04-25T00:13:19.420922+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420922+0000","last_undegraded":"2024-04-25T00:13:19.420922+0000","last_fullsized":"2024-04-25T00:13:19.420922+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.436636+0000","last_change":"2024-04-25T00:13:19.433617+0000","last_active":"2024-04-25T00:13:19.436636+0000","last_peered":"2024-04-25T00:13:19.436636+0000","last_clean":"2024-04-25T00:13:19.436636+0000","last_became_active":"2024-04-25T00:13:19.433063+0000","last_became_peered":"2024-04-25T00:13:19.433063+0000","last_unstale":"2024-04-25T00:13:19.436636+0000","last_undegraded":"2024-04-25T00:13:19.436636+0000","last_fullsized":"2024-04-25T00:13:19.436636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:18.683206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423602+0000","last_change":"2024-04-25T00:13:19.423602+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423602+0000","last_undegraded":"2024-04-25T00:13:19.423602+0000","last_fullsized":"2024-04-25T00:13:19.423602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442465+0000","last_change":"2024-04-25T00:13:19.431998+0000","last_active":"2024-04-25T00:13:19.442465+0000","last_peered":"2024-04-25T00:13:19.442465+0000","last_clean":"2024-04-25T00:13:19.442465+0000","last_became_active":"2024-04-25T00:13:19.431728+0000","last_became_peered":"2024-04-25T00:13:19.431728+0000","last_unstale":"2024-04-25T00:13:19.442465+0000","last_undegraded":"2024-04-25T00:13:19.442465+0000","last_fullsized":"2024-04-25T00:13:19.442465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:04.083138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426189+0000","last_change":"2024-04-25T00:13:19.426189+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426189+0000","last_undegraded":"2024-04-25T00:13:19.426189+0000","last_fullsized":"2024-04-25T00:13:19.426189+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446032+0000","last_change":"2024-04-25T00:13:19.440887+0000","last_active":"2024-04-25T00:13:19.446032+0000","last_peered":"2024-04-25T00:13:19.446032+0000","last_clean":"2024-04-25T00:13:19.446032+0000","last_became_active":"2024-04-25T00:13:19.437895+0000","last_became_peered":"2024-04-25T00:13:19.437895+0000","last_unstale":"2024-04-25T00:13:19.446032+0000","last_undegraded":"2024-04-25T00:13:19.446032+0000","last_fullsized":"2024-04-25T00:13:19.446032+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:47.273633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422117+0000","last_change":"2024-04-25T00:13:19.422117+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422117+0000","last_undegraded":"2024-04-25T00:13:19.422117+0000","last_fullsized":"2024-04-25T00:13:19.422117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.443114+0000","last_change":"2024-04-25T00:13:19.437705+0000","last_active":"2024-04-25T00:13:19.443114+0000","last_peered":"2024-04-25T00:13:19.443114+0000","last_clean":"2024-04-25T00:13:19.443114+0000","last_became_active":"2024-04-25T00:13:19.437390+0000","last_became_peered":"2024-04-25T00:13:19.437390+0000","last_unstale":"2024-04-25T00:13:19.443114+0000","last_undegraded":"2024-04-25T00:13:19.443114+0000","last_fullsized":"2024-04-25T00:13:19.443114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:44.096993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.421441+0000","last_change":"2024-04-25T00:13:19.421441+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.421441+0000","last_undegraded":"2024-04-25T00:13:19.421441+0000","last_fullsized":"2024-04-25T00:13:19.421441+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442484+0000","last_change":"2024-04-25T00:13:19.438746+0000","last_active":"2024-04-25T00:13:19.442484+0000","last_peered":"2024-04-25T00:13:19.442484+0000","last_clean":"2024-04-25T00:13:19.442484+0000","last_became_active":"2024-04-25T00:13:19.437749+0000","last_became_peered":"2024-04-25T00:13:19.437749+0000","last_unstale":"2024-04-25T00:13:19.442484+0000","last_undegraded":"2024-04-25T00:13:19.442484+0000","last_fullsized":"2024-04-25T00:13:19.442484+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:39.640503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426025+0000","last_change":"2024-04-25T00:13:19.426025+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426025+0000","last_undegraded":"2024-04-25T00:13:19.426025+0000","last_fullsized":"2024-04-25T00:13:19.426025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.448034+0000","last_change":"2024-04-25T00:13:19.445212+0000","last_active":"2024-04-25T00:13:19.448034+0000","last_peered":"2024-04-25T00:13:19.448034+0000","last_clean":"2024-04-25T00:13:19.448034+0000","last_became_active":"2024-04-25T00:13:19.444953+0000","last_became_peered":"2024-04-25T00:13:19.444953+0000","last_unstale":"2024-04-25T00:13:19.448034+0000","last_undegraded":"2024-04-25T00:13:19.448034+0000","last_fullsized":"2024-04-25T00:13:19.448034+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:28.280706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426254+0000","last_change":"2024-04-25T00:13:19.426254+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426254+0000","last_undegraded":"2024-04-25T00:13:19.426254+0000","last_fullsized":"2024-04-25T00:13:19.426254+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445679+0000","last_change":"2024-04-25T00:13:19.439464+0000","last_active":"2024-04-25T00:13:19.445679+0000","last_peered":"2024-04-25T00:13:19.445679+0000","last_clean":"2024-04-25T00:13:19.445679+0000","last_became_active":"2024-04-25T00:13:19.439143+0000","last_became_peered":"2024-04-25T00:13:19.439143+0000","last_unstale":"2024-04-25T00:13:19.445679+0000","last_undegraded":"2024-04-25T00:13:19.445679+0000","last_fullsized":"2024-04-25T00:13:19.445679+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:03.413067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.435804+0000","last_change":"2024-04-25T00:13:19.435804+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.435804+0000","last_undegraded":"2024-04-25T00:13:19.435804+0000","last_fullsized":"2024-04-25T00:13:19.435804+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446509+0000","last_change":"2024-04-25T00:13:19.443972+0000","last_active":"2024-04-25T00:13:19.446509+0000","last_peered":"2024-04-25T00:13:19.446509+0000","last_clean":"2024-04-25T00:13:19.446509+0000","last_became_active":"2024-04-25T00:13:19.443634+0000","last_became_peered":"2024-04-25T00:13:19.443634+0000","last_unstale":"2024-04-25T00:13:19.446509+0000","last_undegraded":"2024-04-25T00:13:19.446509+0000","last_fullsized":"2024-04-25T00:13:19.446509+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:03.413067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420470+0000","last_change":"2024-04-25T00:13:19.420470+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420470+0000","last_undegraded":"2024-04-25T00:13:19.420470+0000","last_fullsized":"2024-04-25T00:13:19.420470+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.436847+0000","last_change":"2024-04-25T00:13:19.430607+0000","last_active":"2024-04-25T00:13:19.436847+0000","last_peered":"2024-04-25T00:13:19.436847+0000","last_clean":"2024-04-25T00:13:19.436847+0000","last_became_active":"2024-04-25T00:13:19.430240+0000","last_became_peered":"2024-04-25T00:13:19.430240+0000","last_unstale":"2024-04-25T00:13:19.436847+0000","last_undegraded":"2024-04-25T00:13:19.436847+0000","last_fullsized":"2024-04-25T00:13:19.436847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:07.859795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428661+0000","last_change":"2024-04-25T00:13:19.428661+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428661+0000","last_undegraded":"2024-04-25T00:13:19.428661+0000","last_fullsized":"2024-04-25T00:13:19.428661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.443661+0000","last_change":"2024-04-25T00:13:19.438379+0000","last_active":"2024-04-25T00:13:19.443661+0000","last_peered":"2024-04-25T00:13:19.443661+0000","last_clean":"2024-04-25T00:13:19.443661+0000","last_became_active":"2024-04-25T00:13:19.438093+0000","last_became_peered":"2024-04-25T00:13:19.438093+0000","last_unstale":"2024-04-25T00:13:19.443661+0000","last_undegraded":"2024-04-25T00:13:19.443661+0000","last_fullsized":"2024-04-25T00:13:19.443661+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:28.036116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.424260+0000","last_change":"2024-04-25T00:13:19.424260+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.424260+0000","last_undegraded":"2024-04-25T00:13:19.424260+0000","last_fullsized":"2024-04-25T00:13:19.424260+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445046+0000","last_change":"2024-04-25T00:13:19.440461+0000","last_active":"2024-04-25T00:13:19.445046+0000","last_peered":"2024-04-25T00:13:19.445046+0000","last_clean":"2024-04-25T00:13:19.445046+0000","last_became_active":"2024-04-25T00:13:19.440197+0000","last_became_peered":"2024-04-25T00:13:19.440197+0000","last_unstale":"2024-04-25T00:13:19.445046+0000","last_undegraded":"2024-04-25T00:13:19.445046+0000","last_fullsized":"2024-04-25T00:13:19.445046+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:37.585343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420102+0000","last_change":"2024-04-25T00:13:19.420102+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420102+0000","last_undegraded":"2024-04-25T00:13:19.420102+0000","last_fullsized":"2024-04-25T00:13:19.420102+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442434+0000","last_change":"2024-04-25T00:13:19.438945+0000","last_active":"2024-04-25T00:13:19.442434+0000","last_peered":"2024-04-25T00:13:19.442434+0000","last_clean":"2024-04-25T00:13:19.442434+0000","last_became_active":"2024-04-25T00:13:19.438052+0000","last_became_peered":"2024-04-25T00:13:19.438052+0000","last_unstale":"2024-04-25T00:13:19.442434+0000","last_undegraded":"2024-04-25T00:13:19.442434+0000","last_fullsized":"2024-04-25T00:13:19.442434+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:21.586818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427651+0000","last_change":"2024-04-25T00:13:19.427651+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427651+0000","last_undegraded":"2024-04-25T00:13:19.427651+0000","last_fullsized":"2024-04-25T00:13:19.427651+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445826+0000","last_change":"2024-04-25T00:13:19.440459+0000","last_active":"2024-04-25T00:13:19.445826+0000","last_peered":"2024-04-25T00:13:19.445826+0000","last_clean":"2024-04-25T00:13:19.445826+0000","last_became_active":"2024-04-25T00:13:19.438442+0000","last_became_peered":"2024-04-25T00:13:19.438442+0000","last_unstale":"2024-04-25T00:13:19.445826+0000","last_undegraded":"2024-04-25T00:13:19.445826+0000","last_fullsized":"2024-04-25T00:13:19.445826+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:42.830716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420129+0000","last_change":"2024-04-25T00:13:19.420129+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420129+0000","last_undegraded":"2024-04-25T00:13:19.420129+0000","last_fullsized":"2024-04-25T00:13:19.420129+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442276+0000","last_change":"2024-04-25T00:13:19.440785+0000","last_active":"2024-04-25T00:13:19.442276+0000","last_peered":"2024-04-25T00:13:19.442276+0000","last_clean":"2024-04-25T00:13:19.442276+0000","last_became_active":"2024-04-25T00:13:19.440512+0000","last_became_peered":"2024-04-25T00:13:19.440512+0000","last_unstale":"2024-04-25T00:13:19.442276+0000","last_undegraded":"2024-04-25T00:13:19.442276+0000","last_fullsized":"2024-04-25T00:13:19.442276+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:35.767979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423501+0000","last_change":"2024-04-25T00:13:19.423501+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423501+0000","last_undegraded":"2024-04-25T00:13:19.423501+0000","last_fullsized":"2024-04-25T00:13:19.423501+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447676+0000","last_change":"2024-04-25T00:13:19.443427+0000","last_active":"2024-04-25T00:13:19.447676+0000","last_peered":"2024-04-25T00:13:19.447676+0000","last_clean":"2024-04-25T00:13:19.447676+0000","last_became_active":"2024-04-25T00:13:19.443147+0000","last_became_peered":"2024-04-25T00:13:19.443147+0000","last_unstale":"2024-04-25T00:13:19.447676+0000","last_undegraded":"2024-04-25T00:13:19.447676+0000","last_fullsized":"2024-04-25T00:13:19.447676+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:17.551103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.435341+0000","last_change":"2024-04-25T00:13:19.435341+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.435341+0000","last_undegraded":"2024-04-25T00:13:19.435341+0000","last_fullsized":"2024-04-25T00:13:19.435341+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445110+0000","last_change":"2024-04-25T00:13:19.441302+0000","last_active":"2024-04-25T00:13:19.445110+0000","last_peered":"2024-04-25T00:13:19.445110+0000","last_clean":"2024-04-25T00:13:19.445110+0000","last_became_active":"2024-04-25T00:13:19.440951+0000","last_became_peered":"2024-04-25T00:13:19.440951+0000","last_unstale":"2024-04-25T00:13:19.445110+0000","last_undegraded":"2024-04-25T00:13:19.445110+0000","last_fullsized":"2024-04-25T00:13:19.445110+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:07.859795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419439+0000","last_change":"2024-04-25T00:13:19.419439+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419439+0000","last_undegraded":"2024-04-25T00:13:19.419439+0000","last_fullsized":"2024-04-25T00:13:19.419439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.441866+0000","last_change":"2024-04-25T00:13:19.438811+0000","last_active":"2024-04-25T00:13:19.441866+0000","last_peered":"2024-04-25T00:13:19.441866+0000","last_clean":"2024-04-25T00:13:19.441866+0000","last_became_active":"2024-04-25T00:13:19.438527+0000","last_became_peered":"2024-04-25T00:13:19.438527+0000","last_unstale":"2024-04-25T00:13:19.441866+0000","last_undegraded":"2024-04-25T00:13:19.441866+0000","last_fullsized":"2024-04-25T00:13:19.441866+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:13.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.424634+0000","last_change":"2024-04-25T00:13:19.424634+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.424634+0000","last_undegraded":"2024-04-25T00:13:19.424634+0000","last_fullsized":"2024-04-25T00:13:19.424634+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446400+0000","last_change":"2024-04-25T00:13:19.442162+0000","last_active":"2024-04-25T00:13:19.446400+0000","last_peered":"2024-04-25T00:13:19.446400+0000","last_clean":"2024-04-25T00:13:19.446400+0000","last_became_active":"2024-04-25T00:13:19.441874+0000","last_became_peered":"2024-04-25T00:13:19.441874+0000","last_unstale":"2024-04-25T00:13:19.446400+0000","last_undegraded":"2024-04-25T00:13:19.446400+0000","last_fullsized":"2024-04-25T00:13:19.446400+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:39.640503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430355+0000","last_change":"2024-04-25T00:13:19.430355+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430355+0000","last_undegraded":"2024-04-25T00:13:19.430355+0000","last_fullsized":"2024-04-25T00:13:19.430355+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446460+0000","last_change":"2024-04-25T00:13:19.443257+0000","last_active":"2024-04-25T00:13:19.446460+0000","last_peered":"2024-04-25T00:13:19.446460+0000","last_clean":"2024-04-25T00:13:19.446460+0000","last_became_active":"2024-04-25T00:13:19.442853+0000","last_became_peered":"2024-04-25T00:13:19.442853+0000","last_unstale":"2024-04-25T00:13:19.446460+0000","last_undegraded":"2024-04-25T00:13:19.446460+0000","last_fullsized":"2024-04-25T00:13:19.446460+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:21.586818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427928+0000","last_change":"2024-04-25T00:13:19.427928+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427928+0000","last_undegraded":"2024-04-25T00:13:19.427928+0000","last_fullsized":"2024-04-25T00:13:19.427928+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447682+0000","last_change":"2024-04-25T00:13:19.444471+0000","last_active":"2024-04-25T00:13:19.447682+0000","last_peered":"2024-04-25T00:13:19.447682+0000","last_clean":"2024-04-25T00:13:19.447682+0000","last_became_active":"2024-04-25T00:13:19.444197+0000","last_became_peered":"2024-04-25T00:13:19.444197+0000","last_unstale":"2024-04-25T00:13:19.447682+0000","last_undegraded":"2024-04-25T00:13:19.447682+0000","last_fullsized":"2024-04-25T00:13:19.447682+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:29.987108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419511+0000","last_change":"2024-04-25T00:13:19.419511+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419511+0000","last_undegraded":"2024-04-25T00:13:19.419511+0000","last_fullsized":"2024-04-25T00:13:19.419511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447962+0000","last_change":"2024-04-25T00:13:19.445103+0000","last_active":"2024-04-25T00:13:19.447962+0000","last_peered":"2024-04-25T00:13:19.447962+0000","last_clean":"2024-04-25T00:13:19.447962+0000","last_became_active":"2024-04-25T00:13:19.444652+0000","last_became_peered":"2024-04-25T00:13:19.444652+0000","last_unstale":"2024-04-25T00:13:19.447962+0000","last_undegraded":"2024-04-25T00:13:19.447962+0000","last_fullsized":"2024-04-25T00:13:19.447962+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:39.691842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419832+0000","last_change":"2024-04-25T00:13:19.419832+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419832+0000","last_undegraded":"2024-04-25T00:13:19.419832+0000","last_fullsized":"2024-04-25T00:13:19.419832+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.448638+0000","last_change":"2024-04-25T00:13:19.442646+0000","last_active":"2024-04-25T00:13:19.448638+0000","last_peered":"2024-04-25T00:13:19.448638+0000","last_clean":"2024-04-25T00:13:19.448638+0000","last_became_active":"2024-04-25T00:13:19.438848+0000","last_became_peered":"2024-04-25T00:13:19.438848+0000","last_unstale":"2024-04-25T00:13:19.448638+0000","last_undegraded":"2024-04-25T00:13:19.448638+0000","last_fullsized":"2024-04-25T00:13:19.448638+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:13.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.425579+0000","last_change":"2024-04-25T00:13:19.425579+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.425579+0000","last_undegraded":"2024-04-25T00:13:19.425579+0000","last_fullsized":"2024-04-25T00:13:19.425579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.439322+0000","last_change":"2024-04-25T00:13:19.436516+0000","last_active":"2024-04-25T00:13:19.439322+0000","last_peered":"2024-04-25T00:13:19.439322+0000","last_clean":"2024-04-25T00:13:19.439322+0000","last_became_active":"2024-04-25T00:13:19.435936+0000","last_became_peered":"2024-04-25T00:13:19.435936+0000","last_unstale":"2024-04-25T00:13:19.439322+0000","last_undegraded":"2024-04-25T00:13:19.439322+0000","last_fullsized":"2024-04-25T00:13:19.439322+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:57.960615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.424954+0000","last_change":"2024-04-25T00:13:19.424954+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.424954+0000","last_undegraded":"2024-04-25T00:13:19.424954+0000","last_fullsized":"2024-04-25T00:13:19.424954+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.429368+0000","last_change":"2024-04-25T00:13:19.427399+0000","last_active":"2024-04-25T00:13:19.429368+0000","last_peered":"2024-04-25T00:13:19.429368+0000","last_clean":"2024-04-25T00:13:19.429368+0000","last_became_active":"2024-04-25T00:13:19.427145+0000","last_became_peered":"2024-04-25T00:13:19.427145+0000","last_unstale":"2024-04-25T00:13:19.429368+0000","last_undegraded":"2024-04-25T00:13:19.429368+0000","last_fullsized":"2024-04-25T00:13:19.429368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:19.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430930+0000","last_change":"2024-04-25T00:13:19.430930+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430930+0000","last_undegraded":"2024-04-25T00:13:19.430930+0000","last_fullsized":"2024-04-25T00:13:19.430930+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.441056+0000","last_change":"2024-04-25T00:13:19.438635+0000","last_active":"2024-04-25T00:13:19.441056+0000","last_peered":"2024-04-25T00:13:19.441056+0000","last_clean":"2024-04-25T00:13:19.441056+0000","last_became_active":"2024-04-25T00:13:19.438328+0000","last_became_peered":"2024-04-25T00:13:19.438328+0000","last_unstale":"2024-04-25T00:13:19.441056+0000","last_undegraded":"2024-04-25T00:13:19.441056+0000","last_fullsized":"2024-04-25T00:13:19.441056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:17.223076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419237+0000","last_change":"2024-04-25T00:13:19.419237+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419237+0000","last_undegraded":"2024-04-25T00:13:19.419237+0000","last_fullsized":"2024-04-25T00:13:19.419237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.433512+0000","last_change":"2024-04-25T00:13:19.430683+0000","last_active":"2024-04-25T00:13:19.433512+0000","last_peered":"2024-04-25T00:13:19.433512+0000","last_clean":"2024-04-25T00:13:19.433512+0000","last_became_active":"2024-04-25T00:13:19.430464+0000","last_became_peered":"2024-04-25T00:13:19.430464+0000","last_unstale":"2024-04-25T00:13:19.433512+0000","last_undegraded":"2024-04-25T00:13:19.433512+0000","last_fullsized":"2024-04-25T00:13:19.433512+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:36.678931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":239,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.424193+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:13:19.424193+0000","last_peered":"2024-04-25T00:13:19.424193+0000","last_clean":"2024-04-25T00:13:19.424193+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:13:19.424193+0000","last_undegraded":"2024-04-25T00:13:19.424193+0000","last_fullsized":"2024-04-25T00:13:19.424193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1294336,"data_stored":901561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":700416,"data_stored":311193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1282048,"data_stored":894902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:13:24.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:13:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f771c000640 1 Processor -- start 2024-04-25T00:13:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f771c000640 1 -- start start 2024-04-25T00:13:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f771c000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f771415d8f0 0x7f771415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f771c000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77140d1030 con 0x7f771406c2f0 2024-04-25T00:13:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f771c000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77140cc470 con 0x7f771406f9e0 2024-04-25T00:13:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f771c000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f77140ccda0 con 0x7f771415d8f0 2024-04-25T00:13:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f7718c00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f771406c2f0 0x7f77140d0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54868/0 (socket says 172.21.15.83:54868) 2024-04-25T00:13:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f7718c00640 1 -- 172.21.15.83:0/3299079256 learned_addr learned my addr 172.21.15.83:0/3299079256 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f771a000640 1 --2- 172.21.15.83:0/3299079256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f771415d8f0 0x7f771415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f7713e00640 1 -- 172.21.15.83:0/3299079256 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2049637380 0 0) 0x7f77140d1030 con 0x7f771406c2f0 2024-04-25T00:13:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f7713e00640 1 -- 172.21.15.83:0/3299079256 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76fc003530 con 0x7f771406c2f0 2024-04-25T00:13:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f7713e00640 1 -- 172.21.15.83:0/3299079256 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 188089117 0 0) 0x7f76fc003530 con 0x7f771406c2f0 2024-04-25T00:13:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f771a000640 1 -- 172.21.15.83:0/3299079256 >> v1:172.21.15.86:6789/0 conn(0x7f771406f9e0 legacy=0x7f771406fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f771a000640 1 -- 172.21.15.83:0/3299079256 >> v1:172.21.15.83:6789/0 conn(0x7f771406c2f0 legacy=0x7f77140d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.057+0000 7f771a000640 1 -- 172.21.15.83:0/3299079256 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77140cc750 con 0x7f771415d8f0 2024-04-25T00:13:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.061+0000 7f771a000640 1 --2- 172.21.15.83:0/3299079256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f771415d8f0 0x7f771415dda0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f770c009dc0 tx=0x7f770c01f0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ffd79213441ed1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.061+0000 7f7713e00640 1 -- 172.21.15.83:0/3299079256 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c0b5070 con 0x7f771415d8f0 2024-04-25T00:13:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.061+0000 7f7713e00640 1 -- 172.21.15.83:0/3299079256 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f770c0a8c30 con 0x7f771415d8f0 2024-04-25T00:13:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.061+0000 7f7713e00640 1 -- 172.21.15.83:0/3299079256 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700001680 con 0x7f771415d8f0 2024-04-25T00:13:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.061+0000 7f771c000640 1 -- 172.21.15.83:0/3299079256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f771415d8f0 msgr2=0x7f771415dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.061+0000 7f771c000640 1 --2- 172.21.15.83:0/3299079256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f771415d8f0 0x7f771415dda0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f770c009dc0 tx=0x7f770c01f0a0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.061+0000 7f771c000640 1 -- 172.21.15.83:0/3299079256 shutdown_connections 2024-04-25T00:13:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.061+0000 7f771c000640 1 --2- 172.21.15.83:0/3299079256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f771415d8f0 0x7f771415dda0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.061+0000 7f771c000640 1 -- 172.21.15.83:0/3299079256 >> 172.21.15.83:0/3299079256 conn(0x7f77140d1c80 msgr2=0x7f77140d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.061+0000 7f771c000640 1 -- 172.21.15.83:0/3299079256 shutdown_connections 2024-04-25T00:13:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.061+0000 7f771c000640 1 -- 172.21.15.83:0/3299079256 wait complete. 2024-04-25T00:13:25.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f771c000640 1 Processor -- start 2024-04-25T00:13:25.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f771c000640 1 -- start start 2024-04-25T00:13:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f771c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f771406c2f0 0x7f77141e8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f771c000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f771406f9e0 0x7f771409bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f771c000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f771415d8f0 0x7f771409c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f771c000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f771406f4b0 con 0x7f771406f9e0 2024-04-25T00:13:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f771c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f771406f330 con 0x7f771406c2f0 2024-04-25T00:13:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f771c000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f771415e2e0 con 0x7f771415d8f0 2024-04-25T00:13:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f7718c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f771406f9e0 0x7f771409bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f7718c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f771406f9e0 0x7f771409bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56788/0 (socket says 172.21.15.83:56788) 2024-04-25T00:13:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f7718c00640 1 -- 172.21.15.83:0/1809793299 learned_addr learned my addr 172.21.15.83:0/1809793299 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f771a000640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f771415d8f0 0x7f771409c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f7719600640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f771406c2f0 0x7f77141e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f7718c00640 1 -- 172.21.15.83:0/1809793299 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f771415d8f0 msgr2=0x7f771409c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f7718c00640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f771415d8f0 0x7f771409c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f7718c00640 1 -- 172.21.15.83:0/1809793299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f771406c2f0 msgr2=0x7f77141e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f7718c00640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f771406c2f0 0x7f77141e8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f7718c00640 1 -- 172.21.15.83:0/1809793299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77140ca310 con 0x7f771406f9e0 2024-04-25T00:13:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f7719600640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f771406c2f0 0x7f77141e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f7718c00640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f771406f9e0 0x7f771409bb60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f770000c730 tx=0x7f770000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f7711600640 1 -- 172.21.15.83:0/1809793299 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700017070 con 0x7f771406f9e0 2024-04-25T00:13:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f7711600640 1 -- 172.21.15.83:0/1809793299 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7700005b00 con 0x7f771406f9e0 2024-04-25T00:13:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f7711600640 1 -- 172.21.15.83:0/1809793299 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700007070 con 0x7f771406f9e0 2024-04-25T00:13:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f771c000640 1 -- 172.21.15.83:0/1809793299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77140ca5f0 con 0x7f771406f9e0 2024-04-25T00:13:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.065+0000 7f771c000640 1 -- 172.21.15.83:0/1809793299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77140cabb0 con 0x7f771406f9e0 2024-04-25T00:13:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.069+0000 7f771c000640 1 -- 172.21.15.83:0/1809793299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f771409fdd0 con 0x7f771406f9e0 2024-04-25T00:13:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.069+0000 7f7711600640 1 -- 172.21.15.83:0/1809793299 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f770001a070 con 0x7f771406f9e0 2024-04-25T00:13:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.069+0000 7f7711600640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76f40678f0 0x7f76f4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.069+0000 7f7711600640 1 -- 172.21.15.83:0/1809793299 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f77000852a0 con 0x7f771406f9e0 2024-04-25T00:13:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.069+0000 7f7719600640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76f40678f0 0x7f76f4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.069+0000 7f7719600640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76f40678f0 0x7f76f4069db0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7708000f70 tx=0x7f7708009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.073+0000 7f7711600640 1 -- 172.21.15.83:0/1809793299 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f77000555f0 con 0x7f771406f9e0 2024-04-25T00:13:25.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.225+0000 7f771c000640 1 -- 172.21.15.83:0/1809793299 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f771406f7c0 con 0x7f76f40678f0 2024-04-25T00:13:25.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.229+0000 7f7711600640 1 -- 172.21.15.83:0/1809793299 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246710 (secure 0 0 0) 0x7f771406f7c0 con 0x7f76f40678f0 2024-04-25T00:13:25.234 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:25.235 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:13:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.233+0000 7f771c000640 1 -- 172.21.15.83:0/1809793299 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76f40678f0 msgr2=0x7f76f4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.233+0000 7f771c000640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76f40678f0 0x7f76f4069db0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7708000f70 tx=0x7f7708009040 comp rx=0 tx=0).stop 2024-04-25T00:13:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.233+0000 7f771c000640 1 -- 172.21.15.83:0/1809793299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f771406f9e0 msgr2=0x7f771409bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.233+0000 7f771c000640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f771406f9e0 0x7f771409bb60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f770000c730 tx=0x7f770000cbf0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.233+0000 7f771c000640 1 -- 172.21.15.83:0/1809793299 shutdown_connections 2024-04-25T00:13:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.233+0000 7f771c000640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76f40678f0 0x7f76f4069db0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.233+0000 7f771c000640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f771415d8f0 0x7f771409c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.233+0000 7f771c000640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f771406f9e0 0x7f771409bb60 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.233+0000 7f771c000640 1 --2- 172.21.15.83:0/1809793299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f771406c2f0 0x7f77141e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.233+0000 7f771c000640 1 -- 172.21.15.83:0/1809793299 >> 172.21.15.83:0/1809793299 conn(0x7f77140d1c80 msgr2=0x7f77140d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.233+0000 7f771c000640 1 -- 172.21.15.83:0/1809793299 shutdown_connections 2024-04-25T00:13:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.233+0000 7f771c000640 1 -- 172.21.15.83:0/1809793299 wait complete. 2024-04-25T00:13:25.253 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-04-25T00:13:24.470989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226940,"kb_used_data":6560,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747780,"statfs":{"total":773094113280,"available":772861726720,"internally_reserved":0,"allocated":6717440,"data_stored":3630326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000984"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427611+0000","last_change":"2024-04-25T00:13:19.427611+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427611+0000","last_undegraded":"2024-04-25T00:13:19.427611+0000","last_fullsized":"2024-04-25T00:13:19.427611+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427995+0000","last_change":"2024-04-25T00:13:19.427995+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427995+0000","last_undegraded":"2024-04-25T00:13:19.427995+0000","last_fullsized":"2024-04-25T00:13:19.427995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420276+0000","last_change":"2024-04-25T00:13:19.420276+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420276+0000","last_undegraded":"2024-04-25T00:13:19.420276+0000","last_fullsized":"2024-04-25T00:13:19.420276+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428639+0000","last_change":"2024-04-25T00:13:19.428639+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428639+0000","last_undegraded":"2024-04-25T00:13:19.428639+0000","last_fullsized":"2024-04-25T00:13:19.428639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420673+0000","last_change":"2024-04-25T00:13:19.420673+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420673+0000","last_undegraded":"2024-04-25T00:13:19.420673+0000","last_fullsized":"2024-04-25T00:13:19.420673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422946+0000","last_change":"2024-04-25T00:13:19.422946+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422946+0000","last_undegraded":"2024-04-25T00:13:19.422946+0000","last_fullsized":"2024-04-25T00:13:19.422946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431248+0000","last_change":"2024-04-25T00:13:19.431248+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431248+0000","last_undegraded":"2024-04-25T00:13:19.431248+0000","last_fullsized":"2024-04-25T00:13:19.431248+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.429195+0000","last_change":"2024-04-25T00:13:19.429195+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.429195+0000","last_undegraded":"2024-04-25T00:13:19.429195+0000","last_fullsized":"2024-04-25T00:13:19.429195+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.421312+0000","last_change":"2024-04-25T00:13:19.421312+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.421312+0000","last_undegraded":"2024-04-25T00:13:19.421312+0000","last_fullsized":"2024-04-25T00:13:19.421312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423950+0000","last_change":"2024-04-25T00:13:19.423950+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423950+0000","last_undegraded":"2024-04-25T00:13:19.423950+0000","last_fullsized":"2024-04-25T00:13:19.423950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427090+0000","last_change":"2024-04-25T00:13:19.427090+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427090+0000","last_undegraded":"2024-04-25T00:13:19.427090+0000","last_fullsized":"2024-04-25T00:13:19.427090+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431410+0000","last_change":"2024-04-25T00:13:19.431410+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431410+0000","last_undegraded":"2024-04-25T00:13:19.431410+0000","last_fullsized":"2024-04-25T00:13:19.431410+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431226+0000","last_change":"2024-04-25T00:13:19.431226+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431226+0000","last_undegraded":"2024-04-25T00:13:19.431226+0000","last_fullsized":"2024-04-25T00:13:19.431226+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419593+0000","last_change":"2024-04-25T00:13:19.419593+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419593+0000","last_undegraded":"2024-04-25T00:13:19.419593+0000","last_fullsized":"2024-04-25T00:13:19.419593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419376+0000","last_change":"2024-04-25T00:13:19.419376+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419376+0000","last_undegraded":"2024-04-25T00:13:19.419376+0000","last_fullsized":"2024-04-25T00:13:19.419376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431142+0000","last_change":"2024-04-25T00:13:19.431142+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431142+0000","last_undegraded":"2024-04-25T00:13:19.431142+0000","last_fullsized":"2024-04-25T00:13:19.431142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427186+0000","last_change":"2024-04-25T00:13:19.427186+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427186+0000","last_undegraded":"2024-04-25T00:13:19.427186+0000","last_fullsized":"2024-04-25T00:13:19.427186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428152+0000","last_change":"2024-04-25T00:13:19.428152+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428152+0000","last_undegraded":"2024-04-25T00:13:19.428152+0000","last_fullsized":"2024-04-25T00:13:19.428152+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420621+0000","last_change":"2024-04-25T00:13:19.420621+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420621+0000","last_undegraded":"2024-04-25T00:13:19.420621+0000","last_fullsized":"2024-04-25T00:13:19.420621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423212+0000","last_change":"2024-04-25T00:13:19.423212+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423212+0000","last_undegraded":"2024-04-25T00:13:19.423212+0000","last_fullsized":"2024-04-25T00:13:19.423212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430777+0000","last_change":"2024-04-25T00:13:19.430777+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430777+0000","last_undegraded":"2024-04-25T00:13:19.430777+0000","last_fullsized":"2024-04-25T00:13:19.430777+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422795+0000","last_change":"2024-04-25T00:13:19.422795+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422795+0000","last_undegraded":"2024-04-25T00:13:19.422795+0000","last_fullsized":"2024-04-25T00:13:19.422795+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422637+0000","last_change":"2024-04-25T00:13:19.422637+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422637+0000","last_undegraded":"2024-04-25T00:13:19.422637+0000","last_fullsized":"2024-04-25T00:13:19.422637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423888+0000","last_change":"2024-04-25T00:13:19.423888+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423888+0000","last_undegraded":"2024-04-25T00:13:19.423888+0000","last_fullsized":"2024-04-25T00:13:19.423888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.429151+0000","last_change":"2024-04-25T00:13:19.429151+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.429151+0000","last_undegraded":"2024-04-25T00:13:19.429151+0000","last_fullsized":"2024-04-25T00:13:19.429151+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428403+0000","last_change":"2024-04-25T00:13:19.428403+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428403+0000","last_undegraded":"2024-04-25T00:13:19.428403+0000","last_fullsized":"2024-04-25T00:13:19.428403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426616+0000","last_change":"2024-04-25T00:13:19.426616+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426616+0000","last_undegraded":"2024-04-25T00:13:19.426616+0000","last_fullsized":"2024-04-25T00:13:19.426616+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426605+0000","last_change":"2024-04-25T00:13:19.426605+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426605+0000","last_undegraded":"2024-04-25T00:13:19.426605+0000","last_fullsized":"2024-04-25T00:13:19.426605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420098+0000","last_change":"2024-04-25T00:13:19.420098+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420098+0000","last_undegraded":"2024-04-25T00:13:19.420098+0000","last_fullsized":"2024-04-25T00:13:19.420098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431639+0000","last_change":"2024-04-25T00:13:19.431639+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431639+0000","last_undegraded":"2024-04-25T00:13:19.431639+0000","last_fullsized":"2024-04-25T00:13:19.431639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.429845+0000","last_change":"2024-04-25T00:13:19.429845+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.429845+0000","last_undegraded":"2024-04-25T00:13:19.429845+0000","last_fullsized":"2024-04-25T00:13:19.429845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423891+0000","last_change":"2024-04-25T00:13:19.423891+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423891+0000","last_undegraded":"2024-04-25T00:13:19.423891+0000","last_fullsized":"2024-04-25T00:13:19.423891+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419810+0000","last_change":"2024-04-25T00:13:19.419810+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419810+0000","last_undegraded":"2024-04-25T00:13:19.419810+0000","last_fullsized":"2024-04-25T00:13:19.419810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.448008+0000","last_change":"2024-04-25T00:13:19.445153+0000","last_active":"2024-04-25T00:13:19.448008+0000","last_peered":"2024-04-25T00:13:19.448008+0000","last_clean":"2024-04-25T00:13:19.448008+0000","last_became_active":"2024-04-25T00:13:19.444845+0000","last_became_peered":"2024-04-25T00:13:19.444845+0000","last_unstale":"2024-04-25T00:13:19.448008+0000","last_undegraded":"2024-04-25T00:13:19.448008+0000","last_fullsized":"2024-04-25T00:13:19.448008+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:03.923113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426198+0000","last_change":"2024-04-25T00:13:19.426198+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426198+0000","last_undegraded":"2024-04-25T00:13:19.426198+0000","last_fullsized":"2024-04-25T00:13:19.426198+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.441989+0000","last_change":"2024-04-25T00:13:19.439199+0000","last_active":"2024-04-25T00:13:19.441989+0000","last_peered":"2024-04-25T00:13:19.441989+0000","last_clean":"2024-04-25T00:13:19.441989+0000","last_became_active":"2024-04-25T00:13:19.438895+0000","last_became_peered":"2024-04-25T00:13:19.438895+0000","last_unstale":"2024-04-25T00:13:19.441989+0000","last_undegraded":"2024-04-25T00:13:19.441989+0000","last_fullsized":"2024-04-25T00:13:19.441989+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:20.022144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428957+0000","last_change":"2024-04-25T00:13:19.428957+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428957+0000","last_undegraded":"2024-04-25T00:13:19.428957+0000","last_fullsized":"2024-04-25T00:13:19.428957+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447654+0000","last_change":"2024-04-25T00:13:19.442726+0000","last_active":"2024-04-25T00:13:19.447654+0000","last_peered":"2024-04-25T00:13:19.447654+0000","last_clean":"2024-04-25T00:13:19.447654+0000","last_became_active":"2024-04-25T00:13:19.442433+0000","last_became_peered":"2024-04-25T00:13:19.442433+0000","last_unstale":"2024-04-25T00:13:19.447654+0000","last_undegraded":"2024-04-25T00:13:19.447654+0000","last_fullsized":"2024-04-25T00:13:19.447654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:12.349352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423306+0000","last_change":"2024-04-25T00:13:19.423306+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423306+0000","last_undegraded":"2024-04-25T00:13:19.423306+0000","last_fullsized":"2024-04-25T00:13:19.423306+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.439924+0000","last_change":"2024-04-25T00:13:19.438204+0000","last_active":"2024-04-25T00:13:19.439924+0000","last_peered":"2024-04-25T00:13:19.439924+0000","last_clean":"2024-04-25T00:13:19.439924+0000","last_became_active":"2024-04-25T00:13:19.437845+0000","last_became_peered":"2024-04-25T00:13:19.437845+0000","last_unstale":"2024-04-25T00:13:19.439924+0000","last_undegraded":"2024-04-25T00:13:19.439924+0000","last_fullsized":"2024-04-25T00:13:19.439924+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:47.273633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430288+0000","last_change":"2024-04-25T00:13:19.430288+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430288+0000","last_undegraded":"2024-04-25T00:13:19.430288+0000","last_fullsized":"2024-04-25T00:13:19.430288+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.435286+0000","last_change":"2024-04-25T00:13:19.431189+0000","last_active":"2024-04-25T00:13:19.435286+0000","last_peered":"2024-04-25T00:13:19.435286+0000","last_clean":"2024-04-25T00:13:19.435286+0000","last_became_active":"2024-04-25T00:13:19.430933+0000","last_became_peered":"2024-04-25T00:13:19.430933+0000","last_unstale":"2024-04-25T00:13:19.435286+0000","last_undegraded":"2024-04-25T00:13:19.435286+0000","last_fullsized":"2024-04-25T00:13:19.435286+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:50.255023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.425319+0000","last_change":"2024-04-25T00:13:19.425319+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.425319+0000","last_undegraded":"2024-04-25T00:13:19.425319+0000","last_fullsized":"2024-04-25T00:13:19.425319+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442691+0000","last_change":"2024-04-25T00:13:19.441394+0000","last_active":"2024-04-25T00:13:19.442691+0000","last_peered":"2024-04-25T00:13:19.442691+0000","last_clean":"2024-04-25T00:13:19.442691+0000","last_became_active":"2024-04-25T00:13:19.438688+0000","last_became_peered":"2024-04-25T00:13:19.438688+0000","last_unstale":"2024-04-25T00:13:19.442691+0000","last_undegraded":"2024-04-25T00:13:19.442691+0000","last_fullsized":"2024-04-25T00:13:19.442691+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:28.553754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420922+0000","last_change":"2024-04-25T00:13:19.420922+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420922+0000","last_undegraded":"2024-04-25T00:13:19.420922+0000","last_fullsized":"2024-04-25T00:13:19.420922+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.436636+0000","last_change":"2024-04-25T00:13:19.433617+0000","last_active":"2024-04-25T00:13:19.436636+0000","last_peered":"2024-04-25T00:13:19.436636+0000","last_clean":"2024-04-25T00:13:19.436636+0000","last_became_active":"2024-04-25T00:13:19.433063+0000","last_became_peered":"2024-04-25T00:13:19.433063+0000","last_unstale":"2024-04-25T00:13:19.436636+0000","last_undegraded":"2024-04-25T00:13:19.436636+0000","last_fullsized":"2024-04-25T00:13:19.436636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:18.683206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423602+0000","last_change":"2024-04-25T00:13:19.423602+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423602+0000","last_undegraded":"2024-04-25T00:13:19.423602+0000","last_fullsized":"2024-04-25T00:13:19.423602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442465+0000","last_change":"2024-04-25T00:13:19.431998+0000","last_active":"2024-04-25T00:13:19.442465+0000","last_peered":"2024-04-25T00:13:19.442465+0000","last_clean":"2024-04-25T00:13:19.442465+0000","last_became_active":"2024-04-25T00:13:19.431728+0000","last_became_peered":"2024-04-25T00:13:19.431728+0000","last_unstale":"2024-04-25T00:13:19.442465+0000","last_undegraded":"2024-04-25T00:13:19.442465+0000","last_fullsized":"2024-04-25T00:13:19.442465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:04.083138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426189+0000","last_change":"2024-04-25T00:13:19.426189+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426189+0000","last_undegraded":"2024-04-25T00:13:19.426189+0000","last_fullsized":"2024-04-25T00:13:19.426189+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446032+0000","last_change":"2024-04-25T00:13:19.440887+0000","last_active":"2024-04-25T00:13:19.446032+0000","last_peered":"2024-04-25T00:13:19.446032+0000","last_clean":"2024-04-25T00:13:19.446032+0000","last_became_active":"2024-04-25T00:13:19.437895+0000","last_became_peered":"2024-04-25T00:13:19.437895+0000","last_unstale":"2024-04-25T00:13:19.446032+0000","last_undegraded":"2024-04-25T00:13:19.446032+0000","last_fullsized":"2024-04-25T00:13:19.446032+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:47.273633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422117+0000","last_change":"2024-04-25T00:13:19.422117+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422117+0000","last_undegraded":"2024-04-25T00:13:19.422117+0000","last_fullsized":"2024-04-25T00:13:19.422117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.443114+0000","last_change":"2024-04-25T00:13:19.437705+0000","last_active":"2024-04-25T00:13:19.443114+0000","last_peered":"2024-04-25T00:13:19.443114+0000","last_clean":"2024-04-25T00:13:19.443114+0000","last_became_active":"2024-04-25T00:13:19.437390+0000","last_became_peered":"2024-04-25T00:13:19.437390+0000","last_unstale":"2024-04-25T00:13:19.443114+0000","last_undegraded":"2024-04-25T00:13:19.443114+0000","last_fullsized":"2024-04-25T00:13:19.443114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:44.096993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.421441+0000","last_change":"2024-04-25T00:13:19.421441+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.421441+0000","last_undegraded":"2024-04-25T00:13:19.421441+0000","last_fullsized":"2024-04-25T00:13:19.421441+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442484+0000","last_change":"2024-04-25T00:13:19.438746+0000","last_active":"2024-04-25T00:13:19.442484+0000","last_peered":"2024-04-25T00:13:19.442484+0000","last_clean":"2024-04-25T00:13:19.442484+0000","last_became_active":"2024-04-25T00:13:19.437749+0000","last_became_peered":"2024-04-25T00:13:19.437749+0000","last_unstale":"2024-04-25T00:13:19.442484+0000","last_undegraded":"2024-04-25T00:13:19.442484+0000","last_fullsized":"2024-04-25T00:13:19.442484+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:39.640503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426025+0000","last_change":"2024-04-25T00:13:19.426025+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426025+0000","last_undegraded":"2024-04-25T00:13:19.426025+0000","last_fullsized":"2024-04-25T00:13:19.426025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.448034+0000","last_change":"2024-04-25T00:13:19.445212+0000","last_active":"2024-04-25T00:13:19.448034+0000","last_peered":"2024-04-25T00:13:19.448034+0000","last_clean":"2024-04-25T00:13:19.448034+0000","last_became_active":"2024-04-25T00:13:19.444953+0000","last_became_peered":"2024-04-25T00:13:19.444953+0000","last_unstale":"2024-04-25T00:13:19.448034+0000","last_undegraded":"2024-04-25T00:13:19.448034+0000","last_fullsized":"2024-04-25T00:13:19.448034+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:28.280706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426254+0000","last_change":"2024-04-25T00:13:19.426254+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426254+0000","last_undegraded":"2024-04-25T00:13:19.426254+0000","last_fullsized":"2024-04-25T00:13:19.426254+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445679+0000","last_change":"2024-04-25T00:13:19.439464+0000","last_active":"2024-04-25T00:13:19.445679+0000","last_peered":"2024-04-25T00:13:19.445679+0000","last_clean":"2024-04-25T00:13:19.445679+0000","last_became_active":"2024-04-25T00:13:19.439143+0000","last_became_peered":"2024-04-25T00:13:19.439143+0000","last_unstale":"2024-04-25T00:13:19.445679+0000","last_undegraded":"2024-04-25T00:13:19.445679+0000","last_fullsized":"2024-04-25T00:13:19.445679+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:03.413067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.435804+0000","last_change":"2024-04-25T00:13:19.435804+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.435804+0000","last_undegraded":"2024-04-25T00:13:19.435804+0000","last_fullsized":"2024-04-25T00:13:19.435804+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446509+0000","last_change":"2024-04-25T00:13:19.443972+0000","last_active":"2024-04-25T00:13:19.446509+0000","last_peered":"2024-04-25T00:13:19.446509+0000","last_clean":"2024-04-25T00:13:19.446509+0000","last_became_active":"2024-04-25T00:13:19.443634+0000","last_became_peered":"2024-04-25T00:13:19.443634+0000","last_unstale":"2024-04-25T00:13:19.446509+0000","last_undegraded":"2024-04-25T00:13:19.446509+0000","last_fullsized":"2024-04-25T00:13:19.446509+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:03.413067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420470+0000","last_change":"2024-04-25T00:13:19.420470+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420470+0000","last_undegraded":"2024-04-25T00:13:19.420470+0000","last_fullsized":"2024-04-25T00:13:19.420470+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.436847+0000","last_change":"2024-04-25T00:13:19.430607+0000","last_active":"2024-04-25T00:13:19.436847+0000","last_peered":"2024-04-25T00:13:19.436847+0000","last_clean":"2024-04-25T00:13:19.436847+0000","last_became_active":"2024-04-25T00:13:19.430240+0000","last_became_peered":"2024-04-25T00:13:19.430240+0000","last_unstale":"2024-04-25T00:13:19.436847+0000","last_undegraded":"2024-04-25T00:13:19.436847+0000","last_fullsized":"2024-04-25T00:13:19.436847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:07.859795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428661+0000","last_change":"2024-04-25T00:13:19.428661+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428661+0000","last_undegraded":"2024-04-25T00:13:19.428661+0000","last_fullsized":"2024-04-25T00:13:19.428661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.443661+0000","last_change":"2024-04-25T00:13:19.438379+0000","last_active":"2024-04-25T00:13:19.443661+0000","last_peered":"2024-04-25T00:13:19.443661+0000","last_clean":"2024-04-25T00:13:19.443661+0000","last_became_active":"2024-04-25T00:13:19.438093+0000","last_became_peered":"2024-04-25T00:13:19.438093+0000","last_unstale":"2024-04-25T00:13:19.443661+0000","last_undegraded":"2024-04-25T00:13:19.443661+0000","last_fullsized":"2024-04-25T00:13:19.443661+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:28.036116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.424260+0000","last_change":"2024-04-25T00:13:19.424260+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.424260+0000","last_undegraded":"2024-04-25T00:13:19.424260+0000","last_fullsized":"2024-04-25T00:13:19.424260+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445046+0000","last_change":"2024-04-25T00:13:19.440461+0000","last_active":"2024-04-25T00:13:19.445046+0000","last_peered":"2024-04-25T00:13:19.445046+0000","last_clean":"2024-04-25T00:13:19.445046+0000","last_became_active":"2024-04-25T00:13:19.440197+0000","last_became_peered":"2024-04-25T00:13:19.440197+0000","last_unstale":"2024-04-25T00:13:19.445046+0000","last_undegraded":"2024-04-25T00:13:19.445046+0000","last_fullsized":"2024-04-25T00:13:19.445046+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:37.585343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420102+0000","last_change":"2024-04-25T00:13:19.420102+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420102+0000","last_undegraded":"2024-04-25T00:13:19.420102+0000","last_fullsized":"2024-04-25T00:13:19.420102+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442434+0000","last_change":"2024-04-25T00:13:19.438945+0000","last_active":"2024-04-25T00:13:19.442434+0000","last_peered":"2024-04-25T00:13:19.442434+0000","last_clean":"2024-04-25T00:13:19.442434+0000","last_became_active":"2024-04-25T00:13:19.438052+0000","last_became_peered":"2024-04-25T00:13:19.438052+0000","last_unstale":"2024-04-25T00:13:19.442434+0000","last_undegraded":"2024-04-25T00:13:19.442434+0000","last_fullsized":"2024-04-25T00:13:19.442434+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:21.586818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427651+0000","last_change":"2024-04-25T00:13:19.427651+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427651+0000","last_undegraded":"2024-04-25T00:13:19.427651+0000","last_fullsized":"2024-04-25T00:13:19.427651+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445826+0000","last_change":"2024-04-25T00:13:19.440459+0000","last_active":"2024-04-25T00:13:19.445826+0000","last_peered":"2024-04-25T00:13:19.445826+0000","last_clean":"2024-04-25T00:13:19.445826+0000","last_became_active":"2024-04-25T00:13:19.438442+0000","last_became_peered":"2024-04-25T00:13:19.438442+0000","last_unstale":"2024-04-25T00:13:19.445826+0000","last_undegraded":"2024-04-25T00:13:19.445826+0000","last_fullsized":"2024-04-25T00:13:19.445826+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:42.830716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420129+0000","last_change":"2024-04-25T00:13:19.420129+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420129+0000","last_undegraded":"2024-04-25T00:13:19.420129+0000","last_fullsized":"2024-04-25T00:13:19.420129+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442276+0000","last_change":"2024-04-25T00:13:19.440785+0000","last_active":"2024-04-25T00:13:19.442276+0000","last_peered":"2024-04-25T00:13:19.442276+0000","last_clean":"2024-04-25T00:13:19.442276+0000","last_became_active":"2024-04-25T00:13:19.440512+0000","last_became_peered":"2024-04-25T00:13:19.440512+0000","last_unstale":"2024-04-25T00:13:19.442276+0000","last_undegraded":"2024-04-25T00:13:19.442276+0000","last_fullsized":"2024-04-25T00:13:19.442276+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:35.767979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423501+0000","last_change":"2024-04-25T00:13:19.423501+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423501+0000","last_undegraded":"2024-04-25T00:13:19.423501+0000","last_fullsized":"2024-04-25T00:13:19.423501+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447676+0000","last_change":"2024-04-25T00:13:19.443427+0000","last_active":"2024-04-25T00:13:19.447676+0000","last_peered":"2024-04-25T00:13:19.447676+0000","last_clean":"2024-04-25T00:13:19.447676+0000","last_became_active":"2024-04-25T00:13:19.443147+0000","last_became_peered":"2024-04-25T00:13:19.443147+0000","last_unstale":"2024-04-25T00:13:19.447676+0000","last_undegraded":"2024-04-25T00:13:19.447676+0000","last_fullsized":"2024-04-25T00:13:19.447676+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:17.551103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.435341+0000","last_change":"2024-04-25T00:13:19.435341+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.435341+0000","last_undegraded":"2024-04-25T00:13:19.435341+0000","last_fullsized":"2024-04-25T00:13:19.435341+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445110+0000","last_change":"2024-04-25T00:13:19.441302+0000","last_active":"2024-04-25T00:13:19.445110+0000","last_peered":"2024-04-25T00:13:19.445110+0000","last_clean":"2024-04-25T00:13:19.445110+0000","last_became_active":"2024-04-25T00:13:19.440951+0000","last_became_peered":"2024-04-25T00:13:19.440951+0000","last_unstale":"2024-04-25T00:13:19.445110+0000","last_undegraded":"2024-04-25T00:13:19.445110+0000","last_fullsized":"2024-04-25T00:13:19.445110+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:07.859795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419439+0000","last_change":"2024-04-25T00:13:19.419439+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419439+0000","last_undegraded":"2024-04-25T00:13:19.419439+0000","last_fullsized":"2024-04-25T00:13:19.419439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.441866+0000","last_change":"2024-04-25T00:13:19.438811+0000","last_active":"2024-04-25T00:13:19.441866+0000","last_peered":"2024-04-25T00:13:19.441866+0000","last_clean":"2024-04-25T00:13:19.441866+0000","last_became_active":"2024-04-25T00:13:19.438527+0000","last_became_peered":"2024-04-25T00:13:19.438527+0000","last_unstale":"2024-04-25T00:13:19.441866+0000","last_undegraded":"2024-04-25T00:13:19.441866+0000","last_fullsized":"2024-04-25T00:13:19.441866+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:13.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.424634+0000","last_change":"2024-04-25T00:13:19.424634+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.424634+0000","last_undegraded":"2024-04-25T00:13:19.424634+0000","last_fullsized":"2024-04-25T00:13:19.424634+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446400+0000","last_change":"2024-04-25T00:13:19.442162+0000","last_active":"2024-04-25T00:13:19.446400+0000","last_peered":"2024-04-25T00:13:19.446400+0000","last_clean":"2024-04-25T00:13:19.446400+0000","last_became_active":"2024-04-25T00:13:19.441874+0000","last_became_peered":"2024-04-25T00:13:19.441874+0000","last_unstale":"2024-04-25T00:13:19.446400+0000","last_undegraded":"2024-04-25T00:13:19.446400+0000","last_fullsized":"2024-04-25T00:13:19.446400+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:39.640503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430355+0000","last_change":"2024-04-25T00:13:19.430355+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430355+0000","last_undegraded":"2024-04-25T00:13:19.430355+0000","last_fullsized":"2024-04-25T00:13:19.430355+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446460+0000","last_change":"2024-04-25T00:13:19.443257+0000","last_active":"2024-04-25T00:13:19.446460+0000","last_peered":"2024-04-25T00:13:19.446460+0000","last_clean":"2024-04-25T00:13:19.446460+0000","last_became_active":"2024-04-25T00:13:19.442853+0000","last_became_peered":"2024-04-25T00:13:19.442853+0000","last_unstale":"2024-04-25T00:13:19.446460+0000","last_undegraded":"2024-04-25T00:13:19.446460+0000","last_fullsized":"2024-04-25T00:13:19.446460+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:21.586818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427928+0000","last_change":"2024-04-25T00:13:19.427928+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427928+0000","last_undegraded":"2024-04-25T00:13:19.427928+0000","last_fullsized":"2024-04-25T00:13:19.427928+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447682+0000","last_change":"2024-04-25T00:13:19.444471+0000","last_active":"2024-04-25T00:13:19.447682+0000","last_peered":"2024-04-25T00:13:19.447682+0000","last_clean":"2024-04-25T00:13:19.447682+0000","last_became_active":"2024-04-25T00:13:19.444197+0000","last_became_peered":"2024-04-25T00:13:19.444197+0000","last_unstale":"2024-04-25T00:13:19.447682+0000","last_undegraded":"2024-04-25T00:13:19.447682+0000","last_fullsized":"2024-04-25T00:13:19.447682+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:29.987108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419511+0000","last_change":"2024-04-25T00:13:19.419511+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419511+0000","last_undegraded":"2024-04-25T00:13:19.419511+0000","last_fullsized":"2024-04-25T00:13:19.419511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447962+0000","last_change":"2024-04-25T00:13:19.445103+0000","last_active":"2024-04-25T00:13:19.447962+0000","last_peered":"2024-04-25T00:13:19.447962+0000","last_clean":"2024-04-25T00:13:19.447962+0000","last_became_active":"2024-04-25T00:13:19.444652+0000","last_became_peered":"2024-04-25T00:13:19.444652+0000","last_unstale":"2024-04-25T00:13:19.447962+0000","last_undegraded":"2024-04-25T00:13:19.447962+0000","last_fullsized":"2024-04-25T00:13:19.447962+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:39.691842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419832+0000","last_change":"2024-04-25T00:13:19.419832+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419832+0000","last_undegraded":"2024-04-25T00:13:19.419832+0000","last_fullsized":"2024-04-25T00:13:19.419832+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.448638+0000","last_change":"2024-04-25T00:13:19.442646+0000","last_active":"2024-04-25T00:13:19.448638+0000","last_peered":"2024-04-25T00:13:19.448638+0000","last_clean":"2024-04-25T00:13:19.448638+0000","last_became_active":"2024-04-25T00:13:19.438848+0000","last_became_peered":"2024-04-25T00:13:19.438848+0000","last_unstale":"2024-04-25T00:13:19.448638+0000","last_undegraded":"2024-04-25T00:13:19.448638+0000","last_fullsized":"2024-04-25T00:13:19.448638+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:13.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.425579+0000","last_change":"2024-04-25T00:13:19.425579+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.425579+0000","last_undegraded":"2024-04-25T00:13:19.425579+0000","last_fullsized":"2024-04-25T00:13:19.425579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.439322+0000","last_change":"2024-04-25T00:13:19.436516+0000","last_active":"2024-04-25T00:13:19.439322+0000","last_peered":"2024-04-25T00:13:19.439322+0000","last_clean":"2024-04-25T00:13:19.439322+0000","last_became_active":"2024-04-25T00:13:19.435936+0000","last_became_peered":"2024-04-25T00:13:19.435936+0000","last_unstale":"2024-04-25T00:13:19.439322+0000","last_undegraded":"2024-04-25T00:13:19.439322+0000","last_fullsized":"2024-04-25T00:13:19.439322+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:57.960615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.424954+0000","last_change":"2024-04-25T00:13:19.424954+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.424954+0000","last_undegraded":"2024-04-25T00:13:19.424954+0000","last_fullsized":"2024-04-25T00:13:19.424954+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.429368+0000","last_change":"2024-04-25T00:13:19.427399+0000","last_active":"2024-04-25T00:13:19.429368+0000","last_peered":"2024-04-25T00:13:19.429368+0000","last_clean":"2024-04-25T00:13:19.429368+0000","last_became_active":"2024-04-25T00:13:19.427145+0000","last_became_peered":"2024-04-25T00:13:19.427145+0000","last_unstale":"2024-04-25T00:13:19.429368+0000","last_undegraded":"2024-04-25T00:13:19.429368+0000","last_fullsized":"2024-04-25T00:13:19.429368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:19.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430930+0000","last_change":"2024-04-25T00:13:19.430930+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430930+0000","last_undegraded":"2024-04-25T00:13:19.430930+0000","last_fullsized":"2024-04-25T00:13:19.430930+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.441056+0000","last_change":"2024-04-25T00:13:19.438635+0000","last_active":"2024-04-25T00:13:19.441056+0000","last_peered":"2024-04-25T00:13:19.441056+0000","last_clean":"2024-04-25T00:13:19.441056+0000","last_became_active":"2024-04-25T00:13:19.438328+0000","last_became_peered":"2024-04-25T00:13:19.438328+0000","last_unstale":"2024-04-25T00:13:19.441056+0000","last_undegraded":"2024-04-25T00:13:19.441056+0000","last_fullsized":"2024-04-25T00:13:19.441056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:17.223076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419237+0000","last_change":"2024-04-25T00:13:19.419237+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419237+0000","last_undegraded":"2024-04-25T00:13:19.419237+0000","last_fullsized":"2024-04-25T00:13:19.419237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.433512+0000","last_change":"2024-04-25T00:13:19.430683+0000","last_active":"2024-04-25T00:13:19.433512+0000","last_peered":"2024-04-25T00:13:19.433512+0000","last_clean":"2024-04-25T00:13:19.433512+0000","last_became_active":"2024-04-25T00:13:19.430464+0000","last_became_peered":"2024-04-25T00:13:19.430464+0000","last_unstale":"2024-04-25T00:13:19.433512+0000","last_undegraded":"2024-04-25T00:13:19.433512+0000","last_fullsized":"2024-04-25T00:13:19.433512+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:36.678931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":239,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.424193+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:13:19.424193+0000","last_peered":"2024-04-25T00:13:19.424193+0000","last_clean":"2024-04-25T00:13:19.424193+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:13:19.424193+0000","last_undegraded":"2024-04-25T00:13:19.424193+0000","last_fullsized":"2024-04-25T00:13:19.424193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1294336,"data_stored":901561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":700416,"data_stored":311193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1282048,"data_stored":894902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:13:25.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2024-04-25T00:13:25.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.427611+0000', 'last_change': '2024-04-25T00:13:19.427611+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.427611+0000', 'last_undegraded': '2024-04-25T00:13:19.427611+0000', 'last_fullsized': '2024-04-25T00:13:19.427611+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:13:25.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2024-04-25T00:13:25.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.427995+0000', 'last_change': '2024-04-25T00:13:19.427995+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.427995+0000', 'last_undegraded': '2024-04-25T00:13:19.427995+0000', 'last_fullsized': '2024-04-25T00:13:19.427995+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:13:25.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2024-04-25T00:13:25.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.420276+0000', 'last_change': '2024-04-25T00:13:19.420276+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.420276+0000', 'last_undegraded': '2024-04-25T00:13:19.420276+0000', 'last_fullsized': '2024-04-25T00:13:19.420276+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:13:25.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2024-04-25T00:13:25.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.428639+0000', 'last_change': '2024-04-25T00:13:19.428639+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.428639+0000', 'last_undegraded': '2024-04-25T00:13:19.428639+0000', 'last_fullsized': '2024-04-25T00:13:19.428639+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:13:25.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2024-04-25T00:13:25.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.420673+0000', 'last_change': '2024-04-25T00:13:19.420673+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.420673+0000', 'last_undegraded': '2024-04-25T00:13:19.420673+0000', 'last_fullsized': '2024-04-25T00:13:19.420673+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:13:25.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2024-04-25T00:13:25.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.422946+0000', 'last_change': '2024-04-25T00:13:19.422946+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.422946+0000', 'last_undegraded': '2024-04-25T00:13:19.422946+0000', 'last_fullsized': '2024-04-25T00:13:19.422946+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:13:25.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2024-04-25T00:13:25.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.431248+0000', 'last_change': '2024-04-25T00:13:19.431248+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.431248+0000', 'last_undegraded': '2024-04-25T00:13:19.431248+0000', 'last_fullsized': '2024-04-25T00:13:19.431248+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:13:25.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2024-04-25T00:13:25.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.429195+0000', 'last_change': '2024-04-25T00:13:19.429195+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.429195+0000', 'last_undegraded': '2024-04-25T00:13:19.429195+0000', 'last_fullsized': '2024-04-25T00:13:19.429195+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:13:25.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2024-04-25T00:13:25.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.421312+0000', 'last_change': '2024-04-25T00:13:19.421312+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.421312+0000', 'last_undegraded': '2024-04-25T00:13:19.421312+0000', 'last_fullsized': '2024-04-25T00:13:19.421312+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:13:25.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2024-04-25T00:13:25.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.423950+0000', 'last_change': '2024-04-25T00:13:19.423950+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.423950+0000', 'last_undegraded': '2024-04-25T00:13:19.423950+0000', 'last_fullsized': '2024-04-25T00:13:19.423950+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:13:25.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2024-04-25T00:13:25.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.427090+0000', 'last_change': '2024-04-25T00:13:19.427090+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.427090+0000', 'last_undegraded': '2024-04-25T00:13:19.427090+0000', 'last_fullsized': '2024-04-25T00:13:19.427090+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:13:25.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2024-04-25T00:13:25.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.431410+0000', 'last_change': '2024-04-25T00:13:19.431410+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.431410+0000', 'last_undegraded': '2024-04-25T00:13:19.431410+0000', 'last_fullsized': '2024-04-25T00:13:19.431410+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:13:25.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2024-04-25T00:13:25.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.431226+0000', 'last_change': '2024-04-25T00:13:19.431226+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.431226+0000', 'last_undegraded': '2024-04-25T00:13:19.431226+0000', 'last_fullsized': '2024-04-25T00:13:19.431226+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:13:25.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2024-04-25T00:13:25.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.419593+0000', 'last_change': '2024-04-25T00:13:19.419593+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.419593+0000', 'last_undegraded': '2024-04-25T00:13:19.419593+0000', 'last_fullsized': '2024-04-25T00:13:19.419593+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:13:25.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2024-04-25T00:13:25.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.419376+0000', 'last_change': '2024-04-25T00:13:19.419376+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.419376+0000', 'last_undegraded': '2024-04-25T00:13:19.419376+0000', 'last_fullsized': '2024-04-25T00:13:19.419376+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:13:25.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2024-04-25T00:13:25.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.431142+0000', 'last_change': '2024-04-25T00:13:19.431142+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.431142+0000', 'last_undegraded': '2024-04-25T00:13:19.431142+0000', 'last_fullsized': '2024-04-25T00:13:19.431142+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:13:25.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2024-04-25T00:13:25.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.427186+0000', 'last_change': '2024-04-25T00:13:19.427186+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.427186+0000', 'last_undegraded': '2024-04-25T00:13:19.427186+0000', 'last_fullsized': '2024-04-25T00:13:19.427186+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:13:25.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2024-04-25T00:13:25.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.428152+0000', 'last_change': '2024-04-25T00:13:19.428152+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.428152+0000', 'last_undegraded': '2024-04-25T00:13:19.428152+0000', 'last_fullsized': '2024-04-25T00:13:19.428152+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:13:25.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2024-04-25T00:13:25.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.420621+0000', 'last_change': '2024-04-25T00:13:19.420621+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.420621+0000', 'last_undegraded': '2024-04-25T00:13:19.420621+0000', 'last_fullsized': '2024-04-25T00:13:19.420621+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:13:25.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2c is not active+clean 2024-04-25T00:13:25.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.423212+0000', 'last_change': '2024-04-25T00:13:19.423212+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.423212+0000', 'last_undegraded': '2024-04-25T00:13:19.423212+0000', 'last_fullsized': '2024-04-25T00:13:19.423212+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:13:25.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2024-04-25T00:13:25.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.430777+0000', 'last_change': '2024-04-25T00:13:19.430777+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.430777+0000', 'last_undegraded': '2024-04-25T00:13:19.430777+0000', 'last_fullsized': '2024-04-25T00:13:19.430777+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:13:25.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2024-04-25T00:13:25.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.422795+0000', 'last_change': '2024-04-25T00:13:19.422795+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.422795+0000', 'last_undegraded': '2024-04-25T00:13:19.422795+0000', 'last_fullsized': '2024-04-25T00:13:19.422795+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:13:25.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.29 is not active+clean 2024-04-25T00:13:25.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.422637+0000', 'last_change': '2024-04-25T00:13:19.422637+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.422637+0000', 'last_undegraded': '2024-04-25T00:13:19.422637+0000', 'last_fullsized': '2024-04-25T00:13:19.422637+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:13:25.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2024-04-25T00:13:25.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.423888+0000', 'last_change': '2024-04-25T00:13:19.423888+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.423888+0000', 'last_undegraded': '2024-04-25T00:13:19.423888+0000', 'last_fullsized': '2024-04-25T00:13:19.423888+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:13:25.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2024-04-25T00:13:25.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.429151+0000', 'last_change': '2024-04-25T00:13:19.429151+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.429151+0000', 'last_undegraded': '2024-04-25T00:13:19.429151+0000', 'last_fullsized': '2024-04-25T00:13:19.429151+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:13:25.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2024-04-25T00:13:25.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.428403+0000', 'last_change': '2024-04-25T00:13:19.428403+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.428403+0000', 'last_undegraded': '2024-04-25T00:13:19.428403+0000', 'last_fullsized': '2024-04-25T00:13:19.428403+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:13:25.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2024-04-25T00:13:25.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.426616+0000', 'last_change': '2024-04-25T00:13:19.426616+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.426616+0000', 'last_undegraded': '2024-04-25T00:13:19.426616+0000', 'last_fullsized': '2024-04-25T00:13:19.426616+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:13:25.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2024-04-25T00:13:25.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.426605+0000', 'last_change': '2024-04-25T00:13:19.426605+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.426605+0000', 'last_undegraded': '2024-04-25T00:13:19.426605+0000', 'last_fullsized': '2024-04-25T00:13:19.426605+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:13:25.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2024-04-25T00:13:25.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.420098+0000', 'last_change': '2024-04-25T00:13:19.420098+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.420098+0000', 'last_undegraded': '2024-04-25T00:13:19.420098+0000', 'last_fullsized': '2024-04-25T00:13:19.420098+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:13:25.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2024-04-25T00:13:25.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.431639+0000', 'last_change': '2024-04-25T00:13:19.431639+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.431639+0000', 'last_undegraded': '2024-04-25T00:13:19.431639+0000', 'last_fullsized': '2024-04-25T00:13:19.431639+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:13:25.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2024-04-25T00:13:25.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.429845+0000', 'last_change': '2024-04-25T00:13:19.429845+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.429845+0000', 'last_undegraded': '2024-04-25T00:13:19.429845+0000', 'last_fullsized': '2024-04-25T00:13:19.429845+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:13:25.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2024-04-25T00:13:25.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.423891+0000', 'last_change': '2024-04-25T00:13:19.423891+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.423891+0000', 'last_undegraded': '2024-04-25T00:13:19.423891+0000', 'last_fullsized': '2024-04-25T00:13:19.423891+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:13:25.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2024-04-25T00:13:25.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.419810+0000', 'last_change': '2024-04-25T00:13:19.419810+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.419810+0000', 'last_undegraded': '2024-04-25T00:13:19.419810+0000', 'last_fullsized': '2024-04-25T00:13:19.419810+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:13:25.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2024-04-25T00:13:25.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.426198+0000', 'last_change': '2024-04-25T00:13:19.426198+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.426198+0000', 'last_undegraded': '2024-04-25T00:13:19.426198+0000', 'last_fullsized': '2024-04-25T00:13:19.426198+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:13:25.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2024-04-25T00:13:25.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.428957+0000', 'last_change': '2024-04-25T00:13:19.428957+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.428957+0000', 'last_undegraded': '2024-04-25T00:13:19.428957+0000', 'last_fullsized': '2024-04-25T00:13:19.428957+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:13:25.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2024-04-25T00:13:25.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.423306+0000', 'last_change': '2024-04-25T00:13:19.423306+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.423306+0000', 'last_undegraded': '2024-04-25T00:13:19.423306+0000', 'last_fullsized': '2024-04-25T00:13:19.423306+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:13:25.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2024-04-25T00:13:25.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.430288+0000', 'last_change': '2024-04-25T00:13:19.430288+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.430288+0000', 'last_undegraded': '2024-04-25T00:13:19.430288+0000', 'last_fullsized': '2024-04-25T00:13:19.430288+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:13:25.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2024-04-25T00:13:25.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.425319+0000', 'last_change': '2024-04-25T00:13:19.425319+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.425319+0000', 'last_undegraded': '2024-04-25T00:13:19.425319+0000', 'last_fullsized': '2024-04-25T00:13:19.425319+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:13:25.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2024-04-25T00:13:25.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.420922+0000', 'last_change': '2024-04-25T00:13:19.420922+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.420922+0000', 'last_undegraded': '2024-04-25T00:13:19.420922+0000', 'last_fullsized': '2024-04-25T00:13:19.420922+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:13:25.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2024-04-25T00:13:25.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.423602+0000', 'last_change': '2024-04-25T00:13:19.423602+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.423602+0000', 'last_undegraded': '2024-04-25T00:13:19.423602+0000', 'last_fullsized': '2024-04-25T00:13:19.423602+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:13:25.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2024-04-25T00:13:25.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.426189+0000', 'last_change': '2024-04-25T00:13:19.426189+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.426189+0000', 'last_undegraded': '2024-04-25T00:13:19.426189+0000', 'last_fullsized': '2024-04-25T00:13:19.426189+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:13:25.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2024-04-25T00:13:25.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.422117+0000', 'last_change': '2024-04-25T00:13:19.422117+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.422117+0000', 'last_undegraded': '2024-04-25T00:13:19.422117+0000', 'last_fullsized': '2024-04-25T00:13:19.422117+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:13:25.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2024-04-25T00:13:25.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.421441+0000', 'last_change': '2024-04-25T00:13:19.421441+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.421441+0000', 'last_undegraded': '2024-04-25T00:13:19.421441+0000', 'last_fullsized': '2024-04-25T00:13:19.421441+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:13:25.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2024-04-25T00:13:25.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.426025+0000', 'last_change': '2024-04-25T00:13:19.426025+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.426025+0000', 'last_undegraded': '2024-04-25T00:13:19.426025+0000', 'last_fullsized': '2024-04-25T00:13:19.426025+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:13:25.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2024-04-25T00:13:25.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.426254+0000', 'last_change': '2024-04-25T00:13:19.426254+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.426254+0000', 'last_undegraded': '2024-04-25T00:13:19.426254+0000', 'last_fullsized': '2024-04-25T00:13:19.426254+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:13:25.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2024-04-25T00:13:25.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.435804+0000', 'last_change': '2024-04-25T00:13:19.435804+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.435804+0000', 'last_undegraded': '2024-04-25T00:13:19.435804+0000', 'last_fullsized': '2024-04-25T00:13:19.435804+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:13:25.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2024-04-25T00:13:25.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.420470+0000', 'last_change': '2024-04-25T00:13:19.420470+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.420470+0000', 'last_undegraded': '2024-04-25T00:13:19.420470+0000', 'last_fullsized': '2024-04-25T00:13:19.420470+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:13:25.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2024-04-25T00:13:25.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.428661+0000', 'last_change': '2024-04-25T00:13:19.428661+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.428661+0000', 'last_undegraded': '2024-04-25T00:13:19.428661+0000', 'last_fullsized': '2024-04-25T00:13:19.428661+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:13:25.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2024-04-25T00:13:25.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.424260+0000', 'last_change': '2024-04-25T00:13:19.424260+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.424260+0000', 'last_undegraded': '2024-04-25T00:13:19.424260+0000', 'last_fullsized': '2024-04-25T00:13:19.424260+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:13:25.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2024-04-25T00:13:25.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.420102+0000', 'last_change': '2024-04-25T00:13:19.420102+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.420102+0000', 'last_undegraded': '2024-04-25T00:13:19.420102+0000', 'last_fullsized': '2024-04-25T00:13:19.420102+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:13:25.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2024-04-25T00:13:25.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.427651+0000', 'last_change': '2024-04-25T00:13:19.427651+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.427651+0000', 'last_undegraded': '2024-04-25T00:13:19.427651+0000', 'last_fullsized': '2024-04-25T00:13:19.427651+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:13:25.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2024-04-25T00:13:25.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.420129+0000', 'last_change': '2024-04-25T00:13:19.420129+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.420129+0000', 'last_undegraded': '2024-04-25T00:13:19.420129+0000', 'last_fullsized': '2024-04-25T00:13:19.420129+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:13:25.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2024-04-25T00:13:25.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.423501+0000', 'last_change': '2024-04-25T00:13:19.423501+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.423501+0000', 'last_undegraded': '2024-04-25T00:13:19.423501+0000', 'last_fullsized': '2024-04-25T00:13:19.423501+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:13:25.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2024-04-25T00:13:25.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.435341+0000', 'last_change': '2024-04-25T00:13:19.435341+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.435341+0000', 'last_undegraded': '2024-04-25T00:13:19.435341+0000', 'last_fullsized': '2024-04-25T00:13:19.435341+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:13:25.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2024-04-25T00:13:25.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.419439+0000', 'last_change': '2024-04-25T00:13:19.419439+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.419439+0000', 'last_undegraded': '2024-04-25T00:13:19.419439+0000', 'last_fullsized': '2024-04-25T00:13:19.419439+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:13:25.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2024-04-25T00:13:25.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.424634+0000', 'last_change': '2024-04-25T00:13:19.424634+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.424634+0000', 'last_undegraded': '2024-04-25T00:13:19.424634+0000', 'last_fullsized': '2024-04-25T00:13:19.424634+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:13:25.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2024-04-25T00:13:25.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.430355+0000', 'last_change': '2024-04-25T00:13:19.430355+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.430355+0000', 'last_undegraded': '2024-04-25T00:13:19.430355+0000', 'last_fullsized': '2024-04-25T00:13:19.430355+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:13:25.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2024-04-25T00:13:25.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.427928+0000', 'last_change': '2024-04-25T00:13:19.427928+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.427928+0000', 'last_undegraded': '2024-04-25T00:13:19.427928+0000', 'last_fullsized': '2024-04-25T00:13:19.427928+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:13:25.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2024-04-25T00:13:25.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.419511+0000', 'last_change': '2024-04-25T00:13:19.419511+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.419511+0000', 'last_undegraded': '2024-04-25T00:13:19.419511+0000', 'last_fullsized': '2024-04-25T00:13:19.419511+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:13:25.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2024-04-25T00:13:25.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.419832+0000', 'last_change': '2024-04-25T00:13:19.419832+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.419832+0000', 'last_undegraded': '2024-04-25T00:13:19.419832+0000', 'last_fullsized': '2024-04-25T00:13:19.419832+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:13:25.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2024-04-25T00:13:25.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.425579+0000', 'last_change': '2024-04-25T00:13:19.425579+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.425579+0000', 'last_undegraded': '2024-04-25T00:13:19.425579+0000', 'last_fullsized': '2024-04-25T00:13:19.425579+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:13:25.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2024-04-25T00:13:25.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.424954+0000', 'last_change': '2024-04-25T00:13:19.424954+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.424954+0000', 'last_undegraded': '2024-04-25T00:13:19.424954+0000', 'last_fullsized': '2024-04-25T00:13:19.424954+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:13:25.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2024-04-25T00:13:25.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.430930+0000', 'last_change': '2024-04-25T00:13:19.430930+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.430930+0000', 'last_undegraded': '2024-04-25T00:13:19.430930+0000', 'last_fullsized': '2024-04-25T00:13:19.430930+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:13:25.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2024-04-25T00:13:25.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:13:19.419237+0000', 'last_change': '2024-04-25T00:13:19.419237+0000', 'last_active': '2024-04-25T00:13:19.412448+0000', 'last_peered': '2024-04-25T00:13:19.412448+0000', 'last_clean': '2024-04-25T00:13:19.412448+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:13:19.419237+0000', 'last_undegraded': '2024-04-25T00:13:19.419237+0000', 'last_fullsized': '2024-04-25T00:13:19.419237+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'last_clean_scrub_stamp': '2024-04-25T00:13:19.412448+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:13:25.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:13:25.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.433+0000 7f7b02400640 1 Processor -- start 2024-04-25T00:13:25.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.433+0000 7f7b02400640 1 -- start start 2024-04-25T00:13:25.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.433+0000 7f7b02400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7afc071db0 0x7f7afc06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.433+0000 7f7b02400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7afc0b4210 con 0x7f7afc0b52b0 2024-04-25T00:13:25.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.433+0000 7f7b02400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7afc0b44f0 con 0x7f7afc161630 2024-04-25T00:13:25.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.433+0000 7f7b02400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7afc0b5760 con 0x7f7afc071db0 2024-04-25T00:13:25.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.433+0000 7f7afbe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7afc071db0 0x7f7afc06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.433+0000 7f7afaa00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7afc0b52b0 0x7f7afc0716a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54876/0 (socket says 172.21.15.83:54876) 2024-04-25T00:13:25.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.433+0000 7f7afaa00640 1 -- 172.21.15.83:0/2588961260 learned_addr learned my addr 172.21.15.83:0/2588961260 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:25.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.433+0000 7f7afbe00640 1 -- 172.21.15.83:0/2588961260 >> v1:172.21.15.86:6789/0 conn(0x7f7afc161630 legacy=0x7f7afc0ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.433+0000 7f7afbe00640 1 -- 172.21.15.83:0/2588961260 >> v1:172.21.15.83:6789/0 conn(0x7f7afc0b52b0 legacy=0x7f7afc0716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.433+0000 7f7afbe00640 1 -- 172.21.15.83:0/2588961260 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7afc0b47d0 con 0x7f7afc071db0 2024-04-25T00:13:25.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.433+0000 7f7afa000640 1 -- 172.21.15.83:0/2588961260 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3456071837 0 0) 0x7f7afc0b4210 con 0x7f7afc0b52b0 2024-04-25T00:13:25.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.437+0000 7f7afbe00640 1 --2- 172.21.15.83:0/2588961260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7afc071db0 0x7f7afc06d630 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7af40097c0 tx=0x7f7af401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592ab63921997b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.437+0000 7f7afa000640 1 -- 172.21.15.83:0/2588961260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af40b0070 con 0x7f7afc071db0 2024-04-25T00:13:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.437+0000 7f7afa000640 1 -- 172.21.15.83:0/2588961260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7af40ab600 con 0x7f7afc071db0 2024-04-25T00:13:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.437+0000 7f7afa000640 1 -- 172.21.15.83:0/2588961260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af40ab9d0 con 0x7f7afc071db0 2024-04-25T00:13:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.437+0000 7f7b02400640 1 -- 172.21.15.83:0/2588961260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7afc071db0 msgr2=0x7f7afc06d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.437+0000 7f7b02400640 1 --2- 172.21.15.83:0/2588961260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7afc071db0 0x7f7afc06d630 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7af40097c0 tx=0x7f7af401c380 comp rx=0 tx=0).stop 2024-04-25T00:13:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.437+0000 7f7b02400640 1 -- 172.21.15.83:0/2588961260 shutdown_connections 2024-04-25T00:13:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.437+0000 7f7b02400640 1 --2- 172.21.15.83:0/2588961260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7afc071db0 0x7f7afc06d630 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.437+0000 7f7b02400640 1 -- 172.21.15.83:0/2588961260 >> 172.21.15.83:0/2588961260 conn(0x7f7afc06b1e0 msgr2=0x7f7afc06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.437+0000 7f7b02400640 1 -- 172.21.15.83:0/2588961260 shutdown_connections 2024-04-25T00:13:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.437+0000 7f7b02400640 1 -- 172.21.15.83:0/2588961260 wait complete. 2024-04-25T00:13:25.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7b02400640 1 Processor -- start 2024-04-25T00:13:25.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7b02400640 1 -- start start 2024-04-25T00:13:25.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7b02400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7afc071db0 0x7f7afc1603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7b02400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7afc0b52b0 0x7f7afc154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7b02400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7afc161630 0x7f7afc1593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7b02400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7afc0b3e30 con 0x7f7afc0b52b0 2024-04-25T00:13:25.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7b02400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7afc0ae980 con 0x7f7afc071db0 2024-04-25T00:13:25.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7b02400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7afc0b3fb0 con 0x7f7afc161630 2024-04-25T00:13:25.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7afaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7afc0b52b0 0x7f7afc154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7afaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7afc0b52b0 0x7f7afc154eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56804/0 (socket says 172.21.15.83:56804) 2024-04-25T00:13:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7afaa00640 1 -- 172.21.15.83:0/3749384531 learned_addr learned my addr 172.21.15.83:0/3749384531 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7afb400640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7afc071db0 0x7f7afc1603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7afbe00640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7afc161630 0x7f7afc1593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7afaa00640 1 -- 172.21.15.83:0/3749384531 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7afc161630 msgr2=0x7f7afc1593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7afaa00640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7afc161630 0x7f7afc1593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7afaa00640 1 -- 172.21.15.83:0/3749384531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7afc071db0 msgr2=0x7f7afc1603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7afaa00640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7afc071db0 0x7f7afc1603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7afaa00640 1 -- 172.21.15.83:0/3749384531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7afc156f30 con 0x7f7afc0b52b0 2024-04-25T00:13:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7afb400640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7afc071db0 0x7f7afc1603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7afaa00640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7afc0b52b0 0x7f7afc154eb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800c4a0 tx=0x7f7ae800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7af3400640 1 -- 172.21.15.83:0/3749384531 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8018070 con 0x7f7afc0b52b0 2024-04-25T00:13:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7af3400640 1 -- 172.21.15.83:0/3749384531 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae80074e0 con 0x7f7afc0b52b0 2024-04-25T00:13:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7b02400640 1 -- 172.21.15.83:0/3749384531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7afc157210 con 0x7f7afc0b52b0 2024-04-25T00:13:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7af3400640 1 -- 172.21.15.83:0/3749384531 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8004a30 con 0x7f7afc0b52b0 2024-04-25T00:13:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7b02400640 1 -- 172.21.15.83:0/3749384531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7afc156b70 con 0x7f7afc0b52b0 2024-04-25T00:13:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.441+0000 7f7b02400640 1 -- 172.21.15.83:0/3749384531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7afc072fe0 con 0x7f7afc0b52b0 2024-04-25T00:13:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.445+0000 7f7af3400640 1 -- 172.21.15.83:0/3749384531 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7ae801b070 con 0x7f7afc0b52b0 2024-04-25T00:13:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.445+0000 7f7af3400640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7adc0678f0 0x7f7adc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.445+0000 7f7af3400640 1 -- 172.21.15.83:0/3749384531 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f7ae80851f0 con 0x7f7afc0b52b0 2024-04-25T00:13:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.445+0000 7f7afb400640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7adc0678f0 0x7f7adc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.445+0000 7f7afb400640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7adc0678f0 0x7f7adc069db0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0012f0 tx=0x7f7aec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:25.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.449+0000 7f7af3400640 1 -- 172.21.15.83:0/3749384531 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7ae8055b00 con 0x7f7afc0b52b0 2024-04-25T00:13:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.657+0000 7f7b02400640 1 -- 172.21.15.83:0/3749384531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7afc04f5f0 con 0x7f7afc0b52b0 2024-04-25T00:13:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.661+0000 7f7af3400640 1 -- 172.21.15.83:0/3749384531 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1453 (secure 0 0 0) 0x7f7ae8055400 con 0x7f7afc0b52b0 2024-04-25T00:13:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:25.662 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":164,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003853,"num_in_osds":8,"osd_in_since":1714003843,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":232386560,"bytes_avail":772861726720,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":35,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":6198}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-04-25T00:13:08.468570+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:13:25.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.661+0000 7f7b02400640 1 -- 172.21.15.83:0/3749384531 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7adc0678f0 msgr2=0x7f7adc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.661+0000 7f7b02400640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7adc0678f0 0x7f7adc069db0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0012f0 tx=0x7f7aec009040 comp rx=0 tx=0).stop 2024-04-25T00:13:25.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.661+0000 7f7b02400640 1 -- 172.21.15.83:0/3749384531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7afc0b52b0 msgr2=0x7f7afc154eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.661+0000 7f7b02400640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7afc0b52b0 0x7f7afc154eb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800c4a0 tx=0x7f7ae800c960 comp rx=0 tx=0).stop 2024-04-25T00:13:25.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.661+0000 7f7b02400640 1 -- 172.21.15.83:0/3749384531 shutdown_connections 2024-04-25T00:13:25.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.661+0000 7f7b02400640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7adc0678f0 0x7f7adc069db0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.661+0000 7f7b02400640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7afc161630 0x7f7afc1593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.661+0000 7f7b02400640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7afc0b52b0 0x7f7afc154eb0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.661+0000 7f7b02400640 1 --2- 172.21.15.83:0/3749384531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7afc071db0 0x7f7afc1603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.661+0000 7f7b02400640 1 -- 172.21.15.83:0/3749384531 >> 172.21.15.83:0/3749384531 conn(0x7f7afc06b1e0 msgr2=0x7f7afc1696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:25.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.661+0000 7f7b02400640 1 -- 172.21.15.83:0/3749384531 shutdown_connections 2024-04-25T00:13:25.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.661+0000 7f7b02400640 1 -- 172.21.15.83:0/3749384531 wait complete. 2024-04-25T00:13:25.683 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T00:13:25.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:13:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f357f800640 1 Processor -- start 2024-04-25T00:13:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f357f800640 1 -- start start 2024-04-25T00:13:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f357f800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f35780d0230 0x7f35780d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f357f800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3578163b00 0x7f3578165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f357f800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35780cc9b0 con 0x7f357815bf70 2024-04-25T00:13:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f357f800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f35780d0be0 con 0x7f3578163b00 2024-04-25T00:13:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f357f800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f35780b5ad0 con 0x7f35780d0230 2024-04-25T00:13:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f357d800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3578163b00 0x7f3578165ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f357d800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3578163b00 0x7f3578165ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56818/0 (socket says 172.21.15.83:56818) 2024-04-25T00:13:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f357d800640 1 -- 172.21.15.83:0/2213282526 learned_addr learned my addr 172.21.15.83:0/2213282526 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f3577e00640 1 --2- 172.21.15.83:0/2213282526 >> v2:172.21.15.86:3300/0 conn(0x7f35780d0230 0x7f35780d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f357d800640 1 -- 172.21.15.83:0/2213282526 >> v2:172.21.15.86:3300/0 conn(0x7f35780d0230 msgr2=0x7f35780d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f357d800640 1 --2- 172.21.15.83:0/2213282526 >> v2:172.21.15.86:3300/0 conn(0x7f35780d0230 0x7f35780d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f357d800640 1 -- 172.21.15.83:0/2213282526 >> v1:172.21.15.86:6789/0 conn(0x7f357815bf70 legacy=0x7f35780cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.817+0000 7f357d800640 1 -- 172.21.15.83:0/2213282526 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35780ccc90 con 0x7f3578163b00 2024-04-25T00:13:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.821+0000 7f357d800640 1 --2- 172.21.15.83:0/2213282526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3578163b00 0x7f3578165ef0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f357806ec30 tx=0x7f357001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c3a5f3f67b2df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.821+0000 7f3577400640 1 -- 172.21.15.83:0/2213282526 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35700aa650 con 0x7f3578163b00 2024-04-25T00:13:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.821+0000 7f3577400640 1 -- 172.21.15.83:0/2213282526 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35700af070 con 0x7f3578163b00 2024-04-25T00:13:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.821+0000 7f3577400640 1 -- 172.21.15.83:0/2213282526 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35700aaa10 con 0x7f3578163b00 2024-04-25T00:13:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.821+0000 7f357f800640 1 -- 172.21.15.83:0/2213282526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3578163b00 msgr2=0x7f3578165ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.821+0000 7f357f800640 1 --2- 172.21.15.83:0/2213282526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3578163b00 0x7f3578165ef0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f357806ec30 tx=0x7f357001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.821+0000 7f357f800640 1 -- 172.21.15.83:0/2213282526 shutdown_connections 2024-04-25T00:13:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.821+0000 7f357f800640 1 --2- 172.21.15.83:0/2213282526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3578163b00 0x7f3578165ef0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.821+0000 7f357f800640 1 --2- 172.21.15.83:0/2213282526 >> v2:172.21.15.86:3300/0 conn(0x7f35780d0230 0x7f35780d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.821+0000 7f357f800640 1 -- 172.21.15.83:0/2213282526 >> 172.21.15.83:0/2213282526 conn(0x7f35780d1c70 msgr2=0x7f35780d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:25.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.821+0000 7f357f800640 1 -- 172.21.15.83:0/2213282526 shutdown_connections 2024-04-25T00:13:25.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.821+0000 7f357f800640 1 -- 172.21.15.83:0/2213282526 wait complete. 2024-04-25T00:13:25.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357f800640 1 Processor -- start 2024-04-25T00:13:25.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357f800640 1 -- start start 2024-04-25T00:13:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357f800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35780d0230 0x7f357814d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357815bf70 0x7f35780c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357f800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3578163b00 0x7f357814cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357f800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f357815c350 con 0x7f3578163b00 2024-04-25T00:13:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35780d29d0 con 0x7f357815bf70 2024-04-25T00:13:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357f800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f35780cff50 con 0x7f35780d0230 2024-04-25T00:13:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3578163b00 0x7f357814cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3578163b00 0x7f357814cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56830/0 (socket says 172.21.15.83:56830) 2024-04-25T00:13:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357d800640 1 -- 172.21.15.83:0/3411046923 learned_addr learned my addr 172.21.15.83:0/3411046923 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f3577e00640 1 --2- 172.21.15.83:0/3411046923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357815bf70 0x7f35780c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357ce00640 1 --2- 172.21.15.83:0/3411046923 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35780d0230 0x7f357814d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357d800640 1 -- 172.21.15.83:0/3411046923 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35780d0230 msgr2=0x7f357814d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357d800640 1 --2- 172.21.15.83:0/3411046923 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35780d0230 0x7f357814d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357d800640 1 -- 172.21.15.83:0/3411046923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357815bf70 msgr2=0x7f35780c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357d800640 1 --2- 172.21.15.83:0/3411046923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357815bf70 0x7f35780c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357d800640 1 -- 172.21.15.83:0/3411046923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357814d150 con 0x7f3578163b00 2024-04-25T00:13:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357d800640 1 --2- 172.21.15.83:0/3411046923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3578163b00 0x7f357814cc10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3570019100 tx=0x7f357001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f3574c00640 1 -- 172.21.15.83:0/3411046923 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35700bd5f0 con 0x7f3578163b00 2024-04-25T00:13:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f3574c00640 1 -- 172.21.15.83:0/3411046923 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35700a6040 con 0x7f3578163b00 2024-04-25T00:13:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f3574c00640 1 -- 172.21.15.83:0/3411046923 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35700bd9d0 con 0x7f3578163b00 2024-04-25T00:13:25.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357f800640 1 -- 172.21.15.83:0/3411046923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35780b3880 con 0x7f3578163b00 2024-04-25T00:13:25.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357f800640 1 -- 172.21.15.83:0/3411046923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35780b3e40 con 0x7f3578163b00 2024-04-25T00:13:25.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.825+0000 7f357f800640 1 -- 172.21.15.83:0/3411046923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f357809fd60 con 0x7f3578163b00 2024-04-25T00:13:25.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.829+0000 7f3574c00640 1 -- 172.21.15.83:0/3411046923 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f35700b33f0 con 0x7f3578163b00 2024-04-25T00:13:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.829+0000 7f3574c00640 1 --2- 172.21.15.83:0/3411046923 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35580678c0 0x7f3558069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.829+0000 7f3574c00640 1 -- 172.21.15.83:0/3411046923 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f3570120400 con 0x7f3578163b00 2024-04-25T00:13:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.829+0000 7f357ce00640 1 --2- 172.21.15.83:0/3411046923 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35580678c0 0x7f3558069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.829+0000 7f357ce00640 1 --2- 172.21.15.83:0/3411046923 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35580678c0 0x7f3558069d80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3568001d50 tx=0x7f3568009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:25.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.833+0000 7f3574c00640 1 -- 172.21.15.83:0/3411046923 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f35700c3060 con 0x7f3578163b00 2024-04-25T00:13:25.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.985+0000 7f357f800640 1 -- 172.21.15.83:0/3411046923 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f357804f5f0 con 0x7f35580678c0 2024-04-25T00:13:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.993+0000 7f3574c00640 1 -- 172.21.15.83:0/3411046923 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246710 (secure 0 0 0) 0x7f357804f5f0 con 0x7f35580678c0 2024-04-25T00:13:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:25.998 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:13:25.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.993+0000 7f357f800640 1 -- 172.21.15.83:0/3411046923 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35580678c0 msgr2=0x7f3558069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.993+0000 7f357f800640 1 --2- 172.21.15.83:0/3411046923 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35580678c0 0x7f3558069d80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3568001d50 tx=0x7f3568009040 comp rx=0 tx=0).stop 2024-04-25T00:13:25.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.993+0000 7f357f800640 1 -- 172.21.15.83:0/3411046923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3578163b00 msgr2=0x7f357814cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:26.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.993+0000 7f357f800640 1 --2- 172.21.15.83:0/3411046923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3578163b00 0x7f357814cc10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3570019100 tx=0x7f357001ad30 comp rx=0 tx=0).stop 2024-04-25T00:13:26.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.997+0000 7f357f800640 1 -- 172.21.15.83:0/3411046923 shutdown_connections 2024-04-25T00:13:26.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.997+0000 7f357f800640 1 --2- 172.21.15.83:0/3411046923 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35580678c0 0x7f3558069d80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:26.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.997+0000 7f357f800640 1 --2- 172.21.15.83:0/3411046923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3578163b00 0x7f357814cc10 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:26.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.997+0000 7f357f800640 1 --2- 172.21.15.83:0/3411046923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357815bf70 0x7f35780c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:26.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.997+0000 7f357f800640 1 --2- 172.21.15.83:0/3411046923 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35780d0230 0x7f357814d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:26.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.997+0000 7f357f800640 1 -- 172.21.15.83:0/3411046923 >> 172.21.15.83:0/3411046923 conn(0x7f35780d1c70 msgr2=0x7f357815a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:26.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.997+0000 7f357f800640 1 -- 172.21.15.83:0/3411046923 shutdown_connections 2024-04-25T00:13:26.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:25.997+0000 7f357f800640 1 -- 172.21.15.83:0/3411046923 wait complete. 2024-04-25T00:13:26.016 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-04-25T00:13:24.470989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226940,"kb_used_data":6560,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747780,"statfs":{"total":773094113280,"available":772861726720,"internally_reserved":0,"allocated":6717440,"data_stored":3630326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000984"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427611+0000","last_change":"2024-04-25T00:13:19.427611+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427611+0000","last_undegraded":"2024-04-25T00:13:19.427611+0000","last_fullsized":"2024-04-25T00:13:19.427611+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427995+0000","last_change":"2024-04-25T00:13:19.427995+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427995+0000","last_undegraded":"2024-04-25T00:13:19.427995+0000","last_fullsized":"2024-04-25T00:13:19.427995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420276+0000","last_change":"2024-04-25T00:13:19.420276+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420276+0000","last_undegraded":"2024-04-25T00:13:19.420276+0000","last_fullsized":"2024-04-25T00:13:19.420276+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428639+0000","last_change":"2024-04-25T00:13:19.428639+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428639+0000","last_undegraded":"2024-04-25T00:13:19.428639+0000","last_fullsized":"2024-04-25T00:13:19.428639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420673+0000","last_change":"2024-04-25T00:13:19.420673+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420673+0000","last_undegraded":"2024-04-25T00:13:19.420673+0000","last_fullsized":"2024-04-25T00:13:19.420673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422946+0000","last_change":"2024-04-25T00:13:19.422946+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422946+0000","last_undegraded":"2024-04-25T00:13:19.422946+0000","last_fullsized":"2024-04-25T00:13:19.422946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431248+0000","last_change":"2024-04-25T00:13:19.431248+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431248+0000","last_undegraded":"2024-04-25T00:13:19.431248+0000","last_fullsized":"2024-04-25T00:13:19.431248+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.429195+0000","last_change":"2024-04-25T00:13:19.429195+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.429195+0000","last_undegraded":"2024-04-25T00:13:19.429195+0000","last_fullsized":"2024-04-25T00:13:19.429195+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.421312+0000","last_change":"2024-04-25T00:13:19.421312+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.421312+0000","last_undegraded":"2024-04-25T00:13:19.421312+0000","last_fullsized":"2024-04-25T00:13:19.421312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423950+0000","last_change":"2024-04-25T00:13:19.423950+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423950+0000","last_undegraded":"2024-04-25T00:13:19.423950+0000","last_fullsized":"2024-04-25T00:13:19.423950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427090+0000","last_change":"2024-04-25T00:13:19.427090+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427090+0000","last_undegraded":"2024-04-25T00:13:19.427090+0000","last_fullsized":"2024-04-25T00:13:19.427090+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431410+0000","last_change":"2024-04-25T00:13:19.431410+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431410+0000","last_undegraded":"2024-04-25T00:13:19.431410+0000","last_fullsized":"2024-04-25T00:13:19.431410+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431226+0000","last_change":"2024-04-25T00:13:19.431226+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431226+0000","last_undegraded":"2024-04-25T00:13:19.431226+0000","last_fullsized":"2024-04-25T00:13:19.431226+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419593+0000","last_change":"2024-04-25T00:13:19.419593+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419593+0000","last_undegraded":"2024-04-25T00:13:19.419593+0000","last_fullsized":"2024-04-25T00:13:19.419593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419376+0000","last_change":"2024-04-25T00:13:19.419376+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419376+0000","last_undegraded":"2024-04-25T00:13:19.419376+0000","last_fullsized":"2024-04-25T00:13:19.419376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431142+0000","last_change":"2024-04-25T00:13:19.431142+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431142+0000","last_undegraded":"2024-04-25T00:13:19.431142+0000","last_fullsized":"2024-04-25T00:13:19.431142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427186+0000","last_change":"2024-04-25T00:13:19.427186+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427186+0000","last_undegraded":"2024-04-25T00:13:19.427186+0000","last_fullsized":"2024-04-25T00:13:19.427186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428152+0000","last_change":"2024-04-25T00:13:19.428152+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428152+0000","last_undegraded":"2024-04-25T00:13:19.428152+0000","last_fullsized":"2024-04-25T00:13:19.428152+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420621+0000","last_change":"2024-04-25T00:13:19.420621+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420621+0000","last_undegraded":"2024-04-25T00:13:19.420621+0000","last_fullsized":"2024-04-25T00:13:19.420621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423212+0000","last_change":"2024-04-25T00:13:19.423212+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423212+0000","last_undegraded":"2024-04-25T00:13:19.423212+0000","last_fullsized":"2024-04-25T00:13:19.423212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430777+0000","last_change":"2024-04-25T00:13:19.430777+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430777+0000","last_undegraded":"2024-04-25T00:13:19.430777+0000","last_fullsized":"2024-04-25T00:13:19.430777+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422795+0000","last_change":"2024-04-25T00:13:19.422795+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422795+0000","last_undegraded":"2024-04-25T00:13:19.422795+0000","last_fullsized":"2024-04-25T00:13:19.422795+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422637+0000","last_change":"2024-04-25T00:13:19.422637+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422637+0000","last_undegraded":"2024-04-25T00:13:19.422637+0000","last_fullsized":"2024-04-25T00:13:19.422637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423888+0000","last_change":"2024-04-25T00:13:19.423888+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423888+0000","last_undegraded":"2024-04-25T00:13:19.423888+0000","last_fullsized":"2024-04-25T00:13:19.423888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.429151+0000","last_change":"2024-04-25T00:13:19.429151+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.429151+0000","last_undegraded":"2024-04-25T00:13:19.429151+0000","last_fullsized":"2024-04-25T00:13:19.429151+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428403+0000","last_change":"2024-04-25T00:13:19.428403+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428403+0000","last_undegraded":"2024-04-25T00:13:19.428403+0000","last_fullsized":"2024-04-25T00:13:19.428403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426616+0000","last_change":"2024-04-25T00:13:19.426616+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426616+0000","last_undegraded":"2024-04-25T00:13:19.426616+0000","last_fullsized":"2024-04-25T00:13:19.426616+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426605+0000","last_change":"2024-04-25T00:13:19.426605+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426605+0000","last_undegraded":"2024-04-25T00:13:19.426605+0000","last_fullsized":"2024-04-25T00:13:19.426605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420098+0000","last_change":"2024-04-25T00:13:19.420098+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420098+0000","last_undegraded":"2024-04-25T00:13:19.420098+0000","last_fullsized":"2024-04-25T00:13:19.420098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.431639+0000","last_change":"2024-04-25T00:13:19.431639+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.431639+0000","last_undegraded":"2024-04-25T00:13:19.431639+0000","last_fullsized":"2024-04-25T00:13:19.431639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.429845+0000","last_change":"2024-04-25T00:13:19.429845+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.429845+0000","last_undegraded":"2024-04-25T00:13:19.429845+0000","last_fullsized":"2024-04-25T00:13:19.429845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423891+0000","last_change":"2024-04-25T00:13:19.423891+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423891+0000","last_undegraded":"2024-04-25T00:13:19.423891+0000","last_fullsized":"2024-04-25T00:13:19.423891+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419810+0000","last_change":"2024-04-25T00:13:19.419810+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419810+0000","last_undegraded":"2024-04-25T00:13:19.419810+0000","last_fullsized":"2024-04-25T00:13:19.419810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.448008+0000","last_change":"2024-04-25T00:13:19.445153+0000","last_active":"2024-04-25T00:13:19.448008+0000","last_peered":"2024-04-25T00:13:19.448008+0000","last_clean":"2024-04-25T00:13:19.448008+0000","last_became_active":"2024-04-25T00:13:19.444845+0000","last_became_peered":"2024-04-25T00:13:19.444845+0000","last_unstale":"2024-04-25T00:13:19.448008+0000","last_undegraded":"2024-04-25T00:13:19.448008+0000","last_fullsized":"2024-04-25T00:13:19.448008+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:03.923113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426198+0000","last_change":"2024-04-25T00:13:19.426198+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426198+0000","last_undegraded":"2024-04-25T00:13:19.426198+0000","last_fullsized":"2024-04-25T00:13:19.426198+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.441989+0000","last_change":"2024-04-25T00:13:19.439199+0000","last_active":"2024-04-25T00:13:19.441989+0000","last_peered":"2024-04-25T00:13:19.441989+0000","last_clean":"2024-04-25T00:13:19.441989+0000","last_became_active":"2024-04-25T00:13:19.438895+0000","last_became_peered":"2024-04-25T00:13:19.438895+0000","last_unstale":"2024-04-25T00:13:19.441989+0000","last_undegraded":"2024-04-25T00:13:19.441989+0000","last_fullsized":"2024-04-25T00:13:19.441989+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:20.022144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428957+0000","last_change":"2024-04-25T00:13:19.428957+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428957+0000","last_undegraded":"2024-04-25T00:13:19.428957+0000","last_fullsized":"2024-04-25T00:13:19.428957+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447654+0000","last_change":"2024-04-25T00:13:19.442726+0000","last_active":"2024-04-25T00:13:19.447654+0000","last_peered":"2024-04-25T00:13:19.447654+0000","last_clean":"2024-04-25T00:13:19.447654+0000","last_became_active":"2024-04-25T00:13:19.442433+0000","last_became_peered":"2024-04-25T00:13:19.442433+0000","last_unstale":"2024-04-25T00:13:19.447654+0000","last_undegraded":"2024-04-25T00:13:19.447654+0000","last_fullsized":"2024-04-25T00:13:19.447654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:12.349352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423306+0000","last_change":"2024-04-25T00:13:19.423306+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423306+0000","last_undegraded":"2024-04-25T00:13:19.423306+0000","last_fullsized":"2024-04-25T00:13:19.423306+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.439924+0000","last_change":"2024-04-25T00:13:19.438204+0000","last_active":"2024-04-25T00:13:19.439924+0000","last_peered":"2024-04-25T00:13:19.439924+0000","last_clean":"2024-04-25T00:13:19.439924+0000","last_became_active":"2024-04-25T00:13:19.437845+0000","last_became_peered":"2024-04-25T00:13:19.437845+0000","last_unstale":"2024-04-25T00:13:19.439924+0000","last_undegraded":"2024-04-25T00:13:19.439924+0000","last_fullsized":"2024-04-25T00:13:19.439924+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:47.273633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430288+0000","last_change":"2024-04-25T00:13:19.430288+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430288+0000","last_undegraded":"2024-04-25T00:13:19.430288+0000","last_fullsized":"2024-04-25T00:13:19.430288+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.435286+0000","last_change":"2024-04-25T00:13:19.431189+0000","last_active":"2024-04-25T00:13:19.435286+0000","last_peered":"2024-04-25T00:13:19.435286+0000","last_clean":"2024-04-25T00:13:19.435286+0000","last_became_active":"2024-04-25T00:13:19.430933+0000","last_became_peered":"2024-04-25T00:13:19.430933+0000","last_unstale":"2024-04-25T00:13:19.435286+0000","last_undegraded":"2024-04-25T00:13:19.435286+0000","last_fullsized":"2024-04-25T00:13:19.435286+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:50.255023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.425319+0000","last_change":"2024-04-25T00:13:19.425319+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.425319+0000","last_undegraded":"2024-04-25T00:13:19.425319+0000","last_fullsized":"2024-04-25T00:13:19.425319+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442691+0000","last_change":"2024-04-25T00:13:19.441394+0000","last_active":"2024-04-25T00:13:19.442691+0000","last_peered":"2024-04-25T00:13:19.442691+0000","last_clean":"2024-04-25T00:13:19.442691+0000","last_became_active":"2024-04-25T00:13:19.438688+0000","last_became_peered":"2024-04-25T00:13:19.438688+0000","last_unstale":"2024-04-25T00:13:19.442691+0000","last_undegraded":"2024-04-25T00:13:19.442691+0000","last_fullsized":"2024-04-25T00:13:19.442691+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:28.553754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420922+0000","last_change":"2024-04-25T00:13:19.420922+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420922+0000","last_undegraded":"2024-04-25T00:13:19.420922+0000","last_fullsized":"2024-04-25T00:13:19.420922+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.436636+0000","last_change":"2024-04-25T00:13:19.433617+0000","last_active":"2024-04-25T00:13:19.436636+0000","last_peered":"2024-04-25T00:13:19.436636+0000","last_clean":"2024-04-25T00:13:19.436636+0000","last_became_active":"2024-04-25T00:13:19.433063+0000","last_became_peered":"2024-04-25T00:13:19.433063+0000","last_unstale":"2024-04-25T00:13:19.436636+0000","last_undegraded":"2024-04-25T00:13:19.436636+0000","last_fullsized":"2024-04-25T00:13:19.436636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:18.683206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423602+0000","last_change":"2024-04-25T00:13:19.423602+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423602+0000","last_undegraded":"2024-04-25T00:13:19.423602+0000","last_fullsized":"2024-04-25T00:13:19.423602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442465+0000","last_change":"2024-04-25T00:13:19.431998+0000","last_active":"2024-04-25T00:13:19.442465+0000","last_peered":"2024-04-25T00:13:19.442465+0000","last_clean":"2024-04-25T00:13:19.442465+0000","last_became_active":"2024-04-25T00:13:19.431728+0000","last_became_peered":"2024-04-25T00:13:19.431728+0000","last_unstale":"2024-04-25T00:13:19.442465+0000","last_undegraded":"2024-04-25T00:13:19.442465+0000","last_fullsized":"2024-04-25T00:13:19.442465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:04.083138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426189+0000","last_change":"2024-04-25T00:13:19.426189+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426189+0000","last_undegraded":"2024-04-25T00:13:19.426189+0000","last_fullsized":"2024-04-25T00:13:19.426189+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446032+0000","last_change":"2024-04-25T00:13:19.440887+0000","last_active":"2024-04-25T00:13:19.446032+0000","last_peered":"2024-04-25T00:13:19.446032+0000","last_clean":"2024-04-25T00:13:19.446032+0000","last_became_active":"2024-04-25T00:13:19.437895+0000","last_became_peered":"2024-04-25T00:13:19.437895+0000","last_unstale":"2024-04-25T00:13:19.446032+0000","last_undegraded":"2024-04-25T00:13:19.446032+0000","last_fullsized":"2024-04-25T00:13:19.446032+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:47.273633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.422117+0000","last_change":"2024-04-25T00:13:19.422117+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.422117+0000","last_undegraded":"2024-04-25T00:13:19.422117+0000","last_fullsized":"2024-04-25T00:13:19.422117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.443114+0000","last_change":"2024-04-25T00:13:19.437705+0000","last_active":"2024-04-25T00:13:19.443114+0000","last_peered":"2024-04-25T00:13:19.443114+0000","last_clean":"2024-04-25T00:13:19.443114+0000","last_became_active":"2024-04-25T00:13:19.437390+0000","last_became_peered":"2024-04-25T00:13:19.437390+0000","last_unstale":"2024-04-25T00:13:19.443114+0000","last_undegraded":"2024-04-25T00:13:19.443114+0000","last_fullsized":"2024-04-25T00:13:19.443114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:44.096993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.421441+0000","last_change":"2024-04-25T00:13:19.421441+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.421441+0000","last_undegraded":"2024-04-25T00:13:19.421441+0000","last_fullsized":"2024-04-25T00:13:19.421441+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442484+0000","last_change":"2024-04-25T00:13:19.438746+0000","last_active":"2024-04-25T00:13:19.442484+0000","last_peered":"2024-04-25T00:13:19.442484+0000","last_clean":"2024-04-25T00:13:19.442484+0000","last_became_active":"2024-04-25T00:13:19.437749+0000","last_became_peered":"2024-04-25T00:13:19.437749+0000","last_unstale":"2024-04-25T00:13:19.442484+0000","last_undegraded":"2024-04-25T00:13:19.442484+0000","last_fullsized":"2024-04-25T00:13:19.442484+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:39.640503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426025+0000","last_change":"2024-04-25T00:13:19.426025+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426025+0000","last_undegraded":"2024-04-25T00:13:19.426025+0000","last_fullsized":"2024-04-25T00:13:19.426025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.448034+0000","last_change":"2024-04-25T00:13:19.445212+0000","last_active":"2024-04-25T00:13:19.448034+0000","last_peered":"2024-04-25T00:13:19.448034+0000","last_clean":"2024-04-25T00:13:19.448034+0000","last_became_active":"2024-04-25T00:13:19.444953+0000","last_became_peered":"2024-04-25T00:13:19.444953+0000","last_unstale":"2024-04-25T00:13:19.448034+0000","last_undegraded":"2024-04-25T00:13:19.448034+0000","last_fullsized":"2024-04-25T00:13:19.448034+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:28.280706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.426254+0000","last_change":"2024-04-25T00:13:19.426254+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.426254+0000","last_undegraded":"2024-04-25T00:13:19.426254+0000","last_fullsized":"2024-04-25T00:13:19.426254+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445679+0000","last_change":"2024-04-25T00:13:19.439464+0000","last_active":"2024-04-25T00:13:19.445679+0000","last_peered":"2024-04-25T00:13:19.445679+0000","last_clean":"2024-04-25T00:13:19.445679+0000","last_became_active":"2024-04-25T00:13:19.439143+0000","last_became_peered":"2024-04-25T00:13:19.439143+0000","last_unstale":"2024-04-25T00:13:19.445679+0000","last_undegraded":"2024-04-25T00:13:19.445679+0000","last_fullsized":"2024-04-25T00:13:19.445679+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:03.413067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.435804+0000","last_change":"2024-04-25T00:13:19.435804+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.435804+0000","last_undegraded":"2024-04-25T00:13:19.435804+0000","last_fullsized":"2024-04-25T00:13:19.435804+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446509+0000","last_change":"2024-04-25T00:13:19.443972+0000","last_active":"2024-04-25T00:13:19.446509+0000","last_peered":"2024-04-25T00:13:19.446509+0000","last_clean":"2024-04-25T00:13:19.446509+0000","last_became_active":"2024-04-25T00:13:19.443634+0000","last_became_peered":"2024-04-25T00:13:19.443634+0000","last_unstale":"2024-04-25T00:13:19.446509+0000","last_undegraded":"2024-04-25T00:13:19.446509+0000","last_fullsized":"2024-04-25T00:13:19.446509+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:03.413067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420470+0000","last_change":"2024-04-25T00:13:19.420470+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420470+0000","last_undegraded":"2024-04-25T00:13:19.420470+0000","last_fullsized":"2024-04-25T00:13:19.420470+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.436847+0000","last_change":"2024-04-25T00:13:19.430607+0000","last_active":"2024-04-25T00:13:19.436847+0000","last_peered":"2024-04-25T00:13:19.436847+0000","last_clean":"2024-04-25T00:13:19.436847+0000","last_became_active":"2024-04-25T00:13:19.430240+0000","last_became_peered":"2024-04-25T00:13:19.430240+0000","last_unstale":"2024-04-25T00:13:19.436847+0000","last_undegraded":"2024-04-25T00:13:19.436847+0000","last_fullsized":"2024-04-25T00:13:19.436847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:07.859795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.428661+0000","last_change":"2024-04-25T00:13:19.428661+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.428661+0000","last_undegraded":"2024-04-25T00:13:19.428661+0000","last_fullsized":"2024-04-25T00:13:19.428661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.443661+0000","last_change":"2024-04-25T00:13:19.438379+0000","last_active":"2024-04-25T00:13:19.443661+0000","last_peered":"2024-04-25T00:13:19.443661+0000","last_clean":"2024-04-25T00:13:19.443661+0000","last_became_active":"2024-04-25T00:13:19.438093+0000","last_became_peered":"2024-04-25T00:13:19.438093+0000","last_unstale":"2024-04-25T00:13:19.443661+0000","last_undegraded":"2024-04-25T00:13:19.443661+0000","last_fullsized":"2024-04-25T00:13:19.443661+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:28.036116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.424260+0000","last_change":"2024-04-25T00:13:19.424260+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.424260+0000","last_undegraded":"2024-04-25T00:13:19.424260+0000","last_fullsized":"2024-04-25T00:13:19.424260+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445046+0000","last_change":"2024-04-25T00:13:19.440461+0000","last_active":"2024-04-25T00:13:19.445046+0000","last_peered":"2024-04-25T00:13:19.445046+0000","last_clean":"2024-04-25T00:13:19.445046+0000","last_became_active":"2024-04-25T00:13:19.440197+0000","last_became_peered":"2024-04-25T00:13:19.440197+0000","last_unstale":"2024-04-25T00:13:19.445046+0000","last_undegraded":"2024-04-25T00:13:19.445046+0000","last_fullsized":"2024-04-25T00:13:19.445046+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:37.585343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420102+0000","last_change":"2024-04-25T00:13:19.420102+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420102+0000","last_undegraded":"2024-04-25T00:13:19.420102+0000","last_fullsized":"2024-04-25T00:13:19.420102+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442434+0000","last_change":"2024-04-25T00:13:19.438945+0000","last_active":"2024-04-25T00:13:19.442434+0000","last_peered":"2024-04-25T00:13:19.442434+0000","last_clean":"2024-04-25T00:13:19.442434+0000","last_became_active":"2024-04-25T00:13:19.438052+0000","last_became_peered":"2024-04-25T00:13:19.438052+0000","last_unstale":"2024-04-25T00:13:19.442434+0000","last_undegraded":"2024-04-25T00:13:19.442434+0000","last_fullsized":"2024-04-25T00:13:19.442434+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:21.586818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427651+0000","last_change":"2024-04-25T00:13:19.427651+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427651+0000","last_undegraded":"2024-04-25T00:13:19.427651+0000","last_fullsized":"2024-04-25T00:13:19.427651+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445826+0000","last_change":"2024-04-25T00:13:19.440459+0000","last_active":"2024-04-25T00:13:19.445826+0000","last_peered":"2024-04-25T00:13:19.445826+0000","last_clean":"2024-04-25T00:13:19.445826+0000","last_became_active":"2024-04-25T00:13:19.438442+0000","last_became_peered":"2024-04-25T00:13:19.438442+0000","last_unstale":"2024-04-25T00:13:19.445826+0000","last_undegraded":"2024-04-25T00:13:19.445826+0000","last_fullsized":"2024-04-25T00:13:19.445826+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:42.830716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.420129+0000","last_change":"2024-04-25T00:13:19.420129+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.420129+0000","last_undegraded":"2024-04-25T00:13:19.420129+0000","last_fullsized":"2024-04-25T00:13:19.420129+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.442276+0000","last_change":"2024-04-25T00:13:19.440785+0000","last_active":"2024-04-25T00:13:19.442276+0000","last_peered":"2024-04-25T00:13:19.442276+0000","last_clean":"2024-04-25T00:13:19.442276+0000","last_became_active":"2024-04-25T00:13:19.440512+0000","last_became_peered":"2024-04-25T00:13:19.440512+0000","last_unstale":"2024-04-25T00:13:19.442276+0000","last_undegraded":"2024-04-25T00:13:19.442276+0000","last_fullsized":"2024-04-25T00:13:19.442276+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:35.767979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.423501+0000","last_change":"2024-04-25T00:13:19.423501+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.423501+0000","last_undegraded":"2024-04-25T00:13:19.423501+0000","last_fullsized":"2024-04-25T00:13:19.423501+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447676+0000","last_change":"2024-04-25T00:13:19.443427+0000","last_active":"2024-04-25T00:13:19.447676+0000","last_peered":"2024-04-25T00:13:19.447676+0000","last_clean":"2024-04-25T00:13:19.447676+0000","last_became_active":"2024-04-25T00:13:19.443147+0000","last_became_peered":"2024-04-25T00:13:19.443147+0000","last_unstale":"2024-04-25T00:13:19.447676+0000","last_undegraded":"2024-04-25T00:13:19.447676+0000","last_fullsized":"2024-04-25T00:13:19.447676+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:17.551103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.435341+0000","last_change":"2024-04-25T00:13:19.435341+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.435341+0000","last_undegraded":"2024-04-25T00:13:19.435341+0000","last_fullsized":"2024-04-25T00:13:19.435341+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.445110+0000","last_change":"2024-04-25T00:13:19.441302+0000","last_active":"2024-04-25T00:13:19.445110+0000","last_peered":"2024-04-25T00:13:19.445110+0000","last_clean":"2024-04-25T00:13:19.445110+0000","last_became_active":"2024-04-25T00:13:19.440951+0000","last_became_peered":"2024-04-25T00:13:19.440951+0000","last_unstale":"2024-04-25T00:13:19.445110+0000","last_undegraded":"2024-04-25T00:13:19.445110+0000","last_fullsized":"2024-04-25T00:13:19.445110+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:07.859795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419439+0000","last_change":"2024-04-25T00:13:19.419439+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419439+0000","last_undegraded":"2024-04-25T00:13:19.419439+0000","last_fullsized":"2024-04-25T00:13:19.419439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.441866+0000","last_change":"2024-04-25T00:13:19.438811+0000","last_active":"2024-04-25T00:13:19.441866+0000","last_peered":"2024-04-25T00:13:19.441866+0000","last_clean":"2024-04-25T00:13:19.441866+0000","last_became_active":"2024-04-25T00:13:19.438527+0000","last_became_peered":"2024-04-25T00:13:19.438527+0000","last_unstale":"2024-04-25T00:13:19.441866+0000","last_undegraded":"2024-04-25T00:13:19.441866+0000","last_fullsized":"2024-04-25T00:13:19.441866+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:13.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.424634+0000","last_change":"2024-04-25T00:13:19.424634+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.424634+0000","last_undegraded":"2024-04-25T00:13:19.424634+0000","last_fullsized":"2024-04-25T00:13:19.424634+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446400+0000","last_change":"2024-04-25T00:13:19.442162+0000","last_active":"2024-04-25T00:13:19.446400+0000","last_peered":"2024-04-25T00:13:19.446400+0000","last_clean":"2024-04-25T00:13:19.446400+0000","last_became_active":"2024-04-25T00:13:19.441874+0000","last_became_peered":"2024-04-25T00:13:19.441874+0000","last_unstale":"2024-04-25T00:13:19.446400+0000","last_undegraded":"2024-04-25T00:13:19.446400+0000","last_fullsized":"2024-04-25T00:13:19.446400+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:39.640503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430355+0000","last_change":"2024-04-25T00:13:19.430355+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430355+0000","last_undegraded":"2024-04-25T00:13:19.430355+0000","last_fullsized":"2024-04-25T00:13:19.430355+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.446460+0000","last_change":"2024-04-25T00:13:19.443257+0000","last_active":"2024-04-25T00:13:19.446460+0000","last_peered":"2024-04-25T00:13:19.446460+0000","last_clean":"2024-04-25T00:13:19.446460+0000","last_became_active":"2024-04-25T00:13:19.442853+0000","last_became_peered":"2024-04-25T00:13:19.442853+0000","last_unstale":"2024-04-25T00:13:19.446460+0000","last_undegraded":"2024-04-25T00:13:19.446460+0000","last_fullsized":"2024-04-25T00:13:19.446460+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:21.586818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.427928+0000","last_change":"2024-04-25T00:13:19.427928+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.427928+0000","last_undegraded":"2024-04-25T00:13:19.427928+0000","last_fullsized":"2024-04-25T00:13:19.427928+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447682+0000","last_change":"2024-04-25T00:13:19.444471+0000","last_active":"2024-04-25T00:13:19.447682+0000","last_peered":"2024-04-25T00:13:19.447682+0000","last_clean":"2024-04-25T00:13:19.447682+0000","last_became_active":"2024-04-25T00:13:19.444197+0000","last_became_peered":"2024-04-25T00:13:19.444197+0000","last_unstale":"2024-04-25T00:13:19.447682+0000","last_undegraded":"2024-04-25T00:13:19.447682+0000","last_fullsized":"2024-04-25T00:13:19.447682+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:29.987108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419511+0000","last_change":"2024-04-25T00:13:19.419511+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419511+0000","last_undegraded":"2024-04-25T00:13:19.419511+0000","last_fullsized":"2024-04-25T00:13:19.419511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.447962+0000","last_change":"2024-04-25T00:13:19.445103+0000","last_active":"2024-04-25T00:13:19.447962+0000","last_peered":"2024-04-25T00:13:19.447962+0000","last_clean":"2024-04-25T00:13:19.447962+0000","last_became_active":"2024-04-25T00:13:19.444652+0000","last_became_peered":"2024-04-25T00:13:19.444652+0000","last_unstale":"2024-04-25T00:13:19.447962+0000","last_undegraded":"2024-04-25T00:13:19.447962+0000","last_fullsized":"2024-04-25T00:13:19.447962+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:39.691842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419832+0000","last_change":"2024-04-25T00:13:19.419832+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419832+0000","last_undegraded":"2024-04-25T00:13:19.419832+0000","last_fullsized":"2024-04-25T00:13:19.419832+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.448638+0000","last_change":"2024-04-25T00:13:19.442646+0000","last_active":"2024-04-25T00:13:19.448638+0000","last_peered":"2024-04-25T00:13:19.448638+0000","last_clean":"2024-04-25T00:13:19.448638+0000","last_became_active":"2024-04-25T00:13:19.438848+0000","last_became_peered":"2024-04-25T00:13:19.438848+0000","last_unstale":"2024-04-25T00:13:19.448638+0000","last_undegraded":"2024-04-25T00:13:19.448638+0000","last_fullsized":"2024-04-25T00:13:19.448638+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:13.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.425579+0000","last_change":"2024-04-25T00:13:19.425579+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.425579+0000","last_undegraded":"2024-04-25T00:13:19.425579+0000","last_fullsized":"2024-04-25T00:13:19.425579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.439322+0000","last_change":"2024-04-25T00:13:19.436516+0000","last_active":"2024-04-25T00:13:19.439322+0000","last_peered":"2024-04-25T00:13:19.439322+0000","last_clean":"2024-04-25T00:13:19.439322+0000","last_became_active":"2024-04-25T00:13:19.435936+0000","last_became_peered":"2024-04-25T00:13:19.435936+0000","last_unstale":"2024-04-25T00:13:19.439322+0000","last_undegraded":"2024-04-25T00:13:19.439322+0000","last_fullsized":"2024-04-25T00:13:19.439322+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:57.960615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.424954+0000","last_change":"2024-04-25T00:13:19.424954+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.424954+0000","last_undegraded":"2024-04-25T00:13:19.424954+0000","last_fullsized":"2024-04-25T00:13:19.424954+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.429368+0000","last_change":"2024-04-25T00:13:19.427399+0000","last_active":"2024-04-25T00:13:19.429368+0000","last_peered":"2024-04-25T00:13:19.429368+0000","last_clean":"2024-04-25T00:13:19.429368+0000","last_became_active":"2024-04-25T00:13:19.427145+0000","last_became_peered":"2024-04-25T00:13:19.427145+0000","last_unstale":"2024-04-25T00:13:19.429368+0000","last_undegraded":"2024-04-25T00:13:19.429368+0000","last_fullsized":"2024-04-25T00:13:19.429368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:19.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.430930+0000","last_change":"2024-04-25T00:13:19.430930+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.430930+0000","last_undegraded":"2024-04-25T00:13:19.430930+0000","last_fullsized":"2024-04-25T00:13:19.430930+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.441056+0000","last_change":"2024-04-25T00:13:19.438635+0000","last_active":"2024-04-25T00:13:19.441056+0000","last_peered":"2024-04-25T00:13:19.441056+0000","last_clean":"2024-04-25T00:13:19.441056+0000","last_became_active":"2024-04-25T00:13:19.438328+0000","last_became_peered":"2024-04-25T00:13:19.438328+0000","last_unstale":"2024-04-25T00:13:19.441056+0000","last_undegraded":"2024-04-25T00:13:19.441056+0000","last_fullsized":"2024-04-25T00:13:19.441056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:17.223076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:13:19.419237+0000","last_change":"2024-04-25T00:13:19.419237+0000","last_active":"2024-04-25T00:13:19.412448+0000","last_peered":"2024-04-25T00:13:19.412448+0000","last_clean":"2024-04-25T00:13:19.412448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:13:19.419237+0000","last_undegraded":"2024-04-25T00:13:19.419237+0000","last_fullsized":"2024-04-25T00:13:19.419237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.433512+0000","last_change":"2024-04-25T00:13:19.430683+0000","last_active":"2024-04-25T00:13:19.433512+0000","last_peered":"2024-04-25T00:13:19.433512+0000","last_clean":"2024-04-25T00:13:19.433512+0000","last_became_active":"2024-04-25T00:13:19.430464+0000","last_became_peered":"2024-04-25T00:13:19.430464+0000","last_unstale":"2024-04-25T00:13:19.433512+0000","last_undegraded":"2024-04-25T00:13:19.433512+0000","last_fullsized":"2024-04-25T00:13:19.433512+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:36.678931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":239,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:13:19.424193+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:13:19.424193+0000","last_peered":"2024-04-25T00:13:19.424193+0000","last_clean":"2024-04-25T00:13:19.424193+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:13:19.424193+0000","last_undegraded":"2024-04-25T00:13:19.424193+0000","last_fullsized":"2024-04-25T00:13:19.424193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1294336,"data_stored":901561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":700416,"data_stored":311193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":688128,"data_stored":304534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1282048,"data_stored":894902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:13:29.021 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:13:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d4400640 1 Processor -- start 2024-04-25T00:13:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d4400640 1 -- start start 2024-04-25T00:13:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d4400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f89cc1563c0 0x7f89cc06b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d4400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc15d090 0x7f89cc0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d4400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89cc0ae200 con 0x7f89cc0aef80 2024-04-25T00:13:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d4400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f89cc06baa0 con 0x7f89cc1563c0 2024-04-25T00:13:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d4400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f89cc0ae4a0 con 0x7f89cc15d090 2024-04-25T00:13:29.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d1000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc15d090 0x7f89cc0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d1a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f89cc1563c0 0x7f89cc06b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d1000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc15d090 0x7f89cc0afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:53066/0 (socket says 172.21.15.83:53066) 2024-04-25T00:13:29.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d1a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f89cc1563c0 0x7f89cc06b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:60550/0 (socket says 172.21.15.83:60550) 2024-04-25T00:13:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d1000640 1 -- 172.21.15.83:0/1652597881 learned_addr learned my addr 172.21.15.83:0/1652597881 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d1a00640 1 -- 172.21.15.83:0/1652597881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc15d090 msgr2=0x7f89cc0afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d1a00640 1 --2- 172.21.15.83:0/1652597881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc15d090 0x7f89cc0afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d1a00640 1 -- 172.21.15.83:0/1652597881 >> v1:172.21.15.86:6789/0 conn(0x7f89cc0aef80 legacy=0x7f89cc071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d1a00640 1 -- 172.21.15.83:0/1652597881 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89cc0ae780 con 0x7f89cc1563c0 2024-04-25T00:13:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.164+0000 7f89d1000640 1 --2- 172.21.15.83:0/1652597881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc15d090 0x7f89cc0afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.168+0000 7f89d1a00640 1 --2- 172.21.15.83:0/1652597881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc1563c0 0x7f89cc06b1e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009d90 tx=0x7f89c001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7389294c15c992b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.168+0000 7f89cbe00640 1 -- 172.21.15.83:0/1652597881 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c00b2070 con 0x7f89cc1563c0 2024-04-25T00:13:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.168+0000 7f89cbe00640 1 -- 172.21.15.83:0/1652597881 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c00a5a20 con 0x7f89cc1563c0 2024-04-25T00:13:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.168+0000 7f89cbe00640 1 -- 172.21.15.83:0/1652597881 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c00a5df0 con 0x7f89cc1563c0 2024-04-25T00:13:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.168+0000 7f89d4400640 1 -- 172.21.15.83:0/1652597881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc1563c0 msgr2=0x7f89cc06b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.168+0000 7f89d4400640 1 --2- 172.21.15.83:0/1652597881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc1563c0 0x7f89cc06b1e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009d90 tx=0x7f89c001c940 comp rx=0 tx=0).stop 2024-04-25T00:13:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.168+0000 7f89d4400640 1 -- 172.21.15.83:0/1652597881 shutdown_connections 2024-04-25T00:13:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.168+0000 7f89d4400640 1 --2- 172.21.15.83:0/1652597881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc15d090 0x7f89cc0afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.168+0000 7f89d4400640 1 --2- 172.21.15.83:0/1652597881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc1563c0 0x7f89cc06b1e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.168+0000 7f89d4400640 1 -- 172.21.15.83:0/1652597881 >> 172.21.15.83:0/1652597881 conn(0x7f89cc0d0c20 msgr2=0x7f89cc0d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.168+0000 7f89d4400640 1 -- 172.21.15.83:0/1652597881 shutdown_connections 2024-04-25T00:13:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.168+0000 7f89d4400640 1 -- 172.21.15.83:0/1652597881 wait complete. 2024-04-25T00:13:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d4400640 1 Processor -- start 2024-04-25T00:13:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d4400640 1 -- start start 2024-04-25T00:13:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d4400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc0aef80 0x7f89cc13f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d4400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89cc1563c0 0x7f89cc1f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d4400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc15d090 0x7f89cc15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d4400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc168a30 con 0x7f89cc1563c0 2024-04-25T00:13:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d4400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc0b0680 con 0x7f89cc15d090 2024-04-25T00:13:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d4400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f89cc168bb0 con 0x7f89cc0aef80 2024-04-25T00:13:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89cc1563c0 0x7f89cc1f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89cc1563c0 0x7f89cc1f73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56846/0 (socket says 172.21.15.83:56846) 2024-04-25T00:13:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d1000640 1 -- 172.21.15.83:0/3835943355 learned_addr learned my addr 172.21.15.83:0/3835943355 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d1a00640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc0aef80 0x7f89cc13f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d2400640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc15d090 0x7f89cc15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d1000640 1 -- 172.21.15.83:0/3835943355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc0aef80 msgr2=0x7f89cc13f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d1000640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc0aef80 0x7f89cc13f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d1000640 1 -- 172.21.15.83:0/3835943355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc15d090 msgr2=0x7f89cc15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d1000640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc15d090 0x7f89cc15b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d1000640 1 -- 172.21.15.83:0/3835943355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89cc1f7ec0 con 0x7f89cc1563c0 2024-04-25T00:13:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d2400640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc15d090 0x7f89cc15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d1000640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89cc1563c0 0x7f89cc1f73c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f89b800ea00 tx=0x7f89b800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89c9600640 1 -- 172.21.15.83:0/3835943355 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8017070 con 0x7f89cc1563c0 2024-04-25T00:13:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89c9600640 1 -- 172.21.15.83:0/3835943355 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b800ca00 con 0x7f89cc1563c0 2024-04-25T00:13:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d4400640 1 -- 172.21.15.83:0/3835943355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89cc1f81a0 con 0x7f89cc1563c0 2024-04-25T00:13:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89c9600640 1 -- 172.21.15.83:0/3835943355 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b800cdb0 con 0x7f89cc1563c0 2024-04-25T00:13:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.172+0000 7f89d4400640 1 -- 172.21.15.83:0/3835943355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89cc153e20 con 0x7f89cc1563c0 2024-04-25T00:13:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.176+0000 7f89d4400640 1 -- 172.21.15.83:0/3835943355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89cc09f0b0 con 0x7f89cc1563c0 2024-04-25T00:13:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.176+0000 7f89c9600640 1 -- 172.21.15.83:0/3835943355 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f89b801a070 con 0x7f89cc1563c0 2024-04-25T00:13:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.176+0000 7f89c9600640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f89ac0678c0 0x7f89ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.176+0000 7f89c9600640 1 -- 172.21.15.83:0/3835943355 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f89b8085fe0 con 0x7f89cc1563c0 2024-04-25T00:13:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.176+0000 7f89d1a00640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f89ac0678c0 0x7f89ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.176+0000 7f89d1a00640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f89ac0678c0 0x7f89ac069d80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f89c00af000 tx=0x7f89c00a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:29.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.180+0000 7f89c9600640 1 -- 172.21.15.83:0/3835943355 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f89b8056390 con 0x7f89cc1563c0 2024-04-25T00:13:29.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.332+0000 7f89d4400640 1 -- 172.21.15.83:0/3835943355 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f89cc0b6490 con 0x7f89ac0678c0 2024-04-25T00:13:29.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.340+0000 7f89c9600640 1 -- 172.21.15.83:0/3835943355 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252020 (secure 0 0 0) 0x7f89cc0b6490 con 0x7f89ac0678c0 2024-04-25T00:13:29.345 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:29.346 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:13:29.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.344+0000 7f89d4400640 1 -- 172.21.15.83:0/3835943355 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f89ac0678c0 msgr2=0x7f89ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.344+0000 7f89d4400640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f89ac0678c0 0x7f89ac069d80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f89c00af000 tx=0x7f89c00a7070 comp rx=0 tx=0).stop 2024-04-25T00:13:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.344+0000 7f89d4400640 1 -- 172.21.15.83:0/3835943355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89cc1563c0 msgr2=0x7f89cc1f73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.344+0000 7f89d4400640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89cc1563c0 0x7f89cc1f73c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f89b800ea00 tx=0x7f89b800eec0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.344+0000 7f89d4400640 1 -- 172.21.15.83:0/3835943355 shutdown_connections 2024-04-25T00:13:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.344+0000 7f89d4400640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f89ac0678c0 0x7f89ac069d80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.344+0000 7f89d4400640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89cc15d090 0x7f89cc15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.344+0000 7f89d4400640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89cc1563c0 0x7f89cc1f73c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.344+0000 7f89d4400640 1 --2- 172.21.15.83:0/3835943355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89cc0aef80 0x7f89cc13f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.344+0000 7f89d4400640 1 -- 172.21.15.83:0/3835943355 >> 172.21.15.83:0/3835943355 conn(0x7f89cc0d0c20 msgr2=0x7f89cc0b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.344+0000 7f89d4400640 1 -- 172.21.15.83:0/3835943355 shutdown_connections 2024-04-25T00:13:29.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.344+0000 7f89d4400640 1 -- 172.21.15.83:0/3835943355 wait complete. 2024-04-25T00:13:29.366 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-04-25T00:13:28.471782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227268,"kb_used_data":6888,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747452,"statfs":{"total":773094113280,"available":772861390848,"internally_reserved":0,"allocated":7053312,"data_stored":3785972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.042799"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.832192+0000","last_change":"2024-04-25T00:13:20.455402+0000","last_active":"2024-04-25T00:13:21.832192+0000","last_peered":"2024-04-25T00:13:21.832192+0000","last_clean":"2024-04-25T00:13:21.832192+0000","last_became_active":"2024-04-25T00:13:20.455103+0000","last_became_peered":"2024-04-25T00:13:20.455103+0000","last_unstale":"2024-04-25T00:13:21.832192+0000","last_undegraded":"2024-04-25T00:13:21.832192+0000","last_fullsized":"2024-04-25T00:13:21.832192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:04.226374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.832125+0000","last_change":"2024-04-25T00:13:20.451957+0000","last_active":"2024-04-25T00:13:21.832125+0000","last_peered":"2024-04-25T00:13:21.832125+0000","last_clean":"2024-04-25T00:13:21.832125+0000","last_became_active":"2024-04-25T00:13:20.451650+0000","last_became_peered":"2024-04-25T00:13:20.451650+0000","last_unstale":"2024-04-25T00:13:21.832125+0000","last_undegraded":"2024-04-25T00:13:21.832125+0000","last_fullsized":"2024-04-25T00:13:21.832125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:30.989398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450104+0000","last_change":"2024-04-25T00:13:20.473189+0000","last_active":"2024-04-25T00:13:21.450104+0000","last_peered":"2024-04-25T00:13:21.450104+0000","last_clean":"2024-04-25T00:13:21.450104+0000","last_became_active":"2024-04-25T00:13:20.471706+0000","last_became_peered":"2024-04-25T00:13:20.471706+0000","last_unstale":"2024-04-25T00:13:21.450104+0000","last_undegraded":"2024-04-25T00:13:21.450104+0000","last_fullsized":"2024-04-25T00:13:21.450104+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:52.779277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451418+0000","last_change":"2024-04-25T00:13:20.452257+0000","last_active":"2024-04-25T00:13:21.451418+0000","last_peered":"2024-04-25T00:13:21.451418+0000","last_clean":"2024-04-25T00:13:21.451418+0000","last_became_active":"2024-04-25T00:13:20.451973+0000","last_became_peered":"2024-04-25T00:13:20.451973+0000","last_unstale":"2024-04-25T00:13:21.451418+0000","last_undegraded":"2024-04-25T00:13:21.451418+0000","last_fullsized":"2024-04-25T00:13:21.451418+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:34.804667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451714+0000","last_change":"2024-04-25T00:13:20.444372+0000","last_active":"2024-04-25T00:13:21.451714+0000","last_peered":"2024-04-25T00:13:21.451714+0000","last_clean":"2024-04-25T00:13:21.451714+0000","last_became_active":"2024-04-25T00:13:20.443974+0000","last_became_peered":"2024-04-25T00:13:20.443974+0000","last_unstale":"2024-04-25T00:13:21.451714+0000","last_undegraded":"2024-04-25T00:13:21.451714+0000","last_fullsized":"2024-04-25T00:13:21.451714+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:21.024433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.486007+0000","last_change":"2024-04-25T00:13:20.468518+0000","last_active":"2024-04-25T00:13:21.486007+0000","last_peered":"2024-04-25T00:13:21.486007+0000","last_clean":"2024-04-25T00:13:21.486007+0000","last_became_active":"2024-04-25T00:13:20.468232+0000","last_became_peered":"2024-04-25T00:13:20.468232+0000","last_unstale":"2024-04-25T00:13:21.486007+0000","last_undegraded":"2024-04-25T00:13:21.486007+0000","last_fullsized":"2024-04-25T00:13:21.486007+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:02:33.567639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831320+0000","last_change":"2024-04-25T00:13:20.467128+0000","last_active":"2024-04-25T00:13:21.831320+0000","last_peered":"2024-04-25T00:13:21.831320+0000","last_clean":"2024-04-25T00:13:21.831320+0000","last_became_active":"2024-04-25T00:13:20.466778+0000","last_became_peered":"2024-04-25T00:13:20.466778+0000","last_unstale":"2024-04-25T00:13:21.831320+0000","last_undegraded":"2024-04-25T00:13:21.831320+0000","last_fullsized":"2024-04-25T00:13:21.831320+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:30.627154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831037+0000","last_change":"2024-04-25T00:13:20.466286+0000","last_active":"2024-04-25T00:13:21.831037+0000","last_peered":"2024-04-25T00:13:21.831037+0000","last_clean":"2024-04-25T00:13:21.831037+0000","last_became_active":"2024-04-25T00:13:20.465645+0000","last_became_peered":"2024-04-25T00:13:20.465645+0000","last_unstale":"2024-04-25T00:13:21.831037+0000","last_undegraded":"2024-04-25T00:13:21.831037+0000","last_fullsized":"2024-04-25T00:13:21.831037+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:36.770268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451494+0000","last_change":"2024-04-25T00:13:20.474679+0000","last_active":"2024-04-25T00:13:21.451494+0000","last_peered":"2024-04-25T00:13:21.451494+0000","last_clean":"2024-04-25T00:13:21.451494+0000","last_became_active":"2024-04-25T00:13:20.474404+0000","last_became_peered":"2024-04-25T00:13:20.474404+0000","last_unstale":"2024-04-25T00:13:21.451494+0000","last_undegraded":"2024-04-25T00:13:21.451494+0000","last_fullsized":"2024-04-25T00:13:21.451494+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:11.954462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.449539+0000","last_change":"2024-04-25T00:13:20.467643+0000","last_active":"2024-04-25T00:13:21.449539+0000","last_peered":"2024-04-25T00:13:21.449539+0000","last_clean":"2024-04-25T00:13:21.449539+0000","last_became_active":"2024-04-25T00:13:20.467355+0000","last_became_peered":"2024-04-25T00:13:20.467355+0000","last_unstale":"2024-04-25T00:13:21.449539+0000","last_undegraded":"2024-04-25T00:13:21.449539+0000","last_fullsized":"2024-04-25T00:13:21.449539+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:29.390125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450096+0000","last_change":"2024-04-25T00:13:20.448507+0000","last_active":"2024-04-25T00:13:21.450096+0000","last_peered":"2024-04-25T00:13:21.450096+0000","last_clean":"2024-04-25T00:13:21.450096+0000","last_became_active":"2024-04-25T00:13:20.447950+0000","last_became_peered":"2024-04-25T00:13:20.447950+0000","last_unstale":"2024-04-25T00:13:21.450096+0000","last_undegraded":"2024-04-25T00:13:21.450096+0000","last_fullsized":"2024-04-25T00:13:21.450096+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:49.839648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831399+0000","last_change":"2024-04-25T00:13:20.469089+0000","last_active":"2024-04-25T00:13:21.831399+0000","last_peered":"2024-04-25T00:13:21.831399+0000","last_clean":"2024-04-25T00:13:21.831399+0000","last_became_active":"2024-04-25T00:13:20.468634+0000","last_became_peered":"2024-04-25T00:13:20.468634+0000","last_unstale":"2024-04-25T00:13:21.831399+0000","last_undegraded":"2024-04-25T00:13:21.831399+0000","last_fullsized":"2024-04-25T00:13:21.831399+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:52.779277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831059+0000","last_change":"2024-04-25T00:13:20.454567+0000","last_active":"2024-04-25T00:13:21.831059+0000","last_peered":"2024-04-25T00:13:21.831059+0000","last_clean":"2024-04-25T00:13:21.831059+0000","last_became_active":"2024-04-25T00:13:20.454280+0000","last_became_peered":"2024-04-25T00:13:20.454280+0000","last_unstale":"2024-04-25T00:13:21.831059+0000","last_undegraded":"2024-04-25T00:13:21.831059+0000","last_fullsized":"2024-04-25T00:13:21.831059+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:29.556043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.449941+0000","last_change":"2024-04-25T00:13:20.472575+0000","last_active":"2024-04-25T00:13:21.449941+0000","last_peered":"2024-04-25T00:13:21.449941+0000","last_clean":"2024-04-25T00:13:21.449941+0000","last_became_active":"2024-04-25T00:13:20.471906+0000","last_became_peered":"2024-04-25T00:13:20.471906+0000","last_unstale":"2024-04-25T00:13:21.449941+0000","last_undegraded":"2024-04-25T00:13:21.449941+0000","last_fullsized":"2024-04-25T00:13:21.449941+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:02:33.567639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.449677+0000","last_change":"2024-04-25T00:13:20.468995+0000","last_active":"2024-04-25T00:13:21.449677+0000","last_peered":"2024-04-25T00:13:21.449677+0000","last_clean":"2024-04-25T00:13:21.449677+0000","last_became_active":"2024-04-25T00:13:20.468636+0000","last_became_peered":"2024-04-25T00:13:20.468636+0000","last_unstale":"2024-04-25T00:13:21.449677+0000","last_undegraded":"2024-04-25T00:13:21.449677+0000","last_fullsized":"2024-04-25T00:13:21.449677+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:17.471179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831373+0000","last_change":"2024-04-25T00:13:20.456318+0000","last_active":"2024-04-25T00:13:21.831373+0000","last_peered":"2024-04-25T00:13:21.831373+0000","last_clean":"2024-04-25T00:13:21.831373+0000","last_became_active":"2024-04-25T00:13:20.456024+0000","last_became_peered":"2024-04-25T00:13:20.456024+0000","last_unstale":"2024-04-25T00:13:21.831373+0000","last_undegraded":"2024-04-25T00:13:21.831373+0000","last_fullsized":"2024-04-25T00:13:21.831373+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:51.033816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.832175+0000","last_change":"2024-04-25T00:13:20.451886+0000","last_active":"2024-04-25T00:13:21.832175+0000","last_peered":"2024-04-25T00:13:21.832175+0000","last_clean":"2024-04-25T00:13:21.832175+0000","last_became_active":"2024-04-25T00:13:20.451600+0000","last_became_peered":"2024-04-25T00:13:20.451600+0000","last_unstale":"2024-04-25T00:13:21.832175+0000","last_undegraded":"2024-04-25T00:13:21.832175+0000","last_fullsized":"2024-04-25T00:13:21.832175+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:19:50.664215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450580+0000","last_change":"2024-04-25T00:13:20.455137+0000","last_active":"2024-04-25T00:13:21.450580+0000","last_peered":"2024-04-25T00:13:21.450580+0000","last_clean":"2024-04-25T00:13:21.450580+0000","last_became_active":"2024-04-25T00:13:20.453439+0000","last_became_peered":"2024-04-25T00:13:20.453439+0000","last_unstale":"2024-04-25T00:13:21.450580+0000","last_undegraded":"2024-04-25T00:13:21.450580+0000","last_fullsized":"2024-04-25T00:13:21.450580+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:07.327604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451430+0000","last_change":"2024-04-25T00:13:20.457655+0000","last_active":"2024-04-25T00:13:21.451430+0000","last_peered":"2024-04-25T00:13:21.451430+0000","last_clean":"2024-04-25T00:13:21.451430+0000","last_became_active":"2024-04-25T00:13:20.457141+0000","last_became_peered":"2024-04-25T00:13:20.457141+0000","last_unstale":"2024-04-25T00:13:21.451430+0000","last_undegraded":"2024-04-25T00:13:21.451430+0000","last_fullsized":"2024-04-25T00:13:21.451430+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:40.642792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.452392+0000","last_change":"2024-04-25T00:13:20.447516+0000","last_active":"2024-04-25T00:13:21.452392+0000","last_peered":"2024-04-25T00:13:21.452392+0000","last_clean":"2024-04-25T00:13:21.452392+0000","last_became_active":"2024-04-25T00:13:20.447196+0000","last_became_peered":"2024-04-25T00:13:20.447196+0000","last_unstale":"2024-04-25T00:13:21.452392+0000","last_undegraded":"2024-04-25T00:13:21.452392+0000","last_fullsized":"2024-04-25T00:13:21.452392+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:37.933791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831229+0000","last_change":"2024-04-25T00:13:20.459954+0000","last_active":"2024-04-25T00:13:21.831229+0000","last_peered":"2024-04-25T00:13:21.831229+0000","last_clean":"2024-04-25T00:13:21.831229+0000","last_became_active":"2024-04-25T00:13:20.459656+0000","last_became_peered":"2024-04-25T00:13:20.459656+0000","last_unstale":"2024-04-25T00:13:21.831229+0000","last_undegraded":"2024-04-25T00:13:21.831229+0000","last_fullsized":"2024-04-25T00:13:21.831229+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:05.225481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.452041+0000","last_change":"2024-04-25T00:13:20.470475+0000","last_active":"2024-04-25T00:13:21.452041+0000","last_peered":"2024-04-25T00:13:21.452041+0000","last_clean":"2024-04-25T00:13:21.452041+0000","last_became_active":"2024-04-25T00:13:20.468275+0000","last_became_peered":"2024-04-25T00:13:20.468275+0000","last_unstale":"2024-04-25T00:13:21.452041+0000","last_undegraded":"2024-04-25T00:13:21.452041+0000","last_fullsized":"2024-04-25T00:13:21.452041+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:48.397716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.452002+0000","last_change":"2024-04-25T00:13:20.468498+0000","last_active":"2024-04-25T00:13:21.452002+0000","last_peered":"2024-04-25T00:13:21.452002+0000","last_clean":"2024-04-25T00:13:21.452002+0000","last_became_active":"2024-04-25T00:13:20.468178+0000","last_became_peered":"2024-04-25T00:13:20.468178+0000","last_unstale":"2024-04-25T00:13:21.452002+0000","last_undegraded":"2024-04-25T00:13:21.452002+0000","last_fullsized":"2024-04-25T00:13:21.452002+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:43.994597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.449563+0000","last_change":"2024-04-25T00:13:20.466616+0000","last_active":"2024-04-25T00:13:21.449563+0000","last_peered":"2024-04-25T00:13:21.449563+0000","last_clean":"2024-04-25T00:13:21.449563+0000","last_became_active":"2024-04-25T00:13:20.466369+0000","last_became_peered":"2024-04-25T00:13:20.466369+0000","last_unstale":"2024-04-25T00:13:21.449563+0000","last_undegraded":"2024-04-25T00:13:21.449563+0000","last_fullsized":"2024-04-25T00:13:21.449563+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:05.225481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831538+0000","last_change":"2024-04-25T00:13:20.461227+0000","last_active":"2024-04-25T00:13:21.831538+0000","last_peered":"2024-04-25T00:13:21.831538+0000","last_clean":"2024-04-25T00:13:21.831538+0000","last_became_active":"2024-04-25T00:13:20.460908+0000","last_became_peered":"2024-04-25T00:13:20.460908+0000","last_unstale":"2024-04-25T00:13:21.831538+0000","last_undegraded":"2024-04-25T00:13:21.831538+0000","last_fullsized":"2024-04-25T00:13:21.831538+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:19.316342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831331+0000","last_change":"2024-04-25T00:13:20.461719+0000","last_active":"2024-04-25T00:13:21.831331+0000","last_peered":"2024-04-25T00:13:21.831331+0000","last_clean":"2024-04-25T00:13:21.831331+0000","last_became_active":"2024-04-25T00:13:20.461429+0000","last_became_peered":"2024-04-25T00:13:20.461429+0000","last_unstale":"2024-04-25T00:13:21.831331+0000","last_undegraded":"2024-04-25T00:13:21.831331+0000","last_fullsized":"2024-04-25T00:13:21.831331+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:17.471179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450481+0000","last_change":"2024-04-25T00:13:20.459654+0000","last_active":"2024-04-25T00:13:21.450481+0000","last_peered":"2024-04-25T00:13:21.450481+0000","last_clean":"2024-04-25T00:13:21.450481+0000","last_became_active":"2024-04-25T00:13:20.459403+0000","last_became_peered":"2024-04-25T00:13:20.459403+0000","last_unstale":"2024-04-25T00:13:21.450481+0000","last_undegraded":"2024-04-25T00:13:21.450481+0000","last_fullsized":"2024-04-25T00:13:21.450481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:47.867660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831903+0000","last_change":"2024-04-25T00:13:20.458637+0000","last_active":"2024-04-25T00:13:21.831903+0000","last_peered":"2024-04-25T00:13:21.831903+0000","last_clean":"2024-04-25T00:13:21.831903+0000","last_became_active":"2024-04-25T00:13:20.458309+0000","last_became_peered":"2024-04-25T00:13:20.458309+0000","last_unstale":"2024-04-25T00:13:21.831903+0000","last_undegraded":"2024-04-25T00:13:21.831903+0000","last_fullsized":"2024-04-25T00:13:21.831903+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:14.041295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450845+0000","last_change":"2024-04-25T00:13:20.470516+0000","last_active":"2024-04-25T00:13:21.450845+0000","last_peered":"2024-04-25T00:13:21.450845+0000","last_clean":"2024-04-25T00:13:21.450845+0000","last_became_active":"2024-04-25T00:13:20.468226+0000","last_became_peered":"2024-04-25T00:13:20.468226+0000","last_unstale":"2024-04-25T00:13:21.450845+0000","last_undegraded":"2024-04-25T00:13:21.450845+0000","last_fullsized":"2024-04-25T00:13:21.450845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:48.959878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831000+0000","last_change":"2024-04-25T00:13:20.469627+0000","last_active":"2024-04-25T00:13:21.831000+0000","last_peered":"2024-04-25T00:13:21.831000+0000","last_clean":"2024-04-25T00:13:21.831000+0000","last_became_active":"2024-04-25T00:13:20.469209+0000","last_became_peered":"2024-04-25T00:13:20.469209+0000","last_unstale":"2024-04-25T00:13:21.831000+0000","last_undegraded":"2024-04-25T00:13:21.831000+0000","last_fullsized":"2024-04-25T00:13:21.831000+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:19.685495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.449570+0000","last_change":"2024-04-25T00:13:20.454313+0000","last_active":"2024-04-25T00:13:21.449570+0000","last_peered":"2024-04-25T00:13:21.449570+0000","last_clean":"2024-04-25T00:13:21.449570+0000","last_became_active":"2024-04-25T00:13:20.453873+0000","last_became_peered":"2024-04-25T00:13:20.453873+0000","last_unstale":"2024-04-25T00:13:21.449570+0000","last_undegraded":"2024-04-25T00:13:21.449570+0000","last_fullsized":"2024-04-25T00:13:21.449570+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:04.925403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.485755+0000","last_change":"2024-04-25T00:13:20.451386+0000","last_active":"2024-04-25T00:13:21.485755+0000","last_peered":"2024-04-25T00:13:21.485755+0000","last_clean":"2024-04-25T00:13:21.485755+0000","last_became_active":"2024-04-25T00:13:20.451079+0000","last_became_peered":"2024-04-25T00:13:20.451079+0000","last_unstale":"2024-04-25T00:13:21.485755+0000","last_undegraded":"2024-04-25T00:13:21.485755+0000","last_fullsized":"2024-04-25T00:13:21.485755+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:05.225481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451340+0000","last_change":"2024-04-25T00:13:20.450838+0000","last_active":"2024-04-25T00:13:21.451340+0000","last_peered":"2024-04-25T00:13:21.451340+0000","last_clean":"2024-04-25T00:13:21.451340+0000","last_became_active":"2024-04-25T00:13:20.449009+0000","last_became_peered":"2024-04-25T00:13:20.449009+0000","last_unstale":"2024-04-25T00:13:21.451340+0000","last_undegraded":"2024-04-25T00:13:21.451340+0000","last_fullsized":"2024-04-25T00:13:21.451340+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:08.862085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"59'4","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.832246+0000","last_change":"2024-04-25T00:13:19.445153+0000","last_active":"2024-04-25T00:13:21.832246+0000","last_peered":"2024-04-25T00:13:21.832246+0000","last_clean":"2024-04-25T00:13:21.832246+0000","last_became_active":"2024-04-25T00:13:19.444845+0000","last_became_peered":"2024-04-25T00:13:19.444845+0000","last_unstale":"2024-04-25T00:13:21.832246+0000","last_undegraded":"2024-04-25T00:13:21.832246+0000","last_fullsized":"2024-04-25T00:13:21.832246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:06.325314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450678+0000","last_change":"2024-04-25T00:13:20.448297+0000","last_active":"2024-04-25T00:13:21.450678+0000","last_peered":"2024-04-25T00:13:21.450678+0000","last_clean":"2024-04-25T00:13:21.450678+0000","last_became_active":"2024-04-25T00:13:20.447573+0000","last_became_peered":"2024-04-25T00:13:20.447573+0000","last_unstale":"2024-04-25T00:13:21.450678+0000","last_undegraded":"2024-04-25T00:13:21.450678+0000","last_fullsized":"2024-04-25T00:13:21.450678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:23.589981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.486246+0000","last_change":"2024-04-25T00:13:19.439199+0000","last_active":"2024-04-25T00:13:21.486246+0000","last_peered":"2024-04-25T00:13:21.486246+0000","last_clean":"2024-04-25T00:13:21.486246+0000","last_became_active":"2024-04-25T00:13:19.438895+0000","last_became_peered":"2024-04-25T00:13:19.438895+0000","last_unstale":"2024-04-25T00:13:21.486246+0000","last_undegraded":"2024-04-25T00:13:21.486246+0000","last_fullsized":"2024-04-25T00:13:21.486246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:03.110378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.486189+0000","last_change":"2024-04-25T00:13:20.457145+0000","last_active":"2024-04-25T00:13:21.486189+0000","last_peered":"2024-04-25T00:13:21.486189+0000","last_clean":"2024-04-25T00:13:21.486189+0000","last_became_active":"2024-04-25T00:13:20.451174+0000","last_became_peered":"2024-04-25T00:13:20.451174+0000","last_unstale":"2024-04-25T00:13:21.486189+0000","last_undegraded":"2024-04-25T00:13:21.486189+0000","last_fullsized":"2024-04-25T00:13:21.486189+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:19.316342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.486119+0000","last_change":"2024-04-25T00:13:19.442726+0000","last_active":"2024-04-25T00:13:21.486119+0000","last_peered":"2024-04-25T00:13:21.486119+0000","last_clean":"2024-04-25T00:13:21.486119+0000","last_became_active":"2024-04-25T00:13:19.442433+0000","last_became_peered":"2024-04-25T00:13:19.442433+0000","last_unstale":"2024-04-25T00:13:21.486119+0000","last_undegraded":"2024-04-25T00:13:21.486119+0000","last_fullsized":"2024-04-25T00:13:21.486119+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:03.088525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451645+0000","last_change":"2024-04-25T00:13:20.473151+0000","last_active":"2024-04-25T00:13:21.451645+0000","last_peered":"2024-04-25T00:13:21.451645+0000","last_clean":"2024-04-25T00:13:21.451645+0000","last_became_active":"2024-04-25T00:13:20.472868+0000","last_became_peered":"2024-04-25T00:13:20.472868+0000","last_unstale":"2024-04-25T00:13:21.451645+0000","last_undegraded":"2024-04-25T00:13:21.451645+0000","last_fullsized":"2024-04-25T00:13:21.451645+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:08.765959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.485998+0000","last_change":"2024-04-25T00:13:19.438204+0000","last_active":"2024-04-25T00:13:21.485998+0000","last_peered":"2024-04-25T00:13:21.485998+0000","last_clean":"2024-04-25T00:13:21.485998+0000","last_became_active":"2024-04-25T00:13:19.437845+0000","last_became_peered":"2024-04-25T00:13:19.437845+0000","last_unstale":"2024-04-25T00:13:21.485998+0000","last_undegraded":"2024-04-25T00:13:21.485998+0000","last_fullsized":"2024-04-25T00:13:21.485998+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:36.678931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831333+0000","last_change":"2024-04-25T00:13:20.447234+0000","last_active":"2024-04-25T00:13:21.831333+0000","last_peered":"2024-04-25T00:13:21.831333+0000","last_clean":"2024-04-25T00:13:21.831333+0000","last_became_active":"2024-04-25T00:13:20.446880+0000","last_became_peered":"2024-04-25T00:13:20.446880+0000","last_unstale":"2024-04-25T00:13:21.831333+0000","last_undegraded":"2024-04-25T00:13:21.831333+0000","last_fullsized":"2024-04-25T00:13:21.831333+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:25.250573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.452205+0000","last_change":"2024-04-25T00:13:19.431189+0000","last_active":"2024-04-25T00:13:21.452205+0000","last_peered":"2024-04-25T00:13:21.452205+0000","last_clean":"2024-04-25T00:13:21.452205+0000","last_became_active":"2024-04-25T00:13:19.430933+0000","last_became_peered":"2024-04-25T00:13:19.430933+0000","last_unstale":"2024-04-25T00:13:21.452205+0000","last_undegraded":"2024-04-25T00:13:21.452205+0000","last_fullsized":"2024-04-25T00:13:21.452205+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:47.957588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831686+0000","last_change":"2024-04-25T00:13:20.442697+0000","last_active":"2024-04-25T00:13:21.831686+0000","last_peered":"2024-04-25T00:13:21.831686+0000","last_clean":"2024-04-25T00:13:21.831686+0000","last_became_active":"2024-04-25T00:13:20.442403+0000","last_became_peered":"2024-04-25T00:13:20.442403+0000","last_unstale":"2024-04-25T00:13:21.831686+0000","last_undegraded":"2024-04-25T00:13:21.831686+0000","last_fullsized":"2024-04-25T00:13:21.831686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:21.024433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451996+0000","last_change":"2024-04-25T00:13:19.441394+0000","last_active":"2024-04-25T00:13:21.451996+0000","last_peered":"2024-04-25T00:13:21.451996+0000","last_clean":"2024-04-25T00:13:21.451996+0000","last_became_active":"2024-04-25T00:13:19.438688+0000","last_became_peered":"2024-04-25T00:13:19.438688+0000","last_unstale":"2024-04-25T00:13:21.451996+0000","last_undegraded":"2024-04-25T00:13:21.451996+0000","last_fullsized":"2024-04-25T00:13:21.451996+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:04.223191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451110+0000","last_change":"2024-04-25T00:13:20.446560+0000","last_active":"2024-04-25T00:13:21.451110+0000","last_peered":"2024-04-25T00:13:21.451110+0000","last_clean":"2024-04-25T00:13:21.451110+0000","last_became_active":"2024-04-25T00:13:20.446079+0000","last_became_peered":"2024-04-25T00:13:20.446079+0000","last_unstale":"2024-04-25T00:13:21.451110+0000","last_undegraded":"2024-04-25T00:13:21.451110+0000","last_fullsized":"2024-04-25T00:13:21.451110+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:21.257610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.452015+0000","last_change":"2024-04-25T00:13:19.433617+0000","last_active":"2024-04-25T00:13:21.452015+0000","last_peered":"2024-04-25T00:13:21.452015+0000","last_clean":"2024-04-25T00:13:21.452015+0000","last_became_active":"2024-04-25T00:13:19.433063+0000","last_became_peered":"2024-04-25T00:13:19.433063+0000","last_unstale":"2024-04-25T00:13:21.452015+0000","last_undegraded":"2024-04-25T00:13:21.452015+0000","last_fullsized":"2024-04-25T00:13:21.452015+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:51.776987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.452241+0000","last_change":"2024-04-25T00:13:20.449964+0000","last_active":"2024-04-25T00:13:21.452241+0000","last_peered":"2024-04-25T00:13:21.452241+0000","last_clean":"2024-04-25T00:13:21.452241+0000","last_became_active":"2024-04-25T00:13:20.449681+0000","last_became_peered":"2024-04-25T00:13:20.449681+0000","last_unstale":"2024-04-25T00:13:21.452241+0000","last_undegraded":"2024-04-25T00:13:21.452241+0000","last_fullsized":"2024-04-25T00:13:21.452241+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:36.222747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450215+0000","last_change":"2024-04-25T00:13:19.431998+0000","last_active":"2024-04-25T00:13:21.450215+0000","last_peered":"2024-04-25T00:13:21.450215+0000","last_clean":"2024-04-25T00:13:21.450215+0000","last_became_active":"2024-04-25T00:13:19.431728+0000","last_became_peered":"2024-04-25T00:13:19.431728+0000","last_unstale":"2024-04-25T00:13:21.450215+0000","last_undegraded":"2024-04-25T00:13:21.450215+0000","last_fullsized":"2024-04-25T00:13:21.450215+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:31.858631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.832055+0000","last_change":"2024-04-25T00:13:20.455176+0000","last_active":"2024-04-25T00:13:21.832055+0000","last_peered":"2024-04-25T00:13:21.832055+0000","last_clean":"2024-04-25T00:13:21.832055+0000","last_became_active":"2024-04-25T00:13:20.454878+0000","last_became_peered":"2024-04-25T00:13:20.454878+0000","last_unstale":"2024-04-25T00:13:21.832055+0000","last_undegraded":"2024-04-25T00:13:21.832055+0000","last_fullsized":"2024-04-25T00:13:21.832055+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:18.553393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451275+0000","last_change":"2024-04-25T00:13:19.440887+0000","last_active":"2024-04-25T00:13:21.451275+0000","last_peered":"2024-04-25T00:13:21.451275+0000","last_clean":"2024-04-25T00:13:21.451275+0000","last_became_active":"2024-04-25T00:13:19.437895+0000","last_became_peered":"2024-04-25T00:13:19.437895+0000","last_unstale":"2024-04-25T00:13:21.451275+0000","last_undegraded":"2024-04-25T00:13:21.451275+0000","last_fullsized":"2024-04-25T00:13:21.451275+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:03.088525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.452402+0000","last_change":"2024-04-25T00:13:20.454205+0000","last_active":"2024-04-25T00:13:21.452402+0000","last_peered":"2024-04-25T00:13:21.452402+0000","last_clean":"2024-04-25T00:13:21.452402+0000","last_became_active":"2024-04-25T00:13:20.453666+0000","last_became_peered":"2024-04-25T00:13:20.453666+0000","last_unstale":"2024-04-25T00:13:21.452402+0000","last_undegraded":"2024-04-25T00:13:21.452402+0000","last_fullsized":"2024-04-25T00:13:21.452402+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:47.538641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450979+0000","last_change":"2024-04-25T00:13:19.437705+0000","last_active":"2024-04-25T00:13:21.450979+0000","last_peered":"2024-04-25T00:13:21.450979+0000","last_clean":"2024-04-25T00:13:21.450979+0000","last_became_active":"2024-04-25T00:13:19.437390+0000","last_became_peered":"2024-04-25T00:13:19.437390+0000","last_unstale":"2024-04-25T00:13:21.450979+0000","last_undegraded":"2024-04-25T00:13:21.450979+0000","last_fullsized":"2024-04-25T00:13:21.450979+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:22.922236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451539+0000","last_change":"2024-04-25T00:13:20.453020+0000","last_active":"2024-04-25T00:13:21.451539+0000","last_peered":"2024-04-25T00:13:21.451539+0000","last_clean":"2024-04-25T00:13:21.451539+0000","last_became_active":"2024-04-25T00:13:20.452755+0000","last_became_peered":"2024-04-25T00:13:20.452755+0000","last_unstale":"2024-04-25T00:13:21.451539+0000","last_undegraded":"2024-04-25T00:13:21.451539+0000","last_fullsized":"2024-04-25T00:13:21.451539+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:48.275922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"59'3","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450192+0000","last_change":"2024-04-25T00:13:19.438746+0000","last_active":"2024-04-25T00:13:21.450192+0000","last_peered":"2024-04-25T00:13:21.450192+0000","last_clean":"2024-04-25T00:13:21.450192+0000","last_became_active":"2024-04-25T00:13:19.437749+0000","last_became_peered":"2024-04-25T00:13:19.437749+0000","last_unstale":"2024-04-25T00:13:21.450192+0000","last_undegraded":"2024-04-25T00:13:21.450192+0000","last_fullsized":"2024-04-25T00:13:21.450192+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:17.551103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451115+0000","last_change":"2024-04-25T00:13:20.449577+0000","last_active":"2024-04-25T00:13:21.451115+0000","last_peered":"2024-04-25T00:13:21.451115+0000","last_clean":"2024-04-25T00:13:21.451115+0000","last_became_active":"2024-04-25T00:13:20.449181+0000","last_became_peered":"2024-04-25T00:13:20.449181+0000","last_unstale":"2024-04-25T00:13:21.451115+0000","last_undegraded":"2024-04-25T00:13:21.451115+0000","last_fullsized":"2024-04-25T00:13:21.451115+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:21.257610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"59'6","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.832101+0000","last_change":"2024-04-25T00:13:19.445212+0000","last_active":"2024-04-25T00:13:21.832101+0000","last_peered":"2024-04-25T00:13:21.832101+0000","last_clean":"2024-04-25T00:13:21.832101+0000","last_became_active":"2024-04-25T00:13:19.444953+0000","last_became_peered":"2024-04-25T00:13:19.444953+0000","last_unstale":"2024-04-25T00:13:21.832101+0000","last_undegraded":"2024-04-25T00:13:21.832101+0000","last_fullsized":"2024-04-25T00:13:21.832101+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:22.587691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.449277+0000","last_change":"2024-04-25T00:13:20.471308+0000","last_active":"2024-04-25T00:13:21.449277+0000","last_peered":"2024-04-25T00:13:21.449277+0000","last_clean":"2024-04-25T00:13:21.449277+0000","last_became_active":"2024-04-25T00:13:20.470159+0000","last_became_peered":"2024-04-25T00:13:20.470159+0000","last_unstale":"2024-04-25T00:13:21.449277+0000","last_undegraded":"2024-04-25T00:13:21.449277+0000","last_fullsized":"2024-04-25T00:13:21.449277+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:40.694132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450933+0000","last_change":"2024-04-25T00:13:19.439464+0000","last_active":"2024-04-25T00:13:21.450933+0000","last_peered":"2024-04-25T00:13:21.450933+0000","last_clean":"2024-04-25T00:13:21.450933+0000","last_became_active":"2024-04-25T00:13:19.439143+0000","last_became_peered":"2024-04-25T00:13:19.439143+0000","last_unstale":"2024-04-25T00:13:21.450933+0000","last_undegraded":"2024-04-25T00:13:21.450933+0000","last_fullsized":"2024-04-25T00:13:21.450933+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:20.022144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831007+0000","last_change":"2024-04-25T00:13:20.451529+0000","last_active":"2024-04-25T00:13:21.831007+0000","last_peered":"2024-04-25T00:13:21.831007+0000","last_clean":"2024-04-25T00:13:21.831007+0000","last_became_active":"2024-04-25T00:13:20.451244+0000","last_became_peered":"2024-04-25T00:13:20.451244+0000","last_unstale":"2024-04-25T00:13:21.831007+0000","last_undegraded":"2024-04-25T00:13:21.831007+0000","last_fullsized":"2024-04-25T00:13:21.831007+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:58.962905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.830933+0000","last_change":"2024-04-25T00:13:19.443972+0000","last_active":"2024-04-25T00:13:21.830933+0000","last_peered":"2024-04-25T00:13:21.830933+0000","last_clean":"2024-04-25T00:13:21.830933+0000","last_became_active":"2024-04-25T00:13:19.443634+0000","last_became_peered":"2024-04-25T00:13:19.443634+0000","last_unstale":"2024-04-25T00:13:21.830933+0000","last_undegraded":"2024-04-25T00:13:21.830933+0000","last_fullsized":"2024-04-25T00:13:21.830933+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:42.830716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451503+0000","last_change":"2024-04-25T00:13:20.452366+0000","last_active":"2024-04-25T00:13:21.451503+0000","last_peered":"2024-04-25T00:13:21.451503+0000","last_clean":"2024-04-25T00:13:21.451503+0000","last_became_active":"2024-04-25T00:13:20.452111+0000","last_became_peered":"2024-04-25T00:13:20.452111+0000","last_unstale":"2024-04-25T00:13:21.451503+0000","last_undegraded":"2024-04-25T00:13:21.451503+0000","last_fullsized":"2024-04-25T00:13:21.451503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:43.833006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.449862+0000","last_change":"2024-04-25T00:13:19.430607+0000","last_active":"2024-04-25T00:13:21.449862+0000","last_peered":"2024-04-25T00:13:21.449862+0000","last_clean":"2024-04-25T00:13:21.449862+0000","last_became_active":"2024-04-25T00:13:19.430240+0000","last_became_peered":"2024-04-25T00:13:19.430240+0000","last_unstale":"2024-04-25T00:13:21.449862+0000","last_undegraded":"2024-04-25T00:13:21.449862+0000","last_fullsized":"2024-04-25T00:13:21.449862+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:03.224084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.832052+0000","last_change":"2024-04-25T00:13:20.469354+0000","last_active":"2024-04-25T00:13:21.832052+0000","last_peered":"2024-04-25T00:13:21.832052+0000","last_clean":"2024-04-25T00:13:21.832052+0000","last_became_active":"2024-04-25T00:13:20.469090+0000","last_became_peered":"2024-04-25T00:13:20.469090+0000","last_unstale":"2024-04-25T00:13:21.832052+0000","last_undegraded":"2024-04-25T00:13:21.832052+0000","last_fullsized":"2024-04-25T00:13:21.832052+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:13.351642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"59'8","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.449412+0000","last_change":"2024-04-25T00:13:19.438379+0000","last_active":"2024-04-25T00:13:21.449412+0000","last_peered":"2024-04-25T00:13:21.449412+0000","last_clean":"2024-04-25T00:13:21.449412+0000","last_became_active":"2024-04-25T00:13:19.438093+0000","last_became_peered":"2024-04-25T00:13:19.438093+0000","last_unstale":"2024-04-25T00:13:21.449412+0000","last_undegraded":"2024-04-25T00:13:21.449412+0000","last_fullsized":"2024-04-25T00:13:21.449412+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:43.485877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450737+0000","last_change":"2024-04-25T00:13:20.470591+0000","last_active":"2024-04-25T00:13:21.450737+0000","last_peered":"2024-04-25T00:13:21.450737+0000","last_clean":"2024-04-25T00:13:21.450737+0000","last_became_active":"2024-04-25T00:13:20.470305+0000","last_became_peered":"2024-04-25T00:13:20.470305+0000","last_unstale":"2024-04-25T00:13:21.450737+0000","last_undegraded":"2024-04-25T00:13:21.450737+0000","last_fullsized":"2024-04-25T00:13:21.450737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:19.316342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831411+0000","last_change":"2024-04-25T00:13:19.440461+0000","last_active":"2024-04-25T00:13:21.831411+0000","last_peered":"2024-04-25T00:13:21.831411+0000","last_clean":"2024-04-25T00:13:21.831411+0000","last_became_active":"2024-04-25T00:13:19.440197+0000","last_became_peered":"2024-04-25T00:13:19.440197+0000","last_unstale":"2024-04-25T00:13:21.831411+0000","last_undegraded":"2024-04-25T00:13:21.831411+0000","last_fullsized":"2024-04-25T00:13:21.831411+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:12.349352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451507+0000","last_change":"2024-04-25T00:13:20.457444+0000","last_active":"2024-04-25T00:13:21.451507+0000","last_peered":"2024-04-25T00:13:21.451507+0000","last_clean":"2024-04-25T00:13:21.451507+0000","last_became_active":"2024-04-25T00:13:20.456808+0000","last_became_peered":"2024-04-25T00:13:20.456808+0000","last_unstale":"2024-04-25T00:13:21.451507+0000","last_undegraded":"2024-04-25T00:13:21.451507+0000","last_fullsized":"2024-04-25T00:13:21.451507+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:04.415357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450436+0000","last_change":"2024-04-25T00:13:19.438945+0000","last_active":"2024-04-25T00:13:21.450436+0000","last_peered":"2024-04-25T00:13:21.450436+0000","last_clean":"2024-04-25T00:13:21.450436+0000","last_became_active":"2024-04-25T00:13:19.438052+0000","last_became_peered":"2024-04-25T00:13:19.438052+0000","last_unstale":"2024-04-25T00:13:21.450436+0000","last_undegraded":"2024-04-25T00:13:21.450436+0000","last_fullsized":"2024-04-25T00:13:21.450436+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:19:49.661925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450768+0000","last_change":"2024-04-25T00:13:20.464034+0000","last_active":"2024-04-25T00:13:21.450768+0000","last_peered":"2024-04-25T00:13:21.450768+0000","last_clean":"2024-04-25T00:13:21.450768+0000","last_became_active":"2024-04-25T00:13:20.461916+0000","last_became_peered":"2024-04-25T00:13:20.461916+0000","last_unstale":"2024-04-25T00:13:21.450768+0000","last_undegraded":"2024-04-25T00:13:21.450768+0000","last_fullsized":"2024-04-25T00:13:21.450768+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:06.966244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450288+0000","last_change":"2024-04-25T00:13:19.440459+0000","last_active":"2024-04-25T00:13:21.450288+0000","last_peered":"2024-04-25T00:13:21.450288+0000","last_clean":"2024-04-25T00:13:21.450288+0000","last_became_active":"2024-04-25T00:13:19.438442+0000","last_became_peered":"2024-04-25T00:13:19.438442+0000","last_unstale":"2024-04-25T00:13:21.450288+0000","last_undegraded":"2024-04-25T00:13:21.450288+0000","last_fullsized":"2024-04-25T00:13:21.450288+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:29.987108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451278+0000","last_change":"2024-04-25T00:13:20.448258+0000","last_active":"2024-04-25T00:13:21.451278+0000","last_peered":"2024-04-25T00:13:21.451278+0000","last_clean":"2024-04-25T00:13:21.451278+0000","last_became_active":"2024-04-25T00:13:20.447997+0000","last_became_peered":"2024-04-25T00:13:20.447997+0000","last_unstale":"2024-04-25T00:13:21.451278+0000","last_undegraded":"2024-04-25T00:13:21.451278+0000","last_fullsized":"2024-04-25T00:13:21.451278+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:34.804667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.452231+0000","last_change":"2024-04-25T00:13:19.440785+0000","last_active":"2024-04-25T00:13:21.452231+0000","last_peered":"2024-04-25T00:13:21.452231+0000","last_clean":"2024-04-25T00:13:21.452231+0000","last_became_active":"2024-04-25T00:13:19.440512+0000","last_became_peered":"2024-04-25T00:13:19.440512+0000","last_unstale":"2024-04-25T00:13:21.452231+0000","last_undegraded":"2024-04-25T00:13:21.452231+0000","last_fullsized":"2024-04-25T00:13:21.452231+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:02:32.565349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.485827+0000","last_change":"2024-04-25T00:13:20.447443+0000","last_active":"2024-04-25T00:13:21.485827+0000","last_peered":"2024-04-25T00:13:21.485827+0000","last_clean":"2024-04-25T00:13:21.485827+0000","last_became_active":"2024-04-25T00:13:20.447212+0000","last_became_peered":"2024-04-25T00:13:20.447212+0000","last_unstale":"2024-04-25T00:13:21.485827+0000","last_undegraded":"2024-04-25T00:13:21.485827+0000","last_fullsized":"2024-04-25T00:13:21.485827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:17.471179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.486088+0000","last_change":"2024-04-25T00:13:19.443427+0000","last_active":"2024-04-25T00:13:21.486088+0000","last_peered":"2024-04-25T00:13:21.486088+0000","last_clean":"2024-04-25T00:13:21.486088+0000","last_became_active":"2024-04-25T00:13:19.443147+0000","last_became_peered":"2024-04-25T00:13:19.443147+0000","last_unstale":"2024-04-25T00:13:21.486088+0000","last_undegraded":"2024-04-25T00:13:21.486088+0000","last_fullsized":"2024-04-25T00:13:21.486088+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:53.668553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831209+0000","last_change":"2024-04-25T00:13:20.461335+0000","last_active":"2024-04-25T00:13:21.831209+0000","last_peered":"2024-04-25T00:13:21.831209+0000","last_clean":"2024-04-25T00:13:21.831209+0000","last_became_active":"2024-04-25T00:13:20.460852+0000","last_became_peered":"2024-04-25T00:13:20.460852+0000","last_unstale":"2024-04-25T00:13:21.831209+0000","last_undegraded":"2024-04-25T00:13:21.831209+0000","last_fullsized":"2024-04-25T00:13:21.831209+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:29.390125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831084+0000","last_change":"2024-04-25T00:13:19.441302+0000","last_active":"2024-04-25T00:13:21.831084+0000","last_peered":"2024-04-25T00:13:21.831084+0000","last_clean":"2024-04-25T00:13:21.831084+0000","last_became_active":"2024-04-25T00:13:19.440951+0000","last_became_peered":"2024-04-25T00:13:19.440951+0000","last_unstale":"2024-04-25T00:13:21.831084+0000","last_undegraded":"2024-04-25T00:13:21.831084+0000","last_fullsized":"2024-04-25T00:13:21.831084+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:47.273633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451294+0000","last_change":"2024-04-25T00:13:20.446328+0000","last_active":"2024-04-25T00:13:21.451294+0000","last_peered":"2024-04-25T00:13:21.451294+0000","last_clean":"2024-04-25T00:13:21.451294+0000","last_became_active":"2024-04-25T00:13:20.444953+0000","last_became_peered":"2024-04-25T00:13:20.444953+0000","last_unstale":"2024-04-25T00:13:21.451294+0000","last_undegraded":"2024-04-25T00:13:21.451294+0000","last_fullsized":"2024-04-25T00:13:21.451294+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:38.587633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.485748+0000","last_change":"2024-04-25T00:13:19.438811+0000","last_active":"2024-04-25T00:13:21.485748+0000","last_peered":"2024-04-25T00:13:21.485748+0000","last_clean":"2024-04-25T00:13:21.485748+0000","last_became_active":"2024-04-25T00:13:19.438527+0000","last_became_peered":"2024-04-25T00:13:19.438527+0000","last_unstale":"2024-04-25T00:13:21.485748+0000","last_undegraded":"2024-04-25T00:13:21.485748+0000","last_fullsized":"2024-04-25T00:13:21.485748+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:19.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.486054+0000","last_change":"2024-04-25T00:13:20.448151+0000","last_active":"2024-04-25T00:13:21.486054+0000","last_peered":"2024-04-25T00:13:21.486054+0000","last_clean":"2024-04-25T00:13:21.486054+0000","last_became_active":"2024-04-25T00:13:20.447877+0000","last_became_peered":"2024-04-25T00:13:20.447877+0000","last_unstale":"2024-04-25T00:13:21.486054+0000","last_undegraded":"2024-04-25T00:13:21.486054+0000","last_fullsized":"2024-04-25T00:13:21.486054+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:52.779277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831580+0000","last_change":"2024-04-25T00:13:19.442162+0000","last_active":"2024-04-25T00:13:21.831580+0000","last_peered":"2024-04-25T00:13:21.831580+0000","last_clean":"2024-04-25T00:13:21.831580+0000","last_became_active":"2024-04-25T00:13:19.441874+0000","last_became_peered":"2024-04-25T00:13:19.441874+0000","last_unstale":"2024-04-25T00:13:21.831580+0000","last_undegraded":"2024-04-25T00:13:21.831580+0000","last_fullsized":"2024-04-25T00:13:21.831580+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:17.551103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831544+0000","last_change":"2024-04-25T00:13:20.458145+0000","last_active":"2024-04-25T00:13:21.831544+0000","last_peered":"2024-04-25T00:13:21.831544+0000","last_clean":"2024-04-25T00:13:21.831544+0000","last_became_active":"2024-04-25T00:13:20.457881+0000","last_became_peered":"2024-04-25T00:13:20.457881+0000","last_unstale":"2024-04-25T00:13:21.831544+0000","last_undegraded":"2024-04-25T00:13:21.831544+0000","last_fullsized":"2024-04-25T00:13:21.831544+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:03.413942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831483+0000","last_change":"2024-04-25T00:13:19.443257+0000","last_active":"2024-04-25T00:13:21.831483+0000","last_peered":"2024-04-25T00:13:21.831483+0000","last_clean":"2024-04-25T00:13:21.831483+0000","last_became_active":"2024-04-25T00:13:19.442853+0000","last_became_peered":"2024-04-25T00:13:19.442853+0000","last_unstale":"2024-04-25T00:13:21.831483+0000","last_undegraded":"2024-04-25T00:13:21.831483+0000","last_fullsized":"2024-04-25T00:13:21.831483+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:20.022144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831380+0000","last_change":"2024-04-25T00:13:20.457769+0000","last_active":"2024-04-25T00:13:21.831380+0000","last_peered":"2024-04-25T00:13:21.831380+0000","last_clean":"2024-04-25T00:13:21.831380+0000","last_became_active":"2024-04-25T00:13:20.456926+0000","last_became_peered":"2024-04-25T00:13:20.456926+0000","last_unstale":"2024-04-25T00:13:21.831380+0000","last_undegraded":"2024-04-25T00:13:21.831380+0000","last_fullsized":"2024-04-25T00:13:21.831380+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:02:33.567639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.486021+0000","last_change":"2024-04-25T00:13:19.444471+0000","last_active":"2024-04-25T00:13:21.486021+0000","last_peered":"2024-04-25T00:13:21.486021+0000","last_clean":"2024-04-25T00:13:21.486021+0000","last_became_active":"2024-04-25T00:13:19.444197+0000","last_became_peered":"2024-04-25T00:13:19.444197+0000","last_unstale":"2024-04-25T00:13:21.486021+0000","last_undegraded":"2024-04-25T00:13:21.486021+0000","last_fullsized":"2024-04-25T00:13:21.486021+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:03.224084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451460+0000","last_change":"2024-04-25T00:13:20.447382+0000","last_active":"2024-04-25T00:13:21.451460+0000","last_peered":"2024-04-25T00:13:21.451460+0000","last_clean":"2024-04-25T00:13:21.451460+0000","last_became_active":"2024-04-25T00:13:20.447048+0000","last_became_peered":"2024-04-25T00:13:20.447048+0000","last_unstale":"2024-04-25T00:13:21.451460+0000","last_undegraded":"2024-04-25T00:13:21.451460+0000","last_fullsized":"2024-04-25T00:13:21.451460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:44:22.589107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.832170+0000","last_change":"2024-04-25T00:13:19.445103+0000","last_active":"2024-04-25T00:13:21.832170+0000","last_peered":"2024-04-25T00:13:21.832170+0000","last_clean":"2024-04-25T00:13:21.832170+0000","last_became_active":"2024-04-25T00:13:19.444652+0000","last_became_peered":"2024-04-25T00:13:19.444652+0000","last_unstale":"2024-04-25T00:13:21.832170+0000","last_undegraded":"2024-04-25T00:13:21.832170+0000","last_fullsized":"2024-04-25T00:13:21.832170+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:05.963954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.450121+0000","last_change":"2024-04-25T00:13:20.449358+0000","last_active":"2024-04-25T00:13:21.450121+0000","last_peered":"2024-04-25T00:13:21.450121+0000","last_clean":"2024-04-25T00:13:21.450121+0000","last_became_active":"2024-04-25T00:13:20.446239+0000","last_became_peered":"2024-04-25T00:13:20.446239+0000","last_unstale":"2024-04-25T00:13:21.450121+0000","last_undegraded":"2024-04-25T00:13:21.450121+0000","last_fullsized":"2024-04-25T00:13:21.450121+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:06.966244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451027+0000","last_change":"2024-04-25T00:13:19.442646+0000","last_active":"2024-04-25T00:13:21.451027+0000","last_peered":"2024-04-25T00:13:21.451027+0000","last_clean":"2024-04-25T00:13:21.451027+0000","last_became_active":"2024-04-25T00:13:19.438848+0000","last_became_peered":"2024-04-25T00:13:19.438848+0000","last_unstale":"2024-04-25T00:13:21.451027+0000","last_undegraded":"2024-04-25T00:13:21.451027+0000","last_fullsized":"2024-04-25T00:13:21.451027+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:12.349352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.449720+0000","last_change":"2024-04-25T00:13:20.470884+0000","last_active":"2024-04-25T00:13:21.449720+0000","last_peered":"2024-04-25T00:13:21.449720+0000","last_clean":"2024-04-25T00:13:21.449720+0000","last_became_active":"2024-04-25T00:13:20.469621+0000","last_became_peered":"2024-04-25T00:13:20.469621+0000","last_unstale":"2024-04-25T00:13:21.449720+0000","last_undegraded":"2024-04-25T00:13:21.449720+0000","last_fullsized":"2024-04-25T00:13:21.449720+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:29.282996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.452105+0000","last_change":"2024-04-25T00:13:19.436516+0000","last_active":"2024-04-25T00:13:21.452105+0000","last_peered":"2024-04-25T00:13:21.452105+0000","last_clean":"2024-04-25T00:13:21.452105+0000","last_became_active":"2024-04-25T00:13:19.435936+0000","last_became_peered":"2024-04-25T00:13:19.435936+0000","last_unstale":"2024-04-25T00:13:21.452105+0000","last_undegraded":"2024-04-25T00:13:21.452105+0000","last_fullsized":"2024-04-25T00:13:21.452105+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:16.468889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831819+0000","last_change":"2024-04-25T00:13:20.442933+0000","last_active":"2024-04-25T00:13:21.831819+0000","last_peered":"2024-04-25T00:13:21.831819+0000","last_clean":"2024-04-25T00:13:21.831819+0000","last_became_active":"2024-04-25T00:13:20.442434+0000","last_became_peered":"2024-04-25T00:13:20.442434+0000","last_unstale":"2024-04-25T00:13:21.831819+0000","last_undegraded":"2024-04-25T00:13:21.831819+0000","last_fullsized":"2024-04-25T00:13:21.831819+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:48.275922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.452099+0000","last_change":"2024-04-25T00:13:19.427399+0000","last_active":"2024-04-25T00:13:21.452099+0000","last_peered":"2024-04-25T00:13:21.452099+0000","last_clean":"2024-04-25T00:13:21.452099+0000","last_became_active":"2024-04-25T00:13:19.427145+0000","last_became_peered":"2024-04-25T00:13:19.427145+0000","last_unstale":"2024-04-25T00:13:21.452099+0000","last_undegraded":"2024-04-25T00:13:21.452099+0000","last_fullsized":"2024-04-25T00:13:21.452099+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:18.314052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831375+0000","last_change":"2024-04-25T00:13:20.458578+0000","last_active":"2024-04-25T00:13:21.831375+0000","last_peered":"2024-04-25T00:13:21.831375+0000","last_clean":"2024-04-25T00:13:21.831375+0000","last_became_active":"2024-04-25T00:13:20.458276+0000","last_became_peered":"2024-04-25T00:13:20.458276+0000","last_unstale":"2024-04-25T00:13:21.831375+0000","last_undegraded":"2024-04-25T00:13:21.831375+0000","last_fullsized":"2024-04-25T00:13:21.831375+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:07.016922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.831688+0000","last_change":"2024-04-25T00:13:19.438635+0000","last_active":"2024-04-25T00:13:21.831688+0000","last_peered":"2024-04-25T00:13:21.831688+0000","last_clean":"2024-04-25T00:13:21.831688+0000","last_became_active":"2024-04-25T00:13:19.438328+0000","last_became_peered":"2024-04-25T00:13:19.438328+0000","last_unstale":"2024-04-25T00:13:21.831688+0000","last_undegraded":"2024-04-25T00:13:21.831688+0000","last_fullsized":"2024-04-25T00:13:21.831688+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:46.865370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451713+0000","last_change":"2024-04-25T00:13:20.450441+0000","last_active":"2024-04-25T00:13:21.451713+0000","last_peered":"2024-04-25T00:13:21.451713+0000","last_clean":"2024-04-25T00:13:21.451713+0000","last_became_active":"2024-04-25T00:13:20.447153+0000","last_became_peered":"2024-04-25T00:13:20.447153+0000","last_unstale":"2024-04-25T00:13:21.451713+0000","last_undegraded":"2024-04-25T00:13:21.451713+0000","last_fullsized":"2024-04-25T00:13:21.451713+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:19.412448+0000","last_clean_scrub_stamp":"2024-04-25T00:13:19.412448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:14.041295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.451909+0000","last_change":"2024-04-25T00:13:19.430683+0000","last_active":"2024-04-25T00:13:21.451909+0000","last_peered":"2024-04-25T00:13:21.451909+0000","last_clean":"2024-04-25T00:13:21.451909+0000","last_became_active":"2024-04-25T00:13:19.430464+0000","last_became_peered":"2024-04-25T00:13:19.430464+0000","last_unstale":"2024-04-25T00:13:21.451909+0000","last_undegraded":"2024-04-25T00:13:21.451909+0000","last_fullsized":"2024-04-25T00:13:21.451909+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:18.410158+0000","last_clean_scrub_stamp":"2024-04-25T00:13:18.410158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:28.387835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":243,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:13:21.449672+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:13:21.449672+0000","last_peered":"2024-04-25T00:13:21.449672+0000","last_clean":"2024-04-25T00:13:21.449672+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:13:21.449672+0000","last_undegraded":"2024-04-25T00:13:21.449672+0000","last_fullsized":"2024-04-25T00:13:21.449672+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28328,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343512,"statfs":{"total":96636764160,"available":96607756288,"internally_reserved":0,"allocated":1335296,"data_stored":916196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":729088,"data_stored":325086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":729088,"data_stored":325074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":737280,"data_stored":326664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":724992,"data_stored":325052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":724992,"data_stored":325052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":753664,"data_stored":327428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1318912,"data_stored":915420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:13:29.371 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:13:29.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:13:29.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.548+0000 7faa32000640 1 Processor -- start 2024-04-25T00:13:29.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.548+0000 7faa32000640 1 -- start start 2024-04-25T00:13:29.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa32000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa2c159810 0x7faa2c0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa32000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa2c06e950 0x7faa2c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa32000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faa2c15fe50 0x7faa2c15c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa32000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7faa2c06e210 con 0x7faa2c159810 2024-04-25T00:13:29.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa32000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7faa2c06da00 con 0x7faa2c15fe50 2024-04-25T00:13:29.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa32000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7faa2c0ce000 con 0x7faa2c06e950 2024-04-25T00:13:29.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa2b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa2c159810 0x7faa2c0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa2b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa2c159810 0x7faa2c0cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56862/0 (socket says 172.21.15.83:56862) 2024-04-25T00:13:29.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa2b400640 1 -- 172.21.15.83:0/1395769221 learned_addr learned my addr 172.21.15.83:0/1395769221 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:29.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa2b400640 1 -- 172.21.15.83:0/1395769221 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa2c06e950 msgr2=0x7faa2c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:29.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa2be00640 1 --2- 172.21.15.83:0/1395769221 >> v2:172.21.15.86:3300/0 conn(0x7faa2c15fe50 0x7faa2c15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa2aa00640 1 --2- 172.21.15.83:0/1395769221 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa2c06e950 0x7faa2c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa2b400640 1 --2- 172.21.15.83:0/1395769221 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa2c06e950 0x7faa2c15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa2b400640 1 -- 172.21.15.83:0/1395769221 >> v2:172.21.15.86:3300/0 conn(0x7faa2c15fe50 msgr2=0x7faa2c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:29.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa2b400640 1 --2- 172.21.15.83:0/1395769221 >> v2:172.21.15.86:3300/0 conn(0x7faa2c15fe50 0x7faa2c15c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa2b400640 1 -- 172.21.15.83:0/1395769221 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2c06eec0 con 0x7faa2c159810 2024-04-25T00:13:29.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa2be00640 1 --2- 172.21.15.83:0/1395769221 >> v2:172.21.15.86:3300/0 conn(0x7faa2c15fe50 0x7faa2c15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:29.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.552+0000 7faa2aa00640 1 --2- 172.21.15.83:0/1395769221 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa2c06e950 0x7faa2c15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:29.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa2b400640 1 --2- 172.21.15.83:0/1395769221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2c159810 0x7faa2c0cc4a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0cdb40 tx=0x7faa1c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3897b2b3fe8b72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:29.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa2a000640 1 -- 172.21.15.83:0/1395769221 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0a56d0 con 0x7faa2c159810 2024-04-25T00:13:29.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa2a000640 1 -- 172.21.15.83:0/1395769221 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1c0a5870 con 0x7faa2c159810 2024-04-25T00:13:29.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa2a000640 1 -- 172.21.15.83:0/1395769221 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0a5c40 con 0x7faa2c159810 2024-04-25T00:13:29.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa32000640 1 -- 172.21.15.83:0/1395769221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2c159810 msgr2=0x7faa2c0cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa32000640 1 --2- 172.21.15.83:0/1395769221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2c159810 0x7faa2c0cc4a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0cdb40 tx=0x7faa1c01c960 comp rx=0 tx=0).stop 2024-04-25T00:13:29.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa32000640 1 -- 172.21.15.83:0/1395769221 shutdown_connections 2024-04-25T00:13:29.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa32000640 1 --2- 172.21.15.83:0/1395769221 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa2c06e950 0x7faa2c15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa32000640 1 --2- 172.21.15.83:0/1395769221 >> v2:172.21.15.86:3300/0 conn(0x7faa2c15fe50 0x7faa2c15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa32000640 1 --2- 172.21.15.83:0/1395769221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2c159810 0x7faa2c0cc4a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa32000640 1 -- 172.21.15.83:0/1395769221 >> 172.21.15.83:0/1395769221 conn(0x7faa2c0b1830 msgr2=0x7faa2c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:29.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa32000640 1 -- 172.21.15.83:0/1395769221 shutdown_connections 2024-04-25T00:13:29.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa32000640 1 -- 172.21.15.83:0/1395769221 wait complete. 2024-04-25T00:13:29.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa32000640 1 Processor -- start 2024-04-25T00:13:29.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.556+0000 7faa32000640 1 -- start start 2024-04-25T00:13:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa32000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c06e950 0x7faa2c15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa32000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2c159810 0x7faa2c14e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa32000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa2c15fe50 0x7faa2c14e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa32000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c159d70 con 0x7faa2c159810 2024-04-25T00:13:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa32000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c159bf0 con 0x7faa2c06e950 2024-04-25T00:13:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa32000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7faa2c070da0 con 0x7faa2c15fe50 2024-04-25T00:13:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2c159810 0x7faa2c14e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2c159810 0x7faa2c14e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56876/0 (socket says 172.21.15.83:56876) 2024-04-25T00:13:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa2aa00640 1 -- 172.21.15.83:0/403748869 learned_addr learned my addr 172.21.15.83:0/403748869 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa2b400640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c06e950 0x7faa2c15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa2be00640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa2c15fe50 0x7faa2c14e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa2aa00640 1 -- 172.21.15.83:0/403748869 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa2c15fe50 msgr2=0x7faa2c14e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa2aa00640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa2c15fe50 0x7faa2c14e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa2aa00640 1 -- 172.21.15.83:0/403748869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c06e950 msgr2=0x7faa2c15b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa2aa00640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c06e950 0x7faa2c15b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa2aa00640 1 -- 172.21.15.83:0/403748869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2c153410 con 0x7faa2c159810 2024-04-25T00:13:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa2b400640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c06e950 0x7faa2c15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa2aa00640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2c159810 0x7faa2c14e210 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7faa1800c7f0 tx=0x7faa1800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa23400640 1 -- 172.21.15.83:0/403748869 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18015070 con 0x7faa2c159810 2024-04-25T00:13:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa23400640 1 -- 172.21.15.83:0/403748869 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa18007820 con 0x7faa2c159810 2024-04-25T00:13:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa23400640 1 -- 172.21.15.83:0/403748869 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18007c10 con 0x7faa2c159810 2024-04-25T00:13:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa32000640 1 -- 172.21.15.83:0/403748869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa2c1536f0 con 0x7faa2c159810 2024-04-25T00:13:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa32000640 1 -- 172.21.15.83:0/403748869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa2c0b2dd0 con 0x7faa2c159810 2024-04-25T00:13:29.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa32000640 1 -- 172.21.15.83:0/403748869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa2c0b2fd0 con 0x7faa2c159810 2024-04-25T00:13:29.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.560+0000 7faa23400640 1 -- 172.21.15.83:0/403748869 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faa18018070 con 0x7faa2c159810 2024-04-25T00:13:29.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.564+0000 7faa23400640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faa0c0678c0 0x7faa0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.564+0000 7faa23400640 1 -- 172.21.15.83:0/403748869 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7faa18085fb0 con 0x7faa2c159810 2024-04-25T00:13:29.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.564+0000 7faa2b400640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faa0c0678c0 0x7faa0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.564+0000 7faa2b400640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faa0c0678c0 0x7faa0c069d80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0ae000 tx=0x7faa1c0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:29.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.568+0000 7faa23400640 1 -- 172.21.15.83:0/403748869 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faa2c0b2fd0 con 0x7faa2c159810 2024-04-25T00:13:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.776+0000 7faa32000640 1 -- 172.21.15.83:0/403748869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7faa2c04f510 con 0x7faa2c159810 2024-04-25T00:13:29.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.780+0000 7faa23400640 1 -- 172.21.15.83:0/403748869 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7faa18056270 con 0x7faa2c159810 2024-04-25T00:13:29.782 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-25T00:13:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylmOqooLBAApte+X6YkRmrpe12YdjYekg== 2024-04-25T00:13:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-25T00:13:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-25T00:13:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:13:29.783 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-25T00:13:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.780+0000 7faa32000640 1 -- 172.21.15.83:0/403748869 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faa0c0678c0 msgr2=0x7faa0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.780+0000 7faa32000640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faa0c0678c0 0x7faa0c069d80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0ae000 tx=0x7faa1c0a7070 comp rx=0 tx=0).stop 2024-04-25T00:13:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.780+0000 7faa32000640 1 -- 172.21.15.83:0/403748869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2c159810 msgr2=0x7faa2c14e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.780+0000 7faa32000640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2c159810 0x7faa2c14e210 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7faa1800c7f0 tx=0x7faa1800ccb0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.780+0000 7faa32000640 1 -- 172.21.15.83:0/403748869 shutdown_connections 2024-04-25T00:13:29.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.780+0000 7faa32000640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faa0c0678c0 0x7faa0c069d80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.780+0000 7faa32000640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa2c15fe50 0x7faa2c14e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.780+0000 7faa32000640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2c159810 0x7faa2c14e210 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.780+0000 7faa32000640 1 --2- 172.21.15.83:0/403748869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c06e950 0x7faa2c15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.780+0000 7faa32000640 1 -- 172.21.15.83:0/403748869 >> 172.21.15.83:0/403748869 conn(0x7faa2c0b1830 msgr2=0x7faa2c164f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:29.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.780+0000 7faa32000640 1 -- 172.21.15.83:0/403748869 shutdown_connections 2024-04-25T00:13:29.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.780+0000 7faa32000640 1 -- 172.21.15.83:0/403748869 wait complete. 2024-04-25T00:13:29.804 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.936+0000 7fe7dfe00640 1 Processor -- start 2024-04-25T00:13:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.936+0000 7fe7dfe00640 1 -- start start 2024-04-25T00:13:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7dfe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe7e01597d0 0x7fe7e00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7dfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe7e015c9b0 0x7fe7e00b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7dfe00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e01605c0 con 0x7fe7e00d0010 2024-04-25T00:13:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7dfe00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe7e015cff0 con 0x7fe7e01597d0 2024-04-25T00:13:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7dfe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe7e015f800 con 0x7fe7e015c9b0 2024-04-25T00:13:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7dea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe7e01597d0 0x7fe7e00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7d7e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe7e00d0010 0x7fe7e015c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54890/0 (socket says 172.21.15.83:54890) 2024-04-25T00:13:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7d7e00640 1 -- 172.21.15.83:0/1163238075 learned_addr learned my addr 172.21.15.83:0/1163238075 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7df400640 1 --2- 172.21.15.83:0/1163238075 >> v2:172.21.15.86:3300/0 conn(0x7fe7e015c9b0 0x7fe7e00b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7dea00640 1 -- 172.21.15.83:0/1163238075 >> v2:172.21.15.86:3300/0 conn(0x7fe7e015c9b0 msgr2=0x7fe7e00b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7dea00640 1 --2- 172.21.15.83:0/1163238075 >> v2:172.21.15.86:3300/0 conn(0x7fe7e015c9b0 0x7fe7e00b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7dea00640 1 -- 172.21.15.83:0/1163238075 >> v1:172.21.15.83:6789/0 conn(0x7fe7e00d0010 legacy=0x7fe7e015c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7dea00640 1 -- 172.21.15.83:0/1163238075 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e015fae0 con 0x7fe7e01597d0 2024-04-25T00:13:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7de000640 1 -- 172.21.15.83:0/1163238075 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3683585959 0 0) 0x7fe7e01605c0 con 0x7fe7e00d0010 2024-04-25T00:13:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7df400640 1 --2- 172.21.15.83:0/1163238075 >> v2:172.21.15.86:3300/0 conn(0x7fe7e015c9b0 0x7fe7e00b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.940+0000 7fe7dea00640 1 --2- 172.21.15.83:0/1163238075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e01597d0 0x7fe7e00cfad0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00b5010 tx=0x7fe7d001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d65cbcc16c4028 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7de000640 1 -- 172.21.15.83:0/1163238075 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00b0070 con 0x7fe7e01597d0 2024-04-25T00:13:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7de000640 1 -- 172.21.15.83:0/1163238075 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d00ab600 con 0x7fe7e01597d0 2024-04-25T00:13:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7de000640 1 -- 172.21.15.83:0/1163238075 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00ab9d0 con 0x7fe7e01597d0 2024-04-25T00:13:29.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 -- 172.21.15.83:0/1163238075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e01597d0 msgr2=0x7fe7e00cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 --2- 172.21.15.83:0/1163238075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e01597d0 0x7fe7e00cfad0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00b5010 tx=0x7fe7d001c360 comp rx=0 tx=0).stop 2024-04-25T00:13:29.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 -- 172.21.15.83:0/1163238075 shutdown_connections 2024-04-25T00:13:29.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 --2- 172.21.15.83:0/1163238075 >> v2:172.21.15.86:3300/0 conn(0x7fe7e015c9b0 0x7fe7e00b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 --2- 172.21.15.83:0/1163238075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e01597d0 0x7fe7e00cfad0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 -- 172.21.15.83:0/1163238075 >> 172.21.15.83:0/1163238075 conn(0x7fe7e006b3b0 msgr2=0x7fe7e006b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 -- 172.21.15.83:0/1163238075 shutdown_connections 2024-04-25T00:13:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 -- 172.21.15.83:0/1163238075 wait complete. 2024-04-25T00:13:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 Processor -- start 2024-04-25T00:13:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 -- start start 2024-04-25T00:13:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e00d0010 0x7fe7e0150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7e01597d0 0x7fe7e01528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e015c9b0 0x7fe7e0152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e00b3a30 con 0x7fe7e00d0010 2024-04-25T00:13:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e00b38b0 con 0x7fe7e015c9b0 2024-04-25T00:13:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dfe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e0168a70 con 0x7fe7e01597d0 2024-04-25T00:13:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e00d0010 0x7fe7e0150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e00d0010 0x7fe7e0150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56890/0 (socket says 172.21.15.83:56890) 2024-04-25T00:13:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7dea00640 1 -- 172.21.15.83:0/3542182642 learned_addr learned my addr 172.21.15.83:0/3542182642 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7d7e00640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7e01597d0 0x7fe7e01528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.944+0000 7fe7df400640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e015c9b0 0x7fe7e0152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7dea00640 1 -- 172.21.15.83:0/3542182642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7e01597d0 msgr2=0x7fe7e01528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7dea00640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7e01597d0 0x7fe7e01528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7dea00640 1 -- 172.21.15.83:0/3542182642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e015c9b0 msgr2=0x7fe7e0152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7dea00640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e015c9b0 0x7fe7e0152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7dea00640 1 -- 172.21.15.83:0/3542182642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e01517a0 con 0x7fe7e00d0010 2024-04-25T00:13:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7d7e00640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7e01597d0 0x7fe7e01528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7dea00640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e00d0010 0x7fe7e0150ed0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0019f90 tx=0x7fe7d001aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7d6a00640 1 -- 172.21.15.83:0/3542182642 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00b0050 con 0x7fe7e00d0010 2024-04-25T00:13:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7d6a00640 1 -- 172.21.15.83:0/3542182642 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d00bc070 con 0x7fe7e00d0010 2024-04-25T00:13:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7d6a00640 1 -- 172.21.15.83:0/3542182642 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00b7670 con 0x7fe7e00d0010 2024-04-25T00:13:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7dfe00640 1 -- 172.21.15.83:0/3542182642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e0153450 con 0x7fe7e00d0010 2024-04-25T00:13:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7dfe00640 1 -- 172.21.15.83:0/3542182642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e015b040 con 0x7fe7e00d0010 2024-04-25T00:13:29.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7dfe00640 1 -- 172.21.15.83:0/3542182642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e004f510 con 0x7fe7e00d0010 2024-04-25T00:13:29.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7d6a00640 1 -- 172.21.15.83:0/3542182642 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe7d00b7810 con 0x7fe7e00d0010 2024-04-25T00:13:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7d6a00640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7bc0678c0 0x7fe7bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7d6a00640 1 -- 172.21.15.83:0/3542182642 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fe7d0120570 con 0x7fe7e00d0010 2024-04-25T00:13:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.948+0000 7fe7d7e00640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7bc0678c0 0x7fe7bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.952+0000 7fe7d7e00640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7bc0678c0 0x7fe7bc069d80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe7e01fa070 tx=0x7fe7c8007cd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:29.952+0000 7fe7d6a00640 1 -- 172.21.15.83:0/3542182642 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe7d0016070 con 0x7fe7e00d0010 2024-04-25T00:13:30.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.140+0000 7fe7dfe00640 1 -- 172.21.15.83:0/3542182642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7e0154260 con 0x7fe7e00d0010 2024-04-25T00:13:30.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.140+0000 7fe7d6a00640 1 -- 172.21.15.83:0/3542182642 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5176 (secure 0 0 0) 0x7fe7d00f0e80 con 0x7fe7e00d0010 2024-04-25T00:13:30.144 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:30.144 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3667389928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":3667389928},{"type":"v1","addr":"172.21.15.86:6835","nonce":3667389928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5923,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2174516900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2174516900},{"type":"v1","addr":"172.21.15.83:6837","nonce":2174516900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2877383406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2877383406},{"type":"v1","addr":"172.21.15.83:6836","nonce":2877383406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:13:19.805315+0000","modified":"2024-04-25T00:13:22.834474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6834/52944842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":52944842},{"type":"v1","addr":"172.21.15.86:6834","nonce":52944842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-04-25T00:13:30.145 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 35 2024-04-25T00:13:30.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.144+0000 7fe7dfe00640 1 -- 172.21.15.83:0/3542182642 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7bc0678c0 msgr2=0x7fe7bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.144+0000 7fe7dfe00640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7bc0678c0 0x7fe7bc069d80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe7e01fa070 tx=0x7fe7c8007cd0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.144+0000 7fe7dfe00640 1 -- 172.21.15.83:0/3542182642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e00d0010 msgr2=0x7fe7e0150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.144+0000 7fe7dfe00640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e00d0010 0x7fe7e0150ed0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0019f90 tx=0x7fe7d001aba0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.144+0000 7fe7dfe00640 1 -- 172.21.15.83:0/3542182642 shutdown_connections 2024-04-25T00:13:30.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.144+0000 7fe7dfe00640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7bc0678c0 0x7fe7bc069d80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.144+0000 7fe7dfe00640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7e01597d0 0x7fe7e01528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.144+0000 7fe7dfe00640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e015c9b0 0x7fe7e0152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.144+0000 7fe7dfe00640 1 --2- 172.21.15.83:0/3542182642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e00d0010 0x7fe7e0150ed0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.144+0000 7fe7dfe00640 1 -- 172.21.15.83:0/3542182642 >> 172.21.15.83:0/3542182642 conn(0x7fe7e006b3b0 msgr2=0x7fe7e00ad9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:30.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.144+0000 7fe7dfe00640 1 -- 172.21.15.83:0/3542182642 shutdown_connections 2024-04-25T00:13:30.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.144+0000 7fe7dfe00640 1 -- 172.21.15.83:0/3542182642 wait complete. 2024-04-25T00:13:30.568 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:13:19.805315+0000', 'modified': '2024-04-25T00:13:22.834474+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6198}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6198': {'gid': 6198, 'name': 'd', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6834/52944842', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6832', 'nonce': 52944842}, {'type': 'v1', 'addr': '172.21.15.86:6834', 'nonce': 52944842}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6198, 'qdb_cluster': [6198]} 2024-04-25T00:13:30.568 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:13:30.569 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T00:13:30.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa2be00640 1 Processor -- start 2024-04-25T00:13:30.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa2be00640 1 -- start start 2024-04-25T00:13:30.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa2be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffa241446e0 0x7ffa240cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa2be00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffa2406ee20 0x7ffa24071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa2be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa240cb210 con 0x7ffa2415f350 2024-04-25T00:13:30.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa2be00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ffa240715b0 con 0x7ffa2406ee20 2024-04-25T00:13:30.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa2be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffa24144ac0 con 0x7ffa241446e0 2024-04-25T00:13:30.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa29e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffa2406ee20 0x7ffa24071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa29e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffa2406ee20 0x7ffa24071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56894/0 (socket says 172.21.15.83:56894) 2024-04-25T00:13:30.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa29e00640 1 -- 172.21.15.83:0/3411050944 learned_addr learned my addr 172.21.15.83:0/3411050944 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:30.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa29400640 1 --2- 172.21.15.83:0/3411050944 >> v2:172.21.15.86:3300/0 conn(0x7ffa241446e0 0x7ffa240cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa29e00640 1 -- 172.21.15.83:0/3411050944 >> v2:172.21.15.86:3300/0 conn(0x7ffa241446e0 msgr2=0x7ffa240cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:30.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa29e00640 1 --2- 172.21.15.83:0/3411050944 >> v2:172.21.15.86:3300/0 conn(0x7ffa241446e0 0x7ffa240cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa29e00640 1 -- 172.21.15.83:0/3411050944 >> v1:172.21.15.86:6789/0 conn(0x7ffa2415f350 legacy=0x7ffa2406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa29e00640 1 -- 172.21.15.83:0/3411050944 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa240cb4f0 con 0x7ffa2406ee20 2024-04-25T00:13:30.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.704+0000 7ffa29400640 1 --2- 172.21.15.83:0/3411050944 >> v2:172.21.15.86:3300/0 conn(0x7ffa241446e0 0x7ffa240cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:30.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.708+0000 7ffa29e00640 1 --2- 172.21.15.83:0/3411050944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2406ee20 0x7ffa24071fe0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ffa2404fb60 tx=0x7ffa1c01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6eaf1c119c5d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:30.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.708+0000 7ffa23e00640 1 -- 172.21.15.83:0/3411050944 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c00b2d0 con 0x7ffa2406ee20 2024-04-25T00:13:30.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.708+0000 7ffa23e00640 1 -- 172.21.15.83:0/3411050944 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1c00b470 con 0x7ffa2406ee20 2024-04-25T00:13:30.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.708+0000 7ffa23e00640 1 -- 172.21.15.83:0/3411050944 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c0a7bf0 con 0x7ffa2406ee20 2024-04-25T00:13:30.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.708+0000 7ffa2be00640 1 -- 172.21.15.83:0/3411050944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2406ee20 msgr2=0x7ffa24071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.708+0000 7ffa2be00640 1 --2- 172.21.15.83:0/3411050944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2406ee20 0x7ffa24071fe0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ffa2404fb60 tx=0x7ffa1c01ebf0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.708+0000 7ffa2be00640 1 -- 172.21.15.83:0/3411050944 shutdown_connections 2024-04-25T00:13:30.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.708+0000 7ffa2be00640 1 --2- 172.21.15.83:0/3411050944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2406ee20 0x7ffa24071fe0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.708+0000 7ffa2be00640 1 --2- 172.21.15.83:0/3411050944 >> v2:172.21.15.86:3300/0 conn(0x7ffa241446e0 0x7ffa240cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.708+0000 7ffa2be00640 1 -- 172.21.15.83:0/3411050944 >> 172.21.15.83:0/3411050944 conn(0x7ffa240cfa60 msgr2=0x7ffa240cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:30.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.708+0000 7ffa2be00640 1 -- 172.21.15.83:0/3411050944 shutdown_connections 2024-04-25T00:13:30.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.708+0000 7ffa2be00640 1 -- 172.21.15.83:0/3411050944 wait complete. 2024-04-25T00:13:30.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa2be00640 1 Processor -- start 2024-04-25T00:13:30.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa2be00640 1 -- start start 2024-04-25T00:13:30.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa2be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2406ee20 0x7ffa240b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa2be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa241446e0 0x7ffa2415afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa2be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa2415f350 0x7ffa2415b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa2be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa24072eb0 con 0x7ffa2406ee20 2024-04-25T00:13:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa2be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa24072d30 con 0x7ffa2415f350 2024-04-25T00:13:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa2be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ffa24073030 con 0x7ffa241446e0 2024-04-25T00:13:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa29400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2406ee20 0x7ffa240b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa29400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2406ee20 0x7ffa240b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56904/0 (socket says 172.21.15.83:56904) 2024-04-25T00:13:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa29400640 1 -- 172.21.15.83:0/1068434051 learned_addr learned my addr 172.21.15.83:0/1068434051 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa29e00640 1 --2- 172.21.15.83:0/1068434051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa2415f350 0x7ffa2415b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa28a00640 1 --2- 172.21.15.83:0/1068434051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa241446e0 0x7ffa2415afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa29400640 1 -- 172.21.15.83:0/1068434051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa241446e0 msgr2=0x7ffa2415afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa29400640 1 --2- 172.21.15.83:0/1068434051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa241446e0 0x7ffa2415afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa29400640 1 -- 172.21.15.83:0/1068434051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa2415f350 msgr2=0x7ffa2415b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa29400640 1 --2- 172.21.15.83:0/1068434051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa2415f350 0x7ffa2415b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa29400640 1 -- 172.21.15.83:0/1068434051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2413e2c0 con 0x7ffa2406ee20 2024-04-25T00:13:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa29400640 1 --2- 172.21.15.83:0/1068434051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2406ee20 0x7ffa240b4030 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800e810 tx=0x7ffa1800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa21600640 1 -- 172.21.15.83:0/1068434051 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18017070 con 0x7ffa2406ee20 2024-04-25T00:13:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa21600640 1 -- 172.21.15.83:0/1068434051 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1800c8d0 con 0x7ffa2406ee20 2024-04-25T00:13:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa21600640 1 -- 172.21.15.83:0/1068434051 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1800f040 con 0x7ffa2406ee20 2024-04-25T00:13:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa2be00640 1 -- 172.21.15.83:0/1068434051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2413e5a0 con 0x7ffa2406ee20 2024-04-25T00:13:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.712+0000 7ffa2be00640 1 -- 172.21.15.83:0/1068434051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa241557f0 con 0x7ffa2406ee20 2024-04-25T00:13:30.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.716+0000 7ffa21600640 1 -- 172.21.15.83:0/1068434051 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffa1801a070 con 0x7ffa2406ee20 2024-04-25T00:13:30.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.716+0000 7ffa21600640 1 --2- 172.21.15.83:0/1068434051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffa040678c0 0x7ffa04069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.716+0000 7ffa21600640 1 -- 172.21.15.83:0/1068434051 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7ffa18085b20 con 0x7ffa2406ee20 2024-04-25T00:13:30.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.716+0000 7ffa28a00640 1 --2- 172.21.15.83:0/1068434051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffa040678c0 0x7ffa04069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.716+0000 7ffa2f2e8000 1 Processor -- start 2024-04-25T00:13:30.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.716+0000 7ffa2f2e8000 1 -- start start 2024-04-25T00:13:30.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.716+0000 7ffa28a00640 1 --2- 172.21.15.83:0/1068434051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffa040678c0 0x7ffa04069d80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffa24158cc0 tx=0x7ffa10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:30.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c4ef0 0x561fed0c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed0c5810 0x561fed07b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed07bbc0 0x561fed07bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561fecec95e0 con 0x561fed0c5810 2024-04-25T00:13:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561fece50c50 con 0x561fed0c4ef0 2024-04-25T00:13:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x561fecec55a0 con 0x561fed07bbc0 2024-04-25T00:13:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa28a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed0c5810 0x561fed07b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa28a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed0c5810 0x561fed07b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56910/0 (socket says 172.21.15.83:56910) 2024-04-25T00:13:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa28a00640 1 -- 172.21.15.83:0/1387368718 learned_addr learned my addr 172.21.15.83:0/1387368718 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa29400640 1 --2- 172.21.15.83:0/1387368718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c4ef0 0x561fed0c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa29e00640 1 --2- 172.21.15.83:0/1387368718 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed07bbc0 0x561fed07bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa28a00640 1 -- 172.21.15.83:0/1387368718 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed07bbc0 msgr2=0x561fed07bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa28a00640 1 --2- 172.21.15.83:0/1387368718 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed07bbc0 0x561fed07bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa28a00640 1 -- 172.21.15.83:0/1387368718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c4ef0 msgr2=0x561fed0c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa28a00640 1 --2- 172.21.15.83:0/1387368718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c4ef0 0x561fed0c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa28a00640 1 -- 172.21.15.83:0/1387368718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fece4ed10 con 0x561fed0c5810 2024-04-25T00:13:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa28a00640 1 --2- 172.21.15.83:0/1387368718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed0c5810 0x561fed07b680 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ffa10005280 tx=0x7ffa10011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ff3efd650443f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa16000640 1 -- 172.21.15.83:0/1387368718 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10021450 con 0x561fed0c5810 2024-04-25T00:13:30.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa16000640 1 -- 172.21.15.83:0/1387368718 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa100215f0 con 0x561fed0c5810 2024-04-25T00:13:30.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa16000640 1 -- 172.21.15.83:0/1387368718 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa100219a0 con 0x561fed0c5810 2024-04-25T00:13:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/1387368718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed0c5810 msgr2=0x561fed07b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/1387368718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed0c5810 0x561fed07b680 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ffa10005280 tx=0x7ffa10011000 comp rx=0 tx=0).stop 2024-04-25T00:13:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/1387368718 shutdown_connections 2024-04-25T00:13:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/1387368718 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed07bbc0 0x561fed07bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/1387368718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed0c5810 0x561fed07b680 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/1387368718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c4ef0 0x561fed0c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/1387368718 >> 172.21.15.83:0/1387368718 conn(0x561fed077b70 msgr2=0x561fed079f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/1387368718 shutdown_connections 2024-04-25T00:13:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/1387368718 wait complete. 2024-04-25T00:13:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 -- start start 2024-04-25T00:13:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 Processor -- start 2024-04-25T00:13:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed07bbc0 0x561fed0a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed0c4ef0 0x561fed0aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c5810 0x561fed0aa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561fece83520 con 0x561fed0c4ef0 2024-04-25T00:13:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561fece6f810 con 0x561fed0c5810 2024-04-25T00:13:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.720+0000 7ffa2f2e8000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x561fece96340 con 0x561fed07bbc0 2024-04-25T00:13:30.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa2f2e8000 10 client.0 _ll_register_callbacks cb 0x561fed0665c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:13:30.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa28a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed0c4ef0 0x561fed0aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa28a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed0c4ef0 0x561fed0aa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56922/0 (socket says 172.21.15.83:56922) 2024-04-25T00:13:30.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa29400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed07bbc0 0x561fed0a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa28a00640 1 -- 172.21.15.83:0/4033968447 learned_addr learned my addr 172.21.15.83:0/4033968447 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:30.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa29e00640 1 --2- 172.21.15.83:0/4033968447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c5810 0x561fed0aa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa28a00640 1 -- 172.21.15.83:0/4033968447 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed07bbc0 msgr2=0x561fed0a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:30.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa28a00640 1 --2- 172.21.15.83:0/4033968447 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed07bbc0 0x561fed0a9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa28a00640 1 -- 172.21.15.83:0/4033968447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c5810 msgr2=0x561fed0aa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:30.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa28a00640 1 --2- 172.21.15.83:0/4033968447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c5810 0x561fed0aa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa28a00640 1 -- 172.21.15.83:0/4033968447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fece617b0 con 0x561fed0c4ef0 2024-04-25T00:13:30.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa29e00640 1 --2- 172.21.15.83:0/4033968447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c5810 0x561fed0aa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:30.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa28a00640 1 --2- 172.21.15.83:0/4033968447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed0c4ef0 0x561fed0aa460 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ffa10021a50 tx=0x7ffa10000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1626a5642e015b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:30.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:13:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa0aa00640 1 -- 172.21.15.83:0/4033968447 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1000e660 con 0x561fed0c4ef0 2024-04-25T00:13:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/4033968447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa100215f0 con 0x561fed0c4ef0 2024-04-25T00:13:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa0aa00640 1 -- 172.21.15.83:0/4033968447 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7ffa1000e800 con 0x561fed0c4ef0 2024-04-25T00:13:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa0aa00640 1 -- 172.21.15.83:0/4033968447 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ffa10032050 con 0x561fed0c4ef0 2024-04-25T00:13:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa2f2e8000 10 client.6342 fetch_fsmap learned FSMap version 35 2024-04-25T00:13:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/4033968447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x561fece50ea0 con 0x561fed0c4ef0 2024-04-25T00:13:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa0aa00640 1 -- 172.21.15.83:0/4033968447 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa100215f0 con 0x561fed0c4ef0 2024-04-25T00:13:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa0aa00640 1 -- 172.21.15.83:0/4033968447 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7ffa10035020 con 0x561fed0c4ef0 2024-04-25T00:13:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa2f2e8000 10 client.6342 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T00:13:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa2f2e8000 10 client.6342 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T00:13:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa2f2e8000 20 client.6342 populate_metadata read hostname 'smithi083' 2024-04-25T00:13:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/4033968447 >> [v2:172.21.15.86:6832/52944842,v1:172.21.15.86:6834/52944842] conn(0x561fed0d4f90 0x561fed0d7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa2f2e8000 4 client.6342 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:13:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/4033968447 --> [v2:172.21.15.86:6832/52944842,v1:172.21.15.86:6834/52944842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fece64ab0 con 0x561fed0d4f90 2024-04-25T00:13:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa29400640 1 --2- 172.21.15.83:0/4033968447 >> [v2:172.21.15.86:6832/52944842,v1:172.21.15.86:6834/52944842] conn(0x561fed0d4f90 0x561fed0d7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa29400640 1 --2- 172.21.15.83:0/4033968447 >> [v2:172.21.15.86:6832/52944842,v1:172.21.15.86:6834/52944842] conn(0x561fed0d4f90 0x561fed0d7380 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d74f983df7ed2944 server_cookie=a8adc7e12f86a440 in_seq=0 out_seq=0 2024-04-25T00:13:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa0aa00640 10 client.6342 ms_handle_connect on v2:172.21.15.86:6832/52944842 2024-04-25T00:13:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa0aa00640 1 -- 172.21.15.83:0/4033968447 <== mds.0 v2:172.21.15.86:6832/52944842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561fece64ab0 con 0x561fed0d4f90 2024-04-25T00:13:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa0aa00640 10 client.6342 handle_command_reply: tid=0 2024-04-25T00:13:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.724+0000 7ffa2f2e8000 1 client.6342 shutdown 2024-04-25T00:13:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.728+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/4033968447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed0c4ef0 msgr2=0x561fed0aa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.728+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/4033968447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed0c4ef0 0x561fed0aa460 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ffa10021a50 tx=0x7ffa10000bd0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.728+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/4033968447 shutdown_connections 2024-04-25T00:13:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.728+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/4033968447 >> [v2:172.21.15.86:6832/52944842,v1:172.21.15.86:6834/52944842] conn(0x561fed0d4f90 0x561fed0d7380 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.728+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/4033968447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c5810 0x561fed0aa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.728+0000 7ffa0aa00640 0 client.6342 ms_handle_reset on v2:172.21.15.86:6832/52944842 2024-04-25T00:13:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.728+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/4033968447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed0c4ef0 0x561fed0aa460 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.728+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/4033968447 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed07bbc0 0x561fed0a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.728+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/4033968447 >> 172.21.15.83:0/4033968447 conn(0x561fed077b70 msgr2=0x561fed07a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:30.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.728+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/4033968447 shutdown_connections 2024-04-25T00:13:30.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.728+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/4033968447 wait complete. 2024-04-25T00:13:30.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.728+0000 7ffa2f2e8000 20 client.6342 trim_cache size 0 max 16384 2024-04-25T00:13:30.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa2f2e8000 1 Processor -- start 2024-04-25T00:13:30.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa2f2e8000 1 -- start start 2024-04-25T00:13:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa2f2e8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed07bbc0 0x561fed079f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa2f2e8000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed0c4ef0 0x561fed07a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa2f2e8000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c5810 0x561fed07aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa2f2e8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa241ff5e0 con 0x561fed07bbc0 2024-04-25T00:13:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa2f2e8000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa241fe660 con 0x561fed0c5810 2024-04-25T00:13:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa2f2e8000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2413fd30 con 0x561fed0c4ef0 2024-04-25T00:13:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa29400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed07bbc0 0x561fed079f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa29400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed07bbc0 0x561fed079f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56932/0 (socket says 172.21.15.83:56932) 2024-04-25T00:13:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa29400640 1 -- 172.21.15.83:0/209619174 learned_addr learned my addr 172.21.15.83:0/209619174 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:30.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa29400640 1 -- 172.21.15.83:0/209619174 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed0c4ef0 msgr2=0x561fed07a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:30.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa29400640 1 --2- 172.21.15.83:0/209619174 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed0c4ef0 0x561fed07a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa29400640 1 -- 172.21.15.83:0/209619174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c5810 msgr2=0x561fed07aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:30.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa29400640 1 --2- 172.21.15.83:0/209619174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c5810 0x561fed07aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa29400640 1 -- 172.21.15.83:0/209619174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fece51e80 con 0x561fed07bbc0 2024-04-25T00:13:30.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa29400640 1 --2- 172.21.15.83:0/209619174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed07bbc0 0x561fed079f60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ffa180565e0 tx=0x7ffa18015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928ef36183dfa08f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:30.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa15600640 1 -- 172.21.15.83:0/209619174 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18083f30 con 0x561fed07bbc0 2024-04-25T00:13:30.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa15600640 1 -- 172.21.15.83:0/209619174 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa18054030 con 0x561fed07bbc0 2024-04-25T00:13:30.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.744+0000 7ffa15600640 1 -- 172.21.15.83:0/209619174 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa180523a0 con 0x561fed07bbc0 2024-04-25T00:13:30.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/209619174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed07bbc0 msgr2=0x561fed079f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/209619174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed07bbc0 0x561fed079f60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ffa180565e0 tx=0x7ffa18015040 comp rx=0 tx=0).stop 2024-04-25T00:13:30.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/209619174 shutdown_connections 2024-04-25T00:13:30.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/209619174 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed0c4ef0 0x561fed07a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/209619174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c5810 0x561fed07aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/209619174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed07bbc0 0x561fed079f60 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/209619174 >> 172.21.15.83:0/209619174 conn(0x561fed077b70 msgr2=0x561fed0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/209619174 shutdown_connections 2024-04-25T00:13:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/209619174 wait complete. 2024-04-25T00:13:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 -- start start 2024-04-25T00:13:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 Processor -- start 2024-04-25T00:13:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed07bbc0 0x561fed134c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed0c4ef0 0x561fed1351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c5810 0x561fed131300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561fece61dd0 con 0x561fed07bbc0 2024-04-25T00:13:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa241fe040 con 0x561fed0c5810 2024-04-25T00:13:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x561fece4db10 con 0x561fed0c4ef0 2024-04-25T00:13:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 10 client.0 _ll_register_callbacks cb 0x7ffa1800eec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:13:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa29400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed07bbc0 0x561fed134c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa29400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed07bbc0 0x561fed134c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56936/0 (socket says 172.21.15.83:56936) 2024-04-25T00:13:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa29400640 1 -- 172.21.15.83:0/2045421301 learned_addr learned my addr 172.21.15.83:0/2045421301 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa29400640 1 -- 172.21.15.83:0/2045421301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed0c4ef0 msgr2=0x561fed1351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa29400640 1 --2- 172.21.15.83:0/2045421301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed0c4ef0 0x561fed1351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa29400640 1 -- 172.21.15.83:0/2045421301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c5810 msgr2=0x561fed131300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa29400640 1 --2- 172.21.15.83:0/2045421301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c5810 0x561fed131300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa29400640 1 -- 172.21.15.83:0/2045421301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fece96760 con 0x561fed07bbc0 2024-04-25T00:13:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa29400640 1 --2- 172.21.15.83:0/2045421301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed07bbc0 0x561fed134c90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800c4c0 tx=0x7ffa10005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58e7194bf4e896f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:13:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa0aa00640 1 -- 172.21.15.83:0/2045421301 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa180523a0 con 0x561fed07bbc0 2024-04-25T00:13:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa0aa00640 1 -- 172.21.15.83:0/2045421301 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7ffa1800cca0 con 0x561fed07bbc0 2024-04-25T00:13:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/2045421301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa18054030 con 0x561fed07bbc0 2024-04-25T00:13:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa0aa00640 1 -- 172.21.15.83:0/2045421301 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ffa18052540 con 0x561fed07bbc0 2024-04-25T00:13:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 10 client.6348 fetch_fsmap learned FSMap version 35 2024-04-25T00:13:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/2045421301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x561fecf229e0 con 0x561fed07bbc0 2024-04-25T00:13:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa0aa00640 1 -- 172.21.15.83:0/2045421301 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa18054030 con 0x561fed07bbc0 2024-04-25T00:13:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa0aa00640 1 -- 172.21.15.83:0/2045421301 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7ffa18012050 con 0x561fed07bbc0 2024-04-25T00:13:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 10 client.6348 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T00:13:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 10 client.6348 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T00:13:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 20 client.6348 populate_metadata read hostname 'smithi083' 2024-04-25T00:13:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/2045421301 >> [v2:172.21.15.86:6832/52944842,v1:172.21.15.86:6834/52944842] conn(0x561fed131c30 0x561fed132010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 4 client.6348 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:13:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.748+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/2045421301 --> [v2:172.21.15.86:6832/52944842,v1:172.21.15.86:6834/52944842] -- command(tid 0: {"prefix": "status"}) -- 0x561feced6b20 con 0x561fed131c30 2024-04-25T00:13:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa28a00640 1 --2- 172.21.15.83:0/2045421301 >> [v2:172.21.15.86:6832/52944842,v1:172.21.15.86:6834/52944842] conn(0x561fed131c30 0x561fed132010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa28a00640 1 --2- 172.21.15.83:0/2045421301 >> [v2:172.21.15.86:6832/52944842,v1:172.21.15.86:6834/52944842] conn(0x561fed131c30 0x561fed132010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d9abfec1f4256 server_cookie=c1c3af3ea540facf in_seq=0 out_seq=0 2024-04-25T00:13:30.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa0aa00640 10 client.6348 ms_handle_connect on v2:172.21.15.86:6832/52944842 2024-04-25T00:13:30.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa0aa00640 1 -- 172.21.15.83:0/2045421301 <== mds.0 v2:172.21.15.86:6832/52944842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x561feced6b20 con 0x561fed131c30 2024-04-25T00:13:30.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa0aa00640 10 client.6348 handle_command_reply: tid=0 2024-04-25T00:13:30.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2f2e8000 1 client.6348 shutdown 2024-04-25T00:13:30.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/2045421301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed07bbc0 msgr2=0x561fed134c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/2045421301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed07bbc0 0x561fed134c90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800c4c0 tx=0x7ffa10005000 comp rx=0 tx=0).stop 2024-04-25T00:13:30.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/2045421301 shutdown_connections 2024-04-25T00:13:30.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/2045421301 >> [v2:172.21.15.86:6832/52944842,v1:172.21.15.86:6834/52944842] conn(0x561fed131c30 0x561fed132010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/2045421301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x561fed0c4ef0 0x561fed1351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/2045421301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fed0c5810 0x561fed131300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa0aa00640 0 client.6348 ms_handle_reset on v2:172.21.15.86:6832/52944842 2024-04-25T00:13:30.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2f2e8000 1 --2- 172.21.15.83:0/2045421301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fed07bbc0 0x561fed134c90 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/2045421301 >> 172.21.15.83:0/2045421301 conn(0x561fed077b70 msgr2=0x561fed0c37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:30.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/2045421301 shutdown_connections 2024-04-25T00:13:30.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2f2e8000 1 -- 172.21.15.83:0/2045421301 wait complete. 2024-04-25T00:13:30.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2f2e8000 20 client.6348 trim_cache size 0 max 16384 2024-04-25T00:13:30.759 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:13:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "d58ae83a-ad72-4e18-824a-86e779979c35", 2024-04-25T00:13:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-25T00:13:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6198, 2024-04-25T00:13:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T00:13:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T00:13:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T00:13:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 10.326761872000001, 2024-04-25T00:13:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 35, 2024-04-25T00:13:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 60, 2024-04-25T00:13:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 60, 2024-04-25T00:13:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 15.197386010000001 2024-04-25T00:13:30.760 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:13:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2be00640 1 -- 172.21.15.83:0/1068434051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffa040678c0 msgr2=0x7ffa04069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2be00640 1 --2- 172.21.15.83:0/1068434051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffa040678c0 0x7ffa04069d80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffa24158cc0 tx=0x7ffa10009040 comp rx=0 tx=0).stop 2024-04-25T00:13:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2be00640 1 -- 172.21.15.83:0/1068434051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2406ee20 msgr2=0x7ffa240b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2be00640 1 --2- 172.21.15.83:0/1068434051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2406ee20 0x7ffa240b4030 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800e810 tx=0x7ffa1800ecd0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2be00640 1 -- 172.21.15.83:0/1068434051 shutdown_connections 2024-04-25T00:13:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2be00640 1 --2- 172.21.15.83:0/1068434051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffa040678c0 0x7ffa04069d80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2be00640 1 --2- 172.21.15.83:0/1068434051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa241446e0 0x7ffa2415afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2be00640 1 --2- 172.21.15.83:0/1068434051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa2415f350 0x7ffa2415b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2be00640 1 --2- 172.21.15.83:0/1068434051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2406ee20 0x7ffa240b4030 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2be00640 1 -- 172.21.15.83:0/1068434051 >> 172.21.15.83:0/1068434051 conn(0x7ffa240cfa60 msgr2=0x7ffa2413bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2be00640 1 -- 172.21.15.83:0/1068434051 shutdown_connections 2024-04-25T00:13:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.752+0000 7ffa2be00640 1 -- 172.21.15.83:0/1068434051 wait complete. 2024-04-25T00:13:30.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:13:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.908+0000 7f5596000640 1 Processor -- start 2024-04-25T00:13:30.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.908+0000 7f5596000640 1 -- start start 2024-04-25T00:13:30.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.908+0000 7f5596000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f55901599f0 0x7f55900cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.908+0000 7f5596000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f559015c340 0x7f559015c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.908+0000 7f5596000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55900ccc60 con 0x7f559015ccc0 2024-04-25T00:13:30.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.908+0000 7f5596000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5590159dd0 con 0x7f55901599f0 2024-04-25T00:13:30.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.908+0000 7f5596000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f559006dba0 con 0x7f559015c340 2024-04-25T00:13:30.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.908+0000 7f558f400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f55901599f0 0x7f55900cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.908+0000 7f558f400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f55901599f0 0x7f55900cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56938/0 (socket says 172.21.15.83:56938) 2024-04-25T00:13:30.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.908+0000 7f558f400640 1 -- 172.21.15.83:0/3641218627 learned_addr learned my addr 172.21.15.83:0/3641218627 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:30.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.908+0000 7f558ea00640 1 --2- 172.21.15.83:0/3641218627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f559015c340 0x7f559015c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f558f400640 1 -- 172.21.15.83:0/3641218627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f559015c340 msgr2=0x7f559015c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:30.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f558f400640 1 --2- 172.21.15.83:0/3641218627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f559015c340 0x7f559015c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f558f400640 1 -- 172.21.15.83:0/3641218627 >> v1:172.21.15.86:6789/0 conn(0x7f559015ccc0 legacy=0x7f559015f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:30.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f558f400640 1 -- 172.21.15.83:0/3641218627 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559006de80 con 0x7f55901599f0 2024-04-25T00:13:30.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f558f400640 1 --2- 172.21.15.83:0/3641218627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55901599f0 0x7f55900cc510 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5580009e80 tx=0x7f558001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7329d25630ba8d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:30.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f558e000640 1 -- 172.21.15.83:0/3641218627 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55800ac020 con 0x7f55901599f0 2024-04-25T00:13:30.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f558e000640 1 -- 172.21.15.83:0/3641218627 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55800a64a0 con 0x7f55901599f0 2024-04-25T00:13:30.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f558e000640 1 -- 172.21.15.83:0/3641218627 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55800a6870 con 0x7f55901599f0 2024-04-25T00:13:30.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f5596000640 1 -- 172.21.15.83:0/3641218627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55901599f0 msgr2=0x7f55900cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f5596000640 1 --2- 172.21.15.83:0/3641218627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55901599f0 0x7f55900cc510 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5580009e80 tx=0x7f558001c760 comp rx=0 tx=0).stop 2024-04-25T00:13:30.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f5596000640 1 -- 172.21.15.83:0/3641218627 shutdown_connections 2024-04-25T00:13:30.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f5596000640 1 --2- 172.21.15.83:0/3641218627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f559015c340 0x7f559015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f5596000640 1 --2- 172.21.15.83:0/3641218627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55901599f0 0x7f55900cc510 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f5596000640 1 -- 172.21.15.83:0/3641218627 >> 172.21.15.83:0/3641218627 conn(0x7f55900b1830 msgr2=0x7f55900b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:30.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f5596000640 1 -- 172.21.15.83:0/3641218627 shutdown_connections 2024-04-25T00:13:30.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.912+0000 7f5596000640 1 -- 172.21.15.83:0/3641218627 wait complete. 2024-04-25T00:13:30.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f5596000640 1 Processor -- start 2024-04-25T00:13:30.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f5596000640 1 -- start start 2024-04-25T00:13:30.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f5596000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55901599f0 0x7f559014f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f5596000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f559015c340 0x7f55900b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f5596000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559015ccc0 0x7f559014e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f5596000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55900cf870 con 0x7f559015c340 2024-04-25T00:13:30.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f5596000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55900cf6f0 con 0x7f559015ccc0 2024-04-25T00:13:30.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f5596000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5590168cd0 con 0x7f55901599f0 2024-04-25T00:13:30.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f558ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f559015c340 0x7f55900b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f558ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f559015c340 0x7f55900b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56948/0 (socket says 172.21.15.83:56948) 2024-04-25T00:13:30.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f558ea00640 1 -- 172.21.15.83:0/3275412871 learned_addr learned my addr 172.21.15.83:0/3275412871 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:30.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f558fe00640 1 --2- 172.21.15.83:0/3275412871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559015ccc0 0x7f559014e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f558f400640 1 --2- 172.21.15.83:0/3275412871 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55901599f0 0x7f559014f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f558ea00640 1 -- 172.21.15.83:0/3275412871 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55901599f0 msgr2=0x7f559014f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f558ea00640 1 --2- 172.21.15.83:0/3275412871 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55901599f0 0x7f559014f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f558ea00640 1 -- 172.21.15.83:0/3275412871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559015ccc0 msgr2=0x7f559014e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f558ea00640 1 --2- 172.21.15.83:0/3275412871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559015ccc0 0x7f559014e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f558ea00640 1 -- 172.21.15.83:0/3275412871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55901f75b0 con 0x7f559015c340 2024-04-25T00:13:30.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f558ea00640 1 --2- 172.21.15.83:0/3275412871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f559015c340 0x7f55900b3f20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f557c00d800 tx=0x7f557c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:30.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f5587400640 1 -- 172.21.15.83:0/3275412871 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c00b6c0 con 0x7f559015c340 2024-04-25T00:13:30.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f5587400640 1 -- 172.21.15.83:0/3275412871 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f557c019070 con 0x7f559015c340 2024-04-25T00:13:30.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f5587400640 1 -- 172.21.15.83:0/3275412871 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c00bae0 con 0x7f559015c340 2024-04-25T00:13:30.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f5596000640 1 -- 172.21.15.83:0/3275412871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55900b4460 con 0x7f559015c340 2024-04-25T00:13:30.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.916+0000 7f5596000640 1 -- 172.21.15.83:0/3275412871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55901f7b30 con 0x7f559015c340 2024-04-25T00:13:30.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.920+0000 7f5596000640 1 -- 172.21.15.83:0/3275412871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55900737f0 con 0x7f559015c340 2024-04-25T00:13:30.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.920+0000 7f5587400640 1 -- 172.21.15.83:0/3275412871 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f557c00bc80 con 0x7f559015c340 2024-04-25T00:13:30.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.920+0000 7f5587400640 1 --2- 172.21.15.83:0/3275412871 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f55700678c0 0x7f5570069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:30.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.920+0000 7f5587400640 1 -- 172.21.15.83:0/3275412871 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f557c0863f0 con 0x7f559015c340 2024-04-25T00:13:30.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.920+0000 7f558f400640 1 --2- 172.21.15.83:0/3275412871 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f55700678c0 0x7f5570069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:30.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.920+0000 7f558f400640 1 --2- 172.21.15.83:0/3275412871 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f55700678c0 0x7f5570069d80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f55800ad000 tx=0x7f55800b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:30.924+0000 7f5587400640 1 -- 172.21.15.83:0/3275412871 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f557c056d00 con 0x7f559015c340 2024-04-25T00:13:31.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.112+0000 7f5596000640 1 -- 172.21.15.83:0/3275412871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f559004f5f0 con 0x7f559015c340 2024-04-25T00:13:31.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.112+0000 7f5587400640 1 -- 172.21.15.83:0/3275412871 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f557c056600 con 0x7f559015c340 2024-04-25T00:13:31.117 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:13:31.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.116+0000 7f5596000640 1 -- 172.21.15.83:0/3275412871 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f55700678c0 msgr2=0x7f5570069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:31.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.116+0000 7f5596000640 1 --2- 172.21.15.83:0/3275412871 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f55700678c0 0x7f5570069d80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f55800ad000 tx=0x7f55800b3040 comp rx=0 tx=0).stop 2024-04-25T00:13:31.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.116+0000 7f5596000640 1 -- 172.21.15.83:0/3275412871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f559015c340 msgr2=0x7f55900b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:31.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.116+0000 7f5596000640 1 --2- 172.21.15.83:0/3275412871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f559015c340 0x7f55900b3f20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f557c00d800 tx=0x7f557c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:13:31.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.116+0000 7f5596000640 1 -- 172.21.15.83:0/3275412871 shutdown_connections 2024-04-25T00:13:31.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.116+0000 7f5596000640 1 --2- 172.21.15.83:0/3275412871 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f55700678c0 0x7f5570069d80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:31.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.116+0000 7f5596000640 1 --2- 172.21.15.83:0/3275412871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559015ccc0 0x7f559014e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:31.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.116+0000 7f5596000640 1 --2- 172.21.15.83:0/3275412871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f559015c340 0x7f55900b3f20 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:31.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.116+0000 7f5596000640 1 --2- 172.21.15.83:0/3275412871 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55901599f0 0x7f559014f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:31.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.116+0000 7f5596000640 1 -- 172.21.15.83:0/3275412871 >> 172.21.15.83:0/3275412871 conn(0x7f55900b1830 msgr2=0x7f55901656a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:31.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.116+0000 7f5596000640 1 -- 172.21.15.83:0/3275412871 shutdown_connections 2024-04-25T00:13:31.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.116+0000 7f5596000640 1 -- 172.21.15.83:0/3275412871 wait complete. 2024-04-25T00:13:31.138 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:13:31.138 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:13:31.138 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-25T00:13:31.138 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:13:31.138 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:13:31.138 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:13:31.138 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:13:31.138 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:13:31.138 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:13:31.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:31.139 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-25T00:13:31.148 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:13:31.148 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:13:31.148 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:13:31.148 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:31.148 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-25T00:13:31.148 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:31.148 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:13:31.148 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:31.149 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-25T00:13:31.149 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:13:31.149 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:31.149 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-25T00:13:31.149 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:13:31.149 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:31.149 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:13:31.149 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:31.149 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:13:31.149 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42323sec preferred_lft 42323sec 2024-04-25T00:13:31.149 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-25T00:13:31.149 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:31.149 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:13:31.150 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:8d:d4:7d:b9 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:31.150 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:13:31.150 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:31.150 INFO:teuthology.orchestra.run.smithi083.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:13:31.150 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:31.150 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:13:31.150 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:31.150 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:13:31.150 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:31.150 INFO:teuthology.orchestra.run.smithi083.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:13:31.150 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:13:31.150 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:13:31.150 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:31.151 INFO:teuthology.orchestra.run.smithi083.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:13:31.151 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:13:31.151 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:13:31.151 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:31.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:31.151 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:13:31.206 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-25T00:13:31.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:31.208 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-25T00:13:31.260 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-25T00:13:31.260 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:13:31.260 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:13:31.261 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:13:31.261 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:13:31.261 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:13:31.261 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:13:31.261 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:13:31.261 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:13:31.261 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:13:31.261 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:13:31.261 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:13:31.261 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:13:31.262 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:13:31.262 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:13:31.262 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:13:31.262 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:13:31.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:13:31.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:13:31.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:31.394 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:13:31.403 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:13:31.403 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:13:31.403 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:13:31.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:31.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-25T00:13:31.464 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:13:31.464 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:13:31.513 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:13:31.513 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:13:31.513 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:13:31.564 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T00:13:31.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:31.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:13:31.656 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:13:31.656 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:13:31.669 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:13:31.669 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:13:31.669 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:13:31.670 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:13:31.670 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:13:31.673 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:13:31.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:31.673 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:13:31.694 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:13:31.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:13:31.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:13:31.810 INFO:teuthology.orchestra.run.smithi083.stdout:client6357 2024-04-25T00:13:31.810 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:13:31.810 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:31.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:13:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.992+0000 7f3451800640 1 Processor -- start 2024-04-25T00:13:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.992+0000 7f3451800640 1 -- start start 2024-04-25T00:13:32.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.992+0000 7f3451800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f344c0b47e0 0x7f344c0b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:32.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.992+0000 7f3451800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f344c0b4c70 con 0x7f344c0b4360 2024-04-25T00:13:32.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.992+0000 7f3451800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f344c06ef80 con 0x7f344c1553e0 2024-04-25T00:13:32.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.992+0000 7f3451800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f344c15b510 con 0x7f344c0b47e0 2024-04-25T00:13:32.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f3442000640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f344c0b4360 0x7f344c0b2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54906/0 (socket says 172.21.15.83:54906) 2024-04-25T00:13:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f3442000640 1 -- 172.21.15.83:0/1233708198 learned_addr learned my addr 172.21.15.83:0/1233708198 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344b400640 1 --2- 172.21.15.83:0/1233708198 >> v2:172.21.15.86:3300/0 conn(0x7f344c0b47e0 0x7f344c0b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2694831087 0 0) 0x7f344c0b4c70 con 0x7f344c0b4360 2024-04-25T00:13:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3430005740 con 0x7f344c0b4360 2024-04-25T00:13:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4128150778 0 0) 0x7f344c06ef80 con 0x7f344c1553e0 2024-04-25T00:13:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f344c0b4c70 con 0x7f344c1553e0 2024-04-25T00:13:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 452819586 0 0) 0x7f3430005740 con 0x7f344c0b4360 2024-04-25T00:13:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 >> v2:172.21.15.86:3300/0 conn(0x7f344c0b47e0 msgr2=0x7f344c0b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 --2- 172.21.15.83:0/1233708198 >> v2:172.21.15.86:3300/0 conn(0x7f344c0b47e0 0x7f344c0b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 >> v1:172.21.15.86:6789/0 conn(0x7f344c1553e0 legacy=0x7f344c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344c15b7f0 con 0x7f344c0b4360 2024-04-25T00:13:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f3434001680 con 0x7f344c0b4360 2024-04-25T00:13:32.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344b400640 1 --2- 172.21.15.83:0/1233708198 >> v2:172.21.15.86:3300/0 conn(0x7f344c0b47e0 0x7f344c0b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:32.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 >> v1:172.21.15.83:6789/0 conn(0x7f344c0b4360 legacy=0x7f344c0b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:32.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 --2- 172.21.15.83:0/1233708198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3430005d60 0x7f3430006160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:32.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 --2- 172.21.15.83:0/1233708198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430006a20 0x7f3430006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:32.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 --2- 172.21.15.83:0/1233708198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34300073e0 0x7f343000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:32.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f344c15b510 con 0x7f3430006a20 2024-04-25T00:13:32.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f343000e310 con 0x7f34300073e0 2024-04-25T00:13:32.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f343000e510 con 0x7f3430005d60 2024-04-25T00:13:32.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f3442000640 1 --2- 172.21.15.83:0/1233708198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430006a20 0x7f3430006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:32.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344aa00640 1 --2- 172.21.15.83:0/1233708198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3430005d60 0x7f3430006160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344b400640 1 --2- 172.21.15.83:0/1233708198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34300073e0 0x7f343000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f3442000640 1 -- 172.21.15.83:0/1233708198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3430005d60 msgr2=0x7f3430006160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f3442000640 1 --2- 172.21.15.83:0/1233708198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3430005d60 0x7f3430006160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f3442000640 1 -- 172.21.15.83:0/1233708198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34300073e0 msgr2=0x7f343000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f3442000640 1 --2- 172.21.15.83:0/1233708198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34300073e0 0x7f343000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f3442000640 1 -- 172.21.15.83:0/1233708198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3430005740 con 0x7f3430006a20 2024-04-25T00:13:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344b400640 1 -- 172.21.15.83:0/1233708198 reap_dead start 2024-04-25T00:13:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:31.996+0000 7f344aa00640 1 --2- 172.21.15.83:0/1233708198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3430005d60 0x7f3430006160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f3442000640 1 --2- 172.21.15.83:0/1233708198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430006a20 0x7f3430006ea0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f343400c190 tx=0x7f343401cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9431c5884e7a5683 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434007cd0 con 0x7f3430006a20 2024-04-25T00:13:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34340a9020 con 0x7f3430006a20 2024-04-25T00:13:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f344a000640 1 -- 172.21.15.83:0/1233708198 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434005ae0 con 0x7f3430006a20 2024-04-25T00:13:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f3451800640 1 -- 172.21.15.83:0/1233708198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430006a20 msgr2=0x7f3430006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f3451800640 1 --2- 172.21.15.83:0/1233708198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430006a20 0x7f3430006ea0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f343400c190 tx=0x7f343401cc00 comp rx=0 tx=0).stop 2024-04-25T00:13:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f3451800640 1 -- 172.21.15.83:0/1233708198 shutdown_connections 2024-04-25T00:13:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f3451800640 1 --2- 172.21.15.83:0/1233708198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430006a20 0x7f3430006ea0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f3451800640 1 -- 172.21.15.83:0/1233708198 >> 172.21.15.83:0/1233708198 conn(0x7f344c0b1830 msgr2=0x7f344c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f3451800640 1 -- 172.21.15.83:0/1233708198 shutdown_connections 2024-04-25T00:13:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f3451800640 1 -- 172.21.15.83:0/1233708198 wait complete. 2024-04-25T00:13:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f3451800640 1 Processor -- start 2024-04-25T00:13:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f3451800640 1 -- start start 2024-04-25T00:13:32.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f3451800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3430006a20 0x7f344c1e84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.000+0000 7f3451800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344c157500 0x7f344c1fad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f3451800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344c15a2d0 0x7f344c15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f3451800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f344c155940 con 0x7f344c15a2d0 2024-04-25T00:13:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f3451800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f344c1557c0 con 0x7f344c157500 2024-04-25T00:13:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f3451800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f344c15f200 con 0x7f3430006a20 2024-04-25T00:13:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f344b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344c15a2d0 0x7f344c15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f344b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344c15a2d0 0x7f344c15a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56966/0 (socket says 172.21.15.83:56966) 2024-04-25T00:13:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f344b400640 1 -- 172.21.15.83:0/1859655002 learned_addr learned my addr 172.21.15.83:0/1859655002 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f3442000640 1 --2- 172.21.15.83:0/1859655002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344c157500 0x7f344c1fad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f344aa00640 1 --2- 172.21.15.83:0/1859655002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3430006a20 0x7f344c1e84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f344b400640 1 -- 172.21.15.83:0/1859655002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3430006a20 msgr2=0x7f344c1e84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f344b400640 1 --2- 172.21.15.83:0/1859655002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3430006a20 0x7f344c1e84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f344b400640 1 -- 172.21.15.83:0/1859655002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344c157500 msgr2=0x7f344c1fad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f344b400640 1 --2- 172.21.15.83:0/1859655002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344c157500 0x7f344c1fad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f344b400640 1 -- 172.21.15.83:0/1859655002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344c15ac90 con 0x7f344c15a2d0 2024-04-25T00:13:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f344b400640 1 --2- 172.21.15.83:0/1859655002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344c15a2d0 0x7f344c15a750 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f344c0b4120 tx=0x7f34440115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f3443400640 1 -- 172.21.15.83:0/1859655002 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34440039a0 con 0x7f344c15a2d0 2024-04-25T00:13:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f3443400640 1 -- 172.21.15.83:0/1859655002 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f344401b070 con 0x7f344c15a2d0 2024-04-25T00:13:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f3451800640 1 -- 172.21.15.83:0/1859655002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f344c157c70 con 0x7f344c15a2d0 2024-04-25T00:13:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f3443400640 1 -- 172.21.15.83:0/1859655002 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3444003ba0 con 0x7f344c15a2d0 2024-04-25T00:13:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f3451800640 1 -- 172.21.15.83:0/1859655002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f344c158230 con 0x7f344c15a2d0 2024-04-25T00:13:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f3451800640 1 -- 172.21.15.83:0/1859655002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f344c158430 con 0x7f344c15a2d0 2024-04-25T00:13:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.004+0000 7f3443400640 1 -- 172.21.15.83:0/1859655002 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3444004030 con 0x7f344c15a2d0 2024-04-25T00:13:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.008+0000 7f3443400640 1 --2- 172.21.15.83:0/1859655002 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34280678f0 0x7f3428069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.008+0000 7f3443400640 1 -- 172.21.15.83:0/1859655002 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f34440868b0 con 0x7f344c15a2d0 2024-04-25T00:13:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.008+0000 7f344aa00640 1 --2- 172.21.15.83:0/1859655002 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34280678f0 0x7f3428069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.008+0000 7f344aa00640 1 --2- 172.21.15.83:0/1859655002 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34280678f0 0x7f3428069db0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f344c069f90 tx=0x7f343c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.012+0000 7f3443400640 1 -- 172.21.15.83:0/1859655002 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f344c158430 con 0x7f344c15a2d0 2024-04-25T00:13:32.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.200+0000 7f3451800640 1 -- 172.21.15.83:0/1859655002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f344c0737f0 con 0x7f344c15a2d0 2024-04-25T00:13:32.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.200+0000 7f3443400640 1 -- 172.21.15.83:0/1859655002 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f3444025020 con 0x7f344c15a2d0 2024-04-25T00:13:32.204 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:13:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.204+0000 7f3451800640 1 -- 172.21.15.83:0/1859655002 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34280678f0 msgr2=0x7f3428069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.204+0000 7f3451800640 1 --2- 172.21.15.83:0/1859655002 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34280678f0 0x7f3428069db0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f344c069f90 tx=0x7f343c009040 comp rx=0 tx=0).stop 2024-04-25T00:13:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.204+0000 7f3451800640 1 -- 172.21.15.83:0/1859655002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344c15a2d0 msgr2=0x7f344c15a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.204+0000 7f3451800640 1 --2- 172.21.15.83:0/1859655002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344c15a2d0 0x7f344c15a750 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f344c0b4120 tx=0x7f34440115a0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.204+0000 7f3451800640 1 -- 172.21.15.83:0/1859655002 shutdown_connections 2024-04-25T00:13:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.204+0000 7f3451800640 1 --2- 172.21.15.83:0/1859655002 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34280678f0 0x7f3428069db0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.204+0000 7f3451800640 1 --2- 172.21.15.83:0/1859655002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344c15a2d0 0x7f344c15a750 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.204+0000 7f3451800640 1 --2- 172.21.15.83:0/1859655002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344c157500 0x7f344c1fad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.204+0000 7f3451800640 1 --2- 172.21.15.83:0/1859655002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3430006a20 0x7f344c1e84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.204+0000 7f3451800640 1 -- 172.21.15.83:0/1859655002 >> 172.21.15.83:0/1859655002 conn(0x7f344c0b1830 msgr2=0x7f344c0b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.204+0000 7f3451800640 1 -- 172.21.15.83:0/1859655002 shutdown_connections 2024-04-25T00:13:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:32.204+0000 7f3451800640 1 -- 172.21.15.83:0/1859655002 wait complete. 2024-04-25T00:13:32.225 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:13:32.225 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:13:32.225 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T00:13:32.225 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:13:32.225 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:13:32.225 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:13:32.225 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:13:32.225 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:13:32.225 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:13:32.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:32.226 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T00:13:32.236 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:13:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:13:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:13:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T00:13:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:32.236 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:13:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T00:13:32.236 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:13:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T00:13:32.237 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:13:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:32.237 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:13:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:13:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42316sec preferred_lft 42316sec 2024-04-25T00:13:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T00:13:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:32.237 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:13:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:35:63:ea:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:13:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:32.238 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:13:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:13:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:13:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:32.238 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:13:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:13:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:13:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:32.239 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:13:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:13:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:13:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:32.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:32.240 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:13:32.301 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T00:13:32.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:32.303 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T00:13:32.358 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T00:13:32.359 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:13:32.359 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:13:32.359 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:13:32.359 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:13:32.359 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:13:32.359 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:13:32.359 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:13:32.359 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:13:32.359 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:13:32.359 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:13:32.360 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:13:32.360 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:13:32.360 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:13:32.360 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:13:32.360 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:13:32.360 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:13:32.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:13:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:13:32.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:13:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:13:32.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:32.505 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T00:13:32.513 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:13:32.513 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:13:32.514 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:13:32.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:32.515 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T00:13:32.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:13:32.577 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:13:32.625 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:13:32.625 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:13:32.625 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:13:32.676 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T00:13:32.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:32.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:13:32.765 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:13:32.765 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:13:32.779 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:13:32.779 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:13:32.779 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:13:32.779 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with new device syntax: 1@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:13:32.779 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:13:32.786 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:13:32.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:32.787 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:13:32.806 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:13:32.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:13:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:13:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:13:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:13:32.904 INFO:teuthology.orchestra.run.smithi086.stdout:client5594 2024-04-25T00:13:32.904 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:13:32.904 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:32.913 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:13:32.913 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:13:33.004 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:13:33.092 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:13:33.092 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:13:33.092 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:13:33.092 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:33.101 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:13:33.102 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:33.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fe0c00640 1 Processor -- start 2024-04-25T00:13:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fe0c00640 1 -- start start 2024-04-25T00:13:33.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fe0c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6fdc0ce090 0x7f6fdc0b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fe0c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fdc0b0520 con 0x7f6fdc06b1e0 2024-04-25T00:13:33.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fe0c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fdc15e6f0 con 0x7f6fdc15f6e0 2024-04-25T00:13:33.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fe0c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6fdc15e9d0 con 0x7f6fdc0ce090 2024-04-25T00:13:33.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fdb400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6fdc06b1e0 0x7f6fdc06b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54914/0 (socket says 172.21.15.83:54914) 2024-04-25T00:13:33.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fdb400640 1 -- 172.21.15.83:0/1860720453 learned_addr learned my addr 172.21.15.83:0/1860720453 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:33.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 -- 172.21.15.83:0/1860720453 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1148265016 0 0) 0x7f6fdc0b0520 con 0x7f6fdc06b1e0 2024-04-25T00:13:33.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fdaa00640 1 --2- 172.21.15.83:0/1860720453 >> v2:172.21.15.86:3300/0 conn(0x7f6fdc0ce090 0x7f6fdc0b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 -- 172.21.15.83:0/1860720453 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fc0003530 con 0x7f6fdc06b1e0 2024-04-25T00:13:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 -- 172.21.15.83:0/1860720453 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3926028992 0 0) 0x7f6fc0003530 con 0x7f6fdc06b1e0 2024-04-25T00:13:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 -- 172.21.15.83:0/1860720453 >> v2:172.21.15.86:3300/0 conn(0x7f6fdc0ce090 msgr2=0x7f6fdc0b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 --2- 172.21.15.83:0/1860720453 >> v2:172.21.15.86:3300/0 conn(0x7f6fdc0ce090 0x7f6fdc0b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 -- 172.21.15.83:0/1860720453 >> v1:172.21.15.86:6789/0 conn(0x7f6fdc15f6e0 legacy=0x7f6fdc071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 -- 172.21.15.83:0/1860720453 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc15ecb0 con 0x7f6fdc06b1e0 2024-04-25T00:13:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 -- 172.21.15.83:0/1860720453 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f6fcc001a00 con 0x7f6fdc06b1e0 2024-04-25T00:13:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 -- 172.21.15.83:0/1860720453 >> v1:172.21.15.83:6789/0 conn(0x7f6fdc06b1e0 legacy=0x7f6fdc06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 --2- 172.21.15.83:0/1860720453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc0003a10 0x7f6fc0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 --2- 172.21.15.83:0/1860720453 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fc0004810 0x7f6fc0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 --2- 172.21.15.83:0/1860720453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fc0005220 0x7f6fc000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 -- 172.21.15.83:0/1860720453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc15e9d0 con 0x7f6fc0003a10 2024-04-25T00:13:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 -- 172.21.15.83:0/1860720453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc000c170 con 0x7f6fc0005220 2024-04-25T00:13:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.336+0000 7f6fda000640 1 -- 172.21.15.83:0/1860720453 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc000c370 con 0x7f6fc0004810 2024-04-25T00:13:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fdb400640 1 --2- 172.21.15.83:0/1860720453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc0003a10 0x7f6fc0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fdaa00640 1 --2- 172.21.15.83:0/1860720453 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fc0004810 0x7f6fc0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fdbe00640 1 --2- 172.21.15.83:0/1860720453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fc0005220 0x7f6fc000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fdb400640 1 -- 172.21.15.83:0/1860720453 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fc0004810 msgr2=0x7f6fc0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fdb400640 1 --2- 172.21.15.83:0/1860720453 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fc0004810 0x7f6fc0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fdb400640 1 -- 172.21.15.83:0/1860720453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fc0005220 msgr2=0x7f6fc000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fdb400640 1 --2- 172.21.15.83:0/1860720453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fc0005220 0x7f6fc000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fdb400640 1 -- 172.21.15.83:0/1860720453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc0003530 con 0x7f6fc0003a10 2024-04-25T00:13:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fdbe00640 1 -- 172.21.15.83:0/1860720453 reap_dead start 2024-04-25T00:13:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fdb400640 1 --2- 172.21.15.83:0/1860720453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc0003a10 0x7f6fc0003e30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00a760 tx=0x7f6fcc01d260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ed2bdd14021335 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fda000640 1 -- 172.21.15.83:0/1860720453 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc00a3b0 con 0x7f6fc0003a10 2024-04-25T00:13:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fda000640 1 -- 172.21.15.83:0/1860720453 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fcc0ad650 con 0x7f6fc0003a10 2024-04-25T00:13:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fda000640 1 -- 172.21.15.83:0/1860720453 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc0ada80 con 0x7f6fc0003a10 2024-04-25T00:13:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fe0c00640 1 -- 172.21.15.83:0/1860720453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc0003a10 msgr2=0x7f6fc0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.340+0000 7f6fe0c00640 1 --2- 172.21.15.83:0/1860720453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc0003a10 0x7f6fc0003e30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00a760 tx=0x7f6fcc01d260 comp rx=0 tx=0).stop 2024-04-25T00:13:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fe0c00640 1 -- 172.21.15.83:0/1860720453 shutdown_connections 2024-04-25T00:13:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fe0c00640 1 --2- 172.21.15.83:0/1860720453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc0003a10 0x7f6fc0003e30 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fe0c00640 1 -- 172.21.15.83:0/1860720453 >> 172.21.15.83:0/1860720453 conn(0x7f6fdc0cd500 msgr2=0x7f6fdc0cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:33.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fe0c00640 1 -- 172.21.15.83:0/1860720453 shutdown_connections 2024-04-25T00:13:33.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fe0c00640 1 -- 172.21.15.83:0/1860720453 wait complete. 2024-04-25T00:13:33.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fe0c00640 1 Processor -- start 2024-04-25T00:13:33.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fe0c00640 1 -- start start 2024-04-25T00:13:33.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fe0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fc0003a10 0x7f6fdc0aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fe0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fdc159a00 0x7f6fdc0c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fe0c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fdc15cdc0 0x7f6fdc15d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fe0c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc15fc70 con 0x7f6fdc159a00 2024-04-25T00:13:33.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fe0c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc15faf0 con 0x7f6fc0003a10 2024-04-25T00:13:33.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fe0c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6fdc0727d0 con 0x7f6fdc15cdc0 2024-04-25T00:13:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fdaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fdc159a00 0x7f6fdc0c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fdaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fdc159a00 0x7f6fdc0c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56980/0 (socket says 172.21.15.83:56980) 2024-04-25T00:13:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fdaa00640 1 -- 172.21.15.83:0/28900165 learned_addr learned my addr 172.21.15.83:0/28900165 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fdb400640 1 --2- 172.21.15.83:0/28900165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fc0003a10 0x7f6fdc0aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fdbe00640 1 --2- 172.21.15.83:0/28900165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fdc15cdc0 0x7f6fdc15d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fdaa00640 1 -- 172.21.15.83:0/28900165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fdc15cdc0 msgr2=0x7f6fdc15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fdaa00640 1 --2- 172.21.15.83:0/28900165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fdc15cdc0 0x7f6fdc15d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fdaa00640 1 -- 172.21.15.83:0/28900165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fc0003a10 msgr2=0x7f6fdc0aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fdaa00640 1 --2- 172.21.15.83:0/28900165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fc0003a10 0x7f6fdc0aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fdaa00640 1 -- 172.21.15.83:0/28900165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc0c74d0 con 0x7f6fdc159a00 2024-04-25T00:13:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fdaa00640 1 --2- 172.21.15.83:0/28900165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fdc159a00 0x7f6fdc0c6f90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80118a0 tx=0x7f6fc8011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fd3400640 1 -- 172.21.15.83:0/28900165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8019070 con 0x7f6fdc159a00 2024-04-25T00:13:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fd3400640 1 -- 172.21.15.83:0/28900165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc8004240 con 0x7f6fdc159a00 2024-04-25T00:13:33.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fd3400640 1 -- 172.21.15.83:0/28900165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc800f820 con 0x7f6fdc159a00 2024-04-25T00:13:33.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.344+0000 7f6fe0c00640 1 -- 172.21.15.83:0/28900165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc0af690 con 0x7f6fdc159a00 2024-04-25T00:13:33.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.348+0000 7f6fe0c00640 1 -- 172.21.15.83:0/28900165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc0afc50 con 0x7f6fdc159a00 2024-04-25T00:13:33.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.348+0000 7f6fe0c00640 1 -- 172.21.15.83:0/28900165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc0afe50 con 0x7f6fdc159a00 2024-04-25T00:13:33.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.348+0000 7f6fd3400640 1 -- 172.21.15.83:0/28900165 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6fc801c070 con 0x7f6fdc159a00 2024-04-25T00:13:33.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.348+0000 7f6fd3400640 1 --2- 172.21.15.83:0/28900165 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fbc0678f0 0x7f6fbc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.348+0000 7f6fd3400640 1 -- 172.21.15.83:0/28900165 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f6fc80860b0 con 0x7f6fdc159a00 2024-04-25T00:13:33.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.348+0000 7f6fdb400640 1 --2- 172.21.15.83:0/28900165 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fbc0678f0 0x7f6fbc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.348+0000 7f6fdb400640 1 --2- 172.21.15.83:0/28900165 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fbc0678f0 0x7f6fbc069db0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0ad000 tx=0x7f6fcc0af040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:33.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.352+0000 7f6fd3400640 1 -- 172.21.15.83:0/28900165 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6fdc0afe50 con 0x7f6fdc159a00 2024-04-25T00:13:33.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.512+0000 7f6fe0c00640 1 -- 172.21.15.83:0/28900165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fdc04f510 con 0x7f6fdc159a00 2024-04-25T00:13:33.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.512+0000 7f6fd3400640 1 -- 172.21.15.83:0/28900165 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16800 (secure 0 0 0) 0x7f6fdc0c74d0 con 0x7f6fdc159a00 2024-04-25T00:13:33.515 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:33.515 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":60,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:21.438568+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:18.160239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:18.800529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:33.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.512+0000 7f6fe0c00640 1 -- 172.21.15.83:0/28900165 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fbc0678f0 msgr2=0x7f6fbc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.512+0000 7f6fe0c00640 1 --2- 172.21.15.83:0/28900165 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fbc0678f0 0x7f6fbc069db0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0ad000 tx=0x7f6fcc0af040 comp rx=0 tx=0).stop 2024-04-25T00:13:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.512+0000 7f6fe0c00640 1 -- 172.21.15.83:0/28900165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fdc159a00 msgr2=0x7f6fdc0c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.512+0000 7f6fe0c00640 1 --2- 172.21.15.83:0/28900165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fdc159a00 0x7f6fdc0c6f90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80118a0 tx=0x7f6fc8011d60 comp rx=0 tx=0).stop 2024-04-25T00:13:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.512+0000 7f6fe0c00640 1 -- 172.21.15.83:0/28900165 shutdown_connections 2024-04-25T00:13:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.512+0000 7f6fe0c00640 1 --2- 172.21.15.83:0/28900165 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6fbc0678f0 0x7f6fbc069db0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.512+0000 7f6fe0c00640 1 --2- 172.21.15.83:0/28900165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fdc15cdc0 0x7f6fdc15d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.512+0000 7f6fe0c00640 1 --2- 172.21.15.83:0/28900165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fdc159a00 0x7f6fdc0c6f90 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.512+0000 7f6fe0c00640 1 --2- 172.21.15.83:0/28900165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fc0003a10 0x7f6fdc0aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.512+0000 7f6fe0c00640 1 -- 172.21.15.83:0/28900165 >> 172.21.15.83:0/28900165 conn(0x7f6fdc0cd500 msgr2=0x7f6fdc15b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:33.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.512+0000 7f6fe0c00640 1 -- 172.21.15.83:0/28900165 shutdown_connections 2024-04-25T00:13:33.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.512+0000 7f6fe0c00640 1 -- 172.21.15.83:0/28900165 wait complete. 2024-04-25T00:13:33.536 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:18.160239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:13:18.800529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:13:33.536 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:13:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.672+0000 7fe626e00640 1 Processor -- start 2024-04-25T00:13:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.672+0000 7fe626e00640 1 -- start start 2024-04-25T00:13:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.672+0000 7fe626e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe62015de90 0x7fe620071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.672+0000 7fe626e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe620162a10 con 0x7fe62006e840 2024-04-25T00:13:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.672+0000 7fe626e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe62006dcf0 con 0x7fe62015e270 2024-04-25T00:13:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.672+0000 7fe626e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe6200b4460 con 0x7fe62015de90 2024-04-25T00:13:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.672+0000 7fe61fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe62015de90 0x7fe620071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.672+0000 7fe61fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe62015de90 0x7fe620071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56988/0 (socket says 172.21.15.83:56988) 2024-04-25T00:13:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.672+0000 7fe624e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe62006e840 0x7fe620162300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54924/0 (socket says 172.21.15.83:54924) 2024-04-25T00:13:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.672+0000 7fe61fe00640 1 -- 172.21.15.83:0/1729522133 learned_addr learned my addr 172.21.15.83:0/1729522133 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:33.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.672+0000 7fe61fe00640 1 -- 172.21.15.83:0/1729522133 >> v1:172.21.15.86:6789/0 conn(0x7fe62015e270 legacy=0x7fe620161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.672+0000 7fe61fe00640 1 -- 172.21.15.83:0/1729522133 >> v1:172.21.15.83:6789/0 conn(0x7fe62006e840 legacy=0x7fe620162300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.672+0000 7fe61fe00640 1 -- 172.21.15.83:0/1729522133 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62006dfd0 con 0x7fe62015de90 2024-04-25T00:13:33.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.672+0000 7fe61ea00640 1 -- 172.21.15.83:0/1729522133 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2248925807 0 0) 0x7fe620162a10 con 0x7fe62006e840 2024-04-25T00:13:33.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.676+0000 7fe61fe00640 1 --2- 172.21.15.83:0/1729522133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe62015de90 0x7fe620071740 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe610009870 tx=0x7fe61001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304dbc203dbd7618 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.676+0000 7fe61ea00640 1 -- 172.21.15.83:0/1729522133 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6100b0070 con 0x7fe62015de90 2024-04-25T00:13:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.676+0000 7fe61ea00640 1 -- 172.21.15.83:0/1729522133 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6100ab600 con 0x7fe62015de90 2024-04-25T00:13:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.676+0000 7fe61ea00640 1 -- 172.21.15.83:0/1729522133 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6100ab9d0 con 0x7fe62015de90 2024-04-25T00:13:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.676+0000 7fe626e00640 1 -- 172.21.15.83:0/1729522133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe62015de90 msgr2=0x7fe620071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.676+0000 7fe626e00640 1 --2- 172.21.15.83:0/1729522133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe62015de90 0x7fe620071740 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe610009870 tx=0x7fe61001c4d0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.676+0000 7fe626e00640 1 -- 172.21.15.83:0/1729522133 shutdown_connections 2024-04-25T00:13:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.676+0000 7fe626e00640 1 --2- 172.21.15.83:0/1729522133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe62015de90 0x7fe620071740 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.676+0000 7fe626e00640 1 -- 172.21.15.83:0/1729522133 >> 172.21.15.83:0/1729522133 conn(0x7fe6200b1830 msgr2=0x7fe6200b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.676+0000 7fe626e00640 1 -- 172.21.15.83:0/1729522133 shutdown_connections 2024-04-25T00:13:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.676+0000 7fe626e00640 1 -- 172.21.15.83:0/1729522133 wait complete. 2024-04-25T00:13:33.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.676+0000 7fe626e00640 1 Processor -- start 2024-04-25T00:13:33.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.676+0000 7fe626e00640 1 -- start start 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe626e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62006e840 0x7fe62014f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe626e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe62015de90 0x7fe62014b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe626e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe62015e270 0x7fe620149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe626e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6200ad860 con 0x7fe62015e270 2024-04-25T00:13:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe626e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe620072090 con 0x7fe62006e840 2024-04-25T00:13:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe626e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe62016c580 con 0x7fe62015de90 2024-04-25T00:13:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe624e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe62015e270 0x7fe620149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe624e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe62015e270 0x7fe620149120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57000/0 (socket says 172.21.15.83:57000) 2024-04-25T00:13:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe624e00640 1 -- 172.21.15.83:0/2180284692 learned_addr learned my addr 172.21.15.83:0/2180284692 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe61fe00640 1 --2- 172.21.15.83:0/2180284692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62006e840 0x7fe62014f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe61f400640 1 --2- 172.21.15.83:0/2180284692 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe62015de90 0x7fe62014b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe624e00640 1 -- 172.21.15.83:0/2180284692 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe62015de90 msgr2=0x7fe62014b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe624e00640 1 --2- 172.21.15.83:0/2180284692 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe62015de90 0x7fe62014b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe624e00640 1 -- 172.21.15.83:0/2180284692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62006e840 msgr2=0x7fe62014f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe624e00640 1 --2- 172.21.15.83:0/2180284692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62006e840 0x7fe62014f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe624e00640 1 -- 172.21.15.83:0/2180284692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62014e3e0 con 0x7fe62015e270 2024-04-25T00:13:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe624e00640 1 --2- 172.21.15.83:0/2180284692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe62015e270 0x7fe620149120 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe61800c700 tx=0x7fe61800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe617e00640 1 -- 172.21.15.83:0/2180284692 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618018070 con 0x7fe62015e270 2024-04-25T00:13:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe617e00640 1 -- 172.21.15.83:0/2180284692 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe61800ce10 con 0x7fe62015e270 2024-04-25T00:13:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe617e00640 1 -- 172.21.15.83:0/2180284692 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61800a860 con 0x7fe62015e270 2024-04-25T00:13:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe626e00640 1 -- 172.21.15.83:0/2180284692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62014e6c0 con 0x7fe62015e270 2024-04-25T00:13:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe626e00640 1 -- 172.21.15.83:0/2180284692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62014ec80 con 0x7fe62015e270 2024-04-25T00:13:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe626e00640 1 -- 172.21.15.83:0/2180284692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe62004f510 con 0x7fe62015e270 2024-04-25T00:13:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.680+0000 7fe617e00640 1 -- 172.21.15.83:0/2180284692 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe61800aa00 con 0x7fe62015e270 2024-04-25T00:13:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.684+0000 7fe617e00640 1 --2- 172.21.15.83:0/2180284692 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6000678c0 0x7fe600069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.684+0000 7fe617e00640 1 -- 172.21.15.83:0/2180284692 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fe618085420 con 0x7fe62015e270 2024-04-25T00:13:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.684+0000 7fe61fe00640 1 --2- 172.21.15.83:0/2180284692 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6000678c0 0x7fe600069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.684+0000 7fe61fe00640 1 --2- 172.21.15.83:0/2180284692 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6000678c0 0x7fe600069d80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe6100ad000 tx=0x7fe6100a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:33.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.688+0000 7fe617e00640 1 -- 172.21.15.83:0/2180284692 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe6180556e0 con 0x7fe62015e270 2024-04-25T00:13:33.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.844+0000 7fe626e00640 1 -- 172.21.15.83:0/2180284692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe620151b10 con 0x7fe62015e270 2024-04-25T00:13:33.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.844+0000 7fe617e00640 1 -- 172.21.15.83:0/2180284692 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) ==== 93+0+10 (secure 0 0 0) 0x7fe6180599e0 con 0x7fe62015e270 2024-04-25T00:13:33.847 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:13:33.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.844+0000 7fe626e00640 1 -- 172.21.15.83:0/2180284692 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6000678c0 msgr2=0x7fe600069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.844+0000 7fe626e00640 1 --2- 172.21.15.83:0/2180284692 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6000678c0 0x7fe600069d80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe6100ad000 tx=0x7fe6100a6000 comp rx=0 tx=0).stop 2024-04-25T00:13:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.844+0000 7fe626e00640 1 -- 172.21.15.83:0/2180284692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe62015e270 msgr2=0x7fe620149120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.844+0000 7fe626e00640 1 --2- 172.21.15.83:0/2180284692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe62015e270 0x7fe620149120 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe61800c700 tx=0x7fe61800cbc0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.848+0000 7fe626e00640 1 -- 172.21.15.83:0/2180284692 shutdown_connections 2024-04-25T00:13:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.848+0000 7fe626e00640 1 --2- 172.21.15.83:0/2180284692 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6000678c0 0x7fe600069d80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.848+0000 7fe626e00640 1 --2- 172.21.15.83:0/2180284692 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe62015de90 0x7fe62014b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.848+0000 7fe626e00640 1 --2- 172.21.15.83:0/2180284692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe62015e270 0x7fe620149120 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.848+0000 7fe626e00640 1 --2- 172.21.15.83:0/2180284692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62006e840 0x7fe62014f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.848+0000 7fe626e00640 1 -- 172.21.15.83:0/2180284692 >> 172.21.15.83:0/2180284692 conn(0x7fe6200b1830 msgr2=0x7fe620071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.848+0000 7fe626e00640 1 -- 172.21.15.83:0/2180284692 shutdown_connections 2024-04-25T00:13:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:33.848+0000 7fe626e00640 1 -- 172.21.15.83:0/2180284692 wait complete. 2024-04-25T00:13:33.868 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:13:34.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3be00640 1 Processor -- start 2024-04-25T00:13:34.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3be00640 1 -- start start 2024-04-25T00:13:34.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3c0727b0 0x7fec3c15f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fec3c072c10 0x7fec3c15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec3c070ca0 con 0x7fec3c1455d0 2024-04-25T00:13:34.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fec3c0ce9d0 con 0x7fec3c072c10 2024-04-25T00:13:34.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fec3c070fa0 con 0x7fec3c0727b0 2024-04-25T00:13:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fec3c072c10 0x7fec3c15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3c0727b0 0x7fec3c15f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3c0727b0 0x7fec3c15f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:53236/0 (socket says 172.21.15.83:53236) 2024-04-25T00:13:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fec3c072c10 0x7fec3c15fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:60724/0 (socket says 172.21.15.83:60724) 2024-04-25T00:13:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3a000640 1 -- 172.21.15.83:0/4204483840 learned_addr learned my addr 172.21.15.83:0/4204483840 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec39600640 1 -- 172.21.15.83:0/4204483840 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1966439029 0 0) 0x7fec3c070ca0 con 0x7fec3c1455d0 2024-04-25T00:13:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec39600640 1 -- 172.21.15.83:0/4204483840 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec20003530 con 0x7fec3c1455d0 2024-04-25T00:13:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3b400640 1 -- 172.21.15.83:0/4204483840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3c0727b0 msgr2=0x7fec3c15f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:34.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3b400640 1 --2- 172.21.15.83:0/4204483840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3c0727b0 0x7fec3c15f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3b400640 1 -- 172.21.15.83:0/4204483840 >> v1:172.21.15.86:6789/0 conn(0x7fec3c1455d0 legacy=0x7fec3c06cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3b400640 1 -- 172.21.15.83:0/4204483840 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c1605d0 con 0x7fec3c072c10 2024-04-25T00:13:34.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.004+0000 7fec3a000640 1 --2- 172.21.15.83:0/4204483840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3c0727b0 0x7fec3c15f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:34.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.008+0000 7fec3b400640 1 --2- 172.21.15.83:0/4204483840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3c072c10 0x7fec3c15fd40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fec34009d90 tx=0x7fec3401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88d22a57986d5bae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.008+0000 7fec39600640 1 -- 172.21.15.83:0/4204483840 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec340b2070 con 0x7fec3c072c10 2024-04-25T00:13:34.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.008+0000 7fec39600640 1 -- 172.21.15.83:0/4204483840 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec340a5920 con 0x7fec3c072c10 2024-04-25T00:13:34.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.008+0000 7fec39600640 1 -- 172.21.15.83:0/4204483840 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec340a5cd0 con 0x7fec3c072c10 2024-04-25T00:13:34.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.008+0000 7fec3be00640 1 -- 172.21.15.83:0/4204483840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3c072c10 msgr2=0x7fec3c15fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.008+0000 7fec3be00640 1 --2- 172.21.15.83:0/4204483840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3c072c10 0x7fec3c15fd40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fec34009d90 tx=0x7fec3401c970 comp rx=0 tx=0).stop 2024-04-25T00:13:34.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.008+0000 7fec3be00640 1 -- 172.21.15.83:0/4204483840 shutdown_connections 2024-04-25T00:13:34.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.008+0000 7fec3be00640 1 --2- 172.21.15.83:0/4204483840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3c072c10 0x7fec3c15fd40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.008+0000 7fec3be00640 1 --2- 172.21.15.83:0/4204483840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3c0727b0 0x7fec3c15f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.008+0000 7fec3be00640 1 -- 172.21.15.83:0/4204483840 >> 172.21.15.83:0/4204483840 conn(0x7fec3c0cf980 msgr2=0x7fec3c0cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.008+0000 7fec3be00640 1 -- 172.21.15.83:0/4204483840 shutdown_connections 2024-04-25T00:13:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.008+0000 7fec3be00640 1 -- 172.21.15.83:0/4204483840 wait complete. 2024-04-25T00:13:34.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3be00640 1 Processor -- start 2024-04-25T00:13:34.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3be00640 1 -- start start 2024-04-25T00:13:34.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec3c0727b0 0x7fec3c137040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3c072c10 0x7fec3c135900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3c1455d0 0x7fec3c1593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c071da0 con 0x7fec3c0727b0 2024-04-25T00:13:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c071c20 con 0x7fec3c1455d0 2024-04-25T00:13:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fec3c071f20 con 0x7fec3c072c10 2024-04-25T00:13:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec3c0727b0 0x7fec3c137040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec3c0727b0 0x7fec3c137040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57004/0 (socket says 172.21.15.83:57004) 2024-04-25T00:13:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3aa00640 1 -- 172.21.15.83:0/1067998116 learned_addr learned my addr 172.21.15.83:0/1067998116 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3a000640 1 --2- 172.21.15.83:0/1067998116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3c072c10 0x7fec3c135900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3b400640 1 --2- 172.21.15.83:0/1067998116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3c1455d0 0x7fec3c1593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3aa00640 1 -- 172.21.15.83:0/1067998116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3c072c10 msgr2=0x7fec3c135900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3aa00640 1 --2- 172.21.15.83:0/1067998116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3c072c10 0x7fec3c135900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3aa00640 1 -- 172.21.15.83:0/1067998116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3c1455d0 msgr2=0x7fec3c1593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3aa00640 1 --2- 172.21.15.83:0/1067998116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3c1455d0 0x7fec3c1593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3aa00640 1 -- 172.21.15.83:0/1067998116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c137580 con 0x7fec3c0727b0 2024-04-25T00:13:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3aa00640 1 --2- 172.21.15.83:0/1067998116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec3c0727b0 0x7fec3c137040 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00c500 tx=0x7fec2c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec32a00640 1 -- 172.21.15.83:0/1067998116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0052d0 con 0x7fec3c0727b0 2024-04-25T00:13:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec32a00640 1 -- 172.21.15.83:0/1067998116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2c005490 con 0x7fec3c0727b0 2024-04-25T00:13:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3be00640 1 -- 172.21.15.83:0/1067998116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3c13e180 con 0x7fec3c0727b0 2024-04-25T00:13:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec32a00640 1 -- 172.21.15.83:0/1067998116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c012850 con 0x7fec3c0727b0 2024-04-25T00:13:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3be00640 1 -- 172.21.15.83:0/1067998116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3c136840 con 0x7fec3c0727b0 2024-04-25T00:13:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.012+0000 7fec3be00640 1 -- 172.21.15.83:0/1067998116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3c136370 con 0x7fec3c0727b0 2024-04-25T00:13:34.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.016+0000 7fec32a00640 1 -- 172.21.15.83:0/1067998116 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fec2c0129f0 con 0x7fec3c0727b0 2024-04-25T00:13:34.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.016+0000 7fec32a00640 1 --2- 172.21.15.83:0/1067998116 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec1c0678c0 0x7fec1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.016+0000 7fec32a00640 1 -- 172.21.15.83:0/1067998116 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fec2c085b90 con 0x7fec3c0727b0 2024-04-25T00:13:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.016+0000 7fec3a000640 1 --2- 172.21.15.83:0/1067998116 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec1c0678c0 0x7fec1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.016+0000 7fec3a000640 1 --2- 172.21.15.83:0/1067998116 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec1c0678c0 0x7fec1c069d80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0b0de0 tx=0x7fec28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.020+0000 7fec32a00640 1 -- 172.21.15.83:0/1067998116 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fec2c0564b0 con 0x7fec3c0727b0 2024-04-25T00:13:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.180+0000 7fec3be00640 1 -- 172.21.15.83:0/1067998116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fec3c06e8e0 con 0x7fec3c0727b0 2024-04-25T00:13:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.180+0000 7fec32a00640 1 -- 172.21.15.83:0/1067998116 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) ==== 104+0+11 (secure 0 0 0) 0x7fec2c055db0 con 0x7fec3c0727b0 2024-04-25T00:13:34.188 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:13:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.180+0000 7fec3be00640 1 -- 172.21.15.83:0/1067998116 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec1c0678c0 msgr2=0x7fec1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.180+0000 7fec3be00640 1 --2- 172.21.15.83:0/1067998116 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec1c0678c0 0x7fec1c069d80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0b0de0 tx=0x7fec28008040 comp rx=0 tx=0).stop 2024-04-25T00:13:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.180+0000 7fec3be00640 1 -- 172.21.15.83:0/1067998116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec3c0727b0 msgr2=0x7fec3c137040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.180+0000 7fec3be00640 1 --2- 172.21.15.83:0/1067998116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec3c0727b0 0x7fec3c137040 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00c500 tx=0x7fec2c00c9c0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.180+0000 7fec3be00640 1 -- 172.21.15.83:0/1067998116 shutdown_connections 2024-04-25T00:13:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.180+0000 7fec3be00640 1 --2- 172.21.15.83:0/1067998116 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec1c0678c0 0x7fec1c069d80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.180+0000 7fec3be00640 1 --2- 172.21.15.83:0/1067998116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3c072c10 0x7fec3c135900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.180+0000 7fec3be00640 1 --2- 172.21.15.83:0/1067998116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3c1455d0 0x7fec3c1593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.180+0000 7fec3be00640 1 --2- 172.21.15.83:0/1067998116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec3c0727b0 0x7fec3c137040 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.180+0000 7fec3be00640 1 -- 172.21.15.83:0/1067998116 >> 172.21.15.83:0/1067998116 conn(0x7fec3c0cf980 msgr2=0x7fec3c0afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:34.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.180+0000 7fec3be00640 1 -- 172.21.15.83:0/1067998116 shutdown_connections 2024-04-25T00:13:34.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.180+0000 7fec3be00640 1 -- 172.21.15.83:0/1067998116 wait complete. 2024-04-25T00:13:34.204 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:13:34.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:13:34.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:13:34.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.340+0000 7f3294e00640 1 Processor -- start 2024-04-25T00:13:34.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.340+0000 7f3294e00640 1 -- start start 2024-04-25T00:13:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.340+0000 7f3294e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f329006e950 0x7f329015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.340+0000 7f3294e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32900ce000 con 0x7f329015fe50 2024-04-25T00:13:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.340+0000 7f3294e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f329006eec0 con 0x7f3290159810 2024-04-25T00:13:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.340+0000 7f3294e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f329006e210 con 0x7f329006e950 2024-04-25T00:13:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.344+0000 7f3285400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f329006e950 0x7f329015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.344+0000 7f3285400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f329006e950 0x7f329015f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34762/0 (socket says 172.21.15.83:34762) 2024-04-25T00:13:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.344+0000 7f3285400640 1 -- 172.21.15.83:0/1081486721 learned_addr learned my addr 172.21.15.83:0/1081486721 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.344+0000 7f328e800640 1 --1- 172.21.15.83:0/1081486721 >> v1:172.21.15.83:6789/0 conn(0x7f329015fe50 0x7f32900ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40618/0 (socket says 172.21.15.83:40618) 2024-04-25T00:13:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.344+0000 7f3285400640 1 -- 172.21.15.83:0/1081486721 >> v1:172.21.15.86:6789/0 conn(0x7f3290159810 legacy=0x7f32900cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.344+0000 7f3285400640 1 -- 172.21.15.83:0/1081486721 >> v1:172.21.15.83:6789/0 conn(0x7f329015fe50 legacy=0x7f32900ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.344+0000 7f3285400640 1 -- 172.21.15.83:0/1081486721 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329006dc50 con 0x7f329006e950 2024-04-25T00:13:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.344+0000 7f328d400640 1 -- 172.21.15.83:0/1081486721 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3225852405 0 0) 0x7f32900ce000 con 0x7f329015fe50 2024-04-25T00:13:34.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.344+0000 7f3285400640 1 --2- 172.21.15.83:0/1081486721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f329006e950 0x7f329015f910 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3278009870 tx=0x7f327801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f172477345f233 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.344+0000 7f328d400640 1 -- 172.21.15.83:0/1081486721 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32780b0070 con 0x7f329006e950 2024-04-25T00:13:34.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.344+0000 7f328d400640 1 -- 172.21.15.83:0/1081486721 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32780ab600 con 0x7f329006e950 2024-04-25T00:13:34.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.344+0000 7f328d400640 1 -- 172.21.15.83:0/1081486721 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32780ab9d0 con 0x7f329006e950 2024-04-25T00:13:34.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 -- 172.21.15.83:0/1081486721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f329006e950 msgr2=0x7f329015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 --2- 172.21.15.83:0/1081486721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f329006e950 0x7f329015f910 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3278009870 tx=0x7f327801c4d0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 -- 172.21.15.83:0/1081486721 shutdown_connections 2024-04-25T00:13:34.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 --2- 172.21.15.83:0/1081486721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f329006e950 0x7f329015f910 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 -- 172.21.15.83:0/1081486721 >> 172.21.15.83:0/1081486721 conn(0x7f32900b1830 msgr2=0x7f32900b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:34.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 -- 172.21.15.83:0/1081486721 shutdown_connections 2024-04-25T00:13:34.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 -- 172.21.15.83:0/1081486721 wait complete. 2024-04-25T00:13:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 Processor -- start 2024-04-25T00:13:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 -- start start 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f329006e950 0x7f32901510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3290159810 0x7f3290159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329015fe50 0x7f3290141330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3290169e40 con 0x7f329006e950 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3290169cc0 con 0x7f329015fe50 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3294e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3290169fc0 con 0x7f3290159810 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f328de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f329006e950 0x7f32901510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f328de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f329006e950 0x7f32901510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34776/0 (socket says 172.21.15.83:34776) 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f328de00640 1 -- 172.21.15.83:0/3730658149 learned_addr learned my addr 172.21.15.83:0/3730658149 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f328e800640 1 --2- 172.21.15.83:0/3730658149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329015fe50 0x7f3290141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f3285400640 1 --2- 172.21.15.83:0/3730658149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3290159810 0x7f3290159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f328de00640 1 -- 172.21.15.83:0/3730658149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3290159810 msgr2=0x7f3290159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f328de00640 1 --2- 172.21.15.83:0/3730658149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3290159810 0x7f3290159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f328de00640 1 -- 172.21.15.83:0/3730658149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329015fe50 msgr2=0x7f3290141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f328de00640 1 --2- 172.21.15.83:0/3730658149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329015fe50 0x7f3290141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.348+0000 7f328de00640 1 -- 172.21.15.83:0/3730658149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329015a340 con 0x7f329006e950 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.352+0000 7f328de00640 1 --2- 172.21.15.83:0/3730658149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f329006e950 0x7f32901510e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f328000c3c0 tx=0x7f328000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.352+0000 7f3286a00640 1 -- 172.21.15.83:0/3730658149 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280017070 con 0x7f329006e950 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.352+0000 7f3286a00640 1 -- 172.21.15.83:0/3730658149 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32800052d0 con 0x7f329006e950 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.352+0000 7f3286a00640 1 -- 172.21.15.83:0/3730658149 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280003840 con 0x7f329006e950 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.352+0000 7f3294e00640 1 -- 172.21.15.83:0/3730658149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329015b310 con 0x7f329006e950 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.352+0000 7f3294e00640 1 -- 172.21.15.83:0/3730658149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329015b8d0 con 0x7f329006e950 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.352+0000 7f3294e00640 1 -- 172.21.15.83:0/3730658149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329004f510 con 0x7f329006e950 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.352+0000 7f3286a00640 1 -- 172.21.15.83:0/3730658149 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f328001a070 con 0x7f329006e950 2024-04-25T00:13:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.352+0000 7f3286a00640 1 --2- 172.21.15.83:0/3730658149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f326c0678c0 0x7f326c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.352+0000 7f3286a00640 1 -- 172.21.15.83:0/3730658149 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f32800850b0 con 0x7f329006e950 2024-04-25T00:13:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.356+0000 7f3285400640 1 --2- 172.21.15.83:0/3730658149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f326c0678c0 0x7f326c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.356+0000 7f3285400640 1 --2- 172.21.15.83:0/3730658149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f326c0678c0 0x7f326c069d80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f32780ad000 tx=0x7f32780a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.360+0000 7f3286a00640 1 -- 172.21.15.83:0/3730658149 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3280005480 con 0x7f329006e950 2024-04-25T00:13:34.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.516+0000 7f3294e00640 1 -- 172.21.15.83:0/3730658149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f32900ae310 con 0x7f329006e950 2024-04-25T00:13:34.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.516+0000 7f3286a00640 1 -- 172.21.15.83:0/3730658149 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) ==== 100+0+11 (secure 0 0 0) 0x7f3280055410 con 0x7f329006e950 2024-04-25T00:13:34.520 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:13:34.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.516+0000 7f3294e00640 1 -- 172.21.15.83:0/3730658149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f326c0678c0 msgr2=0x7f326c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.516+0000 7f3294e00640 1 --2- 172.21.15.83:0/3730658149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f326c0678c0 0x7f326c069d80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f32780ad000 tx=0x7f32780a6000 comp rx=0 tx=0).stop 2024-04-25T00:13:34.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.516+0000 7f3294e00640 1 -- 172.21.15.83:0/3730658149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f329006e950 msgr2=0x7f32901510e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.516+0000 7f3294e00640 1 --2- 172.21.15.83:0/3730658149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f329006e950 0x7f32901510e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f328000c3c0 tx=0x7f328000c880 comp rx=0 tx=0).stop 2024-04-25T00:13:34.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.520+0000 7f3294e00640 1 -- 172.21.15.83:0/3730658149 shutdown_connections 2024-04-25T00:13:34.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.520+0000 7f3294e00640 1 --2- 172.21.15.83:0/3730658149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f326c0678c0 0x7f326c069d80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.520+0000 7f3294e00640 1 --2- 172.21.15.83:0/3730658149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3290159810 0x7f3290159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.520+0000 7f3294e00640 1 --2- 172.21.15.83:0/3730658149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329015fe50 0x7f3290141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.520+0000 7f3294e00640 1 --2- 172.21.15.83:0/3730658149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f329006e950 0x7f32901510e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.520+0000 7f3294e00640 1 -- 172.21.15.83:0/3730658149 >> 172.21.15.83:0/3730658149 conn(0x7f32900b1830 msgr2=0x7f3290153db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:34.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.520+0000 7f3294e00640 1 -- 172.21.15.83:0/3730658149 shutdown_connections 2024-04-25T00:13:34.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.520+0000 7f3294e00640 1 -- 172.21.15.83:0/3730658149 wait complete. 2024-04-25T00:13:34.541 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:13:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.716+0000 7f2eeb000640 1 Processor -- start 2024-04-25T00:13:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.716+0000 7f2eeb000640 1 -- start start 2024-04-25T00:13:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.716+0000 7f2eeb000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ee406ee20 0x7f2ee40cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.716+0000 7f2eeb000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2ee415f350 0x7f2ee4071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.716+0000 7f2eeb000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ee40cb210 con 0x7f2ee41446e0 2024-04-25T00:13:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.716+0000 7f2eeb000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2ee40715b0 con 0x7f2ee406ee20 2024-04-25T00:13:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.716+0000 7f2eeb000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2ee4144ac0 con 0x7f2ee415f350 2024-04-25T00:13:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.716+0000 7f2edb400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ee406ee20 0x7f2ee40cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.716+0000 7f2edb400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ee406ee20 0x7f2ee40cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34782/0 (socket says 172.21.15.83:34782) 2024-04-25T00:13:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.716+0000 7f2edb400640 1 -- 172.21.15.83:0/1487116196 learned_addr learned my addr 172.21.15.83:0/1487116196 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:34.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.716+0000 7f2ee9000640 1 --2- 172.21.15.83:0/1487116196 >> v2:172.21.15.86:3300/0 conn(0x7f2ee415f350 0x7f2ee4071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2edb400640 1 -- 172.21.15.83:0/1487116196 >> v2:172.21.15.86:3300/0 conn(0x7f2ee415f350 msgr2=0x7f2ee4071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:34.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2edb400640 1 --2- 172.21.15.83:0/1487116196 >> v2:172.21.15.86:3300/0 conn(0x7f2ee415f350 0x7f2ee4071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2edb400640 1 -- 172.21.15.83:0/1487116196 >> v1:172.21.15.86:6789/0 conn(0x7f2ee41446e0 legacy=0x7f2ee406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:34.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2edb400640 1 -- 172.21.15.83:0/1487116196 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee40cb4f0 con 0x7f2ee406ee20 2024-04-25T00:13:34.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2ee9000640 1 --2- 172.21.15.83:0/1487116196 >> v2:172.21.15.86:3300/0 conn(0x7f2ee415f350 0x7f2ee4071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2edb400640 1 --2- 172.21.15.83:0/1487116196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee406ee20 0x7f2ee40cab50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40af650 tx=0x7f2ecc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f645cfdee993d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2ee3400640 1 -- 172.21.15.83:0/1487116196 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0ab5f0 con 0x7f2ee406ee20 2024-04-25T00:13:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2ee3400640 1 -- 172.21.15.83:0/1487116196 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ecc0b0070 con 0x7f2ee406ee20 2024-04-25T00:13:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2ee3400640 1 -- 172.21.15.83:0/1487116196 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0ab9b0 con 0x7f2ee406ee20 2024-04-25T00:13:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2eeb000640 1 -- 172.21.15.83:0/1487116196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee406ee20 msgr2=0x7f2ee40cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2eeb000640 1 --2- 172.21.15.83:0/1487116196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee406ee20 0x7f2ee40cab50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40af650 tx=0x7f2ecc01c4e0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2eeb000640 1 -- 172.21.15.83:0/1487116196 shutdown_connections 2024-04-25T00:13:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2eeb000640 1 --2- 172.21.15.83:0/1487116196 >> v2:172.21.15.86:3300/0 conn(0x7f2ee415f350 0x7f2ee4071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2eeb000640 1 --2- 172.21.15.83:0/1487116196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee406ee20 0x7f2ee40cab50 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2eeb000640 1 -- 172.21.15.83:0/1487116196 >> 172.21.15.83:0/1487116196 conn(0x7f2ee40cfa60 msgr2=0x7f2ee40cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2eeb000640 1 -- 172.21.15.83:0/1487116196 shutdown_connections 2024-04-25T00:13:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.720+0000 7f2eeb000640 1 -- 172.21.15.83:0/1487116196 wait complete. 2024-04-25T00:13:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2eeb000640 1 Processor -- start 2024-04-25T00:13:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2eeb000640 1 -- start start 2024-04-25T00:13:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2eeb000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ee406ee20 0x7f2ee40b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2eeb000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee41446e0 0x7f2ee413e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2eeb000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ee415f350 0x7f2ee415e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2eeb000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4072eb0 con 0x7f2ee41446e0 2024-04-25T00:13:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2eeb000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4072d30 con 0x7f2ee406ee20 2024-04-25T00:13:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2eeb000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee4073030 con 0x7f2ee415f350 2024-04-25T00:13:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2edb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee41446e0 0x7f2ee413e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2edb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee41446e0 0x7f2ee413e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34784/0 (socket says 172.21.15.83:34784) 2024-04-25T00:13:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ee406ee20 0x7f2ee40b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2edb400640 1 -- 172.21.15.83:0/1903530056 learned_addr learned my addr 172.21.15.83:0/1903530056 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2ee9000640 1 --2- 172.21.15.83:0/1903530056 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ee415f350 0x7f2ee415e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2edb400640 1 -- 172.21.15.83:0/1903530056 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ee415f350 msgr2=0x7f2ee415e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2edb400640 1 --2- 172.21.15.83:0/1903530056 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ee415f350 0x7f2ee415e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2edb400640 1 -- 172.21.15.83:0/1903530056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ee406ee20 msgr2=0x7f2ee40b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2edb400640 1 --2- 172.21.15.83:0/1903530056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ee406ee20 0x7f2ee40b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2edb400640 1 -- 172.21.15.83:0/1903530056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee4141540 con 0x7f2ee41446e0 2024-04-25T00:13:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2edb400640 1 --2- 172.21.15.83:0/1903530056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee41446e0 0x7f2ee413e7b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0ad040 tx=0x7f2ecc01ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2ee0c00640 1 -- 172.21.15.83:0/1903530056 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0be5f0 con 0x7f2ee41446e0 2024-04-25T00:13:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2ee0c00640 1 -- 172.21.15.83:0/1903530056 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ecc0be7b0 con 0x7f2ee41446e0 2024-04-25T00:13:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2eeb000640 1 -- 172.21.15.83:0/1903530056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee4159400 con 0x7f2ee41446e0 2024-04-25T00:13:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.724+0000 7f2ee0c00640 1 -- 172.21.15.83:0/1903530056 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0beba0 con 0x7f2ee41446e0 2024-04-25T00:13:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.728+0000 7f2eeb000640 1 -- 172.21.15.83:0/1903530056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee41578e0 con 0x7f2ee41446e0 2024-04-25T00:13:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.728+0000 7f2eeb000640 1 -- 172.21.15.83:0/1903530056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee4068230 con 0x7f2ee41446e0 2024-04-25T00:13:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.728+0000 7f2ee0c00640 1 -- 172.21.15.83:0/1903530056 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2ecc0b43f0 con 0x7f2ee41446e0 2024-04-25T00:13:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.728+0000 7f2ee0c00640 1 --2- 172.21.15.83:0/1903530056 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2ec00678c0 0x7f2ec0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.728+0000 7f2ee0c00640 1 -- 172.21.15.83:0/1903530056 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f2ecc120680 con 0x7f2ee41446e0 2024-04-25T00:13:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.728+0000 7f2ee3e00640 1 --2- 172.21.15.83:0/1903530056 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2ec00678c0 0x7f2ec0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.728+0000 7f2ee3e00640 1 --2- 172.21.15.83:0/1903530056 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2ec00678c0 0x7f2ec0069d80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40cdcb0 tx=0x7f2ed4007e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.732+0000 7f2ee0c00640 1 -- 172.21.15.83:0/1903530056 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2ecc0c4060 con 0x7f2ee41446e0 2024-04-25T00:13:34.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.900+0000 7f2eeb000640 1 -- 172.21.15.83:0/1903530056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2ee40cdce0 con 0x7f2ee41446e0 2024-04-25T00:13:34.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.904+0000 7f2ee0c00640 1 -- 172.21.15.83:0/1903530056 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) ==== 80+0+0 (secure 0 0 0) 0x7f2ecc0160d0 con 0x7f2ee41446e0 2024-04-25T00:13:34.906 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:13:34.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.904+0000 7f2eeb000640 1 -- 172.21.15.83:0/1903530056 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2ec00678c0 msgr2=0x7f2ec0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.904+0000 7f2eeb000640 1 --2- 172.21.15.83:0/1903530056 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2ec00678c0 0x7f2ec0069d80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40cdcb0 tx=0x7f2ed4007e40 comp rx=0 tx=0).stop 2024-04-25T00:13:34.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.904+0000 7f2eeb000640 1 -- 172.21.15.83:0/1903530056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee41446e0 msgr2=0x7f2ee413e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.904+0000 7f2eeb000640 1 --2- 172.21.15.83:0/1903530056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee41446e0 0x7f2ee413e7b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0ad040 tx=0x7f2ecc01ad20 comp rx=0 tx=0).stop 2024-04-25T00:13:34.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.904+0000 7f2eeb000640 1 -- 172.21.15.83:0/1903530056 shutdown_connections 2024-04-25T00:13:34.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.904+0000 7f2eeb000640 1 --2- 172.21.15.83:0/1903530056 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2ec00678c0 0x7f2ec0069d80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.904+0000 7f2eeb000640 1 --2- 172.21.15.83:0/1903530056 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ee415f350 0x7f2ee415e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.904+0000 7f2eeb000640 1 --2- 172.21.15.83:0/1903530056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee41446e0 0x7f2ee413e7b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.904+0000 7f2eeb000640 1 --2- 172.21.15.83:0/1903530056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ee406ee20 0x7f2ee40b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.904+0000 7f2eeb000640 1 -- 172.21.15.83:0/1903530056 >> 172.21.15.83:0/1903530056 conn(0x7f2ee40cfa60 msgr2=0x7f2ee40af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:34.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.904+0000 7f2eeb000640 1 -- 172.21.15.83:0/1903530056 shutdown_connections 2024-04-25T00:13:34.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:34.904+0000 7f2eeb000640 1 -- 172.21.15.83:0/1903530056 wait complete. 2024-04-25T00:13:34.927 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:13:34.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:34.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:13:34.956 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:13:34.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:34.956 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:13:35.016 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:13:35.017 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:13:35.017 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:35.030 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:13:35.030 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:35.086 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.224+0000 7f325ba00640 1 Processor -- start 2024-04-25T00:13:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.224+0000 7f325ba00640 1 -- start start 2024-04-25T00:13:35.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.224+0000 7f325ba00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32540cc510 0x7f325415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.224+0000 7f325ba00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f325415cf90 0x7f32540ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.224+0000 7f325ba00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32540ccc60 con 0x7f32541599f0 2024-04-25T00:13:35.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.224+0000 7f325ba00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3254159dd0 con 0x7f325415cf90 2024-04-25T00:13:35.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.224+0000 7f325ba00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f325406dba0 con 0x7f32540cc510 2024-04-25T00:13:35.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.224+0000 7f3259000640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f32541599f0 0x7f325415c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40630/0 (socket says 172.21.15.83:40630) 2024-04-25T00:13:35.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.224+0000 7f3259000640 1 -- 172.21.15.83:0/1617272929 learned_addr learned my addr 172.21.15.83:0/1617272929 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:35.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.224+0000 7f3259a00640 1 --2- 172.21.15.83:0/1617272929 >> v2:172.21.15.83:3300/0 conn(0x7f325415cf90 0x7f32540ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.224+0000 7f324be00640 1 --2- 172.21.15.83:0/1617272929 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32540cc510 0x7f325415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.224+0000 7f3253e00640 1 -- 172.21.15.83:0/1617272929 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 376839673 0 0) 0x7f3244001120 con 0x7f32541599f0 2024-04-25T00:13:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.224+0000 7f3259a00640 1 -- 172.21.15.83:0/1617272929 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32540cc510 msgr2=0x7f325415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.224+0000 7f3259a00640 1 --2- 172.21.15.83:0/1617272929 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32540cc510 0x7f325415ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.228+0000 7f3259a00640 1 -- 172.21.15.83:0/1617272929 >> v1:172.21.15.83:6789/0 conn(0x7f32541599f0 legacy=0x7f325415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.228+0000 7f3259a00640 1 -- 172.21.15.83:0/1617272929 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325406de80 con 0x7f325415cf90 2024-04-25T00:13:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.228+0000 7f3259a00640 1 --2- 172.21.15.83:0/1617272929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f325415cf90 0x7f32540ce6e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f324c009d90 tx=0x7f324c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd133893fd9ab21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:35.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.228+0000 7f3253e00640 1 -- 172.21.15.83:0/1617272929 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c0b2070 con 0x7f325415cf90 2024-04-25T00:13:35.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.228+0000 7f3253e00640 1 -- 172.21.15.83:0/1617272929 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f324c0a57f0 con 0x7f325415cf90 2024-04-25T00:13:35.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.228+0000 7f3253e00640 1 -- 172.21.15.83:0/1617272929 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c0a5bc0 con 0x7f325415cf90 2024-04-25T00:13:35.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.228+0000 7f325ba00640 1 -- 172.21.15.83:0/1617272929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f325415cf90 msgr2=0x7f32540ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.228+0000 7f325ba00640 1 --2- 172.21.15.83:0/1617272929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f325415cf90 0x7f32540ce6e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f324c009d90 tx=0x7f324c01c8e0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.228+0000 7f325ba00640 1 -- 172.21.15.83:0/1617272929 shutdown_connections 2024-04-25T00:13:35.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.228+0000 7f325ba00640 1 --2- 172.21.15.83:0/1617272929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f325415cf90 0x7f32540ce6e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.228+0000 7f325ba00640 1 --2- 172.21.15.83:0/1617272929 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32540cc510 0x7f325415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f325ba00640 1 -- 172.21.15.83:0/1617272929 >> 172.21.15.83:0/1617272929 conn(0x7f32540b1830 msgr2=0x7f32540b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f325ba00640 1 -- 172.21.15.83:0/1617272929 shutdown_connections 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f325ba00640 1 -- 172.21.15.83:0/1617272929 wait complete. 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f325ba00640 1 Processor -- start 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f325ba00640 1 -- start start 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f325ba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32540cc510 0x7f32540af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f325ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32541599f0 0x7f325414dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f325ba00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f325415cf90 0x7f32540b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f325ba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32540cf5b0 con 0x7f32540cc510 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f325ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32540cf430 con 0x7f32541599f0 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f325ba00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f32540cf730 con 0x7f325415cf90 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f3259000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32540cc510 0x7f32540af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f3259000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32540cc510 0x7f32540af9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34802/0 (socket says 172.21.15.83:34802) 2024-04-25T00:13:35.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f3259000640 1 -- 172.21.15.83:0/1989677078 learned_addr learned my addr 172.21.15.83:0/1989677078 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f3259a00640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f325415cf90 0x7f32540b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f324be00640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32541599f0 0x7f325414dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f3259000640 1 -- 172.21.15.83:0/1989677078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f325415cf90 msgr2=0x7f32540b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f3259000640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f325415cf90 0x7f32540b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f3259000640 1 -- 172.21.15.83:0/1989677078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32541599f0 msgr2=0x7f325414dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f3259000640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32541599f0 0x7f325414dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f3259000640 1 -- 172.21.15.83:0/1989677078 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32540b3ed0 con 0x7f32540cc510 2024-04-25T00:13:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f324be00640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32541599f0 0x7f325414dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f3259000640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32540cc510 0x7f32540af9b0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f324400c780 tx=0x7f324400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.232+0000 7f3251600640 1 -- 172.21.15.83:0/1989677078 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244018070 con 0x7f32540cc510 2024-04-25T00:13:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.236+0000 7f3251600640 1 -- 172.21.15.83:0/1989677078 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32440072f0 con 0x7f32540cc510 2024-04-25T00:13:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.236+0000 7f3251600640 1 -- 172.21.15.83:0/1989677078 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32440047f0 con 0x7f32540cc510 2024-04-25T00:13:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.236+0000 7f325ba00640 1 -- 172.21.15.83:0/1989677078 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f325414fb10 con 0x7f32540cc510 2024-04-25T00:13:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.236+0000 7f325ba00640 1 -- 172.21.15.83:0/1989677078 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f325414eaf0 con 0x7f32540cc510 2024-04-25T00:13:35.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.236+0000 7f325ba00640 1 -- 172.21.15.83:0/1989677078 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32540737f0 con 0x7f32540cc510 2024-04-25T00:13:35.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.236+0000 7f3251600640 1 -- 172.21.15.83:0/1989677078 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3244004990 con 0x7f32540cc510 2024-04-25T00:13:35.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.236+0000 7f3251600640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f32300678c0 0x7f3230069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.236+0000 7f3251600640 1 -- 172.21.15.83:0/1989677078 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f32440851e0 con 0x7f32540cc510 2024-04-25T00:13:35.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.236+0000 7f324be00640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f32300678c0 0x7f3230069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.240+0000 7f324be00640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f32300678c0 0x7f3230069d80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3254158730 tx=0x7f323c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:35.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.244+0000 7f3251600640 1 -- 172.21.15.83:0/1989677078 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f325414fb10 con 0x7f32540cc510 2024-04-25T00:13:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.400+0000 7f325ba00640 1 -- 172.21.15.83:0/1989677078 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f325404f5f0 con 0x7f32540cc510 2024-04-25T00:13:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.400+0000 7f3251600640 1 -- 172.21.15.83:0/1989677078 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16800 (secure 0 0 0) 0x7f32540ccc60 con 0x7f32540cc510 2024-04-25T00:13:35.405 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:35.405 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":60,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:21.438568+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:18.160239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:18.800529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:35.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.404+0000 7f325ba00640 1 -- 172.21.15.83:0/1989677078 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f32300678c0 msgr2=0x7f3230069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.404+0000 7f325ba00640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f32300678c0 0x7f3230069d80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3254158730 tx=0x7f323c008040 comp rx=0 tx=0).stop 2024-04-25T00:13:35.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.404+0000 7f325ba00640 1 -- 172.21.15.83:0/1989677078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32540cc510 msgr2=0x7f32540af9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.404+0000 7f325ba00640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32540cc510 0x7f32540af9b0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f324400c780 tx=0x7f324400cc40 comp rx=0 tx=0).stop 2024-04-25T00:13:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.404+0000 7f325ba00640 1 -- 172.21.15.83:0/1989677078 shutdown_connections 2024-04-25T00:13:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.404+0000 7f325ba00640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f32300678c0 0x7f3230069d80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.404+0000 7f325ba00640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f325415cf90 0x7f32540b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.404+0000 7f325ba00640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32541599f0 0x7f325414dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.404+0000 7f325ba00640 1 --2- 172.21.15.83:0/1989677078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32540cc510 0x7f32540af9b0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.404+0000 7f325ba00640 1 -- 172.21.15.83:0/1989677078 >> 172.21.15.83:0/1989677078 conn(0x7f32540b1830 msgr2=0x7f32540b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.404+0000 7f325ba00640 1 -- 172.21.15.83:0/1989677078 shutdown_connections 2024-04-25T00:13:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.404+0000 7f325ba00640 1 -- 172.21.15.83:0/1989677078 wait complete. 2024-04-25T00:13:35.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:18.160239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:13:18.800529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:13:35.427 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:13:35.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.564+0000 7fef95400640 1 Processor -- start 2024-04-25T00:13:35.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.564+0000 7fef95400640 1 -- start start 2024-04-25T00:13:35.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.564+0000 7fef95400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fef901551e0 0x7fef9015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.564+0000 7fef95400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef9015c560 con 0x7fef9006e4c0 2024-04-25T00:13:35.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.564+0000 7fef95400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef90072460 con 0x7fef901555c0 2024-04-25T00:13:35.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.564+0000 7fef95400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fef9015c7d0 con 0x7fef901551e0 2024-04-25T00:13:35.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8f200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fef9006e4c0 0x7fef9015cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40642/0 (socket says 172.21.15.83:40642) 2024-04-25T00:13:35.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8f200640 1 -- 172.21.15.83:0/810087804 learned_addr learned my addr 172.21.15.83:0/810087804 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:35.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8e800640 1 --2- 172.21.15.83:0/810087804 >> v2:172.21.15.86:3300/0 conn(0x7fef901551e0 0x7fef9015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 -- 172.21.15.83:0/810087804 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4098121713 0 0) 0x7fef9015c560 con 0x7fef9006e4c0 2024-04-25T00:13:35.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 -- 172.21.15.83:0/810087804 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef74003530 con 0x7fef9006e4c0 2024-04-25T00:13:35.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 -- 172.21.15.83:0/810087804 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3594244588 0 0) 0x7fef74003530 con 0x7fef9006e4c0 2024-04-25T00:13:35.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 -- 172.21.15.83:0/810087804 >> v2:172.21.15.86:3300/0 conn(0x7fef901551e0 msgr2=0x7fef9015bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:35.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 --2- 172.21.15.83:0/810087804 >> v2:172.21.15.86:3300/0 conn(0x7fef901551e0 0x7fef9015bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 -- 172.21.15.83:0/810087804 >> v1:172.21.15.86:6789/0 conn(0x7fef901555c0 legacy=0x7fef90071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 -- 172.21.15.83:0/810087804 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9015cab0 con 0x7fef9006e4c0 2024-04-25T00:13:35.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 -- 172.21.15.83:0/810087804 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fef88001a00 con 0x7fef9006e4c0 2024-04-25T00:13:35.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 -- 172.21.15.83:0/810087804 >> v1:172.21.15.83:6789/0 conn(0x7fef9006e4c0 legacy=0x7fef9015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 --2- 172.21.15.83:0/810087804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef74003a80 0x7fef74003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 --2- 172.21.15.83:0/810087804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef74004810 0x7fef74004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 --2- 172.21.15.83:0/810087804 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef74005220 0x7fef7400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 -- 172.21.15.83:0/810087804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef9015c7d0 con 0x7fef74003a80 2024-04-25T00:13:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 -- 172.21.15.83:0/810087804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef7400c150 con 0x7fef74004810 2024-04-25T00:13:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8de00640 1 -- 172.21.15.83:0/810087804 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fef7400c350 con 0x7fef74005220 2024-04-25T00:13:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8e800640 1 --2- 172.21.15.83:0/810087804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef74003a80 0x7fef74003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef85e00640 1 --2- 172.21.15.83:0/810087804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef74004810 0x7fef74004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8f200640 1 --2- 172.21.15.83:0/810087804 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef74005220 0x7fef7400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8e800640 1 -- 172.21.15.83:0/810087804 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef74005220 msgr2=0x7fef7400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8e800640 1 --2- 172.21.15.83:0/810087804 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef74005220 0x7fef7400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8e800640 1 -- 172.21.15.83:0/810087804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef74004810 msgr2=0x7fef74004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8e800640 1 --2- 172.21.15.83:0/810087804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef74004810 0x7fef74004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8e800640 1 -- 172.21.15.83:0/810087804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef74003530 con 0x7fef74003a80 2024-04-25T00:13:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8f200640 1 --2- 172.21.15.83:0/810087804 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef74005220 0x7fef7400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.568+0000 7fef8f200640 1 -- 172.21.15.83:0/810087804 reap_dead start 2024-04-25T00:13:35.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef8e800640 1 --2- 172.21.15.83:0/810087804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef74003a80 0x7fef74003ea0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fef80009ae0 tx=0x7fef8001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d7f5c2e7323aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:35.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef8de00640 1 -- 172.21.15.83:0/810087804 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800b2070 con 0x7fef74003a80 2024-04-25T00:13:35.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef8de00640 1 -- 172.21.15.83:0/810087804 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef800a5550 con 0x7fef74003a80 2024-04-25T00:13:35.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef8de00640 1 -- 172.21.15.83:0/810087804 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800096b0 con 0x7fef74003a80 2024-04-25T00:13:35.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 -- 172.21.15.83:0/810087804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef74003a80 msgr2=0x7fef74003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 --2- 172.21.15.83:0/810087804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef74003a80 0x7fef74003ea0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fef80009ae0 tx=0x7fef8001ca70 comp rx=0 tx=0).stop 2024-04-25T00:13:35.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 -- 172.21.15.83:0/810087804 shutdown_connections 2024-04-25T00:13:35.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 --2- 172.21.15.83:0/810087804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef74003a80 0x7fef74003ea0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 -- 172.21.15.83:0/810087804 >> 172.21.15.83:0/810087804 conn(0x7fef900b1830 msgr2=0x7fef900b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:35.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 -- 172.21.15.83:0/810087804 shutdown_connections 2024-04-25T00:13:35.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 -- 172.21.15.83:0/810087804 wait complete. 2024-04-25T00:13:35.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 Processor -- start 2024-04-25T00:13:35.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 -- start start 2024-04-25T00:13:35.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef74003a80 0x7fef9015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef900afa00 0x7fef90140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef900afe80 0x7fef90141510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef9015ec80 con 0x7fef900afa00 2024-04-25T00:13:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef9015eb00 con 0x7fef900afe80 2024-04-25T00:13:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.572+0000 7fef95400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fef9015ee00 con 0x7fef74003a80 2024-04-25T00:13:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef85e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef900afa00 0x7fef90140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef85e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef900afa00 0x7fef90140fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34822/0 (socket says 172.21.15.83:34822) 2024-04-25T00:13:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef85e00640 1 -- 172.21.15.83:0/3990990293 learned_addr learned my addr 172.21.15.83:0/3990990293 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef8e800640 1 --2- 172.21.15.83:0/3990990293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef74003a80 0x7fef9015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef8f200640 1 --2- 172.21.15.83:0/3990990293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef900afe80 0x7fef90141510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef85e00640 1 -- 172.21.15.83:0/3990990293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef74003a80 msgr2=0x7fef9015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef85e00640 1 --2- 172.21.15.83:0/3990990293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef74003a80 0x7fef9015a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef85e00640 1 -- 172.21.15.83:0/3990990293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef900afe80 msgr2=0x7fef90141510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef85e00640 1 --2- 172.21.15.83:0/3990990293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef900afe80 0x7fef90141510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef85e00640 1 -- 172.21.15.83:0/3990990293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef90141a50 con 0x7fef900afa00 2024-04-25T00:13:35.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef85e00640 1 --2- 172.21.15.83:0/3990990293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef900afa00 0x7fef90140fd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fef7800f5f0 tx=0x7fef7800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:35.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef87400640 1 -- 172.21.15.83:0/3990990293 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef780052e0 con 0x7fef900afa00 2024-04-25T00:13:35.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef87400640 1 -- 172.21.15.83:0/3990990293 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef78018070 con 0x7fef900afa00 2024-04-25T00:13:35.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef95400640 1 -- 172.21.15.83:0/3990990293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef900ce480 con 0x7fef900afa00 2024-04-25T00:13:35.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef87400640 1 -- 172.21.15.83:0/3990990293 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78003840 con 0x7fef900afa00 2024-04-25T00:13:35.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef95400640 1 -- 172.21.15.83:0/3990990293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef900cea40 con 0x7fef900afa00 2024-04-25T00:13:35.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef95400640 1 -- 172.21.15.83:0/3990990293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef9004f510 con 0x7fef900afa00 2024-04-25T00:13:35.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef87400640 1 -- 172.21.15.83:0/3990990293 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fef780039e0 con 0x7fef900afa00 2024-04-25T00:13:35.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.576+0000 7fef87400640 1 --2- 172.21.15.83:0/3990990293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fef6c0678c0 0x7fef6c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.580+0000 7fef87400640 1 -- 172.21.15.83:0/3990990293 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fef78085be0 con 0x7fef900afa00 2024-04-25T00:13:35.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.580+0000 7fef8e800640 1 --2- 172.21.15.83:0/3990990293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fef6c0678c0 0x7fef6c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.580+0000 7fef8e800640 1 --2- 172.21.15.83:0/3990990293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fef6c0678c0 0x7fef6c069d80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fef800af000 tx=0x7fef80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:35.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.584+0000 7fef87400640 1 -- 172.21.15.83:0/3990990293 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fef780564f0 con 0x7fef900afa00 2024-04-25T00:13:35.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.736+0000 7fef95400640 1 -- 172.21.15.83:0/3990990293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef9015db40 con 0x7fef900afa00 2024-04-25T00:13:35.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.740+0000 7fef87400640 1 -- 172.21.15.83:0/3990990293 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) ==== 93+0+10 (secure 0 0 0) 0x7fef78055df0 con 0x7fef900afa00 2024-04-25T00:13:35.742 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:13:35.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.740+0000 7fef95400640 1 -- 172.21.15.83:0/3990990293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fef6c0678c0 msgr2=0x7fef6c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.740+0000 7fef95400640 1 --2- 172.21.15.83:0/3990990293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fef6c0678c0 0x7fef6c069d80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fef800af000 tx=0x7fef80008040 comp rx=0 tx=0).stop 2024-04-25T00:13:35.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.740+0000 7fef95400640 1 -- 172.21.15.83:0/3990990293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef900afa00 msgr2=0x7fef90140fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.740+0000 7fef95400640 1 --2- 172.21.15.83:0/3990990293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef900afa00 0x7fef90140fd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fef7800f5f0 tx=0x7fef7800fab0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.740+0000 7fef95400640 1 -- 172.21.15.83:0/3990990293 shutdown_connections 2024-04-25T00:13:35.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.740+0000 7fef95400640 1 --2- 172.21.15.83:0/3990990293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fef6c0678c0 0x7fef6c069d80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.740+0000 7fef95400640 1 --2- 172.21.15.83:0/3990990293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef900afe80 0x7fef90141510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.740+0000 7fef95400640 1 --2- 172.21.15.83:0/3990990293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef900afa00 0x7fef90140fd0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.740+0000 7fef95400640 1 --2- 172.21.15.83:0/3990990293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef74003a80 0x7fef9015a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.740+0000 7fef95400640 1 -- 172.21.15.83:0/3990990293 >> 172.21.15.83:0/3990990293 conn(0x7fef900b1830 msgr2=0x7fef900b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:35.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.740+0000 7fef95400640 1 -- 172.21.15.83:0/3990990293 shutdown_connections 2024-04-25T00:13:35.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.740+0000 7fef95400640 1 -- 172.21.15.83:0/3990990293 wait complete. 2024-04-25T00:13:35.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:13:35.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.896+0000 7f8c5e800640 1 Processor -- start 2024-04-25T00:13:35.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.896+0000 7f8c5e800640 1 -- start start 2024-04-25T00:13:35.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c5e800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8c581551e0 0x7f8c5815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c5e800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c581555c0 0x7f8c58071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c5e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c5806e4c0 0x7f8c5806e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c5e800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8c5806ee70 con 0x7f8c581551e0 2024-04-25T00:13:35.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c5e800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8c5815c560 con 0x7f8c5806e4c0 2024-04-25T00:13:35.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c5e800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8c58072460 con 0x7f8c581555c0 2024-04-25T00:13:35.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c57400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8c581551e0 0x7f8c5815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c57400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8c581551e0 0x7f8c5815bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34838/0 (socket says 172.21.15.83:34838) 2024-04-25T00:13:35.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c57400640 1 -- 172.21.15.83:0/4129439577 learned_addr learned my addr 172.21.15.83:0/4129439577 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:35.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c4ea00640 1 --2- 172.21.15.83:0/4129439577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c581555c0 0x7f8c58071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c57e00640 1 --2- 172.21.15.83:0/4129439577 >> v2:172.21.15.86:3300/0 conn(0x7f8c5806e4c0 0x7f8c5806e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c57400640 1 -- 172.21.15.83:0/4129439577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c581555c0 msgr2=0x7f8c58071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:35.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c57400640 1 --2- 172.21.15.83:0/4129439577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c581555c0 0x7f8c58071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c57400640 1 -- 172.21.15.83:0/4129439577 >> v2:172.21.15.86:3300/0 conn(0x7f8c5806e4c0 msgr2=0x7f8c5806e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:35.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c57400640 1 --2- 172.21.15.83:0/4129439577 >> v2:172.21.15.86:3300/0 conn(0x7f8c5806e4c0 0x7f8c5806e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.900+0000 7f8c57400640 1 -- 172.21.15.83:0/4129439577 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5815c7d0 con 0x7f8c581551e0 2024-04-25T00:13:35.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c57400640 1 --2- 172.21.15.83:0/4129439577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c581551e0 0x7f8c5815bea0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8c580cc860 tx=0x7f8c4801ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7cd0b766e49524 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:35.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c56a00640 1 -- 172.21.15.83:0/4129439577 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c480b4070 con 0x7f8c581551e0 2024-04-25T00:13:35.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c56a00640 1 -- 172.21.15.83:0/4129439577 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c480a7b70 con 0x7f8c581551e0 2024-04-25T00:13:35.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c56a00640 1 -- 172.21.15.83:0/4129439577 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c480af440 con 0x7f8c581551e0 2024-04-25T00:13:35.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c5e800640 1 -- 172.21.15.83:0/4129439577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c581551e0 msgr2=0x7f8c5815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c5e800640 1 --2- 172.21.15.83:0/4129439577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c581551e0 0x7f8c5815bea0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8c580cc860 tx=0x7f8c4801ebc0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c5e800640 1 -- 172.21.15.83:0/4129439577 shutdown_connections 2024-04-25T00:13:35.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c5e800640 1 --2- 172.21.15.83:0/4129439577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c581555c0 0x7f8c58071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c5e800640 1 --2- 172.21.15.83:0/4129439577 >> v2:172.21.15.86:3300/0 conn(0x7f8c5806e4c0 0x7f8c5806e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c5e800640 1 --2- 172.21.15.83:0/4129439577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c581551e0 0x7f8c5815bea0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c5e800640 1 -- 172.21.15.83:0/4129439577 >> 172.21.15.83:0/4129439577 conn(0x7f8c580b1830 msgr2=0x7f8c580b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:35.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c5e800640 1 -- 172.21.15.83:0/4129439577 shutdown_connections 2024-04-25T00:13:35.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c5e800640 1 -- 172.21.15.83:0/4129439577 wait complete. 2024-04-25T00:13:35.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c5e800640 1 Processor -- start 2024-04-25T00:13:35.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.904+0000 7f8c5e800640 1 -- start start 2024-04-25T00:13:35.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c5e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c5806e4c0 0x7f8c5814b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c5e800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c581551e0 0x7f8c5815dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c5e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c581555c0 0x7f8c581467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c5e800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5815f280 con 0x7f8c5806e4c0 2024-04-25T00:13:35.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c5e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5815f100 con 0x7f8c581555c0 2024-04-25T00:13:35.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c5e800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8c5815f400 con 0x7f8c581551e0 2024-04-25T00:13:35.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c57400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c5806e4c0 0x7f8c5814b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c57400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c5806e4c0 0x7f8c5814b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34854/0 (socket says 172.21.15.83:34854) 2024-04-25T00:13:35.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c57400640 1 -- 172.21.15.83:0/2092339637 learned_addr learned my addr 172.21.15.83:0/2092339637 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:35.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c57e00640 1 --2- 172.21.15.83:0/2092339637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c581555c0 0x7f8c581467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c4ea00640 1 --2- 172.21.15.83:0/2092339637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c581551e0 0x7f8c5815dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c57400640 1 -- 172.21.15.83:0/2092339637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c581551e0 msgr2=0x7f8c5815dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c57400640 1 --2- 172.21.15.83:0/2092339637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c581551e0 0x7f8c5815dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c57400640 1 -- 172.21.15.83:0/2092339637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c581555c0 msgr2=0x7f8c581467d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:35.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c57400640 1 --2- 172.21.15.83:0/2092339637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c581555c0 0x7f8c581467d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:35.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c57400640 1 -- 172.21.15.83:0/2092339637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5814aa20 con 0x7f8c5806e4c0 2024-04-25T00:13:35.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c57400640 1 --2- 172.21.15.83:0/2092339637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c5806e4c0 0x7f8c5814b7e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800b2c0 tx=0x7f8c4801b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c4fe00640 1 -- 172.21.15.83:0/2092339637 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c480be020 con 0x7f8c5806e4c0 2024-04-25T00:13:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c4fe00640 1 -- 172.21.15.83:0/2092339637 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c480afca0 con 0x7f8c5806e4c0 2024-04-25T00:13:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c4fe00640 1 -- 172.21.15.83:0/2092339637 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c480b4040 con 0x7f8c5806e4c0 2024-04-25T00:13:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c5e800640 1 -- 172.21.15.83:0/2092339637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5814ad00 con 0x7f8c5806e4c0 2024-04-25T00:13:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c5e800640 1 -- 172.21.15.83:0/2092339637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c581461e0 con 0x7f8c5806e4c0 2024-04-25T00:13:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c5e800640 1 -- 172.21.15.83:0/2092339637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c5804f510 con 0x7f8c5806e4c0 2024-04-25T00:13:35.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.908+0000 7f8c4fe00640 1 -- 172.21.15.83:0/2092339637 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8c480c1020 con 0x7f8c5806e4c0 2024-04-25T00:13:35.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.912+0000 7f8c4fe00640 1 --2- 172.21.15.83:0/2092339637 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c340678c0 0x7f8c34069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:35.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.912+0000 7f8c4fe00640 1 -- 172.21.15.83:0/2092339637 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f8c48122eb0 con 0x7f8c5806e4c0 2024-04-25T00:13:35.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.912+0000 7f8c4ea00640 1 --2- 172.21.15.83:0/2092339637 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c340678c0 0x7f8c34069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:35.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.912+0000 7f8c4ea00640 1 --2- 172.21.15.83:0/2092339637 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c340678c0 0x7f8c34069d80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8c580d0200 tx=0x7f8c40008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:35.916+0000 7f8c4fe00640 1 -- 172.21.15.83:0/2092339637 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8c480c8030 con 0x7f8c5806e4c0 2024-04-25T00:13:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.068+0000 7f8c5e800640 1 -- 172.21.15.83:0/2092339637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8c5806ce30 con 0x7f8c5806e4c0 2024-04-25T00:13:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.068+0000 7f8c4fe00640 1 -- 172.21.15.83:0/2092339637 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) ==== 104+0+11 (secure 0 0 0) 0x7f8c480187e0 con 0x7f8c5806e4c0 2024-04-25T00:13:36.074 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:13:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.072+0000 7f8c5e800640 1 -- 172.21.15.83:0/2092339637 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c340678c0 msgr2=0x7f8c34069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.072+0000 7f8c5e800640 1 --2- 172.21.15.83:0/2092339637 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c340678c0 0x7f8c34069d80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8c580d0200 tx=0x7f8c40008040 comp rx=0 tx=0).stop 2024-04-25T00:13:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.072+0000 7f8c5e800640 1 -- 172.21.15.83:0/2092339637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c5806e4c0 msgr2=0x7f8c5814b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.072+0000 7f8c5e800640 1 --2- 172.21.15.83:0/2092339637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c5806e4c0 0x7f8c5814b7e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800b2c0 tx=0x7f8c4801b960 comp rx=0 tx=0).stop 2024-04-25T00:13:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.072+0000 7f8c5e800640 1 -- 172.21.15.83:0/2092339637 shutdown_connections 2024-04-25T00:13:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.072+0000 7f8c5e800640 1 --2- 172.21.15.83:0/2092339637 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8c340678c0 0x7f8c34069d80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.072+0000 7f8c5e800640 1 --2- 172.21.15.83:0/2092339637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c581551e0 0x7f8c5815dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.072+0000 7f8c5e800640 1 --2- 172.21.15.83:0/2092339637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c581555c0 0x7f8c581467d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.072+0000 7f8c5e800640 1 --2- 172.21.15.83:0/2092339637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c5806e4c0 0x7f8c5814b7e0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.072+0000 7f8c5e800640 1 -- 172.21.15.83:0/2092339637 >> 172.21.15.83:0/2092339637 conn(0x7f8c580b1830 msgr2=0x7f8c5814d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.072+0000 7f8c5e800640 1 -- 172.21.15.83:0/2092339637 shutdown_connections 2024-04-25T00:13:36.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.072+0000 7f8c5e800640 1 -- 172.21.15.83:0/2092339637 wait complete. 2024-04-25T00:13:36.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:13:36.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.228+0000 7f8111800640 1 Processor -- start 2024-04-25T00:13:36.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.228+0000 7f8111800640 1 -- start start 2024-04-25T00:13:36.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.232+0000 7f8111800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f810c1551e0 0x7f810c15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.232+0000 7f8111800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f810c06e4c0 0x7f810c06e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.232+0000 7f8111800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f810c15c560 con 0x7f810c1555c0 2024-04-25T00:13:36.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.232+0000 7f8111800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f810c072460 con 0x7f810c06e4c0 2024-04-25T00:13:36.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.232+0000 7f8111800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f810c15c7d0 con 0x7f810c1551e0 2024-04-25T00:13:36.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.232+0000 7f810b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f810c06e4c0 0x7f810c06e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.232+0000 7f810b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f810c06e4c0 0x7f810c06e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34856/0 (socket says 172.21.15.83:34856) 2024-04-25T00:13:36.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.232+0000 7f810b400640 1 -- 172.21.15.83:0/3123693230 learned_addr learned my addr 172.21.15.83:0/3123693230 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:36.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.232+0000 7f810aa00640 1 --2- 172.21.15.83:0/3123693230 >> v2:172.21.15.86:3300/0 conn(0x7f810c1551e0 0x7f810c15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.232+0000 7f810b400640 1 -- 172.21.15.83:0/3123693230 >> v2:172.21.15.86:3300/0 conn(0x7f810c1551e0 msgr2=0x7f810c15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:36.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.232+0000 7f810b400640 1 --2- 172.21.15.83:0/3123693230 >> v2:172.21.15.86:3300/0 conn(0x7f810c1551e0 0x7f810c15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.232+0000 7f810b400640 1 -- 172.21.15.83:0/3123693230 >> v1:172.21.15.86:6789/0 conn(0x7f810c1555c0 legacy=0x7f810c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.232+0000 7f810b400640 1 -- 172.21.15.83:0/3123693230 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810c15cab0 con 0x7f810c06e4c0 2024-04-25T00:13:36.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.232+0000 7f810aa00640 1 --2- 172.21.15.83:0/3123693230 >> v2:172.21.15.86:3300/0 conn(0x7f810c1551e0 0x7f810c15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:36.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.236+0000 7f810b400640 1 --2- 172.21.15.83:0/3123693230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f810c06e4c0 0x7f810c06e950 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f810400d150 tx=0x7f810401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd05d88b41bfa10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:36.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.236+0000 7f810a000640 1 -- 172.21.15.83:0/3123693230 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810400b2d0 con 0x7f810c06e4c0 2024-04-25T00:13:36.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.236+0000 7f810a000640 1 -- 172.21.15.83:0/3123693230 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810400b470 con 0x7f810c06e4c0 2024-04-25T00:13:36.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.236+0000 7f810a000640 1 -- 172.21.15.83:0/3123693230 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81040a79e0 con 0x7f810c06e4c0 2024-04-25T00:13:36.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.236+0000 7f8111800640 1 -- 172.21.15.83:0/3123693230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f810c06e4c0 msgr2=0x7f810c06e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.236+0000 7f8111800640 1 --2- 172.21.15.83:0/3123693230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f810c06e4c0 0x7f810c06e950 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f810400d150 tx=0x7f810401ebf0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.236+0000 7f8111800640 1 -- 172.21.15.83:0/3123693230 shutdown_connections 2024-04-25T00:13:36.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.236+0000 7f8111800640 1 --2- 172.21.15.83:0/3123693230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f810c06e4c0 0x7f810c06e950 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.236+0000 7f8111800640 1 --2- 172.21.15.83:0/3123693230 >> v2:172.21.15.86:3300/0 conn(0x7f810c1551e0 0x7f810c15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.236+0000 7f8111800640 1 -- 172.21.15.83:0/3123693230 >> 172.21.15.83:0/3123693230 conn(0x7f810c0b1830 msgr2=0x7f810c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:36.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.236+0000 7f8111800640 1 -- 172.21.15.83:0/3123693230 shutdown_connections 2024-04-25T00:13:36.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.236+0000 7f8111800640 1 -- 172.21.15.83:0/3123693230 wait complete. 2024-04-25T00:13:36.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.236+0000 7f8111800640 1 Processor -- start 2024-04-25T00:13:36.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.236+0000 7f8111800640 1 -- start start 2024-04-25T00:13:36.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8111800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f810c06e4c0 0x7f810c146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8111800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f810c1551e0 0x7f810c0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8111800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f810c1555c0 0x7f810c0b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8111800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f810c06df50 con 0x7f810c1555c0 2024-04-25T00:13:36.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8111800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f810c06ddd0 con 0x7f810c06e4c0 2024-04-25T00:13:36.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8111800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f810c06e0d0 con 0x7f810c1551e0 2024-04-25T00:13:36.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f810b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f810c1555c0 0x7f810c0b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f810b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f810c1555c0 0x7f810c0b0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34858/0 (socket says 172.21.15.83:34858) 2024-04-25T00:13:36.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f810b400640 1 -- 172.21.15.83:0/4167890068 learned_addr learned my addr 172.21.15.83:0/4167890068 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:36.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f810aa00640 1 --2- 172.21.15.83:0/4167890068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f810c06e4c0 0x7f810c146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8102000640 1 --2- 172.21.15.83:0/4167890068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f810c1551e0 0x7f810c0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f810b400640 1 -- 172.21.15.83:0/4167890068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f810c1551e0 msgr2=0x7f810c0b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f810b400640 1 --2- 172.21.15.83:0/4167890068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f810c1551e0 0x7f810c0b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f810b400640 1 -- 172.21.15.83:0/4167890068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f810c06e4c0 msgr2=0x7f810c146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f810b400640 1 --2- 172.21.15.83:0/4167890068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f810c06e4c0 0x7f810c146e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f810b400640 1 -- 172.21.15.83:0/4167890068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810c15f090 con 0x7f810c1555c0 2024-04-25T00:13:36.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f810b400640 1 --2- 172.21.15.83:0/4167890068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f810c1555c0 0x7f810c0b0d30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f81040b1040 tx=0x7f810401b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:36.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8103400640 1 -- 172.21.15.83:0/4167890068 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810400b720 con 0x7f810c1555c0 2024-04-25T00:13:36.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8103400640 1 -- 172.21.15.83:0/4167890068 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81040be070 con 0x7f810c1555c0 2024-04-25T00:13:36.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8103400640 1 -- 172.21.15.83:0/4167890068 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104009c90 con 0x7f810c1555c0 2024-04-25T00:13:36.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8111800640 1 -- 172.21.15.83:0/4167890068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810c0b2b70 con 0x7f810c1555c0 2024-04-25T00:13:36.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8111800640 1 -- 172.21.15.83:0/4167890068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f810c147380 con 0x7f810c1555c0 2024-04-25T00:13:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8111800640 1 -- 172.21.15.83:0/4167890068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f810c04f510 con 0x7f810c1555c0 2024-04-25T00:13:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8103400640 1 -- 172.21.15.83:0/4167890068 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f81040b6070 con 0x7f810c1555c0 2024-04-25T00:13:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.240+0000 7f8103400640 1 --2- 172.21.15.83:0/4167890068 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80e80678f0 0x7f80e8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.244+0000 7f8103400640 1 -- 172.21.15.83:0/4167890068 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f8104120f20 con 0x7f810c1555c0 2024-04-25T00:13:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.244+0000 7f810aa00640 1 --2- 172.21.15.83:0/4167890068 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80e80678f0 0x7f80e8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.244+0000 7f810aa00640 1 --2- 172.21.15.83:0/4167890068 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80e80678f0 0x7f80e8069db0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0098b0 tx=0x7f80fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:36.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.248+0000 7f8103400640 1 -- 172.21.15.83:0/4167890068 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f81040187e0 con 0x7f810c1555c0 2024-04-25T00:13:36.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.400+0000 7f8111800640 1 -- 172.21.15.83:0/4167890068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f810c06ce30 con 0x7f810c1555c0 2024-04-25T00:13:36.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.400+0000 7f8103400640 1 -- 172.21.15.83:0/4167890068 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) ==== 100+0+11 (secure 0 0 0) 0x7f81040f1a10 con 0x7f810c1555c0 2024-04-25T00:13:36.405 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:13:36.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.404+0000 7f8111800640 1 -- 172.21.15.83:0/4167890068 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80e80678f0 msgr2=0x7f80e8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.404+0000 7f8111800640 1 --2- 172.21.15.83:0/4167890068 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80e80678f0 0x7f80e8069db0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0098b0 tx=0x7f80fc008040 comp rx=0 tx=0).stop 2024-04-25T00:13:36.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.404+0000 7f8111800640 1 -- 172.21.15.83:0/4167890068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f810c1555c0 msgr2=0x7f810c0b0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.404+0000 7f8111800640 1 --2- 172.21.15.83:0/4167890068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f810c1555c0 0x7f810c0b0d30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f81040b1040 tx=0x7f810401b990 comp rx=0 tx=0).stop 2024-04-25T00:13:36.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.404+0000 7f8111800640 1 -- 172.21.15.83:0/4167890068 shutdown_connections 2024-04-25T00:13:36.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.404+0000 7f8111800640 1 --2- 172.21.15.83:0/4167890068 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80e80678f0 0x7f80e8069db0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.404+0000 7f8111800640 1 --2- 172.21.15.83:0/4167890068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f810c1551e0 0x7f810c0b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.404+0000 7f8111800640 1 --2- 172.21.15.83:0/4167890068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f810c1555c0 0x7f810c0b0d30 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.404+0000 7f8111800640 1 --2- 172.21.15.83:0/4167890068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f810c06e4c0 0x7f810c146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.404+0000 7f8111800640 1 -- 172.21.15.83:0/4167890068 >> 172.21.15.83:0/4167890068 conn(0x7f810c0b1830 msgr2=0x7f810c159db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:36.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.404+0000 7f8111800640 1 -- 172.21.15.83:0/4167890068 shutdown_connections 2024-04-25T00:13:36.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.404+0000 7f8111800640 1 -- 172.21.15.83:0/4167890068 wait complete. 2024-04-25T00:13:36.427 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:13:36.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.564+0000 7ff50b000640 1 Processor -- start 2024-04-25T00:13:36.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.564+0000 7ff50b000640 1 -- start start 2024-04-25T00:13:36.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff50b000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5041446e0 0x7ff5040cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff50b000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff50406ee20 0x7ff504071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff50b000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5040cb210 con 0x7ff50415f350 2024-04-25T00:13:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff50b000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff5040715b0 con 0x7ff50406ee20 2024-04-25T00:13:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff50b000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff504144ac0 con 0x7ff5041446e0 2024-04-25T00:13:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff509000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff50406ee20 0x7ff504071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff4fb400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff50415f350 0x7ff50406e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40644/0 (socket says 172.21.15.83:40644) 2024-04-25T00:13:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff4fb400640 1 -- 172.21.15.83:0/309512077 learned_addr learned my addr 172.21.15.83:0/309512077 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff503e00640 1 --2- 172.21.15.83:0/309512077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5041446e0 0x7ff5040cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff503400640 1 -- 172.21.15.83:0/309512077 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2288151057 0 0) 0x7ff5040cb210 con 0x7ff50415f350 2024-04-25T00:13:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff503400640 1 -- 172.21.15.83:0/309512077 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4e8005740 con 0x7ff50415f350 2024-04-25T00:13:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff509000640 1 -- 172.21.15.83:0/309512077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5041446e0 msgr2=0x7ff5040cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff509000640 1 --2- 172.21.15.83:0/309512077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5041446e0 0x7ff5040cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff509000640 1 -- 172.21.15.83:0/309512077 >> v1:172.21.15.83:6789/0 conn(0x7ff50415f350 legacy=0x7ff50406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff509000640 1 -- 172.21.15.83:0/309512077 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5040cb4f0 con 0x7ff50406ee20 2024-04-25T00:13:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.568+0000 7ff503e00640 1 --2- 172.21.15.83:0/309512077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5041446e0 0x7ff5040cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff509000640 1 --2- 172.21.15.83:0/309512077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50406ee20 0x7ff504071fe0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0097c0 tx=0x7ff4fc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f213d0309d46c9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff503400640 1 -- 172.21.15.83:0/309512077 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0b1070 con 0x7ff50406ee20 2024-04-25T00:13:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff503400640 1 -- 172.21.15.83:0/309512077 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4fc0ac410 con 0x7ff50406ee20 2024-04-25T00:13:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff503400640 1 -- 172.21.15.83:0/309512077 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0ac7e0 con 0x7ff50406ee20 2024-04-25T00:13:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 -- 172.21.15.83:0/309512077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50406ee20 msgr2=0x7ff504071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 --2- 172.21.15.83:0/309512077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50406ee20 0x7ff504071fe0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0097c0 tx=0x7ff4fc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 -- 172.21.15.83:0/309512077 shutdown_connections 2024-04-25T00:13:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 --2- 172.21.15.83:0/309512077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50406ee20 0x7ff504071fe0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 --2- 172.21.15.83:0/309512077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5041446e0 0x7ff5040cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 -- 172.21.15.83:0/309512077 >> 172.21.15.83:0/309512077 conn(0x7ff5040cfa60 msgr2=0x7ff5040cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 -- 172.21.15.83:0/309512077 shutdown_connections 2024-04-25T00:13:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 -- 172.21.15.83:0/309512077 wait complete. 2024-04-25T00:13:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 Processor -- start 2024-04-25T00:13:36.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 -- start start 2024-04-25T00:13:36.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50406ee20 0x7ff5041f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5041446e0 0x7ff50415e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff50415f350 0x7ff50413a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff504072eb0 con 0x7ff50406ee20 2024-04-25T00:13:36.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff504072d30 con 0x7ff5041446e0 2024-04-25T00:13:36.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff50b000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff504073030 con 0x7ff50415f350 2024-04-25T00:13:36.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff503e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50406ee20 0x7ff5041f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff503e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50406ee20 0x7ff5041f2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34872/0 (socket says 172.21.15.83:34872) 2024-04-25T00:13:36.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.572+0000 7ff503e00640 1 -- 172.21.15.83:0/4003749260 learned_addr learned my addr 172.21.15.83:0/4003749260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:36.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff509000640 1 --2- 172.21.15.83:0/4003749260 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff50415f350 0x7ff50413a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff4fb400640 1 --2- 172.21.15.83:0/4003749260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5041446e0 0x7ff50415e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff503e00640 1 -- 172.21.15.83:0/4003749260 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff50415f350 msgr2=0x7ff50413a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff503e00640 1 --2- 172.21.15.83:0/4003749260 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff50415f350 0x7ff50413a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff503e00640 1 -- 172.21.15.83:0/4003749260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5041446e0 msgr2=0x7ff50415e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff503e00640 1 --2- 172.21.15.83:0/4003749260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5041446e0 0x7ff50415e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff503e00640 1 -- 172.21.15.83:0/4003749260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50415ef00 con 0x7ff50406ee20 2024-04-25T00:13:36.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff503e00640 1 --2- 172.21.15.83:0/4003749260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50406ee20 0x7ff5041f2970 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4007fa0 tx=0x7ff4f400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:36.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff500c00640 1 -- 172.21.15.83:0/4003749260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f401a070 con 0x7ff50406ee20 2024-04-25T00:13:36.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff500c00640 1 -- 172.21.15.83:0/4003749260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f4003ba0 con 0x7ff50406ee20 2024-04-25T00:13:36.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff500c00640 1 -- 172.21.15.83:0/4003749260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4004030 con 0x7ff50406ee20 2024-04-25T00:13:36.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff50b000640 1 -- 172.21.15.83:0/4003749260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5041415c0 con 0x7ff50406ee20 2024-04-25T00:13:36.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff50b000640 1 -- 172.21.15.83:0/4003749260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5041412e0 con 0x7ff50406ee20 2024-04-25T00:13:36.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff50b000640 1 -- 172.21.15.83:0/4003749260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff504068230 con 0x7ff50406ee20 2024-04-25T00:13:36.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff500c00640 1 -- 172.21.15.83:0/4003749260 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff4f401d070 con 0x7ff50406ee20 2024-04-25T00:13:36.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.576+0000 7ff500c00640 1 --2- 172.21.15.83:0/4003749260 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff4e00678c0 0x7ff4e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.580+0000 7ff500c00640 1 -- 172.21.15.83:0/4003749260 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7ff4f4085ff0 con 0x7ff50406ee20 2024-04-25T00:13:36.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.580+0000 7ff4fb400640 1 --2- 172.21.15.83:0/4003749260 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff4e00678c0 0x7ff4e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.580+0000 7ff4fb400640 1 --2- 172.21.15.83:0/4003749260 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff4e00678c0 0x7ff4e0069d80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff50413b750 tx=0x7ff4ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:36.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.584+0000 7ff500c00640 1 -- 172.21.15.83:0/4003749260 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff50415ef00 con 0x7ff50406ee20 2024-04-25T00:13:36.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.752+0000 7ff50b000640 1 -- 172.21.15.83:0/4003749260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff5040b48c0 con 0x7ff50406ee20 2024-04-25T00:13:36.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.752+0000 7ff500c00640 1 -- 172.21.15.83:0/4003749260 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) ==== 80+0+0 (secure 0 0 0) 0x7ff4f4010070 con 0x7ff50406ee20 2024-04-25T00:13:36.757 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:13:36.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.756+0000 7ff50b000640 1 -- 172.21.15.83:0/4003749260 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff4e00678c0 msgr2=0x7ff4e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.756+0000 7ff50b000640 1 --2- 172.21.15.83:0/4003749260 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff4e00678c0 0x7ff4e0069d80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff50413b750 tx=0x7ff4ec009040 comp rx=0 tx=0).stop 2024-04-25T00:13:36.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.756+0000 7ff50b000640 1 -- 172.21.15.83:0/4003749260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50406ee20 msgr2=0x7ff5041f2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.756+0000 7ff50b000640 1 --2- 172.21.15.83:0/4003749260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50406ee20 0x7ff5041f2970 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4007fa0 tx=0x7ff4f400d560 comp rx=0 tx=0).stop 2024-04-25T00:13:36.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.756+0000 7ff50b000640 1 -- 172.21.15.83:0/4003749260 shutdown_connections 2024-04-25T00:13:36.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.756+0000 7ff50b000640 1 --2- 172.21.15.83:0/4003749260 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff4e00678c0 0x7ff4e0069d80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.756+0000 7ff50b000640 1 --2- 172.21.15.83:0/4003749260 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff50415f350 0x7ff50413a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.756+0000 7ff50b000640 1 --2- 172.21.15.83:0/4003749260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5041446e0 0x7ff50415e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.756+0000 7ff50b000640 1 --2- 172.21.15.83:0/4003749260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50406ee20 0x7ff5041f2970 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.756+0000 7ff50b000640 1 -- 172.21.15.83:0/4003749260 >> 172.21.15.83:0/4003749260 conn(0x7ff5040cfa60 msgr2=0x7ff5040b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:36.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.756+0000 7ff50b000640 1 -- 172.21.15.83:0/4003749260 shutdown_connections 2024-04-25T00:13:36.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:36.756+0000 7ff50b000640 1 -- 172.21.15.83:0/4003749260 wait complete. 2024-04-25T00:13:36.778 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T00:13:36.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:36.778 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T00:13:36.814 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:13:36.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:36.814 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:13:36.866 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:13:36.867 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:13:36.867 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:36.876 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:13:36.876 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:13:36.876 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:13:36.876 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:36.922 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:13:36.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:37.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cbe400640 1 Processor -- start 2024-04-25T00:13:37.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cbe400640 1 -- start start 2024-04-25T00:13:37.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cbe400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cb806f5f0 0x7f5cb815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cbe400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5cb806f9d0 0x7f5cb80cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cbe400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5cb806b1e0 0x7f5cb806b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cbe400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5cb806bb70 con 0x7f5cb806b1e0 2024-04-25T00:13:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cbe400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5cb80cdb20 con 0x7f5cb806f9d0 2024-04-25T00:13:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cbe400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb80d0b60 con 0x7f5cb806f5f0 2024-04-25T00:13:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cb7e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5cb806b1e0 0x7f5cb806b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cb7e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5cb806b1e0 0x7f5cb806b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34884/0 (socket says 172.21.15.83:34884) 2024-04-25T00:13:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cb7e00640 1 -- 172.21.15.83:0/2813203962 learned_addr learned my addr 172.21.15.83:0/2813203962 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cb7400640 1 --2- 172.21.15.83:0/2813203962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cb806f5f0 0x7f5cb815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cb6a00640 1 --2- 172.21.15.83:0/2813203962 >> v2:172.21.15.86:3300/0 conn(0x7f5cb806f9d0 0x7f5cb80cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cb7e00640 1 -- 172.21.15.83:0/2813203962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cb806f5f0 msgr2=0x7f5cb815cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/2813203962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cb806f5f0 0x7f5cb815cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cb7e00640 1 -- 172.21.15.83:0/2813203962 >> v2:172.21.15.86:3300/0 conn(0x7f5cb806f9d0 msgr2=0x7f5cb80cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/2813203962 >> v2:172.21.15.86:3300/0 conn(0x7f5cb806f9d0 0x7f5cb80cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.056+0000 7f5cb7e00640 1 -- 172.21.15.83:0/2813203962 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb80d0e40 con 0x7f5cb806b1e0 2024-04-25T00:13:37.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.060+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/2813203962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb806b1e0 0x7f5cb806b630 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80b3850 tx=0x7f5cb001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6976869da3273673 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:37.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.060+0000 7f5cb6000640 1 -- 172.21.15.83:0/2813203962 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb00a5670 con 0x7f5cb806b1e0 2024-04-25T00:13:37.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.060+0000 7f5cb6000640 1 -- 172.21.15.83:0/2813203962 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb00a5810 con 0x7f5cb806b1e0 2024-04-25T00:13:37.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.060+0000 7f5cb6000640 1 -- 172.21.15.83:0/2813203962 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb00a5be0 con 0x7f5cb806b1e0 2024-04-25T00:13:37.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.060+0000 7f5cbe400640 1 -- 172.21.15.83:0/2813203962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb806b1e0 msgr2=0x7f5cb806b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.060+0000 7f5cbe400640 1 --2- 172.21.15.83:0/2813203962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb806b1e0 0x7f5cb806b630 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80b3850 tx=0x7f5cb001c9f0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.060+0000 7f5cbe400640 1 -- 172.21.15.83:0/2813203962 shutdown_connections 2024-04-25T00:13:37.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.060+0000 7f5cbe400640 1 --2- 172.21.15.83:0/2813203962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb806b1e0 0x7f5cb806b630 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.060+0000 7f5cbe400640 1 --2- 172.21.15.83:0/2813203962 >> v2:172.21.15.86:3300/0 conn(0x7f5cb806f9d0 0x7f5cb80cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.060+0000 7f5cbe400640 1 --2- 172.21.15.83:0/2813203962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cb806f5f0 0x7f5cb815cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.060+0000 7f5cbe400640 1 -- 172.21.15.83:0/2813203962 >> 172.21.15.83:0/2813203962 conn(0x7f5cb806c2f0 msgr2=0x7f5cb806c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:37.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.060+0000 7f5cbe400640 1 -- 172.21.15.83:0/2813203962 shutdown_connections 2024-04-25T00:13:37.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.060+0000 7f5cbe400640 1 -- 172.21.15.83:0/2813203962 wait complete. 2024-04-25T00:13:37.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.060+0000 7f5cbe400640 1 Processor -- start 2024-04-25T00:13:37.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cbe400640 1 -- start start 2024-04-25T00:13:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cbe400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb806b1e0 0x7f5cb815ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cbe400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cb806f5f0 0x7f5cb815e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cbe400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cb806f9d0 0x7f5cb814e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cbe400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb81603f0 con 0x7f5cb806b1e0 2024-04-25T00:13:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cbe400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb8160270 con 0x7f5cb806f5f0 2024-04-25T00:13:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cbe400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb8160570 con 0x7f5cb806f9d0 2024-04-25T00:13:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cb7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb806b1e0 0x7f5cb815ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cb7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb806b1e0 0x7f5cb815ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34898/0 (socket says 172.21.15.83:34898) 2024-04-25T00:13:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cb7400640 1 -- 172.21.15.83:0/1708743006 learned_addr learned my addr 172.21.15.83:0/1708743006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cb6a00640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cb806f5f0 0x7f5cb815e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cb806f9d0 0x7f5cb814e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cb7400640 1 -- 172.21.15.83:0/1708743006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cb806f9d0 msgr2=0x7f5cb814e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cb7400640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cb806f9d0 0x7f5cb814e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cb7400640 1 -- 172.21.15.83:0/1708743006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cb806f5f0 msgr2=0x7f5cb815e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cb7400640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cb806f5f0 0x7f5cb815e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cb7400640 1 -- 172.21.15.83:0/1708743006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb815f370 con 0x7f5cb806b1e0 2024-04-25T00:13:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cb806f9d0 0x7f5cb814e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cb7400640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb806b1e0 0x7f5cb815ec70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800d800 tx=0x7f5ca800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5caf400640 1 -- 172.21.15.83:0/1708743006 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8018020 con 0x7f5cb806b1e0 2024-04-25T00:13:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5caf400640 1 -- 172.21.15.83:0/1708743006 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca80096a0 con 0x7f5cb806b1e0 2024-04-25T00:13:37.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cbe400640 1 -- 172.21.15.83:0/1708743006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb80b3c50 con 0x7f5cb806b1e0 2024-04-25T00:13:37.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5caf400640 1 -- 172.21.15.83:0/1708743006 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8009a50 con 0x7f5cb806b1e0 2024-04-25T00:13:37.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cbe400640 1 -- 172.21.15.83:0/1708743006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb80b4210 con 0x7f5cb806b1e0 2024-04-25T00:13:37.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.064+0000 7f5cbe400640 1 -- 172.21.15.83:0/1708743006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb80736c0 con 0x7f5cb806b1e0 2024-04-25T00:13:37.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.068+0000 7f5caf400640 1 -- 172.21.15.83:0/1708743006 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5ca801b020 con 0x7f5cb806b1e0 2024-04-25T00:13:37.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.068+0000 7f5caf400640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5c980678c0 0x7f5c98069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.068+0000 7f5caf400640 1 -- 172.21.15.83:0/1708743006 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f5ca80863e0 con 0x7f5cb806b1e0 2024-04-25T00:13:37.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.068+0000 7f5cb6a00640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5c980678c0 0x7f5c98069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.068+0000 7f5cb6a00640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5c980678c0 0x7f5c98069d80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5cb81542c0 tx=0x7f5ca4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:37.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.072+0000 7f5caf400640 1 -- 172.21.15.83:0/1708743006 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5ca8022020 con 0x7f5cb806b1e0 2024-04-25T00:13:37.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.260+0000 7f5cbe400640 1 -- 172.21.15.83:0/1708743006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cb80724b0 con 0x7f5cb806b1e0 2024-04-25T00:13:37.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.260+0000 7f5caf400640 1 -- 172.21.15.83:0/1708743006 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5176 (secure 0 0 0) 0x7f5ca8056cf0 con 0x7f5cb806b1e0 2024-04-25T00:13:37.264 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:37.264 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3667389928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":3667389928},{"type":"v1","addr":"172.21.15.86:6835","nonce":3667389928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5923,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2174516900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2174516900},{"type":"v1","addr":"172.21.15.83:6837","nonce":2174516900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2877383406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2877383406},{"type":"v1","addr":"172.21.15.83:6836","nonce":2877383406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:13:19.805315+0000","modified":"2024-04-25T00:13:22.834474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6834/52944842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":52944842},{"type":"v1","addr":"172.21.15.86:6834","nonce":52944842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-04-25T00:13:37.264 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 35 2024-04-25T00:13:37.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.264+0000 7f5cbe400640 1 -- 172.21.15.83:0/1708743006 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5c980678c0 msgr2=0x7f5c98069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.264+0000 7f5cbe400640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5c980678c0 0x7f5c98069d80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5cb81542c0 tx=0x7f5ca4008040 comp rx=0 tx=0).stop 2024-04-25T00:13:37.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.264+0000 7f5cbe400640 1 -- 172.21.15.83:0/1708743006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb806b1e0 msgr2=0x7f5cb815ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.264+0000 7f5cbe400640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb806b1e0 0x7f5cb815ec70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800d800 tx=0x7f5ca800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.264+0000 7f5cbe400640 1 -- 172.21.15.83:0/1708743006 shutdown_connections 2024-04-25T00:13:37.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.264+0000 7f5cbe400640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5c980678c0 0x7f5c98069d80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.264+0000 7f5cbe400640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cb806f9d0 0x7f5cb814e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.264+0000 7f5cbe400640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cb806f5f0 0x7f5cb815e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.264+0000 7f5cbe400640 1 --2- 172.21.15.83:0/1708743006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb806b1e0 0x7f5cb815ec70 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.264+0000 7f5cbe400640 1 -- 172.21.15.83:0/1708743006 >> 172.21.15.83:0/1708743006 conn(0x7f5cb806c2f0 msgr2=0x7f5cb806e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.264+0000 7f5cbe400640 1 -- 172.21.15.83:0/1708743006 shutdown_connections 2024-04-25T00:13:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.264+0000 7f5cbe400640 1 -- 172.21.15.83:0/1708743006 wait complete. 2024-04-25T00:13:37.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.416+0000 7f4a94000640 1 Processor -- start 2024-04-25T00:13:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.416+0000 7f4a94000640 1 -- start start 2024-04-25T00:13:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a94000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4a8c0cc510 0x7f4a8c15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a94000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a8c15cf90 0x7f4a8c0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a94000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a8c0ccc60 con 0x7f4a8c1599f0 2024-04-25T00:13:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a94000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4a8c159dd0 con 0x7f4a8c0cc510 2024-04-25T00:13:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a94000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8c06dba0 con 0x7f4a8c15cf90 2024-04-25T00:13:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a91600640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4a8c1599f0 0x7f4a8c15c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40652/0 (socket says 172.21.15.83:40652) 2024-04-25T00:13:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a91600640 1 -- 172.21.15.83:0/3046675571 learned_addr learned my addr 172.21.15.83:0/3046675571 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a90c00640 1 --2- 172.21.15.83:0/3046675571 >> v2:172.21.15.86:3300/0 conn(0x7f4a8c0cc510 0x7f4a8c15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a92000640 1 --2- 172.21.15.83:0/3046675571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a8c15cf90 0x7f4a8c0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a8be00640 1 -- 172.21.15.83:0/3046675571 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2654331573 0 0) 0x7f4a80001120 con 0x7f4a8c1599f0 2024-04-25T00:13:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a8be00640 1 -- 172.21.15.83:0/3046675571 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a74003530 con 0x7f4a8c1599f0 2024-04-25T00:13:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a90c00640 1 -- 172.21.15.83:0/3046675571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a8c15cf90 msgr2=0x7f4a8c0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a90c00640 1 --2- 172.21.15.83:0/3046675571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a8c15cf90 0x7f4a8c0ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a90c00640 1 -- 172.21.15.83:0/3046675571 >> v1:172.21.15.83:6789/0 conn(0x7f4a8c1599f0 legacy=0x7f4a8c15c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a90c00640 1 -- 172.21.15.83:0/3046675571 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c06de80 con 0x7f4a8c0cc510 2024-04-25T00:13:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.420+0000 7f4a92000640 1 --2- 172.21.15.83:0/3046675571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a8c15cf90 0x7f4a8c0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.424+0000 7f4a90c00640 1 --2- 172.21.15.83:0/3046675571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a8c0cc510 0x7f4a8c15ca50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4a780097f0 tx=0x7f4a7801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f2ee788840e016 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.424+0000 7f4a8be00640 1 -- 172.21.15.83:0/3046675571 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a780b0070 con 0x7f4a8c0cc510 2024-04-25T00:13:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.424+0000 7f4a8be00640 1 -- 172.21.15.83:0/3046675571 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a780ab600 con 0x7f4a8c0cc510 2024-04-25T00:13:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.424+0000 7f4a8be00640 1 -- 172.21.15.83:0/3046675571 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a780ab9d0 con 0x7f4a8c0cc510 2024-04-25T00:13:37.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.424+0000 7f4a94000640 1 -- 172.21.15.83:0/3046675571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a8c0cc510 msgr2=0x7f4a8c15ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.424+0000 7f4a94000640 1 --2- 172.21.15.83:0/3046675571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a8c0cc510 0x7f4a8c15ca50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4a780097f0 tx=0x7f4a7801c410 comp rx=0 tx=0).stop 2024-04-25T00:13:37.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.424+0000 7f4a94000640 1 -- 172.21.15.83:0/3046675571 shutdown_connections 2024-04-25T00:13:37.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.424+0000 7f4a94000640 1 --2- 172.21.15.83:0/3046675571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a8c15cf90 0x7f4a8c0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.424+0000 7f4a94000640 1 --2- 172.21.15.83:0/3046675571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a8c0cc510 0x7f4a8c15ca50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.424+0000 7f4a94000640 1 -- 172.21.15.83:0/3046675571 >> 172.21.15.83:0/3046675571 conn(0x7f4a8c0b1830 msgr2=0x7f4a8c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:37.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.424+0000 7f4a94000640 1 -- 172.21.15.83:0/3046675571 shutdown_connections 2024-04-25T00:13:37.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.424+0000 7f4a94000640 1 -- 172.21.15.83:0/3046675571 wait complete. 2024-04-25T00:13:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.424+0000 7f4a94000640 1 Processor -- start 2024-04-25T00:13:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.424+0000 7f4a94000640 1 -- start start 2024-04-25T00:13:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a94000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a8c0cc510 0x7f4a8c154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a94000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a8c1599f0 0x7f4a8c15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a94000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a8c15cf90 0x7f4a8c15b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a94000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c0cf5b0 con 0x7f4a8c0cc510 2024-04-25T00:13:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a94000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c0cf430 con 0x7f4a8c1599f0 2024-04-25T00:13:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a94000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8c0cf730 con 0x7f4a8c15cf90 2024-04-25T00:13:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a91600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a8c0cc510 0x7f4a8c154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a91600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a8c0cc510 0x7f4a8c154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34900/0 (socket says 172.21.15.83:34900) 2024-04-25T00:13:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a91600640 1 -- 172.21.15.83:0/343499836 learned_addr learned my addr 172.21.15.83:0/343499836 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a90c00640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a8c1599f0 0x7f4a8c15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a92000640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a8c15cf90 0x7f4a8c15b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a91600640 1 -- 172.21.15.83:0/343499836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a8c15cf90 msgr2=0x7f4a8c15b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a91600640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a8c15cf90 0x7f4a8c15b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a91600640 1 -- 172.21.15.83:0/343499836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a8c1599f0 msgr2=0x7f4a8c15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a91600640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a8c1599f0 0x7f4a8c15afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a91600640 1 -- 172.21.15.83:0/343499836 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c0b3830 con 0x7f4a8c0cc510 2024-04-25T00:13:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a92000640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a8c15cf90 0x7f4a8c15b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a91600640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a8c0cc510 0x7f4a8c154240 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000c760 tx=0x7f4a8000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a89600640 1 -- 172.21.15.83:0/343499836 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80017070 con 0x7f4a8c0cc510 2024-04-25T00:13:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a89600640 1 -- 172.21.15.83:0/343499836 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a80005ae0 con 0x7f4a8c0cc510 2024-04-25T00:13:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a89600640 1 -- 172.21.15.83:0/343499836 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80007e30 con 0x7f4a8c0cc510 2024-04-25T00:13:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a94000640 1 -- 172.21.15.83:0/343499836 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8c0b3b10 con 0x7f4a8c0cc510 2024-04-25T00:13:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a94000640 1 -- 172.21.15.83:0/343499836 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a8c0b40d0 con 0x7f4a8c0cc510 2024-04-25T00:13:37.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a94000640 1 -- 172.21.15.83:0/343499836 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a8c0737f0 con 0x7f4a8c0cc510 2024-04-25T00:13:37.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.428+0000 7f4a89600640 1 -- 172.21.15.83:0/343499836 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4a8001a070 con 0x7f4a8c0cc510 2024-04-25T00:13:37.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.432+0000 7f4a89600640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4a6c0678c0 0x7f4a6c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.432+0000 7f4a89600640 1 -- 172.21.15.83:0/343499836 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f4a80085190 con 0x7f4a8c0cc510 2024-04-25T00:13:37.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.432+0000 7f4a90c00640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4a6c0678c0 0x7f4a6c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.432+0000 7f4a90c00640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4a6c0678c0 0x7f4a6c069d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4a780ad000 tx=0x7f4a78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:37.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.436+0000 7f4a89600640 1 -- 172.21.15.83:0/343499836 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4a8c0ccc60 con 0x7f4a8c0cc510 2024-04-25T00:13:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.592+0000 7f4a94000640 1 -- 172.21.15.83:0/343499836 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a8c072c40 con 0x7f4a8c0cc510 2024-04-25T00:13:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.592+0000 7f4a89600640 1 -- 172.21.15.83:0/343499836 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16800 (secure 0 0 0) 0x7f4a800554d0 con 0x7f4a8c0cc510 2024-04-25T00:13:37.596 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:37.597 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":60,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:21.438568+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:18.160239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:18.800529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.596+0000 7f4a94000640 1 -- 172.21.15.83:0/343499836 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4a6c0678c0 msgr2=0x7f4a6c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.596+0000 7f4a94000640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4a6c0678c0 0x7f4a6c069d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4a780ad000 tx=0x7f4a78008040 comp rx=0 tx=0).stop 2024-04-25T00:13:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.596+0000 7f4a94000640 1 -- 172.21.15.83:0/343499836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a8c0cc510 msgr2=0x7f4a8c154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.596+0000 7f4a94000640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a8c0cc510 0x7f4a8c154240 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000c760 tx=0x7f4a8000cc20 comp rx=0 tx=0).stop 2024-04-25T00:13:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.596+0000 7f4a94000640 1 -- 172.21.15.83:0/343499836 shutdown_connections 2024-04-25T00:13:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.596+0000 7f4a94000640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4a6c0678c0 0x7f4a6c069d80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.596+0000 7f4a94000640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a8c15cf90 0x7f4a8c15b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.596+0000 7f4a94000640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a8c1599f0 0x7f4a8c15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.596+0000 7f4a94000640 1 --2- 172.21.15.83:0/343499836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a8c0cc510 0x7f4a8c154240 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.596+0000 7f4a94000640 1 -- 172.21.15.83:0/343499836 >> 172.21.15.83:0/343499836 conn(0x7f4a8c0b1830 msgr2=0x7f4a8c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:37.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.596+0000 7f4a94000640 1 -- 172.21.15.83:0/343499836 shutdown_connections 2024-04-25T00:13:37.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.596+0000 7f4a94000640 1 -- 172.21.15.83:0/343499836 wait complete. 2024-04-25T00:13:37.617 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:18.160239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:13:18.800529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:13:37.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:13:37.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.752+0000 7f3877e00640 1 Processor -- start 2024-04-25T00:13:37.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.752+0000 7f3877e00640 1 -- start start 2024-04-25T00:13:37.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.752+0000 7f3877e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f38701551e0 0x7f387015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.752+0000 7f3877e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38701555c0 0x7f3870071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.752+0000 7f3877e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f387015c560 con 0x7f387006e4c0 2024-04-25T00:13:37.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.752+0000 7f3877e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3870072460 con 0x7f38701555c0 2024-04-25T00:13:37.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.752+0000 7f3877e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f387015c7d0 con 0x7f38701551e0 2024-04-25T00:13:37.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.752+0000 7f3874a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38701555c0 0x7f3870071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.752+0000 7f3874a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38701555c0 0x7f3870071b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34912/0 (socket says 172.21.15.83:34912) 2024-04-25T00:13:37.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.752+0000 7f3874a00640 1 -- 172.21.15.83:0/1737027474 learned_addr learned my addr 172.21.15.83:0/1737027474 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:37.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.752+0000 7f3875400640 1 --2- 172.21.15.83:0/1737027474 >> v2:172.21.15.86:3300/0 conn(0x7f38701551e0 0x7f387015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f3874a00640 1 -- 172.21.15.83:0/1737027474 >> v2:172.21.15.86:3300/0 conn(0x7f38701551e0 msgr2=0x7f387015bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:37.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f3874a00640 1 --2- 172.21.15.83:0/1737027474 >> v2:172.21.15.86:3300/0 conn(0x7f38701551e0 0x7f387015bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f3874a00640 1 -- 172.21.15.83:0/1737027474 >> v1:172.21.15.86:6789/0 conn(0x7f387006e4c0 legacy=0x7f387015e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f3874a00640 1 -- 172.21.15.83:0/1737027474 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387015cab0 con 0x7f38701555c0 2024-04-25T00:13:37.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f3875400640 1 --2- 172.21.15.83:0/1737027474 >> v2:172.21.15.86:3300/0 conn(0x7f38701551e0 0x7f387015bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:37.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f3874a00640 1 --2- 172.21.15.83:0/1737027474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38701555c0 0x7f3870071b70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f385c009870 tx=0x7f385c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427b4cf3d00623ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:37.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f386fe00640 1 -- 172.21.15.83:0/1737027474 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c0ab640 con 0x7f38701555c0 2024-04-25T00:13:37.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f386fe00640 1 -- 172.21.15.83:0/1737027474 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f385c0b0070 con 0x7f38701555c0 2024-04-25T00:13:37.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f386fe00640 1 -- 172.21.15.83:0/1737027474 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c0aba00 con 0x7f38701555c0 2024-04-25T00:13:37.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f3877e00640 1 -- 172.21.15.83:0/1737027474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38701555c0 msgr2=0x7f3870071b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f3877e00640 1 --2- 172.21.15.83:0/1737027474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38701555c0 0x7f3870071b70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f385c009870 tx=0x7f385c01c520 comp rx=0 tx=0).stop 2024-04-25T00:13:37.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f3877e00640 1 -- 172.21.15.83:0/1737027474 shutdown_connections 2024-04-25T00:13:37.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f3877e00640 1 --2- 172.21.15.83:0/1737027474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38701555c0 0x7f3870071b70 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f3877e00640 1 --2- 172.21.15.83:0/1737027474 >> v2:172.21.15.86:3300/0 conn(0x7f38701551e0 0x7f387015bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.756+0000 7f3877e00640 1 -- 172.21.15.83:0/1737027474 >> 172.21.15.83:0/1737027474 conn(0x7f38700b1830 msgr2=0x7f38700b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:37.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3877e00640 1 -- 172.21.15.83:0/1737027474 shutdown_connections 2024-04-25T00:13:37.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3877e00640 1 -- 172.21.15.83:0/1737027474 wait complete. 2024-04-25T00:13:37.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3877e00640 1 Processor -- start 2024-04-25T00:13:37.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3877e00640 1 -- start start 2024-04-25T00:13:37.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3877e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f387006e4c0 0x7f387014eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3877e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38701551e0 0x7f387013e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3877e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38701555c0 0x7f38700b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3877e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3870167e10 con 0x7f38701551e0 2024-04-25T00:13:37.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3877e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3870167c90 con 0x7f387006e4c0 2024-04-25T00:13:37.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3877e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3870167f90 con 0x7f38701555c0 2024-04-25T00:13:37.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3874a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38701551e0 0x7f387013e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3874a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38701551e0 0x7f387013e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34922/0 (socket says 172.21.15.83:34922) 2024-04-25T00:13:37.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3874a00640 1 -- 172.21.15.83:0/806096846 learned_addr learned my addr 172.21.15.83:0/806096846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:37.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3875400640 1 --2- 172.21.15.83:0/806096846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f387006e4c0 0x7f387014eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3875e00640 1 --2- 172.21.15.83:0/806096846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38701555c0 0x7f38700b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3874a00640 1 -- 172.21.15.83:0/806096846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38701555c0 msgr2=0x7f38700b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3874a00640 1 --2- 172.21.15.83:0/806096846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38701555c0 0x7f38700b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3874a00640 1 -- 172.21.15.83:0/806096846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f387006e4c0 msgr2=0x7f387014eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3874a00640 1 --2- 172.21.15.83:0/806096846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f387006e4c0 0x7f387014eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3874a00640 1 -- 172.21.15.83:0/806096846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387013f300 con 0x7f38701551e0 2024-04-25T00:13:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f3874a00640 1 --2- 172.21.15.83:0/806096846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38701551e0 0x7f387013e920 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f385c0ad040 tx=0x7f385c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f386d600640 1 -- 172.21.15.83:0/806096846 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c0be3f0 con 0x7f38701551e0 2024-04-25T00:13:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f386d600640 1 -- 172.21.15.83:0/806096846 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f385c0be590 con 0x7f38701551e0 2024-04-25T00:13:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.760+0000 7f386d600640 1 -- 172.21.15.83:0/806096846 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c0be980 con 0x7f38701551e0 2024-04-25T00:13:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.764+0000 7f3877e00640 1 -- 172.21.15.83:0/806096846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387013f5e0 con 0x7f38701551e0 2024-04-25T00:13:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.764+0000 7f3877e00640 1 -- 172.21.15.83:0/806096846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387013fba0 con 0x7f38701551e0 2024-04-25T00:13:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.764+0000 7f3877e00640 1 -- 172.21.15.83:0/806096846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387004f510 con 0x7f38701551e0 2024-04-25T00:13:37.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.764+0000 7f386d600640 1 -- 172.21.15.83:0/806096846 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f385c0b43f0 con 0x7f38701551e0 2024-04-25T00:13:37.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.764+0000 7f386d600640 1 --2- 172.21.15.83:0/806096846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38500678c0 0x7f3850069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.764+0000 7f386d600640 1 -- 172.21.15.83:0/806096846 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f385c120370 con 0x7f38701551e0 2024-04-25T00:13:37.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.764+0000 7f3875400640 1 --2- 172.21.15.83:0/806096846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38500678c0 0x7f3850069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.764+0000 7f3875400640 1 --2- 172.21.15.83:0/806096846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38500678c0 0x7f3850069d80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3864003b90 tx=0x7f3864009910 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.768+0000 7f386d600640 1 -- 172.21.15.83:0/806096846 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f385c0c4060 con 0x7f38701551e0 2024-04-25T00:13:37.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.924+0000 7f3877e00640 1 -- 172.21.15.83:0/806096846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3870154ca0 con 0x7f38701551e0 2024-04-25T00:13:37.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.924+0000 7f386d600640 1 -- 172.21.15.83:0/806096846 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) ==== 93+0+10 (secure 0 0 0) 0x7f385c016110 con 0x7f38701551e0 2024-04-25T00:13:37.928 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:13:37.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.928+0000 7f3877e00640 1 -- 172.21.15.83:0/806096846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38500678c0 msgr2=0x7f3850069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.928+0000 7f3877e00640 1 --2- 172.21.15.83:0/806096846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38500678c0 0x7f3850069d80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3864003b90 tx=0x7f3864009910 comp rx=0 tx=0).stop 2024-04-25T00:13:37.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.928+0000 7f3877e00640 1 -- 172.21.15.83:0/806096846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38701551e0 msgr2=0x7f387013e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.928+0000 7f3877e00640 1 --2- 172.21.15.83:0/806096846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38701551e0 0x7f387013e920 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f385c0ad040 tx=0x7f385c01ad60 comp rx=0 tx=0).stop 2024-04-25T00:13:37.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.928+0000 7f3877e00640 1 -- 172.21.15.83:0/806096846 shutdown_connections 2024-04-25T00:13:37.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.928+0000 7f3877e00640 1 --2- 172.21.15.83:0/806096846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38500678c0 0x7f3850069d80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.928+0000 7f3877e00640 1 --2- 172.21.15.83:0/806096846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38701555c0 0x7f38700b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.928+0000 7f3877e00640 1 --2- 172.21.15.83:0/806096846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38701551e0 0x7f387013e920 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.928+0000 7f3877e00640 1 --2- 172.21.15.83:0/806096846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f387006e4c0 0x7f387014eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.928+0000 7f3877e00640 1 -- 172.21.15.83:0/806096846 >> 172.21.15.83:0/806096846 conn(0x7f38700b1830 msgr2=0x7f38700b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:37.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.928+0000 7f3877e00640 1 -- 172.21.15.83:0/806096846 shutdown_connections 2024-04-25T00:13:37.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:37.928+0000 7f3877e00640 1 -- 172.21.15.83:0/806096846 wait complete. 2024-04-25T00:13:37.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:13:38.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.084+0000 7f1335e00640 1 Processor -- start 2024-04-25T00:13:38.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.084+0000 7f1335e00640 1 -- start start 2024-04-25T00:13:38.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.084+0000 7f1335e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f133006e950 0x7f133015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.084+0000 7f1335e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f133015fe50 0x7f13300ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.084+0000 7f1335e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13300ce000 con 0x7f1330159810 2024-04-25T00:13:38.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.084+0000 7f1335e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f133006e210 con 0x7f133015fe50 2024-04-25T00:13:38.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.084+0000 7f1335e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f133006da00 con 0x7f133006e950 2024-04-25T00:13:38.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.084+0000 7f132ea00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1330159810 0x7f13300cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40668/0 (socket says 172.21.15.83:40668) 2024-04-25T00:13:38.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.084+0000 7f132ea00640 1 -- 172.21.15.83:0/2158177442 learned_addr learned my addr 172.21.15.83:0/2158177442 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:38.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.084+0000 7f132f400640 1 --2- 172.21.15.83:0/2158177442 >> v2:172.21.15.86:3300/0 conn(0x7f133015fe50 0x7f13300ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.084+0000 7f132e000640 1 --2- 172.21.15.83:0/2158177442 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f133006e950 0x7f133015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.084+0000 7f132d600640 1 -- 172.21.15.83:0/2158177442 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1798617277 0 0) 0x7f13300ce000 con 0x7f1330159810 2024-04-25T00:13:38.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.084+0000 7f132d600640 1 -- 172.21.15.83:0/2158177442 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1314003530 con 0x7f1330159810 2024-04-25T00:13:38.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 -- 172.21.15.83:0/2158177442 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1016595336 0 0) 0x7f1314003530 con 0x7f1330159810 2024-04-25T00:13:38.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 -- 172.21.15.83:0/2158177442 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f133006e950 msgr2=0x7f133015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 --2- 172.21.15.83:0/2158177442 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f133006e950 0x7f133015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 -- 172.21.15.83:0/2158177442 >> v2:172.21.15.86:3300/0 conn(0x7f133015fe50 msgr2=0x7f13300ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 --2- 172.21.15.83:0/2158177442 >> v2:172.21.15.86:3300/0 conn(0x7f133015fe50 0x7f13300ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 -- 172.21.15.83:0/2158177442 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133006eec0 con 0x7f1330159810 2024-04-25T00:13:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 -- 172.21.15.83:0/2158177442 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f1320001680 con 0x7f1330159810 2024-04-25T00:13:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 -- 172.21.15.83:0/2158177442 >> v1:172.21.15.83:6789/0 conn(0x7f1330159810 legacy=0x7f13300cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 --2- 172.21.15.83:0/2158177442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1314003a30 0x7f1314003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 --2- 172.21.15.83:0/2158177442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1314004810 0x7f1314004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 --2- 172.21.15.83:0/2158177442 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f133006e950 0x7f13140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 -- 172.21.15.83:0/2158177442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f133006e210 con 0x7f1314003a30 2024-04-25T00:13:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 -- 172.21.15.83:0/2158177442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f133006da00 con 0x7f1314004810 2024-04-25T00:13:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 -- 172.21.15.83:0/2158177442 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f131400ba50 con 0x7f133006e950 2024-04-25T00:13:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132ea00640 1 --2- 172.21.15.83:0/2158177442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1314003a30 0x7f1314003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132f400640 1 --2- 172.21.15.83:0/2158177442 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f133006e950 0x7f13140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132e000640 1 --2- 172.21.15.83:0/2158177442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1314004810 0x7f1314004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132ea00640 1 -- 172.21.15.83:0/2158177442 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f133006e950 msgr2=0x7f13140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132ea00640 1 --2- 172.21.15.83:0/2158177442 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f133006e950 0x7f13140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132ea00640 1 -- 172.21.15.83:0/2158177442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1314004810 msgr2=0x7f1314004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132ea00640 1 --2- 172.21.15.83:0/2158177442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1314004810 0x7f1314004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132ea00640 1 -- 172.21.15.83:0/2158177442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1314003530 con 0x7f1314003a30 2024-04-25T00:13:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132f400640 1 --2- 172.21.15.83:0/2158177442 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f133006e950 0x7f13140051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132ea00640 1 --2- 172.21.15.83:0/2158177442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1314003a30 0x7f1314003e50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f132000c2c0 tx=0x7f132001cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75abce716e4f8241 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 -- 172.21.15.83:0/2158177442 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320007d60 con 0x7f1314003a30 2024-04-25T00:13:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 -- 172.21.15.83:0/2158177442 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13200a9020 con 0x7f1314003a30 2024-04-25T00:13:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.088+0000 7f132d600640 1 -- 172.21.15.83:0/2158177442 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320005ae0 con 0x7f1314003a30 2024-04-25T00:13:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f1335e00640 1 -- 172.21.15.83:0/2158177442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1314003a30 msgr2=0x7f1314003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f1335e00640 1 --2- 172.21.15.83:0/2158177442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1314003a30 0x7f1314003e50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f132000c2c0 tx=0x7f132001cd40 comp rx=0 tx=0).stop 2024-04-25T00:13:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f132f400640 1 -- 172.21.15.83:0/2158177442 reap_dead start 2024-04-25T00:13:38.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f1335e00640 1 -- 172.21.15.83:0/2158177442 shutdown_connections 2024-04-25T00:13:38.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f1335e00640 1 -- 172.21.15.83:0/2158177442 >> 172.21.15.83:0/2158177442 conn(0x7f13300b1830 msgr2=0x7f13300b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:38.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f1335e00640 1 -- 172.21.15.83:0/2158177442 shutdown_connections 2024-04-25T00:13:38.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f1335e00640 1 -- 172.21.15.83:0/2158177442 wait complete. 2024-04-25T00:13:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f1335e00640 1 Processor -- start 2024-04-25T00:13:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f1335e00640 1 -- start start 2024-04-25T00:13:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f1335e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1314003a30 0x7f13301f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f1335e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f133013afb0 0x7f1330136f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f1335e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133015e680 0x7f133015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f1335e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1330070f20 con 0x7f133013afb0 2024-04-25T00:13:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f1335e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1330070da0 con 0x7f133015e680 2024-04-25T00:13:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f1335e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1330166ca0 con 0x7f1314003a30 2024-04-25T00:13:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f132e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f133013afb0 0x7f1330136f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f132ea00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1314003a30 0x7f13301f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f132e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f133013afb0 0x7f1330136f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34940/0 (socket says 172.21.15.83:34940) 2024-04-25T00:13:38.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f132e000640 1 -- 172.21.15.83:0/2324399370 learned_addr learned my addr 172.21.15.83:0/2324399370 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:38.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f132f400640 1 --2- 172.21.15.83:0/2324399370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133015e680 0x7f133015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f132e000640 1 -- 172.21.15.83:0/2324399370 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1314003a30 msgr2=0x7f13301f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f132e000640 1 --2- 172.21.15.83:0/2324399370 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1314003a30 0x7f13301f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f132e000640 1 -- 172.21.15.83:0/2324399370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133015e680 msgr2=0x7f133015eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f132e000640 1 --2- 172.21.15.83:0/2324399370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133015e680 0x7f133015eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.092+0000 7f132e000640 1 -- 172.21.15.83:0/2324399370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1330134f10 con 0x7f133013afb0 2024-04-25T00:13:38.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.096+0000 7f132e000640 1 --2- 172.21.15.83:0/2324399370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f133013afb0 0x7f1330136f60 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f13180115c0 tx=0x7f1318011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:38.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.096+0000 7f1327400640 1 -- 172.21.15.83:0/2324399370 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318015930 con 0x7f133013afb0 2024-04-25T00:13:38.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.096+0000 7f1327400640 1 -- 172.21.15.83:0/2324399370 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1318015ad0 con 0x7f133013afb0 2024-04-25T00:13:38.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.096+0000 7f1327400640 1 -- 172.21.15.83:0/2324399370 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131801d400 con 0x7f133013afb0 2024-04-25T00:13:38.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.096+0000 7f1335e00640 1 -- 172.21.15.83:0/2324399370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13301f5720 con 0x7f133013afb0 2024-04-25T00:13:38.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.096+0000 7f1335e00640 1 -- 172.21.15.83:0/2324399370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1330137ad0 con 0x7f133013afb0 2024-04-25T00:13:38.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.096+0000 7f1335e00640 1 -- 172.21.15.83:0/2324399370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133004f510 con 0x7f133013afb0 2024-04-25T00:13:38.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.096+0000 7f1327400640 1 -- 172.21.15.83:0/2324399370 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f131801d5a0 con 0x7f133013afb0 2024-04-25T00:13:38.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.096+0000 7f1327400640 1 --2- 172.21.15.83:0/2324399370 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f130c0678c0 0x7f130c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.096+0000 7f1327400640 1 -- 172.21.15.83:0/2324399370 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f13180866f0 con 0x7f133013afb0 2024-04-25T00:13:38.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.096+0000 7f132ea00640 1 --2- 172.21.15.83:0/2324399370 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f130c0678c0 0x7f130c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.100+0000 7f132ea00640 1 --2- 172.21.15.83:0/2324399370 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f130c0678c0 0x7f130c069d80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f13200aa000 tx=0x7f13200b1040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:38.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.100+0000 7f1327400640 1 -- 172.21.15.83:0/2324399370 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1318019030 con 0x7f133013afb0 2024-04-25T00:13:38.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.256+0000 7f1335e00640 1 -- 172.21.15.83:0/2324399370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1330165a90 con 0x7f133013afb0 2024-04-25T00:13:38.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.256+0000 7f1327400640 1 -- 172.21.15.83:0/2324399370 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) ==== 104+0+11 (secure 0 0 0) 0x7f1318057000 con 0x7f133013afb0 2024-04-25T00:13:38.260 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:13:38.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.260+0000 7f1335e00640 1 -- 172.21.15.83:0/2324399370 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f130c0678c0 msgr2=0x7f130c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.260+0000 7f1335e00640 1 --2- 172.21.15.83:0/2324399370 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f130c0678c0 0x7f130c069d80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f13200aa000 tx=0x7f13200b1040 comp rx=0 tx=0).stop 2024-04-25T00:13:38.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.260+0000 7f1335e00640 1 -- 172.21.15.83:0/2324399370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f133013afb0 msgr2=0x7f1330136f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.260+0000 7f1335e00640 1 --2- 172.21.15.83:0/2324399370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f133013afb0 0x7f1330136f60 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f13180115c0 tx=0x7f1318011a80 comp rx=0 tx=0).stop 2024-04-25T00:13:38.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.260+0000 7f1335e00640 1 -- 172.21.15.83:0/2324399370 shutdown_connections 2024-04-25T00:13:38.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.260+0000 7f1335e00640 1 --2- 172.21.15.83:0/2324399370 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f130c0678c0 0x7f130c069d80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.260+0000 7f1335e00640 1 --2- 172.21.15.83:0/2324399370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133015e680 0x7f133015eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.260+0000 7f1335e00640 1 --2- 172.21.15.83:0/2324399370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f133013afb0 0x7f1330136f60 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.260+0000 7f1335e00640 1 --2- 172.21.15.83:0/2324399370 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1314003a30 0x7f13301f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.260+0000 7f1335e00640 1 -- 172.21.15.83:0/2324399370 >> 172.21.15.83:0/2324399370 conn(0x7f13300b1830 msgr2=0x7f13300d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:38.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.260+0000 7f1335e00640 1 -- 172.21.15.83:0/2324399370 shutdown_connections 2024-04-25T00:13:38.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.260+0000 7f1335e00640 1 -- 172.21.15.83:0/2324399370 wait complete. 2024-04-25T00:13:38.281 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:13:38.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ae8c00640 1 Processor -- start 2024-04-25T00:13:38.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ae8c00640 1 -- start start 2024-04-25T00:13:38.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ae8c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4ae415a330 0x7f4ae40b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ae8c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ae415a710 0x7f4ae4070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ae8c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ae4071850 con 0x7f4ae406ddc0 2024-04-25T00:13:38.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ae8c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4ae40cc690 con 0x7f4ae415a330 2024-04-25T00:13:38.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ae8c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae406e920 con 0x7f4ae415a710 2024-04-25T00:13:38.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ae1e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4ae415a330 0x7f4ae40b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ae1e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4ae415a330 0x7f4ae40b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34952/0 (socket says 172.21.15.83:34952) 2024-04-25T00:13:38.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ae1e00640 1 -- 172.21.15.83:0/1142468675 learned_addr learned my addr 172.21.15.83:0/1142468675 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:38.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ad9400640 1 --2- 172.21.15.83:0/1142468675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ae415a710 0x7f4ae4070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ae1e00640 1 -- 172.21.15.83:0/1142468675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ae415a710 msgr2=0x7f4ae4070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:38.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ae1e00640 1 --2- 172.21.15.83:0/1142468675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ae415a710 0x7f4ae4070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ae1e00640 1 -- 172.21.15.83:0/1142468675 >> v1:172.21.15.86:6789/0 conn(0x7f4ae406ddc0 legacy=0x7f4ae406e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.416+0000 7f4ae1e00640 1 -- 172.21.15.83:0/1142468675 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae40b41b0 con 0x7f4ae415a330 2024-04-25T00:13:38.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.420+0000 7f4ae1e00640 1 --2- 172.21.15.83:0/1142468675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ae415a330 0x7f4ae40b4a40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009e50 tx=0x7f4ad401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db21e904320e1a11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:38.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.420+0000 7f4ae1400640 1 -- 172.21.15.83:0/1142468675 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad40a5700 con 0x7f4ae415a330 2024-04-25T00:13:38.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.420+0000 7f4ae1400640 1 -- 172.21.15.83:0/1142468675 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad40a58a0 con 0x7f4ae415a330 2024-04-25T00:13:38.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.420+0000 7f4ae1400640 1 -- 172.21.15.83:0/1142468675 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad40a5c70 con 0x7f4ae415a330 2024-04-25T00:13:38.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.420+0000 7f4ae8c00640 1 -- 172.21.15.83:0/1142468675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ae415a330 msgr2=0x7f4ae40b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.420+0000 7f4ae8c00640 1 --2- 172.21.15.83:0/1142468675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ae415a330 0x7f4ae40b4a40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009e50 tx=0x7f4ad401ca30 comp rx=0 tx=0).stop 2024-04-25T00:13:38.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.420+0000 7f4ae8c00640 1 -- 172.21.15.83:0/1142468675 shutdown_connections 2024-04-25T00:13:38.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.420+0000 7f4ae8c00640 1 --2- 172.21.15.83:0/1142468675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ae415a710 0x7f4ae4070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.420+0000 7f4ae8c00640 1 --2- 172.21.15.83:0/1142468675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ae415a330 0x7f4ae40b4a40 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.420+0000 7f4ae8c00640 1 -- 172.21.15.83:0/1142468675 >> 172.21.15.83:0/1142468675 conn(0x7f4ae40b1830 msgr2=0x7f4ae40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:38.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.420+0000 7f4ae8c00640 1 -- 172.21.15.83:0/1142468675 shutdown_connections 2024-04-25T00:13:38.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.420+0000 7f4ae8c00640 1 -- 172.21.15.83:0/1142468675 wait complete. 2024-04-25T00:13:38.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae8c00640 1 Processor -- start 2024-04-25T00:13:38.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae8c00640 1 -- start start 2024-04-25T00:13:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ae406ddc0 0x7f4ae41f9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ae415a330 0x7f4ae40ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae8c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ae40ae7b0 0x7f4ae413f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae8c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae415e200 con 0x7f4ae406ddc0 2024-04-25T00:13:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae8c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae415e080 con 0x7f4ae415a330 2024-04-25T00:13:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae8c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae415e380 con 0x7f4ae40ae7b0 2024-04-25T00:13:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ae406ddc0 0x7f4ae41f9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ae406ddc0 0x7f4ae41f9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34964/0 (socket says 172.21.15.83:34964) 2024-04-25T00:13:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae1e00640 1 -- 172.21.15.83:0/3110068530 learned_addr learned my addr 172.21.15.83:0/3110068530 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ad9400640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ae415a330 0x7f4ae40ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae2800640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ae40ae7b0 0x7f4ae413f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae1e00640 1 -- 172.21.15.83:0/3110068530 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ae40ae7b0 msgr2=0x7f4ae413f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae1e00640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ae40ae7b0 0x7f4ae413f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae1e00640 1 -- 172.21.15.83:0/3110068530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ae415a330 msgr2=0x7f4ae40ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae1e00640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ae415a330 0x7f4ae40ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae1e00640 1 -- 172.21.15.83:0/3110068530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae40aed30 con 0x7f4ae406ddc0 2024-04-25T00:13:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ad9400640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ae415a330 0x7f4ae40ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae1e00640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ae406ddc0 0x7f4ae41f9c50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4ad401a660 tx=0x7f4ad40197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4adaa00640 1 -- 172.21.15.83:0/3110068530 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad40bc030 con 0x7f4ae406ddc0 2024-04-25T00:13:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4adaa00640 1 -- 172.21.15.83:0/3110068530 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad40b8470 con 0x7f4ae406ddc0 2024-04-25T00:13:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae8c00640 1 -- 172.21.15.83:0/3110068530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae40af010 con 0x7f4ae406ddc0 2024-04-25T00:13:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4adaa00640 1 -- 172.21.15.83:0/3110068530 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad40b88a0 con 0x7f4ae406ddc0 2024-04-25T00:13:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.424+0000 7f4ae8c00640 1 -- 172.21.15.83:0/3110068530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae413d610 con 0x7f4ae406ddc0 2024-04-25T00:13:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.428+0000 7f4ae8c00640 1 -- 172.21.15.83:0/3110068530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae413d120 con 0x7f4ae406ddc0 2024-04-25T00:13:38.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.428+0000 7f4adaa00640 1 -- 172.21.15.83:0/3110068530 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4ad40bf070 con 0x7f4ae406ddc0 2024-04-25T00:13:38.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.428+0000 7f4adaa00640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ac00678f0 0x7f4ac0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.428+0000 7f4adaa00640 1 -- 172.21.15.83:0/3110068530 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f4ad41208d0 con 0x7f4ae406ddc0 2024-04-25T00:13:38.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.428+0000 7f4ad9400640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ac00678f0 0x7f4ac0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.428+0000 7f4ad9400640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ac00678f0 0x7f4ac0069db0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40b1df0 tx=0x7f4acc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:38.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.432+0000 7f4adaa00640 1 -- 172.21.15.83:0/3110068530 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4ad4016620 con 0x7f4ae406ddc0 2024-04-25T00:13:38.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.588+0000 7f4ae8c00640 1 -- 172.21.15.83:0/3110068530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4ae404f510 con 0x7f4ae406ddc0 2024-04-25T00:13:38.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.588+0000 7f4adaa00640 1 -- 172.21.15.83:0/3110068530 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) ==== 100+0+11 (secure 0 0 0) 0x7f4ad40f1330 con 0x7f4ae406ddc0 2024-04-25T00:13:38.593 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:13:38.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.592+0000 7f4ae8c00640 1 -- 172.21.15.83:0/3110068530 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ac00678f0 msgr2=0x7f4ac0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.592+0000 7f4ae8c00640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ac00678f0 0x7f4ac0069db0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40b1df0 tx=0x7f4acc008040 comp rx=0 tx=0).stop 2024-04-25T00:13:38.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.592+0000 7f4ae8c00640 1 -- 172.21.15.83:0/3110068530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ae406ddc0 msgr2=0x7f4ae41f9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.592+0000 7f4ae8c00640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ae406ddc0 0x7f4ae41f9c50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4ad401a660 tx=0x7f4ad40197d0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.592+0000 7f4ae8c00640 1 -- 172.21.15.83:0/3110068530 shutdown_connections 2024-04-25T00:13:38.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.592+0000 7f4ae8c00640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ac00678f0 0x7f4ac0069db0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.592+0000 7f4ae8c00640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ae40ae7b0 0x7f4ae413f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.592+0000 7f4ae8c00640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ae415a330 0x7f4ae40ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.592+0000 7f4ae8c00640 1 --2- 172.21.15.83:0/3110068530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ae406ddc0 0x7f4ae41f9c50 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.592+0000 7f4ae8c00640 1 -- 172.21.15.83:0/3110068530 >> 172.21.15.83:0/3110068530 conn(0x7f4ae40b1830 msgr2=0x7f4ae406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:38.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.592+0000 7f4ae8c00640 1 -- 172.21.15.83:0/3110068530 shutdown_connections 2024-04-25T00:13:38.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.592+0000 7f4ae8c00640 1 -- 172.21.15.83:0/3110068530 wait complete. 2024-04-25T00:13:38.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e25200640 1 Processor -- start 2024-04-25T00:13:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e25200640 1 -- start start 2024-04-25T00:13:38.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e25200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e2015e4f0 0x7f5e200ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e25200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e200d0ad0 con 0x7f5e2006fdc0 2024-04-25T00:13:38.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e25200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e200d0db0 con 0x7f5e2015e110 2024-04-25T00:13:38.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e25200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5e200cc3b0 con 0x7f5e2015e4f0 2024-04-25T00:13:38.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1fe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5e2006fdc0 0x7f5e200ccbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40684/0 (socket says 172.21.15.83:40684) 2024-04-25T00:13:38.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1fe00640 1 -- 172.21.15.83:0/3624987047 learned_addr learned my addr 172.21.15.83:0/3624987047 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1ea00640 1 --2- 172.21.15.83:0/3624987047 >> v2:172.21.15.86:3300/0 conn(0x7f5e2015e4f0 0x7f5e200ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 -- 172.21.15.83:0/3624987047 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 544058676 0 0) 0x7f5e200d0ad0 con 0x7f5e2006fdc0 2024-04-25T00:13:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 -- 172.21.15.83:0/3624987047 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e04003530 con 0x7f5e2006fdc0 2024-04-25T00:13:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 -- 172.21.15.83:0/3624987047 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1905287832 0 0) 0x7f5e04003530 con 0x7f5e2006fdc0 2024-04-25T00:13:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 -- 172.21.15.83:0/3624987047 >> v2:172.21.15.86:3300/0 conn(0x7f5e2015e4f0 msgr2=0x7f5e200ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 --2- 172.21.15.83:0/3624987047 >> v2:172.21.15.86:3300/0 conn(0x7f5e2015e4f0 0x7f5e200ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 -- 172.21.15.83:0/3624987047 >> v1:172.21.15.86:6789/0 conn(0x7f5e2015e110 legacy=0x7f5e200ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 -- 172.21.15.83:0/3624987047 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e200d0760 con 0x7f5e2006fdc0 2024-04-25T00:13:38.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 -- 172.21.15.83:0/3624987047 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f5e18001a00 con 0x7f5e2006fdc0 2024-04-25T00:13:38.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 -- 172.21.15.83:0/3624987047 >> v1:172.21.15.83:6789/0 conn(0x7f5e2006fdc0 legacy=0x7f5e200ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 --2- 172.21.15.83:0/3624987047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e040039c0 0x7f5e04003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 --2- 172.21.15.83:0/3624987047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e04004810 0x7f5e04004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 --2- 172.21.15.83:0/3624987047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e040052f0 0x7f5e0400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 -- 172.21.15.83:0/3624987047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e200cc3b0 con 0x7f5e040039c0 2024-04-25T00:13:38.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 -- 172.21.15.83:0/3624987047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0400c1c0 con 0x7f5e040052f0 2024-04-25T00:13:38.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1e000640 1 -- 172.21.15.83:0/3624987047 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5e0400c3c0 con 0x7f5e04004810 2024-04-25T00:13:38.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1f400640 1 --2- 172.21.15.83:0/3624987047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e040039c0 0x7f5e04003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1fe00640 1 --2- 172.21.15.83:0/3624987047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e040052f0 0x7f5e0400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1ea00640 1 --2- 172.21.15.83:0/3624987047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e04004810 0x7f5e04004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1f400640 1 -- 172.21.15.83:0/3624987047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e04004810 msgr2=0x7f5e04004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:38.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1f400640 1 --2- 172.21.15.83:0/3624987047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e04004810 0x7f5e04004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1f400640 1 -- 172.21.15.83:0/3624987047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e040052f0 msgr2=0x7f5e0400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:38.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1f400640 1 --2- 172.21.15.83:0/3624987047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e040052f0 0x7f5e0400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1f400640 1 -- 172.21.15.83:0/3624987047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e04003530 con 0x7f5e040039c0 2024-04-25T00:13:38.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.748+0000 7f5e1fe00640 1 -- 172.21.15.83:0/3624987047 reap_dead start 2024-04-25T00:13:38.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.752+0000 7f5e1f400640 1 --2- 172.21.15.83:0/3624987047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e040039c0 0x7f5e04003de0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5e10001750 tx=0x7f5e1001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72cbfe6f122c1715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:38.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.752+0000 7f5e1e000640 1 -- 172.21.15.83:0/3624987047 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100013a0 con 0x7f5e040039c0 2024-04-25T00:13:38.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.752+0000 7f5e1e000640 1 -- 172.21.15.83:0/3624987047 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e100052f0 con 0x7f5e040039c0 2024-04-25T00:13:38.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.752+0000 7f5e1e000640 1 -- 172.21.15.83:0/3624987047 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10003840 con 0x7f5e040039c0 2024-04-25T00:13:38.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.752+0000 7f5e25200640 1 -- 172.21.15.83:0/3624987047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e040039c0 msgr2=0x7f5e04003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.752+0000 7f5e25200640 1 --2- 172.21.15.83:0/3624987047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e040039c0 0x7f5e04003de0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5e10001750 tx=0x7f5e1001c3c0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.752+0000 7f5e25200640 1 -- 172.21.15.83:0/3624987047 shutdown_connections 2024-04-25T00:13:38.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.752+0000 7f5e25200640 1 --2- 172.21.15.83:0/3624987047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e040039c0 0x7f5e04003de0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.752+0000 7f5e25200640 1 -- 172.21.15.83:0/3624987047 >> 172.21.15.83:0/3624987047 conn(0x7f5e200d1bc0 msgr2=0x7f5e200d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:38.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.752+0000 7f5e25200640 1 -- 172.21.15.83:0/3624987047 shutdown_connections 2024-04-25T00:13:38.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.752+0000 7f5e25200640 1 -- 172.21.15.83:0/3624987047 wait complete. 2024-04-25T00:13:38.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e25200640 1 Processor -- start 2024-04-25T00:13:38.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e25200640 1 -- start start 2024-04-25T00:13:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e25200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e040039c0 0x7f5e2015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e25200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e20161380 0x7f5e20161800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e25200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e200cd000 0x7f5e200cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e25200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e200cefe0 con 0x7f5e20161380 2024-04-25T00:13:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e25200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e200cee60 con 0x7f5e040039c0 2024-04-25T00:13:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e25200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5e200cf160 con 0x7f5e200cd000 2024-04-25T00:13:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e1ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e20161380 0x7f5e20161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e1ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e20161380 0x7f5e20161800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34980/0 (socket says 172.21.15.83:34980) 2024-04-25T00:13:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e1ea00640 1 -- 172.21.15.83:0/3294225293 learned_addr learned my addr 172.21.15.83:0/3294225293 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e1f400640 1 --2- 172.21.15.83:0/3294225293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e040039c0 0x7f5e2015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e1fe00640 1 --2- 172.21.15.83:0/3294225293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e200cd000 0x7f5e200cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e1ea00640 1 -- 172.21.15.83:0/3294225293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e200cd000 msgr2=0x7f5e200cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e1ea00640 1 --2- 172.21.15.83:0/3294225293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e200cd000 0x7f5e200cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e1ea00640 1 -- 172.21.15.83:0/3294225293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e040039c0 msgr2=0x7f5e2015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e1ea00640 1 --2- 172.21.15.83:0/3294225293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e040039c0 0x7f5e2015ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e1ea00640 1 -- 172.21.15.83:0/3294225293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e201f9bc0 con 0x7f5e20161380 2024-04-25T00:13:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e1ea00640 1 --2- 172.21.15.83:0/3294225293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e20161380 0x7f5e20161800 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c011530 tx=0x7f5e0c0119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e17400640 1 -- 172.21.15.83:0/3294225293 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c015870 con 0x7f5e20161380 2024-04-25T00:13:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e17400640 1 -- 172.21.15.83:0/3294225293 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e0c015a10 con 0x7f5e20161380 2024-04-25T00:13:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e17400640 1 -- 172.21.15.83:0/3294225293 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c015dc0 con 0x7f5e20161380 2024-04-25T00:13:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e25200640 1 -- 172.21.15.83:0/3294225293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e201f9ea0 con 0x7f5e20161380 2024-04-25T00:13:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.756+0000 7f5e25200640 1 -- 172.21.15.83:0/3294225293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e201fa460 con 0x7f5e20161380 2024-04-25T00:13:38.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.760+0000 7f5e25200640 1 -- 172.21.15.83:0/3294225293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e2004f510 con 0x7f5e20161380 2024-04-25T00:13:38.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.760+0000 7f5e17400640 1 -- 172.21.15.83:0/3294225293 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5e0c013400 con 0x7f5e20161380 2024-04-25T00:13:38.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.760+0000 7f5e17400640 1 --2- 172.21.15.83:0/3294225293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5e000678c0 0x7f5e00069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:38.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.760+0000 7f5e17400640 1 -- 172.21.15.83:0/3294225293 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f5e0c0865b0 con 0x7f5e20161380 2024-04-25T00:13:38.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.760+0000 7f5e1f400640 1 --2- 172.21.15.83:0/3294225293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5e000678c0 0x7f5e00069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:38.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.760+0000 7f5e1f400640 1 --2- 172.21.15.83:0/3294225293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5e000678c0 0x7f5e00069d80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5e040057f0 tx=0x7f5e100011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:38.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.764+0000 7f5e17400640 1 -- 172.21.15.83:0/3294225293 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5e0c019030 con 0x7f5e20161380 2024-04-25T00:13:38.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.952+0000 7f5e25200640 1 -- 172.21.15.83:0/3294225293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e200d1550 con 0x7f5e20161380 2024-04-25T00:13:38.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.956+0000 7f5e17400640 1 -- 172.21.15.83:0/3294225293 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5176 (secure 0 0 0) 0x7f5e0c056f60 con 0x7f5e20161380 2024-04-25T00:13:38.958 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:38.958 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3667389928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":3667389928},{"type":"v1","addr":"172.21.15.86:6835","nonce":3667389928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5923,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2174516900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2174516900},{"type":"v1","addr":"172.21.15.83:6837","nonce":2174516900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2877383406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2877383406},{"type":"v1","addr":"172.21.15.83:6836","nonce":2877383406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:13:19.805315+0000","modified":"2024-04-25T00:13:22.834474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6834/52944842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":52944842},{"type":"v1","addr":"172.21.15.86:6834","nonce":52944842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-04-25T00:13:38.958 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 35 2024-04-25T00:13:38.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.956+0000 7f5e25200640 1 -- 172.21.15.83:0/3294225293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5e000678c0 msgr2=0x7f5e00069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.956+0000 7f5e25200640 1 --2- 172.21.15.83:0/3294225293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5e000678c0 0x7f5e00069d80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5e040057f0 tx=0x7f5e100011a0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.956+0000 7f5e25200640 1 -- 172.21.15.83:0/3294225293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e20161380 msgr2=0x7f5e20161800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:38.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.956+0000 7f5e25200640 1 --2- 172.21.15.83:0/3294225293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e20161380 0x7f5e20161800 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c011530 tx=0x7f5e0c0119f0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.956+0000 7f5e25200640 1 -- 172.21.15.83:0/3294225293 shutdown_connections 2024-04-25T00:13:38.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.956+0000 7f5e25200640 1 --2- 172.21.15.83:0/3294225293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5e000678c0 0x7f5e00069d80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.956+0000 7f5e25200640 1 --2- 172.21.15.83:0/3294225293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e200cd000 0x7f5e200cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.956+0000 7f5e25200640 1 --2- 172.21.15.83:0/3294225293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e20161380 0x7f5e20161800 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.956+0000 7f5e25200640 1 --2- 172.21.15.83:0/3294225293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e040039c0 0x7f5e2015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:38.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.956+0000 7f5e25200640 1 -- 172.21.15.83:0/3294225293 >> 172.21.15.83:0/3294225293 conn(0x7f5e200d1bc0 msgr2=0x7f5e2015d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:38.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.956+0000 7f5e25200640 1 -- 172.21.15.83:0/3294225293 shutdown_connections 2024-04-25T00:13:38.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:38.956+0000 7f5e25200640 1 -- 172.21.15.83:0/3294225293 wait complete. 2024-04-25T00:13:38.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:39.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.108+0000 7fdf65800640 1 Processor -- start 2024-04-25T00:13:39.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.112+0000 7fdf65800640 1 -- start start 2024-04-25T00:13:39.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.112+0000 7fdf65800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdf6006b1e0 0x7fdf600b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.112+0000 7fdf65800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf600b0520 con 0x7fdf6015f6e0 2024-04-25T00:13:39.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.112+0000 7fdf65800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf6015e6f0 con 0x7fdf600ce090 2024-04-25T00:13:39.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.112+0000 7fdf65800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fdf6015e9d0 con 0x7fdf6006b1e0 2024-04-25T00:13:39.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.112+0000 7fdf5fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdf6006b1e0 0x7fdf600b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.112+0000 7fdf64e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdf6015f6e0 0x7fdf60071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40692/0 (socket says 172.21.15.83:40692) 2024-04-25T00:13:39.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.112+0000 7fdf64e00640 1 -- 172.21.15.83:0/1245996234 learned_addr learned my addr 172.21.15.83:0/1245996234 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:39.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.112+0000 7fdf5fe00640 1 -- 172.21.15.83:0/1245996234 >> v1:172.21.15.86:6789/0 conn(0x7fdf600ce090 legacy=0x7fdf6006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.112+0000 7fdf5fe00640 1 -- 172.21.15.83:0/1245996234 >> v1:172.21.15.83:6789/0 conn(0x7fdf6015f6e0 legacy=0x7fdf60071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.112+0000 7fdf5fe00640 1 -- 172.21.15.83:0/1245996234 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6015ecb0 con 0x7fdf6006b1e0 2024-04-25T00:13:39.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.112+0000 7fdf5ea00640 1 -- 172.21.15.83:0/1245996234 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4016740101 0 0) 0x7fdf600b0520 con 0x7fdf6015f6e0 2024-04-25T00:13:39.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.116+0000 7fdf5fe00640 1 --2- 172.21.15.83:0/1245996234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6006b1e0 0x7fdf600b59c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009d90 tx=0x7fdf5001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970cbc90ae2f1b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:39.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.116+0000 7fdf5ea00640 1 -- 172.21.15.83:0/1245996234 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf500b2070 con 0x7fdf6006b1e0 2024-04-25T00:13:39.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.116+0000 7fdf5ea00640 1 -- 172.21.15.83:0/1245996234 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf500a5870 con 0x7fdf6006b1e0 2024-04-25T00:13:39.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.116+0000 7fdf5ea00640 1 -- 172.21.15.83:0/1245996234 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf500a5c40 con 0x7fdf6006b1e0 2024-04-25T00:13:39.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.116+0000 7fdf65800640 1 -- 172.21.15.83:0/1245996234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6006b1e0 msgr2=0x7fdf600b59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.116+0000 7fdf65800640 1 --2- 172.21.15.83:0/1245996234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6006b1e0 0x7fdf600b59c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009d90 tx=0x7fdf5001c8e0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.116+0000 7fdf65800640 1 -- 172.21.15.83:0/1245996234 shutdown_connections 2024-04-25T00:13:39.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.116+0000 7fdf65800640 1 --2- 172.21.15.83:0/1245996234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6006b1e0 0x7fdf600b59c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.116+0000 7fdf65800640 1 -- 172.21.15.83:0/1245996234 >> 172.21.15.83:0/1245996234 conn(0x7fdf600cd500 msgr2=0x7fdf600cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.116+0000 7fdf65800640 1 -- 172.21.15.83:0/1245996234 shutdown_connections 2024-04-25T00:13:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.116+0000 7fdf65800640 1 -- 172.21.15.83:0/1245996234 wait complete. 2024-04-25T00:13:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.116+0000 7fdf65800640 1 Processor -- start 2024-04-25T00:13:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.116+0000 7fdf65800640 1 -- start start 2024-04-25T00:13:39.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf65800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf6006b1e0 0x7fdf6006f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf65800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf600ce090 0x7fdf60154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf65800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6015f6e0 0x7fdf601430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf65800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6015e4a0 con 0x7fdf6015f6e0 2024-04-25T00:13:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf65800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6015e320 con 0x7fdf600ce090 2024-04-25T00:13:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf65800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6016c4c0 con 0x7fdf6006b1e0 2024-04-25T00:13:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf64e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6015f6e0 0x7fdf601430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf64e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6015f6e0 0x7fdf601430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34998/0 (socket says 172.21.15.83:34998) 2024-04-25T00:13:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf64e00640 1 -- 172.21.15.83:0/2768782436 learned_addr learned my addr 172.21.15.83:0/2768782436 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf5f400640 1 --2- 172.21.15.83:0/2768782436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf600ce090 0x7fdf60154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf5fe00640 1 --2- 172.21.15.83:0/2768782436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf6006b1e0 0x7fdf6006f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf64e00640 1 -- 172.21.15.83:0/2768782436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf6006b1e0 msgr2=0x7fdf6006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf64e00640 1 --2- 172.21.15.83:0/2768782436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf6006b1e0 0x7fdf6006f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf64e00640 1 -- 172.21.15.83:0/2768782436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf600ce090 msgr2=0x7fdf60154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf64e00640 1 --2- 172.21.15.83:0/2768782436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf600ce090 0x7fdf60154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf64e00640 1 -- 172.21.15.83:0/2768782436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf60155950 con 0x7fdf6015f6e0 2024-04-25T00:13:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf64e00640 1 --2- 172.21.15.83:0/2768782436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6015f6e0 0x7fdf601430a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800c4b0 tx=0x7fdf5800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:39.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf57e00640 1 -- 172.21.15.83:0/2768782436 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58018070 con 0x7fdf6015f6e0 2024-04-25T00:13:39.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf57e00640 1 -- 172.21.15.83:0/2768782436 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf580074e0 con 0x7fdf6015f6e0 2024-04-25T00:13:39.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf65800640 1 -- 172.21.15.83:0/2768782436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf60155c30 con 0x7fdf6015f6e0 2024-04-25T00:13:39.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf57e00640 1 -- 172.21.15.83:0/2768782436 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58004a30 con 0x7fdf6015f6e0 2024-04-25T00:13:39.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf65800640 1 -- 172.21.15.83:0/2768782436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf601561f0 con 0x7fdf6015f6e0 2024-04-25T00:13:39.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf65800640 1 -- 172.21.15.83:0/2768782436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf6004f510 con 0x7fdf6015f6e0 2024-04-25T00:13:39.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.120+0000 7fdf57e00640 1 -- 172.21.15.83:0/2768782436 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdf5801b070 con 0x7fdf6015f6e0 2024-04-25T00:13:39.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.124+0000 7fdf57e00640 1 --2- 172.21.15.83:0/2768782436 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdf400678c0 0x7fdf40069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.124+0000 7fdf57e00640 1 -- 172.21.15.83:0/2768782436 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fdf58085110 con 0x7fdf6015f6e0 2024-04-25T00:13:39.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.124+0000 7fdf5fe00640 1 --2- 172.21.15.83:0/2768782436 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdf400678c0 0x7fdf40069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.124+0000 7fdf5fe00640 1 --2- 172.21.15.83:0/2768782436 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdf400678c0 0x7fdf40069d80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdf500af000 tx=0x7fdf500a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.128+0000 7fdf57e00640 1 -- 172.21.15.83:0/2768782436 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdf58055a20 con 0x7fdf6015f6e0 2024-04-25T00:13:39.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.284+0000 7fdf65800640 1 -- 172.21.15.83:0/2768782436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf601699a0 con 0x7fdf6015f6e0 2024-04-25T00:13:39.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.284+0000 7fdf57e00640 1 -- 172.21.15.83:0/2768782436 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16800 (secure 0 0 0) 0x7fdf58055320 con 0x7fdf6015f6e0 2024-04-25T00:13:39.289 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:39.289 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":60,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:21.438568+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:18.160239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:18.800529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.288+0000 7fdf65800640 1 -- 172.21.15.83:0/2768782436 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdf400678c0 msgr2=0x7fdf40069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.288+0000 7fdf65800640 1 --2- 172.21.15.83:0/2768782436 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdf400678c0 0x7fdf40069d80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdf500af000 tx=0x7fdf500a7070 comp rx=0 tx=0).stop 2024-04-25T00:13:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.288+0000 7fdf65800640 1 -- 172.21.15.83:0/2768782436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6015f6e0 msgr2=0x7fdf601430a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.288+0000 7fdf65800640 1 --2- 172.21.15.83:0/2768782436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6015f6e0 0x7fdf601430a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800c4b0 tx=0x7fdf5800c970 comp rx=0 tx=0).stop 2024-04-25T00:13:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.288+0000 7fdf65800640 1 -- 172.21.15.83:0/2768782436 shutdown_connections 2024-04-25T00:13:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.288+0000 7fdf65800640 1 --2- 172.21.15.83:0/2768782436 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdf400678c0 0x7fdf40069d80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.288+0000 7fdf65800640 1 --2- 172.21.15.83:0/2768782436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6015f6e0 0x7fdf601430a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.288+0000 7fdf65800640 1 --2- 172.21.15.83:0/2768782436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf600ce090 0x7fdf60154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.288+0000 7fdf65800640 1 --2- 172.21.15.83:0/2768782436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf6006b1e0 0x7fdf6006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.288+0000 7fdf65800640 1 -- 172.21.15.83:0/2768782436 >> 172.21.15.83:0/2768782436 conn(0x7fdf600cd500 msgr2=0x7fdf6015fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.288+0000 7fdf65800640 1 -- 172.21.15.83:0/2768782436 shutdown_connections 2024-04-25T00:13:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.288+0000 7fdf65800640 1 -- 172.21.15.83:0/2768782436 wait complete. 2024-04-25T00:13:39.311 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:13:39.311 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:39.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc17600640 1 Processor -- start 2024-04-25T00:13:39.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc17600640 1 -- start start 2024-04-25T00:13:39.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc17600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc100b4360 0x7fdc100b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc17600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc1006ef80 con 0x7fdc101553e0 2024-04-25T00:13:39.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc17600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc1015b510 con 0x7fdc100b2b30 2024-04-25T00:13:39.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc17600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1015b7f0 con 0x7fdc100b4360 2024-04-25T00:13:39.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc14c00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdc101553e0 0x7fdc1006e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40702/0 (socket says 172.21.15.83:40702) 2024-04-25T00:13:39.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc14c00640 1 -- 172.21.15.83:0/3149356663 learned_addr learned my addr 172.21.15.83:0/3149356663 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:39.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc0fe00640 1 --2- 172.21.15.83:0/3149356663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc100b4360 0x7fdc100b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc0f400640 1 -- 172.21.15.83:0/3149356663 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 574259963 0 0) 0x7fdc1006ef80 con 0x7fdc101553e0 2024-04-25T00:13:39.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc0f400640 1 -- 172.21.15.83:0/3149356663 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbf8003530 con 0x7fdc101553e0 2024-04-25T00:13:39.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc0f400640 1 -- 172.21.15.83:0/3149356663 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2612779032 0 0) 0x7fdc1015b510 con 0x7fdc100b2b30 2024-04-25T00:13:39.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc0f400640 1 -- 172.21.15.83:0/3149356663 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc1006ef80 con 0x7fdc100b2b30 2024-04-25T00:13:39.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc0fe00640 1 -- 172.21.15.83:0/3149356663 >> v1:172.21.15.86:6789/0 conn(0x7fdc100b2b30 legacy=0x7fdc100b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc0fe00640 1 -- 172.21.15.83:0/3149356663 >> v1:172.21.15.83:6789/0 conn(0x7fdc101553e0 legacy=0x7fdc1006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.444+0000 7fdc0fe00640 1 -- 172.21.15.83:0/3149356663 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1015bad0 con 0x7fdc100b4360 2024-04-25T00:13:39.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.448+0000 7fdc0fe00640 1 --2- 172.21.15.83:0/3149356663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc100b4360 0x7fdc100b47e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0097f0 tx=0x7fdbfc01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58242c91fd44f936 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:39.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.448+0000 7fdc0f400640 1 -- 172.21.15.83:0/3149356663 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc0b0070 con 0x7fdc100b4360 2024-04-25T00:13:39.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.448+0000 7fdc0f400640 1 -- 172.21.15.83:0/3149356663 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbfc0ab600 con 0x7fdc100b4360 2024-04-25T00:13:39.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.448+0000 7fdc0f400640 1 -- 172.21.15.83:0/3149356663 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc0ab9d0 con 0x7fdc100b4360 2024-04-25T00:13:39.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.448+0000 7fdc17600640 1 -- 172.21.15.83:0/3149356663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc100b4360 msgr2=0x7fdc100b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.448+0000 7fdc17600640 1 --2- 172.21.15.83:0/3149356663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc100b4360 0x7fdc100b47e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0097f0 tx=0x7fdbfc01c450 comp rx=0 tx=0).stop 2024-04-25T00:13:39.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.448+0000 7fdc17600640 1 -- 172.21.15.83:0/3149356663 shutdown_connections 2024-04-25T00:13:39.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.448+0000 7fdc17600640 1 --2- 172.21.15.83:0/3149356663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc100b4360 0x7fdc100b47e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.448+0000 7fdc17600640 1 -- 172.21.15.83:0/3149356663 >> 172.21.15.83:0/3149356663 conn(0x7fdc100b1830 msgr2=0x7fdc100b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:39.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.448+0000 7fdc17600640 1 -- 172.21.15.83:0/3149356663 shutdown_connections 2024-04-25T00:13:39.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.448+0000 7fdc17600640 1 -- 172.21.15.83:0/3149356663 wait complete. 2024-04-25T00:13:39.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.448+0000 7fdc17600640 1 Processor -- start 2024-04-25T00:13:39.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.448+0000 7fdc17600640 1 -- start start 2024-04-25T00:13:39.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc17600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc100b2b30 0x7fdc1015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc17600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc100b4360 0x7fdc100cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc17600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc101553e0 0x7fdc1014e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc17600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc10155940 con 0x7fdc100b2b30 2024-04-25T00:13:39.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc17600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc101557c0 con 0x7fdc101553e0 2024-04-25T00:13:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc17600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1006ebd0 con 0x7fdc100b4360 2024-04-25T00:13:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc14c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc100b2b30 0x7fdc1015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc14c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc100b2b30 0x7fdc1015c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35008/0 (socket says 172.21.15.83:35008) 2024-04-25T00:13:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc14c00640 1 -- 172.21.15.83:0/1640575353 learned_addr learned my addr 172.21.15.83:0/1640575353 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc0fe00640 1 --2- 172.21.15.83:0/1640575353 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc100b4360 0x7fdc100cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc15600640 1 --2- 172.21.15.83:0/1640575353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc101553e0 0x7fdc1014e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc14c00640 1 -- 172.21.15.83:0/1640575353 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc100b4360 msgr2=0x7fdc100cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc14c00640 1 --2- 172.21.15.83:0/1640575353 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc100b4360 0x7fdc100cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc14c00640 1 -- 172.21.15.83:0/1640575353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc101553e0 msgr2=0x7fdc1014e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc14c00640 1 --2- 172.21.15.83:0/1640575353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc101553e0 0x7fdc1014e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc14c00640 1 -- 172.21.15.83:0/1640575353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc10151a70 con 0x7fdc100b2b30 2024-04-25T00:13:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc14c00640 1 --2- 172.21.15.83:0/1640575353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc100b2b30 0x7fdc1015c7d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400c420 tx=0x7fdc0400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc0cc00640 1 -- 172.21.15.83:0/1640575353 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04017070 con 0x7fdc100b2b30 2024-04-25T00:13:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc0cc00640 1 -- 172.21.15.83:0/1640575353 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc04005cf0 con 0x7fdc100b2b30 2024-04-25T00:13:39.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc17600640 1 -- 172.21.15.83:0/1640575353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc10151d50 con 0x7fdc100b2b30 2024-04-25T00:13:39.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc0cc00640 1 -- 172.21.15.83:0/1640575353 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04007310 con 0x7fdc100b2b30 2024-04-25T00:13:39.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc17600640 1 -- 172.21.15.83:0/1640575353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc10152310 con 0x7fdc100b2b30 2024-04-25T00:13:39.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.452+0000 7fdc17600640 1 -- 172.21.15.83:0/1640575353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc101517e0 con 0x7fdc100b2b30 2024-04-25T00:13:39.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.456+0000 7fdc0cc00640 1 -- 172.21.15.83:0/1640575353 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdc0401a070 con 0x7fdc100b2b30 2024-04-25T00:13:39.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.456+0000 7fdc0cc00640 1 --2- 172.21.15.83:0/1640575353 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdbf00678f0 0x7fdbf0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.456+0000 7fdc0cc00640 1 -- 172.21.15.83:0/1640575353 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fdc040851e0 con 0x7fdc100b2b30 2024-04-25T00:13:39.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.456+0000 7fdc0fe00640 1 --2- 172.21.15.83:0/1640575353 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdbf00678f0 0x7fdbf0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.456+0000 7fdc0fe00640 1 --2- 172.21.15.83:0/1640575353 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdbf00678f0 0x7fdbf0069db0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0ad000 tx=0x7fdbfc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:39.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.460+0000 7fdc0cc00640 1 -- 172.21.15.83:0/1640575353 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdc040554a0 con 0x7fdc100b2b30 2024-04-25T00:13:39.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.648+0000 7fdc17600640 1 -- 172.21.15.83:0/1640575353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc100737f0 con 0x7fdc100b2b30 2024-04-25T00:13:39.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.648+0000 7fdc0cc00640 1 -- 172.21.15.83:0/1640575353 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5176 (secure 0 0 0) 0x7fdc040597a0 con 0x7fdc100b2b30 2024-04-25T00:13:39.651 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:39.651 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3667389928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":3667389928},{"type":"v1","addr":"172.21.15.86:6835","nonce":3667389928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5923,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2174516900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2174516900},{"type":"v1","addr":"172.21.15.83:6837","nonce":2174516900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2877383406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2877383406},{"type":"v1","addr":"172.21.15.83:6836","nonce":2877383406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:13:19.805315+0000","modified":"2024-04-25T00:13:22.834474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6834/52944842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":52944842},{"type":"v1","addr":"172.21.15.86:6834","nonce":52944842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-04-25T00:13:39.651 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 35 2024-04-25T00:13:39.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.648+0000 7fdc17600640 1 -- 172.21.15.83:0/1640575353 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdbf00678f0 msgr2=0x7fdbf0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.648+0000 7fdc17600640 1 --2- 172.21.15.83:0/1640575353 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdbf00678f0 0x7fdbf0069db0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0ad000 tx=0x7fdbfc0a6000 comp rx=0 tx=0).stop 2024-04-25T00:13:39.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.648+0000 7fdc17600640 1 -- 172.21.15.83:0/1640575353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc100b2b30 msgr2=0x7fdc1015c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.648+0000 7fdc17600640 1 --2- 172.21.15.83:0/1640575353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc100b2b30 0x7fdc1015c7d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400c420 tx=0x7fdc0400c8e0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.648+0000 7fdc17600640 1 -- 172.21.15.83:0/1640575353 shutdown_connections 2024-04-25T00:13:39.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.648+0000 7fdc17600640 1 --2- 172.21.15.83:0/1640575353 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdbf00678f0 0x7fdbf0069db0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.648+0000 7fdc17600640 1 --2- 172.21.15.83:0/1640575353 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc100b4360 0x7fdc100cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.648+0000 7fdc17600640 1 --2- 172.21.15.83:0/1640575353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc101553e0 0x7fdc1014e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.648+0000 7fdc17600640 1 --2- 172.21.15.83:0/1640575353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc100b2b30 0x7fdc1015c7d0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.648+0000 7fdc17600640 1 -- 172.21.15.83:0/1640575353 >> 172.21.15.83:0/1640575353 conn(0x7fdc100b1830 msgr2=0x7fdc10154500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:39.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.648+0000 7fdc17600640 1 -- 172.21.15.83:0/1640575353 shutdown_connections 2024-04-25T00:13:39.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.648+0000 7fdc17600640 1 -- 172.21.15.83:0/1640575353 wait complete. 2024-04-25T00:13:39.672 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:39.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b92e00640 1 Processor -- start 2024-04-25T00:13:39.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b92e00640 1 -- start start 2024-04-25T00:13:39.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b92e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0b8c15be10 0x7f0b8c1602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b92e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b8c06de00 0x7f0b8c06b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b92e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0b8c06e260 0x7f0b8c06b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b92e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0b8c0cd610 con 0x7f0b8c15be10 2024-04-25T00:13:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b92e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0b8c160800 con 0x7f0b8c06e260 2024-04-25T00:13:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b92e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8c0cd8b0 con 0x7f0b8c06de00 2024-04-25T00:13:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b8be00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0b8c15be10 0x7f0b8c1602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b8be00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0b8c15be10 0x7f0b8c1602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35010/0 (socket says 172.21.15.83:35010) 2024-04-25T00:13:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b8be00640 1 -- 172.21.15.83:0/643405208 learned_addr learned my addr 172.21.15.83:0/643405208 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b90e00640 1 --2- 172.21.15.83:0/643405208 >> v2:172.21.15.86:3300/0 conn(0x7f0b8c06e260 0x7f0b8c06b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b83400640 1 --2- 172.21.15.83:0/643405208 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b8c06de00 0x7f0b8c06b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b8be00640 1 -- 172.21.15.83:0/643405208 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b8c06de00 msgr2=0x7f0b8c06b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b8be00640 1 --2- 172.21.15.83:0/643405208 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b8c06de00 0x7f0b8c06b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b8be00640 1 -- 172.21.15.83:0/643405208 >> v2:172.21.15.86:3300/0 conn(0x7f0b8c06e260 msgr2=0x7f0b8c06b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b8be00640 1 --2- 172.21.15.83:0/643405208 >> v2:172.21.15.86:3300/0 conn(0x7f0b8c06e260 0x7f0b8c06b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.804+0000 7f0b8be00640 1 -- 172.21.15.83:0/643405208 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c0cdb90 con 0x7f0b8c15be10 2024-04-25T00:13:39.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.808+0000 7f0b8be00640 1 --2- 172.21.15.83:0/643405208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b8c15be10 0x7f0b8c1602c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0d1ce0 tx=0x7f0b7c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523d985ccac6fcd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:39.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.808+0000 7f0b8b400640 1 -- 172.21.15.83:0/643405208 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c0a5760 con 0x7f0b8c15be10 2024-04-25T00:13:39.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.808+0000 7f0b8b400640 1 -- 172.21.15.83:0/643405208 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b7c0a5900 con 0x7f0b8c15be10 2024-04-25T00:13:39.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.808+0000 7f0b8b400640 1 -- 172.21.15.83:0/643405208 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c0a5cd0 con 0x7f0b8c15be10 2024-04-25T00:13:39.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.808+0000 7f0b92e00640 1 -- 172.21.15.83:0/643405208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b8c15be10 msgr2=0x7f0b8c1602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.808+0000 7f0b92e00640 1 --2- 172.21.15.83:0/643405208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b8c15be10 0x7f0b8c1602c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0d1ce0 tx=0x7f0b7c01c9f0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.808+0000 7f0b92e00640 1 -- 172.21.15.83:0/643405208 shutdown_connections 2024-04-25T00:13:39.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.808+0000 7f0b92e00640 1 --2- 172.21.15.83:0/643405208 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b8c06de00 0x7f0b8c06b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.808+0000 7f0b92e00640 1 --2- 172.21.15.83:0/643405208 >> v2:172.21.15.86:3300/0 conn(0x7f0b8c06e260 0x7f0b8c06b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.808+0000 7f0b92e00640 1 --2- 172.21.15.83:0/643405208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b8c15be10 0x7f0b8c1602c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.808+0000 7f0b92e00640 1 -- 172.21.15.83:0/643405208 >> 172.21.15.83:0/643405208 conn(0x7f0b8c06c2f0 msgr2=0x7f0b8c06c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:39.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.808+0000 7f0b92e00640 1 -- 172.21.15.83:0/643405208 shutdown_connections 2024-04-25T00:13:39.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.808+0000 7f0b92e00640 1 -- 172.21.15.83:0/643405208 wait complete. 2024-04-25T00:13:39.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b92e00640 1 Processor -- start 2024-04-25T00:13:39.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b92e00640 1 -- start start 2024-04-25T00:13:39.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b92e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b8c06de00 0x7f0b8c1392e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b92e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b8c06e260 0x7f0b8c1505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b92e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b8c15be10 0x7f0b8c15e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b92e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c06fab0 con 0x7f0b8c06de00 2024-04-25T00:13:39.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b92e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c06f930 con 0x7f0b8c15be10 2024-04-25T00:13:39.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b92e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8c0d0b50 con 0x7f0b8c06e260 2024-04-25T00:13:39.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b8c06de00 0x7f0b8c1392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b8c06de00 0x7f0b8c1392e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35016/0 (socket says 172.21.15.83:35016) 2024-04-25T00:13:39.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b8be00640 1 -- 172.21.15.83:0/1486819909 learned_addr learned my addr 172.21.15.83:0/1486819909 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:39.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b83400640 1 --2- 172.21.15.83:0/1486819909 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b8c06e260 0x7f0b8c1505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b90e00640 1 --2- 172.21.15.83:0/1486819909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b8c15be10 0x7f0b8c15e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b8be00640 1 -- 172.21.15.83:0/1486819909 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b8c06e260 msgr2=0x7f0b8c1505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b8be00640 1 --2- 172.21.15.83:0/1486819909 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b8c06e260 0x7f0b8c1505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b8be00640 1 -- 172.21.15.83:0/1486819909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b8c15be10 msgr2=0x7f0b8c15e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b8be00640 1 --2- 172.21.15.83:0/1486819909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b8c15be10 0x7f0b8c15e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b8be00640 1 -- 172.21.15.83:0/1486819909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c150f00 con 0x7f0b8c06de00 2024-04-25T00:13:39.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.812+0000 7f0b8be00640 1 --2- 172.21.15.83:0/1486819909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b8c06de00 0x7f0b8c1392e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0af040 tx=0x7f0b7c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.816+0000 7f0b88c00640 1 -- 172.21.15.83:0/1486819909 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c0b8670 con 0x7f0b8c06de00 2024-04-25T00:13:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.816+0000 7f0b88c00640 1 -- 172.21.15.83:0/1486819909 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b7c0be070 con 0x7f0b8c06de00 2024-04-25T00:13:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.816+0000 7f0b92e00640 1 -- 172.21.15.83:0/1486819909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8c15dec0 con 0x7f0b8c06de00 2024-04-25T00:13:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.816+0000 7f0b88c00640 1 -- 172.21.15.83:0/1486819909 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c0b8a90 con 0x7f0b8c06de00 2024-04-25T00:13:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.816+0000 7f0b92e00640 1 -- 172.21.15.83:0/1486819909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8c15e480 con 0x7f0b8c06de00 2024-04-25T00:13:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.816+0000 7f0b92e00640 1 -- 172.21.15.83:0/1486819909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b8c0736c0 con 0x7f0b8c06de00 2024-04-25T00:13:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.816+0000 7f0b88c00640 1 -- 172.21.15.83:0/1486819909 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0b7c0b8c30 con 0x7f0b8c06de00 2024-04-25T00:13:39.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.820+0000 7f0b88c00640 1 --2- 172.21.15.83:0/1486819909 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0b680678c0 0x7f0b68069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.820+0000 7f0b88c00640 1 -- 172.21.15.83:0/1486819909 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f0b7c120bf0 con 0x7f0b8c06de00 2024-04-25T00:13:39.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.820+0000 7f0b83400640 1 --2- 172.21.15.83:0/1486819909 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0b680678c0 0x7f0b68069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.820+0000 7f0b83400640 1 --2- 172.21.15.83:0/1486819909 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0b680678c0 0x7f0b68069d80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c15d960 tx=0x7f0b74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:39.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:39.824+0000 7f0b88c00640 1 -- 172.21.15.83:0/1486819909 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0b7c0165e0 con 0x7f0b8c06de00 2024-04-25T00:13:40.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.008+0000 7f0b92e00640 1 -- 172.21.15.83:0/1486819909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b8c04f5f0 con 0x7f0b8c06de00 2024-04-25T00:13:40.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.012+0000 7f0b88c00640 1 -- 172.21.15.83:0/1486819909 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5176 (secure 0 0 0) 0x7f0b7c0f0fd0 con 0x7f0b8c06de00 2024-04-25T00:13:40.014 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:40.014 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3667389928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":3667389928},{"type":"v1","addr":"172.21.15.86:6835","nonce":3667389928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5923,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2174516900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2174516900},{"type":"v1","addr":"172.21.15.83:6837","nonce":2174516900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2877383406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2877383406},{"type":"v1","addr":"172.21.15.83:6836","nonce":2877383406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:13:19.805315+0000","modified":"2024-04-25T00:13:22.834474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6834/52944842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":52944842},{"type":"v1","addr":"172.21.15.86:6834","nonce":52944842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-04-25T00:13:40.014 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 35 2024-04-25T00:13:40.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.012+0000 7f0b92e00640 1 -- 172.21.15.83:0/1486819909 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0b680678c0 msgr2=0x7f0b68069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.012+0000 7f0b92e00640 1 --2- 172.21.15.83:0/1486819909 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0b680678c0 0x7f0b68069d80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c15d960 tx=0x7f0b74008040 comp rx=0 tx=0).stop 2024-04-25T00:13:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.012+0000 7f0b92e00640 1 -- 172.21.15.83:0/1486819909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b8c06de00 msgr2=0x7f0b8c1392e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.012+0000 7f0b92e00640 1 --2- 172.21.15.83:0/1486819909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b8c06de00 0x7f0b8c1392e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0af040 tx=0x7f0b7c019790 comp rx=0 tx=0).stop 2024-04-25T00:13:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.012+0000 7f0b92e00640 1 -- 172.21.15.83:0/1486819909 shutdown_connections 2024-04-25T00:13:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.012+0000 7f0b92e00640 1 --2- 172.21.15.83:0/1486819909 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0b680678c0 0x7f0b68069d80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.012+0000 7f0b92e00640 1 --2- 172.21.15.83:0/1486819909 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b8c06e260 0x7f0b8c1505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.012+0000 7f0b92e00640 1 --2- 172.21.15.83:0/1486819909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b8c15be10 0x7f0b8c15e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.012+0000 7f0b92e00640 1 --2- 172.21.15.83:0/1486819909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b8c06de00 0x7f0b8c1392e0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.012+0000 7f0b92e00640 1 -- 172.21.15.83:0/1486819909 >> 172.21.15.83:0/1486819909 conn(0x7f0b8c06c2f0 msgr2=0x7f0b8c15f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.012+0000 7f0b92e00640 1 -- 172.21.15.83:0/1486819909 shutdown_connections 2024-04-25T00:13:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.012+0000 7f0b92e00640 1 -- 172.21.15.83:0/1486819909 wait complete. 2024-04-25T00:13:40.034 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.164+0000 7f9a03000640 1 Processor -- start 2024-04-25T00:13:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.164+0000 7f9a03000640 1 -- start start 2024-04-25T00:13:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f9a03000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc1553e0 0x7f99fc06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f9a03000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f99fc06ea00 0x7f99fc0b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f9a03000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f99fc0b48a0 0x7f99fc15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f9a03000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f99fc06ef80 con 0x7f99fc06ea00 2024-04-25T00:13:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f9a03000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f99fc15b510 con 0x7f99fc0b48a0 2024-04-25T00:13:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f9a03000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f99fc15b7f0 con 0x7f99fc1553e0 2024-04-25T00:13:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f99fb400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f99fc06ea00 0x7f99fc0b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f99fb400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f99fc06ea00 0x7f99fc0b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35024/0 (socket says 172.21.15.83:35024) 2024-04-25T00:13:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f99fb400640 1 -- 172.21.15.83:0/2987035063 learned_addr learned my addr 172.21.15.83:0/2987035063 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f99fbe00640 1 --2- 172.21.15.83:0/2987035063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc1553e0 0x7f99fc06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f99fb400640 1 -- 172.21.15.83:0/2987035063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc1553e0 msgr2=0x7f99fc06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f9a01000640 1 --2- 172.21.15.83:0/2987035063 >> v2:172.21.15.86:3300/0 conn(0x7f99fc0b48a0 0x7f99fc15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f99fb400640 1 --2- 172.21.15.83:0/2987035063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc1553e0 0x7f99fc06e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f99fb400640 1 -- 172.21.15.83:0/2987035063 >> v2:172.21.15.86:3300/0 conn(0x7f99fc0b48a0 msgr2=0x7f99fc15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f99fb400640 1 --2- 172.21.15.83:0/2987035063 >> v2:172.21.15.86:3300/0 conn(0x7f99fc0b48a0 0x7f99fc15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f99fb400640 1 -- 172.21.15.83:0/2987035063 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99fc15bad0 con 0x7f99fc06ea00 2024-04-25T00:13:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f99fbe00640 1 --2- 172.21.15.83:0/2987035063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc1553e0 0x7f99fc06e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.168+0000 7f9a01000640 1 --2- 172.21.15.83:0/2987035063 >> v2:172.21.15.86:3300/0 conn(0x7f99fc0b48a0 0x7f99fc15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:40.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f99fb400640 1 --2- 172.21.15.83:0/2987035063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99fc06ea00 0x7f99fc0b4360 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0ccdf0 tx=0x7f99e801c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f465333c716545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:40.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f99faa00640 1 -- 172.21.15.83:0/2987035063 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e80b0070 con 0x7f99fc06ea00 2024-04-25T00:13:40.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f99faa00640 1 -- 172.21.15.83:0/2987035063 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e80ab600 con 0x7f99fc06ea00 2024-04-25T00:13:40.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f99faa00640 1 -- 172.21.15.83:0/2987035063 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e80ab9d0 con 0x7f99fc06ea00 2024-04-25T00:13:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f9a03000640 1 -- 172.21.15.83:0/2987035063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99fc06ea00 msgr2=0x7f99fc0b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f9a03000640 1 --2- 172.21.15.83:0/2987035063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99fc06ea00 0x7f99fc0b4360 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0ccdf0 tx=0x7f99e801c1d0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f9a03000640 1 -- 172.21.15.83:0/2987035063 shutdown_connections 2024-04-25T00:13:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f9a03000640 1 --2- 172.21.15.83:0/2987035063 >> v2:172.21.15.86:3300/0 conn(0x7f99fc0b48a0 0x7f99fc15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f9a03000640 1 --2- 172.21.15.83:0/2987035063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99fc06ea00 0x7f99fc0b4360 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f9a03000640 1 --2- 172.21.15.83:0/2987035063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc1553e0 0x7f99fc06e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f9a03000640 1 -- 172.21.15.83:0/2987035063 >> 172.21.15.83:0/2987035063 conn(0x7f99fc0b1830 msgr2=0x7f99fc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f9a03000640 1 -- 172.21.15.83:0/2987035063 shutdown_connections 2024-04-25T00:13:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f9a03000640 1 -- 172.21.15.83:0/2987035063 wait complete. 2024-04-25T00:13:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f9a03000640 1 Processor -- start 2024-04-25T00:13:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.172+0000 7f9a03000640 1 -- start start 2024-04-25T00:13:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a03000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc06ea00 0x7f99fc0cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a03000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99fc0b48a0 0x7f99fc13c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a03000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99fc1553e0 0x7f99fc0b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a03000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc06d530 con 0x7f99fc1553e0 2024-04-25T00:13:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a03000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc06d3b0 con 0x7f99fc0b48a0 2024-04-25T00:13:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a03000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f99fc06d6b0 con 0x7f99fc06ea00 2024-04-25T00:13:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a01000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99fc1553e0 0x7f99fc0b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a01000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99fc1553e0 0x7f99fc0b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35026/0 (socket says 172.21.15.83:35026) 2024-04-25T00:13:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a01000640 1 -- 172.21.15.83:0/3548382669 learned_addr learned my addr 172.21.15.83:0/3548382669 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f99fb400640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99fc0b48a0 0x7f99fc13c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f99fbe00640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc06ea00 0x7f99fc0cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a01000640 1 -- 172.21.15.83:0/3548382669 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc06ea00 msgr2=0x7f99fc0cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a01000640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc06ea00 0x7f99fc0cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a01000640 1 -- 172.21.15.83:0/3548382669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99fc0b48a0 msgr2=0x7f99fc13c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a01000640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99fc0b48a0 0x7f99fc13c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a01000640 1 -- 172.21.15.83:0/3548382669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99fc06f6b0 con 0x7f99fc1553e0 2024-04-25T00:13:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f99fb400640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99fc0b48a0 0x7f99fc13c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a01000640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99fc1553e0 0x7f99fc0b3970 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c820 tx=0x7f99f400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f99f3e00640 1 -- 172.21.15.83:0/3548382669 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4015070 con 0x7f99fc1553e0 2024-04-25T00:13:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f99f3e00640 1 -- 172.21.15.83:0/3548382669 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f4007a30 con 0x7f99fc1553e0 2024-04-25T00:13:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a03000640 1 -- 172.21.15.83:0/3548382669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99fc156460 con 0x7f99fc1553e0 2024-04-25T00:13:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f99f3e00640 1 -- 172.21.15.83:0/3548382669 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4007e20 con 0x7f99fc1553e0 2024-04-25T00:13:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a03000640 1 -- 172.21.15.83:0/3548382669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99fc156a20 con 0x7f99fc1553e0 2024-04-25T00:13:40.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f9a03000640 1 -- 172.21.15.83:0/3548382669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99fc0737f0 con 0x7f99fc1553e0 2024-04-25T00:13:40.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.176+0000 7f99f3e00640 1 -- 172.21.15.83:0/3548382669 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f99f4018070 con 0x7f99fc1553e0 2024-04-25T00:13:40.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.180+0000 7f99f3e00640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99dc0678c0 0x7f99dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.180+0000 7f99f3e00640 1 -- 172.21.15.83:0/3548382669 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f99f4085c70 con 0x7f99fc1553e0 2024-04-25T00:13:40.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.180+0000 7f99fbe00640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99dc0678c0 0x7f99dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.180+0000 7f99fbe00640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99dc0678c0 0x7f99dc069d80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f99ec007510 tx=0x7f99ec0074a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:40.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.184+0000 7f99f3e00640 1 -- 172.21.15.83:0/3548382669 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f99f4056770 con 0x7f99fc1553e0 2024-04-25T00:13:40.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.340+0000 7f9a03000640 1 -- 172.21.15.83:0/3548382669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99fc04f5f0 con 0x7f99fc1553e0 2024-04-25T00:13:40.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.340+0000 7f99f3e00640 1 -- 172.21.15.83:0/3548382669 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16800 (secure 0 0 0) 0x7f99f4056070 con 0x7f99fc1553e0 2024-04-25T00:13:40.344 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:40.344 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":60,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:21.438568+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:18.160239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:18.800529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:40.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.344+0000 7f9a03000640 1 -- 172.21.15.83:0/3548382669 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99dc0678c0 msgr2=0x7f99dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.344+0000 7f9a03000640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99dc0678c0 0x7f99dc069d80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f99ec007510 tx=0x7f99ec0074a0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.344+0000 7f9a03000640 1 -- 172.21.15.83:0/3548382669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99fc1553e0 msgr2=0x7f99fc0b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.344+0000 7f9a03000640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99fc1553e0 0x7f99fc0b3970 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c820 tx=0x7f99f400cce0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.344+0000 7f9a03000640 1 -- 172.21.15.83:0/3548382669 shutdown_connections 2024-04-25T00:13:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.344+0000 7f9a03000640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99dc0678c0 0x7f99dc069d80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.344+0000 7f9a03000640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99fc1553e0 0x7f99fc0b3970 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.344+0000 7f9a03000640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99fc0b48a0 0x7f99fc13c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.344+0000 7f9a03000640 1 --2- 172.21.15.83:0/3548382669 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc06ea00 0x7f99fc0cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.344+0000 7f9a03000640 1 -- 172.21.15.83:0/3548382669 >> 172.21.15.83:0/3548382669 conn(0x7f99fc0b1830 msgr2=0x7f99fc15dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.344+0000 7f9a03000640 1 -- 172.21.15.83:0/3548382669 shutdown_connections 2024-04-25T00:13:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.344+0000 7f9a03000640 1 -- 172.21.15.83:0/3548382669 wait complete. 2024-04-25T00:13:40.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:13:40.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.496+0000 7f7196200640 1 Processor -- start 2024-04-25T00:13:40.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.496+0000 7f7196200640 1 -- start start 2024-04-25T00:13:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.500+0000 7f7196200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71900cb640 0x7f719006dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.500+0000 7f7196200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7190071180 con 0x7f719006cfd0 2024-04-25T00:13:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.500+0000 7f7196200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f719006e160 con 0x7f7190070d10 2024-04-25T00:13:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.500+0000 7f7196200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f719006ea10 con 0x7f71900cb640 2024-04-25T00:13:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.500+0000 7f718f400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f719006cfd0 0x7f719006d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40704/0 (socket says 172.21.15.83:40704) 2024-04-25T00:13:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.500+0000 7f718f400640 1 -- 172.21.15.83:0/185738284 learned_addr learned my addr 172.21.15.83:0/185738284 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.500+0000 7f7186a00640 1 --2- 172.21.15.83:0/185738284 >> v2:172.21.15.83:3300/0 conn(0x7f71900cb640 0x7f719006dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.500+0000 7f718ea00640 1 -- 172.21.15.83:0/185738284 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 875341236 0 0) 0x7f7190071180 con 0x7f719006cfd0 2024-04-25T00:13:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.500+0000 7f718ea00640 1 -- 172.21.15.83:0/185738284 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7174003530 con 0x7f719006cfd0 2024-04-25T00:13:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.500+0000 7f7186a00640 1 -- 172.21.15.83:0/185738284 >> v1:172.21.15.86:6789/0 conn(0x7f7190070d10 legacy=0x7f7190165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.500+0000 7f7186a00640 1 -- 172.21.15.83:0/185738284 >> v1:172.21.15.83:6789/0 conn(0x7f719006cfd0 legacy=0x7f719006d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.500+0000 7f7186a00640 1 -- 172.21.15.83:0/185738284 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719006ecf0 con 0x7f71900cb640 2024-04-25T00:13:40.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.504+0000 7f7186a00640 1 --2- 172.21.15.83:0/185738284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900cb640 0x7f719006dac0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f71780097c0 tx=0x7f717801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ae9200192c8be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:40.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.504+0000 7f718ea00640 1 -- 172.21.15.83:0/185738284 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71780b1070 con 0x7f71900cb640 2024-04-25T00:13:40.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.504+0000 7f718ea00640 1 -- 172.21.15.83:0/185738284 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71780ac410 con 0x7f71900cb640 2024-04-25T00:13:40.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.504+0000 7f718ea00640 1 -- 172.21.15.83:0/185738284 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71780ac7e0 con 0x7f71900cb640 2024-04-25T00:13:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.504+0000 7f7196200640 1 -- 172.21.15.83:0/185738284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900cb640 msgr2=0x7f719006dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.504+0000 7f7196200640 1 --2- 172.21.15.83:0/185738284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900cb640 0x7f719006dac0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f71780097c0 tx=0x7f717801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.504+0000 7f7196200640 1 -- 172.21.15.83:0/185738284 shutdown_connections 2024-04-25T00:13:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.504+0000 7f7196200640 1 --2- 172.21.15.83:0/185738284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900cb640 0x7f719006dac0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.504+0000 7f7196200640 1 -- 172.21.15.83:0/185738284 >> 172.21.15.83:0/185738284 conn(0x7f71900cfa60 msgr2=0x7f71900cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.504+0000 7f7196200640 1 -- 172.21.15.83:0/185738284 shutdown_connections 2024-04-25T00:13:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.504+0000 7f7196200640 1 -- 172.21.15.83:0/185738284 wait complete. 2024-04-25T00:13:40.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.504+0000 7f7196200640 1 Processor -- start 2024-04-25T00:13:40.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.504+0000 7f7196200640 1 -- start start 2024-04-25T00:13:40.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f7196200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719006cfd0 0x7f7190159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f7196200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7190070d10 0x7f7190136ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f7196200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71900cb640 0x7f71900c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f7196200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7190169c00 con 0x7f719006cfd0 2024-04-25T00:13:40.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f7196200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7190169a80 con 0x7f71900cb640 2024-04-25T00:13:40.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f7196200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7190169d80 con 0x7f7190070d10 2024-04-25T00:13:40.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f718f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719006cfd0 0x7f7190159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f718f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719006cfd0 0x7f7190159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35048/0 (socket says 172.21.15.83:35048) 2024-04-25T00:13:40.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f718f400640 1 -- 172.21.15.83:0/2368467023 learned_addr learned my addr 172.21.15.83:0/2368467023 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:40.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f718fe00640 1 --2- 172.21.15.83:0/2368467023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71900cb640 0x7f71900c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f7186a00640 1 --2- 172.21.15.83:0/2368467023 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7190070d10 0x7f7190136ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f718f400640 1 -- 172.21.15.83:0/2368467023 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7190070d10 msgr2=0x7f7190136ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f718f400640 1 --2- 172.21.15.83:0/2368467023 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7190070d10 0x7f7190136ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f718f400640 1 -- 172.21.15.83:0/2368467023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71900cb640 msgr2=0x7f71900c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f718f400640 1 --2- 172.21.15.83:0/2368467023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71900cb640 0x7f71900c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f718f400640 1 -- 172.21.15.83:0/2368467023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71901378f0 con 0x7f719006cfd0 2024-04-25T00:13:40.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f718f400640 1 --2- 172.21.15.83:0/2368467023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719006cfd0 0x7f7190159780 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f718000c420 tx=0x7f718000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:40.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f7187e00640 1 -- 172.21.15.83:0/2368467023 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180017070 con 0x7f719006cfd0 2024-04-25T00:13:40.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f7187e00640 1 -- 172.21.15.83:0/2368467023 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7180005cf0 con 0x7f719006cfd0 2024-04-25T00:13:40.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f7196200640 1 -- 172.21.15.83:0/2368467023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7190137bd0 con 0x7f719006cfd0 2024-04-25T00:13:40.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f7187e00640 1 -- 172.21.15.83:0/2368467023 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71800072d0 con 0x7f719006cfd0 2024-04-25T00:13:40.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f7196200640 1 -- 172.21.15.83:0/2368467023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7190138190 con 0x7f719006cfd0 2024-04-25T00:13:40.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f7196200640 1 -- 172.21.15.83:0/2368467023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7190068230 con 0x7f719006cfd0 2024-04-25T00:13:40.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.508+0000 7f7187e00640 1 -- 172.21.15.83:0/2368467023 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f718001a070 con 0x7f719006cfd0 2024-04-25T00:13:40.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.512+0000 7f7187e00640 1 --2- 172.21.15.83:0/2368467023 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f716c0678f0 0x7f716c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.512+0000 7f7187e00640 1 -- 172.21.15.83:0/2368467023 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f7180085240 con 0x7f719006cfd0 2024-04-25T00:13:40.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.512+0000 7f7186a00640 1 --2- 172.21.15.83:0/2368467023 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f716c0678f0 0x7f716c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.512+0000 7f7186a00640 1 --2- 172.21.15.83:0/2368467023 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f716c0678f0 0x7f716c069db0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f71780ae000 tx=0x7f7178008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:40.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.512+0000 7f7187e00640 1 -- 172.21.15.83:0/2368467023 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7180055470 con 0x7f719006cfd0 2024-04-25T00:13:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.700+0000 7f7196200640 1 -- 172.21.15.83:0/2368467023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7190139af0 con 0x7f719006cfd0 2024-04-25T00:13:40.705 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-25T00:13:40.700+0000 7efeeee00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:13:40.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.704+0000 7f7187e00640 1 -- 172.21.15.83:0/2368467023 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7f7180059770 con 0x7f719006cfd0 2024-04-25T00:13:40.707 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:13:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.708+0000 7f7196200640 1 -- 172.21.15.83:0/2368467023 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f716c0678f0 msgr2=0x7f716c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.708+0000 7f7196200640 1 --2- 172.21.15.83:0/2368467023 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f716c0678f0 0x7f716c069db0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f71780ae000 tx=0x7f7178008040 comp rx=0 tx=0).stop 2024-04-25T00:13:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.708+0000 7f7196200640 1 -- 172.21.15.83:0/2368467023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719006cfd0 msgr2=0x7f7190159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.708+0000 7f7196200640 1 --2- 172.21.15.83:0/2368467023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719006cfd0 0x7f7190159780 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f718000c420 tx=0x7f718000c8e0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.708+0000 7f7196200640 1 -- 172.21.15.83:0/2368467023 shutdown_connections 2024-04-25T00:13:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.708+0000 7f7196200640 1 --2- 172.21.15.83:0/2368467023 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f716c0678f0 0x7f716c069db0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.708+0000 7f7196200640 1 --2- 172.21.15.83:0/2368467023 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7190070d10 0x7f7190136ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.708+0000 7f7196200640 1 --2- 172.21.15.83:0/2368467023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71900cb640 0x7f71900c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.708+0000 7f7196200640 1 --2- 172.21.15.83:0/2368467023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719006cfd0 0x7f7190159780 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.708+0000 7f7196200640 1 -- 172.21.15.83:0/2368467023 >> 172.21.15.83:0/2368467023 conn(0x7f71900cfa60 msgr2=0x7f719013d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.708+0000 7f7196200640 1 -- 172.21.15.83:0/2368467023 shutdown_connections 2024-04-25T00:13:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.708+0000 7f7196200640 1 -- 172.21.15.83:0/2368467023 wait complete. 2024-04-25T00:13:40.716 INFO:tasks.ceph.mds.d.smithi086.stderr: -2387> 2024-04-25T00:13:22.392+0000 7f47c0800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:13:40.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:13:40.763 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2024-04-25T00:13:40.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.864+0000 7f439ba00640 1 Processor -- start 2024-04-25T00:13:40.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.864+0000 7f439ba00640 1 -- start start 2024-04-25T00:13:40.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f439ba00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4394159810 0x7f43940cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f439ba00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43940ce000 con 0x7f439406e950 2024-04-25T00:13:40.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f439ba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f439406eec0 con 0x7f4394160020 2024-04-25T00:13:40.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f439ba00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f439406e210 con 0x7f4394159810 2024-04-25T00:13:40.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f4393e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f439406e950 0x7f439415f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40712/0 (socket says 172.21.15.83:40712) 2024-04-25T00:13:40.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f4393e00640 1 -- 172.21.15.83:0/2671307130 learned_addr learned my addr 172.21.15.83:0/2671307130 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:40.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f4399000640 1 --2- 172.21.15.83:0/2671307130 >> v2:172.21.15.86:3300/0 conn(0x7f4394159810 0x7f43940cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f4393400640 1 -- 172.21.15.83:0/2671307130 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1846799479 0 0) 0x7f43940ce000 con 0x7f439406e950 2024-04-25T00:13:40.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f4393400640 1 -- 172.21.15.83:0/2671307130 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f437c003530 con 0x7f439406e950 2024-04-25T00:13:40.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f4393400640 1 -- 172.21.15.83:0/2671307130 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2271080778 0 0) 0x7f439406eec0 con 0x7f4394160020 2024-04-25T00:13:40.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f4393400640 1 -- 172.21.15.83:0/2671307130 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43940ce000 con 0x7f4394160020 2024-04-25T00:13:40.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f4399000640 1 -- 172.21.15.83:0/2671307130 >> v1:172.21.15.86:6789/0 conn(0x7f4394160020 legacy=0x7f43940ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f4399000640 1 -- 172.21.15.83:0/2671307130 >> v1:172.21.15.83:6789/0 conn(0x7f439406e950 legacy=0x7f439415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f4393400640 1 -- 172.21.15.83:0/2671307130 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 35592735 0 0) 0x7f437c003530 con 0x7f439406e950 2024-04-25T00:13:40.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.868+0000 7f4399000640 1 -- 172.21.15.83:0/2671307130 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439406dc50 con 0x7f4394159810 2024-04-25T00:13:40.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.872+0000 7f4399000640 1 --2- 172.21.15.83:0/2671307130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4394159810 0x7f43940cc4a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f43880097c0 tx=0x7f438801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a018fa848f6ce936 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:40.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.872+0000 7f4393400640 1 -- 172.21.15.83:0/2671307130 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43880b0070 con 0x7f4394159810 2024-04-25T00:13:40.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.872+0000 7f4393400640 1 -- 172.21.15.83:0/2671307130 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43880a4e50 con 0x7f4394159810 2024-04-25T00:13:40.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.872+0000 7f4393400640 1 -- 172.21.15.83:0/2671307130 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380001680 con 0x7f4394159810 2024-04-25T00:13:40.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.872+0000 7f439ba00640 1 -- 172.21.15.83:0/2671307130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4394159810 msgr2=0x7f43940cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.872+0000 7f439ba00640 1 --2- 172.21.15.83:0/2671307130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4394159810 0x7f43940cc4a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f43880097c0 tx=0x7f438801c330 comp rx=0 tx=0).stop 2024-04-25T00:13:40.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.872+0000 7f439ba00640 1 -- 172.21.15.83:0/2671307130 shutdown_connections 2024-04-25T00:13:40.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.872+0000 7f439ba00640 1 --2- 172.21.15.83:0/2671307130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4394159810 0x7f43940cc4a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.872+0000 7f439ba00640 1 -- 172.21.15.83:0/2671307130 >> 172.21.15.83:0/2671307130 conn(0x7f43940b1830 msgr2=0x7f43940b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:40.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.872+0000 7f439ba00640 1 -- 172.21.15.83:0/2671307130 shutdown_connections 2024-04-25T00:13:40.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.872+0000 7f439ba00640 1 -- 172.21.15.83:0/2671307130 wait complete. 2024-04-25T00:13:40.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.872+0000 7f439ba00640 1 Processor -- start 2024-04-25T00:13:40.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.872+0000 7f439ba00640 1 -- start start 2024-04-25T00:13:40.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f439ba00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439406e950 0x7f43940b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f439ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4394159810 0x7f439414e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f439ba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4394160020 0x7f4394150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f439ba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4394070f20 con 0x7f4394160020 2024-04-25T00:13:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f439ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4394070da0 con 0x7f4394159810 2024-04-25T00:13:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f439ba00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4394169cc0 con 0x7f439406e950 2024-04-25T00:13:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4399a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4394160020 0x7f4394150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4399a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4394160020 0x7f4394150520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35052/0 (socket says 172.21.15.83:35052) 2024-04-25T00:13:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4399a00640 1 -- 172.21.15.83:0/240319701 learned_addr learned my addr 172.21.15.83:0/240319701 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4393e00640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4394159810 0x7f439414e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4399000640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439406e950 0x7f43940b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4399a00640 1 -- 172.21.15.83:0/240319701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439406e950 msgr2=0x7f43940b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4399a00640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439406e950 0x7f43940b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4399a00640 1 -- 172.21.15.83:0/240319701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4394159810 msgr2=0x7f439414e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4399a00640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4394159810 0x7f439414e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4399a00640 1 -- 172.21.15.83:0/240319701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43940b3730 con 0x7f4394160020 2024-04-25T00:13:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4393e00640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4394159810 0x7f439414e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:40.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4399a00640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4394160020 0x7f4394150520 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f438c00c6c0 tx=0x7f438c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:40.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4390c00640 1 -- 172.21.15.83:0/240319701 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c003b80 con 0x7f4394160020 2024-04-25T00:13:40.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4390c00640 1 -- 172.21.15.83:0/240319701 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f438c003d20 con 0x7f4394160020 2024-04-25T00:13:40.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4390c00640 1 -- 172.21.15.83:0/240319701 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c0052f0 con 0x7f4394160020 2024-04-25T00:13:40.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f439ba00640 1 -- 172.21.15.83:0/240319701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43940b3a10 con 0x7f4394160020 2024-04-25T00:13:40.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f439ba00640 1 -- 172.21.15.83:0/240319701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43940b3fd0 con 0x7f4394160020 2024-04-25T00:13:40.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f439ba00640 1 -- 172.21.15.83:0/240319701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f439404f510 con 0x7f4394160020 2024-04-25T00:13:40.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.876+0000 7f4390c00640 1 -- 172.21.15.83:0/240319701 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f438c005490 con 0x7f4394160020 2024-04-25T00:13:40.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.880+0000 7f4390c00640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f43740678c0 0x7f4374069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:40.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.880+0000 7f4390c00640 1 -- 172.21.15.83:0/240319701 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6187+0+0 (secure 0 0 0) 0x7f438c085e00 con 0x7f4394160020 2024-04-25T00:13:40.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.880+0000 7f4399000640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f43740678c0 0x7f4374069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:40.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.880+0000 7f4399000640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f43740678c0 0x7f4374069d80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f43880ad000 tx=0x7f43880a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:40.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:40.884+0000 7f4390c00640 1 -- 172.21.15.83:0/240319701 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f438c0566c0 con 0x7f4394160020 2024-04-25T00:13:41.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.076+0000 7f439ba00640 1 -- 172.21.15.83:0/240319701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4394072c90 con 0x7f4394160020 2024-04-25T00:13:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.704+0000 7f4390c00640 1 -- 172.21.15.83:0/240319701 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7f438c055fc0 con 0x7f4394160020 2024-04-25T00:13:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.708+0000 7f439ba00640 1 -- 172.21.15.83:0/240319701 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f43740678c0 msgr2=0x7f4374069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.708+0000 7f439ba00640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f43740678c0 0x7f4374069d80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f43880ad000 tx=0x7f43880a6000 comp rx=0 tx=0).stop 2024-04-25T00:13:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.708+0000 7f439ba00640 1 -- 172.21.15.83:0/240319701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4394160020 msgr2=0x7f4394150520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.708+0000 7f439ba00640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4394160020 0x7f4394150520 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f438c00c6c0 tx=0x7f438c00cb80 comp rx=0 tx=0).stop 2024-04-25T00:13:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.708+0000 7f439ba00640 1 -- 172.21.15.83:0/240319701 shutdown_connections 2024-04-25T00:13:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.708+0000 7f439ba00640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f43740678c0 0x7f4374069d80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.708+0000 7f439ba00640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4394160020 0x7f4394150520 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.708+0000 7f439ba00640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4394159810 0x7f439414e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.708+0000 7f439ba00640 1 --2- 172.21.15.83:0/240319701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439406e950 0x7f43940b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.708+0000 7f439ba00640 1 -- 172.21.15.83:0/240319701 >> 172.21.15.83:0/240319701 conn(0x7f43940b1830 msgr2=0x7f43940aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:41.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.708+0000 7f439ba00640 1 -- 172.21.15.83:0/240319701 shutdown_connections 2024-04-25T00:13:41.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.708+0000 7f439ba00640 1 -- 172.21.15.83:0/240319701 wait complete. 2024-04-25T00:13:41.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:13:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.876+0000 7f2e5f400640 1 Processor -- start 2024-04-25T00:13:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.876+0000 7f2e5f400640 1 -- start start 2024-04-25T00:13:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e5f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e580ce080 0x7f2e580cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e5f400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e580d0280 0x7f2e580b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e5f400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e580ce610 con 0x7f2e581455d0 2024-04-25T00:13:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e5f400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2e580ce8f0 con 0x7f2e580ce080 2024-04-25T00:13:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e5f400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2e580cebd0 con 0x7f2e580d0280 2024-04-25T00:13:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e5d400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e580d0280 0x7f2e580b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e4fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e580ce080 0x7f2e580cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e5ca00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2e581455d0 0x7f2e5806d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:35654/0 (socket says 172.21.15.83:35654) 2024-04-25T00:13:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e5ca00640 1 -- 172.21.15.83:0/3239702478 learned_addr learned my addr 172.21.15.83:0/3239702478 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e5d400640 1 -- 172.21.15.83:0/3239702478 >> v2:172.21.15.86:3300/0 conn(0x7f2e580ce080 msgr2=0x7f2e580cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e5d400640 1 --2- 172.21.15.83:0/3239702478 >> v2:172.21.15.86:3300/0 conn(0x7f2e580ce080 0x7f2e580cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e5d400640 1 -- 172.21.15.83:0/3239702478 >> v1:172.21.15.86:6789/0 conn(0x7f2e581455d0 legacy=0x7f2e5806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e5d400640 1 -- 172.21.15.83:0/3239702478 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e580ceeb0 con 0x7f2e580d0280 2024-04-25T00:13:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e57e00640 1 -- 172.21.15.83:0/3239702478 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1365066062 0 0) 0x7f2e580ce610 con 0x7f2e581455d0 2024-04-25T00:13:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.880+0000 7f2e4fe00640 1 --2- 172.21.15.83:0/3239702478 >> v2:172.21.15.86:3300/0 conn(0x7f2e580ce080 0x7f2e580cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.884+0000 7f2e5d400640 1 --2- 172.21.15.83:0/3239702478 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e580d0280 0x7f2e580b2b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009d90 tx=0x7f2e5001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd4374ac3a37d549 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.884+0000 7f2e57e00640 1 -- 172.21.15.83:0/3239702478 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e500b2070 con 0x7f2e580d0280 2024-04-25T00:13:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.884+0000 7f2e57e00640 1 -- 172.21.15.83:0/3239702478 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e500a5940 con 0x7f2e580d0280 2024-04-25T00:13:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.884+0000 7f2e57e00640 1 -- 172.21.15.83:0/3239702478 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e500a5d10 con 0x7f2e580d0280 2024-04-25T00:13:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.884+0000 7f2e5f400640 1 -- 172.21.15.83:0/3239702478 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e580d0280 msgr2=0x7f2e580b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.884+0000 7f2e5f400640 1 --2- 172.21.15.83:0/3239702478 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e580d0280 0x7f2e580b2b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009d90 tx=0x7f2e5001c970 comp rx=0 tx=0).stop 2024-04-25T00:13:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.884+0000 7f2e5f400640 1 -- 172.21.15.83:0/3239702478 shutdown_connections 2024-04-25T00:13:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.884+0000 7f2e5f400640 1 --2- 172.21.15.83:0/3239702478 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e580d0280 0x7f2e580b2b30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.884+0000 7f2e5f400640 1 --2- 172.21.15.83:0/3239702478 >> v2:172.21.15.86:3300/0 conn(0x7f2e580ce080 0x7f2e580cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.884+0000 7f2e5f400640 1 -- 172.21.15.83:0/3239702478 >> 172.21.15.83:0/3239702478 conn(0x7f2e5806b3b0 msgr2=0x7f2e5806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.884+0000 7f2e5f400640 1 -- 172.21.15.83:0/3239702478 shutdown_connections 2024-04-25T00:13:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.884+0000 7f2e5f400640 1 -- 172.21.15.83:0/3239702478 wait complete. 2024-04-25T00:13:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.884+0000 7f2e5f400640 1 Processor -- start 2024-04-25T00:13:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.884+0000 7f2e5f400640 1 -- start start 2024-04-25T00:13:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e5f400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e580ce080 0x7f2e581577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e5f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e580d0280 0x7f2e58157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e5f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e581455d0 0x7f2e5815e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e5f400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e580b3c50 con 0x7f2e580d0280 2024-04-25T00:13:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e5f400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e580b3ad0 con 0x7f2e581455d0 2024-04-25T00:13:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e5f400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2e580b3dd0 con 0x7f2e580ce080 2024-04-25T00:13:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e4fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e580d0280 0x7f2e58157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e4fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e580d0280 0x7f2e58157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35058/0 (socket says 172.21.15.83:35058) 2024-04-25T00:13:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e4fe00640 1 -- 172.21.15.83:0/3088986829 learned_addr learned my addr 172.21.15.83:0/3088986829 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e5ca00640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e580ce080 0x7f2e581577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e5d400640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e581455d0 0x7f2e5815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e4fe00640 1 -- 172.21.15.83:0/3088986829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e580ce080 msgr2=0x7f2e581577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e4fe00640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e580ce080 0x7f2e581577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e4fe00640 1 -- 172.21.15.83:0/3088986829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e581455d0 msgr2=0x7f2e5815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e4fe00640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e581455d0 0x7f2e5815e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e4fe00640 1 -- 172.21.15.83:0/3088986829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e58158320 con 0x7f2e580d0280 2024-04-25T00:13:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e5d400640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e581455d0 0x7f2e5815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e4fe00640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e580d0280 0x7f2e58157de0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000ec10 tx=0x7f2e4000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e55600640 1 -- 172.21.15.83:0/3088986829 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e40017070 con 0x7f2e580d0280 2024-04-25T00:13:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e55600640 1 -- 172.21.15.83:0/3088986829 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e4000cc30 con 0x7f2e580d0280 2024-04-25T00:13:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e55600640 1 -- 172.21.15.83:0/3088986829 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e40013470 con 0x7f2e580d0280 2024-04-25T00:13:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e5f400640 1 -- 172.21.15.83:0/3088986829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e581585c0 con 0x7f2e580d0280 2024-04-25T00:13:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e5f400640 1 -- 172.21.15.83:0/3088986829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e58158b80 con 0x7f2e580d0280 2024-04-25T00:13:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e5f400640 1 -- 172.21.15.83:0/3088986829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e580736f0 con 0x7f2e580d0280 2024-04-25T00:13:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.888+0000 7f2e55600640 1 -- 172.21.15.83:0/3088986829 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2e4001a070 con 0x7f2e580d0280 2024-04-25T00:13:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.892+0000 7f2e55600640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e340678c0 0x7f2e34069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.892+0000 7f2e55600640 1 -- 172.21.15.83:0/3088986829 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6187+0+0 (secure 0 0 0) 0x7f2e400861b0 con 0x7f2e580d0280 2024-04-25T00:13:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.892+0000 7f2e5ca00640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e340678c0 0x7f2e34069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.892+0000 7f2e5ca00640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e340678c0 0x7f2e34069d80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2e48000f70 tx=0x7f2e48007d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:41.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:41.896+0000 7f2e55600640 1 -- 172.21.15.83:0/3088986829 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2e40056b80 con 0x7f2e580d0280 2024-04-25T00:13:42.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:42.052+0000 7f2e5f400640 1 -- 172.21.15.83:0/3088986829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2e5804f5f0 con 0x7f2e580d0280 2024-04-25T00:13:42.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:42.828+0000 7f2e55600640 1 -- 172.21.15.83:0/3088986829 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v62) ==== 181+0+0 (secure 0 0 0) 0x7f2e40056480 con 0x7f2e580d0280 2024-04-25T00:13:42.832 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:13:42.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:42.832+0000 7f2e5f400640 1 -- 172.21.15.83:0/3088986829 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e340678c0 msgr2=0x7f2e34069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:42.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:42.832+0000 7f2e5f400640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e340678c0 0x7f2e34069d80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2e48000f70 tx=0x7f2e48007d10 comp rx=0 tx=0).stop 2024-04-25T00:13:42.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:42.832+0000 7f2e5f400640 1 -- 172.21.15.83:0/3088986829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e580d0280 msgr2=0x7f2e58157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:42.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:42.832+0000 7f2e5f400640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e580d0280 0x7f2e58157de0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000ec10 tx=0x7f2e4000c520 comp rx=0 tx=0).stop 2024-04-25T00:13:42.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:42.832+0000 7f2e5f400640 1 -- 172.21.15.83:0/3088986829 shutdown_connections 2024-04-25T00:13:42.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:42.832+0000 7f2e5f400640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e340678c0 0x7f2e34069d80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:42.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:42.832+0000 7f2e5f400640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e581455d0 0x7f2e5815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:42.832+0000 7f2e5f400640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e580d0280 0x7f2e58157de0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:42.832+0000 7f2e5f400640 1 --2- 172.21.15.83:0/3088986829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e580ce080 0x7f2e581577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:42.832+0000 7f2e5f400640 1 -- 172.21.15.83:0/3088986829 >> 172.21.15.83:0/3088986829 conn(0x7f2e5806b3b0 msgr2=0x7f2e5815b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:42.832+0000 7f2e5f400640 1 -- 172.21.15.83:0/3088986829 shutdown_connections 2024-04-25T00:13:42.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:42.832+0000 7f2e5f400640 1 -- 172.21.15.83:0/3088986829 wait complete. 2024-04-25T00:13:42.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:13:43.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.000+0000 7fe7c5200640 1 Processor -- start 2024-04-25T00:13:43.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.000+0000 7fe7c5200640 1 -- start start 2024-04-25T00:13:43.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.004+0000 7fe7c5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe7c00b2070 0x7fe7c006c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:43.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.004+0000 7fe7c5200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe7c00b29a0 0x7fe7c006c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.004+0000 7fe7c5200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7c00b31f0 con 0x7fe7c00b2450 2024-04-25T00:13:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.004+0000 7fe7c5200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe7c00d0900 con 0x7fe7c00b29a0 2024-04-25T00:13:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.004+0000 7fe7c5200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe7c00d0be0 con 0x7fe7c00b2070 2024-04-25T00:13:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.004+0000 7fe7bea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe7c00b29a0 0x7fe7c006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.004+0000 7fe7bea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe7c00b29a0 0x7fe7c006c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35068/0 (socket says 172.21.15.83:35068) 2024-04-25T00:13:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.004+0000 7fe7bea00640 1 -- 172.21.15.83:0/800938910 learned_addr learned my addr 172.21.15.83:0/800938910 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.004+0000 7fe7bf400640 1 --2- 172.21.15.83:0/800938910 >> v2:172.21.15.86:3300/0 conn(0x7fe7c00b2070 0x7fe7c006c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.004+0000 7fe7bea00640 1 -- 172.21.15.83:0/800938910 >> v2:172.21.15.86:3300/0 conn(0x7fe7c00b2070 msgr2=0x7fe7c006c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.004+0000 7fe7bea00640 1 --2- 172.21.15.83:0/800938910 >> v2:172.21.15.86:3300/0 conn(0x7fe7c00b2070 0x7fe7c006c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.004+0000 7fe7bea00640 1 -- 172.21.15.83:0/800938910 >> v1:172.21.15.86:6789/0 conn(0x7fe7c00b2450 legacy=0x7fe7c0161d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.004+0000 7fe7bea00640 1 -- 172.21.15.83:0/800938910 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c00b2e10 con 0x7fe7c00b29a0 2024-04-25T00:13:43.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.008+0000 7fe7bea00640 1 --2- 172.21.15.83:0/800938910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c00b29a0 0x7fe7c006c830 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0068f30 tx=0x7fe7ac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc024a9c04f626d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:43.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.008+0000 7fe7be000640 1 -- 172.21.15.83:0/800938910 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0b0070 con 0x7fe7c00b29a0 2024-04-25T00:13:43.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.008+0000 7fe7be000640 1 -- 172.21.15.83:0/800938910 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ac0ab600 con 0x7fe7c00b29a0 2024-04-25T00:13:43.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.008+0000 7fe7be000640 1 -- 172.21.15.83:0/800938910 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0ab9d0 con 0x7fe7c00b29a0 2024-04-25T00:13:43.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.008+0000 7fe7c5200640 1 -- 172.21.15.83:0/800938910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c00b29a0 msgr2=0x7fe7c006c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:43.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.008+0000 7fe7c5200640 1 --2- 172.21.15.83:0/800938910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c00b29a0 0x7fe7c006c830 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0068f30 tx=0x7fe7ac01c520 comp rx=0 tx=0).stop 2024-04-25T00:13:43.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.008+0000 7fe7c5200640 1 -- 172.21.15.83:0/800938910 shutdown_connections 2024-04-25T00:13:43.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.008+0000 7fe7c5200640 1 --2- 172.21.15.83:0/800938910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c00b29a0 0x7fe7c006c830 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.008+0000 7fe7c5200640 1 --2- 172.21.15.83:0/800938910 >> v2:172.21.15.86:3300/0 conn(0x7fe7c00b2070 0x7fe7c006c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.008+0000 7fe7c5200640 1 -- 172.21.15.83:0/800938910 >> 172.21.15.83:0/800938910 conn(0x7fe7c00d1b30 msgr2=0x7fe7c00d1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:43.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.008+0000 7fe7c5200640 1 -- 172.21.15.83:0/800938910 shutdown_connections 2024-04-25T00:13:43.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.008+0000 7fe7c5200640 1 -- 172.21.15.83:0/800938910 wait complete. 2024-04-25T00:13:43.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.008+0000 7fe7c5200640 1 Processor -- start 2024-04-25T00:13:43.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.008+0000 7fe7c5200640 1 -- start start 2024-04-25T00:13:43.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7c5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c00b2070 0x7fe7c01613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:43.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7c5200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c00b2450 0x7fe7c0157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:43.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7c5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c00b29a0 0x7fe7c00b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:43.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7c5200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c016b2e0 con 0x7fe7c00b29a0 2024-04-25T00:13:43.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7c5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0162cb0 con 0x7fe7c00b2070 2024-04-25T00:13:43.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7c5200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c016b460 con 0x7fe7c00b2450 2024-04-25T00:13:43.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7bfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c00b29a0 0x7fe7c00b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:43.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c00b2070 0x7fe7c01613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:43.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7bea00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c00b2450 0x7fe7c0157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:43.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7bea00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c00b2450 0x7fe7c0157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:41342/0 (socket says 172.21.15.83:41342) 2024-04-25T00:13:43.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c00b2070 0x7fe7c01613a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:49958/0 (socket says 172.21.15.83:49958) 2024-04-25T00:13:43.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7bea00640 1 -- 172.21.15.83:0/4045530202 learned_addr learned my addr 172.21.15.83:0/4045530202 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:43.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7bfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c00b29a0 0x7fe7c00b0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35074/0 (socket says 172.21.15.83:35074) 2024-04-25T00:13:43.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7bfe00640 1 -- 172.21.15.83:0/4045530202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c00b2450 msgr2=0x7fe7c0157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:43.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7bfe00640 1 --2- 172.21.15.83:0/4045530202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c00b2450 0x7fe7c0157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7bfe00640 1 -- 172.21.15.83:0/4045530202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c00b2070 msgr2=0x7fe7c01613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:43.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7bfe00640 1 --2- 172.21.15.83:0/4045530202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c00b2070 0x7fe7c01613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7bfe00640 1 -- 172.21.15.83:0/4045530202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c00b0a90 con 0x7fe7c00b29a0 2024-04-25T00:13:43.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7bfe00640 1 --2- 172.21.15.83:0/4045530202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c00b29a0 0x7fe7c00b0310 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c700 tx=0x7fe7b800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:43.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7b7400640 1 -- 172.21.15.83:0/4045530202 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8003b80 con 0x7fe7c00b29a0 2024-04-25T00:13:43.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7b7400640 1 -- 172.21.15.83:0/4045530202 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b8003d20 con 0x7fe7c00b29a0 2024-04-25T00:13:43.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7b7400640 1 -- 172.21.15.83:0/4045530202 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8005310 con 0x7fe7c00b29a0 2024-04-25T00:13:43.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7c5200640 1 -- 172.21.15.83:0/4045530202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c00b0d70 con 0x7fe7c00b29a0 2024-04-25T00:13:43.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7c5200640 1 -- 172.21.15.83:0/4045530202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c00b1330 con 0x7fe7c00b29a0 2024-04-25T00:13:43.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7c5200640 1 -- 172.21.15.83:0/4045530202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c004f510 con 0x7fe7c00b29a0 2024-04-25T00:13:43.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.012+0000 7fe7b7400640 1 -- 172.21.15.83:0/4045530202 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe7b80054b0 con 0x7fe7c00b29a0 2024-04-25T00:13:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.016+0000 7fe7b7400640 1 --2- 172.21.15.83:0/4045530202 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7a00678c0 0x7fe7a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.016+0000 7fe7b7400640 1 -- 172.21.15.83:0/4045530202 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5749+0+0 (secure 0 0 0) 0x7fe7b8085950 con 0x7fe7c00b29a0 2024-04-25T00:13:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.016+0000 7fe7bf400640 1 --2- 172.21.15.83:0/4045530202 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7a00678c0 0x7fe7a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:43.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.016+0000 7fe7bf400640 1 --2- 172.21.15.83:0/4045530202 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7a00678c0 0x7fe7a0069d80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00b6da0 tx=0x7fe7b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:43.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.020+0000 7fe7b7400640 1 -- 172.21.15.83:0/4045530202 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe7b8055e00 con 0x7fe7c00b29a0 2024-04-25T00:13:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.176+0000 7fe7c5200640 1 -- 172.21.15.83:0/4045530202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe7c0074180 con 0x7fe7c00b29a0 2024-04-25T00:13:43.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.832+0000 7fe7b7400640 1 -- 172.21.15.83:0/4045530202 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v63) ==== 169+0+0 (secure 0 0 0) 0x7fe7b805a100 con 0x7fe7c00b29a0 2024-04-25T00:13:43.835 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-25T00:13:43.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.836+0000 7fe7c5200640 1 -- 172.21.15.83:0/4045530202 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7a00678c0 msgr2=0x7fe7a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:43.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.836+0000 7fe7c5200640 1 --2- 172.21.15.83:0/4045530202 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7a00678c0 0x7fe7a0069d80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00b6da0 tx=0x7fe7b0008040 comp rx=0 tx=0).stop 2024-04-25T00:13:43.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.836+0000 7fe7c5200640 1 -- 172.21.15.83:0/4045530202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c00b29a0 msgr2=0x7fe7c00b0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:43.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.836+0000 7fe7c5200640 1 --2- 172.21.15.83:0/4045530202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c00b29a0 0x7fe7c00b0310 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c700 tx=0x7fe7b800cbc0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.836+0000 7fe7c5200640 1 -- 172.21.15.83:0/4045530202 shutdown_connections 2024-04-25T00:13:43.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.836+0000 7fe7c5200640 1 --2- 172.21.15.83:0/4045530202 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe7a00678c0 0x7fe7a0069d80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.836+0000 7fe7c5200640 1 --2- 172.21.15.83:0/4045530202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7c00b2450 0x7fe7c0157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.836+0000 7fe7c5200640 1 --2- 172.21.15.83:0/4045530202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7c00b29a0 0x7fe7c00b0310 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.836+0000 7fe7c5200640 1 --2- 172.21.15.83:0/4045530202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c00b2070 0x7fe7c01613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.836+0000 7fe7c5200640 1 -- 172.21.15.83:0/4045530202 >> 172.21.15.83:0/4045530202 conn(0x7fe7c00d1b30 msgr2=0x7fe7c015de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:43.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.836+0000 7fe7c5200640 1 -- 172.21.15.83:0/4045530202 shutdown_connections 2024-04-25T00:13:43.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:43.836+0000 7fe7c5200640 1 -- 172.21.15.83:0/4045530202 wait complete. 2024-04-25T00:13:43.868 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:13:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.004+0000 7f59bca00640 1 Processor -- start 2024-04-25T00:13:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.004+0000 7f59bca00640 1 -- start start 2024-04-25T00:13:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59bca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f59b80727b0 0x7f59b815f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59bca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59b8072c10 0x7f59b815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59bca00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59b8070ca0 con 0x7f59b81455d0 2024-04-25T00:13:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59bca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f59b80ce9d0 con 0x7f59b80727b0 2024-04-25T00:13:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59bca00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f59b8070fa0 con 0x7f59b8072c10 2024-04-25T00:13:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59b7400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f59b81455d0 0x7f59b806cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40724/0 (socket says 172.21.15.83:40724) 2024-04-25T00:13:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59b7400640 1 -- 172.21.15.83:0/2235892396 learned_addr learned my addr 172.21.15.83:0/2235892396 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59aea00640 1 --2- 172.21.15.83:0/2235892396 >> v2:172.21.15.86:3300/0 conn(0x7f59b80727b0 0x7f59b815f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59b7e00640 1 --2- 172.21.15.83:0/2235892396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59b8072c10 0x7f59b815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59b6a00640 1 -- 172.21.15.83:0/2235892396 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1830226401 0 0) 0x7f59b8070ca0 con 0x7f59b81455d0 2024-04-25T00:13:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59b6a00640 1 -- 172.21.15.83:0/2235892396 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f599c003530 con 0x7f59b81455d0 2024-04-25T00:13:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59aea00640 1 -- 172.21.15.83:0/2235892396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59b8072c10 msgr2=0x7f59b815fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59aea00640 1 --2- 172.21.15.83:0/2235892396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59b8072c10 0x7f59b815fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59aea00640 1 -- 172.21.15.83:0/2235892396 >> v1:172.21.15.83:6789/0 conn(0x7f59b81455d0 legacy=0x7f59b806cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59b6a00640 1 -- 172.21.15.83:0/2235892396 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3329598854 0 0) 0x7f599c003530 con 0x7f59b81455d0 2024-04-25T00:13:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.008+0000 7f59aea00640 1 -- 172.21.15.83:0/2235892396 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b81605d0 con 0x7f59b80727b0 2024-04-25T00:13:44.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.012+0000 7f59aea00640 1 --2- 172.21.15.83:0/2235892396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b80727b0 0x7f59b815f800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f59a00097c0 tx=0x7f59a001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e095e4861f48593d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.012+0000 7f59b6a00640 1 -- 172.21.15.83:0/2235892396 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a00b2070 con 0x7f59b80727b0 2024-04-25T00:13:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.012+0000 7f59b6a00640 1 -- 172.21.15.83:0/2235892396 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a00ad640 con 0x7f59b80727b0 2024-04-25T00:13:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.012+0000 7f59b6a00640 1 -- 172.21.15.83:0/2235892396 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a00ada10 con 0x7f59b80727b0 2024-04-25T00:13:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.012+0000 7f59bca00640 1 -- 172.21.15.83:0/2235892396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b80727b0 msgr2=0x7f59b815f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.012+0000 7f59bca00640 1 --2- 172.21.15.83:0/2235892396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b80727b0 0x7f59b815f800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f59a00097c0 tx=0x7f59a001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.012+0000 7f59bca00640 1 -- 172.21.15.83:0/2235892396 shutdown_connections 2024-04-25T00:13:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.012+0000 7f59bca00640 1 --2- 172.21.15.83:0/2235892396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59b8072c10 0x7f59b815fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.012+0000 7f59bca00640 1 --2- 172.21.15.83:0/2235892396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b80727b0 0x7f59b815f800 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.012+0000 7f59bca00640 1 -- 172.21.15.83:0/2235892396 >> 172.21.15.83:0/2235892396 conn(0x7f59b80cf980 msgr2=0x7f59b80cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.012+0000 7f59bca00640 1 -- 172.21.15.83:0/2235892396 shutdown_connections 2024-04-25T00:13:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.012+0000 7f59bca00640 1 -- 172.21.15.83:0/2235892396 wait complete. 2024-04-25T00:13:44.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.012+0000 7f59bca00640 1 Processor -- start 2024-04-25T00:13:44.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.012+0000 7f59bca00640 1 -- start start 2024-04-25T00:13:44.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59bca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b80727b0 0x7f59b80aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:44.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59bca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b8072c10 0x7f59b80ac240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:44.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59bca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59b81455d0 0x7f59b813e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:44.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59bca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8071dc0 con 0x7f59b80727b0 2024-04-25T00:13:44.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59bca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8071c40 con 0x7f59b8072c10 2024-04-25T00:13:44.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59bca00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f59b8071f40 con 0x7f59b81455d0 2024-04-25T00:13:44.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59b7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b80727b0 0x7f59b80aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:44.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59b7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b80727b0 0x7f59b80aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35080/0 (socket says 172.21.15.83:35080) 2024-04-25T00:13:44.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59b7400640 1 -- 172.21.15.83:0/3735274837 learned_addr learned my addr 172.21.15.83:0/3735274837 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:44.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59aea00640 1 --2- 172.21.15.83:0/3735274837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b8072c10 0x7f59b80ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:44.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59b7e00640 1 --2- 172.21.15.83:0/3735274837 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59b81455d0 0x7f59b813e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:44.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59b7400640 1 -- 172.21.15.83:0/3735274837 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59b81455d0 msgr2=0x7f59b813e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59b7400640 1 --2- 172.21.15.83:0/3735274837 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59b81455d0 0x7f59b813e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59b7400640 1 -- 172.21.15.83:0/3735274837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b8072c10 msgr2=0x7f59b80ac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59b7400640 1 --2- 172.21.15.83:0/3735274837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b8072c10 0x7f59b80ac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59b7400640 1 -- 172.21.15.83:0/3735274837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b80b3460 con 0x7f59b80727b0 2024-04-25T00:13:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59b7400640 1 --2- 172.21.15.83:0/3735274837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b80727b0 0x7f59b80aebb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f59a800c780 tx=0x7f59a800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59afe00640 1 -- 172.21.15.83:0/3735274837 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8005ac0 con 0x7f59b80727b0 2024-04-25T00:13:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59afe00640 1 -- 172.21.15.83:0/3735274837 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a8005c60 con 0x7f59b80727b0 2024-04-25T00:13:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59afe00640 1 -- 172.21.15.83:0/3735274837 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a80072b0 con 0x7f59b80727b0 2024-04-25T00:13:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59bca00640 1 -- 172.21.15.83:0/3735274837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b80b3700 con 0x7f59b80727b0 2024-04-25T00:13:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59bca00640 1 -- 172.21.15.83:0/3735274837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b80aabb0 con 0x7f59b80727b0 2024-04-25T00:13:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59bca00640 1 -- 172.21.15.83:0/3735274837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b813ec80 con 0x7f59b80727b0 2024-04-25T00:13:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.016+0000 7f59afe00640 1 -- 172.21.15.83:0/3735274837 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f59a8007450 con 0x7f59b80727b0 2024-04-25T00:13:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.020+0000 7f59afe00640 1 --2- 172.21.15.83:0/3735274837 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59940678f0 0x7f5994069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.020+0000 7f59afe00640 1 -- 172.21.15.83:0/3735274837 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5335+0+0 (secure 0 0 0) 0x7f59a80858a0 con 0x7f59b80727b0 2024-04-25T00:13:44.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.020+0000 7f59aea00640 1 --2- 172.21.15.83:0/3735274837 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59940678f0 0x7f5994069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:44.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.020+0000 7f59aea00640 1 --2- 172.21.15.83:0/3735274837 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59940678f0 0x7f5994069db0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f59a00af000 tx=0x7f59a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:44.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.024+0000 7f59afe00640 1 -- 172.21.15.83:0/3735274837 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f59b813ec80 con 0x7f59b80727b0 2024-04-25T00:13:44.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.176+0000 7f59bca00640 1 -- 172.21.15.83:0/3735274837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f59b804f510 con 0x7f59b80727b0 2024-04-25T00:13:44.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.184+0000 7f59afe00640 1 -- 172.21.15.83:0/3735274837 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f59a8010300 con 0x7f59b80727b0 2024-04-25T00:13:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.188+0000 7f59bca00640 1 -- 172.21.15.83:0/3735274837 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59940678f0 msgr2=0x7f5994069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.188+0000 7f59bca00640 1 --2- 172.21.15.83:0/3735274837 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59940678f0 0x7f5994069db0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f59a00af000 tx=0x7f59a0008040 comp rx=0 tx=0).stop 2024-04-25T00:13:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.188+0000 7f59bca00640 1 -- 172.21.15.83:0/3735274837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b80727b0 msgr2=0x7f59b80aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.188+0000 7f59bca00640 1 --2- 172.21.15.83:0/3735274837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b80727b0 0x7f59b80aebb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f59a800c780 tx=0x7f59a800cc40 comp rx=0 tx=0).stop 2024-04-25T00:13:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.188+0000 7f59bca00640 1 -- 172.21.15.83:0/3735274837 shutdown_connections 2024-04-25T00:13:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.188+0000 7f59bca00640 1 --2- 172.21.15.83:0/3735274837 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59940678f0 0x7f5994069db0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.188+0000 7f59bca00640 1 --2- 172.21.15.83:0/3735274837 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59b81455d0 0x7f59b813e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.188+0000 7f59bca00640 1 --2- 172.21.15.83:0/3735274837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b8072c10 0x7f59b80ac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.188+0000 7f59bca00640 1 --2- 172.21.15.83:0/3735274837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b80727b0 0x7f59b80aebb0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.188+0000 7f59bca00640 1 -- 172.21.15.83:0/3735274837 >> 172.21.15.83:0/3735274837 conn(0x7f59b80cf980 msgr2=0x7f59b80cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.188+0000 7f59bca00640 1 -- 172.21.15.83:0/3735274837 shutdown_connections 2024-04-25T00:13:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.188+0000 7f59bca00640 1 -- 172.21.15.83:0/3735274837 wait complete. 2024-04-25T00:13:44.210 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject' 2024-04-25T00:13:44.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.340+0000 7fbd3fc00640 1 Processor -- start 2024-04-25T00:13:44.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.340+0000 7fbd3fc00640 1 -- start start 2024-04-25T00:13:44.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd3fc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbd381599f0 0x7fbd380cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:44.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd3fc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3815c340 0x7fbd3815c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:44.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd3fc00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd380ccc60 con 0x7fbd3815ccc0 2024-04-25T00:13:44.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd3fc00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbd38159dd0 con 0x7fbd381599f0 2024-04-25T00:13:44.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd3fc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3806dba0 con 0x7fbd3815c340 2024-04-25T00:13:44.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd3d200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbd381599f0 0x7fbd380cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:44.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd3dc00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbd3815ccc0 0x7fbd3815f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41594/0 (socket says 172.21.15.83:41594) 2024-04-25T00:13:44.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd3d200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbd381599f0 0x7fbd380cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43930/0 (socket says 172.21.15.83:43930) 2024-04-25T00:13:44.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd3dc00640 1 -- 172.21.15.83:0/4232778529 learned_addr learned my addr 172.21.15.83:0/4232778529 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:44.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd2fe00640 1 --2- 172.21.15.83:0/4232778529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3815c340 0x7fbd3815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:44.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd3d200640 1 -- 172.21.15.83:0/4232778529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3815c340 msgr2=0x7fbd3815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:44.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd3d200640 1 --2- 172.21.15.83:0/4232778529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3815c340 0x7fbd3815c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd3d200640 1 -- 172.21.15.83:0/4232778529 >> v1:172.21.15.83:6789/0 conn(0x7fbd3815ccc0 legacy=0x7fbd3815f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:44.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd3d200640 1 -- 172.21.15.83:0/4232778529 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3806de80 con 0x7fbd381599f0 2024-04-25T00:13:44.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd2fe00640 1 --2- 172.21.15.83:0/4232778529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3815c340 0x7fbd3815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:44.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.344+0000 7fbd37e00640 1 -- 172.21.15.83:0/4232778529 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3435301701 0 0) 0x7fbd300014a0 con 0x7fbd3815ccc0 2024-04-25T00:13:44.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.348+0000 7fbd3d200640 1 --2- 172.21.15.83:0/4232778529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd381599f0 0x7fbd380cc510 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbd280097f0 tx=0x7fbd2801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46eebae1aaa4b90a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:44.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.348+0000 7fbd37e00640 1 -- 172.21.15.83:0/4232778529 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd280b0070 con 0x7fbd381599f0 2024-04-25T00:13:44.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.348+0000 7fbd37e00640 1 -- 172.21.15.83:0/4232778529 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd280ab600 con 0x7fbd381599f0 2024-04-25T00:13:44.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.348+0000 7fbd37e00640 1 -- 172.21.15.83:0/4232778529 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd280ab9d0 con 0x7fbd381599f0 2024-04-25T00:13:44.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.348+0000 7fbd3fc00640 1 -- 172.21.15.83:0/4232778529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd381599f0 msgr2=0x7fbd380cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:44.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.348+0000 7fbd3fc00640 1 --2- 172.21.15.83:0/4232778529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd381599f0 0x7fbd380cc510 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbd280097f0 tx=0x7fbd2801c410 comp rx=0 tx=0).stop 2024-04-25T00:13:44.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.348+0000 7fbd3fc00640 1 -- 172.21.15.83:0/4232778529 shutdown_connections 2024-04-25T00:13:44.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.348+0000 7fbd3fc00640 1 --2- 172.21.15.83:0/4232778529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3815c340 0x7fbd3815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.348+0000 7fbd3fc00640 1 --2- 172.21.15.83:0/4232778529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd381599f0 0x7fbd380cc510 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.348+0000 7fbd3fc00640 1 -- 172.21.15.83:0/4232778529 >> 172.21.15.83:0/4232778529 conn(0x7fbd380b1830 msgr2=0x7fbd380b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:44.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.348+0000 7fbd3fc00640 1 -- 172.21.15.83:0/4232778529 shutdown_connections 2024-04-25T00:13:44.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.348+0000 7fbd3fc00640 1 -- 172.21.15.83:0/4232778529 wait complete. 2024-04-25T00:13:44.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.348+0000 7fbd3fc00640 1 Processor -- start 2024-04-25T00:13:44.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.348+0000 7fbd3fc00640 1 -- start start 2024-04-25T00:13:44.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3fc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd381599f0 0x7fbd38145af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:44.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3815c340 0x7fbd38157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:44.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3fc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3815ccc0 0x7fbd38150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3fc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd380cf870 con 0x7fbd381599f0 2024-04-25T00:13:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd380cf6f0 con 0x7fbd3815c340 2024-04-25T00:13:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3fc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbd38168cd0 con 0x7fbd3815ccc0 2024-04-25T00:13:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd381599f0 0x7fbd38145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd381599f0 0x7fbd38145af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43932/0 (socket says 172.21.15.83:43932) 2024-04-25T00:13:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3d200640 1 -- 172.21.15.83:0/253402917 learned_addr learned my addr 172.21.15.83:0/253402917 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd2fe00640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3815c340 0x7fbd38157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3dc00640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3815ccc0 0x7fbd38150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3d200640 1 -- 172.21.15.83:0/253402917 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3815ccc0 msgr2=0x7fbd38150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3d200640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3815ccc0 0x7fbd38150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3d200640 1 -- 172.21.15.83:0/253402917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3815c340 msgr2=0x7fbd38157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3d200640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3815c340 0x7fbd38157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3d200640 1 -- 172.21.15.83:0/253402917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd381579d0 con 0x7fbd381599f0 2024-04-25T00:13:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd2fe00640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3815c340 0x7fbd38157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3d200640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd381599f0 0x7fbd38145af0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbd2801a040 tx=0x7fbd2801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd35600640 1 -- 172.21.15.83:0/253402917 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd280b0050 con 0x7fbd381599f0 2024-04-25T00:13:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd35600640 1 -- 172.21.15.83:0/253402917 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd280b1040 con 0x7fbd381599f0 2024-04-25T00:13:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd35600640 1 -- 172.21.15.83:0/253402917 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd280b5610 con 0x7fbd381599f0 2024-04-25T00:13:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3fc00640 1 -- 172.21.15.83:0/253402917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd38157cb0 con 0x7fbd381599f0 2024-04-25T00:13:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3fc00640 1 -- 172.21.15.83:0/253402917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd38149a70 con 0x7fbd381599f0 2024-04-25T00:13:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd3fc00640 1 -- 172.21.15.83:0/253402917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd380737f0 con 0x7fbd381599f0 2024-04-25T00:13:44.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.352+0000 7fbd35600640 1 -- 172.21.15.83:0/253402917 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbd280be030 con 0x7fbd381599f0 2024-04-25T00:13:44.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.356+0000 7fbd35600640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd140678c0 0x7fbd14069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:44.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.356+0000 7fbd35600640 1 -- 172.21.15.83:0/253402917 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5335+0+0 (secure 0 0 0) 0x7fbd2811f880 con 0x7fbd381599f0 2024-04-25T00:13:44.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.356+0000 7fbd2fe00640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd140678c0 0x7fbd14069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:44.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.356+0000 7fbd2fe00640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd140678c0 0x7fbd14069d80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbd380cfeb0 tx=0x7fbd20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:44.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.360+0000 7fbd35600640 1 -- 172.21.15.83:0/253402917 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbd280c5060 con 0x7fbd381599f0 2024-04-25T00:13:44.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.572+0000 7fbd3fc00640 1 -- 172.21.15.83:0/253402917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]} v 0) -- 0x7fbd38157e80 con 0x7fbd381599f0 2024-04-25T00:13:44.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.840+0000 7fbd35600640 1 -- 172.21.15.83:0/253402917 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]}]=0 v179) ==== 142+0+0 (secure 0 0 0) 0x7fbd28016090 con 0x7fbd381599f0 2024-04-25T00:13:44.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.844+0000 7fbd3fc00640 1 -- 172.21.15.83:0/253402917 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd140678c0 msgr2=0x7fbd14069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:44.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.844+0000 7fbd3fc00640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd140678c0 0x7fbd14069d80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbd380cfeb0 tx=0x7fbd20008040 comp rx=0 tx=0).stop 2024-04-25T00:13:44.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.844+0000 7fbd3fc00640 1 -- 172.21.15.83:0/253402917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd381599f0 msgr2=0x7fbd38145af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:44.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.844+0000 7fbd3fc00640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd381599f0 0x7fbd38145af0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbd2801a040 tx=0x7fbd2801ac50 comp rx=0 tx=0).stop 2024-04-25T00:13:44.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.844+0000 7fbd3fc00640 1 -- 172.21.15.83:0/253402917 shutdown_connections 2024-04-25T00:13:44.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.844+0000 7fbd3fc00640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbd140678c0 0x7fbd14069d80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.844+0000 7fbd3fc00640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3815ccc0 0x7fbd38150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.844+0000 7fbd3fc00640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3815c340 0x7fbd38157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.844+0000 7fbd3fc00640 1 --2- 172.21.15.83:0/253402917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd381599f0 0x7fbd38145af0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.844+0000 7fbd3fc00640 1 -- 172.21.15.83:0/253402917 >> 172.21.15.83:0/253402917 conn(0x7fbd380b1830 msgr2=0x7fbd3806d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:44.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.844+0000 7fbd3fc00640 1 -- 172.21.15.83:0/253402917 shutdown_connections 2024-04-25T00:13:44.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:44.844+0000 7fbd3fc00640 1 -- 172.21.15.83:0/253402917 wait complete. 2024-04-25T00:13:44.870 INFO:tasks.cephfs_test_runner:test_session_reject (tasks.cephfs.test_sessionmap.TestSessionMap) ... skipped 'Requires FUSE client to inject client metadata' 2024-04-25T00:13:44.870 INFO:tasks.cephfs_test_runner:Starting test: test_tell_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) That when a `tell` command is sent using the python CLI, 2024-04-25T00:13:44.870 INFO:tasks.cephfs_test_runner:test_tell_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-04-25T00:13:44.871 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:45.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.000+0000 7f0a1ea00640 1 Processor -- start 2024-04-25T00:13:45.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.000+0000 7f0a1ea00640 1 -- start start 2024-04-25T00:13:45.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a1ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0a18164cb0 0x7f0a181670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a1ea00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a180ccc60 con 0x7f0a181599f0 2024-04-25T00:13:45.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a1ea00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a1806dba0 con 0x7f0a180cc510 2024-04-25T00:13:45.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a1ea00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0a18159dd0 con 0x7f0a18164cb0 2024-04-25T00:13:45.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a17e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0a181599f0 0x7f0a1815c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41604/0 (socket says 172.21.15.83:41604) 2024-04-25T00:13:45.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a17e00640 1 -- 172.21.15.83:0/3888540565 learned_addr learned my addr 172.21.15.83:0/3888540565 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:45.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a1ca00640 1 --2- 172.21.15.83:0/3888540565 >> v2:172.21.15.86:3300/0 conn(0x7f0a18164cb0 0x7f0a181670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 -- 172.21.15.83:0/3888540565 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2551311189 0 0) 0x7f0a080014a0 con 0x7f0a181599f0 2024-04-25T00:13:45.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 -- 172.21.15.83:0/3888540565 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09fc003530 con 0x7f0a181599f0 2024-04-25T00:13:45.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 -- 172.21.15.83:0/3888540565 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 254895373 0 0) 0x7f09fc003530 con 0x7f0a181599f0 2024-04-25T00:13:45.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 -- 172.21.15.83:0/3888540565 >> v2:172.21.15.86:3300/0 conn(0x7f0a18164cb0 msgr2=0x7f0a181670a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:45.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 --2- 172.21.15.83:0/3888540565 >> v2:172.21.15.86:3300/0 conn(0x7f0a18164cb0 0x7f0a181670a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 -- 172.21.15.83:0/3888540565 >> v1:172.21.15.86:6789/0 conn(0x7f0a180cc510 legacy=0x7f0a1815ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 -- 172.21.15.83:0/3888540565 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1806de80 con 0x7f0a181599f0 2024-04-25T00:13:45.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 -- 172.21.15.83:0/3888540565 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f0a08001bd0 con 0x7f0a181599f0 2024-04-25T00:13:45.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 -- 172.21.15.83:0/3888540565 >> v1:172.21.15.83:6789/0 conn(0x7f0a181599f0 legacy=0x7f0a1815c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 --2- 172.21.15.83:0/3888540565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc0039c0 0x7f09fc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 --2- 172.21.15.83:0/3888540565 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09fc0043c0 0x7f09fc004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 --2- 172.21.15.83:0/3888540565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09fc005130 0x7f09fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 -- 172.21.15.83:0/3888540565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc00c040 con 0x7f09fc005130 2024-04-25T00:13:45.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 -- 172.21.15.83:0/3888540565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc00c1f0 con 0x7f09fc0039c0 2024-04-25T00:13:45.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a16a00640 1 -- 172.21.15.83:0/3888540565 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f09fc00c3f0 con 0x7f09fc0043c0 2024-04-25T00:13:45.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a1ca00640 1 --2- 172.21.15.83:0/3888540565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09fc005130 0x7f09fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a17e00640 1 --2- 172.21.15.83:0/3888540565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc0039c0 0x7f09fc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a17400640 1 --2- 172.21.15.83:0/3888540565 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09fc0043c0 0x7f09fc004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a1ca00640 1 -- 172.21.15.83:0/3888540565 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09fc0043c0 msgr2=0x7f09fc004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:45.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a1ca00640 1 --2- 172.21.15.83:0/3888540565 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09fc0043c0 0x7f09fc004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a1ca00640 1 -- 172.21.15.83:0/3888540565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc0039c0 msgr2=0x7f09fc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:45.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a1ca00640 1 --2- 172.21.15.83:0/3888540565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc0039c0 0x7f09fc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a1ca00640 1 -- 172.21.15.83:0/3888540565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc003530 con 0x7f09fc005130 2024-04-25T00:13:45.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.004+0000 7f0a17400640 1 --2- 172.21.15.83:0/3888540565 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09fc0043c0 0x7f09fc004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:45.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.008+0000 7f0a1ca00640 1 -- 172.21.15.83:0/3888540565 reap_dead start 2024-04-25T00:13:45.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.008+0000 7f0a1ca00640 1 --2- 172.21.15.83:0/3888540565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09fc005130 0x7f09fc00b9d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009ae0 tx=0x7f0a1001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c6dbbb9becc98f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:45.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.008+0000 7f0a16a00640 1 -- 172.21.15.83:0/3888540565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a100096b0 con 0x7f09fc005130 2024-04-25T00:13:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.008+0000 7f0a16a00640 1 -- 172.21.15.83:0/3888540565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a10004020 con 0x7f09fc005130 2024-04-25T00:13:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.008+0000 7f0a16a00640 1 -- 172.21.15.83:0/3888540565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10004410 con 0x7f09fc005130 2024-04-25T00:13:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.008+0000 7f0a1ea00640 1 -- 172.21.15.83:0/3888540565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09fc005130 msgr2=0x7f09fc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.008+0000 7f0a1ea00640 1 --2- 172.21.15.83:0/3888540565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09fc005130 0x7f09fc00b9d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009ae0 tx=0x7f0a1001cab0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.008+0000 7f0a1ea00640 1 -- 172.21.15.83:0/3888540565 shutdown_connections 2024-04-25T00:13:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.008+0000 7f0a1ea00640 1 --2- 172.21.15.83:0/3888540565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09fc005130 0x7f09fc00b9d0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.008+0000 7f0a1ea00640 1 -- 172.21.15.83:0/3888540565 >> 172.21.15.83:0/3888540565 conn(0x7f0a180b1830 msgr2=0x7f0a180b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.008+0000 7f0a1ea00640 1 -- 172.21.15.83:0/3888540565 shutdown_connections 2024-04-25T00:13:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.008+0000 7f0a1ea00640 1 -- 172.21.15.83:0/3888540565 wait complete. 2024-04-25T00:13:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.008+0000 7f0a1ea00640 1 Processor -- start 2024-04-25T00:13:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a1ea00640 1 -- start start 2024-04-25T00:13:45.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a1ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09fc005130 0x7f0a181463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a1ea00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a18160af0 0x7f0a180afe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a1ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a18160f70 0x7f0a180b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a1ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a180ce0c0 con 0x7f09fc005130 2024-04-25T00:13:45.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a1ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a180cc980 con 0x7f0a18160f70 2024-04-25T00:13:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a1ea00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0a180ce240 con 0x7f0a18160af0 2024-04-25T00:13:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a17e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09fc005130 0x7f0a181463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a17e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09fc005130 0x7f0a181463b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43952/0 (socket says 172.21.15.83:43952) 2024-04-25T00:13:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a17e00640 1 -- 172.21.15.83:0/2616634916 learned_addr learned my addr 172.21.15.83:0/2616634916 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a17400640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a18160af0 0x7f0a180afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a1ca00640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a18160f70 0x7f0a180b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a17e00640 1 -- 172.21.15.83:0/2616634916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a18160af0 msgr2=0x7f0a180afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a17e00640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a18160af0 0x7f0a180afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a17e00640 1 -- 172.21.15.83:0/2616634916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a18160f70 msgr2=0x7f0a180b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a17e00640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a18160f70 0x7f0a180b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a17e00640 1 -- 172.21.15.83:0/2616634916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1815fd30 con 0x7f09fc005130 2024-04-25T00:13:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a1ca00640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a18160f70 0x7f0a180b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a17e00640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09fc005130 0x7f0a181463b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800aae0 tx=0x7f0a08007060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a0fe00640 1 -- 172.21.15.83:0/2616634916 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08007a40 con 0x7f09fc005130 2024-04-25T00:13:45.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a0fe00640 1 -- 172.21.15.83:0/2616634916 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a08018070 con 0x7f09fc005130 2024-04-25T00:13:45.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a0fe00640 1 -- 172.21.15.83:0/2616634916 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08013630 con 0x7f09fc005130 2024-04-25T00:13:45.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a1ea00640 1 -- 172.21.15.83:0/2616634916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a18160010 con 0x7f09fc005130 2024-04-25T00:13:45.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a1ea00640 1 -- 172.21.15.83:0/2616634916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a181605d0 con 0x7f09fc005130 2024-04-25T00:13:45.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.012+0000 7f0a1ea00640 1 -- 172.21.15.83:0/2616634916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a180737f0 con 0x7f09fc005130 2024-04-25T00:13:45.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.016+0000 7f0a0fe00640 1 -- 172.21.15.83:0/2616634916 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0a080137d0 con 0x7f09fc005130 2024-04-25T00:13:45.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.016+0000 7f0a0fe00640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f09f80678f0 0x7f09f8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.016+0000 7f0a0fe00640 1 -- 172.21.15.83:0/2616634916 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5335+0+0 (secure 0 0 0) 0x7f0a080863b0 con 0x7f09fc005130 2024-04-25T00:13:45.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.016+0000 7f0a17400640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f09f80678f0 0x7f09f8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.016+0000 7f0a17400640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f09f80678f0 0x7f09f8069db0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0a04001210 tx=0x7f0a0400a990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:45.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.020+0000 7f0a0fe00640 1 -- 172.21.15.83:0/2616634916 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0a180ccc60 con 0x7f09fc005130 2024-04-25T00:13:45.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.176+0000 7f0a1ea00640 1 -- 172.21.15.83:0/2616634916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a181607a0 con 0x7f09fc005130 2024-04-25T00:13:45.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.176+0000 7f0a0fe00640 1 -- 172.21.15.83:0/2616634916 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+13206 (secure 0 0 0) 0x7f0a08056fc0 con 0x7f09fc005130 2024-04-25T00:13:45.179 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:45.180 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":63,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:43.833282+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6834/52944842":"2024-04-26T00:13:40.706470+0000","172.21.15.86:6832/52944842":"2024-04-26T00:13:40.706470+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:45.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.176+0000 7f0a1ea00640 1 -- 172.21.15.83:0/2616634916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f09f80678f0 msgr2=0x7f09f8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.176+0000 7f0a1ea00640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f09f80678f0 0x7f09f8069db0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0a04001210 tx=0x7f0a0400a990 comp rx=0 tx=0).stop 2024-04-25T00:13:45.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.176+0000 7f0a1ea00640 1 -- 172.21.15.83:0/2616634916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09fc005130 msgr2=0x7f0a181463b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.176+0000 7f0a1ea00640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09fc005130 0x7f0a181463b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800aae0 tx=0x7f0a08007060 comp rx=0 tx=0).stop 2024-04-25T00:13:45.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.176+0000 7f0a1ea00640 1 -- 172.21.15.83:0/2616634916 shutdown_connections 2024-04-25T00:13:45.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.176+0000 7f0a1ea00640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f09f80678f0 0x7f09f8069db0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.176+0000 7f0a1ea00640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a18160af0 0x7f0a180afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.176+0000 7f0a1ea00640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a18160f70 0x7f0a180b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.176+0000 7f0a1ea00640 1 --2- 172.21.15.83:0/2616634916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09fc005130 0x7f0a181463b0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.176+0000 7f0a1ea00640 1 -- 172.21.15.83:0/2616634916 >> 172.21.15.83:0/2616634916 conn(0x7f0a180b1830 msgr2=0x7f0a181577d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:45.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.180+0000 7f0a1ea00640 1 -- 172.21.15.83:0/2616634916 shutdown_connections 2024-04-25T00:13:45.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.180+0000 7f0a1ea00640 1 -- 172.21.15.83:0/2616634916 wait complete. 2024-04-25T00:13:45.200 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:13:45.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:13:45.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60ee00640 1 Processor -- start 2024-04-25T00:13:45.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60ee00640 1 -- start start 2024-04-25T00:13:45.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60ee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff60815e790 0x7ff60815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60ee00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6080719f0 con 0x7ff608155310 2024-04-25T00:13:45.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60ee00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff608071cd0 con 0x7ff6080b2da0 2024-04-25T00:13:45.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60ee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff60815f610 con 0x7ff60815e790 2024-04-25T00:13:45.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60da00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff608155310 0x7ff60806cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41610/0 (socket says 172.21.15.83:41610) 2024-04-25T00:13:45.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60da00640 1 -- 172.21.15.83:0/648829114 learned_addr learned my addr 172.21.15.83:0/648829114 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:45.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 -- 172.21.15.83:0/648829114 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3359371123 0 0) 0x7ff6080719f0 con 0x7ff608155310 2024-04-25T00:13:45.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 -- 172.21.15.83:0/648829114 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5f0003530 con 0x7ff608155310 2024-04-25T00:13:45.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60d000640 1 --2- 172.21.15.83:0/648829114 >> v2:172.21.15.86:3300/0 conn(0x7ff60815e790 0x7ff60815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 -- 172.21.15.83:0/648829114 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3283675099 0 0) 0x7ff5f0003530 con 0x7ff608155310 2024-04-25T00:13:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 -- 172.21.15.83:0/648829114 >> v2:172.21.15.86:3300/0 conn(0x7ff60815e790 msgr2=0x7ff60815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 --2- 172.21.15.83:0/648829114 >> v2:172.21.15.86:3300/0 conn(0x7ff60815e790 0x7ff60815ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 -- 172.21.15.83:0/648829114 >> v1:172.21.15.86:6789/0 conn(0x7ff6080b2da0 legacy=0x7ff6080b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 -- 172.21.15.83:0/648829114 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff608071fb0 con 0x7ff608155310 2024-04-25T00:13:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 -- 172.21.15.83:0/648829114 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7ff5fc001680 con 0x7ff608155310 2024-04-25T00:13:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60d000640 1 --2- 172.21.15.83:0/648829114 >> v2:172.21.15.86:3300/0 conn(0x7ff60815e790 0x7ff60815ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 -- 172.21.15.83:0/648829114 >> v1:172.21.15.83:6789/0 conn(0x7ff608155310 legacy=0x7ff60806cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 --2- 172.21.15.83:0/648829114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5f0003a10 0x7ff5f0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 --2- 172.21.15.83:0/648829114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f0004810 0x7ff5f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 --2- 172.21.15.83:0/648829114 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5f00052f0 0x7ff5f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 -- 172.21.15.83:0/648829114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f000c1e0 con 0x7ff5f0003a10 2024-04-25T00:13:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 -- 172.21.15.83:0/648829114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f000c390 con 0x7ff5f0004810 2024-04-25T00:13:45.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff607e00640 1 -- 172.21.15.83:0/648829114 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f000c590 con 0x7ff5f00052f0 2024-04-25T00:13:45.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60da00640 1 --2- 172.21.15.83:0/648829114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5f0003a10 0x7ff5f0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60da00640 1 -- 172.21.15.83:0/648829114 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5f00052f0 msgr2=0x7ff5f000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:45.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60da00640 1 --2- 172.21.15.83:0/648829114 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5f00052f0 0x7ff5f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60da00640 1 -- 172.21.15.83:0/648829114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f0004810 msgr2=0x7ff5f0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:45.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60da00640 1 --2- 172.21.15.83:0/648829114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f0004810 0x7ff5f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60da00640 1 -- 172.21.15.83:0/648829114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f0003530 con 0x7ff5f0003a10 2024-04-25T00:13:45.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.336+0000 7ff60e400640 1 -- 172.21.15.83:0/648829114 reap_dead start 2024-04-25T00:13:45.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.340+0000 7ff60da00640 1 --2- 172.21.15.83:0/648829114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5f0003a10 0x7ff5f0003e30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc007c30 tx=0x7ff5fc01edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d19b4a330df5845 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:45.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.340+0000 7ff607e00640 1 -- 172.21.15.83:0/648829114 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc007c70 con 0x7ff5f0003a10 2024-04-25T00:13:45.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.340+0000 7ff607e00640 1 -- 172.21.15.83:0/648829114 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5fc0ab070 con 0x7ff5f0003a10 2024-04-25T00:13:45.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.340+0000 7ff607e00640 1 -- 172.21.15.83:0/648829114 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc005da0 con 0x7ff5f0003a10 2024-04-25T00:13:45.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.340+0000 7ff60ee00640 1 -- 172.21.15.83:0/648829114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5f0003a10 msgr2=0x7ff5f0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.340+0000 7ff60ee00640 1 --2- 172.21.15.83:0/648829114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5f0003a10 0x7ff5f0003e30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc007c30 tx=0x7ff5fc01edf0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.340+0000 7ff60ee00640 1 -- 172.21.15.83:0/648829114 shutdown_connections 2024-04-25T00:13:45.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.340+0000 7ff60ee00640 1 --2- 172.21.15.83:0/648829114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5f0003a10 0x7ff5f0003e30 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.340+0000 7ff60ee00640 1 -- 172.21.15.83:0/648829114 >> 172.21.15.83:0/648829114 conn(0x7ff6080cf980 msgr2=0x7ff6080cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:45.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.340+0000 7ff60ee00640 1 -- 172.21.15.83:0/648829114 shutdown_connections 2024-04-25T00:13:45.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.340+0000 7ff60ee00640 1 -- 172.21.15.83:0/648829114 wait complete. 2024-04-25T00:13:45.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60ee00640 1 Processor -- start 2024-04-25T00:13:45.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60ee00640 1 -- start start 2024-04-25T00:13:45.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60ee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5f0003a10 0x7ff60806d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608144f20 0x7ff6081453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60ee00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6080ad180 0x7ff6080ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60ee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff608071740 con 0x7ff5f0003a10 2024-04-25T00:13:45.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6080715c0 con 0x7ff608144f20 2024-04-25T00:13:45.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60ee00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff60815b1a0 con 0x7ff6080ad180 2024-04-25T00:13:45.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5f0003a10 0x7ff60806d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5f0003a10 0x7ff60806d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43964/0 (socket says 172.21.15.83:43964) 2024-04-25T00:13:45.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60da00640 1 -- 172.21.15.83:0/2003688252 learned_addr learned my addr 172.21.15.83:0/2003688252 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:45.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60e400640 1 --2- 172.21.15.83:0/2003688252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6080ad180 0x7ff6080ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60d000640 1 --2- 172.21.15.83:0/2003688252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608144f20 0x7ff6081453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60da00640 1 -- 172.21.15.83:0/2003688252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6080ad180 msgr2=0x7ff6080ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60da00640 1 --2- 172.21.15.83:0/2003688252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6080ad180 0x7ff6080ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60da00640 1 -- 172.21.15.83:0/2003688252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608144f20 msgr2=0x7ff6081453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60da00640 1 --2- 172.21.15.83:0/2003688252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608144f20 0x7ff6081453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60da00640 1 -- 172.21.15.83:0/2003688252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff608147cc0 con 0x7ff5f0003a10 2024-04-25T00:13:45.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60da00640 1 --2- 172.21.15.83:0/2003688252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5f0003a10 0x7ff60806d750 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc09a5c0 tx=0x7ff5f0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:45.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff605600640 1 -- 172.21.15.83:0/2003688252 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc007c70 con 0x7ff5f0003a10 2024-04-25T00:13:45.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff605600640 1 -- 172.21.15.83:0/2003688252 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5fc0a8e20 con 0x7ff5f0003a10 2024-04-25T00:13:45.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff605600640 1 -- 172.21.15.83:0/2003688252 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc001680 con 0x7ff5f0003a10 2024-04-25T00:13:45.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60ee00640 1 -- 172.21.15.83:0/2003688252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff608147f60 con 0x7ff5f0003a10 2024-04-25T00:13:45.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.344+0000 7ff60ee00640 1 -- 172.21.15.83:0/2003688252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff608148520 con 0x7ff5f0003a10 2024-04-25T00:13:45.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.348+0000 7ff60ee00640 1 -- 172.21.15.83:0/2003688252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60804f510 con 0x7ff5f0003a10 2024-04-25T00:13:45.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.348+0000 7ff605600640 1 -- 172.21.15.83:0/2003688252 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff5fc0049f0 con 0x7ff5f0003a10 2024-04-25T00:13:45.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.348+0000 7ff605600640 1 --2- 172.21.15.83:0/2003688252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5e80678f0 0x7ff5e8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.348+0000 7ff605600640 1 -- 172.21.15.83:0/2003688252 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5335+0+0 (secure 0 0 0) 0x7ff5fc11f340 con 0x7ff5f0003a10 2024-04-25T00:13:45.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.348+0000 7ff60d000640 1 --2- 172.21.15.83:0/2003688252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5e80678f0 0x7ff5e8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.348+0000 7ff60d000640 1 --2- 172.21.15.83:0/2003688252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5e80678f0 0x7ff5e8069db0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff6081fab40 tx=0x7ff5f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:45.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.352+0000 7ff605600640 1 -- 172.21.15.83:0/2003688252 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff5fc0c1030 con 0x7ff5f0003a10 2024-04-25T00:13:45.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.508+0000 7ff60ee00640 1 -- 172.21.15.83:0/2003688252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff608070d10 con 0x7ff5f0003a10 2024-04-25T00:13:45.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.508+0000 7ff605600640 1 -- 172.21.15.83:0/2003688252 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) ==== 93+0+10 (secure 0 0 0) 0x7ff5fc018b00 con 0x7ff5f0003a10 2024-04-25T00:13:45.512 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:13:45.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.508+0000 7ff60ee00640 1 -- 172.21.15.83:0/2003688252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5e80678f0 msgr2=0x7ff5e8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.508+0000 7ff60ee00640 1 --2- 172.21.15.83:0/2003688252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5e80678f0 0x7ff5e8069db0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff6081fab40 tx=0x7ff5f4008040 comp rx=0 tx=0).stop 2024-04-25T00:13:45.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.508+0000 7ff60ee00640 1 -- 172.21.15.83:0/2003688252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5f0003a10 msgr2=0x7ff60806d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.508+0000 7ff60ee00640 1 --2- 172.21.15.83:0/2003688252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5f0003a10 0x7ff60806d750 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc09a5c0 tx=0x7ff5f0005840 comp rx=0 tx=0).stop 2024-04-25T00:13:45.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.512+0000 7ff60ee00640 1 -- 172.21.15.83:0/2003688252 shutdown_connections 2024-04-25T00:13:45.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.512+0000 7ff60ee00640 1 --2- 172.21.15.83:0/2003688252 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5e80678f0 0x7ff5e8069db0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.512+0000 7ff60ee00640 1 --2- 172.21.15.83:0/2003688252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6080ad180 0x7ff6080ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.512+0000 7ff60ee00640 1 --2- 172.21.15.83:0/2003688252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff608144f20 0x7ff6081453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.512+0000 7ff60ee00640 1 --2- 172.21.15.83:0/2003688252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5f0003a10 0x7ff60806d750 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.512+0000 7ff60ee00640 1 -- 172.21.15.83:0/2003688252 >> 172.21.15.83:0/2003688252 conn(0x7ff6080cf980 msgr2=0x7ff6080ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:45.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.512+0000 7ff60ee00640 1 -- 172.21.15.83:0/2003688252 shutdown_connections 2024-04-25T00:13:45.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.512+0000 7ff60ee00640 1 -- 172.21.15.83:0/2003688252 wait complete. 2024-04-25T00:13:45.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close' 2024-04-25T00:13:45.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.664+0000 7f310ee00640 1 Processor -- start 2024-04-25T00:13:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.664+0000 7f310ee00640 1 -- start start 2024-04-25T00:13:45.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f310ee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f31081599f0 0x7f31080cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f310ee00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f310815c340 0x7f310815c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f310ee00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31080ccc60 con 0x7f310815ccc0 2024-04-25T00:13:45.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f310ee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3108159dd0 con 0x7f31081599f0 2024-04-25T00:13:45.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f310ee00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f310806dba0 con 0x7f310815c340 2024-04-25T00:13:45.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f310ce00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f310815ccc0 0x7f310815f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41618/0 (socket says 172.21.15.83:41618) 2024-04-25T00:13:45.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f310ce00640 1 -- 172.21.15.83:0/2111309372 learned_addr learned my addr 172.21.15.83:0/2111309372 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:45.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f30ff400640 1 --2- 172.21.15.83:0/2111309372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f310815c340 0x7f310815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f3107e00640 1 --2- 172.21.15.83:0/2111309372 >> v2:172.21.15.86:3300/0 conn(0x7f31081599f0 0x7f31080cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f3107400640 1 -- 172.21.15.83:0/2111309372 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 515948533 0 0) 0x7f31000014a0 con 0x7f310815ccc0 2024-04-25T00:13:45.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f3107400640 1 -- 172.21.15.83:0/2111309372 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30ec003530 con 0x7f310815ccc0 2024-04-25T00:13:45.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f30ff400640 1 -- 172.21.15.83:0/2111309372 >> v2:172.21.15.86:3300/0 conn(0x7f31081599f0 msgr2=0x7f31080cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:45.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f30ff400640 1 --2- 172.21.15.83:0/2111309372 >> v2:172.21.15.86:3300/0 conn(0x7f31081599f0 0x7f31080cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f30ff400640 1 -- 172.21.15.83:0/2111309372 >> v1:172.21.15.83:6789/0 conn(0x7f310815ccc0 legacy=0x7f310815f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f3107400640 1 -- 172.21.15.83:0/2111309372 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 85326375 0 0) 0x7f30ec003530 con 0x7f310815ccc0 2024-04-25T00:13:45.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f30ff400640 1 -- 172.21.15.83:0/2111309372 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310806de80 con 0x7f310815c340 2024-04-25T00:13:45.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.668+0000 7f3107e00640 1 --2- 172.21.15.83:0/2111309372 >> v2:172.21.15.86:3300/0 conn(0x7f31081599f0 0x7f31080cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:45.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f30ff400640 1 --2- 172.21.15.83:0/2111309372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f310815c340 0x7f310815c780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f30f00097f0 tx=0x7f30f001c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa823b61ee0c131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:45.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f3107400640 1 -- 172.21.15.83:0/2111309372 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f00b0070 con 0x7f310815c340 2024-04-25T00:13:45.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f3107400640 1 -- 172.21.15.83:0/2111309372 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f00ab400 con 0x7f310815c340 2024-04-25T00:13:45.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f3107400640 1 -- 172.21.15.83:0/2111309372 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100001bd0 con 0x7f310815c340 2024-04-25T00:13:45.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 -- 172.21.15.83:0/2111309372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f310815c340 msgr2=0x7f310815c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 --2- 172.21.15.83:0/2111309372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f310815c340 0x7f310815c780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f30f00097f0 tx=0x7f30f001c330 comp rx=0 tx=0).stop 2024-04-25T00:13:45.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 -- 172.21.15.83:0/2111309372 shutdown_connections 2024-04-25T00:13:45.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 --2- 172.21.15.83:0/2111309372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f310815c340 0x7f310815c780 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 --2- 172.21.15.83:0/2111309372 >> v2:172.21.15.86:3300/0 conn(0x7f31081599f0 0x7f31080cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 -- 172.21.15.83:0/2111309372 >> 172.21.15.83:0/2111309372 conn(0x7f31080b1830 msgr2=0x7f31080b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:45.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 -- 172.21.15.83:0/2111309372 shutdown_connections 2024-04-25T00:13:45.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 -- 172.21.15.83:0/2111309372 wait complete. 2024-04-25T00:13:45.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 Processor -- start 2024-04-25T00:13:45.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 -- start start 2024-04-25T00:13:45.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31081599f0 0x7f3108160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310815c340 0x7f3108165e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f310815ccc0 0x7f310813d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31080cf870 con 0x7f31081599f0 2024-04-25T00:13:45.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31080cf6f0 con 0x7f310815c340 2024-04-25T00:13:45.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f310ee00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3108168cd0 con 0x7f310815ccc0 2024-04-25T00:13:45.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f3107e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31081599f0 0x7f3108160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f3107e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31081599f0 0x7f3108160b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43978/0 (socket says 172.21.15.83:43978) 2024-04-25T00:13:45.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.672+0000 7f3107e00640 1 -- 172.21.15.83:0/3709563355 learned_addr learned my addr 172.21.15.83:0/3709563355 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:45.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f30ff400640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310815c340 0x7f3108165e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f310ce00640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f310815ccc0 0x7f310813d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f3107e00640 1 -- 172.21.15.83:0/3709563355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f310815ccc0 msgr2=0x7f310813d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f3107e00640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f310815ccc0 0x7f310813d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f3107e00640 1 -- 172.21.15.83:0/3709563355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310815c340 msgr2=0x7f3108165e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:45.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f3107e00640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310815c340 0x7f3108165e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:45.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f3107e00640 1 -- 172.21.15.83:0/3709563355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31081665b0 con 0x7f31081599f0 2024-04-25T00:13:45.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f30ff400640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310815c340 0x7f3108165e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:45.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f310ce00640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f310815ccc0 0x7f310813d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:45.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f3107e00640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31081599f0 0x7f3108160b80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f30f800ebe0 tx=0x7f30f800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:45.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f3104c00640 1 -- 172.21.15.83:0/3709563355 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8017070 con 0x7f31081599f0 2024-04-25T00:13:45.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f3104c00640 1 -- 172.21.15.83:0/3709563355 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f800cc30 con 0x7f31081599f0 2024-04-25T00:13:45.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f310ee00640 1 -- 172.21.15.83:0/3709563355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3108166890 con 0x7f31081599f0 2024-04-25T00:13:45.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f3104c00640 1 -- 172.21.15.83:0/3709563355 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8013470 con 0x7f31081599f0 2024-04-25T00:13:45.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f310ee00640 1 -- 172.21.15.83:0/3709563355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31080b3120 con 0x7f31081599f0 2024-04-25T00:13:45.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f310ee00640 1 -- 172.21.15.83:0/3709563355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31080b2c30 con 0x7f31081599f0 2024-04-25T00:13:45.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.676+0000 7f3104c00640 1 -- 172.21.15.83:0/3709563355 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f30f801a070 con 0x7f31081599f0 2024-04-25T00:13:45.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.680+0000 7f3104c00640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f30e40678c0 0x7f30e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:45.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.680+0000 7f3104c00640 1 -- 172.21.15.83:0/3709563355 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5335+0+0 (secure 0 0 0) 0x7f30f8085f70 con 0x7f31081599f0 2024-04-25T00:13:45.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.680+0000 7f30ff400640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f30e40678c0 0x7f30e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:45.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.680+0000 7f30ff400640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f30e40678c0 0x7f30e4069d80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f30f00ad000 tx=0x7f30f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:45.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.684+0000 7f3104c00640 1 -- 172.21.15.83:0/3709563355 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f30f8056ba0 con 0x7f31081599f0 2024-04-25T00:13:45.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:45.892+0000 7f310ee00640 1 -- 172.21.15.83:0/3709563355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]} v 0) -- 0x7f31080737f0 con 0x7f31081599f0 2024-04-25T00:13:46.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:46.852+0000 7f3104c00640 1 -- 172.21.15.83:0/3709563355 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]}]=0 v181) ==== 146+0+0 (secure 0 0 0) 0x7f30f80564a0 con 0x7f31081599f0 2024-04-25T00:13:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:46.852+0000 7f310ee00640 1 -- 172.21.15.83:0/3709563355 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f30e40678c0 msgr2=0x7f30e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:46.852+0000 7f310ee00640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f30e40678c0 0x7f30e4069d80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f30f00ad000 tx=0x7f30f0008040 comp rx=0 tx=0).stop 2024-04-25T00:13:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:46.852+0000 7f310ee00640 1 -- 172.21.15.83:0/3709563355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31081599f0 msgr2=0x7f3108160b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:46.852+0000 7f310ee00640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31081599f0 0x7f3108160b80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f30f800ebe0 tx=0x7f30f800c520 comp rx=0 tx=0).stop 2024-04-25T00:13:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:46.852+0000 7f310ee00640 1 -- 172.21.15.83:0/3709563355 shutdown_connections 2024-04-25T00:13:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:46.852+0000 7f310ee00640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f30e40678c0 0x7f30e4069d80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:46.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:46.852+0000 7f310ee00640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f310815ccc0 0x7f310813d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:46.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:46.852+0000 7f310ee00640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310815c340 0x7f3108165e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:46.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:46.852+0000 7f310ee00640 1 --2- 172.21.15.83:0/3709563355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31081599f0 0x7f3108160b80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:46.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:46.852+0000 7f310ee00640 1 -- 172.21.15.83:0/3709563355 >> 172.21.15.83:0/3709563355 conn(0x7f31080b1830 msgr2=0x7f3108072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:46.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:46.852+0000 7f310ee00640 1 -- 172.21.15.83:0/3709563355 shutdown_connections 2024-04-25T00:13:46.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:46.852+0000 7f310ee00640 1 -- 172.21.15.83:0/3709563355 wait complete. 2024-04-25T00:13:46.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:13:47.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.020+0000 7fb5f0e00640 1 Processor -- start 2024-04-25T00:13:47.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.020+0000 7fb5f0e00640 1 -- start start 2024-04-25T00:13:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5f0e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5ec159810 0x7fb5ec0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5f0e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb5ec06e950 0x7fb5ec15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5f0e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5ec0ce000 con 0x7fb5ec15fe50 2024-04-25T00:13:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5f0e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb5ec06e210 con 0x7fb5ec06e950 2024-04-25T00:13:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5f0e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ec06da00 con 0x7fb5ec159810 2024-04-25T00:13:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5ea800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb5ec15fe50 0x7fb5ec15c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41632/0 (socket says 172.21.15.83:41632) 2024-04-25T00:13:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5ea800640 1 -- 172.21.15.83:0/2085084779 learned_addr learned my addr 172.21.15.83:0/2085084779 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5e9e00640 1 --2- 172.21.15.83:0/2085084779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5ec159810 0x7fb5ec0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5e1400640 1 --2- 172.21.15.83:0/2085084779 >> v2:172.21.15.86:3300/0 conn(0x7fb5ec06e950 0x7fb5ec15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5e9400640 1 -- 172.21.15.83:0/2085084779 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1894344567 0 0) 0x7fb5ec0ce000 con 0x7fb5ec15fe50 2024-04-25T00:13:47.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5e9400640 1 -- 172.21.15.83:0/2085084779 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5d0003530 con 0x7fb5ec15fe50 2024-04-25T00:13:47.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5e1400640 1 -- 172.21.15.83:0/2085084779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5ec159810 msgr2=0x7fb5ec0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:47.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5e1400640 1 --2- 172.21.15.83:0/2085084779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5ec159810 0x7fb5ec0cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5e9400640 1 -- 172.21.15.83:0/2085084779 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3107822598 0 0) 0x7fb5d0003530 con 0x7fb5ec15fe50 2024-04-25T00:13:47.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5e1400640 1 -- 172.21.15.83:0/2085084779 >> v1:172.21.15.83:6789/0 conn(0x7fb5ec15fe50 legacy=0x7fb5ec15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5e1400640 1 -- 172.21.15.83:0/2085084779 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec06eec0 con 0x7fb5ec06e950 2024-04-25T00:13:47.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.024+0000 7fb5e9e00640 1 --2- 172.21.15.83:0/2085084779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5ec159810 0x7fb5ec0cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.028+0000 7fb5e1400640 1 --2- 172.21.15.83:0/2085084779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ec06e950 0x7fb5ec15f910 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009d90 tx=0x7fb5d401c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8edc724d9a3910fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.028+0000 7fb5e9400640 1 -- 172.21.15.83:0/2085084779 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d40b2070 con 0x7fb5ec06e950 2024-04-25T00:13:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.028+0000 7fb5e9400640 1 -- 172.21.15.83:0/2085084779 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d40a5510 con 0x7fb5ec06e950 2024-04-25T00:13:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.028+0000 7fb5e9400640 1 -- 172.21.15.83:0/2085084779 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4001680 con 0x7fb5ec06e950 2024-04-25T00:13:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.028+0000 7fb5f0e00640 1 -- 172.21.15.83:0/2085084779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ec06e950 msgr2=0x7fb5ec15f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.028+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/2085084779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ec06e950 0x7fb5ec15f910 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009d90 tx=0x7fb5d401c800 comp rx=0 tx=0).stop 2024-04-25T00:13:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.028+0000 7fb5f0e00640 1 -- 172.21.15.83:0/2085084779 shutdown_connections 2024-04-25T00:13:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.028+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/2085084779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ec06e950 0x7fb5ec15f910 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.028+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/2085084779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5ec159810 0x7fb5ec0cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.028+0000 7fb5f0e00640 1 -- 172.21.15.83:0/2085084779 >> 172.21.15.83:0/2085084779 conn(0x7fb5ec0b1830 msgr2=0x7fb5ec0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.028+0000 7fb5f0e00640 1 -- 172.21.15.83:0/2085084779 shutdown_connections 2024-04-25T00:13:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.028+0000 7fb5f0e00640 1 -- 172.21.15.83:0/2085084779 wait complete. 2024-04-25T00:13:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.028+0000 7fb5f0e00640 1 Processor -- start 2024-04-25T00:13:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.028+0000 7fb5f0e00640 1 -- start start 2024-04-25T00:13:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5f0e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5ec06e950 0x7fb5ec154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5f0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ec159810 0x7fb5ec0b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5f0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5ec15fe50 0x7fb5ec142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5f0e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec159d70 con 0x7fb5ec15fe50 2024-04-25T00:13:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5f0e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec159bf0 con 0x7fb5ec159810 2024-04-25T00:13:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5f0e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ec070da0 con 0x7fb5ec06e950 2024-04-25T00:13:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5ea800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5ec15fe50 0x7fb5ec142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5ea800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5ec15fe50 0x7fb5ec142600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43984/0 (socket says 172.21.15.83:43984) 2024-04-25T00:13:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5ea800640 1 -- 172.21.15.83:0/3713093734 learned_addr learned my addr 172.21.15.83:0/3713093734 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5e1400640 1 --2- 172.21.15.83:0/3713093734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ec159810 0x7fb5ec0b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5e9e00640 1 --2- 172.21.15.83:0/3713093734 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5ec06e950 0x7fb5ec154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5ea800640 1 -- 172.21.15.83:0/3713093734 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5ec06e950 msgr2=0x7fb5ec154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5ea800640 1 --2- 172.21.15.83:0/3713093734 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5ec06e950 0x7fb5ec154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5ea800640 1 -- 172.21.15.83:0/3713093734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ec159810 msgr2=0x7fb5ec0b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5ea800640 1 --2- 172.21.15.83:0/3713093734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ec159810 0x7fb5ec0b09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5ea800640 1 -- 172.21.15.83:0/3713093734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec154b10 con 0x7fb5ec15fe50 2024-04-25T00:13:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5ea800640 1 --2- 172.21.15.83:0/3713093734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5ec15fe50 0x7fb5ec142600 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400c750 tx=0x7fb5e400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5e2a00640 1 -- 172.21.15.83:0/3713093734 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4005ae0 con 0x7fb5ec15fe50 2024-04-25T00:13:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5e2a00640 1 -- 172.21.15.83:0/3713093734 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e4005ca0 con 0x7fb5ec15fe50 2024-04-25T00:13:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5f0e00640 1 -- 172.21.15.83:0/3713093734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ec154df0 con 0x7fb5ec15fe50 2024-04-25T00:13:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5e2a00640 1 -- 172.21.15.83:0/3713093734 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e40072b0 con 0x7fb5ec15fe50 2024-04-25T00:13:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5f0e00640 1 -- 172.21.15.83:0/3713093734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ec1553b0 con 0x7fb5ec15fe50 2024-04-25T00:13:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.032+0000 7fb5f0e00640 1 -- 172.21.15.83:0/3713093734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5ec04f510 con 0x7fb5ec15fe50 2024-04-25T00:13:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.036+0000 7fb5e2a00640 1 -- 172.21.15.83:0/3713093734 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb5e4007450 con 0x7fb5ec15fe50 2024-04-25T00:13:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.036+0000 7fb5e2a00640 1 --2- 172.21.15.83:0/3713093734 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb5c80678c0 0x7fb5c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.036+0000 7fb5e2a00640 1 -- 172.21.15.83:0/3713093734 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5335+0+0 (secure 0 0 0) 0x7fb5e4085860 con 0x7fb5ec15fe50 2024-04-25T00:13:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.036+0000 7fb5e9e00640 1 --2- 172.21.15.83:0/3713093734 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb5c80678c0 0x7fb5c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.036+0000 7fb5e9e00640 1 --2- 172.21.15.83:0/3713093734 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb5c80678c0 0x7fb5c8069d80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009580 tx=0x7fb5dc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:47.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.040+0000 7fb5e2a00640 1 -- 172.21.15.83:0/3713093734 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb5e4055ec0 con 0x7fb5ec15fe50 2024-04-25T00:13:47.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.192+0000 7fb5f0e00640 1 -- 172.21.15.83:0/3713093734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb5ec06d450 con 0x7fb5ec15fe50 2024-04-25T00:13:47.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.196+0000 7fb5e2a00640 1 -- 172.21.15.83:0/3713093734 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fb5e405a1c0 con 0x7fb5ec15fe50 2024-04-25T00:13:47.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.200+0000 7fb5f0e00640 1 -- 172.21.15.83:0/3713093734 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb5c80678c0 msgr2=0x7fb5c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.200+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/3713093734 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb5c80678c0 0x7fb5c8069d80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009580 tx=0x7fb5dc009190 comp rx=0 tx=0).stop 2024-04-25T00:13:47.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.200+0000 7fb5f0e00640 1 -- 172.21.15.83:0/3713093734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5ec15fe50 msgr2=0x7fb5ec142600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.200+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/3713093734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5ec15fe50 0x7fb5ec142600 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400c750 tx=0x7fb5e400cc10 comp rx=0 tx=0).stop 2024-04-25T00:13:47.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.200+0000 7fb5f0e00640 1 -- 172.21.15.83:0/3713093734 shutdown_connections 2024-04-25T00:13:47.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.200+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/3713093734 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb5c80678c0 0x7fb5c8069d80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.200+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/3713093734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5ec15fe50 0x7fb5ec142600 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.200+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/3713093734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ec159810 0x7fb5ec0b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.200+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/3713093734 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5ec06e950 0x7fb5ec154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.200+0000 7fb5f0e00640 1 -- 172.21.15.83:0/3713093734 >> 172.21.15.83:0/3713093734 conn(0x7fb5ec0b1830 msgr2=0x7fb5ec15d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:47.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.200+0000 7fb5f0e00640 1 -- 172.21.15.83:0/3713093734 shutdown_connections 2024-04-25T00:13:47.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.200+0000 7fb5f0e00640 1 -- 172.21.15.83:0/3713093734 wait complete. 2024-04-25T00:13:47.221 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:13:47.222 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:13:47.312 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:13:47.412 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:13:47.412 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:13:47.412 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:47.421 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:13:47.421 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:13:47.421 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:47.430 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:13:47.431 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:13:47.431 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:47.472 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:13:47.473 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:13:47.473 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:47.485 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:13:47.485 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:13:47.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:47.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.644+0000 7f42d6a00640 1 Processor -- start 2024-04-25T00:13:47.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.644+0000 7f42d6a00640 1 -- start start 2024-04-25T00:13:47.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42d6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f42d00b4360 0x7f42d00b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42d6a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42d00b2b30 0x7f42d00b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42d6a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d006ef80 con 0x7f42d01553e0 2024-04-25T00:13:47.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42d6a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f42d015b510 con 0x7f42d00b4360 2024-04-25T00:13:47.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42d6a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f42d015b7f0 con 0x7f42d00b2b30 2024-04-25T00:13:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42d4a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42d00b2b30 0x7f42d00b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42d4a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42d00b2b30 0x7f42d00b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:52570/0 (socket says 172.21.15.83:52570) 2024-04-25T00:13:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42d4a00640 1 -- 172.21.15.83:0/3876518393 learned_addr learned my addr 172.21.15.83:0/3876518393 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42cfe00640 1 --1- 172.21.15.83:0/3876518393 >> v1:172.21.15.86:6789/0 conn(0x7f42d01553e0 0x7f42d006e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:55348/0 (socket says 172.21.15.83:55348) 2024-04-25T00:13:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42cf400640 1 --2- 172.21.15.83:0/3876518393 >> v2:172.21.15.86:3300/0 conn(0x7f42d00b4360 0x7f42d00b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42cea00640 1 -- 172.21.15.83:0/3876518393 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2749503955 0 0) 0x7f42d006ef80 con 0x7f42d01553e0 2024-04-25T00:13:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42cea00640 1 -- 172.21.15.83:0/3876518393 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42b4003530 con 0x7f42d01553e0 2024-04-25T00:13:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42d4a00640 1 -- 172.21.15.83:0/3876518393 >> v2:172.21.15.86:3300/0 conn(0x7f42d00b4360 msgr2=0x7f42d00b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42d4a00640 1 --2- 172.21.15.83:0/3876518393 >> v2:172.21.15.86:3300/0 conn(0x7f42d00b4360 0x7f42d00b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42d4a00640 1 -- 172.21.15.83:0/3876518393 >> v1:172.21.15.86:6789/0 conn(0x7f42d01553e0 legacy=0x7f42d006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42d4a00640 1 -- 172.21.15.83:0/3876518393 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d015bad0 con 0x7f42d00b2b30 2024-04-25T00:13:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.648+0000 7f42cf400640 1 --2- 172.21.15.83:0/3876518393 >> v2:172.21.15.86:3300/0 conn(0x7f42d00b4360 0x7f42d00b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:47.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.652+0000 7f42d4a00640 1 --2- 172.21.15.83:0/3876518393 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42d00b2b30 0x7f42d00b2fc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009e50 tx=0x7f42c801c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8cef01850b1c1ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:47.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.652+0000 7f42cea00640 1 -- 172.21.15.83:0/3876518393 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c80a5620 con 0x7f42d00b2b30 2024-04-25T00:13:47.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.652+0000 7f42cea00640 1 -- 172.21.15.83:0/3876518393 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c80a57c0 con 0x7f42d00b2b30 2024-04-25T00:13:47.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.652+0000 7f42cea00640 1 -- 172.21.15.83:0/3876518393 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c80a5b90 con 0x7f42d00b2b30 2024-04-25T00:13:47.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.652+0000 7f42d6a00640 1 -- 172.21.15.83:0/3876518393 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42d00b2b30 msgr2=0x7f42d00b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.652+0000 7f42d6a00640 1 --2- 172.21.15.83:0/3876518393 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42d00b2b30 0x7f42d00b2fc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009e50 tx=0x7f42c801c9a0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.652+0000 7f42d6a00640 1 -- 172.21.15.83:0/3876518393 shutdown_connections 2024-04-25T00:13:47.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.652+0000 7f42d6a00640 1 --2- 172.21.15.83:0/3876518393 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42d00b2b30 0x7f42d00b2fc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.652+0000 7f42d6a00640 1 --2- 172.21.15.83:0/3876518393 >> v2:172.21.15.86:3300/0 conn(0x7f42d00b4360 0x7f42d00b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.652+0000 7f42d6a00640 1 -- 172.21.15.83:0/3876518393 >> 172.21.15.83:0/3876518393 conn(0x7f42d00b1830 msgr2=0x7f42d00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:47.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.652+0000 7f42d6a00640 1 -- 172.21.15.83:0/3876518393 shutdown_connections 2024-04-25T00:13:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.652+0000 7f42d6a00640 1 -- 172.21.15.83:0/3876518393 wait complete. 2024-04-25T00:13:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.652+0000 7f42d6a00640 1 Processor -- start 2024-04-25T00:13:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.652+0000 7f42d6a00640 1 -- start start 2024-04-25T00:13:47.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42d6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d00b2b30 0x7f42d006d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d00b4360 0x7f42d015aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42d6a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42d01553e0 0x7f42d01f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42d6a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42d006ebd0 con 0x7f42d00b2b30 2024-04-25T00:13:47.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42d6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42d006d3b0 con 0x7f42d00b4360 2024-04-25T00:13:47.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42d6a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f42d00ce9c0 con 0x7f42d01553e0 2024-04-25T00:13:47.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d00b2b30 0x7f42d006d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d00b2b30 0x7f42d006d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43998/0 (socket says 172.21.15.83:43998) 2024-04-25T00:13:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42cfe00640 1 -- 172.21.15.83:0/2251637012 learned_addr learned my addr 172.21.15.83:0/2251637012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42d4a00640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42d01553e0 0x7f42d01f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42cf400640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d00b4360 0x7f42d015aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42cfe00640 1 -- 172.21.15.83:0/2251637012 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42d01553e0 msgr2=0x7f42d01f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42cfe00640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42d01553e0 0x7f42d01f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42cfe00640 1 -- 172.21.15.83:0/2251637012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d00b4360 msgr2=0x7f42d015aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42cfe00640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d00b4360 0x7f42d015aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42cfe00640 1 -- 172.21.15.83:0/2251637012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d00ceca0 con 0x7f42d00b2b30 2024-04-25T00:13:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42cf400640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d00b4360 0x7f42d015aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42cfe00640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d00b2b30 0x7f42d006d910 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f42c000c390 tx=0x7f42c000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42c7e00640 1 -- 172.21.15.83:0/2251637012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c00074e0 con 0x7f42d00b2b30 2024-04-25T00:13:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42c7e00640 1 -- 172.21.15.83:0/2251637012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c00047f0 con 0x7f42d00b2b30 2024-04-25T00:13:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42d6a00640 1 -- 172.21.15.83:0/2251637012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d015a240 con 0x7f42d00b2b30 2024-04-25T00:13:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42c7e00640 1 -- 172.21.15.83:0/2251637012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c00076a0 con 0x7f42d00b2b30 2024-04-25T00:13:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42d6a00640 1 -- 172.21.15.83:0/2251637012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d00ce650 con 0x7f42d00b2b30 2024-04-25T00:13:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42d6a00640 1 -- 172.21.15.83:0/2251637012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d00737f0 con 0x7f42d00b2b30 2024-04-25T00:13:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.656+0000 7f42c7e00640 1 -- 172.21.15.83:0/2251637012 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f42c001a020 con 0x7f42d00b2b30 2024-04-25T00:13:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.660+0000 7f42c7e00640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f42b00678c0 0x7f42b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.660+0000 7f42c7e00640 1 -- 172.21.15.83:0/2251637012 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5335+0+0 (secure 0 0 0) 0x7f42c0085360 con 0x7f42d00b2b30 2024-04-25T00:13:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.660+0000 7f42cf400640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f42b00678c0 0x7f42b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.660+0000 7f42cf400640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f42b00678c0 0x7f42b0069d80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f42d0150e00 tx=0x7f42bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.664+0000 7f42c7e00640 1 -- 172.21.15.83:0/2251637012 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f42c0055f80 con 0x7f42d00b2b30 2024-04-25T00:13:47.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.848+0000 7f42d6a00640 1 -- 172.21.15.83:0/2251637012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42d004f5f0 con 0x7f42d00b2b30 2024-04-25T00:13:47.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.848+0000 7f42c7e00640 1 -- 172.21.15.83:0/2251637012 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+3869 (secure 0 0 0) 0x7f42c0083e20 con 0x7f42d00b2b30 2024-04-25T00:13:47.855 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:47.856 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5486,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3667389928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":3667389928},{"type":"v1","addr":"172.21.15.86:6835","nonce":3667389928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5923,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2174516900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2174516900},{"type":"v1","addr":"172.21.15.83:6837","nonce":2174516900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2877383406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2877383406},{"type":"v1","addr":"172.21.15.83:6836","nonce":2877383406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2232770749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2232770749},{"type":"v1","addr":"172.21.15.86:6834","nonce":2232770749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[]} 2024-04-25T00:13:47.858 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 37 2024-04-25T00:13:47.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.852+0000 7f42d6a00640 1 -- 172.21.15.83:0/2251637012 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f42b00678c0 msgr2=0x7f42b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.852+0000 7f42d6a00640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f42b00678c0 0x7f42b0069d80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f42d0150e00 tx=0x7f42bc008040 comp rx=0 tx=0).stop 2024-04-25T00:13:47.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.852+0000 7f42d6a00640 1 -- 172.21.15.83:0/2251637012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d00b2b30 msgr2=0x7f42d006d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.852+0000 7f42d6a00640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d00b2b30 0x7f42d006d910 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f42c000c390 tx=0x7f42c000c850 comp rx=0 tx=0).stop 2024-04-25T00:13:47.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.852+0000 7f42d6a00640 1 -- 172.21.15.83:0/2251637012 shutdown_connections 2024-04-25T00:13:47.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.852+0000 7f42d6a00640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f42b00678c0 0x7f42b0069d80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.852+0000 7f42d6a00640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42d01553e0 0x7f42d01f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.852+0000 7f42d6a00640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d00b4360 0x7f42d015aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.852+0000 7f42d6a00640 1 --2- 172.21.15.83:0/2251637012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d00b2b30 0x7f42d006d910 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.852+0000 7f42d6a00640 1 -- 172.21.15.83:0/2251637012 >> 172.21.15.83:0/2251637012 conn(0x7f42d00b1830 msgr2=0x7f42d00cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:47.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.852+0000 7f42d6a00640 1 -- 172.21.15.83:0/2251637012 shutdown_connections 2024-04-25T00:13:47.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:47.852+0000 7f42d6a00640 1 -- 172.21.15.83:0/2251637012 wait complete. 2024-04-25T00:13:47.877 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:13:47.877 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:13:47.877 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:13:47.877 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:13:47.878 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:13:47.878 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:13:47.878 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:13:47.878 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:13:47.878 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:13:47.878 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:13:47.878 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:13:47.879 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:13:47.879 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:13:47.879 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:13:47.879 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:13:47.879 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:13:47.891 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:13:47.891 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:13:47.892 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:13:47.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:13:47.894 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:13:47.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:13:47.895 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:13:47.895 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:13:47.936 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:13:47.936 DEBUG:teuthology.parallel:result is None 2024-04-25T00:13:47.937 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:13:47.938 DEBUG:teuthology.parallel:result is None 2024-04-25T00:13:47.941 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:13:47.941 DEBUG:teuthology.parallel:result is None 2024-04-25T00:13:47.942 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:13:47.942 DEBUG:teuthology.parallel:result is None 2024-04-25T00:13:47.943 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:13:48.023 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-25T00:13:48.023 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-25T00:13:48.031 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2024-04-25T00:13:48.031 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2024-04-25T00:13:48.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c8200640 1 Processor -- start 2024-04-25T00:13:48.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c8200640 1 -- start start 2024-04-25T00:13:48.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c8200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22c015e160 0x7f22c00c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:48.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c8200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f22c015f420 0x7f22c00c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:48.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c8200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22c00cb8b0 con 0x7f22c00ceff0 2024-04-25T00:13:48.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c8200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f22c00ca490 con 0x7f22c015e160 2024-04-25T00:13:48.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c8200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f22c015e940 con 0x7f22c015f420 2024-04-25T00:13:48.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c5800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22c015e160 0x7f22c00c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:48.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c6200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f22c00ceff0 0x7f22c0161c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41640/0 (socket says 172.21.15.83:41640) 2024-04-25T00:13:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c6200640 1 -- 172.21.15.83:0/838094943 learned_addr learned my addr 172.21.15.83:0/838094943 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c4e00640 1 --2- 172.21.15.83:0/838094943 >> v2:172.21.15.86:3300/0 conn(0x7f22c015f420 0x7f22c00c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c5800640 1 -- 172.21.15.83:0/838094943 >> v2:172.21.15.86:3300/0 conn(0x7f22c015f420 msgr2=0x7f22c00c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c5800640 1 --2- 172.21.15.83:0/838094943 >> v2:172.21.15.86:3300/0 conn(0x7f22c015f420 0x7f22c00c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c5800640 1 -- 172.21.15.83:0/838094943 >> v1:172.21.15.83:6789/0 conn(0x7f22c00ceff0 legacy=0x7f22c0161c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c5800640 1 -- 172.21.15.83:0/838094943 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c015ec20 con 0x7f22c015e160 2024-04-25T00:13:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22c4e00640 1 --2- 172.21.15.83:0/838094943 >> v2:172.21.15.86:3300/0 conn(0x7f22c015f420 0x7f22c00c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.084+0000 7f22bfe00640 1 -- 172.21.15.83:0/838094943 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2638091619 0 0) 0x7f22c00cb8b0 con 0x7f22c00ceff0 2024-04-25T00:13:48.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.088+0000 7f22c5800640 1 --2- 172.21.15.83:0/838094943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c015e160 0x7f22c00c8db0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f22c00cf660 tx=0x7f22b401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4903586cd138665a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:48.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.088+0000 7f22bfe00640 1 -- 172.21.15.83:0/838094943 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b40a6710 con 0x7f22c015e160 2024-04-25T00:13:48.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.088+0000 7f22bfe00640 1 -- 172.21.15.83:0/838094943 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b40a68b0 con 0x7f22c015e160 2024-04-25T00:13:48.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.088+0000 7f22bfe00640 1 -- 172.21.15.83:0/838094943 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b40a6c80 con 0x7f22c015e160 2024-04-25T00:13:48.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.088+0000 7f22c8200640 1 -- 172.21.15.83:0/838094943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c015e160 msgr2=0x7f22c00c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:48.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.088+0000 7f22c8200640 1 --2- 172.21.15.83:0/838094943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c015e160 0x7f22c00c8db0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f22c00cf660 tx=0x7f22b401c810 comp rx=0 tx=0).stop 2024-04-25T00:13:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.088+0000 7f22c8200640 1 -- 172.21.15.83:0/838094943 shutdown_connections 2024-04-25T00:13:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.088+0000 7f22c8200640 1 --2- 172.21.15.83:0/838094943 >> v2:172.21.15.86:3300/0 conn(0x7f22c015f420 0x7f22c00c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.088+0000 7f22c8200640 1 --2- 172.21.15.83:0/838094943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c015e160 0x7f22c00c8db0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.088+0000 7f22c8200640 1 -- 172.21.15.83:0/838094943 >> 172.21.15.83:0/838094943 conn(0x7f22c00cfa60 msgr2=0x7f22c00cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.088+0000 7f22c8200640 1 -- 172.21.15.83:0/838094943 shutdown_connections 2024-04-25T00:13:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.088+0000 7f22c8200640 1 -- 172.21.15.83:0/838094943 wait complete. 2024-04-25T00:13:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c8200640 1 Processor -- start 2024-04-25T00:13:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c8200640 1 -- start start 2024-04-25T00:13:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c8200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c00ceff0 0x7f22c01573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c8200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22c015e160 0x7f22c0141360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c8200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c015f420 0x7f22c00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c8200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22c00c9da0 con 0x7f22c00ceff0 2024-04-25T00:13:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c8200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22c00c9c20 con 0x7f22c015f420 2024-04-25T00:13:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c8200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f22c016b010 con 0x7f22c015e160 2024-04-25T00:13:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c00ceff0 0x7f22c01573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c00ceff0 0x7f22c01573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44068/0 (socket says 172.21.15.83:44068) 2024-04-25T00:13:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c5800640 1 -- 172.21.15.83:0/1840540583 learned_addr learned my addr 172.21.15.83:0/1840540583 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c4e00640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22c015e160 0x7f22c0141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c6200640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c015f420 0x7f22c00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c5800640 1 -- 172.21.15.83:0/1840540583 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22c015e160 msgr2=0x7f22c0141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c5800640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22c015e160 0x7f22c0141360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c5800640 1 -- 172.21.15.83:0/1840540583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c015f420 msgr2=0x7f22c00b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c5800640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c015f420 0x7f22c00b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c5800640 1 -- 172.21.15.83:0/1840540583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c00b3d30 con 0x7f22c00ceff0 2024-04-25T00:13:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c6200640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c015f420 0x7f22c00b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c5800640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c00ceff0 0x7f22c01573c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f22b40b0040 tx=0x7f22b40195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22bd600640 1 -- 172.21.15.83:0/1840540583 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b40be070 con 0x7f22c00ceff0 2024-04-25T00:13:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22bd600640 1 -- 172.21.15.83:0/1840540583 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b40b8860 con 0x7f22c00ceff0 2024-04-25T00:13:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22bd600640 1 -- 172.21.15.83:0/1840540583 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b40b8c50 con 0x7f22c00ceff0 2024-04-25T00:13:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c8200640 1 -- 172.21.15.83:0/1840540583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c00b4010 con 0x7f22c00ceff0 2024-04-25T00:13:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.092+0000 7f22c8200640 1 -- 172.21.15.83:0/1840540583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c00b45d0 con 0x7f22c00ceff0 2024-04-25T00:13:48.100 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T00:13:48.100 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T00:13:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.096+0000 7f22c8200640 1 -- 172.21.15.83:0/1840540583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22c0068230 con 0x7f22c00ceff0 2024-04-25T00:13:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.096+0000 7f22bd600640 1 -- 172.21.15.83:0/1840540583 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f22b40c1070 con 0x7f22c00ceff0 2024-04-25T00:13:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.096+0000 7f22bd600640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22a00678c0 0x7f22a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.096+0000 7f22bd600640 1 -- 172.21.15.83:0/1840540583 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5335+0+0 (secure 0 0 0) 0x7f22b4120e60 con 0x7f22c00ceff0 2024-04-25T00:13:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.096+0000 7f22c4e00640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22a00678c0 0x7f22a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.096+0000 7f22c4e00640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22a00678c0 0x7f22a0069d80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f22c00718b0 tx=0x7f22ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:48.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.100+0000 7f22bd600640 1 -- 172.21.15.83:0/1840540583 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f22b40c8030 con 0x7f22c00ceff0 2024-04-25T00:13:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.268+0000 7f22c8200640 1 -- 172.21.15.83:0/1840540583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f22c00c9a00 con 0x7f22c00ceff0 2024-04-25T00:13:48.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.868+0000 7f22bd600640 1 -- 172.21.15.83:0/1840540583 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v65) ==== 97+0+0 (secure 0 0 0) 0x7f22b4016520 con 0x7f22c00ceff0 2024-04-25T00:13:48.873 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-25T00:13:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.872+0000 7f22c8200640 1 -- 172.21.15.83:0/1840540583 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22a00678c0 msgr2=0x7f22a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.872+0000 7f22c8200640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22a00678c0 0x7f22a0069d80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f22c00718b0 tx=0x7f22ac008040 comp rx=0 tx=0).stop 2024-04-25T00:13:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.872+0000 7f22c8200640 1 -- 172.21.15.83:0/1840540583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c00ceff0 msgr2=0x7f22c01573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.872+0000 7f22c8200640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c00ceff0 0x7f22c01573c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f22b40b0040 tx=0x7f22b40195b0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.872+0000 7f22c8200640 1 -- 172.21.15.83:0/1840540583 shutdown_connections 2024-04-25T00:13:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.872+0000 7f22c8200640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f22a00678c0 0x7f22a0069d80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.872+0000 7f22c8200640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22c015e160 0x7f22c0141360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.872+0000 7f22c8200640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c015f420 0x7f22c00b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.872+0000 7f22c8200640 1 --2- 172.21.15.83:0/1840540583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c00ceff0 0x7f22c01573c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.872+0000 7f22c8200640 1 -- 172.21.15.83:0/1840540583 >> 172.21.15.83:0/1840540583 conn(0x7f22c00cfa60 msgr2=0x7f22c0161c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:48.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.872+0000 7f22c8200640 1 -- 172.21.15.83:0/1840540583 shutdown_connections 2024-04-25T00:13:48.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:48.872+0000 7f22c8200640 1 -- 172.21.15.83:0/1840540583 wait complete. 2024-04-25T00:13:48.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:13:49.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.032+0000 7fb71ca00640 1 Processor -- start 2024-04-25T00:13:49.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.032+0000 7fb71ca00640 1 -- start start 2024-04-25T00:13:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.032+0000 7fb71ca00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7180d0230 0x7fb7180d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.032+0000 7fb71ca00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7180cc9b0 con 0x7fb718163b00 2024-04-25T00:13:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.032+0000 7fb71ca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7180ccc90 con 0x7fb71815bf70 2024-04-25T00:13:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.032+0000 7fb71ca00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb7180d0be0 con 0x7fb7180d0230 2024-04-25T00:13:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.032+0000 7fb70fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7180d0230 0x7fb7180d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.032+0000 7fb716800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb718163b00 0x7fb718165ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41642/0 (socket says 172.21.15.83:41642) 2024-04-25T00:13:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.032+0000 7fb716800640 1 -- 172.21.15.83:0/18167508 learned_addr learned my addr 172.21.15.83:0/18167508 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.032+0000 7fb70fe00640 1 -- 172.21.15.83:0/18167508 >> v1:172.21.15.86:6789/0 conn(0x7fb71815bf70 legacy=0x7fb7180cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.032+0000 7fb70fe00640 1 -- 172.21.15.83:0/18167508 >> v1:172.21.15.83:6789/0 conn(0x7fb718163b00 legacy=0x7fb718165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.032+0000 7fb70fe00640 1 -- 172.21.15.83:0/18167508 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7180ccf70 con 0x7fb7180d0230 2024-04-25T00:13:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.032+0000 7fb715400640 1 -- 172.21.15.83:0/18167508 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2638439353 0 0) 0x7fb7180cc9b0 con 0x7fb718163b00 2024-04-25T00:13:49.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.036+0000 7fb70fe00640 1 --2- 172.21.15.83:0/18167508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7180d0230 0x7fb7180d06a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb7000097c0 tx=0x7fb70001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7eb3933a84740f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:49.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.036+0000 7fb715400640 1 -- 172.21.15.83:0/18167508 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7000b1070 con 0x7fb7180d0230 2024-04-25T00:13:49.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.036+0000 7fb715400640 1 -- 172.21.15.83:0/18167508 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7000ac5f0 con 0x7fb7180d0230 2024-04-25T00:13:49.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.036+0000 7fb715400640 1 -- 172.21.15.83:0/18167508 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7000ac9c0 con 0x7fb7180d0230 2024-04-25T00:13:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.036+0000 7fb71ca00640 1 -- 172.21.15.83:0/18167508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7180d0230 msgr2=0x7fb7180d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.036+0000 7fb71ca00640 1 --2- 172.21.15.83:0/18167508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7180d0230 0x7fb7180d06a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb7000097c0 tx=0x7fb70001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.036+0000 7fb71ca00640 1 -- 172.21.15.83:0/18167508 shutdown_connections 2024-04-25T00:13:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.036+0000 7fb71ca00640 1 --2- 172.21.15.83:0/18167508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7180d0230 0x7fb7180d06a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.036+0000 7fb71ca00640 1 -- 172.21.15.83:0/18167508 >> 172.21.15.83:0/18167508 conn(0x7fb7180d1c70 msgr2=0x7fb7180d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.036+0000 7fb71ca00640 1 -- 172.21.15.83:0/18167508 shutdown_connections 2024-04-25T00:13:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.036+0000 7fb71ca00640 1 -- 172.21.15.83:0/18167508 wait complete. 2024-04-25T00:13:49.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb71ca00640 1 Processor -- start 2024-04-25T00:13:49.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb71ca00640 1 -- start start 2024-04-25T00:13:49.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb71ca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7180d0230 0x7fb71814c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb71ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb71815bf70 0x7fb7180c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb71ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb718163b00 0x7fb718154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb71ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7180cff50 con 0x7fb718163b00 2024-04-25T00:13:49.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb71ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb71815c350 con 0x7fb71815bf70 2024-04-25T00:13:49.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb71ca00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb7180cda50 con 0x7fb7180d0230 2024-04-25T00:13:49.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb716800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb718163b00 0x7fb718154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb716800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb718163b00 0x7fb718154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44078/0 (socket says 172.21.15.83:44078) 2024-04-25T00:13:49.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb716800640 1 -- 172.21.15.83:0/482346740 learned_addr learned my addr 172.21.15.83:0/482346740 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb715e00640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7180d0230 0x7fb71814c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb70fe00640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb71815bf70 0x7fb7180c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb716800640 1 -- 172.21.15.83:0/482346740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7180d0230 msgr2=0x7fb71814c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb716800640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7180d0230 0x7fb71814c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb716800640 1 -- 172.21.15.83:0/482346740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb71815bf70 msgr2=0x7fb7180c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb716800640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb71815bf70 0x7fb7180c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb716800640 1 -- 172.21.15.83:0/482346740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7180b3560 con 0x7fb718163b00 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb70fe00640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb71815bf70 0x7fb7180c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb715e00640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7180d0230 0x7fb71814c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb716800640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb718163b00 0x7fb718154440 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb71000c730 tx=0x7fb71000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb70e000640 1 -- 172.21.15.83:0/482346740 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710018070 con 0x7fb718163b00 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb70e000640 1 -- 172.21.15.83:0/482346740 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71000ce40 con 0x7fb718163b00 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb70e000640 1 -- 172.21.15.83:0/482346740 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71000a860 con 0x7fb718163b00 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb71ca00640 1 -- 172.21.15.83:0/482346740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7180b3840 con 0x7fb718163b00 2024-04-25T00:13:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb71ca00640 1 -- 172.21.15.83:0/482346740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7180b3e00 con 0x7fb718163b00 2024-04-25T00:13:49.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.040+0000 7fb71ca00640 1 -- 172.21.15.83:0/482346740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71809fd60 con 0x7fb718163b00 2024-04-25T00:13:49.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.044+0000 7fb70e000640 1 -- 172.21.15.83:0/482346740 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb71000aa00 con 0x7fb718163b00 2024-04-25T00:13:49.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.044+0000 7fb70e000640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6f40678c0 0x7fb6f4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.044+0000 7fb70e000640 1 -- 172.21.15.83:0/482346740 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5249+0+0 (secure 0 0 0) 0x7fb7100850a0 con 0x7fb718163b00 2024-04-25T00:13:49.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.044+0000 7fb715e00640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6f40678c0 0x7fb6f4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.044+0000 7fb715e00640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6f40678c0 0x7fb6f4069d80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb708000f70 tx=0x7fb708009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:49.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.048+0000 7fb70e000640 1 -- 172.21.15.83:0/482346740 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb710055e30 con 0x7fb718163b00 2024-04-25T00:13:49.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.260+0000 7fb71ca00640 1 -- 172.21.15.83:0/482346740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb71804f5f0 con 0x7fb718163b00 2024-04-25T00:13:49.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.260+0000 7fb70e000640 1 -- 172.21.15.83:0/482346740 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fb710055730 con 0x7fb718163b00 2024-04-25T00:13:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:49.262 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:13:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-25T00:13:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-25T00:13:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmytlRFRAA3vHLqK3vHSB+8+TZHAuj7Q==", 2024-04-25T00:13:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:13:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:13:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-25T00:13:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm088MIRAApJvvePXyFoO8JnVErd57zQ==", 2024-04-25T00:13:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:13:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:13:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-25T00:13:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmCqZXHBAAPy5JGbEU11i4gc3gFtbGIQ==", 2024-04-25T00:13:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:13:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:13:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-25T00:13:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmiJZGKBAA5CBwH/cA+LOAj+inS/Hx2g==", 2024-04-25T00:13:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:13:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:13:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-25T00:13:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBhnylmMOiqLRAAdqrCTz4KfghoJ7cZc+b4qg==", 2024-04-25T00:13:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-25T00:13:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBknylmlp8OOBAAl+y4vBrCzDyV/ne9qICQ+w==", 2024-04-25T00:13:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-25T00:13:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBonylm85MOBhAA3ciY1UT/Qg4Pe72OA0IvnA==", 2024-04-25T00:13:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBrnylmnAbDEhAAkCQXjbiKINiftX71HBKvQA==", 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBxnylmxhy2KxAAs6qjxZA+Fj5umHU/4FVqGw==", 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-25T00:13:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB0nylmGfSlNRAA45tuuh+5ISOMlhaFdCE5rQ==", 2024-04-25T00:13:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-25T00:13:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB4nylmouyBARAAz4s4S1BSiDU9YxQ/Vj0d0A==", 2024-04-25T00:13:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-25T00:13:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB7nylmtIh0CxAApWJLvC8Zlt298VJ55gPrqw==", 2024-04-25T00:13:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:13:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:13:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-25T00:13:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmOqooLBAApte+X6YkRmrpe12YdjYekg==", 2024-04-25T00:13:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-04-25T00:13:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:13:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-25T00:13:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmDH93NRAASQzJNC3nI0NeU8cv951eBA==", 2024-04-25T00:13:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:13:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:13:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:13:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-25T00:13:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmHw3+MRAAciXRmexPxmrf7U2jKwpZQQ==", 2024-04-25T00:13:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:13:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:13:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:13:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-25T00:13:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBfnylmIsMCABAAu+yzPRupFFqA/yH4GSZ+8w==", 2024-04-25T00:13:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:13:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:13:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:13:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:13:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-25T00:13:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdnylmZ7SKNRAArVHYbh7fBiOUNRTenIwT1w==", 2024-04-25T00:13:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:13:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:13:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:13:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-25T00:13:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm6TMqERAAiPL81HRFrlz4ku20HemS4w==", 2024-04-25T00:13:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm9VRpDRAA5XUtYwjiWBpF1ObJ/Y5g0w==", 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T00:13:49.275 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:13:49.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.260+0000 7fb71ca00640 1 -- 172.21.15.83:0/482346740 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6f40678c0 msgr2=0x7fb6f4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.260+0000 7fb71ca00640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6f40678c0 0x7fb6f4069d80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb708000f70 tx=0x7fb708009040 comp rx=0 tx=0).stop 2024-04-25T00:13:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.260+0000 7fb71ca00640 1 -- 172.21.15.83:0/482346740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb718163b00 msgr2=0x7fb718154440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.260+0000 7fb71ca00640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb718163b00 0x7fb718154440 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb71000c730 tx=0x7fb71000cbf0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.260+0000 7fb71ca00640 1 -- 172.21.15.83:0/482346740 shutdown_connections 2024-04-25T00:13:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.260+0000 7fb71ca00640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6f40678c0 0x7fb6f4069d80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.260+0000 7fb71ca00640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb718163b00 0x7fb718154440 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.260+0000 7fb71ca00640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb71815bf70 0x7fb7180c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.260+0000 7fb71ca00640 1 --2- 172.21.15.83:0/482346740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7180d0230 0x7fb71814c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.260+0000 7fb71ca00640 1 -- 172.21.15.83:0/482346740 >> 172.21.15.83:0/482346740 conn(0x7fb7180d1c70 msgr2=0x7fb7180d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.260+0000 7fb71ca00640 1 -- 172.21.15.83:0/482346740 shutdown_connections 2024-04-25T00:13:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.260+0000 7fb71ca00640 1 -- 172.21.15.83:0/482346740 wait complete. 2024-04-25T00:13:49.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:49.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf42400640 1 Processor -- start 2024-04-25T00:13:49.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf42400640 1 -- start start 2024-04-25T00:13:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf42400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcf3c06fc10 0x7fcf3c0cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf42400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf3c1452c0 0x7fcf3c0cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf42400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf3c0d1850 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf42400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fcf3c0d1b30 con 0x7fcf3c06fc10 2024-04-25T00:13:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf42400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3c0cd710 con 0x7fcf3c1452c0 2024-04-25T00:13:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf3b400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcf3c06b1e0 0x7fcf3c06b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41656/0 (socket says 172.21.15.83:41656) 2024-04-25T00:13:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf3b400640 1 -- 172.21.15.83:0/196172290 learned_addr learned my addr 172.21.15.83:0/196172290 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf3aa00640 1 --2- 172.21.15.83:0/196172290 >> v2:172.21.15.83:3300/0 conn(0x7fcf3c06fc10 0x7fcf3c0cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf3a000640 1 -- 172.21.15.83:0/196172290 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1906711815 0 0) 0x7fcf3c0d1850 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf3a000640 1 -- 172.21.15.83:0/196172290 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf20003530 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf3be00640 1 --2- 172.21.15.83:0/196172290 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf3c1452c0 0x7fcf3c0cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf3aa00640 1 -- 172.21.15.83:0/196172290 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf3c1452c0 msgr2=0x7fcf3c0cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf3aa00640 1 --2- 172.21.15.83:0/196172290 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf3c1452c0 0x7fcf3c0cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf3aa00640 1 -- 172.21.15.83:0/196172290 >> v1:172.21.15.83:6789/0 conn(0x7fcf3c06b1e0 legacy=0x7fcf3c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf3aa00640 1 -- 172.21.15.83:0/196172290 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c0cd9f0 con 0x7fcf3c06fc10 2024-04-25T00:13:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.416+0000 7fcf3be00640 1 --2- 172.21.15.83:0/196172290 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf3c1452c0 0x7fcf3c0cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:49.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.420+0000 7fcf3aa00640 1 --2- 172.21.15.83:0/196172290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf3c06fc10 0x7fcf3c0cf260 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcf280097c0 tx=0x7fcf2801c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d1cb24179a7416 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:49.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.420+0000 7fcf3a000640 1 -- 172.21.15.83:0/196172290 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf280b1070 con 0x7fcf3c06fc10 2024-04-25T00:13:49.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.420+0000 7fcf3a000640 1 -- 172.21.15.83:0/196172290 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf280a4810 con 0x7fcf3c06fc10 2024-04-25T00:13:49.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.420+0000 7fcf3a000640 1 -- 172.21.15.83:0/196172290 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf280a4be0 con 0x7fcf3c06fc10 2024-04-25T00:13:49.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.420+0000 7fcf42400640 1 -- 172.21.15.83:0/196172290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf3c06fc10 msgr2=0x7fcf3c0cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.420+0000 7fcf42400640 1 --2- 172.21.15.83:0/196172290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf3c06fc10 0x7fcf3c0cf260 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcf280097c0 tx=0x7fcf2801c060 comp rx=0 tx=0).stop 2024-04-25T00:13:49.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.420+0000 7fcf42400640 1 -- 172.21.15.83:0/196172290 shutdown_connections 2024-04-25T00:13:49.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.420+0000 7fcf42400640 1 --2- 172.21.15.83:0/196172290 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf3c1452c0 0x7fcf3c0cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.420+0000 7fcf42400640 1 --2- 172.21.15.83:0/196172290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf3c06fc10 0x7fcf3c0cf260 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.420+0000 7fcf42400640 1 -- 172.21.15.83:0/196172290 >> 172.21.15.83:0/196172290 conn(0x7fcf3c0d0b90 msgr2=0x7fcf3c0d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:49.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.420+0000 7fcf42400640 1 -- 172.21.15.83:0/196172290 shutdown_connections 2024-04-25T00:13:49.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.420+0000 7fcf42400640 1 -- 172.21.15.83:0/196172290 wait complete. 2024-04-25T00:13:49.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf42400640 1 Processor -- start 2024-04-25T00:13:49.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf42400640 1 -- start start 2024-04-25T00:13:49.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf42400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf3c06b1e0 0x7fcf3c13ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf42400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf3c06fc10 0x7fcf3c15f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf42400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf3c1452c0 0x7fcf3c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf42400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c169ad0 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf42400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c169950 con 0x7fcf3c1452c0 2024-04-25T00:13:49.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf42400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3c169c50 con 0x7fcf3c06fc10 2024-04-25T00:13:49.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf3b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf3c06b1e0 0x7fcf3c13ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf3b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf3c06b1e0 0x7fcf3c13ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44090/0 (socket says 172.21.15.83:44090) 2024-04-25T00:13:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf3b400640 1 -- 172.21.15.83:0/3464236624 learned_addr learned my addr 172.21.15.83:0/3464236624 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf3b400640 1 -- 172.21.15.83:0/3464236624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf3c06fc10 msgr2=0x7fcf3c15f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf3b400640 1 --2- 172.21.15.83:0/3464236624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf3c06fc10 0x7fcf3c15f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf3b400640 1 -- 172.21.15.83:0/3464236624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf3c1452c0 msgr2=0x7fcf3c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf3b400640 1 --2- 172.21.15.83:0/3464236624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf3c1452c0 0x7fcf3c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf3b400640 1 -- 172.21.15.83:0/3464236624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c164730 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf3b400640 1 --2- 172.21.15.83:0/3464236624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf3c06b1e0 0x7fcf3c13ef20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00c3c0 tx=0x7fcf2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf33400640 1 -- 172.21.15.83:0/3464236624 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c017070 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf33400640 1 -- 172.21.15.83:0/3464236624 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2c005cf0 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf33400640 1 -- 172.21.15.83:0/3464236624 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c0072f0 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf42400640 1 -- 172.21.15.83:0/3464236624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3c164a10 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf42400640 1 -- 172.21.15.83:0/3464236624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3c164fd0 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.424+0000 7fcf42400640 1 -- 172.21.15.83:0/3464236624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3c0737d0 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.428+0000 7fcf33400640 1 -- 172.21.15.83:0/3464236624 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcf2c01a070 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.428+0000 7fcf33400640 1 --2- 172.21.15.83:0/3464236624 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcf1c0678c0 0x7fcf1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.428+0000 7fcf33400640 1 -- 172.21.15.83:0/3464236624 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5249+0+0 (secure 0 0 0) 0x7fcf2c084f40 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.428+0000 7fcf3aa00640 1 --2- 172.21.15.83:0/3464236624 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcf1c0678c0 0x7fcf1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.428+0000 7fcf3aa00640 1 --2- 172.21.15.83:0/3464236624 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcf1c0678c0 0x7fcf1c069d80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcf280ae000 tx=0x7fcf28004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:49.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.432+0000 7fcf33400640 1 -- 172.21.15.83:0/3464236624 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcf2c055bb0 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.592+0000 7fcf42400640 1 -- 172.21.15.83:0/3464236624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf3c04f5f0 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.592+0000 7fcf33400640 1 -- 172.21.15.83:0/3464236624 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+13081 (secure 0 0 0) 0x7fcf2c0554b0 con 0x7fcf3c06b1e0 2024-04-25T00:13:49.596 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:49.596 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":67,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:48.884263+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.596+0000 7fcf42400640 1 -- 172.21.15.83:0/3464236624 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcf1c0678c0 msgr2=0x7fcf1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.596+0000 7fcf42400640 1 --2- 172.21.15.83:0/3464236624 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcf1c0678c0 0x7fcf1c069d80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcf280ae000 tx=0x7fcf28004740 comp rx=0 tx=0).stop 2024-04-25T00:13:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.596+0000 7fcf42400640 1 -- 172.21.15.83:0/3464236624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf3c06b1e0 msgr2=0x7fcf3c13ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.596+0000 7fcf42400640 1 --2- 172.21.15.83:0/3464236624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf3c06b1e0 0x7fcf3c13ef20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00c3c0 tx=0x7fcf2c00c880 comp rx=0 tx=0).stop 2024-04-25T00:13:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.596+0000 7fcf42400640 1 -- 172.21.15.83:0/3464236624 shutdown_connections 2024-04-25T00:13:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.596+0000 7fcf42400640 1 --2- 172.21.15.83:0/3464236624 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcf1c0678c0 0x7fcf1c069d80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.596+0000 7fcf42400640 1 --2- 172.21.15.83:0/3464236624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf3c06fc10 0x7fcf3c15f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.596+0000 7fcf42400640 1 --2- 172.21.15.83:0/3464236624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf3c1452c0 0x7fcf3c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.596+0000 7fcf42400640 1 --2- 172.21.15.83:0/3464236624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf3c06b1e0 0x7fcf3c13ef20 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.596+0000 7fcf42400640 1 -- 172.21.15.83:0/3464236624 >> 172.21.15.83:0/3464236624 conn(0x7fcf3c0d0b90 msgr2=0x7fcf3c0d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.596+0000 7fcf42400640 1 -- 172.21.15.83:0/3464236624 shutdown_connections 2024-04-25T00:13:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.596+0000 7fcf42400640 1 -- 172.21.15.83:0/3464236624 wait complete. 2024-04-25T00:13:49.617 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:13:49.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:13:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.756+0000 7fc910200640 1 Processor -- start 2024-04-25T00:13:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.756+0000 7fc910200640 1 -- start start 2024-04-25T00:13:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.756+0000 7fc910200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc9080d05a0 0x7fc9080cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.756+0000 7fc910200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc9080cd280 0x7fc9080b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.756+0000 7fc910200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9080cda00 con 0x7fc90815c150 2024-04-25T00:13:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.756+0000 7fc910200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc90815c530 con 0x7fc9080d05a0 2024-04-25T00:13:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.756+0000 7fc910200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc9080cdce0 con 0x7fc9080cd280 2024-04-25T00:13:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.756+0000 7fc90ce00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc9080d05a0 0x7fc9080cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.756+0000 7fc90d800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc90815c150 0x7fc9080cfe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41666/0 (socket says 172.21.15.83:41666) 2024-04-25T00:13:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.756+0000 7fc90d800640 1 -- 172.21.15.83:0/2216727224 learned_addr learned my addr 172.21.15.83:0/2216727224 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.756+0000 7fc90e200640 1 --2- 172.21.15.83:0/2216727224 >> v2:172.21.15.86:3300/0 conn(0x7fc9080cd280 0x7fc9080b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.756+0000 7fc90ce00640 1 -- 172.21.15.83:0/2216727224 >> v2:172.21.15.86:3300/0 conn(0x7fc9080cd280 msgr2=0x7fc9080b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc90ce00640 1 --2- 172.21.15.83:0/2216727224 >> v2:172.21.15.86:3300/0 conn(0x7fc9080cd280 0x7fc9080b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc90ce00640 1 -- 172.21.15.83:0/2216727224 >> v1:172.21.15.83:6789/0 conn(0x7fc90815c150 legacy=0x7fc9080cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc907e00640 1 -- 172.21.15.83:0/2216727224 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1222613203 0 0) 0x7fc9080cda00 con 0x7fc90815c150 2024-04-25T00:13:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc90ce00640 1 -- 172.21.15.83:0/2216727224 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9080b1230 con 0x7fc9080d05a0 2024-04-25T00:13:49.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc90ce00640 1 --2- 172.21.15.83:0/2216727224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9080d05a0 0x7fc9080cf860 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc9080cf740 tx=0x7fc8f401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2157a540fce6ad4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:49.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc907e00640 1 -- 172.21.15.83:0/2216727224 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40b0070 con 0x7fc9080d05a0 2024-04-25T00:13:49.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc907e00640 1 -- 172.21.15.83:0/2216727224 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8f40ab600 con 0x7fc9080d05a0 2024-04-25T00:13:49.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc907e00640 1 -- 172.21.15.83:0/2216727224 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40ab9d0 con 0x7fc9080d05a0 2024-04-25T00:13:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc910200640 1 -- 172.21.15.83:0/2216727224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9080d05a0 msgr2=0x7fc9080cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc910200640 1 --2- 172.21.15.83:0/2216727224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9080d05a0 0x7fc9080cf860 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc9080cf740 tx=0x7fc8f401c330 comp rx=0 tx=0).stop 2024-04-25T00:13:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc910200640 1 -- 172.21.15.83:0/2216727224 shutdown_connections 2024-04-25T00:13:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc910200640 1 --2- 172.21.15.83:0/2216727224 >> v2:172.21.15.86:3300/0 conn(0x7fc9080cd280 0x7fc9080b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc910200640 1 --2- 172.21.15.83:0/2216727224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9080d05a0 0x7fc9080cf860 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc910200640 1 -- 172.21.15.83:0/2216727224 >> 172.21.15.83:0/2216727224 conn(0x7fc9080d1c90 msgr2=0x7fc9080d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc910200640 1 -- 172.21.15.83:0/2216727224 shutdown_connections 2024-04-25T00:13:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.760+0000 7fc910200640 1 -- 172.21.15.83:0/2216727224 wait complete. 2024-04-25T00:13:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc910200640 1 Processor -- start 2024-04-25T00:13:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc910200640 1 -- start start 2024-04-25T00:13:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc910200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9080cd280 0x7fc908154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc910200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9080d05a0 0x7fc9080c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc910200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc90815c150 0x7fc9081654e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc910200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc9080b2be0 con 0x7fc90815c150 2024-04-25T00:13:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc910200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9080b2a60 con 0x7fc9080cd280 2024-04-25T00:13:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc910200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc9080b2d60 con 0x7fc9080d05a0 2024-04-25T00:13:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc90e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc90815c150 0x7fc9081654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc90e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc90815c150 0x7fc9081654e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44110/0 (socket says 172.21.15.83:44110) 2024-04-25T00:13:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc90e200640 1 -- 172.21.15.83:0/42486553 learned_addr learned my addr 172.21.15.83:0/42486553 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc90d800640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9080cd280 0x7fc908154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc90ce00640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9080d05a0 0x7fc9080c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc90e200640 1 -- 172.21.15.83:0/42486553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9080d05a0 msgr2=0x7fc9080c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc90e200640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9080d05a0 0x7fc9080c7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc90e200640 1 -- 172.21.15.83:0/42486553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9080cd280 msgr2=0x7fc908154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc90e200640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9080cd280 0x7fc908154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc90e200640 1 -- 172.21.15.83:0/42486553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc908155240 con 0x7fc90815c150 2024-04-25T00:13:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc90d800640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9080cd280 0x7fc908154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc90e200640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc90815c150 0x7fc9081654e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc90000d800 tx=0x7fc90000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc905600640 1 -- 172.21.15.83:0/42486553 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90000b6c0 con 0x7fc90815c150 2024-04-25T00:13:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc905600640 1 -- 172.21.15.83:0/42486553 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc900019070 con 0x7fc90815c150 2024-04-25T00:13:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc905600640 1 -- 172.21.15.83:0/42486553 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90000baa0 con 0x7fc90815c150 2024-04-25T00:13:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc910200640 1 -- 172.21.15.83:0/42486553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90815d440 con 0x7fc90815c150 2024-04-25T00:13:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.764+0000 7fc910200640 1 -- 172.21.15.83:0/42486553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90815da00 con 0x7fc90815c150 2024-04-25T00:13:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.768+0000 7fc910200640 1 -- 172.21.15.83:0/42486553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90804f510 con 0x7fc90815c150 2024-04-25T00:13:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.768+0000 7fc905600640 1 -- 172.21.15.83:0/42486553 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc90000bc40 con 0x7fc90815c150 2024-04-25T00:13:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.768+0000 7fc905600640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8e80678c0 0x7fc8e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:49.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.768+0000 7fc905600640 1 -- 172.21.15.83:0/42486553 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5249+0+0 (secure 0 0 0) 0x7fc900085f40 con 0x7fc90815c150 2024-04-25T00:13:49.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.768+0000 7fc90d800640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8e80678c0 0x7fc8e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:49.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.768+0000 7fc90d800640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8e80678c0 0x7fc8e8069d80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc007fd0 tx=0x7fc8fc005b80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:49.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.772+0000 7fc905600640 1 -- 172.21.15.83:0/42486553 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc900056ba0 con 0x7fc90815c150 2024-04-25T00:13:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.932+0000 7fc910200640 1 -- 172.21.15.83:0/42486553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc9080cc760 con 0x7fc90815c150 2024-04-25T00:13:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.932+0000 7fc905600640 1 -- 172.21.15.83:0/42486553 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) ==== 93+0+10 (secure 0 0 0) 0x7fc9000564a0 con 0x7fc90815c150 2024-04-25T00:13:49.937 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:13:49.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.936+0000 7fc910200640 1 -- 172.21.15.83:0/42486553 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8e80678c0 msgr2=0x7fc8e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.936+0000 7fc910200640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8e80678c0 0x7fc8e8069d80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc007fd0 tx=0x7fc8fc005b80 comp rx=0 tx=0).stop 2024-04-25T00:13:49.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.936+0000 7fc910200640 1 -- 172.21.15.83:0/42486553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc90815c150 msgr2=0x7fc9081654e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:49.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.936+0000 7fc910200640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc90815c150 0x7fc9081654e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc90000d800 tx=0x7fc90000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.936+0000 7fc910200640 1 -- 172.21.15.83:0/42486553 shutdown_connections 2024-04-25T00:13:49.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.936+0000 7fc910200640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc8e80678c0 0x7fc8e8069d80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.936+0000 7fc910200640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9080d05a0 0x7fc9080c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.936+0000 7fc910200640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc90815c150 0x7fc9081654e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.936+0000 7fc910200640 1 --2- 172.21.15.83:0/42486553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9080cd280 0x7fc908154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.936+0000 7fc910200640 1 -- 172.21.15.83:0/42486553 >> 172.21.15.83:0/42486553 conn(0x7fc9080d1c90 msgr2=0x7fc90806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.936+0000 7fc910200640 1 -- 172.21.15.83:0/42486553 shutdown_connections 2024-04-25T00:13:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:49.936+0000 7fc910200640 1 -- 172.21.15.83:0/42486553 wait complete. 2024-04-25T00:13:49.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:13:50.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.088+0000 7faccc200640 1 Processor -- start 2024-04-25T00:13:50.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7faccc200640 1 -- start start 2024-04-25T00:13:50.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7faccc200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facc406fa50 0x7facc40b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7faccc200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7facc415e320 0x7facc40ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7faccc200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facc406fee0 con 0x7facc415df40 2024-04-25T00:13:50.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7faccc200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7facc40704a0 con 0x7facc415e320 2024-04-25T00:13:50.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7faccc200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7facc40701c0 con 0x7facc406fa50 2024-04-25T00:13:50.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7facc9800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7facc415df40 0x7facc40cc2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41670/0 (socket says 172.21.15.83:41670) 2024-04-25T00:13:50.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7facc9800640 1 -- 172.21.15.83:0/3495341650 learned_addr learned my addr 172.21.15.83:0/3495341650 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:50.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7facca200640 1 --2- 172.21.15.83:0/3495341650 >> v2:172.21.15.83:3300/0 conn(0x7facc415e320 0x7facc40ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7facc8e00640 1 --2- 172.21.15.83:0/3495341650 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facc406fa50 0x7facc40b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7facca200640 1 -- 172.21.15.83:0/3495341650 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facc406fa50 msgr2=0x7facc40b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:50.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7facca200640 1 --2- 172.21.15.83:0/3495341650 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facc406fa50 0x7facc40b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7facca200640 1 -- 172.21.15.83:0/3495341650 >> v1:172.21.15.83:6789/0 conn(0x7facc415df40 legacy=0x7facc40cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7facca200640 1 -- 172.21.15.83:0/3495341650 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc40d00a0 con 0x7facc415e320 2024-04-25T00:13:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7facc3e00640 1 -- 172.21.15.83:0/3495341650 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1294997446 0 0) 0x7facc406fee0 con 0x7facc415df40 2024-04-25T00:13:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.092+0000 7facc8e00640 1 --2- 172.21.15.83:0/3495341650 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facc406fa50 0x7facc40b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.096+0000 7facca200640 1 --2- 172.21.15.83:0/3495341650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc415e320 0x7facc40ce510 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7facbc009d90 tx=0x7facbc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c0c9c706fe82d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.096+0000 7facc3e00640 1 -- 172.21.15.83:0/3495341650 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc0b2070 con 0x7facc415e320 2024-04-25T00:13:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.096+0000 7facc3e00640 1 -- 172.21.15.83:0/3495341650 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facbc0a5940 con 0x7facc415e320 2024-04-25T00:13:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.096+0000 7facc3e00640 1 -- 172.21.15.83:0/3495341650 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc0a5d10 con 0x7facc415e320 2024-04-25T00:13:50.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.096+0000 7faccc200640 1 -- 172.21.15.83:0/3495341650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc415e320 msgr2=0x7facc40ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.096+0000 7faccc200640 1 --2- 172.21.15.83:0/3495341650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc415e320 0x7facc40ce510 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7facbc009d90 tx=0x7facbc01c970 comp rx=0 tx=0).stop 2024-04-25T00:13:50.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.096+0000 7faccc200640 1 -- 172.21.15.83:0/3495341650 shutdown_connections 2024-04-25T00:13:50.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.096+0000 7faccc200640 1 --2- 172.21.15.83:0/3495341650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc415e320 0x7facc40ce510 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.096+0000 7faccc200640 1 --2- 172.21.15.83:0/3495341650 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facc406fa50 0x7facc40b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.096+0000 7faccc200640 1 -- 172.21.15.83:0/3495341650 >> 172.21.15.83:0/3495341650 conn(0x7facc40b37e0 msgr2=0x7facc40b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:50.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.096+0000 7faccc200640 1 -- 172.21.15.83:0/3495341650 shutdown_connections 2024-04-25T00:13:50.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.096+0000 7faccc200640 1 -- 172.21.15.83:0/3495341650 wait complete. 2024-04-25T00:13:50.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7faccc200640 1 Processor -- start 2024-04-25T00:13:50.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7faccc200640 1 -- start start 2024-04-25T00:13:50.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7faccc200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facc415df40 0x7facc40b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7faccc200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc415e320 0x7facc4160f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7faccc200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc4152dd0 0x7facc40b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7faccc200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7facc416b020 con 0x7facc415e320 2024-04-25T00:13:50.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7faccc200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facc40b15b0 con 0x7facc4152dd0 2024-04-25T00:13:50.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7faccc200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7facc416b1a0 con 0x7facc415df40 2024-04-25T00:13:50.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7facc8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc415e320 0x7facc4160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7facc8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc415e320 0x7facc4160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44122/0 (socket says 172.21.15.83:44122) 2024-04-25T00:13:50.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7facc8e00640 1 -- 172.21.15.83:0/2402515414 learned_addr learned my addr 172.21.15.83:0/2402515414 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:50.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7facca200640 1 --2- 172.21.15.83:0/2402515414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc4152dd0 0x7facc40b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7facc9800640 1 --2- 172.21.15.83:0/2402515414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facc415df40 0x7facc40b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7facc8e00640 1 -- 172.21.15.83:0/2402515414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facc415df40 msgr2=0x7facc40b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7facc8e00640 1 --2- 172.21.15.83:0/2402515414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facc415df40 0x7facc40b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7facc8e00640 1 -- 172.21.15.83:0/2402515414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc4152dd0 msgr2=0x7facc40b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7facc8e00640 1 --2- 172.21.15.83:0/2402515414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc4152dd0 0x7facc40b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7facc8e00640 1 -- 172.21.15.83:0/2402515414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc40afde0 con 0x7facc415e320 2024-04-25T00:13:50.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7facc8e00640 1 --2- 172.21.15.83:0/2402515414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc415e320 0x7facc4160f20 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7facb000e980 tx=0x7facb000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:50.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7facc1600640 1 -- 172.21.15.83:0/2402515414 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0017070 con 0x7facc415e320 2024-04-25T00:13:50.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7facc1600640 1 -- 172.21.15.83:0/2402515414 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb000ca00 con 0x7facc415e320 2024-04-25T00:13:50.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7facc1600640 1 -- 172.21.15.83:0/2402515414 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb000cdd0 con 0x7facc415e320 2024-04-25T00:13:50.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7faccc200640 1 -- 172.21.15.83:0/2402515414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc40b00c0 con 0x7facc415e320 2024-04-25T00:13:50.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7faccc200640 1 -- 172.21.15.83:0/2402515414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc4153860 con 0x7facc415e320 2024-04-25T00:13:50.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.100+0000 7faccc200640 1 -- 172.21.15.83:0/2402515414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc409fe40 con 0x7facc415e320 2024-04-25T00:13:50.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.104+0000 7facc1600640 1 -- 172.21.15.83:0/2402515414 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7facb001a070 con 0x7facc415e320 2024-04-25T00:13:50.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.104+0000 7facc1600640 1 --2- 172.21.15.83:0/2402515414 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faca40678c0 0x7faca4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.104+0000 7facc1600640 1 -- 172.21.15.83:0/2402515414 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5249+0+0 (secure 0 0 0) 0x7facb0085c90 con 0x7facc415e320 2024-04-25T00:13:50.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.104+0000 7facc9800640 1 --2- 172.21.15.83:0/2402515414 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faca40678c0 0x7faca4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.104+0000 7facc9800640 1 --2- 172.21.15.83:0/2402515414 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faca40678c0 0x7faca4069d80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7facb8007fd0 tx=0x7facb8007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:50.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.112+0000 7facc1600640 1 -- 172.21.15.83:0/2402515414 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7facb0056330 con 0x7facc415e320 2024-04-25T00:13:50.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.268+0000 7faccc200640 1 -- 172.21.15.83:0/2402515414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7facc4073280 con 0x7facc415e320 2024-04-25T00:13:50.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.268+0000 7facc1600640 1 -- 172.21.15.83:0/2402515414 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v67) ==== 84+0+65 (secure 0 0 0) 0x7facb005a630 con 0x7facc415e320 2024-04-25T00:13:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:50.273 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-25T00:13:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:13:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-25T00:13:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-25T00:13:50.273 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:13:50.273 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-25T00:13:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.272+0000 7faccc200640 1 -- 172.21.15.83:0/2402515414 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faca40678c0 msgr2=0x7faca4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.272+0000 7faccc200640 1 --2- 172.21.15.83:0/2402515414 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faca40678c0 0x7faca4069d80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7facb8007fd0 tx=0x7facb8007d60 comp rx=0 tx=0).stop 2024-04-25T00:13:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.272+0000 7faccc200640 1 -- 172.21.15.83:0/2402515414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc415e320 msgr2=0x7facc4160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.272+0000 7faccc200640 1 --2- 172.21.15.83:0/2402515414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc415e320 0x7facc4160f20 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7facb000e980 tx=0x7facb000ee40 comp rx=0 tx=0).stop 2024-04-25T00:13:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.272+0000 7faccc200640 1 -- 172.21.15.83:0/2402515414 shutdown_connections 2024-04-25T00:13:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.272+0000 7faccc200640 1 --2- 172.21.15.83:0/2402515414 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7faca40678c0 0x7faca4069d80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.272+0000 7faccc200640 1 --2- 172.21.15.83:0/2402515414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc4152dd0 0x7facc40b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.272+0000 7faccc200640 1 --2- 172.21.15.83:0/2402515414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facc415e320 0x7facc4160f20 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.272+0000 7faccc200640 1 --2- 172.21.15.83:0/2402515414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facc415df40 0x7facc40b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.272+0000 7faccc200640 1 -- 172.21.15.83:0/2402515414 >> 172.21.15.83:0/2402515414 conn(0x7facc40b37e0 msgr2=0x7facc40cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.272+0000 7faccc200640 1 -- 172.21.15.83:0/2402515414 shutdown_connections 2024-04-25T00:13:50.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.272+0000 7faccc200640 1 -- 172.21.15.83:0/2402515414 wait complete. 2024-04-25T00:13:50.294 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:13:50.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:13:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.432+0000 7f2183400640 1 Processor -- start 2024-04-25T00:13:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.432+0000 7f2183400640 1 -- start start 2024-04-25T00:13:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.432+0000 7f2183400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f217c15f5d0 0x7f217c0b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.432+0000 7f2183400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f217c0d2870 con 0x7f217c0b1fa0 2024-04-25T00:13:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.432+0000 7f2183400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f217c0d2b10 con 0x7f217c0b1410 2024-04-25T00:13:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.432+0000 7f2183400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f217c0aea70 con 0x7f217c15f5d0 2024-04-25T00:13:50.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.432+0000 7f2181400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f217c0b1fa0 0x7f217c0af4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41672/0 (socket says 172.21.15.83:41672) 2024-04-25T00:13:50.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.432+0000 7f2180a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f217c15f5d0 0x7f217c0b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.432+0000 7f2181400640 1 -- 172.21.15.83:0/3054236231 learned_addr learned my addr 172.21.15.83:0/3054236231 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:50.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.432+0000 7f217be00640 1 --1- 172.21.15.83:0/3054236231 >> v1:172.21.15.86:6789/0 conn(0x7f217c0b1410 0x7f217c0b1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:55376/0 (socket says 172.21.15.83:55376) 2024-04-25T00:13:50.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.432+0000 7f2180a00640 1 -- 172.21.15.83:0/3054236231 >> v1:172.21.15.86:6789/0 conn(0x7f217c0b1410 legacy=0x7f217c0b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.432+0000 7f2180a00640 1 -- 172.21.15.83:0/3054236231 >> v1:172.21.15.83:6789/0 conn(0x7f217c0b1fa0 legacy=0x7f217c0af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.432+0000 7f217b400640 1 -- 172.21.15.83:0/3054236231 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 705216540 0 0) 0x7f217c0d2870 con 0x7f217c0b1fa0 2024-04-25T00:13:50.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.432+0000 7f2180a00640 1 -- 172.21.15.83:0/3054236231 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217c0aecc0 con 0x7f217c15f5d0 2024-04-25T00:13:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.436+0000 7f2180a00640 1 --2- 172.21.15.83:0/3054236231 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f217c15f5d0 0x7f217c0b2f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2170009770 tx=0x7f217001c280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f59f6f0195fdb6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.436+0000 7f217b400640 1 -- 172.21.15.83:0/3054236231 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21700b0070 con 0x7f217c15f5d0 2024-04-25T00:13:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.436+0000 7f217b400640 1 -- 172.21.15.83:0/3054236231 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21700ab600 con 0x7f217c15f5d0 2024-04-25T00:13:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.436+0000 7f217b400640 1 -- 172.21.15.83:0/3054236231 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21700ab9d0 con 0x7f217c15f5d0 2024-04-25T00:13:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.436+0000 7f2183400640 1 -- 172.21.15.83:0/3054236231 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f217c15f5d0 msgr2=0x7f217c0b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.436+0000 7f2183400640 1 --2- 172.21.15.83:0/3054236231 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f217c15f5d0 0x7f217c0b2f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2170009770 tx=0x7f217001c280 comp rx=0 tx=0).stop 2024-04-25T00:13:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.436+0000 7f2183400640 1 -- 172.21.15.83:0/3054236231 shutdown_connections 2024-04-25T00:13:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.436+0000 7f2183400640 1 --2- 172.21.15.83:0/3054236231 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f217c15f5d0 0x7f217c0b2f50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.436+0000 7f2183400640 1 -- 172.21.15.83:0/3054236231 >> 172.21.15.83:0/3054236231 conn(0x7f217c0d1d50 msgr2=0x7f217c0d2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.436+0000 7f2183400640 1 -- 172.21.15.83:0/3054236231 shutdown_connections 2024-04-25T00:13:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.436+0000 7f2183400640 1 -- 172.21.15.83:0/3054236231 wait complete. 2024-04-25T00:13:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2183400640 1 Processor -- start 2024-04-25T00:13:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2183400640 1 -- start start 2024-04-25T00:13:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2183400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c0b1410 0x7f217c0c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2183400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c0b1fa0 0x7f217c167620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2183400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f217c15f5d0 0x7f217c0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2183400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f217c0b07f0 con 0x7f217c0b1410 2024-04-25T00:13:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2183400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f217c0b0670 con 0x7f217c0b1fa0 2024-04-25T00:13:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2183400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f217c169bb0 con 0x7f217c15f5d0 2024-04-25T00:13:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2180a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c0b1410 0x7f217c0c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2180a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c0b1410 0x7f217c0c5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44138/0 (socket says 172.21.15.83:44138) 2024-04-25T00:13:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2180a00640 1 -- 172.21.15.83:0/3974706624 learned_addr learned my addr 172.21.15.83:0/3974706624 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f217be00640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c0b1fa0 0x7f217c167620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2181400640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f217c15f5d0 0x7f217c0c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2180a00640 1 -- 172.21.15.83:0/3974706624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f217c15f5d0 msgr2=0x7f217c0c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2180a00640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f217c15f5d0 0x7f217c0c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2180a00640 1 -- 172.21.15.83:0/3974706624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c0b1fa0 msgr2=0x7f217c167620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2180a00640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c0b1fa0 0x7f217c167620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2180a00640 1 -- 172.21.15.83:0/3974706624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217c0c7b60 con 0x7f217c0b1410 2024-04-25T00:13:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2181400640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f217c15f5d0 0x7f217c0c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2180a00640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c0b1410 0x7f217c0c5880 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f217001a930 tx=0x7f2170019020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2178c00640 1 -- 172.21.15.83:0/3974706624 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21700bb070 con 0x7f217c0b1410 2024-04-25T00:13:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2178c00640 1 -- 172.21.15.83:0/3974706624 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21700b7680 con 0x7f217c0b1410 2024-04-25T00:13:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2178c00640 1 -- 172.21.15.83:0/3974706624 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21700b0040 con 0x7f217c0b1410 2024-04-25T00:13:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2183400640 1 -- 172.21.15.83:0/3974706624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217c0cb980 con 0x7f217c0b1410 2024-04-25T00:13:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2183400640 1 -- 172.21.15.83:0/3974706624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217c14b400 con 0x7f217c0b1410 2024-04-25T00:13:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.440+0000 7f2183400640 1 -- 172.21.15.83:0/3974706624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f217c09fdf0 con 0x7f217c0b1410 2024-04-25T00:13:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.444+0000 7f2178c00640 1 -- 172.21.15.83:0/3974706624 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f21700be070 con 0x7f217c0b1410 2024-04-25T00:13:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.444+0000 7f2178c00640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f215c0678c0 0x7f215c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.444+0000 7f2178c00640 1 -- 172.21.15.83:0/3974706624 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5249+0+0 (secure 0 0 0) 0x7f21701200d0 con 0x7f217c0b1410 2024-04-25T00:13:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.444+0000 7f217be00640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f215c0678c0 0x7f215c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.444+0000 7f217be00640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f215c0678c0 0x7f215c069d80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f217c159110 tx=0x7f2168007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.448+0000 7f2178c00640 1 -- 172.21.15.83:0/3974706624 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2170015f90 con 0x7f217c0b1410 2024-04-25T00:13:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.608+0000 7f2183400640 1 -- 172.21.15.83:0/3974706624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f217c04f5f0 con 0x7f217c0b1410 2024-04-25T00:13:50.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.872+0000 7f2178c00640 1 -- 172.21.15.83:0/3974706624 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v68) ==== 138+0+0 (secure 0 0 0) 0x7f21700f0890 con 0x7f217c0b1410 2024-04-25T00:13:50.876 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-25T00:13:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.876+0000 7f2183400640 1 -- 172.21.15.83:0/3974706624 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f215c0678c0 msgr2=0x7f215c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.876+0000 7f2183400640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f215c0678c0 0x7f215c069d80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f217c159110 tx=0x7f2168007df0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.876+0000 7f2183400640 1 -- 172.21.15.83:0/3974706624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c0b1410 msgr2=0x7f217c0c5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.876+0000 7f2183400640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c0b1410 0x7f217c0c5880 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f217001a930 tx=0x7f2170019020 comp rx=0 tx=0).stop 2024-04-25T00:13:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.876+0000 7f2183400640 1 -- 172.21.15.83:0/3974706624 shutdown_connections 2024-04-25T00:13:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.876+0000 7f2183400640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f215c0678c0 0x7f215c069d80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.876+0000 7f2183400640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f217c15f5d0 0x7f217c0c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.876+0000 7f2183400640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c0b1fa0 0x7f217c167620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.876+0000 7f2183400640 1 --2- 172.21.15.83:0/3974706624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c0b1410 0x7f217c0c5880 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.876+0000 7f2183400640 1 -- 172.21.15.83:0/3974706624 >> 172.21.15.83:0/3974706624 conn(0x7f217c0d1d50 msgr2=0x7f217c15d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.876+0000 7f2183400640 1 -- 172.21.15.83:0/3974706624 shutdown_connections 2024-04-25T00:13:50.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:50.876+0000 7f2183400640 1 -- 172.21.15.83:0/3974706624 wait complete. 2024-04-25T00:13:50.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:13:51.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a68200640 1 Processor -- start 2024-04-25T00:13:51.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a68200640 1 -- start start 2024-04-25T00:13:51.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a68200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5a6006ebd0 0x7f5a6015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:51.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a68200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a6015c560 con 0x7f5a601551e0 2024-04-25T00:13:51.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a68200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a60072460 con 0x7f5a601555c0 2024-04-25T00:13:51.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a68200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5a6015c7d0 con 0x7f5a6006ebd0 2024-04-25T00:13:51.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a65800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5a601551e0 0x7f5a60071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41674/0 (socket says 172.21.15.83:41674) 2024-04-25T00:13:51.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a65800640 1 -- 172.21.15.83:0/2228566857 learned_addr learned my addr 172.21.15.83:0/2228566857 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:51.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a66200640 1 --2- 172.21.15.83:0/2228566857 >> v2:172.21.15.86:3300/0 conn(0x7f5a6006ebd0 0x7f5a6015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:51.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a5fe00640 1 -- 172.21.15.83:0/2228566857 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2993456390 0 0) 0x7f5a6015c560 con 0x7f5a601551e0 2024-04-25T00:13:51.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a5fe00640 1 -- 172.21.15.83:0/2228566857 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a48003530 con 0x7f5a601551e0 2024-04-25T00:13:51.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a5fe00640 1 -- 172.21.15.83:0/2228566857 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4218149064 0 0) 0x7f5a60072460 con 0x7f5a601555c0 2024-04-25T00:13:51.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a5fe00640 1 -- 172.21.15.83:0/2228566857 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a6015c560 con 0x7f5a601555c0 2024-04-25T00:13:51.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a66200640 1 -- 172.21.15.83:0/2228566857 >> v1:172.21.15.86:6789/0 conn(0x7f5a601555c0 legacy=0x7f5a6006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:51.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a5fe00640 1 -- 172.21.15.83:0/2228566857 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1711931238 0 0) 0x7f5a48003530 con 0x7f5a601551e0 2024-04-25T00:13:51.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a66200640 1 -- 172.21.15.83:0/2228566857 >> v1:172.21.15.83:6789/0 conn(0x7f5a601551e0 legacy=0x7f5a60071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:51.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.044+0000 7f5a66200640 1 -- 172.21.15.83:0/2228566857 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6015cab0 con 0x7f5a6006ebd0 2024-04-25T00:13:51.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.048+0000 7f5a66200640 1 --2- 172.21.15.83:0/2228566857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a6006ebd0 0x7f5a6015bea0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009dc0 tx=0x7f5a5801c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9c95426ba461aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:51.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.048+0000 7f5a5fe00640 1 -- 172.21.15.83:0/2228566857 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a580b2070 con 0x7f5a6006ebd0 2024-04-25T00:13:51.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.048+0000 7f5a5fe00640 1 -- 172.21.15.83:0/2228566857 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a580a5780 con 0x7f5a6006ebd0 2024-04-25T00:13:51.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.048+0000 7f5a5fe00640 1 -- 172.21.15.83:0/2228566857 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a580a5b50 con 0x7f5a6006ebd0 2024-04-25T00:13:51.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.048+0000 7f5a68200640 1 -- 172.21.15.83:0/2228566857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a6006ebd0 msgr2=0x7f5a6015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:51.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.048+0000 7f5a68200640 1 --2- 172.21.15.83:0/2228566857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a6006ebd0 0x7f5a6015bea0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009dc0 tx=0x7f5a5801c8c0 comp rx=0 tx=0).stop 2024-04-25T00:13:51.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.048+0000 7f5a68200640 1 -- 172.21.15.83:0/2228566857 shutdown_connections 2024-04-25T00:13:51.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.048+0000 7f5a68200640 1 --2- 172.21.15.83:0/2228566857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a6006ebd0 0x7f5a6015bea0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:51.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.048+0000 7f5a68200640 1 -- 172.21.15.83:0/2228566857 >> 172.21.15.83:0/2228566857 conn(0x7f5a600b1830 msgr2=0x7f5a600b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:51.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.048+0000 7f5a68200640 1 -- 172.21.15.83:0/2228566857 shutdown_connections 2024-04-25T00:13:51.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.048+0000 7f5a68200640 1 -- 172.21.15.83:0/2228566857 wait complete. 2024-04-25T00:13:51.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a68200640 1 Processor -- start 2024-04-25T00:13:51.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a68200640 1 -- start start 2024-04-25T00:13:51.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a68200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a6006ebd0 0x7f5a601437a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:51.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a68200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a601551e0 0x7f5a6014b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:51.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a68200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a601555c0 0x7f5a60147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:51.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a68200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6006d7c0 con 0x7f5a601551e0 2024-04-25T00:13:51.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a68200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6006d640 con 0x7f5a601555c0 2024-04-25T00:13:51.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a68200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5a6006d940 con 0x7f5a6006ebd0 2024-04-25T00:13:51.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a64e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a601551e0 0x7f5a6014b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:51.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a64e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a601551e0 0x7f5a6014b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44140/0 (socket says 172.21.15.83:44140) 2024-04-25T00:13:51.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a64e00640 1 -- 172.21.15.83:0/134948118 learned_addr learned my addr 172.21.15.83:0/134948118 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:51.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a66200640 1 --2- 172.21.15.83:0/134948118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a601555c0 0x7f5a60147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:51.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a65800640 1 --2- 172.21.15.83:0/134948118 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a6006ebd0 0x7f5a601437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:51.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a64e00640 1 -- 172.21.15.83:0/134948118 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a6006ebd0 msgr2=0x7f5a601437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:51.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a64e00640 1 --2- 172.21.15.83:0/134948118 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a6006ebd0 0x7f5a601437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:51.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a64e00640 1 -- 172.21.15.83:0/134948118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a601555c0 msgr2=0x7f5a60147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:51.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a64e00640 1 --2- 172.21.15.83:0/134948118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a601555c0 0x7f5a60147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:51.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a64e00640 1 -- 172.21.15.83:0/134948118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a60143ce0 con 0x7f5a601551e0 2024-04-25T00:13:51.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a64e00640 1 --2- 172.21.15.83:0/134948118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a601551e0 0x7f5a6014b3c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c460 tx=0x7f5a4c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:51.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a5d600640 1 -- 172.21.15.83:0/134948118 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c017070 con 0x7f5a601551e0 2024-04-25T00:13:51.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a5d600640 1 -- 172.21.15.83:0/134948118 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a4c0052f0 con 0x7f5a601551e0 2024-04-25T00:13:51.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a68200640 1 -- 172.21.15.83:0/134948118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6014bd30 con 0x7f5a601551e0 2024-04-25T00:13:51.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a5d600640 1 -- 172.21.15.83:0/134948118 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c003840 con 0x7f5a601551e0 2024-04-25T00:13:51.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a68200640 1 -- 172.21.15.83:0/134948118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6014c2f0 con 0x7f5a601551e0 2024-04-25T00:13:51.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.052+0000 7f5a68200640 1 -- 172.21.15.83:0/134948118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a6004f510 con 0x7f5a601551e0 2024-04-25T00:13:51.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.056+0000 7f5a5d600640 1 -- 172.21.15.83:0/134948118 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5a4c01a070 con 0x7f5a601551e0 2024-04-25T00:13:51.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.056+0000 7f5a5d600640 1 --2- 172.21.15.83:0/134948118 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5a400678c0 0x7f5a40069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:51.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.056+0000 7f5a5d600640 1 -- 172.21.15.83:0/134948118 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5619+0+0 (secure 0 0 0) 0x7f5a4c0850e0 con 0x7f5a601551e0 2024-04-25T00:13:51.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.056+0000 7f5a65800640 1 --2- 172.21.15.83:0/134948118 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5a400678c0 0x7f5a40069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:51.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.056+0000 7f5a65800640 1 --2- 172.21.15.83:0/134948118 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5a400678c0 0x7f5a40069d80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5a54000fc0 tx=0x7f5a540078a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:51.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.060+0000 7f5a5d600640 1 -- 172.21.15.83:0/134948118 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5a4c055bd0 con 0x7f5a601551e0 2024-04-25T00:13:51.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.220+0000 7f5a68200640 1 -- 172.21.15.83:0/134948118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5a600b0270 con 0x7f5a601551e0 2024-04-25T00:13:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.880+0000 7f5a5d600640 1 -- 172.21.15.83:0/134948118 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v69) ==== 170+0+0 (secure 0 0 0) 0x7f5a4c0554d0 con 0x7f5a601551e0 2024-04-25T00:13:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-25T00:13:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.884+0000 7f5a68200640 1 -- 172.21.15.83:0/134948118 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5a400678c0 msgr2=0x7f5a40069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.884+0000 7f5a68200640 1 --2- 172.21.15.83:0/134948118 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5a400678c0 0x7f5a40069d80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5a54000fc0 tx=0x7f5a540078a0 comp rx=0 tx=0).stop 2024-04-25T00:13:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.884+0000 7f5a68200640 1 -- 172.21.15.83:0/134948118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a601551e0 msgr2=0x7f5a6014b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.884+0000 7f5a68200640 1 --2- 172.21.15.83:0/134948118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a601551e0 0x7f5a6014b3c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c460 tx=0x7f5a4c00c920 comp rx=0 tx=0).stop 2024-04-25T00:13:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.884+0000 7f5a68200640 1 -- 172.21.15.83:0/134948118 shutdown_connections 2024-04-25T00:13:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.884+0000 7f5a68200640 1 --2- 172.21.15.83:0/134948118 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5a400678c0 0x7f5a40069d80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.884+0000 7f5a68200640 1 --2- 172.21.15.83:0/134948118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a601555c0 0x7f5a60147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.884+0000 7f5a68200640 1 --2- 172.21.15.83:0/134948118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a601551e0 0x7f5a6014b3c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.884+0000 7f5a68200640 1 --2- 172.21.15.83:0/134948118 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a6006ebd0 0x7f5a601437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.884+0000 7f5a68200640 1 -- 172.21.15.83:0/134948118 >> 172.21.15.83:0/134948118 conn(0x7f5a600b1830 msgr2=0x7f5a600afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.884+0000 7f5a68200640 1 -- 172.21.15.83:0/134948118 shutdown_connections 2024-04-25T00:13:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:51.884+0000 7f5a68200640 1 -- 172.21.15.83:0/134948118 wait complete. 2024-04-25T00:13:51.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:13:52.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.064+0000 7fe9a1400640 1 Processor -- start 2024-04-25T00:13:52.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.064+0000 7fe9a1400640 1 -- start start 2024-04-25T00:13:52.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.064+0000 7fe9a1400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99c1551e0 0x7fe99c15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:52.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.064+0000 7fe9a1400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe99c1555c0 0x7fe99c071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:52.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe9a1400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe99c15c560 con 0x7fe99c06e4c0 2024-04-25T00:13:52.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe9a1400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe99c072460 con 0x7fe99c1555c0 2024-04-25T00:13:52.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe9a1400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe99c15c7d0 con 0x7fe99c1551e0 2024-04-25T00:13:52.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe999e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe99c1555c0 0x7fe99c071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:52.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe99b200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe99c06e4c0 0x7fe99c15e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41680/0 (socket says 172.21.15.83:41680) 2024-04-25T00:13:52.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe99b200640 1 -- 172.21.15.83:0/2154064668 learned_addr learned my addr 172.21.15.83:0/2154064668 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:52.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe99a800640 1 --2- 172.21.15.83:0/2154064668 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99c1551e0 0x7fe99c15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:52.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe999e00640 1 -- 172.21.15.83:0/2154064668 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99c1551e0 msgr2=0x7fe99c15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:52.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe999e00640 1 --2- 172.21.15.83:0/2154064668 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99c1551e0 0x7fe99c15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:52.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe999e00640 1 -- 172.21.15.83:0/2154064668 >> v1:172.21.15.83:6789/0 conn(0x7fe99c06e4c0 legacy=0x7fe99c15e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:52.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe999e00640 1 -- 172.21.15.83:0/2154064668 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99c15cab0 con 0x7fe99c1555c0 2024-04-25T00:13:52.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe999400640 1 -- 172.21.15.83:0/2154064668 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 858809956 0 0) 0x7fe99c15c560 con 0x7fe99c06e4c0 2024-04-25T00:13:52.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe999e00640 1 --2- 172.21.15.83:0/2154064668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe99c1555c0 0x7fe99c071b70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe9880097c0 tx=0x7fe98801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ceb45f2c0cffdbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:52.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe999400640 1 -- 172.21.15.83:0/2154064668 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9880b1070 con 0x7fe99c1555c0 2024-04-25T00:13:52.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe999400640 1 -- 172.21.15.83:0/2154064668 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9880ac620 con 0x7fe99c1555c0 2024-04-25T00:13:52.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.068+0000 7fe999400640 1 -- 172.21.15.83:0/2154064668 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9880ac9f0 con 0x7fe99c1555c0 2024-04-25T00:13:52.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 -- 172.21.15.83:0/2154064668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe99c1555c0 msgr2=0x7fe99c071b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:52.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 --2- 172.21.15.83:0/2154064668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe99c1555c0 0x7fe99c071b70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe9880097c0 tx=0x7fe98801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:13:52.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 -- 172.21.15.83:0/2154064668 shutdown_connections 2024-04-25T00:13:52.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 --2- 172.21.15.83:0/2154064668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe99c1555c0 0x7fe99c071b70 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:52.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 --2- 172.21.15.83:0/2154064668 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99c1551e0 0x7fe99c15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:52.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 -- 172.21.15.83:0/2154064668 >> 172.21.15.83:0/2154064668 conn(0x7fe99c0b1830 msgr2=0x7fe99c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:52.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 -- 172.21.15.83:0/2154064668 shutdown_connections 2024-04-25T00:13:52.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 -- 172.21.15.83:0/2154064668 wait complete. 2024-04-25T00:13:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 Processor -- start 2024-04-25T00:13:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 -- start start 2024-04-25T00:13:52.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe99c06e4c0 0x7fe99c14ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:52.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99c1551e0 0x7fe99c156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:52.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe99c1555c0 0x7fe99c14bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:52.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c167e10 con 0x7fe99c06e4c0 2024-04-25T00:13:52.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c167c90 con 0x7fe99c1555c0 2024-04-25T00:13:52.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe9a1400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe99c167f90 con 0x7fe99c1551e0 2024-04-25T00:13:52.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe99a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe99c06e4c0 0x7fe99c14ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:52.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe99a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe99c06e4c0 0x7fe99c14ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44162/0 (socket says 172.21.15.83:44162) 2024-04-25T00:13:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe99a800640 1 -- 172.21.15.83:0/2060393202 learned_addr learned my addr 172.21.15.83:0/2060393202 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe99b200640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe99c1555c0 0x7fe99c14bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe999e00640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99c1551e0 0x7fe99c156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe99a800640 1 -- 172.21.15.83:0/2060393202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99c1551e0 msgr2=0x7fe99c156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe99a800640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99c1551e0 0x7fe99c156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe99a800640 1 -- 172.21.15.83:0/2060393202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe99c1555c0 msgr2=0x7fe99c14bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe99a800640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe99c1555c0 0x7fe99c14bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe99a800640 1 -- 172.21.15.83:0/2060393202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99c14dc30 con 0x7fe99c06e4c0 2024-04-25T00:13:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.072+0000 7fe99b200640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe99c1555c0 0x7fe99c14bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.076+0000 7fe99a800640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe99c06e4c0 0x7fe99c14ecd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe98c007ec0 tx=0x7fe98c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.076+0000 7fe992a00640 1 -- 172.21.15.83:0/2060393202 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c014620 con 0x7fe99c06e4c0 2024-04-25T00:13:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.076+0000 7fe992a00640 1 -- 172.21.15.83:0/2060393202 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe98c019070 con 0x7fe99c06e4c0 2024-04-25T00:13:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.076+0000 7fe992a00640 1 -- 172.21.15.83:0/2060393202 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c014a20 con 0x7fe99c06e4c0 2024-04-25T00:13:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.076+0000 7fe9a1400640 1 -- 172.21.15.83:0/2060393202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99c14df10 con 0x7fe99c06e4c0 2024-04-25T00:13:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.076+0000 7fe9a1400640 1 -- 172.21.15.83:0/2060393202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe99c14e4d0 con 0x7fe99c06e4c0 2024-04-25T00:13:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.076+0000 7fe9a1400640 1 -- 172.21.15.83:0/2060393202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe99c04f510 con 0x7fe99c06e4c0 2024-04-25T00:13:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.076+0000 7fe992a00640 1 -- 172.21.15.83:0/2060393202 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe98c014bc0 con 0x7fe99c06e4c0 2024-04-25T00:13:52.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.080+0000 7fe992a00640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe97c0678c0 0x7fe97c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:52.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.080+0000 7fe992a00640 1 -- 172.21.15.83:0/2060393202 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6001+0+0 (secure 0 0 0) 0x7fe98c087690 con 0x7fe99c06e4c0 2024-04-25T00:13:52.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.080+0000 7fe999e00640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe97c0678c0 0x7fe97c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:52.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.080+0000 7fe999e00640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe97c0678c0 0x7fe97c069d80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe9880ae000 tx=0x7fe9880a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:52.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.084+0000 7fe992a00640 1 -- 172.21.15.83:0/2060393202 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe98c057ff0 con 0x7fe99c06e4c0 2024-04-25T00:13:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.268+0000 7fe9a1400640 1 -- 172.21.15.83:0/2060393202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe99c06ce30 con 0x7fe99c06e4c0 2024-04-25T00:13:52.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.892+0000 7fe992a00640 1 -- 172.21.15.83:0/2060393202 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v40) ==== 174+0+0 (secure 0 0 0) 0x7fe98c0578f0 con 0x7fe99c06e4c0 2024-04-25T00:13:52.897 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T00:13:52.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.900+0000 7fe9a1400640 1 -- 172.21.15.83:0/2060393202 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe97c0678c0 msgr2=0x7fe97c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:52.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.900+0000 7fe9a1400640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe97c0678c0 0x7fe97c069d80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe9880ae000 tx=0x7fe9880a7000 comp rx=0 tx=0).stop 2024-04-25T00:13:52.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.900+0000 7fe9a1400640 1 -- 172.21.15.83:0/2060393202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe99c06e4c0 msgr2=0x7fe99c14ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:52.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.900+0000 7fe9a1400640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe99c06e4c0 0x7fe99c14ecd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe98c007ec0 tx=0x7fe98c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:13:52.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.904+0000 7fe9a1400640 1 -- 172.21.15.83:0/2060393202 shutdown_connections 2024-04-25T00:13:52.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.904+0000 7fe9a1400640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe97c0678c0 0x7fe97c069d80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:52.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.904+0000 7fe9a1400640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99c1551e0 0x7fe99c156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:52.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.904+0000 7fe9a1400640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe99c1555c0 0x7fe99c14bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:52.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.904+0000 7fe9a1400640 1 --2- 172.21.15.83:0/2060393202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe99c06e4c0 0x7fe99c14ecd0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:52.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.904+0000 7fe9a1400640 1 -- 172.21.15.83:0/2060393202 >> 172.21.15.83:0/2060393202 conn(0x7fe99c0b1830 msgr2=0x7fe99c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:52.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.904+0000 7fe9a1400640 1 -- 172.21.15.83:0/2060393202 shutdown_connections 2024-04-25T00:13:52.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:52.904+0000 7fe9a1400640 1 -- 172.21.15.83:0/2060393202 wait complete. 2024-04-25T00:13:52.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:53.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.072+0000 7fe165600640 1 Processor -- start 2024-04-25T00:13:53.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.072+0000 7fe165600640 1 -- start start 2024-04-25T00:13:53.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.072+0000 7fe165600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe1601553e0 0x7fe16006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.072+0000 7fe165600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe16006ef80 con 0x7fe16006ea00 2024-04-25T00:13:53.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.072+0000 7fe165600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe16015b510 con 0x7fe1600b4a70 2024-04-25T00:13:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.072+0000 7fe165600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe16015b7f0 con 0x7fe1601553e0 2024-04-25T00:13:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.072+0000 7fe15ea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe1601553e0 0x7fe16006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.072+0000 7fe156000640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe16006ea00 0x7fe1600b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41694/0 (socket says 172.21.15.83:41694) 2024-04-25T00:13:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.072+0000 7fe156000640 1 -- 172.21.15.83:0/3632789879 learned_addr learned my addr 172.21.15.83:0/3632789879 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.072+0000 7fe15ea00640 1 -- 172.21.15.83:0/3632789879 >> v1:172.21.15.86:6789/0 conn(0x7fe1600b4a70 legacy=0x7fe1600b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.072+0000 7fe15ea00640 1 -- 172.21.15.83:0/3632789879 >> v1:172.21.15.83:6789/0 conn(0x7fe16006ea00 legacy=0x7fe1600b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.072+0000 7fe15e000640 1 -- 172.21.15.83:0/3632789879 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3565200496 0 0) 0x7fe16006ef80 con 0x7fe16006ea00 2024-04-25T00:13:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.072+0000 7fe15ea00640 1 -- 172.21.15.83:0/3632789879 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16015bad0 con 0x7fe1601553e0 2024-04-25T00:13:53.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.076+0000 7fe15ea00640 1 --2- 172.21.15.83:0/3632789879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1601553e0 0x7fe16006e4c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe1500097c0 tx=0x7fe15001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f653eea0276a18e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:53.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.076+0000 7fe15e000640 1 -- 172.21.15.83:0/3632789879 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1500b0070 con 0x7fe1601553e0 2024-04-25T00:13:53.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.076+0000 7fe15e000640 1 -- 172.21.15.83:0/3632789879 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1500ab600 con 0x7fe1601553e0 2024-04-25T00:13:53.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.076+0000 7fe15e000640 1 -- 172.21.15.83:0/3632789879 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1500ab9d0 con 0x7fe1601553e0 2024-04-25T00:13:53.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.076+0000 7fe165600640 1 -- 172.21.15.83:0/3632789879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1601553e0 msgr2=0x7fe16006e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.076+0000 7fe165600640 1 --2- 172.21.15.83:0/3632789879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1601553e0 0x7fe16006e4c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe1500097c0 tx=0x7fe15001c3d0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.076+0000 7fe165600640 1 -- 172.21.15.83:0/3632789879 shutdown_connections 2024-04-25T00:13:53.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.076+0000 7fe165600640 1 --2- 172.21.15.83:0/3632789879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1601553e0 0x7fe16006e4c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.076+0000 7fe165600640 1 -- 172.21.15.83:0/3632789879 >> 172.21.15.83:0/3632789879 conn(0x7fe1600b1830 msgr2=0x7fe1600b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:53.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.076+0000 7fe165600640 1 -- 172.21.15.83:0/3632789879 shutdown_connections 2024-04-25T00:13:53.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.076+0000 7fe165600640 1 -- 172.21.15.83:0/3632789879 wait complete. 2024-04-25T00:13:53.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe165600640 1 Processor -- start 2024-04-25T00:13:53.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe165600640 1 -- start start 2024-04-25T00:13:53.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe165600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe16006ea00 0x7fe160144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe165600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1600b4a70 0x7fe160153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe165600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1601553e0 0x7fe160154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe165600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe16006d530 con 0x7fe1600b4a70 2024-04-25T00:13:53.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe165600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe16006d3b0 con 0x7fe1601553e0 2024-04-25T00:13:53.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe165600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe16006d6b0 con 0x7fe16006ea00 2024-04-25T00:13:53.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe156000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1600b4a70 0x7fe160153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe156000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1600b4a70 0x7fe160153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44174/0 (socket says 172.21.15.83:44174) 2024-04-25T00:13:53.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe156000640 1 -- 172.21.15.83:0/2026812673 learned_addr learned my addr 172.21.15.83:0/2026812673 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:53.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe15ea00640 1 --2- 172.21.15.83:0/2026812673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe16006ea00 0x7fe160144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe15f400640 1 --2- 172.21.15.83:0/2026812673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1601553e0 0x7fe160154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe156000640 1 -- 172.21.15.83:0/2026812673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe16006ea00 msgr2=0x7fe160144790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe156000640 1 --2- 172.21.15.83:0/2026812673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe16006ea00 0x7fe160144790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe156000640 1 -- 172.21.15.83:0/2026812673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1601553e0 msgr2=0x7fe160154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe156000640 1 --2- 172.21.15.83:0/2026812673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1601553e0 0x7fe160154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe156000640 1 -- 172.21.15.83:0/2026812673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16015a720 con 0x7fe1600b4a70 2024-04-25T00:13:53.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe156000640 1 --2- 172.21.15.83:0/2026812673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1600b4a70 0x7fe160153f20 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe1480011f0 tx=0x7fe14800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:53.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe157400640 1 -- 172.21.15.83:0/2026812673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148018070 con 0x7fe1600b4a70 2024-04-25T00:13:53.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe157400640 1 -- 172.21.15.83:0/2026812673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1480074e0 con 0x7fe1600b4a70 2024-04-25T00:13:53.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe157400640 1 -- 172.21.15.83:0/2026812673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148004a10 con 0x7fe1600b4a70 2024-04-25T00:13:53.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe165600640 1 -- 172.21.15.83:0/2026812673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16015aa00 con 0x7fe1600b4a70 2024-04-25T00:13:53.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.080+0000 7fe165600640 1 -- 172.21.15.83:0/2026812673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16015afc0 con 0x7fe1600b4a70 2024-04-25T00:13:53.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.084+0000 7fe165600640 1 -- 172.21.15.83:0/2026812673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe16015b1c0 con 0x7fe1600b4a70 2024-04-25T00:13:53.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.084+0000 7fe157400640 1 -- 172.21.15.83:0/2026812673 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe148004bb0 con 0x7fe1600b4a70 2024-04-25T00:13:53.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.084+0000 7fe157400640 1 --2- 172.21.15.83:0/2026812673 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe13c0678c0 0x7fe13c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.084+0000 7fe157400640 1 -- 172.21.15.83:0/2026812673 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fe148085280 con 0x7fe1600b4a70 2024-04-25T00:13:53.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.084+0000 7fe15ea00640 1 --2- 172.21.15.83:0/2026812673 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe13c0678c0 0x7fe13c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.084+0000 7fe15ea00640 1 --2- 172.21.15.83:0/2026812673 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe13c0678c0 0x7fe13c069d80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe1500ad000 tx=0x7fe1500a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:53.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.088+0000 7fe157400640 1 -- 172.21.15.83:0/2026812673 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe16015b1c0 con 0x7fe1600b4a70 2024-04-25T00:13:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.244+0000 7fe165600640 1 -- 172.21.15.83:0/2026812673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1600737f0 con 0x7fe1600b4a70 2024-04-25T00:13:53.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.244+0000 7fe157400640 1 -- 172.21.15.83:0/2026812673 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16874 (secure 0 0 0) 0x7fe1480554b0 con 0x7fe1600b4a70 2024-04-25T00:13:53.249 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:53.249 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:52.273360+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:50.655187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:51.268160+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.248+0000 7fe165600640 1 -- 172.21.15.83:0/2026812673 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe13c0678c0 msgr2=0x7fe13c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.248+0000 7fe165600640 1 --2- 172.21.15.83:0/2026812673 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe13c0678c0 0x7fe13c069d80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe1500ad000 tx=0x7fe1500a6000 comp rx=0 tx=0).stop 2024-04-25T00:13:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.248+0000 7fe165600640 1 -- 172.21.15.83:0/2026812673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1600b4a70 msgr2=0x7fe160153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.248+0000 7fe165600640 1 --2- 172.21.15.83:0/2026812673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1600b4a70 0x7fe160153f20 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe1480011f0 tx=0x7fe14800c9b0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.248+0000 7fe165600640 1 -- 172.21.15.83:0/2026812673 shutdown_connections 2024-04-25T00:13:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.248+0000 7fe165600640 1 --2- 172.21.15.83:0/2026812673 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe13c0678c0 0x7fe13c069d80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.248+0000 7fe165600640 1 --2- 172.21.15.83:0/2026812673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1601553e0 0x7fe160154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.248+0000 7fe165600640 1 --2- 172.21.15.83:0/2026812673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1600b4a70 0x7fe160153f20 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.248+0000 7fe165600640 1 --2- 172.21.15.83:0/2026812673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe16006ea00 0x7fe160144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.248+0000 7fe165600640 1 -- 172.21.15.83:0/2026812673 >> 172.21.15.83:0/2026812673 conn(0x7fe1600b1830 msgr2=0x7fe16015c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.248+0000 7fe165600640 1 -- 172.21.15.83:0/2026812673 shutdown_connections 2024-04-25T00:13:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.248+0000 7fe165600640 1 -- 172.21.15.83:0/2026812673 wait complete. 2024-04-25T00:13:53.269 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:53.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.400+0000 7f3348400640 1 Processor -- start 2024-04-25T00:13:53.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.400+0000 7f3348400640 1 -- start start 2024-04-25T00:13:53.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.400+0000 7f3348400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f33401553e0 0x7f334006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.400+0000 7f3348400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f334006ea00 0x7f33400b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.400+0000 7f3348400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33400b3b40 con 0x7f33400b48a0 2024-04-25T00:13:53.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.400+0000 7f3348400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f334006ef80 con 0x7f334006ea00 2024-04-25T00:13:53.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.400+0000 7f3348400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f334015b510 con 0x7f33401553e0 2024-04-25T00:13:53.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.400+0000 7f3345a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f33401553e0 0x7f334006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.404+0000 7f3345a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f33401553e0 0x7f334006e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:52726/0 (socket says 172.21.15.83:52726) 2024-04-25T00:13:53.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.404+0000 7f3345000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f334006ea00 0x7f33400b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.404+0000 7f3346400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f33400b48a0 0x7f334015e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:55406/0 (socket says 172.21.15.83:55406) 2024-04-25T00:13:53.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.404+0000 7f3345a00640 1 -- 172.21.15.83:0/402012126 learned_addr learned my addr 172.21.15.83:0/402012126 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:53.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.404+0000 7f3345000640 1 -- 172.21.15.83:0/402012126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f33401553e0 msgr2=0x7f334006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:53.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.404+0000 7f3345000640 1 --2- 172.21.15.83:0/402012126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f33401553e0 0x7f334006e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.404+0000 7f3345000640 1 -- 172.21.15.83:0/402012126 >> v1:172.21.15.86:6789/0 conn(0x7f33400b48a0 legacy=0x7f334015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.404+0000 7f3345000640 1 -- 172.21.15.83:0/402012126 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334015b7f0 con 0x7f334006ea00 2024-04-25T00:13:53.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.404+0000 7f3345a00640 1 --2- 172.21.15.83:0/402012126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f33401553e0 0x7f334006e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:53.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.404+0000 7f3345000640 1 --2- 172.21.15.83:0/402012126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f334006ea00 0x7f33400b4360 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f332c0097c0 tx=0x7f332c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2745b32e84d74286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:53.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.404+0000 7f333fe00640 1 -- 172.21.15.83:0/402012126 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c0b0070 con 0x7f334006ea00 2024-04-25T00:13:53.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.404+0000 7f333fe00640 1 -- 172.21.15.83:0/402012126 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f332c0ab600 con 0x7f334006ea00 2024-04-25T00:13:53.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.404+0000 7f333fe00640 1 -- 172.21.15.83:0/402012126 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c0ab9d0 con 0x7f334006ea00 2024-04-25T00:13:53.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 -- 172.21.15.83:0/402012126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f334006ea00 msgr2=0x7f33400b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 --2- 172.21.15.83:0/402012126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f334006ea00 0x7f33400b4360 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f332c0097c0 tx=0x7f332c01c410 comp rx=0 tx=0).stop 2024-04-25T00:13:53.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 -- 172.21.15.83:0/402012126 shutdown_connections 2024-04-25T00:13:53.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 --2- 172.21.15.83:0/402012126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f334006ea00 0x7f33400b4360 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 --2- 172.21.15.83:0/402012126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f33401553e0 0x7f334006e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 -- 172.21.15.83:0/402012126 >> 172.21.15.83:0/402012126 conn(0x7f33400b1830 msgr2=0x7f33400b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:53.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 -- 172.21.15.83:0/402012126 shutdown_connections 2024-04-25T00:13:53.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 -- 172.21.15.83:0/402012126 wait complete. 2024-04-25T00:13:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 Processor -- start 2024-04-25T00:13:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 -- start start 2024-04-25T00:13:53.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f334006ea00 0x7f3340144570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33400b48a0 0x7f334006d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33401553e0 0x7f334006de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3340155940 con 0x7f33401553e0 2024-04-25T00:13:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33401557c0 con 0x7f33400b48a0 2024-04-25T00:13:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3348400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3340167d90 con 0x7f334006ea00 2024-04-25T00:13:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3346400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33401553e0 0x7f334006de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3346400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33401553e0 0x7f334006de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44176/0 (socket says 172.21.15.83:44176) 2024-04-25T00:13:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3346400640 1 -- 172.21.15.83:0/2656297636 learned_addr learned my addr 172.21.15.83:0/2656297636 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3345a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f334006ea00 0x7f3340144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3345000640 1 --2- 172.21.15.83:0/2656297636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33400b48a0 0x7f334006d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3346400640 1 -- 172.21.15.83:0/2656297636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f334006ea00 msgr2=0x7f3340144570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3346400640 1 --2- 172.21.15.83:0/2656297636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f334006ea00 0x7f3340144570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3346400640 1 -- 172.21.15.83:0/2656297636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33400b48a0 msgr2=0x7f334006d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3346400640 1 --2- 172.21.15.83:0/2656297636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33400b48a0 0x7f334006d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.408+0000 7f3346400640 1 -- 172.21.15.83:0/2656297636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3340144ab0 con 0x7f33401553e0 2024-04-25T00:13:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.412+0000 7f3346400640 1 --2- 172.21.15.83:0/2656297636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33401553e0 0x7f334006de90 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f333800c730 tx=0x7f333800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.412+0000 7f333d600640 1 -- 172.21.15.83:0/2656297636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338017070 con 0x7f33401553e0 2024-04-25T00:13:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.412+0000 7f333d600640 1 -- 172.21.15.83:0/2656297636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f333800ce40 con 0x7f33401553e0 2024-04-25T00:13:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.412+0000 7f333d600640 1 -- 172.21.15.83:0/2656297636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333800a840 con 0x7f33401553e0 2024-04-25T00:13:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.412+0000 7f3348400640 1 -- 172.21.15.83:0/2656297636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3340164b70 con 0x7f33401553e0 2024-04-25T00:13:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.412+0000 7f3348400640 1 -- 172.21.15.83:0/2656297636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334014f230 con 0x7f33401553e0 2024-04-25T00:13:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.412+0000 7f3348400640 1 -- 172.21.15.83:0/2656297636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33400737f0 con 0x7f33401553e0 2024-04-25T00:13:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.412+0000 7f333d600640 1 -- 172.21.15.83:0/2656297636 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f333801a070 con 0x7f33401553e0 2024-04-25T00:13:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.412+0000 7f333d600640 1 --2- 172.21.15.83:0/2656297636 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f33200678c0 0x7f3320069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.412+0000 7f333d600640 1 -- 172.21.15.83:0/2656297636 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f3338085350 con 0x7f33401553e0 2024-04-25T00:13:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.412+0000 7f3345a00640 1 --2- 172.21.15.83:0/2656297636 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f33200678c0 0x7f3320069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.416+0000 7f3345a00640 1 --2- 172.21.15.83:0/2656297636 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f33200678c0 0x7f3320069d80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3334006cb0 tx=0x7f3334009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.416+0000 7f333d600640 1 -- 172.21.15.83:0/2656297636 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f33380556a0 con 0x7f33401553e0 2024-04-25T00:13:53.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.572+0000 7f3348400640 1 -- 172.21.15.83:0/2656297636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f334004f5f0 con 0x7f33401553e0 2024-04-25T00:13:53.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.576+0000 7f333d600640 1 -- 172.21.15.83:0/2656297636 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16874 (secure 0 0 0) 0x7f33380599a0 con 0x7f33401553e0 2024-04-25T00:13:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:53.578 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:52.273360+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:50.655187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:51.268160+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:53.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.576+0000 7f3348400640 1 -- 172.21.15.83:0/2656297636 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f33200678c0 msgr2=0x7f3320069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.576+0000 7f3348400640 1 --2- 172.21.15.83:0/2656297636 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f33200678c0 0x7f3320069d80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3334006cb0 tx=0x7f3334009240 comp rx=0 tx=0).stop 2024-04-25T00:13:53.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.576+0000 7f3348400640 1 -- 172.21.15.83:0/2656297636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33401553e0 msgr2=0x7f334006de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.576+0000 7f3348400640 1 --2- 172.21.15.83:0/2656297636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33401553e0 0x7f334006de90 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f333800c730 tx=0x7f333800cbf0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.576+0000 7f3348400640 1 -- 172.21.15.83:0/2656297636 shutdown_connections 2024-04-25T00:13:53.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.576+0000 7f3348400640 1 --2- 172.21.15.83:0/2656297636 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f33200678c0 0x7f3320069d80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.576+0000 7f3348400640 1 --2- 172.21.15.83:0/2656297636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33401553e0 0x7f334006de90 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.576+0000 7f3348400640 1 --2- 172.21.15.83:0/2656297636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33400b48a0 0x7f334006d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.576+0000 7f3348400640 1 --2- 172.21.15.83:0/2656297636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f334006ea00 0x7f3340144570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.576+0000 7f3348400640 1 -- 172.21.15.83:0/2656297636 >> 172.21.15.83:0/2656297636 conn(0x7f33400b1830 msgr2=0x7f334014e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:53.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.576+0000 7f3348400640 1 -- 172.21.15.83:0/2656297636 shutdown_connections 2024-04-25T00:13:53.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.576+0000 7f3348400640 1 -- 172.21.15.83:0/2656297636 wait complete. 2024-04-25T00:13:53.599 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:13:53.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.732+0000 7ff7dd600640 1 Processor -- start 2024-04-25T00:13:53.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.732+0000 7ff7dd600640 1 -- start start 2024-04-25T00:13:53.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7dd600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7d81551e0 0x7ff7d815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7dd600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff7d81555c0 0x7ff7d8071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7dd600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff7d806e4c0 0x7ff7d806e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7dd600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff7d806ee70 con 0x7ff7d806e4c0 2024-04-25T00:13:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7dd600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff7d815c560 con 0x7ff7d81555c0 2024-04-25T00:13:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7dd600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d8072460 con 0x7ff7d81551e0 2024-04-25T00:13:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7d7400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff7d806e4c0 0x7ff7d806e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7d7400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff7d806e4c0 0x7ff7d806e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44188/0 (socket says 172.21.15.83:44188) 2024-04-25T00:13:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7d7400640 1 -- 172.21.15.83:0/4092827990 learned_addr learned my addr 172.21.15.83:0/4092827990 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7d6a00640 1 --2- 172.21.15.83:0/4092827990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7d81551e0 0x7ff7d815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7ce000640 1 --2- 172.21.15.83:0/4092827990 >> v2:172.21.15.86:3300/0 conn(0x7ff7d81555c0 0x7ff7d8071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7d7400640 1 -- 172.21.15.83:0/4092827990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7d81551e0 msgr2=0x7ff7d815bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7d7400640 1 --2- 172.21.15.83:0/4092827990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7d81551e0 0x7ff7d815bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7d7400640 1 -- 172.21.15.83:0/4092827990 >> v2:172.21.15.86:3300/0 conn(0x7ff7d81555c0 msgr2=0x7ff7d8071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7d7400640 1 --2- 172.21.15.83:0/4092827990 >> v2:172.21.15.86:3300/0 conn(0x7ff7d81555c0 0x7ff7d8071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7d7400640 1 -- 172.21.15.83:0/4092827990 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d815c7d0 con 0x7ff7d806e4c0 2024-04-25T00:13:53.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.736+0000 7ff7ce000640 1 --2- 172.21.15.83:0/4092827990 >> v2:172.21.15.86:3300/0 conn(0x7ff7d81555c0 0x7ff7d8071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:53.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7d7400640 1 --2- 172.21.15.83:0/4092827990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d806e4c0 0x7ff7d806e930 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8070a20 tx=0x7ff7d001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a282bb2edf959ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:53.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7d6000640 1 -- 172.21.15.83:0/4092827990 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d00b1070 con 0x7ff7d806e4c0 2024-04-25T00:13:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7d6000640 1 -- 172.21.15.83:0/4092827990 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7d00ac630 con 0x7ff7d806e4c0 2024-04-25T00:13:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7d6000640 1 -- 172.21.15.83:0/4092827990 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d00a8050 con 0x7ff7d806e4c0 2024-04-25T00:13:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 -- 172.21.15.83:0/4092827990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d806e4c0 msgr2=0x7ff7d806e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 --2- 172.21.15.83:0/4092827990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d806e4c0 0x7ff7d806e930 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8070a20 tx=0x7ff7d001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 -- 172.21.15.83:0/4092827990 shutdown_connections 2024-04-25T00:13:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 --2- 172.21.15.83:0/4092827990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d806e4c0 0x7ff7d806e930 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 --2- 172.21.15.83:0/4092827990 >> v2:172.21.15.86:3300/0 conn(0x7ff7d81555c0 0x7ff7d8071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 --2- 172.21.15.83:0/4092827990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7d81551e0 0x7ff7d815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 -- 172.21.15.83:0/4092827990 >> 172.21.15.83:0/4092827990 conn(0x7ff7d80b1830 msgr2=0x7ff7d80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 -- 172.21.15.83:0/4092827990 shutdown_connections 2024-04-25T00:13:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 -- 172.21.15.83:0/4092827990 wait complete. 2024-04-25T00:13:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 Processor -- start 2024-04-25T00:13:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 -- start start 2024-04-25T00:13:53.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7d806e4c0 0x7ff7d813d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d81551e0 0x7ff7d814c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7d81555c0 0x7ff7d814cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d815f280 con 0x7ff7d81551e0 2024-04-25T00:13:53.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d815f100 con 0x7ff7d81555c0 2024-04-25T00:13:53.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7dd600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d815f400 con 0x7ff7d806e4c0 2024-04-25T00:13:53.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7ce000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d81551e0 0x7ff7d814c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7ce000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d81551e0 0x7ff7d814c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44198/0 (socket says 172.21.15.83:44198) 2024-04-25T00:13:53.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7ce000640 1 -- 172.21.15.83:0/3769097706 learned_addr learned my addr 172.21.15.83:0/3769097706 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:53.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7d6a00640 1 --2- 172.21.15.83:0/3769097706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7d806e4c0 0x7ff7d813d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.740+0000 7ff7d7400640 1 --2- 172.21.15.83:0/3769097706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7d81555c0 0x7ff7d814cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7ce000640 1 -- 172.21.15.83:0/3769097706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7d806e4c0 msgr2=0x7ff7d813d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7ce000640 1 --2- 172.21.15.83:0/3769097706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7d806e4c0 0x7ff7d813d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7ce000640 1 -- 172.21.15.83:0/3769097706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7d81555c0 msgr2=0x7ff7d814cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7ce000640 1 --2- 172.21.15.83:0/3769097706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7d81555c0 0x7ff7d814cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7ce000640 1 -- 172.21.15.83:0/3769097706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d81414b0 con 0x7ff7d81551e0 2024-04-25T00:13:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7ce000640 1 --2- 172.21.15.83:0/3769097706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d81551e0 0x7ff7d814c7d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000d800 tx=0x7ff7c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7cf400640 1 -- 172.21.15.83:0/3769097706 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c000b6c0 con 0x7ff7d81551e0 2024-04-25T00:13:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7cf400640 1 -- 172.21.15.83:0/3769097706 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7c0019070 con 0x7ff7d81551e0 2024-04-25T00:13:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7cf400640 1 -- 172.21.15.83:0/3769097706 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c000baa0 con 0x7ff7d81551e0 2024-04-25T00:13:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7dd600640 1 -- 172.21.15.83:0/3769097706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d8141790 con 0x7ff7d81551e0 2024-04-25T00:13:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7dd600640 1 -- 172.21.15.83:0/3769097706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d8141d50 con 0x7ff7d81551e0 2024-04-25T00:13:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7dd600640 1 -- 172.21.15.83:0/3769097706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d804f510 con 0x7ff7d81551e0 2024-04-25T00:13:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7cf400640 1 -- 172.21.15.83:0/3769097706 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff7c000bc40 con 0x7ff7d81551e0 2024-04-25T00:13:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7cf400640 1 --2- 172.21.15.83:0/3769097706 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff7b40678c0 0x7ff7b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7cf400640 1 -- 172.21.15.83:0/3769097706 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7ff7c0086170 con 0x7ff7d81551e0 2024-04-25T00:13:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.744+0000 7ff7d6a00640 1 --2- 172.21.15.83:0/3769097706 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff7b40678c0 0x7ff7b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.748+0000 7ff7d6a00640 1 --2- 172.21.15.83:0/3769097706 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff7b40678c0 0x7ff7b4069d80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800bfd0 tx=0x7ff7c800f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.748+0000 7ff7cf400640 1 -- 172.21.15.83:0/3769097706 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff7c0056430 con 0x7ff7d81551e0 2024-04-25T00:13:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:53.944+0000 7ff7dd600640 1 -- 172.21.15.83:0/3769097706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff7d80b2610 con 0x7ff7d81551e0 2024-04-25T00:13:54.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T00:13:54.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T00:13:54.877 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-04-25T00:13:54.876+0000 7fda22200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:13:54.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:54.904+0000 7ff7cf400640 1 -- 172.21.15.83:0/3769097706 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) ==== 118+0+0 (secure 0 0 0) 0x7ff7c005a730 con 0x7ff7d81551e0 2024-04-25T00:13:54.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:54.908+0000 7ff7dd600640 1 -- 172.21.15.83:0/3769097706 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff7b40678c0 msgr2=0x7ff7b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:54.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:54.908+0000 7ff7dd600640 1 --2- 172.21.15.83:0/3769097706 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff7b40678c0 0x7ff7b4069d80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800bfd0 tx=0x7ff7c800f040 comp rx=0 tx=0).stop 2024-04-25T00:13:54.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:54.908+0000 7ff7dd600640 1 -- 172.21.15.83:0/3769097706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d81551e0 msgr2=0x7ff7d814c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:54.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:54.908+0000 7ff7dd600640 1 --2- 172.21.15.83:0/3769097706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d81551e0 0x7ff7d814c7d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000d800 tx=0x7ff7c000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:13:54.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:54.908+0000 7ff7dd600640 1 -- 172.21.15.83:0/3769097706 shutdown_connections 2024-04-25T00:13:54.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:54.908+0000 7ff7dd600640 1 --2- 172.21.15.83:0/3769097706 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff7b40678c0 0x7ff7b4069d80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:54.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:54.908+0000 7ff7dd600640 1 --2- 172.21.15.83:0/3769097706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7d81555c0 0x7ff7d814cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:54.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:54.908+0000 7ff7dd600640 1 --2- 172.21.15.83:0/3769097706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d81551e0 0x7ff7d814c7d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:54.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:54.908+0000 7ff7dd600640 1 --2- 172.21.15.83:0/3769097706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7d806e4c0 0x7ff7d813d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:54.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:54.908+0000 7ff7dd600640 1 -- 172.21.15.83:0/3769097706 >> 172.21.15.83:0/3769097706 conn(0x7ff7d80b1830 msgr2=0x7ff7d80afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:54.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:54.908+0000 7ff7dd600640 1 -- 172.21.15.83:0/3769097706 shutdown_connections 2024-04-25T00:13:54.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:54.908+0000 7ff7dd600640 1 -- 172.21.15.83:0/3769097706 wait complete. 2024-04-25T00:13:54.933 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:13:54.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:13:55.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdec3a00640 1 Processor -- start 2024-04-25T00:13:55.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdec3a00640 1 -- start start 2024-04-25T00:13:55.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdec3a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdebc1551e0 0x7fdebc15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:55.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdec3a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdebc1555c0 0x7fdebc071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:55.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdec3a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdebc15c560 con 0x7fdebc06e4c0 2024-04-25T00:13:55.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdec3a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fdebc072460 con 0x7fdebc1555c0 2024-04-25T00:13:55.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdec3a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdebc15c7d0 con 0x7fdebc1551e0 2024-04-25T00:13:55.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdeb3e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdebc1555c0 0x7fdebc071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:55.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdeb3e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdebc1555c0 0x7fdebc071b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45074/0 (socket says 172.21.15.83:45074) 2024-04-25T00:13:55.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdeb3e00640 1 -- 172.21.15.83:0/2844866885 learned_addr learned my addr 172.21.15.83:0/2844866885 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:55.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdec1000640 1 --2- 172.21.15.83:0/2844866885 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdebc1551e0 0x7fdebc15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:55.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdeb3e00640 1 -- 172.21.15.83:0/2844866885 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdebc1551e0 msgr2=0x7fdebc15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:55.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdeb3e00640 1 --2- 172.21.15.83:0/2844866885 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdebc1551e0 0x7fdebc15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdeb3e00640 1 -- 172.21.15.83:0/2844866885 >> v1:172.21.15.86:6789/0 conn(0x7fdebc06e4c0 legacy=0x7fdebc15e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:55.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdeb3e00640 1 -- 172.21.15.83:0/2844866885 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc15cab0 con 0x7fdebc1555c0 2024-04-25T00:13:55.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.072+0000 7fdec1000640 1 --2- 172.21.15.83:0/2844866885 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdebc1551e0 0x7fdebc15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:55.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.076+0000 7fdeb3e00640 1 --2- 172.21.15.83:0/2844866885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc1555c0 0x7fdebc071b70 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009870 tx=0x7fdea401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8f6f3ce763200d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:55.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.076+0000 7fdebbe00640 1 -- 172.21.15.83:0/2844866885 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea40ab600 con 0x7fdebc1555c0 2024-04-25T00:13:55.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.076+0000 7fdebbe00640 1 -- 172.21.15.83:0/2844866885 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea40b0070 con 0x7fdebc1555c0 2024-04-25T00:13:55.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.076+0000 7fdebbe00640 1 -- 172.21.15.83:0/2844866885 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea40ab9c0 con 0x7fdebc1555c0 2024-04-25T00:13:55.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.076+0000 7fdec3a00640 1 -- 172.21.15.83:0/2844866885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc1555c0 msgr2=0x7fdebc071b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:55.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.076+0000 7fdec3a00640 1 --2- 172.21.15.83:0/2844866885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc1555c0 0x7fdebc071b70 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009870 tx=0x7fdea401c520 comp rx=0 tx=0).stop 2024-04-25T00:13:55.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.076+0000 7fdec3a00640 1 -- 172.21.15.83:0/2844866885 shutdown_connections 2024-04-25T00:13:55.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.076+0000 7fdec3a00640 1 --2- 172.21.15.83:0/2844866885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc1555c0 0x7fdebc071b70 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.076+0000 7fdec3a00640 1 --2- 172.21.15.83:0/2844866885 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdebc1551e0 0x7fdebc15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.076+0000 7fdec3a00640 1 -- 172.21.15.83:0/2844866885 >> 172.21.15.83:0/2844866885 conn(0x7fdebc0b1830 msgr2=0x7fdebc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:55.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.076+0000 7fdec3a00640 1 -- 172.21.15.83:0/2844866885 shutdown_connections 2024-04-25T00:13:55.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.076+0000 7fdec3a00640 1 -- 172.21.15.83:0/2844866885 wait complete. 2024-04-25T00:13:55.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec3a00640 1 Processor -- start 2024-04-25T00:13:55.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec3a00640 1 -- start start 2024-04-25T00:13:55.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec3a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdebc06e4c0 0x7fdebc15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:55.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec3a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc1551e0 0x7fdebc0713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:55.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec3a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc1555c0 0x7fdebc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec3a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc167e10 con 0x7fdebc1555c0 2024-04-25T00:13:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec3a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc167c90 con 0x7fdebc1551e0 2024-04-25T00:13:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec3a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdebc167f90 con 0x7fdebc06e4c0 2024-04-25T00:13:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc1555c0 0x7fdebc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc1555c0 0x7fdebc071910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45078/0 (socket says 172.21.15.83:45078) 2024-04-25T00:13:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec1a00640 1 -- 172.21.15.83:0/1425781041 learned_addr learned my addr 172.21.15.83:0/1425781041 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec1000640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdebc06e4c0 0x7fdebc15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdeb3e00640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc1551e0 0x7fdebc0713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec1a00640 1 -- 172.21.15.83:0/1425781041 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdebc06e4c0 msgr2=0x7fdebc15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec1a00640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdebc06e4c0 0x7fdebc15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec1a00640 1 -- 172.21.15.83:0/1425781041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc1551e0 msgr2=0x7fdebc0713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec1a00640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc1551e0 0x7fdebc0713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec1a00640 1 -- 172.21.15.83:0/1425781041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc14e4d0 con 0x7fdebc1555c0 2024-04-25T00:13:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdeb3e00640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc1551e0 0x7fdebc0713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:55.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec1000640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdebc06e4c0 0x7fdebc15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:55.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec1a00640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc1555c0 0x7fdebc071910 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400b9a0 tx=0x7fdeb400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:55.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdeb9600640 1 -- 172.21.15.83:0/1425781041 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4019070 con 0x7fdebc1555c0 2024-04-25T00:13:55.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdeb9600640 1 -- 172.21.15.83:0/1425781041 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb400cba0 con 0x7fdebc1555c0 2024-04-25T00:13:55.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdeb9600640 1 -- 172.21.15.83:0/1425781041 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4014440 con 0x7fdebc1555c0 2024-04-25T00:13:55.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec3a00640 1 -- 172.21.15.83:0/1425781041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdebc14e7b0 con 0x7fdebc1555c0 2024-04-25T00:13:55.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.080+0000 7fdec3a00640 1 -- 172.21.15.83:0/1425781041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc14ed70 con 0x7fdebc1555c0 2024-04-25T00:13:55.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.084+0000 7fdec3a00640 1 -- 172.21.15.83:0/1425781041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdebc04f510 con 0x7fdebc1555c0 2024-04-25T00:13:55.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.084+0000 7fdeb9600640 1 -- 172.21.15.83:0/1425781041 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdeb401c070 con 0x7fdebc1555c0 2024-04-25T00:13:55.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.084+0000 7fdeb9600640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fde980678c0 0x7fde98069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:55.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.084+0000 7fdeb9600640 1 -- 172.21.15.83:0/1425781041 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7fdeb4085aa0 con 0x7fdebc1555c0 2024-04-25T00:13:55.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.084+0000 7fdec1000640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fde980678c0 0x7fde98069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:55.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.084+0000 7fdec1000640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fde980678c0 0x7fde98069d80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdeac008930 tx=0x7fdeac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:55.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.088+0000 7fdeb9600640 1 -- 172.21.15.83:0/1425781041 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdeb4056440 con 0x7fdebc1555c0 2024-04-25T00:13:55.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.280+0000 7fdec3a00640 1 -- 172.21.15.83:0/1425781041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdebc06ce30 con 0x7fdebc1555c0 2024-04-25T00:13:55.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.908+0000 7fdeb9600640 1 -- 172.21.15.83:0/1425781041 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7fdeb4055d40 con 0x7fdebc1555c0 2024-04-25T00:13:55.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.908+0000 7fdec3a00640 1 -- 172.21.15.83:0/1425781041 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fde980678c0 msgr2=0x7fde98069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:55.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.908+0000 7fdec3a00640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fde980678c0 0x7fde98069d80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdeac008930 tx=0x7fdeac008040 comp rx=0 tx=0).stop 2024-04-25T00:13:55.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.908+0000 7fdec3a00640 1 -- 172.21.15.83:0/1425781041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc1555c0 msgr2=0x7fdebc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:55.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.908+0000 7fdec3a00640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc1555c0 0x7fdebc071910 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400b9a0 tx=0x7fdeb400be60 comp rx=0 tx=0).stop 2024-04-25T00:13:55.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.908+0000 7fdec3a00640 1 -- 172.21.15.83:0/1425781041 shutdown_connections 2024-04-25T00:13:55.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.908+0000 7fdec3a00640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fde980678c0 0x7fde98069d80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.908+0000 7fdec3a00640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc1555c0 0x7fdebc071910 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.908+0000 7fdec3a00640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc1551e0 0x7fdebc0713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.908+0000 7fdec3a00640 1 --2- 172.21.15.83:0/1425781041 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdebc06e4c0 0x7fdebc15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.908+0000 7fdec3a00640 1 -- 172.21.15.83:0/1425781041 >> 172.21.15.83:0/1425781041 conn(0x7fdebc0b1830 msgr2=0x7fdebc0cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:55.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.908+0000 7fdec3a00640 1 -- 172.21.15.83:0/1425781041 shutdown_connections 2024-04-25T00:13:55.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:55.908+0000 7fdec3a00640 1 -- 172.21.15.83:0/1425781041 wait complete. 2024-04-25T00:13:55.935 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:13:55.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:56.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa589200640 1 Processor -- start 2024-04-25T00:13:56.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa589200640 1 -- start start 2024-04-25T00:13:56.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa589200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa5841599f0 0x7fa5840cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa589200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa58415cec0 0x7fa5840ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa589200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5840ccc60 con 0x7fa58415c340 2024-04-25T00:13:56.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa589200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa584159dd0 con 0x7fa58415cec0 2024-04-25T00:13:56.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa589200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa58406dba0 con 0x7fa5841599f0 2024-04-25T00:13:56.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa582800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa58415cec0 0x7fa5840ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa582800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa58415cec0 0x7fa5840ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45092/0 (socket says 172.21.15.83:45092) 2024-04-25T00:13:56.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa582800640 1 -- 172.21.15.83:0/1445805777 learned_addr learned my addr 172.21.15.83:0/1445805777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:56.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa581e00640 1 --2- 172.21.15.83:0/1445805777 >> v2:172.21.15.86:3300/0 conn(0x7fa5841599f0 0x7fa5840cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa582800640 1 -- 172.21.15.83:0/1445805777 >> v2:172.21.15.86:3300/0 conn(0x7fa5841599f0 msgr2=0x7fa5840cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:56.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa582800640 1 --2- 172.21.15.83:0/1445805777 >> v2:172.21.15.86:3300/0 conn(0x7fa5841599f0 0x7fa5840cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa582800640 1 -- 172.21.15.83:0/1445805777 >> v1:172.21.15.86:6789/0 conn(0x7fa58415c340 legacy=0x7fa58415c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:56.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa582800640 1 -- 172.21.15.83:0/1445805777 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58406de80 con 0x7fa58415cec0 2024-04-25T00:13:56.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.068+0000 7fa581e00640 1 --2- 172.21.15.83:0/1445805777 >> v2:172.21.15.86:3300/0 conn(0x7fa5841599f0 0x7fa5840cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:56.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.072+0000 7fa582800640 1 --2- 172.21.15.83:0/1445805777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58415cec0 0x7fa5840ce6e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009870 tx=0x7fa57c01c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60965142d70a300a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:56.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.072+0000 7fa581400640 1 -- 172.21.15.83:0/1445805777 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0b0070 con 0x7fa58415cec0 2024-04-25T00:13:56.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.072+0000 7fa581400640 1 -- 172.21.15.83:0/1445805777 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa57c0ab600 con 0x7fa58415cec0 2024-04-25T00:13:56.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.072+0000 7fa581400640 1 -- 172.21.15.83:0/1445805777 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0ab9d0 con 0x7fa58415cec0 2024-04-25T00:13:56.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.072+0000 7fa589200640 1 -- 172.21.15.83:0/1445805777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58415cec0 msgr2=0x7fa5840ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.072+0000 7fa589200640 1 --2- 172.21.15.83:0/1445805777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58415cec0 0x7fa5840ce6e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009870 tx=0x7fa57c01c270 comp rx=0 tx=0).stop 2024-04-25T00:13:56.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.072+0000 7fa589200640 1 -- 172.21.15.83:0/1445805777 shutdown_connections 2024-04-25T00:13:56.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.072+0000 7fa589200640 1 --2- 172.21.15.83:0/1445805777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58415cec0 0x7fa5840ce6e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.072+0000 7fa589200640 1 --2- 172.21.15.83:0/1445805777 >> v2:172.21.15.86:3300/0 conn(0x7fa5841599f0 0x7fa5840cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.072+0000 7fa589200640 1 -- 172.21.15.83:0/1445805777 >> 172.21.15.83:0/1445805777 conn(0x7fa5840b1830 msgr2=0x7fa5840b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:56.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.072+0000 7fa589200640 1 -- 172.21.15.83:0/1445805777 shutdown_connections 2024-04-25T00:13:56.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.072+0000 7fa589200640 1 -- 172.21.15.83:0/1445805777 wait complete. 2024-04-25T00:13:56.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa589200640 1 Processor -- start 2024-04-25T00:13:56.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa589200640 1 -- start start 2024-04-25T00:13:56.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa589200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5841599f0 0x7fa5840b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa589200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58415c340 0x7fa58414d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa589200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58415cec0 0x7fa5841fa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa589200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa5840cf800 con 0x7fa58415c340 2024-04-25T00:13:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa589200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5840cf680 con 0x7fa58415cec0 2024-04-25T00:13:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa589200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa584168cd0 con 0x7fa5841599f0 2024-04-25T00:13:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa579400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58415c340 0x7fa58414d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa579400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58415c340 0x7fa58414d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45102/0 (socket says 172.21.15.83:45102) 2024-04-25T00:13:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa579400640 1 -- 172.21.15.83:0/685417555 learned_addr learned my addr 172.21.15.83:0/685417555 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa581e00640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5841599f0 0x7fa5840b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa582800640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58415cec0 0x7fa5841fa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa579400640 1 -- 172.21.15.83:0/685417555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5841599f0 msgr2=0x7fa5840b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa579400640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5841599f0 0x7fa5840b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa579400640 1 -- 172.21.15.83:0/685417555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58415cec0 msgr2=0x7fa5841fa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa579400640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58415cec0 0x7fa5841fa6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa579400640 1 -- 172.21.15.83:0/685417555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58414def0 con 0x7fa58415c340 2024-04-25T00:13:56.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa581e00640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5841599f0 0x7fa5840b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:56.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa582800640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58415cec0 0x7fa5841fa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:56.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa579400640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58415c340 0x7fa58414d9b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00c450 tx=0x7fa56c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:56.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa57aa00640 1 -- 172.21.15.83:0/685417555 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c012830 con 0x7fa58415c340 2024-04-25T00:13:56.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa57aa00640 1 -- 172.21.15.83:0/685417555 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa56c017070 con 0x7fa58415c340 2024-04-25T00:13:56.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa57aa00640 1 -- 172.21.15.83:0/685417555 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c012c30 con 0x7fa58415c340 2024-04-25T00:13:56.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa589200640 1 -- 172.21.15.83:0/685417555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5841faef0 con 0x7fa58415c340 2024-04-25T00:13:56.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.076+0000 7fa589200640 1 -- 172.21.15.83:0/685417555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5841fb4b0 con 0x7fa58415c340 2024-04-25T00:13:56.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.080+0000 7fa589200640 1 -- 172.21.15.83:0/685417555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5840737f0 con 0x7fa58415c340 2024-04-25T00:13:56.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.080+0000 7fa57aa00640 1 -- 172.21.15.83:0/685417555 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa56c01c460 con 0x7fa58415c340 2024-04-25T00:13:56.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.080+0000 7fa57aa00640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5600678c0 0x7fa560069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.080+0000 7fa57aa00640 1 -- 172.21.15.83:0/685417555 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7fa56c085c40 con 0x7fa58415c340 2024-04-25T00:13:56.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.080+0000 7fa581e00640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5600678c0 0x7fa560069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.080+0000 7fa581e00640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5600678c0 0x7fa560069d80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa574007f40 tx=0x7fa574007490 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:56.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.084+0000 7fa57aa00640 1 -- 172.21.15.83:0/685417555 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa56c056550 con 0x7fa58415c340 2024-04-25T00:13:56.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.268+0000 7fa589200640 1 -- 172.21.15.83:0/685417555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa584072c40 con 0x7fa58415c340 2024-04-25T00:13:56.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.268+0000 7fa57aa00640 1 -- 172.21.15.83:0/685417555 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5183 (secure 0 0 0) 0x7fa56c055e50 con 0x7fa58415c340 2024-04-25T00:13:56.273 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:56.273 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2852678498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2852678498},{"type":"v1","addr":"172.21.15.86:6834","nonce":2852678498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2956252746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2956252746},{"type":"v1","addr":"172.21.15.86:6835","nonce":2956252746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6588,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2678835998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2678835998},{"type":"v1","addr":"172.21.15.83:6836","nonce":2678835998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:13:52.273680+0000","modified":"2024-04-25T00:13:55.285817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2879143105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2879143105},{"type":"v1","addr":"172.21.15.83:6837","nonce":2879143105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6591,"qdb_cluster":[6591]},"id":5}]} 2024-04-25T00:13:56.273 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 44 2024-04-25T00:13:56.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.272+0000 7fa589200640 1 -- 172.21.15.83:0/685417555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5600678c0 msgr2=0x7fa560069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.272+0000 7fa589200640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5600678c0 0x7fa560069d80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa574007f40 tx=0x7fa574007490 comp rx=0 tx=0).stop 2024-04-25T00:13:56.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.272+0000 7fa589200640 1 -- 172.21.15.83:0/685417555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58415c340 msgr2=0x7fa58414d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.272+0000 7fa589200640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58415c340 0x7fa58414d9b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00c450 tx=0x7fa56c00c910 comp rx=0 tx=0).stop 2024-04-25T00:13:56.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.272+0000 7fa589200640 1 -- 172.21.15.83:0/685417555 shutdown_connections 2024-04-25T00:13:56.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.272+0000 7fa589200640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5600678c0 0x7fa560069d80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.272+0000 7fa589200640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58415cec0 0x7fa5841fa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.272+0000 7fa589200640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58415c340 0x7fa58414d9b0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.272+0000 7fa589200640 1 --2- 172.21.15.83:0/685417555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5841599f0 0x7fa5840b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.272+0000 7fa589200640 1 -- 172.21.15.83:0/685417555 >> 172.21.15.83:0/685417555 conn(0x7fa5840b1830 msgr2=0x7fa58414f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.272+0000 7fa589200640 1 -- 172.21.15.83:0/685417555 shutdown_connections 2024-04-25T00:13:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.272+0000 7fa589200640 1 -- 172.21.15.83:0/685417555 wait complete. 2024-04-25T00:13:56.295 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.428+0000 7f387f200640 1 Processor -- start 2024-04-25T00:13:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.428+0000 7f387f200640 1 -- start start 2024-04-25T00:13:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.428+0000 7f387f200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f388015e110 0x7f38800ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.428+0000 7f387f200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38800d0ad0 con 0x7f388006fdc0 2024-04-25T00:13:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.428+0000 7f387f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38800d0db0 con 0x7f388015e4f0 2024-04-25T00:13:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.428+0000 7f387f200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f38800cc3b0 con 0x7f388015e110 2024-04-25T00:13:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.428+0000 7f387e800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f388006fdc0 0x7f38800ccbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33126/0 (socket says 172.21.15.83:33126) 2024-04-25T00:13:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.428+0000 7f387e800640 1 -- 172.21.15.83:0/581439384 learned_addr learned my addr 172.21.15.83:0/581439384 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.428+0000 7f387de00640 1 --2- 172.21.15.83:0/581439384 >> v2:172.21.15.83:3300/0 conn(0x7f388015e110 0x7f38800ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.428+0000 7f387de00640 1 -- 172.21.15.83:0/581439384 >> v1:172.21.15.86:6789/0 conn(0x7f388015e4f0 legacy=0x7f38800ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.428+0000 7f387de00640 1 -- 172.21.15.83:0/581439384 >> v1:172.21.15.83:6789/0 conn(0x7f388006fdc0 legacy=0x7f38800ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.428+0000 7f387de00640 1 -- 172.21.15.83:0/581439384 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38800d0760 con 0x7f388015e110 2024-04-25T00:13:56.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.428+0000 7f387ca00640 1 -- 172.21.15.83:0/581439384 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1266030938 0 0) 0x7f38800d0ad0 con 0x7f388006fdc0 2024-04-25T00:13:56.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.432+0000 7f387de00640 1 --2- 172.21.15.83:0/581439384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388015e110 0x7f38800ce440 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3870009d90 tx=0x7f387001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ab6f1aa56b57fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:56.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.432+0000 7f387ca00640 1 -- 172.21.15.83:0/581439384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38700b2070 con 0x7f388015e110 2024-04-25T00:13:56.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.432+0000 7f387ca00640 1 -- 172.21.15.83:0/581439384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38700a5710 con 0x7f388015e110 2024-04-25T00:13:56.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.432+0000 7f387ca00640 1 -- 172.21.15.83:0/581439384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38700a5ae0 con 0x7f388015e110 2024-04-25T00:13:56.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.432+0000 7f387f200640 1 -- 172.21.15.83:0/581439384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388015e110 msgr2=0x7f38800ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.432+0000 7f387f200640 1 --2- 172.21.15.83:0/581439384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388015e110 0x7f38800ce440 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3870009d90 tx=0x7f387001c850 comp rx=0 tx=0).stop 2024-04-25T00:13:56.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.432+0000 7f387f200640 1 -- 172.21.15.83:0/581439384 shutdown_connections 2024-04-25T00:13:56.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.432+0000 7f387f200640 1 --2- 172.21.15.83:0/581439384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388015e110 0x7f38800ce440 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.432+0000 7f387f200640 1 -- 172.21.15.83:0/581439384 >> 172.21.15.83:0/581439384 conn(0x7f38800d1bc0 msgr2=0x7f38800d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:56.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.432+0000 7f387f200640 1 -- 172.21.15.83:0/581439384 shutdown_connections 2024-04-25T00:13:56.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.432+0000 7f387f200640 1 -- 172.21.15.83:0/581439384 wait complete. 2024-04-25T00:13:56.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387f200640 1 Processor -- start 2024-04-25T00:13:56.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387f200640 1 -- start start 2024-04-25T00:13:56.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f388006fdc0 0x7f388014c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f388015e110 0x7f3880154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388015e4f0 0x7f388015b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38800cdee0 con 0x7f388015e4f0 2024-04-25T00:13:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38800cdd60 con 0x7f388006fdc0 2024-04-25T00:13:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f38800cf2a0 con 0x7f388015e110 2024-04-25T00:13:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388015e4f0 0x7f388015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388015e4f0 0x7f388015b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45134/0 (socket says 172.21.15.83:45134) 2024-04-25T00:13:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387e800640 1 -- 172.21.15.83:0/3854656683 learned_addr learned my addr 172.21.15.83:0/3854656683 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387e800640 1 -- 172.21.15.83:0/3854656683 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f388015e110 msgr2=0x7f3880154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387e800640 1 --2- 172.21.15.83:0/3854656683 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f388015e110 0x7f3880154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387e800640 1 -- 172.21.15.83:0/3854656683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f388006fdc0 msgr2=0x7f388014c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387e800640 1 --2- 172.21.15.83:0/3854656683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f388006fdc0 0x7f388014c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387e800640 1 -- 172.21.15.83:0/3854656683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388015a460 con 0x7f388015e4f0 2024-04-25T00:13:56.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387e800640 1 --2- 172.21.15.83:0/3854656683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388015e4f0 0x7f388015b500 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f387800c3e0 tx=0x7f387800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:56.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f3876000640 1 -- 172.21.15.83:0/3854656683 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38780074e0 con 0x7f388015e4f0 2024-04-25T00:13:56.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f3876000640 1 -- 172.21.15.83:0/3854656683 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38780047f0 con 0x7f388015e4f0 2024-04-25T00:13:56.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f3876000640 1 -- 172.21.15.83:0/3854656683 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878004be0 con 0x7f388015e4f0 2024-04-25T00:13:56.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387f200640 1 -- 172.21.15.83:0/3854656683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388015a740 con 0x7f388015e4f0 2024-04-25T00:13:56.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.436+0000 7f387f200640 1 -- 172.21.15.83:0/3854656683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388015afe0 con 0x7f388015e4f0 2024-04-25T00:13:56.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.440+0000 7f387f200640 1 -- 172.21.15.83:0/3854656683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388015a910 con 0x7f388015e4f0 2024-04-25T00:13:56.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.440+0000 7f3876000640 1 -- 172.21.15.83:0/3854656683 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3878011070 con 0x7f388015e4f0 2024-04-25T00:13:56.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.440+0000 7f3876000640 1 --2- 172.21.15.83:0/3854656683 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38600678c0 0x7f3860069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.440+0000 7f3876000640 1 -- 172.21.15.83:0/3854656683 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f38780859f0 con 0x7f388015e4f0 2024-04-25T00:13:56.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.440+0000 7f387de00640 1 --2- 172.21.15.83:0/3854656683 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38600678c0 0x7f3860069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.440+0000 7f387de00640 1 --2- 172.21.15.83:0/3854656683 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38600678c0 0x7f3860069d80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f38700af000 tx=0x7f3870004ac0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:56.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.444+0000 7f3876000640 1 -- 172.21.15.83:0/3854656683 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f38780563a0 con 0x7f388015e4f0 2024-04-25T00:13:56.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.612+0000 7f387f200640 1 -- 172.21.15.83:0/3854656683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f388015b1e0 con 0x7f388015e4f0 2024-04-25T00:13:56.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.612+0000 7f3876000640 1 -- 172.21.15.83:0/3854656683 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16861 (secure 0 0 0) 0x7f388015b1e0 con 0x7f388015e4f0 2024-04-25T00:13:56.616 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:56.616 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":71,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:53.896317+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:50.655187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:51.268160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.616+0000 7f387f200640 1 -- 172.21.15.83:0/3854656683 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38600678c0 msgr2=0x7f3860069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.616+0000 7f387f200640 1 --2- 172.21.15.83:0/3854656683 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38600678c0 0x7f3860069d80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f38700af000 tx=0x7f3870004ac0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.616+0000 7f387f200640 1 -- 172.21.15.83:0/3854656683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388015e4f0 msgr2=0x7f388015b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.616+0000 7f387f200640 1 --2- 172.21.15.83:0/3854656683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388015e4f0 0x7f388015b500 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f387800c3e0 tx=0x7f387800c8a0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.616+0000 7f387f200640 1 -- 172.21.15.83:0/3854656683 shutdown_connections 2024-04-25T00:13:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.616+0000 7f387f200640 1 --2- 172.21.15.83:0/3854656683 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38600678c0 0x7f3860069d80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.616+0000 7f387f200640 1 --2- 172.21.15.83:0/3854656683 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f388015e110 0x7f3880154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.616+0000 7f387f200640 1 --2- 172.21.15.83:0/3854656683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388015e4f0 0x7f388015b500 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.616+0000 7f387f200640 1 --2- 172.21.15.83:0/3854656683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f388006fdc0 0x7f388014c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.616+0000 7f387f200640 1 -- 172.21.15.83:0/3854656683 >> 172.21.15.83:0/3854656683 conn(0x7f38800d1bc0 msgr2=0x7f388006c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.616+0000 7f387f200640 1 -- 172.21.15.83:0/3854656683 shutdown_connections 2024-04-25T00:13:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.616+0000 7f387f200640 1 -- 172.21.15.83:0/3854656683 wait complete. 2024-04-25T00:13:56.637 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:13:56.637 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:13:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.768+0000 7f5be3a00640 1 Processor -- start 2024-04-25T00:13:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.768+0000 7f5be3a00640 1 -- start start 2024-04-25T00:13:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.768+0000 7f5be3a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bdc1553e0 0x7f5bdc06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.768+0000 7f5be3a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5bdc06ea00 0x7f5bdc0b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.768+0000 7f5be3a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bdc0b3b40 con 0x7f5bdc0b48a0 2024-04-25T00:13:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.772+0000 7f5be3a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5bdc06ef80 con 0x7f5bdc06ea00 2024-04-25T00:13:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.772+0000 7f5be3a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5bdc15b510 con 0x7f5bdc1553e0 2024-04-25T00:13:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.772+0000 7f5be1a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5bdc0b48a0 0x7f5bdc15e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33132/0 (socket says 172.21.15.83:33132) 2024-04-25T00:13:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.772+0000 7f5be1a00640 1 -- 172.21.15.83:0/4200098748 learned_addr learned my addr 172.21.15.83:0/4200098748 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.772+0000 7f5bd3e00640 1 --2- 172.21.15.83:0/4200098748 >> v2:172.21.15.86:3300/0 conn(0x7f5bdc06ea00 0x7f5bdc0b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.772+0000 7f5be1000640 1 --2- 172.21.15.83:0/4200098748 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bdc1553e0 0x7f5bdc06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.772+0000 7f5bdbe00640 1 -- 172.21.15.83:0/4200098748 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1738252007 0 0) 0x7f5bdc0b3b40 con 0x7f5bdc0b48a0 2024-04-25T00:13:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.772+0000 7f5bdbe00640 1 -- 172.21.15.83:0/4200098748 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bc0003530 con 0x7f5bdc0b48a0 2024-04-25T00:13:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.772+0000 7f5bd3e00640 1 -- 172.21.15.83:0/4200098748 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bdc1553e0 msgr2=0x7f5bdc06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.772+0000 7f5bd3e00640 1 --2- 172.21.15.83:0/4200098748 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bdc1553e0 0x7f5bdc06e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.772+0000 7f5bd3e00640 1 -- 172.21.15.83:0/4200098748 >> v1:172.21.15.83:6789/0 conn(0x7f5bdc0b48a0 legacy=0x7f5bdc15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.772+0000 7f5bdbe00640 1 -- 172.21.15.83:0/4200098748 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 736503579 0 0) 0x7f5bc0003530 con 0x7f5bdc0b48a0 2024-04-25T00:13:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.772+0000 7f5bd3e00640 1 -- 172.21.15.83:0/4200098748 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bdc15b7f0 con 0x7f5bdc06ea00 2024-04-25T00:13:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5bd3e00640 1 --2- 172.21.15.83:0/4200098748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bdc06ea00 0x7f5bdc0b4360 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40097c0 tx=0x7f5bc401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9a7d3156b89bfee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5bdbe00640 1 -- 172.21.15.83:0/4200098748 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40b0070 con 0x7f5bdc06ea00 2024-04-25T00:13:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5bdbe00640 1 -- 172.21.15.83:0/4200098748 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc40ab400 con 0x7f5bdc06ea00 2024-04-25T00:13:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5bdbe00640 1 -- 172.21.15.83:0/4200098748 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4001a00 con 0x7f5bdc06ea00 2024-04-25T00:13:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 -- 172.21.15.83:0/4200098748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bdc06ea00 msgr2=0x7f5bdc0b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 --2- 172.21.15.83:0/4200098748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bdc06ea00 0x7f5bdc0b4360 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40097c0 tx=0x7f5bc401c330 comp rx=0 tx=0).stop 2024-04-25T00:13:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 -- 172.21.15.83:0/4200098748 shutdown_connections 2024-04-25T00:13:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 --2- 172.21.15.83:0/4200098748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bdc06ea00 0x7f5bdc0b4360 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 --2- 172.21.15.83:0/4200098748 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bdc1553e0 0x7f5bdc06e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 -- 172.21.15.83:0/4200098748 >> 172.21.15.83:0/4200098748 conn(0x7f5bdc0b1830 msgr2=0x7f5bdc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 -- 172.21.15.83:0/4200098748 shutdown_connections 2024-04-25T00:13:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 -- 172.21.15.83:0/4200098748 wait complete. 2024-04-25T00:13:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 Processor -- start 2024-04-25T00:13:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 -- start start 2024-04-25T00:13:56.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bdc06ea00 0x7f5bdc1fadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bdc0b48a0 0x7f5bdc0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bdc1553e0 0x7f5bdc14b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc155940 con 0x7f5bdc06ea00 2024-04-25T00:13:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc1557c0 con 0x7f5bdc1553e0 2024-04-25T00:13:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be3a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5bdc06d3b0 con 0x7f5bdc0b48a0 2024-04-25T00:13:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bdc06ea00 0x7f5bdc1fadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bdc06ea00 0x7f5bdc1fadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45138/0 (socket says 172.21.15.83:45138) 2024-04-25T00:13:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.776+0000 7f5be1000640 1 -- 172.21.15.83:0/3101521329 learned_addr learned my addr 172.21.15.83:0/3101521329 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5bd3e00640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bdc0b48a0 0x7f5bdc0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5be1a00640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bdc1553e0 0x7f5bdc14b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5be1000640 1 -- 172.21.15.83:0/3101521329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bdc0b48a0 msgr2=0x7f5bdc0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5be1000640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bdc0b48a0 0x7f5bdc0ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5be1000640 1 -- 172.21.15.83:0/3101521329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bdc1553e0 msgr2=0x7f5bdc14b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5be1000640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bdc1553e0 0x7f5bdc14b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5be1000640 1 -- 172.21.15.83:0/3101521329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bdc156080 con 0x7f5bdc06ea00 2024-04-25T00:13:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5be1a00640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bdc1553e0 0x7f5bdc14b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5be1000640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bdc06ea00 0x7f5bdc1fadc0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00ea40 tx=0x7f5bcc00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5bd9600640 1 -- 172.21.15.83:0/3101521329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc00de60 con 0x7f5bdc06ea00 2024-04-25T00:13:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5bd9600640 1 -- 172.21.15.83:0/3101521329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bcc010440 con 0x7f5bdc06ea00 2024-04-25T00:13:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5bd9600640 1 -- 172.21.15.83:0/3101521329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc010810 con 0x7f5bdc06ea00 2024-04-25T00:13:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5be3a00640 1 -- 172.21.15.83:0/3101521329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bdc156870 con 0x7f5bdc06ea00 2024-04-25T00:13:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5be3a00640 1 -- 172.21.15.83:0/3101521329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bdc155da0 con 0x7f5bdc06ea00 2024-04-25T00:13:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5be3a00640 1 -- 172.21.15.83:0/3101521329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bdc0737f0 con 0x7f5bdc06ea00 2024-04-25T00:13:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5bd9600640 1 -- 172.21.15.83:0/3101521329 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5bcc0109b0 con 0x7f5bdc06ea00 2024-04-25T00:13:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5bd9600640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5bb80678c0 0x7f5bb8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5bd9600640 1 -- 172.21.15.83:0/3101521329 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f5bcc0863d0 con 0x7f5bdc06ea00 2024-04-25T00:13:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.780+0000 7f5bd3e00640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5bb80678c0 0x7f5bb8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.784+0000 7f5bd3e00640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5bb80678c0 0x7f5bb8069d80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40ad000 tx=0x7f5bc40a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:56.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.784+0000 7f5bd9600640 1 -- 172.21.15.83:0/3101521329 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5bcc014030 con 0x7f5bdc06ea00 2024-04-25T00:13:56.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.936+0000 7f5be3a00640 1 -- 172.21.15.83:0/3101521329 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5bdc06ce10 con 0x7f5bb80678c0 2024-04-25T00:13:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.944+0000 7f5bd9600640 1 -- 172.21.15.83:0/3101521329 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252067 (secure 0 0 0) 0x7f5bdc06ce10 con 0x7f5bb80678c0 2024-04-25T00:13:56.950 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:13:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.948+0000 7f5be3a00640 1 -- 172.21.15.83:0/3101521329 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5bb80678c0 msgr2=0x7f5bb8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.948+0000 7f5be3a00640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5bb80678c0 0x7f5bb8069d80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40ad000 tx=0x7f5bc40a6000 comp rx=0 tx=0).stop 2024-04-25T00:13:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.948+0000 7f5be3a00640 1 -- 172.21.15.83:0/3101521329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bdc06ea00 msgr2=0x7f5bdc1fadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.948+0000 7f5be3a00640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bdc06ea00 0x7f5bdc1fadc0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00ea40 tx=0x7f5bcc00ef00 comp rx=0 tx=0).stop 2024-04-25T00:13:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.948+0000 7f5be3a00640 1 -- 172.21.15.83:0/3101521329 shutdown_connections 2024-04-25T00:13:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.948+0000 7f5be3a00640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5bb80678c0 0x7f5bb8069d80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.948+0000 7f5be3a00640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5bdc0b48a0 0x7f5bdc0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.948+0000 7f5be3a00640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bdc1553e0 0x7f5bdc14b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.948+0000 7f5be3a00640 1 --2- 172.21.15.83:0/3101521329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bdc06ea00 0x7f5bdc1fadc0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.948+0000 7f5be3a00640 1 -- 172.21.15.83:0/3101521329 >> 172.21.15.83:0/3101521329 conn(0x7f5bdc0b1830 msgr2=0x7f5bdc0b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.948+0000 7f5be3a00640 1 -- 172.21.15.83:0/3101521329 shutdown_connections 2024-04-25T00:13:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:56.948+0000 7f5be3a00640 1 -- 172.21.15.83:0/3101521329 wait complete. 2024-04-25T00:13:56.970 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-04-25T00:13:56.476921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236592,"kb_used_data":7936,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754738128,"statfs":{"total":773094113280,"available":772851843072,"internally_reserved":0,"allocated":8126464,"data_stored":4285916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.598678"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458342+0000","last_change":"2024-04-25T00:13:52.906825+0000","last_active":"2024-04-25T00:13:54.458342+0000","last_peered":"2024-04-25T00:13:54.458342+0000","last_clean":"2024-04-25T00:13:54.458342+0000","last_became_active":"2024-04-25T00:13:52.906492+0000","last_became_peered":"2024-04-25T00:13:52.906492+0000","last_unstale":"2024-04-25T00:13:54.458342+0000","last_undegraded":"2024-04-25T00:13:54.458342+0000","last_fullsized":"2024-04-25T00:13:54.458342+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:38.492799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.912107+0000","last_change":"2024-04-25T00:13:52.919890+0000","last_active":"2024-04-25T00:13:53.912107+0000","last_peered":"2024-04-25T00:13:53.912107+0000","last_clean":"2024-04-25T00:13:53.912107+0000","last_became_active":"2024-04-25T00:13:52.919094+0000","last_became_peered":"2024-04-25T00:13:52.919094+0000","last_unstale":"2024-04-25T00:13:53.912107+0000","last_undegraded":"2024-04-25T00:13:53.912107+0000","last_fullsized":"2024-04-25T00:13:53.912107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:07.478047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.909170+0000","last_change":"2024-04-25T00:13:52.904105+0000","last_active":"2024-04-25T00:13:53.909170+0000","last_peered":"2024-04-25T00:13:53.909170+0000","last_clean":"2024-04-25T00:13:53.909170+0000","last_became_active":"2024-04-25T00:13:52.903840+0000","last_became_peered":"2024-04-25T00:13:52.903840+0000","last_unstale":"2024-04-25T00:13:53.909170+0000","last_undegraded":"2024-04-25T00:13:53.909170+0000","last_fullsized":"2024-04-25T00:13:53.909170+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:20.863218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353424+0000","last_change":"2024-04-25T00:13:52.910517+0000","last_active":"2024-04-25T00:13:54.353424+0000","last_peered":"2024-04-25T00:13:54.353424+0000","last_clean":"2024-04-25T00:13:54.353424+0000","last_became_active":"2024-04-25T00:13:52.910158+0000","last_became_peered":"2024-04-25T00:13:52.910158+0000","last_unstale":"2024-04-25T00:13:54.353424+0000","last_undegraded":"2024-04-25T00:13:54.353424+0000","last_fullsized":"2024-04-25T00:13:54.353424+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:15.092144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907283+0000","last_change":"2024-04-25T00:13:52.916818+0000","last_active":"2024-04-25T00:13:53.907283+0000","last_peered":"2024-04-25T00:13:53.907283+0000","last_clean":"2024-04-25T00:13:53.907283+0000","last_became_active":"2024-04-25T00:13:52.916362+0000","last_became_peered":"2024-04-25T00:13:52.916362+0000","last_unstale":"2024-04-25T00:13:53.907283+0000","last_undegraded":"2024-04-25T00:13:53.907283+0000","last_fullsized":"2024-04-25T00:13:53.907283+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:23.129717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457379+0000","last_change":"2024-04-25T00:13:52.928102+0000","last_active":"2024-04-25T00:13:54.457379+0000","last_peered":"2024-04-25T00:13:54.457379+0000","last_clean":"2024-04-25T00:13:54.457379+0000","last_became_active":"2024-04-25T00:13:52.927402+0000","last_became_peered":"2024-04-25T00:13:52.927402+0000","last_unstale":"2024-04-25T00:13:54.457379+0000","last_undegraded":"2024-04-25T00:13:54.457379+0000","last_fullsized":"2024-04-25T00:13:54.457379+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:07.478047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906167+0000","last_change":"2024-04-25T00:13:52.915025+0000","last_active":"2024-04-25T00:13:53.906167+0000","last_peered":"2024-04-25T00:13:53.906167+0000","last_clean":"2024-04-25T00:13:53.906167+0000","last_became_active":"2024-04-25T00:13:52.914745+0000","last_became_peered":"2024-04-25T00:13:52.914745+0000","last_unstale":"2024-04-25T00:13:53.906167+0000","last_undegraded":"2024-04-25T00:13:53.906167+0000","last_fullsized":"2024-04-25T00:13:53.906167+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:08.688249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907175+0000","last_change":"2024-04-25T00:13:52.910058+0000","last_active":"2024-04-25T00:13:53.907175+0000","last_peered":"2024-04-25T00:13:53.907175+0000","last_clean":"2024-04-25T00:13:53.907175+0000","last_became_active":"2024-04-25T00:13:52.909787+0000","last_became_peered":"2024-04-25T00:13:52.909787+0000","last_unstale":"2024-04-25T00:13:53.907175+0000","last_undegraded":"2024-04-25T00:13:53.907175+0000","last_fullsized":"2024-04-25T00:13:53.907175+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:51.018895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457631+0000","last_change":"2024-04-25T00:13:52.927269+0000","last_active":"2024-04-25T00:13:54.457631+0000","last_peered":"2024-04-25T00:13:54.457631+0000","last_clean":"2024-04-25T00:13:54.457631+0000","last_became_active":"2024-04-25T00:13:52.926935+0000","last_became_peered":"2024-04-25T00:13:52.926935+0000","last_unstale":"2024-04-25T00:13:54.457631+0000","last_undegraded":"2024-04-25T00:13:54.457631+0000","last_fullsized":"2024-04-25T00:13:54.457631+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:21.425380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353305+0000","last_change":"2024-04-25T00:13:52.912802+0000","last_active":"2024-04-25T00:13:54.353305+0000","last_peered":"2024-04-25T00:13:54.353305+0000","last_clean":"2024-04-25T00:13:54.353305+0000","last_became_active":"2024-04-25T00:13:52.912228+0000","last_became_peered":"2024-04-25T00:13:52.912228+0000","last_unstale":"2024-04-25T00:13:54.353305+0000","last_undegraded":"2024-04-25T00:13:54.353305+0000","last_fullsized":"2024-04-25T00:13:54.353305+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:34.148169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911934+0000","last_change":"2024-04-25T00:13:52.922317+0000","last_active":"2024-04-25T00:13:53.911934+0000","last_peered":"2024-04-25T00:13:53.911934+0000","last_clean":"2024-04-25T00:13:53.911934+0000","last_became_active":"2024-04-25T00:13:52.921948+0000","last_became_peered":"2024-04-25T00:13:52.921948+0000","last_unstale":"2024-04-25T00:13:53.911934+0000","last_undegraded":"2024-04-25T00:13:53.911934+0000","last_fullsized":"2024-04-25T00:13:53.911934+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:36.719641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353483+0000","last_change":"2024-04-25T00:13:52.908021+0000","last_active":"2024-04-25T00:13:54.353483+0000","last_peered":"2024-04-25T00:13:54.353483+0000","last_clean":"2024-04-25T00:13:54.353483+0000","last_became_active":"2024-04-25T00:13:52.907675+0000","last_became_peered":"2024-04-25T00:13:52.907675+0000","last_unstale":"2024-04-25T00:13:54.353483+0000","last_undegraded":"2024-04-25T00:13:54.353483+0000","last_fullsized":"2024-04-25T00:13:54.353483+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:48.303380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907248+0000","last_change":"2024-04-25T00:13:52.932246+0000","last_active":"2024-04-25T00:13:53.907248+0000","last_peered":"2024-04-25T00:13:53.907248+0000","last_clean":"2024-04-25T00:13:53.907248+0000","last_became_active":"2024-04-25T00:13:52.932013+0000","last_became_peered":"2024-04-25T00:13:52.932013+0000","last_unstale":"2024-04-25T00:13:53.907248+0000","last_undegraded":"2024-04-25T00:13:53.907248+0000","last_fullsized":"2024-04-25T00:13:53.907248+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:03.454900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457501+0000","last_change":"2024-04-25T00:13:52.927403+0000","last_active":"2024-04-25T00:13:54.457501+0000","last_peered":"2024-04-25T00:13:54.457501+0000","last_clean":"2024-04-25T00:13:54.457501+0000","last_became_active":"2024-04-25T00:13:52.927154+0000","last_became_peered":"2024-04-25T00:13:52.927154+0000","last_unstale":"2024-04-25T00:13:54.457501+0000","last_undegraded":"2024-04-25T00:13:54.457501+0000","last_fullsized":"2024-04-25T00:13:54.457501+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:16:19.823094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911584+0000","last_change":"2024-04-25T00:13:52.902243+0000","last_active":"2024-04-25T00:13:53.911584+0000","last_peered":"2024-04-25T00:13:53.911584+0000","last_clean":"2024-04-25T00:13:53.911584+0000","last_became_active":"2024-04-25T00:13:52.901954+0000","last_became_peered":"2024-04-25T00:13:52.901954+0000","last_unstale":"2024-04-25T00:13:53.911584+0000","last_undegraded":"2024-04-25T00:13:53.911584+0000","last_fullsized":"2024-04-25T00:13:53.911584+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:14.782886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457576+0000","last_change":"2024-04-25T00:13:52.913541+0000","last_active":"2024-04-25T00:13:54.457576+0000","last_peered":"2024-04-25T00:13:54.457576+0000","last_clean":"2024-04-25T00:13:54.457576+0000","last_became_active":"2024-04-25T00:13:52.913289+0000","last_became_peered":"2024-04-25T00:13:52.913289+0000","last_unstale":"2024-04-25T00:13:54.457576+0000","last_undegraded":"2024-04-25T00:13:54.457576+0000","last_fullsized":"2024-04-25T00:13:54.457576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:17.850811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907078+0000","last_change":"2024-04-25T00:13:52.918163+0000","last_active":"2024-04-25T00:13:53.907078+0000","last_peered":"2024-04-25T00:13:53.907078+0000","last_clean":"2024-04-25T00:13:53.907078+0000","last_became_active":"2024-04-25T00:13:52.917900+0000","last_became_peered":"2024-04-25T00:13:52.917900+0000","last_unstale":"2024-04-25T00:13:53.907078+0000","last_undegraded":"2024-04-25T00:13:53.907078+0000","last_fullsized":"2024-04-25T00:13:53.907078+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:53.489936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.909147+0000","last_change":"2024-04-25T00:13:52.901921+0000","last_active":"2024-04-25T00:13:53.909147+0000","last_peered":"2024-04-25T00:13:53.909147+0000","last_clean":"2024-04-25T00:13:53.909147+0000","last_became_active":"2024-04-25T00:13:52.901665+0000","last_became_peered":"2024-04-25T00:13:52.901665+0000","last_unstale":"2024-04-25T00:13:53.909147+0000","last_undegraded":"2024-04-25T00:13:53.909147+0000","last_fullsized":"2024-04-25T00:13:53.909147+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:16.460099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457458+0000","last_change":"2024-04-25T00:13:52.934265+0000","last_active":"2024-04-25T00:13:54.457458+0000","last_peered":"2024-04-25T00:13:54.457458+0000","last_clean":"2024-04-25T00:13:54.457458+0000","last_became_active":"2024-04-25T00:13:52.933898+0000","last_became_peered":"2024-04-25T00:13:52.933898+0000","last_unstale":"2024-04-25T00:13:54.457458+0000","last_undegraded":"2024-04-25T00:13:54.457458+0000","last_fullsized":"2024-04-25T00:13:54.457458+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:40.637159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907107+0000","last_change":"2024-04-25T00:13:52.914462+0000","last_active":"2024-04-25T00:13:53.907107+0000","last_peered":"2024-04-25T00:13:53.907107+0000","last_clean":"2024-04-25T00:13:53.907107+0000","last_became_active":"2024-04-25T00:13:52.914256+0000","last_became_peered":"2024-04-25T00:13:52.914256+0000","last_unstale":"2024-04-25T00:13:53.907107+0000","last_undegraded":"2024-04-25T00:13:53.907107+0000","last_fullsized":"2024-04-25T00:13:53.907107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:45.817144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457251+0000","last_change":"2024-04-25T00:13:52.905025+0000","last_active":"2024-04-25T00:13:54.457251+0000","last_peered":"2024-04-25T00:13:54.457251+0000","last_clean":"2024-04-25T00:13:54.457251+0000","last_became_active":"2024-04-25T00:13:52.904769+0000","last_became_peered":"2024-04-25T00:13:52.904769+0000","last_unstale":"2024-04-25T00:13:54.457251+0000","last_undegraded":"2024-04-25T00:13:54.457251+0000","last_fullsized":"2024-04-25T00:13:54.457251+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:49.936682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907014+0000","last_change":"2024-04-25T00:13:52.931792+0000","last_active":"2024-04-25T00:13:53.907014+0000","last_peered":"2024-04-25T00:13:53.907014+0000","last_clean":"2024-04-25T00:13:53.907014+0000","last_became_active":"2024-04-25T00:13:52.931565+0000","last_became_peered":"2024-04-25T00:13:52.931565+0000","last_unstale":"2024-04-25T00:13:53.907014+0000","last_undegraded":"2024-04-25T00:13:53.907014+0000","last_fullsized":"2024-04-25T00:13:53.907014+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:46.506797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457174+0000","last_change":"2024-04-25T00:13:52.906720+0000","last_active":"2024-04-25T00:13:54.457174+0000","last_peered":"2024-04-25T00:13:54.457174+0000","last_clean":"2024-04-25T00:13:54.457174+0000","last_became_active":"2024-04-25T00:13:52.906425+0000","last_became_peered":"2024-04-25T00:13:52.906425+0000","last_unstale":"2024-04-25T00:13:54.457174+0000","last_undegraded":"2024-04-25T00:13:54.457174+0000","last_fullsized":"2024-04-25T00:13:54.457174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:01.855627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911539+0000","last_change":"2024-04-25T00:13:52.929015+0000","last_active":"2024-04-25T00:13:53.911539+0000","last_peered":"2024-04-25T00:13:53.911539+0000","last_clean":"2024-04-25T00:13:53.911539+0000","last_became_active":"2024-04-25T00:13:52.927512+0000","last_became_peered":"2024-04-25T00:13:52.927512+0000","last_unstale":"2024-04-25T00:13:53.911539+0000","last_undegraded":"2024-04-25T00:13:53.911539+0000","last_fullsized":"2024-04-25T00:13:53.911539+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:48.303380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911925+0000","last_change":"2024-04-25T00:13:52.922121+0000","last_active":"2024-04-25T00:13:53.911925+0000","last_peered":"2024-04-25T00:13:53.911925+0000","last_clean":"2024-04-25T00:13:53.911925+0000","last_became_active":"2024-04-25T00:13:52.921848+0000","last_became_peered":"2024-04-25T00:13:52.921848+0000","last_unstale":"2024-04-25T00:13:53.911925+0000","last_undegraded":"2024-04-25T00:13:53.911925+0000","last_fullsized":"2024-04-25T00:13:53.911925+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:08.628566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353477+0000","last_change":"2024-04-25T00:13:52.908067+0000","last_active":"2024-04-25T00:13:54.353477+0000","last_peered":"2024-04-25T00:13:54.353477+0000","last_clean":"2024-04-25T00:13:54.353477+0000","last_became_active":"2024-04-25T00:13:52.907787+0000","last_became_peered":"2024-04-25T00:13:52.907787+0000","last_unstale":"2024-04-25T00:13:54.353477+0000","last_undegraded":"2024-04-25T00:13:54.353477+0000","last_fullsized":"2024-04-25T00:13:54.353477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:14.879011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458148+0000","last_change":"2024-04-25T00:13:52.912562+0000","last_active":"2024-04-25T00:13:54.458148+0000","last_peered":"2024-04-25T00:13:54.458148+0000","last_clean":"2024-04-25T00:13:54.458148+0000","last_became_active":"2024-04-25T00:13:52.912227+0000","last_became_peered":"2024-04-25T00:13:52.912227+0000","last_unstale":"2024-04-25T00:13:54.458148+0000","last_undegraded":"2024-04-25T00:13:54.458148+0000","last_fullsized":"2024-04-25T00:13:54.458148+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:08.628566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906845+0000","last_change":"2024-04-25T00:13:52.914826+0000","last_active":"2024-04-25T00:13:53.906845+0000","last_peered":"2024-04-25T00:13:53.906845+0000","last_clean":"2024-04-25T00:13:53.906845+0000","last_became_active":"2024-04-25T00:13:52.912612+0000","last_became_peered":"2024-04-25T00:13:52.912612+0000","last_unstale":"2024-04-25T00:13:53.906845+0000","last_undegraded":"2024-04-25T00:13:53.906845+0000","last_fullsized":"2024-04-25T00:13:53.906845+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:20.741425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911706+0000","last_change":"2024-04-25T00:13:52.905235+0000","last_active":"2024-04-25T00:13:53.911706+0000","last_peered":"2024-04-25T00:13:53.911706+0000","last_clean":"2024-04-25T00:13:53.911706+0000","last_became_active":"2024-04-25T00:13:52.904986+0000","last_became_peered":"2024-04-25T00:13:52.904986+0000","last_unstale":"2024-04-25T00:13:53.911706+0000","last_undegraded":"2024-04-25T00:13:53.911706+0000","last_fullsized":"2024-04-25T00:13:53.911706+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:17.850811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457984+0000","last_change":"2024-04-25T00:13:52.908675+0000","last_active":"2024-04-25T00:13:54.457984+0000","last_peered":"2024-04-25T00:13:54.457984+0000","last_clean":"2024-04-25T00:13:54.457984+0000","last_became_active":"2024-04-25T00:13:52.907831+0000","last_became_peered":"2024-04-25T00:13:52.907831+0000","last_unstale":"2024-04-25T00:13:54.457984+0000","last_undegraded":"2024-04-25T00:13:54.457984+0000","last_fullsized":"2024-04-25T00:13:54.457984+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:04.136398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.908713+0000","last_change":"2024-04-25T00:13:52.910077+0000","last_active":"2024-04-25T00:13:53.908713+0000","last_peered":"2024-04-25T00:13:53.908713+0000","last_clean":"2024-04-25T00:13:53.908713+0000","last_became_active":"2024-04-25T00:13:52.909826+0000","last_became_peered":"2024-04-25T00:13:52.909826+0000","last_unstale":"2024-04-25T00:13:53.908713+0000","last_undegraded":"2024-04-25T00:13:53.908713+0000","last_fullsized":"2024-04-25T00:13:53.908713+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:47.596604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457670+0000","last_change":"2024-04-25T00:13:52.907120+0000","last_active":"2024-04-25T00:13:54.457670+0000","last_peered":"2024-04-25T00:13:54.457670+0000","last_clean":"2024-04-25T00:13:54.457670+0000","last_became_active":"2024-04-25T00:13:52.906545+0000","last_became_peered":"2024-04-25T00:13:52.906545+0000","last_unstale":"2024-04-25T00:13:54.457670+0000","last_undegraded":"2024-04-25T00:13:54.457670+0000","last_fullsized":"2024-04-25T00:13:54.457670+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:41.000967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.908716+0000","last_change":"2024-04-25T00:13:52.900681+0000","last_active":"2024-04-25T00:13:53.908716+0000","last_peered":"2024-04-25T00:13:53.908716+0000","last_clean":"2024-04-25T00:13:53.908716+0000","last_became_active":"2024-04-25T00:13:52.900393+0000","last_became_peered":"2024-04-25T00:13:52.900393+0000","last_unstale":"2024-04-25T00:13:53.908716+0000","last_undegraded":"2024-04-25T00:13:53.908716+0000","last_fullsized":"2024-04-25T00:13:53.908716+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:26.234351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"70'4","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907169+0000","last_change":"2024-04-25T00:13:51.907393+0000","last_active":"2024-04-25T00:13:53.907169+0000","last_peered":"2024-04-25T00:13:53.907169+0000","last_clean":"2024-04-25T00:13:53.907169+0000","last_became_active":"2024-04-25T00:13:51.907136+0000","last_became_peered":"2024-04-25T00:13:51.907136+0000","last_unstale":"2024-04-25T00:13:53.907169+0000","last_undegraded":"2024-04-25T00:13:53.907169+0000","last_fullsized":"2024-04-25T00:13:53.907169+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:38.787274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911834+0000","last_change":"2024-04-25T00:13:52.902781+0000","last_active":"2024-04-25T00:13:53.911834+0000","last_peered":"2024-04-25T00:13:53.911834+0000","last_clean":"2024-04-25T00:13:53.911834+0000","last_became_active":"2024-04-25T00:13:52.902530+0000","last_became_peered":"2024-04-25T00:13:52.902530+0000","last_unstale":"2024-04-25T00:13:53.911834+0000","last_undegraded":"2024-04-25T00:13:53.911834+0000","last_fullsized":"2024-04-25T00:13:53.911834+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:04.136398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458290+0000","last_change":"2024-04-25T00:13:51.914058+0000","last_active":"2024-04-25T00:13:54.458290+0000","last_peered":"2024-04-25T00:13:54.458290+0000","last_clean":"2024-04-25T00:13:54.458290+0000","last_became_active":"2024-04-25T00:13:51.913776+0000","last_became_peered":"2024-04-25T00:13:51.913776+0000","last_unstale":"2024-04-25T00:13:54.458290+0000","last_undegraded":"2024-04-25T00:13:54.458290+0000","last_fullsized":"2024-04-25T00:13:54.458290+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:05.397845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457433+0000","last_change":"2024-04-25T00:13:52.931111+0000","last_active":"2024-04-25T00:13:54.457433+0000","last_peered":"2024-04-25T00:13:54.457433+0000","last_clean":"2024-04-25T00:13:54.457433+0000","last_became_active":"2024-04-25T00:13:52.930874+0000","last_became_peered":"2024-04-25T00:13:52.930874+0000","last_unstale":"2024-04-25T00:13:54.457433+0000","last_undegraded":"2024-04-25T00:13:54.457433+0000","last_fullsized":"2024-04-25T00:13:54.457433+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:06.033141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458189+0000","last_change":"2024-04-25T00:13:51.914765+0000","last_active":"2024-04-25T00:13:54.458189+0000","last_peered":"2024-04-25T00:13:54.458189+0000","last_clean":"2024-04-25T00:13:54.458189+0000","last_became_active":"2024-04-25T00:13:51.914501+0000","last_became_peered":"2024-04-25T00:13:51.914501+0000","last_unstale":"2024-04-25T00:13:54.458189+0000","last_undegraded":"2024-04-25T00:13:54.458189+0000","last_fullsized":"2024-04-25T00:13:54.458189+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:28.273299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457432+0000","last_change":"2024-04-25T00:13:52.906553+0000","last_active":"2024-04-25T00:13:54.457432+0000","last_peered":"2024-04-25T00:13:54.457432+0000","last_clean":"2024-04-25T00:13:54.457432+0000","last_became_active":"2024-04-25T00:13:52.905948+0000","last_became_peered":"2024-04-25T00:13:52.905948+0000","last_unstale":"2024-04-25T00:13:54.457432+0000","last_undegraded":"2024-04-25T00:13:54.457432+0000","last_fullsized":"2024-04-25T00:13:54.457432+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:25.244779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906258+0000","last_change":"2024-04-25T00:13:51.906926+0000","last_active":"2024-04-25T00:13:53.906258+0000","last_peered":"2024-04-25T00:13:53.906258+0000","last_clean":"2024-04-25T00:13:53.906258+0000","last_became_active":"2024-04-25T00:13:51.906684+0000","last_became_peered":"2024-04-25T00:13:51.906684+0000","last_unstale":"2024-04-25T00:13:53.906258+0000","last_undegraded":"2024-04-25T00:13:53.906258+0000","last_fullsized":"2024-04-25T00:13:53.906258+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:36.685151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.912163+0000","last_change":"2024-04-25T00:13:52.922296+0000","last_active":"2024-04-25T00:13:53.912163+0000","last_peered":"2024-04-25T00:13:53.912163+0000","last_clean":"2024-04-25T00:13:53.912163+0000","last_became_active":"2024-04-25T00:13:52.921315+0000","last_became_peered":"2024-04-25T00:13:52.921315+0000","last_unstale":"2024-04-25T00:13:53.912163+0000","last_undegraded":"2024-04-25T00:13:53.912163+0000","last_fullsized":"2024-04-25T00:13:53.912163+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:40.637159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457510+0000","last_change":"2024-04-25T00:13:51.914571+0000","last_active":"2024-04-25T00:13:54.457510+0000","last_peered":"2024-04-25T00:13:54.457510+0000","last_clean":"2024-04-25T00:13:54.457510+0000","last_became_active":"2024-04-25T00:13:51.914275+0000","last_became_peered":"2024-04-25T00:13:51.914275+0000","last_unstale":"2024-04-25T00:13:54.457510+0000","last_undegraded":"2024-04-25T00:13:54.457510+0000","last_fullsized":"2024-04-25T00:13:54.457510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:35.572339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911994+0000","last_change":"2024-04-25T00:13:52.922215+0000","last_active":"2024-04-25T00:13:53.911994+0000","last_peered":"2024-04-25T00:13:53.911994+0000","last_clean":"2024-04-25T00:13:53.911994+0000","last_became_active":"2024-04-25T00:13:52.921444+0000","last_became_peered":"2024-04-25T00:13:52.921444+0000","last_unstale":"2024-04-25T00:13:53.911994+0000","last_undegraded":"2024-04-25T00:13:53.911994+0000","last_fullsized":"2024-04-25T00:13:53.911994+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:23.826879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353167+0000","last_change":"2024-04-25T00:13:51.907948+0000","last_active":"2024-04-25T00:13:54.353167+0000","last_peered":"2024-04-25T00:13:54.353167+0000","last_clean":"2024-04-25T00:13:54.353167+0000","last_became_active":"2024-04-25T00:13:51.906221+0000","last_became_peered":"2024-04-25T00:13:51.906221+0000","last_unstale":"2024-04-25T00:13:54.353167+0000","last_undegraded":"2024-04-25T00:13:54.353167+0000","last_fullsized":"2024-04-25T00:13:54.353167+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:40.645229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457761+0000","last_change":"2024-04-25T00:13:52.923747+0000","last_active":"2024-04-25T00:13:54.457761+0000","last_peered":"2024-04-25T00:13:54.457761+0000","last_clean":"2024-04-25T00:13:54.457761+0000","last_became_active":"2024-04-25T00:13:52.923460+0000","last_became_peered":"2024-04-25T00:13:52.923460+0000","last_unstale":"2024-04-25T00:13:54.457761+0000","last_undegraded":"2024-04-25T00:13:54.457761+0000","last_fullsized":"2024-04-25T00:13:54.457761+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:44.419964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911905+0000","last_change":"2024-04-25T00:13:51.902322+0000","last_active":"2024-04-25T00:13:53.911905+0000","last_peered":"2024-04-25T00:13:53.911905+0000","last_clean":"2024-04-25T00:13:53.911905+0000","last_became_active":"2024-04-25T00:13:51.902053+0000","last_became_peered":"2024-04-25T00:13:51.902053+0000","last_unstale":"2024-04-25T00:13:53.911905+0000","last_undegraded":"2024-04-25T00:13:53.911905+0000","last_fullsized":"2024-04-25T00:13:53.911905+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:19.259348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906613+0000","last_change":"2024-04-25T00:13:52.911436+0000","last_active":"2024-04-25T00:13:53.906613+0000","last_peered":"2024-04-25T00:13:53.906613+0000","last_clean":"2024-04-25T00:13:53.906613+0000","last_became_active":"2024-04-25T00:13:52.911166+0000","last_became_peered":"2024-04-25T00:13:52.911166+0000","last_unstale":"2024-04-25T00:13:53.906613+0000","last_undegraded":"2024-04-25T00:13:53.906613+0000","last_fullsized":"2024-04-25T00:13:53.906613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:04.818049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906629+0000","last_change":"2024-04-25T00:13:51.905908+0000","last_active":"2024-04-25T00:13:53.906629+0000","last_peered":"2024-04-25T00:13:53.906629+0000","last_clean":"2024-04-25T00:13:53.906629+0000","last_became_active":"2024-04-25T00:13:51.905597+0000","last_became_peered":"2024-04-25T00:13:51.905597+0000","last_unstale":"2024-04-25T00:13:53.906629+0000","last_undegraded":"2024-04-25T00:13:53.906629+0000","last_fullsized":"2024-04-25T00:13:53.906629+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:49.685037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911539+0000","last_change":"2024-04-25T00:13:52.906493+0000","last_active":"2024-04-25T00:13:53.911539+0000","last_peered":"2024-04-25T00:13:53.911539+0000","last_clean":"2024-04-25T00:13:53.911539+0000","last_became_active":"2024-04-25T00:13:52.906238+0000","last_became_peered":"2024-04-25T00:13:52.906238+0000","last_unstale":"2024-04-25T00:13:53.911539+0000","last_undegraded":"2024-04-25T00:13:53.911539+0000","last_fullsized":"2024-04-25T00:13:53.911539+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:48.458973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458387+0000","last_change":"2024-04-25T00:13:51.914190+0000","last_active":"2024-04-25T00:13:54.458387+0000","last_peered":"2024-04-25T00:13:54.458387+0000","last_clean":"2024-04-25T00:13:54.458387+0000","last_became_active":"2024-04-25T00:13:51.913929+0000","last_became_peered":"2024-04-25T00:13:51.913929+0000","last_unstale":"2024-04-25T00:13:54.458387+0000","last_undegraded":"2024-04-25T00:13:54.458387+0000","last_fullsized":"2024-04-25T00:13:54.458387+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:10.915459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353276+0000","last_change":"2024-04-25T00:13:52.910690+0000","last_active":"2024-04-25T00:13:54.353276+0000","last_peered":"2024-04-25T00:13:54.353276+0000","last_clean":"2024-04-25T00:13:54.353276+0000","last_became_active":"2024-04-25T00:13:52.910122+0000","last_became_peered":"2024-04-25T00:13:52.910122+0000","last_unstale":"2024-04-25T00:13:54.353276+0000","last_undegraded":"2024-04-25T00:13:54.353276+0000","last_fullsized":"2024-04-25T00:13:54.353276+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:16.820300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353474+0000","last_change":"2024-04-25T00:13:51.909299+0000","last_active":"2024-04-25T00:13:54.353474+0000","last_peered":"2024-04-25T00:13:54.353474+0000","last_clean":"2024-04-25T00:13:54.353474+0000","last_became_active":"2024-04-25T00:13:51.908909+0000","last_became_peered":"2024-04-25T00:13:51.908909+0000","last_unstale":"2024-04-25T00:13:54.353474+0000","last_undegraded":"2024-04-25T00:13:54.353474+0000","last_fullsized":"2024-04-25T00:13:54.353474+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:25.345470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457623+0000","last_change":"2024-04-25T00:13:52.932015+0000","last_active":"2024-04-25T00:13:54.457623+0000","last_peered":"2024-04-25T00:13:54.457623+0000","last_clean":"2024-04-25T00:13:54.457623+0000","last_became_active":"2024-04-25T00:13:52.931712+0000","last_became_peered":"2024-04-25T00:13:52.931712+0000","last_unstale":"2024-04-25T00:13:54.457623+0000","last_undegraded":"2024-04-25T00:13:54.457623+0000","last_fullsized":"2024-04-25T00:13:54.457623+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:35.604240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"70'3","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907076+0000","last_change":"2024-04-25T00:13:51.910876+0000","last_active":"2024-04-25T00:13:53.907076+0000","last_peered":"2024-04-25T00:13:53.907076+0000","last_clean":"2024-04-25T00:13:53.907076+0000","last_became_active":"2024-04-25T00:13:51.910611+0000","last_became_peered":"2024-04-25T00:13:51.910611+0000","last_unstale":"2024-04-25T00:13:53.907076+0000","last_undegraded":"2024-04-25T00:13:53.907076+0000","last_fullsized":"2024-04-25T00:13:53.907076+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:36.385074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.908899+0000","last_change":"2024-04-25T00:13:52.903692+0000","last_active":"2024-04-25T00:13:53.908899+0000","last_peered":"2024-04-25T00:13:53.908899+0000","last_clean":"2024-04-25T00:13:53.908899+0000","last_became_active":"2024-04-25T00:13:52.903405+0000","last_became_peered":"2024-04-25T00:13:52.903405+0000","last_unstale":"2024-04-25T00:13:53.908899+0000","last_undegraded":"2024-04-25T00:13:53.908899+0000","last_fullsized":"2024-04-25T00:13:53.908899+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:07.076263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"70'6","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906385+0000","last_change":"2024-04-25T00:13:51.905851+0000","last_active":"2024-04-25T00:13:53.906385+0000","last_peered":"2024-04-25T00:13:53.906385+0000","last_clean":"2024-04-25T00:13:53.906385+0000","last_became_active":"2024-04-25T00:13:51.904311+0000","last_became_peered":"2024-04-25T00:13:51.904311+0000","last_unstale":"2024-04-25T00:13:53.906385+0000","last_undegraded":"2024-04-25T00:13:53.906385+0000","last_fullsized":"2024-04-25T00:13:53.906385+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:05.027309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353553+0000","last_change":"2024-04-25T00:13:52.926495+0000","last_active":"2024-04-25T00:13:54.353553+0000","last_peered":"2024-04-25T00:13:54.353553+0000","last_clean":"2024-04-25T00:13:54.353553+0000","last_became_active":"2024-04-25T00:13:52.926240+0000","last_became_peered":"2024-04-25T00:13:52.926240+0000","last_unstale":"2024-04-25T00:13:54.353553+0000","last_undegraded":"2024-04-25T00:13:54.353553+0000","last_fullsized":"2024-04-25T00:13:54.353553+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:48.458973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353147+0000","last_change":"2024-04-25T00:13:51.904959+0000","last_active":"2024-04-25T00:13:54.353147+0000","last_peered":"2024-04-25T00:13:54.353147+0000","last_clean":"2024-04-25T00:13:54.353147+0000","last_became_active":"2024-04-25T00:13:51.904681+0000","last_became_peered":"2024-04-25T00:13:51.904681+0000","last_unstale":"2024-04-25T00:13:54.353147+0000","last_undegraded":"2024-04-25T00:13:54.353147+0000","last_fullsized":"2024-04-25T00:13:54.353147+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:03.937735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906104+0000","last_change":"2024-04-25T00:13:52.909363+0000","last_active":"2024-04-25T00:13:53.906104+0000","last_peered":"2024-04-25T00:13:53.906104+0000","last_clean":"2024-04-25T00:13:53.906104+0000","last_became_active":"2024-04-25T00:13:52.908357+0000","last_became_peered":"2024-04-25T00:13:52.908357+0000","last_unstale":"2024-04-25T00:13:53.906104+0000","last_undegraded":"2024-04-25T00:13:53.906104+0000","last_fullsized":"2024-04-25T00:13:53.906104+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:16.460099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457910+0000","last_change":"2024-04-25T00:13:51.913490+0000","last_active":"2024-04-25T00:13:54.457910+0000","last_peered":"2024-04-25T00:13:54.457910+0000","last_clean":"2024-04-25T00:13:54.457910+0000","last_became_active":"2024-04-25T00:13:51.913007+0000","last_became_peered":"2024-04-25T00:13:51.913007+0000","last_unstale":"2024-04-25T00:13:54.457910+0000","last_undegraded":"2024-04-25T00:13:54.457910+0000","last_fullsized":"2024-04-25T00:13:54.457910+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:23.164680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906553+0000","last_change":"2024-04-25T00:13:52.910658+0000","last_active":"2024-04-25T00:13:53.906553+0000","last_peered":"2024-04-25T00:13:53.906553+0000","last_clean":"2024-04-25T00:13:53.906553+0000","last_became_active":"2024-04-25T00:13:52.910334+0000","last_became_peered":"2024-04-25T00:13:52.910334+0000","last_unstale":"2024-04-25T00:13:53.906553+0000","last_undegraded":"2024-04-25T00:13:53.906553+0000","last_fullsized":"2024-04-25T00:13:53.906553+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:20.265179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906752+0000","last_change":"2024-04-25T00:13:51.908752+0000","last_active":"2024-04-25T00:13:53.906752+0000","last_peered":"2024-04-25T00:13:53.906752+0000","last_clean":"2024-04-25T00:13:53.906752+0000","last_became_active":"2024-04-25T00:13:51.906157+0000","last_became_peered":"2024-04-25T00:13:51.906157+0000","last_unstale":"2024-04-25T00:13:53.906752+0000","last_undegraded":"2024-04-25T00:13:53.906752+0000","last_fullsized":"2024-04-25T00:13:53.906752+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:12.153802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906585+0000","last_change":"2024-04-25T00:13:52.930550+0000","last_active":"2024-04-25T00:13:53.906585+0000","last_peered":"2024-04-25T00:13:53.906585+0000","last_clean":"2024-04-25T00:13:53.906585+0000","last_became_active":"2024-04-25T00:13:52.930277+0000","last_became_peered":"2024-04-25T00:13:52.930277+0000","last_unstale":"2024-04-25T00:13:53.906585+0000","last_undegraded":"2024-04-25T00:13:53.906585+0000","last_fullsized":"2024-04-25T00:13:53.906585+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:41.651061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"70'8","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911735+0000","last_change":"2024-04-25T00:13:51.910161+0000","last_active":"2024-04-25T00:13:53.911735+0000","last_peered":"2024-04-25T00:13:53.911735+0000","last_clean":"2024-04-25T00:13:53.911735+0000","last_became_active":"2024-04-25T00:13:51.909865+0000","last_became_peered":"2024-04-25T00:13:51.909865+0000","last_unstale":"2024-04-25T00:13:53.911735+0000","last_undegraded":"2024-04-25T00:13:53.911735+0000","last_fullsized":"2024-04-25T00:13:53.911735+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:14:18.998312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457185+0000","last_change":"2024-04-25T00:13:52.921126+0000","last_active":"2024-04-25T00:13:54.457185+0000","last_peered":"2024-04-25T00:13:54.457185+0000","last_clean":"2024-04-25T00:13:54.457185+0000","last_became_active":"2024-04-25T00:13:52.920870+0000","last_became_peered":"2024-04-25T00:13:52.920870+0000","last_unstale":"2024-04-25T00:13:54.457185+0000","last_undegraded":"2024-04-25T00:13:54.457185+0000","last_fullsized":"2024-04-25T00:13:54.457185+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:23.826879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458445+0000","last_change":"2024-04-25T00:13:51.916352+0000","last_active":"2024-04-25T00:13:54.458445+0000","last_peered":"2024-04-25T00:13:54.458445+0000","last_clean":"2024-04-25T00:13:54.458445+0000","last_became_active":"2024-04-25T00:13:51.915993+0000","last_became_peered":"2024-04-25T00:13:51.915993+0000","last_unstale":"2024-04-25T00:13:54.458445+0000","last_undegraded":"2024-04-25T00:13:54.458445+0000","last_fullsized":"2024-04-25T00:13:54.458445+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:05.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906685+0000","last_change":"2024-04-25T00:13:52.911234+0000","last_active":"2024-04-25T00:13:53.906685+0000","last_peered":"2024-04-25T00:13:53.906685+0000","last_clean":"2024-04-25T00:13:53.906685+0000","last_became_active":"2024-04-25T00:13:52.910959+0000","last_became_peered":"2024-04-25T00:13:52.910959+0000","last_unstale":"2024-04-25T00:13:53.906685+0000","last_undegraded":"2024-04-25T00:13:53.906685+0000","last_fullsized":"2024-04-25T00:13:53.906685+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:36.880859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457738+0000","last_change":"2024-04-25T00:13:51.909081+0000","last_active":"2024-04-25T00:13:54.457738+0000","last_peered":"2024-04-25T00:13:54.457738+0000","last_clean":"2024-04-25T00:13:54.457738+0000","last_became_active":"2024-04-25T00:13:51.908817+0000","last_became_peered":"2024-04-25T00:13:51.908817+0000","last_unstale":"2024-04-25T00:13:54.457738+0000","last_undegraded":"2024-04-25T00:13:54.457738+0000","last_fullsized":"2024-04-25T00:13:54.457738+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:35.550486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906729+0000","last_change":"2024-04-25T00:13:52.915676+0000","last_active":"2024-04-25T00:13:53.906729+0000","last_peered":"2024-04-25T00:13:53.906729+0000","last_clean":"2024-04-25T00:13:53.906729+0000","last_became_active":"2024-04-25T00:13:52.913972+0000","last_became_peered":"2024-04-25T00:13:52.913972+0000","last_unstale":"2024-04-25T00:13:53.906729+0000","last_undegraded":"2024-04-25T00:13:53.906729+0000","last_fullsized":"2024-04-25T00:13:53.906729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:16.298509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907119+0000","last_change":"2024-04-25T00:13:51.908325+0000","last_active":"2024-04-25T00:13:53.907119+0000","last_peered":"2024-04-25T00:13:53.907119+0000","last_clean":"2024-04-25T00:13:53.907119+0000","last_became_active":"2024-04-25T00:13:51.907027+0000","last_became_peered":"2024-04-25T00:13:51.907027+0000","last_unstale":"2024-04-25T00:13:53.907119+0000","last_undegraded":"2024-04-25T00:13:53.907119+0000","last_fullsized":"2024-04-25T00:13:53.907119+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:21.299318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457367+0000","last_change":"2024-04-25T00:13:52.923300+0000","last_active":"2024-04-25T00:13:54.457367+0000","last_peered":"2024-04-25T00:13:54.457367+0000","last_clean":"2024-04-25T00:13:54.457367+0000","last_became_active":"2024-04-25T00:13:52.922706+0000","last_became_peered":"2024-04-25T00:13:52.922706+0000","last_unstale":"2024-04-25T00:13:54.457367+0000","last_undegraded":"2024-04-25T00:13:54.457367+0000","last_fullsized":"2024-04-25T00:13:54.457367+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:14:01.301770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.908887+0000","last_change":"2024-04-25T00:13:51.906615+0000","last_active":"2024-04-25T00:13:53.908887+0000","last_peered":"2024-04-25T00:13:53.908887+0000","last_clean":"2024-04-25T00:13:53.908887+0000","last_became_active":"2024-04-25T00:13:51.906366+0000","last_became_peered":"2024-04-25T00:13:51.906366+0000","last_unstale":"2024-04-25T00:13:53.908887+0000","last_undegraded":"2024-04-25T00:13:53.908887+0000","last_fullsized":"2024-04-25T00:13:53.908887+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:48.930850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911954+0000","last_change":"2024-04-25T00:13:52.902737+0000","last_active":"2024-04-25T00:13:53.911954+0000","last_peered":"2024-04-25T00:13:53.911954+0000","last_clean":"2024-04-25T00:13:53.911954+0000","last_became_active":"2024-04-25T00:13:52.902435+0000","last_became_peered":"2024-04-25T00:13:52.902435+0000","last_unstale":"2024-04-25T00:13:53.911954+0000","last_undegraded":"2024-04-25T00:13:53.911954+0000","last_fullsized":"2024-04-25T00:13:53.911954+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:38.492799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457380+0000","last_change":"2024-04-25T00:13:51.898425+0000","last_active":"2024-04-25T00:13:54.457380+0000","last_peered":"2024-04-25T00:13:54.457380+0000","last_clean":"2024-04-25T00:13:54.457380+0000","last_became_active":"2024-04-25T00:13:51.898156+0000","last_became_peered":"2024-04-25T00:13:51.898156+0000","last_unstale":"2024-04-25T00:13:54.457380+0000","last_undegraded":"2024-04-25T00:13:54.457380+0000","last_fullsized":"2024-04-25T00:13:54.457380+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:13.777054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.912080+0000","last_change":"2024-04-25T00:13:52.897712+0000","last_active":"2024-04-25T00:13:53.912080+0000","last_peered":"2024-04-25T00:13:53.912080+0000","last_clean":"2024-04-25T00:13:53.912080+0000","last_became_active":"2024-04-25T00:13:52.897448+0000","last_became_peered":"2024-04-25T00:13:52.897448+0000","last_unstale":"2024-04-25T00:13:53.912080+0000","last_undegraded":"2024-04-25T00:13:53.912080+0000","last_fullsized":"2024-04-25T00:13:53.912080+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:14:01.301770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457216+0000","last_change":"2024-04-25T00:13:51.904095+0000","last_active":"2024-04-25T00:13:54.457216+0000","last_peered":"2024-04-25T00:13:54.457216+0000","last_clean":"2024-04-25T00:13:54.457216+0000","last_became_active":"2024-04-25T00:13:51.902301+0000","last_became_peered":"2024-04-25T00:13:51.902301+0000","last_unstale":"2024-04-25T00:13:54.457216+0000","last_undegraded":"2024-04-25T00:13:54.457216+0000","last_fullsized":"2024-04-25T00:13:54.457216+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:47.297549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906750+0000","last_change":"2024-04-25T00:13:52.910412+0000","last_active":"2024-04-25T00:13:53.906750+0000","last_peered":"2024-04-25T00:13:53.906750+0000","last_clean":"2024-04-25T00:13:53.906750+0000","last_became_active":"2024-04-25T00:13:52.910103+0000","last_became_peered":"2024-04-25T00:13:52.910103+0000","last_unstale":"2024-04-25T00:13:53.906750+0000","last_undegraded":"2024-04-25T00:13:53.906750+0000","last_fullsized":"2024-04-25T00:13:53.906750+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:04.943567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457158+0000","last_change":"2024-04-25T00:13:51.901915+0000","last_active":"2024-04-25T00:13:54.457158+0000","last_peered":"2024-04-25T00:13:54.457158+0000","last_clean":"2024-04-25T00:13:54.457158+0000","last_became_active":"2024-04-25T00:13:51.901676+0000","last_became_peered":"2024-04-25T00:13:51.901676+0000","last_unstale":"2024-04-25T00:13:54.457158+0000","last_undegraded":"2024-04-25T00:13:54.457158+0000","last_fullsized":"2024-04-25T00:13:54.457158+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:13.873180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457190+0000","last_change":"2024-04-25T00:13:52.906083+0000","last_active":"2024-04-25T00:13:54.457190+0000","last_peered":"2024-04-25T00:13:54.457190+0000","last_clean":"2024-04-25T00:13:54.457190+0000","last_became_active":"2024-04-25T00:13:52.905804+0000","last_became_peered":"2024-04-25T00:13:52.905804+0000","last_unstale":"2024-04-25T00:13:54.457190+0000","last_undegraded":"2024-04-25T00:13:54.457190+0000","last_fullsized":"2024-04-25T00:13:54.457190+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:37.690983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353423+0000","last_change":"2024-04-25T00:13:51.910660+0000","last_active":"2024-04-25T00:13:54.353423+0000","last_peered":"2024-04-25T00:13:54.353423+0000","last_clean":"2024-04-25T00:13:54.353423+0000","last_became_active":"2024-04-25T00:13:51.910397+0000","last_became_peered":"2024-04-25T00:13:51.910397+0000","last_unstale":"2024-04-25T00:13:54.353423+0000","last_undegraded":"2024-04-25T00:13:54.353423+0000","last_fullsized":"2024-04-25T00:13:54.353423+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:03.812218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906331+0000","last_change":"2024-04-25T00:13:52.925029+0000","last_active":"2024-04-25T00:13:53.906331+0000","last_peered":"2024-04-25T00:13:53.906331+0000","last_clean":"2024-04-25T00:13:53.906331+0000","last_became_active":"2024-04-25T00:13:52.924705+0000","last_became_peered":"2024-04-25T00:13:52.924705+0000","last_unstale":"2024-04-25T00:13:53.906331+0000","last_undegraded":"2024-04-25T00:13:53.906331+0000","last_fullsized":"2024-04-25T00:13:53.906331+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:14:20.004143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"71'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.913412+0000","last_change":"2024-04-25T00:13:51.908015+0000","last_active":"2024-04-25T00:13:53.913412+0000","last_peered":"2024-04-25T00:13:53.913412+0000","last_clean":"2024-04-25T00:13:53.913412+0000","last_became_active":"2024-04-25T00:13:51.906803+0000","last_became_peered":"2024-04-25T00:13:51.906803+0000","last_unstale":"2024-04-25T00:13:53.913412+0000","last_undegraded":"2024-04-25T00:13:53.913412+0000","last_fullsized":"2024-04-25T00:13:53.913412+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:40.645229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457618+0000","last_change":"2024-04-25T00:13:52.924432+0000","last_active":"2024-04-25T00:13:54.457618+0000","last_peered":"2024-04-25T00:13:54.457618+0000","last_clean":"2024-04-25T00:13:54.457618+0000","last_became_active":"2024-04-25T00:13:52.924165+0000","last_became_peered":"2024-04-25T00:13:52.924165+0000","last_unstale":"2024-04-25T00:13:54.457618+0000","last_undegraded":"2024-04-25T00:13:54.457618+0000","last_fullsized":"2024-04-25T00:13:54.457618+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:51.781844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458336+0000","last_change":"2024-04-25T00:13:51.915284+0000","last_active":"2024-04-25T00:13:54.458336+0000","last_peered":"2024-04-25T00:13:54.458336+0000","last_clean":"2024-04-25T00:13:54.458336+0000","last_became_active":"2024-04-25T00:13:51.915014+0000","last_became_peered":"2024-04-25T00:13:51.915014+0000","last_unstale":"2024-04-25T00:13:54.458336+0000","last_undegraded":"2024-04-25T00:13:54.458336+0000","last_fullsized":"2024-04-25T00:13:54.458336+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:15.814468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911792+0000","last_change":"2024-04-25T00:13:52.906600+0000","last_active":"2024-04-25T00:13:53.911792+0000","last_peered":"2024-04-25T00:13:53.911792+0000","last_clean":"2024-04-25T00:13:53.911792+0000","last_became_active":"2024-04-25T00:13:52.906329+0000","last_became_peered":"2024-04-25T00:13:52.906329+0000","last_unstale":"2024-04-25T00:13:53.911792+0000","last_undegraded":"2024-04-25T00:13:53.911792+0000","last_fullsized":"2024-04-25T00:13:53.911792+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:41.000967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458005+0000","last_change":"2024-04-25T00:13:51.914352+0000","last_active":"2024-04-25T00:13:54.458005+0000","last_peered":"2024-04-25T00:13:54.458005+0000","last_clean":"2024-04-25T00:13:54.458005+0000","last_became_active":"2024-04-25T00:13:51.914019+0000","last_became_peered":"2024-04-25T00:13:51.914019+0000","last_unstale":"2024-04-25T00:13:54.458005+0000","last_undegraded":"2024-04-25T00:13:54.458005+0000","last_fullsized":"2024-04-25T00:13:54.458005+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:43.614223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906140+0000","last_change":"2024-04-25T00:13:52.930622+0000","last_active":"2024-04-25T00:13:53.906140+0000","last_peered":"2024-04-25T00:13:53.906140+0000","last_clean":"2024-04-25T00:13:53.906140+0000","last_became_active":"2024-04-25T00:13:52.930272+0000","last_became_peered":"2024-04-25T00:13:52.930272+0000","last_unstale":"2024-04-25T00:13:53.906140+0000","last_undegraded":"2024-04-25T00:13:53.906140+0000","last_fullsized":"2024-04-25T00:13:53.906140+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:20.863218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906440+0000","last_change":"2024-04-25T00:13:51.905739+0000","last_active":"2024-04-25T00:13:53.906440+0000","last_peered":"2024-04-25T00:13:53.906440+0000","last_clean":"2024-04-25T00:13:53.906440+0000","last_became_active":"2024-04-25T00:13:51.905453+0000","last_became_peered":"2024-04-25T00:13:51.905453+0000","last_unstale":"2024-04-25T00:13:53.906440+0000","last_undegraded":"2024-04-25T00:13:53.906440+0000","last_fullsized":"2024-04-25T00:13:53.906440+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:48.930850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906191+0000","last_change":"2024-04-25T00:13:52.911888+0000","last_active":"2024-04-25T00:13:53.906191+0000","last_peered":"2024-04-25T00:13:53.906191+0000","last_clean":"2024-04-25T00:13:53.906191+0000","last_became_active":"2024-04-25T00:13:52.911620+0000","last_became_peered":"2024-04-25T00:13:52.911620+0000","last_unstale":"2024-04-25T00:13:53.906191+0000","last_undegraded":"2024-04-25T00:13:53.906191+0000","last_fullsized":"2024-04-25T00:13:53.906191+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:10.399293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907155+0000","last_change":"2024-04-25T00:13:51.910063+0000","last_active":"2024-04-25T00:13:53.907155+0000","last_peered":"2024-04-25T00:13:53.907155+0000","last_clean":"2024-04-25T00:13:53.907155+0000","last_became_active":"2024-04-25T00:13:51.909814+0000","last_became_peered":"2024-04-25T00:13:51.909814+0000","last_unstale":"2024-04-25T00:13:53.907155+0000","last_undegraded":"2024-04-25T00:13:53.907155+0000","last_fullsized":"2024-04-25T00:13:53.907155+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:00.742666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.909206+0000","last_change":"2024-04-25T00:13:52.902612+0000","last_active":"2024-04-25T00:13:53.909206+0000","last_peered":"2024-04-25T00:13:53.909206+0000","last_clean":"2024-04-25T00:13:53.909206+0000","last_became_active":"2024-04-25T00:13:52.902290+0000","last_became_peered":"2024-04-25T00:13:52.902290+0000","last_unstale":"2024-04-25T00:13:53.909206+0000","last_undegraded":"2024-04-25T00:13:53.909206+0000","last_fullsized":"2024-04-25T00:13:53.909206+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:08.688249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906356+0000","last_change":"2024-04-25T00:13:51.902090+0000","last_active":"2024-04-25T00:13:53.906356+0000","last_peered":"2024-04-25T00:13:53.906356+0000","last_clean":"2024-04-25T00:13:53.906356+0000","last_became_active":"2024-04-25T00:13:51.901819+0000","last_became_peered":"2024-04-25T00:13:51.901819+0000","last_unstale":"2024-04-25T00:13:53.906356+0000","last_undegraded":"2024-04-25T00:13:53.906356+0000","last_fullsized":"2024-04-25T00:13:53.906356+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:00.849796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.909284+0000","last_change":"2024-04-25T00:13:52.907368+0000","last_active":"2024-04-25T00:13:53.909284+0000","last_peered":"2024-04-25T00:13:53.909284+0000","last_clean":"2024-04-25T00:13:53.909284+0000","last_became_active":"2024-04-25T00:13:52.907038+0000","last_became_peered":"2024-04-25T00:13:52.907038+0000","last_unstale":"2024-04-25T00:13:53.909284+0000","last_undegraded":"2024-04-25T00:13:53.909284+0000","last_fullsized":"2024-04-25T00:13:53.909284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:10.399293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.908648+0000","last_change":"2024-04-25T00:13:51.905204+0000","last_active":"2024-04-25T00:13:53.908648+0000","last_peered":"2024-04-25T00:13:53.908648+0000","last_clean":"2024-04-25T00:13:53.908648+0000","last_became_active":"2024-04-25T00:13:51.904954+0000","last_became_peered":"2024-04-25T00:13:51.904954+0000","last_unstale":"2024-04-25T00:13:53.908648+0000","last_undegraded":"2024-04-25T00:13:53.908648+0000","last_fullsized":"2024-04-25T00:13:53.908648+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:36.685151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458331+0000","last_change":"2024-04-25T00:13:52.915444+0000","last_active":"2024-04-25T00:13:54.458331+0000","last_peered":"2024-04-25T00:13:54.458331+0000","last_clean":"2024-04-25T00:13:54.458331+0000","last_became_active":"2024-04-25T00:13:52.915151+0000","last_became_peered":"2024-04-25T00:13:52.915151+0000","last_unstale":"2024-04-25T00:13:54.458331+0000","last_undegraded":"2024-04-25T00:13:54.458331+0000","last_fullsized":"2024-04-25T00:13:54.458331+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:36.719641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457626+0000","last_change":"2024-04-25T00:13:51.906121+0000","last_active":"2024-04-25T00:13:54.457626+0000","last_peered":"2024-04-25T00:13:54.457626+0000","last_clean":"2024-04-25T00:13:54.457626+0000","last_became_active":"2024-04-25T00:13:51.905829+0000","last_became_peered":"2024-04-25T00:13:51.905829+0000","last_unstale":"2024-04-25T00:13:54.457626+0000","last_undegraded":"2024-04-25T00:13:54.457626+0000","last_fullsized":"2024-04-25T00:13:54.457626+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:47.453141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":263,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457701+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:13:54.457701+0000","last_peered":"2024-04-25T00:13:54.457701+0000","last_clean":"2024-04-25T00:13:54.457701+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:13:54.457701+0000","last_undegraded":"2024-04-25T00:13:54.457701+0000","last_fullsized":"2024-04-25T00:13:54.457701+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1454080,"data_stored":976262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":387484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":392598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":387588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":386454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":386614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":392676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1449984,"data_stored":976240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:13:56.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:13:57.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.148+0000 7f478a000640 1 Processor -- start 2024-04-25T00:13:57.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.148+0000 7f478a000640 1 -- start start 2024-04-25T00:13:57.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.148+0000 7f478a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4784164cb0 0x7f47841670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.148+0000 7f478a000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47840ccc60 con 0x7f47840cc510 2024-04-25T00:13:57.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.148+0000 7f478a000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f478406dba0 con 0x7f47841599f0 2024-04-25T00:13:57.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.148+0000 7f478a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4784159dd0 con 0x7f4784164cb0 2024-04-25T00:13:57.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.152+0000 7f477aa00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f47840cc510 0x7f478415ca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33144/0 (socket says 172.21.15.83:33144) 2024-04-25T00:13:57.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.152+0000 7f477aa00640 1 -- 172.21.15.83:0/3445107143 learned_addr learned my addr 172.21.15.83:0/3445107143 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:57.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.152+0000 7f4783e00640 1 --2- 172.21.15.83:0/3445107143 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4784164cb0 0x7f47841670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.152+0000 7f4782a00640 1 -- 172.21.15.83:0/3445107143 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1736898984 0 0) 0x7f476c001120 con 0x7f47840cc510 2024-04-25T00:13:57.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.152+0000 7f4782a00640 1 -- 172.21.15.83:0/3445107143 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4768003530 con 0x7f47840cc510 2024-04-25T00:13:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.152+0000 7f4782a00640 1 -- 172.21.15.83:0/3445107143 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 493056440 0 0) 0x7f478406dba0 con 0x7f47841599f0 2024-04-25T00:13:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.152+0000 7f4782a00640 1 -- 172.21.15.83:0/3445107143 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f476c001120 con 0x7f47841599f0 2024-04-25T00:13:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.152+0000 7f4783e00640 1 -- 172.21.15.83:0/3445107143 >> v1:172.21.15.86:6789/0 conn(0x7f47841599f0 legacy=0x7f478415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.152+0000 7f4783e00640 1 -- 172.21.15.83:0/3445107143 >> v1:172.21.15.83:6789/0 conn(0x7f47840cc510 legacy=0x7f478415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.152+0000 7f4783e00640 1 -- 172.21.15.83:0/3445107143 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478406de80 con 0x7f4784164cb0 2024-04-25T00:13:57.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.152+0000 7f4783e00640 1 --2- 172.21.15.83:0/3445107143 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4784164cb0 0x7f47841670a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f477c009820 tx=0x7f477c01c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c8932d334abad08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:57.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f4782a00640 1 -- 172.21.15.83:0/3445107143 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c0b0070 con 0x7f4784164cb0 2024-04-25T00:13:57.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f4782a00640 1 -- 172.21.15.83:0/3445107143 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477c0ab600 con 0x7f4784164cb0 2024-04-25T00:13:57.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f4782a00640 1 -- 172.21.15.83:0/3445107143 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c0ab9b0 con 0x7f4784164cb0 2024-04-25T00:13:57.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 -- 172.21.15.83:0/3445107143 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4784164cb0 msgr2=0x7f47841670a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 --2- 172.21.15.83:0/3445107143 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4784164cb0 0x7f47841670a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f477c009820 tx=0x7f477c01c430 comp rx=0 tx=0).stop 2024-04-25T00:13:57.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 -- 172.21.15.83:0/3445107143 shutdown_connections 2024-04-25T00:13:57.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 --2- 172.21.15.83:0/3445107143 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4784164cb0 0x7f47841670a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 -- 172.21.15.83:0/3445107143 >> 172.21.15.83:0/3445107143 conn(0x7f47840b1830 msgr2=0x7f47840b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:57.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 -- 172.21.15.83:0/3445107143 shutdown_connections 2024-04-25T00:13:57.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 -- 172.21.15.83:0/3445107143 wait complete. 2024-04-25T00:13:57.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 Processor -- start 2024-04-25T00:13:57.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 -- start start 2024-04-25T00:13:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47840cc510 0x7f4784165220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47841599f0 0x7f4784150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4784164cb0 0x7f4784154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f47840ce240 con 0x7f4784164cb0 2024-04-25T00:13:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47840ce0c0 con 0x7f47840cc510 2024-04-25T00:13:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f478a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f47840cf6f0 con 0x7f47841599f0 2024-04-25T00:13:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f4783e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4784164cb0 0x7f4784154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f4783e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4784164cb0 0x7f4784154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45144/0 (socket says 172.21.15.83:45144) 2024-04-25T00:13:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f4783e00640 1 -- 172.21.15.83:0/3732504463 learned_addr learned my addr 172.21.15.83:0/3732504463 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f477aa00640 1 --2- 172.21.15.83:0/3732504463 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47841599f0 0x7f4784150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.156+0000 7f4783400640 1 --2- 172.21.15.83:0/3732504463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47840cc510 0x7f4784165220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f4783e00640 1 -- 172.21.15.83:0/3732504463 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47841599f0 msgr2=0x7f4784150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f4783e00640 1 --2- 172.21.15.83:0/3732504463 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47841599f0 0x7f4784150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f4783e00640 1 -- 172.21.15.83:0/3732504463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47840cc510 msgr2=0x7f4784165220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f4783e00640 1 --2- 172.21.15.83:0/3732504463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47840cc510 0x7f4784165220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f4783e00640 1 -- 172.21.15.83:0/3732504463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47840b14c0 con 0x7f4784164cb0 2024-04-25T00:13:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f4783e00640 1 --2- 172.21.15.83:0/3732504463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4784164cb0 0x7f4784154510 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f477c009820 tx=0x7f477c0191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f477be00640 1 -- 172.21.15.83:0/3732504463 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c0b0040 con 0x7f4784164cb0 2024-04-25T00:13:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f477be00640 1 -- 172.21.15.83:0/3732504463 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477c0ab7b0 con 0x7f4784164cb0 2024-04-25T00:13:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f477be00640 1 -- 172.21.15.83:0/3732504463 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c0abb60 con 0x7f4784164cb0 2024-04-25T00:13:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f478a000640 1 -- 172.21.15.83:0/3732504463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47840b34d0 con 0x7f4784164cb0 2024-04-25T00:13:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f478a000640 1 -- 172.21.15.83:0/3732504463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4784165a00 con 0x7f4784164cb0 2024-04-25T00:13:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f478a000640 1 -- 172.21.15.83:0/3732504463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47840737f0 con 0x7f4784164cb0 2024-04-25T00:13:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f477be00640 1 -- 172.21.15.83:0/3732504463 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f477c0abd00 con 0x7f4784164cb0 2024-04-25T00:13:57.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f477be00640 1 --2- 172.21.15.83:0/3732504463 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47600678c0 0x7f4760069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.160+0000 7f477be00640 1 -- 172.21.15.83:0/3732504463 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f477c11fd30 con 0x7f4784164cb0 2024-04-25T00:13:57.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.164+0000 7f4783400640 1 --2- 172.21.15.83:0/3732504463 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47600678c0 0x7f4760069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.164+0000 7f4783400640 1 --2- 172.21.15.83:0/3732504463 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47600678c0 0x7f4760069d80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4774007fd0 tx=0x7f4774009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:57.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.164+0000 7f477be00640 1 -- 172.21.15.83:0/3732504463 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f477c0160b0 con 0x7f4784164cb0 2024-04-25T00:13:57.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.316+0000 7f478a000640 1 -- 172.21.15.83:0/3732504463 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f47840716c0 con 0x7f47600678c0 2024-04-25T00:13:57.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.324+0000 7f477be00640 1 -- 172.21.15.83:0/3732504463 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252067 (secure 0 0 0) 0x7f47840716c0 con 0x7f47600678c0 2024-04-25T00:13:57.329 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:57.331 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:13:57.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.328+0000 7f478a000640 1 -- 172.21.15.83:0/3732504463 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47600678c0 msgr2=0x7f4760069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.328+0000 7f478a000640 1 --2- 172.21.15.83:0/3732504463 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47600678c0 0x7f4760069d80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4774007fd0 tx=0x7f4774009040 comp rx=0 tx=0).stop 2024-04-25T00:13:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.328+0000 7f478a000640 1 -- 172.21.15.83:0/3732504463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4784164cb0 msgr2=0x7f4784154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.328+0000 7f478a000640 1 --2- 172.21.15.83:0/3732504463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4784164cb0 0x7f4784154510 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f477c009820 tx=0x7f477c0191d0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.328+0000 7f478a000640 1 -- 172.21.15.83:0/3732504463 shutdown_connections 2024-04-25T00:13:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.328+0000 7f478a000640 1 --2- 172.21.15.83:0/3732504463 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47600678c0 0x7f4760069d80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.328+0000 7f478a000640 1 --2- 172.21.15.83:0/3732504463 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47841599f0 0x7f4784150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.328+0000 7f478a000640 1 --2- 172.21.15.83:0/3732504463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4784164cb0 0x7f4784154510 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.328+0000 7f478a000640 1 --2- 172.21.15.83:0/3732504463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47840cc510 0x7f4784165220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.328+0000 7f478a000640 1 -- 172.21.15.83:0/3732504463 >> 172.21.15.83:0/3732504463 conn(0x7f47840b1830 msgr2=0x7f478415c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.328+0000 7f478a000640 1 -- 172.21.15.83:0/3732504463 shutdown_connections 2024-04-25T00:13:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.328+0000 7f478a000640 1 -- 172.21.15.83:0/3732504463 wait complete. 2024-04-25T00:13:57.350 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-04-25T00:13:56.476921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236592,"kb_used_data":7936,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754738128,"statfs":{"total":773094113280,"available":772851843072,"internally_reserved":0,"allocated":8126464,"data_stored":4285916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.598678"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458342+0000","last_change":"2024-04-25T00:13:52.906825+0000","last_active":"2024-04-25T00:13:54.458342+0000","last_peered":"2024-04-25T00:13:54.458342+0000","last_clean":"2024-04-25T00:13:54.458342+0000","last_became_active":"2024-04-25T00:13:52.906492+0000","last_became_peered":"2024-04-25T00:13:52.906492+0000","last_unstale":"2024-04-25T00:13:54.458342+0000","last_undegraded":"2024-04-25T00:13:54.458342+0000","last_fullsized":"2024-04-25T00:13:54.458342+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:38.492799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.912107+0000","last_change":"2024-04-25T00:13:52.919890+0000","last_active":"2024-04-25T00:13:53.912107+0000","last_peered":"2024-04-25T00:13:53.912107+0000","last_clean":"2024-04-25T00:13:53.912107+0000","last_became_active":"2024-04-25T00:13:52.919094+0000","last_became_peered":"2024-04-25T00:13:52.919094+0000","last_unstale":"2024-04-25T00:13:53.912107+0000","last_undegraded":"2024-04-25T00:13:53.912107+0000","last_fullsized":"2024-04-25T00:13:53.912107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:07.478047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.909170+0000","last_change":"2024-04-25T00:13:52.904105+0000","last_active":"2024-04-25T00:13:53.909170+0000","last_peered":"2024-04-25T00:13:53.909170+0000","last_clean":"2024-04-25T00:13:53.909170+0000","last_became_active":"2024-04-25T00:13:52.903840+0000","last_became_peered":"2024-04-25T00:13:52.903840+0000","last_unstale":"2024-04-25T00:13:53.909170+0000","last_undegraded":"2024-04-25T00:13:53.909170+0000","last_fullsized":"2024-04-25T00:13:53.909170+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:20.863218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353424+0000","last_change":"2024-04-25T00:13:52.910517+0000","last_active":"2024-04-25T00:13:54.353424+0000","last_peered":"2024-04-25T00:13:54.353424+0000","last_clean":"2024-04-25T00:13:54.353424+0000","last_became_active":"2024-04-25T00:13:52.910158+0000","last_became_peered":"2024-04-25T00:13:52.910158+0000","last_unstale":"2024-04-25T00:13:54.353424+0000","last_undegraded":"2024-04-25T00:13:54.353424+0000","last_fullsized":"2024-04-25T00:13:54.353424+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:15.092144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907283+0000","last_change":"2024-04-25T00:13:52.916818+0000","last_active":"2024-04-25T00:13:53.907283+0000","last_peered":"2024-04-25T00:13:53.907283+0000","last_clean":"2024-04-25T00:13:53.907283+0000","last_became_active":"2024-04-25T00:13:52.916362+0000","last_became_peered":"2024-04-25T00:13:52.916362+0000","last_unstale":"2024-04-25T00:13:53.907283+0000","last_undegraded":"2024-04-25T00:13:53.907283+0000","last_fullsized":"2024-04-25T00:13:53.907283+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:23.129717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457379+0000","last_change":"2024-04-25T00:13:52.928102+0000","last_active":"2024-04-25T00:13:54.457379+0000","last_peered":"2024-04-25T00:13:54.457379+0000","last_clean":"2024-04-25T00:13:54.457379+0000","last_became_active":"2024-04-25T00:13:52.927402+0000","last_became_peered":"2024-04-25T00:13:52.927402+0000","last_unstale":"2024-04-25T00:13:54.457379+0000","last_undegraded":"2024-04-25T00:13:54.457379+0000","last_fullsized":"2024-04-25T00:13:54.457379+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:07.478047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906167+0000","last_change":"2024-04-25T00:13:52.915025+0000","last_active":"2024-04-25T00:13:53.906167+0000","last_peered":"2024-04-25T00:13:53.906167+0000","last_clean":"2024-04-25T00:13:53.906167+0000","last_became_active":"2024-04-25T00:13:52.914745+0000","last_became_peered":"2024-04-25T00:13:52.914745+0000","last_unstale":"2024-04-25T00:13:53.906167+0000","last_undegraded":"2024-04-25T00:13:53.906167+0000","last_fullsized":"2024-04-25T00:13:53.906167+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:08.688249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907175+0000","last_change":"2024-04-25T00:13:52.910058+0000","last_active":"2024-04-25T00:13:53.907175+0000","last_peered":"2024-04-25T00:13:53.907175+0000","last_clean":"2024-04-25T00:13:53.907175+0000","last_became_active":"2024-04-25T00:13:52.909787+0000","last_became_peered":"2024-04-25T00:13:52.909787+0000","last_unstale":"2024-04-25T00:13:53.907175+0000","last_undegraded":"2024-04-25T00:13:53.907175+0000","last_fullsized":"2024-04-25T00:13:53.907175+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:51.018895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457631+0000","last_change":"2024-04-25T00:13:52.927269+0000","last_active":"2024-04-25T00:13:54.457631+0000","last_peered":"2024-04-25T00:13:54.457631+0000","last_clean":"2024-04-25T00:13:54.457631+0000","last_became_active":"2024-04-25T00:13:52.926935+0000","last_became_peered":"2024-04-25T00:13:52.926935+0000","last_unstale":"2024-04-25T00:13:54.457631+0000","last_undegraded":"2024-04-25T00:13:54.457631+0000","last_fullsized":"2024-04-25T00:13:54.457631+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:21.425380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353305+0000","last_change":"2024-04-25T00:13:52.912802+0000","last_active":"2024-04-25T00:13:54.353305+0000","last_peered":"2024-04-25T00:13:54.353305+0000","last_clean":"2024-04-25T00:13:54.353305+0000","last_became_active":"2024-04-25T00:13:52.912228+0000","last_became_peered":"2024-04-25T00:13:52.912228+0000","last_unstale":"2024-04-25T00:13:54.353305+0000","last_undegraded":"2024-04-25T00:13:54.353305+0000","last_fullsized":"2024-04-25T00:13:54.353305+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:34.148169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911934+0000","last_change":"2024-04-25T00:13:52.922317+0000","last_active":"2024-04-25T00:13:53.911934+0000","last_peered":"2024-04-25T00:13:53.911934+0000","last_clean":"2024-04-25T00:13:53.911934+0000","last_became_active":"2024-04-25T00:13:52.921948+0000","last_became_peered":"2024-04-25T00:13:52.921948+0000","last_unstale":"2024-04-25T00:13:53.911934+0000","last_undegraded":"2024-04-25T00:13:53.911934+0000","last_fullsized":"2024-04-25T00:13:53.911934+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:36.719641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353483+0000","last_change":"2024-04-25T00:13:52.908021+0000","last_active":"2024-04-25T00:13:54.353483+0000","last_peered":"2024-04-25T00:13:54.353483+0000","last_clean":"2024-04-25T00:13:54.353483+0000","last_became_active":"2024-04-25T00:13:52.907675+0000","last_became_peered":"2024-04-25T00:13:52.907675+0000","last_unstale":"2024-04-25T00:13:54.353483+0000","last_undegraded":"2024-04-25T00:13:54.353483+0000","last_fullsized":"2024-04-25T00:13:54.353483+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:48.303380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907248+0000","last_change":"2024-04-25T00:13:52.932246+0000","last_active":"2024-04-25T00:13:53.907248+0000","last_peered":"2024-04-25T00:13:53.907248+0000","last_clean":"2024-04-25T00:13:53.907248+0000","last_became_active":"2024-04-25T00:13:52.932013+0000","last_became_peered":"2024-04-25T00:13:52.932013+0000","last_unstale":"2024-04-25T00:13:53.907248+0000","last_undegraded":"2024-04-25T00:13:53.907248+0000","last_fullsized":"2024-04-25T00:13:53.907248+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:03.454900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457501+0000","last_change":"2024-04-25T00:13:52.927403+0000","last_active":"2024-04-25T00:13:54.457501+0000","last_peered":"2024-04-25T00:13:54.457501+0000","last_clean":"2024-04-25T00:13:54.457501+0000","last_became_active":"2024-04-25T00:13:52.927154+0000","last_became_peered":"2024-04-25T00:13:52.927154+0000","last_unstale":"2024-04-25T00:13:54.457501+0000","last_undegraded":"2024-04-25T00:13:54.457501+0000","last_fullsized":"2024-04-25T00:13:54.457501+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:16:19.823094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911584+0000","last_change":"2024-04-25T00:13:52.902243+0000","last_active":"2024-04-25T00:13:53.911584+0000","last_peered":"2024-04-25T00:13:53.911584+0000","last_clean":"2024-04-25T00:13:53.911584+0000","last_became_active":"2024-04-25T00:13:52.901954+0000","last_became_peered":"2024-04-25T00:13:52.901954+0000","last_unstale":"2024-04-25T00:13:53.911584+0000","last_undegraded":"2024-04-25T00:13:53.911584+0000","last_fullsized":"2024-04-25T00:13:53.911584+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:14.782886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457576+0000","last_change":"2024-04-25T00:13:52.913541+0000","last_active":"2024-04-25T00:13:54.457576+0000","last_peered":"2024-04-25T00:13:54.457576+0000","last_clean":"2024-04-25T00:13:54.457576+0000","last_became_active":"2024-04-25T00:13:52.913289+0000","last_became_peered":"2024-04-25T00:13:52.913289+0000","last_unstale":"2024-04-25T00:13:54.457576+0000","last_undegraded":"2024-04-25T00:13:54.457576+0000","last_fullsized":"2024-04-25T00:13:54.457576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:17.850811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907078+0000","last_change":"2024-04-25T00:13:52.918163+0000","last_active":"2024-04-25T00:13:53.907078+0000","last_peered":"2024-04-25T00:13:53.907078+0000","last_clean":"2024-04-25T00:13:53.907078+0000","last_became_active":"2024-04-25T00:13:52.917900+0000","last_became_peered":"2024-04-25T00:13:52.917900+0000","last_unstale":"2024-04-25T00:13:53.907078+0000","last_undegraded":"2024-04-25T00:13:53.907078+0000","last_fullsized":"2024-04-25T00:13:53.907078+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:53.489936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.909147+0000","last_change":"2024-04-25T00:13:52.901921+0000","last_active":"2024-04-25T00:13:53.909147+0000","last_peered":"2024-04-25T00:13:53.909147+0000","last_clean":"2024-04-25T00:13:53.909147+0000","last_became_active":"2024-04-25T00:13:52.901665+0000","last_became_peered":"2024-04-25T00:13:52.901665+0000","last_unstale":"2024-04-25T00:13:53.909147+0000","last_undegraded":"2024-04-25T00:13:53.909147+0000","last_fullsized":"2024-04-25T00:13:53.909147+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:16.460099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457458+0000","last_change":"2024-04-25T00:13:52.934265+0000","last_active":"2024-04-25T00:13:54.457458+0000","last_peered":"2024-04-25T00:13:54.457458+0000","last_clean":"2024-04-25T00:13:54.457458+0000","last_became_active":"2024-04-25T00:13:52.933898+0000","last_became_peered":"2024-04-25T00:13:52.933898+0000","last_unstale":"2024-04-25T00:13:54.457458+0000","last_undegraded":"2024-04-25T00:13:54.457458+0000","last_fullsized":"2024-04-25T00:13:54.457458+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:40.637159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907107+0000","last_change":"2024-04-25T00:13:52.914462+0000","last_active":"2024-04-25T00:13:53.907107+0000","last_peered":"2024-04-25T00:13:53.907107+0000","last_clean":"2024-04-25T00:13:53.907107+0000","last_became_active":"2024-04-25T00:13:52.914256+0000","last_became_peered":"2024-04-25T00:13:52.914256+0000","last_unstale":"2024-04-25T00:13:53.907107+0000","last_undegraded":"2024-04-25T00:13:53.907107+0000","last_fullsized":"2024-04-25T00:13:53.907107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:45.817144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457251+0000","last_change":"2024-04-25T00:13:52.905025+0000","last_active":"2024-04-25T00:13:54.457251+0000","last_peered":"2024-04-25T00:13:54.457251+0000","last_clean":"2024-04-25T00:13:54.457251+0000","last_became_active":"2024-04-25T00:13:52.904769+0000","last_became_peered":"2024-04-25T00:13:52.904769+0000","last_unstale":"2024-04-25T00:13:54.457251+0000","last_undegraded":"2024-04-25T00:13:54.457251+0000","last_fullsized":"2024-04-25T00:13:54.457251+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:49.936682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907014+0000","last_change":"2024-04-25T00:13:52.931792+0000","last_active":"2024-04-25T00:13:53.907014+0000","last_peered":"2024-04-25T00:13:53.907014+0000","last_clean":"2024-04-25T00:13:53.907014+0000","last_became_active":"2024-04-25T00:13:52.931565+0000","last_became_peered":"2024-04-25T00:13:52.931565+0000","last_unstale":"2024-04-25T00:13:53.907014+0000","last_undegraded":"2024-04-25T00:13:53.907014+0000","last_fullsized":"2024-04-25T00:13:53.907014+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:46.506797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457174+0000","last_change":"2024-04-25T00:13:52.906720+0000","last_active":"2024-04-25T00:13:54.457174+0000","last_peered":"2024-04-25T00:13:54.457174+0000","last_clean":"2024-04-25T00:13:54.457174+0000","last_became_active":"2024-04-25T00:13:52.906425+0000","last_became_peered":"2024-04-25T00:13:52.906425+0000","last_unstale":"2024-04-25T00:13:54.457174+0000","last_undegraded":"2024-04-25T00:13:54.457174+0000","last_fullsized":"2024-04-25T00:13:54.457174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:01.855627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911539+0000","last_change":"2024-04-25T00:13:52.929015+0000","last_active":"2024-04-25T00:13:53.911539+0000","last_peered":"2024-04-25T00:13:53.911539+0000","last_clean":"2024-04-25T00:13:53.911539+0000","last_became_active":"2024-04-25T00:13:52.927512+0000","last_became_peered":"2024-04-25T00:13:52.927512+0000","last_unstale":"2024-04-25T00:13:53.911539+0000","last_undegraded":"2024-04-25T00:13:53.911539+0000","last_fullsized":"2024-04-25T00:13:53.911539+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:48.303380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911925+0000","last_change":"2024-04-25T00:13:52.922121+0000","last_active":"2024-04-25T00:13:53.911925+0000","last_peered":"2024-04-25T00:13:53.911925+0000","last_clean":"2024-04-25T00:13:53.911925+0000","last_became_active":"2024-04-25T00:13:52.921848+0000","last_became_peered":"2024-04-25T00:13:52.921848+0000","last_unstale":"2024-04-25T00:13:53.911925+0000","last_undegraded":"2024-04-25T00:13:53.911925+0000","last_fullsized":"2024-04-25T00:13:53.911925+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:08.628566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353477+0000","last_change":"2024-04-25T00:13:52.908067+0000","last_active":"2024-04-25T00:13:54.353477+0000","last_peered":"2024-04-25T00:13:54.353477+0000","last_clean":"2024-04-25T00:13:54.353477+0000","last_became_active":"2024-04-25T00:13:52.907787+0000","last_became_peered":"2024-04-25T00:13:52.907787+0000","last_unstale":"2024-04-25T00:13:54.353477+0000","last_undegraded":"2024-04-25T00:13:54.353477+0000","last_fullsized":"2024-04-25T00:13:54.353477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:14.879011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458148+0000","last_change":"2024-04-25T00:13:52.912562+0000","last_active":"2024-04-25T00:13:54.458148+0000","last_peered":"2024-04-25T00:13:54.458148+0000","last_clean":"2024-04-25T00:13:54.458148+0000","last_became_active":"2024-04-25T00:13:52.912227+0000","last_became_peered":"2024-04-25T00:13:52.912227+0000","last_unstale":"2024-04-25T00:13:54.458148+0000","last_undegraded":"2024-04-25T00:13:54.458148+0000","last_fullsized":"2024-04-25T00:13:54.458148+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:08.628566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906845+0000","last_change":"2024-04-25T00:13:52.914826+0000","last_active":"2024-04-25T00:13:53.906845+0000","last_peered":"2024-04-25T00:13:53.906845+0000","last_clean":"2024-04-25T00:13:53.906845+0000","last_became_active":"2024-04-25T00:13:52.912612+0000","last_became_peered":"2024-04-25T00:13:52.912612+0000","last_unstale":"2024-04-25T00:13:53.906845+0000","last_undegraded":"2024-04-25T00:13:53.906845+0000","last_fullsized":"2024-04-25T00:13:53.906845+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:20.741425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911706+0000","last_change":"2024-04-25T00:13:52.905235+0000","last_active":"2024-04-25T00:13:53.911706+0000","last_peered":"2024-04-25T00:13:53.911706+0000","last_clean":"2024-04-25T00:13:53.911706+0000","last_became_active":"2024-04-25T00:13:52.904986+0000","last_became_peered":"2024-04-25T00:13:52.904986+0000","last_unstale":"2024-04-25T00:13:53.911706+0000","last_undegraded":"2024-04-25T00:13:53.911706+0000","last_fullsized":"2024-04-25T00:13:53.911706+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:17.850811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457984+0000","last_change":"2024-04-25T00:13:52.908675+0000","last_active":"2024-04-25T00:13:54.457984+0000","last_peered":"2024-04-25T00:13:54.457984+0000","last_clean":"2024-04-25T00:13:54.457984+0000","last_became_active":"2024-04-25T00:13:52.907831+0000","last_became_peered":"2024-04-25T00:13:52.907831+0000","last_unstale":"2024-04-25T00:13:54.457984+0000","last_undegraded":"2024-04-25T00:13:54.457984+0000","last_fullsized":"2024-04-25T00:13:54.457984+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:04.136398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.908713+0000","last_change":"2024-04-25T00:13:52.910077+0000","last_active":"2024-04-25T00:13:53.908713+0000","last_peered":"2024-04-25T00:13:53.908713+0000","last_clean":"2024-04-25T00:13:53.908713+0000","last_became_active":"2024-04-25T00:13:52.909826+0000","last_became_peered":"2024-04-25T00:13:52.909826+0000","last_unstale":"2024-04-25T00:13:53.908713+0000","last_undegraded":"2024-04-25T00:13:53.908713+0000","last_fullsized":"2024-04-25T00:13:53.908713+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:47.596604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457670+0000","last_change":"2024-04-25T00:13:52.907120+0000","last_active":"2024-04-25T00:13:54.457670+0000","last_peered":"2024-04-25T00:13:54.457670+0000","last_clean":"2024-04-25T00:13:54.457670+0000","last_became_active":"2024-04-25T00:13:52.906545+0000","last_became_peered":"2024-04-25T00:13:52.906545+0000","last_unstale":"2024-04-25T00:13:54.457670+0000","last_undegraded":"2024-04-25T00:13:54.457670+0000","last_fullsized":"2024-04-25T00:13:54.457670+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:41.000967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.908716+0000","last_change":"2024-04-25T00:13:52.900681+0000","last_active":"2024-04-25T00:13:53.908716+0000","last_peered":"2024-04-25T00:13:53.908716+0000","last_clean":"2024-04-25T00:13:53.908716+0000","last_became_active":"2024-04-25T00:13:52.900393+0000","last_became_peered":"2024-04-25T00:13:52.900393+0000","last_unstale":"2024-04-25T00:13:53.908716+0000","last_undegraded":"2024-04-25T00:13:53.908716+0000","last_fullsized":"2024-04-25T00:13:53.908716+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:26.234351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"70'4","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907169+0000","last_change":"2024-04-25T00:13:51.907393+0000","last_active":"2024-04-25T00:13:53.907169+0000","last_peered":"2024-04-25T00:13:53.907169+0000","last_clean":"2024-04-25T00:13:53.907169+0000","last_became_active":"2024-04-25T00:13:51.907136+0000","last_became_peered":"2024-04-25T00:13:51.907136+0000","last_unstale":"2024-04-25T00:13:53.907169+0000","last_undegraded":"2024-04-25T00:13:53.907169+0000","last_fullsized":"2024-04-25T00:13:53.907169+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:38.787274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911834+0000","last_change":"2024-04-25T00:13:52.902781+0000","last_active":"2024-04-25T00:13:53.911834+0000","last_peered":"2024-04-25T00:13:53.911834+0000","last_clean":"2024-04-25T00:13:53.911834+0000","last_became_active":"2024-04-25T00:13:52.902530+0000","last_became_peered":"2024-04-25T00:13:52.902530+0000","last_unstale":"2024-04-25T00:13:53.911834+0000","last_undegraded":"2024-04-25T00:13:53.911834+0000","last_fullsized":"2024-04-25T00:13:53.911834+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:04.136398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458290+0000","last_change":"2024-04-25T00:13:51.914058+0000","last_active":"2024-04-25T00:13:54.458290+0000","last_peered":"2024-04-25T00:13:54.458290+0000","last_clean":"2024-04-25T00:13:54.458290+0000","last_became_active":"2024-04-25T00:13:51.913776+0000","last_became_peered":"2024-04-25T00:13:51.913776+0000","last_unstale":"2024-04-25T00:13:54.458290+0000","last_undegraded":"2024-04-25T00:13:54.458290+0000","last_fullsized":"2024-04-25T00:13:54.458290+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:05.397845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457433+0000","last_change":"2024-04-25T00:13:52.931111+0000","last_active":"2024-04-25T00:13:54.457433+0000","last_peered":"2024-04-25T00:13:54.457433+0000","last_clean":"2024-04-25T00:13:54.457433+0000","last_became_active":"2024-04-25T00:13:52.930874+0000","last_became_peered":"2024-04-25T00:13:52.930874+0000","last_unstale":"2024-04-25T00:13:54.457433+0000","last_undegraded":"2024-04-25T00:13:54.457433+0000","last_fullsized":"2024-04-25T00:13:54.457433+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:06.033141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458189+0000","last_change":"2024-04-25T00:13:51.914765+0000","last_active":"2024-04-25T00:13:54.458189+0000","last_peered":"2024-04-25T00:13:54.458189+0000","last_clean":"2024-04-25T00:13:54.458189+0000","last_became_active":"2024-04-25T00:13:51.914501+0000","last_became_peered":"2024-04-25T00:13:51.914501+0000","last_unstale":"2024-04-25T00:13:54.458189+0000","last_undegraded":"2024-04-25T00:13:54.458189+0000","last_fullsized":"2024-04-25T00:13:54.458189+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:35:28.273299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457432+0000","last_change":"2024-04-25T00:13:52.906553+0000","last_active":"2024-04-25T00:13:54.457432+0000","last_peered":"2024-04-25T00:13:54.457432+0000","last_clean":"2024-04-25T00:13:54.457432+0000","last_became_active":"2024-04-25T00:13:52.905948+0000","last_became_peered":"2024-04-25T00:13:52.905948+0000","last_unstale":"2024-04-25T00:13:54.457432+0000","last_undegraded":"2024-04-25T00:13:54.457432+0000","last_fullsized":"2024-04-25T00:13:54.457432+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:25.244779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906258+0000","last_change":"2024-04-25T00:13:51.906926+0000","last_active":"2024-04-25T00:13:53.906258+0000","last_peered":"2024-04-25T00:13:53.906258+0000","last_clean":"2024-04-25T00:13:53.906258+0000","last_became_active":"2024-04-25T00:13:51.906684+0000","last_became_peered":"2024-04-25T00:13:51.906684+0000","last_unstale":"2024-04-25T00:13:53.906258+0000","last_undegraded":"2024-04-25T00:13:53.906258+0000","last_fullsized":"2024-04-25T00:13:53.906258+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:36.685151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.912163+0000","last_change":"2024-04-25T00:13:52.922296+0000","last_active":"2024-04-25T00:13:53.912163+0000","last_peered":"2024-04-25T00:13:53.912163+0000","last_clean":"2024-04-25T00:13:53.912163+0000","last_became_active":"2024-04-25T00:13:52.921315+0000","last_became_peered":"2024-04-25T00:13:52.921315+0000","last_unstale":"2024-04-25T00:13:53.912163+0000","last_undegraded":"2024-04-25T00:13:53.912163+0000","last_fullsized":"2024-04-25T00:13:53.912163+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:40.637159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457510+0000","last_change":"2024-04-25T00:13:51.914571+0000","last_active":"2024-04-25T00:13:54.457510+0000","last_peered":"2024-04-25T00:13:54.457510+0000","last_clean":"2024-04-25T00:13:54.457510+0000","last_became_active":"2024-04-25T00:13:51.914275+0000","last_became_peered":"2024-04-25T00:13:51.914275+0000","last_unstale":"2024-04-25T00:13:54.457510+0000","last_undegraded":"2024-04-25T00:13:54.457510+0000","last_fullsized":"2024-04-25T00:13:54.457510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:35.572339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911994+0000","last_change":"2024-04-25T00:13:52.922215+0000","last_active":"2024-04-25T00:13:53.911994+0000","last_peered":"2024-04-25T00:13:53.911994+0000","last_clean":"2024-04-25T00:13:53.911994+0000","last_became_active":"2024-04-25T00:13:52.921444+0000","last_became_peered":"2024-04-25T00:13:52.921444+0000","last_unstale":"2024-04-25T00:13:53.911994+0000","last_undegraded":"2024-04-25T00:13:53.911994+0000","last_fullsized":"2024-04-25T00:13:53.911994+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:23.826879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353167+0000","last_change":"2024-04-25T00:13:51.907948+0000","last_active":"2024-04-25T00:13:54.353167+0000","last_peered":"2024-04-25T00:13:54.353167+0000","last_clean":"2024-04-25T00:13:54.353167+0000","last_became_active":"2024-04-25T00:13:51.906221+0000","last_became_peered":"2024-04-25T00:13:51.906221+0000","last_unstale":"2024-04-25T00:13:54.353167+0000","last_undegraded":"2024-04-25T00:13:54.353167+0000","last_fullsized":"2024-04-25T00:13:54.353167+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:40.645229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457761+0000","last_change":"2024-04-25T00:13:52.923747+0000","last_active":"2024-04-25T00:13:54.457761+0000","last_peered":"2024-04-25T00:13:54.457761+0000","last_clean":"2024-04-25T00:13:54.457761+0000","last_became_active":"2024-04-25T00:13:52.923460+0000","last_became_peered":"2024-04-25T00:13:52.923460+0000","last_unstale":"2024-04-25T00:13:54.457761+0000","last_undegraded":"2024-04-25T00:13:54.457761+0000","last_fullsized":"2024-04-25T00:13:54.457761+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:44.419964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911905+0000","last_change":"2024-04-25T00:13:51.902322+0000","last_active":"2024-04-25T00:13:53.911905+0000","last_peered":"2024-04-25T00:13:53.911905+0000","last_clean":"2024-04-25T00:13:53.911905+0000","last_became_active":"2024-04-25T00:13:51.902053+0000","last_became_peered":"2024-04-25T00:13:51.902053+0000","last_unstale":"2024-04-25T00:13:53.911905+0000","last_undegraded":"2024-04-25T00:13:53.911905+0000","last_fullsized":"2024-04-25T00:13:53.911905+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:19.259348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906613+0000","last_change":"2024-04-25T00:13:52.911436+0000","last_active":"2024-04-25T00:13:53.906613+0000","last_peered":"2024-04-25T00:13:53.906613+0000","last_clean":"2024-04-25T00:13:53.906613+0000","last_became_active":"2024-04-25T00:13:52.911166+0000","last_became_peered":"2024-04-25T00:13:52.911166+0000","last_unstale":"2024-04-25T00:13:53.906613+0000","last_undegraded":"2024-04-25T00:13:53.906613+0000","last_fullsized":"2024-04-25T00:13:53.906613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:04.818049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906629+0000","last_change":"2024-04-25T00:13:51.905908+0000","last_active":"2024-04-25T00:13:53.906629+0000","last_peered":"2024-04-25T00:13:53.906629+0000","last_clean":"2024-04-25T00:13:53.906629+0000","last_became_active":"2024-04-25T00:13:51.905597+0000","last_became_peered":"2024-04-25T00:13:51.905597+0000","last_unstale":"2024-04-25T00:13:53.906629+0000","last_undegraded":"2024-04-25T00:13:53.906629+0000","last_fullsized":"2024-04-25T00:13:53.906629+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:49.685037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911539+0000","last_change":"2024-04-25T00:13:52.906493+0000","last_active":"2024-04-25T00:13:53.911539+0000","last_peered":"2024-04-25T00:13:53.911539+0000","last_clean":"2024-04-25T00:13:53.911539+0000","last_became_active":"2024-04-25T00:13:52.906238+0000","last_became_peered":"2024-04-25T00:13:52.906238+0000","last_unstale":"2024-04-25T00:13:53.911539+0000","last_undegraded":"2024-04-25T00:13:53.911539+0000","last_fullsized":"2024-04-25T00:13:53.911539+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:48.458973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458387+0000","last_change":"2024-04-25T00:13:51.914190+0000","last_active":"2024-04-25T00:13:54.458387+0000","last_peered":"2024-04-25T00:13:54.458387+0000","last_clean":"2024-04-25T00:13:54.458387+0000","last_became_active":"2024-04-25T00:13:51.913929+0000","last_became_peered":"2024-04-25T00:13:51.913929+0000","last_unstale":"2024-04-25T00:13:54.458387+0000","last_undegraded":"2024-04-25T00:13:54.458387+0000","last_fullsized":"2024-04-25T00:13:54.458387+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:10.915459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353276+0000","last_change":"2024-04-25T00:13:52.910690+0000","last_active":"2024-04-25T00:13:54.353276+0000","last_peered":"2024-04-25T00:13:54.353276+0000","last_clean":"2024-04-25T00:13:54.353276+0000","last_became_active":"2024-04-25T00:13:52.910122+0000","last_became_peered":"2024-04-25T00:13:52.910122+0000","last_unstale":"2024-04-25T00:13:54.353276+0000","last_undegraded":"2024-04-25T00:13:54.353276+0000","last_fullsized":"2024-04-25T00:13:54.353276+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:16.820300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353474+0000","last_change":"2024-04-25T00:13:51.909299+0000","last_active":"2024-04-25T00:13:54.353474+0000","last_peered":"2024-04-25T00:13:54.353474+0000","last_clean":"2024-04-25T00:13:54.353474+0000","last_became_active":"2024-04-25T00:13:51.908909+0000","last_became_peered":"2024-04-25T00:13:51.908909+0000","last_unstale":"2024-04-25T00:13:54.353474+0000","last_undegraded":"2024-04-25T00:13:54.353474+0000","last_fullsized":"2024-04-25T00:13:54.353474+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:25.345470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457623+0000","last_change":"2024-04-25T00:13:52.932015+0000","last_active":"2024-04-25T00:13:54.457623+0000","last_peered":"2024-04-25T00:13:54.457623+0000","last_clean":"2024-04-25T00:13:54.457623+0000","last_became_active":"2024-04-25T00:13:52.931712+0000","last_became_peered":"2024-04-25T00:13:52.931712+0000","last_unstale":"2024-04-25T00:13:54.457623+0000","last_undegraded":"2024-04-25T00:13:54.457623+0000","last_fullsized":"2024-04-25T00:13:54.457623+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:35.604240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"70'3","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907076+0000","last_change":"2024-04-25T00:13:51.910876+0000","last_active":"2024-04-25T00:13:53.907076+0000","last_peered":"2024-04-25T00:13:53.907076+0000","last_clean":"2024-04-25T00:13:53.907076+0000","last_became_active":"2024-04-25T00:13:51.910611+0000","last_became_peered":"2024-04-25T00:13:51.910611+0000","last_unstale":"2024-04-25T00:13:53.907076+0000","last_undegraded":"2024-04-25T00:13:53.907076+0000","last_fullsized":"2024-04-25T00:13:53.907076+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:36.385074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.908899+0000","last_change":"2024-04-25T00:13:52.903692+0000","last_active":"2024-04-25T00:13:53.908899+0000","last_peered":"2024-04-25T00:13:53.908899+0000","last_clean":"2024-04-25T00:13:53.908899+0000","last_became_active":"2024-04-25T00:13:52.903405+0000","last_became_peered":"2024-04-25T00:13:52.903405+0000","last_unstale":"2024-04-25T00:13:53.908899+0000","last_undegraded":"2024-04-25T00:13:53.908899+0000","last_fullsized":"2024-04-25T00:13:53.908899+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:07.076263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"70'6","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906385+0000","last_change":"2024-04-25T00:13:51.905851+0000","last_active":"2024-04-25T00:13:53.906385+0000","last_peered":"2024-04-25T00:13:53.906385+0000","last_clean":"2024-04-25T00:13:53.906385+0000","last_became_active":"2024-04-25T00:13:51.904311+0000","last_became_peered":"2024-04-25T00:13:51.904311+0000","last_unstale":"2024-04-25T00:13:53.906385+0000","last_undegraded":"2024-04-25T00:13:53.906385+0000","last_fullsized":"2024-04-25T00:13:53.906385+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:05.027309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353553+0000","last_change":"2024-04-25T00:13:52.926495+0000","last_active":"2024-04-25T00:13:54.353553+0000","last_peered":"2024-04-25T00:13:54.353553+0000","last_clean":"2024-04-25T00:13:54.353553+0000","last_became_active":"2024-04-25T00:13:52.926240+0000","last_became_peered":"2024-04-25T00:13:52.926240+0000","last_unstale":"2024-04-25T00:13:54.353553+0000","last_undegraded":"2024-04-25T00:13:54.353553+0000","last_fullsized":"2024-04-25T00:13:54.353553+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:48.458973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353147+0000","last_change":"2024-04-25T00:13:51.904959+0000","last_active":"2024-04-25T00:13:54.353147+0000","last_peered":"2024-04-25T00:13:54.353147+0000","last_clean":"2024-04-25T00:13:54.353147+0000","last_became_active":"2024-04-25T00:13:51.904681+0000","last_became_peered":"2024-04-25T00:13:51.904681+0000","last_unstale":"2024-04-25T00:13:54.353147+0000","last_undegraded":"2024-04-25T00:13:54.353147+0000","last_fullsized":"2024-04-25T00:13:54.353147+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:03.937735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906104+0000","last_change":"2024-04-25T00:13:52.909363+0000","last_active":"2024-04-25T00:13:53.906104+0000","last_peered":"2024-04-25T00:13:53.906104+0000","last_clean":"2024-04-25T00:13:53.906104+0000","last_became_active":"2024-04-25T00:13:52.908357+0000","last_became_peered":"2024-04-25T00:13:52.908357+0000","last_unstale":"2024-04-25T00:13:53.906104+0000","last_undegraded":"2024-04-25T00:13:53.906104+0000","last_fullsized":"2024-04-25T00:13:53.906104+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:16.460099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457910+0000","last_change":"2024-04-25T00:13:51.913490+0000","last_active":"2024-04-25T00:13:54.457910+0000","last_peered":"2024-04-25T00:13:54.457910+0000","last_clean":"2024-04-25T00:13:54.457910+0000","last_became_active":"2024-04-25T00:13:51.913007+0000","last_became_peered":"2024-04-25T00:13:51.913007+0000","last_unstale":"2024-04-25T00:13:54.457910+0000","last_undegraded":"2024-04-25T00:13:54.457910+0000","last_fullsized":"2024-04-25T00:13:54.457910+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:23.164680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906553+0000","last_change":"2024-04-25T00:13:52.910658+0000","last_active":"2024-04-25T00:13:53.906553+0000","last_peered":"2024-04-25T00:13:53.906553+0000","last_clean":"2024-04-25T00:13:53.906553+0000","last_became_active":"2024-04-25T00:13:52.910334+0000","last_became_peered":"2024-04-25T00:13:52.910334+0000","last_unstale":"2024-04-25T00:13:53.906553+0000","last_undegraded":"2024-04-25T00:13:53.906553+0000","last_fullsized":"2024-04-25T00:13:53.906553+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:20.265179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906752+0000","last_change":"2024-04-25T00:13:51.908752+0000","last_active":"2024-04-25T00:13:53.906752+0000","last_peered":"2024-04-25T00:13:53.906752+0000","last_clean":"2024-04-25T00:13:53.906752+0000","last_became_active":"2024-04-25T00:13:51.906157+0000","last_became_peered":"2024-04-25T00:13:51.906157+0000","last_unstale":"2024-04-25T00:13:53.906752+0000","last_undegraded":"2024-04-25T00:13:53.906752+0000","last_fullsized":"2024-04-25T00:13:53.906752+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:12.153802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906585+0000","last_change":"2024-04-25T00:13:52.930550+0000","last_active":"2024-04-25T00:13:53.906585+0000","last_peered":"2024-04-25T00:13:53.906585+0000","last_clean":"2024-04-25T00:13:53.906585+0000","last_became_active":"2024-04-25T00:13:52.930277+0000","last_became_peered":"2024-04-25T00:13:52.930277+0000","last_unstale":"2024-04-25T00:13:53.906585+0000","last_undegraded":"2024-04-25T00:13:53.906585+0000","last_fullsized":"2024-04-25T00:13:53.906585+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:41.651061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"70'8","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911735+0000","last_change":"2024-04-25T00:13:51.910161+0000","last_active":"2024-04-25T00:13:53.911735+0000","last_peered":"2024-04-25T00:13:53.911735+0000","last_clean":"2024-04-25T00:13:53.911735+0000","last_became_active":"2024-04-25T00:13:51.909865+0000","last_became_peered":"2024-04-25T00:13:51.909865+0000","last_unstale":"2024-04-25T00:13:53.911735+0000","last_undegraded":"2024-04-25T00:13:53.911735+0000","last_fullsized":"2024-04-25T00:13:53.911735+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:14:18.998312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457185+0000","last_change":"2024-04-25T00:13:52.921126+0000","last_active":"2024-04-25T00:13:54.457185+0000","last_peered":"2024-04-25T00:13:54.457185+0000","last_clean":"2024-04-25T00:13:54.457185+0000","last_became_active":"2024-04-25T00:13:52.920870+0000","last_became_peered":"2024-04-25T00:13:52.920870+0000","last_unstale":"2024-04-25T00:13:54.457185+0000","last_undegraded":"2024-04-25T00:13:54.457185+0000","last_fullsized":"2024-04-25T00:13:54.457185+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:23.826879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458445+0000","last_change":"2024-04-25T00:13:51.916352+0000","last_active":"2024-04-25T00:13:54.458445+0000","last_peered":"2024-04-25T00:13:54.458445+0000","last_clean":"2024-04-25T00:13:54.458445+0000","last_became_active":"2024-04-25T00:13:51.915993+0000","last_became_peered":"2024-04-25T00:13:51.915993+0000","last_unstale":"2024-04-25T00:13:54.458445+0000","last_undegraded":"2024-04-25T00:13:54.458445+0000","last_fullsized":"2024-04-25T00:13:54.458445+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:05.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906685+0000","last_change":"2024-04-25T00:13:52.911234+0000","last_active":"2024-04-25T00:13:53.906685+0000","last_peered":"2024-04-25T00:13:53.906685+0000","last_clean":"2024-04-25T00:13:53.906685+0000","last_became_active":"2024-04-25T00:13:52.910959+0000","last_became_peered":"2024-04-25T00:13:52.910959+0000","last_unstale":"2024-04-25T00:13:53.906685+0000","last_undegraded":"2024-04-25T00:13:53.906685+0000","last_fullsized":"2024-04-25T00:13:53.906685+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:36.880859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457738+0000","last_change":"2024-04-25T00:13:51.909081+0000","last_active":"2024-04-25T00:13:54.457738+0000","last_peered":"2024-04-25T00:13:54.457738+0000","last_clean":"2024-04-25T00:13:54.457738+0000","last_became_active":"2024-04-25T00:13:51.908817+0000","last_became_peered":"2024-04-25T00:13:51.908817+0000","last_unstale":"2024-04-25T00:13:54.457738+0000","last_undegraded":"2024-04-25T00:13:54.457738+0000","last_fullsized":"2024-04-25T00:13:54.457738+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:35.550486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906729+0000","last_change":"2024-04-25T00:13:52.915676+0000","last_active":"2024-04-25T00:13:53.906729+0000","last_peered":"2024-04-25T00:13:53.906729+0000","last_clean":"2024-04-25T00:13:53.906729+0000","last_became_active":"2024-04-25T00:13:52.913972+0000","last_became_peered":"2024-04-25T00:13:52.913972+0000","last_unstale":"2024-04-25T00:13:53.906729+0000","last_undegraded":"2024-04-25T00:13:53.906729+0000","last_fullsized":"2024-04-25T00:13:53.906729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:16.298509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907119+0000","last_change":"2024-04-25T00:13:51.908325+0000","last_active":"2024-04-25T00:13:53.907119+0000","last_peered":"2024-04-25T00:13:53.907119+0000","last_clean":"2024-04-25T00:13:53.907119+0000","last_became_active":"2024-04-25T00:13:51.907027+0000","last_became_peered":"2024-04-25T00:13:51.907027+0000","last_unstale":"2024-04-25T00:13:53.907119+0000","last_undegraded":"2024-04-25T00:13:53.907119+0000","last_fullsized":"2024-04-25T00:13:53.907119+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:21.299318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457367+0000","last_change":"2024-04-25T00:13:52.923300+0000","last_active":"2024-04-25T00:13:54.457367+0000","last_peered":"2024-04-25T00:13:54.457367+0000","last_clean":"2024-04-25T00:13:54.457367+0000","last_became_active":"2024-04-25T00:13:52.922706+0000","last_became_peered":"2024-04-25T00:13:52.922706+0000","last_unstale":"2024-04-25T00:13:54.457367+0000","last_undegraded":"2024-04-25T00:13:54.457367+0000","last_fullsized":"2024-04-25T00:13:54.457367+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:14:01.301770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.908887+0000","last_change":"2024-04-25T00:13:51.906615+0000","last_active":"2024-04-25T00:13:53.908887+0000","last_peered":"2024-04-25T00:13:53.908887+0000","last_clean":"2024-04-25T00:13:53.908887+0000","last_became_active":"2024-04-25T00:13:51.906366+0000","last_became_peered":"2024-04-25T00:13:51.906366+0000","last_unstale":"2024-04-25T00:13:53.908887+0000","last_undegraded":"2024-04-25T00:13:53.908887+0000","last_fullsized":"2024-04-25T00:13:53.908887+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:48.930850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911954+0000","last_change":"2024-04-25T00:13:52.902737+0000","last_active":"2024-04-25T00:13:53.911954+0000","last_peered":"2024-04-25T00:13:53.911954+0000","last_clean":"2024-04-25T00:13:53.911954+0000","last_became_active":"2024-04-25T00:13:52.902435+0000","last_became_peered":"2024-04-25T00:13:52.902435+0000","last_unstale":"2024-04-25T00:13:53.911954+0000","last_undegraded":"2024-04-25T00:13:53.911954+0000","last_fullsized":"2024-04-25T00:13:53.911954+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:38.492799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457380+0000","last_change":"2024-04-25T00:13:51.898425+0000","last_active":"2024-04-25T00:13:54.457380+0000","last_peered":"2024-04-25T00:13:54.457380+0000","last_clean":"2024-04-25T00:13:54.457380+0000","last_became_active":"2024-04-25T00:13:51.898156+0000","last_became_peered":"2024-04-25T00:13:51.898156+0000","last_unstale":"2024-04-25T00:13:54.457380+0000","last_undegraded":"2024-04-25T00:13:54.457380+0000","last_fullsized":"2024-04-25T00:13:54.457380+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:13.777054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.912080+0000","last_change":"2024-04-25T00:13:52.897712+0000","last_active":"2024-04-25T00:13:53.912080+0000","last_peered":"2024-04-25T00:13:53.912080+0000","last_clean":"2024-04-25T00:13:53.912080+0000","last_became_active":"2024-04-25T00:13:52.897448+0000","last_became_peered":"2024-04-25T00:13:52.897448+0000","last_unstale":"2024-04-25T00:13:53.912080+0000","last_undegraded":"2024-04-25T00:13:53.912080+0000","last_fullsized":"2024-04-25T00:13:53.912080+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:14:01.301770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457216+0000","last_change":"2024-04-25T00:13:51.904095+0000","last_active":"2024-04-25T00:13:54.457216+0000","last_peered":"2024-04-25T00:13:54.457216+0000","last_clean":"2024-04-25T00:13:54.457216+0000","last_became_active":"2024-04-25T00:13:51.902301+0000","last_became_peered":"2024-04-25T00:13:51.902301+0000","last_unstale":"2024-04-25T00:13:54.457216+0000","last_undegraded":"2024-04-25T00:13:54.457216+0000","last_fullsized":"2024-04-25T00:13:54.457216+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:47.297549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906750+0000","last_change":"2024-04-25T00:13:52.910412+0000","last_active":"2024-04-25T00:13:53.906750+0000","last_peered":"2024-04-25T00:13:53.906750+0000","last_clean":"2024-04-25T00:13:53.906750+0000","last_became_active":"2024-04-25T00:13:52.910103+0000","last_became_peered":"2024-04-25T00:13:52.910103+0000","last_unstale":"2024-04-25T00:13:53.906750+0000","last_undegraded":"2024-04-25T00:13:53.906750+0000","last_fullsized":"2024-04-25T00:13:53.906750+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:04.943567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457158+0000","last_change":"2024-04-25T00:13:51.901915+0000","last_active":"2024-04-25T00:13:54.457158+0000","last_peered":"2024-04-25T00:13:54.457158+0000","last_clean":"2024-04-25T00:13:54.457158+0000","last_became_active":"2024-04-25T00:13:51.901676+0000","last_became_peered":"2024-04-25T00:13:51.901676+0000","last_unstale":"2024-04-25T00:13:54.457158+0000","last_undegraded":"2024-04-25T00:13:54.457158+0000","last_fullsized":"2024-04-25T00:13:54.457158+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:13.873180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457190+0000","last_change":"2024-04-25T00:13:52.906083+0000","last_active":"2024-04-25T00:13:54.457190+0000","last_peered":"2024-04-25T00:13:54.457190+0000","last_clean":"2024-04-25T00:13:54.457190+0000","last_became_active":"2024-04-25T00:13:52.905804+0000","last_became_peered":"2024-04-25T00:13:52.905804+0000","last_unstale":"2024-04-25T00:13:54.457190+0000","last_undegraded":"2024-04-25T00:13:54.457190+0000","last_fullsized":"2024-04-25T00:13:54.457190+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:37.690983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.353423+0000","last_change":"2024-04-25T00:13:51.910660+0000","last_active":"2024-04-25T00:13:54.353423+0000","last_peered":"2024-04-25T00:13:54.353423+0000","last_clean":"2024-04-25T00:13:54.353423+0000","last_became_active":"2024-04-25T00:13:51.910397+0000","last_became_peered":"2024-04-25T00:13:51.910397+0000","last_unstale":"2024-04-25T00:13:54.353423+0000","last_undegraded":"2024-04-25T00:13:54.353423+0000","last_fullsized":"2024-04-25T00:13:54.353423+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:03.812218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906331+0000","last_change":"2024-04-25T00:13:52.925029+0000","last_active":"2024-04-25T00:13:53.906331+0000","last_peered":"2024-04-25T00:13:53.906331+0000","last_clean":"2024-04-25T00:13:53.906331+0000","last_became_active":"2024-04-25T00:13:52.924705+0000","last_became_peered":"2024-04-25T00:13:52.924705+0000","last_unstale":"2024-04-25T00:13:53.906331+0000","last_undegraded":"2024-04-25T00:13:53.906331+0000","last_fullsized":"2024-04-25T00:13:53.906331+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:14:20.004143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"71'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.913412+0000","last_change":"2024-04-25T00:13:51.908015+0000","last_active":"2024-04-25T00:13:53.913412+0000","last_peered":"2024-04-25T00:13:53.913412+0000","last_clean":"2024-04-25T00:13:53.913412+0000","last_became_active":"2024-04-25T00:13:51.906803+0000","last_became_peered":"2024-04-25T00:13:51.906803+0000","last_unstale":"2024-04-25T00:13:53.913412+0000","last_undegraded":"2024-04-25T00:13:53.913412+0000","last_fullsized":"2024-04-25T00:13:53.913412+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:40.645229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457618+0000","last_change":"2024-04-25T00:13:52.924432+0000","last_active":"2024-04-25T00:13:54.457618+0000","last_peered":"2024-04-25T00:13:54.457618+0000","last_clean":"2024-04-25T00:13:54.457618+0000","last_became_active":"2024-04-25T00:13:52.924165+0000","last_became_peered":"2024-04-25T00:13:52.924165+0000","last_unstale":"2024-04-25T00:13:54.457618+0000","last_undegraded":"2024-04-25T00:13:54.457618+0000","last_fullsized":"2024-04-25T00:13:54.457618+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:51.781844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458336+0000","last_change":"2024-04-25T00:13:51.915284+0000","last_active":"2024-04-25T00:13:54.458336+0000","last_peered":"2024-04-25T00:13:54.458336+0000","last_clean":"2024-04-25T00:13:54.458336+0000","last_became_active":"2024-04-25T00:13:51.915014+0000","last_became_peered":"2024-04-25T00:13:51.915014+0000","last_unstale":"2024-04-25T00:13:54.458336+0000","last_undegraded":"2024-04-25T00:13:54.458336+0000","last_fullsized":"2024-04-25T00:13:54.458336+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:15.814468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.911792+0000","last_change":"2024-04-25T00:13:52.906600+0000","last_active":"2024-04-25T00:13:53.911792+0000","last_peered":"2024-04-25T00:13:53.911792+0000","last_clean":"2024-04-25T00:13:53.911792+0000","last_became_active":"2024-04-25T00:13:52.906329+0000","last_became_peered":"2024-04-25T00:13:52.906329+0000","last_unstale":"2024-04-25T00:13:53.911792+0000","last_undegraded":"2024-04-25T00:13:53.911792+0000","last_fullsized":"2024-04-25T00:13:53.911792+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:41.000967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458005+0000","last_change":"2024-04-25T00:13:51.914352+0000","last_active":"2024-04-25T00:13:54.458005+0000","last_peered":"2024-04-25T00:13:54.458005+0000","last_clean":"2024-04-25T00:13:54.458005+0000","last_became_active":"2024-04-25T00:13:51.914019+0000","last_became_peered":"2024-04-25T00:13:51.914019+0000","last_unstale":"2024-04-25T00:13:54.458005+0000","last_undegraded":"2024-04-25T00:13:54.458005+0000","last_fullsized":"2024-04-25T00:13:54.458005+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:43.614223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906140+0000","last_change":"2024-04-25T00:13:52.930622+0000","last_active":"2024-04-25T00:13:53.906140+0000","last_peered":"2024-04-25T00:13:53.906140+0000","last_clean":"2024-04-25T00:13:53.906140+0000","last_became_active":"2024-04-25T00:13:52.930272+0000","last_became_peered":"2024-04-25T00:13:52.930272+0000","last_unstale":"2024-04-25T00:13:53.906140+0000","last_undegraded":"2024-04-25T00:13:53.906140+0000","last_fullsized":"2024-04-25T00:13:53.906140+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:20.863218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906440+0000","last_change":"2024-04-25T00:13:51.905739+0000","last_active":"2024-04-25T00:13:53.906440+0000","last_peered":"2024-04-25T00:13:53.906440+0000","last_clean":"2024-04-25T00:13:53.906440+0000","last_became_active":"2024-04-25T00:13:51.905453+0000","last_became_peered":"2024-04-25T00:13:51.905453+0000","last_unstale":"2024-04-25T00:13:53.906440+0000","last_undegraded":"2024-04-25T00:13:53.906440+0000","last_fullsized":"2024-04-25T00:13:53.906440+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:48.930850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906191+0000","last_change":"2024-04-25T00:13:52.911888+0000","last_active":"2024-04-25T00:13:53.906191+0000","last_peered":"2024-04-25T00:13:53.906191+0000","last_clean":"2024-04-25T00:13:53.906191+0000","last_became_active":"2024-04-25T00:13:52.911620+0000","last_became_peered":"2024-04-25T00:13:52.911620+0000","last_unstale":"2024-04-25T00:13:53.906191+0000","last_undegraded":"2024-04-25T00:13:53.906191+0000","last_fullsized":"2024-04-25T00:13:53.906191+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:10.399293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.907155+0000","last_change":"2024-04-25T00:13:51.910063+0000","last_active":"2024-04-25T00:13:53.907155+0000","last_peered":"2024-04-25T00:13:53.907155+0000","last_clean":"2024-04-25T00:13:53.907155+0000","last_became_active":"2024-04-25T00:13:51.909814+0000","last_became_peered":"2024-04-25T00:13:51.909814+0000","last_unstale":"2024-04-25T00:13:53.907155+0000","last_undegraded":"2024-04-25T00:13:53.907155+0000","last_fullsized":"2024-04-25T00:13:53.907155+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:00.742666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.909206+0000","last_change":"2024-04-25T00:13:52.902612+0000","last_active":"2024-04-25T00:13:53.909206+0000","last_peered":"2024-04-25T00:13:53.909206+0000","last_clean":"2024-04-25T00:13:53.909206+0000","last_became_active":"2024-04-25T00:13:52.902290+0000","last_became_peered":"2024-04-25T00:13:52.902290+0000","last_unstale":"2024-04-25T00:13:53.909206+0000","last_undegraded":"2024-04-25T00:13:53.909206+0000","last_fullsized":"2024-04-25T00:13:53.909206+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:08.688249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.906356+0000","last_change":"2024-04-25T00:13:51.902090+0000","last_active":"2024-04-25T00:13:53.906356+0000","last_peered":"2024-04-25T00:13:53.906356+0000","last_clean":"2024-04-25T00:13:53.906356+0000","last_became_active":"2024-04-25T00:13:51.901819+0000","last_became_peered":"2024-04-25T00:13:51.901819+0000","last_unstale":"2024-04-25T00:13:53.906356+0000","last_undegraded":"2024-04-25T00:13:53.906356+0000","last_fullsized":"2024-04-25T00:13:53.906356+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:00.849796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.909284+0000","last_change":"2024-04-25T00:13:52.907368+0000","last_active":"2024-04-25T00:13:53.909284+0000","last_peered":"2024-04-25T00:13:53.909284+0000","last_clean":"2024-04-25T00:13:53.909284+0000","last_became_active":"2024-04-25T00:13:52.907038+0000","last_became_peered":"2024-04-25T00:13:52.907038+0000","last_unstale":"2024-04-25T00:13:53.909284+0000","last_undegraded":"2024-04-25T00:13:53.909284+0000","last_fullsized":"2024-04-25T00:13:53.909284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:10.399293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:53.908648+0000","last_change":"2024-04-25T00:13:51.905204+0000","last_active":"2024-04-25T00:13:53.908648+0000","last_peered":"2024-04-25T00:13:53.908648+0000","last_clean":"2024-04-25T00:13:53.908648+0000","last_became_active":"2024-04-25T00:13:51.904954+0000","last_became_peered":"2024-04-25T00:13:51.904954+0000","last_unstale":"2024-04-25T00:13:53.908648+0000","last_undegraded":"2024-04-25T00:13:53.908648+0000","last_fullsized":"2024-04-25T00:13:53.908648+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:36.685151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.458331+0000","last_change":"2024-04-25T00:13:52.915444+0000","last_active":"2024-04-25T00:13:54.458331+0000","last_peered":"2024-04-25T00:13:54.458331+0000","last_clean":"2024-04-25T00:13:54.458331+0000","last_became_active":"2024-04-25T00:13:52.915151+0000","last_became_peered":"2024-04-25T00:13:52.915151+0000","last_unstale":"2024-04-25T00:13:54.458331+0000","last_undegraded":"2024-04-25T00:13:54.458331+0000","last_fullsized":"2024-04-25T00:13:54.458331+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:51.877950+0000","last_clean_scrub_stamp":"2024-04-25T00:13:51.877950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:36.719641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457626+0000","last_change":"2024-04-25T00:13:51.906121+0000","last_active":"2024-04-25T00:13:54.457626+0000","last_peered":"2024-04-25T00:13:54.457626+0000","last_clean":"2024-04-25T00:13:54.457626+0000","last_became_active":"2024-04-25T00:13:51.905829+0000","last_became_peered":"2024-04-25T00:13:51.905829+0000","last_unstale":"2024-04-25T00:13:54.457626+0000","last_undegraded":"2024-04-25T00:13:54.457626+0000","last_fullsized":"2024-04-25T00:13:54.457626+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:13:50.872118+0000","last_clean_scrub_stamp":"2024-04-25T00:13:50.872118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:47.453141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":263,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T00:13:54.457701+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:13:54.457701+0000","last_peered":"2024-04-25T00:13:54.457701+0000","last_clean":"2024-04-25T00:13:54.457701+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:13:54.457701+0000","last_undegraded":"2024-04-25T00:13:54.457701+0000","last_fullsized":"2024-04-25T00:13:54.457701+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1454080,"data_stored":976262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":387484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":392598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":387588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":386454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":386614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":392676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1449984,"data_stored":976240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:13:57.354 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:13:57.354 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:13:57.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe446e00640 1 Processor -- start 2024-04-25T00:13:57.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe446e00640 1 -- start start 2024-04-25T00:13:57.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe446e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe4401551e0 0x7fe44015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe446e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe4401555c0 0x7fe440071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe446e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe44006e4c0 0x7fe44006e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe446e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe44015c560 con 0x7fe4401551e0 2024-04-25T00:13:57.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe446e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe440072460 con 0x7fe4401555c0 2024-04-25T00:13:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe446e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe44015c7d0 con 0x7fe44006e4c0 2024-04-25T00:13:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe43fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe4401551e0 0x7fe44015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe43fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe4401551e0 0x7fe44015bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45152/0 (socket says 172.21.15.83:45152) 2024-04-25T00:13:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe43fe00640 1 -- 172.21.15.83:0/1569026326 learned_addr learned my addr 172.21.15.83:0/1569026326 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe437400640 1 --2- 172.21.15.83:0/1569026326 >> v2:172.21.15.86:3300/0 conn(0x7fe4401555c0 0x7fe440071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe43fe00640 1 -- 172.21.15.83:0/1569026326 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe44006e4c0 msgr2=0x7fe44006e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe444e00640 1 --2- 172.21.15.83:0/1569026326 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe44006e4c0 0x7fe44006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe43fe00640 1 --2- 172.21.15.83:0/1569026326 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe44006e4c0 0x7fe44006e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe43fe00640 1 -- 172.21.15.83:0/1569026326 >> v2:172.21.15.86:3300/0 conn(0x7fe4401555c0 msgr2=0x7fe440071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe43fe00640 1 --2- 172.21.15.83:0/1569026326 >> v2:172.21.15.86:3300/0 conn(0x7fe4401555c0 0x7fe440071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe43fe00640 1 -- 172.21.15.83:0/1569026326 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44015cab0 con 0x7fe4401551e0 2024-04-25T00:13:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe437400640 1 --2- 172.21.15.83:0/1569026326 >> v2:172.21.15.86:3300/0 conn(0x7fe4401555c0 0x7fe440071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.536+0000 7fe444e00640 1 --2- 172.21.15.83:0/1569026326 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe44006e4c0 0x7fe44006e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.540+0000 7fe43fe00640 1 --2- 172.21.15.83:0/1569026326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4401551e0 0x7fe44015bea0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe430009e50 tx=0x7fe43001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7018b8150d59b04c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.540+0000 7fe43f400640 1 -- 172.21.15.83:0/1569026326 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4300a5780 con 0x7fe4401551e0 2024-04-25T00:13:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.540+0000 7fe43f400640 1 -- 172.21.15.83:0/1569026326 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4300a5920 con 0x7fe4401551e0 2024-04-25T00:13:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.540+0000 7fe43f400640 1 -- 172.21.15.83:0/1569026326 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4300a5cf0 con 0x7fe4401551e0 2024-04-25T00:13:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.540+0000 7fe446e00640 1 -- 172.21.15.83:0/1569026326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4401551e0 msgr2=0x7fe44015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.540+0000 7fe446e00640 1 --2- 172.21.15.83:0/1569026326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4401551e0 0x7fe44015bea0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe430009e50 tx=0x7fe43001c9f0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.540+0000 7fe446e00640 1 -- 172.21.15.83:0/1569026326 shutdown_connections 2024-04-25T00:13:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.540+0000 7fe446e00640 1 --2- 172.21.15.83:0/1569026326 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe44006e4c0 0x7fe44006e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.540+0000 7fe446e00640 1 --2- 172.21.15.83:0/1569026326 >> v2:172.21.15.86:3300/0 conn(0x7fe4401555c0 0x7fe440071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.540+0000 7fe446e00640 1 --2- 172.21.15.83:0/1569026326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4401551e0 0x7fe44015bea0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.540+0000 7fe446e00640 1 -- 172.21.15.83:0/1569026326 >> 172.21.15.83:0/1569026326 conn(0x7fe4400b1830 msgr2=0x7fe4400b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.540+0000 7fe446e00640 1 -- 172.21.15.83:0/1569026326 shutdown_connections 2024-04-25T00:13:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.540+0000 7fe446e00640 1 -- 172.21.15.83:0/1569026326 wait complete. 2024-04-25T00:13:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe446e00640 1 Processor -- start 2024-04-25T00:13:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe446e00640 1 -- start start 2024-04-25T00:13:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe446e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44006e4c0 0x7fe44014e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe446e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4401551e0 0x7fe44014fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe446e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4401555c0 0x7fe440150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe446e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe44015f490 con 0x7fe4401551e0 2024-04-25T00:13:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe446e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe44015f310 con 0x7fe44006e4c0 2024-04-25T00:13:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe446e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe44015f610 con 0x7fe4401555c0 2024-04-25T00:13:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe437400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4401551e0 0x7fe44014fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe437400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4401551e0 0x7fe44014fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45156/0 (socket says 172.21.15.83:45156) 2024-04-25T00:13:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe437400640 1 -- 172.21.15.83:0/2726329307 learned_addr learned my addr 172.21.15.83:0/2726329307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe444e00640 1 --2- 172.21.15.83:0/2726329307 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4401555c0 0x7fe440150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe43fe00640 1 --2- 172.21.15.83:0/2726329307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44006e4c0 0x7fe44014e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe437400640 1 -- 172.21.15.83:0/2726329307 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4401555c0 msgr2=0x7fe440150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe437400640 1 --2- 172.21.15.83:0/2726329307 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4401555c0 0x7fe440150cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe437400640 1 -- 172.21.15.83:0/2726329307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44006e4c0 msgr2=0x7fe44014e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe437400640 1 --2- 172.21.15.83:0/2726329307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44006e4c0 0x7fe44014e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe437400640 1 -- 172.21.15.83:0/2726329307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4400b2b70 con 0x7fe4401551e0 2024-04-25T00:13:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe437400640 1 --2- 172.21.15.83:0/2726329307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4401551e0 0x7fe44014fff0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe42800c820 tx=0x7fe42800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe43cc00640 1 -- 172.21.15.83:0/2726329307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428015070 con 0x7fe4401551e0 2024-04-25T00:13:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe43cc00640 1 -- 172.21.15.83:0/2726329307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe428004220 con 0x7fe4401551e0 2024-04-25T00:13:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe43cc00640 1 -- 172.21.15.83:0/2726329307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42800ce10 con 0x7fe4401551e0 2024-04-25T00:13:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe446e00640 1 -- 172.21.15.83:0/2726329307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4400b2e50 con 0x7fe4401551e0 2024-04-25T00:13:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe446e00640 1 -- 172.21.15.83:0/2726329307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44014f2e0 con 0x7fe4401551e0 2024-04-25T00:13:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.544+0000 7fe446e00640 1 -- 172.21.15.83:0/2726329307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44004f510 con 0x7fe4401551e0 2024-04-25T00:13:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.548+0000 7fe43cc00640 1 -- 172.21.15.83:0/2726329307 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe428007830 con 0x7fe4401551e0 2024-04-25T00:13:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.548+0000 7fe43cc00640 1 --2- 172.21.15.83:0/2726329307 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe41c0678f0 0x7fe41c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.548+0000 7fe43cc00640 1 -- 172.21.15.83:0/2726329307 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7fe428085c80 con 0x7fe4401551e0 2024-04-25T00:13:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.548+0000 7fe43fe00640 1 --2- 172.21.15.83:0/2726329307 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe41c0678f0 0x7fe41c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.548+0000 7fe43fe00640 1 --2- 172.21.15.83:0/2726329307 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe41c0678f0 0x7fe41c069db0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe4300ae000 tx=0x7fe4300a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:57.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.552+0000 7fe43cc00640 1 -- 172.21.15.83:0/2726329307 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe428056590 con 0x7fe4401551e0 2024-04-25T00:13:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.760+0000 7fe446e00640 1 -- 172.21.15.83:0/2726329307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe4400b28a0 con 0x7fe4401551e0 2024-04-25T00:13:57.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.764+0000 7fe43cc00640 1 -- 172.21.15.83:0/2726329307 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7fe428055e90 con 0x7fe4401551e0 2024-04-25T00:13:57.768 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-25T00:13:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylmOqooLBAApte+X6YkRmrpe12YdjYekg== 2024-04-25T00:13:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-25T00:13:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-25T00:13:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:13:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-25T00:13:57.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.768+0000 7fe446e00640 1 -- 172.21.15.83:0/2726329307 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe41c0678f0 msgr2=0x7fe41c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.768+0000 7fe446e00640 1 --2- 172.21.15.83:0/2726329307 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe41c0678f0 0x7fe41c069db0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe4300ae000 tx=0x7fe4300a7070 comp rx=0 tx=0).stop 2024-04-25T00:13:57.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.768+0000 7fe446e00640 1 -- 172.21.15.83:0/2726329307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4401551e0 msgr2=0x7fe44014fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.768+0000 7fe446e00640 1 --2- 172.21.15.83:0/2726329307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4401551e0 0x7fe44014fff0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe42800c820 tx=0x7fe42800cce0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.768+0000 7fe446e00640 1 -- 172.21.15.83:0/2726329307 shutdown_connections 2024-04-25T00:13:57.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.768+0000 7fe446e00640 1 --2- 172.21.15.83:0/2726329307 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe41c0678f0 0x7fe41c069db0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.768+0000 7fe446e00640 1 --2- 172.21.15.83:0/2726329307 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4401555c0 0x7fe440150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.768+0000 7fe446e00640 1 --2- 172.21.15.83:0/2726329307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4401551e0 0x7fe44014fff0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.768+0000 7fe446e00640 1 --2- 172.21.15.83:0/2726329307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44006e4c0 0x7fe44014e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.768+0000 7fe446e00640 1 -- 172.21.15.83:0/2726329307 >> 172.21.15.83:0/2726329307 conn(0x7fe4400b1830 msgr2=0x7fe4400b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.768+0000 7fe446e00640 1 -- 172.21.15.83:0/2726329307 shutdown_connections 2024-04-25T00:13:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.768+0000 7fe446e00640 1 -- 172.21.15.83:0/2726329307 wait complete. 2024-04-25T00:13:57.790 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:57.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa459a00640 1 Processor -- start 2024-04-25T00:13:57.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa459a00640 1 -- start start 2024-04-25T00:13:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa459a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa454145940 0x7fa454072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa459a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa45406de00 0x7fa4540d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa459a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4540d1a40 con 0x7fa4540cd2a0 2024-04-25T00:13:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa459a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa45406e260 con 0x7fa45406de00 2024-04-25T00:13:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa459a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa4540d0f00 con 0x7fa454145940 2024-04-25T00:13:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa453400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa4540cd2a0 0x7fa4540cf910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33158/0 (socket says 172.21.15.83:33158) 2024-04-25T00:13:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa453400640 1 -- 172.21.15.83:0/180854003 learned_addr learned my addr 172.21.15.83:0/180854003 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:13:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa452000640 1 --2- 172.21.15.83:0/180854003 >> v2:172.21.15.86:3300/0 conn(0x7fa45406de00 0x7fa4540d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa452a00640 1 --2- 172.21.15.83:0/180854003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa454145940 0x7fa454072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 -- 172.21.15.83:0/180854003 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 95005833 0 0) 0x7fa4540d1a40 con 0x7fa4540cd2a0 2024-04-25T00:13:57.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 -- 172.21.15.83:0/180854003 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa438003530 con 0x7fa4540cd2a0 2024-04-25T00:13:57.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 -- 172.21.15.83:0/180854003 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2859735968 0 0) 0x7fa438003530 con 0x7fa4540cd2a0 2024-04-25T00:13:57.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 -- 172.21.15.83:0/180854003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa454145940 msgr2=0x7fa454072ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:57.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 --2- 172.21.15.83:0/180854003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa454145940 0x7fa454072ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 -- 172.21.15.83:0/180854003 >> v2:172.21.15.86:3300/0 conn(0x7fa45406de00 msgr2=0x7fa4540d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:57.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 --2- 172.21.15.83:0/180854003 >> v2:172.21.15.86:3300/0 conn(0x7fa45406de00 0x7fa4540d09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 -- 172.21.15.83:0/180854003 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4540d11a0 con 0x7fa4540cd2a0 2024-04-25T00:13:57.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 -- 172.21.15.83:0/180854003 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fa44c001680 con 0x7fa4540cd2a0 2024-04-25T00:13:57.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 -- 172.21.15.83:0/180854003 >> v1:172.21.15.83:6789/0 conn(0x7fa4540cd2a0 legacy=0x7fa4540cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 --2- 172.21.15.83:0/180854003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4380039c0 0x7fa438003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 --2- 172.21.15.83:0/180854003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa438004810 0x7fa438004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 --2- 172.21.15.83:0/180854003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa454145940 0x7fa4380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 -- 172.21.15.83:0/180854003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa45406e260 con 0x7fa4380039c0 2024-04-25T00:13:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 -- 172.21.15.83:0/180854003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4540d0f00 con 0x7fa438004810 2024-04-25T00:13:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa451600640 1 -- 172.21.15.83:0/180854003 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa43800b9d0 con 0x7fa454145940 2024-04-25T00:13:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa452000640 1 --2- 172.21.15.83:0/180854003 >> v2:172.21.15.86:3300/0 conn(0x7fa45406de00 0x7fa4540d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.924+0000 7fa452a00640 1 --2- 172.21.15.83:0/180854003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4380039c0 0x7fa438003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa452a00640 1 -- 172.21.15.83:0/180854003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa454145940 msgr2=0x7fa4380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa452a00640 1 --2- 172.21.15.83:0/180854003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa454145940 0x7fa4380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa452a00640 1 -- 172.21.15.83:0/180854003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa438004810 msgr2=0x7fa438004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa452a00640 1 --2- 172.21.15.83:0/180854003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa438004810 0x7fa438004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa452a00640 1 -- 172.21.15.83:0/180854003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa438003530 con 0x7fa4380039c0 2024-04-25T00:13:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa452a00640 1 --2- 172.21.15.83:0/180854003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4380039c0 0x7fa438003de0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa444009fd0 tx=0x7fa44401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd544ffadf4f5ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa451600640 1 -- 172.21.15.83:0/180854003 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4440b2070 con 0x7fa4380039c0 2024-04-25T00:13:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa451600640 1 -- 172.21.15.83:0/180854003 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4440a5a50 con 0x7fa4380039c0 2024-04-25T00:13:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa451600640 1 -- 172.21.15.83:0/180854003 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444009bf0 con 0x7fa4380039c0 2024-04-25T00:13:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa459a00640 1 -- 172.21.15.83:0/180854003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4380039c0 msgr2=0x7fa438003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa459a00640 1 --2- 172.21.15.83:0/180854003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4380039c0 0x7fa438003de0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa444009fd0 tx=0x7fa44401cc80 comp rx=0 tx=0).stop 2024-04-25T00:13:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa453400640 1 -- 172.21.15.83:0/180854003 reap_dead start 2024-04-25T00:13:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa459a00640 1 -- 172.21.15.83:0/180854003 shutdown_connections 2024-04-25T00:13:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa459a00640 1 -- 172.21.15.83:0/180854003 >> 172.21.15.83:0/180854003 conn(0x7fa45406c2f0 msgr2=0x7fa45406c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.928+0000 7fa459a00640 1 -- 172.21.15.83:0/180854003 shutdown_connections 2024-04-25T00:13:57.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa459a00640 1 -- 172.21.15.83:0/180854003 wait complete. 2024-04-25T00:13:57.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa459a00640 1 Processor -- start 2024-04-25T00:13:57.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa459a00640 1 -- start start 2024-04-25T00:13:57.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa459a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4380039c0 0x7fa4540cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa459a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4540b4550 0x7fa4540b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa459a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4540b2c60 0x7fa454157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa459a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4540d0020 con 0x7fa4380039c0 2024-04-25T00:13:57.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa459a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4540cdae0 con 0x7fa4540b4550 2024-04-25T00:13:57.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa459a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa4540d01a0 con 0x7fa4540b2c60 2024-04-25T00:13:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa452a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4380039c0 0x7fa4540cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa452a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4380039c0 0x7fa4540cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45176/0 (socket says 172.21.15.83:45176) 2024-04-25T00:13:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa452a00640 1 -- 172.21.15.83:0/825078937 learned_addr learned my addr 172.21.15.83:0/825078937 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa453400640 1 --2- 172.21.15.83:0/825078937 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4540b2c60 0x7fa454157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa452000640 1 --2- 172.21.15.83:0/825078937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4540b4550 0x7fa4540b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa452a00640 1 -- 172.21.15.83:0/825078937 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4540b2c60 msgr2=0x7fa454157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa452a00640 1 --2- 172.21.15.83:0/825078937 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4540b2c60 0x7fa454157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa452a00640 1 -- 172.21.15.83:0/825078937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4540b4550 msgr2=0x7fa4540b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa452a00640 1 --2- 172.21.15.83:0/825078937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4540b4550 0x7fa4540b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa452a00640 1 -- 172.21.15.83:0/825078937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4540cc320 con 0x7fa4380039c0 2024-04-25T00:13:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.932+0000 7fa452a00640 1 --2- 172.21.15.83:0/825078937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4380039c0 0x7fa4540cbd40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa4440b0040 tx=0x7fa438005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.936+0000 7fa44aa00640 1 -- 172.21.15.83:0/825078937 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4440b2050 con 0x7fa4380039c0 2024-04-25T00:13:57.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.936+0000 7fa44aa00640 1 -- 172.21.15.83:0/825078937 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4440be070 con 0x7fa4380039c0 2024-04-25T00:13:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.936+0000 7fa44aa00640 1 -- 172.21.15.83:0/825078937 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444009bf0 con 0x7fa4380039c0 2024-04-25T00:13:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.936+0000 7fa459a00640 1 -- 172.21.15.83:0/825078937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4540cc600 con 0x7fa4380039c0 2024-04-25T00:13:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.936+0000 7fa459a00640 1 -- 172.21.15.83:0/825078937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4540b4030 con 0x7fa4380039c0 2024-04-25T00:13:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.936+0000 7fa459a00640 1 -- 172.21.15.83:0/825078937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4540736c0 con 0x7fa4380039c0 2024-04-25T00:13:57.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.936+0000 7fa44aa00640 1 -- 172.21.15.83:0/825078937 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa4440c3460 con 0x7fa4380039c0 2024-04-25T00:13:57.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.936+0000 7fa44aa00640 1 --2- 172.21.15.83:0/825078937 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4340678c0 0x7fa434069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.936+0000 7fa44aa00640 1 -- 172.21.15.83:0/825078937 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7fa4441211b0 con 0x7fa4380039c0 2024-04-25T00:13:57.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.936+0000 7fa452000640 1 --2- 172.21.15.83:0/825078937 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4340678c0 0x7fa434069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.936+0000 7fa452000640 1 --2- 172.21.15.83:0/825078937 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4340678c0 0x7fa434069d80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa4540b52d0 tx=0x7fa440009650 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:57.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:57.940+0000 7fa44aa00640 1 -- 172.21.15.83:0/825078937 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa4440169d0 con 0x7fa4380039c0 2024-04-25T00:13:58.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.128+0000 7fa459a00640 1 -- 172.21.15.83:0/825078937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4540cc7d0 con 0x7fa4380039c0 2024-04-25T00:13:58.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.128+0000 7fa44aa00640 1 -- 172.21.15.83:0/825078937 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5183 (secure 0 0 0) 0x7fa4440f13e0 con 0x7fa4380039c0 2024-04-25T00:13:58.132 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:13:58.132 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2852678498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2852678498},{"type":"v1","addr":"172.21.15.86:6834","nonce":2852678498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2956252746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2956252746},{"type":"v1","addr":"172.21.15.86:6835","nonce":2956252746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6588,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2678835998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2678835998},{"type":"v1","addr":"172.21.15.83:6836","nonce":2678835998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:13:52.273680+0000","modified":"2024-04-25T00:13:55.285817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2879143105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2879143105},{"type":"v1","addr":"172.21.15.83:6837","nonce":2879143105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6591,"qdb_cluster":[6591]},"id":5}]} 2024-04-25T00:13:58.132 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 44 2024-04-25T00:13:58.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.132+0000 7fa459a00640 1 -- 172.21.15.83:0/825078937 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4340678c0 msgr2=0x7fa434069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.132+0000 7fa459a00640 1 --2- 172.21.15.83:0/825078937 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4340678c0 0x7fa434069d80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa4540b52d0 tx=0x7fa440009650 comp rx=0 tx=0).stop 2024-04-25T00:13:58.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.132+0000 7fa459a00640 1 -- 172.21.15.83:0/825078937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4380039c0 msgr2=0x7fa4540cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.132+0000 7fa459a00640 1 --2- 172.21.15.83:0/825078937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4380039c0 0x7fa4540cbd40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa4440b0040 tx=0x7fa438005840 comp rx=0 tx=0).stop 2024-04-25T00:13:58.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.132+0000 7fa459a00640 1 -- 172.21.15.83:0/825078937 shutdown_connections 2024-04-25T00:13:58.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.132+0000 7fa459a00640 1 --2- 172.21.15.83:0/825078937 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa4340678c0 0x7fa434069d80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.132+0000 7fa459a00640 1 --2- 172.21.15.83:0/825078937 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4540b2c60 0x7fa454157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.132+0000 7fa459a00640 1 --2- 172.21.15.83:0/825078937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4540b4550 0x7fa4540b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.132+0000 7fa459a00640 1 --2- 172.21.15.83:0/825078937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4380039c0 0x7fa4540cbd40 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.132+0000 7fa459a00640 1 -- 172.21.15.83:0/825078937 >> 172.21.15.83:0/825078937 conn(0x7fa45406c2f0 msgr2=0x7fa45415f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:58.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.132+0000 7fa459a00640 1 -- 172.21.15.83:0/825078937 shutdown_connections 2024-04-25T00:13:58.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.132+0000 7fa459a00640 1 -- 172.21.15.83:0/825078937 wait complete. 2024-04-25T00:13:58.556 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:13:52.273680+0000', 'modified': '2024-04-25T00:13:55.285817+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6591}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6591': {'gid': 6591, 'name': 'a', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/2879143105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6835', 'nonce': 2879143105}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 2879143105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6591, 'qdb_cluster': [6591]} 2024-04-25T00:13:58.556 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:13:58.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:13:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.692+0000 7f4d23c00640 1 Processor -- start 2024-04-25T00:13:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.692+0000 7f4d23c00640 1 -- start start 2024-04-25T00:13:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.692+0000 7f4d23c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4d1c06ee20 0x7f4d1c0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.692+0000 7f4d23c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4d1c15f350 0x7f4d1c071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.692+0000 7f4d23c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d1c0cb210 con 0x7f4d1c1446e0 2024-04-25T00:13:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.692+0000 7f4d23c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4d1c0715b0 con 0x7f4d1c15f350 2024-04-25T00:13:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.692+0000 7f4d23c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4d1c144ac0 con 0x7f4d1c06ee20 2024-04-25T00:13:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.692+0000 7f4d21c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4d1c15f350 0x7f4d1c071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.692+0000 7f4d21c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4d1c15f350 0x7f4d1c071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45182/0 (socket says 172.21.15.83:45182) 2024-04-25T00:13:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.692+0000 7f4d21c00640 1 -- 172.21.15.83:0/4038306063 learned_addr learned my addr 172.21.15.83:0/4038306063 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.692+0000 7f4d1be00640 1 --2- 172.21.15.83:0/4038306063 >> v2:172.21.15.86:3300/0 conn(0x7f4d1c06ee20 0x7f4d1c0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d21c00640 1 -- 172.21.15.83:0/4038306063 >> v2:172.21.15.86:3300/0 conn(0x7f4d1c06ee20 msgr2=0x7f4d1c0cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d21c00640 1 --2- 172.21.15.83:0/4038306063 >> v2:172.21.15.86:3300/0 conn(0x7f4d1c06ee20 0x7f4d1c0cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d21c00640 1 -- 172.21.15.83:0/4038306063 >> v1:172.21.15.86:6789/0 conn(0x7f4d1c1446e0 legacy=0x7f4d1c06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d21c00640 1 -- 172.21.15.83:0/4038306063 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c0cb4f0 con 0x7f4d1c15f350 2024-04-25T00:13:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d1be00640 1 --2- 172.21.15.83:0/4038306063 >> v2:172.21.15.86:3300/0 conn(0x7f4d1c06ee20 0x7f4d1c0cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:58.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d21c00640 1 --2- 172.21.15.83:0/4038306063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c15f350 0x7f4d1c071fe0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c068a60 tx=0x7f4d1401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa41a1f9ecf55f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d1b400640 1 -- 172.21.15.83:0/4038306063 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140a5750 con 0x7f4d1c15f350 2024-04-25T00:13:58.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d1b400640 1 -- 172.21.15.83:0/4038306063 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d140a58f0 con 0x7f4d1c15f350 2024-04-25T00:13:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d1b400640 1 -- 172.21.15.83:0/4038306063 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140a5cc0 con 0x7f4d1c15f350 2024-04-25T00:13:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d23c00640 1 -- 172.21.15.83:0/4038306063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c15f350 msgr2=0x7f4d1c071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d23c00640 1 --2- 172.21.15.83:0/4038306063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c15f350 0x7f4d1c071fe0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c068a60 tx=0x7f4d1401c9f0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d23c00640 1 -- 172.21.15.83:0/4038306063 shutdown_connections 2024-04-25T00:13:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d23c00640 1 --2- 172.21.15.83:0/4038306063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c15f350 0x7f4d1c071fe0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d23c00640 1 --2- 172.21.15.83:0/4038306063 >> v2:172.21.15.86:3300/0 conn(0x7f4d1c06ee20 0x7f4d1c0cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d23c00640 1 -- 172.21.15.83:0/4038306063 >> 172.21.15.83:0/4038306063 conn(0x7f4d1c0cfa60 msgr2=0x7f4d1c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d23c00640 1 -- 172.21.15.83:0/4038306063 shutdown_connections 2024-04-25T00:13:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.696+0000 7f4d23c00640 1 -- 172.21.15.83:0/4038306063 wait complete. 2024-04-25T00:13:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d23c00640 1 Processor -- start 2024-04-25T00:13:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d23c00640 1 -- start start 2024-04-25T00:13:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d23c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d1c06ee20 0x7f4d1c15e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d23c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d1c1446e0 0x7f4d1c1414c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d23c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c15f350 0x7f4d1c141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d23c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c072eb0 con 0x7f4d1c15f350 2024-04-25T00:13:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d23c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c072d30 con 0x7f4d1c06ee20 2024-04-25T00:13:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d23c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4d1c073030 con 0x7f4d1c1446e0 2024-04-25T00:13:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d21c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c15f350 0x7f4d1c141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d21c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c15f350 0x7f4d1c141a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45192/0 (socket says 172.21.15.83:45192) 2024-04-25T00:13:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d21c00640 1 -- 172.21.15.83:0/1119249840 learned_addr learned my addr 172.21.15.83:0/1119249840 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d21200640 1 --2- 172.21.15.83:0/1119249840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d1c06ee20 0x7f4d1c15e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d1be00640 1 --2- 172.21.15.83:0/1119249840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d1c1446e0 0x7f4d1c1414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d21c00640 1 -- 172.21.15.83:0/1119249840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d1c1446e0 msgr2=0x7f4d1c1414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d21c00640 1 --2- 172.21.15.83:0/1119249840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d1c1446e0 0x7f4d1c1414c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d21c00640 1 -- 172.21.15.83:0/1119249840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d1c06ee20 msgr2=0x7f4d1c15e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d21c00640 1 --2- 172.21.15.83:0/1119249840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d1c06ee20 0x7f4d1c15e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d21c00640 1 -- 172.21.15.83:0/1119249840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c13c020 con 0x7f4d1c15f350 2024-04-25T00:13:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d21c00640 1 --2- 172.21.15.83:0/1119249840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c15f350 0x7f4d1c141a00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4d140af040 tx=0x7f4d14019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d18c00640 1 -- 172.21.15.83:0/1119249840 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140bc020 con 0x7f4d1c15f350 2024-04-25T00:13:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d18c00640 1 -- 172.21.15.83:0/1119249840 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d140b8690 con 0x7f4d1c15f350 2024-04-25T00:13:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d23c00640 1 -- 172.21.15.83:0/1119249840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1c13c300 con 0x7f4d1c15f350 2024-04-25T00:13:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d18c00640 1 -- 172.21.15.83:0/1119249840 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140b8a80 con 0x7f4d1c15f350 2024-04-25T00:13:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.700+0000 7f4d23c00640 1 -- 172.21.15.83:0/1119249840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d1c13c8c0 con 0x7f4d1c15f350 2024-04-25T00:13:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.704+0000 7f4d18c00640 1 -- 172.21.15.83:0/1119249840 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4d140bf020 con 0x7f4d1c15f350 2024-04-25T00:13:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.704+0000 7f4d18c00640 1 --2- 172.21.15.83:0/1119249840 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4cfc0678c0 0x7f4cfc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.704+0000 7f4d18c00640 1 -- 172.21.15.83:0/1119249840 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f4d14120bc0 con 0x7f4d1c15f350 2024-04-25T00:13:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.704+0000 7f4d21200640 1 --2- 172.21.15.83:0/1119249840 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4cfc0678c0 0x7f4cfc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.704+0000 7f4d270e2000 1 Processor -- start 2024-04-25T00:13:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.704+0000 7f4d270e2000 1 -- start start 2024-04-25T00:13:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.704+0000 7f4d21200640 1 --2- 172.21.15.83:0/1119249840 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4cfc0678c0 0x7f4cfc069d80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4d10007fd0 tx=0x7f4d10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 0x558a16dada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db3a90 0x558a16db3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db43b0 0x558a16df74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558a16bf75e0 con 0x558a16db43b0 2024-04-25T00:13:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558a16b7ec50 con 0x558a16dad6b0 2024-04-25T00:13:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x558a16bf35a0 con 0x558a16db3a90 2024-04-25T00:13:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d21c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db43b0 0x558a16df74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d21200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 0x558a16dada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d21c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db43b0 0x558a16df74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45206/0 (socket says 172.21.15.83:45206) 2024-04-25T00:13:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d1be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db3a90 0x558a16db3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d21c00640 1 -- 172.21.15.83:0/3704548202 learned_addr learned my addr 172.21.15.83:0/3704548202 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d21c00640 1 -- 172.21.15.83:0/3704548202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db3a90 msgr2=0x558a16db3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d21c00640 1 --2- 172.21.15.83:0/3704548202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db3a90 0x558a16db3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d21c00640 1 -- 172.21.15.83:0/3704548202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 msgr2=0x558a16dada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d21c00640 1 --2- 172.21.15.83:0/3704548202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 0x558a16dada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d21c00640 1 -- 172.21.15.83:0/3704548202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a16b7cd10 con 0x558a16db43b0 2024-04-25T00:13:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d21c00640 1 --2- 172.21.15.83:0/3704548202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db43b0 0x558a16df74f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4d140f95b0 tx=0x7f4d140f0eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79681f4fe020c1e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d0d600640 1 -- 172.21.15.83:0/3704548202 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140f52b0 con 0x558a16db43b0 2024-04-25T00:13:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d0d600640 1 -- 172.21.15.83:0/3704548202 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d140f9ae0 con 0x558a16db43b0 2024-04-25T00:13:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d0d600640 1 -- 172.21.15.83:0/3704548202 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140ed620 con 0x558a16db43b0 2024-04-25T00:13:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 -- 172.21.15.83:0/3704548202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db43b0 msgr2=0x558a16df74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 --2- 172.21.15.83:0/3704548202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db43b0 0x558a16df74f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4d140f95b0 tx=0x7f4d140f0eb0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 -- 172.21.15.83:0/3704548202 shutdown_connections 2024-04-25T00:13:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 --2- 172.21.15.83:0/3704548202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db3a90 0x558a16db3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 --2- 172.21.15.83:0/3704548202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db43b0 0x558a16df74f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 --2- 172.21.15.83:0/3704548202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 0x558a16dada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 -- 172.21.15.83:0/3704548202 >> 172.21.15.83:0/3704548202 conn(0x558a16daed00 msgr2=0x558a16dabf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:58.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 -- 172.21.15.83:0/3704548202 shutdown_connections 2024-04-25T00:13:58.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 -- 172.21.15.83:0/3704548202 wait complete. 2024-04-25T00:13:58.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 -- start start 2024-04-25T00:13:58.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 Processor -- start 2024-04-25T00:13:58.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 0x558a16de0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.708+0000 7f4d270e2000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db3a90 0x558a16de1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db43b0 0x558a16de1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558a16bb1520 con 0x558a16db43b0 2024-04-25T00:13:58.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558a16bf68f0 con 0x558a16dad6b0 2024-04-25T00:13:58.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x558a16bb16a0 con 0x558a16db3a90 2024-04-25T00:13:58.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 10 client.0 _ll_register_callbacks cb 0x558a16d945c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:13:58.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d21c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db43b0 0x558a16de1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d21c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db43b0 0x558a16de1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45222/0 (socket says 172.21.15.83:45222) 2024-04-25T00:13:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d21c00640 1 -- 172.21.15.83:0/1534159011 learned_addr learned my addr 172.21.15.83:0/1534159011 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d1be00640 1 --2- 172.21.15.83:0/1534159011 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db3a90 0x558a16de1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d21200640 1 --2- 172.21.15.83:0/1534159011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 0x558a16de0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d21c00640 1 -- 172.21.15.83:0/1534159011 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db3a90 msgr2=0x558a16de1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d21c00640 1 --2- 172.21.15.83:0/1534159011 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db3a90 0x558a16de1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d21c00640 1 -- 172.21.15.83:0/1534159011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 msgr2=0x558a16de0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d21c00640 1 --2- 172.21.15.83:0/1534159011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 0x558a16de0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d21c00640 1 -- 172.21.15.83:0/1534159011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a16b8f7b0 con 0x558a16db43b0 2024-04-25T00:13:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d21c00640 1 --2- 172.21.15.83:0/1534159011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db43b0 0x558a16de1c40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4d140984f0 tx=0x7f4d14009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca0ad281f0e36a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d02000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:13:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d02000640 1 -- 172.21.15.83:0/1534159011 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140f1410 con 0x558a16db43b0 2024-04-25T00:13:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d02000640 1 -- 172.21.15.83:0/1534159011 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f4d140f9650 con 0x558a16db43b0 2024-04-25T00:13:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 -- 172.21.15.83:0/1534159011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d140f9ae0 con 0x558a16db43b0 2024-04-25T00:13:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d02000640 1 -- 172.21.15.83:0/1534159011 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4d140f9850 con 0x558a16db43b0 2024-04-25T00:13:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 10 client.6738 fetch_fsmap learned FSMap version 44 2024-04-25T00:13:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 -- 172.21.15.83:0/1534159011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x558a16b7eea0 con 0x558a16db43b0 2024-04-25T00:13:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d02000640 1 -- 172.21.15.83:0/1534159011 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d140f9ae0 con 0x558a16db43b0 2024-04-25T00:13:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d02000640 1 -- 172.21.15.83:0/1534159011 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f4d140f0400 con 0x558a16db43b0 2024-04-25T00:13:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 10 client.6738 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T00:13:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 10 client.6738 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:13:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 20 client.6738 populate_metadata read hostname 'smithi083' 2024-04-25T00:13:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 --2- 172.21.15.83:0/1534159011 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x558a16de2650 0x558a16de2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 4 client.6738 mds_command: new command op to 6591 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:13:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 -- 172.21.15.83:0/1534159011 --> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a16b92ab0 con 0x558a16de2650 2024-04-25T00:13:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d21200640 1 --2- 172.21.15.83:0/1534159011 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x558a16de2650 0x558a16de2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d02000640 10 client.6738 ms_handle_connect on v2:172.21.15.83:6835/2879143105 2024-04-25T00:13:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d21200640 1 --2- 172.21.15.83:0/1534159011 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x558a16de2650 0x558a16de2a30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27ad4db9a6df101e server_cookie=96cab2bad00fc146 in_seq=0 out_seq=0 2024-04-25T00:13:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d02000640 1 -- 172.21.15.83:0/1534159011 <== mds.0 v2:172.21.15.83:6835/2879143105 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a16b92ab0 con 0x558a16de2650 2024-04-25T00:13:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d02000640 10 client.6738 handle_command_reply: tid=0 2024-04-25T00:13:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 client.6738 shutdown 2024-04-25T00:13:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 -- 172.21.15.83:0/1534159011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db43b0 msgr2=0x558a16de1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 --2- 172.21.15.83:0/1534159011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db43b0 0x558a16de1c40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4d140984f0 tx=0x7f4d14009810 comp rx=0 tx=0).stop 2024-04-25T00:13:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 -- 172.21.15.83:0/1534159011 shutdown_connections 2024-04-25T00:13:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 --2- 172.21.15.83:0/1534159011 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x558a16de2650 0x558a16de2a30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 --2- 172.21.15.83:0/1534159011 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db3a90 0x558a16de1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d02000640 0 client.6738 ms_handle_reset on v2:172.21.15.83:6835/2879143105 2024-04-25T00:13:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 --2- 172.21.15.83:0/1534159011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db43b0 0x558a16de1c40 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 --2- 172.21.15.83:0/1534159011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 0x558a16de0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 -- 172.21.15.83:0/1534159011 >> 172.21.15.83:0/1534159011 conn(0x558a16daed00 msgr2=0x558a16dae320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 -- 172.21.15.83:0/1534159011 shutdown_connections 2024-04-25T00:13:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 1 -- 172.21.15.83:0/1534159011 wait complete. 2024-04-25T00:13:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.712+0000 7f4d270e2000 20 client.6738 trim_cache size 0 max 16384 2024-04-25T00:13:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d270e2000 1 Processor -- start 2024-04-25T00:13:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d270e2000 1 -- start start 2024-04-25T00:13:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d270e2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 0x558a16dac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d270e2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db3a90 0x558a16dacbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d270e2000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db43b0 0x558a16e02550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d270e2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1ff4f0 con 0x558a16db3a90 2024-04-25T00:13:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d270e2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c0b3b00 con 0x558a16dad6b0 2024-04-25T00:13:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d270e2000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4d1c1641b0 con 0x558a16db43b0 2024-04-25T00:13:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db3a90 0x558a16dacbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db3a90 0x558a16dacbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45228/0 (socket says 172.21.15.83:45228) 2024-04-25T00:13:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d1be00640 1 -- 172.21.15.83:0/582905649 learned_addr learned my addr 172.21.15.83:0/582905649 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d21c00640 1 --2- 172.21.15.83:0/582905649 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db43b0 0x558a16e02550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d21200640 1 --2- 172.21.15.83:0/582905649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 0x558a16dac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d1be00640 1 -- 172.21.15.83:0/582905649 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db43b0 msgr2=0x558a16e02550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d1be00640 1 --2- 172.21.15.83:0/582905649 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db43b0 0x558a16e02550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d1be00640 1 -- 172.21.15.83:0/582905649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 msgr2=0x558a16dac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d1be00640 1 --2- 172.21.15.83:0/582905649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 0x558a16dac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d1be00640 1 -- 172.21.15.83:0/582905649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a16b7fe80 con 0x558a16db3a90 2024-04-25T00:13:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d21200640 1 --2- 172.21.15.83:0/582905649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 0x558a16dac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d1be00640 1 --2- 172.21.15.83:0/582905649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db3a90 0x558a16dacbe0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009ac0 tx=0x7f4d08009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51b9e165be4b180 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d0cc00640 1 -- 172.21.15.83:0/582905649 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0800c070 con 0x558a16db3a90 2024-04-25T00:13:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d0cc00640 1 -- 172.21.15.83:0/582905649 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d08017d80 con 0x558a16db3a90 2024-04-25T00:13:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.732+0000 7f4d0cc00640 1 -- 172.21.15.83:0/582905649 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0800f040 con 0x558a16db3a90 2024-04-25T00:13:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 -- 172.21.15.83:0/582905649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db3a90 msgr2=0x558a16dacbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 --2- 172.21.15.83:0/582905649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db3a90 0x558a16dacbe0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009ac0 tx=0x7f4d08009f80 comp rx=0 tx=0).stop 2024-04-25T00:13:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 -- 172.21.15.83:0/582905649 shutdown_connections 2024-04-25T00:13:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 --2- 172.21.15.83:0/582905649 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db43b0 0x558a16e02550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 --2- 172.21.15.83:0/582905649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16db3a90 0x558a16dacbe0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 --2- 172.21.15.83:0/582905649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16dad6b0 0x558a16dac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 -- 172.21.15.83:0/582905649 >> 172.21.15.83:0/582905649 conn(0x558a16daed00 msgr2=0x558a16dabf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 -- 172.21.15.83:0/582905649 shutdown_connections 2024-04-25T00:13:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 -- 172.21.15.83:0/582905649 wait complete. 2024-04-25T00:13:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 -- start start 2024-04-25T00:13:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 Processor -- start 2024-04-25T00:13:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16dad6b0 0x558a16e66770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16db3a90 0x558a16e66cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db43b0 0x558a16e09880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558a16bb1520 con 0x558a16dad6b0 2024-04-25T00:13:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c164800 con 0x558a16db3a90 2024-04-25T00:13:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x558a16bf4570 con 0x558a16db43b0 2024-04-25T00:13:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 10 client.0 _ll_register_callbacks cb 0x7f4d10017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:13:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d21200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16dad6b0 0x558a16e66770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d21200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16dad6b0 0x558a16e66770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45240/0 (socket says 172.21.15.83:45240) 2024-04-25T00:13:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d21200640 1 -- 172.21.15.83:0/3076727082 learned_addr learned my addr 172.21.15.83:0/3076727082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d1be00640 1 --2- 172.21.15.83:0/3076727082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16db3a90 0x558a16e66cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d21c00640 1 --2- 172.21.15.83:0/3076727082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db43b0 0x558a16e09880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d21200640 1 -- 172.21.15.83:0/3076727082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db43b0 msgr2=0x558a16e09880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d21200640 1 --2- 172.21.15.83:0/3076727082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db43b0 0x558a16e09880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d21200640 1 -- 172.21.15.83:0/3076727082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16db3a90 msgr2=0x558a16e66cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d21200640 1 --2- 172.21.15.83:0/3076727082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16db3a90 0x558a16e66cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d21200640 1 -- 172.21.15.83:0/3076727082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a16bc4760 con 0x558a16dad6b0 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d21200640 1 --2- 172.21.15.83:0/3076727082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16dad6b0 0x558a16e66770 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4d100013a0 tx=0x7f4d10001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a493a4af207386 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d02000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d02000640 1 -- 172.21.15.83:0/3076727082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1002e070 con 0x558a16dad6b0 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d02000640 1 -- 172.21.15.83:0/3076727082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f4d1002ad60 con 0x558a16dad6b0 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 -- 172.21.15.83:0/3076727082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d08017d80 con 0x558a16dad6b0 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d02000640 1 -- 172.21.15.83:0/3076727082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4d10039050 con 0x558a16dad6b0 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 10 client.6744 fetch_fsmap learned FSMap version 44 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 -- 172.21.15.83:0/3076727082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x558a16c509e0 con 0x558a16dad6b0 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d02000640 1 -- 172.21.15.83:0/3076727082 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d08017d80 con 0x558a16dad6b0 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d02000640 1 -- 172.21.15.83:0/3076727082 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f4d1003c020 con 0x558a16dad6b0 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 10 client.6744 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 10 client.6744 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 20 client.6744 populate_metadata read hostname 'smithi083' 2024-04-25T00:13:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 --2- 172.21.15.83:0/3076727082 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x558a16e6d640 0x558a16e0a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 4 client.6744 mds_command: new command op to 6591 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:13:58.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.736+0000 7f4d270e2000 1 -- 172.21.15.83:0/3076727082 --> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] -- command(tid 0: {"prefix": "status"}) -- 0x558a16c04b20 con 0x558a16e6d640 2024-04-25T00:13:58.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d1be00640 1 --2- 172.21.15.83:0/3076727082 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x558a16e6d640 0x558a16e0a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d1be00640 1 --2- 172.21.15.83:0/3076727082 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x558a16e6d640 0x558a16e0a500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab063ff1a289cced server_cookie=2a0768f470da2296 in_seq=0 out_seq=0 2024-04-25T00:13:58.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d02000640 10 client.6744 ms_handle_connect on v2:172.21.15.83:6835/2879143105 2024-04-25T00:13:58.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d02000640 1 -- 172.21.15.83:0/3076727082 <== mds.0 v2:172.21.15.83:6835/2879143105 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x7f4d08026150 con 0x558a16e6d640 2024-04-25T00:13:58.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d02000640 10 client.6744 handle_command_reply: tid=0 2024-04-25T00:13:58.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d270e2000 1 client.6744 shutdown 2024-04-25T00:13:58.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d270e2000 1 -- 172.21.15.83:0/3076727082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16dad6b0 msgr2=0x558a16e66770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d270e2000 1 --2- 172.21.15.83:0/3076727082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16dad6b0 0x558a16e66770 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4d100013a0 tx=0x7f4d10001130 comp rx=0 tx=0).stop 2024-04-25T00:13:58.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d270e2000 1 -- 172.21.15.83:0/3076727082 shutdown_connections 2024-04-25T00:13:58.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d270e2000 1 --2- 172.21.15.83:0/3076727082 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x558a16e6d640 0x558a16e0a500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d270e2000 1 --2- 172.21.15.83:0/3076727082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558a16db43b0 0x558a16e09880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d270e2000 1 --2- 172.21.15.83:0/3076727082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a16db3a90 0x558a16e66cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d02000640 0 client.6744 ms_handle_reset on v2:172.21.15.83:6835/2879143105 2024-04-25T00:13:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d270e2000 1 --2- 172.21.15.83:0/3076727082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a16dad6b0 0x558a16e66770 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d270e2000 1 -- 172.21.15.83:0/3076727082 >> 172.21.15.83:0/3076727082 conn(0x558a16daed00 msgr2=0x558a16db5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d270e2000 1 -- 172.21.15.83:0/3076727082 shutdown_connections 2024-04-25T00:13:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d270e2000 1 -- 172.21.15.83:0/3076727082 wait complete. 2024-04-25T00:13:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d270e2000 20 client.6744 trim_cache size 0 max 16384 2024-04-25T00:13:58.747 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:13:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "d58ae83a-ad72-4e18-824a-86e779979c35", 2024-04-25T00:13:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-25T00:13:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6591, 2024-04-25T00:13:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T00:13:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T00:13:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T00:13:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.8453400660000003, 2024-04-25T00:13:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 44, 2024-04-25T00:13:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 71, 2024-04-25T00:13:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 71, 2024-04-25T00:13:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 10.71380196 2024-04-25T00:13:58.748 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:13:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d23c00640 1 -- 172.21.15.83:0/1119249840 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4cfc0678c0 msgr2=0x7f4cfc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d23c00640 1 --2- 172.21.15.83:0/1119249840 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4cfc0678c0 0x7f4cfc069d80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4d10007fd0 tx=0x7f4d10009040 comp rx=0 tx=0).stop 2024-04-25T00:13:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d23c00640 1 -- 172.21.15.83:0/1119249840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c15f350 msgr2=0x7f4d1c141a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d23c00640 1 --2- 172.21.15.83:0/1119249840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c15f350 0x7f4d1c141a00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4d140af040 tx=0x7f4d14019790 comp rx=0 tx=0).stop 2024-04-25T00:13:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d23c00640 1 -- 172.21.15.83:0/1119249840 shutdown_connections 2024-04-25T00:13:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d23c00640 1 --2- 172.21.15.83:0/1119249840 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4cfc0678c0 0x7f4cfc069d80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d23c00640 1 --2- 172.21.15.83:0/1119249840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d1c1446e0 0x7f4d1c1414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d23c00640 1 --2- 172.21.15.83:0/1119249840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c15f350 0x7f4d1c141a00 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d23c00640 1 --2- 172.21.15.83:0/1119249840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d1c06ee20 0x7f4d1c15e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d23c00640 1 -- 172.21.15.83:0/1119249840 >> 172.21.15.83:0/1119249840 conn(0x7f4d1c0cfa60 msgr2=0x7f4d1c0ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d23c00640 1 -- 172.21.15.83:0/1119249840 shutdown_connections 2024-04-25T00:13:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.740+0000 7f4d23c00640 1 -- 172.21.15.83:0/1119249840 wait complete. 2024-04-25T00:13:58.761 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:13:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb200e00640 1 Processor -- start 2024-04-25T00:13:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb200e00640 1 -- start start 2024-04-25T00:13:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb200e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb1fc15e4f0 0x7fb1fc0ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb200e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1fc06fdc0 0x7fb1fc0ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb200e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1fc0d0ad0 con 0x7fb1fc15e110 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb200e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb1fc0d0db0 con 0x7fb1fc15e4f0 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb200e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb1fc0cc3b0 con 0x7fb1fc06fdc0 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb1fbe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1fc06fdc0 0x7fb1fc0ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb1fbe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1fc06fdc0 0x7fb1fc0ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:45254/0 (socket says 172.21.15.83:45254) 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb1fbe00640 1 -- 172.21.15.83:0/3790513962 learned_addr learned my addr 172.21.15.83:0/3790513962 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb1faa00640 1 --2- 172.21.15.83:0/3790513962 >> v2:172.21.15.86:3300/0 conn(0x7fb1fc15e4f0 0x7fb1fc0ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb1fbe00640 1 -- 172.21.15.83:0/3790513962 >> v2:172.21.15.86:3300/0 conn(0x7fb1fc15e4f0 msgr2=0x7fb1fc0ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb1fbe00640 1 --2- 172.21.15.83:0/3790513962 >> v2:172.21.15.86:3300/0 conn(0x7fb1fc15e4f0 0x7fb1fc0ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb1fa000640 1 -- 172.21.15.83:0/3790513962 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 40530584 0 0) 0x7fb1fc0d0ad0 con 0x7fb1fc15e110 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb1fbe00640 1 -- 172.21.15.83:0/3790513962 >> v1:172.21.15.86:6789/0 conn(0x7fb1fc15e110 legacy=0x7fb1fc0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb1fbe00640 1 -- 172.21.15.83:0/3790513962 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc0d0760 con 0x7fb1fc06fdc0 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.896+0000 7fb1faa00640 1 --2- 172.21.15.83:0/3790513962 >> v2:172.21.15.86:3300/0 conn(0x7fb1fc15e4f0 0x7fb1fc0ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.900+0000 7fb1fbe00640 1 --2- 172.21.15.83:0/3790513962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1fc06fdc0 0x7fb1fc0ccbb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009e50 tx=0x7fb1f401c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c392467062570578 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.900+0000 7fb1fa000640 1 -- 172.21.15.83:0/3790513962 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f40a5660 con 0x7fb1fc06fdc0 2024-04-25T00:13:58.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.900+0000 7fb1fa000640 1 -- 172.21.15.83:0/3790513962 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f40a5800 con 0x7fb1fc06fdc0 2024-04-25T00:13:58.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.900+0000 7fb1fa000640 1 -- 172.21.15.83:0/3790513962 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f40a5bd0 con 0x7fb1fc06fdc0 2024-04-25T00:13:58.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.900+0000 7fb200e00640 1 -- 172.21.15.83:0/3790513962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1fc06fdc0 msgr2=0x7fb1fc0ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.900+0000 7fb200e00640 1 --2- 172.21.15.83:0/3790513962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1fc06fdc0 0x7fb1fc0ccbb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009e50 tx=0x7fb1f401c9a0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.900+0000 7fb200e00640 1 -- 172.21.15.83:0/3790513962 shutdown_connections 2024-04-25T00:13:58.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.900+0000 7fb200e00640 1 --2- 172.21.15.83:0/3790513962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1fc06fdc0 0x7fb1fc0ccbb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.900+0000 7fb200e00640 1 --2- 172.21.15.83:0/3790513962 >> v2:172.21.15.86:3300/0 conn(0x7fb1fc15e4f0 0x7fb1fc0ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.900+0000 7fb200e00640 1 -- 172.21.15.83:0/3790513962 >> 172.21.15.83:0/3790513962 conn(0x7fb1fc0d1bc0 msgr2=0x7fb1fc0d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:58.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.900+0000 7fb200e00640 1 -- 172.21.15.83:0/3790513962 shutdown_connections 2024-04-25T00:13:58.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.900+0000 7fb200e00640 1 -- 172.21.15.83:0/3790513962 wait complete. 2024-04-25T00:13:58.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb200e00640 1 Processor -- start 2024-04-25T00:13:58.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb200e00640 1 -- start start 2024-04-25T00:13:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb200e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1fc06fdc0 0x7fb1fc151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb200e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1fc15e110 0x7fb1fc0c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb200e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1fc15e4f0 0x7fb1fc151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb200e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc0cdcd0 con 0x7fb1fc15e4f0 2024-04-25T00:13:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb200e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc0cdb50 con 0x7fb1fc06fdc0 2024-04-25T00:13:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb200e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb1fc0cde50 con 0x7fb1fc15e110 2024-04-25T00:13:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1fc15e4f0 0x7fb1fc151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1faa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1fc15e110 0x7fb1fc0c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1fc15e4f0 0x7fb1fc151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45248/0 (socket says 172.21.15.83:45248) 2024-04-25T00:13:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1fbe00640 1 -- 172.21.15.83:0/494186227 learned_addr learned my addr 172.21.15.83:0/494186227 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:13:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1fb400640 1 --2- 172.21.15.83:0/494186227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1fc06fdc0 0x7fb1fc151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1fbe00640 1 -- 172.21.15.83:0/494186227 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1fc15e110 msgr2=0x7fb1fc0c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1fbe00640 1 --2- 172.21.15.83:0/494186227 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1fc15e110 0x7fb1fc0c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1fbe00640 1 -- 172.21.15.83:0/494186227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1fc06fdc0 msgr2=0x7fb1fc151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1fbe00640 1 --2- 172.21.15.83:0/494186227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1fc06fdc0 0x7fb1fc151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1fbe00640 1 -- 172.21.15.83:0/494186227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc153750 con 0x7fb1fc15e4f0 2024-04-25T00:13:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1fb400640 1 --2- 172.21.15.83:0/494186227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1fc06fdc0 0x7fb1fc151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1fbe00640 1 --2- 172.21.15.83:0/494186227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1fc15e4f0 0x7fb1fc151f50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009e50 tx=0x7fb1f4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1f3400640 1 -- 172.21.15.83:0/494186227 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f40bc020 con 0x7fb1fc15e4f0 2024-04-25T00:13:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1f3400640 1 -- 172.21.15.83:0/494186227 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f40b8880 con 0x7fb1fc15e4f0 2024-04-25T00:13:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb1f3400640 1 -- 172.21.15.83:0/494186227 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f40a5660 con 0x7fb1fc15e4f0 2024-04-25T00:13:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb200e00640 1 -- 172.21.15.83:0/494186227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1fc0ca390 con 0x7fb1fc15e4f0 2024-04-25T00:13:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb200e00640 1 -- 172.21.15.83:0/494186227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1fc0ca950 con 0x7fb1fc15e4f0 2024-04-25T00:13:58.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.904+0000 7fb200e00640 1 -- 172.21.15.83:0/494186227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1fc04f510 con 0x7fb1fc15e4f0 2024-04-25T00:13:58.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.908+0000 7fb1f3400640 1 -- 172.21.15.83:0/494186227 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb1f40bf020 con 0x7fb1fc15e4f0 2024-04-25T00:13:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.908+0000 7fb1f3400640 1 --2- 172.21.15.83:0/494186227 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb1dc0678c0 0x7fb1dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.908+0000 7fb1f3400640 1 -- 172.21.15.83:0/494186227 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7fb1f4120f80 con 0x7fb1fc15e4f0 2024-04-25T00:13:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.908+0000 7fb1fb400640 1 --2- 172.21.15.83:0/494186227 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb1dc0678c0 0x7fb1dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.908+0000 7fb1fb400640 1 --2- 172.21.15.83:0/494186227 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb1dc0678c0 0x7fb1dc069d80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec000f70 tx=0x7fb1ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:58.912+0000 7fb1f3400640 1 -- 172.21.15.83:0/494186227 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb1f40c6030 con 0x7fb1fc15e4f0 2024-04-25T00:13:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.108+0000 7fb200e00640 1 -- 172.21.15.83:0/494186227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb1fc0c8750 con 0x7fb1fc15e4f0 2024-04-25T00:13:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.108+0000 7fb1f3400640 1 -- 172.21.15.83:0/494186227 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) ==== 53+0+73 (secure 0 0 0) 0x7fb1f4016620 con 0x7fb1fc15e4f0 2024-04-25T00:13:59.111 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:13:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.108+0000 7fb200e00640 1 -- 172.21.15.83:0/494186227 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb1dc0678c0 msgr2=0x7fb1dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.108+0000 7fb200e00640 1 --2- 172.21.15.83:0/494186227 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb1dc0678c0 0x7fb1dc069d80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec000f70 tx=0x7fb1ec009040 comp rx=0 tx=0).stop 2024-04-25T00:13:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.108+0000 7fb200e00640 1 -- 172.21.15.83:0/494186227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1fc15e4f0 msgr2=0x7fb1fc151f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.108+0000 7fb200e00640 1 --2- 172.21.15.83:0/494186227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1fc15e4f0 0x7fb1fc151f50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009e50 tx=0x7fb1f4019740 comp rx=0 tx=0).stop 2024-04-25T00:13:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.108+0000 7fb200e00640 1 -- 172.21.15.83:0/494186227 shutdown_connections 2024-04-25T00:13:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.108+0000 7fb200e00640 1 --2- 172.21.15.83:0/494186227 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb1dc0678c0 0x7fb1dc069d80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.108+0000 7fb200e00640 1 --2- 172.21.15.83:0/494186227 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1fc15e110 0x7fb1fc0c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.108+0000 7fb200e00640 1 --2- 172.21.15.83:0/494186227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1fc15e4f0 0x7fb1fc151f50 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.108+0000 7fb200e00640 1 --2- 172.21.15.83:0/494186227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1fc06fdc0 0x7fb1fc151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.108+0000 7fb200e00640 1 -- 172.21.15.83:0/494186227 >> 172.21.15.83:0/494186227 conn(0x7fb1fc0d1bc0 msgr2=0x7fb1fc06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.108+0000 7fb200e00640 1 -- 172.21.15.83:0/494186227 shutdown_connections 2024-04-25T00:13:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.108+0000 7fb200e00640 1 -- 172.21.15.83:0/494186227 wait complete. 2024-04-25T00:13:59.132 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:13:59.132 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:13:59.132 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-25T00:13:59.132 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:13:59.132 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:13:59.132 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:13:59.132 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:13:59.132 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:13:59.132 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:13:59.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:59.133 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-25T00:13:59.142 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:13:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:13:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:13:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-25T00:13:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:59.143 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:13:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-25T00:13:59.143 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:13:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-25T00:13:59.143 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:13:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:59.143 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:13:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:13:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42295sec preferred_lft 42295sec 2024-04-25T00:13:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-25T00:13:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:59.144 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:13:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:8d:d4:7d:b9 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:13:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:59.144 INFO:teuthology.orchestra.run.smithi083.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:13:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:13:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:13:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:13:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:59.144 INFO:teuthology.orchestra.run.smithi083.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:13:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:13:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:13:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:59.145 INFO:teuthology.orchestra.run.smithi083.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:13:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:13:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:13:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:13:59.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:59.146 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:13:59.202 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-25T00:13:59.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:59.204 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-25T00:13:59.256 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-25T00:13:59.256 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:13:59.257 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:13:59.257 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:13:59.257 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:13:59.257 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:13:59.257 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:13:59.257 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:13:59.257 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:13:59.257 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:13:59.257 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:13:59.257 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:13:59.258 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:13:59.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:13:59.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:13:59.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:13:59.258 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:13:59.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:13:59.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:13:59.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:59.390 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:13:59.398 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:13:59.399 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:13:59.399 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:13:59.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:59.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-25T00:13:59.460 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:13:59.461 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:13:59.509 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:13:59.510 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:13:59.510 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:13:59.560 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T00:13:59.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:59.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:13:59.648 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:13:59.649 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:13:59.662 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:13:59.662 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:13:59.662 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:13:59.662 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:13:59.662 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:13:59.666 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:13:59.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:13:59.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:13:59.686 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:13:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:13:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:13:59.791 INFO:teuthology.orchestra.run.smithi083.stdout:client5846 2024-04-25T00:13:59.791 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:13:59.791 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:13:59.800 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:13:59.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e3d800640 1 Processor -- start 2024-04-25T00:14:00.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e3d800640 1 -- start start 2024-04-25T00:14:00.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e3d800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e38071dc0 0x7f3e3806d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e3d800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e3806e7b0 0x7f3e380ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e3d800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e3806e0c0 con 0x7f3e3815df00 2024-04-25T00:14:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e3d800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3e3806f150 con 0x7f3e38071dc0 2024-04-25T00:14:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e3d800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3e3806e3a0 con 0x7f3e3806e7b0 2024-04-25T00:14:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e36000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e38071dc0 0x7f3e3806d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e36000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e38071dc0 0x7f3e3806d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45264/0 (socket says 172.21.15.83:45264) 2024-04-25T00:14:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e36000640 1 -- 172.21.15.83:0/3686344963 learned_addr learned my addr 172.21.15.83:0/3686344963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e37400640 1 --2- 172.21.15.83:0/3686344963 >> v2:172.21.15.86:3300/0 conn(0x7f3e3806e7b0 0x7f3e380ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e36000640 1 -- 172.21.15.83:0/3686344963 >> v2:172.21.15.86:3300/0 conn(0x7f3e3806e7b0 msgr2=0x7f3e380ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e36000640 1 --2- 172.21.15.83:0/3686344963 >> v2:172.21.15.86:3300/0 conn(0x7f3e3806e7b0 0x7f3e380ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e36000640 1 -- 172.21.15.83:0/3686344963 >> v1:172.21.15.86:6789/0 conn(0x7f3e3815df00 legacy=0x7f3e380716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.976+0000 7f3e36000640 1 -- 172.21.15.83:0/3686344963 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e380ad990 con 0x7f3e38071dc0 2024-04-25T00:14:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.980+0000 7f3e36000640 1 --2- 172.21.15.83:0/3686344963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e38071dc0 0x7f3e3806d970 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3e3806c860 tx=0x7f3e2401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9267586a34202e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.980+0000 7f3e35600640 1 -- 172.21.15.83:0/3686344963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e240b0070 con 0x7f3e38071dc0 2024-04-25T00:14:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.980+0000 7f3e35600640 1 -- 172.21.15.83:0/3686344963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e240ab600 con 0x7f3e38071dc0 2024-04-25T00:14:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.980+0000 7f3e35600640 1 -- 172.21.15.83:0/3686344963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e240ab9d0 con 0x7f3e38071dc0 2024-04-25T00:14:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.980+0000 7f3e3d800640 1 -- 172.21.15.83:0/3686344963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e38071dc0 msgr2=0x7f3e3806d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.980+0000 7f3e3d800640 1 --2- 172.21.15.83:0/3686344963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e38071dc0 0x7f3e3806d970 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3e3806c860 tx=0x7f3e2401c400 comp rx=0 tx=0).stop 2024-04-25T00:14:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.980+0000 7f3e3d800640 1 -- 172.21.15.83:0/3686344963 shutdown_connections 2024-04-25T00:14:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.980+0000 7f3e3d800640 1 --2- 172.21.15.83:0/3686344963 >> v2:172.21.15.86:3300/0 conn(0x7f3e3806e7b0 0x7f3e380ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.980+0000 7f3e3d800640 1 --2- 172.21.15.83:0/3686344963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e38071dc0 0x7f3e3806d970 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.980+0000 7f3e3d800640 1 -- 172.21.15.83:0/3686344963 >> 172.21.15.83:0/3686344963 conn(0x7f3e380b1830 msgr2=0x7f3e380b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.980+0000 7f3e3d800640 1 -- 172.21.15.83:0/3686344963 shutdown_connections 2024-04-25T00:14:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.980+0000 7f3e3d800640 1 -- 172.21.15.83:0/3686344963 wait complete. 2024-04-25T00:14:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e3d800640 1 Processor -- start 2024-04-25T00:14:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e3d800640 1 -- start start 2024-04-25T00:14:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e3d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3806e7b0 0x7f3e3814fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e3d800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e38071dc0 0x7f3e38151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e3d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e3815df00 0x7f3e3815e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e3d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e380b32a0 con 0x7f3e3806e7b0 2024-04-25T00:14:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e3d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e380b3120 con 0x7f3e3815df00 2024-04-25T00:14:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e3d800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3816ae80 con 0x7f3e38071dc0 2024-04-25T00:14:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e36a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3806e7b0 0x7f3e3814fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e36a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3806e7b0 0x7f3e3814fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45266/0 (socket says 172.21.15.83:45266) 2024-04-25T00:14:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e36a00640 1 -- 172.21.15.83:0/2053039276 learned_addr learned my addr 172.21.15.83:0/2053039276 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e36000640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e38071dc0 0x7f3e38151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e37400640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e3815df00 0x7f3e3815e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e36a00640 1 -- 172.21.15.83:0/2053039276 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e38071dc0 msgr2=0x7f3e38151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e36a00640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e38071dc0 0x7f3e38151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e36a00640 1 -- 172.21.15.83:0/2053039276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e3815df00 msgr2=0x7f3e3815e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e36a00640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e3815df00 0x7f3e3815e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e36a00640 1 -- 172.21.15.83:0/2053039276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e380b1660 con 0x7f3e3806e7b0 2024-04-25T00:14:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e36000640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e38071dc0 0x7f3e38151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e36a00640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3806e7b0 0x7f3e3814fe80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800c3a0 tx=0x7f3e2800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e2ea00640 1 -- 172.21.15.83:0/2053039276 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28017070 con 0x7f3e3806e7b0 2024-04-25T00:14:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e2ea00640 1 -- 172.21.15.83:0/2053039276 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e280052f0 con 0x7f3e3806e7b0 2024-04-25T00:14:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e2ea00640 1 -- 172.21.15.83:0/2053039276 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28003860 con 0x7f3e3806e7b0 2024-04-25T00:14:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e3d800640 1 -- 172.21.15.83:0/2053039276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3814f0c0 con 0x7f3e3806e7b0 2024-04-25T00:14:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.984+0000 7f3e3d800640 1 -- 172.21.15.83:0/2053039276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3814f960 con 0x7f3e3806e7b0 2024-04-25T00:14:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.988+0000 7f3e3d800640 1 -- 172.21.15.83:0/2053039276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e3814f290 con 0x7f3e3806e7b0 2024-04-25T00:14:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.988+0000 7f3e2ea00640 1 -- 172.21.15.83:0/2053039276 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3e2801a070 con 0x7f3e3806e7b0 2024-04-25T00:14:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.988+0000 7f3e2ea00640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e180678f0 0x7f3e18069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.988+0000 7f3e2ea00640 1 -- 172.21.15.83:0/2053039276 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f3e28085170 con 0x7f3e3806e7b0 2024-04-25T00:14:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.988+0000 7f3e36000640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e180678f0 0x7f3e18069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.988+0000 7f3e36000640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e180678f0 0x7f3e18069db0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3e240ad000 tx=0x7f3e24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:00.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:13:59.992+0000 7f3e2ea00640 1 -- 172.21.15.83:0/2053039276 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3e28055b00 con 0x7f3e3806e7b0 2024-04-25T00:14:00.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:00.180+0000 7f3e3d800640 1 -- 172.21.15.83:0/2053039276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3e380737f0 con 0x7f3e3806e7b0 2024-04-25T00:14:00.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:00.184+0000 7f3e2ea00640 1 -- 172.21.15.83:0/2053039276 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) ==== 53+0+73 (secure 0 0 0) 0x7f3e28055400 con 0x7f3e3806e7b0 2024-04-25T00:14:00.186 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:14:00.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:00.184+0000 7f3e3d800640 1 -- 172.21.15.83:0/2053039276 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e180678f0 msgr2=0x7f3e18069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:00.184+0000 7f3e3d800640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e180678f0 0x7f3e18069db0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3e240ad000 tx=0x7f3e24008040 comp rx=0 tx=0).stop 2024-04-25T00:14:00.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:00.184+0000 7f3e3d800640 1 -- 172.21.15.83:0/2053039276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3806e7b0 msgr2=0x7f3e3814fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:00.184+0000 7f3e3d800640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3806e7b0 0x7f3e3814fe80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800c3a0 tx=0x7f3e2800c860 comp rx=0 tx=0).stop 2024-04-25T00:14:00.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:00.184+0000 7f3e3d800640 1 -- 172.21.15.83:0/2053039276 shutdown_connections 2024-04-25T00:14:00.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:00.184+0000 7f3e3d800640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e180678f0 0x7f3e18069db0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:00.184+0000 7f3e3d800640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e38071dc0 0x7f3e38151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:00.184+0000 7f3e3d800640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e3815df00 0x7f3e3815e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:00.184+0000 7f3e3d800640 1 --2- 172.21.15.83:0/2053039276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3806e7b0 0x7f3e3814fe80 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:00.184+0000 7f3e3d800640 1 -- 172.21.15.83:0/2053039276 >> 172.21.15.83:0/2053039276 conn(0x7f3e380b1830 msgr2=0x7f3e3815bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:00.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:00.184+0000 7f3e3d800640 1 -- 172.21.15.83:0/2053039276 shutdown_connections 2024-04-25T00:14:00.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:00.184+0000 7f3e3d800640 1 -- 172.21.15.83:0/2053039276 wait complete. 2024-04-25T00:14:00.206 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:14:00.206 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:14:00.206 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T00:14:00.206 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:14:00.206 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:14:00.206 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:14:00.207 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:14:00.207 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:14:00.207 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:14:00.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:00.207 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T00:14:00.218 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:14:00.218 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:14:00.218 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:14:00.219 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:00.219 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T00:14:00.219 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:00.219 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:00.219 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:00.219 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T00:14:00.219 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:00.219 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:00.219 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T00:14:00.219 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:00.219 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:00.219 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:14:00.219 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:00.219 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:14:00.220 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42288sec preferred_lft 42288sec 2024-04-25T00:14:00.220 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T00:14:00.220 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:00.220 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:14:00.220 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:35:63:ea:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:00.220 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:14:00.220 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:00.220 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:14:00.220 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:00.220 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:14:00.220 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:00.220 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:14:00.220 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:00.221 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:14:00.221 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:14:00.221 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:14:00.221 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:00.221 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:14:00.221 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:14:00.221 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:14:00.221 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:00.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:00.221 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:14:00.281 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T00:14:00.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:00.284 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T00:14:00.337 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T00:14:00.338 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:14:00.338 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:14:00.338 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:14:00.338 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:14:00.338 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:14:00.338 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:14:00.338 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:00.338 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:00.338 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:00.338 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:14:00.339 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:14:00.339 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:14:00.339 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:14:00.339 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:14:00.339 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:14:00.339 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:14:00.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:14:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:14:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:14:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:00.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:00.488 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T00:14:00.497 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:14:00.497 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:14:00.498 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:14:00.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:00.498 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T00:14:00.565 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:14:00.566 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:14:00.617 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:14:00.617 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:14:00.617 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:14:00.669 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T00:14:00.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:00.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:14:00.766 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:14:00.766 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:14:00.779 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:14:00.779 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:14:00.779 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:14:00.779 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with new device syntax: 1@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:14:00.779 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:14:00.783 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:14:00.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:00.783 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:14:00.802 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:14:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:14:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:14:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:14:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:00.900 INFO:teuthology.orchestra.run.smithi086.stdout:client6436 2024-04-25T00:14:00.900 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:14:00.901 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:00.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds ms_async_reap_threshold 1 2024-04-25T00:14:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70f6e00640 1 Processor -- start 2024-04-25T00:14:01.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70f6e00640 1 -- start start 2024-04-25T00:14:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70f6e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70f00cf4b0 0x7f70f0163400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70f6e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f70f00712b0 0x7f70f006cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70f6e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70f0163940 con 0x7f70f006d060 2024-04-25T00:14:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70f6e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f70f006f100 con 0x7f70f00712b0 2024-04-25T00:14:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70f6e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f70f00cf890 con 0x7f70f00cf4b0 2024-04-25T00:14:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70ef400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f70f00712b0 0x7f70f006cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70f4e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f70f006d060 0x7f70f0165f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33174/0 (socket says 172.21.15.83:33174) 2024-04-25T00:14:01.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70f4e00640 1 -- 172.21.15.83:0/1366936861 learned_addr learned my addr 172.21.15.83:0/1366936861 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:01.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70efe00640 1 --2- 172.21.15.83:0/1366936861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70f00cf4b0 0x7f70f0163400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70ef400640 1 -- 172.21.15.83:0/1366936861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70f00cf4b0 msgr2=0x7f70f0163400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:01.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70ef400640 1 --2- 172.21.15.83:0/1366936861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70f00cf4b0 0x7f70f0163400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70ef400640 1 -- 172.21.15.83:0/1366936861 >> v1:172.21.15.83:6789/0 conn(0x7f70f006d060 legacy=0x7f70f0165f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70ef400640 1 -- 172.21.15.83:0/1366936861 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f0071770 con 0x7f70f00712b0 2024-04-25T00:14:01.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.056+0000 7f70eea00640 1 -- 172.21.15.83:0/1366936861 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1884393210 0 0) 0x7f70f0163940 con 0x7f70f006d060 2024-04-25T00:14:01.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.060+0000 7f70ef400640 1 --2- 172.21.15.83:0/1366936861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70f00712b0 0x7f70f006cb20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0097c0 tx=0x7f70dc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0ddf67d55789bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:01.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.060+0000 7f70eea00640 1 -- 172.21.15.83:0/1366936861 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0b0070 con 0x7f70f00712b0 2024-04-25T00:14:01.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.060+0000 7f70eea00640 1 -- 172.21.15.83:0/1366936861 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70dc0ab600 con 0x7f70f00712b0 2024-04-25T00:14:01.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.060+0000 7f70eea00640 1 -- 172.21.15.83:0/1366936861 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0ab9d0 con 0x7f70f00712b0 2024-04-25T00:14:01.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.060+0000 7f70f6e00640 1 -- 172.21.15.83:0/1366936861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70f00712b0 msgr2=0x7f70f006cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.060+0000 7f70f6e00640 1 --2- 172.21.15.83:0/1366936861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70f00712b0 0x7f70f006cb20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0097c0 tx=0x7f70dc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.060+0000 7f70f6e00640 1 -- 172.21.15.83:0/1366936861 shutdown_connections 2024-04-25T00:14:01.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.060+0000 7f70f6e00640 1 --2- 172.21.15.83:0/1366936861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70f00712b0 0x7f70f006cb20 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.060+0000 7f70f6e00640 1 --2- 172.21.15.83:0/1366936861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70f00cf4b0 0x7f70f0163400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.060+0000 7f70f6e00640 1 -- 172.21.15.83:0/1366936861 >> 172.21.15.83:0/1366936861 conn(0x7f70f00b1830 msgr2=0x7f70f00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:01.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.060+0000 7f70f6e00640 1 -- 172.21.15.83:0/1366936861 shutdown_connections 2024-04-25T00:14:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.060+0000 7f70f6e00640 1 -- 172.21.15.83:0/1366936861 wait complete. 2024-04-25T00:14:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70f6e00640 1 Processor -- start 2024-04-25T00:14:01.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70f6e00640 1 -- start start 2024-04-25T00:14:01.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70f6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f006d060 0x7f70f014f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70f6e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70f00712b0 0x7f70f0146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70f6e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70f00cf4b0 0x7f70f014d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70f6e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f70f00b3b40 con 0x7f70f00712b0 2024-04-25T00:14:01.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70f6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70f00722f0 con 0x7f70f006d060 2024-04-25T00:14:01.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70f6e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f70f00b3cc0 con 0x7f70f00cf4b0 2024-04-25T00:14:01.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70f00712b0 0x7f70f0146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70f00712b0 0x7f70f0146d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45290/0 (socket says 172.21.15.83:45290) 2024-04-25T00:14:01.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70ef400640 1 -- 172.21.15.83:0/2167576333 learned_addr learned my addr 172.21.15.83:0/2167576333 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:01.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70efe00640 1 --2- 172.21.15.83:0/2167576333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f006d060 0x7f70f014f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70f4e00640 1 --2- 172.21.15.83:0/2167576333 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70f00cf4b0 0x7f70f014d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70ef400640 1 -- 172.21.15.83:0/2167576333 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70f00cf4b0 msgr2=0x7f70f014d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70ef400640 1 --2- 172.21.15.83:0/2167576333 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70f00cf4b0 0x7f70f014d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70ef400640 1 -- 172.21.15.83:0/2167576333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f006d060 msgr2=0x7f70f014f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70ef400640 1 --2- 172.21.15.83:0/2167576333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f006d060 0x7f70f014f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70ef400640 1 -- 172.21.15.83:0/2167576333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f0147830 con 0x7f70f00712b0 2024-04-25T00:14:01.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70ef400640 1 --2- 172.21.15.83:0/2167576333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70f00712b0 0x7f70f0146d60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f70dc01a0d0 tx=0x7f70dc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:01.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70e7e00640 1 -- 172.21.15.83:0/2167576333 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0b0040 con 0x7f70f00712b0 2024-04-25T00:14:01.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70e7e00640 1 -- 172.21.15.83:0/2167576333 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70dc0ba070 con 0x7f70f00712b0 2024-04-25T00:14:01.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70e7e00640 1 -- 172.21.15.83:0/2167576333 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0b4630 con 0x7f70f00712b0 2024-04-25T00:14:01.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70f6e00640 1 -- 172.21.15.83:0/2167576333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f0147b10 con 0x7f70f00712b0 2024-04-25T00:14:01.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70f6e00640 1 -- 172.21.15.83:0/2167576333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f01480d0 con 0x7f70f00712b0 2024-04-25T00:14:01.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.064+0000 7f70f6e00640 1 -- 172.21.15.83:0/2167576333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f004f510 con 0x7f70f00712b0 2024-04-25T00:14:01.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.068+0000 7f70e7e00640 1 -- 172.21.15.83:0/2167576333 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f70dc0b47d0 con 0x7f70f00712b0 2024-04-25T00:14:01.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.068+0000 7f70e7e00640 1 --2- 172.21.15.83:0/2167576333 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f70d00678c0 0x7f70d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.068+0000 7f70e7e00640 1 -- 172.21.15.83:0/2167576333 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f70dc11f9e0 con 0x7f70f00712b0 2024-04-25T00:14:01.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.068+0000 7f70efe00640 1 --2- 172.21.15.83:0/2167576333 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f70d00678c0 0x7f70d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.068+0000 7f70efe00640 1 --2- 172.21.15.83:0/2167576333 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f70d00678c0 0x7f70d0069d80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f70f01524d0 tx=0x7f70e0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:01.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.072+0000 7f70e7e00640 1 -- 172.21.15.83:0/2167576333 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f70dc016090 con 0x7f70f00712b0 2024-04-25T00:14:01.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.228+0000 7f70f6e00640 1 -- 172.21.15.83:0/2167576333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=ms_async_reap_threshold}] v 0) -- 0x7f70f0156640 con 0x7f70f00712b0 2024-04-25T00:14:01.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.228+0000 7f70e7e00640 1 -- 172.21.15.83:0/2167576333 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=ms_async_reap_threshold}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f70dc0efca0 con 0x7f70f00712b0 2024-04-25T00:14:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.232+0000 7f70f6e00640 1 -- 172.21.15.83:0/2167576333 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f70d00678c0 msgr2=0x7f70d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.232+0000 7f70f6e00640 1 --2- 172.21.15.83:0/2167576333 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f70d00678c0 0x7f70d0069d80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f70f01524d0 tx=0x7f70e0009190 comp rx=0 tx=0).stop 2024-04-25T00:14:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.232+0000 7f70f6e00640 1 -- 172.21.15.83:0/2167576333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70f00712b0 msgr2=0x7f70f0146d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.232+0000 7f70f6e00640 1 --2- 172.21.15.83:0/2167576333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70f00712b0 0x7f70f0146d60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f70dc01a0d0 tx=0x7f70dc019240 comp rx=0 tx=0).stop 2024-04-25T00:14:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.232+0000 7f70f6e00640 1 -- 172.21.15.83:0/2167576333 shutdown_connections 2024-04-25T00:14:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.232+0000 7f70f6e00640 1 --2- 172.21.15.83:0/2167576333 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f70d00678c0 0x7f70d0069d80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.232+0000 7f70f6e00640 1 --2- 172.21.15.83:0/2167576333 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70f00cf4b0 0x7f70f014d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.232+0000 7f70f6e00640 1 --2- 172.21.15.83:0/2167576333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70f00712b0 0x7f70f0146d60 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.232+0000 7f70f6e00640 1 --2- 172.21.15.83:0/2167576333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f006d060 0x7f70f014f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.232+0000 7f70f6e00640 1 -- 172.21.15.83:0/2167576333 >> 172.21.15.83:0/2167576333 conn(0x7f70f00b1830 msgr2=0x7f70f0165400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:01.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.232+0000 7f70f6e00640 1 -- 172.21.15.83:0/2167576333 shutdown_connections 2024-04-25T00:14:01.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.232+0000 7f70f6e00640 1 -- 172.21.15.83:0/2167576333 wait complete. 2024-04-25T00:14:01.254 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:14:01.254 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:01.263 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:14:01.263 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:01.317 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.492+0000 7f1970c00640 1 Processor -- start 2024-04-25T00:14:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.492+0000 7f1970c00640 1 -- start start 2024-04-25T00:14:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.492+0000 7f1970c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f196c15e6b0 0x7f196c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.492+0000 7f1970c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f196c0cf3c0 0x7f196c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.492+0000 7f1970c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f196c0b4c30 con 0x7f196c1581d0 2024-04-25T00:14:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.492+0000 7f1970c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f196c0b3440 con 0x7f196c0cf3c0 2024-04-25T00:14:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.492+0000 7f1970c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f196c0b3720 con 0x7f196c15e6b0 2024-04-25T00:14:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.492+0000 7f196a800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f196c0cf3c0 0x7f196c06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.496+0000 7f196a800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f196c0cf3c0 0x7f196c06d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45306/0 (socket says 172.21.15.83:45306) 2024-04-25T00:14:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.496+0000 7f196a800640 1 -- 172.21.15.83:0/3521840442 learned_addr learned my addr 172.21.15.83:0/3521840442 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.496+0000 7f1969400640 1 --2- 172.21.15.83:0/3521840442 >> v2:172.21.15.86:3300/0 conn(0x7f196c15e6b0 0x7f196c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.496+0000 7f196a800640 1 -- 172.21.15.83:0/3521840442 >> v2:172.21.15.86:3300/0 conn(0x7f196c15e6b0 msgr2=0x7f196c0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.496+0000 7f196a800640 1 --2- 172.21.15.83:0/3521840442 >> v2:172.21.15.86:3300/0 conn(0x7f196c15e6b0 0x7f196c0b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.496+0000 7f196a800640 1 -- 172.21.15.83:0/3521840442 >> v1:172.21.15.86:6789/0 conn(0x7f196c1581d0 legacy=0x7f196c15dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.496+0000 7f196a800640 1 -- 172.21.15.83:0/3521840442 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c0b3a00 con 0x7f196c0cf3c0 2024-04-25T00:14:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.496+0000 7f196a800640 1 --2- 172.21.15.83:0/3521840442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196c0cf3c0 0x7f196c06d3b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f196c0cc2a0 tx=0x7f196401daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cebd98e6c9d52f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.496+0000 7f1968a00640 1 -- 172.21.15.83:0/3521840442 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196400a2d0 con 0x7f196c0cf3c0 2024-04-25T00:14:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.496+0000 7f1968a00640 1 -- 172.21.15.83:0/3521840442 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19640a6970 con 0x7f196c0cf3c0 2024-04-25T00:14:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.496+0000 7f1968a00640 1 -- 172.21.15.83:0/3521840442 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19640a6d40 con 0x7f196c0cf3c0 2024-04-25T00:14:01.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.496+0000 7f1970c00640 1 -- 172.21.15.83:0/3521840442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196c0cf3c0 msgr2=0x7f196c06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.496+0000 7f1970c00640 1 --2- 172.21.15.83:0/3521840442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196c0cf3c0 0x7f196c06d3b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f196c0cc2a0 tx=0x7f196401daf0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1970c00640 1 -- 172.21.15.83:0/3521840442 shutdown_connections 2024-04-25T00:14:01.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1970c00640 1 --2- 172.21.15.83:0/3521840442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196c0cf3c0 0x7f196c06d3b0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1970c00640 1 --2- 172.21.15.83:0/3521840442 >> v2:172.21.15.86:3300/0 conn(0x7f196c15e6b0 0x7f196c0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1970c00640 1 -- 172.21.15.83:0/3521840442 >> 172.21.15.83:0/3521840442 conn(0x7f196c0b1830 msgr2=0x7f196c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:01.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1970c00640 1 -- 172.21.15.83:0/3521840442 shutdown_connections 2024-04-25T00:14:01.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1970c00640 1 -- 172.21.15.83:0/3521840442 wait complete. 2024-04-25T00:14:01.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1970c00640 1 Processor -- start 2024-04-25T00:14:01.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1970c00640 1 -- start start 2024-04-25T00:14:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1970c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f196c0cf3c0 0x7f196c13d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1970c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c1581d0 0x7f196c153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1970c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196c15e6b0 0x7f196c06dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1970c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f196c06da70 con 0x7f196c15e6b0 2024-04-25T00:14:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1970c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f196c06d8f0 con 0x7f196c1581d0 2024-04-25T00:14:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1970c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f196c06dbf0 con 0x7f196c0cf3c0 2024-04-25T00:14:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f196a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196c15e6b0 0x7f196c06dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f196a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196c15e6b0 0x7f196c06dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45312/0 (socket says 172.21.15.83:45312) 2024-04-25T00:14:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f196a800640 1 -- 172.21.15.83:0/2494135446 learned_addr learned my addr 172.21.15.83:0/2494135446 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1969400640 1 --2- 172.21.15.83:0/2494135446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c1581d0 0x7f196c153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f1969e00640 1 --2- 172.21.15.83:0/2494135446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f196c0cf3c0 0x7f196c13d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f196a800640 1 -- 172.21.15.83:0/2494135446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f196c0cf3c0 msgr2=0x7f196c13d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f196a800640 1 --2- 172.21.15.83:0/2494135446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f196c0cf3c0 0x7f196c13d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f196a800640 1 -- 172.21.15.83:0/2494135446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c1581d0 msgr2=0x7f196c153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f196a800640 1 --2- 172.21.15.83:0/2494135446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c1581d0 0x7f196c153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f196a800640 1 -- 172.21.15.83:0/2494135446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c13eee0 con 0x7f196c15e6b0 2024-04-25T00:14:01.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.500+0000 7f196a800640 1 --2- 172.21.15.83:0/2494135446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196c15e6b0 0x7f196c06dfb0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f196401a700 tx=0x7f196401a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:01.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.504+0000 7f1962000640 1 -- 172.21.15.83:0/2494135446 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964008700 con 0x7f196c15e6b0 2024-04-25T00:14:01.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.504+0000 7f1962000640 1 -- 172.21.15.83:0/2494135446 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19640bd070 con 0x7f196c15e6b0 2024-04-25T00:14:01.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.504+0000 7f1962000640 1 -- 172.21.15.83:0/2494135446 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964008b20 con 0x7f196c15e6b0 2024-04-25T00:14:01.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.504+0000 7f1970c00640 1 -- 172.21.15.83:0/2494135446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196c13e120 con 0x7f196c15e6b0 2024-04-25T00:14:01.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.504+0000 7f1970c00640 1 -- 172.21.15.83:0/2494135446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196c13e6e0 con 0x7f196c15e6b0 2024-04-25T00:14:01.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.504+0000 7f1970c00640 1 -- 172.21.15.83:0/2494135446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196c0737f0 con 0x7f196c15e6b0 2024-04-25T00:14:01.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.504+0000 7f1962000640 1 -- 172.21.15.83:0/2494135446 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1964008cc0 con 0x7f196c15e6b0 2024-04-25T00:14:01.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.504+0000 7f1962000640 1 --2- 172.21.15.83:0/2494135446 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f19480678f0 0x7f1948069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.504+0000 7f1962000640 1 -- 172.21.15.83:0/2494135446 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f1964120e80 con 0x7f196c15e6b0 2024-04-25T00:14:01.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.504+0000 7f1969e00640 1 --2- 172.21.15.83:0/2494135446 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f19480678f0 0x7f1948069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.504+0000 7f1969e00640 1 --2- 172.21.15.83:0/2494135446 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f19480678f0 0x7f1948069db0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1954001810 tx=0x7f1954009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.508+0000 7f1962000640 1 -- 172.21.15.83:0/2494135446 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f19640176e0 con 0x7f196c15e6b0 2024-04-25T00:14:01.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.664+0000 7f1970c00640 1 -- 172.21.15.83:0/2494135446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f196c04f5f0 con 0x7f196c15e6b0 2024-04-25T00:14:01.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.668+0000 7f1962000640 1 -- 172.21.15.83:0/2494135446 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16861 (secure 0 0 0) 0x7f19640f1200 con 0x7f196c15e6b0 2024-04-25T00:14:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:01.671 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":71,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:53.896317+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:50.655187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:51.268160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:01.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.668+0000 7f1970c00640 1 -- 172.21.15.83:0/2494135446 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f19480678f0 msgr2=0x7f1948069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.668+0000 7f1970c00640 1 --2- 172.21.15.83:0/2494135446 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f19480678f0 0x7f1948069db0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1954001810 tx=0x7f1954009040 comp rx=0 tx=0).stop 2024-04-25T00:14:01.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.668+0000 7f1970c00640 1 -- 172.21.15.83:0/2494135446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196c15e6b0 msgr2=0x7f196c06dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.668+0000 7f1970c00640 1 --2- 172.21.15.83:0/2494135446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196c15e6b0 0x7f196c06dfb0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f196401a700 tx=0x7f196401a890 comp rx=0 tx=0).stop 2024-04-25T00:14:01.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.668+0000 7f1970c00640 1 -- 172.21.15.83:0/2494135446 shutdown_connections 2024-04-25T00:14:01.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.668+0000 7f1970c00640 1 --2- 172.21.15.83:0/2494135446 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f19480678f0 0x7f1948069db0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.668+0000 7f1970c00640 1 --2- 172.21.15.83:0/2494135446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196c15e6b0 0x7f196c06dfb0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.668+0000 7f1970c00640 1 --2- 172.21.15.83:0/2494135446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c1581d0 0x7f196c153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.668+0000 7f1970c00640 1 --2- 172.21.15.83:0/2494135446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f196c0cf3c0 0x7f196c13d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.668+0000 7f1970c00640 1 -- 172.21.15.83:0/2494135446 >> 172.21.15.83:0/2494135446 conn(0x7f196c0b1830 msgr2=0x7f196c069060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:01.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.668+0000 7f1970c00640 1 -- 172.21.15.83:0/2494135446 shutdown_connections 2024-04-25T00:14:01.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.668+0000 7f1970c00640 1 -- 172.21.15.83:0/2494135446 wait complete. 2024-04-25T00:14:01.692 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:50.655187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:13:51.268160+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:14:01.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:14:01.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29d5600640 1 Processor -- start 2024-04-25T00:14:01.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29d5600640 1 -- start start 2024-04-25T00:14:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29d5600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f29d0159810 0x7f29d00cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29d5600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f29d0160020 0x7f29d00ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29d5600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d00ce000 con 0x7f29d006e950 2024-04-25T00:14:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29d5600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f29d006e210 con 0x7f29d0160020 2024-04-25T00:14:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29d5600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f29d006da00 con 0x7f29d0159810 2024-04-25T00:14:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29cf400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f29d0160020 0x7f29d00ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29ce000640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f29d006e950 0x7f29d015f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33184/0 (socket says 172.21.15.83:33184) 2024-04-25T00:14:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29ce000640 1 -- 172.21.15.83:0/3455529912 learned_addr learned my addr 172.21.15.83:0/3455529912 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:01.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29cea00640 1 --2- 172.21.15.83:0/3455529912 >> v2:172.21.15.86:3300/0 conn(0x7f29d0159810 0x7f29d00cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29cf400640 1 -- 172.21.15.83:0/3455529912 >> v2:172.21.15.86:3300/0 conn(0x7f29d0159810 msgr2=0x7f29d00cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:01.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29cf400640 1 --2- 172.21.15.83:0/3455529912 >> v2:172.21.15.86:3300/0 conn(0x7f29d0159810 0x7f29d00cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29cf400640 1 -- 172.21.15.83:0/3455529912 >> v1:172.21.15.83:6789/0 conn(0x7f29d006e950 legacy=0x7f29d015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29cf400640 1 -- 172.21.15.83:0/3455529912 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d006eec0 con 0x7f29d0160020 2024-04-25T00:14:01.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.828+0000 7f29cd600640 1 -- 172.21.15.83:0/3455529912 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1323180961 0 0) 0x7f29d00ce000 con 0x7f29d006e950 2024-04-25T00:14:01.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.832+0000 7f29cf400640 1 --2- 172.21.15.83:0/3455529912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d0160020 0x7f29d00ce4a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f29d006db80 tx=0x7f29c801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f10ee1904bcf5ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:01.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.832+0000 7f29cd600640 1 -- 172.21.15.83:0/3455529912 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c80b2070 con 0x7f29d0160020 2024-04-25T00:14:01.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.832+0000 7f29cd600640 1 -- 172.21.15.83:0/3455529912 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c80a5930 con 0x7f29d0160020 2024-04-25T00:14:01.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.832+0000 7f29cd600640 1 -- 172.21.15.83:0/3455529912 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c80a5d00 con 0x7f29d0160020 2024-04-25T00:14:01.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.832+0000 7f29d5600640 1 -- 172.21.15.83:0/3455529912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d0160020 msgr2=0x7f29d00ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.832+0000 7f29d5600640 1 --2- 172.21.15.83:0/3455529912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d0160020 0x7f29d00ce4a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f29d006db80 tx=0x7f29c801c870 comp rx=0 tx=0).stop 2024-04-25T00:14:01.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.832+0000 7f29d5600640 1 -- 172.21.15.83:0/3455529912 shutdown_connections 2024-04-25T00:14:01.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.832+0000 7f29d5600640 1 --2- 172.21.15.83:0/3455529912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d0160020 0x7f29d00ce4a0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.832+0000 7f29d5600640 1 --2- 172.21.15.83:0/3455529912 >> v2:172.21.15.86:3300/0 conn(0x7f29d0159810 0x7f29d00cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.832+0000 7f29d5600640 1 -- 172.21.15.83:0/3455529912 >> 172.21.15.83:0/3455529912 conn(0x7f29d00b1830 msgr2=0x7f29d00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.832+0000 7f29d5600640 1 -- 172.21.15.83:0/3455529912 shutdown_connections 2024-04-25T00:14:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.832+0000 7f29d5600640 1 -- 172.21.15.83:0/3455529912 wait complete. 2024-04-25T00:14:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29d5600640 1 Processor -- start 2024-04-25T00:14:01.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29d5600640 1 -- start start 2024-04-25T00:14:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29d5600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29d006e950 0x7f29d0150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29d5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d0159810 0x7f29d0142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29d5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d0160020 0x7f29d00b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29d5600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29d0159d70 con 0x7f29d0159810 2024-04-25T00:14:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29d5600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29d0159bf0 con 0x7f29d0160020 2024-04-25T00:14:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29d5600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f29d0070da0 con 0x7f29d006e950 2024-04-25T00:14:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29ce000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d0159810 0x7f29d0142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29ce000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d0159810 0x7f29d0142570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45332/0 (socket says 172.21.15.83:45332) 2024-04-25T00:14:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29ce000640 1 -- 172.21.15.83:0/1001784455 learned_addr learned my addr 172.21.15.83:0/1001784455 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29cf400640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d0160020 0x7f29d00b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29cea00640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29d006e950 0x7f29d0150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29ce000640 1 -- 172.21.15.83:0/1001784455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29d006e950 msgr2=0x7f29d0150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29ce000640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29d006e950 0x7f29d0150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29ce000640 1 -- 172.21.15.83:0/1001784455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d0160020 msgr2=0x7f29d00b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29ce000640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d0160020 0x7f29d00b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29ce000640 1 -- 172.21.15.83:0/1001784455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d00b4bf0 con 0x7f29d0159810 2024-04-25T00:14:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29cf400640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d0160020 0x7f29d00b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29ce000640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d0159810 0x7f29d0142570 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00c4b0 tx=0x7f29bc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29c6a00640 1 -- 172.21.15.83:0/1001784455 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc018070 con 0x7f29d0159810 2024-04-25T00:14:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29c6a00640 1 -- 172.21.15.83:0/1001784455 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29bc007500 con 0x7f29d0159810 2024-04-25T00:14:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29c6a00640 1 -- 172.21.15.83:0/1001784455 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc004a30 con 0x7f29d0159810 2024-04-25T00:14:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29d5600640 1 -- 172.21.15.83:0/1001784455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d00b4910 con 0x7f29d0159810 2024-04-25T00:14:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29d5600640 1 -- 172.21.15.83:0/1001784455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d01651f0 con 0x7f29d0159810 2024-04-25T00:14:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.836+0000 7f29d5600640 1 -- 172.21.15.83:0/1001784455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d004f510 con 0x7f29d0159810 2024-04-25T00:14:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.840+0000 7f29c6a00640 1 -- 172.21.15.83:0/1001784455 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f29bc01b070 con 0x7f29d0159810 2024-04-25T00:14:01.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.840+0000 7f29c6a00640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29b00678c0 0x7f29b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.840+0000 7f29c6a00640 1 -- 172.21.15.83:0/1001784455 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f29bc084fe0 con 0x7f29d0159810 2024-04-25T00:14:01.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.840+0000 7f29cea00640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29b00678c0 0x7f29b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.840+0000 7f29cea00640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29b00678c0 0x7f29b0069d80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f29c0006aa0 tx=0x7f29c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:01.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:01.844+0000 7f29c6a00640 1 -- 172.21.15.83:0/1001784455 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f29bc055330 con 0x7f29d0159810 2024-04-25T00:14:02.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.000+0000 7f29d5600640 1 -- 172.21.15.83:0/1001784455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f29d0156c60 con 0x7f29d0159810 2024-04-25T00:14:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.000+0000 7f29c6a00640 1 -- 172.21.15.83:0/1001784455 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f29bc059630 con 0x7f29d0159810 2024-04-25T00:14:02.004 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:14:02.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.004+0000 7f29d5600640 1 -- 172.21.15.83:0/1001784455 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29b00678c0 msgr2=0x7f29b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.004+0000 7f29d5600640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29b00678c0 0x7f29b0069d80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f29c0006aa0 tx=0x7f29c0008040 comp rx=0 tx=0).stop 2024-04-25T00:14:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.004+0000 7f29d5600640 1 -- 172.21.15.83:0/1001784455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d0159810 msgr2=0x7f29d0142570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.004+0000 7f29d5600640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d0159810 0x7f29d0142570 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00c4b0 tx=0x7f29bc00c970 comp rx=0 tx=0).stop 2024-04-25T00:14:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.004+0000 7f29d5600640 1 -- 172.21.15.83:0/1001784455 shutdown_connections 2024-04-25T00:14:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.004+0000 7f29d5600640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29b00678c0 0x7f29b0069d80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.004+0000 7f29d5600640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d0160020 0x7f29d00b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.004+0000 7f29d5600640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d0159810 0x7f29d0142570 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.004+0000 7f29d5600640 1 --2- 172.21.15.83:0/1001784455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29d006e950 0x7f29d0150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.004+0000 7f29d5600640 1 -- 172.21.15.83:0/1001784455 >> 172.21.15.83:0/1001784455 conn(0x7f29d00b1830 msgr2=0x7f29d015d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.004+0000 7f29d5600640 1 -- 172.21.15.83:0/1001784455 shutdown_connections 2024-04-25T00:14:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.004+0000 7f29d5600640 1 -- 172.21.15.83:0/1001784455 wait complete. 2024-04-25T00:14:02.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:14:02.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.160+0000 7ff63e600640 1 Processor -- start 2024-04-25T00:14:02.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.160+0000 7ff63e600640 1 -- start start 2024-04-25T00:14:02.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.160+0000 7ff63e600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff6381551e0 0x7ff63815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.160+0000 7ff63e600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff63806e4c0 0x7ff63806e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.160+0000 7ff63e600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff63815c560 con 0x7ff6381555c0 2024-04-25T00:14:02.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.160+0000 7ff63e600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff638072460 con 0x7ff63806e4c0 2024-04-25T00:14:02.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.160+0000 7ff63e600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff63815c7d0 con 0x7ff6381551e0 2024-04-25T00:14:02.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.160+0000 7ff637e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff63806e4c0 0x7ff63806e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.160+0000 7ff636a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff6381555c0 0x7ff638071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33200/0 (socket says 172.21.15.83:33200) 2024-04-25T00:14:02.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.160+0000 7ff636a00640 1 -- 172.21.15.83:0/3998198205 learned_addr learned my addr 172.21.15.83:0/3998198205 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:02.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.160+0000 7ff637400640 1 --2- 172.21.15.83:0/3998198205 >> v2:172.21.15.86:3300/0 conn(0x7ff6381551e0 0x7ff63815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.164+0000 7ff636000640 1 -- 172.21.15.83:0/3998198205 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3213717201 0 0) 0x7ff63815c560 con 0x7ff6381555c0 2024-04-25T00:14:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.164+0000 7ff636000640 1 -- 172.21.15.83:0/3998198205 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff61c003530 con 0x7ff6381555c0 2024-04-25T00:14:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.164+0000 7ff637e00640 1 -- 172.21.15.83:0/3998198205 >> v2:172.21.15.86:3300/0 conn(0x7ff6381551e0 msgr2=0x7ff63815bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.164+0000 7ff637e00640 1 --2- 172.21.15.83:0/3998198205 >> v2:172.21.15.86:3300/0 conn(0x7ff6381551e0 0x7ff63815bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.164+0000 7ff637e00640 1 -- 172.21.15.83:0/3998198205 >> v1:172.21.15.83:6789/0 conn(0x7ff6381555c0 legacy=0x7ff638071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.164+0000 7ff637e00640 1 -- 172.21.15.83:0/3998198205 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63815cab0 con 0x7ff63806e4c0 2024-04-25T00:14:02.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.164+0000 7ff637e00640 1 --2- 172.21.15.83:0/3998198205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63806e4c0 0x7ff63806e950 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff630009d90 tx=0x7ff63001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14afc05e6a3e4db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:02.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.164+0000 7ff636000640 1 -- 172.21.15.83:0/3998198205 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6300b2070 con 0x7ff63806e4c0 2024-04-25T00:14:02.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.164+0000 7ff636000640 1 -- 172.21.15.83:0/3998198205 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6300a57f0 con 0x7ff63806e4c0 2024-04-25T00:14:02.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.164+0000 7ff636000640 1 -- 172.21.15.83:0/3998198205 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6300a5bc0 con 0x7ff63806e4c0 2024-04-25T00:14:02.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.164+0000 7ff63e600640 1 -- 172.21.15.83:0/3998198205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63806e4c0 msgr2=0x7ff63806e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.164+0000 7ff63e600640 1 --2- 172.21.15.83:0/3998198205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63806e4c0 0x7ff63806e950 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff630009d90 tx=0x7ff63001c930 comp rx=0 tx=0).stop 2024-04-25T00:14:02.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff63e600640 1 -- 172.21.15.83:0/3998198205 shutdown_connections 2024-04-25T00:14:02.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff63e600640 1 --2- 172.21.15.83:0/3998198205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63806e4c0 0x7ff63806e950 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff63e600640 1 --2- 172.21.15.83:0/3998198205 >> v2:172.21.15.86:3300/0 conn(0x7ff6381551e0 0x7ff63815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff63e600640 1 -- 172.21.15.83:0/3998198205 >> 172.21.15.83:0/3998198205 conn(0x7ff6380b1830 msgr2=0x7ff6380b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:02.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff63e600640 1 -- 172.21.15.83:0/3998198205 shutdown_connections 2024-04-25T00:14:02.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff63e600640 1 -- 172.21.15.83:0/3998198205 wait complete. 2024-04-25T00:14:02.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff63e600640 1 Processor -- start 2024-04-25T00:14:02.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff63e600640 1 -- start start 2024-04-25T00:14:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff63e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63806e4c0 0x7ff6381512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff63e600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6381551e0 0x7ff6381517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff63e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6381555c0 0x7ff63813e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff63e600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff63815db70 con 0x7ff6381555c0 2024-04-25T00:14:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff63e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff63815d9f0 con 0x7ff63806e4c0 2024-04-25T00:14:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff63e600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff63815dcf0 con 0x7ff6381551e0 2024-04-25T00:14:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff637e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6381555c0 0x7ff63813e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff637e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6381555c0 0x7ff63813e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45348/0 (socket says 172.21.15.83:45348) 2024-04-25T00:14:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff637e00640 1 -- 172.21.15.83:0/1812016500 learned_addr learned my addr 172.21.15.83:0/1812016500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff636a00640 1 --2- 172.21.15.83:0/1812016500 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6381551e0 0x7ff6381517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff637400640 1 --2- 172.21.15.83:0/1812016500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63806e4c0 0x7ff6381512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff637e00640 1 -- 172.21.15.83:0/1812016500 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6381551e0 msgr2=0x7ff6381517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff637e00640 1 --2- 172.21.15.83:0/1812016500 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6381551e0 0x7ff6381517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff637e00640 1 -- 172.21.15.83:0/1812016500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63806e4c0 msgr2=0x7ff6381512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff637e00640 1 --2- 172.21.15.83:0/1812016500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63806e4c0 0x7ff6381512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff637e00640 1 -- 172.21.15.83:0/1812016500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63813f420 con 0x7ff6381555c0 2024-04-25T00:14:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.168+0000 7ff637e00640 1 --2- 172.21.15.83:0/1812016500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6381555c0 0x7ff63813e070 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff630009d90 tx=0x7ff6300196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.172+0000 7ff62f400640 1 -- 172.21.15.83:0/1812016500 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6300b2050 con 0x7ff6381555c0 2024-04-25T00:14:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.172+0000 7ff62f400640 1 -- 172.21.15.83:0/1812016500 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6300be070 con 0x7ff6381555c0 2024-04-25T00:14:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.172+0000 7ff62f400640 1 -- 172.21.15.83:0/1812016500 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630009450 con 0x7ff6381555c0 2024-04-25T00:14:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.172+0000 7ff63e600640 1 -- 172.21.15.83:0/1812016500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63813f700 con 0x7ff6381555c0 2024-04-25T00:14:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.172+0000 7ff63e600640 1 -- 172.21.15.83:0/1812016500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63813fcc0 con 0x7ff6381555c0 2024-04-25T00:14:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.172+0000 7ff63e600640 1 -- 172.21.15.83:0/1812016500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63804f510 con 0x7ff6381555c0 2024-04-25T00:14:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.172+0000 7ff62f400640 1 -- 172.21.15.83:0/1812016500 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff6300adc40 con 0x7ff6381555c0 2024-04-25T00:14:02.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.172+0000 7ff62f400640 1 --2- 172.21.15.83:0/1812016500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff6180678c0 0x7ff618069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.172+0000 7ff62f400640 1 -- 172.21.15.83:0/1812016500 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7ff630120ff0 con 0x7ff6381555c0 2024-04-25T00:14:02.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.172+0000 7ff637400640 1 --2- 172.21.15.83:0/1812016500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff6180678c0 0x7ff618069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.172+0000 7ff637400640 1 --2- 172.21.15.83:0/1812016500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff6180678c0 0x7ff618069d80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff628009b60 tx=0x7ff628009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:02.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.176+0000 7ff62f400640 1 -- 172.21.15.83:0/1812016500 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff630016520 con 0x7ff6381555c0 2024-04-25T00:14:02.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.340+0000 7ff63e600640 1 -- 172.21.15.83:0/1812016500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff63806ce30 con 0x7ff6381555c0 2024-04-25T00:14:02.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.340+0000 7ff62f400640 1 -- 172.21.15.83:0/1812016500 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7ff6300f12d0 con 0x7ff6381555c0 2024-04-25T00:14:02.344 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:14:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.344+0000 7ff63e600640 1 -- 172.21.15.83:0/1812016500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff6180678c0 msgr2=0x7ff618069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.344+0000 7ff63e600640 1 --2- 172.21.15.83:0/1812016500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff6180678c0 0x7ff618069d80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff628009b60 tx=0x7ff628009190 comp rx=0 tx=0).stop 2024-04-25T00:14:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.344+0000 7ff63e600640 1 -- 172.21.15.83:0/1812016500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6381555c0 msgr2=0x7ff63813e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.344+0000 7ff63e600640 1 --2- 172.21.15.83:0/1812016500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6381555c0 0x7ff63813e070 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff630009d90 tx=0x7ff6300196d0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.344+0000 7ff63e600640 1 -- 172.21.15.83:0/1812016500 shutdown_connections 2024-04-25T00:14:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.344+0000 7ff63e600640 1 --2- 172.21.15.83:0/1812016500 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff6180678c0 0x7ff618069d80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.344+0000 7ff63e600640 1 --2- 172.21.15.83:0/1812016500 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6381551e0 0x7ff6381517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.344+0000 7ff63e600640 1 --2- 172.21.15.83:0/1812016500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6381555c0 0x7ff63813e070 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.344+0000 7ff63e600640 1 --2- 172.21.15.83:0/1812016500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63806e4c0 0x7ff6381512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.344+0000 7ff63e600640 1 -- 172.21.15.83:0/1812016500 >> 172.21.15.83:0/1812016500 conn(0x7ff6380b1830 msgr2=0x7ff6380713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.344+0000 7ff63e600640 1 -- 172.21.15.83:0/1812016500 shutdown_connections 2024-04-25T00:14:02.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.344+0000 7ff63e600640 1 -- 172.21.15.83:0/1812016500 wait complete. 2024-04-25T00:14:02.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:14:02.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.500+0000 7f3912600640 1 Processor -- start 2024-04-25T00:14:02.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.500+0000 7f3912600640 1 -- start start 2024-04-25T00:14:02.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.500+0000 7f3912600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f390c160020 0x7f390c166ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.500+0000 7f3912600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f390c0ce000 con 0x7f390c159810 2024-04-25T00:14:02.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.500+0000 7f3912600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f390c06eec0 con 0x7f390c06e950 2024-04-25T00:14:02.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.500+0000 7f3912600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f390c06e210 con 0x7f390c160020 2024-04-25T00:14:02.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.500+0000 7f390b400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f390c159810 0x7f390c0cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33204/0 (socket says 172.21.15.83:33204) 2024-04-25T00:14:02.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.500+0000 7f390b400640 1 -- 172.21.15.83:0/2707769145 learned_addr learned my addr 172.21.15.83:0/2707769145 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:02.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.500+0000 7f390be00640 1 --2- 172.21.15.83:0/2707769145 >> v2:172.21.15.86:3300/0 conn(0x7f390c160020 0x7f390c166ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.500+0000 7f390aa00640 1 -- 172.21.15.83:0/2707769145 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 601570243 0 0) 0x7f390c0ce000 con 0x7f390c159810 2024-04-25T00:14:02.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.500+0000 7f390aa00640 1 -- 172.21.15.83:0/2707769145 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38f0003530 con 0x7f390c159810 2024-04-25T00:14:02.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.500+0000 7f390be00640 1 -- 172.21.15.83:0/2707769145 >> v1:172.21.15.86:6789/0 conn(0x7f390c06e950 legacy=0x7f390c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.500+0000 7f390be00640 1 -- 172.21.15.83:0/2707769145 >> v1:172.21.15.83:6789/0 conn(0x7f390c159810 legacy=0x7f390c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.500+0000 7f390be00640 1 -- 172.21.15.83:0/2707769145 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c06dc50 con 0x7f390c160020 2024-04-25T00:14:02.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.504+0000 7f390be00640 1 --2- 172.21.15.83:0/2707769145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c160020 0x7f390c166ca0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3904009d90 tx=0x7f390401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0fa2f4d6e89bbc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:02.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.504+0000 7f390aa00640 1 -- 172.21.15.83:0/2707769145 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39040b2070 con 0x7f390c160020 2024-04-25T00:14:02.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.504+0000 7f390aa00640 1 -- 172.21.15.83:0/2707769145 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39040a58b0 con 0x7f390c160020 2024-04-25T00:14:02.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.504+0000 7f390aa00640 1 -- 172.21.15.83:0/2707769145 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39040a5c80 con 0x7f390c160020 2024-04-25T00:14:02.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.504+0000 7f3912600640 1 -- 172.21.15.83:0/2707769145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c160020 msgr2=0x7f390c166ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.504+0000 7f3912600640 1 --2- 172.21.15.83:0/2707769145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c160020 0x7f390c166ca0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3904009d90 tx=0x7f390401c8e0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.504+0000 7f3912600640 1 -- 172.21.15.83:0/2707769145 shutdown_connections 2024-04-25T00:14:02.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.504+0000 7f3912600640 1 --2- 172.21.15.83:0/2707769145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c160020 0x7f390c166ca0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.504+0000 7f3912600640 1 -- 172.21.15.83:0/2707769145 >> 172.21.15.83:0/2707769145 conn(0x7f390c0b1830 msgr2=0x7f390c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:02.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.504+0000 7f3912600640 1 -- 172.21.15.83:0/2707769145 shutdown_connections 2024-04-25T00:14:02.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.504+0000 7f3912600640 1 -- 172.21.15.83:0/2707769145 wait complete. 2024-04-25T00:14:02.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f3912600640 1 Processor -- start 2024-04-25T00:14:02.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f3912600640 1 -- start start 2024-04-25T00:14:02.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f3912600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c06e950 0x7f390c14feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f3912600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f390c159810 0x7f390c0b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f3912600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f390c160020 0x7f390c154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f3912600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f390c169b80 con 0x7f390c160020 2024-04-25T00:14:02.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f3912600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f390c169a00 con 0x7f390c06e950 2024-04-25T00:14:02.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f3912600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f390c169d00 con 0x7f390c159810 2024-04-25T00:14:02.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f390be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f390c160020 0x7f390c154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f390be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f390c160020 0x7f390c154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45362/0 (socket says 172.21.15.83:45362) 2024-04-25T00:14:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f390be00640 1 -- 172.21.15.83:0/2036367247 learned_addr learned my addr 172.21.15.83:0/2036367247 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f390be00640 1 -- 172.21.15.83:0/2036367247 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f390c159810 msgr2=0x7f390c0b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f390b400640 1 --2- 172.21.15.83:0/2036367247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c06e950 0x7f390c14feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f390be00640 1 --2- 172.21.15.83:0/2036367247 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f390c159810 0x7f390c0b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f390be00640 1 -- 172.21.15.83:0/2036367247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c06e950 msgr2=0x7f390c14feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f390be00640 1 --2- 172.21.15.83:0/2036367247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c06e950 0x7f390c14feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f390be00640 1 -- 172.21.15.83:0/2036367247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c0b31e0 con 0x7f390c160020 2024-04-25T00:14:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f390b400640 1 --2- 172.21.15.83:0/2036367247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c06e950 0x7f390c14feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f390be00640 1 --2- 172.21.15.83:0/2036367247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f390c160020 0x7f390c154540 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f39040b0040 tx=0x7f3904019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f3903e00640 1 -- 172.21.15.83:0/2036367247 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39040b2050 con 0x7f390c160020 2024-04-25T00:14:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f3903e00640 1 -- 172.21.15.83:0/2036367247 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39040be070 con 0x7f390c160020 2024-04-25T00:14:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f3903e00640 1 -- 172.21.15.83:0/2036367247 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39040ad850 con 0x7f390c160020 2024-04-25T00:14:02.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f3912600640 1 -- 172.21.15.83:0/2036367247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c0b34c0 con 0x7f390c160020 2024-04-25T00:14:02.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f3912600640 1 -- 172.21.15.83:0/2036367247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c0b3a80 con 0x7f390c160020 2024-04-25T00:14:02.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.508+0000 7f3912600640 1 -- 172.21.15.83:0/2036367247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390c04f510 con 0x7f390c160020 2024-04-25T00:14:02.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.512+0000 7f3903e00640 1 -- 172.21.15.83:0/2036367247 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f39040ad9f0 con 0x7f390c160020 2024-04-25T00:14:02.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.512+0000 7f3903e00640 1 --2- 172.21.15.83:0/2036367247 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38e80678c0 0x7f38e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.512+0000 7f3903e00640 1 -- 172.21.15.83:0/2036367247 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f3904120e60 con 0x7f390c160020 2024-04-25T00:14:02.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.512+0000 7f390b400640 1 --2- 172.21.15.83:0/2036367247 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38e80678c0 0x7f38e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.512+0000 7f390b400640 1 --2- 172.21.15.83:0/2036367247 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38e80678c0 0x7f38e8069d80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0011f0 tx=0x7f38fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:02.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.516+0000 7f3903e00640 1 -- 172.21.15.83:0/2036367247 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3904016560 con 0x7f390c160020 2024-04-25T00:14:02.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.672+0000 7f3912600640 1 -- 172.21.15.83:0/2036367247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f390c072c90 con 0x7f390c160020 2024-04-25T00:14:02.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.672+0000 7f3903e00640 1 -- 172.21.15.83:0/2036367247 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7f39040f1800 con 0x7f390c160020 2024-04-25T00:14:02.675 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:14:02.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.672+0000 7f3912600640 1 -- 172.21.15.83:0/2036367247 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38e80678c0 msgr2=0x7f38e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.672+0000 7f3912600640 1 --2- 172.21.15.83:0/2036367247 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38e80678c0 0x7f38e8069d80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0011f0 tx=0x7f38fc009040 comp rx=0 tx=0).stop 2024-04-25T00:14:02.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.672+0000 7f3912600640 1 -- 172.21.15.83:0/2036367247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f390c160020 msgr2=0x7f390c154540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.672+0000 7f3912600640 1 --2- 172.21.15.83:0/2036367247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f390c160020 0x7f390c154540 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f39040b0040 tx=0x7f3904019680 comp rx=0 tx=0).stop 2024-04-25T00:14:02.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.672+0000 7f3912600640 1 -- 172.21.15.83:0/2036367247 shutdown_connections 2024-04-25T00:14:02.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.672+0000 7f3912600640 1 --2- 172.21.15.83:0/2036367247 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38e80678c0 0x7f38e8069d80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.672+0000 7f3912600640 1 --2- 172.21.15.83:0/2036367247 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f390c159810 0x7f390c0b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.672+0000 7f3912600640 1 --2- 172.21.15.83:0/2036367247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f390c160020 0x7f390c154540 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.672+0000 7f3912600640 1 --2- 172.21.15.83:0/2036367247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c06e950 0x7f390c14feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.672+0000 7f3912600640 1 -- 172.21.15.83:0/2036367247 >> 172.21.15.83:0/2036367247 conn(0x7f390c0b1830 msgr2=0x7f390c15e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:02.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.672+0000 7f3912600640 1 -- 172.21.15.83:0/2036367247 shutdown_connections 2024-04-25T00:14:02.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.672+0000 7f3912600640 1 -- 172.21.15.83:0/2036367247 wait complete. 2024-04-25T00:14:02.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:14:02.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.828+0000 7f086ee00640 1 Processor -- start 2024-04-25T00:14:02.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.828+0000 7f086ee00640 1 -- start start 2024-04-25T00:14:02.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.832+0000 7f086ee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f086806cfd0 0x7f0868070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.832+0000 7f086ee00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f086806e160 con 0x7f086806d3b0 2024-04-25T00:14:02.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.832+0000 7f086ee00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f086806ea10 con 0x7f08680cb640 2024-04-25T00:14:02.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.832+0000 7f086ee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f086806df20 con 0x7f086806cfd0 2024-04-25T00:14:02.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.832+0000 7f085fe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f086806d3b0 0x7f086806d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33216/0 (socket says 172.21.15.83:33216) 2024-04-25T00:14:02.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.832+0000 7f085fe00640 1 -- 172.21.15.83:0/1885911392 learned_addr learned my addr 172.21.15.83:0/1885911392 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:02.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.832+0000 7f0867e00640 1 --2- 172.21.15.83:0/1885911392 >> v2:172.21.15.86:3300/0 conn(0x7f086806cfd0 0x7f0868070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.832+0000 7f0867400640 1 -- 172.21.15.83:0/1885911392 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3944001282 0 0) 0x7f086806e160 con 0x7f086806d3b0 2024-04-25T00:14:02.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.832+0000 7f0867400640 1 -- 172.21.15.83:0/1885911392 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f084c003530 con 0x7f086806d3b0 2024-04-25T00:14:02.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.832+0000 7f0867400640 1 -- 172.21.15.83:0/1885911392 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3920063288 0 0) 0x7f086806ea10 con 0x7f08680cb640 2024-04-25T00:14:02.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.832+0000 7f0867400640 1 -- 172.21.15.83:0/1885911392 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f086806e160 con 0x7f08680cb640 2024-04-25T00:14:02.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.832+0000 7f0867e00640 1 -- 172.21.15.83:0/1885911392 >> v1:172.21.15.86:6789/0 conn(0x7f08680cb640 legacy=0x7f0868165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.832+0000 7f0867e00640 1 -- 172.21.15.83:0/1885911392 >> v1:172.21.15.83:6789/0 conn(0x7f086806d3b0 legacy=0x7f086806d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.832+0000 7f0867e00640 1 -- 172.21.15.83:0/1885911392 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086806ecf0 con 0x7f086806cfd0 2024-04-25T00:14:02.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.836+0000 7f0867e00640 1 --2- 172.21.15.83:0/1885911392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f086806cfd0 0x7f0868070d10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f08580097c0 tx=0x7f085801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b77915737efd1746 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:02.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.836+0000 7f0867400640 1 -- 172.21.15.83:0/1885911392 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08580b0070 con 0x7f086806cfd0 2024-04-25T00:14:02.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.836+0000 7f0867400640 1 -- 172.21.15.83:0/1885911392 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08580ab600 con 0x7f086806cfd0 2024-04-25T00:14:02.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.836+0000 7f0867400640 1 -- 172.21.15.83:0/1885911392 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08580ab9d0 con 0x7f086806cfd0 2024-04-25T00:14:02.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.836+0000 7f086ee00640 1 -- 172.21.15.83:0/1885911392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f086806cfd0 msgr2=0x7f0868070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.836+0000 7f086ee00640 1 --2- 172.21.15.83:0/1885911392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f086806cfd0 0x7f0868070d10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f08580097c0 tx=0x7f085801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.836+0000 7f086ee00640 1 -- 172.21.15.83:0/1885911392 shutdown_connections 2024-04-25T00:14:02.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.836+0000 7f086ee00640 1 --2- 172.21.15.83:0/1885911392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f086806cfd0 0x7f0868070d10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.836+0000 7f086ee00640 1 -- 172.21.15.83:0/1885911392 >> 172.21.15.83:0/1885911392 conn(0x7f08680cfa60 msgr2=0x7f08680cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:02.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.836+0000 7f086ee00640 1 -- 172.21.15.83:0/1885911392 shutdown_connections 2024-04-25T00:14:02.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.836+0000 7f086ee00640 1 -- 172.21.15.83:0/1885911392 wait complete. 2024-04-25T00:14:02.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.836+0000 7f086ee00640 1 Processor -- start 2024-04-25T00:14:02.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.836+0000 7f086ee00640 1 -- start start 2024-04-25T00:14:02.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f086ee00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f086806cfd0 0x7f086815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f086ee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f086806d3b0 0x7f086813bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f086ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08680cb640 0x7f08680b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f086ee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0868169a80 con 0x7f086806d3b0 2024-04-25T00:14:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f086ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0868145970 con 0x7f08680cb640 2024-04-25T00:14:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f086ee00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0868169c00 con 0x7f086806cfd0 2024-04-25T00:14:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f085fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f086806d3b0 0x7f086813bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f085fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f086806d3b0 0x7f086813bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45364/0 (socket says 172.21.15.83:45364) 2024-04-25T00:14:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f085fe00640 1 -- 172.21.15.83:0/3065297526 learned_addr learned my addr 172.21.15.83:0/3065297526 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f0867e00640 1 --2- 172.21.15.83:0/3065297526 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f086806cfd0 0x7f086815b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f086ce00640 1 --2- 172.21.15.83:0/3065297526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08680cb640 0x7f08680b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f085fe00640 1 -- 172.21.15.83:0/3065297526 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f086806cfd0 msgr2=0x7f086815b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f085fe00640 1 --2- 172.21.15.83:0/3065297526 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f086806cfd0 0x7f086815b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f085fe00640 1 -- 172.21.15.83:0/3065297526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08680cb640 msgr2=0x7f08680b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f085fe00640 1 --2- 172.21.15.83:0/3065297526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08680cb640 0x7f08680b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f085fe00640 1 -- 172.21.15.83:0/3065297526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086815b930 con 0x7f086806d3b0 2024-04-25T00:14:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f085fe00640 1 --2- 172.21.15.83:0/3065297526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f086806d3b0 0x7f086813bc30 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f085000c4d0 tx=0x7f085000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f0864c00640 1 -- 172.21.15.83:0/3065297526 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850017070 con 0x7f086806d3b0 2024-04-25T00:14:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f0864c00640 1 -- 172.21.15.83:0/3065297526 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0850005cf0 con 0x7f086806d3b0 2024-04-25T00:14:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f0864c00640 1 -- 172.21.15.83:0/3065297526 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850007270 con 0x7f086806d3b0 2024-04-25T00:14:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f086ee00640 1 -- 172.21.15.83:0/3065297526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086813c6d0 con 0x7f086806d3b0 2024-04-25T00:14:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f086ee00640 1 -- 172.21.15.83:0/3065297526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f086813a930 con 0x7f086806d3b0 2024-04-25T00:14:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f086ee00640 1 -- 172.21.15.83:0/3065297526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0868068230 con 0x7f086806d3b0 2024-04-25T00:14:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.840+0000 7f0864c00640 1 -- 172.21.15.83:0/3065297526 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f085001a070 con 0x7f086806d3b0 2024-04-25T00:14:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.844+0000 7f0864c00640 1 --2- 172.21.15.83:0/3065297526 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08440678c0 0x7f0844069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.844+0000 7f0864c00640 1 -- 172.21.15.83:0/3065297526 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f0850085110 con 0x7f086806d3b0 2024-04-25T00:14:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.844+0000 7f0867e00640 1 --2- 172.21.15.83:0/3065297526 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08440678c0 0x7f0844069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.844+0000 7f0867e00640 1 --2- 172.21.15.83:0/3065297526 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08440678c0 0x7f0844069d80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f08580ad000 tx=0x7f0858008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:02.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:02.848+0000 7f0864c00640 1 -- 172.21.15.83:0/3065297526 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f086815b930 con 0x7f086806d3b0 2024-04-25T00:14:03.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.012+0000 7f086ee00640 1 -- 172.21.15.83:0/3065297526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f086815bb30 con 0x7f086806d3b0 2024-04-25T00:14:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.016+0000 7f0864c00640 1 -- 172.21.15.83:0/3065297526 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7f0850055460 con 0x7f086806d3b0 2024-04-25T00:14:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:14:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.016+0000 7f086ee00640 1 -- 172.21.15.83:0/3065297526 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08440678c0 msgr2=0x7f0844069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.016+0000 7f086ee00640 1 --2- 172.21.15.83:0/3065297526 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08440678c0 0x7f0844069d80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f08580ad000 tx=0x7f0858008040 comp rx=0 tx=0).stop 2024-04-25T00:14:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.016+0000 7f086ee00640 1 -- 172.21.15.83:0/3065297526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f086806d3b0 msgr2=0x7f086813bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.016+0000 7f086ee00640 1 --2- 172.21.15.83:0/3065297526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f086806d3b0 0x7f086813bc30 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f085000c4d0 tx=0x7f085000c990 comp rx=0 tx=0).stop 2024-04-25T00:14:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.016+0000 7f086ee00640 1 -- 172.21.15.83:0/3065297526 shutdown_connections 2024-04-25T00:14:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.016+0000 7f086ee00640 1 --2- 172.21.15.83:0/3065297526 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08440678c0 0x7f0844069d80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.016+0000 7f086ee00640 1 --2- 172.21.15.83:0/3065297526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08680cb640 0x7f08680b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.016+0000 7f086ee00640 1 --2- 172.21.15.83:0/3065297526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f086806d3b0 0x7f086813bc30 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.016+0000 7f086ee00640 1 --2- 172.21.15.83:0/3065297526 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f086806cfd0 0x7f086815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.016+0000 7f086ee00640 1 -- 172.21.15.83:0/3065297526 >> 172.21.15.83:0/3065297526 conn(0x7f08680cfa60 msgr2=0x7f086813d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.016+0000 7f086ee00640 1 -- 172.21.15.83:0/3065297526 shutdown_connections 2024-04-25T00:14:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.016+0000 7f086ee00640 1 -- 172.21.15.83:0/3065297526 wait complete. 2024-04-25T00:14:03.039 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:14:03.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:03.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:14:03.066 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:14:03.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:03.066 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:14:03.122 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:14:03.122 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:03.132 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:14:03.132 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:03.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6789800640 1 Processor -- start 2024-04-25T00:14:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6789800640 1 -- start start 2024-04-25T00:14:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6789800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67841519d0 0x7f67840cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6789800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f678415b700 con 0x7f678406bb20 2024-04-25T00:14:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6789800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f678415b9e0 con 0x7f6784151db0 2024-04-25T00:14:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6789800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f678415bcc0 con 0x7f67841519d0 2024-04-25T00:14:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6788e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f678406bb20 0x7f678415ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33220/0 (socket says 172.21.15.83:33220) 2024-04-25T00:14:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6788e00640 1 -- 172.21.15.83:0/2444455639 learned_addr learned my addr 172.21.15.83:0/2444455639 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6783e00640 1 --2- 172.21.15.83:0/2444455639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67841519d0 0x7f67840cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6782a00640 1 -- 172.21.15.83:0/2444455639 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 974416731 0 0) 0x7f678415b700 con 0x7f678406bb20 2024-04-25T00:14:03.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6782a00640 1 -- 172.21.15.83:0/2444455639 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6768005740 con 0x7f678406bb20 2024-04-25T00:14:03.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6782a00640 1 -- 172.21.15.83:0/2444455639 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3345898971 0 0) 0x7f678415b9e0 con 0x7f6784151db0 2024-04-25T00:14:03.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6782a00640 1 -- 172.21.15.83:0/2444455639 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f678415b700 con 0x7f6784151db0 2024-04-25T00:14:03.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6783e00640 1 -- 172.21.15.83:0/2444455639 >> v1:172.21.15.86:6789/0 conn(0x7f6784151db0 legacy=0x7f678406b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6783e00640 1 -- 172.21.15.83:0/2444455639 >> v1:172.21.15.83:6789/0 conn(0x7f678406bb20 legacy=0x7f678415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.320+0000 7f6783e00640 1 -- 172.21.15.83:0/2444455639 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678415bfa0 con 0x7f67841519d0 2024-04-25T00:14:03.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.324+0000 7f6783e00640 1 --2- 172.21.15.83:0/2444455639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67841519d0 0x7f67840cdec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f67740097f0 tx=0x7f677401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72c0d612d94b5872 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.324+0000 7f6782a00640 1 -- 172.21.15.83:0/2444455639 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67740b0070 con 0x7f67841519d0 2024-04-25T00:14:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.324+0000 7f6782a00640 1 -- 172.21.15.83:0/2444455639 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67740ab600 con 0x7f67841519d0 2024-04-25T00:14:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.324+0000 7f6782a00640 1 -- 172.21.15.83:0/2444455639 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67740ab9b0 con 0x7f67841519d0 2024-04-25T00:14:03.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.324+0000 7f6789800640 1 -- 172.21.15.83:0/2444455639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67841519d0 msgr2=0x7f67840cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.324+0000 7f6789800640 1 --2- 172.21.15.83:0/2444455639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67841519d0 0x7f67840cdec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f67740097f0 tx=0x7f677401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.324+0000 7f6789800640 1 -- 172.21.15.83:0/2444455639 shutdown_connections 2024-04-25T00:14:03.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.324+0000 7f6789800640 1 --2- 172.21.15.83:0/2444455639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67841519d0 0x7f67840cdec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.324+0000 7f6789800640 1 -- 172.21.15.83:0/2444455639 >> 172.21.15.83:0/2444455639 conn(0x7f67840b5860 msgr2=0x7f67840b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:03.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.324+0000 7f6789800640 1 -- 172.21.15.83:0/2444455639 shutdown_connections 2024-04-25T00:14:03.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.324+0000 7f6789800640 1 -- 172.21.15.83:0/2444455639 wait complete. 2024-04-25T00:14:03.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.324+0000 7f6789800640 1 Processor -- start 2024-04-25T00:14:03.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.324+0000 7f6789800640 1 -- start start 2024-04-25T00:14:03.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6789800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f678406bb20 0x7f6784154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6789800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67841519d0 0x7f678415d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6789800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6784151db0 0x7f678415d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6789800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67840cd160 con 0x7f67841519d0 2024-04-25T00:14:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6789800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67840ccfe0 con 0x7f6784151db0 2024-04-25T00:14:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6789800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f67840cd2e0 con 0x7f678406bb20 2024-04-25T00:14:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6783400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67841519d0 0x7f678415d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6783400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67841519d0 0x7f678415d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45376/0 (socket says 172.21.15.83:45376) 2024-04-25T00:14:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6783400640 1 -- 172.21.15.83:0/256809679 learned_addr learned my addr 172.21.15.83:0/256809679 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6788e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6784151db0 0x7f678415d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6783e00640 1 --2- 172.21.15.83:0/256809679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f678406bb20 0x7f6784154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6783400640 1 -- 172.21.15.83:0/256809679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f678406bb20 msgr2=0x7f6784154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6783400640 1 --2- 172.21.15.83:0/256809679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f678406bb20 0x7f6784154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6783400640 1 -- 172.21.15.83:0/256809679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6784151db0 msgr2=0x7f678415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6783400640 1 --2- 172.21.15.83:0/256809679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6784151db0 0x7f678415d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6783400640 1 -- 172.21.15.83:0/256809679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67841507c0 con 0x7f67841519d0 2024-04-25T00:14:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6783400640 1 --2- 172.21.15.83:0/256809679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67841519d0 0x7f678415d110 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f677000c450 tx=0x7f677000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f677be00640 1 -- 172.21.15.83:0/256809679 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770005480 con 0x7f67841519d0 2024-04-25T00:14:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f677be00640 1 -- 172.21.15.83:0/256809679 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6770003600 con 0x7f67841519d0 2024-04-25T00:14:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6789800640 1 -- 172.21.15.83:0/256809679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6784150aa0 con 0x7f67841519d0 2024-04-25T00:14:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f677be00640 1 -- 172.21.15.83:0/256809679 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67700039d0 con 0x7f67841519d0 2024-04-25T00:14:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6789800640 1 -- 172.21.15.83:0/256809679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6784151060 con 0x7f67841519d0 2024-04-25T00:14:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.328+0000 7f6789800640 1 -- 172.21.15.83:0/256809679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f678404f510 con 0x7f67841519d0 2024-04-25T00:14:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.332+0000 7f677be00640 1 -- 172.21.15.83:0/256809679 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6770003b70 con 0x7f67841519d0 2024-04-25T00:14:03.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.332+0000 7f677be00640 1 --2- 172.21.15.83:0/256809679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f67640678c0 0x7f6764069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.332+0000 7f677be00640 1 -- 172.21.15.83:0/256809679 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f67700859b0 con 0x7f67841519d0 2024-04-25T00:14:03.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.332+0000 7f6783e00640 1 --2- 172.21.15.83:0/256809679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f67640678c0 0x7f6764069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.332+0000 7f6783e00640 1 --2- 172.21.15.83:0/256809679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f67640678c0 0x7f6764069d80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f67740ad000 tx=0x7f67740a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:03.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.336+0000 7f677be00640 1 -- 172.21.15.83:0/256809679 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6770055c70 con 0x7f67841519d0 2024-04-25T00:14:03.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.492+0000 7f6789800640 1 -- 172.21.15.83:0/256809679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6784155080 con 0x7f67841519d0 2024-04-25T00:14:03.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.492+0000 7f677be00640 1 -- 172.21.15.83:0/256809679 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16861 (secure 0 0 0) 0x7f6770059f70 con 0x7f67841519d0 2024-04-25T00:14:03.496 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:03.496 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":71,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:53.896317+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:50.655187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:51.268160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:03.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.496+0000 7f6789800640 1 -- 172.21.15.83:0/256809679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f67640678c0 msgr2=0x7f6764069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.496+0000 7f6789800640 1 --2- 172.21.15.83:0/256809679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f67640678c0 0x7f6764069d80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f67740ad000 tx=0x7f67740a6000 comp rx=0 tx=0).stop 2024-04-25T00:14:03.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.496+0000 7f6789800640 1 -- 172.21.15.83:0/256809679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67841519d0 msgr2=0x7f678415d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.496+0000 7f6789800640 1 --2- 172.21.15.83:0/256809679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67841519d0 0x7f678415d110 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f677000c450 tx=0x7f677000c910 comp rx=0 tx=0).stop 2024-04-25T00:14:03.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.496+0000 7f6789800640 1 -- 172.21.15.83:0/256809679 shutdown_connections 2024-04-25T00:14:03.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.496+0000 7f6789800640 1 --2- 172.21.15.83:0/256809679 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f67640678c0 0x7f6764069d80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.496+0000 7f6789800640 1 --2- 172.21.15.83:0/256809679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6784151db0 0x7f678415d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.496+0000 7f6789800640 1 --2- 172.21.15.83:0/256809679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67841519d0 0x7f678415d110 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.496+0000 7f6789800640 1 --2- 172.21.15.83:0/256809679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f678406bb20 0x7f6784154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.496+0000 7f6789800640 1 -- 172.21.15.83:0/256809679 >> 172.21.15.83:0/256809679 conn(0x7f67840b5860 msgr2=0x7f678406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:03.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.496+0000 7f6789800640 1 -- 172.21.15.83:0/256809679 shutdown_connections 2024-04-25T00:14:03.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.496+0000 7f6789800640 1 -- 172.21.15.83:0/256809679 wait complete. 2024-04-25T00:14:03.517 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:50.655187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:13:51.268160+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:14:03.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:14:03.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f350fc00640 1 Processor -- start 2024-04-25T00:14:03.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f350fc00640 1 -- start start 2024-04-25T00:14:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f350fc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3508159810 0x7f35080cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f350fc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3508160020 0x7f35080ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f350fc00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35080ce000 con 0x7f350806e950 2024-04-25T00:14:03.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f350fc00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f350806e210 con 0x7f3508159810 2024-04-25T00:14:03.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f350fc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f350806da00 con 0x7f3508160020 2024-04-25T00:14:03.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f34ffe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f350806e950 0x7f350815f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33236/0 (socket says 172.21.15.83:33236) 2024-04-25T00:14:03.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f350d200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3508159810 0x7f35080cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f34ffe00640 1 -- 172.21.15.83:0/1254051328 learned_addr learned my addr 172.21.15.83:0/1254051328 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:03.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f350dc00640 1 --2- 172.21.15.83:0/1254051328 >> v2:172.21.15.86:3300/0 conn(0x7f3508160020 0x7f35080ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f350d200640 1 -- 172.21.15.83:0/1254051328 >> v2:172.21.15.86:3300/0 conn(0x7f3508160020 msgr2=0x7f35080ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:03.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f350d200640 1 --2- 172.21.15.83:0/1254051328 >> v2:172.21.15.86:3300/0 conn(0x7f3508160020 0x7f35080ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f350d200640 1 -- 172.21.15.83:0/1254051328 >> v1:172.21.15.83:6789/0 conn(0x7f350806e950 legacy=0x7f350815f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f350d200640 1 -- 172.21.15.83:0/1254051328 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350806eec0 con 0x7f3508159810 2024-04-25T00:14:03.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f3507e00640 1 -- 172.21.15.83:0/1254051328 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3689645037 0 0) 0x7f35080ce000 con 0x7f350806e950 2024-04-25T00:14:03.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.652+0000 7f350dc00640 1 --2- 172.21.15.83:0/1254051328 >> v2:172.21.15.86:3300/0 conn(0x7f3508160020 0x7f35080ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:03.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.656+0000 7f350d200640 1 --2- 172.21.15.83:0/1254051328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3508159810 0x7f35080cc4a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f350806db80 tx=0x7f34f801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594800682706f9e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:03.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.656+0000 7f3507e00640 1 -- 172.21.15.83:0/1254051328 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f80b1070 con 0x7f3508159810 2024-04-25T00:14:03.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.656+0000 7f3507e00640 1 -- 172.21.15.83:0/1254051328 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f80ac600 con 0x7f3508159810 2024-04-25T00:14:03.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.656+0000 7f3507e00640 1 -- 172.21.15.83:0/1254051328 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f80ac9d0 con 0x7f3508159810 2024-04-25T00:14:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.656+0000 7f350fc00640 1 -- 172.21.15.83:0/1254051328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3508159810 msgr2=0x7f35080cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.656+0000 7f350fc00640 1 --2- 172.21.15.83:0/1254051328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3508159810 0x7f35080cc4a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f350806db80 tx=0x7f34f801c3a0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.656+0000 7f350fc00640 1 -- 172.21.15.83:0/1254051328 shutdown_connections 2024-04-25T00:14:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.656+0000 7f350fc00640 1 --2- 172.21.15.83:0/1254051328 >> v2:172.21.15.86:3300/0 conn(0x7f3508160020 0x7f35080ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.656+0000 7f350fc00640 1 --2- 172.21.15.83:0/1254051328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3508159810 0x7f35080cc4a0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.656+0000 7f350fc00640 1 -- 172.21.15.83:0/1254051328 >> 172.21.15.83:0/1254051328 conn(0x7f35080b1830 msgr2=0x7f35080b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.656+0000 7f350fc00640 1 -- 172.21.15.83:0/1254051328 shutdown_connections 2024-04-25T00:14:03.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.656+0000 7f350fc00640 1 -- 172.21.15.83:0/1254051328 wait complete. 2024-04-25T00:14:03.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350fc00640 1 Processor -- start 2024-04-25T00:14:03.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350fc00640 1 -- start start 2024-04-25T00:14:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350fc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f350806e950 0x7f35080b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3508159810 0x7f350815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350fc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3508160020 0x7f3508155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350fc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3508159d70 con 0x7f3508160020 2024-04-25T00:14:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3508159bf0 con 0x7f3508159810 2024-04-25T00:14:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350fc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3508070da0 con 0x7f350806e950 2024-04-25T00:14:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3508160020 0x7f3508155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3508160020 0x7f3508155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45386/0 (socket says 172.21.15.83:45386) 2024-04-25T00:14:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350dc00640 1 -- 172.21.15.83:0/1851670864 learned_addr learned my addr 172.21.15.83:0/1851670864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350d200640 1 --2- 172.21.15.83:0/1851670864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f350806e950 0x7f35080b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f34ffe00640 1 --2- 172.21.15.83:0/1851670864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3508159810 0x7f350815b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350dc00640 1 -- 172.21.15.83:0/1851670864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f350806e950 msgr2=0x7f35080b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350dc00640 1 --2- 172.21.15.83:0/1851670864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f350806e950 0x7f35080b2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350dc00640 1 -- 172.21.15.83:0/1851670864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3508159810 msgr2=0x7f350815b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350dc00640 1 --2- 172.21.15.83:0/1851670864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3508159810 0x7f350815b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350dc00640 1 -- 172.21.15.83:0/1851670864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350815d660 con 0x7f3508160020 2024-04-25T00:14:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350dc00640 1 --2- 172.21.15.83:0/1851670864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3508160020 0x7f3508155ec0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f350000ce80 tx=0x7f350000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:03.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f3505600640 1 -- 172.21.15.83:0/1851670864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350001a070 con 0x7f3508160020 2024-04-25T00:14:03.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f3505600640 1 -- 172.21.15.83:0/1851670864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f350000d880 con 0x7f3508160020 2024-04-25T00:14:03.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350fc00640 1 -- 172.21.15.83:0/1851670864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350815d940 con 0x7f3508160020 2024-04-25T00:14:03.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f3505600640 1 -- 172.21.15.83:0/1851670864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350000dc70 con 0x7f3508160020 2024-04-25T00:14:03.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.660+0000 7f350fc00640 1 -- 172.21.15.83:0/1851670864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f350815e1e0 con 0x7f3508160020 2024-04-25T00:14:03.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.664+0000 7f350fc00640 1 -- 172.21.15.83:0/1851670864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f350815db10 con 0x7f3508160020 2024-04-25T00:14:03.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.664+0000 7f3505600640 1 -- 172.21.15.83:0/1851670864 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f350001d070 con 0x7f3508160020 2024-04-25T00:14:03.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.664+0000 7f3505600640 1 --2- 172.21.15.83:0/1851670864 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34e40678c0 0x7f34e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.664+0000 7f3505600640 1 -- 172.21.15.83:0/1851670864 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f3500086660 con 0x7f3508160020 2024-04-25T00:14:03.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.664+0000 7f350d200640 1 --2- 172.21.15.83:0/1851670864 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34e40678c0 0x7f34e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.664+0000 7f350d200640 1 --2- 172.21.15.83:0/1851670864 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34e40678c0 0x7f34e4069d80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f34f80ae000 tx=0x7f34f80a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:03.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.668+0000 7f3505600640 1 -- 172.21.15.83:0/1851670864 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3500056920 con 0x7f3508160020 2024-04-25T00:14:03.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.824+0000 7f350fc00640 1 -- 172.21.15.83:0/1851670864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3508072c90 con 0x7f3508160020 2024-04-25T00:14:03.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.824+0000 7f3505600640 1 -- 172.21.15.83:0/1851670864 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f350005ac20 con 0x7f3508160020 2024-04-25T00:14:03.828 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:14:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.828+0000 7f350fc00640 1 -- 172.21.15.83:0/1851670864 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34e40678c0 msgr2=0x7f34e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.828+0000 7f350fc00640 1 --2- 172.21.15.83:0/1851670864 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34e40678c0 0x7f34e4069d80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f34f80ae000 tx=0x7f34f80a7000 comp rx=0 tx=0).stop 2024-04-25T00:14:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.828+0000 7f350fc00640 1 -- 172.21.15.83:0/1851670864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3508160020 msgr2=0x7f3508155ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.828+0000 7f350fc00640 1 --2- 172.21.15.83:0/1851670864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3508160020 0x7f3508155ec0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f350000ce80 tx=0x7f350000a710 comp rx=0 tx=0).stop 2024-04-25T00:14:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.828+0000 7f350fc00640 1 -- 172.21.15.83:0/1851670864 shutdown_connections 2024-04-25T00:14:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.828+0000 7f350fc00640 1 --2- 172.21.15.83:0/1851670864 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f34e40678c0 0x7f34e4069d80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.828+0000 7f350fc00640 1 --2- 172.21.15.83:0/1851670864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3508160020 0x7f3508155ec0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.828+0000 7f350fc00640 1 --2- 172.21.15.83:0/1851670864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3508159810 0x7f350815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.828+0000 7f350fc00640 1 --2- 172.21.15.83:0/1851670864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f350806e950 0x7f35080b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.828+0000 7f350fc00640 1 -- 172.21.15.83:0/1851670864 >> 172.21.15.83:0/1851670864 conn(0x7f35080b1830 msgr2=0x7f35081544c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:03.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.828+0000 7f350fc00640 1 -- 172.21.15.83:0/1851670864 shutdown_connections 2024-04-25T00:14:03.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.828+0000 7f350fc00640 1 -- 172.21.15.83:0/1851670864 wait complete. 2024-04-25T00:14:03.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:14:03.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.984+0000 7f384f600640 1 Processor -- start 2024-04-25T00:14:03.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.984+0000 7f384f600640 1 -- start start 2024-04-25T00:14:03.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.984+0000 7f384f600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3848159810 0x7f38480cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.984+0000 7f384f600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3848160020 0x7f38480ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.984+0000 7f384f600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38480ce000 con 0x7f384806e950 2024-04-25T00:14:03.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.984+0000 7f384f600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f384806e210 con 0x7f3848159810 2024-04-25T00:14:03.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.984+0000 7f384f600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f384806da00 con 0x7f3848160020 2024-04-25T00:14:03.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.984+0000 7f384cc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3848159810 0x7f38480cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.984+0000 7f383fe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f384806e950 0x7f384815f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33238/0 (socket says 172.21.15.83:33238) 2024-04-25T00:14:03.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.984+0000 7f383fe00640 1 -- 172.21.15.83:0/2063402022 learned_addr learned my addr 172.21.15.83:0/2063402022 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:03.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.988+0000 7f384d600640 1 --2- 172.21.15.83:0/2063402022 >> v2:172.21.15.86:3300/0 conn(0x7f3848160020 0x7f38480ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.988+0000 7f384cc00640 1 -- 172.21.15.83:0/2063402022 >> v2:172.21.15.86:3300/0 conn(0x7f3848160020 msgr2=0x7f38480ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:03.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.988+0000 7f384cc00640 1 --2- 172.21.15.83:0/2063402022 >> v2:172.21.15.86:3300/0 conn(0x7f3848160020 0x7f38480ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.988+0000 7f384cc00640 1 -- 172.21.15.83:0/2063402022 >> v1:172.21.15.83:6789/0 conn(0x7f384806e950 legacy=0x7f384815f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.988+0000 7f3847e00640 1 -- 172.21.15.83:0/2063402022 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3340741033 0 0) 0x7f38480ce000 con 0x7f384806e950 2024-04-25T00:14:03.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.988+0000 7f384cc00640 1 -- 172.21.15.83:0/2063402022 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384806eec0 con 0x7f3848159810 2024-04-25T00:14:03.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.988+0000 7f384d600640 1 --2- 172.21.15.83:0/2063402022 >> v2:172.21.15.86:3300/0 conn(0x7f3848160020 0x7f38480ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:03.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.988+0000 7f384cc00640 1 --2- 172.21.15.83:0/2063402022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3848159810 0x7f38480cc4a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f384806db80 tx=0x7f383801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4ab92b085b8d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:03.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.988+0000 7f3847e00640 1 -- 172.21.15.83:0/2063402022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38380b1070 con 0x7f3848159810 2024-04-25T00:14:03.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.988+0000 7f3847e00640 1 -- 172.21.15.83:0/2063402022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38380ac600 con 0x7f3848159810 2024-04-25T00:14:03.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.988+0000 7f3847e00640 1 -- 172.21.15.83:0/2063402022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38380ac9d0 con 0x7f3848159810 2024-04-25T00:14:03.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.988+0000 7f384f600640 1 -- 172.21.15.83:0/2063402022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3848159810 msgr2=0x7f38480cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.988+0000 7f384f600640 1 --2- 172.21.15.83:0/2063402022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3848159810 0x7f38480cc4a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f384806db80 tx=0x7f383801c3a0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384f600640 1 -- 172.21.15.83:0/2063402022 shutdown_connections 2024-04-25T00:14:03.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384f600640 1 --2- 172.21.15.83:0/2063402022 >> v2:172.21.15.86:3300/0 conn(0x7f3848160020 0x7f38480ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384f600640 1 --2- 172.21.15.83:0/2063402022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3848159810 0x7f38480cc4a0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384f600640 1 -- 172.21.15.83:0/2063402022 >> 172.21.15.83:0/2063402022 conn(0x7f38480b1830 msgr2=0x7f38480b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:03.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384f600640 1 -- 172.21.15.83:0/2063402022 shutdown_connections 2024-04-25T00:14:03.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384f600640 1 -- 172.21.15.83:0/2063402022 wait complete. 2024-04-25T00:14:03.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384f600640 1 Processor -- start 2024-04-25T00:14:03.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384f600640 1 -- start start 2024-04-25T00:14:03.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384f600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f384806e950 0x7f38480b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3848159810 0x7f38480ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384f600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3848160020 0x7f3848150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384f600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3848159d70 con 0x7f3848160020 2024-04-25T00:14:03.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3848159bf0 con 0x7f3848159810 2024-04-25T00:14:03.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384f600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3848070da0 con 0x7f384806e950 2024-04-25T00:14:03.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3848160020 0x7f3848150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3848160020 0x7f3848150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45400/0 (socket says 172.21.15.83:45400) 2024-04-25T00:14:03.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384d600640 1 -- 172.21.15.83:0/252460554 learned_addr learned my addr 172.21.15.83:0/252460554 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:03.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384cc00640 1 --2- 172.21.15.83:0/252460554 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f384806e950 0x7f38480b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f383fe00640 1 --2- 172.21.15.83:0/252460554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3848159810 0x7f38480ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384d600640 1 -- 172.21.15.83:0/252460554 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f384806e950 msgr2=0x7f38480b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384d600640 1 --2- 172.21.15.83:0/252460554 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f384806e950 0x7f38480b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384d600640 1 -- 172.21.15.83:0/252460554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3848159810 msgr2=0x7f38480ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384d600640 1 --2- 172.21.15.83:0/252460554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3848159810 0x7f38480ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384d600640 1 -- 172.21.15.83:0/252460554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38480ae9e0 con 0x7f3848160020 2024-04-25T00:14:03.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.992+0000 7f384d600640 1 --2- 172.21.15.83:0/252460554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3848160020 0x7f3848150870 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f3840009f70 tx=0x7f384000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:03.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.996+0000 7f3845600640 1 -- 172.21.15.83:0/252460554 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384001a070 con 0x7f3848160020 2024-04-25T00:14:03.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.996+0000 7f3845600640 1 -- 172.21.15.83:0/252460554 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3840003bc0 con 0x7f3848160020 2024-04-25T00:14:03.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.996+0000 7f384f600640 1 -- 172.21.15.83:0/252460554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38480aecc0 con 0x7f3848160020 2024-04-25T00:14:03.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.996+0000 7f3845600640 1 -- 172.21.15.83:0/252460554 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840004030 con 0x7f3848160020 2024-04-25T00:14:03.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.996+0000 7f384f600640 1 -- 172.21.15.83:0/252460554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f384815a090 con 0x7f3848160020 2024-04-25T00:14:03.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.996+0000 7f384f600640 1 -- 172.21.15.83:0/252460554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f384804f510 con 0x7f3848160020 2024-04-25T00:14:03.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.996+0000 7f3845600640 1 -- 172.21.15.83:0/252460554 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f384001d070 con 0x7f3848160020 2024-04-25T00:14:04.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.996+0000 7f3845600640 1 --2- 172.21.15.83:0/252460554 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38240678c0 0x7f3824069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.996+0000 7f3845600640 1 -- 172.21.15.83:0/252460554 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f3840086150 con 0x7f3848160020 2024-04-25T00:14:04.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.996+0000 7f384cc00640 1 --2- 172.21.15.83:0/252460554 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38240678c0 0x7f3824069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:03.996+0000 7f384cc00640 1 --2- 172.21.15.83:0/252460554 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38240678c0 0x7f3824069d80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f38380ae000 tx=0x7f38380a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:04.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.000+0000 7f3845600640 1 -- 172.21.15.83:0/252460554 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3840010070 con 0x7f3848160020 2024-04-25T00:14:04.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.156+0000 7f384f600640 1 -- 172.21.15.83:0/252460554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3848072c90 con 0x7f3848160020 2024-04-25T00:14:04.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.156+0000 7f3845600640 1 -- 172.21.15.83:0/252460554 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7f3840056410 con 0x7f3848160020 2024-04-25T00:14:04.161 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:14:04.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.160+0000 7f384f600640 1 -- 172.21.15.83:0/252460554 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38240678c0 msgr2=0x7f3824069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.160+0000 7f384f600640 1 --2- 172.21.15.83:0/252460554 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38240678c0 0x7f3824069d80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f38380ae000 tx=0x7f38380a7000 comp rx=0 tx=0).stop 2024-04-25T00:14:04.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.160+0000 7f384f600640 1 -- 172.21.15.83:0/252460554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3848160020 msgr2=0x7f3848150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.160+0000 7f384f600640 1 --2- 172.21.15.83:0/252460554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3848160020 0x7f3848150870 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f3840009f70 tx=0x7f384000d560 comp rx=0 tx=0).stop 2024-04-25T00:14:04.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.160+0000 7f384f600640 1 -- 172.21.15.83:0/252460554 shutdown_connections 2024-04-25T00:14:04.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.160+0000 7f384f600640 1 --2- 172.21.15.83:0/252460554 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38240678c0 0x7f3824069d80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.160+0000 7f384f600640 1 --2- 172.21.15.83:0/252460554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3848160020 0x7f3848150870 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.160+0000 7f384f600640 1 --2- 172.21.15.83:0/252460554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3848159810 0x7f38480ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.160+0000 7f384f600640 1 --2- 172.21.15.83:0/252460554 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f384806e950 0x7f38480b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.160+0000 7f384f600640 1 -- 172.21.15.83:0/252460554 >> 172.21.15.83:0/252460554 conn(0x7f38480b1830 msgr2=0x7f384806cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.160+0000 7f384f600640 1 -- 172.21.15.83:0/252460554 shutdown_connections 2024-04-25T00:14:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.160+0000 7f384f600640 1 -- 172.21.15.83:0/252460554 wait complete. 2024-04-25T00:14:04.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:14:04.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.316+0000 7f3762800640 1 Processor -- start 2024-04-25T00:14:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.316+0000 7f3762800640 1 -- start start 2024-04-25T00:14:04.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f3762800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f375c15e1e0 0x7f375c0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f3762800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f375c0cfb90 0x7f375c0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f3762800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f375c0d0070 con 0x7f375c0ce8c0 2024-04-25T00:14:04.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f3762800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f375c0d0c80 con 0x7f375c15e1e0 2024-04-25T00:14:04.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f3762800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f375c0d0350 con 0x7f375c0cfb90 2024-04-25T00:14:04.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f3760a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f375c0cfb90 0x7f375c0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f3761400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f375c15e1e0 0x7f375c0b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f3760a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f375c0cfb90 0x7f375c0ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:50772/0 (socket says 172.21.15.83:50772) 2024-04-25T00:14:04.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f3760a00640 1 -- 172.21.15.83:0/660561363 learned_addr learned my addr 172.21.15.83:0/660561363 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:04.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f3760a00640 1 -- 172.21.15.83:0/660561363 >> v2:172.21.15.86:3300/0 conn(0x7f375c15e1e0 msgr2=0x7f375c0b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f3760a00640 1 --2- 172.21.15.83:0/660561363 >> v2:172.21.15.86:3300/0 conn(0x7f375c15e1e0 0x7f375c0b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f3760a00640 1 -- 172.21.15.83:0/660561363 >> v1:172.21.15.86:6789/0 conn(0x7f375c0ce8c0 legacy=0x7f375c06c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f375be00640 1 -- 172.21.15.83:0/660561363 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1910632028 0 0) 0x7f375c0d0070 con 0x7f375c0ce8c0 2024-04-25T00:14:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f3760a00640 1 -- 172.21.15.83:0/660561363 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c0d0630 con 0x7f375c0cfb90 2024-04-25T00:14:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.320+0000 7f3761400640 1 --2- 172.21.15.83:0/660561363 >> v2:172.21.15.86:3300/0 conn(0x7f375c15e1e0 0x7f375c0b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:04.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3760a00640 1 --2- 172.21.15.83:0/660561363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f375c0cfb90 0x7f375c0ce380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3748009870 tx=0x7f374801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47502b27a342ffaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:04.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f375be00640 1 -- 172.21.15.83:0/660561363 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37480a7070 con 0x7f375c0cfb90 2024-04-25T00:14:04.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f375be00640 1 -- 172.21.15.83:0/660561363 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37480b0070 con 0x7f375c0cfb90 2024-04-25T00:14:04.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f375be00640 1 -- 172.21.15.83:0/660561363 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37480ab890 con 0x7f375c0cfb90 2024-04-25T00:14:04.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 -- 172.21.15.83:0/660561363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f375c0cfb90 msgr2=0x7f375c0ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 --2- 172.21.15.83:0/660561363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f375c0cfb90 0x7f375c0ce380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3748009870 tx=0x7f374801c560 comp rx=0 tx=0).stop 2024-04-25T00:14:04.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 -- 172.21.15.83:0/660561363 shutdown_connections 2024-04-25T00:14:04.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 --2- 172.21.15.83:0/660561363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f375c0cfb90 0x7f375c0ce380 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 --2- 172.21.15.83:0/660561363 >> v2:172.21.15.86:3300/0 conn(0x7f375c15e1e0 0x7f375c0b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 -- 172.21.15.83:0/660561363 >> 172.21.15.83:0/660561363 conn(0x7f375c0b36b0 msgr2=0x7f375c0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:04.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 -- 172.21.15.83:0/660561363 shutdown_connections 2024-04-25T00:14:04.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 -- 172.21.15.83:0/660561363 wait complete. 2024-04-25T00:14:04.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 Processor -- start 2024-04-25T00:14:04.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 -- start start 2024-04-25T00:14:04.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f375c0ce8c0 0x7f375c15c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f375c0cfb90 0x7f375c0b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f375c15e1e0 0x7f375c154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f375c16ae40 con 0x7f375c0ce8c0 2024-04-25T00:14:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f375c0cf210 con 0x7f375c0cfb90 2024-04-25T00:14:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3762800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f375c16afc0 con 0x7f375c15e1e0 2024-04-25T00:14:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3761400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f375c0ce8c0 0x7f375c15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3761400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f375c0ce8c0 0x7f375c15c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55342/0 (socket says 172.21.15.83:55342) 2024-04-25T00:14:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.324+0000 7f3761400640 1 -- 172.21.15.83:0/3787302127 learned_addr learned my addr 172.21.15.83:0/3787302127 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3760a00640 1 --2- 172.21.15.83:0/3787302127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f375c0cfb90 0x7f375c0b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3761e00640 1 --2- 172.21.15.83:0/3787302127 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f375c15e1e0 0x7f375c154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3761400640 1 -- 172.21.15.83:0/3787302127 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f375c15e1e0 msgr2=0x7f375c154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3761400640 1 --2- 172.21.15.83:0/3787302127 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f375c15e1e0 0x7f375c154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3761400640 1 -- 172.21.15.83:0/3787302127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f375c0cfb90 msgr2=0x7f375c0b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3761400640 1 --2- 172.21.15.83:0/3787302127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f375c0cfb90 0x7f375c0b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3761400640 1 -- 172.21.15.83:0/3787302127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c0b55b0 con 0x7f375c0ce8c0 2024-04-25T00:14:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3761400640 1 --2- 172.21.15.83:0/3787302127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f375c0ce8c0 0x7f375c15c800 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f375000efc0 tx=0x7f375000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:04.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3759600640 1 -- 172.21.15.83:0/3787302127 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750018020 con 0x7f375c0ce8c0 2024-04-25T00:14:04.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3759600640 1 -- 172.21.15.83:0/3787302127 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3750014680 con 0x7f375c0ce8c0 2024-04-25T00:14:04.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3762800640 1 -- 172.21.15.83:0/3787302127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375c0b4ff0 con 0x7f375c0ce8c0 2024-04-25T00:14:04.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3759600640 1 -- 172.21.15.83:0/3787302127 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750014a50 con 0x7f375c0ce8c0 2024-04-25T00:14:04.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3762800640 1 -- 172.21.15.83:0/3787302127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375c15c000 con 0x7f375c0ce8c0 2024-04-25T00:14:04.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:14:04.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3762800640 1 -- 172.21.15.83:0/3787302127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f375c04f510 con 0x7f375c0ce8c0 2024-04-25T00:14:04.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3759600640 1 -- 172.21.15.83:0/3787302127 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f375001b020 con 0x7f375c0ce8c0 2024-04-25T00:14:04.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3759600640 1 --2- 172.21.15.83:0/3787302127 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f373c0678c0 0x7f373c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3759600640 1 -- 172.21.15.83:0/3787302127 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f3750086aa0 con 0x7f375c0ce8c0 2024-04-25T00:14:04.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.328+0000 7f3760a00640 1 --2- 172.21.15.83:0/3787302127 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f373c0678c0 0x7f373c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.332+0000 7f3760a00640 1 --2- 172.21.15.83:0/3787302127 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f373c0678c0 0x7f373c069d80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f37480af000 tx=0x7f37480a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:04.336 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:14:04.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.336+0000 7f3759600640 1 -- 172.21.15.83:0/3787302127 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3750022030 con 0x7f375c0ce8c0 2024-04-25T00:14:04.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.492+0000 7f3762800640 1 -- 172.21.15.83:0/3787302127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f375c06f1a0 con 0x7f375c0ce8c0 2024-04-25T00:14:04.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.492+0000 7f3759600640 1 -- 172.21.15.83:0/3787302127 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7f3750010040 con 0x7f375c0ce8c0 2024-04-25T00:14:04.495 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:14:04.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.492+0000 7f3762800640 1 -- 172.21.15.83:0/3787302127 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f373c0678c0 msgr2=0x7f373c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.492+0000 7f3762800640 1 --2- 172.21.15.83:0/3787302127 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f373c0678c0 0x7f373c069d80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f37480af000 tx=0x7f37480a7000 comp rx=0 tx=0).stop 2024-04-25T00:14:04.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.492+0000 7f3762800640 1 -- 172.21.15.83:0/3787302127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f375c0ce8c0 msgr2=0x7f375c15c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.492+0000 7f3762800640 1 --2- 172.21.15.83:0/3787302127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f375c0ce8c0 0x7f375c15c800 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f375000efc0 tx=0x7f375000c490 comp rx=0 tx=0).stop 2024-04-25T00:14:04.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.492+0000 7f3762800640 1 -- 172.21.15.83:0/3787302127 shutdown_connections 2024-04-25T00:14:04.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.492+0000 7f3762800640 1 --2- 172.21.15.83:0/3787302127 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f373c0678c0 0x7f373c069d80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.492+0000 7f3762800640 1 --2- 172.21.15.83:0/3787302127 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f375c15e1e0 0x7f375c154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.492+0000 7f3762800640 1 --2- 172.21.15.83:0/3787302127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f375c0cfb90 0x7f375c0b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.492+0000 7f3762800640 1 --2- 172.21.15.83:0/3787302127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f375c0ce8c0 0x7f375c15c800 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.492+0000 7f3762800640 1 -- 172.21.15.83:0/3787302127 >> 172.21.15.83:0/3787302127 conn(0x7f375c0b36b0 msgr2=0x7f375c15a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:04.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.492+0000 7f3762800640 1 -- 172.21.15.83:0/3787302127 shutdown_connections 2024-04-25T00:14:04.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.492+0000 7f3762800640 1 -- 172.21.15.83:0/3787302127 wait complete. 2024-04-25T00:14:04.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:14:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.648+0000 7efeb3800640 1 Processor -- start 2024-04-25T00:14:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.648+0000 7efeb3800640 1 -- start start 2024-04-25T00:14:04.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.648+0000 7efeb3800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efeac15e160 0x7efeac0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.648+0000 7efeb3800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efeac15f420 0x7efeac0c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efeb3800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeac0cb8b0 con 0x7efeac0ceff0 2024-04-25T00:14:04.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efeb3800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7efeac0ca490 con 0x7efeac15e160 2024-04-25T00:14:04.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efeb3800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efeac15e940 con 0x7efeac15f420 2024-04-25T00:14:04.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efeb0e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efeac15e160 0x7efeac0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efeb0e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efeac15e160 0x7efeac0c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55344/0 (socket says 172.21.15.83:55344) 2024-04-25T00:14:04.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efeb0e00640 1 -- 172.21.15.83:0/86124786 learned_addr learned my addr 172.21.15.83:0/86124786 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:04.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efea3e00640 1 --2- 172.21.15.83:0/86124786 >> v2:172.21.15.86:3300/0 conn(0x7efeac15f420 0x7efeac0c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efeb0e00640 1 -- 172.21.15.83:0/86124786 >> v2:172.21.15.86:3300/0 conn(0x7efeac15f420 msgr2=0x7efeac0c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:04.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efeb0e00640 1 --2- 172.21.15.83:0/86124786 >> v2:172.21.15.86:3300/0 conn(0x7efeac15f420 0x7efeac0c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efeb0e00640 1 -- 172.21.15.83:0/86124786 >> v1:172.21.15.86:6789/0 conn(0x7efeac0ceff0 legacy=0x7efeac161c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:04.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efeb0e00640 1 -- 172.21.15.83:0/86124786 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac15ec20 con 0x7efeac15e160 2024-04-25T00:14:04.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efeb0e00640 1 --2- 172.21.15.83:0/86124786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeac15e160 0x7efeac0c8db0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7efeac0cf660 tx=0x7efe9c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a339ed19d3154a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:04.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efeabe00640 1 -- 172.21.15.83:0/86124786 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0b0070 con 0x7efeac15e160 2024-04-25T00:14:04.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efeabe00640 1 -- 172.21.15.83:0/86124786 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe9c0ab600 con 0x7efeac15e160 2024-04-25T00:14:04.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.652+0000 7efeabe00640 1 -- 172.21.15.83:0/86124786 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0ab9d0 con 0x7efeac15e160 2024-04-25T00:14:04.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 -- 172.21.15.83:0/86124786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeac15e160 msgr2=0x7efeac0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 --2- 172.21.15.83:0/86124786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeac15e160 0x7efeac0c8db0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7efeac0cf660 tx=0x7efe9c01c3c0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 -- 172.21.15.83:0/86124786 shutdown_connections 2024-04-25T00:14:04.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 --2- 172.21.15.83:0/86124786 >> v2:172.21.15.86:3300/0 conn(0x7efeac15f420 0x7efeac0c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 --2- 172.21.15.83:0/86124786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeac15e160 0x7efeac0c8db0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 -- 172.21.15.83:0/86124786 >> 172.21.15.83:0/86124786 conn(0x7efeac0cfa60 msgr2=0x7efeac0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:04.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 -- 172.21.15.83:0/86124786 shutdown_connections 2024-04-25T00:14:04.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 -- 172.21.15.83:0/86124786 wait complete. 2024-04-25T00:14:04.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 Processor -- start 2024-04-25T00:14:04.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 -- start start 2024-04-25T00:14:04.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeac0ceff0 0x7efeac15d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efeac15e160 0x7efeac152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeac15f420 0x7efeac152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efeac0c9da0 con 0x7efeac15f420 2024-04-25T00:14:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efeac0c9c20 con 0x7efeac0ceff0 2024-04-25T00:14:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb3800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efeac16b010 con 0x7efeac15e160 2024-04-25T00:14:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeac15f420 0x7efeac152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeac15f420 0x7efeac152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55348/0 (socket says 172.21.15.83:55348) 2024-04-25T00:14:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb1800640 1 -- 172.21.15.83:0/2353714613 learned_addr learned my addr 172.21.15.83:0/2353714613 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efea3e00640 1 --2- 172.21.15.83:0/2353714613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efeac15e160 0x7efeac152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb0e00640 1 --2- 172.21.15.83:0/2353714613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeac0ceff0 0x7efeac15d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb1800640 1 -- 172.21.15.83:0/2353714613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efeac15e160 msgr2=0x7efeac152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb1800640 1 --2- 172.21.15.83:0/2353714613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efeac15e160 0x7efeac152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb1800640 1 -- 172.21.15.83:0/2353714613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeac0ceff0 msgr2=0x7efeac15d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb1800640 1 --2- 172.21.15.83:0/2353714613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeac0ceff0 0x7efeac15d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb1800640 1 -- 172.21.15.83:0/2353714613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac156210 con 0x7efeac15f420 2024-04-25T00:14:04.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.656+0000 7efeb1800640 1 --2- 172.21.15.83:0/2353714613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeac15f420 0x7efeac152da0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7efea400b9a0 tx=0x7efea400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:04.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.660+0000 7efea9600640 1 -- 172.21.15.83:0/2353714613 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4019070 con 0x7efeac15f420 2024-04-25T00:14:04.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.660+0000 7efea9600640 1 -- 172.21.15.83:0/2353714613 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea400cba0 con 0x7efeac15f420 2024-04-25T00:14:04.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.660+0000 7efea9600640 1 -- 172.21.15.83:0/2353714613 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4014440 con 0x7efeac15f420 2024-04-25T00:14:04.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.660+0000 7efeb3800640 1 -- 172.21.15.83:0/2353714613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeac1564f0 con 0x7efeac15f420 2024-04-25T00:14:04.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.660+0000 7efeb3800640 1 -- 172.21.15.83:0/2353714613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeac156ab0 con 0x7efeac15f420 2024-04-25T00:14:04.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.660+0000 7efeb3800640 1 -- 172.21.15.83:0/2353714613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeac068230 con 0x7efeac15f420 2024-04-25T00:14:04.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.660+0000 7efea9600640 1 -- 172.21.15.83:0/2353714613 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efea401c070 con 0x7efeac15f420 2024-04-25T00:14:04.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.660+0000 7efea9600640 1 --2- 172.21.15.83:0/2353714613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efe880678c0 0x7efe88069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.660+0000 7efea9600640 1 -- 172.21.15.83:0/2353714613 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7efea40859f0 con 0x7efeac15f420 2024-04-25T00:14:04.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.660+0000 7efeb0e00640 1 --2- 172.21.15.83:0/2353714613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efe880678c0 0x7efe88069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.660+0000 7efeb0e00640 1 --2- 172.21.15.83:0/2353714613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efe880678c0 0x7efe88069d80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0ad000 tx=0x7efe9c0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:04.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.664+0000 7efea9600640 1 -- 172.21.15.83:0/2353714613 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efea4055cb0 con 0x7efeac15f420 2024-04-25T00:14:04.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.832+0000 7efeb3800640 1 -- 172.21.15.83:0/2353714613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efeac06e890 con 0x7efeac15f420 2024-04-25T00:14:04.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.832+0000 7efea9600640 1 -- 172.21.15.83:0/2353714613 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7efea4059fb0 con 0x7efeac15f420 2024-04-25T00:14:04.835 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:14:04.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.832+0000 7efeb3800640 1 -- 172.21.15.83:0/2353714613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efe880678c0 msgr2=0x7efe88069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.832+0000 7efeb3800640 1 --2- 172.21.15.83:0/2353714613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efe880678c0 0x7efe88069d80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0ad000 tx=0x7efe9c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:14:04.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.832+0000 7efeb3800640 1 -- 172.21.15.83:0/2353714613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeac15f420 msgr2=0x7efeac152da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.832+0000 7efeb3800640 1 --2- 172.21.15.83:0/2353714613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeac15f420 0x7efeac152da0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7efea400b9a0 tx=0x7efea400be60 comp rx=0 tx=0).stop 2024-04-25T00:14:04.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.832+0000 7efeb3800640 1 -- 172.21.15.83:0/2353714613 shutdown_connections 2024-04-25T00:14:04.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.832+0000 7efeb3800640 1 --2- 172.21.15.83:0/2353714613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efe880678c0 0x7efe88069d80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.832+0000 7efeb3800640 1 --2- 172.21.15.83:0/2353714613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efeac15e160 0x7efeac152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.832+0000 7efeb3800640 1 --2- 172.21.15.83:0/2353714613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeac15f420 0x7efeac152da0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.832+0000 7efeb3800640 1 --2- 172.21.15.83:0/2353714613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeac0ceff0 0x7efeac15d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.832+0000 7efeb3800640 1 -- 172.21.15.83:0/2353714613 >> 172.21.15.83:0/2353714613 conn(0x7efeac0cfa60 msgr2=0x7efeac0cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:04.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.832+0000 7efeb3800640 1 -- 172.21.15.83:0/2353714613 shutdown_connections 2024-04-25T00:14:04.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:04.832+0000 7efeb3800640 1 -- 172.21.15.83:0/2353714613 wait complete. 2024-04-25T00:14:04.857 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T00:14:04.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:04.857 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T00:14:04.885 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:14:04.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:04.885 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:14:04.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:05.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.076+0000 7ff3f8600640 1 Processor -- start 2024-04-25T00:14:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.076+0000 7ff3f8600640 1 -- start start 2024-04-25T00:14:05.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.076+0000 7ff3f8600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3f00d0230 0x7ff3f00d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.076+0000 7ff3f8600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3f00cc9b0 con 0x7ff3f0163b00 2024-04-25T00:14:05.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.076+0000 7ff3f8600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3f00ccc90 con 0x7ff3f015bf70 2024-04-25T00:14:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.076+0000 7ff3f8600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff3f00d0be0 con 0x7ff3f00d0230 2024-04-25T00:14:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.076+0000 7ff3f5200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3f00d0230 0x7ff3f00d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.076+0000 7ff3f6600640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff3f0163b00 0x7ff3f0165ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46228/0 (socket says 172.21.15.83:46228) 2024-04-25T00:14:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.076+0000 7ff3f6600640 1 -- 172.21.15.83:0/3847276371 learned_addr learned my addr 172.21.15.83:0/3847276371 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.076+0000 7ff3f5200640 1 -- 172.21.15.83:0/3847276371 >> v1:172.21.15.86:6789/0 conn(0x7ff3f015bf70 legacy=0x7ff3f00cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.076+0000 7ff3f5200640 1 -- 172.21.15.83:0/3847276371 >> v1:172.21.15.83:6789/0 conn(0x7ff3f0163b00 legacy=0x7ff3f0165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.076+0000 7ff3f5200640 1 -- 172.21.15.83:0/3847276371 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f00ccf70 con 0x7ff3f00d0230 2024-04-25T00:14:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.076+0000 7ff3efe00640 1 -- 172.21.15.83:0/3847276371 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3604599351 0 0) 0x7ff3f00cc9b0 con 0x7ff3f0163b00 2024-04-25T00:14:05.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.080+0000 7ff3f5200640 1 --2- 172.21.15.83:0/3847276371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3f00d0230 0x7ff3f00d06a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0097c0 tx=0x7ff3dc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55002d1ca923fd86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:05.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.080+0000 7ff3efe00640 1 -- 172.21.15.83:0/3847276371 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0b1070 con 0x7ff3f00d0230 2024-04-25T00:14:05.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.080+0000 7ff3efe00640 1 -- 172.21.15.83:0/3847276371 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3dc0ac5f0 con 0x7ff3f00d0230 2024-04-25T00:14:05.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.080+0000 7ff3efe00640 1 -- 172.21.15.83:0/3847276371 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0ac9c0 con 0x7ff3f00d0230 2024-04-25T00:14:05.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.080+0000 7ff3f8600640 1 -- 172.21.15.83:0/3847276371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3f00d0230 msgr2=0x7ff3f00d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.080+0000 7ff3f8600640 1 --2- 172.21.15.83:0/3847276371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3f00d0230 0x7ff3f00d06a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0097c0 tx=0x7ff3dc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.080+0000 7ff3f8600640 1 -- 172.21.15.83:0/3847276371 shutdown_connections 2024-04-25T00:14:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.080+0000 7ff3f8600640 1 --2- 172.21.15.83:0/3847276371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3f00d0230 0x7ff3f00d06a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.080+0000 7ff3f8600640 1 -- 172.21.15.83:0/3847276371 >> 172.21.15.83:0/3847276371 conn(0x7ff3f00d1c70 msgr2=0x7ff3f00d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.080+0000 7ff3f8600640 1 -- 172.21.15.83:0/3847276371 shutdown_connections 2024-04-25T00:14:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.080+0000 7ff3f8600640 1 -- 172.21.15.83:0/3847276371 wait complete. 2024-04-25T00:14:05.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f8600640 1 Processor -- start 2024-04-25T00:14:05.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f8600640 1 -- start start 2024-04-25T00:14:05.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f8600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3f00d0230 0x7ff3f01535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f8600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3f015bf70 0x7ff3f00cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f8600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3f0163b00 0x7ff3f00c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f8600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f00cff50 con 0x7ff3f015bf70 2024-04-25T00:14:05.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f8600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f015c350 con 0x7ff3f0163b00 2024-04-25T00:14:05.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f8600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f00cda50 con 0x7ff3f00d0230 2024-04-25T00:14:05.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3f015bf70 0x7ff3f00cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3f015bf70 0x7ff3f00cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55358/0 (socket says 172.21.15.83:55358) 2024-04-25T00:14:05.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f5200640 1 -- 172.21.15.83:0/3325836616 learned_addr learned my addr 172.21.15.83:0/3325836616 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:05.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f6600640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3f0163b00 0x7ff3f00c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f5c00640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3f00d0230 0x7ff3f01535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f5200640 1 -- 172.21.15.83:0/3325836616 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3f00d0230 msgr2=0x7ff3f01535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f5200640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3f00d0230 0x7ff3f01535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f5200640 1 -- 172.21.15.83:0/3325836616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3f0163b00 msgr2=0x7ff3f00c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f5200640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3f0163b00 0x7ff3f00c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f5200640 1 -- 172.21.15.83:0/3325836616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f00ca470 con 0x7ff3f015bf70 2024-04-25T00:14:05.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f5c00640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3f00d0230 0x7ff3f01535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:05.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f5200640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3f015bf70 0x7ff3f00cb510 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0af040 tx=0x7ff3dc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:05.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3ed600640 1 -- 172.21.15.83:0/3325836616 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0b9020 con 0x7ff3f015bf70 2024-04-25T00:14:05.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3ed600640 1 -- 172.21.15.83:0/3325836616 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3dc0b5630 con 0x7ff3f015bf70 2024-04-25T00:14:05.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f8600640 1 -- 172.21.15.83:0/3325836616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f00ca750 con 0x7ff3f015bf70 2024-04-25T00:14:05.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3ed600640 1 -- 172.21.15.83:0/3325836616 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0b1040 con 0x7ff3f015bf70 2024-04-25T00:14:05.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f8600640 1 -- 172.21.15.83:0/3325836616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f00cad10 con 0x7ff3f015bf70 2024-04-25T00:14:05.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.084+0000 7ff3f8600640 1 -- 172.21.15.83:0/3325836616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f009fd60 con 0x7ff3f015bf70 2024-04-25T00:14:05.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.088+0000 7ff3ed600640 1 -- 172.21.15.83:0/3325836616 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff3dc0bc020 con 0x7ff3f015bf70 2024-04-25T00:14:05.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.088+0000 7ff3ed600640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff3d00678c0 0x7ff3d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.088+0000 7ff3ed600640 1 -- 172.21.15.83:0/3325836616 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7ff3dc11f8c0 con 0x7ff3f015bf70 2024-04-25T00:14:05.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.088+0000 7ff3f5c00640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff3d00678c0 0x7ff3d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.088+0000 7ff3f5c00640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff3d00678c0 0x7ff3d0069d80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4000f70 tx=0x7ff3e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:05.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.092+0000 7ff3ed600640 1 -- 172.21.15.83:0/3325836616 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff3dc0c3030 con 0x7ff3f015bf70 2024-04-25T00:14:05.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.280+0000 7ff3f8600640 1 -- 172.21.15.83:0/3325836616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3f004f5f0 con 0x7ff3f015bf70 2024-04-25T00:14:05.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.280+0000 7ff3ed600640 1 -- 172.21.15.83:0/3325836616 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5183 (secure 0 0 0) 0x7ff3dc0a9070 con 0x7ff3f015bf70 2024-04-25T00:14:05.283 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:05.284 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2852678498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2852678498},{"type":"v1","addr":"172.21.15.86:6834","nonce":2852678498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2956252746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2956252746},{"type":"v1","addr":"172.21.15.86:6835","nonce":2956252746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6588,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2678835998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2678835998},{"type":"v1","addr":"172.21.15.83:6836","nonce":2678835998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:13:52.273680+0000","modified":"2024-04-25T00:13:55.285817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2879143105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2879143105},{"type":"v1","addr":"172.21.15.83:6837","nonce":2879143105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6591,"qdb_cluster":[6591]},"id":5}]} 2024-04-25T00:14:05.284 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 44 2024-04-25T00:14:05.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.280+0000 7ff3f8600640 1 -- 172.21.15.83:0/3325836616 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff3d00678c0 msgr2=0x7ff3d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.280+0000 7ff3f8600640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff3d00678c0 0x7ff3d0069d80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4000f70 tx=0x7ff3e4009040 comp rx=0 tx=0).stop 2024-04-25T00:14:05.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.280+0000 7ff3f8600640 1 -- 172.21.15.83:0/3325836616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3f015bf70 msgr2=0x7ff3f00cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.280+0000 7ff3f8600640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3f015bf70 0x7ff3f00cb510 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0af040 tx=0x7ff3dc019240 comp rx=0 tx=0).stop 2024-04-25T00:14:05.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.280+0000 7ff3f8600640 1 -- 172.21.15.83:0/3325836616 shutdown_connections 2024-04-25T00:14:05.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.280+0000 7ff3f8600640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff3d00678c0 0x7ff3d0069d80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.280+0000 7ff3f8600640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3f0163b00 0x7ff3f00c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.280+0000 7ff3f8600640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3f015bf70 0x7ff3f00cb510 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.280+0000 7ff3f8600640 1 --2- 172.21.15.83:0/3325836616 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3f00d0230 0x7ff3f01535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.280+0000 7ff3f8600640 1 -- 172.21.15.83:0/3325836616 >> 172.21.15.83:0/3325836616 conn(0x7ff3f00d1c70 msgr2=0x7ff3f00b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:05.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.280+0000 7ff3f8600640 1 -- 172.21.15.83:0/3325836616 shutdown_connections 2024-04-25T00:14:05.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.280+0000 7ff3f8600640 1 -- 172.21.15.83:0/3325836616 wait complete. 2024-04-25T00:14:05.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-04-25T00:14:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":23,"msgr_send_messages":25,"msgr_recv_bytes":6215,"msgr_send_bytes":13540,"msgr_created_connections":7,"msgr_active_connections":2,"msgr_running_total_time":0.008931600,"msgr_running_send_time":0.000536464,"msgr_running_recv_time":0.003908494,"msgr_running_fast_dispatch_time":0.000564766,"msgr_send_messages_queue_lat":{"avgcount":23,"sum":0.189881756,"avgtime":0.008255728},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":101,"msgr_send_messages":91,"msgr_recv_bytes":123613,"msgr_send_bytes":52003,"msgr_created_connections":7,"msgr_active_connections":3,"msgr_running_total_time":0.015483185,"msgr_running_send_time":0.004537837,"msgr_running_recv_time":0.016032421,"msgr_running_fast_dispatch_time":0.000493107,"msgr_send_messages_queue_lat":{"avgcount":91,"sum":0.042175991,"avgtime":0.000463472},"msgr_handle_ack_lat":{"avgcount":58,"sum":0.000009808,"avgtime":0.000000169},"msgr_recv_encrypted_bytes":112688,"msgr_send_encrypted_bytes":12176},"AsyncMessenger::Worker-2":{"msgr_recv_messages":24,"msgr_send_messages":26,"msgr_recv_bytes":6585,"msgr_send_bytes":108811,"msgr_created_connections":6,"msgr_active_connections":3,"msgr_running_total_time":0.006872301,"msgr_running_send_time":0.001027859,"msgr_running_recv_time":0.005200883,"msgr_running_fast_dispatch_time":0.000417946,"msgr_send_messages_queue_lat":{"avgcount":24,"sum":0.031941606,"avgtime":0.001330900},"msgr_handle_ack_lat":{"avgcount":14,"sum":0.000001945,"avgtime":0.000000138},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":94704},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":41,"sum":0.001953214,"avgtime":0.000047639}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000017165,"avgtime":0.000008582}},"mds":{"request":29,"reply":29,"reply_latency":{"avgcount":29,"sum":0.004130285,"avgtime":0.000142423},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":2,"subtrees":2,"traverse":55,"traverse_hit":53,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":8,"evex":0,"evtrm":0,"ev":8,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197029,"rdpos":0,"jlat":{"avgcount":5,"sum":0.011713745,"avgtime":0.002342749},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":0,"cap+":2,"cap-":2,"rss":43816,"heap":231732},"mds_server":{"dispatch_client_request":29,"dispatch_server_request":0,"handle_client_request":29,"handle_client_session":6,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":26,"sum":0.003575379,"avgtime":0.000137514},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":2,"sum":0.000228926,"avgtime":0.000114463},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000325980,"avgtime":0.000325980},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":28,"average_load":14,"avg_session_uptime":2,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":217341,"buffer_anon_items":85,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":53576,"mds_co_items":218,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":43,"op_send_bytes":13845,"op_resend":37,"op_reply":43,"op_latency":{"avgcount":43,"sum":0.711980839,"avgtime":0.016557693},"op_inflight":0,"oplen_avg":{"avgcount":43,"sum":67},"op":43,"op_r":0,"op_w":43,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":71,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":144,"get_sum":119143,"get_or_fail_fail":0,"get_or_fail_success":144,"take":0,"take_sum":0,"put":144,"put_sum":119143,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":13845,"put":43,"put_sum":13845,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":43,"put":43,"put_sum":43,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x556c1d1a8780":{"val":0,"max":3758096384,"get_started":0,"get":8,"get_sum":2725,"get_or_fail_fail":0,"get_or_fail_success":8,"take":0,"take_sum":0,"put":5,"put_sum":2725,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-04-25T00:14:05.465 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 6215, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 23, "msgr_running_fast_dispatch_time": 0.000564766, "msgr_running_recv_time": 0.003908494, "msgr_running_send_time": 0.000536464, "msgr_running_total_time": 0.0089316, "msgr_send_bytes": 13540, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 25, "msgr_send_messages_queue_lat": { "avgcount": 23, "avgtime": 0.008255728, "sum": 0.189881756 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 3, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 58, "avgtime": 1.69e-07, "sum": 9.808e-06 }, "msgr_recv_bytes": 123613, "msgr_recv_encrypted_bytes": 112688, "msgr_recv_messages": 101, "msgr_running_fast_dispatch_time": 0.000493107, "msgr_running_recv_time": 0.016032421, "msgr_running_send_time": 0.004537837, "msgr_running_total_time": 0.015483185, "msgr_send_bytes": 52003, "msgr_send_encrypted_bytes": 12176, "msgr_send_messages": 91, "msgr_send_messages_queue_lat": { "avgcount": 91, "avgtime": 0.000463472, "sum": 0.042175991 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 14, "avgtime": 1.38e-07, "sum": 1.945e-06 }, "msgr_recv_bytes": 6585, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 24, "msgr_running_fast_dispatch_time": 0.000417946, "msgr_running_recv_time": 0.005200883, "msgr_running_send_time": 0.001027859, "msgr_running_total_time": 0.006872301, "msgr_send_bytes": 108811, "msgr_send_encrypted_bytes": 94704, "msgr_send_messages": 26, "msgr_send_messages_queue_lat": { "avgcount": 24, "avgtime": 0.0013309, "sum": 0.031941606 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 41, "avgtime": 4.7639e-05, "sum": 0.001953214 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 8.582e-06, "sum": 1.7165e-05 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 29, "reply_latency": { "avgcount": 29, "avgtime": 0.000142423, "sum": 0.004130285 }, "request": 29, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 55, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 53, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 8, "evadd": 8, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.002342749, "sum": 0.011713745 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197029 }, "mds_mem": { "cap": 0, "cap+": 2, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 231732, "ino": 13, "ino+": 13, "ino-": 0, "rss": 43816 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 29, "dispatch_server_request": 0, "handle_client_request": 29, "handle_client_session": 6, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 26, "avgtime": 0.000137514, "sum": 0.003575379 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 2, "avgtime": 0.000114463, "sum": 0.000228926 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.00032598, "sum": 0.00032598 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 14, "avg_session_uptime": 2, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 28 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 217341, "buffer_anon_items": 85, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 53576, "mds_co_items": 218, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 71, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 43, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 43, "avgtime": 0.016557693, "sum": 0.711980839 }, "op_pg": 0, "op_r": 0, "op_reply": 43, "op_resend": 37, "op_rmw": 0, "op_send": 43, "op_send_bytes": 13845, "op_w": 43, "oplen_avg": { "avgcount": 43, "sum": 67 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 144, "get_or_fail_fail": 0, "get_or_fail_success": 144, "get_started": 0, "get_sum": 119143, "max": 104857600, "put": 144, "put_sum": 119143, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 43, "put_sum": 13845, "take": 43, "take_sum": 13845, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 43, "put_sum": 43, "take": 43, "take_sum": 43, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x556c1d1a8780": { "get": 8, "get_or_fail_fail": 0, "get_or_fail_success": 8, "get_started": 0, "get_sum": 2725, "max": 3758096384, "put": 5, "put_sum": 2725, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-04-25T00:14:05.465 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 session ls 2024-04-25T00:14:05.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bbfe00640 1 Processor -- start 2024-04-25T00:14:05.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bbfe00640 1 -- start start 2024-04-25T00:14:05.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bbfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3bb8159810 0x7f3bb80cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bbfe00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb80ce000 con 0x7f3bb806e950 2024-04-25T00:14:05.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bbfe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb806eec0 con 0x7f3bb8160020 2024-04-25T00:14:05.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bbfe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb806e210 con 0x7f3bb8159810 2024-04-25T00:14:05.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bbca00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3bb806e950 0x7f3bb815f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46232/0 (socket says 172.21.15.83:46232) 2024-04-25T00:14:05.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bbca00640 1 -- 172.21.15.83:0/120263829 learned_addr learned my addr 172.21.15.83:0/120263829 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:05.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bbd400640 1 --2- 172.21.15.83:0/120263829 >> v2:172.21.15.86:3300/0 conn(0x7f3bb8159810 0x7f3bb80cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 -- 172.21.15.83:0/120263829 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3592858850 0 0) 0x7f3bb80ce000 con 0x7f3bb806e950 2024-04-25T00:14:05.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 -- 172.21.15.83:0/120263829 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ba0003530 con 0x7f3bb806e950 2024-04-25T00:14:05.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 -- 172.21.15.83:0/120263829 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2049097089 0 0) 0x7f3ba0003530 con 0x7f3bb806e950 2024-04-25T00:14:05.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 -- 172.21.15.83:0/120263829 >> v2:172.21.15.86:3300/0 conn(0x7f3bb8159810 msgr2=0x7f3bb80cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:05.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 --2- 172.21.15.83:0/120263829 >> v2:172.21.15.86:3300/0 conn(0x7f3bb8159810 0x7f3bb80cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 -- 172.21.15.83:0/120263829 >> v1:172.21.15.86:6789/0 conn(0x7f3bb8160020 legacy=0x7f3bb80ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 -- 172.21.15.83:0/120263829 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb806dc50 con 0x7f3bb806e950 2024-04-25T00:14:05.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 -- 172.21.15.83:0/120263829 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f3ba4001680 con 0x7f3bb806e950 2024-04-25T00:14:05.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 -- 172.21.15.83:0/120263829 >> v1:172.21.15.83:6789/0 conn(0x7f3bb806e950 legacy=0x7f3bb815f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 --2- 172.21.15.83:0/120263829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ba0003a30 0x7f3ba0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 --2- 172.21.15.83:0/120263829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ba0004810 0x7f3ba0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 --2- 172.21.15.83:0/120263829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ba00051d0 0x7f3ba000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 -- 172.21.15.83:0/120263829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb806e210 con 0x7f3ba0003a30 2024-04-25T00:14:05.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 -- 172.21.15.83:0/120263829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba000c100 con 0x7f3ba0004810 2024-04-25T00:14:05.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bb7e00640 1 -- 172.21.15.83:0/120263829 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba000c300 con 0x7f3ba00051d0 2024-04-25T00:14:05.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bbd400640 1 --2- 172.21.15.83:0/120263829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ba0003a30 0x7f3ba0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bbca00640 1 --2- 172.21.15.83:0/120263829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ba0004810 0x7f3ba0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.600+0000 7f3bbde00640 1 --2- 172.21.15.83:0/120263829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ba00051d0 0x7f3ba000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bbd400640 1 -- 172.21.15.83:0/120263829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ba00051d0 msgr2=0x7f3ba000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:05.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bbd400640 1 --2- 172.21.15.83:0/120263829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ba00051d0 0x7f3ba000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bbd400640 1 -- 172.21.15.83:0/120263829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ba0004810 msgr2=0x7f3ba0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:05.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bbd400640 1 --2- 172.21.15.83:0/120263829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ba0004810 0x7f3ba0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bbd400640 1 -- 172.21.15.83:0/120263829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba0003530 con 0x7f3ba0003a30 2024-04-25T00:14:05.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bbde00640 1 -- 172.21.15.83:0/120263829 reap_dead start 2024-04-25T00:14:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bbd400640 1 --2- 172.21.15.83:0/120263829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ba0003a30 0x7f3ba0003e10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009ac0 tx=0x7f3bac01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb19a4d2d4ca1fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bb7e00640 1 -- 172.21.15.83:0/120263829 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac009690 con 0x7f3ba0003a30 2024-04-25T00:14:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bb7e00640 1 -- 172.21.15.83:0/120263829 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bac0a6430 con 0x7f3ba0003a30 2024-04-25T00:14:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bb7e00640 1 -- 172.21.15.83:0/120263829 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0a6800 con 0x7f3ba0003a30 2024-04-25T00:14:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bbfe00640 1 -- 172.21.15.83:0/120263829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ba0003a30 msgr2=0x7f3ba0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bbfe00640 1 --2- 172.21.15.83:0/120263829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ba0003a30 0x7f3ba0003e10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009ac0 tx=0x7f3bac01ca30 comp rx=0 tx=0).stop 2024-04-25T00:14:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bbfe00640 1 -- 172.21.15.83:0/120263829 shutdown_connections 2024-04-25T00:14:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bbfe00640 1 --2- 172.21.15.83:0/120263829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ba0003a30 0x7f3ba0003e10 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bbfe00640 1 -- 172.21.15.83:0/120263829 >> 172.21.15.83:0/120263829 conn(0x7f3bb80b1830 msgr2=0x7f3bb80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:05.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bbfe00640 1 -- 172.21.15.83:0/120263829 shutdown_connections 2024-04-25T00:14:05.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.604+0000 7f3bbfe00640 1 -- 172.21.15.83:0/120263829 wait complete. 2024-04-25T00:14:05.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbfe00640 1 Processor -- start 2024-04-25T00:14:05.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbfe00640 1 -- start start 2024-04-25T00:14:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ba0003a30 0x7f3bb80ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbfe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bb81450e0 0x7f3bb8148390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb80cf7e0 0x7f3bb80cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbfe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb8070f20 con 0x7f3ba0003a30 2024-04-25T00:14:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb8070da0 con 0x7f3bb80cf7e0 2024-04-25T00:14:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbfe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb80cc8b0 con 0x7f3bb81450e0 2024-04-25T00:14:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ba0003a30 0x7f3bb80ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ba0003a30 0x7f3bb80ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55384/0 (socket says 172.21.15.83:55384) 2024-04-25T00:14:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbd400640 1 -- 172.21.15.83:0/2171031754 learned_addr learned my addr 172.21.15.83:0/2171031754 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbca00640 1 --2- 172.21.15.83:0/2171031754 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bb81450e0 0x7f3bb8148390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbde00640 1 --2- 172.21.15.83:0/2171031754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb80cf7e0 0x7f3bb80cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbd400640 1 -- 172.21.15.83:0/2171031754 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bb81450e0 msgr2=0x7f3bb8148390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbd400640 1 --2- 172.21.15.83:0/2171031754 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bb81450e0 0x7f3bb8148390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbd400640 1 -- 172.21.15.83:0/2171031754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb80cf7e0 msgr2=0x7f3bb80cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbd400640 1 --2- 172.21.15.83:0/2171031754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb80cf7e0 0x7f3bb80cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbd400640 1 -- 172.21.15.83:0/2171031754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb80aece0 con 0x7f3ba0003a30 2024-04-25T00:14:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbd400640 1 --2- 172.21.15.83:0/2171031754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ba0003a30 0x7f3bb80ae600 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f3bac01c860 tx=0x7f3bac0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bb5600640 1 -- 172.21.15.83:0/2171031754 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0a6c60 con 0x7f3ba0003a30 2024-04-25T00:14:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bb5600640 1 -- 172.21.15.83:0/2171031754 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bac0a6e00 con 0x7f3ba0003a30 2024-04-25T00:14:05.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bb5600640 1 -- 172.21.15.83:0/2171031754 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0a7610 con 0x7f3ba0003a30 2024-04-25T00:14:05.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbfe00640 1 -- 172.21.15.83:0/2171031754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb80aefc0 con 0x7f3ba0003a30 2024-04-25T00:14:05.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.608+0000 7f3bbfe00640 1 -- 172.21.15.83:0/2171031754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb8145560 con 0x7f3ba0003a30 2024-04-25T00:14:05.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.612+0000 7f3bb5600640 1 -- 172.21.15.83:0/2171031754 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3bac0c1060 con 0x7f3ba0003a30 2024-04-25T00:14:05.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.612+0000 7f3bb5600640 1 --2- 172.21.15.83:0/2171031754 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3b980678c0 0x7f3b98069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.612+0000 7f3bb5600640 1 -- 172.21.15.83:0/2171031754 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f3bac11fc80 con 0x7f3ba0003a30 2024-04-25T00:14:05.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.612+0000 7f3bbca00640 1 --2- 172.21.15.83:0/2171031754 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3b980678c0 0x7f3b98069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.612+0000 7f3bc32ba000 1 Processor -- start 2024-04-25T00:14:05.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.612+0000 7f3bc32ba000 1 -- start start 2024-04-25T00:14:05.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.612+0000 7f3bbca00640 1 --2- 172.21.15.83:0/2171031754 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3b980678c0 0x7f3b98069d80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8158f40 tx=0x7f3ba4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a64d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a64db40 0x56470a6008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a600e20 0x56470a601200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56470a451830 con 0x56470a64d220 2024-04-25T00:14:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56470a3eee70 con 0x56470a600e20 2024-04-25T00:14:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x56470a44d7f0 con 0x56470a64db40 2024-04-25T00:14:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bbd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a64d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bbd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a64d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55400/0 (socket says 172.21.15.83:55400) 2024-04-25T00:14:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bbd400640 1 -- 172.21.15.83:0/144470815 learned_addr learned my addr 172.21.15.83:0/144470815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bbca00640 1 --2- 172.21.15.83:0/144470815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a64db40 0x56470a6008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bbde00640 1 --2- 172.21.15.83:0/144470815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a600e20 0x56470a601200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bbd400640 1 -- 172.21.15.83:0/144470815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a64db40 msgr2=0x56470a6008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bbd400640 1 --2- 172.21.15.83:0/144470815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a64db40 0x56470a6008e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bbd400640 1 -- 172.21.15.83:0/144470815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a600e20 msgr2=0x56470a601200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bbd400640 1 --2- 172.21.15.83:0/144470815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a600e20 0x56470a601200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bbd400640 1 -- 172.21.15.83:0/144470815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56470a40c150 con 0x56470a64d220 2024-04-25T00:14:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bbd400640 1 --2- 172.21.15.83:0/144470815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a64d600 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f3bac01cfa0 tx=0x7f3bac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d578bd8c7ded77a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3baa000640 1 -- 172.21.15.83:0/144470815 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0efe90 con 0x56470a64d220 2024-04-25T00:14:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3baa000640 1 -- 172.21.15.83:0/144470815 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bac0f4240 con 0x56470a64d220 2024-04-25T00:14:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3baa000640 1 -- 172.21.15.83:0/144470815 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0f8a70 con 0x56470a64d220 2024-04-25T00:14:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 -- 172.21.15.83:0/144470815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 msgr2=0x56470a64d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/144470815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a64d600 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f3bac01cfa0 tx=0x7f3bac009750 comp rx=0 tx=0).stop 2024-04-25T00:14:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 -- 172.21.15.83:0/144470815 shutdown_connections 2024-04-25T00:14:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/144470815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a64db40 0x56470a6008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/144470815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a600e20 0x56470a601200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/144470815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a64d600 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 -- 172.21.15.83:0/144470815 >> 172.21.15.83:0/144470815 conn(0x56470a62ddb0 msgr2=0x56470a6301a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 -- 172.21.15.83:0/144470815 shutdown_connections 2024-04-25T00:14:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 -- 172.21.15.83:0/144470815 wait complete. 2024-04-25T00:14:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 -- start start 2024-04-25T00:14:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.616+0000 7f3bc32ba000 1 Processor -- start 2024-04-25T00:14:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a600e20 0x56470a654580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a654ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a64db40 0x56470a6065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56470a11c680 con 0x56470a64d220 2024-04-25T00:14:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56470a40f0a0 con 0x56470a64db40 2024-04-25T00:14:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x56470a45f8d0 con 0x56470a600e20 2024-04-25T00:14:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 10 client.0 _ll_register_callbacks cb 0x56470a3fe330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bbca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a654ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bbca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a654ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55410/0 (socket says 172.21.15.83:55410) 2024-04-25T00:14:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bbd400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a600e20 0x56470a654580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bbca00640 1 -- 172.21.15.83:0/343169656 learned_addr learned my addr 172.21.15.83:0/343169656 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bbde00640 1 --2- 172.21.15.83:0/343169656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a64db40 0x56470a6065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bbca00640 1 -- 172.21.15.83:0/343169656 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a600e20 msgr2=0x56470a654580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bbca00640 1 --2- 172.21.15.83:0/343169656 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a600e20 0x56470a654580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bbca00640 1 -- 172.21.15.83:0/343169656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a64db40 msgr2=0x56470a6065d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bbca00640 1 --2- 172.21.15.83:0/343169656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a64db40 0x56470a6065d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bbca00640 1 -- 172.21.15.83:0/343169656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56470a3f3ab0 con 0x56470a64d220 2024-04-25T00:14:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bbca00640 1 --2- 172.21.15.83:0/343169656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a654ac0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4001e90 tx=0x7f3bb806f090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83dabead0ccff7cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3b9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:14:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3b9ea00640 1 -- 172.21.15.83:0/343169656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4027060 con 0x56470a64d220 2024-04-25T00:14:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3b9ea00640 1 -- 172.21.15.83:0/343169656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f3ba4022c30 con 0x56470a64d220 2024-04-25T00:14:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 -- 172.21.15.83:0/343169656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bac0f4240 con 0x56470a64d220 2024-04-25T00:14:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3b9ea00640 1 -- 172.21.15.83:0/343169656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3ba401a070 con 0x56470a64d220 2024-04-25T00:14:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 10 client.6852 fetch_fsmap learned FSMap version 44 2024-04-25T00:14:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 -- 172.21.15.83:0/343169656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x56470a4d3230 con 0x56470a64d220 2024-04-25T00:14:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3b9ea00640 1 -- 172.21.15.83:0/343169656 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bac0f4240 con 0x56470a64d220 2024-04-25T00:14:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3b9ea00640 1 -- 172.21.15.83:0/343169656 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f3ba4020070 con 0x56470a64d220 2024-04-25T00:14:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 10 client.6852 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T00:14:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 10 client.6852 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T00:14:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 20 client.6852 populate_metadata read hostname 'smithi083' 2024-04-25T00:14:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/343169656 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x56470a607370 0x56470a607780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 4 client.6852 mds_command: new command op to 6591 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:14:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 -- 172.21.15.83:0/343169656 --> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56470a3eccd0 con 0x56470a607370 2024-04-25T00:14:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bbd400640 1 --2- 172.21.15.83:0/343169656 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x56470a607370 0x56470a607780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bbd400640 1 --2- 172.21.15.83:0/343169656 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x56470a607370 0x56470a607780 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef423a268cea4a2f server_cookie=3e1fedef4bfd18c1 in_seq=0 out_seq=0 2024-04-25T00:14:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3b9ea00640 10 client.6852 ms_handle_connect on v2:172.21.15.83:6835/2879143105 2024-04-25T00:14:05.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3b9ea00640 1 -- 172.21.15.83:0/343169656 <== mds.0 v2:172.21.15.83:6835/2879143105 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56470a3eccd0 con 0x56470a607370 2024-04-25T00:14:05.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3b9ea00640 10 client.6852 handle_command_reply: tid=0 2024-04-25T00:14:05.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 client.6852 shutdown 2024-04-25T00:14:05.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 -- 172.21.15.83:0/343169656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 msgr2=0x56470a654ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/343169656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a654ac0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4001e90 tx=0x7f3bb806f090 comp rx=0 tx=0).stop 2024-04-25T00:14:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 -- 172.21.15.83:0/343169656 shutdown_connections 2024-04-25T00:14:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/343169656 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x56470a607370 0x56470a607780 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/343169656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a64db40 0x56470a6065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/343169656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a654ac0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3b9ea00640 0 client.6852 ms_handle_reset on v2:172.21.15.83:6835/2879143105 2024-04-25T00:14:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/343169656 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a600e20 0x56470a654580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 -- 172.21.15.83:0/343169656 >> 172.21.15.83:0/343169656 conn(0x56470a62ddb0 msgr2=0x56470a6301a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.620+0000 7f3bc32ba000 1 -- 172.21.15.83:0/343169656 shutdown_connections 2024-04-25T00:14:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.624+0000 7f3bc32ba000 1 -- 172.21.15.83:0/343169656 wait complete. 2024-04-25T00:14:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.624+0000 7f3bc32ba000 20 client.6852 trim_cache size 0 max 16384 2024-04-25T00:14:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bc32ba000 1 Processor -- start 2024-04-25T00:14:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bc32ba000 1 -- start start 2024-04-25T00:14:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bc32ba000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a600e20 0x56470a6b9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bc32ba000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a6b9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bc32ba000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a64db40 0x56470a6b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bc32ba000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb82046c0 con 0x56470a64d220 2024-04-25T00:14:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bc32ba000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb8203740 con 0x56470a600e20 2024-04-25T00:14:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bc32ba000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb8147110 con 0x56470a64db40 2024-04-25T00:14:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bbca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a6b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bbca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a6b9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55424/0 (socket says 172.21.15.83:55424) 2024-04-25T00:14:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bbca00640 1 -- 172.21.15.83:0/1956280750 learned_addr learned my addr 172.21.15.83:0/1956280750 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bbde00640 1 --2- 172.21.15.83:0/1956280750 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a64db40 0x56470a6b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bbd400640 1 --2- 172.21.15.83:0/1956280750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a600e20 0x56470a6b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bbca00640 1 -- 172.21.15.83:0/1956280750 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a64db40 msgr2=0x56470a6b9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bbca00640 1 --2- 172.21.15.83:0/1956280750 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a64db40 0x56470a6b9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bbca00640 1 -- 172.21.15.83:0/1956280750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a600e20 msgr2=0x56470a6b9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bbca00640 1 --2- 172.21.15.83:0/1956280750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a600e20 0x56470a6b9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bbca00640 1 -- 172.21.15.83:0/1956280750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56470a4aa550 con 0x56470a64d220 2024-04-25T00:14:05.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bbd400640 1 --2- 172.21.15.83:0/1956280750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a600e20 0x56470a6b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:05.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3bbca00640 1 --2- 172.21.15.83:0/1956280750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a6b9880 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4001980 tx=0x7f3ba4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43966f0f845254e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:05.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3ba9600640 1 -- 172.21.15.83:0/1956280750 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4010c00 con 0x56470a64d220 2024-04-25T00:14:05.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3ba9600640 1 -- 172.21.15.83:0/1956280750 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba4020070 con 0x56470a64d220 2024-04-25T00:14:05.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.640+0000 7f3ba9600640 1 -- 172.21.15.83:0/1956280750 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4020420 con 0x56470a64d220 2024-04-25T00:14:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 -- 172.21.15.83:0/1956280750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 msgr2=0x56470a6b9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/1956280750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a6b9880 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4001980 tx=0x7f3ba4020000 comp rx=0 tx=0).stop 2024-04-25T00:14:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 -- 172.21.15.83:0/1956280750 shutdown_connections 2024-04-25T00:14:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/1956280750 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a64db40 0x56470a6b9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/1956280750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a6b9880 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/1956280750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a600e20 0x56470a6b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 -- 172.21.15.83:0/1956280750 >> 172.21.15.83:0/1956280750 conn(0x56470a62ddb0 msgr2=0x56470a663fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 -- 172.21.15.83:0/1956280750 shutdown_connections 2024-04-25T00:14:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 -- 172.21.15.83:0/1956280750 wait complete. 2024-04-25T00:14:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 -- start start 2024-04-25T00:14:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 Processor -- start 2024-04-25T00:14:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a600e20 0x56470a651490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a6519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a64db40 0x56470a651f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56470a11c680 con 0x56470a64d220 2024-04-25T00:14:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb82030f0 con 0x56470a64db40 2024-04-25T00:14:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x56470a50ddc0 con 0x56470a600e20 2024-04-25T00:14:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 10 client.0 _ll_register_callbacks cb 0x7f3ba4001560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bbca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a6519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bbca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a6519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55430/0 (socket says 172.21.15.83:55430) 2024-04-25T00:14:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bbca00640 1 -- 172.21.15.83:0/1255895818 learned_addr learned my addr 172.21.15.83:0/1255895818 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bbd400640 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a600e20 0x56470a651490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bbde00640 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a64db40 0x56470a651f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bbca00640 1 -- 172.21.15.83:0/1255895818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a600e20 msgr2=0x56470a651490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bbca00640 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a600e20 0x56470a651490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bbca00640 1 -- 172.21.15.83:0/1255895818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a64db40 msgr2=0x56470a651f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bbca00640 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a64db40 0x56470a651f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bbca00640 1 -- 172.21.15.83:0/1255895818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56470a3f81f0 con 0x56470a64d220 2024-04-25T00:14:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bbde00640 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a64db40 0x56470a651f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bbd400640 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a600e20 0x56470a651490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bbca00640 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a6519d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4010630 tx=0x7f3ba4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb51338332d90270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3b9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:14:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3b9ea00640 1 -- 172.21.15.83:0/1255895818 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4010c00 con 0x56470a64d220 2024-04-25T00:14:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3b9ea00640 1 -- 172.21.15.83:0/1255895818 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f3ba4032040 con 0x56470a64d220 2024-04-25T00:14:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 -- 172.21.15.83:0/1255895818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba4020070 con 0x56470a64d220 2024-04-25T00:14:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3b9ea00640 1 -- 172.21.15.83:0/1255895818 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3ba4020cf0 con 0x56470a64d220 2024-04-25T00:14:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 10 client.6858 fetch_fsmap learned FSMap version 44 2024-04-25T00:14:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 -- 172.21.15.83:0/1255895818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x56470a417850 con 0x56470a64d220 2024-04-25T00:14:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3b9ea00640 1 -- 172.21.15.83:0/1255895818 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba4020070 con 0x56470a64d220 2024-04-25T00:14:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3b9ea00640 1 -- 172.21.15.83:0/1255895818 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f3ba402d430 con 0x56470a64d220 2024-04-25T00:14:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 10 client.6858 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T00:14:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 10 client.6858 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T00:14:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 20 client.6858 populate_metadata read hostname 'smithi083' 2024-04-25T00:14:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x56470a652900 0x56470a6c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 4 client.6858 mds_command: new command op to 6591 tid=0 multi_id=0 [{"prefix": "session ls"}] 2024-04-25T00:14:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.644+0000 7f3bc32ba000 1 -- 172.21.15.83:0/1255895818 --> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] -- command(tid 0: {"prefix": "session ls"}) -- 0x56470a45ed70 con 0x56470a652900 2024-04-25T00:14:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bbd400640 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x56470a652900 0x56470a6c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bbd400640 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x56470a652900 0x56470a6c51b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a91b1b9940f7ffe0 server_cookie=d2caded4beaed148 in_seq=0 out_seq=0 2024-04-25T00:14:05.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3b9ea00640 10 client.6858 ms_handle_connect on v2:172.21.15.83:6835/2879143105 2024-04-25T00:14:05.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3b9ea00640 1 -- 172.21.15.83:0/1255895818 <== mds.0 v2:172.21.15.83:6835/2879143105 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56470a45ed70 con 0x56470a652900 2024-04-25T00:14:05.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3b9ea00640 10 client.6858 handle_command_reply: tid=0 2024-04-25T00:14:05.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bc32ba000 1 client.6858 shutdown 2024-04-25T00:14:05.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bc32ba000 1 -- 172.21.15.83:0/1255895818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 msgr2=0x56470a6519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a6519d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4010630 tx=0x7f3ba4014000 comp rx=0 tx=0).stop 2024-04-25T00:14:05.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bc32ba000 1 -- 172.21.15.83:0/1255895818 shutdown_connections 2024-04-25T00:14:05.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.83:6835/2879143105,v1:172.21.15.83:6837/2879143105] conn(0x56470a652900 0x56470a6c51b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56470a64db40 0x56470a651f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56470a64d220 0x56470a6519d0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3b9ea00640 0 client.6858 ms_handle_reset on v2:172.21.15.83:6835/2879143105 2024-04-25T00:14:05.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bc32ba000 1 --2- 172.21.15.83:0/1255895818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56470a600e20 0x56470a651490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bc32ba000 1 -- 172.21.15.83:0/1255895818 >> 172.21.15.83:0/1255895818 conn(0x56470a62ddb0 msgr2=0x56470a5fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:05.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bc32ba000 1 -- 172.21.15.83:0/1255895818 shutdown_connections 2024-04-25T00:14:05.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bc32ba000 1 -- 172.21.15.83:0/1255895818 wait complete. 2024-04-25T00:14:05.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bc32ba000 20 client.6858 trim_cache size 0 max 16384 2024-04-25T00:14:05.655 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-25T00:14:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bbfe00640 1 -- 172.21.15.83:0/2171031754 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3b980678c0 msgr2=0x7f3b98069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bbfe00640 1 --2- 172.21.15.83:0/2171031754 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3b980678c0 0x7f3b98069d80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8158f40 tx=0x7f3ba4009040 comp rx=0 tx=0).stop 2024-04-25T00:14:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bbfe00640 1 -- 172.21.15.83:0/2171031754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ba0003a30 msgr2=0x7f3bb80ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bbfe00640 1 --2- 172.21.15.83:0/2171031754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ba0003a30 0x7f3bb80ae600 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f3bac01c860 tx=0x7f3bac0197d0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bbfe00640 1 -- 172.21.15.83:0/2171031754 shutdown_connections 2024-04-25T00:14:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bbfe00640 1 --2- 172.21.15.83:0/2171031754 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3b980678c0 0x7f3b98069d80 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bbfe00640 1 --2- 172.21.15.83:0/2171031754 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bb81450e0 0x7f3bb8148390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bbfe00640 1 --2- 172.21.15.83:0/2171031754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb80cf7e0 0x7f3bb80cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bbfe00640 1 --2- 172.21.15.83:0/2171031754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ba0003a30 0x7f3bb80ae600 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bbfe00640 1 -- 172.21.15.83:0/2171031754 >> 172.21.15.83:0/2171031754 conn(0x7f3bb80b1830 msgr2=0x7f3bb80b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:05.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bbfe00640 1 -- 172.21.15.83:0/2171031754 shutdown_connections 2024-04-25T00:14:05.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:05.648+0000 7f3bbfe00640 1 -- 172.21.15.83:0/2171031754 wait complete. 2024-04-25T00:14:05.670 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-04-25T00:14:05.827 INFO:teuthology.orchestra.run.smithi083.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":23,"msgr_send_messages":25,"msgr_recv_bytes":6215,"msgr_send_bytes":13540,"msgr_created_connections":7,"msgr_active_connections":2,"msgr_running_total_time":0.009146248,"msgr_running_send_time":0.000536464,"msgr_running_recv_time":0.003908494,"msgr_running_fast_dispatch_time":0.000564766,"msgr_send_messages_queue_lat":{"avgcount":23,"sum":0.189881756,"avgtime":0.008255728},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":103,"msgr_send_messages":93,"msgr_recv_bytes":123903,"msgr_send_bytes":75448,"msgr_created_connections":9,"msgr_active_connections":3,"msgr_running_total_time":0.016475203,"msgr_running_send_time":0.004636748,"msgr_running_recv_time":0.016778421,"msgr_running_fast_dispatch_time":0.000493107,"msgr_send_messages_queue_lat":{"avgcount":93,"sum":0.042203952,"avgtime":0.000453805},"msgr_handle_ack_lat":{"avgcount":60,"sum":0.000010004,"avgtime":0.000000166},"msgr_recv_encrypted_bytes":112688,"msgr_send_encrypted_bytes":12176},"AsyncMessenger::Worker-2":{"msgr_recv_messages":24,"msgr_send_messages":26,"msgr_recv_bytes":6585,"msgr_send_bytes":108811,"msgr_created_connections":6,"msgr_active_connections":3,"msgr_running_total_time":0.006872301,"msgr_running_send_time":0.001027859,"msgr_running_recv_time":0.005200883,"msgr_running_fast_dispatch_time":0.000417946,"msgr_send_messages_queue_lat":{"avgcount":24,"sum":0.031941606,"avgtime":0.001330900},"msgr_handle_ack_lat":{"avgcount":14,"sum":0.000001945,"avgtime":0.000000138},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":94704},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":41,"sum":0.001953214,"avgtime":0.000047639}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000017165,"avgtime":0.000008582}},"mds":{"request":29,"reply":29,"reply_latency":{"avgcount":29,"sum":0.004130285,"avgtime":0.000142423},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":2,"subtrees":2,"traverse":55,"traverse_hit":53,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":8,"evex":0,"evtrm":0,"ev":8,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197029,"rdpos":0,"jlat":{"avgcount":5,"sum":0.011713745,"avgtime":0.002342749},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":0,"cap+":2,"cap-":2,"rss":43816,"heap":231732},"mds_server":{"dispatch_client_request":29,"dispatch_server_request":0,"handle_client_request":29,"handle_client_session":6,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":26,"sum":0.003575379,"avgtime":0.000137514},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":2,"sum":0.000228926,"avgtime":0.000114463},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000325980,"avgtime":0.000325980},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":28,"average_load":14,"avg_session_uptime":2,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":217341,"buffer_anon_items":85,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":53576,"mds_co_items":218,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":43,"op_send_bytes":13845,"op_resend":37,"op_reply":43,"op_latency":{"avgcount":43,"sum":0.711980839,"avgtime":0.016557693},"op_inflight":0,"oplen_avg":{"avgcount":43,"sum":67},"op":43,"op_r":0,"op_w":43,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":71,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":146,"get_sum":119253,"get_or_fail_fail":0,"get_or_fail_success":146,"take":0,"take_sum":0,"put":146,"put_sum":119253,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":13845,"put":43,"put_sum":13845,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":43,"put":43,"put_sum":43,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x556c1d1a8780":{"val":0,"max":3758096384,"get_started":0,"get":8,"get_sum":2725,"get_or_fail_fail":0,"get_or_fail_success":8,"take":0,"take_sum":0,"put":5,"put_sum":2725,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-04-25T00:14:05.828 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 6215, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 23, "msgr_running_fast_dispatch_time": 0.000564766, "msgr_running_recv_time": 0.003908494, "msgr_running_send_time": 0.000536464, "msgr_running_total_time": 0.009146248, "msgr_send_bytes": 13540, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 25, "msgr_send_messages_queue_lat": { "avgcount": 23, "avgtime": 0.008255728, "sum": 0.189881756 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 3, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 60, "avgtime": 1.66e-07, "sum": 1.0004e-05 }, "msgr_recv_bytes": 123903, "msgr_recv_encrypted_bytes": 112688, "msgr_recv_messages": 103, "msgr_running_fast_dispatch_time": 0.000493107, "msgr_running_recv_time": 0.016778421, "msgr_running_send_time": 0.004636748, "msgr_running_total_time": 0.016475203, "msgr_send_bytes": 75448, "msgr_send_encrypted_bytes": 12176, "msgr_send_messages": 93, "msgr_send_messages_queue_lat": { "avgcount": 93, "avgtime": 0.000453805, "sum": 0.042203952 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 14, "avgtime": 1.38e-07, "sum": 1.945e-06 }, "msgr_recv_bytes": 6585, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 24, "msgr_running_fast_dispatch_time": 0.000417946, "msgr_running_recv_time": 0.005200883, "msgr_running_send_time": 0.001027859, "msgr_running_total_time": 0.006872301, "msgr_send_bytes": 108811, "msgr_send_encrypted_bytes": 94704, "msgr_send_messages": 26, "msgr_send_messages_queue_lat": { "avgcount": 24, "avgtime": 0.0013309, "sum": 0.031941606 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 41, "avgtime": 4.7639e-05, "sum": 0.001953214 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 8.582e-06, "sum": 1.7165e-05 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 29, "reply_latency": { "avgcount": 29, "avgtime": 0.000142423, "sum": 0.004130285 }, "request": 29, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 55, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 53, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 8, "evadd": 8, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.002342749, "sum": 0.011713745 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197029 }, "mds_mem": { "cap": 0, "cap+": 2, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 231732, "ino": 13, "ino+": 13, "ino-": 0, "rss": 43816 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 29, "dispatch_server_request": 0, "handle_client_request": 29, "handle_client_session": 6, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 26, "avgtime": 0.000137514, "sum": 0.003575379 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 2, "avgtime": 0.000114463, "sum": 0.000228926 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.00032598, "sum": 0.00032598 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 14, "avg_session_uptime": 2, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 28 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 217341, "buffer_anon_items": 85, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 53576, "mds_co_items": 218, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 71, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 43, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 43, "avgtime": 0.016557693, "sum": 0.711980839 }, "op_pg": 0, "op_r": 0, "op_reply": 43, "op_resend": 37, "op_rmw": 0, "op_send": 43, "op_send_bytes": 13845, "op_w": 43, "oplen_avg": { "avgcount": 43, "sum": 67 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 146, "get_or_fail_fail": 0, "get_or_fail_success": 146, "get_started": 0, "get_sum": 119253, "max": 104857600, "put": 146, "put_sum": 119253, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 43, "put_sum": 13845, "take": 43, "take_sum": 13845, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 43, "put_sum": 43, "take": 43, "take_sum": 43, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x556c1d1a8780": { "get": 8, "get_or_fail_fail": 0, "get_or_fail_success": 8, "get_started": 0, "get_sum": 2725, "max": 3758096384, "put": 5, "put_sum": 2725, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-04-25T00:14:05.828 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T00:14:05.829 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:14:05.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:14:05.916 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:14:06.012 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:14:06.012 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:14:06.012 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:14:06.012 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:06.021 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:14:06.021 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:14:06.021 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:14:06.021 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:06.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:14:06.030 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:14:06.030 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:14:06.030 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:06.069 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:14:06.069 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:14:06.069 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:14:06.069 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:06.080 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:14:06.081 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:06.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.244+0000 7f8894a00640 1 Processor -- start 2024-04-25T00:14:06.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.244+0000 7f8894a00640 1 -- start start 2024-04-25T00:14:06.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.244+0000 7f8894a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f889015f0c0 0x7f88900cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.244+0000 7f8894a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88900d0010 0x7f88900ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.244+0000 7f8894a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88900b3f40 con 0x7f8890144470 2024-04-25T00:14:06.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.244+0000 7f8894a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f88900d0450 con 0x7f889015f0c0 2024-04-25T00:14:06.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.244+0000 7f8894a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8890144850 con 0x7f88900d0010 2024-04-25T00:14:06.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.244+0000 7f888fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88900d0010 0x7f88900ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.244+0000 7f888ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f889015f0c0 0x7f88900cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.244+0000 7f888f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8890144470 0x7f88900b34f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:33266/0 (socket says 172.21.15.83:33266) 2024-04-25T00:14:06.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.244+0000 7f888f400640 1 -- 172.21.15.83:0/1003939132 learned_addr learned my addr 172.21.15.83:0/1003939132 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:06.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.248+0000 7f888ea00640 1 -- 172.21.15.83:0/1003939132 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88900d0010 msgr2=0x7f88900ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:06.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.248+0000 7f888ea00640 1 --2- 172.21.15.83:0/1003939132 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88900d0010 0x7f88900ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.248+0000 7f888ea00640 1 -- 172.21.15.83:0/1003939132 >> v1:172.21.15.86:6789/0 conn(0x7f8890144470 legacy=0x7f88900b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.248+0000 7f888ea00640 1 -- 172.21.15.83:0/1003939132 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88900b4220 con 0x7f889015f0c0 2024-04-25T00:14:06.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.248+0000 7f888fe00640 1 --2- 172.21.15.83:0/1003939132 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88900d0010 0x7f88900ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:06.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.248+0000 7f888ea00640 1 --2- 172.21.15.83:0/1003939132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f889015f0c0 0x7f88900cfad0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f887c0097c0 tx=0x7f887c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4de0040aa4d3e103 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:06.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.248+0000 7f888e000640 1 -- 172.21.15.83:0/1003939132 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c0a7070 con 0x7f889015f0c0 2024-04-25T00:14:06.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.248+0000 7f888e000640 1 -- 172.21.15.83:0/1003939132 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f887c0b0070 con 0x7f889015f0c0 2024-04-25T00:14:06.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.248+0000 7f888e000640 1 -- 172.21.15.83:0/1003939132 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c0ab890 con 0x7f889015f0c0 2024-04-25T00:14:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 -- 172.21.15.83:0/1003939132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f889015f0c0 msgr2=0x7f88900cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 --2- 172.21.15.83:0/1003939132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f889015f0c0 0x7f88900cfad0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f887c0097c0 tx=0x7f887c01c470 comp rx=0 tx=0).stop 2024-04-25T00:14:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 -- 172.21.15.83:0/1003939132 shutdown_connections 2024-04-25T00:14:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 --2- 172.21.15.83:0/1003939132 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88900d0010 0x7f88900ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 --2- 172.21.15.83:0/1003939132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f889015f0c0 0x7f88900cfad0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 -- 172.21.15.83:0/1003939132 >> 172.21.15.83:0/1003939132 conn(0x7f889006b3b0 msgr2=0x7f889006b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 -- 172.21.15.83:0/1003939132 shutdown_connections 2024-04-25T00:14:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 -- 172.21.15.83:0/1003939132 wait complete. 2024-04-25T00:14:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 Processor -- start 2024-04-25T00:14:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 -- start start 2024-04-25T00:14:06.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88900d0010 0x7f8890158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8890144470 0x7f889015a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889015f0c0 0x7f889015a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88900b2ae0 con 0x7f889015f0c0 2024-04-25T00:14:06.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88900b3c00 con 0x7f8890144470 2024-04-25T00:14:06.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f8894a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f88900b2c60 con 0x7f88900d0010 2024-04-25T00:14:06.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f888fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889015f0c0 0x7f889015a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f888fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889015f0c0 0x7f889015a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55434/0 (socket says 172.21.15.83:55434) 2024-04-25T00:14:06.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f888fe00640 1 -- 172.21.15.83:0/3266398612 learned_addr learned my addr 172.21.15.83:0/3266398612 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:06.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f888f400640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88900d0010 0x7f8890158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f888ea00640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8890144470 0x7f889015a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f888fe00640 1 -- 172.21.15.83:0/3266398612 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88900d0010 msgr2=0x7f8890158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f888fe00640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88900d0010 0x7f8890158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f888fe00640 1 -- 172.21.15.83:0/3266398612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8890144470 msgr2=0x7f889015a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f888fe00640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8890144470 0x7f889015a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f888fe00640 1 -- 172.21.15.83:0/3266398612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8890142930 con 0x7f889015f0c0 2024-04-25T00:14:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.252+0000 7f888ea00640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8890144470 0x7f889015a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.256+0000 7f888fe00640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889015f0c0 0x7f889015a7f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f888800efc0 tx=0x7f888800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.256+0000 7f8887400640 1 -- 172.21.15.83:0/3266398612 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888018030 con 0x7f889015f0c0 2024-04-25T00:14:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.256+0000 7f8887400640 1 -- 172.21.15.83:0/3266398612 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f888800cdf0 con 0x7f889015f0c0 2024-04-25T00:14:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.256+0000 7f8894a00640 1 -- 172.21.15.83:0/3266398612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8890142c10 con 0x7f889015f0c0 2024-04-25T00:14:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.256+0000 7f8887400640 1 -- 172.21.15.83:0/3266398612 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888003bc0 con 0x7f889015f0c0 2024-04-25T00:14:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.256+0000 7f8894a00640 1 -- 172.21.15.83:0/3266398612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88901431d0 con 0x7f889015f0c0 2024-04-25T00:14:06.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.256+0000 7f8894a00640 1 -- 172.21.15.83:0/3266398612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889004f510 con 0x7f889015f0c0 2024-04-25T00:14:06.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.256+0000 7f8887400640 1 -- 172.21.15.83:0/3266398612 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f888801b070 con 0x7f889015f0c0 2024-04-25T00:14:06.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.256+0000 7f8887400640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88700678c0 0x7f8870069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.256+0000 7f8887400640 1 -- 172.21.15.83:0/3266398612 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f8888086840 con 0x7f889015f0c0 2024-04-25T00:14:06.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.256+0000 7f888f400640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88700678c0 0x7f8870069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.260+0000 7f888f400640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88700678c0 0x7f8870069d80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8880005890 tx=0x7f8880009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:06.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.264+0000 7f8887400640 1 -- 172.21.15.83:0/3266398612 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8888010040 con 0x7f889015f0c0 2024-04-25T00:14:06.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.452+0000 7f8894a00640 1 -- 172.21.15.83:0/3266398612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f889006b910 con 0x7f889015f0c0 2024-04-25T00:14:06.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.452+0000 7f8887400640 1 -- 172.21.15.83:0/3266398612 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5183 (secure 0 0 0) 0x7f8888056b00 con 0x7f889015f0c0 2024-04-25T00:14:06.454 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:06.455 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2852678498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2852678498},{"type":"v1","addr":"172.21.15.86:6834","nonce":2852678498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2956252746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2956252746},{"type":"v1","addr":"172.21.15.86:6835","nonce":2956252746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6588,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2678835998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2678835998},{"type":"v1","addr":"172.21.15.83:6836","nonce":2678835998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:13:52.273680+0000","modified":"2024-04-25T00:13:55.285817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2879143105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2879143105},{"type":"v1","addr":"172.21.15.83:6837","nonce":2879143105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6591,"qdb_cluster":[6591]},"id":5}]} 2024-04-25T00:14:06.455 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 44 2024-04-25T00:14:06.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.452+0000 7f8894a00640 1 -- 172.21.15.83:0/3266398612 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88700678c0 msgr2=0x7f8870069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.452+0000 7f8894a00640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88700678c0 0x7f8870069d80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8880005890 tx=0x7f8880009040 comp rx=0 tx=0).stop 2024-04-25T00:14:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.452+0000 7f8894a00640 1 -- 172.21.15.83:0/3266398612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889015f0c0 msgr2=0x7f889015a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.452+0000 7f8894a00640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889015f0c0 0x7f889015a7f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f888800efc0 tx=0x7f888800c490 comp rx=0 tx=0).stop 2024-04-25T00:14:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.452+0000 7f8894a00640 1 -- 172.21.15.83:0/3266398612 shutdown_connections 2024-04-25T00:14:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.452+0000 7f8894a00640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f88700678c0 0x7f8870069d80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.452+0000 7f8894a00640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889015f0c0 0x7f889015a7f0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.452+0000 7f8894a00640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8890144470 0x7f889015a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.452+0000 7f8894a00640 1 --2- 172.21.15.83:0/3266398612 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88900d0010 0x7f8890158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.452+0000 7f8894a00640 1 -- 172.21.15.83:0/3266398612 >> 172.21.15.83:0/3266398612 conn(0x7f889006b3b0 msgr2=0x7f8890071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.452+0000 7f8894a00640 1 -- 172.21.15.83:0/3266398612 shutdown_connections 2024-04-25T00:14:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.452+0000 7f8894a00640 1 -- 172.21.15.83:0/3266398612 wait complete. 2024-04-25T00:14:06.476 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:06.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.608+0000 7f9f39a00640 1 Processor -- start 2024-04-25T00:14:06.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.608+0000 7f9f39a00640 1 -- start start 2024-04-25T00:14:06.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.612+0000 7f9f39a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9f34144470 0x7f9f3415f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.612+0000 7f9f39a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f340d01e0 0x7f9f340ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.612+0000 7f9f39a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f340b3f40 con 0x7f9f340b34f0 2024-04-25T00:14:06.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.612+0000 7f9f39a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9f34144850 con 0x7f9f340d01e0 2024-04-25T00:14:06.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.612+0000 7f9f39a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9f340b4220 con 0x7f9f34144470 2024-04-25T00:14:06.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.612+0000 7f9f39000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f340d01e0 0x7f9f340ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.612+0000 7f9f39000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f340d01e0 0x7f9f340ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55450/0 (socket says 172.21.15.83:55450) 2024-04-25T00:14:06.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.612+0000 7f9f39000640 1 -- 172.21.15.83:0/1759475819 learned_addr learned my addr 172.21.15.83:0/1759475819 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:06.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.612+0000 7f9f33e00640 1 --2- 172.21.15.83:0/1759475819 >> v2:172.21.15.86:3300/0 conn(0x7f9f34144470 0x7f9f3415f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.612+0000 7f9f39000640 1 -- 172.21.15.83:0/1759475819 >> v2:172.21.15.86:3300/0 conn(0x7f9f34144470 msgr2=0x7f9f3415f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:06.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.612+0000 7f9f39000640 1 --2- 172.21.15.83:0/1759475819 >> v2:172.21.15.86:3300/0 conn(0x7f9f34144470 0x7f9f3415f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.612+0000 7f9f39000640 1 -- 172.21.15.83:0/1759475819 >> v1:172.21.15.86:6789/0 conn(0x7f9f340b34f0 legacy=0x7f9f340cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:06.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.612+0000 7f9f39000640 1 -- 172.21.15.83:0/1759475819 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3415e5e0 con 0x7f9f340d01e0 2024-04-25T00:14:06.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.612+0000 7f9f33e00640 1 --2- 172.21.15.83:0/1759475819 >> v2:172.21.15.86:3300/0 conn(0x7f9f34144470 0x7f9f3415f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39000640 1 --2- 172.21.15.83:0/1759475819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f340d01e0 0x7f9f340ccbd0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9f340b5040 tx=0x7f9f2c01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f8620755f6e440 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f32a00640 1 -- 172.21.15.83:0/1759475819 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0a57d0 con 0x7f9f340d01e0 2024-04-25T00:14:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f32a00640 1 -- 172.21.15.83:0/1759475819 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2c0a5970 con 0x7f9f340d01e0 2024-04-25T00:14:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f32a00640 1 -- 172.21.15.83:0/1759475819 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0a5d40 con 0x7f9f340d01e0 2024-04-25T00:14:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 -- 172.21.15.83:0/1759475819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f340d01e0 msgr2=0x7f9f340ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 --2- 172.21.15.83:0/1759475819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f340d01e0 0x7f9f340ccbd0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9f340b5040 tx=0x7f9f2c01c8f0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 -- 172.21.15.83:0/1759475819 shutdown_connections 2024-04-25T00:14:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 --2- 172.21.15.83:0/1759475819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f340d01e0 0x7f9f340ccbd0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 --2- 172.21.15.83:0/1759475819 >> v2:172.21.15.86:3300/0 conn(0x7f9f34144470 0x7f9f3415f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 -- 172.21.15.83:0/1759475819 >> 172.21.15.83:0/1759475819 conn(0x7f9f3406b3b0 msgr2=0x7f9f3406b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 -- 172.21.15.83:0/1759475819 shutdown_connections 2024-04-25T00:14:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 -- 172.21.15.83:0/1759475819 wait complete. 2024-04-25T00:14:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 Processor -- start 2024-04-25T00:14:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 -- start start 2024-04-25T00:14:06.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f340b34f0 0x7f9f340b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f340d01e0 0x7f9f34140f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f34144470 0x7f9f34159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f34167860 con 0x7f9f340b34f0 2024-04-25T00:14:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f341676e0 con 0x7f9f34144470 2024-04-25T00:14:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f39a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9f341679e0 con 0x7f9f340d01e0 2024-04-25T00:14:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f33e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f340b34f0 0x7f9f340b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f33e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f340b34f0 0x7f9f340b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55464/0 (socket says 172.21.15.83:55464) 2024-04-25T00:14:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.616+0000 7f9f33e00640 1 -- 172.21.15.83:0/1285600373 learned_addr learned my addr 172.21.15.83:0/1285600373 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f33400640 1 --2- 172.21.15.83:0/1285600373 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f340d01e0 0x7f9f34140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f39000640 1 --2- 172.21.15.83:0/1285600373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f34144470 0x7f9f34159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f33e00640 1 -- 172.21.15.83:0/1285600373 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f340d01e0 msgr2=0x7f9f34140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f33e00640 1 --2- 172.21.15.83:0/1285600373 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f340d01e0 0x7f9f34140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f33e00640 1 -- 172.21.15.83:0/1285600373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f34144470 msgr2=0x7f9f34159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f33e00640 1 --2- 172.21.15.83:0/1285600373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f34144470 0x7f9f34159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f33e00640 1 -- 172.21.15.83:0/1285600373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f340b0fc0 con 0x7f9f340b34f0 2024-04-25T00:14:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f33e00640 1 --2- 172.21.15.83:0/1285600373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f340b34f0 0x7f9f340b08e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400c7f0 tx=0x7f9f2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f2be00640 1 -- 172.21.15.83:0/1285600373 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24015070 con 0x7f9f340b34f0 2024-04-25T00:14:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f2be00640 1 -- 172.21.15.83:0/1285600373 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f24007820 con 0x7f9f340b34f0 2024-04-25T00:14:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f39a00640 1 -- 172.21.15.83:0/1285600373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f340b12a0 con 0x7f9f340b34f0 2024-04-25T00:14:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f2be00640 1 -- 172.21.15.83:0/1285600373 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24007c10 con 0x7f9f340b34f0 2024-04-25T00:14:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f39a00640 1 -- 172.21.15.83:0/1285600373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f340b1860 con 0x7f9f340b34f0 2024-04-25T00:14:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f39a00640 1 -- 172.21.15.83:0/1285600373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3404f510 con 0x7f9f340b34f0 2024-04-25T00:14:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f2be00640 1 -- 172.21.15.83:0/1285600373 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9f24018070 con 0x7f9f340b34f0 2024-04-25T00:14:06.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f2be00640 1 --2- 172.21.15.83:0/1285600373 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9f140678c0 0x7f9f14069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f2be00640 1 -- 172.21.15.83:0/1285600373 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f9f24086060 con 0x7f9f340b34f0 2024-04-25T00:14:06.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.620+0000 7f9f33400640 1 --2- 172.21.15.83:0/1285600373 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9f140678c0 0x7f9f14069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.624+0000 7f9f33400640 1 --2- 172.21.15.83:0/1285600373 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9f140678c0 0x7f9f14069d80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9f341fa4b0 tx=0x7f9f20009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:06.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.624+0000 7f9f2be00640 1 -- 172.21.15.83:0/1285600373 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9f24056970 con 0x7f9f340b34f0 2024-04-25T00:14:06.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.784+0000 7f9f39a00640 1 -- 172.21.15.83:0/1285600373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f3406ceb0 con 0x7f9f340b34f0 2024-04-25T00:14:06.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.784+0000 7f9f2be00640 1 -- 172.21.15.83:0/1285600373 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16861 (secure 0 0 0) 0x7f9f24056270 con 0x7f9f340b34f0 2024-04-25T00:14:06.787 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:06.787 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":71,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:53.896317+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:50.655187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:51.268160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.784+0000 7f9f39a00640 1 -- 172.21.15.83:0/1285600373 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9f140678c0 msgr2=0x7f9f14069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.784+0000 7f9f39a00640 1 --2- 172.21.15.83:0/1285600373 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9f140678c0 0x7f9f14069d80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9f341fa4b0 tx=0x7f9f20009040 comp rx=0 tx=0).stop 2024-04-25T00:14:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.784+0000 7f9f39a00640 1 -- 172.21.15.83:0/1285600373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f340b34f0 msgr2=0x7f9f340b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.784+0000 7f9f39a00640 1 --2- 172.21.15.83:0/1285600373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f340b34f0 0x7f9f340b08e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400c7f0 tx=0x7f9f2400ccb0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.784+0000 7f9f39a00640 1 -- 172.21.15.83:0/1285600373 shutdown_connections 2024-04-25T00:14:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.784+0000 7f9f39a00640 1 --2- 172.21.15.83:0/1285600373 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9f140678c0 0x7f9f14069d80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.784+0000 7f9f39a00640 1 --2- 172.21.15.83:0/1285600373 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f340d01e0 0x7f9f34140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.784+0000 7f9f39a00640 1 --2- 172.21.15.83:0/1285600373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f34144470 0x7f9f34159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.784+0000 7f9f39a00640 1 --2- 172.21.15.83:0/1285600373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f340b34f0 0x7f9f340b08e0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.784+0000 7f9f39a00640 1 -- 172.21.15.83:0/1285600373 >> 172.21.15.83:0/1285600373 conn(0x7f9f3406b3b0 msgr2=0x7f9f3406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:06.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.784+0000 7f9f39a00640 1 -- 172.21.15.83:0/1285600373 shutdown_connections 2024-04-25T00:14:06.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.784+0000 7f9f39a00640 1 -- 172.21.15.83:0/1285600373 wait complete. 2024-04-25T00:14:06.807 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:50.655187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:13:51.268160+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:14:06.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:14:06.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.940+0000 7feedd800640 1 Processor -- start 2024-04-25T00:14:06.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feedd800640 1 -- start start 2024-04-25T00:14:06.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feedd800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feed815e270 0x7feed80b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feedd800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feed8071e50 0x7feed80b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feedd800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed806dcf0 con 0x7feed815de90 2024-04-25T00:14:06.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feedd800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7feed80b2ce0 con 0x7feed8071e50 2024-04-25T00:14:06.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feedd800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7feed806dfd0 con 0x7feed815e270 2024-04-25T00:14:06.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feece000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feed815e270 0x7feed80b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feece000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feed815e270 0x7feed80b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:50878/0 (socket says 172.21.15.83:50878) 2024-04-25T00:14:06.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feece000640 1 -- 172.21.15.83:0/359918032 learned_addr learned my addr 172.21.15.83:0/359918032 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:06.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feed7400640 1 --2- 172.21.15.83:0/359918032 >> v2:172.21.15.86:3300/0 conn(0x7feed8071e50 0x7feed80b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feece000640 1 -- 172.21.15.83:0/359918032 >> v2:172.21.15.86:3300/0 conn(0x7feed8071e50 msgr2=0x7feed80b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:06.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feed6000640 1 -- 172.21.15.83:0/359918032 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 525432596 0 0) 0x7feed806dcf0 con 0x7feed815de90 2024-04-25T00:14:06.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feece000640 1 --2- 172.21.15.83:0/359918032 >> v2:172.21.15.86:3300/0 conn(0x7feed8071e50 0x7feed80b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feece000640 1 -- 172.21.15.83:0/359918032 >> v1:172.21.15.86:6789/0 conn(0x7feed815de90 legacy=0x7feed8071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.944+0000 7feece000640 1 -- 172.21.15.83:0/359918032 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed806e2b0 con 0x7feed815e270 2024-04-25T00:14:06.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.948+0000 7feece000640 1 --2- 172.21.15.83:0/359918032 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feed815e270 0x7feed80b3450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feec0009870 tx=0x7feec001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=640ccf64f9d65111 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:06.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.948+0000 7feed6000640 1 -- 172.21.15.83:0/359918032 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00b2070 con 0x7feed815e270 2024-04-25T00:14:06.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.948+0000 7feed6000640 1 -- 172.21.15.83:0/359918032 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec00a9070 con 0x7feed815e270 2024-04-25T00:14:06.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.948+0000 7feed6000640 1 -- 172.21.15.83:0/359918032 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00ad880 con 0x7feed815e270 2024-04-25T00:14:06.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.948+0000 7feedd800640 1 -- 172.21.15.83:0/359918032 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feed815e270 msgr2=0x7feed80b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.948+0000 7feedd800640 1 --2- 172.21.15.83:0/359918032 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feed815e270 0x7feed80b3450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feec0009870 tx=0x7feec001c4d0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.948+0000 7feedd800640 1 -- 172.21.15.83:0/359918032 shutdown_connections 2024-04-25T00:14:06.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.948+0000 7feedd800640 1 --2- 172.21.15.83:0/359918032 >> v2:172.21.15.86:3300/0 conn(0x7feed8071e50 0x7feed80b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.948+0000 7feedd800640 1 --2- 172.21.15.83:0/359918032 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feed815e270 0x7feed80b3450 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.948+0000 7feedd800640 1 -- 172.21.15.83:0/359918032 >> 172.21.15.83:0/359918032 conn(0x7feed80b1830 msgr2=0x7feed80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:06.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.948+0000 7feedd800640 1 -- 172.21.15.83:0/359918032 shutdown_connections 2024-04-25T00:14:06.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.948+0000 7feedd800640 1 -- 172.21.15.83:0/359918032 wait complete. 2024-04-25T00:14:06.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feedd800640 1 Processor -- start 2024-04-25T00:14:06.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feedd800640 1 -- start start 2024-04-25T00:14:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feedd800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feed8071e50 0x7feed81445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feedd800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feed815de90 0x7feed81524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feedd800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed815e270 0x7feed80af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feedd800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feed80b4050 con 0x7feed815de90 2024-04-25T00:14:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feedd800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feed80b3ed0 con 0x7feed815e270 2024-04-25T00:14:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feedd800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7feed80b41d0 con 0x7feed8071e50 2024-04-25T00:14:06.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feece000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feed815de90 0x7feed81524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feece000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feed815de90 0x7feed81524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55478/0 (socket says 172.21.15.83:55478) 2024-04-25T00:14:06.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feece000640 1 -- 172.21.15.83:0/3602383715 learned_addr learned my addr 172.21.15.83:0/3602383715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:06.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feed7400640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed815e270 0x7feed80af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feece000640 1 -- 172.21.15.83:0/3602383715 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feed8071e50 msgr2=0x7feed81445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feed6a00640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feed8071e50 0x7feed81445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feece000640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feed8071e50 0x7feed81445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feece000640 1 -- 172.21.15.83:0/3602383715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed815e270 msgr2=0x7feed80af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feece000640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed815e270 0x7feed80af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feece000640 1 -- 172.21.15.83:0/3602383715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed8152ed0 con 0x7feed815de90 2024-04-25T00:14:06.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feed6a00640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feed8071e50 0x7feed81445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:06.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feed7400640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed815e270 0x7feed80af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:06.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feece000640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feed815de90 0x7feed81524f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7feec0009870 tx=0x7feec001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feecf400640 1 -- 172.21.15.83:0/3602383715 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00bc070 con 0x7feed815de90 2024-04-25T00:14:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feecf400640 1 -- 172.21.15.83:0/3602383715 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec00add00 con 0x7feed815de90 2024-04-25T00:14:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feecf400640 1 -- 172.21.15.83:0/3602383715 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00b2040 con 0x7feed815de90 2024-04-25T00:14:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feedd800640 1 -- 172.21.15.83:0/3602383715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed81531b0 con 0x7feed815de90 2024-04-25T00:14:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feedd800640 1 -- 172.21.15.83:0/3602383715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed8153770 con 0x7feed815de90 2024-04-25T00:14:06.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.952+0000 7feedd800640 1 -- 172.21.15.83:0/3602383715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed804f510 con 0x7feed815de90 2024-04-25T00:14:06.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.956+0000 7feecf400640 1 -- 172.21.15.83:0/3602383715 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feec00ab070 con 0x7feed815de90 2024-04-25T00:14:06.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.956+0000 7feecf400640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feeb40678c0 0x7feeb4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.956+0000 7feecf400640 1 -- 172.21.15.83:0/3602383715 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7feec011fd30 con 0x7feed815de90 2024-04-25T00:14:06.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.956+0000 7feed6a00640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feeb40678c0 0x7feeb4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.956+0000 7feed6a00640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feeb40678c0 0x7feeb4069d80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feec8007fd0 tx=0x7feec8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:06.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:06.960+0000 7feecf400640 1 -- 172.21.15.83:0/3602383715 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7feec0016150 con 0x7feed815de90 2024-04-25T00:14:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.116+0000 7feedd800640 1 -- 172.21.15.83:0/3602383715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feed80719c0 con 0x7feed815de90 2024-04-25T00:14:07.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.116+0000 7feecf400640 1 -- 172.21.15.83:0/3602383715 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7feec00ab360 con 0x7feed815de90 2024-04-25T00:14:07.119 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:14:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.116+0000 7feedd800640 1 -- 172.21.15.83:0/3602383715 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feeb40678c0 msgr2=0x7feeb4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.116+0000 7feedd800640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feeb40678c0 0x7feeb4069d80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feec8007fd0 tx=0x7feec8009040 comp rx=0 tx=0).stop 2024-04-25T00:14:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.116+0000 7feedd800640 1 -- 172.21.15.83:0/3602383715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feed815de90 msgr2=0x7feed81524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.116+0000 7feedd800640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feed815de90 0x7feed81524f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7feec0009870 tx=0x7feec001ad10 comp rx=0 tx=0).stop 2024-04-25T00:14:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.116+0000 7feedd800640 1 -- 172.21.15.83:0/3602383715 shutdown_connections 2024-04-25T00:14:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.116+0000 7feedd800640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feeb40678c0 0x7feeb4069d80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.116+0000 7feedd800640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed815e270 0x7feed80af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.116+0000 7feedd800640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feed815de90 0x7feed81524f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.116+0000 7feedd800640 1 --2- 172.21.15.83:0/3602383715 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feed8071e50 0x7feed81445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.116+0000 7feedd800640 1 -- 172.21.15.83:0/3602383715 >> 172.21.15.83:0/3602383715 conn(0x7feed80b1830 msgr2=0x7feed80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.116+0000 7feedd800640 1 -- 172.21.15.83:0/3602383715 shutdown_connections 2024-04-25T00:14:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.116+0000 7feedd800640 1 -- 172.21.15.83:0/3602383715 wait complete. 2024-04-25T00:14:07.140 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:14:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3ea5200640 1 Processor -- start 2024-04-25T00:14:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3ea5200640 1 -- start start 2024-04-25T00:14:07.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3ea5200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ea0159810 0x7f3ea00cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3ea5200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3ea006e950 0x7f3ea015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3ea5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3ea015fe50 0x7f3ea015c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3ea5200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3ea006e210 con 0x7f3ea006e950 2024-04-25T00:14:07.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3ea5200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3ea006da00 con 0x7f3ea015fe50 2024-04-25T00:14:07.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3ea5200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea00ce000 con 0x7f3ea0159810 2024-04-25T00:14:07.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3e95400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3ea006e950 0x7f3ea015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3e95400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3ea006e950 0x7f3ea015f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55494/0 (socket says 172.21.15.83:55494) 2024-04-25T00:14:07.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3e95400640 1 -- 172.21.15.83:0/2461474620 learned_addr learned my addr 172.21.15.83:0/2461474620 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:07.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3e9e800640 1 --2- 172.21.15.83:0/2461474620 >> v2:172.21.15.86:3300/0 conn(0x7f3ea015fe50 0x7f3ea015c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3e9de00640 1 --2- 172.21.15.83:0/2461474620 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ea0159810 0x7f3ea00cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3e95400640 1 -- 172.21.15.83:0/2461474620 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ea0159810 msgr2=0x7f3ea00cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:07.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3e95400640 1 --2- 172.21.15.83:0/2461474620 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ea0159810 0x7f3ea00cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3e95400640 1 -- 172.21.15.83:0/2461474620 >> v2:172.21.15.86:3300/0 conn(0x7f3ea015fe50 msgr2=0x7f3ea015c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:07.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3e95400640 1 --2- 172.21.15.83:0/2461474620 >> v2:172.21.15.86:3300/0 conn(0x7f3ea015fe50 0x7f3ea015c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3e95400640 1 -- 172.21.15.83:0/2461474620 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea006eec0 con 0x7f3ea006e950 2024-04-25T00:14:07.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.276+0000 7f3e9e800640 1 --2- 172.21.15.83:0/2461474620 >> v2:172.21.15.86:3300/0 conn(0x7f3ea015fe50 0x7f3ea015c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:07.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.280+0000 7f3e95400640 1 --2- 172.21.15.83:0/2461474620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea006e950 0x7f3ea015f910 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00cdb40 tx=0x7f3e8801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3722fea2d894c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:07.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.280+0000 7f3e9d400640 1 -- 172.21.15.83:0/2461474620 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e880b0070 con 0x7f3ea006e950 2024-04-25T00:14:07.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.280+0000 7f3e9d400640 1 -- 172.21.15.83:0/2461474620 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e880ab600 con 0x7f3ea006e950 2024-04-25T00:14:07.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.280+0000 7f3e9d400640 1 -- 172.21.15.83:0/2461474620 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e880ab9d0 con 0x7f3ea006e950 2024-04-25T00:14:07.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.280+0000 7f3ea5200640 1 -- 172.21.15.83:0/2461474620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea006e950 msgr2=0x7f3ea015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.280+0000 7f3ea5200640 1 --2- 172.21.15.83:0/2461474620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea006e950 0x7f3ea015f910 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00cdb40 tx=0x7f3e8801c490 comp rx=0 tx=0).stop 2024-04-25T00:14:07.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.280+0000 7f3ea5200640 1 -- 172.21.15.83:0/2461474620 shutdown_connections 2024-04-25T00:14:07.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.280+0000 7f3ea5200640 1 --2- 172.21.15.83:0/2461474620 >> v2:172.21.15.86:3300/0 conn(0x7f3ea015fe50 0x7f3ea015c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.280+0000 7f3ea5200640 1 --2- 172.21.15.83:0/2461474620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea006e950 0x7f3ea015f910 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.280+0000 7f3ea5200640 1 --2- 172.21.15.83:0/2461474620 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ea0159810 0x7f3ea00cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.280+0000 7f3ea5200640 1 -- 172.21.15.83:0/2461474620 >> 172.21.15.83:0/2461474620 conn(0x7f3ea00b1830 msgr2=0x7f3ea00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:07.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.280+0000 7f3ea5200640 1 -- 172.21.15.83:0/2461474620 shutdown_connections 2024-04-25T00:14:07.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.280+0000 7f3ea5200640 1 -- 172.21.15.83:0/2461474620 wait complete. 2024-04-25T00:14:07.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3ea5200640 1 Processor -- start 2024-04-25T00:14:07.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3ea5200640 1 -- start start 2024-04-25T00:14:07.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3ea5200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ea006e950 0x7f3ea0165880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3ea5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ea0159810 0x7f3ea01511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3ea5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea015fe50 0x7f3ea01516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3ea5200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0159d70 con 0x7f3ea015fe50 2024-04-25T00:14:07.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3ea5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0159bf0 con 0x7f3ea0159810 2024-04-25T00:14:07.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3ea5200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea0070da0 con 0x7f3ea006e950 2024-04-25T00:14:07.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3e9e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea015fe50 0x7f3ea01516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3e9de00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ea006e950 0x7f3ea0165880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3e9e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea015fe50 0x7f3ea01516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55510/0 (socket says 172.21.15.83:55510) 2024-04-25T00:14:07.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3e9e800640 1 -- 172.21.15.83:0/3788040384 learned_addr learned my addr 172.21.15.83:0/3788040384 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:07.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3e95400640 1 --2- 172.21.15.83:0/3788040384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ea0159810 0x7f3ea01511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3e9e800640 1 -- 172.21.15.83:0/3788040384 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ea006e950 msgr2=0x7f3ea0165880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3e9e800640 1 --2- 172.21.15.83:0/3788040384 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ea006e950 0x7f3ea0165880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3e9e800640 1 -- 172.21.15.83:0/3788040384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ea0159810 msgr2=0x7f3ea01511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3e9e800640 1 --2- 172.21.15.83:0/3788040384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ea0159810 0x7f3ea01511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3e9e800640 1 -- 172.21.15.83:0/3788040384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea01581f0 con 0x7f3ea015fe50 2024-04-25T00:14:07.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3e9e800640 1 --2- 172.21.15.83:0/3788040384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea015fe50 0x7f3ea01516e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3e98007e70 tx=0x7f3e9800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3e96a00640 1 -- 172.21.15.83:0/3788040384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9801a070 con 0x7f3ea015fe50 2024-04-25T00:14:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3e96a00640 1 -- 172.21.15.83:0/3788040384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e98003bc0 con 0x7f3ea015fe50 2024-04-25T00:14:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3ea5200640 1 -- 172.21.15.83:0/3788040384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea01584d0 con 0x7f3ea015fe50 2024-04-25T00:14:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3e96a00640 1 -- 172.21.15.83:0/3788040384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98004030 con 0x7f3ea015fe50 2024-04-25T00:14:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3ea5200640 1 -- 172.21.15.83:0/3788040384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea0158a90 con 0x7f3ea015fe50 2024-04-25T00:14:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.284+0000 7f3ea5200640 1 -- 172.21.15.83:0/3788040384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea004f510 con 0x7f3ea015fe50 2024-04-25T00:14:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.288+0000 7f3e96a00640 1 -- 172.21.15.83:0/3788040384 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3e9801d070 con 0x7f3ea015fe50 2024-04-25T00:14:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.288+0000 7f3e96a00640 1 --2- 172.21.15.83:0/3788040384 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e7c0678c0 0x7f3e7c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.288+0000 7f3e96a00640 1 -- 172.21.15.83:0/3788040384 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f3e98085fe0 con 0x7f3ea015fe50 2024-04-25T00:14:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.288+0000 7f3e9de00640 1 --2- 172.21.15.83:0/3788040384 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e7c0678c0 0x7f3e7c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.288+0000 7f3e9de00640 1 --2- 172.21.15.83:0/3788040384 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e7c0678c0 0x7f3e7c069d80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3ea01fa2a0 tx=0x7f3e90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.292+0000 7f3e96a00640 1 -- 172.21.15.83:0/3788040384 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3e98010070 con 0x7f3ea015fe50 2024-04-25T00:14:07.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.444+0000 7f3ea5200640 1 -- 172.21.15.83:0/3788040384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3ea00cc280 con 0x7f3ea015fe50 2024-04-25T00:14:07.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.444+0000 7f3e96a00640 1 -- 172.21.15.83:0/3788040384 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7f3e980562e0 con 0x7f3ea015fe50 2024-04-25T00:14:07.449 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:14:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.448+0000 7f3ea5200640 1 -- 172.21.15.83:0/3788040384 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e7c0678c0 msgr2=0x7f3e7c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.448+0000 7f3ea5200640 1 --2- 172.21.15.83:0/3788040384 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e7c0678c0 0x7f3e7c069d80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3ea01fa2a0 tx=0x7f3e90008040 comp rx=0 tx=0).stop 2024-04-25T00:14:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.448+0000 7f3ea5200640 1 -- 172.21.15.83:0/3788040384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea015fe50 msgr2=0x7f3ea01516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.448+0000 7f3ea5200640 1 --2- 172.21.15.83:0/3788040384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea015fe50 0x7f3ea01516e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3e98007e70 tx=0x7f3e9800d4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.448+0000 7f3ea5200640 1 -- 172.21.15.83:0/3788040384 shutdown_connections 2024-04-25T00:14:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.448+0000 7f3ea5200640 1 --2- 172.21.15.83:0/3788040384 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e7c0678c0 0x7f3e7c069d80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.448+0000 7f3ea5200640 1 --2- 172.21.15.83:0/3788040384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea015fe50 0x7f3ea01516e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.448+0000 7f3ea5200640 1 --2- 172.21.15.83:0/3788040384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ea0159810 0x7f3ea01511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.448+0000 7f3ea5200640 1 --2- 172.21.15.83:0/3788040384 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ea006e950 0x7f3ea0165880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.448+0000 7f3ea5200640 1 -- 172.21.15.83:0/3788040384 >> 172.21.15.83:0/3788040384 conn(0x7f3ea00b1830 msgr2=0x7f3ea0071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.448+0000 7f3ea5200640 1 -- 172.21.15.83:0/3788040384 shutdown_connections 2024-04-25T00:14:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.448+0000 7f3ea5200640 1 -- 172.21.15.83:0/3788040384 wait complete. 2024-04-25T00:14:07.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:14:07.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.604+0000 7f2dd5800640 1 Processor -- start 2024-04-25T00:14:07.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.604+0000 7f2dd5800640 1 -- start start 2024-04-25T00:14:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.604+0000 7f2dd5800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2dd006e950 0x7f2dd015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.604+0000 7f2dd5800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd015fe50 0x7f2dd00ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.604+0000 7f2dd5800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dd00ce000 con 0x7f2dd0159810 2024-04-25T00:14:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.604+0000 7f2dd5800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2dd006e210 con 0x7f2dd006e950 2024-04-25T00:14:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.604+0000 7f2dd5800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2dd006da00 con 0x7f2dd015fe50 2024-04-25T00:14:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.608+0000 7f2dce000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2dd006e950 0x7f2dd015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.608+0000 7f2dcea00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2dd0159810 0x7f2dd00cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46238/0 (socket says 172.21.15.83:46238) 2024-04-25T00:14:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.608+0000 7f2dcea00640 1 -- 172.21.15.83:0/2369517347 learned_addr learned my addr 172.21.15.83:0/2369517347 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:07.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.608+0000 7f2dcf400640 1 --2- 172.21.15.83:0/2369517347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd015fe50 0x7f2dd00ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.608+0000 7f2dce000640 1 -- 172.21.15.83:0/2369517347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd015fe50 msgr2=0x7f2dd00ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:07.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.608+0000 7f2dce000640 1 --2- 172.21.15.83:0/2369517347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd015fe50 0x7f2dd00ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.608+0000 7f2dce000640 1 -- 172.21.15.83:0/2369517347 >> v1:172.21.15.83:6789/0 conn(0x7f2dd0159810 legacy=0x7f2dd00cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.608+0000 7f2dce000640 1 -- 172.21.15.83:0/2369517347 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd006eec0 con 0x7f2dd006e950 2024-04-25T00:14:07.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.608+0000 7f2dce000640 1 --2- 172.21.15.83:0/2369517347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd006e950 0x7f2dd015f910 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2dd006db80 tx=0x7f2dbc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16b15ec752eccfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:07.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.608+0000 7f2dcd600640 1 -- 172.21.15.83:0/2369517347 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0b0070 con 0x7f2dd006e950 2024-04-25T00:14:07.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.608+0000 7f2dcd600640 1 -- 172.21.15.83:0/2369517347 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dbc0ab600 con 0x7f2dd006e950 2024-04-25T00:14:07.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.608+0000 7f2dcd600640 1 -- 172.21.15.83:0/2369517347 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0ab9d0 con 0x7f2dd006e950 2024-04-25T00:14:07.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 -- 172.21.15.83:0/2369517347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd006e950 msgr2=0x7f2dd015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 --2- 172.21.15.83:0/2369517347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd006e950 0x7f2dd015f910 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2dd006db80 tx=0x7f2dbc01c410 comp rx=0 tx=0).stop 2024-04-25T00:14:07.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 -- 172.21.15.83:0/2369517347 shutdown_connections 2024-04-25T00:14:07.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 --2- 172.21.15.83:0/2369517347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd015fe50 0x7f2dd00ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 --2- 172.21.15.83:0/2369517347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd006e950 0x7f2dd015f910 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 -- 172.21.15.83:0/2369517347 >> 172.21.15.83:0/2369517347 conn(0x7f2dd00b1830 msgr2=0x7f2dd00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:07.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 -- 172.21.15.83:0/2369517347 shutdown_connections 2024-04-25T00:14:07.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 -- 172.21.15.83:0/2369517347 wait complete. 2024-04-25T00:14:07.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 Processor -- start 2024-04-25T00:14:07.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 -- start start 2024-04-25T00:14:07.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd006e950 0x7f2dd01446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd0159810 0x7f2dd0155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd015fe50 0x7f2dd0156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0070f20 con 0x7f2dd0159810 2024-04-25T00:14:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0070da0 con 0x7f2dd015fe50 2024-04-25T00:14:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dd5800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2dd0168cb0 con 0x7f2dd006e950 2024-04-25T00:14:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dce000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd0159810 0x7f2dd0155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dce000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd0159810 0x7f2dd0155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55536/0 (socket says 172.21.15.83:55536) 2024-04-25T00:14:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dce000640 1 -- 172.21.15.83:0/3968938139 learned_addr learned my addr 172.21.15.83:0/3968938139 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dcea00640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd006e950 0x7f2dd01446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dce000640 1 -- 172.21.15.83:0/3968938139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd006e950 msgr2=0x7f2dd01446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dce000640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd006e950 0x7f2dd01446b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dce000640 1 -- 172.21.15.83:0/3968938139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd015fe50 msgr2=0x7f2dd0156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dcf400640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd015fe50 0x7f2dd0156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dce000640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd015fe50 0x7f2dd0156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dce000640 1 -- 172.21.15.83:0/3968938139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd015b730 con 0x7f2dd0159810 2024-04-25T00:14:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dcea00640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd006e950 0x7f2dd01446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:07.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dcf400640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd015fe50 0x7f2dd0156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:07.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dce000640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd0159810 0x7f2dd0155fc0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0ae040 tx=0x7f2dbc0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:07.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.612+0000 7f2dc6a00640 1 -- 172.21.15.83:0/3968938139 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0b8020 con 0x7f2dd0159810 2024-04-25T00:14:07.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.616+0000 7f2dc6a00640 1 -- 172.21.15.83:0/3968938139 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dbc0b4410 con 0x7f2dd0159810 2024-04-25T00:14:07.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.616+0000 7f2dc6a00640 1 -- 172.21.15.83:0/3968938139 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0b0040 con 0x7f2dd0159810 2024-04-25T00:14:07.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.616+0000 7f2dd5800640 1 -- 172.21.15.83:0/3968938139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd015abb0 con 0x7f2dd0159810 2024-04-25T00:14:07.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.616+0000 7f2dd5800640 1 -- 172.21.15.83:0/3968938139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd015b170 con 0x7f2dd0159810 2024-04-25T00:14:07.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.616+0000 7f2dd5800640 1 -- 172.21.15.83:0/3968938139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd004f510 con 0x7f2dd0159810 2024-04-25T00:14:07.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.616+0000 7f2dc6a00640 1 -- 172.21.15.83:0/3968938139 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2dbc0bb020 con 0x7f2dd0159810 2024-04-25T00:14:07.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.616+0000 7f2dc6a00640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2db00678c0 0x7f2db0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.616+0000 7f2dc6a00640 1 -- 172.21.15.83:0/3968938139 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f2dbc11fa60 con 0x7f2dd0159810 2024-04-25T00:14:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.616+0000 7f2dcea00640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2db00678c0 0x7f2db0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.620+0000 7f2dcea00640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2db00678c0 0x7f2db0069d80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2dd00728f0 tx=0x7f2dc0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:07.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.620+0000 7f2dc6a00640 1 -- 172.21.15.83:0/3968938139 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2dbc0c2030 con 0x7f2dd0159810 2024-04-25T00:14:07.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.776+0000 7f2dd5800640 1 -- 172.21.15.83:0/3968938139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2dd0072c90 con 0x7f2dd0159810 2024-04-25T00:14:07.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.776+0000 7f2dc6a00640 1 -- 172.21.15.83:0/3968938139 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7f2dbc016090 con 0x7f2dd0159810 2024-04-25T00:14:07.781 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:14:07.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.780+0000 7f2dd5800640 1 -- 172.21.15.83:0/3968938139 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2db00678c0 msgr2=0x7f2db0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.780+0000 7f2dd5800640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2db00678c0 0x7f2db0069d80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2dd00728f0 tx=0x7f2dc0009040 comp rx=0 tx=0).stop 2024-04-25T00:14:07.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.780+0000 7f2dd5800640 1 -- 172.21.15.83:0/3968938139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd0159810 msgr2=0x7f2dd0155fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.780+0000 7f2dd5800640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd0159810 0x7f2dd0155fc0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0ae040 tx=0x7f2dbc0191b0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.780+0000 7f2dd5800640 1 -- 172.21.15.83:0/3968938139 shutdown_connections 2024-04-25T00:14:07.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.780+0000 7f2dd5800640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2db00678c0 0x7f2db0069d80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.780+0000 7f2dd5800640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd015fe50 0x7f2dd0156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.780+0000 7f2dd5800640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd0159810 0x7f2dd0155fc0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.780+0000 7f2dd5800640 1 --2- 172.21.15.83:0/3968938139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd006e950 0x7f2dd01446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.780+0000 7f2dd5800640 1 -- 172.21.15.83:0/3968938139 >> 172.21.15.83:0/3968938139 conn(0x7f2dd00b1830 msgr2=0x7f2dd00cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:07.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.780+0000 7f2dd5800640 1 -- 172.21.15.83:0/3968938139 shutdown_connections 2024-04-25T00:14:07.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.780+0000 7f2dd5800640 1 -- 172.21.15.83:0/3968938139 wait complete. 2024-04-25T00:14:07.802 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:07.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff223800640 1 Processor -- start 2024-04-25T00:14:07.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff223800640 1 -- start start 2024-04-25T00:14:07.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff223800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff21c0cfe80 0x7ff21c072520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff223800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff21c0d0260 0x7ff21c15ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff223800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff21c15e590 con 0x7ff21c15f2b0 2024-04-25T00:14:07.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff223800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff21c155530 con 0x7ff21c0d0260 2024-04-25T00:14:07.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff223800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff21c0afe10 con 0x7ff21c0cfe80 2024-04-25T00:14:07.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff221800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff21c15f2b0 0x7ff21c06b1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46246/0 (socket says 172.21.15.83:46246) 2024-04-25T00:14:07.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff221800640 1 -- 172.21.15.83:0/1664913057 learned_addr learned my addr 172.21.15.83:0/1664913057 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:07.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff213e00640 1 --2- 172.21.15.83:0/1664913057 >> v2:172.21.15.83:3300/0 conn(0x7ff21c0d0260 0x7ff21c15ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff220e00640 1 --2- 172.21.15.83:0/1664913057 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff21c0cfe80 0x7ff21c072520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff213e00640 1 -- 172.21.15.83:0/1664913057 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff21c0cfe80 msgr2=0x7ff21c072520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:07.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff213e00640 1 --2- 172.21.15.83:0/1664913057 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff21c0cfe80 0x7ff21c072520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff213e00640 1 -- 172.21.15.83:0/1664913057 >> v1:172.21.15.83:6789/0 conn(0x7ff21c15f2b0 legacy=0x7ff21c06b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff213e00640 1 -- 172.21.15.83:0/1664913057 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c0b00f0 con 0x7ff21c0d0260 2024-04-25T00:14:07.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.936+0000 7ff21be00640 1 -- 172.21.15.83:0/1664913057 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3158323669 0 0) 0x7ff21c15e590 con 0x7ff21c15f2b0 2024-04-25T00:14:07.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.940+0000 7ff213e00640 1 --2- 172.21.15.83:0/1664913057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff21c0d0260 0x7ff21c15ed70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff2040097c0 tx=0x7ff20401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fb94bda3d9b344 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.940+0000 7ff21be00640 1 -- 172.21.15.83:0/1664913057 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040a7070 con 0x7ff21c0d0260 2024-04-25T00:14:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.940+0000 7ff21be00640 1 -- 172.21.15.83:0/1664913057 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2040b0070 con 0x7ff21c0d0260 2024-04-25T00:14:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.940+0000 7ff21be00640 1 -- 172.21.15.83:0/1664913057 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040ab890 con 0x7ff21c0d0260 2024-04-25T00:14:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.940+0000 7ff223800640 1 -- 172.21.15.83:0/1664913057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff21c0d0260 msgr2=0x7ff21c15ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.940+0000 7ff223800640 1 --2- 172.21.15.83:0/1664913057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff21c0d0260 0x7ff21c15ed70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff2040097c0 tx=0x7ff20401c470 comp rx=0 tx=0).stop 2024-04-25T00:14:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.940+0000 7ff223800640 1 -- 172.21.15.83:0/1664913057 shutdown_connections 2024-04-25T00:14:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.940+0000 7ff223800640 1 --2- 172.21.15.83:0/1664913057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff21c0d0260 0x7ff21c15ed70 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.940+0000 7ff223800640 1 --2- 172.21.15.83:0/1664913057 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff21c0cfe80 0x7ff21c072520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.940+0000 7ff223800640 1 -- 172.21.15.83:0/1664913057 >> 172.21.15.83:0/1664913057 conn(0x7ff21c06c2f0 msgr2=0x7ff21c06c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:07.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.940+0000 7ff223800640 1 -- 172.21.15.83:0/1664913057 shutdown_connections 2024-04-25T00:14:07.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.940+0000 7ff223800640 1 -- 172.21.15.83:0/1664913057 wait complete. 2024-04-25T00:14:07.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff223800640 1 Processor -- start 2024-04-25T00:14:07.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff223800640 1 -- start start 2024-04-25T00:14:07.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff223800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff21c0cfe80 0x7ff21c0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff223800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff21c0d0260 0x7ff21c149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff223800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c157020 0x7ff21c071da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff223800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c06e790 con 0x7ff21c0d0260 2024-04-25T00:14:07.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff223800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c06e610 con 0x7ff21c157020 2024-04-25T00:14:07.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff223800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff21c06b8f0 con 0x7ff21c0cfe80 2024-04-25T00:14:07.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff213e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff21c0d0260 0x7ff21c149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff213e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff21c0d0260 0x7ff21c149830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55540/0 (socket says 172.21.15.83:55540) 2024-04-25T00:14:07.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff213e00640 1 -- 172.21.15.83:0/2277953634 learned_addr learned my addr 172.21.15.83:0/2277953634 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:07.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff220e00640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff21c0cfe80 0x7ff21c0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff221800640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c157020 0x7ff21c071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff213e00640 1 -- 172.21.15.83:0/2277953634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff21c0cfe80 msgr2=0x7ff21c0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff213e00640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff21c0cfe80 0x7ff21c0b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff213e00640 1 -- 172.21.15.83:0/2277953634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c157020 msgr2=0x7ff21c071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff213e00640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c157020 0x7ff21c071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff213e00640 1 -- 172.21.15.83:0/2277953634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c156260 con 0x7ff21c0d0260 2024-04-25T00:14:07.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff221800640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c157020 0x7ff21c071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:07.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff213e00640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff21c0d0260 0x7ff21c149830 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff2040ad040 tx=0x7ff20401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:07.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff219600640 1 -- 172.21.15.83:0/2277953634 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040bf610 con 0x7ff21c0d0260 2024-04-25T00:14:07.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff219600640 1 -- 172.21.15.83:0/2277953634 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2040bf7d0 con 0x7ff21c0d0260 2024-04-25T00:14:07.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff223800640 1 -- 172.21.15.83:0/2277953634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21c156540 con 0x7ff21c0d0260 2024-04-25T00:14:07.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff219600640 1 -- 172.21.15.83:0/2277953634 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040bfbc0 con 0x7ff21c0d0260 2024-04-25T00:14:07.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff223800640 1 -- 172.21.15.83:0/2277953634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21c0b2630 con 0x7ff21c0d0260 2024-04-25T00:14:07.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.944+0000 7ff223800640 1 -- 172.21.15.83:0/2277953634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff21c0736c0 con 0x7ff21c0d0260 2024-04-25T00:14:07.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.948+0000 7ff219600640 1 -- 172.21.15.83:0/2277953634 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff2040bd020 con 0x7ff21c0d0260 2024-04-25T00:14:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.948+0000 7ff219600640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff1f80678f0 0x7ff1f8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.948+0000 7ff219600640 1 -- 172.21.15.83:0/2277953634 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7ff204120760 con 0x7ff21c0d0260 2024-04-25T00:14:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.948+0000 7ff220e00640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff1f80678f0 0x7ff1f8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.948+0000 7ff220e00640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff1f80678f0 0x7ff1f8069db0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff21c15e400 tx=0x7ff20c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:07.952+0000 7ff219600640 1 -- 172.21.15.83:0/2277953634 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff204016090 con 0x7ff21c0d0260 2024-04-25T00:14:08.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.144+0000 7ff223800640 1 -- 172.21.15.83:0/2277953634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff21c04f5f0 con 0x7ff21c0d0260 2024-04-25T00:14:08.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.144+0000 7ff219600640 1 -- 172.21.15.83:0/2277953634 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5183 (secure 0 0 0) 0x7ff2040f0ab0 con 0x7ff21c0d0260 2024-04-25T00:14:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:08.149 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2852678498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2852678498},{"type":"v1","addr":"172.21.15.86:6834","nonce":2852678498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2956252746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2956252746},{"type":"v1","addr":"172.21.15.86:6835","nonce":2956252746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6588,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2678835998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2678835998},{"type":"v1","addr":"172.21.15.83:6836","nonce":2678835998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:13:52.273680+0000","modified":"2024-04-25T00:13:55.285817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2879143105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2879143105},{"type":"v1","addr":"172.21.15.83:6837","nonce":2879143105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6591,"qdb_cluster":[6591]},"id":5}]} 2024-04-25T00:14:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 44 2024-04-25T00:14:08.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.148+0000 7ff223800640 1 -- 172.21.15.83:0/2277953634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff1f80678f0 msgr2=0x7ff1f8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.148+0000 7ff223800640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff1f80678f0 0x7ff1f8069db0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff21c15e400 tx=0x7ff20c009190 comp rx=0 tx=0).stop 2024-04-25T00:14:08.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.148+0000 7ff223800640 1 -- 172.21.15.83:0/2277953634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff21c0d0260 msgr2=0x7ff21c149830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.148+0000 7ff223800640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff21c0d0260 0x7ff21c149830 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff2040ad040 tx=0x7ff20401acb0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.148+0000 7ff223800640 1 -- 172.21.15.83:0/2277953634 shutdown_connections 2024-04-25T00:14:08.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.148+0000 7ff223800640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff1f80678f0 0x7ff1f8069db0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.148+0000 7ff223800640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c157020 0x7ff21c071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.148+0000 7ff223800640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff21c0d0260 0x7ff21c149830 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.148+0000 7ff223800640 1 --2- 172.21.15.83:0/2277953634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff21c0cfe80 0x7ff21c0b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.148+0000 7ff223800640 1 -- 172.21.15.83:0/2277953634 >> 172.21.15.83:0/2277953634 conn(0x7ff21c06c2f0 msgr2=0x7ff21c06c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:08.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.148+0000 7ff223800640 1 -- 172.21.15.83:0/2277953634 shutdown_connections 2024-04-25T00:14:08.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.148+0000 7ff223800640 1 -- 172.21.15.83:0/2277953634 wait complete. 2024-04-25T00:14:08.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:08.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.300+0000 7ff36fc00640 1 Processor -- start 2024-04-25T00:14:08.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.300+0000 7ff36fc00640 1 -- start start 2024-04-25T00:14:08.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.304+0000 7ff36fc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3680b4360 0x7ff3680b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.304+0000 7ff36fc00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff36806ef80 con 0x7ff3681553e0 2024-04-25T00:14:08.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.304+0000 7ff36fc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff36815b510 con 0x7ff3680b2b30 2024-04-25T00:14:08.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.304+0000 7ff36fc00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff36815b7f0 con 0x7ff3680b4360 2024-04-25T00:14:08.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.304+0000 7ff36d200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff3681553e0 0x7ff36806e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46254/0 (socket says 172.21.15.83:46254) 2024-04-25T00:14:08.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.304+0000 7ff36d200640 1 -- 172.21.15.83:0/3410453779 learned_addr learned my addr 172.21.15.83:0/3410453779 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:08.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.304+0000 7ff367e00640 1 --2- 172.21.15.83:0/3410453779 >> v2:172.21.15.83:3300/0 conn(0x7ff3680b4360 0x7ff3680b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.304+0000 7ff367400640 1 -- 172.21.15.83:0/3410453779 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 834818432 0 0) 0x7ff36806ef80 con 0x7ff3681553e0 2024-04-25T00:14:08.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.304+0000 7ff367400640 1 -- 172.21.15.83:0/3410453779 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff34c003530 con 0x7ff3681553e0 2024-04-25T00:14:08.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.304+0000 7ff367e00640 1 -- 172.21.15.83:0/3410453779 >> v1:172.21.15.86:6789/0 conn(0x7ff3680b2b30 legacy=0x7ff3680b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.304+0000 7ff367e00640 1 -- 172.21.15.83:0/3410453779 >> v1:172.21.15.83:6789/0 conn(0x7ff3681553e0 legacy=0x7ff36806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.304+0000 7ff367e00640 1 -- 172.21.15.83:0/3410453779 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff36815bad0 con 0x7ff3680b4360 2024-04-25T00:14:08.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff367e00640 1 --2- 172.21.15.83:0/3410453779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3680b4360 0x7ff3680b47e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff3540097c0 tx=0x7ff35401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4b0dd40bc84896 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:08.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff367400640 1 -- 172.21.15.83:0/3410453779 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3540b0070 con 0x7ff3680b4360 2024-04-25T00:14:08.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff367400640 1 -- 172.21.15.83:0/3410453779 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3540ab600 con 0x7ff3680b4360 2024-04-25T00:14:08.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff367400640 1 -- 172.21.15.83:0/3410453779 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3540ab9b0 con 0x7ff3680b4360 2024-04-25T00:14:08.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff36fc00640 1 -- 172.21.15.83:0/3410453779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3680b4360 msgr2=0x7ff3680b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff36fc00640 1 --2- 172.21.15.83:0/3410453779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3680b4360 0x7ff3680b47e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff3540097c0 tx=0x7ff35401c410 comp rx=0 tx=0).stop 2024-04-25T00:14:08.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff36fc00640 1 -- 172.21.15.83:0/3410453779 shutdown_connections 2024-04-25T00:14:08.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff36fc00640 1 --2- 172.21.15.83:0/3410453779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3680b4360 0x7ff3680b47e0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff36fc00640 1 -- 172.21.15.83:0/3410453779 >> 172.21.15.83:0/3410453779 conn(0x7ff3680b1830 msgr2=0x7ff3680b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:08.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff36fc00640 1 -- 172.21.15.83:0/3410453779 shutdown_connections 2024-04-25T00:14:08.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff36fc00640 1 -- 172.21.15.83:0/3410453779 wait complete. 2024-04-25T00:14:08.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff36fc00640 1 Processor -- start 2024-04-25T00:14:08.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff36fc00640 1 -- start start 2024-04-25T00:14:08.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff36fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3680b2b30 0x7ff368150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.308+0000 7ff36fc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3680b4360 0x7ff368165a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff36fc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3681553e0 0x7ff36806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff36fc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff368155940 con 0x7ff3680b4360 2024-04-25T00:14:08.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff36fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3681557c0 con 0x7ff3680b2b30 2024-04-25T00:14:08.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff36fc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff36806ebd0 con 0x7ff3681553e0 2024-04-25T00:14:08.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff367e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3680b4360 0x7ff368165a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff367e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3680b4360 0x7ff368165a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55570/0 (socket says 172.21.15.83:55570) 2024-04-25T00:14:08.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff367e00640 1 -- 172.21.15.83:0/479589779 learned_addr learned my addr 172.21.15.83:0/479589779 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:08.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff36d200640 1 --2- 172.21.15.83:0/479589779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3680b2b30 0x7ff368150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff36dc00640 1 --2- 172.21.15.83:0/479589779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3681553e0 0x7ff36806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff367e00640 1 -- 172.21.15.83:0/479589779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3681553e0 msgr2=0x7ff36806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff367e00640 1 --2- 172.21.15.83:0/479589779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3681553e0 0x7ff36806d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff367e00640 1 -- 172.21.15.83:0/479589779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3680b2b30 msgr2=0x7ff368150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff367e00640 1 --2- 172.21.15.83:0/479589779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3680b2b30 0x7ff368150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff367e00640 1 -- 172.21.15.83:0/479589779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff368150b70 con 0x7ff3680b4360 2024-04-25T00:14:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff367e00640 1 --2- 172.21.15.83:0/479589779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3680b4360 0x7ff368165a10 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff3540097c0 tx=0x7ff35401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff364c00640 1 -- 172.21.15.83:0/479589779 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3540b0040 con 0x7ff3680b4360 2024-04-25T00:14:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff364c00640 1 -- 172.21.15.83:0/479589779 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff354004030 con 0x7ff3680b4360 2024-04-25T00:14:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff36fc00640 1 -- 172.21.15.83:0/479589779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff368159270 con 0x7ff3680b4360 2024-04-25T00:14:08.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff364c00640 1 -- 172.21.15.83:0/479589779 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354006e20 con 0x7ff3680b4360 2024-04-25T00:14:08.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff36fc00640 1 -- 172.21.15.83:0/479589779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff368159830 con 0x7ff3680b4360 2024-04-25T00:14:08.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff36fc00640 1 -- 172.21.15.83:0/479589779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3680737f0 con 0x7ff3680b4360 2024-04-25T00:14:08.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.312+0000 7ff364c00640 1 -- 172.21.15.83:0/479589779 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff354003620 con 0x7ff3680b4360 2024-04-25T00:14:08.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.316+0000 7ff364c00640 1 --2- 172.21.15.83:0/479589779 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff3480678c0 0x7ff348069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.316+0000 7ff364c00640 1 -- 172.21.15.83:0/479589779 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7ff3541207b0 con 0x7ff3680b4360 2024-04-25T00:14:08.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.316+0000 7ff36d200640 1 --2- 172.21.15.83:0/479589779 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff3480678c0 0x7ff348069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.316+0000 7ff36d200640 1 --2- 172.21.15.83:0/479589779 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff3480678c0 0x7ff348069d80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff3580011f0 tx=0x7ff358007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:08.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.320+0000 7ff364c00640 1 -- 172.21.15.83:0/479589779 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff354016090 con 0x7ff3680b4360 2024-04-25T00:14:08.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.476+0000 7ff36fc00640 1 -- 172.21.15.83:0/479589779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff36804f5f0 con 0x7ff3680b4360 2024-04-25T00:14:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.476+0000 7ff364c00640 1 -- 172.21.15.83:0/479589779 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16861 (secure 0 0 0) 0x7ff3540f1270 con 0x7ff3680b4360 2024-04-25T00:14:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:08.480 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":71,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:53.896317+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:50.655187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:51.268160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.480+0000 7ff36fc00640 1 -- 172.21.15.83:0/479589779 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff3480678c0 msgr2=0x7ff348069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.480+0000 7ff36fc00640 1 --2- 172.21.15.83:0/479589779 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff3480678c0 0x7ff348069d80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff3580011f0 tx=0x7ff358007d40 comp rx=0 tx=0).stop 2024-04-25T00:14:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.480+0000 7ff36fc00640 1 -- 172.21.15.83:0/479589779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3680b4360 msgr2=0x7ff368165a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.480+0000 7ff36fc00640 1 --2- 172.21.15.83:0/479589779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3680b4360 0x7ff368165a10 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff3540097c0 tx=0x7ff35401ac50 comp rx=0 tx=0).stop 2024-04-25T00:14:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.480+0000 7ff36fc00640 1 -- 172.21.15.83:0/479589779 shutdown_connections 2024-04-25T00:14:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.480+0000 7ff36fc00640 1 --2- 172.21.15.83:0/479589779 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff3480678c0 0x7ff348069d80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.480+0000 7ff36fc00640 1 --2- 172.21.15.83:0/479589779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3681553e0 0x7ff36806d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.480+0000 7ff36fc00640 1 --2- 172.21.15.83:0/479589779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3680b4360 0x7ff368165a10 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.480+0000 7ff36fc00640 1 --2- 172.21.15.83:0/479589779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3680b2b30 0x7ff368150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.480+0000 7ff36fc00640 1 -- 172.21.15.83:0/479589779 >> 172.21.15.83:0/479589779 conn(0x7ff3680b1830 msgr2=0x7ff3680b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.480+0000 7ff36fc00640 1 -- 172.21.15.83:0/479589779 shutdown_connections 2024-04-25T00:14:08.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.480+0000 7ff36fc00640 1 -- 172.21.15.83:0/479589779 wait complete. 2024-04-25T00:14:08.501 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:14:08.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:08.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.632+0000 7ffb8be00640 1 Processor -- start 2024-04-25T00:14:08.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.632+0000 7ffb8be00640 1 -- start start 2024-04-25T00:14:08.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb8be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb8415e2d0 0x7ffb8406f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb8be00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb8406d830 0x7ffb8406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb8be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb8415e6b0 0x7ffb840ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb8be00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ffb8406dda0 con 0x7ffb8406d830 2024-04-25T00:14:08.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb8be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffb8406fef0 con 0x7ffb8415e6b0 2024-04-25T00:14:08.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb8be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8406e2d0 con 0x7ffb8415e2d0 2024-04-25T00:14:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb88a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb8406d830 0x7ffb8406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb88a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb8406d830 0x7ffb8406cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55576/0 (socket says 172.21.15.83:55576) 2024-04-25T00:14:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb88a00640 1 -- 172.21.15.83:0/806982321 learned_addr learned my addr 172.21.15.83:0/806982321 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb89e00640 1 --2- 172.21.15.83:0/806982321 >> v2:172.21.15.86:3300/0 conn(0x7ffb8415e6b0 0x7ffb840ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb89400640 1 --2- 172.21.15.83:0/806982321 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb8415e2d0 0x7ffb8406f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb88a00640 1 -- 172.21.15.83:0/806982321 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb8415e2d0 msgr2=0x7ffb8406f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb88a00640 1 --2- 172.21.15.83:0/806982321 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb8415e2d0 0x7ffb8406f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb88a00640 1 -- 172.21.15.83:0/806982321 >> v2:172.21.15.86:3300/0 conn(0x7ffb8415e6b0 msgr2=0x7ffb840ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb88a00640 1 --2- 172.21.15.83:0/806982321 >> v2:172.21.15.86:3300/0 conn(0x7ffb8415e6b0 0x7ffb840ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb88a00640 1 -- 172.21.15.83:0/806982321 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8406dff0 con 0x7ffb8406d830 2024-04-25T00:14:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.636+0000 7ffb89e00640 1 --2- 172.21.15.83:0/806982321 >> v2:172.21.15.86:3300/0 conn(0x7ffb8415e6b0 0x7ffb840ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:08.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb88a00640 1 --2- 172.21.15.83:0/806982321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8406d830 0x7ffb8406cd00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ffb84072320 tx=0x7ffb7001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4f19433ccc3f59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:08.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb83e00640 1 -- 172.21.15.83:0/806982321 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb700b0070 con 0x7ffb8406d830 2024-04-25T00:14:08.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb83e00640 1 -- 172.21.15.83:0/806982321 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb700ab600 con 0x7ffb8406d830 2024-04-25T00:14:08.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb83e00640 1 -- 172.21.15.83:0/806982321 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb700ab9d0 con 0x7ffb8406d830 2024-04-25T00:14:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb8be00640 1 -- 172.21.15.83:0/806982321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8406d830 msgr2=0x7ffb8406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb8be00640 1 --2- 172.21.15.83:0/806982321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8406d830 0x7ffb8406cd00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ffb84072320 tx=0x7ffb7001c490 comp rx=0 tx=0).stop 2024-04-25T00:14:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb8be00640 1 -- 172.21.15.83:0/806982321 shutdown_connections 2024-04-25T00:14:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb8be00640 1 --2- 172.21.15.83:0/806982321 >> v2:172.21.15.86:3300/0 conn(0x7ffb8415e6b0 0x7ffb840ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb8be00640 1 --2- 172.21.15.83:0/806982321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8406d830 0x7ffb8406cd00 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb8be00640 1 --2- 172.21.15.83:0/806982321 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb8415e2d0 0x7ffb8406f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb8be00640 1 -- 172.21.15.83:0/806982321 >> 172.21.15.83:0/806982321 conn(0x7ffb840d1d10 msgr2=0x7ffb840d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb8be00640 1 -- 172.21.15.83:0/806982321 shutdown_connections 2024-04-25T00:14:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb8be00640 1 -- 172.21.15.83:0/806982321 wait complete. 2024-04-25T00:14:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb8be00640 1 Processor -- start 2024-04-25T00:14:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.640+0000 7ffb8be00640 1 -- start start 2024-04-25T00:14:08.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb8be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb8415e2d0 0x7ffb840b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb8be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8415e6b0 0x7ffb840aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb841550b0 0x7ffb84155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb8be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffb840cee60 con 0x7ffb841550b0 2024-04-25T00:14:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb8be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb840cece0 con 0x7ffb8415e6b0 2024-04-25T00:14:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb8be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ffb840d05b0 con 0x7ffb8415e2d0 2024-04-25T00:14:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb89e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb841550b0 0x7ffb84155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb89e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb841550b0 0x7ffb84155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55578/0 (socket says 172.21.15.83:55578) 2024-04-25T00:14:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb89e00640 1 -- 172.21.15.83:0/2429381241 learned_addr learned my addr 172.21.15.83:0/2429381241 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb88a00640 1 --2- 172.21.15.83:0/2429381241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8415e6b0 0x7ffb840aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb89400640 1 --2- 172.21.15.83:0/2429381241 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb8415e2d0 0x7ffb840b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb89e00640 1 -- 172.21.15.83:0/2429381241 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb8415e2d0 msgr2=0x7ffb840b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb89e00640 1 --2- 172.21.15.83:0/2429381241 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb8415e2d0 0x7ffb840b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb89e00640 1 -- 172.21.15.83:0/2429381241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8415e6b0 msgr2=0x7ffb840aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb89e00640 1 --2- 172.21.15.83:0/2429381241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8415e6b0 0x7ffb840aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb89e00640 1 -- 172.21.15.83:0/2429381241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb84155de0 con 0x7ffb841550b0 2024-04-25T00:14:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb89e00640 1 --2- 172.21.15.83:0/2429381241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb841550b0 0x7ffb84155560 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c007dc0 tx=0x7ffb7c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb81600640 1 -- 172.21.15.83:0/2429381241 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c01a070 con 0x7ffb841550b0 2024-04-25T00:14:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb81600640 1 -- 172.21.15.83:0/2429381241 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c0039a0 con 0x7ffb841550b0 2024-04-25T00:14:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb8be00640 1 -- 172.21.15.83:0/2429381241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb84151450 con 0x7ffb841550b0 2024-04-25T00:14:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb81600640 1 -- 172.21.15.83:0/2429381241 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c003d90 con 0x7ffb841550b0 2024-04-25T00:14:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb8be00640 1 -- 172.21.15.83:0/2429381241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb84155aa0 con 0x7ffb841550b0 2024-04-25T00:14:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb8be00640 1 -- 172.21.15.83:0/2429381241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb8409fe40 con 0x7ffb841550b0 2024-04-25T00:14:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.644+0000 7ffb81600640 1 -- 172.21.15.83:0/2429381241 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffb7c01d070 con 0x7ffb841550b0 2024-04-25T00:14:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.648+0000 7ffb81600640 1 --2- 172.21.15.83:0/2429381241 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffb640678c0 0x7ffb64069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.648+0000 7ffb81600640 1 -- 172.21.15.83:0/2429381241 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7ffb7c0860b0 con 0x7ffb841550b0 2024-04-25T00:14:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.648+0000 7ffb89400640 1 --2- 172.21.15.83:0/2429381241 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffb640678c0 0x7ffb64069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.648+0000 7ffb89400640 1 --2- 172.21.15.83:0/2429381241 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffb640678c0 0x7ffb64069d80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800ba30 tx=0x7ffb78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:08.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.652+0000 7ffb81600640 1 -- 172.21.15.83:0/2429381241 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffb7c010070 con 0x7ffb841550b0 2024-04-25T00:14:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.840+0000 7ffb8be00640 1 -- 172.21.15.83:0/2429381241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb8404f5f0 con 0x7ffb841550b0 2024-04-25T00:14:08.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.840+0000 7ffb81600640 1 -- 172.21.15.83:0/2429381241 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5183 (secure 0 0 0) 0x7ffb7c056370 con 0x7ffb841550b0 2024-04-25T00:14:08.842 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:08.842 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2852678498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2852678498},{"type":"v1","addr":"172.21.15.86:6834","nonce":2852678498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2956252746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2956252746},{"type":"v1","addr":"172.21.15.86:6835","nonce":2956252746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6588,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2678835998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2678835998},{"type":"v1","addr":"172.21.15.83:6836","nonce":2678835998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:13:52.273680+0000","modified":"2024-04-25T00:13:55.285817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2879143105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2879143105},{"type":"v1","addr":"172.21.15.83:6837","nonce":2879143105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6591,"qdb_cluster":[6591]},"id":5}]} 2024-04-25T00:14:08.843 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 44 2024-04-25T00:14:08.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.840+0000 7ffb8be00640 1 -- 172.21.15.83:0/2429381241 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffb640678c0 msgr2=0x7ffb64069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.840+0000 7ffb8be00640 1 --2- 172.21.15.83:0/2429381241 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffb640678c0 0x7ffb64069d80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800ba30 tx=0x7ffb78008040 comp rx=0 tx=0).stop 2024-04-25T00:14:08.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.840+0000 7ffb8be00640 1 -- 172.21.15.83:0/2429381241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb841550b0 msgr2=0x7ffb84155560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.840+0000 7ffb8be00640 1 --2- 172.21.15.83:0/2429381241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb841550b0 0x7ffb84155560 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c007dc0 tx=0x7ffb7c00d4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.840+0000 7ffb8be00640 1 -- 172.21.15.83:0/2429381241 shutdown_connections 2024-04-25T00:14:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.840+0000 7ffb8be00640 1 --2- 172.21.15.83:0/2429381241 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffb640678c0 0x7ffb64069d80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.840+0000 7ffb8be00640 1 --2- 172.21.15.83:0/2429381241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb841550b0 0x7ffb84155560 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.840+0000 7ffb8be00640 1 --2- 172.21.15.83:0/2429381241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8415e6b0 0x7ffb840aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.840+0000 7ffb8be00640 1 --2- 172.21.15.83:0/2429381241 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb8415e2d0 0x7ffb840b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.840+0000 7ffb8be00640 1 -- 172.21.15.83:0/2429381241 >> 172.21.15.83:0/2429381241 conn(0x7ffb840d1d10 msgr2=0x7ffb840b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.840+0000 7ffb8be00640 1 -- 172.21.15.83:0/2429381241 shutdown_connections 2024-04-25T00:14:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:08.840+0000 7ffb8be00640 1 -- 172.21.15.83:0/2429381241 wait complete. 2024-04-25T00:14:08.864 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.000+0000 7fd1f7000640 1 Processor -- start 2024-04-25T00:14:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.000+0000 7fd1f7000640 1 -- start start 2024-04-25T00:14:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.000+0000 7fd1f7000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd1f01599f0 0x7fd1f00cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.000+0000 7fd1f7000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1f015cec0 0x7fd1f00ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.000+0000 7fd1f7000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1f00ccc60 con 0x7fd1f015c340 2024-04-25T00:14:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.000+0000 7fd1f7000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd1f0159dd0 con 0x7fd1f01599f0 2024-04-25T00:14:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.000+0000 7fd1f7000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f006dba0 con 0x7fd1f015cec0 2024-04-25T00:14:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.000+0000 7fd1ef400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd1f015c340 0x7fd1f015c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46266/0 (socket says 172.21.15.83:46266) 2024-04-25T00:14:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.000+0000 7fd1ef400640 1 -- 172.21.15.83:0/1079864240 learned_addr learned my addr 172.21.15.83:0/1079864240 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.000+0000 7fd1efe00640 1 --2- 172.21.15.83:0/1079864240 >> v2:172.21.15.86:3300/0 conn(0x7fd1f01599f0 0x7fd1f00cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.000+0000 7fd1f5000640 1 --2- 172.21.15.83:0/1079864240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1f015cec0 0x7fd1f00ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.000+0000 7fd1eea00640 1 -- 172.21.15.83:0/1079864240 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2373947144 0 0) 0x7fd1dc001120 con 0x7fd1f015c340 2024-04-25T00:14:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.000+0000 7fd1eea00640 1 -- 172.21.15.83:0/1079864240 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1d4003530 con 0x7fd1f015c340 2024-04-25T00:14:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 -- 172.21.15.83:0/1079864240 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 579786560 0 0) 0x7fd1d4003530 con 0x7fd1f015c340 2024-04-25T00:14:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 -- 172.21.15.83:0/1079864240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1f015cec0 msgr2=0x7fd1f00ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 --2- 172.21.15.83:0/1079864240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1f015cec0 0x7fd1f00ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 -- 172.21.15.83:0/1079864240 >> v2:172.21.15.86:3300/0 conn(0x7fd1f01599f0 msgr2=0x7fd1f00cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 --2- 172.21.15.83:0/1079864240 >> v2:172.21.15.86:3300/0 conn(0x7fd1f01599f0 0x7fd1f00cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 -- 172.21.15.83:0/1079864240 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f006de80 con 0x7fd1f015c340 2024-04-25T00:14:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 -- 172.21.15.83:0/1079864240 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fd1dc001850 con 0x7fd1f015c340 2024-04-25T00:14:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 -- 172.21.15.83:0/1079864240 >> v1:172.21.15.83:6789/0 conn(0x7fd1f015c340 legacy=0x7fd1f015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 --2- 172.21.15.83:0/1079864240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1d40039c0 0x7fd1d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 --2- 172.21.15.83:0/1079864240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d4004810 0x7fd1d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 --2- 172.21.15.83:0/1079864240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1f015cec0 0x7fd1d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 -- 172.21.15.83:0/1079864240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f006dba0 con 0x7fd1d4004810 2024-04-25T00:14:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 -- 172.21.15.83:0/1079864240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d400b9d0 con 0x7fd1d40039c0 2024-04-25T00:14:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 -- 172.21.15.83:0/1079864240 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d400bba0 con 0x7fd1f015cec0 2024-04-25T00:14:09.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1ef400640 1 --2- 172.21.15.83:0/1079864240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d4004810 0x7fd1d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1f5000640 1 --2- 172.21.15.83:0/1079864240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1f015cec0 0x7fd1d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1efe00640 1 --2- 172.21.15.83:0/1079864240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1d40039c0 0x7fd1d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1ef400640 1 -- 172.21.15.83:0/1079864240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1f015cec0 msgr2=0x7fd1d40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1ef400640 1 --2- 172.21.15.83:0/1079864240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1f015cec0 0x7fd1d40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1ef400640 1 -- 172.21.15.83:0/1079864240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1d40039c0 msgr2=0x7fd1d4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1ef400640 1 --2- 172.21.15.83:0/1079864240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1d40039c0 0x7fd1d4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1ef400640 1 -- 172.21.15.83:0/1079864240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d4003530 con 0x7fd1d4004810 2024-04-25T00:14:09.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1efe00640 1 --2- 172.21.15.83:0/1079864240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1d40039c0 0x7fd1d4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:09.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1ef400640 1 --2- 172.21.15.83:0/1079864240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d4004810 0x7fd1d4004c90 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c350 tx=0x7fd1dc01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809f7a83c9cd03fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 -- 172.21.15.83:0/1079864240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc007e20 con 0x7fd1d4004810 2024-04-25T00:14:09.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 -- 172.21.15.83:0/1079864240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1dc0a9070 con 0x7fd1d4004810 2024-04-25T00:14:09.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.004+0000 7fd1eea00640 1 -- 172.21.15.83:0/1079864240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc005d40 con 0x7fd1d4004810 2024-04-25T00:14:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f7000640 1 -- 172.21.15.83:0/1079864240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d4004810 msgr2=0x7fd1d4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f7000640 1 --2- 172.21.15.83:0/1079864240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d4004810 0x7fd1d4004c90 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c350 tx=0x7fd1dc01cba0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f5000640 1 -- 172.21.15.83:0/1079864240 reap_dead start 2024-04-25T00:14:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f7000640 1 -- 172.21.15.83:0/1079864240 shutdown_connections 2024-04-25T00:14:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f7000640 1 -- 172.21.15.83:0/1079864240 >> 172.21.15.83:0/1079864240 conn(0x7fd1f00b1830 msgr2=0x7fd1f00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f7000640 1 -- 172.21.15.83:0/1079864240 shutdown_connections 2024-04-25T00:14:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f7000640 1 -- 172.21.15.83:0/1079864240 wait complete. 2024-04-25T00:14:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f7000640 1 Processor -- start 2024-04-25T00:14:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f7000640 1 -- start start 2024-04-25T00:14:09.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f7000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d4004810 0x7fd1f015ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f7000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f0157710 0x7fd1f0155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f7000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1f00b27f0 0x7fd1f00b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f7000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f00cf2c0 con 0x7fd1d4004810 2024-04-25T00:14:09.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f7000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f00cf140 con 0x7fd1f0157710 2024-04-25T00:14:09.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f7000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f00cf440 con 0x7fd1f00b27f0 2024-04-25T00:14:09.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d4004810 0x7fd1f015ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d4004810 0x7fd1f015ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55596/0 (socket says 172.21.15.83:55596) 2024-04-25T00:14:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1efe00640 1 -- 172.21.15.83:0/2559041993 learned_addr learned my addr 172.21.15.83:0/2559041993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1f5000640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1f00b27f0 0x7fd1f00b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1ef400640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f0157710 0x7fd1f0155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1efe00640 1 -- 172.21.15.83:0/2559041993 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1f00b27f0 msgr2=0x7fd1f00b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1efe00640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1f00b27f0 0x7fd1f00b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1efe00640 1 -- 172.21.15.83:0/2559041993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f0157710 msgr2=0x7fd1f0155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1efe00640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f0157710 0x7fd1f0155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1efe00640 1 -- 172.21.15.83:0/2559041993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f015bed0 con 0x7fd1d4004810 2024-04-25T00:14:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.008+0000 7fd1ef400640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f0157710 0x7fd1f0155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.012+0000 7fd1efe00640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d4004810 0x7fd1f015ac00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0007f30 tx=0x7fd1e0010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.012+0000 7fd1e7e00640 1 -- 172.21.15.83:0/2559041993 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e001a030 con 0x7fd1d4004810 2024-04-25T00:14:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.012+0000 7fd1e7e00640 1 -- 172.21.15.83:0/2559041993 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e00039a0 con 0x7fd1d4004810 2024-04-25T00:14:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.012+0000 7fd1e7e00640 1 -- 172.21.15.83:0/2559041993 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0003d70 con 0x7fd1d4004810 2024-04-25T00:14:09.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.012+0000 7fd1f7000640 1 -- 172.21.15.83:0/2559041993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f015b910 con 0x7fd1d4004810 2024-04-25T00:14:09.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.012+0000 7fd1f7000640 1 -- 172.21.15.83:0/2559041993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f01fabe0 con 0x7fd1d4004810 2024-04-25T00:14:09.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.012+0000 7fd1f7000640 1 -- 172.21.15.83:0/2559041993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f00737f0 con 0x7fd1d4004810 2024-04-25T00:14:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.012+0000 7fd1e7e00640 1 -- 172.21.15.83:0/2559041993 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd1e001d070 con 0x7fd1d4004810 2024-04-25T00:14:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.012+0000 7fd1e7e00640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd1d00678c0 0x7fd1d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.012+0000 7fd1e7e00640 1 -- 172.21.15.83:0/2559041993 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7fd1e0086c00 con 0x7fd1d4004810 2024-04-25T00:14:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.012+0000 7fd1ef400640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd1d00678c0 0x7fd1d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.016+0000 7fd1ef400640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd1d00678c0 0x7fd1d0069d80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0ab000 tx=0x7fd1dc0b1040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.016+0000 7fd1e7e00640 1 -- 172.21.15.83:0/2559041993 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd1e00575a0 con 0x7fd1d4004810 2024-04-25T00:14:09.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.204+0000 7fd1f7000640 1 -- 172.21.15.83:0/2559041993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1f004f5f0 con 0x7fd1d4004810 2024-04-25T00:14:09.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.204+0000 7fd1e7e00640 1 -- 172.21.15.83:0/2559041993 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5183 (secure 0 0 0) 0x7fd1e0056ea0 con 0x7fd1d4004810 2024-04-25T00:14:09.207 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:09.208 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2852678498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2852678498},{"type":"v1","addr":"172.21.15.86:6834","nonce":2852678498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2956252746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2956252746},{"type":"v1","addr":"172.21.15.86:6835","nonce":2956252746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6588,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2678835998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2678835998},{"type":"v1","addr":"172.21.15.83:6836","nonce":2678835998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:13:52.273680+0000","modified":"2024-04-25T00:13:55.285817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2879143105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2879143105},{"type":"v1","addr":"172.21.15.83:6837","nonce":2879143105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6591,"qdb_cluster":[6591]},"id":5}]} 2024-04-25T00:14:09.208 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 44 2024-04-25T00:14:09.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.208+0000 7fd1f7000640 1 -- 172.21.15.83:0/2559041993 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd1d00678c0 msgr2=0x7fd1d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.208+0000 7fd1f7000640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd1d00678c0 0x7fd1d0069d80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0ab000 tx=0x7fd1dc0b1040 comp rx=0 tx=0).stop 2024-04-25T00:14:09.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.208+0000 7fd1f7000640 1 -- 172.21.15.83:0/2559041993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d4004810 msgr2=0x7fd1f015ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.208+0000 7fd1f7000640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d4004810 0x7fd1f015ac00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0007f30 tx=0x7fd1e0010650 comp rx=0 tx=0).stop 2024-04-25T00:14:09.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.208+0000 7fd1f7000640 1 -- 172.21.15.83:0/2559041993 shutdown_connections 2024-04-25T00:14:09.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.208+0000 7fd1f7000640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd1d00678c0 0x7fd1d0069d80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.208+0000 7fd1f7000640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1f00b27f0 0x7fd1f00b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.208+0000 7fd1f7000640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f0157710 0x7fd1f0155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.208+0000 7fd1f7000640 1 --2- 172.21.15.83:0/2559041993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1d4004810 0x7fd1f015ac00 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.208+0000 7fd1f7000640 1 -- 172.21.15.83:0/2559041993 >> 172.21.15.83:0/2559041993 conn(0x7fd1f00b1830 msgr2=0x7fd1f0070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.208+0000 7fd1f7000640 1 -- 172.21.15.83:0/2559041993 shutdown_connections 2024-04-25T00:14:09.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.208+0000 7fd1f7000640 1 -- 172.21.15.83:0/2559041993 wait complete. 2024-04-25T00:14:09.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:09.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.360+0000 7f7bb4600640 1 Processor -- start 2024-04-25T00:14:09.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.360+0000 7f7bb4600640 1 -- start start 2024-04-25T00:14:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.360+0000 7f7bb4600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bac0cc510 0x7f7bac15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.360+0000 7f7bb4600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7bac15cf90 0x7f7bac0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.360+0000 7f7bb4600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bac0ccc60 con 0x7f7bac1599f0 2024-04-25T00:14:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.360+0000 7f7bb4600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7bac159dd0 con 0x7f7bac15cf90 2024-04-25T00:14:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.360+0000 7f7bb4600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7bac06dba0 con 0x7f7bac0cc510 2024-04-25T00:14:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.364+0000 7f7bb1c00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7bac1599f0 0x7f7bac15c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46282/0 (socket says 172.21.15.83:46282) 2024-04-25T00:14:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.364+0000 7f7bb2600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7bac15cf90 0x7f7bac0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.364+0000 7f7bb1c00640 1 -- 172.21.15.83:0/1563574005 learned_addr learned my addr 172.21.15.83:0/1563574005 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.364+0000 7f7bb1200640 1 --2- 172.21.15.83:0/1563574005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bac0cc510 0x7f7bac15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.364+0000 7f7bb2600640 1 -- 172.21.15.83:0/1563574005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bac0cc510 msgr2=0x7f7bac15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.364+0000 7f7bb2600640 1 --2- 172.21.15.83:0/1563574005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bac0cc510 0x7f7bac15ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.364+0000 7f7bb2600640 1 -- 172.21.15.83:0/1563574005 >> v1:172.21.15.83:6789/0 conn(0x7f7bac1599f0 legacy=0x7f7bac15c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.364+0000 7f7bb2600640 1 -- 172.21.15.83:0/1563574005 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac06de80 con 0x7f7bac15cf90 2024-04-25T00:14:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.364+0000 7f7babe00640 1 -- 172.21.15.83:0/1563574005 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819952352 0 0) 0x7f7ba0001120 con 0x7f7bac1599f0 2024-04-25T00:14:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.364+0000 7f7bb2600640 1 --2- 172.21.15.83:0/1563574005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bac15cf90 0x7f7bac0ce6e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009d90 tx=0x7f7ba401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a71755bf3dbc66f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.364+0000 7f7babe00640 1 -- 172.21.15.83:0/1563574005 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40b2070 con 0x7f7bac15cf90 2024-04-25T00:14:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.364+0000 7f7babe00640 1 -- 172.21.15.83:0/1563574005 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba40a57f0 con 0x7f7bac15cf90 2024-04-25T00:14:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.364+0000 7f7babe00640 1 -- 172.21.15.83:0/1563574005 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40a5bc0 con 0x7f7bac15cf90 2024-04-25T00:14:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 -- 172.21.15.83:0/1563574005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bac15cf90 msgr2=0x7f7bac0ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 --2- 172.21.15.83:0/1563574005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bac15cf90 0x7f7bac0ce6e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009d90 tx=0x7f7ba401c8e0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 -- 172.21.15.83:0/1563574005 shutdown_connections 2024-04-25T00:14:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 --2- 172.21.15.83:0/1563574005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bac15cf90 0x7f7bac0ce6e0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 --2- 172.21.15.83:0/1563574005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bac0cc510 0x7f7bac15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 -- 172.21.15.83:0/1563574005 >> 172.21.15.83:0/1563574005 conn(0x7f7bac0b1830 msgr2=0x7f7bac0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 -- 172.21.15.83:0/1563574005 shutdown_connections 2024-04-25T00:14:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 -- 172.21.15.83:0/1563574005 wait complete. 2024-04-25T00:14:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 Processor -- start 2024-04-25T00:14:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 -- start start 2024-04-25T00:14:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bac0cc510 0x7f7bac156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bac1599f0 0x7f7bac14d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bac15cf90 0x7f7bac1f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac0cf5b0 con 0x7f7bac0cc510 2024-04-25T00:14:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac0cf430 con 0x7f7bac15cf90 2024-04-25T00:14:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb4600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7bac0cf730 con 0x7f7bac1599f0 2024-04-25T00:14:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bac0cc510 0x7f7bac156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bac0cc510 0x7f7bac156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55608/0 (socket says 172.21.15.83:55608) 2024-04-25T00:14:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb1c00640 1 -- 172.21.15.83:0/1051673556 learned_addr learned my addr 172.21.15.83:0/1051673556 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb1200640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bac1599f0 0x7f7bac14d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb2600640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bac15cf90 0x7f7bac1f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb1c00640 1 -- 172.21.15.83:0/1051673556 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bac1599f0 msgr2=0x7f7bac14d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb1c00640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bac1599f0 0x7f7bac14d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb1c00640 1 -- 172.21.15.83:0/1051673556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bac15cf90 msgr2=0x7f7bac1f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb1c00640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bac15cf90 0x7f7bac1f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb1c00640 1 -- 172.21.15.83:0/1051673556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac14dd80 con 0x7f7bac0cc510 2024-04-25T00:14:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.368+0000 7f7bb2600640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bac15cf90 0x7f7bac1f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.372+0000 7f7bb1c00640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bac0cc510 0x7f7bac156770 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000c7b0 tx=0x7f7ba000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.372+0000 7f7ba9600640 1 -- 172.21.15.83:0/1051673556 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0018070 con 0x7f7bac0cc510 2024-04-25T00:14:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.372+0000 7f7ba9600640 1 -- 172.21.15.83:0/1051673556 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba00072f0 con 0x7f7bac0cc510 2024-04-25T00:14:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.372+0000 7f7bb4600640 1 -- 172.21.15.83:0/1051673556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bac1f7580 con 0x7f7bac0cc510 2024-04-25T00:14:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.372+0000 7f7ba9600640 1 -- 172.21.15.83:0/1051673556 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00047f0 con 0x7f7bac0cc510 2024-04-25T00:14:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.372+0000 7f7bb4600640 1 -- 172.21.15.83:0/1051673556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bac157110 con 0x7f7bac0cc510 2024-04-25T00:14:09.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.372+0000 7f7bb4600640 1 -- 172.21.15.83:0/1051673556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bac0737f0 con 0x7f7bac0cc510 2024-04-25T00:14:09.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.372+0000 7f7ba9600640 1 -- 172.21.15.83:0/1051673556 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7ba001b070 con 0x7f7bac0cc510 2024-04-25T00:14:09.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.372+0000 7f7ba9600640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7b8c0678c0 0x7f7b8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.372+0000 7f7ba9600640 1 -- 172.21.15.83:0/1051673556 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f7ba0085220 con 0x7f7bac0cc510 2024-04-25T00:14:09.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.372+0000 7f7bb1200640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7b8c0678c0 0x7f7b8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.376+0000 7f7bb1200640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7b8c0678c0 0x7f7b8c069d80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0b4580 tx=0x7f7b98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.376+0000 7f7ba9600640 1 -- 172.21.15.83:0/1051673556 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7bac14dd80 con 0x7f7bac0cc510 2024-04-25T00:14:09.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.536+0000 7f7bb4600640 1 -- 172.21.15.83:0/1051673556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7bac072c40 con 0x7f7bac0cc510 2024-04-25T00:14:09.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.536+0000 7f7ba9600640 1 -- 172.21.15.83:0/1051673556 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16861 (secure 0 0 0) 0x7f7bac0ccc60 con 0x7f7bac0cc510 2024-04-25T00:14:09.540 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:09.540 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":71,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:13:53.896317+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:50.655187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T00:13:51.268160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.540+0000 7f7bb4600640 1 -- 172.21.15.83:0/1051673556 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7b8c0678c0 msgr2=0x7f7b8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.540+0000 7f7bb4600640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7b8c0678c0 0x7f7b8c069d80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0b4580 tx=0x7f7b98008040 comp rx=0 tx=0).stop 2024-04-25T00:14:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.540+0000 7f7bb4600640 1 -- 172.21.15.83:0/1051673556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bac0cc510 msgr2=0x7f7bac156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.540+0000 7f7bb4600640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bac0cc510 0x7f7bac156770 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000c7b0 tx=0x7f7ba000cc70 comp rx=0 tx=0).stop 2024-04-25T00:14:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.540+0000 7f7bb4600640 1 -- 172.21.15.83:0/1051673556 shutdown_connections 2024-04-25T00:14:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.540+0000 7f7bb4600640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7b8c0678c0 0x7f7b8c069d80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.540+0000 7f7bb4600640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bac1599f0 0x7f7bac14d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.540+0000 7f7bb4600640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bac15cf90 0x7f7bac1f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.540+0000 7f7bb4600640 1 --2- 172.21.15.83:0/1051673556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bac0cc510 0x7f7bac156770 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.540+0000 7f7bb4600640 1 -- 172.21.15.83:0/1051673556 >> 172.21.15.83:0/1051673556 conn(0x7f7bac0b1830 msgr2=0x7f7bac160050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.540+0000 7f7bb4600640 1 -- 172.21.15.83:0/1051673556 shutdown_connections 2024-04-25T00:14:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.540+0000 7f7bb4600640 1 -- 172.21.15.83:0/1051673556 wait complete. 2024-04-25T00:14:09.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:14:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eb2c00640 1 Processor -- start 2024-04-25T00:14:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eb2c00640 1 -- start start 2024-04-25T00:14:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eb2c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac1446e0 0x7f5eac0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eb2c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5eac15f350 0x7f5eac06e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eb2c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5eac06ec50 0x7f5eac15b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eb2c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5eac0715b0 con 0x7f5eac15f350 2024-04-25T00:14:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eb2c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5eac144ac0 con 0x7f5eac06ec50 2024-04-25T00:14:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eb2c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5eac071b50 con 0x7f5eac1446e0 2024-04-25T00:14:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eab400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5eac15f350 0x7f5eac06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eab400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5eac15f350 0x7f5eac06e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55618/0 (socket says 172.21.15.83:55618) 2024-04-25T00:14:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eab400640 1 -- 172.21.15.83:0/4089742308 learned_addr learned my addr 172.21.15.83:0/4089742308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eabe00640 1 --2- 172.21.15.83:0/4089742308 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac1446e0 0x7f5eac0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eb0c00640 1 --2- 172.21.15.83:0/4089742308 >> v2:172.21.15.86:3300/0 conn(0x7f5eac06ec50 0x7f5eac15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eab400640 1 -- 172.21.15.83:0/4089742308 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac1446e0 msgr2=0x7f5eac0cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eab400640 1 --2- 172.21.15.83:0/4089742308 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac1446e0 0x7f5eac0cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eab400640 1 -- 172.21.15.83:0/4089742308 >> v2:172.21.15.86:3300/0 conn(0x7f5eac06ec50 msgr2=0x7f5eac15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eab400640 1 --2- 172.21.15.83:0/4089742308 >> v2:172.21.15.86:3300/0 conn(0x7f5eac06ec50 0x7f5eac15b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.696+0000 7f5eab400640 1 -- 172.21.15.83:0/4089742308 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac0cb210 con 0x7f5eac15f350 2024-04-25T00:14:09.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.700+0000 7f5eab400640 1 --2- 172.21.15.83:0/4089742308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac15f350 0x7f5eac06e710 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0d09c0 tx=0x7f5e9801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d136475d41a5cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.700+0000 7f5eaaa00640 1 -- 172.21.15.83:0/4089742308 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e980b1070 con 0x7f5eac15f350 2024-04-25T00:14:09.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.700+0000 7f5eaaa00640 1 -- 172.21.15.83:0/4089742308 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e980ac630 con 0x7f5eac15f350 2024-04-25T00:14:09.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.700+0000 7f5eaaa00640 1 -- 172.21.15.83:0/4089742308 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e980a8050 con 0x7f5eac15f350 2024-04-25T00:14:09.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.700+0000 7f5eb2c00640 1 -- 172.21.15.83:0/4089742308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac15f350 msgr2=0x7f5eac06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.700+0000 7f5eb2c00640 1 --2- 172.21.15.83:0/4089742308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac15f350 0x7f5eac06e710 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0d09c0 tx=0x7f5e9801c520 comp rx=0 tx=0).stop 2024-04-25T00:14:09.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.700+0000 7f5eb2c00640 1 -- 172.21.15.83:0/4089742308 shutdown_connections 2024-04-25T00:14:09.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.700+0000 7f5eb2c00640 1 --2- 172.21.15.83:0/4089742308 >> v2:172.21.15.86:3300/0 conn(0x7f5eac06ec50 0x7f5eac15b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.700+0000 7f5eb2c00640 1 --2- 172.21.15.83:0/4089742308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac15f350 0x7f5eac06e710 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.700+0000 7f5eb2c00640 1 --2- 172.21.15.83:0/4089742308 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac1446e0 0x7f5eac0cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.700+0000 7f5eb2c00640 1 -- 172.21.15.83:0/4089742308 >> 172.21.15.83:0/4089742308 conn(0x7f5eac0cfa60 msgr2=0x7f5eac0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.700+0000 7f5eb2c00640 1 -- 172.21.15.83:0/4089742308 shutdown_connections 2024-04-25T00:14:09.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.700+0000 7f5eb2c00640 1 -- 172.21.15.83:0/4089742308 wait complete. 2024-04-25T00:14:09.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eb2c00640 1 Processor -- start 2024-04-25T00:14:09.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eb2c00640 1 -- start start 2024-04-25T00:14:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eb2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eac06ec50 0x7f5eac15c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eb2c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac1446e0 0x7f5eac1f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eb2c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac15f350 0x7f5eac15a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eb2c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac06d5a0 con 0x7f5eac1446e0 2024-04-25T00:14:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eb2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac0cd630 con 0x7f5eac06ec50 2024-04-25T00:14:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eb2c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5eac06d720 con 0x7f5eac15f350 2024-04-25T00:14:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac1446e0 0x7f5eac1f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac1446e0 0x7f5eac1f7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55628/0 (socket says 172.21.15.83:55628) 2024-04-25T00:14:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eab400640 1 -- 172.21.15.83:0/2810148853 learned_addr learned my addr 172.21.15.83:0/2810148853 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eabe00640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eac06ec50 0x7f5eac15c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eb0c00640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac15f350 0x7f5eac15a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eab400640 1 -- 172.21.15.83:0/2810148853 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac15f350 msgr2=0x7f5eac15a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eab400640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac15f350 0x7f5eac15a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eab400640 1 -- 172.21.15.83:0/2810148853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eac06ec50 msgr2=0x7f5eac15c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eab400640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eac06ec50 0x7f5eac15c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eab400640 1 -- 172.21.15.83:0/2810148853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac1f7e90 con 0x7f5eac1446e0 2024-04-25T00:14:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eabe00640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eac06ec50 0x7f5eac15c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eab400640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac1446e0 0x7f5eac1f7950 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f5e980af040 tx=0x7f5e980192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5ea3e00640 1 -- 172.21.15.83:0/2810148853 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e980bc070 con 0x7f5eac1446e0 2024-04-25T00:14:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5ea3e00640 1 -- 172.21.15.83:0/2810148853 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e980ace30 con 0x7f5eac1446e0 2024-04-25T00:14:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5ea3e00640 1 -- 172.21.15.83:0/2810148853 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e980b1070 con 0x7f5eac1446e0 2024-04-25T00:14:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eb2c00640 1 -- 172.21.15.83:0/2810148853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eac13b350 con 0x7f5eac1446e0 2024-04-25T00:14:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eb2c00640 1 -- 172.21.15.83:0/2810148853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eac135280 con 0x7f5eac1446e0 2024-04-25T00:14:09.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.704+0000 7f5eb2c00640 1 -- 172.21.15.83:0/2810148853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eac068230 con 0x7f5eac1446e0 2024-04-25T00:14:09.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.708+0000 7f5ea3e00640 1 -- 172.21.15.83:0/2810148853 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5e980bf070 con 0x7f5eac1446e0 2024-04-25T00:14:09.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.708+0000 7f5ea3e00640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5e8c0678c0 0x7f5e8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.708+0000 7f5ea3e00640 1 -- 172.21.15.83:0/2810148853 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f5e9811f590 con 0x7f5eac1446e0 2024-04-25T00:14:09.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.708+0000 7f5eabe00640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5e8c0678c0 0x7f5e8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.708+0000 7f5eabe00640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5e8c0678c0 0x7f5e8c069d80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0d0790 tx=0x7f5e9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.712+0000 7f5ea3e00640 1 -- 172.21.15.83:0/2810148853 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5e98016110 con 0x7f5eac1446e0 2024-04-25T00:14:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.900+0000 7f5eb2c00640 1 -- 172.21.15.83:0/2810148853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5eac13c5f0 con 0x7f5eac1446e0 2024-04-25T00:14:09.902 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-25T00:14:09.900+0000 7efeeee00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:14:09.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.900+0000 7f5ea3e00640 1 -- 172.21.15.83:0/2810148853 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7f5e980ef980 con 0x7f5eac1446e0 2024-04-25T00:14:09.904 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:14:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.904+0000 7f5eb2c00640 1 -- 172.21.15.83:0/2810148853 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5e8c0678c0 msgr2=0x7f5e8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.904+0000 7f5eb2c00640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5e8c0678c0 0x7f5e8c069d80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0d0790 tx=0x7f5e9c008040 comp rx=0 tx=0).stop 2024-04-25T00:14:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.904+0000 7f5eb2c00640 1 -- 172.21.15.83:0/2810148853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac1446e0 msgr2=0x7f5eac1f7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.904+0000 7f5eb2c00640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac1446e0 0x7f5eac1f7950 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f5e980af040 tx=0x7f5e980192c0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.904+0000 7f5eb2c00640 1 -- 172.21.15.83:0/2810148853 shutdown_connections 2024-04-25T00:14:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.904+0000 7f5eb2c00640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5e8c0678c0 0x7f5e8c069d80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.904+0000 7f5eb2c00640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac15f350 0x7f5eac15a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.904+0000 7f5eb2c00640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac1446e0 0x7f5eac1f7950 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.904+0000 7f5eb2c00640 1 --2- 172.21.15.83:0/2810148853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eac06ec50 0x7f5eac15c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.904+0000 7f5eb2c00640 1 -- 172.21.15.83:0/2810148853 >> 172.21.15.83:0/2810148853 conn(0x7f5eac0cfa60 msgr2=0x7f5eac13f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.904+0000 7f5eb2c00640 1 -- 172.21.15.83:0/2810148853 shutdown_connections 2024-04-25T00:14:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:09.904+0000 7f5eb2c00640 1 -- 172.21.15.83:0/2810148853 wait complete. 2024-04-25T00:14:09.913 INFO:tasks.ceph.mds.a.smithi083.stderr: -2329> 2024-04-25T00:13:54.876+0000 7fda22200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:14:09.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:14:09.960 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-25T00:14:10.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.064+0000 7f95a1c00640 1 Processor -- start 2024-04-25T00:14:10.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.064+0000 7f95a1c00640 1 -- start start 2024-04-25T00:14:10.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.064+0000 7f95a1c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f959c06e950 0x7f959c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:10.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.064+0000 7f95a1c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f959c15fe50 0x7f959c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:10.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.064+0000 7f95a1c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f959c0ce000 con 0x7f959c159810 2024-04-25T00:14:10.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.064+0000 7f95a1c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f959c06e210 con 0x7f959c15fe50 2024-04-25T00:14:10.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.064+0000 7f95a1c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f959c06da00 con 0x7f959c06e950 2024-04-25T00:14:10.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.068+0000 7f959aa00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f959c159810 0x7f959c0cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46314/0 (socket says 172.21.15.83:46314) 2024-04-25T00:14:10.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.068+0000 7f959aa00640 1 -- 172.21.15.83:0/2839838954 learned_addr learned my addr 172.21.15.83:0/2839838954 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.068+0000 7f959b400640 1 --2- 172.21.15.83:0/2839838954 >> v2:172.21.15.86:3300/0 conn(0x7f959c15fe50 0x7f959c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.068+0000 7f9592000640 1 --2- 172.21.15.83:0/2839838954 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f959c06e950 0x7f959c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.068+0000 7f959a000640 1 -- 172.21.15.83:0/2839838954 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4184522594 0 0) 0x7f959c0ce000 con 0x7f959c159810 2024-04-25T00:14:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.068+0000 7f959a000640 1 -- 172.21.15.83:0/2839838954 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9580003530 con 0x7f959c159810 2024-04-25T00:14:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.068+0000 7f959b400640 1 -- 172.21.15.83:0/2839838954 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f959c06e950 msgr2=0x7f959c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.068+0000 7f959b400640 1 --2- 172.21.15.83:0/2839838954 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f959c06e950 0x7f959c15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.068+0000 7f959b400640 1 -- 172.21.15.83:0/2839838954 >> v1:172.21.15.83:6789/0 conn(0x7f959c159810 legacy=0x7f959c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.068+0000 7f959b400640 1 -- 172.21.15.83:0/2839838954 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c06eec0 con 0x7f959c15fe50 2024-04-25T00:14:10.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.068+0000 7f959b400640 1 --2- 172.21.15.83:0/2839838954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f959c15fe50 0x7f959c0ce4a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9594009d90 tx=0x7f959401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df2d723dc49c3556 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.068+0000 7f959a000640 1 -- 172.21.15.83:0/2839838954 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95940b2070 con 0x7f959c15fe50 2024-04-25T00:14:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.068+0000 7f959a000640 1 -- 172.21.15.83:0/2839838954 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95940a5890 con 0x7f959c15fe50 2024-04-25T00:14:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.068+0000 7f959a000640 1 -- 172.21.15.83:0/2839838954 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95940a5c60 con 0x7f959c15fe50 2024-04-25T00:14:10.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 -- 172.21.15.83:0/2839838954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f959c15fe50 msgr2=0x7f959c0ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:10.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 --2- 172.21.15.83:0/2839838954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f959c15fe50 0x7f959c0ce4a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9594009d90 tx=0x7f959401c8e0 comp rx=0 tx=0).stop 2024-04-25T00:14:10.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 -- 172.21.15.83:0/2839838954 shutdown_connections 2024-04-25T00:14:10.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 --2- 172.21.15.83:0/2839838954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f959c15fe50 0x7f959c0ce4a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:10.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 --2- 172.21.15.83:0/2839838954 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f959c06e950 0x7f959c15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:10.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 -- 172.21.15.83:0/2839838954 >> 172.21.15.83:0/2839838954 conn(0x7f959c0b1830 msgr2=0x7f959c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:10.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 -- 172.21.15.83:0/2839838954 shutdown_connections 2024-04-25T00:14:10.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 -- 172.21.15.83:0/2839838954 wait complete. 2024-04-25T00:14:10.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 Processor -- start 2024-04-25T00:14:10.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 -- start start 2024-04-25T00:14:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959c06e950 0x7f959c166500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f959c159810 0x7f959c138b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f959c15fe50 0x7f959c14dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f959c070f20 con 0x7f959c06e950 2024-04-25T00:14:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f959c070da0 con 0x7f959c159810 2024-04-25T00:14:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f95a1c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f959c168cb0 con 0x7f959c15fe50 2024-04-25T00:14:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f959aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959c06e950 0x7f959c166500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f959aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959c06e950 0x7f959c166500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55632/0 (socket says 172.21.15.83:55632) 2024-04-25T00:14:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f959aa00640 1 -- 172.21.15.83:0/565052179 learned_addr learned my addr 172.21.15.83:0/565052179 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f9592000640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f959c159810 0x7f959c138b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f959b400640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f959c15fe50 0x7f959c14dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f959aa00640 1 -- 172.21.15.83:0/565052179 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f959c15fe50 msgr2=0x7f959c14dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f959aa00640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f959c15fe50 0x7f959c14dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f959aa00640 1 -- 172.21.15.83:0/565052179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f959c159810 msgr2=0x7f959c138b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f959aa00640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f959c159810 0x7f959c138b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f959aa00640 1 -- 172.21.15.83:0/565052179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c1390d0 con 0x7f959c06e950 2024-04-25T00:14:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.072+0000 7f959b400640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f959c15fe50 0x7f959c14dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.076+0000 7f959aa00640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959c06e950 0x7f959c166500 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f958c00c450 tx=0x7f958c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.076+0000 7f9593400640 1 -- 172.21.15.83:0/565052179 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c017070 con 0x7f959c06e950 2024-04-25T00:14:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.076+0000 7f9593400640 1 -- 172.21.15.83:0/565052179 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f958c005cf0 con 0x7f959c06e950 2024-04-25T00:14:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.076+0000 7f9593400640 1 -- 172.21.15.83:0/565052179 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c007290 con 0x7f959c06e950 2024-04-25T00:14:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.076+0000 7f95a1c00640 1 -- 172.21.15.83:0/565052179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959c13b8e0 con 0x7f959c06e950 2024-04-25T00:14:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.076+0000 7f95a1c00640 1 -- 172.21.15.83:0/565052179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959c13bea0 con 0x7f959c06e950 2024-04-25T00:14:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.076+0000 7f95a1c00640 1 -- 172.21.15.83:0/565052179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f959c04f510 con 0x7f959c06e950 2024-04-25T00:14:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.076+0000 7f9593400640 1 -- 172.21.15.83:0/565052179 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f958c01a070 con 0x7f959c06e950 2024-04-25T00:14:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.076+0000 7f9593400640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95780678c0 0x7f9578069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.076+0000 7f9593400640 1 -- 172.21.15.83:0/565052179 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6187+0+0 (secure 0 0 0) 0x7f958c085280 con 0x7f959c06e950 2024-04-25T00:14:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.076+0000 7f9592000640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95780678c0 0x7f9578069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.080+0000 7f9592000640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95780678c0 0x7f9578069d80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f959c1662c0 tx=0x7f9584008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.080+0000 7f9593400640 1 -- 172.21.15.83:0/565052179 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f958c055bd0 con 0x7f959c06e950 2024-04-25T00:14:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.268+0000 7f95a1c00640 1 -- 172.21.15.83:0/565052179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f959c072c90 con 0x7f959c06e950 2024-04-25T00:14:10.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.904+0000 7f9593400640 1 -- 172.21.15.83:0/565052179 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) ==== 104+0+0 (secure 0 0 0) 0x7f958c0554d0 con 0x7f959c06e950 2024-04-25T00:14:10.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.908+0000 7f95a1c00640 1 -- 172.21.15.83:0/565052179 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95780678c0 msgr2=0x7f9578069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:10.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.908+0000 7f95a1c00640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95780678c0 0x7f9578069d80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f959c1662c0 tx=0x7f9584008040 comp rx=0 tx=0).stop 2024-04-25T00:14:10.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.908+0000 7f95a1c00640 1 -- 172.21.15.83:0/565052179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959c06e950 msgr2=0x7f959c166500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:10.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.908+0000 7f95a1c00640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959c06e950 0x7f959c166500 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f958c00c450 tx=0x7f958c00c910 comp rx=0 tx=0).stop 2024-04-25T00:14:10.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.908+0000 7f95a1c00640 1 -- 172.21.15.83:0/565052179 shutdown_connections 2024-04-25T00:14:10.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.908+0000 7f95a1c00640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95780678c0 0x7f9578069d80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:10.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.908+0000 7f95a1c00640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f959c15fe50 0x7f959c14dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:10.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.908+0000 7f95a1c00640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f959c159810 0x7f959c138b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:10.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.908+0000 7f95a1c00640 1 --2- 172.21.15.83:0/565052179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959c06e950 0x7f959c166500 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:10.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.908+0000 7f95a1c00640 1 -- 172.21.15.83:0/565052179 >> 172.21.15.83:0/565052179 conn(0x7f959c0b1830 msgr2=0x7f959c15ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:10.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.908+0000 7f95a1c00640 1 -- 172.21.15.83:0/565052179 shutdown_connections 2024-04-25T00:14:10.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:10.908+0000 7f95a1c00640 1 -- 172.21.15.83:0/565052179 wait complete. 2024-04-25T00:14:10.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:14:11.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f479ee00640 1 Processor -- start 2024-04-25T00:14:11.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f479ee00640 1 -- start start 2024-04-25T00:14:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f479ee00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4798161850 0x7f47980b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f479ee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47980cfd10 0x7f47980d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f479ee00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47980d1a90 con 0x7f47980b2830 2024-04-25T00:14:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f479ee00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f479806ce30 con 0x7f4798161850 2024-04-25T00:14:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f479ee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f47980d17b0 con 0x7f47980cfd10 2024-04-25T00:14:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f478f400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f47980b2830 0x7f47980b1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46322/0 (socket says 172.21.15.83:46322) 2024-04-25T00:14:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f4797e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4798161850 0x7f47980b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f478f400640 1 -- 172.21.15.83:0/1468257844 learned_addr learned my addr 172.21.15.83:0/1468257844 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f479ce00640 1 --2- 172.21.15.83:0/1468257844 >> v2:172.21.15.86:3300/0 conn(0x7f47980cfd10 0x7f47980d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f4797400640 1 -- 172.21.15.83:0/1468257844 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 29871517 0 0) 0x7f47980d1a90 con 0x7f47980b2830 2024-04-25T00:14:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f4797e00640 1 -- 172.21.15.83:0/1468257844 >> v2:172.21.15.86:3300/0 conn(0x7f47980cfd10 msgr2=0x7f47980d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f4797e00640 1 --2- 172.21.15.83:0/1468257844 >> v2:172.21.15.86:3300/0 conn(0x7f47980cfd10 0x7f47980d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f4797e00640 1 -- 172.21.15.83:0/1468257844 >> v1:172.21.15.83:6789/0 conn(0x7f47980b2830 legacy=0x7f47980b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f4797e00640 1 -- 172.21.15.83:0/1468257844 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479806d0d0 con 0x7f4798161850 2024-04-25T00:14:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.116+0000 7f479ce00640 1 --2- 172.21.15.83:0/1468257844 >> v2:172.21.15.86:3300/0 conn(0x7f47980cfd10 0x7f47980d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:11.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.120+0000 7f4797e00640 1 --2- 172.21.15.83:0/1468257844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798161850 0x7f47980b15e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f479806e4b0 tx=0x7f478801c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe92194a5967aa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:11.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.120+0000 7f4797400640 1 -- 172.21.15.83:0/1468257844 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47880a66e0 con 0x7f4798161850 2024-04-25T00:14:11.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.120+0000 7f4797400640 1 -- 172.21.15.83:0/1468257844 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47880a6880 con 0x7f4798161850 2024-04-25T00:14:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.120+0000 7f4797400640 1 -- 172.21.15.83:0/1468257844 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47880a6c50 con 0x7f4798161850 2024-04-25T00:14:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.120+0000 7f479ee00640 1 -- 172.21.15.83:0/1468257844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798161850 msgr2=0x7f47980b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.120+0000 7f479ee00640 1 --2- 172.21.15.83:0/1468257844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798161850 0x7f47980b15e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f479806e4b0 tx=0x7f478801c810 comp rx=0 tx=0).stop 2024-04-25T00:14:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.120+0000 7f479ee00640 1 -- 172.21.15.83:0/1468257844 shutdown_connections 2024-04-25T00:14:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.120+0000 7f479ee00640 1 --2- 172.21.15.83:0/1468257844 >> v2:172.21.15.86:3300/0 conn(0x7f47980cfd10 0x7f47980d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.120+0000 7f479ee00640 1 --2- 172.21.15.83:0/1468257844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798161850 0x7f47980b15e0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.120+0000 7f479ee00640 1 -- 172.21.15.83:0/1468257844 >> 172.21.15.83:0/1468257844 conn(0x7f47980d1c80 msgr2=0x7f47980d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.120+0000 7f479ee00640 1 -- 172.21.15.83:0/1468257844 shutdown_connections 2024-04-25T00:14:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.120+0000 7f479ee00640 1 -- 172.21.15.83:0/1468257844 wait complete. 2024-04-25T00:14:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ee00640 1 Processor -- start 2024-04-25T00:14:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ee00640 1 -- start start 2024-04-25T00:14:11.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ee00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47980b2830 0x7f4798155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:11.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47980cfd10 0x7f47980c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:11.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798161850 0x7f47980c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:11.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f479816b060 con 0x7f4798161850 2024-04-25T00:14:11.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47980b2440 con 0x7f47980cfd10 2024-04-25T00:14:11.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ee00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f479816b1e0 con 0x7f47980b2830 2024-04-25T00:14:11.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798161850 0x7f47980c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:11.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798161850 0x7f47980c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55658/0 (socket says 172.21.15.83:55658) 2024-04-25T00:14:11.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ce00640 1 -- 172.21.15.83:0/1462006028 learned_addr learned my addr 172.21.15.83:0/1462006028 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:11.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f478f400640 1 --2- 172.21.15.83:0/1462006028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47980cfd10 0x7f47980c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:11.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f4797e00640 1 --2- 172.21.15.83:0/1462006028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47980b2830 0x7f4798155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:11.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ce00640 1 -- 172.21.15.83:0/1462006028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47980b2830 msgr2=0x7f4798155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:11.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ce00640 1 --2- 172.21.15.83:0/1462006028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47980b2830 0x7f4798155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ce00640 1 -- 172.21.15.83:0/1462006028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47980cfd10 msgr2=0x7f47980c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:11.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ce00640 1 --2- 172.21.15.83:0/1462006028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47980cfd10 0x7f47980c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ce00640 1 -- 172.21.15.83:0/1462006028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47980ce2c0 con 0x7f4798161850 2024-04-25T00:14:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ce00640 1 --2- 172.21.15.83:0/1462006028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798161850 0x7f47980c9570 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f479000c820 tx=0x7f479000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f4794c00640 1 -- 172.21.15.83:0/1462006028 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790015020 con 0x7f4798161850 2024-04-25T00:14:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f4794c00640 1 -- 172.21.15.83:0/1462006028 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4790004240 con 0x7f4798161850 2024-04-25T00:14:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f4794c00640 1 -- 172.21.15.83:0/1462006028 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790003620 con 0x7f4798161850 2024-04-25T00:14:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ee00640 1 -- 172.21.15.83:0/1462006028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47980c6610 con 0x7f4798161850 2024-04-25T00:14:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.124+0000 7f479ee00640 1 -- 172.21.15.83:0/1462006028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47980c6bd0 con 0x7f4798161850 2024-04-25T00:14:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.128+0000 7f479ee00640 1 -- 172.21.15.83:0/1462006028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f479809fdd0 con 0x7f4798161850 2024-04-25T00:14:11.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.128+0000 7f4794c00640 1 -- 172.21.15.83:0/1462006028 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4790018020 con 0x7f4798161850 2024-04-25T00:14:11.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.128+0000 7f4794c00640 1 --2- 172.21.15.83:0/1462006028 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47740678c0 0x7f4774069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:11.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.128+0000 7f4794c00640 1 -- 172.21.15.83:0/1462006028 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6187+0+0 (secure 0 0 0) 0x7f4790085e50 con 0x7f4798161850 2024-04-25T00:14:11.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.128+0000 7f4797e00640 1 --2- 172.21.15.83:0/1462006028 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47740678c0 0x7f4774069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:11.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.128+0000 7f4797e00640 1 --2- 172.21.15.83:0/1462006028 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47740678c0 0x7f4774069d80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f47880af000 tx=0x7f47880a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:11.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.132+0000 7f4794c00640 1 -- 172.21.15.83:0/1462006028 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f479001f020 con 0x7f4798161850 2024-04-25T00:14:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:11.292+0000 7f479ee00640 1 -- 172.21.15.83:0/1462006028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4798155e80 con 0x7f4798161850 2024-04-25T00:14:12.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.275+0000 7f4794c00640 1 -- 172.21.15.83:0/1462006028 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) ==== 181+0+0 (secure 0 0 0) 0x7f47900560c0 con 0x7f4798161850 2024-04-25T00:14:12.278 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:14:12.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.279+0000 7f479ee00640 1 -- 172.21.15.83:0/1462006028 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47740678c0 msgr2=0x7f4774069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:12.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.279+0000 7f479ee00640 1 --2- 172.21.15.83:0/1462006028 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47740678c0 0x7f4774069d80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f47880af000 tx=0x7f47880a7000 comp rx=0 tx=0).stop 2024-04-25T00:14:12.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.279+0000 7f479ee00640 1 -- 172.21.15.83:0/1462006028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798161850 msgr2=0x7f47980c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:12.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.279+0000 7f479ee00640 1 --2- 172.21.15.83:0/1462006028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798161850 0x7f47980c9570 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f479000c820 tx=0x7f479000cce0 comp rx=0 tx=0).stop 2024-04-25T00:14:12.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.279+0000 7f479ee00640 1 -- 172.21.15.83:0/1462006028 shutdown_connections 2024-04-25T00:14:12.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.279+0000 7f479ee00640 1 --2- 172.21.15.83:0/1462006028 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f47740678c0 0x7f4774069d80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:12.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.279+0000 7f479ee00640 1 --2- 172.21.15.83:0/1462006028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798161850 0x7f47980c9570 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:12.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.279+0000 7f479ee00640 1 --2- 172.21.15.83:0/1462006028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47980cfd10 0x7f47980c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:12.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.279+0000 7f479ee00640 1 --2- 172.21.15.83:0/1462006028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47980b2830 0x7f4798155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:12.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.279+0000 7f479ee00640 1 -- 172.21.15.83:0/1462006028 >> 172.21.15.83:0/1462006028 conn(0x7f47980d1c80 msgr2=0x7f47980c4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:12.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.279+0000 7f479ee00640 1 -- 172.21.15.83:0/1462006028 shutdown_connections 2024-04-25T00:14:12.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.279+0000 7f479ee00640 1 -- 172.21.15.83:0/1462006028 wait complete. 2024-04-25T00:14:12.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:14:12.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f22a6600640 1 Processor -- start 2024-04-25T00:14:12.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f22a6600640 1 -- start start 2024-04-25T00:14:12.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f22a6600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f22a01445e0 0x7f22a00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:12.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f22a6600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22a0072120 0x7f22a015bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f22a6600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a015c1a0 0x7f22a00cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f22a6600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f22a015b7d0 con 0x7f22a0072120 2024-04-25T00:14:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f22a6600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f22a015b480 con 0x7f22a01445e0 2024-04-25T00:14:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f22a6600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f22a01449c0 con 0x7f22a015c1a0 2024-04-25T00:14:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f2297e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22a0072120 0x7f22a015bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f2297e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22a0072120 0x7f22a015bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55674/0 (socket says 172.21.15.83:55674) 2024-04-25T00:14:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f2297e00640 1 -- 172.21.15.83:0/3445761871 learned_addr learned my addr 172.21.15.83:0/3445761871 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f229fe00640 1 --2- 172.21.15.83:0/3445761871 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a015c1a0 0x7f22a00cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f229f400640 1 --2- 172.21.15.83:0/3445761871 >> v2:172.21.15.86:3300/0 conn(0x7f22a01445e0 0x7f22a00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:12.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f2297e00640 1 -- 172.21.15.83:0/3445761871 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a015c1a0 msgr2=0x7f22a00cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:12.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f2297e00640 1 --2- 172.21.15.83:0/3445761871 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a015c1a0 0x7f22a00cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:12.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f2297e00640 1 -- 172.21.15.83:0/3445761871 >> v2:172.21.15.86:3300/0 conn(0x7f22a01445e0 msgr2=0x7f22a00b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:12.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f2297e00640 1 --2- 172.21.15.83:0/3445761871 >> v2:172.21.15.86:3300/0 conn(0x7f22a01445e0 0x7f22a00b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:12.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.451+0000 7f2297e00640 1 -- 172.21.15.83:0/3445761871 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a0072910 con 0x7f22a0072120 2024-04-25T00:14:12.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.455+0000 7f2297e00640 1 --2- 172.21.15.83:0/3445761871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a0072120 0x7f22a015bc60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f22a006f000 tx=0x7f228801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1e2dc27c2579e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.455+0000 7f229ea00640 1 -- 172.21.15.83:0/3445761871 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22880b2070 con 0x7f22a0072120 2024-04-25T00:14:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.455+0000 7f229ea00640 1 -- 172.21.15.83:0/3445761871 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22880a9070 con 0x7f22a0072120 2024-04-25T00:14:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.455+0000 7f229ea00640 1 -- 172.21.15.83:0/3445761871 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22880ad880 con 0x7f22a0072120 2024-04-25T00:14:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.455+0000 7f22a6600640 1 -- 172.21.15.83:0/3445761871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a0072120 msgr2=0x7f22a015bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.455+0000 7f22a6600640 1 --2- 172.21.15.83:0/3445761871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a0072120 0x7f22a015bc60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f22a006f000 tx=0x7f228801c490 comp rx=0 tx=0).stop 2024-04-25T00:14:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.455+0000 7f22a6600640 1 -- 172.21.15.83:0/3445761871 shutdown_connections 2024-04-25T00:14:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.455+0000 7f22a6600640 1 --2- 172.21.15.83:0/3445761871 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a015c1a0 0x7f22a00cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.455+0000 7f22a6600640 1 --2- 172.21.15.83:0/3445761871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a0072120 0x7f22a015bc60 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.455+0000 7f22a6600640 1 --2- 172.21.15.83:0/3445761871 >> v2:172.21.15.86:3300/0 conn(0x7f22a01445e0 0x7f22a00b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.455+0000 7f22a6600640 1 -- 172.21.15.83:0/3445761871 >> 172.21.15.83:0/3445761871 conn(0x7f22a006b3b0 msgr2=0x7f22a006b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.455+0000 7f22a6600640 1 -- 172.21.15.83:0/3445761871 shutdown_connections 2024-04-25T00:14:12.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.455+0000 7f22a6600640 1 -- 172.21.15.83:0/3445761871 wait complete. 2024-04-25T00:14:12.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f22a6600640 1 Processor -- start 2024-04-25T00:14:12.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f22a6600640 1 -- start start 2024-04-25T00:14:12.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f22a6600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0072120 0x7f22a00cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:12.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f22a6600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a01445e0 0x7f22a013e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:12.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f22a6600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a01fae30 0x7f22a01f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:12.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f22a6600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22a006ce20 con 0x7f22a01fae30 2024-04-25T00:14:12.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f22a6600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22a00725d0 con 0x7f22a0072120 2024-04-25T00:14:12.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f22a6600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f22a006cfa0 con 0x7f22a01445e0 2024-04-25T00:14:12.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f229fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a01fae30 0x7f22a01f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:12.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f229fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a01fae30 0x7f22a01f77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55678/0 (socket says 172.21.15.83:55678) 2024-04-25T00:14:12.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f229fe00640 1 -- 172.21.15.83:0/880974443 learned_addr learned my addr 172.21.15.83:0/880974443 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:12.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f229f400640 1 --2- 172.21.15.83:0/880974443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0072120 0x7f22a00cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:12.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f2297e00640 1 --2- 172.21.15.83:0/880974443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a01445e0 0x7f22a013e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:12.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f229fe00640 1 -- 172.21.15.83:0/880974443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a01445e0 msgr2=0x7f22a013e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:12.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f229fe00640 1 --2- 172.21.15.83:0/880974443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a01445e0 0x7f22a013e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:12.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f229fe00640 1 -- 172.21.15.83:0/880974443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0072120 msgr2=0x7f22a00cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:12.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f229fe00640 1 --2- 172.21.15.83:0/880974443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0072120 0x7f22a00cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:12.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f229fe00640 1 -- 172.21.15.83:0/880974443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a013ef00 con 0x7f22a01fae30 2024-04-25T00:14:12.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f229fe00640 1 --2- 172.21.15.83:0/880974443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a01fae30 0x7f22a01f77d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f229800efc0 tx=0x7f229800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:12.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f2297400640 1 -- 172.21.15.83:0/880974443 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298018020 con 0x7f22a01fae30 2024-04-25T00:14:12.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f2297400640 1 -- 172.21.15.83:0/880974443 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2298014480 con 0x7f22a01fae30 2024-04-25T00:14:12.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f22a6600640 1 -- 172.21.15.83:0/880974443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a01f8070 con 0x7f22a01fae30 2024-04-25T00:14:12.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f2297400640 1 -- 172.21.15.83:0/880974443 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298009e60 con 0x7f22a01fae30 2024-04-25T00:14:12.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.459+0000 7f22a6600640 1 -- 172.21.15.83:0/880974443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a01f7d10 con 0x7f22a01fae30 2024-04-25T00:14:12.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.463+0000 7f22a6600640 1 -- 172.21.15.83:0/880974443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a00736f0 con 0x7f22a01fae30 2024-04-25T00:14:12.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.463+0000 7f2297400640 1 -- 172.21.15.83:0/880974443 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f229801b020 con 0x7f22a01fae30 2024-04-25T00:14:12.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.463+0000 7f2297400640 1 --2- 172.21.15.83:0/880974443 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f227c0678c0 0x7f227c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:12.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.463+0000 7f2297400640 1 -- 172.21.15.83:0/880974443 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5749+0+0 (secure 0 0 0) 0x7f2298086910 con 0x7f22a01fae30 2024-04-25T00:14:12.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.463+0000 7f229f400640 1 --2- 172.21.15.83:0/880974443 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f227c0678c0 0x7f227c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:12.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.463+0000 7f229f400640 1 --2- 172.21.15.83:0/880974443 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f227c0678c0 0x7f227c069d80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2290009ae0 tx=0x7f2290009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:12.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.467+0000 7f2297400640 1 -- 172.21.15.83:0/880974443 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2298022030 con 0x7f22a01fae30 2024-04-25T00:14:12.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:12.623+0000 7f22a6600640 1 -- 172.21.15.83:0/880974443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f22a004f5f0 con 0x7f22a01fae30 2024-04-25T00:14:13.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.279+0000 7f2297400640 1 -- 172.21.15.83:0/880974443 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) ==== 169+0+0 (secure 0 0 0) 0x7f2298010040 con 0x7f22a01fae30 2024-04-25T00:14:13.284 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-25T00:14:13.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.283+0000 7f22a6600640 1 -- 172.21.15.83:0/880974443 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f227c0678c0 msgr2=0x7f227c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.283+0000 7f22a6600640 1 --2- 172.21.15.83:0/880974443 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f227c0678c0 0x7f227c069d80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2290009ae0 tx=0x7f2290009190 comp rx=0 tx=0).stop 2024-04-25T00:14:13.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.283+0000 7f22a6600640 1 -- 172.21.15.83:0/880974443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a01fae30 msgr2=0x7f22a01f77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.283+0000 7f22a6600640 1 --2- 172.21.15.83:0/880974443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a01fae30 0x7f22a01f77d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f229800efc0 tx=0x7f229800c490 comp rx=0 tx=0).stop 2024-04-25T00:14:13.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.283+0000 7f22a6600640 1 -- 172.21.15.83:0/880974443 shutdown_connections 2024-04-25T00:14:13.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.283+0000 7f22a6600640 1 --2- 172.21.15.83:0/880974443 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f227c0678c0 0x7f227c069d80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.283+0000 7f22a6600640 1 --2- 172.21.15.83:0/880974443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22a01445e0 0x7f22a013e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.283+0000 7f22a6600640 1 --2- 172.21.15.83:0/880974443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a01fae30 0x7f22a01f77d0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.283+0000 7f22a6600640 1 --2- 172.21.15.83:0/880974443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a0072120 0x7f22a00cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.283+0000 7f22a6600640 1 -- 172.21.15.83:0/880974443 >> 172.21.15.83:0/880974443 conn(0x7f22a006b3b0 msgr2=0x7f22a0142710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:13.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.283+0000 7f22a6600640 1 -- 172.21.15.83:0/880974443 shutdown_connections 2024-04-25T00:14:13.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.283+0000 7f22a6600640 1 -- 172.21.15.83:0/880974443 wait complete. 2024-04-25T00:14:13.312 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:14:13.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.447+0000 7f4592800640 1 Processor -- start 2024-04-25T00:14:13.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.447+0000 7f4592800640 1 -- start start 2024-04-25T00:14:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f4592800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f458c06e950 0x7f458c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f4592800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458c0ce000 con 0x7f458c15fe50 2024-04-25T00:14:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f4592800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458c06eec0 con 0x7f458c159810 2024-04-25T00:14:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f4592800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f458c06e210 con 0x7f458c06e950 2024-04-25T00:14:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458be00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f458c15fe50 0x7f458c0ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46330/0 (socket says 172.21.15.83:46330) 2024-04-25T00:14:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458be00640 1 -- 172.21.15.83:0/1775407242 learned_addr learned my addr 172.21.15.83:0/1775407242 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458aa00640 1 --2- 172.21.15.83:0/1775407242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f458c06e950 0x7f458c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003268279 0 0) 0x7f458c0ce000 con 0x7f458c15fe50 2024-04-25T00:14:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4570003530 con 0x7f458c15fe50 2024-04-25T00:14:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 501397707 0 0) 0x7f458c06eec0 con 0x7f458c159810 2024-04-25T00:14:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f458c0ce000 con 0x7f458c159810 2024-04-25T00:14:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3442840546 0 0) 0x7f4570003530 con 0x7f458c15fe50 2024-04-25T00:14:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f458c06e950 msgr2=0x7f458c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 --2- 172.21.15.83:0/1775407242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f458c06e950 0x7f458c15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 >> v1:172.21.15.86:6789/0 conn(0x7f458c159810 legacy=0x7f458c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c06dc50 con 0x7f458c15fe50 2024-04-25T00:14:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f4584001a00 con 0x7f458c15fe50 2024-04-25T00:14:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 >> v1:172.21.15.83:6789/0 conn(0x7f458c15fe50 legacy=0x7f458c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 --2- 172.21.15.83:0/1775407242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f458c06e950 0x7f4570003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 --2- 172.21.15.83:0/1775407242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4570004150 0x7f4570004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 --2- 172.21.15.83:0/1775407242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4570004d50 0x7f45700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f458c06e210 con 0x7f4570004d50 2024-04-25T00:14:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f457000bbc0 con 0x7f4570004150 2024-04-25T00:14:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f457000bdc0 con 0x7f458c06e950 2024-04-25T00:14:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458b400640 1 --2- 172.21.15.83:0/1775407242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f458c06e950 0x7f4570003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458be00640 1 --2- 172.21.15.83:0/1775407242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4570004d50 0x7f45700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458aa00640 1 --2- 172.21.15.83:0/1775407242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4570004150 0x7f4570004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458be00640 1 -- 172.21.15.83:0/1775407242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f458c06e950 msgr2=0x7f4570003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458be00640 1 --2- 172.21.15.83:0/1775407242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f458c06e950 0x7f4570003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458be00640 1 -- 172.21.15.83:0/1775407242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4570004150 msgr2=0x7f4570004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458be00640 1 --2- 172.21.15.83:0/1775407242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4570004150 0x7f4570004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.451+0000 7f458be00640 1 -- 172.21.15.83:0/1775407242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4570003530 con 0x7f4570004d50 2024-04-25T00:14:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.455+0000 7f458be00640 1 --2- 172.21.15.83:0/1775407242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4570004d50 0x7f45700051d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f458400a7f0 tx=0x7f458401d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d52aaada759684 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.455+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458400a410 con 0x7f4570004d50 2024-04-25T00:14:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.455+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45840ad650 con 0x7f4570004d50 2024-04-25T00:14:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.455+0000 7f458a000640 1 -- 172.21.15.83:0/1775407242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45840ada80 con 0x7f4570004d50 2024-04-25T00:14:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.455+0000 7f4592800640 1 -- 172.21.15.83:0/1775407242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4570004d50 msgr2=0x7f45700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.455+0000 7f4592800640 1 --2- 172.21.15.83:0/1775407242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4570004d50 0x7f45700051d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f458400a7f0 tx=0x7f458401d300 comp rx=0 tx=0).stop 2024-04-25T00:14:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.455+0000 7f458be00640 1 -- 172.21.15.83:0/1775407242 reap_dead start 2024-04-25T00:14:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.455+0000 7f4592800640 1 -- 172.21.15.83:0/1775407242 shutdown_connections 2024-04-25T00:14:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.455+0000 7f4592800640 1 -- 172.21.15.83:0/1775407242 >> 172.21.15.83:0/1775407242 conn(0x7f458c0b1830 msgr2=0x7f458c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.455+0000 7f4592800640 1 -- 172.21.15.83:0/1775407242 shutdown_connections 2024-04-25T00:14:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.455+0000 7f4592800640 1 -- 172.21.15.83:0/1775407242 wait complete. 2024-04-25T00:14:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f4592800640 1 Processor -- start 2024-04-25T00:14:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f4592800640 1 -- start start 2024-04-25T00:14:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f4592800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4570004d50 0x7f458c140e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f4592800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f458c153530 0x7f458c153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f4592800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f458c1576d0 0x7f458c157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f4592800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f458c159bf0 con 0x7f458c1576d0 2024-04-25T00:14:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f4592800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f458c06da00 con 0x7f458c153530 2024-04-25T00:14:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f4592800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f458c159d70 con 0x7f4570004d50 2024-04-25T00:14:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f458be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f458c1576d0 0x7f458c157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f458be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f458c1576d0 0x7f458c157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55708/0 (socket says 172.21.15.83:55708) 2024-04-25T00:14:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f458be00640 1 -- 172.21.15.83:0/3307835243 learned_addr learned my addr 172.21.15.83:0/3307835243 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f458b400640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4570004d50 0x7f458c140e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f458aa00640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f458c153530 0x7f458c153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f458be00640 1 -- 172.21.15.83:0/3307835243 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4570004d50 msgr2=0x7f458c140e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f458be00640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4570004d50 0x7f458c140e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f458be00640 1 -- 172.21.15.83:0/3307835243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f458c153530 msgr2=0x7f458c153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f458be00640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f458c153530 0x7f458c153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f458be00640 1 -- 172.21.15.83:0/3307835243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c0afad0 con 0x7f458c1576d0 2024-04-25T00:14:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f458b400640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4570004d50 0x7f458c140e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f458be00640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f458c1576d0 0x7f458c157b80 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f458401dba0 tx=0x7f4570005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f4583400640 1 -- 172.21.15.83:0/3307835243 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45840bc070 con 0x7f458c1576d0 2024-04-25T00:14:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f4583400640 1 -- 172.21.15.83:0/3307835243 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45840013b0 con 0x7f458c1576d0 2024-04-25T00:14:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f4583400640 1 -- 172.21.15.83:0/3307835243 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584001800 con 0x7f458c1576d0 2024-04-25T00:14:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f4592800640 1 -- 172.21.15.83:0/3307835243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c0afdb0 con 0x7f458c1576d0 2024-04-25T00:14:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f4592800640 1 -- 172.21.15.83:0/3307835243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c0b0370 con 0x7f458c1576d0 2024-04-25T00:14:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.459+0000 7f4592800640 1 -- 172.21.15.83:0/3307835243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f458c04f510 con 0x7f458c1576d0 2024-04-25T00:14:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.463+0000 7f4583400640 1 -- 172.21.15.83:0/3307835243 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f45840bf070 con 0x7f458c1576d0 2024-04-25T00:14:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.463+0000 7f4583400640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f456c0678c0 0x7f456c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.463+0000 7f4583400640 1 -- 172.21.15.83:0/3307835243 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7f458411fb40 con 0x7f458c1576d0 2024-04-25T00:14:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.463+0000 7f458b400640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f456c0678c0 0x7f456c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.463+0000 7f458b400640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f456c0678c0 0x7f456c069d80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f457c000c10 tx=0x7f457c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.467+0000 7f4583400640 1 -- 172.21.15.83:0/3307835243 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4584017010 con 0x7f458c1576d0 2024-04-25T00:14:13.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.631+0000 7f4592800640 1 -- 172.21.15.83:0/3307835243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f458c072c90 con 0x7f458c1576d0 2024-04-25T00:14:13.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.631+0000 7f4583400640 1 -- 172.21.15.83:0/3307835243 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f45840f06c0 con 0x7f458c1576d0 2024-04-25T00:14:13.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.635+0000 7f4592800640 1 -- 172.21.15.83:0/3307835243 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f456c0678c0 msgr2=0x7f456c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.635+0000 7f4592800640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f456c0678c0 0x7f456c069d80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f457c000c10 tx=0x7f457c009040 comp rx=0 tx=0).stop 2024-04-25T00:14:13.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.635+0000 7f4592800640 1 -- 172.21.15.83:0/3307835243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f458c1576d0 msgr2=0x7f458c157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.635+0000 7f4592800640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f458c1576d0 0x7f458c157b80 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f458401dba0 tx=0x7f4570005840 comp rx=0 tx=0).stop 2024-04-25T00:14:13.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.635+0000 7f4592800640 1 -- 172.21.15.83:0/3307835243 shutdown_connections 2024-04-25T00:14:13.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.635+0000 7f4592800640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f456c0678c0 0x7f456c069d80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.635+0000 7f4592800640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f458c1576d0 0x7f458c157b80 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.635+0000 7f4592800640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f458c153530 0x7f458c153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.635+0000 7f4592800640 1 --2- 172.21.15.83:0/3307835243 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4570004d50 0x7f458c140e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.635+0000 7f4592800640 1 -- 172.21.15.83:0/3307835243 >> 172.21.15.83:0/3307835243 conn(0x7f458c0b1830 msgr2=0x7f458c0ce530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:13.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.635+0000 7f4592800640 1 -- 172.21.15.83:0/3307835243 shutdown_connections 2024-04-25T00:14:13.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.635+0000 7f4592800640 1 -- 172.21.15.83:0/3307835243 wait complete. 2024-04-25T00:14:13.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds ms_async_reap_threshold 2024-04-25T00:14:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.791+0000 7fb8ff200640 1 Processor -- start 2024-04-25T00:14:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.791+0000 7fb8ff200640 1 -- start start 2024-04-25T00:14:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.791+0000 7fb8ff200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb9001455d0 0x7fb90006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.791+0000 7fb8ff200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9000727b0 0x7fb90015f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.791+0000 7fb8ff200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb900070ca0 con 0x7fb900072c10 2024-04-25T00:14:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.791+0000 7fb8ff200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb9000ce9d0 con 0x7fb9001455d0 2024-04-25T00:14:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.791+0000 7fb8ff200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb900070fa0 con 0x7fb9000727b0 2024-04-25T00:14:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.791+0000 7fb8fde00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb9001455d0 0x7fb90006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.791+0000 7fb8fe800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb900072c10 0x7fb900071430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46334/0 (socket says 172.21.15.83:46334) 2024-04-25T00:14:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.791+0000 7fb8fe800640 1 -- 172.21.15.83:0/2162047073 learned_addr learned my addr 172.21.15.83:0/2162047073 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.795+0000 7fb8f5400640 1 --2- 172.21.15.83:0/2162047073 >> v2:172.21.15.86:3300/0 conn(0x7fb9000727b0 0x7fb90015f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.795+0000 7fb8fde00640 1 -- 172.21.15.83:0/2162047073 >> v2:172.21.15.86:3300/0 conn(0x7fb9000727b0 msgr2=0x7fb90015f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.795+0000 7fb8fde00640 1 --2- 172.21.15.83:0/2162047073 >> v2:172.21.15.86:3300/0 conn(0x7fb9000727b0 0x7fb90015f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.795+0000 7fb8fde00640 1 -- 172.21.15.83:0/2162047073 >> v1:172.21.15.83:6789/0 conn(0x7fb900072c10 legacy=0x7fb900071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.795+0000 7fb8fde00640 1 -- 172.21.15.83:0/2162047073 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90006d1a0 con 0x7fb9001455d0 2024-04-25T00:14:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.795+0000 7fb8fd400640 1 -- 172.21.15.83:0/2162047073 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637614151 0 0) 0x7fb900070ca0 con 0x7fb900072c10 2024-04-25T00:14:13.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.795+0000 7fb8fde00640 1 --2- 172.21.15.83:0/2162047073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9001455d0 0x7fb90006cc60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb90006e8b0 tx=0x7fb8f001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75123cedac752721 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:13.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.795+0000 7fb8fd400640 1 -- 172.21.15.83:0/2162047073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00b0070 con 0x7fb9001455d0 2024-04-25T00:14:13.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.795+0000 7fb8fd400640 1 -- 172.21.15.83:0/2162047073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f00ab600 con 0x7fb9001455d0 2024-04-25T00:14:13.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.795+0000 7fb8fd400640 1 -- 172.21.15.83:0/2162047073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00ab9d0 con 0x7fb9001455d0 2024-04-25T00:14:13.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.795+0000 7fb8ff200640 1 -- 172.21.15.83:0/2162047073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9001455d0 msgr2=0x7fb90006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.795+0000 7fb8ff200640 1 --2- 172.21.15.83:0/2162047073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9001455d0 0x7fb90006cc60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb90006e8b0 tx=0x7fb8f001c480 comp rx=0 tx=0).stop 2024-04-25T00:14:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8ff200640 1 -- 172.21.15.83:0/2162047073 shutdown_connections 2024-04-25T00:14:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8ff200640 1 --2- 172.21.15.83:0/2162047073 >> v2:172.21.15.86:3300/0 conn(0x7fb9000727b0 0x7fb90015f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8ff200640 1 --2- 172.21.15.83:0/2162047073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9001455d0 0x7fb90006cc60 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8ff200640 1 -- 172.21.15.83:0/2162047073 >> 172.21.15.83:0/2162047073 conn(0x7fb9000cf980 msgr2=0x7fb9000cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8ff200640 1 -- 172.21.15.83:0/2162047073 shutdown_connections 2024-04-25T00:14:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8ff200640 1 -- 172.21.15.83:0/2162047073 wait complete. 2024-04-25T00:14:13.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8ff200640 1 Processor -- start 2024-04-25T00:14:13.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8ff200640 1 -- start start 2024-04-25T00:14:13.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8ff200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9000727b0 0x7fb90015b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8ff200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb900072c10 0x7fb90013e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8ff200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9001455d0 0x7fb9000b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8ff200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9000cf1e0 con 0x7fb9000727b0 2024-04-25T00:14:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8ff200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb900071f50 con 0x7fb900072c10 2024-04-25T00:14:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8ff200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb9001693b0 con 0x7fb9001455d0 2024-04-25T00:14:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8fde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9000727b0 0x7fb90015b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8fde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9000727b0 0x7fb90015b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55726/0 (socket says 172.21.15.83:55726) 2024-04-25T00:14:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8fde00640 1 -- 172.21.15.83:0/3519597503 learned_addr learned my addr 172.21.15.83:0/3519597503 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8f5400640 1 --2- 172.21.15.83:0/3519597503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb900072c10 0x7fb90013e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8fe800640 1 --2- 172.21.15.83:0/3519597503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9001455d0 0x7fb9000b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8fde00640 1 -- 172.21.15.83:0/3519597503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9001455d0 msgr2=0x7fb9000b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8fde00640 1 --2- 172.21.15.83:0/3519597503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9001455d0 0x7fb9000b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8fde00640 1 -- 172.21.15.83:0/3519597503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb900072c10 msgr2=0x7fb90013e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8fde00640 1 --2- 172.21.15.83:0/3519597503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb900072c10 0x7fb90013e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8fde00640 1 -- 172.21.15.83:0/3519597503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90013ece0 con 0x7fb9000727b0 2024-04-25T00:14:13.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8fde00640 1 --2- 172.21.15.83:0/3519597503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9000727b0 0x7fb90015b400 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb8f001a0b0 tx=0x7fb8f001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:13.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.799+0000 7fb8f6a00640 1 -- 172.21.15.83:0/3519597503 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00bd070 con 0x7fb9000727b0 2024-04-25T00:14:13.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.803+0000 7fb8f6a00640 1 -- 172.21.15.83:0/3519597503 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f0004030 con 0x7fb9000727b0 2024-04-25T00:14:13.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.803+0000 7fb8f6a00640 1 -- 172.21.15.83:0/3519597503 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00b0040 con 0x7fb9000727b0 2024-04-25T00:14:13.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.803+0000 7fb8ff200640 1 -- 172.21.15.83:0/3519597503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90013efc0 con 0x7fb9000727b0 2024-04-25T00:14:13.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.803+0000 7fb8ff200640 1 -- 172.21.15.83:0/3519597503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb90013f580 con 0x7fb9000727b0 2024-04-25T00:14:13.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.803+0000 7fb8ff200640 1 -- 172.21.15.83:0/3519597503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb90004f510 con 0x7fb9000727b0 2024-04-25T00:14:13.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.803+0000 7fb8f6a00640 1 -- 172.21.15.83:0/3519597503 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb8f0003620 con 0x7fb9000727b0 2024-04-25T00:14:13.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.803+0000 7fb8f6a00640 1 --2- 172.21.15.83:0/3519597503 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8dc0678f0 0x7fb8dc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.803+0000 7fb8f6a00640 1 -- 172.21.15.83:0/3519597503 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7fb8f011f700 con 0x7fb9000727b0 2024-04-25T00:14:13.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.803+0000 7fb8f5400640 1 --2- 172.21.15.83:0/3519597503 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8dc0678f0 0x7fb8dc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.803+0000 7fb8f5400640 1 --2- 172.21.15.83:0/3519597503 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8dc0678f0 0x7fb8dc069db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb90015bc80 tx=0x7fb8e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:13.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.807+0000 7fb8f6a00640 1 -- 172.21.15.83:0/3519597503 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb8f0016070 con 0x7fb9000727b0 2024-04-25T00:14:13.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.963+0000 7fb8ff200640 1 -- 172.21.15.83:0/3519597503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"} v 0) -- 0x7fb90006e8e0 con 0x7fb9000727b0 2024-04-25T00:14:13.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.967+0000 7fb8f6a00640 1 -- 172.21.15.83:0/3519597503 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"}]=0 v21) ==== 106+0+0 (secure 0 0 0) 0x7fb8f00efcc0 con 0x7fb9000727b0 2024-04-25T00:14:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.971+0000 7fb8ff200640 1 -- 172.21.15.83:0/3519597503 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8dc0678f0 msgr2=0x7fb8dc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.971+0000 7fb8ff200640 1 --2- 172.21.15.83:0/3519597503 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8dc0678f0 0x7fb8dc069db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb90015bc80 tx=0x7fb8e8008040 comp rx=0 tx=0).stop 2024-04-25T00:14:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.971+0000 7fb8ff200640 1 -- 172.21.15.83:0/3519597503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9000727b0 msgr2=0x7fb90015b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.971+0000 7fb8ff200640 1 --2- 172.21.15.83:0/3519597503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9000727b0 0x7fb90015b400 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb8f001a0b0 tx=0x7fb8f001acc0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.971+0000 7fb8ff200640 1 -- 172.21.15.83:0/3519597503 shutdown_connections 2024-04-25T00:14:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.971+0000 7fb8ff200640 1 --2- 172.21.15.83:0/3519597503 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8dc0678f0 0x7fb8dc069db0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.971+0000 7fb8ff200640 1 --2- 172.21.15.83:0/3519597503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9001455d0 0x7fb9000b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.971+0000 7fb8ff200640 1 --2- 172.21.15.83:0/3519597503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb900072c10 0x7fb90013e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.971+0000 7fb8ff200640 1 --2- 172.21.15.83:0/3519597503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9000727b0 0x7fb90015b400 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.971+0000 7fb8ff200640 1 -- 172.21.15.83:0/3519597503 >> 172.21.15.83:0/3519597503 conn(0x7fb9000cf980 msgr2=0x7fb90013c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:13.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.971+0000 7fb8ff200640 1 -- 172.21.15.83:0/3519597503 shutdown_connections 2024-04-25T00:14:13.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:13.971+0000 7fb8ff200640 1 -- 172.21.15.83:0/3519597503 wait complete. 2024-04-25T00:14:13.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close' 2024-04-25T00:14:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.123+0000 7f41b8c00640 1 Processor -- start 2024-04-25T00:14:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.123+0000 7f41b8c00640 1 -- start start 2024-04-25T00:14:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.123+0000 7f41b8c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41b40cc510 0x7f41b415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:14.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.123+0000 7f41b8c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f41b415cf90 0x7f41b40ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:14.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.123+0000 7f41b8c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b40ccc60 con 0x7f41b41599f0 2024-04-25T00:14:14.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.123+0000 7f41b8c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f41b4159dd0 con 0x7f41b415cf90 2024-04-25T00:14:14.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.123+0000 7f41b8c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f41b406dba0 con 0x7f41b40cc510 2024-04-25T00:14:14.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.123+0000 7f41b2800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f41b415cf90 0x7f41b40ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:14.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.127+0000 7f41b2800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f41b415cf90 0x7f41b40ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55736/0 (socket says 172.21.15.83:55736) 2024-04-25T00:14:14.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.127+0000 7f41b2800640 1 -- 172.21.15.83:0/2103199357 learned_addr learned my addr 172.21.15.83:0/2103199357 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:14.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.127+0000 7f41b1400640 1 --2- 172.21.15.83:0/2103199357 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41b40cc510 0x7f41b415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:14.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.127+0000 7f41b2800640 1 -- 172.21.15.83:0/2103199357 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41b40cc510 msgr2=0x7f41b415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:14.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.127+0000 7f41b2800640 1 --2- 172.21.15.83:0/2103199357 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41b40cc510 0x7f41b415ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.127+0000 7f41b2800640 1 -- 172.21.15.83:0/2103199357 >> v1:172.21.15.86:6789/0 conn(0x7f41b41599f0 legacy=0x7f41b415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:14.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.127+0000 7f41b2800640 1 -- 172.21.15.83:0/2103199357 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b406de80 con 0x7f41b415cf90 2024-04-25T00:14:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.127+0000 7f41b2800640 1 --2- 172.21.15.83:0/2103199357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b415cf90 0x7f41b40ce6e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009e50 tx=0x7f41ac01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe424b956bdb30a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.127+0000 7f41b0a00640 1 -- 172.21.15.83:0/2103199357 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0a56b0 con 0x7f41b415cf90 2024-04-25T00:14:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.127+0000 7f41b0a00640 1 -- 172.21.15.83:0/2103199357 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ac0a5850 con 0x7f41b415cf90 2024-04-25T00:14:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.127+0000 7f41b0a00640 1 -- 172.21.15.83:0/2103199357 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0a5c20 con 0x7f41b415cf90 2024-04-25T00:14:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 -- 172.21.15.83:0/2103199357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b415cf90 msgr2=0x7f41b40ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 --2- 172.21.15.83:0/2103199357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b415cf90 0x7f41b40ce6e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009e50 tx=0x7f41ac01c9a0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 -- 172.21.15.83:0/2103199357 shutdown_connections 2024-04-25T00:14:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 --2- 172.21.15.83:0/2103199357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b415cf90 0x7f41b40ce6e0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 --2- 172.21.15.83:0/2103199357 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41b40cc510 0x7f41b415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 -- 172.21.15.83:0/2103199357 >> 172.21.15.83:0/2103199357 conn(0x7f41b40b1830 msgr2=0x7f41b40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 -- 172.21.15.83:0/2103199357 shutdown_connections 2024-04-25T00:14:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 -- 172.21.15.83:0/2103199357 wait complete. 2024-04-25T00:14:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 Processor -- start 2024-04-25T00:14:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 -- start start 2024-04-25T00:14:14.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41b40cc510 0x7f41b415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:14.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41b41599f0 0x7f41b414a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:14.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b415cf90 0x7f41b4151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:14.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41b40cf5b0 con 0x7f41b415cf90 2024-04-25T00:14:14.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41b40cf430 con 0x7f41b41599f0 2024-04-25T00:14:14.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b8c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f41b40cf730 con 0x7f41b40cc510 2024-04-25T00:14:14.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b415cf90 0x7f41b4151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b415cf90 0x7f41b4151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55750/0 (socket says 172.21.15.83:55750) 2024-04-25T00:14:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b2800640 1 -- 172.21.15.83:0/1115737468 learned_addr learned my addr 172.21.15.83:0/1115737468 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b1e00640 1 --2- 172.21.15.83:0/1115737468 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41b40cc510 0x7f41b415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b1400640 1 --2- 172.21.15.83:0/1115737468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41b41599f0 0x7f41b414a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b2800640 1 -- 172.21.15.83:0/1115737468 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41b40cc510 msgr2=0x7f41b415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b2800640 1 --2- 172.21.15.83:0/1115737468 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41b40cc510 0x7f41b415d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b2800640 1 -- 172.21.15.83:0/1115737468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41b41599f0 msgr2=0x7f41b414a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b2800640 1 --2- 172.21.15.83:0/1115737468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41b41599f0 0x7f41b414a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.131+0000 7f41b2800640 1 -- 172.21.15.83:0/1115737468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b41f8470 con 0x7f41b415cf90 2024-04-25T00:14:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.135+0000 7f41b2800640 1 --2- 172.21.15.83:0/1115737468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b415cf90 0x7f41b4151240 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f41ac01a5d0 tx=0x7f41ac019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.135+0000 7f41aa000640 1 -- 172.21.15.83:0/1115737468 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0bc070 con 0x7f41b415cf90 2024-04-25T00:14:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.135+0000 7f41aa000640 1 -- 172.21.15.83:0/1115737468 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ac0b8660 con 0x7f41b415cf90 2024-04-25T00:14:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.135+0000 7f41b8c00640 1 -- 172.21.15.83:0/1115737468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b415da20 con 0x7f41b415cf90 2024-04-25T00:14:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.135+0000 7f41aa000640 1 -- 172.21.15.83:0/1115737468 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac009510 con 0x7f41b415cf90 2024-04-25T00:14:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.135+0000 7f41b8c00640 1 -- 172.21.15.83:0/1115737468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b41f7990 con 0x7f41b415cf90 2024-04-25T00:14:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.135+0000 7f41b8c00640 1 -- 172.21.15.83:0/1115737468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b40737f0 con 0x7f41b415cf90 2024-04-25T00:14:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.135+0000 7f41aa000640 1 -- 172.21.15.83:0/1115737468 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f41ac0bf070 con 0x7f41b415cf90 2024-04-25T00:14:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.135+0000 7f41aa000640 1 --2- 172.21.15.83:0/1115737468 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f41940678c0 0x7f4194069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.135+0000 7f41aa000640 1 -- 172.21.15.83:0/1115737468 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7f41ac1208f0 con 0x7f41b415cf90 2024-04-25T00:14:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.135+0000 7f41b1e00640 1 --2- 172.21.15.83:0/1115737468 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f41940678c0 0x7f4194069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.139+0000 7f41b1e00640 1 --2- 172.21.15.83:0/1115737468 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f41940678c0 0x7f4194069d80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f41b414b140 tx=0x7f41a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:14.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.139+0000 7f41aa000640 1 -- 172.21.15.83:0/1115737468 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f41ac0c6030 con 0x7f41b415cf90 2024-04-25T00:14:14.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:14.351+0000 7f41b8c00640 1 -- 172.21.15.83:0/1115737468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]} v 0) -- 0x7f41b406d980 con 0x7f41b415cf90 2024-04-25T00:14:15.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.287+0000 7f41aa000640 1 -- 172.21.15.83:0/1115737468 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]}]=0 v209) ==== 143+0+0 (secure 0 0 0) 0x7f41ac016620 con 0x7f41b415cf90 2024-04-25T00:14:15.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.291+0000 7f41b8c00640 1 -- 172.21.15.83:0/1115737468 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f41940678c0 msgr2=0x7f4194069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.291+0000 7f41b8c00640 1 --2- 172.21.15.83:0/1115737468 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f41940678c0 0x7f4194069d80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f41b414b140 tx=0x7f41a4009040 comp rx=0 tx=0).stop 2024-04-25T00:14:15.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.291+0000 7f41b8c00640 1 -- 172.21.15.83:0/1115737468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b415cf90 msgr2=0x7f41b4151240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.291+0000 7f41b8c00640 1 --2- 172.21.15.83:0/1115737468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b415cf90 0x7f41b4151240 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f41ac01a5d0 tx=0x7f41ac019740 comp rx=0 tx=0).stop 2024-04-25T00:14:15.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.291+0000 7f41b8c00640 1 -- 172.21.15.83:0/1115737468 shutdown_connections 2024-04-25T00:14:15.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.291+0000 7f41b8c00640 1 --2- 172.21.15.83:0/1115737468 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f41940678c0 0x7f4194069d80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.291+0000 7f41b8c00640 1 --2- 172.21.15.83:0/1115737468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b415cf90 0x7f41b4151240 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.291+0000 7f41b8c00640 1 --2- 172.21.15.83:0/1115737468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41b41599f0 0x7f41b414a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.291+0000 7f41b8c00640 1 --2- 172.21.15.83:0/1115737468 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41b40cc510 0x7f41b415d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.291+0000 7f41b8c00640 1 -- 172.21.15.83:0/1115737468 >> 172.21.15.83:0/1115737468 conn(0x7f41b40b1830 msgr2=0x7f41b415a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:15.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.291+0000 7f41b8c00640 1 -- 172.21.15.83:0/1115737468 shutdown_connections 2024-04-25T00:14:15.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.291+0000 7f41b8c00640 1 -- 172.21.15.83:0/1115737468 wait complete. 2024-04-25T00:14:15.318 INFO:tasks.cephfs_test_runner:That when a `tell` command is sent using the python CLI, ... ok 2024-04-25T00:14:15.318 INFO:tasks.cephfs_test_runner:Starting test: test_tell_session_drop (tasks.cephfs.test_sessionmap.TestSessionMap) That when a `tell` command is sent using the python CLI, 2024-04-25T00:14:15.318 INFO:tasks.cephfs_test_runner:test_tell_session_drop (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-04-25T00:14:15.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:15.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa38d400640 1 Processor -- start 2024-04-25T00:14:15.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa38d400640 1 -- start start 2024-04-25T00:14:15.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa38d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa3881618d0 0x7fa3880ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa38d400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3880d1930 0x7fa3880ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa38d400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa388161cb0 con 0x7fa3880cf070 2024-04-25T00:14:15.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa38d400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa3880d11d0 con 0x7fa3881618d0 2024-04-25T00:14:15.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa38d400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa3880d0970 con 0x7fa3880d1930 2024-04-25T00:14:15.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa387200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa3880cf070 0x7fa38806c2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38230/0 (socket says 172.21.15.83:38230) 2024-04-25T00:14:15.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa387200640 1 -- 172.21.15.83:0/3409477691 learned_addr learned my addr 172.21.15.83:0/3409477691 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:15.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa386800640 1 --2- 172.21.15.83:0/3409477691 >> v2:172.21.15.86:3300/0 conn(0x7fa3881618d0 0x7fa3880ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa385e00640 1 --2- 172.21.15.83:0/3409477691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3880d1930 0x7fa3880ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa385400640 1 -- 172.21.15.83:0/3409477691 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3821206381 0 0) 0x7fa388161cb0 con 0x7fa3880cf070 2024-04-25T00:14:15.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa385400640 1 -- 172.21.15.83:0/3409477691 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa36c003530 con 0x7fa3880cf070 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa386800640 1 -- 172.21.15.83:0/3409477691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3880d1930 msgr2=0x7fa3880ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa386800640 1 --2- 172.21.15.83:0/3409477691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3880d1930 0x7fa3880ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa385400640 1 -- 172.21.15.83:0/3409477691 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2530697140 0 0) 0x7fa36c003530 con 0x7fa3880cf070 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa386800640 1 -- 172.21.15.83:0/3409477691 >> v1:172.21.15.83:6789/0 conn(0x7fa3880cf070 legacy=0x7fa38806c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.455+0000 7fa386800640 1 -- 172.21.15.83:0/3409477691 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3880d0c10 con 0x7fa3881618d0 2024-04-25T00:14:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.459+0000 7fa386800640 1 --2- 172.21.15.83:0/3409477691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3881618d0 0x7fa3880ce5f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa3780097c0 tx=0x7fa37801cdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb43ebf2e3625a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:15.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.459+0000 7fa385400640 1 -- 172.21.15.83:0/3409477691 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3780b0070 con 0x7fa3881618d0 2024-04-25T00:14:15.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.459+0000 7fa385400640 1 -- 172.21.15.83:0/3409477691 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3780ab400 con 0x7fa3881618d0 2024-04-25T00:14:15.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.459+0000 7fa385400640 1 -- 172.21.15.83:0/3409477691 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380001a00 con 0x7fa3881618d0 2024-04-25T00:14:15.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.459+0000 7fa38d400640 1 -- 172.21.15.83:0/3409477691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3881618d0 msgr2=0x7fa3880ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.459+0000 7fa38d400640 1 --2- 172.21.15.83:0/3409477691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3881618d0 0x7fa3880ce5f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa3780097c0 tx=0x7fa37801cdf0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.459+0000 7fa38d400640 1 -- 172.21.15.83:0/3409477691 shutdown_connections 2024-04-25T00:14:15.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.459+0000 7fa38d400640 1 --2- 172.21.15.83:0/3409477691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3880d1930 0x7fa3880ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.459+0000 7fa38d400640 1 --2- 172.21.15.83:0/3409477691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3881618d0 0x7fa3880ce5f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.459+0000 7fa38d400640 1 -- 172.21.15.83:0/3409477691 >> 172.21.15.83:0/3409477691 conn(0x7fa3880d1d70 msgr2=0x7fa3880d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:15.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.459+0000 7fa38d400640 1 -- 172.21.15.83:0/3409477691 shutdown_connections 2024-04-25T00:14:15.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.459+0000 7fa38d400640 1 -- 172.21.15.83:0/3409477691 wait complete. 2024-04-25T00:14:15.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa38d400640 1 Processor -- start 2024-04-25T00:14:15.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa38d400640 1 -- start start 2024-04-25T00:14:15.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa38d400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3880cf070 0x7fa3880c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa38d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3880d1930 0x7fa3880c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa38d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3881618d0 0x7fa388153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa38d400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa38816b2d0 con 0x7fa3880d1930 2024-04-25T00:14:15.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa38d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa38816b150 con 0x7fa3881618d0 2024-04-25T00:14:15.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa38d400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa38816b450 con 0x7fa3880cf070 2024-04-25T00:14:15.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa385e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3880d1930 0x7fa3880c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa385e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3880d1930 0x7fa3880c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54762/0 (socket says 172.21.15.83:54762) 2024-04-25T00:14:15.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa385e00640 1 -- 172.21.15.83:0/1271421815 learned_addr learned my addr 172.21.15.83:0/1271421815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:15.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa387200640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3881618d0 0x7fa388153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa386800640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3880cf070 0x7fa3880c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa385e00640 1 -- 172.21.15.83:0/1271421815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3880cf070 msgr2=0x7fa3880c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa385e00640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3880cf070 0x7fa3880c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa385e00640 1 -- 172.21.15.83:0/1271421815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3881618d0 msgr2=0x7fa388153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa385e00640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3881618d0 0x7fa388153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa385e00640 1 -- 172.21.15.83:0/1271421815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3881537c0 con 0x7fa3880d1930 2024-04-25T00:14:15.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa387200640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3881618d0 0x7fa388153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:15.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa385e00640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3880d1930 0x7fa3880c86d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa37400e930 tx=0x7fa37400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:15.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa37ea00640 1 -- 172.21.15.83:0/1271421815 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374017070 con 0x7fa3880d1930 2024-04-25T00:14:15.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa37ea00640 1 -- 172.21.15.83:0/1271421815 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa37400ca00 con 0x7fa3880d1930 2024-04-25T00:14:15.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa38d400640 1 -- 172.21.15.83:0/1271421815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa388155230 con 0x7fa3880d1930 2024-04-25T00:14:15.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa37ea00640 1 -- 172.21.15.83:0/1271421815 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37400cdb0 con 0x7fa3880d1930 2024-04-25T00:14:15.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.463+0000 7fa38d400640 1 -- 172.21.15.83:0/1271421815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3881539b0 con 0x7fa3880d1930 2024-04-25T00:14:15.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.467+0000 7fa38d400640 1 -- 172.21.15.83:0/1271421815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa38809fdd0 con 0x7fa3880d1930 2024-04-25T00:14:15.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.467+0000 7fa37ea00640 1 -- 172.21.15.83:0/1271421815 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa37401a070 con 0x7fa3880d1930 2024-04-25T00:14:15.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.467+0000 7fa37ea00640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3680678c0 0x7fa368069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.467+0000 7fa37ea00640 1 -- 172.21.15.83:0/1271421815 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7fa374085be0 con 0x7fa3880d1930 2024-04-25T00:14:15.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.467+0000 7fa386800640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3680678c0 0x7fa368069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.467+0000 7fa386800640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3680678c0 0x7fa368069d80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa3780ad000 tx=0x7fa378004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:15.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.471+0000 7fa37ea00640 1 -- 172.21.15.83:0/1271421815 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa3740568b0 con 0x7fa3880d1930 2024-04-25T00:14:15.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.627+0000 7fa38d400640 1 -- 172.21.15.83:0/1271421815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa38815c120 con 0x7fa3880d1930 2024-04-25T00:14:15.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.631+0000 7fa37ea00640 1 -- 172.21.15.83:0/1271421815 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+13210 (secure 0 0 0) 0x7fa3740561b0 con 0x7fa3880d1930 2024-04-25T00:14:15.634 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:15.635 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":74,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:13.280107+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/2879143105":"2024-04-26T00:14:09.903820+0000","172.21.15.83:6835/2879143105":"2024-04-26T00:14:09.903820+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:15.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.631+0000 7fa38d400640 1 -- 172.21.15.83:0/1271421815 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3680678c0 msgr2=0x7fa368069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.631+0000 7fa38d400640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3680678c0 0x7fa368069d80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa3780ad000 tx=0x7fa378004740 comp rx=0 tx=0).stop 2024-04-25T00:14:15.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.631+0000 7fa38d400640 1 -- 172.21.15.83:0/1271421815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3880d1930 msgr2=0x7fa3880c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.631+0000 7fa38d400640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3880d1930 0x7fa3880c86d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa37400e930 tx=0x7fa37400edf0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.631+0000 7fa38d400640 1 -- 172.21.15.83:0/1271421815 shutdown_connections 2024-04-25T00:14:15.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.631+0000 7fa38d400640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa3680678c0 0x7fa368069d80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.631+0000 7fa38d400640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3881618d0 0x7fa388153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.631+0000 7fa38d400640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3880d1930 0x7fa3880c86d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.631+0000 7fa38d400640 1 --2- 172.21.15.83:0/1271421815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3880cf070 0x7fa3880c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.631+0000 7fa38d400640 1 -- 172.21.15.83:0/1271421815 >> 172.21.15.83:0/1271421815 conn(0x7fa3880d1d70 msgr2=0x7fa388070910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:15.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.631+0000 7fa38d400640 1 -- 172.21.15.83:0/1271421815 shutdown_connections 2024-04-25T00:14:15.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.631+0000 7fa38d400640 1 -- 172.21.15.83:0/1271421815 wait complete. 2024-04-25T00:14:15.655 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:14:15.656 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:14:15.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c58e00640 1 Processor -- start 2024-04-25T00:14:15.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c58e00640 1 -- start start 2024-04-25T00:14:15.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c58e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c54159810 0x7f7c540cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c58e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c540ce000 con 0x7f7c54160020 2024-04-25T00:14:15.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c58e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c5406eec0 con 0x7f7c5406e950 2024-04-25T00:14:15.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c58e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5406e210 con 0x7f7c54159810 2024-04-25T00:14:15.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c52800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7c54160020 0x7f7c540ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38238/0 (socket says 172.21.15.83:38238) 2024-04-25T00:14:15.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c52800640 1 -- 172.21.15.83:0/734166484 learned_addr learned my addr 172.21.15.83:0/734166484 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:15.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c51e00640 1 --2- 172.21.15.83:0/734166484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c54159810 0x7f7c540cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4078911160 0 0) 0x7f7c540ce000 con 0x7f7c54160020 2024-04-25T00:14:15.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c38003530 con 0x7f7c54160020 2024-04-25T00:14:15.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 303648225 0 0) 0x7f7c5406eec0 con 0x7f7c5406e950 2024-04-25T00:14:15.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c540ce000 con 0x7f7c5406e950 2024-04-25T00:14:15.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 646423784 0 0) 0x7f7c38003530 con 0x7f7c54160020 2024-04-25T00:14:15.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c54159810 msgr2=0x7f7c540cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:15.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.791+0000 7f7c51400640 1 --2- 172.21.15.83:0/734166484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c54159810 0x7f7c540cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 >> v1:172.21.15.86:6789/0 conn(0x7f7c5406e950 legacy=0x7f7c5415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5406dc50 con 0x7f7c54160020 2024-04-25T00:14:15.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f7c4c001a00 con 0x7f7c54160020 2024-04-25T00:14:15.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51e00640 1 --2- 172.21.15.83:0/734166484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c54159810 0x7f7c540cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:15.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 >> v1:172.21.15.83:6789/0 conn(0x7f7c54160020 legacy=0x7f7c540ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51400640 1 --2- 172.21.15.83:0/734166484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c38003bf0 0x7f7c38003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51400640 1 --2- 172.21.15.83:0/734166484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c38004810 0x7f7c38004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51400640 1 --2- 172.21.15.83:0/734166484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c54159810 0x7f7c38005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5406e210 con 0x7f7c38004810 2024-04-25T00:14:15.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3800bb60 con 0x7f7c38003bf0 2024-04-25T00:14:15.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3800bd10 con 0x7f7c54159810 2024-04-25T00:14:15.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c49400640 1 --2- 172.21.15.83:0/734166484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c38004810 0x7f7c38004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51e00640 1 --2- 172.21.15.83:0/734166484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c38003bf0 0x7f7c38003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c52800640 1 --2- 172.21.15.83:0/734166484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c54159810 0x7f7c38005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c49400640 1 -- 172.21.15.83:0/734166484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c54159810 msgr2=0x7f7c38005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:15.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c49400640 1 --2- 172.21.15.83:0/734166484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c54159810 0x7f7c38005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c49400640 1 -- 172.21.15.83:0/734166484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c38003bf0 msgr2=0x7f7c38003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:15.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c49400640 1 --2- 172.21.15.83:0/734166484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c38003bf0 0x7f7c38003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c49400640 1 -- 172.21.15.83:0/734166484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c38003530 con 0x7f7c38004810 2024-04-25T00:14:15.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c49400640 1 --2- 172.21.15.83:0/734166484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c38004810 0x7f7c38004c90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0018d0 tx=0x7f7c3c01c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e971306be68304cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:15.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c001520 con 0x7f7c38004810 2024-04-25T00:14:15.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c3c0050d0 con 0x7f7c38004810 2024-04-25T00:14:15.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.795+0000 7f7c51400640 1 -- 172.21.15.83:0/734166484 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c0054a0 con 0x7f7c38004810 2024-04-25T00:14:15.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c58e00640 1 -- 172.21.15.83:0/734166484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c38004810 msgr2=0x7f7c38004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c58e00640 1 --2- 172.21.15.83:0/734166484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c38004810 0x7f7c38004c90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0018d0 tx=0x7f7c3c01c580 comp rx=0 tx=0).stop 2024-04-25T00:14:15.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c52800640 1 -- 172.21.15.83:0/734166484 reap_dead start 2024-04-25T00:14:15.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c58e00640 1 -- 172.21.15.83:0/734166484 shutdown_connections 2024-04-25T00:14:15.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c58e00640 1 -- 172.21.15.83:0/734166484 >> 172.21.15.83:0/734166484 conn(0x7f7c540b1830 msgr2=0x7f7c540b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:15.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c58e00640 1 -- 172.21.15.83:0/734166484 shutdown_connections 2024-04-25T00:14:15.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c58e00640 1 -- 172.21.15.83:0/734166484 wait complete. 2024-04-25T00:14:15.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c58e00640 1 Processor -- start 2024-04-25T00:14:15.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c58e00640 1 -- start start 2024-04-25T00:14:15.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c58e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c38004810 0x7f7c5414b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c58e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54162d30 0x7f7c541f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c58e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540b3270 0x7f7c54158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c58e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c540cc8b0 con 0x7f7c54162d30 2024-04-25T00:14:15.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c58e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c540cca30 con 0x7f7c540b3270 2024-04-25T00:14:15.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c58e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7c54167cb0 con 0x7f7c38004810 2024-04-25T00:14:15.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c49400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54162d30 0x7f7c541f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c49400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54162d30 0x7f7c541f7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54792/0 (socket says 172.21.15.83:54792) 2024-04-25T00:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c49400640 1 -- 172.21.15.83:0/234376799 learned_addr learned my addr 172.21.15.83:0/234376799 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c52800640 1 --2- 172.21.15.83:0/234376799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540b3270 0x7f7c54158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c51e00640 1 --2- 172.21.15.83:0/234376799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c38004810 0x7f7c5414b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c49400640 1 -- 172.21.15.83:0/234376799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c38004810 msgr2=0x7f7c5414b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c49400640 1 --2- 172.21.15.83:0/234376799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c38004810 0x7f7c5414b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c49400640 1 -- 172.21.15.83:0/234376799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540b3270 msgr2=0x7f7c54158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c49400640 1 --2- 172.21.15.83:0/234376799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540b3270 0x7f7c54158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.799+0000 7f7c49400640 1 -- 172.21.15.83:0/234376799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c540cd5d0 con 0x7f7c54162d30 2024-04-25T00:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.803+0000 7f7c49400640 1 --2- 172.21.15.83:0/234376799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54162d30 0x7f7c541f7210 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0ae040 tx=0x7f7c38005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.803+0000 7f7c4aa00640 1 -- 172.21.15.83:0/234376799 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c0bc020 con 0x7f7c54162d30 2024-04-25T00:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.803+0000 7f7c4aa00640 1 -- 172.21.15.83:0/234376799 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c3c003740 con 0x7f7c54162d30 2024-04-25T00:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.803+0000 7f7c4aa00640 1 -- 172.21.15.83:0/234376799 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c003b30 con 0x7f7c54162d30 2024-04-25T00:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.803+0000 7f7c58e00640 1 -- 172.21.15.83:0/234376799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c54069410 con 0x7f7c54162d30 2024-04-25T00:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.803+0000 7f7c58e00640 1 -- 172.21.15.83:0/234376799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c541f8200 con 0x7f7c54162d30 2024-04-25T00:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.803+0000 7f7c58e00640 1 -- 172.21.15.83:0/234376799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c5404f510 con 0x7f7c54162d30 2024-04-25T00:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.803+0000 7f7c4aa00640 1 -- 172.21.15.83:0/234376799 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7c3c0bf020 con 0x7f7c54162d30 2024-04-25T00:14:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.803+0000 7f7c4aa00640 1 --2- 172.21.15.83:0/234376799 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c300678c0 0x7f7c30069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.803+0000 7f7c4aa00640 1 -- 172.21.15.83:0/234376799 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7f7c3c11ff00 con 0x7f7c54162d30 2024-04-25T00:14:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.803+0000 7f7c51e00640 1 --2- 172.21.15.83:0/234376799 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c300678c0 0x7f7c30069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.803+0000 7f7c51e00640 1 --2- 172.21.15.83:0/234376799 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c300678c0 0x7f7c30069d80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009580 tx=0x7f7c44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:15.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.807+0000 7f7c4aa00640 1 -- 172.21.15.83:0/234376799 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7c3c0c6030 con 0x7f7c54162d30 2024-04-25T00:14:15.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.963+0000 7f7c58e00640 1 -- 172.21.15.83:0/234376799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7c54072c90 con 0x7f7c54162d30 2024-04-25T00:14:15.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.963+0000 7f7c4aa00640 1 -- 172.21.15.83:0/234376799 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f7c3c016290 con 0x7f7c54162d30 2024-04-25T00:14:15.967 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:14:15.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.963+0000 7f7c58e00640 1 -- 172.21.15.83:0/234376799 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c300678c0 msgr2=0x7f7c30069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.963+0000 7f7c58e00640 1 --2- 172.21.15.83:0/234376799 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c300678c0 0x7f7c30069d80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009580 tx=0x7f7c44008040 comp rx=0 tx=0).stop 2024-04-25T00:14:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.963+0000 7f7c58e00640 1 -- 172.21.15.83:0/234376799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54162d30 msgr2=0x7f7c541f7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.963+0000 7f7c58e00640 1 --2- 172.21.15.83:0/234376799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54162d30 0x7f7c541f7210 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0ae040 tx=0x7f7c38005840 comp rx=0 tx=0).stop 2024-04-25T00:14:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.963+0000 7f7c58e00640 1 -- 172.21.15.83:0/234376799 shutdown_connections 2024-04-25T00:14:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.963+0000 7f7c58e00640 1 --2- 172.21.15.83:0/234376799 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c300678c0 0x7f7c30069d80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.963+0000 7f7c58e00640 1 --2- 172.21.15.83:0/234376799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540b3270 0x7f7c54158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.963+0000 7f7c58e00640 1 --2- 172.21.15.83:0/234376799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54162d30 0x7f7c541f7210 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.963+0000 7f7c58e00640 1 --2- 172.21.15.83:0/234376799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c38004810 0x7f7c5414b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.963+0000 7f7c58e00640 1 -- 172.21.15.83:0/234376799 >> 172.21.15.83:0/234376799 conn(0x7f7c540b1830 msgr2=0x7f7c540b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.967+0000 7f7c58e00640 1 -- 172.21.15.83:0/234376799 shutdown_connections 2024-04-25T00:14:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:15.967+0000 7f7c58e00640 1 -- 172.21.15.83:0/234376799 wait complete. 2024-04-25T00:14:15.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop' 2024-04-25T00:14:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f7800640 1 Processor -- start 2024-04-25T00:14:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f7800640 1 -- start start 2024-04-25T00:14:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f7800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2f01599f0 0x7fc2f00cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f7800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2f015cec0 0x7fc2f00ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f7800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f00ccc60 con 0x7fc2f015c340 2024-04-25T00:14:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f7800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f0159dd0 con 0x7fc2f01599f0 2024-04-25T00:14:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f7800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f006dba0 con 0x7fc2f015cec0 2024-04-25T00:14:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f4e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2f01599f0 0x7fc2f00cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f4e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2f01599f0 0x7fc2f00cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54806/0 (socket says 172.21.15.83:54806) 2024-04-25T00:14:16.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f4e00640 1 -- 172.21.15.83:0/374293513 learned_addr learned my addr 172.21.15.83:0/374293513 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:16.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f5800640 1 --2- 172.21.15.83:0/374293513 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2f015cec0 0x7fc2f00ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f4e00640 1 -- 172.21.15.83:0/374293513 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2f015cec0 msgr2=0x7fc2f00ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:16.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f4e00640 1 --2- 172.21.15.83:0/374293513 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2f015cec0 0x7fc2f00ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f4e00640 1 -- 172.21.15.83:0/374293513 >> v1:172.21.15.86:6789/0 conn(0x7fc2f015c340 legacy=0x7fc2f015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.119+0000 7fc2f4e00640 1 -- 172.21.15.83:0/374293513 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f006de80 con 0x7fc2f01599f0 2024-04-25T00:14:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.123+0000 7fc2f4e00640 1 --2- 172.21.15.83:0/374293513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2f01599f0 0x7fc2f00cc510 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009e80 tx=0x7fc2e001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a258fee478302ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.123+0000 7fc2ef400640 1 -- 172.21.15.83:0/374293513 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00a5640 con 0x7fc2f01599f0 2024-04-25T00:14:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.123+0000 7fc2ef400640 1 -- 172.21.15.83:0/374293513 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e00a57e0 con 0x7fc2f01599f0 2024-04-25T00:14:16.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.123+0000 7fc2ef400640 1 -- 172.21.15.83:0/374293513 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00a5bb0 con 0x7fc2f01599f0 2024-04-25T00:14:16.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.123+0000 7fc2f7800640 1 -- 172.21.15.83:0/374293513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2f01599f0 msgr2=0x7fc2f00cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.123+0000 7fc2f7800640 1 --2- 172.21.15.83:0/374293513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2f01599f0 0x7fc2f00cc510 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009e80 tx=0x7fc2e001c980 comp rx=0 tx=0).stop 2024-04-25T00:14:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.123+0000 7fc2f7800640 1 -- 172.21.15.83:0/374293513 shutdown_connections 2024-04-25T00:14:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.123+0000 7fc2f7800640 1 --2- 172.21.15.83:0/374293513 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2f015cec0 0x7fc2f00ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.123+0000 7fc2f7800640 1 --2- 172.21.15.83:0/374293513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2f01599f0 0x7fc2f00cc510 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.123+0000 7fc2f7800640 1 -- 172.21.15.83:0/374293513 >> 172.21.15.83:0/374293513 conn(0x7fc2f00b1830 msgr2=0x7fc2f00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.123+0000 7fc2f7800640 1 -- 172.21.15.83:0/374293513 shutdown_connections 2024-04-25T00:14:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.123+0000 7fc2f7800640 1 -- 172.21.15.83:0/374293513 wait complete. 2024-04-25T00:14:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f7800640 1 Processor -- start 2024-04-25T00:14:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f7800640 1 -- start start 2024-04-25T00:14:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f7800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2f01599f0 0x7fc2f0142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f7800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2f015c340 0x7fc2f01f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f7800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f015cec0 0x7fc2f015aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f7800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f00cf7c0 con 0x7fc2f015c340 2024-04-25T00:14:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f7800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f00cf640 con 0x7fc2f015cec0 2024-04-25T00:14:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f7800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f0168cd0 con 0x7fc2f01599f0 2024-04-25T00:14:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2f015c340 0x7fc2f01f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2f015c340 0x7fc2f01f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54808/0 (socket says 172.21.15.83:54808) 2024-04-25T00:14:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2efe00640 1 -- 172.21.15.83:0/2093077613 learned_addr learned my addr 172.21.15.83:0/2093077613 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f5800640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f015cec0 0x7fc2f015aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f4e00640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2f01599f0 0x7fc2f0142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2efe00640 1 -- 172.21.15.83:0/2093077613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2f01599f0 msgr2=0x7fc2f0142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2efe00640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2f01599f0 0x7fc2f0142ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2efe00640 1 -- 172.21.15.83:0/2093077613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f015cec0 msgr2=0x7fc2f015aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2efe00640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f015cec0 0x7fc2f015aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2efe00640 1 -- 172.21.15.83:0/2093077613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f014c120 con 0x7fc2f015c340 2024-04-25T00:14:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f5800640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f015cec0 0x7fc2f015aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f4e00640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2f01599f0 0x7fc2f0142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2efe00640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2f015c340 0x7fc2f01f5f80 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00c3a0 tx=0x7fc2dc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2ecc00640 1 -- 172.21.15.83:0/2093077613 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0052d0 con 0x7fc2f015c340 2024-04-25T00:14:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2ecc00640 1 -- 172.21.15.83:0/2093077613 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2dc005470 con 0x7fc2f015c340 2024-04-25T00:14:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2ecc00640 1 -- 172.21.15.83:0/2093077613 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc007260 con 0x7fc2f015c340 2024-04-25T00:14:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f7800640 1 -- 172.21.15.83:0/2093077613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f014c3c0 con 0x7fc2f015c340 2024-04-25T00:14:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f7800640 1 -- 172.21.15.83:0/2093077613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f014c940 con 0x7fc2f015c340 2024-04-25T00:14:16.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.127+0000 7fc2f7800640 1 -- 172.21.15.83:0/2093077613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f00737f0 con 0x7fc2f015c340 2024-04-25T00:14:16.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.131+0000 7fc2ecc00640 1 -- 172.21.15.83:0/2093077613 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc2dc007400 con 0x7fc2f015c340 2024-04-25T00:14:16.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.131+0000 7fc2ecc00640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2d00678c0 0x7fc2d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.131+0000 7fc2ecc00640 1 -- 172.21.15.83:0/2093077613 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7fc2dc084f00 con 0x7fc2f015c340 2024-04-25T00:14:16.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.131+0000 7fc2f4e00640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2d00678c0 0x7fc2d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.131+0000 7fc2f4e00640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2d00678c0 0x7fc2d0069d80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00ae000 tx=0x7fc2e00a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:16.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.135+0000 7fc2ecc00640 1 -- 172.21.15.83:0/2093077613 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc2f00ccc60 con 0x7fc2f015c340 2024-04-25T00:14:16.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:16.347+0000 7fc2f7800640 1 -- 172.21.15.83:0/2093077613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]} v 0) -- 0x7fc2f004f5f0 con 0x7fc2f015c340 2024-04-25T00:14:17.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.295+0000 7fc2ecc00640 1 -- 172.21.15.83:0/2093077613 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]}]=0 v211) ==== 148+0+0 (secure 0 0 0) 0x7fc2dc055c40 con 0x7fc2f015c340 2024-04-25T00:14:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.299+0000 7fc2f7800640 1 -- 172.21.15.83:0/2093077613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2d00678c0 msgr2=0x7fc2d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.299+0000 7fc2f7800640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2d00678c0 0x7fc2d0069d80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00ae000 tx=0x7fc2e00a7070 comp rx=0 tx=0).stop 2024-04-25T00:14:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.299+0000 7fc2f7800640 1 -- 172.21.15.83:0/2093077613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2f015c340 msgr2=0x7fc2f01f5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.299+0000 7fc2f7800640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2f015c340 0x7fc2f01f5f80 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00c3a0 tx=0x7fc2dc00c860 comp rx=0 tx=0).stop 2024-04-25T00:14:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.299+0000 7fc2f7800640 1 -- 172.21.15.83:0/2093077613 shutdown_connections 2024-04-25T00:14:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.299+0000 7fc2f7800640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2d00678c0 0x7fc2d0069d80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.299+0000 7fc2f7800640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f015cec0 0x7fc2f015aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.299+0000 7fc2f7800640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2f015c340 0x7fc2f01f5f80 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.299+0000 7fc2f7800640 1 --2- 172.21.15.83:0/2093077613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2f01599f0 0x7fc2f0142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.299+0000 7fc2f7800640 1 -- 172.21.15.83:0/2093077613 >> 172.21.15.83:0/2093077613 conn(0x7fc2f00b1830 msgr2=0x7fc2f0156850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.299+0000 7fc2f7800640 1 -- 172.21.15.83:0/2093077613 shutdown_connections 2024-04-25T00:14:17.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.299+0000 7fc2f7800640 1 -- 172.21.15.83:0/2093077613 wait complete. 2024-04-25T00:14:17.324 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:14:17.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.463+0000 7f9984600640 1 Processor -- start 2024-04-25T00:14:17.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.463+0000 7f9984600640 1 -- start start 2024-04-25T00:14:17.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.463+0000 7f9984600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c0d0990 0x7f997c0d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.463+0000 7f9984600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f997c15c540 con 0x7f997c0b2ed0 2024-04-25T00:14:17.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.463+0000 7f9984600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f997c0cfa30 con 0x7f997c15c160 2024-04-25T00:14:17.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.463+0000 7f9984600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f997c0cf660 con 0x7f997c0d0990 2024-04-25T00:14:17.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.463+0000 7f9982600640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f997c0b2ed0 0x7f997c0cc7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38244/0 (socket says 172.21.15.83:38244) 2024-04-25T00:14:17.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.463+0000 7f9982600640 1 -- 172.21.15.83:0/356212699 learned_addr learned my addr 172.21.15.83:0/356212699 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:17.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.463+0000 7f9981200640 1 --2- 172.21.15.83:0/356212699 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c0d0990 0x7f997c0d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.463+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3374191330 0 0) 0x7f997c15c540 con 0x7f997c0b2ed0 2024-04-25T00:14:17.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.463+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9964003530 con 0x7f997c0b2ed0 2024-04-25T00:14:17.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.463+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2883048880 0 0) 0x7f997c0cfa30 con 0x7f997c15c160 2024-04-25T00:14:17.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f997c15c540 con 0x7f997c15c160 2024-04-25T00:14:17.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 955876148 0 0) 0x7f9964003530 con 0x7f997c0b2ed0 2024-04-25T00:14:17.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c0d0990 msgr2=0x7f997c0d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:17.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f997be00640 1 --2- 172.21.15.83:0/356212699 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c0d0990 0x7f997c0d27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 >> v1:172.21.15.86:6789/0 conn(0x7f997c15c160 legacy=0x7f997c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c0cfd10 con 0x7f997c0b2ed0 2024-04-25T00:14:17.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f9974001a00 con 0x7f997c0b2ed0 2024-04-25T00:14:17.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f9981200640 1 --2- 172.21.15.83:0/356212699 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c0d0990 0x7f997c0d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:17.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 >> v1:172.21.15.83:6789/0 conn(0x7f997c0b2ed0 legacy=0x7f997c0cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f997be00640 1 --2- 172.21.15.83:0/356212699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9964003bf0 0x7f9964003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f997be00640 1 --2- 172.21.15.83:0/356212699 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c0d0990 0x7f9964004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f997be00640 1 --2- 172.21.15.83:0/356212699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9964004d50 0x7f99640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f996400bae0 con 0x7f9964004d50 2024-04-25T00:14:17.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f996400bc90 con 0x7f9964003bf0 2024-04-25T00:14:17.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f996400be90 con 0x7f997c0d0990 2024-04-25T00:14:17.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f9982600640 1 --2- 172.21.15.83:0/356212699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9964004d50 0x7f99640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f9981c00640 1 --2- 172.21.15.83:0/356212699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9964003bf0 0x7f9964003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f9981200640 1 --2- 172.21.15.83:0/356212699 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c0d0990 0x7f9964004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f9982600640 1 -- 172.21.15.83:0/356212699 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c0d0990 msgr2=0x7f9964004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:17.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f9982600640 1 --2- 172.21.15.83:0/356212699 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c0d0990 0x7f9964004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f9982600640 1 -- 172.21.15.83:0/356212699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9964003bf0 msgr2=0x7f9964003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:17.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f9982600640 1 --2- 172.21.15.83:0/356212699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9964003bf0 0x7f9964003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f9982600640 1 -- 172.21.15.83:0/356212699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9964003530 con 0x7f9964004d50 2024-04-25T00:14:17.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.467+0000 7f9982600640 1 --2- 172.21.15.83:0/356212699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9964004d50 0x7f99640051d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f997400a720 tx=0x7f997401f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2cbacefc863d07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:17.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997400a370 con 0x7f9964004d50 2024-04-25T00:14:17.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99740b1070 con 0x7f9964004d50 2024-04-25T00:14:17.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f997be00640 1 -- 172.21.15.83:0/356212699 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99740ac8b0 con 0x7f9964004d50 2024-04-25T00:14:17.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9984600640 1 -- 172.21.15.83:0/356212699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9964004d50 msgr2=0x7f99640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9984600640 1 --2- 172.21.15.83:0/356212699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9964004d50 0x7f99640051d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f997400a720 tx=0x7f997401f9a0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9982600640 1 -- 172.21.15.83:0/356212699 reap_dead start 2024-04-25T00:14:17.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9984600640 1 -- 172.21.15.83:0/356212699 shutdown_connections 2024-04-25T00:14:17.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9984600640 1 -- 172.21.15.83:0/356212699 >> 172.21.15.83:0/356212699 conn(0x7f997c0d1cc0 msgr2=0x7f997c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:17.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9984600640 1 -- 172.21.15.83:0/356212699 shutdown_connections 2024-04-25T00:14:17.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9984600640 1 -- 172.21.15.83:0/356212699 wait complete. 2024-04-25T00:14:17.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9984600640 1 Processor -- start 2024-04-25T00:14:17.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9984600640 1 -- start start 2024-04-25T00:14:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9984600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f997c06f3f0 0x7f997c0b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9984600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c06f7d0 0x7f997c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9984600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f997c0cafd0 0x7f997c151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9984600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f997c0b19a0 con 0x7f997c0cafd0 2024-04-25T00:14:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9984600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f997c0b16c0 con 0x7f997c06f3f0 2024-04-25T00:14:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9984600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f997c0d27e0 con 0x7f997c06f7d0 2024-04-25T00:14:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9982600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f997c0cafd0 0x7f997c151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9982600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f997c0cafd0 0x7f997c151d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54836/0 (socket says 172.21.15.83:54836) 2024-04-25T00:14:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9982600640 1 -- 172.21.15.83:0/3049063382 learned_addr learned my addr 172.21.15.83:0/3049063382 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9981c00640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f997c06f3f0 0x7f997c0b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9982600640 1 -- 172.21.15.83:0/3049063382 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c06f7d0 msgr2=0x7f997c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.471+0000 7f9981200640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c06f7d0 0x7f997c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9982600640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c06f7d0 0x7f997c0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9982600640 1 -- 172.21.15.83:0/3049063382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f997c06f3f0 msgr2=0x7f997c0b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9982600640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f997c06f3f0 0x7f997c0b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9982600640 1 -- 172.21.15.83:0/3049063382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c1f8210 con 0x7f997c0cafd0 2024-04-25T00:14:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9981c00640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f997c06f3f0 0x7f997c0b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9981200640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c06f7d0 0x7f997c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9982600640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f997c0cafd0 0x7f997c151d50 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f997401f9e0 tx=0x7f9964005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9979600640 1 -- 172.21.15.83:0/3049063382 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99740011d0 con 0x7f997c0cafd0 2024-04-25T00:14:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9979600640 1 -- 172.21.15.83:0/3049063382 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99740ba020 con 0x7f997c0cafd0 2024-04-25T00:14:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9979600640 1 -- 172.21.15.83:0/3049063382 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974004d70 con 0x7f997c0cafd0 2024-04-25T00:14:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9984600640 1 -- 172.21.15.83:0/3049063382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997c0c5990 con 0x7f997c0cafd0 2024-04-25T00:14:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9984600640 1 -- 172.21.15.83:0/3049063382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997c0c5f50 con 0x7f997c0cafd0 2024-04-25T00:14:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9984600640 1 -- 172.21.15.83:0/3049063382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f997c04f510 con 0x7f997c0cafd0 2024-04-25T00:14:17.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9979600640 1 -- 172.21.15.83:0/3049063382 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f99740ac450 con 0x7f997c0cafd0 2024-04-25T00:14:17.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9979600640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f995c0678c0 0x7f995c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9979600640 1 -- 172.21.15.83:0/3049063382 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7f9974123330 con 0x7f997c0cafd0 2024-04-25T00:14:17.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.475+0000 7f9981c00640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f995c0678c0 0x7f995c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.479+0000 7f9981c00640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f995c0678c0 0x7f995c069d80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f997c0ca9c0 tx=0x7f9970009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.479+0000 7f9979600640 1 -- 172.21.15.83:0/3049063382 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f99740c4030 con 0x7f997c0cafd0 2024-04-25T00:14:17.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.639+0000 7f9984600640 1 -- 172.21.15.83:0/3049063382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f997c067320 con 0x7f997c0cafd0 2024-04-25T00:14:17.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.639+0000 7f9979600640 1 -- 172.21.15.83:0/3049063382 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f99740196b0 con 0x7f997c0cafd0 2024-04-25T00:14:17.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.643+0000 7f9984600640 1 -- 172.21.15.83:0/3049063382 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f995c0678c0 msgr2=0x7f995c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.643+0000 7f9984600640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f995c0678c0 0x7f995c069d80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f997c0ca9c0 tx=0x7f9970009040 comp rx=0 tx=0).stop 2024-04-25T00:14:17.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.643+0000 7f9984600640 1 -- 172.21.15.83:0/3049063382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f997c0cafd0 msgr2=0x7f997c151d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.643+0000 7f9984600640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f997c0cafd0 0x7f997c151d50 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f997401f9e0 tx=0x7f9964005840 comp rx=0 tx=0).stop 2024-04-25T00:14:17.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.643+0000 7f9984600640 1 -- 172.21.15.83:0/3049063382 shutdown_connections 2024-04-25T00:14:17.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.643+0000 7f9984600640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f995c0678c0 0x7f995c069d80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.643+0000 7f9984600640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f997c06f7d0 0x7f997c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.643+0000 7f9984600640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f997c0cafd0 0x7f997c151d50 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.643+0000 7f9984600640 1 --2- 172.21.15.83:0/3049063382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f997c06f3f0 0x7f997c0b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.643+0000 7f9984600640 1 -- 172.21.15.83:0/3049063382 >> 172.21.15.83:0/3049063382 conn(0x7f997c0d1cc0 msgr2=0x7f997c0691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:17.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.643+0000 7f9984600640 1 -- 172.21.15.83:0/3049063382 shutdown_connections 2024-04-25T00:14:17.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:17.643+0000 7f9984600640 1 -- 172.21.15.83:0/3049063382 wait complete. 2024-04-25T00:14:17.667 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:14:17.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:14:17.720 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:14:17.785 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:14:17.785 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:14:17.785 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:17.793 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:14:17.794 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:14:17.794 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:17.803 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:14:17.803 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:14:17.803 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:17.850 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:14:17.850 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:14:17.850 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:17.860 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:14:17.861 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:14:17.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.027+0000 7f23e0e00640 1 Processor -- start 2024-04-25T00:14:18.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.027+0000 7f23e0e00640 1 -- start start 2024-04-25T00:14:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.027+0000 7f23e0e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f23dc0cf420 0x7f23dc071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.027+0000 7f23e0e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23dc1635e0 0x7f23dc06cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.027+0000 7f23e0e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23dc072750 con 0x7f23dc06d060 2024-04-25T00:14:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.027+0000 7f23e0e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f23dc071b40 con 0x7f23dc0cf420 2024-04-25T00:14:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.027+0000 7f23e0e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc071e20 con 0x7f23dc1635e0 2024-04-25T00:14:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.027+0000 7f23da800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f23dc06d060 0x7f23dc165fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38260/0 (socket says 172.21.15.83:38260) 2024-04-25T00:14:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.027+0000 7f23da800640 1 -- 172.21.15.83:0/1606194116 learned_addr learned my addr 172.21.15.83:0/1606194116 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.027+0000 7f23d9e00640 1 --2- 172.21.15.83:0/1606194116 >> v2:172.21.15.86:3300/0 conn(0x7f23dc0cf420 0x7f23dc071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.027+0000 7f23d1400640 1 --2- 172.21.15.83:0/1606194116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23dc1635e0 0x7f23dc06cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.027+0000 7f23d9400640 1 -- 172.21.15.83:0/1606194116 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4249363575 0 0) 0x7f23dc072750 con 0x7f23dc06d060 2024-04-25T00:14:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.027+0000 7f23d9400640 1 -- 172.21.15.83:0/1606194116 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23c0003530 con 0x7f23dc06d060 2024-04-25T00:14:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9400640 1 -- 172.21.15.83:0/1606194116 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1434141041 0 0) 0x7f23c0003530 con 0x7f23dc06d060 2024-04-25T00:14:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9400640 1 -- 172.21.15.83:0/1606194116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23dc1635e0 msgr2=0x7f23dc06cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9400640 1 --2- 172.21.15.83:0/1606194116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23dc1635e0 0x7f23dc06cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9400640 1 -- 172.21.15.83:0/1606194116 >> v2:172.21.15.86:3300/0 conn(0x7f23dc0cf420 msgr2=0x7f23dc071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9400640 1 --2- 172.21.15.83:0/1606194116 >> v2:172.21.15.86:3300/0 conn(0x7f23dc0cf420 0x7f23dc071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9400640 1 -- 172.21.15.83:0/1606194116 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc072100 con 0x7f23dc06d060 2024-04-25T00:14:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9400640 1 -- 172.21.15.83:0/1606194116 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f23d4001a00 con 0x7f23dc06d060 2024-04-25T00:14:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9e00640 1 --2- 172.21.15.83:0/1606194116 >> v2:172.21.15.86:3300/0 conn(0x7f23dc0cf420 0x7f23dc071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9400640 1 -- 172.21.15.83:0/1606194116 >> v1:172.21.15.83:6789/0 conn(0x7f23dc06d060 legacy=0x7f23dc165fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9400640 1 --2- 172.21.15.83:0/1606194116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c00039c0 0x7f23c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9400640 1 --2- 172.21.15.83:0/1606194116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c0004810 0x7f23c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9400640 1 --2- 172.21.15.83:0/1606194116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23dc1635e0 0x7f23c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9400640 1 -- 172.21.15.83:0/1606194116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc071b40 con 0x7f23c00039c0 2024-04-25T00:14:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9400640 1 -- 172.21.15.83:0/1606194116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc071e20 con 0x7f23c0004810 2024-04-25T00:14:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9400640 1 -- 172.21.15.83:0/1606194116 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f23c000b9d0 con 0x7f23dc1635e0 2024-04-25T00:14:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9e00640 1 --2- 172.21.15.83:0/1606194116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c00039c0 0x7f23c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23da800640 1 --2- 172.21.15.83:0/1606194116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23dc1635e0 0x7f23c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d1400640 1 --2- 172.21.15.83:0/1606194116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c0004810 0x7f23c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9e00640 1 -- 172.21.15.83:0/1606194116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23dc1635e0 msgr2=0x7f23c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9e00640 1 --2- 172.21.15.83:0/1606194116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23dc1635e0 0x7f23c00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9e00640 1 -- 172.21.15.83:0/1606194116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c0004810 msgr2=0x7f23c0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9e00640 1 --2- 172.21.15.83:0/1606194116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c0004810 0x7f23c0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9e00640 1 -- 172.21.15.83:0/1606194116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c0003530 con 0x7f23c00039c0 2024-04-25T00:14:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d1400640 1 --2- 172.21.15.83:0/1606194116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c0004810 0x7f23c0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.031+0000 7f23d9e00640 1 --2- 172.21.15.83:0/1606194116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c00039c0 0x7f23c0003de0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009cb0 tx=0x7f23cc01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63473a28a55b3749 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23d9400640 1 -- 172.21.15.83:0/1606194116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc0b2070 con 0x7f23c00039c0 2024-04-25T00:14:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23d9400640 1 -- 172.21.15.83:0/1606194116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23cc0a5650 con 0x7f23c00039c0 2024-04-25T00:14:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23d9400640 1 -- 172.21.15.83:0/1606194116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc009880 con 0x7f23c00039c0 2024-04-25T00:14:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23e0e00640 1 -- 172.21.15.83:0/1606194116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c00039c0 msgr2=0x7f23c0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23e0e00640 1 --2- 172.21.15.83:0/1606194116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c00039c0 0x7f23c0003de0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009cb0 tx=0x7f23cc01cc30 comp rx=0 tx=0).stop 2024-04-25T00:14:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23da800640 1 -- 172.21.15.83:0/1606194116 reap_dead start 2024-04-25T00:14:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23e0e00640 1 -- 172.21.15.83:0/1606194116 shutdown_connections 2024-04-25T00:14:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23e0e00640 1 -- 172.21.15.83:0/1606194116 >> 172.21.15.83:0/1606194116 conn(0x7f23dc0b1830 msgr2=0x7f23dc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23e0e00640 1 -- 172.21.15.83:0/1606194116 shutdown_connections 2024-04-25T00:14:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23e0e00640 1 -- 172.21.15.83:0/1606194116 wait complete. 2024-04-25T00:14:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23e0e00640 1 Processor -- start 2024-04-25T00:14:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23e0e00640 1 -- start start 2024-04-25T00:14:18.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23e0e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23c00039c0 0x7f23dc164310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23e0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23dc157ac0 0x7f23dc14f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23e0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc14e830 0x7f23dc14ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23e0e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc16d4d0 con 0x7f23dc157ac0 2024-04-25T00:14:18.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23e0e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc16d350 con 0x7f23dc14e830 2024-04-25T00:14:18.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23e0e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc16d650 con 0x7f23c00039c0 2024-04-25T00:14:18.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23d1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23dc157ac0 0x7f23dc14f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23d1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23dc157ac0 0x7f23dc14f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54860/0 (socket says 172.21.15.83:54860) 2024-04-25T00:14:18.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23d1400640 1 -- 172.21.15.83:0/416325662 learned_addr learned my addr 172.21.15.83:0/416325662 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:18.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23d9e00640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23c00039c0 0x7f23dc164310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.035+0000 7f23da800640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc14e830 0x7f23dc14ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23d1400640 1 -- 172.21.15.83:0/416325662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23c00039c0 msgr2=0x7f23dc164310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:18.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23d1400640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23c00039c0 0x7f23dc164310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23d1400640 1 -- 172.21.15.83:0/416325662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc14e830 msgr2=0x7f23dc14ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:18.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23d1400640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc14e830 0x7f23dc14ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23d1400640 1 -- 172.21.15.83:0/416325662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc156960 con 0x7f23dc157ac0 2024-04-25T00:14:18.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23da800640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc14e830 0x7f23dc14ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:18.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23d1400640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23dc157ac0 0x7f23dc14f610 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f23c40116a0 tx=0x7f23c4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:18.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23d2a00640 1 -- 172.21.15.83:0/416325662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c400f9c0 con 0x7f23dc157ac0 2024-04-25T00:14:18.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23d2a00640 1 -- 172.21.15.83:0/416325662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c400fb60 con 0x7f23dc157ac0 2024-04-25T00:14:18.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23e0e00640 1 -- 172.21.15.83:0/416325662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc156c40 con 0x7f23dc157ac0 2024-04-25T00:14:18.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23d2a00640 1 -- 172.21.15.83:0/416325662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4015460 con 0x7f23dc157ac0 2024-04-25T00:14:18.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23e0e00640 1 -- 172.21.15.83:0/416325662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc157f20 con 0x7f23dc157ac0 2024-04-25T00:14:18.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23e0e00640 1 -- 172.21.15.83:0/416325662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23dc0737f0 con 0x7f23dc157ac0 2024-04-25T00:14:18.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23d2a00640 1 -- 172.21.15.83:0/416325662 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f23c4015600 con 0x7f23dc157ac0 2024-04-25T00:14:18.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23d2a00640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f23b80678f0 0x7f23b8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23d2a00640 1 -- 172.21.15.83:0/416325662 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7f23c4086580 con 0x7f23dc157ac0 2024-04-25T00:14:18.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.039+0000 7f23d9e00640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f23b80678f0 0x7f23b8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.043+0000 7f23d9e00640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f23b80678f0 0x7f23b8069db0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f23cc0af000 tx=0x7f23cc0046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:18.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.043+0000 7f23d2a00640 1 -- 172.21.15.83:0/416325662 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f23c4019030 con 0x7f23dc157ac0 2024-04-25T00:14:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.231+0000 7f23e0e00640 1 -- 172.21.15.83:0/416325662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23dc157160 con 0x7f23dc157ac0 2024-04-25T00:14:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.231+0000 7f23d2a00640 1 -- 172.21.15.83:0/416325662 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+3869 (secure 0 0 0) 0x7f23c4057190 con 0x7f23dc157ac0 2024-04-25T00:14:18.235 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:18.235 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2852678498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2852678498},{"type":"v1","addr":"172.21.15.86:6834","nonce":2852678498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2956252746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":2956252746},{"type":"v1","addr":"172.21.15.86:6835","nonce":2956252746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6588,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2678835998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2678835998},{"type":"v1","addr":"172.21.15.83:6836","nonce":2678835998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6936,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3772875899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3772875899},{"type":"v1","addr":"172.21.15.83:6837","nonce":3772875899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[]} 2024-04-25T00:14:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 46 2024-04-25T00:14:18.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.231+0000 7f23e0e00640 1 -- 172.21.15.83:0/416325662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f23b80678f0 msgr2=0x7f23b8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:18.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.231+0000 7f23e0e00640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f23b80678f0 0x7f23b8069db0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f23cc0af000 tx=0x7f23cc0046d0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.231+0000 7f23e0e00640 1 -- 172.21.15.83:0/416325662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23dc157ac0 msgr2=0x7f23dc14f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:18.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.231+0000 7f23e0e00640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23dc157ac0 0x7f23dc14f610 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f23c40116a0 tx=0x7f23c4011b60 comp rx=0 tx=0).stop 2024-04-25T00:14:18.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.231+0000 7f23e0e00640 1 -- 172.21.15.83:0/416325662 shutdown_connections 2024-04-25T00:14:18.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.231+0000 7f23e0e00640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f23b80678f0 0x7f23b8069db0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.231+0000 7f23e0e00640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc14e830 0x7f23dc14ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.231+0000 7f23e0e00640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23dc157ac0 0x7f23dc14f610 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.231+0000 7f23e0e00640 1 --2- 172.21.15.83:0/416325662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23c00039c0 0x7f23dc164310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.231+0000 7f23e0e00640 1 -- 172.21.15.83:0/416325662 >> 172.21.15.83:0/416325662 conn(0x7f23dc0b1830 msgr2=0x7f23dc0af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:18.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.231+0000 7f23e0e00640 1 -- 172.21.15.83:0/416325662 shutdown_connections 2024-04-25T00:14:18.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.231+0000 7f23e0e00640 1 -- 172.21.15.83:0/416325662 wait complete. 2024-04-25T00:14:18.255 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:14:18.255 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:14:18.255 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:14:18.256 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:14:18.256 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:14:18.256 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:14:18.256 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:14:18.256 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:14:18.256 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:14:18.256 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:14:18.257 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:14:18.257 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:14:18.257 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:14:18.257 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:14:18.257 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:14:18.257 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:14:18.269 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:14:18.269 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:14:18.271 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:14:18.271 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:14:18.273 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:14:18.273 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:14:18.274 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:14:18.274 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:14:18.319 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:14:18.319 DEBUG:teuthology.parallel:result is None 2024-04-25T00:14:18.320 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:14:18.320 DEBUG:teuthology.parallel:result is None 2024-04-25T00:14:18.321 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:14:18.321 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:14:18.321 DEBUG:teuthology.parallel:result is None 2024-04-25T00:14:18.321 DEBUG:teuthology.parallel:result is None 2024-04-25T00:14:18.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:14:18.406 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-25T00:14:18.407 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2024-04-25T00:14:18.408 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-25T00:14:18.408 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2024-04-25T00:14:18.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.459+0000 7fe6f2800640 1 Processor -- start 2024-04-25T00:14:18.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.459+0000 7fe6f2800640 1 -- start start 2024-04-25T00:14:18.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6f2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe6ec06cfd0 0x7fe6ec070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6f2800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6ec0cb640 0x7fe6ec165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6f2800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6ec06e160 con 0x7fe6ec06d3b0 2024-04-25T00:14:18.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6f2800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe6ec06df20 con 0x7fe6ec06cfd0 2024-04-25T00:14:18.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6f2800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe6ec06ea10 con 0x7fe6ec0cb640 2024-04-25T00:14:18.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6e2a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe6ec06d3b0 0x7fe6ec06d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38264/0 (socket says 172.21.15.83:38264) 2024-04-25T00:14:18.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6e2a00640 1 -- 172.21.15.83:0/524205598 learned_addr learned my addr 172.21.15.83:0/524205598 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:18.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6eb400640 1 --2- 172.21.15.83:0/524205598 >> v2:172.21.15.86:3300/0 conn(0x7fe6ec06cfd0 0x7fe6ec070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6ebe00640 1 --2- 172.21.15.83:0/524205598 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6ec0cb640 0x7fe6ec165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6eaa00640 1 -- 172.21.15.83:0/524205598 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 83945990 0 0) 0x7fe6ec06e160 con 0x7fe6ec06d3b0 2024-04-25T00:14:18.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6eaa00640 1 -- 172.21.15.83:0/524205598 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6d0003530 con 0x7fe6ec06d3b0 2024-04-25T00:14:18.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6eb400640 1 -- 172.21.15.83:0/524205598 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6ec0cb640 msgr2=0x7fe6ec165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:18.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6eb400640 1 --2- 172.21.15.83:0/524205598 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6ec0cb640 0x7fe6ec165930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6eb400640 1 -- 172.21.15.83:0/524205598 >> v1:172.21.15.83:6789/0 conn(0x7fe6ec06d3b0 legacy=0x7fe6ec06d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:18.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6eb400640 1 -- 172.21.15.83:0/524205598 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ec06ecf0 con 0x7fe6ec06cfd0 2024-04-25T00:14:18.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.463+0000 7fe6eb400640 1 --2- 172.21.15.83:0/524205598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec06cfd0 0x7fe6ec070d10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0097c0 tx=0x7fe6dc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12ac7c8c2997f946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:18.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6eaa00640 1 -- 172.21.15.83:0/524205598 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc0b0070 con 0x7fe6ec06cfd0 2024-04-25T00:14:18.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6eaa00640 1 -- 172.21.15.83:0/524205598 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6dc0ab600 con 0x7fe6ec06cfd0 2024-04-25T00:14:18.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6eaa00640 1 -- 172.21.15.83:0/524205598 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc0ab9d0 con 0x7fe6ec06cfd0 2024-04-25T00:14:18.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 -- 172.21.15.83:0/524205598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec06cfd0 msgr2=0x7fe6ec070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:18.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 --2- 172.21.15.83:0/524205598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec06cfd0 0x7fe6ec070d10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0097c0 tx=0x7fe6dc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 -- 172.21.15.83:0/524205598 shutdown_connections 2024-04-25T00:14:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 --2- 172.21.15.83:0/524205598 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6ec0cb640 0x7fe6ec165930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 --2- 172.21.15.83:0/524205598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec06cfd0 0x7fe6ec070d10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 -- 172.21.15.83:0/524205598 >> 172.21.15.83:0/524205598 conn(0x7fe6ec0cfa60 msgr2=0x7fe6ec0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 -- 172.21.15.83:0/524205598 shutdown_connections 2024-04-25T00:14:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 -- 172.21.15.83:0/524205598 wait complete. 2024-04-25T00:14:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 Processor -- start 2024-04-25T00:14:18.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 -- start start 2024-04-25T00:14:18.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec06cfd0 0x7fe6ec159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6ec06d3b0 0x7fe6ec139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6ec0cb640 0x7fe6ec1574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec1688a0 con 0x7fe6ec0cb640 2024-04-25T00:14:18.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec145970 con 0x7fe6ec06cfd0 2024-04-25T00:14:18.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6f2800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe6ec168a20 con 0x7fe6ec06d3b0 2024-04-25T00:14:18.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6ebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6ec0cb640 0x7fe6ec1574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6ebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6ec0cb640 0x7fe6ec1574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54920/0 (socket says 172.21.15.83:54920) 2024-04-25T00:14:18.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6ebe00640 1 -- 172.21.15.83:0/2040884457 learned_addr learned my addr 172.21.15.83:0/2040884457 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:18.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6eb400640 1 --2- 172.21.15.83:0/2040884457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec06cfd0 0x7fe6ec159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.467+0000 7fe6e2a00640 1 --2- 172.21.15.83:0/2040884457 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6ec06d3b0 0x7fe6ec139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6ebe00640 1 -- 172.21.15.83:0/2040884457 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6ec06d3b0 msgr2=0x7fe6ec139bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6ebe00640 1 --2- 172.21.15.83:0/2040884457 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6ec06d3b0 0x7fe6ec139bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6ebe00640 1 -- 172.21.15.83:0/2040884457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec06cfd0 msgr2=0x7fe6ec159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6ebe00640 1 --2- 172.21.15.83:0/2040884457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec06cfd0 0x7fe6ec159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6ebe00640 1 -- 172.21.15.83:0/2040884457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ec157eb0 con 0x7fe6ec0cb640 2024-04-25T00:14:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6ebe00640 1 --2- 172.21.15.83:0/2040884457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6ec0cb640 0x7fe6ec1574e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400efc0 tx=0x7fe6e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6e3e00640 1 -- 172.21.15.83:0/2040884457 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4018020 con 0x7fe6ec0cb640 2024-04-25T00:14:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6e3e00640 1 -- 172.21.15.83:0/2040884457 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e4014480 con 0x7fe6ec0cb640 2024-04-25T00:14:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6f2800640 1 -- 172.21.15.83:0/2040884457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ec158190 con 0x7fe6ec0cb640 2024-04-25T00:14:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6e3e00640 1 -- 172.21.15.83:0/2040884457 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4009e60 con 0x7fe6ec0cb640 2024-04-25T00:14:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6f2800640 1 -- 172.21.15.83:0/2040884457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ec158750 con 0x7fe6ec0cb640 2024-04-25T00:14:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6f2800640 1 -- 172.21.15.83:0/2040884457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6ec068230 con 0x7fe6ec0cb640 2024-04-25T00:14:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6e3e00640 1 -- 172.21.15.83:0/2040884457 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe6e401b020 con 0x7fe6ec0cb640 2024-04-25T00:14:18.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/2040884457 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6c80678c0 0x7fe6c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:18.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6e3e00640 1 -- 172.21.15.83:0/2040884457 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7fe6e40867a0 con 0x7fe6ec0cb640 2024-04-25T00:14:18.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.471+0000 7fe6eb400640 1 --2- 172.21.15.83:0/2040884457 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6c80678c0 0x7fe6c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:18.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.475+0000 7fe6eb400640 1 --2- 172.21.15.83:0/2040884457 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6c80678c0 0x7fe6c8069d80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0ad000 tx=0x7fe6dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.475+0000 7fe6e3e00640 1 -- 172.21.15.83:0/2040884457 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe6e4022030 con 0x7fe6ec0cb640 2024-04-25T00:14:18.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:18.643+0000 7fe6f2800640 1 -- 172.21.15.83:0/2040884457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe6ec13b2d0 con 0x7fe6ec0cb640 2024-04-25T00:14:19.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.311+0000 7fe6e3e00640 1 -- 172.21.15.83:0/2040884457 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v76) ==== 97+0+0 (secure 0 0 0) 0x7fe6e4010040 con 0x7fe6ec0cb640 2024-04-25T00:14:19.315 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-25T00:14:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.315+0000 7fe6f2800640 1 -- 172.21.15.83:0/2040884457 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6c80678c0 msgr2=0x7fe6c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.315+0000 7fe6f2800640 1 --2- 172.21.15.83:0/2040884457 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6c80678c0 0x7fe6c8069d80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0ad000 tx=0x7fe6dc008040 comp rx=0 tx=0).stop 2024-04-25T00:14:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.315+0000 7fe6f2800640 1 -- 172.21.15.83:0/2040884457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6ec0cb640 msgr2=0x7fe6ec1574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.315+0000 7fe6f2800640 1 --2- 172.21.15.83:0/2040884457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6ec0cb640 0x7fe6ec1574e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400efc0 tx=0x7fe6e400c490 comp rx=0 tx=0).stop 2024-04-25T00:14:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.315+0000 7fe6f2800640 1 -- 172.21.15.83:0/2040884457 shutdown_connections 2024-04-25T00:14:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.315+0000 7fe6f2800640 1 --2- 172.21.15.83:0/2040884457 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe6c80678c0 0x7fe6c8069d80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.315+0000 7fe6f2800640 1 --2- 172.21.15.83:0/2040884457 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe6ec06d3b0 0x7fe6ec139bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.315+0000 7fe6f2800640 1 --2- 172.21.15.83:0/2040884457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6ec0cb640 0x7fe6ec1574e0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.315+0000 7fe6f2800640 1 --2- 172.21.15.83:0/2040884457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec06cfd0 0x7fe6ec159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.315+0000 7fe6f2800640 1 -- 172.21.15.83:0/2040884457 >> 172.21.15.83:0/2040884457 conn(0x7fe6ec0cfa60 msgr2=0x7fe6ec13a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.315+0000 7fe6f2800640 1 -- 172.21.15.83:0/2040884457 shutdown_connections 2024-04-25T00:14:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.315+0000 7fe6f2800640 1 -- 172.21.15.83:0/2040884457 wait complete. 2024-04-25T00:14:19.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:14:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.475+0000 7f356e200640 1 Processor -- start 2024-04-25T00:14:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.475+0000 7f356e200640 1 -- start start 2024-04-25T00:14:19.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.475+0000 7f356e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f356815e330 0x7f35680cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:19.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.475+0000 7f356e200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35680d0010 0x7f35680b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:19.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.475+0000 7f356e200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35680b2b00 con 0x7f35681585b0 2024-04-25T00:14:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.475+0000 7f356e200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f35680d0420 con 0x7f35680d0010 2024-04-25T00:14:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.475+0000 7f356e200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f356815e830 con 0x7f356815e330 2024-04-25T00:14:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.475+0000 7f356d800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35680d0010 0x7f35680b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.475+0000 7f356d800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35680d0010 0x7f35680b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54932/0 (socket says 172.21.15.83:54932) 2024-04-25T00:14:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.475+0000 7f356d800640 1 -- 172.21.15.83:0/4275098396 learned_addr learned my addr 172.21.15.83:0/4275098396 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.479+0000 7f355fe00640 1 --2- 172.21.15.83:0/4275098396 >> v2:172.21.15.86:3300/0 conn(0x7f356815e330 0x7f35680cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.479+0000 7f356d800640 1 -- 172.21.15.83:0/4275098396 >> v2:172.21.15.86:3300/0 conn(0x7f356815e330 msgr2=0x7f35680cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.479+0000 7f356d800640 1 --2- 172.21.15.83:0/4275098396 >> v2:172.21.15.86:3300/0 conn(0x7f356815e330 0x7f35680cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.479+0000 7f356d800640 1 -- 172.21.15.83:0/4275098396 >> v1:172.21.15.86:6789/0 conn(0x7f35681585b0 legacy=0x7f356806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.479+0000 7f356d800640 1 -- 172.21.15.83:0/4275098396 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356815eb10 con 0x7f35680d0010 2024-04-25T00:14:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.479+0000 7f355fe00640 1 --2- 172.21.15.83:0/4275098396 >> v2:172.21.15.86:3300/0 conn(0x7f356815e330 0x7f35680cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.479+0000 7f356d800640 1 --2- 172.21.15.83:0/4275098396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35680d0010 0x7f35680b2fc0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f35680b29e0 tx=0x7f356001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34383e2fe018d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.479+0000 7f3567e00640 1 -- 172.21.15.83:0/4275098396 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35600b4070 con 0x7f35680d0010 2024-04-25T00:14:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.479+0000 7f3567e00640 1 -- 172.21.15.83:0/4275098396 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35600af400 con 0x7f35680d0010 2024-04-25T00:14:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.479+0000 7f3567e00640 1 -- 172.21.15.83:0/4275098396 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35600af7d0 con 0x7f35680d0010 2024-04-25T00:14:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.479+0000 7f356e200640 1 -- 172.21.15.83:0/4275098396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35680d0010 msgr2=0x7f35680b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.479+0000 7f356e200640 1 --2- 172.21.15.83:0/4275098396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35680d0010 0x7f35680b2fc0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f35680b29e0 tx=0x7f356001f1e0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356e200640 1 -- 172.21.15.83:0/4275098396 shutdown_connections 2024-04-25T00:14:19.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356e200640 1 --2- 172.21.15.83:0/4275098396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35680d0010 0x7f35680b2fc0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356e200640 1 --2- 172.21.15.83:0/4275098396 >> v2:172.21.15.86:3300/0 conn(0x7f356815e330 0x7f35680cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356e200640 1 -- 172.21.15.83:0/4275098396 >> 172.21.15.83:0/4275098396 conn(0x7f356806b3b0 msgr2=0x7f356806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:19.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356e200640 1 -- 172.21.15.83:0/4275098396 shutdown_connections 2024-04-25T00:14:19.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356e200640 1 -- 172.21.15.83:0/4275098396 wait complete. 2024-04-25T00:14:19.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356e200640 1 Processor -- start 2024-04-25T00:14:19.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356e200640 1 -- start start 2024-04-25T00:14:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35680d0010 0x7f356814a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356e200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35681585b0 0x7f356815d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f356815e330 0x7f3568159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356e200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3568071580 con 0x7f356815e330 2024-04-25T00:14:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f356815f0d0 con 0x7f35680d0010 2024-04-25T00:14:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356e200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f356806dca0 con 0x7f35681585b0 2024-04-25T00:14:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f356815e330 0x7f3568159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f356815e330 0x7f3568159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54944/0 (socket says 172.21.15.83:54944) 2024-04-25T00:14:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356d800640 1 -- 172.21.15.83:0/1793931818 learned_addr learned my addr 172.21.15.83:0/1793931818 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f355fe00640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35681585b0 0x7f356815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356ce00640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35680d0010 0x7f356814a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356d800640 1 -- 172.21.15.83:0/1793931818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35681585b0 msgr2=0x7f356815d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356d800640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35681585b0 0x7f356815d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356d800640 1 -- 172.21.15.83:0/1793931818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35680d0010 msgr2=0x7f356814a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356d800640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35680d0010 0x7f356814a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356d800640 1 -- 172.21.15.83:0/1793931818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3568158b90 con 0x7f356815e330 2024-04-25T00:14:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f355fe00640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35681585b0 0x7f356815d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356ce00640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35680d0010 0x7f356814a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.483+0000 7f356d800640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f356815e330 0x7f3568159910 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f35600b2040 tx=0x7f356001bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.487+0000 7f3565600640 1 -- 172.21.15.83:0/1793931818 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35600b4040 con 0x7f356815e330 2024-04-25T00:14:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.487+0000 7f3565600640 1 -- 172.21.15.83:0/1793931818 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35600c0070 con 0x7f356815e330 2024-04-25T00:14:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.487+0000 7f3565600640 1 -- 172.21.15.83:0/1793931818 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560009510 con 0x7f356815e330 2024-04-25T00:14:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.487+0000 7f356e200640 1 -- 172.21.15.83:0/1793931818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3568158e30 con 0x7f356815e330 2024-04-25T00:14:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.487+0000 7f356e200640 1 -- 172.21.15.83:0/1793931818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3568159e50 con 0x7f356815e330 2024-04-25T00:14:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.487+0000 7f356e200640 1 -- 172.21.15.83:0/1793931818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f356804f510 con 0x7f356815e330 2024-04-25T00:14:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.487+0000 7f3565600640 1 -- 172.21.15.83:0/1793931818 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f35600c5460 con 0x7f356815e330 2024-04-25T00:14:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.487+0000 7f3565600640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35440678c0 0x7f3544069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.487+0000 7f3565600640 1 -- 172.21.15.83:0/1793931818 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5249+0+0 (secure 0 0 0) 0x7f3560123e70 con 0x7f356815e330 2024-04-25T00:14:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.487+0000 7f356ce00640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35440678c0 0x7f3544069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.491+0000 7f356ce00640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35440678c0 0x7f3544069d80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3558001710 tx=0x7f3558009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.495+0000 7f3565600640 1 -- 172.21.15.83:0/1793931818 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3560018d40 con 0x7f356815e330 2024-04-25T00:14:19.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.703+0000 7f356e200640 1 -- 172.21.15.83:0/1793931818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f35680aecc0 con 0x7f356815e330 2024-04-25T00:14:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.703+0000 7f3565600640 1 -- 172.21.15.83:0/1793931818 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f35600f4b90 con 0x7f356815e330 2024-04-25T00:14:19.707 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:19.707 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:14:19.707 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-25T00:14:19.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.707 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-25T00:14:19.707 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmytlRFRAA3vHLqK3vHSB+8+TZHAuj7Q==", 2024-04-25T00:14:19.708 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.708 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:19.708 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:14:19.708 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:14:19.708 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.708 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-25T00:14:19.708 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm088MIRAApJvvePXyFoO8JnVErd57zQ==", 2024-04-25T00:14:19.708 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.708 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:14:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:14:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-25T00:14:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmCqZXHBAAPy5JGbEU11i4gc3gFtbGIQ==", 2024-04-25T00:14:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:14:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:14:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.710 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-25T00:14:19.710 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmiJZGKBAA5CBwH/cA+LOAj+inS/Hx2g==", 2024-04-25T00:14:19.710 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.710 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:19.710 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:14:19.710 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:14:19.710 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.711 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-25T00:14:19.711 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBhnylmMOiqLRAAdqrCTz4KfghoJ7cZc+b4qg==", 2024-04-25T00:14:19.711 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.711 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:19.711 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:19.711 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.711 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.711 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-25T00:14:19.711 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBknylmlp8OOBAAl+y4vBrCzDyV/ne9qICQ+w==", 2024-04-25T00:14:19.711 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.711 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBonylm85MOBhAA3ciY1UT/Qg4Pe72OA0IvnA==", 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.712 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-25T00:14:19.713 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBrnylmnAbDEhAAkCQXjbiKINiftX71HBKvQA==", 2024-04-25T00:14:19.713 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.713 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:19.713 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:19.713 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.713 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-25T00:14:19.713 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBxnylmxhy2KxAAs6qjxZA+Fj5umHU/4FVqGw==", 2024-04-25T00:14:19.713 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.713 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:19.713 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:19.713 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.714 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-25T00:14:19.714 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB0nylmGfSlNRAA45tuuh+5ISOMlhaFdCE5rQ==", 2024-04-25T00:14:19.714 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.714 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:19.714 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:19.714 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.715 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-25T00:14:19.715 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB4nylmouyBARAAz4s4S1BSiDU9YxQ/Vj0d0A==", 2024-04-25T00:14:19.715 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.715 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:19.715 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:19.715 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.715 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.715 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-25T00:14:19.715 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB7nylmtIh0CxAApWJLvC8Zlt298VJ55gPrqw==", 2024-04-25T00:14:19.715 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.716 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:19.716 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:19.716 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.716 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-25T00:14:19.716 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmOqooLBAApte+X6YkRmrpe12YdjYekg==", 2024-04-25T00:14:19.716 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.716 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:19.716 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-04-25T00:14:19.716 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:14:19.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.717 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-25T00:14:19.717 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmDH93NRAASQzJNC3nI0NeU8cv951eBA==", 2024-04-25T00:14:19.717 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.717 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:19.717 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:14:19.717 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:14:19.717 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:14:19.717 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.717 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-25T00:14:19.717 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmHw3+MRAAciXRmexPxmrf7U2jKwpZQQ==", 2024-04-25T00:14:19.717 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.718 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:19.718 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:14:19.718 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:14:19.718 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:14:19.718 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.718 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-25T00:14:19.718 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBfnylmIsMCABAAu+yzPRupFFqA/yH4GSZ+8w==", 2024-04-25T00:14:19.718 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.718 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:19.718 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:14:19.718 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:14:19.718 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:14:19.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.719 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-25T00:14:19.719 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdnylmZ7SKNRAArVHYbh7fBiOUNRTenIwT1w==", 2024-04-25T00:14:19.719 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.719 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:14:19.719 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:14:19.719 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:14:19.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.719 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-25T00:14:19.720 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm6TMqERAAiPL81HRFrlz4ku20HemS4w==", 2024-04-25T00:14:19.720 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.720 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:14:19.720 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-25T00:14:19.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:19.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:19.720 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-25T00:14:19.720 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm9VRpDRAA5XUtYwjiWBpF1ObJ/Y5g0w==", 2024-04-25T00:14:19.720 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:19.720 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:14:19.720 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-25T00:14:19.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:19.721 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.721 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:19.721 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T00:14:19.721 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:14:19.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.703+0000 7f356e200640 1 -- 172.21.15.83:0/1793931818 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35440678c0 msgr2=0x7f3544069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:19.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.703+0000 7f356e200640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35440678c0 0x7f3544069d80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3558001710 tx=0x7f3558009040 comp rx=0 tx=0).stop 2024-04-25T00:14:19.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.703+0000 7f356e200640 1 -- 172.21.15.83:0/1793931818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f356815e330 msgr2=0x7f3568159910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:19.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.703+0000 7f356e200640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f356815e330 0x7f3568159910 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f35600b2040 tx=0x7f356001bf80 comp rx=0 tx=0).stop 2024-04-25T00:14:19.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.703+0000 7f356e200640 1 -- 172.21.15.83:0/1793931818 shutdown_connections 2024-04-25T00:14:19.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.703+0000 7f356e200640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35440678c0 0x7f3544069d80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.703+0000 7f356e200640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35681585b0 0x7f356815d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.703+0000 7f356e200640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f356815e330 0x7f3568159910 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.703+0000 7f356e200640 1 --2- 172.21.15.83:0/1793931818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35680d0010 0x7f356814a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.703+0000 7f356e200640 1 -- 172.21.15.83:0/1793931818 >> 172.21.15.83:0/1793931818 conn(0x7f356806b3b0 msgr2=0x7f35680b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:19.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.703+0000 7f356e200640 1 -- 172.21.15.83:0/1793931818 shutdown_connections 2024-04-25T00:14:19.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.707+0000 7f356e200640 1 -- 172.21.15.83:0/1793931818 wait complete. 2024-04-25T00:14:19.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:19.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.859+0000 7f7d9be00640 1 Processor -- start 2024-04-25T00:14:19.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.859+0000 7f7d9be00640 1 -- start start 2024-04-25T00:14:19.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.859+0000 7f7d9be00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7d940b47e0 0x7f7d940b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:19.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.859+0000 7f7d9be00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d940b4c70 con 0x7f7d941553e0 2024-04-25T00:14:19.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.859+0000 7f7d9be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d9406ef80 con 0x7f7d940b4360 2024-04-25T00:14:19.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.859+0000 7f7d9be00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7d9415b510 con 0x7f7d940b47e0 2024-04-25T00:14:19.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.859+0000 7f7d99400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7d941553e0 0x7f7d9406e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38278/0 (socket says 172.21.15.83:38278) 2024-04-25T00:14:19.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.863+0000 7f7d99400640 1 -- 172.21.15.83:0/3201167745 learned_addr learned my addr 172.21.15.83:0/3201167745 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:19.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.863+0000 7f7d99e00640 1 --2- 172.21.15.83:0/3201167745 >> v2:172.21.15.83:3300/0 conn(0x7f7d940b47e0 0x7f7d940b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:19.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.863+0000 7f7d99e00640 1 -- 172.21.15.83:0/3201167745 >> v1:172.21.15.86:6789/0 conn(0x7f7d940b4360 legacy=0x7f7d940b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:19.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.863+0000 7f7d99e00640 1 -- 172.21.15.83:0/3201167745 >> v1:172.21.15.83:6789/0 conn(0x7f7d941553e0 legacy=0x7f7d9406e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:19.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.863+0000 7f7d99e00640 1 -- 172.21.15.83:0/3201167745 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9415b7f0 con 0x7f7d940b47e0 2024-04-25T00:14:19.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.863+0000 7f7d99e00640 1 --2- 172.21.15.83:0/3201167745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d940b47e0 0x7f7d940b3240 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009d90 tx=0x7f7d8c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69d552e4c143373 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:19.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.863+0000 7f7d93e00640 1 -- 172.21.15.83:0/3201167745 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c0b2070 con 0x7f7d940b47e0 2024-04-25T00:14:19.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.863+0000 7f7d93e00640 1 -- 172.21.15.83:0/3201167745 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d8c0a57a0 con 0x7f7d940b47e0 2024-04-25T00:14:19.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.863+0000 7f7d93e00640 1 -- 172.21.15.83:0/3201167745 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c0a5b70 con 0x7f7d940b47e0 2024-04-25T00:14:19.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 -- 172.21.15.83:0/3201167745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d940b47e0 msgr2=0x7f7d940b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:19.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 --2- 172.21.15.83:0/3201167745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d940b47e0 0x7f7d940b3240 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009d90 tx=0x7f7d8c01c8e0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 -- 172.21.15.83:0/3201167745 shutdown_connections 2024-04-25T00:14:19.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 --2- 172.21.15.83:0/3201167745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d940b47e0 0x7f7d940b3240 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 -- 172.21.15.83:0/3201167745 >> 172.21.15.83:0/3201167745 conn(0x7f7d940b1830 msgr2=0x7f7d940b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:19.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 -- 172.21.15.83:0/3201167745 shutdown_connections 2024-04-25T00:14:19.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 -- 172.21.15.83:0/3201167745 wait complete. 2024-04-25T00:14:19.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 Processor -- start 2024-04-25T00:14:19.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 -- start start 2024-04-25T00:14:19.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d940b4360 0x7f7d940cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:19.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d940b47e0 0x7f7d94153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:19.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d941553e0 0x7f7d9413c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:19.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d94155940 con 0x7f7d940b4360 2024-04-25T00:14:19.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d941557c0 con 0x7f7d941553e0 2024-04-25T00:14:19.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d9be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7d9406ebd0 con 0x7f7d940b47e0 2024-04-25T00:14:19.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d99400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d940b4360 0x7f7d940cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:19.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d99400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d940b4360 0x7f7d940cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54950/0 (socket says 172.21.15.83:54950) 2024-04-25T00:14:19.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d99400640 1 -- 172.21.15.83:0/202415367 learned_addr learned my addr 172.21.15.83:0/202415367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:19.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d99e00640 1 --2- 172.21.15.83:0/202415367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d941553e0 0x7f7d9413c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:19.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d98a00640 1 --2- 172.21.15.83:0/202415367 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d940b47e0 0x7f7d94153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:19.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d99400640 1 -- 172.21.15.83:0/202415367 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d940b47e0 msgr2=0x7f7d94153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:19.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d99400640 1 --2- 172.21.15.83:0/202415367 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d940b47e0 0x7f7d94153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d99400640 1 -- 172.21.15.83:0/202415367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d941553e0 msgr2=0x7f7d9413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:19.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d99400640 1 --2- 172.21.15.83:0/202415367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d941553e0 0x7f7d9413c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:19.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d99400640 1 -- 172.21.15.83:0/202415367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d94154630 con 0x7f7d940b4360 2024-04-25T00:14:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d99400640 1 --2- 172.21.15.83:0/202415367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d940b4360 0x7f7d940cbd20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800c400 tx=0x7f7d8800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.867+0000 7f7d91600640 1 -- 172.21.15.83:0/202415367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88017070 con 0x7f7d940b4360 2024-04-25T00:14:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.871+0000 7f7d91600640 1 -- 172.21.15.83:0/202415367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d88005cf0 con 0x7f7d940b4360 2024-04-25T00:14:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.871+0000 7f7d91600640 1 -- 172.21.15.83:0/202415367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d880072d0 con 0x7f7d940b4360 2024-04-25T00:14:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.871+0000 7f7d9be00640 1 -- 172.21.15.83:0/202415367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d940caf60 con 0x7f7d940b4360 2024-04-25T00:14:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.871+0000 7f7d9be00640 1 -- 172.21.15.83:0/202415367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d940cb520 con 0x7f7d940b4360 2024-04-25T00:14:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.871+0000 7f7d9be00640 1 -- 172.21.15.83:0/202415367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d940737f0 con 0x7f7d940b4360 2024-04-25T00:14:19.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.871+0000 7f7d91600640 1 -- 172.21.15.83:0/202415367 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7d8801a070 con 0x7f7d940b4360 2024-04-25T00:14:19.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.871+0000 7f7d91600640 1 --2- 172.21.15.83:0/202415367 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7d740678c0 0x7f7d74069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:19.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.871+0000 7f7d91600640 1 -- 172.21.15.83:0/202415367 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5249+0+0 (secure 0 0 0) 0x7f7d88084c80 con 0x7f7d940b4360 2024-04-25T00:14:19.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.871+0000 7f7d98a00640 1 --2- 172.21.15.83:0/202415367 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7d740678c0 0x7f7d74069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:19.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.871+0000 7f7d98a00640 1 --2- 172.21.15.83:0/202415367 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7d740678c0 0x7f7d74069d80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7d94154bc0 tx=0x7f7d80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:19.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:19.875+0000 7f7d91600640 1 -- 172.21.15.83:0/202415367 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7d880553c0 con 0x7f7d940b4360 2024-04-25T00:14:20.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.031+0000 7f7d9be00640 1 -- 172.21.15.83:0/202415367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d9404f5f0 con 0x7f7d940b4360 2024-04-25T00:14:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.031+0000 7f7d91600640 1 -- 172.21.15.83:0/202415367 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+13081 (secure 0 0 0) 0x7f7d880596c0 con 0x7f7d940b4360 2024-04-25T00:14:20.038 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:20.038 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":78,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:19.324221+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.035+0000 7f7d9be00640 1 -- 172.21.15.83:0/202415367 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7d740678c0 msgr2=0x7f7d74069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.035+0000 7f7d9be00640 1 --2- 172.21.15.83:0/202415367 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7d740678c0 0x7f7d74069d80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7d94154bc0 tx=0x7f7d80009040 comp rx=0 tx=0).stop 2024-04-25T00:14:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.035+0000 7f7d9be00640 1 -- 172.21.15.83:0/202415367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d940b4360 msgr2=0x7f7d940cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.035+0000 7f7d9be00640 1 --2- 172.21.15.83:0/202415367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d940b4360 0x7f7d940cbd20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800c400 tx=0x7f7d8800c8c0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.035+0000 7f7d9be00640 1 -- 172.21.15.83:0/202415367 shutdown_connections 2024-04-25T00:14:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.035+0000 7f7d9be00640 1 --2- 172.21.15.83:0/202415367 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7d740678c0 0x7f7d74069d80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.035+0000 7f7d9be00640 1 --2- 172.21.15.83:0/202415367 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d940b47e0 0x7f7d94153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.035+0000 7f7d9be00640 1 --2- 172.21.15.83:0/202415367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d941553e0 0x7f7d9413c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.035+0000 7f7d9be00640 1 --2- 172.21.15.83:0/202415367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d940b4360 0x7f7d940cbd20 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.035+0000 7f7d9be00640 1 -- 172.21.15.83:0/202415367 >> 172.21.15.83:0/202415367 conn(0x7f7d940b1830 msgr2=0x7f7d9415cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.035+0000 7f7d9be00640 1 -- 172.21.15.83:0/202415367 shutdown_connections 2024-04-25T00:14:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.035+0000 7f7d9be00640 1 -- 172.21.15.83:0/202415367 wait complete. 2024-04-25T00:14:20.057 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:14:20.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:14:20.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.191+0000 7f2dcf400640 1 Processor -- start 2024-04-25T00:14:20.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.191+0000 7f2dcf400640 1 -- start start 2024-04-25T00:14:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.191+0000 7f2dcf400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2dc8159810 0x7f2dc80cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.191+0000 7f2dcf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2dc806e950 0x7f2dc815f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.191+0000 7f2dcf400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dc80ce000 con 0x7f2dc815fe50 2024-04-25T00:14:20.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.191+0000 7f2dcf400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2dc806e210 con 0x7f2dc8159810 2024-04-25T00:14:20.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.191+0000 7f2dcf400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2dc81602a0 con 0x7f2dc806e950 2024-04-25T00:14:20.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.191+0000 7f2dcca00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2dc8159810 0x7f2dc80cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.191+0000 7f2dcd400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2dc815fe50 0x7f2dc815c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38288/0 (socket says 172.21.15.83:38288) 2024-04-25T00:14:20.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.191+0000 7f2dcd400640 1 -- 172.21.15.83:0/3300689405 learned_addr learned my addr 172.21.15.83:0/3300689405 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:20.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.195+0000 7f2dbfe00640 1 --2- 172.21.15.83:0/3300689405 >> v2:172.21.15.86:3300/0 conn(0x7f2dc806e950 0x7f2dc815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.195+0000 7f2dcca00640 1 -- 172.21.15.83:0/3300689405 >> v2:172.21.15.86:3300/0 conn(0x7f2dc806e950 msgr2=0x7f2dc815f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:20.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.195+0000 7f2dcca00640 1 --2- 172.21.15.83:0/3300689405 >> v2:172.21.15.86:3300/0 conn(0x7f2dc806e950 0x7f2dc815f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.195+0000 7f2dcca00640 1 -- 172.21.15.83:0/3300689405 >> v1:172.21.15.83:6789/0 conn(0x7f2dc815fe50 legacy=0x7f2dc815c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.195+0000 7f2dcca00640 1 -- 172.21.15.83:0/3300689405 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc806eec0 con 0x7f2dc8159810 2024-04-25T00:14:20.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.195+0000 7f2dc7e00640 1 -- 172.21.15.83:0/3300689405 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1754861270 0 0) 0x7f2dc80ce000 con 0x7f2dc815fe50 2024-04-25T00:14:20.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.195+0000 7f2dcca00640 1 --2- 172.21.15.83:0/3300689405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc8159810 0x7f2dc80cc4a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80709a0 tx=0x7f2db801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3994ebb3da30e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:20.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.195+0000 7f2dc7e00640 1 -- 172.21.15.83:0/3300689405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db80b0070 con 0x7f2dc8159810 2024-04-25T00:14:20.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.195+0000 7f2dc7e00640 1 -- 172.21.15.83:0/3300689405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db80ab600 con 0x7f2dc8159810 2024-04-25T00:14:20.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.195+0000 7f2dc7e00640 1 -- 172.21.15.83:0/3300689405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db80ab9d0 con 0x7f2dc8159810 2024-04-25T00:14:20.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.195+0000 7f2dcf400640 1 -- 172.21.15.83:0/3300689405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc8159810 msgr2=0x7f2dc80cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.195+0000 7f2dcf400640 1 --2- 172.21.15.83:0/3300689405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc8159810 0x7f2dc80cc4a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80709a0 tx=0x7f2db801c3f0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 -- 172.21.15.83:0/3300689405 shutdown_connections 2024-04-25T00:14:20.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 --2- 172.21.15.83:0/3300689405 >> v2:172.21.15.86:3300/0 conn(0x7f2dc806e950 0x7f2dc815f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 --2- 172.21.15.83:0/3300689405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc8159810 0x7f2dc80cc4a0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 -- 172.21.15.83:0/3300689405 >> 172.21.15.83:0/3300689405 conn(0x7f2dc80b1830 msgr2=0x7f2dc80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:20.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 -- 172.21.15.83:0/3300689405 shutdown_connections 2024-04-25T00:14:20.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 -- 172.21.15.83:0/3300689405 wait complete. 2024-04-25T00:14:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 Processor -- start 2024-04-25T00:14:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 -- start start 2024-04-25T00:14:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc806e950 0x7f2dc8148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dc8159810 0x7f2dc80b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc815fe50 0x7f2dc80b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc8159d70 con 0x7f2dc815fe50 2024-04-25T00:14:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc8159bf0 con 0x7f2dc806e950 2024-04-25T00:14:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc8070da0 con 0x7f2dc8159810 2024-04-25T00:14:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc815fe50 0x7f2dc80b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc815fe50 0x7f2dc80b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54958/0 (socket says 172.21.15.83:54958) 2024-04-25T00:14:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcd400640 1 -- 172.21.15.83:0/613997662 learned_addr learned my addr 172.21.15.83:0/613997662 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcca00640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc806e950 0x7f2dc8148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dbfe00640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dc8159810 0x7f2dc80b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcd400640 1 -- 172.21.15.83:0/613997662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dc8159810 msgr2=0x7f2dc80b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcd400640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dc8159810 0x7f2dc80b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcd400640 1 -- 172.21.15.83:0/613997662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc806e950 msgr2=0x7f2dc8148110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcd400640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc806e950 0x7f2dc8148110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcd400640 1 -- 172.21.15.83:0/613997662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc81541e0 con 0x7f2dc815fe50 2024-04-25T00:14:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dbfe00640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dc8159810 0x7f2dc80b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcca00640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc806e950 0x7f2dc8148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcd400640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc815fe50 0x7f2dc80b0f60 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000caf0 tx=0x7f2dc00021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dc5600640 1 -- 172.21.15.83:0/613997662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc000ac00 con 0x7f2dc815fe50 2024-04-25T00:14:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dc5600640 1 -- 172.21.15.83:0/613997662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc000ada0 con 0x7f2dc815fe50 2024-04-25T00:14:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dc5600640 1 -- 172.21.15.83:0/613997662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0012640 con 0x7f2dc815fe50 2024-04-25T00:14:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.199+0000 7f2dcf400640 1 -- 172.21.15.83:0/613997662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc81544c0 con 0x7f2dc815fe50 2024-04-25T00:14:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.203+0000 7f2dcf400640 1 -- 172.21.15.83:0/613997662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc8154a80 con 0x7f2dc815fe50 2024-04-25T00:14:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.203+0000 7f2dcf400640 1 -- 172.21.15.83:0/613997662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc804f510 con 0x7f2dc815fe50 2024-04-25T00:14:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.203+0000 7f2dc5600640 1 -- 172.21.15.83:0/613997662 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2dc00127e0 con 0x7f2dc815fe50 2024-04-25T00:14:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.203+0000 7f2dc5600640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2da40678c0 0x7f2da4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.203+0000 7f2dc5600640 1 -- 172.21.15.83:0/613997662 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5249+0+0 (secure 0 0 0) 0x7f2dc0086220 con 0x7f2dc815fe50 2024-04-25T00:14:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.203+0000 7f2dcca00640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2da40678c0 0x7f2da4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.203+0000 7f2dcca00640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2da40678c0 0x7f2da4069d80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2db80ad000 tx=0x7f2db8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:20.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.207+0000 7f2dc5600640 1 -- 172.21.15.83:0/613997662 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2dc0056e00 con 0x7f2dc815fe50 2024-04-25T00:14:20.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.371+0000 7f2dcf400640 1 -- 172.21.15.83:0/613997662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2dc8072c90 con 0x7f2dc815fe50 2024-04-25T00:14:20.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.371+0000 7f2dc5600640 1 -- 172.21.15.83:0/613997662 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f2dc0056700 con 0x7f2dc815fe50 2024-04-25T00:14:20.374 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:14:20.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.371+0000 7f2dcf400640 1 -- 172.21.15.83:0/613997662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2da40678c0 msgr2=0x7f2da4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.371+0000 7f2dcf400640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2da40678c0 0x7f2da4069d80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2db80ad000 tx=0x7f2db8008040 comp rx=0 tx=0).stop 2024-04-25T00:14:20.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.371+0000 7f2dcf400640 1 -- 172.21.15.83:0/613997662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc815fe50 msgr2=0x7f2dc80b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.371+0000 7f2dcf400640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc815fe50 0x7f2dc80b0f60 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000caf0 tx=0x7f2dc00021d0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.371+0000 7f2dcf400640 1 -- 172.21.15.83:0/613997662 shutdown_connections 2024-04-25T00:14:20.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.371+0000 7f2dcf400640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2da40678c0 0x7f2da4069d80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.371+0000 7f2dcf400640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dc8159810 0x7f2dc80b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.371+0000 7f2dcf400640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dc815fe50 0x7f2dc80b0f60 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.371+0000 7f2dcf400640 1 --2- 172.21.15.83:0/613997662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc806e950 0x7f2dc8148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.371+0000 7f2dcf400640 1 -- 172.21.15.83:0/613997662 >> 172.21.15.83:0/613997662 conn(0x7f2dc80b1830 msgr2=0x7f2dc815dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:20.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.371+0000 7f2dcf400640 1 -- 172.21.15.83:0/613997662 shutdown_connections 2024-04-25T00:14:20.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.371+0000 7f2dcf400640 1 -- 172.21.15.83:0/613997662 wait complete. 2024-04-25T00:14:20.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:14:20.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecb3e00640 1 Processor -- start 2024-04-25T00:14:20.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecb3e00640 1 -- start start 2024-04-25T00:14:20.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecb3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fecb4154aa0 0x7fecb40cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecb3e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecb406e1a0 0x7fecb40b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecb3e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecb406e950 con 0x7fecb406d5c0 2024-04-25T00:14:20.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecb3e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fecb406ec30 con 0x7fecb406e1a0 2024-04-25T00:14:20.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecb3e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fecb40b3570 con 0x7fecb4154aa0 2024-04-25T00:14:20.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecaa000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecb406e1a0 0x7fecb40b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecb3400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fecb406d5c0 0x7fecb4070bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38296/0 (socket says 172.21.15.83:38296) 2024-04-25T00:14:20.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecb3400640 1 -- 172.21.15.83:0/1855508953 learned_addr learned my addr 172.21.15.83:0/1855508953 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:20.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecb2a00640 1 --2- 172.21.15.83:0/1855508953 >> v2:172.21.15.86:3300/0 conn(0x7fecb4154aa0 0x7fecb40cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecaa000640 1 -- 172.21.15.83:0/1855508953 >> v2:172.21.15.86:3300/0 conn(0x7fecb4154aa0 msgr2=0x7fecb40cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:20.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecaa000640 1 --2- 172.21.15.83:0/1855508953 >> v2:172.21.15.86:3300/0 conn(0x7fecb4154aa0 0x7fecb40cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecaa000640 1 -- 172.21.15.83:0/1855508953 >> v1:172.21.15.83:6789/0 conn(0x7fecb406d5c0 legacy=0x7fecb4070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecaa000640 1 -- 172.21.15.83:0/1855508953 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb40b3850 con 0x7fecb406e1a0 2024-04-25T00:14:20.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.527+0000 7fecb2a00640 1 --2- 172.21.15.83:0/1855508953 >> v2:172.21.15.86:3300/0 conn(0x7fecb4154aa0 0x7fecb40cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:20.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.531+0000 7fecaa000640 1 --2- 172.21.15.83:0/1855508953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb406e1a0 0x7fecb40b2a50 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fecb4072c30 tx=0x7fec9c01d500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa76f875478bc584 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:20.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.531+0000 7fecb2000640 1 -- 172.21.15.83:0/1855508953 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0b2070 con 0x7fecb406e1a0 2024-04-25T00:14:20.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.531+0000 7fecb2000640 1 -- 172.21.15.83:0/1855508953 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec9c0ad640 con 0x7fecb406e1a0 2024-04-25T00:14:20.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.531+0000 7fecb2000640 1 -- 172.21.15.83:0/1855508953 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0ada10 con 0x7fecb406e1a0 2024-04-25T00:14:20.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.531+0000 7fecb3e00640 1 -- 172.21.15.83:0/1855508953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb406e1a0 msgr2=0x7fecb40b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.531+0000 7fecb3e00640 1 --2- 172.21.15.83:0/1855508953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb406e1a0 0x7fecb40b2a50 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fecb4072c30 tx=0x7fec9c01d500 comp rx=0 tx=0).stop 2024-04-25T00:14:20.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.531+0000 7fecb3e00640 1 -- 172.21.15.83:0/1855508953 shutdown_connections 2024-04-25T00:14:20.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.531+0000 7fecb3e00640 1 --2- 172.21.15.83:0/1855508953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb406e1a0 0x7fecb40b2a50 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.531+0000 7fecb3e00640 1 --2- 172.21.15.83:0/1855508953 >> v2:172.21.15.86:3300/0 conn(0x7fecb4154aa0 0x7fecb40cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.531+0000 7fecb3e00640 1 -- 172.21.15.83:0/1855508953 >> 172.21.15.83:0/1855508953 conn(0x7fecb406b3b0 msgr2=0x7fecb406b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:20.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.531+0000 7fecb3e00640 1 -- 172.21.15.83:0/1855508953 shutdown_connections 2024-04-25T00:14:20.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.531+0000 7fecb3e00640 1 -- 172.21.15.83:0/1855508953 wait complete. 2024-04-25T00:14:20.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3e00640 1 Processor -- start 2024-04-25T00:14:20.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3e00640 1 -- start start 2024-04-25T00:14:20.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb406d5c0 0x7fecb4141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fecb406e1a0 0x7fecb4148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb4154aa0 0x7fecb4148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fecb4166ff0 con 0x7fecb4154aa0 2024-04-25T00:14:20.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fecb4166e70 con 0x7fecb406d5c0 2024-04-25T00:14:20.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fecb4167170 con 0x7fecb406e1a0 2024-04-25T00:14:20.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb4154aa0 0x7fecb4148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb4154aa0 0x7fecb4148d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54980/0 (socket says 172.21.15.83:54980) 2024-04-25T00:14:20.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb406d5c0 0x7fecb4141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3400640 1 -- 172.21.15.83:0/4199780564 learned_addr learned my addr 172.21.15.83:0/4199780564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:20.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecaa000640 1 --2- 172.21.15.83:0/4199780564 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fecb406e1a0 0x7fecb4148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3400640 1 -- 172.21.15.83:0/4199780564 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fecb406e1a0 msgr2=0x7fecb4148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3400640 1 --2- 172.21.15.83:0/4199780564 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fecb406e1a0 0x7fecb4148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3400640 1 -- 172.21.15.83:0/4199780564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb406d5c0 msgr2=0x7fecb4141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3400640 1 --2- 172.21.15.83:0/4199780564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb406d5c0 0x7fecb4141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3400640 1 -- 172.21.15.83:0/4199780564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb4142200 con 0x7fecb4154aa0 2024-04-25T00:14:20.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3400640 1 --2- 172.21.15.83:0/4199780564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb4154aa0 0x7fecb4148d50 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fecac00ca80 tx=0x7fecac00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:20.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecab400640 1 -- 172.21.15.83:0/4199780564 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac017070 con 0x7fecb4154aa0 2024-04-25T00:14:20.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecab400640 1 -- 172.21.15.83:0/4199780564 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecac00ac30 con 0x7fecb4154aa0 2024-04-25T00:14:20.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecab400640 1 -- 172.21.15.83:0/4199780564 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac012400 con 0x7fecb4154aa0 2024-04-25T00:14:20.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3e00640 1 -- 172.21.15.83:0/4199780564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb4145050 con 0x7fecb4154aa0 2024-04-25T00:14:20.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.535+0000 7fecb3e00640 1 -- 172.21.15.83:0/4199780564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb4145610 con 0x7fecb4154aa0 2024-04-25T00:14:20.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.539+0000 7fecb3e00640 1 -- 172.21.15.83:0/4199780564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb404f510 con 0x7fecb4154aa0 2024-04-25T00:14:20.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.539+0000 7fecab400640 1 -- 172.21.15.83:0/4199780564 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fecac01a070 con 0x7fecb4154aa0 2024-04-25T00:14:20.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.539+0000 7fecab400640 1 --2- 172.21.15.83:0/4199780564 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec900678c0 0x7fec90069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.539+0000 7fecab400640 1 -- 172.21.15.83:0/4199780564 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5249+0+0 (secure 0 0 0) 0x7fecac085770 con 0x7fecb4154aa0 2024-04-25T00:14:20.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.539+0000 7fecb2a00640 1 --2- 172.21.15.83:0/4199780564 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec900678c0 0x7fec90069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.539+0000 7fecb2a00640 1 --2- 172.21.15.83:0/4199780564 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec900678c0 0x7fec90069d80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fecb414c220 tx=0x7feca4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:20.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.543+0000 7fecab400640 1 -- 172.21.15.83:0/4199780564 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fecac055d80 con 0x7fecb4154aa0 2024-04-25T00:14:20.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.699+0000 7fecb3e00640 1 -- 172.21.15.83:0/4199780564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fecb40b3350 con 0x7fecb4154aa0 2024-04-25T00:14:20.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.703+0000 7fecab400640 1 -- 172.21.15.83:0/4199780564 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7fecac05a080 con 0x7fecb4154aa0 2024-04-25T00:14:20.706 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:20.706 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-25T00:14:20.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:20.706 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-25T00:14:20.706 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-25T00:14:20.706 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:20.706 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-25T00:14:20.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.703+0000 7fecb3e00640 1 -- 172.21.15.83:0/4199780564 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec900678c0 msgr2=0x7fec90069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.703+0000 7fecb3e00640 1 --2- 172.21.15.83:0/4199780564 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec900678c0 0x7fec90069d80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fecb414c220 tx=0x7feca4008040 comp rx=0 tx=0).stop 2024-04-25T00:14:20.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.703+0000 7fecb3e00640 1 -- 172.21.15.83:0/4199780564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb4154aa0 msgr2=0x7fecb4148d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.703+0000 7fecb3e00640 1 --2- 172.21.15.83:0/4199780564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb4154aa0 0x7fecb4148d50 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fecac00ca80 tx=0x7fecac00cf40 comp rx=0 tx=0).stop 2024-04-25T00:14:20.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.703+0000 7fecb3e00640 1 -- 172.21.15.83:0/4199780564 shutdown_connections 2024-04-25T00:14:20.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.703+0000 7fecb3e00640 1 --2- 172.21.15.83:0/4199780564 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec900678c0 0x7fec90069d80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.703+0000 7fecb3e00640 1 --2- 172.21.15.83:0/4199780564 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fecb406e1a0 0x7fecb4148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.703+0000 7fecb3e00640 1 --2- 172.21.15.83:0/4199780564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb4154aa0 0x7fecb4148d50 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.703+0000 7fecb3e00640 1 --2- 172.21.15.83:0/4199780564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb406d5c0 0x7fecb4141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.703+0000 7fecb3e00640 1 -- 172.21.15.83:0/4199780564 >> 172.21.15.83:0/4199780564 conn(0x7fecb406b3b0 msgr2=0x7fecb4072340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:20.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.703+0000 7fecb3e00640 1 -- 172.21.15.83:0/4199780564 shutdown_connections 2024-04-25T00:14:20.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.703+0000 7fecb3e00640 1 -- 172.21.15.83:0/4199780564 wait complete. 2024-04-25T00:14:20.727 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:14:20.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:14:20.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4479200640 1 Processor -- start 2024-04-25T00:14:20.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4479200640 1 -- start start 2024-04-25T00:14:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4479200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44741455d0 0x7f447406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4479200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f44740d0450 0x7f44740b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4479200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44740ce610 con 0x7f44740ce080 2024-04-25T00:14:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4479200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f44740ce8f0 con 0x7f44740d0450 2024-04-25T00:14:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4479200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f44740cebd0 con 0x7f44741455d0 2024-04-25T00:14:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4472800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f44740d0450 0x7f44740b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4471400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f44740ce080 0x7f44740cfd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38308/0 (socket says 172.21.15.83:38308) 2024-04-25T00:14:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4471400640 1 -- 172.21.15.83:0/3508177957 learned_addr learned my addr 172.21.15.83:0/3508177957 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4471e00640 1 --2- 172.21.15.83:0/3508177957 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44741455d0 0x7f447406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4472800640 1 -- 172.21.15.83:0/3508177957 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44741455d0 msgr2=0x7f447406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4472800640 1 --2- 172.21.15.83:0/3508177957 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44741455d0 0x7f447406d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4472800640 1 -- 172.21.15.83:0/3508177957 >> v1:172.21.15.83:6789/0 conn(0x7f44740ce080 legacy=0x7f44740cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4472800640 1 -- 172.21.15.83:0/3508177957 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44740ceeb0 con 0x7f44740d0450 2024-04-25T00:14:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.863+0000 7f4470a00640 1 -- 172.21.15.83:0/3508177957 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1695488348 0 0) 0x7f44740ce610 con 0x7f44740ce080 2024-04-25T00:14:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.867+0000 7f4472800640 1 --2- 172.21.15.83:0/3508177957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44740d0450 0x7f44740b2b30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f446c009d90 tx=0x7f446c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a93b21ec2d7b6e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.867+0000 7f4470a00640 1 -- 172.21.15.83:0/3508177957 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c0b2070 con 0x7f44740d0450 2024-04-25T00:14:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.867+0000 7f4470a00640 1 -- 172.21.15.83:0/3508177957 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446c0a5990 con 0x7f44740d0450 2024-04-25T00:14:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.867+0000 7f4470a00640 1 -- 172.21.15.83:0/3508177957 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c0a5d60 con 0x7f44740d0450 2024-04-25T00:14:20.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.867+0000 7f4479200640 1 -- 172.21.15.83:0/3508177957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44740d0450 msgr2=0x7f44740b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.867+0000 7f4479200640 1 --2- 172.21.15.83:0/3508177957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44740d0450 0x7f44740b2b30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f446c009d90 tx=0x7f446c01c970 comp rx=0 tx=0).stop 2024-04-25T00:14:20.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.867+0000 7f4479200640 1 -- 172.21.15.83:0/3508177957 shutdown_connections 2024-04-25T00:14:20.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.867+0000 7f4479200640 1 --2- 172.21.15.83:0/3508177957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44740d0450 0x7f44740b2b30 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.867+0000 7f4479200640 1 --2- 172.21.15.83:0/3508177957 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44741455d0 0x7f447406d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.867+0000 7f4479200640 1 -- 172.21.15.83:0/3508177957 >> 172.21.15.83:0/3508177957 conn(0x7f447406b3b0 msgr2=0x7f447406b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.867+0000 7f4479200640 1 -- 172.21.15.83:0/3508177957 shutdown_connections 2024-04-25T00:14:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.867+0000 7f4479200640 1 -- 172.21.15.83:0/3508177957 wait complete. 2024-04-25T00:14:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4479200640 1 Processor -- start 2024-04-25T00:14:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4479200640 1 -- start start 2024-04-25T00:14:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4479200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44740ce080 0x7f4474158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4479200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44740d0450 0x7f4474153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4479200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44741455d0 0x7f447413d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4479200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f44740b3a00 con 0x7f44740d0450 2024-04-25T00:14:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4479200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44740b3880 con 0x7f44740ce080 2024-04-25T00:14:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4479200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f44740b3b80 con 0x7f44741455d0 2024-04-25T00:14:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4471400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44740d0450 0x7f4474153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4471400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44740d0450 0x7f4474153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55006/0 (socket says 172.21.15.83:55006) 2024-04-25T00:14:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4471400640 1 -- 172.21.15.83:0/2635523289 learned_addr learned my addr 172.21.15.83:0/2635523289 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4472800640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44741455d0 0x7f447413d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4471e00640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44740ce080 0x7f4474158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4471400640 1 -- 172.21.15.83:0/2635523289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44741455d0 msgr2=0x7f447413d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4471400640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44741455d0 0x7f447413d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4471400640 1 -- 172.21.15.83:0/2635523289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44740ce080 msgr2=0x7f4474158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4471400640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44740ce080 0x7f4474158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4471400640 1 -- 172.21.15.83:0/2635523289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44740cd430 con 0x7f44740d0450 2024-04-25T00:14:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4472800640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44741455d0 0x7f447413d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4471e00640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44740ce080 0x7f4474158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4471400640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44740d0450 0x7f4474153db0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f44600011f0 tx=0x7f446000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f446a000640 1 -- 172.21.15.83:0/2635523289 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460018070 con 0x7f44740d0450 2024-04-25T00:14:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f446a000640 1 -- 172.21.15.83:0/2635523289 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44600074e0 con 0x7f44740d0450 2024-04-25T00:14:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f446a000640 1 -- 172.21.15.83:0/2635523289 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460004a10 con 0x7f44740d0450 2024-04-25T00:14:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4479200640 1 -- 172.21.15.83:0/2635523289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447414ad50 con 0x7f44740d0450 2024-04-25T00:14:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4479200640 1 -- 172.21.15.83:0/2635523289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447414b310 con 0x7f44740d0450 2024-04-25T00:14:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.871+0000 7f4479200640 1 -- 172.21.15.83:0/2635523289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44740736f0 con 0x7f44740d0450 2024-04-25T00:14:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.875+0000 7f446a000640 1 -- 172.21.15.83:0/2635523289 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f446001b070 con 0x7f44740d0450 2024-04-25T00:14:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.875+0000 7f446a000640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44540678c0 0x7f4454069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:20.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.875+0000 7f446a000640 1 -- 172.21.15.83:0/2635523289 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5249+0+0 (secure 0 0 0) 0x7f4460084df0 con 0x7f44740d0450 2024-04-25T00:14:20.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.875+0000 7f4471e00640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44540678c0 0x7f4454069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:20.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.875+0000 7f4471e00640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44540678c0 0x7f4454069d80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4464009b10 tx=0x7f4464008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:20.879+0000 7f446a000640 1 -- 172.21.15.83:0/2635523289 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4460055af0 con 0x7f44740d0450 2024-04-25T00:14:21.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.035+0000 7f4479200640 1 -- 172.21.15.83:0/2635523289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f447404f5f0 con 0x7f44740d0450 2024-04-25T00:14:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.315+0000 7f446a000640 1 -- 172.21.15.83:0/2635523289 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7f44600553f0 con 0x7f44740d0450 2024-04-25T00:14:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-25T00:14:21.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.315+0000 7f4479200640 1 -- 172.21.15.83:0/2635523289 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44540678c0 msgr2=0x7f4454069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:21.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.315+0000 7f4479200640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44540678c0 0x7f4454069d80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4464009b10 tx=0x7f4464008040 comp rx=0 tx=0).stop 2024-04-25T00:14:21.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.315+0000 7f4479200640 1 -- 172.21.15.83:0/2635523289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44740d0450 msgr2=0x7f4474153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:21.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.315+0000 7f4479200640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44740d0450 0x7f4474153db0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f44600011f0 tx=0x7f446000c9b0 comp rx=0 tx=0).stop 2024-04-25T00:14:21.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.315+0000 7f4479200640 1 -- 172.21.15.83:0/2635523289 shutdown_connections 2024-04-25T00:14:21.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.315+0000 7f4479200640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44540678c0 0x7f4454069d80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:21.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.315+0000 7f4479200640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44741455d0 0x7f447413d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:21.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.315+0000 7f4479200640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44740d0450 0x7f4474153db0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:21.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.315+0000 7f4479200640 1 --2- 172.21.15.83:0/2635523289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44740ce080 0x7f4474158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:21.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.315+0000 7f4479200640 1 -- 172.21.15.83:0/2635523289 >> 172.21.15.83:0/2635523289 conn(0x7f447406b3b0 msgr2=0x7f44741555c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:21.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.315+0000 7f4479200640 1 -- 172.21.15.83:0/2635523289 shutdown_connections 2024-04-25T00:14:21.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.315+0000 7f4479200640 1 -- 172.21.15.83:0/2635523289 wait complete. 2024-04-25T00:14:21.345 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:14:21.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.483+0000 7f1644e00640 1 Processor -- start 2024-04-25T00:14:21.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.487+0000 7f1644e00640 1 -- start start 2024-04-25T00:14:21.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.487+0000 7f1644e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f164006e950 0x7f164015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:21.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.487+0000 7f1644e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16400ce000 con 0x7f1640159810 2024-04-25T00:14:21.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.487+0000 7f1644e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f164006eec0 con 0x7f164015fe50 2024-04-25T00:14:21.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.487+0000 7f1644e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f164006e210 con 0x7f164006e950 2024-04-25T00:14:21.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.487+0000 7f163de00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1640159810 0x7f16400cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38318/0 (socket says 172.21.15.83:38318) 2024-04-25T00:14:21.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.487+0000 7f163de00640 1 -- 172.21.15.83:0/2169778082 learned_addr learned my addr 172.21.15.83:0/2169778082 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:21.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.487+0000 7f163d400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f164006e950 0x7f164015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.487+0000 7f163ca00640 1 -- 172.21.15.83:0/2169778082 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2239656195 0 0) 0x7f16400ce000 con 0x7f1640159810 2024-04-25T00:14:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.487+0000 7f163ca00640 1 -- 172.21.15.83:0/2169778082 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1624003530 con 0x7f1640159810 2024-04-25T00:14:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.487+0000 7f163d400640 1 -- 172.21.15.83:0/2169778082 >> v1:172.21.15.86:6789/0 conn(0x7f164015fe50 legacy=0x7f16400ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.487+0000 7f163d400640 1 -- 172.21.15.83:0/2169778082 >> v1:172.21.15.83:6789/0 conn(0x7f1640159810 legacy=0x7f16400cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.487+0000 7f163d400640 1 -- 172.21.15.83:0/2169778082 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164006dc50 con 0x7f164006e950 2024-04-25T00:14:21.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.491+0000 7f163d400640 1 --2- 172.21.15.83:0/2169778082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164006e950 0x7f164015f910 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f162c0097c0 tx=0x7f162c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a46a8c840e629e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.491+0000 7f163ca00640 1 -- 172.21.15.83:0/2169778082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c0b0070 con 0x7f164006e950 2024-04-25T00:14:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.491+0000 7f163ca00640 1 -- 172.21.15.83:0/2169778082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f162c0ab600 con 0x7f164006e950 2024-04-25T00:14:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.491+0000 7f163ca00640 1 -- 172.21.15.83:0/2169778082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c0ab9d0 con 0x7f164006e950 2024-04-25T00:14:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.491+0000 7f1644e00640 1 -- 172.21.15.83:0/2169778082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164006e950 msgr2=0x7f164015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.491+0000 7f1644e00640 1 --2- 172.21.15.83:0/2169778082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164006e950 0x7f164015f910 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f162c0097c0 tx=0x7f162c01c410 comp rx=0 tx=0).stop 2024-04-25T00:14:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.491+0000 7f1644e00640 1 -- 172.21.15.83:0/2169778082 shutdown_connections 2024-04-25T00:14:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.491+0000 7f1644e00640 1 --2- 172.21.15.83:0/2169778082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164006e950 0x7f164015f910 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.491+0000 7f1644e00640 1 -- 172.21.15.83:0/2169778082 >> 172.21.15.83:0/2169778082 conn(0x7f16400b1830 msgr2=0x7f16400b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.491+0000 7f1644e00640 1 -- 172.21.15.83:0/2169778082 shutdown_connections 2024-04-25T00:14:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.491+0000 7f1644e00640 1 -- 172.21.15.83:0/2169778082 wait complete. 2024-04-25T00:14:21.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.491+0000 7f1644e00640 1 Processor -- start 2024-04-25T00:14:21.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.491+0000 7f1644e00640 1 -- start start 2024-04-25T00:14:21.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f1644e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164006e950 0x7f164014dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:21.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f1644e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640159810 0x7f164015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:21.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f1644e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f164015fe50 0x7f164015af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:21.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f1644e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1640169e40 con 0x7f164006e950 2024-04-25T00:14:21.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f1644e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1640169cc0 con 0x7f1640159810 2024-04-25T00:14:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f1644e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1640169fc0 con 0x7f164015fe50 2024-04-25T00:14:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f163de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164006e950 0x7f164014dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f163de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164006e950 0x7f164014dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55028/0 (socket says 172.21.15.83:55028) 2024-04-25T00:14:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f163de00640 1 -- 172.21.15.83:0/494180199 learned_addr learned my addr 172.21.15.83:0/494180199 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f163d400640 1 --2- 172.21.15.83:0/494180199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640159810 0x7f164015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f163e800640 1 --2- 172.21.15.83:0/494180199 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f164015fe50 0x7f164015af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f163de00640 1 -- 172.21.15.83:0/494180199 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f164015fe50 msgr2=0x7f164015af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f163de00640 1 --2- 172.21.15.83:0/494180199 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f164015fe50 0x7f164015af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f163de00640 1 -- 172.21.15.83:0/494180199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640159810 msgr2=0x7f164015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f163de00640 1 --2- 172.21.15.83:0/494180199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640159810 0x7f164015dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f163de00640 1 -- 172.21.15.83:0/494180199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164014e1d0 con 0x7f164006e950 2024-04-25T00:14:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f163de00640 1 --2- 172.21.15.83:0/494180199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164006e950 0x7f164014dc90 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f163000c450 tx=0x7f163000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f1636000640 1 -- 172.21.15.83:0/494180199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630017070 con 0x7f164006e950 2024-04-25T00:14:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f1636000640 1 -- 172.21.15.83:0/494180199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1630005cf0 con 0x7f164006e950 2024-04-25T00:14:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f1644e00640 1 -- 172.21.15.83:0/494180199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16400b0250 con 0x7f164006e950 2024-04-25T00:14:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f1636000640 1 -- 172.21.15.83:0/494180199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630007270 con 0x7f164006e950 2024-04-25T00:14:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f1644e00640 1 -- 172.21.15.83:0/494180199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16400b0ab0 con 0x7f164006e950 2024-04-25T00:14:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.495+0000 7f1644e00640 1 -- 172.21.15.83:0/494180199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16400b0600 con 0x7f164006e950 2024-04-25T00:14:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.499+0000 7f1636000640 1 -- 172.21.15.83:0/494180199 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f163001a070 con 0x7f164006e950 2024-04-25T00:14:21.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.499+0000 7f1636000640 1 --2- 172.21.15.83:0/494180199 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16200678c0 0x7f1620069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:21.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.499+0000 7f1636000640 1 -- 172.21.15.83:0/494180199 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5619+0+0 (secure 0 0 0) 0x7f16300852f0 con 0x7f164006e950 2024-04-25T00:14:21.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.499+0000 7f163d400640 1 --2- 172.21.15.83:0/494180199 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16200678c0 0x7f1620069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.499+0000 7f163d400640 1 --2- 172.21.15.83:0/494180199 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16200678c0 0x7f1620069d80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f162c0ad000 tx=0x7f162c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:21.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.503+0000 7f1636000640 1 -- 172.21.15.83:0/494180199 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1630021020 con 0x7f164006e950 2024-04-25T00:14:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:21.667+0000 7f1644e00640 1 -- 172.21.15.83:0/494180199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f164004f510 con 0x7f164006e950 2024-04-25T00:14:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.315+0000 7f1636000640 1 -- 172.21.15.83:0/494180199 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7f16300559d0 con 0x7f164006e950 2024-04-25T00:14:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-25T00:14:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.319+0000 7f1644e00640 1 -- 172.21.15.83:0/494180199 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16200678c0 msgr2=0x7f1620069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.319+0000 7f1644e00640 1 --2- 172.21.15.83:0/494180199 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16200678c0 0x7f1620069d80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f162c0ad000 tx=0x7f162c008040 comp rx=0 tx=0).stop 2024-04-25T00:14:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.319+0000 7f1644e00640 1 -- 172.21.15.83:0/494180199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164006e950 msgr2=0x7f164014dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.319+0000 7f1644e00640 1 --2- 172.21.15.83:0/494180199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164006e950 0x7f164014dc90 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f163000c450 tx=0x7f163000c910 comp rx=0 tx=0).stop 2024-04-25T00:14:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.319+0000 7f1644e00640 1 -- 172.21.15.83:0/494180199 shutdown_connections 2024-04-25T00:14:22.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.319+0000 7f1644e00640 1 --2- 172.21.15.83:0/494180199 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f16200678c0 0x7f1620069d80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:22.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.319+0000 7f1644e00640 1 --2- 172.21.15.83:0/494180199 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f164015fe50 0x7f164015af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:22.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.319+0000 7f1644e00640 1 --2- 172.21.15.83:0/494180199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640159810 0x7f164015dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:22.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.319+0000 7f1644e00640 1 --2- 172.21.15.83:0/494180199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164006e950 0x7f164014dc90 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:22.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.319+0000 7f1644e00640 1 -- 172.21.15.83:0/494180199 >> 172.21.15.83:0/494180199 conn(0x7f16400b1830 msgr2=0x7f16400ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:22.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.319+0000 7f1644e00640 1 -- 172.21.15.83:0/494180199 shutdown_connections 2024-04-25T00:14:22.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.319+0000 7f1644e00640 1 -- 172.21.15.83:0/494180199 wait complete. 2024-04-25T00:14:22.360 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:14:22.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.495+0000 7f1825800640 1 Processor -- start 2024-04-25T00:14:22.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.495+0000 7f1825800640 1 -- start start 2024-04-25T00:14:22.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.495+0000 7f1825800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18201551e0 0x7f182015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:22.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.495+0000 7f1825800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18201555c0 0x7f1820071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:22.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.495+0000 7f1825800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f182015c560 con 0x7f182006e4c0 2024-04-25T00:14:22.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.495+0000 7f1825800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1820072460 con 0x7f18201551e0 2024-04-25T00:14:22.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.495+0000 7f1825800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f182015c7d0 con 0x7f18201555c0 2024-04-25T00:14:22.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.495+0000 7f181ea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18201551e0 0x7f182015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:22.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.495+0000 7f181ea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18201551e0 0x7f182015bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55036/0 (socket says 172.21.15.83:55036) 2024-04-25T00:14:22.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.495+0000 7f181ea00640 1 -- 172.21.15.83:0/471639468 learned_addr learned my addr 172.21.15.83:0/471639468 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:22.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.495+0000 7f1816000640 1 --2- 172.21.15.83:0/471639468 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18201555c0 0x7f1820071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:22.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f181ea00640 1 -- 172.21.15.83:0/471639468 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18201555c0 msgr2=0x7f1820071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:22.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f181ea00640 1 --2- 172.21.15.83:0/471639468 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18201555c0 0x7f1820071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:22.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f181ea00640 1 -- 172.21.15.83:0/471639468 >> v1:172.21.15.83:6789/0 conn(0x7f182006e4c0 legacy=0x7f182015e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:22.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f181ea00640 1 -- 172.21.15.83:0/471639468 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f182015cab0 con 0x7f18201551e0 2024-04-25T00:14:22.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f181e000640 1 -- 172.21.15.83:0/471639468 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 474477466 0 0) 0x7f182015c560 con 0x7f182006e4c0 2024-04-25T00:14:22.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f181ea00640 1 --2- 172.21.15.83:0/471639468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18201551e0 0x7f182015bea0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1810009e50 tx=0x7f181001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cea26e8ad9ae6da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:22.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f181e000640 1 -- 172.21.15.83:0/471639468 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18100a5820 con 0x7f18201551e0 2024-04-25T00:14:22.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f181e000640 1 -- 172.21.15.83:0/471639468 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18100a59c0 con 0x7f18201551e0 2024-04-25T00:14:22.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f181e000640 1 -- 172.21.15.83:0/471639468 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18100a5d90 con 0x7f18201551e0 2024-04-25T00:14:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f1825800640 1 -- 172.21.15.83:0/471639468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18201551e0 msgr2=0x7f182015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f1825800640 1 --2- 172.21.15.83:0/471639468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18201551e0 0x7f182015bea0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1810009e50 tx=0x7f181001ca30 comp rx=0 tx=0).stop 2024-04-25T00:14:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f1825800640 1 -- 172.21.15.83:0/471639468 shutdown_connections 2024-04-25T00:14:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f1825800640 1 --2- 172.21.15.83:0/471639468 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18201555c0 0x7f1820071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f1825800640 1 --2- 172.21.15.83:0/471639468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18201551e0 0x7f182015bea0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f1825800640 1 -- 172.21.15.83:0/471639468 >> 172.21.15.83:0/471639468 conn(0x7f18200b1830 msgr2=0x7f18200b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f1825800640 1 -- 172.21.15.83:0/471639468 shutdown_connections 2024-04-25T00:14:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.499+0000 7f1825800640 1 -- 172.21.15.83:0/471639468 wait complete. 2024-04-25T00:14:22.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f1825800640 1 Processor -- start 2024-04-25T00:14:22.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f1825800640 1 -- start start 2024-04-25T00:14:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f1825800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f182006e4c0 0x7f1820153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f1825800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18201551e0 0x7f1820154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f1825800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18201555c0 0x7f182014adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f1825800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1820167e10 con 0x7f18201555c0 2024-04-25T00:14:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f1825800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1820167c90 con 0x7f182006e4c0 2024-04-25T00:14:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f1825800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1820167f90 con 0x7f18201551e0 2024-04-25T00:14:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f181f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18201555c0 0x7f182014adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f181f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18201555c0 0x7f182014adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55050/0 (socket says 172.21.15.83:55050) 2024-04-25T00:14:22.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f181f400640 1 -- 172.21.15.83:0/1383843040 learned_addr learned my addr 172.21.15.83:0/1383843040 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:22.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f181ea00640 1 --2- 172.21.15.83:0/1383843040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f182006e4c0 0x7f1820153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:22.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f1816000640 1 --2- 172.21.15.83:0/1383843040 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18201551e0 0x7f1820154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:22.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f181f400640 1 -- 172.21.15.83:0/1383843040 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18201551e0 msgr2=0x7f1820154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:22.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f181f400640 1 --2- 172.21.15.83:0/1383843040 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18201551e0 0x7f1820154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:22.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f181f400640 1 -- 172.21.15.83:0/1383843040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f182006e4c0 msgr2=0x7f1820153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:22.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f181f400640 1 --2- 172.21.15.83:0/1383843040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f182006e4c0 0x7f1820153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:22.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f181f400640 1 -- 172.21.15.83:0/1383843040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18200b2aa0 con 0x7f18201555c0 2024-04-25T00:14:22.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f181f400640 1 --2- 172.21.15.83:0/1383843040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18201555c0 0x7f182014adf0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f181800c480 tx=0x7f181800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f1817400640 1 -- 172.21.15.83:0/1383843040 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818005480 con 0x7f18201555c0 2024-04-25T00:14:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f1817400640 1 -- 172.21.15.83:0/1383843040 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1818003600 con 0x7f18201555c0 2024-04-25T00:14:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f1817400640 1 -- 172.21.15.83:0/1383843040 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18180039f0 con 0x7f18201555c0 2024-04-25T00:14:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.503+0000 7f1825800640 1 -- 172.21.15.83:0/1383843040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18200b2d80 con 0x7f18201555c0 2024-04-25T00:14:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.507+0000 7f1825800640 1 -- 172.21.15.83:0/1383843040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18200b3340 con 0x7f18201555c0 2024-04-25T00:14:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.507+0000 7f1825800640 1 -- 172.21.15.83:0/1383843040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f182004f510 con 0x7f18201555c0 2024-04-25T00:14:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.507+0000 7f1817400640 1 -- 172.21.15.83:0/1383843040 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1818003b90 con 0x7f18201555c0 2024-04-25T00:14:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.507+0000 7f1817400640 1 --2- 172.21.15.83:0/1383843040 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17fc0678c0 0x7f17fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.507+0000 7f1817400640 1 -- 172.21.15.83:0/1383843040 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6001+0+0 (secure 0 0 0) 0x7f1818085890 con 0x7f18201555c0 2024-04-25T00:14:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.507+0000 7f181ea00640 1 --2- 172.21.15.83:0/1383843040 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17fc0678c0 0x7f17fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.511+0000 7f181ea00640 1 --2- 172.21.15.83:0/1383843040 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17fc0678c0 0x7f17fc069d80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f18100ae000 tx=0x7f18100a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.515+0000 7f1817400640 1 -- 172.21.15.83:0/1383843040 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1818055ce0 con 0x7f18201555c0 2024-04-25T00:14:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:22.699+0000 7f1825800640 1 -- 172.21.15.83:0/1383843040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f18200b3510 con 0x7f18201555c0 2024-04-25T00:14:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T00:14:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.347+0000 7f1817400640 1 -- 172.21.15.83:0/1383843040 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v49) ==== 174+0+0 (secure 0 0 0) 0x7f1818059fe0 con 0x7f18201555c0 2024-04-25T00:14:23.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.355+0000 7f1825800640 1 -- 172.21.15.83:0/1383843040 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17fc0678c0 msgr2=0x7f17fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:23.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.355+0000 7f1825800640 1 --2- 172.21.15.83:0/1383843040 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17fc0678c0 0x7f17fc069d80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f18100ae000 tx=0x7f18100a7070 comp rx=0 tx=0).stop 2024-04-25T00:14:23.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.355+0000 7f1825800640 1 -- 172.21.15.83:0/1383843040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18201555c0 msgr2=0x7f182014adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:23.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.355+0000 7f1825800640 1 --2- 172.21.15.83:0/1383843040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18201555c0 0x7f182014adf0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f181800c480 tx=0x7f181800c940 comp rx=0 tx=0).stop 2024-04-25T00:14:23.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.359+0000 7f1825800640 1 -- 172.21.15.83:0/1383843040 shutdown_connections 2024-04-25T00:14:23.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.359+0000 7f1825800640 1 --2- 172.21.15.83:0/1383843040 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17fc0678c0 0x7f17fc069d80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.359+0000 7f1825800640 1 --2- 172.21.15.83:0/1383843040 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18201551e0 0x7f1820154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.359+0000 7f1825800640 1 --2- 172.21.15.83:0/1383843040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18201555c0 0x7f182014adf0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.359+0000 7f1825800640 1 --2- 172.21.15.83:0/1383843040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f182006e4c0 0x7f1820153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.359+0000 7f1825800640 1 -- 172.21.15.83:0/1383843040 >> 172.21.15.83:0/1383843040 conn(0x7f18200b1830 msgr2=0x7f18200cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:23.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.359+0000 7f1825800640 1 -- 172.21.15.83:0/1383843040 shutdown_connections 2024-04-25T00:14:23.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.359+0000 7f1825800640 1 -- 172.21.15.83:0/1383843040 wait complete. 2024-04-25T00:14:23.398 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6e0200640 1 Processor -- start 2024-04-25T00:14:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6e0200640 1 -- start start 2024-04-25T00:14:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6e0200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6d80cf420 0x7ff6d8071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6e0200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff6d81635e0 0x7ff6d806cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6e0200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6d8072750 con 0x7ff6d806d060 2024-04-25T00:14:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6e0200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff6d8071b40 con 0x7ff6d81635e0 2024-04-25T00:14:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6e0200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d8071e20 con 0x7ff6d80cf420 2024-04-25T00:14:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6de200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff6d806d060 0x7ff6d8165fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38326/0 (socket says 172.21.15.83:38326) 2024-04-25T00:14:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6de200640 1 -- 172.21.15.83:0/3048019091 learned_addr learned my addr 172.21.15.83:0/3048019091 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6dd800640 1 --2- 172.21.15.83:0/3048019091 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6d80cf420 0x7ff6d8071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6dce00640 1 --2- 172.21.15.83:0/3048019091 >> v2:172.21.15.86:3300/0 conn(0x7ff6d81635e0 0x7ff6d806cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6d7e00640 1 -- 172.21.15.83:0/3048019091 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1186480567 0 0) 0x7ff6d8072750 con 0x7ff6d806d060 2024-04-25T00:14:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6d7e00640 1 -- 172.21.15.83:0/3048019091 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6c0003530 con 0x7ff6d806d060 2024-04-25T00:14:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6dd800640 1 -- 172.21.15.83:0/3048019091 >> v2:172.21.15.86:3300/0 conn(0x7ff6d81635e0 msgr2=0x7ff6d806cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6dd800640 1 --2- 172.21.15.83:0/3048019091 >> v2:172.21.15.86:3300/0 conn(0x7ff6d81635e0 0x7ff6d806cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6dd800640 1 -- 172.21.15.83:0/3048019091 >> v1:172.21.15.83:6789/0 conn(0x7ff6d806d060 legacy=0x7ff6d8165fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6dd800640 1 -- 172.21.15.83:0/3048019091 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d8072100 con 0x7ff6d80cf420 2024-04-25T00:14:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.531+0000 7ff6dce00640 1 --2- 172.21.15.83:0/3048019091 >> v2:172.21.15.86:3300/0 conn(0x7ff6d81635e0 0x7ff6d806cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:23.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.535+0000 7ff6dd800640 1 --2- 172.21.15.83:0/3048019091 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6d80cf420 0x7ff6d8071240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0097c0 tx=0x7ff6cc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2b8333b8b7999ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:23.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.535+0000 7ff6d7e00640 1 -- 172.21.15.83:0/3048019091 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc0b0070 con 0x7ff6d80cf420 2024-04-25T00:14:23.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.535+0000 7ff6d7e00640 1 -- 172.21.15.83:0/3048019091 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6cc0ab600 con 0x7ff6d80cf420 2024-04-25T00:14:23.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.535+0000 7ff6d7e00640 1 -- 172.21.15.83:0/3048019091 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc0ab9d0 con 0x7ff6d80cf420 2024-04-25T00:14:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.535+0000 7ff6e0200640 1 -- 172.21.15.83:0/3048019091 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6d80cf420 msgr2=0x7ff6d8071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.535+0000 7ff6e0200640 1 --2- 172.21.15.83:0/3048019091 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6d80cf420 0x7ff6d8071240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0097c0 tx=0x7ff6cc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.535+0000 7ff6e0200640 1 -- 172.21.15.83:0/3048019091 shutdown_connections 2024-04-25T00:14:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.535+0000 7ff6e0200640 1 --2- 172.21.15.83:0/3048019091 >> v2:172.21.15.86:3300/0 conn(0x7ff6d81635e0 0x7ff6d806cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.535+0000 7ff6e0200640 1 --2- 172.21.15.83:0/3048019091 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6d80cf420 0x7ff6d8071240 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.535+0000 7ff6e0200640 1 -- 172.21.15.83:0/3048019091 >> 172.21.15.83:0/3048019091 conn(0x7ff6d80b1830 msgr2=0x7ff6d80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.535+0000 7ff6e0200640 1 -- 172.21.15.83:0/3048019091 shutdown_connections 2024-04-25T00:14:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.535+0000 7ff6e0200640 1 -- 172.21.15.83:0/3048019091 wait complete. 2024-04-25T00:14:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6e0200640 1 Processor -- start 2024-04-25T00:14:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6e0200640 1 -- start start 2024-04-25T00:14:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6e0200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d806d060 0x7ff6d814b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6e0200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6d80cf420 0x7ff6d81538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6e0200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6d81635e0 0x7ff6d8164b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6e0200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d816f4e0 con 0x7ff6d80cf420 2024-04-25T00:14:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6e0200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d816f360 con 0x7ff6d806d060 2024-04-25T00:14:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6e0200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d816f660 con 0x7ff6d81635e0 2024-04-25T00:14:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6dce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6d80cf420 0x7ff6d81538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6dce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6d80cf420 0x7ff6d81538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55060/0 (socket says 172.21.15.83:55060) 2024-04-25T00:14:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6dce00640 1 -- 172.21.15.83:0/2147502456 learned_addr learned my addr 172.21.15.83:0/2147502456 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6dd800640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d806d060 0x7ff6d814b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6de200640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6d81635e0 0x7ff6d8164b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6dce00640 1 -- 172.21.15.83:0/2147502456 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6d81635e0 msgr2=0x7ff6d8164b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6dce00640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6d81635e0 0x7ff6d8164b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6dce00640 1 -- 172.21.15.83:0/2147502456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d806d060 msgr2=0x7ff6d814b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6dce00640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d806d060 0x7ff6d814b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6dce00640 1 -- 172.21.15.83:0/2147502456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d8072f10 con 0x7ff6d80cf420 2024-04-25T00:14:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6dd800640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d806d060 0x7ff6d814b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6dce00640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6d80cf420 0x7ff6d81538f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400ebe0 tx=0x7ff6c400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6d5600640 1 -- 172.21.15.83:0/2147502456 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4017070 con 0x7ff6d80cf420 2024-04-25T00:14:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6d5600640 1 -- 172.21.15.83:0/2147502456 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6c400cc30 con 0x7ff6d80cf420 2024-04-25T00:14:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6d5600640 1 -- 172.21.15.83:0/2147502456 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4013470 con 0x7ff6d80cf420 2024-04-25T00:14:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6e0200640 1 -- 172.21.15.83:0/2147502456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d81542d0 con 0x7ff6d80cf420 2024-04-25T00:14:23.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6e0200640 1 -- 172.21.15.83:0/2147502456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d8154890 con 0x7ff6d80cf420 2024-04-25T00:14:23.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.539+0000 7ff6e0200640 1 -- 172.21.15.83:0/2147502456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d80737f0 con 0x7ff6d80cf420 2024-04-25T00:14:23.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.543+0000 7ff6d5600640 1 -- 172.21.15.83:0/2147502456 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff6c401a070 con 0x7ff6d80cf420 2024-04-25T00:14:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.543+0000 7ff6d5600640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff6b80678c0 0x7ff6b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.543+0000 7ff6d5600640 1 -- 172.21.15.83:0/2147502456 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7ff6c4085dd0 con 0x7ff6d80cf420 2024-04-25T00:14:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.543+0000 7ff6dd800640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff6b80678c0 0x7ff6b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.543+0000 7ff6dd800640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff6b80678c0 0x7ff6b8069d80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0ad000 tx=0x7ff6cc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.547+0000 7ff6d5600640 1 -- 172.21.15.83:0/2147502456 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff6c408a070 con 0x7ff6d80cf420 2024-04-25T00:14:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.703+0000 7ff6e0200640 1 -- 172.21.15.83:0/2147502456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6d816d0c0 con 0x7ff6d80cf420 2024-04-25T00:14:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.703+0000 7ff6d5600640 1 -- 172.21.15.83:0/2147502456 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16814 (secure 0 0 0) 0x7ff6c4056090 con 0x7ff6d80cf420 2024-04-25T00:14:23.708 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:23.709 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:22.706409+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:21.082598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:21.713421+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:23.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.707+0000 7ff6e0200640 1 -- 172.21.15.83:0/2147502456 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff6b80678c0 msgr2=0x7ff6b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:23.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.707+0000 7ff6e0200640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff6b80678c0 0x7ff6b8069d80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0ad000 tx=0x7ff6cc0a6000 comp rx=0 tx=0).stop 2024-04-25T00:14:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.707+0000 7ff6e0200640 1 -- 172.21.15.83:0/2147502456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6d80cf420 msgr2=0x7ff6d81538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.707+0000 7ff6e0200640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6d80cf420 0x7ff6d81538f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400ebe0 tx=0x7ff6c400c520 comp rx=0 tx=0).stop 2024-04-25T00:14:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.707+0000 7ff6e0200640 1 -- 172.21.15.83:0/2147502456 shutdown_connections 2024-04-25T00:14:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.707+0000 7ff6e0200640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff6b80678c0 0x7ff6b8069d80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.707+0000 7ff6e0200640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6d81635e0 0x7ff6d8164b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.707+0000 7ff6e0200640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6d80cf420 0x7ff6d81538f0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.707+0000 7ff6e0200640 1 --2- 172.21.15.83:0/2147502456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d806d060 0x7ff6d814b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.707+0000 7ff6e0200640 1 -- 172.21.15.83:0/2147502456 >> 172.21.15.83:0/2147502456 conn(0x7ff6d80b1830 msgr2=0x7ff6d8159090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.707+0000 7ff6e0200640 1 -- 172.21.15.83:0/2147502456 shutdown_connections 2024-04-25T00:14:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.707+0000 7ff6e0200640 1 -- 172.21.15.83:0/2147502456 wait complete. 2024-04-25T00:14:23.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.859+0000 7f38ea400640 1 Processor -- start 2024-04-25T00:14:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.859+0000 7f38ea400640 1 -- start start 2024-04-25T00:14:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.863+0000 7f38ea400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38e41553e0 0x7f38e406e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.863+0000 7f38ea400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f38e406ea00 0x7f38e40b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.863+0000 7f38ea400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38e40b3b40 con 0x7f38e40b48a0 2024-04-25T00:14:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.863+0000 7f38ea400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f38e406ef80 con 0x7f38e406ea00 2024-04-25T00:14:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.863+0000 7f38ea400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f38e415b510 con 0x7f38e41553e0 2024-04-25T00:14:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.863+0000 7f38daa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f38e406ea00 0x7f38e40b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.863+0000 7f38e3400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38e41553e0 0x7f38e406e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.863+0000 7f38e3400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38e41553e0 0x7f38e406e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:44186/0 (socket says 172.21.15.83:44186) 2024-04-25T00:14:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.863+0000 7f38e3400640 1 -- 172.21.15.83:0/1637053525 learned_addr learned my addr 172.21.15.83:0/1637053525 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.863+0000 7f38daa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f38e406ea00 0x7f38e40b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:43492/0 (socket says 172.21.15.83:43492) 2024-04-25T00:14:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.863+0000 7f38e3400640 1 -- 172.21.15.83:0/1637053525 >> v2:172.21.15.86:3300/0 conn(0x7f38e406ea00 msgr2=0x7f38e40b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.863+0000 7f38e3400640 1 --2- 172.21.15.83:0/1637053525 >> v2:172.21.15.86:3300/0 conn(0x7f38e406ea00 0x7f38e40b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.863+0000 7f38e3400640 1 -- 172.21.15.83:0/1637053525 >> v1:172.21.15.86:6789/0 conn(0x7f38e40b48a0 legacy=0x7f38e415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.863+0000 7f38e3400640 1 -- 172.21.15.83:0/1637053525 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e415b7f0 con 0x7f38e41553e0 2024-04-25T00:14:23.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.867+0000 7f38e3400640 1 --2- 172.21.15.83:0/1637053525 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38e41553e0 0x7f38e406e4c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f38d400d150 tx=0x7f38d401eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8d4171155201edf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:23.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.867+0000 7f38e2a00640 1 -- 172.21.15.83:0/1637053525 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d400b2d0 con 0x7f38e41553e0 2024-04-25T00:14:23.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.867+0000 7f38e2a00640 1 -- 172.21.15.83:0/1637053525 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38d400b470 con 0x7f38e41553e0 2024-04-25T00:14:23.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.867+0000 7f38e2a00640 1 -- 172.21.15.83:0/1637053525 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d40a79b0 con 0x7f38e41553e0 2024-04-25T00:14:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.867+0000 7f38ea400640 1 -- 172.21.15.83:0/1637053525 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38e41553e0 msgr2=0x7f38e406e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.867+0000 7f38ea400640 1 --2- 172.21.15.83:0/1637053525 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38e41553e0 0x7f38e406e4c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f38d400d150 tx=0x7f38d401eba0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.867+0000 7f38ea400640 1 -- 172.21.15.83:0/1637053525 shutdown_connections 2024-04-25T00:14:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.867+0000 7f38ea400640 1 --2- 172.21.15.83:0/1637053525 >> v2:172.21.15.86:3300/0 conn(0x7f38e406ea00 0x7f38e40b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.867+0000 7f38ea400640 1 --2- 172.21.15.83:0/1637053525 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38e41553e0 0x7f38e406e4c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.867+0000 7f38ea400640 1 -- 172.21.15.83:0/1637053525 >> 172.21.15.83:0/1637053525 conn(0x7f38e40b1830 msgr2=0x7f38e40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.867+0000 7f38ea400640 1 -- 172.21.15.83:0/1637053525 shutdown_connections 2024-04-25T00:14:23.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.867+0000 7f38ea400640 1 -- 172.21.15.83:0/1637053525 wait complete. 2024-04-25T00:14:23.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.867+0000 7f38ea400640 1 Processor -- start 2024-04-25T00:14:23.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.867+0000 7f38ea400640 1 -- start start 2024-04-25T00:14:23.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38ea400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e406ea00 0x7f38e4157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:23.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38ea400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38e40b48a0 0x7f38e41435f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:23.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38ea400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e41553e0 0x7f38e41502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:23.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38ea400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38e4155940 con 0x7f38e41553e0 2024-04-25T00:14:23.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38ea400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38e41557c0 con 0x7f38e406ea00 2024-04-25T00:14:23.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38ea400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f38e406d3b0 con 0x7f38e40b48a0 2024-04-25T00:14:23.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e41553e0 0x7f38e41502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:23.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e41553e0 0x7f38e41502c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55066/0 (socket says 172.21.15.83:55066) 2024-04-25T00:14:23.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38e3e00640 1 -- 172.21.15.83:0/4120812025 learned_addr learned my addr 172.21.15.83:0/4120812025 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:23.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38daa00640 1 --2- 172.21.15.83:0/4120812025 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38e40b48a0 0x7f38e41435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:23.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38e3400640 1 --2- 172.21.15.83:0/4120812025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e406ea00 0x7f38e4157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:23.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38e3e00640 1 -- 172.21.15.83:0/4120812025 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38e40b48a0 msgr2=0x7f38e41435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:23.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38e3e00640 1 --2- 172.21.15.83:0/4120812025 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38e40b48a0 0x7f38e41435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38e3e00640 1 -- 172.21.15.83:0/4120812025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e406ea00 msgr2=0x7f38e4157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:23.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38e3e00640 1 --2- 172.21.15.83:0/4120812025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e406ea00 0x7f38e4157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38e3e00640 1 -- 172.21.15.83:0/4120812025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e4158100 con 0x7f38e41553e0 2024-04-25T00:14:23.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38e3e00640 1 --2- 172.21.15.83:0/4120812025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e41553e0 0x7f38e41502c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00c460 tx=0x7f38dc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:23.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38dbe00640 1 -- 172.21.15.83:0/4120812025 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc016020 con 0x7f38e41553e0 2024-04-25T00:14:23.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38dbe00640 1 -- 172.21.15.83:0/4120812025 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38dc005220 con 0x7f38e41553e0 2024-04-25T00:14:23.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38ea400640 1 -- 172.21.15.83:0/4120812025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e41583e0 con 0x7f38e41553e0 2024-04-25T00:14:23.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38dbe00640 1 -- 172.21.15.83:0/4120812025 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc007030 con 0x7f38e41553e0 2024-04-25T00:14:23.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38ea400640 1 -- 172.21.15.83:0/4120812025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e4143b30 con 0x7f38e41553e0 2024-04-25T00:14:23.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38ea400640 1 -- 172.21.15.83:0/4120812025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e4158900 con 0x7f38e41553e0 2024-04-25T00:14:23.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.871+0000 7f38dbe00640 1 -- 172.21.15.83:0/4120812025 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f38dc0071d0 con 0x7f38e41553e0 2024-04-25T00:14:23.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.875+0000 7f38dbe00640 1 --2- 172.21.15.83:0/4120812025 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38c00678c0 0x7f38c0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:23.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.875+0000 7f38dbe00640 1 -- 172.21.15.83:0/4120812025 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f38dc085650 con 0x7f38e41553e0 2024-04-25T00:14:23.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.875+0000 7f38e3400640 1 --2- 172.21.15.83:0/4120812025 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38c00678c0 0x7f38c0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:23.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.875+0000 7f38e3400640 1 --2- 172.21.15.83:0/4120812025 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38c00678c0 0x7f38c0069d80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f38d40b0000 tx=0x7f38d40a9070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:23.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:23.879+0000 7f38dbe00640 1 -- 172.21.15.83:0/4120812025 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f38e4158900 con 0x7f38e41553e0 2024-04-25T00:14:24.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.035+0000 7f38ea400640 1 -- 172.21.15.83:0/4120812025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38e40737f0 con 0x7f38e41553e0 2024-04-25T00:14:24.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.035+0000 7f38dbe00640 1 -- 172.21.15.83:0/4120812025 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16814 (secure 0 0 0) 0x7f38dc020020 con 0x7f38e41553e0 2024-04-25T00:14:24.040 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:24.040 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:22.706409+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:21.082598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:21.713421+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.039+0000 7f38ea400640 1 -- 172.21.15.83:0/4120812025 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38c00678c0 msgr2=0x7f38c0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.039+0000 7f38ea400640 1 --2- 172.21.15.83:0/4120812025 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38c00678c0 0x7f38c0069d80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f38d40b0000 tx=0x7f38d40a9070 comp rx=0 tx=0).stop 2024-04-25T00:14:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.039+0000 7f38ea400640 1 -- 172.21.15.83:0/4120812025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e41553e0 msgr2=0x7f38e41502c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.039+0000 7f38ea400640 1 --2- 172.21.15.83:0/4120812025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e41553e0 0x7f38e41502c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00c460 tx=0x7f38dc00c920 comp rx=0 tx=0).stop 2024-04-25T00:14:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.039+0000 7f38ea400640 1 -- 172.21.15.83:0/4120812025 shutdown_connections 2024-04-25T00:14:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.039+0000 7f38ea400640 1 --2- 172.21.15.83:0/4120812025 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f38c00678c0 0x7f38c0069d80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.039+0000 7f38ea400640 1 --2- 172.21.15.83:0/4120812025 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38e40b48a0 0x7f38e41435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.039+0000 7f38ea400640 1 --2- 172.21.15.83:0/4120812025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e41553e0 0x7f38e41502c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.039+0000 7f38ea400640 1 --2- 172.21.15.83:0/4120812025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e406ea00 0x7f38e4157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.039+0000 7f38ea400640 1 -- 172.21.15.83:0/4120812025 >> 172.21.15.83:0/4120812025 conn(0x7f38e40b1830 msgr2=0x7f38e40b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.039+0000 7f38ea400640 1 -- 172.21.15.83:0/4120812025 shutdown_connections 2024-04-25T00:14:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.039+0000 7f38ea400640 1 -- 172.21.15.83:0/4120812025 wait complete. 2024-04-25T00:14:24.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:14:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.195+0000 7feaa9000640 1 Processor -- start 2024-04-25T00:14:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.195+0000 7feaa9000640 1 -- start start 2024-04-25T00:14:24.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.195+0000 7feaa9000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feaa415a710 0x7feaa40b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:24.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.195+0000 7feaa9000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaa4071850 con 0x7feaa406ddc0 2024-04-25T00:14:24.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.195+0000 7feaa9000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaa40b41b0 con 0x7feaa415a330 2024-04-25T00:14:24.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.195+0000 7feaa9000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7feaa40cc690 con 0x7feaa415a710 2024-04-25T00:14:24.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.195+0000 7fea99400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feaa415a710 0x7feaa40b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:24.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.195+0000 7fea99400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feaa415a710 0x7feaa40b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55074/0 (socket says 172.21.15.83:55074) 2024-04-25T00:14:24.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.195+0000 7fea99400640 1 -- 172.21.15.83:0/1264976062 learned_addr learned my addr 172.21.15.83:0/1264976062 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:24.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.195+0000 7feaa2800640 1 --1- 172.21.15.83:0/1264976062 >> v1:172.21.15.83:6789/0 conn(0x7feaa406ddc0 0x7feaa406e210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38338/0 (socket says 172.21.15.83:38338) 2024-04-25T00:14:24.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.199+0000 7fea99400640 1 -- 172.21.15.83:0/1264976062 >> v1:172.21.15.86:6789/0 conn(0x7feaa415a330 legacy=0x7feaa4070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.199+0000 7fea99400640 1 -- 172.21.15.83:0/1264976062 >> v1:172.21.15.83:6789/0 conn(0x7feaa406ddc0 legacy=0x7feaa406e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.199+0000 7fea99400640 1 -- 172.21.15.83:0/1264976062 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa4071ac0 con 0x7feaa415a710 2024-04-25T00:14:24.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.199+0000 7fea99400640 1 --2- 172.21.15.83:0/1264976062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa415a710 0x7feaa40b4a40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00aa40 tx=0x7fea8c01d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9227d848a35d07b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:24.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.199+0000 7feaa1400640 1 -- 172.21.15.83:0/1264976062 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0b3070 con 0x7feaa415a710 2024-04-25T00:14:24.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.199+0000 7feaa1400640 1 -- 172.21.15.83:0/1264976062 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea8c0a6770 con 0x7feaa415a710 2024-04-25T00:14:24.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.199+0000 7feaa1400640 1 -- 172.21.15.83:0/1264976062 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00b040 con 0x7feaa415a710 2024-04-25T00:14:24.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.199+0000 7feaa9000640 1 -- 172.21.15.83:0/1264976062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa415a710 msgr2=0x7feaa40b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.199+0000 7feaa9000640 1 --2- 172.21.15.83:0/1264976062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa415a710 0x7feaa40b4a40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00aa40 tx=0x7fea8c01d910 comp rx=0 tx=0).stop 2024-04-25T00:14:24.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.199+0000 7feaa9000640 1 -- 172.21.15.83:0/1264976062 shutdown_connections 2024-04-25T00:14:24.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.199+0000 7feaa9000640 1 --2- 172.21.15.83:0/1264976062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa415a710 0x7feaa40b4a40 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.199+0000 7feaa9000640 1 -- 172.21.15.83:0/1264976062 >> 172.21.15.83:0/1264976062 conn(0x7feaa40b1830 msgr2=0x7feaa40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:24.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.199+0000 7feaa9000640 1 -- 172.21.15.83:0/1264976062 shutdown_connections 2024-04-25T00:14:24.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.199+0000 7feaa9000640 1 -- 172.21.15.83:0/1264976062 wait complete. 2024-04-25T00:14:24.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7feaa9000640 1 Processor -- start 2024-04-25T00:14:24.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7feaa9000640 1 -- start start 2024-04-25T00:14:24.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7feaa9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa406ddc0 0x7feaa4139400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:24.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7feaa9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa415a330 0x7feaa414e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:24.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7feaa9000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feaa414eec0 0x7feaa4158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:24.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7feaa9000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4167e40 con 0x7feaa415a330 2024-04-25T00:14:24.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7feaa9000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4167fc0 con 0x7feaa406ddc0 2024-04-25T00:14:24.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7feaa9000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7feaa4168140 con 0x7feaa414eec0 2024-04-25T00:14:24.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7fea99400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa415a330 0x7feaa414e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:24.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7fea99400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa415a330 0x7feaa414e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55084/0 (socket says 172.21.15.83:55084) 2024-04-25T00:14:24.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7fea99400640 1 -- 172.21.15.83:0/206301097 learned_addr learned my addr 172.21.15.83:0/206301097 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:24.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7feaa1e00640 1 --2- 172.21.15.83:0/206301097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa406ddc0 0x7feaa4139400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:24.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7feaa2800640 1 --2- 172.21.15.83:0/206301097 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feaa414eec0 0x7feaa4158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:24.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7fea99400640 1 -- 172.21.15.83:0/206301097 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feaa414eec0 msgr2=0x7feaa4158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7fea99400640 1 --2- 172.21.15.83:0/206301097 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feaa414eec0 0x7feaa4158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7fea99400640 1 -- 172.21.15.83:0/206301097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa406ddc0 msgr2=0x7feaa4139400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7fea99400640 1 --2- 172.21.15.83:0/206301097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa406ddc0 0x7feaa4139400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7fea99400640 1 -- 172.21.15.83:0/206301097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa41591b0 con 0x7feaa415a330 2024-04-25T00:14:24.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7fea99400640 1 --2- 172.21.15.83:0/206301097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa415a330 0x7feaa414e980 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00aa40 tx=0x7fea8c01c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:24.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7fea9aa00640 1 -- 172.21.15.83:0/206301097 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0be070 con 0x7feaa415a330 2024-04-25T00:14:24.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7fea9aa00640 1 -- 172.21.15.83:0/206301097 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea8c0b5050 con 0x7feaa415a330 2024-04-25T00:14:24.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7fea9aa00640 1 -- 172.21.15.83:0/206301097 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0b3030 con 0x7feaa415a330 2024-04-25T00:14:24.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.203+0000 7feaa9000640 1 -- 172.21.15.83:0/206301097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa4159490 con 0x7feaa415a330 2024-04-25T00:14:24.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.207+0000 7feaa9000640 1 -- 172.21.15.83:0/206301097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa415bce0 con 0x7feaa415a330 2024-04-25T00:14:24.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.207+0000 7feaa9000640 1 -- 172.21.15.83:0/206301097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa414f2d0 con 0x7feaa415a330 2024-04-25T00:14:24.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.207+0000 7fea9aa00640 1 -- 172.21.15.83:0/206301097 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fea8c0aea80 con 0x7feaa415a330 2024-04-25T00:14:24.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.207+0000 7fea9aa00640 1 --2- 172.21.15.83:0/206301097 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fea800678c0 0x7fea80069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:24.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.207+0000 7fea9aa00640 1 -- 172.21.15.83:0/206301097 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fea8c11f7f0 con 0x7feaa415a330 2024-04-25T00:14:24.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.207+0000 7feaa1e00640 1 --2- 172.21.15.83:0/206301097 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fea800678c0 0x7fea80069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:24.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.211+0000 7feaa1e00640 1 --2- 172.21.15.83:0/206301097 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fea800678c0 0x7fea80069d80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7feaa41fb020 tx=0x7fea94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:24.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.211+0000 7fea9aa00640 1 -- 172.21.15.83:0/206301097 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7feaa414f2d0 con 0x7feaa415a330 2024-04-25T00:14:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:24.407+0000 7feaa9000640 1 -- 172.21.15.83:0/206301097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7feaa404f510 con 0x7feaa415a330 2024-04-25T00:14:25.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.359+0000 7fea9aa00640 1 -- 172.21.15.83:0/206301097 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) ==== 118+0+0 (secure 0 0 0) 0x7fea8c017500 con 0x7feaa415a330 2024-04-25T00:14:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.363+0000 7feaa9000640 1 -- 172.21.15.83:0/206301097 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fea800678c0 msgr2=0x7fea80069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.363+0000 7feaa9000640 1 --2- 172.21.15.83:0/206301097 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fea800678c0 0x7fea80069d80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7feaa41fb020 tx=0x7fea94009040 comp rx=0 tx=0).stop 2024-04-25T00:14:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.363+0000 7feaa9000640 1 -- 172.21.15.83:0/206301097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa415a330 msgr2=0x7feaa414e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.363+0000 7feaa9000640 1 --2- 172.21.15.83:0/206301097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa415a330 0x7feaa414e980 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00aa40 tx=0x7fea8c01c150 comp rx=0 tx=0).stop 2024-04-25T00:14:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.363+0000 7feaa9000640 1 -- 172.21.15.83:0/206301097 shutdown_connections 2024-04-25T00:14:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.363+0000 7feaa9000640 1 --2- 172.21.15.83:0/206301097 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fea800678c0 0x7fea80069d80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.363+0000 7feaa9000640 1 --2- 172.21.15.83:0/206301097 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feaa414eec0 0x7feaa4158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.363+0000 7feaa9000640 1 --2- 172.21.15.83:0/206301097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaa415a330 0x7feaa414e980 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.363+0000 7feaa9000640 1 --2- 172.21.15.83:0/206301097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa406ddc0 0x7feaa4139400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.363+0000 7feaa9000640 1 -- 172.21.15.83:0/206301097 >> 172.21.15.83:0/206301097 conn(0x7feaa40b1830 msgr2=0x7feaa406e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.363+0000 7feaa9000640 1 -- 172.21.15.83:0/206301097 shutdown_connections 2024-04-25T00:14:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.363+0000 7feaa9000640 1 -- 172.21.15.83:0/206301097 wait complete. 2024-04-25T00:14:25.388 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:14:25.388 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:14:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.523+0000 7fac9ae00640 1 Processor -- start 2024-04-25T00:14:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.523+0000 7fac9ae00640 1 -- start start 2024-04-25T00:14:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.523+0000 7fac9ae00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac94159810 0x7fac940cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.523+0000 7fac9ae00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9406e950 0x7fac9415f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac9ae00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac940ce000 con 0x7fac9415fe50 2024-04-25T00:14:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac9ae00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fac9406e210 con 0x7fac94159810 2024-04-25T00:14:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac9ae00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fac9406da00 con 0x7fac9406e950 2024-04-25T00:14:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac93e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac94159810 0x7fac940cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac98e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fac9415fe50 0x7fac9415c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41052/0 (socket says 172.21.15.83:41052) 2024-04-25T00:14:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac98e00640 1 -- 172.21.15.83:0/1591839733 learned_addr learned my addr 172.21.15.83:0/1591839733 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac93e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac94159810 0x7fac940cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40802/0 (socket says 172.21.15.83:40802) 2024-04-25T00:14:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac8b400640 1 --2- 172.21.15.83:0/1591839733 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9406e950 0x7fac9415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac93e00640 1 -- 172.21.15.83:0/1591839733 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9406e950 msgr2=0x7fac9415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac93e00640 1 --2- 172.21.15.83:0/1591839733 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9406e950 0x7fac9415f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac93e00640 1 -- 172.21.15.83:0/1591839733 >> v1:172.21.15.83:6789/0 conn(0x7fac9415fe50 legacy=0x7fac9415c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac93e00640 1 -- 172.21.15.83:0/1591839733 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9406eec0 con 0x7fac94159810 2024-04-25T00:14:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac93400640 1 -- 172.21.15.83:0/1591839733 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 316371141 0 0) 0x7fac940ce000 con 0x7fac9415fe50 2024-04-25T00:14:25.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac93e00640 1 --2- 172.21.15.83:0/1591839733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac94159810 0x7fac940cc4a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fac84009d90 tx=0x7fac8401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c805122e6aff7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:25.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac93400640 1 -- 172.21.15.83:0/1591839733 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac840b2070 con 0x7fac94159810 2024-04-25T00:14:25.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac93400640 1 -- 172.21.15.83:0/1591839733 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac840a58b0 con 0x7fac94159810 2024-04-25T00:14:25.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.527+0000 7fac93400640 1 -- 172.21.15.83:0/1591839733 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac840a5c80 con 0x7fac94159810 2024-04-25T00:14:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 -- 172.21.15.83:0/1591839733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac94159810 msgr2=0x7fac940cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 --2- 172.21.15.83:0/1591839733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac94159810 0x7fac940cc4a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fac84009d90 tx=0x7fac8401c8e0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 -- 172.21.15.83:0/1591839733 shutdown_connections 2024-04-25T00:14:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 --2- 172.21.15.83:0/1591839733 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9406e950 0x7fac9415f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 --2- 172.21.15.83:0/1591839733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac94159810 0x7fac940cc4a0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 -- 172.21.15.83:0/1591839733 >> 172.21.15.83:0/1591839733 conn(0x7fac940b1830 msgr2=0x7fac940b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 -- 172.21.15.83:0/1591839733 shutdown_connections 2024-04-25T00:14:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 -- 172.21.15.83:0/1591839733 wait complete. 2024-04-25T00:14:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 Processor -- start 2024-04-25T00:14:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 -- start start 2024-04-25T00:14:25.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9406e950 0x7fac941510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:25.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac94159810 0x7fac940b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac9415fe50 0x7fac9415a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fac94159d70 con 0x7fac94159810 2024-04-25T00:14:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac94159bf0 con 0x7fac9415fe50 2024-04-25T00:14:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac9ae00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fac94070da0 con 0x7fac9406e950 2024-04-25T00:14:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac8b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac94159810 0x7fac940b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac8b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac94159810 0x7fac940b28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40808/0 (socket says 172.21.15.83:40808) 2024-04-25T00:14:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac8b400640 1 -- 172.21.15.83:0/4812163 learned_addr learned my addr 172.21.15.83:0/4812163 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac8b400640 1 -- 172.21.15.83:0/4812163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9406e950 msgr2=0x7fac941510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac8b400640 1 --2- 172.21.15.83:0/4812163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9406e950 0x7fac941510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac8b400640 1 -- 172.21.15.83:0/4812163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac9415fe50 msgr2=0x7fac9415a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac8b400640 1 --2- 172.21.15.83:0/4812163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac9415fe50 0x7fac9415a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.531+0000 7fac8b400640 1 -- 172.21.15.83:0/4812163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac94157410 con 0x7fac94159810 2024-04-25T00:14:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.535+0000 7fac8b400640 1 --2- 172.21.15.83:0/4812163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac94159810 0x7fac940b28e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00e9b0 tx=0x7fac7c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.535+0000 7fac90c00640 1 -- 172.21.15.83:0/4812163 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c017070 con 0x7fac94159810 2024-04-25T00:14:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.535+0000 7fac90c00640 1 -- 172.21.15.83:0/4812163 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7c00c9a0 con 0x7fac94159810 2024-04-25T00:14:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.535+0000 7fac9ae00640 1 -- 172.21.15.83:0/4812163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac940b1200 con 0x7fac94159810 2024-04-25T00:14:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.535+0000 7fac90c00640 1 -- 172.21.15.83:0/4812163 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c00cd70 con 0x7fac94159810 2024-04-25T00:14:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.535+0000 7fac9ae00640 1 -- 172.21.15.83:0/4812163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9415b330 con 0x7fac94159810 2024-04-25T00:14:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.535+0000 7fac9ae00640 1 -- 172.21.15.83:0/4812163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac940b1530 con 0x7fac94159810 2024-04-25T00:14:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.535+0000 7fac90c00640 1 -- 172.21.15.83:0/4812163 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fac7c01a070 con 0x7fac94159810 2024-04-25T00:14:25.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.535+0000 7fac90c00640 1 --2- 172.21.15.83:0/4812163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fac700678c0 0x7fac70069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:25.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.535+0000 7fac90c00640 1 -- 172.21.15.83:0/4812163 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fac7c086300 con 0x7fac94159810 2024-04-25T00:14:25.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.535+0000 7fac93e00640 1 --2- 172.21.15.83:0/4812163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fac700678c0 0x7fac70069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:25.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.539+0000 7fac93e00640 1 --2- 172.21.15.83:0/4812163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fac700678c0 0x7fac70069d80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fac840af000 tx=0x7fac84004ac0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:25.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.539+0000 7fac90c00640 1 -- 172.21.15.83:0/4812163 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fac7c056c10 con 0x7fac94159810 2024-04-25T00:14:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:25.727+0000 7fac9ae00640 1 -- 172.21.15.83:0/4812163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fac9404f510 con 0x7fac94159810 2024-04-25T00:14:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.367+0000 7fac90c00640 1 -- 172.21.15.83:0/4812163 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) ==== 122+0+0 (secure 0 0 0) 0x7fac7c056510 con 0x7fac94159810 2024-04-25T00:14:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.367+0000 7fac9ae00640 1 -- 172.21.15.83:0/4812163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fac700678c0 msgr2=0x7fac70069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.367+0000 7fac9ae00640 1 --2- 172.21.15.83:0/4812163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fac700678c0 0x7fac70069d80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fac840af000 tx=0x7fac84004ac0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.367+0000 7fac9ae00640 1 -- 172.21.15.83:0/4812163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac94159810 msgr2=0x7fac940b28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.367+0000 7fac9ae00640 1 --2- 172.21.15.83:0/4812163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac94159810 0x7fac940b28e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00e9b0 tx=0x7fac7c00ee70 comp rx=0 tx=0).stop 2024-04-25T00:14:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.367+0000 7fac9ae00640 1 -- 172.21.15.83:0/4812163 shutdown_connections 2024-04-25T00:14:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.367+0000 7fac9ae00640 1 --2- 172.21.15.83:0/4812163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fac700678c0 0x7fac70069d80 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.367+0000 7fac9ae00640 1 --2- 172.21.15.83:0/4812163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac9415fe50 0x7fac9415a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.367+0000 7fac9ae00640 1 --2- 172.21.15.83:0/4812163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac94159810 0x7fac940b28e0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.367+0000 7fac9ae00640 1 --2- 172.21.15.83:0/4812163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9406e950 0x7fac941510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.367+0000 7fac9ae00640 1 -- 172.21.15.83:0/4812163 >> 172.21.15.83:0/4812163 conn(0x7fac940b1830 msgr2=0x7fac94164e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.367+0000 7fac9ae00640 1 -- 172.21.15.83:0/4812163 shutdown_connections 2024-04-25T00:14:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.367+0000 7fac9ae00640 1 -- 172.21.15.83:0/4812163 wait complete. 2024-04-25T00:14:26.392 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:14:26.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:26.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.523+0000 7f3d68200640 1 Processor -- start 2024-04-25T00:14:26.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.523+0000 7f3d68200640 1 -- start start 2024-04-25T00:14:26.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d68200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d601553e0 0x7f3d6006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d68200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6006ef80 con 0x7f3d600b4a70 2024-04-25T00:14:26.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d68200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6015b510 con 0x7f3d6006ea00 2024-04-25T00:14:26.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d68200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6015b7f0 con 0x7f3d601553e0 2024-04-25T00:14:26.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d66200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3d600b4a70 0x7f3d600b2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41064/0 (socket says 172.21.15.83:41064) 2024-04-25T00:14:26.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d66200640 1 -- 172.21.15.83:0/3521481787 learned_addr learned my addr 172.21.15.83:0/3521481787 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:26.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d65800640 1 --2- 172.21.15.83:0/3521481787 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d601553e0 0x7f3d6006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d5fe00640 1 -- 172.21.15.83:0/3521481787 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3475882324 0 0) 0x7f3d6006ef80 con 0x7f3d600b4a70 2024-04-25T00:14:26.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d5fe00640 1 -- 172.21.15.83:0/3521481787 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d48003530 con 0x7f3d600b4a70 2024-04-25T00:14:26.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d5fe00640 1 -- 172.21.15.83:0/3521481787 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 96229940 0 0) 0x7f3d6015b510 con 0x7f3d6006ea00 2024-04-25T00:14:26.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d5fe00640 1 -- 172.21.15.83:0/3521481787 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d6006ef80 con 0x7f3d6006ea00 2024-04-25T00:14:26.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d65800640 1 -- 172.21.15.83:0/3521481787 >> v1:172.21.15.86:6789/0 conn(0x7f3d6006ea00 legacy=0x7f3d600b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d65800640 1 -- 172.21.15.83:0/3521481787 >> v1:172.21.15.83:6789/0 conn(0x7f3d600b4a70 legacy=0x7f3d600b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d5fe00640 1 -- 172.21.15.83:0/3521481787 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1200637904 0 0) 0x7f3d48003530 con 0x7f3d600b4a70 2024-04-25T00:14:26.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.527+0000 7f3d65800640 1 -- 172.21.15.83:0/3521481787 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6015bad0 con 0x7f3d601553e0 2024-04-25T00:14:26.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.531+0000 7f3d65800640 1 --2- 172.21.15.83:0/3521481787 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d601553e0 0x7f3d6006e4c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3d540097f0 tx=0x7f3d5401c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=224d71cb0b9e6c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.531+0000 7f3d5fe00640 1 -- 172.21.15.83:0/3521481787 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d540b0070 con 0x7f3d601553e0 2024-04-25T00:14:26.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.531+0000 7f3d5fe00640 1 -- 172.21.15.83:0/3521481787 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d540ab400 con 0x7f3d601553e0 2024-04-25T00:14:26.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.531+0000 7f3d5fe00640 1 -- 172.21.15.83:0/3521481787 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58001a00 con 0x7f3d601553e0 2024-04-25T00:14:26.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.531+0000 7f3d68200640 1 -- 172.21.15.83:0/3521481787 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d601553e0 msgr2=0x7f3d6006e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.531+0000 7f3d68200640 1 --2- 172.21.15.83:0/3521481787 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d601553e0 0x7f3d6006e4c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3d540097f0 tx=0x7f3d5401c370 comp rx=0 tx=0).stop 2024-04-25T00:14:26.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.531+0000 7f3d68200640 1 -- 172.21.15.83:0/3521481787 shutdown_connections 2024-04-25T00:14:26.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.531+0000 7f3d68200640 1 --2- 172.21.15.83:0/3521481787 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d601553e0 0x7f3d6006e4c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.531+0000 7f3d68200640 1 -- 172.21.15.83:0/3521481787 >> 172.21.15.83:0/3521481787 conn(0x7f3d600b1830 msgr2=0x7f3d600b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:26.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.531+0000 7f3d68200640 1 -- 172.21.15.83:0/3521481787 shutdown_connections 2024-04-25T00:14:26.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.531+0000 7f3d68200640 1 -- 172.21.15.83:0/3521481787 wait complete. 2024-04-25T00:14:26.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.531+0000 7f3d68200640 1 Processor -- start 2024-04-25T00:14:26.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.531+0000 7f3d68200640 1 -- start start 2024-04-25T00:14:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d68200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d6006ea00 0x7f3d60157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d68200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d600b4a70 0x7f3d601423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d68200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d601553e0 0x7f3d601fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d68200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d60168f20 con 0x7f3d6006ea00 2024-04-25T00:14:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d68200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d60168da0 con 0x7f3d600b4a70 2024-04-25T00:14:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d68200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3d601690a0 con 0x7f3d601553e0 2024-04-25T00:14:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d65800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d6006ea00 0x7f3d60157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d65800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d6006ea00 0x7f3d60157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40820/0 (socket says 172.21.15.83:40820) 2024-04-25T00:14:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d65800640 1 -- 172.21.15.83:0/1415099834 learned_addr learned my addr 172.21.15.83:0/1415099834 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:26.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d64e00640 1 --2- 172.21.15.83:0/1415099834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d600b4a70 0x7f3d601423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d66200640 1 --2- 172.21.15.83:0/1415099834 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d601553e0 0x7f3d601fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d65800640 1 -- 172.21.15.83:0/1415099834 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d601553e0 msgr2=0x7f3d601fa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d65800640 1 --2- 172.21.15.83:0/1415099834 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d601553e0 0x7f3d601fa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d65800640 1 -- 172.21.15.83:0/1415099834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d600b4a70 msgr2=0x7f3d601423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d65800640 1 --2- 172.21.15.83:0/1415099834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d600b4a70 0x7f3d601423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d65800640 1 -- 172.21.15.83:0/1415099834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d601fb200 con 0x7f3d6006ea00 2024-04-25T00:14:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d65800640 1 --2- 172.21.15.83:0/1415099834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d6006ea00 0x7f3d60157bf0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f3d540097f0 tx=0x7f3d5401abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d5d600640 1 -- 172.21.15.83:0/1415099834 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d540b0050 con 0x7f3d6006ea00 2024-04-25T00:14:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d5d600640 1 -- 172.21.15.83:0/1415099834 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d540b1040 con 0x7f3d6006ea00 2024-04-25T00:14:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d5d600640 1 -- 172.21.15.83:0/1415099834 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54004220 con 0x7f3d6006ea00 2024-04-25T00:14:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d68200640 1 -- 172.21.15.83:0/1415099834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d601fb4e0 con 0x7f3d6006ea00 2024-04-25T00:14:26.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d68200640 1 -- 172.21.15.83:0/1415099834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d601576d0 con 0x7f3d6006ea00 2024-04-25T00:14:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d68200640 1 -- 172.21.15.83:0/1415099834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d600737f0 con 0x7f3d6006ea00 2024-04-25T00:14:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.535+0000 7f3d5d600640 1 -- 172.21.15.83:0/1415099834 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3d540be020 con 0x7f3d6006ea00 2024-04-25T00:14:26.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.539+0000 7f3d5d600640 1 --2- 172.21.15.83:0/1415099834 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3d400678c0 0x7f3d40069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.539+0000 7f3d5d600640 1 -- 172.21.15.83:0/1415099834 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f3d5411fbf0 con 0x7f3d6006ea00 2024-04-25T00:14:26.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.539+0000 7f3d64e00640 1 --2- 172.21.15.83:0/1415099834 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3d400678c0 0x7f3d40069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.539+0000 7f3d64e00640 1 --2- 172.21.15.83:0/1415099834 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3d400678c0 0x7f3d40069d80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3d6015db50 tx=0x7f3d4c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.543+0000 7f3d5d600640 1 -- 172.21.15.83:0/1415099834 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3d54015ff0 con 0x7f3d6006ea00 2024-04-25T00:14:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.735+0000 7f3d68200640 1 -- 172.21.15.83:0/1415099834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d6006ce10 con 0x7f3d6006ea00 2024-04-25T00:14:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.735+0000 7f3d5d600640 1 -- 172.21.15.83:0/1415099834 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5174 (secure 0 0 0) 0x7f3d540bf370 con 0x7f3d6006ea00 2024-04-25T00:14:26.738 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:26.738 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/3856734881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3856734881},{"type":"v1","addr":"172.21.15.86:6834","nonce":3856734881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/12764228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":12764228},{"type":"v1","addr":"172.21.15.86:6835","nonce":12764228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1680607346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1680607346},{"type":"v1","addr":"172.21.15.83:6837","nonce":1680607346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:22.706751+0000","modified":"2024-04-25T00:14:25.734428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7023},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7023":{"gid":7023,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/121840722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":121840722},{"type":"v1","addr":"172.21.15.83:6836","nonce":121840722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7023,"qdb_cluster":[7023]},"id":6}]} 2024-04-25T00:14:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 53 2024-04-25T00:14:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.735+0000 7f3d68200640 1 -- 172.21.15.83:0/1415099834 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3d400678c0 msgr2=0x7f3d40069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.735+0000 7f3d68200640 1 --2- 172.21.15.83:0/1415099834 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3d400678c0 0x7f3d40069d80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3d6015db50 tx=0x7f3d4c009040 comp rx=0 tx=0).stop 2024-04-25T00:14:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.735+0000 7f3d68200640 1 -- 172.21.15.83:0/1415099834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d6006ea00 msgr2=0x7f3d60157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.735+0000 7f3d68200640 1 --2- 172.21.15.83:0/1415099834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d6006ea00 0x7f3d60157bf0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f3d540097f0 tx=0x7f3d5401abb0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.735+0000 7f3d68200640 1 -- 172.21.15.83:0/1415099834 shutdown_connections 2024-04-25T00:14:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.735+0000 7f3d68200640 1 --2- 172.21.15.83:0/1415099834 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3d400678c0 0x7f3d40069d80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.735+0000 7f3d68200640 1 --2- 172.21.15.83:0/1415099834 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d601553e0 0x7f3d601fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.735+0000 7f3d68200640 1 --2- 172.21.15.83:0/1415099834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d600b4a70 0x7f3d601423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.735+0000 7f3d68200640 1 --2- 172.21.15.83:0/1415099834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d6006ea00 0x7f3d60157bf0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.735+0000 7f3d68200640 1 -- 172.21.15.83:0/1415099834 >> 172.21.15.83:0/1415099834 conn(0x7f3d600b1830 msgr2=0x7f3d60071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.735+0000 7f3d68200640 1 -- 172.21.15.83:0/1415099834 shutdown_connections 2024-04-25T00:14:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.735+0000 7f3d68200640 1 -- 172.21.15.83:0/1415099834 wait complete. 2024-04-25T00:14:26.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:26.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.891+0000 7f3b03000640 1 Processor -- start 2024-04-25T00:14:26.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.891+0000 7f3b03000640 1 -- start start 2024-04-25T00:14:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.891+0000 7f3b03000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3afc15ad80 0x7f3afc0af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.891+0000 7f3b03000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3afc0ae280 con 0x7f3afc072530 2024-04-25T00:14:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.891+0000 7f3b03000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3afc0ae560 con 0x7f3afc0add30 2024-04-25T00:14:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.891+0000 7f3b03000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3afc0aee90 con 0x7f3afc15ad80 2024-04-25T00:14:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.891+0000 7f3afbe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3afc15ad80 0x7f3afc0af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.891+0000 7f3b01000640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3afc072530 0x7f3afc06e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41070/0 (socket says 172.21.15.83:41070) 2024-04-25T00:14:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.891+0000 7f3b01000640 1 -- 172.21.15.83:0/1901537539 learned_addr learned my addr 172.21.15.83:0/1901537539 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.891+0000 7f3afbe00640 1 -- 172.21.15.83:0/1901537539 >> v1:172.21.15.86:6789/0 conn(0x7f3afc0add30 legacy=0x7f3afc071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.891+0000 7f3afbe00640 1 -- 172.21.15.83:0/1901537539 >> v1:172.21.15.83:6789/0 conn(0x7f3afc072530 legacy=0x7f3afc06e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.891+0000 7f3afbe00640 1 -- 172.21.15.83:0/1901537539 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3afc0ae840 con 0x7f3afc15ad80 2024-04-25T00:14:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.891+0000 7f3afaa00640 1 -- 172.21.15.83:0/1901537539 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3862664665 0 0) 0x7f3afc0ae280 con 0x7f3afc072530 2024-04-25T00:14:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.895+0000 7f3afbe00640 1 --2- 172.21.15.83:0/1901537539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3afc15ad80 0x7f3afc0af320 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0097c0 tx=0x7f3aec01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6670c86ed32c2cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.895+0000 7f3afaa00640 1 -- 172.21.15.83:0/1901537539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec0ab650 con 0x7f3afc15ad80 2024-04-25T00:14:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.895+0000 7f3afaa00640 1 -- 172.21.15.83:0/1901537539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aec0b0070 con 0x7f3afc15ad80 2024-04-25T00:14:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.895+0000 7f3afaa00640 1 -- 172.21.15.83:0/1901537539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec0aba10 con 0x7f3afc15ad80 2024-04-25T00:14:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.895+0000 7f3b03000640 1 -- 172.21.15.83:0/1901537539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3afc15ad80 msgr2=0x7f3afc0af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.895+0000 7f3b03000640 1 --2- 172.21.15.83:0/1901537539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3afc15ad80 0x7f3afc0af320 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0097c0 tx=0x7f3aec01c430 comp rx=0 tx=0).stop 2024-04-25T00:14:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.895+0000 7f3b03000640 1 -- 172.21.15.83:0/1901537539 shutdown_connections 2024-04-25T00:14:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.895+0000 7f3b03000640 1 --2- 172.21.15.83:0/1901537539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3afc15ad80 0x7f3afc0af320 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.895+0000 7f3b03000640 1 -- 172.21.15.83:0/1901537539 >> 172.21.15.83:0/1901537539 conn(0x7f3afc0d0c20 msgr2=0x7f3afc0d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.895+0000 7f3b03000640 1 -- 172.21.15.83:0/1901537539 shutdown_connections 2024-04-25T00:14:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.895+0000 7f3b03000640 1 -- 172.21.15.83:0/1901537539 wait complete. 2024-04-25T00:14:26.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.895+0000 7f3b03000640 1 Processor -- start 2024-04-25T00:14:26.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.895+0000 7f3b03000640 1 -- start start 2024-04-25T00:14:26.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3b03000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3afc072530 0x7f3afc155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3b03000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3afc0add30 0x7f3afc155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3b03000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc15ad80 0x7f3afc15d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3b03000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc070850 con 0x7f3afc0add30 2024-04-25T00:14:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3b03000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc0706d0 con 0x7f3afc15ad80 2024-04-25T00:14:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3b03000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3afc06efa0 con 0x7f3afc072530 2024-04-25T00:14:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3afb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3afc0add30 0x7f3afc155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3afb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3afc0add30 0x7f3afc155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40830/0 (socket says 172.21.15.83:40830) 2024-04-25T00:14:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3afb400640 1 -- 172.21.15.83:0/3291680777 learned_addr learned my addr 172.21.15.83:0/3291680777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3afbe00640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3afc072530 0x7f3afc155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3afb400640 1 -- 172.21.15.83:0/3291680777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3afc072530 msgr2=0x7f3afc155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3b01000640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc15ad80 0x7f3afc15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3afb400640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3afc072530 0x7f3afc155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3afb400640 1 -- 172.21.15.83:0/3291680777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc15ad80 msgr2=0x7f3afc15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3afb400640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc15ad80 0x7f3afc15d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3afb400640 1 -- 172.21.15.83:0/3291680777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3afc14f8a0 con 0x7f3afc0add30 2024-04-25T00:14:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3afbe00640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3afc072530 0x7f3afc155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3b01000640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc15ad80 0x7f3afc15d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3afb400640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3afc0add30 0x7f3afc155ea0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800c440 tx=0x7f3ae800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3af3e00640 1 -- 172.21.15.83:0/3291680777 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8016020 con 0x7f3afc0add30 2024-04-25T00:14:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3af3e00640 1 -- 172.21.15.83:0/3291680777 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae8012830 con 0x7f3afc0add30 2024-04-25T00:14:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3af3e00640 1 -- 172.21.15.83:0/3291680777 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8012c20 con 0x7f3afc0add30 2024-04-25T00:14:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3b03000640 1 -- 172.21.15.83:0/3291680777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3afc15df70 con 0x7f3afc0add30 2024-04-25T00:14:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3b03000640 1 -- 172.21.15.83:0/3291680777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3afc154840 con 0x7f3afc0add30 2024-04-25T00:14:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.899+0000 7f3b03000640 1 -- 172.21.15.83:0/3291680777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3afc09f0b0 con 0x7f3afc0add30 2024-04-25T00:14:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.903+0000 7f3af3e00640 1 -- 172.21.15.83:0/3291680777 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3ae8019020 con 0x7f3afc0add30 2024-04-25T00:14:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.903+0000 7f3af3e00640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3adc0678c0 0x7f3adc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.903+0000 7f3af3e00640 1 -- 172.21.15.83:0/3291680777 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f3ae8085c10 con 0x7f3afc0add30 2024-04-25T00:14:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.903+0000 7f3afbe00640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3adc0678c0 0x7f3adc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.903+0000 7f3afbe00640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3adc0678c0 0x7f3adc069d80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0af000 tx=0x7f3aec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:26.907+0000 7f3af3e00640 1 -- 172.21.15.83:0/3291680777 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3ae8020030 con 0x7f3afc0add30 2024-04-25T00:14:27.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.063+0000 7f3b03000640 1 -- 172.21.15.83:0/3291680777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3afc04f5f0 con 0x7f3afc0add30 2024-04-25T00:14:27.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.063+0000 7f3af3e00640 1 -- 172.21.15.83:0/3291680777 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16801 (secure 0 0 0) 0x7f3ae80182c0 con 0x7f3afc0add30 2024-04-25T00:14:27.068 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:27.068 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:24.352251+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:21.082598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:21.713421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.067+0000 7f3b03000640 1 -- 172.21.15.83:0/3291680777 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3adc0678c0 msgr2=0x7f3adc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.067+0000 7f3b03000640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3adc0678c0 0x7f3adc069d80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0af000 tx=0x7f3aec008040 comp rx=0 tx=0).stop 2024-04-25T00:14:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.067+0000 7f3b03000640 1 -- 172.21.15.83:0/3291680777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3afc0add30 msgr2=0x7f3afc155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.067+0000 7f3b03000640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3afc0add30 0x7f3afc155ea0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800c440 tx=0x7f3ae800c900 comp rx=0 tx=0).stop 2024-04-25T00:14:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.067+0000 7f3b03000640 1 -- 172.21.15.83:0/3291680777 shutdown_connections 2024-04-25T00:14:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.067+0000 7f3b03000640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3adc0678c0 0x7f3adc069d80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.067+0000 7f3b03000640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3afc15ad80 0x7f3afc15d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.067+0000 7f3b03000640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3afc0add30 0x7f3afc155ea0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.067+0000 7f3b03000640 1 --2- 172.21.15.83:0/3291680777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3afc072530 0x7f3afc155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.067+0000 7f3b03000640 1 -- 172.21.15.83:0/3291680777 >> 172.21.15.83:0/3291680777 conn(0x7f3afc0d0c20 msgr2=0x7f3afc152320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.067+0000 7f3b03000640 1 -- 172.21.15.83:0/3291680777 shutdown_connections 2024-04-25T00:14:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.067+0000 7f3b03000640 1 -- 172.21.15.83:0/3291680777 wait complete. 2024-04-25T00:14:27.089 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:14:27.089 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:14:27.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.219+0000 7fdc83600640 1 Processor -- start 2024-04-25T00:14:27.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.219+0000 7fdc83600640 1 -- start start 2024-04-25T00:14:27.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.219+0000 7fdc83600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc7c0b47e0 0x7fdc7c0b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.219+0000 7fdc83600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc7c0b4c70 con 0x7fdc7c1553e0 2024-04-25T00:14:27.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.219+0000 7fdc83600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc7c06ef80 con 0x7fdc7c0b4360 2024-04-25T00:14:27.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.219+0000 7fdc83600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdc7c15b510 con 0x7fdc7c0b47e0 2024-04-25T00:14:27.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc80c00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdc7c1553e0 0x7fdc7c06e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41086/0 (socket says 172.21.15.83:41086) 2024-04-25T00:14:27.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc80c00640 1 -- 172.21.15.83:0/3743423988 learned_addr learned my addr 172.21.15.83:0/3743423988 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:27.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc81600640 1 --2- 172.21.15.83:0/3743423988 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc7c0b47e0 0x7fdc7c0b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 -- 172.21.15.83:0/3743423988 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3624302424 0 0) 0x7fdc7c0b4c70 con 0x7fdc7c1553e0 2024-04-25T00:14:27.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 -- 172.21.15.83:0/3743423988 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc60003530 con 0x7fdc7c1553e0 2024-04-25T00:14:27.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 -- 172.21.15.83:0/3743423988 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2453463234 0 0) 0x7fdc60003530 con 0x7fdc7c1553e0 2024-04-25T00:14:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 -- 172.21.15.83:0/3743423988 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc7c0b47e0 msgr2=0x7fdc7c0b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 --2- 172.21.15.83:0/3743423988 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc7c0b47e0 0x7fdc7c0b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 -- 172.21.15.83:0/3743423988 >> v1:172.21.15.86:6789/0 conn(0x7fdc7c0b4360 legacy=0x7fdc7c0b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 -- 172.21.15.83:0/3743423988 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c15b7f0 con 0x7fdc7c1553e0 2024-04-25T00:14:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 -- 172.21.15.83:0/3743423988 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fdc6c001680 con 0x7fdc7c1553e0 2024-04-25T00:14:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 -- 172.21.15.83:0/3743423988 >> v1:172.21.15.83:6789/0 conn(0x7fdc7c1553e0 legacy=0x7fdc7c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 --2- 172.21.15.83:0/3743423988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc600039f0 0x7fdc60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 --2- 172.21.15.83:0/3743423988 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc7c0b47e0 0x7fdc60004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 --2- 172.21.15.83:0/3743423988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc60004d50 0x7fdc600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 -- 172.21.15.83:0/3743423988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c15b510 con 0x7fdc600039f0 2024-04-25T00:14:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 -- 172.21.15.83:0/3743423988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6000ba80 con 0x7fdc60004d50 2024-04-25T00:14:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc7be00640 1 -- 172.21.15.83:0/3743423988 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdc6000de90 con 0x7fdc7c0b47e0 2024-04-25T00:14:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc80c00640 1 --2- 172.21.15.83:0/3743423988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc600039f0 0x7fdc60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc81600640 1 --2- 172.21.15.83:0/3743423988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc60004d50 0x7fdc600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc73e00640 1 --2- 172.21.15.83:0/3743423988 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc7c0b47e0 0x7fdc60004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc80c00640 1 -- 172.21.15.83:0/3743423988 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc7c0b47e0 msgr2=0x7fdc60004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:27.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc80c00640 1 --2- 172.21.15.83:0/3743423988 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc7c0b47e0 0x7fdc60004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc80c00640 1 -- 172.21.15.83:0/3743423988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc60004d50 msgr2=0x7fdc600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:27.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc80c00640 1 --2- 172.21.15.83:0/3743423988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc60004d50 0x7fdc600051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.223+0000 7fdc80c00640 1 -- 172.21.15.83:0/3743423988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc60003530 con 0x7fdc600039f0 2024-04-25T00:14:27.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.227+0000 7fdc80c00640 1 --2- 172.21.15.83:0/3743423988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc600039f0 0x7fdc60003dd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c004fd0 tx=0x7fdc6c01d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619294573649127a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:27.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.227+0000 7fdc7be00640 1 -- 172.21.15.83:0/3743423988 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c004bd0 con 0x7fdc600039f0 2024-04-25T00:14:27.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.227+0000 7fdc7be00640 1 -- 172.21.15.83:0/3743423988 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6c005ae0 con 0x7fdc600039f0 2024-04-25T00:14:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.227+0000 7fdc7be00640 1 -- 172.21.15.83:0/3743423988 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c008110 con 0x7fdc600039f0 2024-04-25T00:14:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.227+0000 7fdc83600640 1 -- 172.21.15.83:0/3743423988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc600039f0 msgr2=0x7fdc60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.227+0000 7fdc83600640 1 --2- 172.21.15.83:0/3743423988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc600039f0 0x7fdc60003dd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c004fd0 tx=0x7fdc6c01d280 comp rx=0 tx=0).stop 2024-04-25T00:14:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.227+0000 7fdc81600640 1 -- 172.21.15.83:0/3743423988 reap_dead start 2024-04-25T00:14:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.227+0000 7fdc83600640 1 -- 172.21.15.83:0/3743423988 shutdown_connections 2024-04-25T00:14:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.227+0000 7fdc83600640 1 -- 172.21.15.83:0/3743423988 >> 172.21.15.83:0/3743423988 conn(0x7fdc7c0b1830 msgr2=0x7fdc7c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.227+0000 7fdc83600640 1 -- 172.21.15.83:0/3743423988 shutdown_connections 2024-04-25T00:14:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.227+0000 7fdc83600640 1 -- 172.21.15.83:0/3743423988 wait complete. 2024-04-25T00:14:27.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.227+0000 7fdc83600640 1 Processor -- start 2024-04-25T00:14:27.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.227+0000 7fdc83600640 1 -- start start 2024-04-25T00:14:27.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc83600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc600039f0 0x7fdc7c144500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc83600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc7c151680 0x7fdc7c151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc83600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc7c14bef0 0x7fdc7c14c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc83600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c155940 con 0x7fdc7c151680 2024-04-25T00:14:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc83600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c1557c0 con 0x7fdc600039f0 2024-04-25T00:14:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc83600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdc7c06ebd0 con 0x7fdc7c14bef0 2024-04-25T00:14:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc73e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc7c151680 0x7fdc7c151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc73e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc7c151680 0x7fdc7c151b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40850/0 (socket says 172.21.15.83:40850) 2024-04-25T00:14:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc73e00640 1 -- 172.21.15.83:0/496662219 learned_addr learned my addr 172.21.15.83:0/496662219 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc80c00640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc600039f0 0x7fdc7c144500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc81600640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc7c14bef0 0x7fdc7c14c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc73e00640 1 -- 172.21.15.83:0/496662219 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc7c14bef0 msgr2=0x7fdc7c14c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc73e00640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc7c14bef0 0x7fdc7c14c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc73e00640 1 -- 172.21.15.83:0/496662219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc600039f0 msgr2=0x7fdc7c144500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc73e00640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc600039f0 0x7fdc7c144500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc73e00640 1 -- 172.21.15.83:0/496662219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c14cca0 con 0x7fdc7c151680 2024-04-25T00:14:27.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc81600640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc7c14bef0 0x7fdc7c14c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:27.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc80c00640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc600039f0 0x7fdc7c144500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:27.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc73e00640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc7c151680 0x7fdc7c151b00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400f7d0 tx=0x7fdc6400fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:27.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc79600640 1 -- 172.21.15.83:0/496662219 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc640052e0 con 0x7fdc7c151680 2024-04-25T00:14:27.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc79600640 1 -- 172.21.15.83:0/496662219 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc64018070 con 0x7fdc7c151680 2024-04-25T00:14:27.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc79600640 1 -- 172.21.15.83:0/496662219 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc640054a0 con 0x7fdc7c151680 2024-04-25T00:14:27.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc83600640 1 -- 172.21.15.83:0/496662219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7c14cf80 con 0x7fdc7c151680 2024-04-25T00:14:27.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc83600640 1 -- 172.21.15.83:0/496662219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7c14d540 con 0x7fdc7c151680 2024-04-25T00:14:27.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc83600640 1 -- 172.21.15.83:0/496662219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc7c0737f0 con 0x7fdc7c151680 2024-04-25T00:14:27.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.231+0000 7fdc79600640 1 -- 172.21.15.83:0/496662219 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdc6400d970 con 0x7fdc7c151680 2024-04-25T00:14:27.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.235+0000 7fdc79600640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdc580678c0 0x7fdc58069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.235+0000 7fdc79600640 1 -- 172.21.15.83:0/496662219 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fdc64085de0 con 0x7fdc7c151680 2024-04-25T00:14:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.235+0000 7fdc80c00640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdc580678c0 0x7fdc58069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.235+0000 7fdc80c00640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdc580678c0 0x7fdc58069d80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c004fd0 tx=0x7fdc6c0b0040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:27.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.239+0000 7fdc79600640 1 -- 172.21.15.83:0/496662219 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdc6401a310 con 0x7fdc7c151680 2024-04-25T00:14:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.391+0000 7fdc83600640 1 -- 172.21.15.83:0/496662219 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdc7c04f5f0 con 0x7fdc580678c0 2024-04-25T00:14:27.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.399+0000 7fdc79600640 1 -- 172.21.15.83:0/496662219 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252033 (secure 0 0 0) 0x7fdc7c04f5f0 con 0x7fdc580678c0 2024-04-25T00:14:27.406 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:27.407 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:14:27.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.403+0000 7fdc83600640 1 -- 172.21.15.83:0/496662219 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdc580678c0 msgr2=0x7fdc58069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.403+0000 7fdc83600640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdc580678c0 0x7fdc58069d80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c004fd0 tx=0x7fdc6c0b0040 comp rx=0 tx=0).stop 2024-04-25T00:14:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.403+0000 7fdc83600640 1 -- 172.21.15.83:0/496662219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc7c151680 msgr2=0x7fdc7c151b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.403+0000 7fdc83600640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc7c151680 0x7fdc7c151b00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400f7d0 tx=0x7fdc6400fc90 comp rx=0 tx=0).stop 2024-04-25T00:14:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.403+0000 7fdc83600640 1 -- 172.21.15.83:0/496662219 shutdown_connections 2024-04-25T00:14:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.403+0000 7fdc83600640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdc580678c0 0x7fdc58069d80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.403+0000 7fdc83600640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc7c14bef0 0x7fdc7c14c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.403+0000 7fdc83600640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc7c151680 0x7fdc7c151b00 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.403+0000 7fdc83600640 1 --2- 172.21.15.83:0/496662219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc600039f0 0x7fdc7c144500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.403+0000 7fdc83600640 1 -- 172.21.15.83:0/496662219 >> 172.21.15.83:0/496662219 conn(0x7fdc7c0b1830 msgr2=0x7fdc7c15ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.403+0000 7fdc83600640 1 -- 172.21.15.83:0/496662219 shutdown_connections 2024-04-25T00:14:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.403+0000 7fdc83600640 1 -- 172.21.15.83:0/496662219 wait complete. 2024-04-25T00:14:27.426 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-04-25T00:14:26.482423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241856,"kb_used_data":9032,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754732864,"statfs":{"total":773094113280,"available":772846452736,"internally_reserved":0,"allocated":9248768,"data_stored":4812628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.162573"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.374763+0000","last_change":"2024-04-25T00:14:23.372323+0000","last_active":"2024-04-25T00:14:23.374763+0000","last_peered":"2024-04-25T00:14:23.374763+0000","last_clean":"2024-04-25T00:14:23.374763+0000","last_became_active":"2024-04-25T00:14:23.372023+0000","last_became_peered":"2024-04-25T00:14:23.372023+0000","last_unstale":"2024-04-25T00:14:23.374763+0000","last_undegraded":"2024-04-25T00:14:23.374763+0000","last_fullsized":"2024-04-25T00:14:23.374763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:18.241067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368569+0000","last_change":"2024-04-25T00:14:23.364050+0000","last_active":"2024-04-25T00:14:24.368569+0000","last_peered":"2024-04-25T00:14:24.368569+0000","last_clean":"2024-04-25T00:14:24.368569+0000","last_became_active":"2024-04-25T00:14:23.363803+0000","last_became_peered":"2024-04-25T00:14:23.363803+0000","last_unstale":"2024-04-25T00:14:24.368569+0000","last_undegraded":"2024-04-25T00:14:24.368569+0000","last_fullsized":"2024-04-25T00:14:24.368569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:21.815636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.377205+0000","last_change":"2024-04-25T00:14:23.373591+0000","last_active":"2024-04-25T00:14:23.377205+0000","last_peered":"2024-04-25T00:14:23.377205+0000","last_clean":"2024-04-25T00:14:23.377205+0000","last_became_active":"2024-04-25T00:14:23.373186+0000","last_became_peered":"2024-04-25T00:14:23.373186+0000","last_unstale":"2024-04-25T00:14:23.377205+0000","last_undegraded":"2024-04-25T00:14:23.377205+0000","last_fullsized":"2024-04-25T00:14:23.377205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:32.704007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.377741+0000","last_change":"2024-04-25T00:14:23.367538+0000","last_active":"2024-04-25T00:14:23.377741+0000","last_peered":"2024-04-25T00:14:23.377741+0000","last_clean":"2024-04-25T00:14:23.377741+0000","last_became_active":"2024-04-25T00:14:23.367221+0000","last_became_peered":"2024-04-25T00:14:23.367221+0000","last_unstale":"2024-04-25T00:14:23.377741+0000","last_undegraded":"2024-04-25T00:14:23.377741+0000","last_fullsized":"2024-04-25T00:14:23.377741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:07.160836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365202+0000","last_change":"2024-04-25T00:14:23.365270+0000","last_active":"2024-04-25T00:14:24.365202+0000","last_peered":"2024-04-25T00:14:24.365202+0000","last_clean":"2024-04-25T00:14:24.365202+0000","last_became_active":"2024-04-25T00:14:23.364878+0000","last_became_peered":"2024-04-25T00:14:23.364878+0000","last_unstale":"2024-04-25T00:14:24.365202+0000","last_undegraded":"2024-04-25T00:14:24.365202+0000","last_fullsized":"2024-04-25T00:14:24.365202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:39.465602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362129+0000","last_change":"2024-04-25T00:14:23.378640+0000","last_active":"2024-04-25T00:14:24.362129+0000","last_peered":"2024-04-25T00:14:24.362129+0000","last_clean":"2024-04-25T00:14:24.362129+0000","last_became_active":"2024-04-25T00:14:23.378315+0000","last_became_peered":"2024-04-25T00:14:23.378315+0000","last_unstale":"2024-04-25T00:14:24.362129+0000","last_undegraded":"2024-04-25T00:14:24.362129+0000","last_fullsized":"2024-04-25T00:14:24.362129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:21.672662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.380510+0000","last_change":"2024-04-25T00:14:23.375200+0000","last_active":"2024-04-25T00:14:23.380510+0000","last_peered":"2024-04-25T00:14:23.380510+0000","last_clean":"2024-04-25T00:14:23.380510+0000","last_became_active":"2024-04-25T00:14:23.374850+0000","last_became_peered":"2024-04-25T00:14:23.374850+0000","last_unstale":"2024-04-25T00:14:23.380510+0000","last_undegraded":"2024-04-25T00:14:23.380510+0000","last_fullsized":"2024-04-25T00:14:23.380510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:18.241067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.378861+0000","last_change":"2024-04-25T00:14:23.368812+0000","last_active":"2024-04-25T00:14:23.378861+0000","last_peered":"2024-04-25T00:14:23.378861+0000","last_clean":"2024-04-25T00:14:23.378861+0000","last_became_active":"2024-04-25T00:14:23.368194+0000","last_became_peered":"2024-04-25T00:14:23.368194+0000","last_unstale":"2024-04-25T00:14:23.378861+0000","last_undegraded":"2024-04-25T00:14:23.378861+0000","last_fullsized":"2024-04-25T00:14:23.378861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:08.933993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364604+0000","last_change":"2024-04-25T00:14:23.364808+0000","last_active":"2024-04-25T00:14:24.364604+0000","last_peered":"2024-04-25T00:14:24.364604+0000","last_clean":"2024-04-25T00:14:24.364604+0000","last_became_active":"2024-04-25T00:14:23.364485+0000","last_became_peered":"2024-04-25T00:14:23.364485+0000","last_unstale":"2024-04-25T00:14:24.364604+0000","last_undegraded":"2024-04-25T00:14:24.364604+0000","last_fullsized":"2024-04-25T00:14:24.364604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:08.933993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.372335+0000","last_change":"2024-04-25T00:14:23.368462+0000","last_active":"2024-04-25T00:14:23.372335+0000","last_peered":"2024-04-25T00:14:23.372335+0000","last_clean":"2024-04-25T00:14:23.372335+0000","last_became_active":"2024-04-25T00:14:23.368209+0000","last_became_peered":"2024-04-25T00:14:23.368209+0000","last_unstale":"2024-04-25T00:14:23.372335+0000","last_undegraded":"2024-04-25T00:14:23.372335+0000","last_fullsized":"2024-04-25T00:14:23.372335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:55.685973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364719+0000","last_change":"2024-04-25T00:14:23.376839+0000","last_active":"2024-04-25T00:14:24.364719+0000","last_peered":"2024-04-25T00:14:24.364719+0000","last_clean":"2024-04-25T00:14:24.364719+0000","last_became_active":"2024-04-25T00:14:23.376560+0000","last_became_peered":"2024-04-25T00:14:23.376560+0000","last_unstale":"2024-04-25T00:14:24.364719+0000","last_undegraded":"2024-04-25T00:14:24.364719+0000","last_fullsized":"2024-04-25T00:14:24.364719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:14:31.742965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.376808+0000","last_change":"2024-04-25T00:14:23.375797+0000","last_active":"2024-04-25T00:14:23.376808+0000","last_peered":"2024-04-25T00:14:23.376808+0000","last_clean":"2024-04-25T00:14:23.376808+0000","last_became_active":"2024-04-25T00:14:23.375476+0000","last_became_peered":"2024-04-25T00:14:23.375476+0000","last_unstale":"2024-04-25T00:14:23.376808+0000","last_undegraded":"2024-04-25T00:14:23.376808+0000","last_fullsized":"2024-04-25T00:14:23.376808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:14.861158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.369175+0000","last_change":"2024-04-25T00:14:23.364656+0000","last_active":"2024-04-25T00:14:23.369175+0000","last_peered":"2024-04-25T00:14:23.369175+0000","last_clean":"2024-04-25T00:14:23.369175+0000","last_became_active":"2024-04-25T00:14:23.364375+0000","last_became_peered":"2024-04-25T00:14:23.364375+0000","last_unstale":"2024-04-25T00:14:23.369175+0000","last_undegraded":"2024-04-25T00:14:23.369175+0000","last_fullsized":"2024-04-25T00:14:23.369175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:34.577592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364349+0000","last_change":"2024-04-25T00:14:23.373638+0000","last_active":"2024-04-25T00:14:24.364349+0000","last_peered":"2024-04-25T00:14:24.364349+0000","last_clean":"2024-04-25T00:14:24.364349+0000","last_became_active":"2024-04-25T00:14:23.373332+0000","last_became_peered":"2024-04-25T00:14:23.373332+0000","last_unstale":"2024-04-25T00:14:24.364349+0000","last_undegraded":"2024-04-25T00:14:24.364349+0000","last_fullsized":"2024-04-25T00:14:24.364349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:39.069760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.379909+0000","last_change":"2024-04-25T00:14:23.370894+0000","last_active":"2024-04-25T00:14:23.379909+0000","last_peered":"2024-04-25T00:14:23.379909+0000","last_clean":"2024-04-25T00:14:23.379909+0000","last_became_active":"2024-04-25T00:14:23.370562+0000","last_became_peered":"2024-04-25T00:14:23.370562+0000","last_unstale":"2024-04-25T00:14:23.379909+0000","last_undegraded":"2024-04-25T00:14:23.379909+0000","last_fullsized":"2024-04-25T00:14:23.379909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:37.919241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362097+0000","last_change":"2024-04-25T00:14:23.383343+0000","last_active":"2024-04-25T00:14:24.362097+0000","last_peered":"2024-04-25T00:14:24.362097+0000","last_clean":"2024-04-25T00:14:24.362097+0000","last_became_active":"2024-04-25T00:14:23.383106+0000","last_became_peered":"2024-04-25T00:14:23.383106+0000","last_unstale":"2024-04-25T00:14:24.362097+0000","last_undegraded":"2024-04-25T00:14:24.362097+0000","last_fullsized":"2024-04-25T00:14:24.362097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:07.812257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362589+0000","last_change":"2024-04-25T00:14:23.372181+0000","last_active":"2024-04-25T00:14:24.362589+0000","last_peered":"2024-04-25T00:14:24.362589+0000","last_clean":"2024-04-25T00:14:24.362589+0000","last_became_active":"2024-04-25T00:14:23.371910+0000","last_became_peered":"2024-04-25T00:14:23.371910+0000","last_unstale":"2024-04-25T00:14:24.362589+0000","last_undegraded":"2024-04-25T00:14:24.362589+0000","last_fullsized":"2024-04-25T00:14:24.362589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:12.434926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.369256+0000","last_change":"2024-04-25T00:14:23.365671+0000","last_active":"2024-04-25T00:14:23.369256+0000","last_peered":"2024-04-25T00:14:23.369256+0000","last_clean":"2024-04-25T00:14:23.369256+0000","last_became_active":"2024-04-25T00:14:23.365194+0000","last_became_peered":"2024-04-25T00:14:23.365194+0000","last_unstale":"2024-04-25T00:14:23.369256+0000","last_undegraded":"2024-04-25T00:14:23.369256+0000","last_fullsized":"2024-04-25T00:14:23.369256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:27.842724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364875+0000","last_change":"2024-04-25T00:14:23.373531+0000","last_active":"2024-04-25T00:14:24.364875+0000","last_peered":"2024-04-25T00:14:24.364875+0000","last_clean":"2024-04-25T00:14:24.364875+0000","last_became_active":"2024-04-25T00:14:23.373188+0000","last_became_peered":"2024-04-25T00:14:23.373188+0000","last_unstale":"2024-04-25T00:14:24.364875+0000","last_undegraded":"2024-04-25T00:14:24.364875+0000","last_fullsized":"2024-04-25T00:14:24.364875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:16:50.264288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.380453+0000","last_change":"2024-04-25T00:14:23.377562+0000","last_active":"2024-04-25T00:14:23.380453+0000","last_peered":"2024-04-25T00:14:23.380453+0000","last_clean":"2024-04-25T00:14:23.380453+0000","last_became_active":"2024-04-25T00:14:23.377309+0000","last_became_peered":"2024-04-25T00:14:23.377309+0000","last_unstale":"2024-04-25T00:14:23.380453+0000","last_undegraded":"2024-04-25T00:14:23.380453+0000","last_fullsized":"2024-04-25T00:14:23.380453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:11.078353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365418+0000","last_change":"2024-04-25T00:14:23.365665+0000","last_active":"2024-04-25T00:14:24.365418+0000","last_peered":"2024-04-25T00:14:24.365418+0000","last_clean":"2024-04-25T00:14:24.365418+0000","last_became_active":"2024-04-25T00:14:23.365272+0000","last_became_peered":"2024-04-25T00:14:23.365272+0000","last_unstale":"2024-04-25T00:14:24.365418+0000","last_undegraded":"2024-04-25T00:14:24.365418+0000","last_fullsized":"2024-04-25T00:14:24.365418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:40.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364532+0000","last_change":"2024-04-25T00:14:23.378289+0000","last_active":"2024-04-25T00:14:24.364532+0000","last_peered":"2024-04-25T00:14:24.364532+0000","last_clean":"2024-04-25T00:14:24.364532+0000","last_became_active":"2024-04-25T00:14:23.377889+0000","last_became_peered":"2024-04-25T00:14:23.377889+0000","last_unstale":"2024-04-25T00:14:24.364532+0000","last_undegraded":"2024-04-25T00:14:24.364532+0000","last_fullsized":"2024-04-25T00:14:24.364532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:07.160836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365753+0000","last_change":"2024-04-25T00:14:23.356266+0000","last_active":"2024-04-25T00:14:24.365753+0000","last_peered":"2024-04-25T00:14:24.365753+0000","last_clean":"2024-04-25T00:14:24.365753+0000","last_became_active":"2024-04-25T00:14:23.355912+0000","last_became_peered":"2024-04-25T00:14:23.355912+0000","last_unstale":"2024-04-25T00:14:24.365753+0000","last_undegraded":"2024-04-25T00:14:24.365753+0000","last_fullsized":"2024-04-25T00:14:24.365753+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:16:50.264288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.374621+0000","last_change":"2024-04-25T00:14:23.367374+0000","last_active":"2024-04-25T00:14:23.374621+0000","last_peered":"2024-04-25T00:14:23.374621+0000","last_clean":"2024-04-25T00:14:23.374621+0000","last_became_active":"2024-04-25T00:14:23.366227+0000","last_became_peered":"2024-04-25T00:14:23.366227+0000","last_unstale":"2024-04-25T00:14:23.374621+0000","last_undegraded":"2024-04-25T00:14:23.374621+0000","last_fullsized":"2024-04-25T00:14:23.374621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:09.799071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368542+0000","last_change":"2024-04-25T00:14:23.359743+0000","last_active":"2024-04-25T00:14:24.368542+0000","last_peered":"2024-04-25T00:14:24.368542+0000","last_clean":"2024-04-25T00:14:24.368542+0000","last_became_active":"2024-04-25T00:14:23.359158+0000","last_became_peered":"2024-04-25T00:14:23.359158+0000","last_unstale":"2024-04-25T00:14:24.368542+0000","last_undegraded":"2024-04-25T00:14:24.368542+0000","last_fullsized":"2024-04-25T00:14:24.368542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:04.628530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364726+0000","last_change":"2024-04-25T00:14:23.368453+0000","last_active":"2024-04-25T00:14:24.364726+0000","last_peered":"2024-04-25T00:14:24.364726+0000","last_clean":"2024-04-25T00:14:24.364726+0000","last_became_active":"2024-04-25T00:14:23.368154+0000","last_became_peered":"2024-04-25T00:14:23.368154+0000","last_unstale":"2024-04-25T00:14:24.364726+0000","last_undegraded":"2024-04-25T00:14:24.364726+0000","last_fullsized":"2024-04-25T00:14:24.364726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:37.919241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.352873+0000","last_change":"2024-04-25T00:14:23.350710+0000","last_active":"2024-04-25T00:14:23.352873+0000","last_peered":"2024-04-25T00:14:23.352873+0000","last_clean":"2024-04-25T00:14:23.352873+0000","last_became_active":"2024-04-25T00:14:23.350423+0000","last_became_peered":"2024-04-25T00:14:23.350423+0000","last_unstale":"2024-04-25T00:14:23.352873+0000","last_undegraded":"2024-04-25T00:14:23.352873+0000","last_fullsized":"2024-04-25T00:14:23.352873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:20.377876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.379579+0000","last_change":"2024-04-25T00:14:23.369797+0000","last_active":"2024-04-25T00:14:23.379579+0000","last_peered":"2024-04-25T00:14:23.379579+0000","last_clean":"2024-04-25T00:14:23.379579+0000","last_became_active":"2024-04-25T00:14:23.369544+0000","last_became_peered":"2024-04-25T00:14:23.369544+0000","last_unstale":"2024-04-25T00:14:23.379579+0000","last_undegraded":"2024-04-25T00:14:23.379579+0000","last_fullsized":"2024-04-25T00:14:23.379579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:14:31.742965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.363966+0000","last_change":"2024-04-25T00:14:23.361046+0000","last_active":"2024-04-25T00:14:23.363966+0000","last_peered":"2024-04-25T00:14:23.363966+0000","last_clean":"2024-04-25T00:14:23.363966+0000","last_became_active":"2024-04-25T00:14:23.360481+0000","last_became_peered":"2024-04-25T00:14:23.360481+0000","last_unstale":"2024-04-25T00:14:23.363966+0000","last_undegraded":"2024-04-25T00:14:23.363966+0000","last_fullsized":"2024-04-25T00:14:23.363966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:21.815636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364688+0000","last_change":"2024-04-25T00:14:23.367748+0000","last_active":"2024-04-25T00:14:24.364688+0000","last_peered":"2024-04-25T00:14:24.364688+0000","last_clean":"2024-04-25T00:14:24.364688+0000","last_became_active":"2024-04-25T00:14:23.364892+0000","last_became_peered":"2024-04-25T00:14:23.364892+0000","last_unstale":"2024-04-25T00:14:24.364688+0000","last_undegraded":"2024-04-25T00:14:24.364688+0000","last_fullsized":"2024-04-25T00:14:24.364688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:54.268073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368233+0000","last_change":"2024-04-25T00:14:23.366683+0000","last_active":"2024-04-25T00:14:24.368233+0000","last_peered":"2024-04-25T00:14:24.368233+0000","last_clean":"2024-04-25T00:14:24.368233+0000","last_became_active":"2024-04-25T00:14:23.366287+0000","last_became_peered":"2024-04-25T00:14:23.366287+0000","last_unstale":"2024-04-25T00:14:24.368233+0000","last_undegraded":"2024-04-25T00:14:24.368233+0000","last_fullsized":"2024-04-25T00:14:24.368233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:27.227370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362101+0000","last_change":"2024-04-25T00:14:23.371472+0000","last_active":"2024-04-25T00:14:24.362101+0000","last_peered":"2024-04-25T00:14:24.362101+0000","last_clean":"2024-04-25T00:14:24.362101+0000","last_became_active":"2024-04-25T00:14:23.371229+0000","last_became_peered":"2024-04-25T00:14:23.371229+0000","last_unstale":"2024-04-25T00:14:24.362101+0000","last_undegraded":"2024-04-25T00:14:24.362101+0000","last_fullsized":"2024-04-25T00:14:24.362101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:47.177380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362453+0000","last_change":"2024-04-25T00:14:23.369972+0000","last_active":"2024-04-25T00:14:24.362453+0000","last_peered":"2024-04-25T00:14:24.362453+0000","last_clean":"2024-04-25T00:14:24.362453+0000","last_became_active":"2024-04-25T00:14:23.369705+0000","last_became_peered":"2024-04-25T00:14:23.369705+0000","last_unstale":"2024-04-25T00:14:24.362453+0000","last_undegraded":"2024-04-25T00:14:24.362453+0000","last_fullsized":"2024-04-25T00:14:24.362453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:27.570041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.366248+0000","last_change":"2024-04-25T00:14:22.340389+0000","last_active":"2024-04-25T00:14:24.366248+0000","last_peered":"2024-04-25T00:14:24.366248+0000","last_clean":"2024-04-25T00:14:24.366248+0000","last_became_active":"2024-04-25T00:14:22.339975+0000","last_became_peered":"2024-04-25T00:14:22.339975+0000","last_unstale":"2024-04-25T00:14:24.366248+0000","last_undegraded":"2024-04-25T00:14:24.366248+0000","last_fullsized":"2024-04-25T00:14:24.366248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:47.286733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362723+0000","last_change":"2024-04-25T00:14:23.367598+0000","last_active":"2024-04-25T00:14:24.362723+0000","last_peered":"2024-04-25T00:14:24.362723+0000","last_clean":"2024-04-25T00:14:24.362723+0000","last_became_active":"2024-04-25T00:14:23.367295+0000","last_became_peered":"2024-04-25T00:14:23.367295+0000","last_unstale":"2024-04-25T00:14:24.362723+0000","last_undegraded":"2024-04-25T00:14:24.362723+0000","last_fullsized":"2024-04-25T00:14:24.362723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:53.768938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364844+0000","last_change":"2024-04-25T00:14:22.342510+0000","last_active":"2024-04-25T00:14:24.364844+0000","last_peered":"2024-04-25T00:14:24.364844+0000","last_clean":"2024-04-25T00:14:24.364844+0000","last_became_active":"2024-04-25T00:14:22.342193+0000","last_became_peered":"2024-04-25T00:14:22.342193+0000","last_unstale":"2024-04-25T00:14:24.364844+0000","last_undegraded":"2024-04-25T00:14:24.364844+0000","last_fullsized":"2024-04-25T00:14:24.364844+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:34.379489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.370331+0000","last_change":"2024-04-25T00:14:23.368492+0000","last_active":"2024-04-25T00:14:23.370331+0000","last_peered":"2024-04-25T00:14:23.370331+0000","last_clean":"2024-04-25T00:14:23.370331+0000","last_became_active":"2024-04-25T00:14:23.368211+0000","last_became_peered":"2024-04-25T00:14:23.368211+0000","last_unstale":"2024-04-25T00:14:23.370331+0000","last_undegraded":"2024-04-25T00:14:23.370331+0000","last_fullsized":"2024-04-25T00:14:23.370331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:27.842724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.385933+0000","last_change":"2024-04-25T00:14:22.345792+0000","last_active":"2024-04-25T00:14:23.385933+0000","last_peered":"2024-04-25T00:14:23.385933+0000","last_clean":"2024-04-25T00:14:23.385933+0000","last_became_active":"2024-04-25T00:14:22.345288+0000","last_became_peered":"2024-04-25T00:14:22.345288+0000","last_unstale":"2024-04-25T00:14:23.385933+0000","last_undegraded":"2024-04-25T00:14:23.385933+0000","last_fullsized":"2024-04-25T00:14:23.385933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:45.792758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362768+0000","last_change":"2024-04-25T00:14:23.367302+0000","last_active":"2024-04-25T00:14:24.362768+0000","last_peered":"2024-04-25T00:14:24.362768+0000","last_clean":"2024-04-25T00:14:24.362768+0000","last_became_active":"2024-04-25T00:14:23.367026+0000","last_became_peered":"2024-04-25T00:14:23.367026+0000","last_unstale":"2024-04-25T00:14:24.362768+0000","last_undegraded":"2024-04-25T00:14:24.362768+0000","last_fullsized":"2024-04-25T00:14:24.362768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:30.600953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.366101+0000","last_change":"2024-04-25T00:14:22.338007+0000","last_active":"2024-04-25T00:14:24.366101+0000","last_peered":"2024-04-25T00:14:24.366101+0000","last_clean":"2024-04-25T00:14:24.366101+0000","last_became_active":"2024-04-25T00:14:22.337563+0000","last_became_peered":"2024-04-25T00:14:22.337563+0000","last_unstale":"2024-04-25T00:14:24.366101+0000","last_undegraded":"2024-04-25T00:14:24.366101+0000","last_fullsized":"2024-04-25T00:14:24.366101+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:10.436890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.378883+0000","last_change":"2024-04-25T00:14:23.369053+0000","last_active":"2024-04-25T00:14:23.378883+0000","last_peered":"2024-04-25T00:14:23.378883+0000","last_clean":"2024-04-25T00:14:23.378883+0000","last_became_active":"2024-04-25T00:14:23.368533+0000","last_became_peered":"2024-04-25T00:14:23.368533+0000","last_unstale":"2024-04-25T00:14:23.378883+0000","last_undegraded":"2024-04-25T00:14:23.378883+0000","last_fullsized":"2024-04-25T00:14:23.378883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:54.268073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365302+0000","last_change":"2024-04-25T00:14:22.343797+0000","last_active":"2024-04-25T00:14:24.365302+0000","last_peered":"2024-04-25T00:14:24.365302+0000","last_clean":"2024-04-25T00:14:24.365302+0000","last_became_active":"2024-04-25T00:14:22.343077+0000","last_became_peered":"2024-04-25T00:14:22.343077+0000","last_unstale":"2024-04-25T00:14:24.365302+0000","last_undegraded":"2024-04-25T00:14:24.365302+0000","last_fullsized":"2024-04-25T00:14:24.365302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:09.512347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365835+0000","last_change":"2024-04-25T00:14:23.368185+0000","last_active":"2024-04-25T00:14:24.365835+0000","last_peered":"2024-04-25T00:14:24.365835+0000","last_clean":"2024-04-25T00:14:24.365835+0000","last_became_active":"2024-04-25T00:14:23.367850+0000","last_became_peered":"2024-04-25T00:14:23.367850+0000","last_unstale":"2024-04-25T00:14:24.365835+0000","last_undegraded":"2024-04-25T00:14:24.365835+0000","last_fullsized":"2024-04-25T00:14:24.365835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:06.045434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.387449+0000","last_change":"2024-04-25T00:14:22.348575+0000","last_active":"2024-04-25T00:14:23.387449+0000","last_peered":"2024-04-25T00:14:23.387449+0000","last_clean":"2024-04-25T00:14:23.387449+0000","last_became_active":"2024-04-25T00:14:22.346712+0000","last_became_peered":"2024-04-25T00:14:22.346712+0000","last_unstale":"2024-04-25T00:14:23.387449+0000","last_undegraded":"2024-04-25T00:14:23.387449+0000","last_fullsized":"2024-04-25T00:14:23.387449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:44.218808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362150+0000","last_change":"2024-04-25T00:14:23.375834+0000","last_active":"2024-04-25T00:14:24.362150+0000","last_peered":"2024-04-25T00:14:24.362150+0000","last_clean":"2024-04-25T00:14:24.362150+0000","last_became_active":"2024-04-25T00:14:23.375581+0000","last_became_peered":"2024-04-25T00:14:23.375581+0000","last_unstale":"2024-04-25T00:14:24.362150+0000","last_undegraded":"2024-04-25T00:14:24.362150+0000","last_fullsized":"2024-04-25T00:14:24.362150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:35.757421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.358125+0000","last_change":"2024-04-25T00:14:22.349355+0000","last_active":"2024-04-25T00:14:23.358125+0000","last_peered":"2024-04-25T00:14:23.358125+0000","last_clean":"2024-04-25T00:14:23.358125+0000","last_became_active":"2024-04-25T00:14:22.349100+0000","last_became_peered":"2024-04-25T00:14:22.349100+0000","last_unstale":"2024-04-25T00:14:23.358125+0000","last_undegraded":"2024-04-25T00:14:23.358125+0000","last_fullsized":"2024-04-25T00:14:23.358125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:20.810365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.364546+0000","last_change":"2024-04-25T00:14:23.358714+0000","last_active":"2024-04-25T00:14:23.364546+0000","last_peered":"2024-04-25T00:14:23.364546+0000","last_clean":"2024-04-25T00:14:23.364546+0000","last_became_active":"2024-04-25T00:14:23.358427+0000","last_became_peered":"2024-04-25T00:14:23.358427+0000","last_unstale":"2024-04-25T00:14:23.364546+0000","last_undegraded":"2024-04-25T00:14:23.364546+0000","last_fullsized":"2024-04-25T00:14:23.364546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:48.292005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.351787+0000","last_change":"2024-04-25T00:14:22.343802+0000","last_active":"2024-04-25T00:14:23.351787+0000","last_peered":"2024-04-25T00:14:23.351787+0000","last_clean":"2024-04-25T00:14:23.351787+0000","last_became_active":"2024-04-25T00:14:22.343539+0000","last_became_peered":"2024-04-25T00:14:22.343539+0000","last_unstale":"2024-04-25T00:14:23.351787+0000","last_undegraded":"2024-04-25T00:14:23.351787+0000","last_fullsized":"2024-04-25T00:14:23.351787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:38.540125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364819+0000","last_change":"2024-04-25T00:14:23.375118+0000","last_active":"2024-04-25T00:14:24.364819+0000","last_peered":"2024-04-25T00:14:24.364819+0000","last_clean":"2024-04-25T00:14:24.364819+0000","last_became_active":"2024-04-25T00:14:23.374696+0000","last_became_peered":"2024-04-25T00:14:23.374696+0000","last_unstale":"2024-04-25T00:14:24.364819+0000","last_undegraded":"2024-04-25T00:14:24.364819+0000","last_fullsized":"2024-04-25T00:14:24.364819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:11.078353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368613+0000","last_change":"2024-04-25T00:14:22.342276+0000","last_active":"2024-04-25T00:14:24.368613+0000","last_peered":"2024-04-25T00:14:24.368613+0000","last_clean":"2024-04-25T00:14:24.368613+0000","last_became_active":"2024-04-25T00:14:22.341787+0000","last_became_peered":"2024-04-25T00:14:22.341787+0000","last_unstale":"2024-04-25T00:14:24.368613+0000","last_undegraded":"2024-04-25T00:14:24.368613+0000","last_fullsized":"2024-04-25T00:14:24.368613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:38.460331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.371351+0000","last_change":"2024-04-25T00:14:23.369380+0000","last_active":"2024-04-25T00:14:23.371351+0000","last_peered":"2024-04-25T00:14:23.371351+0000","last_clean":"2024-04-25T00:14:23.371351+0000","last_became_active":"2024-04-25T00:14:23.369093+0000","last_became_peered":"2024-04-25T00:14:23.369093+0000","last_unstale":"2024-04-25T00:14:23.371351+0000","last_undegraded":"2024-04-25T00:14:23.371351+0000","last_fullsized":"2024-04-25T00:14:23.371351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:09.799071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.357270+0000","last_change":"2024-04-25T00:14:22.349310+0000","last_active":"2024-04-25T00:14:23.357270+0000","last_peered":"2024-04-25T00:14:23.357270+0000","last_clean":"2024-04-25T00:14:23.357270+0000","last_became_active":"2024-04-25T00:14:22.347510+0000","last_became_peered":"2024-04-25T00:14:22.347510+0000","last_unstale":"2024-04-25T00:14:23.357270+0000","last_undegraded":"2024-04-25T00:14:23.357270+0000","last_fullsized":"2024-04-25T00:14:23.357270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:45.792758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.377858+0000","last_change":"2024-04-25T00:14:23.373096+0000","last_active":"2024-04-25T00:14:23.377858+0000","last_peered":"2024-04-25T00:14:23.377858+0000","last_clean":"2024-04-25T00:14:23.377858+0000","last_became_active":"2024-04-25T00:14:23.372844+0000","last_became_peered":"2024-04-25T00:14:23.372844+0000","last_unstale":"2024-04-25T00:14:23.377858+0000","last_undegraded":"2024-04-25T00:14:23.377858+0000","last_fullsized":"2024-04-25T00:14:23.377858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:24.423768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364984+0000","last_change":"2024-04-25T00:14:22.345472+0000","last_active":"2024-04-25T00:14:24.364984+0000","last_peered":"2024-04-25T00:14:24.364984+0000","last_clean":"2024-04-25T00:14:24.364984+0000","last_became_active":"2024-04-25T00:14:22.344909+0000","last_became_peered":"2024-04-25T00:14:22.344909+0000","last_unstale":"2024-04-25T00:14:24.364984+0000","last_undegraded":"2024-04-25T00:14:24.364984+0000","last_fullsized":"2024-04-25T00:14:24.364984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:14.055977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.364354+0000","last_change":"2024-04-25T00:14:23.360218+0000","last_active":"2024-04-25T00:14:23.364354+0000","last_peered":"2024-04-25T00:14:23.364354+0000","last_clean":"2024-04-25T00:14:23.364354+0000","last_became_active":"2024-04-25T00:14:23.359915+0000","last_became_peered":"2024-04-25T00:14:23.359915+0000","last_unstale":"2024-04-25T00:14:23.364354+0000","last_undegraded":"2024-04-25T00:14:23.364354+0000","last_fullsized":"2024-04-25T00:14:23.364354+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:11.442161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362558+0000","last_change":"2024-04-25T00:14:22.344458+0000","last_active":"2024-04-25T00:14:24.362558+0000","last_peered":"2024-04-25T00:14:24.362558+0000","last_clean":"2024-04-25T00:14:24.362558+0000","last_became_active":"2024-04-25T00:14:22.343093+0000","last_became_peered":"2024-04-25T00:14:22.343093+0000","last_unstale":"2024-04-25T00:14:24.362558+0000","last_undegraded":"2024-04-25T00:14:24.362558+0000","last_fullsized":"2024-04-25T00:14:24.362558+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:38.540125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.369170+0000","last_change":"2024-04-25T00:14:23.364973+0000","last_active":"2024-04-25T00:14:23.369170+0000","last_peered":"2024-04-25T00:14:23.369170+0000","last_clean":"2024-04-25T00:14:23.369170+0000","last_became_active":"2024-04-25T00:14:23.364727+0000","last_became_peered":"2024-04-25T00:14:23.364727+0000","last_unstale":"2024-04-25T00:14:23.369170+0000","last_undegraded":"2024-04-25T00:14:23.369170+0000","last_fullsized":"2024-04-25T00:14:23.369170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:39.069760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.390560+0000","last_change":"2024-04-25T00:14:22.353296+0000","last_active":"2024-04-25T00:14:23.390560+0000","last_peered":"2024-04-25T00:14:23.390560+0000","last_clean":"2024-04-25T00:14:23.390560+0000","last_became_active":"2024-04-25T00:14:22.353015+0000","last_became_peered":"2024-04-25T00:14:22.353015+0000","last_unstale":"2024-04-25T00:14:23.390560+0000","last_undegraded":"2024-04-25T00:14:23.390560+0000","last_fullsized":"2024-04-25T00:14:23.390560+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:26.222098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364714+0000","last_change":"2024-04-25T00:14:23.364620+0000","last_active":"2024-04-25T00:14:24.364714+0000","last_peered":"2024-04-25T00:14:24.364714+0000","last_clean":"2024-04-25T00:14:24.364714+0000","last_became_active":"2024-04-25T00:14:23.364322+0000","last_became_peered":"2024-04-25T00:14:23.364322+0000","last_unstale":"2024-04-25T00:14:24.364714+0000","last_undegraded":"2024-04-25T00:14:24.364714+0000","last_fullsized":"2024-04-25T00:14:24.364714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:34.577592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362235+0000","last_change":"2024-04-25T00:14:22.349316+0000","last_active":"2024-04-25T00:14:24.362235+0000","last_peered":"2024-04-25T00:14:24.362235+0000","last_clean":"2024-04-25T00:14:24.362235+0000","last_became_active":"2024-04-25T00:14:22.347911+0000","last_became_peered":"2024-04-25T00:14:22.347911+0000","last_unstale":"2024-04-25T00:14:24.362235+0000","last_undegraded":"2024-04-25T00:14:24.362235+0000","last_fullsized":"2024-04-25T00:14:24.362235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:03.623259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.372371+0000","last_change":"2024-04-25T00:14:23.369744+0000","last_active":"2024-04-25T00:14:23.372371+0000","last_peered":"2024-04-25T00:14:23.372371+0000","last_clean":"2024-04-25T00:14:23.372371+0000","last_became_active":"2024-04-25T00:14:23.369484+0000","last_became_peered":"2024-04-25T00:14:23.369484+0000","last_unstale":"2024-04-25T00:14:23.372371+0000","last_undegraded":"2024-04-25T00:14:23.372371+0000","last_fullsized":"2024-04-25T00:14:23.372371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:22.223038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.352206+0000","last_change":"2024-04-25T00:14:22.337009+0000","last_active":"2024-04-25T00:14:23.352206+0000","last_peered":"2024-04-25T00:14:23.352206+0000","last_clean":"2024-04-25T00:14:23.352206+0000","last_became_active":"2024-04-25T00:14:22.336671+0000","last_became_peered":"2024-04-25T00:14:22.336671+0000","last_unstale":"2024-04-25T00:14:23.352206+0000","last_undegraded":"2024-04-25T00:14:23.352206+0000","last_fullsized":"2024-04-25T00:14:23.352206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:17.894895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.371707+0000","last_change":"2024-04-25T00:14:23.367167+0000","last_active":"2024-04-25T00:14:23.371707+0000","last_peered":"2024-04-25T00:14:23.371707+0000","last_clean":"2024-04-25T00:14:23.371707+0000","last_became_active":"2024-04-25T00:14:23.366866+0000","last_became_peered":"2024-04-25T00:14:23.366866+0000","last_unstale":"2024-04-25T00:14:23.371707+0000","last_undegraded":"2024-04-25T00:14:23.371707+0000","last_fullsized":"2024-04-25T00:14:23.371707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:36.474335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"81'8","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.388415+0000","last_change":"2024-04-25T00:14:22.348088+0000","last_active":"2024-04-25T00:14:23.388415+0000","last_peered":"2024-04-25T00:14:23.388415+0000","last_clean":"2024-04-25T00:14:23.388415+0000","last_became_active":"2024-04-25T00:14:22.347792+0000","last_became_peered":"2024-04-25T00:14:22.347792+0000","last_unstale":"2024-04-25T00:14:23.388415+0000","last_undegraded":"2024-04-25T00:14:23.388415+0000","last_fullsized":"2024-04-25T00:14:23.388415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:44.314934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362555+0000","last_change":"2024-04-25T00:14:23.370011+0000","last_active":"2024-04-25T00:14:24.362555+0000","last_peered":"2024-04-25T00:14:24.362555+0000","last_clean":"2024-04-25T00:14:24.362555+0000","last_became_active":"2024-04-25T00:14:23.369735+0000","last_became_peered":"2024-04-25T00:14:23.369735+0000","last_unstale":"2024-04-25T00:14:24.362555+0000","last_undegraded":"2024-04-25T00:14:24.362555+0000","last_fullsized":"2024-04-25T00:14:24.362555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:09.031156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.366338+0000","last_change":"2024-04-25T00:14:22.339608+0000","last_active":"2024-04-25T00:14:24.366338+0000","last_peered":"2024-04-25T00:14:24.366338+0000","last_clean":"2024-04-25T00:14:24.366338+0000","last_became_active":"2024-04-25T00:14:22.339299+0000","last_became_peered":"2024-04-25T00:14:22.339299+0000","last_unstale":"2024-04-25T00:14:24.366338+0000","last_undegraded":"2024-04-25T00:14:24.366338+0000","last_fullsized":"2024-04-25T00:14:24.366338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:38.064488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.366087+0000","last_change":"2024-04-25T00:14:23.357737+0000","last_active":"2024-04-25T00:14:24.366087+0000","last_peered":"2024-04-25T00:14:24.366087+0000","last_clean":"2024-04-25T00:14:24.366087+0000","last_became_active":"2024-04-25T00:14:23.357296+0000","last_became_peered":"2024-04-25T00:14:23.357296+0000","last_unstale":"2024-04-25T00:14:24.366087+0000","last_undegraded":"2024-04-25T00:14:24.366087+0000","last_fullsized":"2024-04-25T00:14:24.366087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:18.985534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.388993+0000","last_change":"2024-04-25T00:14:22.348678+0000","last_active":"2024-04-25T00:14:23.388993+0000","last_peered":"2024-04-25T00:14:23.388993+0000","last_clean":"2024-04-25T00:14:23.388993+0000","last_became_active":"2024-04-25T00:14:22.348431+0000","last_became_peered":"2024-04-25T00:14:22.348431+0000","last_unstale":"2024-04-25T00:14:23.388993+0000","last_undegraded":"2024-04-25T00:14:23.388993+0000","last_fullsized":"2024-04-25T00:14:23.388993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:03.623259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365023+0000","last_change":"2024-04-25T00:14:23.365490+0000","last_active":"2024-04-25T00:14:24.365023+0000","last_peered":"2024-04-25T00:14:24.365023+0000","last_clean":"2024-04-25T00:14:24.365023+0000","last_became_active":"2024-04-25T00:14:23.364805+0000","last_became_peered":"2024-04-25T00:14:23.364805+0000","last_unstale":"2024-04-25T00:14:24.365023+0000","last_undegraded":"2024-04-25T00:14:24.365023+0000","last_fullsized":"2024-04-25T00:14:24.365023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:06.045434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365065+0000","last_change":"2024-04-25T00:14:22.341018+0000","last_active":"2024-04-25T00:14:24.365065+0000","last_peered":"2024-04-25T00:14:24.365065+0000","last_clean":"2024-04-25T00:14:24.365065+0000","last_became_active":"2024-04-25T00:14:22.339802+0000","last_became_peered":"2024-04-25T00:14:22.339802+0000","last_unstale":"2024-04-25T00:14:24.365065+0000","last_undegraded":"2024-04-25T00:14:24.365065+0000","last_fullsized":"2024-04-25T00:14:24.365065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:35.839599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362292+0000","last_change":"2024-04-25T00:14:23.374816+0000","last_active":"2024-04-25T00:14:24.362292+0000","last_peered":"2024-04-25T00:14:24.362292+0000","last_clean":"2024-04-25T00:14:24.362292+0000","last_became_active":"2024-04-25T00:14:23.374564+0000","last_became_peered":"2024-04-25T00:14:23.374564+0000","last_unstale":"2024-04-25T00:14:24.362292+0000","last_undegraded":"2024-04-25T00:14:24.362292+0000","last_fullsized":"2024-04-25T00:14:24.362292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:30.903650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368220+0000","last_change":"2024-04-25T00:14:22.345796+0000","last_active":"2024-04-25T00:14:24.368220+0000","last_peered":"2024-04-25T00:14:24.368220+0000","last_clean":"2024-04-25T00:14:24.368220+0000","last_became_active":"2024-04-25T00:14:22.345474+0000","last_became_peered":"2024-04-25T00:14:22.345474+0000","last_unstale":"2024-04-25T00:14:24.368220+0000","last_undegraded":"2024-04-25T00:14:24.368220+0000","last_fullsized":"2024-04-25T00:14:24.368220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:05.040163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368268+0000","last_change":"2024-04-25T00:14:23.364454+0000","last_active":"2024-04-25T00:14:24.368268+0000","last_peered":"2024-04-25T00:14:24.368268+0000","last_clean":"2024-04-25T00:14:24.368268+0000","last_became_active":"2024-04-25T00:14:23.364176+0000","last_became_peered":"2024-04-25T00:14:23.364176+0000","last_unstale":"2024-04-25T00:14:24.368268+0000","last_undegraded":"2024-04-25T00:14:24.368268+0000","last_fullsized":"2024-04-25T00:14:24.368268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:39.545397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.346755+0000","last_change":"2024-04-25T00:14:22.333635+0000","last_active":"2024-04-25T00:14:23.346755+0000","last_peered":"2024-04-25T00:14:23.346755+0000","last_clean":"2024-04-25T00:14:23.346755+0000","last_became_active":"2024-04-25T00:14:22.333349+0000","last_became_peered":"2024-04-25T00:14:22.333349+0000","last_unstale":"2024-04-25T00:14:23.346755+0000","last_undegraded":"2024-04-25T00:14:23.346755+0000","last_fullsized":"2024-04-25T00:14:23.346755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:07.126905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368247+0000","last_change":"2024-04-25T00:14:23.370193+0000","last_active":"2024-04-25T00:14:24.368247+0000","last_peered":"2024-04-25T00:14:24.368247+0000","last_clean":"2024-04-25T00:14:24.368247+0000","last_became_active":"2024-04-25T00:14:23.369877+0000","last_became_peered":"2024-04-25T00:14:23.369877+0000","last_unstale":"2024-04-25T00:14:24.368247+0000","last_undegraded":"2024-04-25T00:14:24.368247+0000","last_fullsized":"2024-04-25T00:14:24.368247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.855680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368342+0000","last_change":"2024-04-25T00:14:22.336426+0000","last_active":"2024-04-25T00:14:24.368342+0000","last_peered":"2024-04-25T00:14:24.368342+0000","last_clean":"2024-04-25T00:14:24.368342+0000","last_became_active":"2024-04-25T00:14:22.336161+0000","last_became_peered":"2024-04-25T00:14:22.336161+0000","last_unstale":"2024-04-25T00:14:24.368342+0000","last_undegraded":"2024-04-25T00:14:24.368342+0000","last_fullsized":"2024-04-25T00:14:24.368342+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:39.032147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362385+0000","last_change":"2024-04-25T00:14:23.368361+0000","last_active":"2024-04-25T00:14:24.362385+0000","last_peered":"2024-04-25T00:14:24.362385+0000","last_clean":"2024-04-25T00:14:24.362385+0000","last_became_active":"2024-04-25T00:14:23.368052+0000","last_became_peered":"2024-04-25T00:14:23.368052+0000","last_unstale":"2024-04-25T00:14:24.362385+0000","last_undegraded":"2024-04-25T00:14:24.362385+0000","last_fullsized":"2024-04-25T00:14:24.362385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:38.819101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.346727+0000","last_change":"2024-04-25T00:14:22.341894+0000","last_active":"2024-04-25T00:14:23.346727+0000","last_peered":"2024-04-25T00:14:23.346727+0000","last_clean":"2024-04-25T00:14:23.346727+0000","last_became_active":"2024-04-25T00:14:22.340075+0000","last_became_peered":"2024-04-25T00:14:22.340075+0000","last_unstale":"2024-04-25T00:14:23.346727+0000","last_undegraded":"2024-04-25T00:14:23.346727+0000","last_fullsized":"2024-04-25T00:14:23.346727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:31.291550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365995+0000","last_change":"2024-04-25T00:14:23.358008+0000","last_active":"2024-04-25T00:14:24.365995+0000","last_peered":"2024-04-25T00:14:24.365995+0000","last_clean":"2024-04-25T00:14:24.365995+0000","last_became_active":"2024-04-25T00:14:23.357682+0000","last_became_peered":"2024-04-25T00:14:23.357682+0000","last_unstale":"2024-04-25T00:14:24.365995+0000","last_undegraded":"2024-04-25T00:14:24.365995+0000","last_fullsized":"2024-04-25T00:14:24.365995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:40.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.388850+0000","last_change":"2024-04-25T00:14:22.340464+0000","last_active":"2024-04-25T00:14:23.388850+0000","last_peered":"2024-04-25T00:14:23.388850+0000","last_clean":"2024-04-25T00:14:23.388850+0000","last_became_active":"2024-04-25T00:14:22.339360+0000","last_became_peered":"2024-04-25T00:14:22.339360+0000","last_unstale":"2024-04-25T00:14:23.388850+0000","last_undegraded":"2024-04-25T00:14:23.388850+0000","last_fullsized":"2024-04-25T00:14:23.388850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:20.850408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362771+0000","last_change":"2024-04-25T00:14:23.375013+0000","last_active":"2024-04-25T00:14:24.362771+0000","last_peered":"2024-04-25T00:14:24.362771+0000","last_clean":"2024-04-25T00:14:24.362771+0000","last_became_active":"2024-04-25T00:14:23.374687+0000","last_became_peered":"2024-04-25T00:14:23.374687+0000","last_unstale":"2024-04-25T00:14:24.362771+0000","last_undegraded":"2024-04-25T00:14:24.362771+0000","last_fullsized":"2024-04-25T00:14:24.362771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:17.343132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.367641+0000","last_change":"2024-04-25T00:14:22.339303+0000","last_active":"2024-04-25T00:14:24.367641+0000","last_peered":"2024-04-25T00:14:24.367641+0000","last_clean":"2024-04-25T00:14:24.367641+0000","last_became_active":"2024-04-25T00:14:22.339037+0000","last_became_peered":"2024-04-25T00:14:22.339037+0000","last_unstale":"2024-04-25T00:14:24.367641+0000","last_undegraded":"2024-04-25T00:14:24.367641+0000","last_fullsized":"2024-04-25T00:14:24.367641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:55.787224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362579+0000","last_change":"2024-04-25T00:14:23.370316+0000","last_active":"2024-04-25T00:14:24.362579+0000","last_peered":"2024-04-25T00:14:24.362579+0000","last_clean":"2024-04-25T00:14:24.362579+0000","last_became_active":"2024-04-25T00:14:23.370071+0000","last_became_peered":"2024-04-25T00:14:23.370071+0000","last_unstale":"2024-04-25T00:14:24.362579+0000","last_undegraded":"2024-04-25T00:14:24.362579+0000","last_fullsized":"2024-04-25T00:14:24.362579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:45.667262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365091+0000","last_change":"2024-04-25T00:14:22.344977+0000","last_active":"2024-04-25T00:14:24.365091+0000","last_peered":"2024-04-25T00:14:24.365091+0000","last_clean":"2024-04-25T00:14:24.365091+0000","last_became_active":"2024-04-25T00:14:22.344692+0000","last_became_peered":"2024-04-25T00:14:22.344692+0000","last_unstale":"2024-04-25T00:14:24.365091+0000","last_undegraded":"2024-04-25T00:14:24.365091+0000","last_fullsized":"2024-04-25T00:14:24.365091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:41.357213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368555+0000","last_change":"2024-04-25T00:14:23.367032+0000","last_active":"2024-04-25T00:14:24.368555+0000","last_peered":"2024-04-25T00:14:24.368555+0000","last_clean":"2024-04-25T00:14:24.368555+0000","last_became_active":"2024-04-25T00:14:23.366783+0000","last_became_peered":"2024-04-25T00:14:23.366783+0000","last_unstale":"2024-04-25T00:14:24.368555+0000","last_undegraded":"2024-04-25T00:14:24.368555+0000","last_fullsized":"2024-04-25T00:14:24.368555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.798029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.366192+0000","last_change":"2024-04-25T00:14:22.334227+0000","last_active":"2024-04-25T00:14:24.366192+0000","last_peered":"2024-04-25T00:14:24.366192+0000","last_clean":"2024-04-25T00:14:24.366192+0000","last_became_active":"2024-04-25T00:14:22.333833+0000","last_became_peered":"2024-04-25T00:14:22.333833+0000","last_unstale":"2024-04-25T00:14:24.366192+0000","last_undegraded":"2024-04-25T00:14:24.366192+0000","last_fullsized":"2024-04-25T00:14:24.366192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:33.572320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.372274+0000","last_change":"2024-04-25T00:14:23.371020+0000","last_active":"2024-04-25T00:14:23.372274+0000","last_peered":"2024-04-25T00:14:23.372274+0000","last_clean":"2024-04-25T00:14:23.372274+0000","last_became_active":"2024-04-25T00:14:23.370728+0000","last_became_peered":"2024-04-25T00:14:23.370728+0000","last_unstale":"2024-04-25T00:14:23.372274+0000","last_undegraded":"2024-04-25T00:14:23.372274+0000","last_fullsized":"2024-04-25T00:14:23.372274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:51.866574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362499+0000","last_change":"2024-04-25T00:14:22.346044+0000","last_active":"2024-04-25T00:14:24.362499+0000","last_peered":"2024-04-25T00:14:24.362499+0000","last_clean":"2024-04-25T00:14:24.362499+0000","last_became_active":"2024-04-25T00:14:22.345794+0000","last_became_peered":"2024-04-25T00:14:22.345794+0000","last_unstale":"2024-04-25T00:14:24.362499+0000","last_undegraded":"2024-04-25T00:14:24.362499+0000","last_fullsized":"2024-04-25T00:14:24.362499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:20.850408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.387185+0000","last_change":"2024-04-25T00:14:23.381001+0000","last_active":"2024-04-25T00:14:23.387185+0000","last_peered":"2024-04-25T00:14:23.387185+0000","last_clean":"2024-04-25T00:14:23.387185+0000","last_became_active":"2024-04-25T00:14:23.380716+0000","last_became_peered":"2024-04-25T00:14:23.380716+0000","last_unstale":"2024-04-25T00:14:23.387185+0000","last_undegraded":"2024-04-25T00:14:23.387185+0000","last_fullsized":"2024-04-25T00:14:23.387185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:44.213908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368264+0000","last_change":"2024-04-25T00:14:22.342686+0000","last_active":"2024-04-25T00:14:24.368264+0000","last_peered":"2024-04-25T00:14:24.368264+0000","last_clean":"2024-04-25T00:14:24.368264+0000","last_became_active":"2024-04-25T00:14:22.342353+0000","last_became_peered":"2024-04-25T00:14:22.342353+0000","last_unstale":"2024-04-25T00:14:24.368264+0000","last_undegraded":"2024-04-25T00:14:24.368264+0000","last_fullsized":"2024-04-25T00:14:24.368264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:16:49.259016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365835+0000","last_change":"2024-04-25T00:14:23.356293+0000","last_active":"2024-04-25T00:14:24.365835+0000","last_peered":"2024-04-25T00:14:24.365835+0000","last_clean":"2024-04-25T00:14:24.365835+0000","last_became_active":"2024-04-25T00:14:23.355930+0000","last_became_peered":"2024-04-25T00:14:23.355930+0000","last_unstale":"2024-04-25T00:14:24.365835+0000","last_undegraded":"2024-04-25T00:14:24.365835+0000","last_fullsized":"2024-04-25T00:14:24.365835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:39.465602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365878+0000","last_change":"2024-04-25T00:14:22.334025+0000","last_active":"2024-04-25T00:14:24.365878+0000","last_peered":"2024-04-25T00:14:24.365878+0000","last_clean":"2024-04-25T00:14:24.365878+0000","last_became_active":"2024-04-25T00:14:22.333627+0000","last_became_peered":"2024-04-25T00:14:22.333627+0000","last_unstale":"2024-04-25T00:14:24.365878+0000","last_undegraded":"2024-04-25T00:14:24.365878+0000","last_fullsized":"2024-04-25T00:14:24.365878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:06.155564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362402+0000","last_change":"2024-04-25T00:14:23.381188+0000","last_active":"2024-04-25T00:14:24.362402+0000","last_peered":"2024-04-25T00:14:24.362402+0000","last_clean":"2024-04-25T00:14:24.362402+0000","last_became_active":"2024-04-25T00:14:23.380913+0000","last_became_peered":"2024-04-25T00:14:23.380913+0000","last_unstale":"2024-04-25T00:14:24.362402+0000","last_undegraded":"2024-04-25T00:14:24.362402+0000","last_fullsized":"2024-04-25T00:14:24.362402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:53.475676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.351095+0000","last_change":"2024-04-25T00:14:22.352063+0000","last_active":"2024-04-25T00:14:23.351095+0000","last_peered":"2024-04-25T00:14:23.351095+0000","last_clean":"2024-04-25T00:14:23.351095+0000","last_became_active":"2024-04-25T00:14:22.351776+0000","last_became_peered":"2024-04-25T00:14:22.351776+0000","last_unstale":"2024-04-25T00:14:23.351095+0000","last_undegraded":"2024-04-25T00:14:23.351095+0000","last_fullsized":"2024-04-25T00:14:23.351095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:06.247583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.382137+0000","last_change":"2024-04-25T00:14:23.375210+0000","last_active":"2024-04-25T00:14:23.382137+0000","last_peered":"2024-04-25T00:14:23.382137+0000","last_clean":"2024-04-25T00:14:23.382137+0000","last_became_active":"2024-04-25T00:14:23.374934+0000","last_became_peered":"2024-04-25T00:14:23.374934+0000","last_unstale":"2024-04-25T00:14:23.382137+0000","last_undegraded":"2024-04-25T00:14:23.382137+0000","last_fullsized":"2024-04-25T00:14:23.382137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:24.423768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.352190+0000","last_change":"2024-04-25T00:14:22.341760+0000","last_active":"2024-04-25T00:14:23.352190+0000","last_peered":"2024-04-25T00:14:23.352190+0000","last_clean":"2024-04-25T00:14:23.352190+0000","last_became_active":"2024-04-25T00:14:22.340148+0000","last_became_peered":"2024-04-25T00:14:22.340148+0000","last_unstale":"2024-04-25T00:14:23.352190+0000","last_undegraded":"2024-04-25T00:14:23.352190+0000","last_fullsized":"2024-04-25T00:14:23.352190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:17.739303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":281,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.351152+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:14:23.351152+0000","last_peered":"2024-04-25T00:14:23.351152+0000","last_clean":"2024-04-25T00:14:23.351152+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:14:23.351152+0000","last_undegraded":"2024-04-25T00:14:23.351152+0000","last_fullsized":"2024-04-25T00:14:23.351152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1601536,"data_stored":1043808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":455600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":453384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":453544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":453384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":453384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1003520,"data_stored":453454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1617920,"data_stored":1046070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:14:27.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:14:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.603+0000 7f5a98200640 1 Processor -- start 2024-04-25T00:14:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.603+0000 7f5a98200640 1 -- start start 2024-04-25T00:14:27.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.603+0000 7f5a98200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5a901599f0 0x7f5a900cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.603+0000 7f5a98200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5a9015c340 0x7f5a9015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.603+0000 7f5a98200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a900ccc60 con 0x7f5a9015ccf0 2024-04-25T00:14:27.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.603+0000 7f5a98200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5a90159dd0 con 0x7f5a9015c340 2024-04-25T00:14:27.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.603+0000 7f5a98200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5a9006dba0 con 0x7f5a901599f0 2024-04-25T00:14:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.607+0000 7f5a94e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5a9015c340 0x7f5a9015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.607+0000 7f5a94e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5a9015c340 0x7f5a9015c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40858/0 (socket says 172.21.15.83:40858) 2024-04-25T00:14:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.607+0000 7f5a94e00640 1 -- 172.21.15.83:0/3648173705 learned_addr learned my addr 172.21.15.83:0/3648173705 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.607+0000 7f5a95800640 1 --2- 172.21.15.83:0/3648173705 >> v2:172.21.15.86:3300/0 conn(0x7f5a901599f0 0x7f5a900cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.607+0000 7f5a94e00640 1 -- 172.21.15.83:0/3648173705 >> v2:172.21.15.86:3300/0 conn(0x7f5a901599f0 msgr2=0x7f5a900cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.607+0000 7f5a94e00640 1 --2- 172.21.15.83:0/3648173705 >> v2:172.21.15.86:3300/0 conn(0x7f5a901599f0 0x7f5a900cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.607+0000 7f5a8fe00640 1 -- 172.21.15.83:0/3648173705 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4138199655 0 0) 0x7f5a880014a0 con 0x7f5a9015ccf0 2024-04-25T00:14:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.607+0000 7f5a94e00640 1 -- 172.21.15.83:0/3648173705 >> v1:172.21.15.83:6789/0 conn(0x7f5a9015ccf0 legacy=0x7f5a9015f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.607+0000 7f5a94e00640 1 -- 172.21.15.83:0/3648173705 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9006de80 con 0x7f5a9015c340 2024-04-25T00:14:27.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.607+0000 7f5a94e00640 1 --2- 172.21.15.83:0/3648173705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9015c340 0x7f5a9015c7b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009870 tx=0x7f5a7c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584262abfbafead9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:27.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.607+0000 7f5a8fe00640 1 -- 172.21.15.83:0/3648173705 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c0b1070 con 0x7f5a9015c340 2024-04-25T00:14:27.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.607+0000 7f5a8fe00640 1 -- 172.21.15.83:0/3648173705 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a7c0ac600 con 0x7f5a9015c340 2024-04-25T00:14:27.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.607+0000 7f5a8fe00640 1 -- 172.21.15.83:0/3648173705 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c0ac9d0 con 0x7f5a9015c340 2024-04-25T00:14:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 -- 172.21.15.83:0/3648173705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9015c340 msgr2=0x7f5a9015c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 --2- 172.21.15.83:0/3648173705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9015c340 0x7f5a9015c7b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009870 tx=0x7f5a7c01c490 comp rx=0 tx=0).stop 2024-04-25T00:14:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 -- 172.21.15.83:0/3648173705 shutdown_connections 2024-04-25T00:14:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 --2- 172.21.15.83:0/3648173705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9015c340 0x7f5a9015c7b0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 --2- 172.21.15.83:0/3648173705 >> v2:172.21.15.86:3300/0 conn(0x7f5a901599f0 0x7f5a900cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 -- 172.21.15.83:0/3648173705 >> 172.21.15.83:0/3648173705 conn(0x7f5a900b1830 msgr2=0x7f5a900b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 -- 172.21.15.83:0/3648173705 shutdown_connections 2024-04-25T00:14:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 -- 172.21.15.83:0/3648173705 wait complete. 2024-04-25T00:14:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 Processor -- start 2024-04-25T00:14:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 -- start start 2024-04-25T00:14:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a901599f0 0x7f5a9013c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9015c340 0x7f5a9013e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a9015ccf0 0x7f5a900b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a900cf870 con 0x7f5a9015c340 2024-04-25T00:14:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a900cf6f0 con 0x7f5a9015ccf0 2024-04-25T00:14:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a98200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5a90168cd0 con 0x7f5a901599f0 2024-04-25T00:14:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a94e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9015c340 0x7f5a9013e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a94e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9015c340 0x7f5a9013e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40862/0 (socket says 172.21.15.83:40862) 2024-04-25T00:14:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a94e00640 1 -- 172.21.15.83:0/3651340563 learned_addr learned my addr 172.21.15.83:0/3651340563 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a94e00640 1 -- 172.21.15.83:0/3651340563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a901599f0 msgr2=0x7f5a9013c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a94e00640 1 --2- 172.21.15.83:0/3651340563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a901599f0 0x7f5a9013c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a94e00640 1 -- 172.21.15.83:0/3651340563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a9015ccf0 msgr2=0x7f5a900b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a94e00640 1 --2- 172.21.15.83:0/3651340563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a9015ccf0 0x7f5a900b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a94e00640 1 -- 172.21.15.83:0/3651340563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a90068440 con 0x7f5a9015c340 2024-04-25T00:14:27.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.611+0000 7f5a94e00640 1 --2- 172.21.15.83:0/3651340563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9015c340 0x7f5a9013e0d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c01a0c0 tx=0x7f5a7c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.615+0000 7f5a8d600640 1 -- 172.21.15.83:0/3651340563 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c0b1040 con 0x7f5a9015c340 2024-04-25T00:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.615+0000 7f5a8d600640 1 -- 172.21.15.83:0/3651340563 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a7c0bc070 con 0x7f5a9015c340 2024-04-25T00:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.615+0000 7f5a8d600640 1 -- 172.21.15.83:0/3651340563 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c0b7690 con 0x7f5a9015c340 2024-04-25T00:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.615+0000 7f5a98200640 1 -- 172.21.15.83:0/3651340563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9013e7c0 con 0x7f5a9015c340 2024-04-25T00:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.615+0000 7f5a98200640 1 -- 172.21.15.83:0/3651340563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a9013ed80 con 0x7f5a9015c340 2024-04-25T00:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.615+0000 7f5a98200640 1 -- 172.21.15.83:0/3651340563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a900737f0 con 0x7f5a9015c340 2024-04-25T00:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.615+0000 7f5a8d600640 1 -- 172.21.15.83:0/3651340563 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5a7c0b7830 con 0x7f5a9015c340 2024-04-25T00:14:27.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.615+0000 7f5a8d600640 1 --2- 172.21.15.83:0/3651340563 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5a700678c0 0x7f5a70069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.615+0000 7f5a8d600640 1 -- 172.21.15.83:0/3651340563 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f5a7c11fb60 con 0x7f5a9015c340 2024-04-25T00:14:27.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.615+0000 7f5a95800640 1 --2- 172.21.15.83:0/3651340563 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5a700678c0 0x7f5a70069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.615+0000 7f5a95800640 1 --2- 172.21.15.83:0/3651340563 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5a700678c0 0x7f5a70069d80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5a84006aa0 tx=0x7f5a84006a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:27.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.619+0000 7f5a8d600640 1 -- 172.21.15.83:0/3651340563 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5a7c0c5020 con 0x7f5a9015c340 2024-04-25T00:14:27.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.775+0000 7f5a98200640 1 -- 172.21.15.83:0/3651340563 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a9004f5f0 con 0x7f5a700678c0 2024-04-25T00:14:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.783+0000 7f5a8d600640 1 -- 172.21.15.83:0/3651340563 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252033 (secure 0 0 0) 0x7f5a9004f5f0 con 0x7f5a700678c0 2024-04-25T00:14:27.788 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:14:27.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.787+0000 7f5a98200640 1 -- 172.21.15.83:0/3651340563 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5a700678c0 msgr2=0x7f5a70069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.787+0000 7f5a98200640 1 --2- 172.21.15.83:0/3651340563 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5a700678c0 0x7f5a70069d80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5a84006aa0 tx=0x7f5a84006a30 comp rx=0 tx=0).stop 2024-04-25T00:14:27.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.787+0000 7f5a98200640 1 -- 172.21.15.83:0/3651340563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9015c340 msgr2=0x7f5a9013e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.787+0000 7f5a98200640 1 --2- 172.21.15.83:0/3651340563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9015c340 0x7f5a9013e0d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c01a0c0 tx=0x7f5a7c01acd0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.787+0000 7f5a98200640 1 -- 172.21.15.83:0/3651340563 shutdown_connections 2024-04-25T00:14:27.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.787+0000 7f5a98200640 1 --2- 172.21.15.83:0/3651340563 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5a700678c0 0x7f5a70069d80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.787+0000 7f5a98200640 1 --2- 172.21.15.83:0/3651340563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a9015ccf0 0x7f5a900b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.787+0000 7f5a98200640 1 --2- 172.21.15.83:0/3651340563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9015c340 0x7f5a9013e0d0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.787+0000 7f5a98200640 1 --2- 172.21.15.83:0/3651340563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a901599f0 0x7f5a9013c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.787+0000 7f5a98200640 1 -- 172.21.15.83:0/3651340563 >> 172.21.15.83:0/3651340563 conn(0x7f5a900b1830 msgr2=0x7f5a9006e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:27.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.787+0000 7f5a98200640 1 -- 172.21.15.83:0/3651340563 shutdown_connections 2024-04-25T00:14:27.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.787+0000 7f5a98200640 1 -- 172.21.15.83:0/3651340563 wait complete. 2024-04-25T00:14:27.809 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-04-25T00:14:26.482423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241856,"kb_used_data":9032,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754732864,"statfs":{"total":773094113280,"available":772846452736,"internally_reserved":0,"allocated":9248768,"data_stored":4812628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.162573"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.374763+0000","last_change":"2024-04-25T00:14:23.372323+0000","last_active":"2024-04-25T00:14:23.374763+0000","last_peered":"2024-04-25T00:14:23.374763+0000","last_clean":"2024-04-25T00:14:23.374763+0000","last_became_active":"2024-04-25T00:14:23.372023+0000","last_became_peered":"2024-04-25T00:14:23.372023+0000","last_unstale":"2024-04-25T00:14:23.374763+0000","last_undegraded":"2024-04-25T00:14:23.374763+0000","last_fullsized":"2024-04-25T00:14:23.374763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:18.241067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368569+0000","last_change":"2024-04-25T00:14:23.364050+0000","last_active":"2024-04-25T00:14:24.368569+0000","last_peered":"2024-04-25T00:14:24.368569+0000","last_clean":"2024-04-25T00:14:24.368569+0000","last_became_active":"2024-04-25T00:14:23.363803+0000","last_became_peered":"2024-04-25T00:14:23.363803+0000","last_unstale":"2024-04-25T00:14:24.368569+0000","last_undegraded":"2024-04-25T00:14:24.368569+0000","last_fullsized":"2024-04-25T00:14:24.368569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:21.815636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.377205+0000","last_change":"2024-04-25T00:14:23.373591+0000","last_active":"2024-04-25T00:14:23.377205+0000","last_peered":"2024-04-25T00:14:23.377205+0000","last_clean":"2024-04-25T00:14:23.377205+0000","last_became_active":"2024-04-25T00:14:23.373186+0000","last_became_peered":"2024-04-25T00:14:23.373186+0000","last_unstale":"2024-04-25T00:14:23.377205+0000","last_undegraded":"2024-04-25T00:14:23.377205+0000","last_fullsized":"2024-04-25T00:14:23.377205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:32.704007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.377741+0000","last_change":"2024-04-25T00:14:23.367538+0000","last_active":"2024-04-25T00:14:23.377741+0000","last_peered":"2024-04-25T00:14:23.377741+0000","last_clean":"2024-04-25T00:14:23.377741+0000","last_became_active":"2024-04-25T00:14:23.367221+0000","last_became_peered":"2024-04-25T00:14:23.367221+0000","last_unstale":"2024-04-25T00:14:23.377741+0000","last_undegraded":"2024-04-25T00:14:23.377741+0000","last_fullsized":"2024-04-25T00:14:23.377741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:07.160836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365202+0000","last_change":"2024-04-25T00:14:23.365270+0000","last_active":"2024-04-25T00:14:24.365202+0000","last_peered":"2024-04-25T00:14:24.365202+0000","last_clean":"2024-04-25T00:14:24.365202+0000","last_became_active":"2024-04-25T00:14:23.364878+0000","last_became_peered":"2024-04-25T00:14:23.364878+0000","last_unstale":"2024-04-25T00:14:24.365202+0000","last_undegraded":"2024-04-25T00:14:24.365202+0000","last_fullsized":"2024-04-25T00:14:24.365202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:39.465602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362129+0000","last_change":"2024-04-25T00:14:23.378640+0000","last_active":"2024-04-25T00:14:24.362129+0000","last_peered":"2024-04-25T00:14:24.362129+0000","last_clean":"2024-04-25T00:14:24.362129+0000","last_became_active":"2024-04-25T00:14:23.378315+0000","last_became_peered":"2024-04-25T00:14:23.378315+0000","last_unstale":"2024-04-25T00:14:24.362129+0000","last_undegraded":"2024-04-25T00:14:24.362129+0000","last_fullsized":"2024-04-25T00:14:24.362129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:21.672662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.380510+0000","last_change":"2024-04-25T00:14:23.375200+0000","last_active":"2024-04-25T00:14:23.380510+0000","last_peered":"2024-04-25T00:14:23.380510+0000","last_clean":"2024-04-25T00:14:23.380510+0000","last_became_active":"2024-04-25T00:14:23.374850+0000","last_became_peered":"2024-04-25T00:14:23.374850+0000","last_unstale":"2024-04-25T00:14:23.380510+0000","last_undegraded":"2024-04-25T00:14:23.380510+0000","last_fullsized":"2024-04-25T00:14:23.380510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:18.241067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.378861+0000","last_change":"2024-04-25T00:14:23.368812+0000","last_active":"2024-04-25T00:14:23.378861+0000","last_peered":"2024-04-25T00:14:23.378861+0000","last_clean":"2024-04-25T00:14:23.378861+0000","last_became_active":"2024-04-25T00:14:23.368194+0000","last_became_peered":"2024-04-25T00:14:23.368194+0000","last_unstale":"2024-04-25T00:14:23.378861+0000","last_undegraded":"2024-04-25T00:14:23.378861+0000","last_fullsized":"2024-04-25T00:14:23.378861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:08.933993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364604+0000","last_change":"2024-04-25T00:14:23.364808+0000","last_active":"2024-04-25T00:14:24.364604+0000","last_peered":"2024-04-25T00:14:24.364604+0000","last_clean":"2024-04-25T00:14:24.364604+0000","last_became_active":"2024-04-25T00:14:23.364485+0000","last_became_peered":"2024-04-25T00:14:23.364485+0000","last_unstale":"2024-04-25T00:14:24.364604+0000","last_undegraded":"2024-04-25T00:14:24.364604+0000","last_fullsized":"2024-04-25T00:14:24.364604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:08.933993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.372335+0000","last_change":"2024-04-25T00:14:23.368462+0000","last_active":"2024-04-25T00:14:23.372335+0000","last_peered":"2024-04-25T00:14:23.372335+0000","last_clean":"2024-04-25T00:14:23.372335+0000","last_became_active":"2024-04-25T00:14:23.368209+0000","last_became_peered":"2024-04-25T00:14:23.368209+0000","last_unstale":"2024-04-25T00:14:23.372335+0000","last_undegraded":"2024-04-25T00:14:23.372335+0000","last_fullsized":"2024-04-25T00:14:23.372335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:55.685973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364719+0000","last_change":"2024-04-25T00:14:23.376839+0000","last_active":"2024-04-25T00:14:24.364719+0000","last_peered":"2024-04-25T00:14:24.364719+0000","last_clean":"2024-04-25T00:14:24.364719+0000","last_became_active":"2024-04-25T00:14:23.376560+0000","last_became_peered":"2024-04-25T00:14:23.376560+0000","last_unstale":"2024-04-25T00:14:24.364719+0000","last_undegraded":"2024-04-25T00:14:24.364719+0000","last_fullsized":"2024-04-25T00:14:24.364719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:14:31.742965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.376808+0000","last_change":"2024-04-25T00:14:23.375797+0000","last_active":"2024-04-25T00:14:23.376808+0000","last_peered":"2024-04-25T00:14:23.376808+0000","last_clean":"2024-04-25T00:14:23.376808+0000","last_became_active":"2024-04-25T00:14:23.375476+0000","last_became_peered":"2024-04-25T00:14:23.375476+0000","last_unstale":"2024-04-25T00:14:23.376808+0000","last_undegraded":"2024-04-25T00:14:23.376808+0000","last_fullsized":"2024-04-25T00:14:23.376808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:14.861158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.369175+0000","last_change":"2024-04-25T00:14:23.364656+0000","last_active":"2024-04-25T00:14:23.369175+0000","last_peered":"2024-04-25T00:14:23.369175+0000","last_clean":"2024-04-25T00:14:23.369175+0000","last_became_active":"2024-04-25T00:14:23.364375+0000","last_became_peered":"2024-04-25T00:14:23.364375+0000","last_unstale":"2024-04-25T00:14:23.369175+0000","last_undegraded":"2024-04-25T00:14:23.369175+0000","last_fullsized":"2024-04-25T00:14:23.369175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:34.577592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364349+0000","last_change":"2024-04-25T00:14:23.373638+0000","last_active":"2024-04-25T00:14:24.364349+0000","last_peered":"2024-04-25T00:14:24.364349+0000","last_clean":"2024-04-25T00:14:24.364349+0000","last_became_active":"2024-04-25T00:14:23.373332+0000","last_became_peered":"2024-04-25T00:14:23.373332+0000","last_unstale":"2024-04-25T00:14:24.364349+0000","last_undegraded":"2024-04-25T00:14:24.364349+0000","last_fullsized":"2024-04-25T00:14:24.364349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:39.069760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.379909+0000","last_change":"2024-04-25T00:14:23.370894+0000","last_active":"2024-04-25T00:14:23.379909+0000","last_peered":"2024-04-25T00:14:23.379909+0000","last_clean":"2024-04-25T00:14:23.379909+0000","last_became_active":"2024-04-25T00:14:23.370562+0000","last_became_peered":"2024-04-25T00:14:23.370562+0000","last_unstale":"2024-04-25T00:14:23.379909+0000","last_undegraded":"2024-04-25T00:14:23.379909+0000","last_fullsized":"2024-04-25T00:14:23.379909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:37.919241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362097+0000","last_change":"2024-04-25T00:14:23.383343+0000","last_active":"2024-04-25T00:14:24.362097+0000","last_peered":"2024-04-25T00:14:24.362097+0000","last_clean":"2024-04-25T00:14:24.362097+0000","last_became_active":"2024-04-25T00:14:23.383106+0000","last_became_peered":"2024-04-25T00:14:23.383106+0000","last_unstale":"2024-04-25T00:14:24.362097+0000","last_undegraded":"2024-04-25T00:14:24.362097+0000","last_fullsized":"2024-04-25T00:14:24.362097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:07.812257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362589+0000","last_change":"2024-04-25T00:14:23.372181+0000","last_active":"2024-04-25T00:14:24.362589+0000","last_peered":"2024-04-25T00:14:24.362589+0000","last_clean":"2024-04-25T00:14:24.362589+0000","last_became_active":"2024-04-25T00:14:23.371910+0000","last_became_peered":"2024-04-25T00:14:23.371910+0000","last_unstale":"2024-04-25T00:14:24.362589+0000","last_undegraded":"2024-04-25T00:14:24.362589+0000","last_fullsized":"2024-04-25T00:14:24.362589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:12.434926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.369256+0000","last_change":"2024-04-25T00:14:23.365671+0000","last_active":"2024-04-25T00:14:23.369256+0000","last_peered":"2024-04-25T00:14:23.369256+0000","last_clean":"2024-04-25T00:14:23.369256+0000","last_became_active":"2024-04-25T00:14:23.365194+0000","last_became_peered":"2024-04-25T00:14:23.365194+0000","last_unstale":"2024-04-25T00:14:23.369256+0000","last_undegraded":"2024-04-25T00:14:23.369256+0000","last_fullsized":"2024-04-25T00:14:23.369256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:27.842724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364875+0000","last_change":"2024-04-25T00:14:23.373531+0000","last_active":"2024-04-25T00:14:24.364875+0000","last_peered":"2024-04-25T00:14:24.364875+0000","last_clean":"2024-04-25T00:14:24.364875+0000","last_became_active":"2024-04-25T00:14:23.373188+0000","last_became_peered":"2024-04-25T00:14:23.373188+0000","last_unstale":"2024-04-25T00:14:24.364875+0000","last_undegraded":"2024-04-25T00:14:24.364875+0000","last_fullsized":"2024-04-25T00:14:24.364875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:16:50.264288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.380453+0000","last_change":"2024-04-25T00:14:23.377562+0000","last_active":"2024-04-25T00:14:23.380453+0000","last_peered":"2024-04-25T00:14:23.380453+0000","last_clean":"2024-04-25T00:14:23.380453+0000","last_became_active":"2024-04-25T00:14:23.377309+0000","last_became_peered":"2024-04-25T00:14:23.377309+0000","last_unstale":"2024-04-25T00:14:23.380453+0000","last_undegraded":"2024-04-25T00:14:23.380453+0000","last_fullsized":"2024-04-25T00:14:23.380453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:11.078353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365418+0000","last_change":"2024-04-25T00:14:23.365665+0000","last_active":"2024-04-25T00:14:24.365418+0000","last_peered":"2024-04-25T00:14:24.365418+0000","last_clean":"2024-04-25T00:14:24.365418+0000","last_became_active":"2024-04-25T00:14:23.365272+0000","last_became_peered":"2024-04-25T00:14:23.365272+0000","last_unstale":"2024-04-25T00:14:24.365418+0000","last_undegraded":"2024-04-25T00:14:24.365418+0000","last_fullsized":"2024-04-25T00:14:24.365418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:40.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364532+0000","last_change":"2024-04-25T00:14:23.378289+0000","last_active":"2024-04-25T00:14:24.364532+0000","last_peered":"2024-04-25T00:14:24.364532+0000","last_clean":"2024-04-25T00:14:24.364532+0000","last_became_active":"2024-04-25T00:14:23.377889+0000","last_became_peered":"2024-04-25T00:14:23.377889+0000","last_unstale":"2024-04-25T00:14:24.364532+0000","last_undegraded":"2024-04-25T00:14:24.364532+0000","last_fullsized":"2024-04-25T00:14:24.364532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:07.160836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365753+0000","last_change":"2024-04-25T00:14:23.356266+0000","last_active":"2024-04-25T00:14:24.365753+0000","last_peered":"2024-04-25T00:14:24.365753+0000","last_clean":"2024-04-25T00:14:24.365753+0000","last_became_active":"2024-04-25T00:14:23.355912+0000","last_became_peered":"2024-04-25T00:14:23.355912+0000","last_unstale":"2024-04-25T00:14:24.365753+0000","last_undegraded":"2024-04-25T00:14:24.365753+0000","last_fullsized":"2024-04-25T00:14:24.365753+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:16:50.264288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.374621+0000","last_change":"2024-04-25T00:14:23.367374+0000","last_active":"2024-04-25T00:14:23.374621+0000","last_peered":"2024-04-25T00:14:23.374621+0000","last_clean":"2024-04-25T00:14:23.374621+0000","last_became_active":"2024-04-25T00:14:23.366227+0000","last_became_peered":"2024-04-25T00:14:23.366227+0000","last_unstale":"2024-04-25T00:14:23.374621+0000","last_undegraded":"2024-04-25T00:14:23.374621+0000","last_fullsized":"2024-04-25T00:14:23.374621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:09.799071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368542+0000","last_change":"2024-04-25T00:14:23.359743+0000","last_active":"2024-04-25T00:14:24.368542+0000","last_peered":"2024-04-25T00:14:24.368542+0000","last_clean":"2024-04-25T00:14:24.368542+0000","last_became_active":"2024-04-25T00:14:23.359158+0000","last_became_peered":"2024-04-25T00:14:23.359158+0000","last_unstale":"2024-04-25T00:14:24.368542+0000","last_undegraded":"2024-04-25T00:14:24.368542+0000","last_fullsized":"2024-04-25T00:14:24.368542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:04.628530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364726+0000","last_change":"2024-04-25T00:14:23.368453+0000","last_active":"2024-04-25T00:14:24.364726+0000","last_peered":"2024-04-25T00:14:24.364726+0000","last_clean":"2024-04-25T00:14:24.364726+0000","last_became_active":"2024-04-25T00:14:23.368154+0000","last_became_peered":"2024-04-25T00:14:23.368154+0000","last_unstale":"2024-04-25T00:14:24.364726+0000","last_undegraded":"2024-04-25T00:14:24.364726+0000","last_fullsized":"2024-04-25T00:14:24.364726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:37.919241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.352873+0000","last_change":"2024-04-25T00:14:23.350710+0000","last_active":"2024-04-25T00:14:23.352873+0000","last_peered":"2024-04-25T00:14:23.352873+0000","last_clean":"2024-04-25T00:14:23.352873+0000","last_became_active":"2024-04-25T00:14:23.350423+0000","last_became_peered":"2024-04-25T00:14:23.350423+0000","last_unstale":"2024-04-25T00:14:23.352873+0000","last_undegraded":"2024-04-25T00:14:23.352873+0000","last_fullsized":"2024-04-25T00:14:23.352873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:20.377876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.379579+0000","last_change":"2024-04-25T00:14:23.369797+0000","last_active":"2024-04-25T00:14:23.379579+0000","last_peered":"2024-04-25T00:14:23.379579+0000","last_clean":"2024-04-25T00:14:23.379579+0000","last_became_active":"2024-04-25T00:14:23.369544+0000","last_became_peered":"2024-04-25T00:14:23.369544+0000","last_unstale":"2024-04-25T00:14:23.379579+0000","last_undegraded":"2024-04-25T00:14:23.379579+0000","last_fullsized":"2024-04-25T00:14:23.379579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:14:31.742965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.363966+0000","last_change":"2024-04-25T00:14:23.361046+0000","last_active":"2024-04-25T00:14:23.363966+0000","last_peered":"2024-04-25T00:14:23.363966+0000","last_clean":"2024-04-25T00:14:23.363966+0000","last_became_active":"2024-04-25T00:14:23.360481+0000","last_became_peered":"2024-04-25T00:14:23.360481+0000","last_unstale":"2024-04-25T00:14:23.363966+0000","last_undegraded":"2024-04-25T00:14:23.363966+0000","last_fullsized":"2024-04-25T00:14:23.363966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:21.815636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364688+0000","last_change":"2024-04-25T00:14:23.367748+0000","last_active":"2024-04-25T00:14:24.364688+0000","last_peered":"2024-04-25T00:14:24.364688+0000","last_clean":"2024-04-25T00:14:24.364688+0000","last_became_active":"2024-04-25T00:14:23.364892+0000","last_became_peered":"2024-04-25T00:14:23.364892+0000","last_unstale":"2024-04-25T00:14:24.364688+0000","last_undegraded":"2024-04-25T00:14:24.364688+0000","last_fullsized":"2024-04-25T00:14:24.364688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:54.268073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368233+0000","last_change":"2024-04-25T00:14:23.366683+0000","last_active":"2024-04-25T00:14:24.368233+0000","last_peered":"2024-04-25T00:14:24.368233+0000","last_clean":"2024-04-25T00:14:24.368233+0000","last_became_active":"2024-04-25T00:14:23.366287+0000","last_became_peered":"2024-04-25T00:14:23.366287+0000","last_unstale":"2024-04-25T00:14:24.368233+0000","last_undegraded":"2024-04-25T00:14:24.368233+0000","last_fullsized":"2024-04-25T00:14:24.368233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:27.227370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362101+0000","last_change":"2024-04-25T00:14:23.371472+0000","last_active":"2024-04-25T00:14:24.362101+0000","last_peered":"2024-04-25T00:14:24.362101+0000","last_clean":"2024-04-25T00:14:24.362101+0000","last_became_active":"2024-04-25T00:14:23.371229+0000","last_became_peered":"2024-04-25T00:14:23.371229+0000","last_unstale":"2024-04-25T00:14:24.362101+0000","last_undegraded":"2024-04-25T00:14:24.362101+0000","last_fullsized":"2024-04-25T00:14:24.362101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:47.177380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362453+0000","last_change":"2024-04-25T00:14:23.369972+0000","last_active":"2024-04-25T00:14:24.362453+0000","last_peered":"2024-04-25T00:14:24.362453+0000","last_clean":"2024-04-25T00:14:24.362453+0000","last_became_active":"2024-04-25T00:14:23.369705+0000","last_became_peered":"2024-04-25T00:14:23.369705+0000","last_unstale":"2024-04-25T00:14:24.362453+0000","last_undegraded":"2024-04-25T00:14:24.362453+0000","last_fullsized":"2024-04-25T00:14:24.362453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:27.570041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.366248+0000","last_change":"2024-04-25T00:14:22.340389+0000","last_active":"2024-04-25T00:14:24.366248+0000","last_peered":"2024-04-25T00:14:24.366248+0000","last_clean":"2024-04-25T00:14:24.366248+0000","last_became_active":"2024-04-25T00:14:22.339975+0000","last_became_peered":"2024-04-25T00:14:22.339975+0000","last_unstale":"2024-04-25T00:14:24.366248+0000","last_undegraded":"2024-04-25T00:14:24.366248+0000","last_fullsized":"2024-04-25T00:14:24.366248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:47.286733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362723+0000","last_change":"2024-04-25T00:14:23.367598+0000","last_active":"2024-04-25T00:14:24.362723+0000","last_peered":"2024-04-25T00:14:24.362723+0000","last_clean":"2024-04-25T00:14:24.362723+0000","last_became_active":"2024-04-25T00:14:23.367295+0000","last_became_peered":"2024-04-25T00:14:23.367295+0000","last_unstale":"2024-04-25T00:14:24.362723+0000","last_undegraded":"2024-04-25T00:14:24.362723+0000","last_fullsized":"2024-04-25T00:14:24.362723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:53.768938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364844+0000","last_change":"2024-04-25T00:14:22.342510+0000","last_active":"2024-04-25T00:14:24.364844+0000","last_peered":"2024-04-25T00:14:24.364844+0000","last_clean":"2024-04-25T00:14:24.364844+0000","last_became_active":"2024-04-25T00:14:22.342193+0000","last_became_peered":"2024-04-25T00:14:22.342193+0000","last_unstale":"2024-04-25T00:14:24.364844+0000","last_undegraded":"2024-04-25T00:14:24.364844+0000","last_fullsized":"2024-04-25T00:14:24.364844+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:34.379489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.370331+0000","last_change":"2024-04-25T00:14:23.368492+0000","last_active":"2024-04-25T00:14:23.370331+0000","last_peered":"2024-04-25T00:14:23.370331+0000","last_clean":"2024-04-25T00:14:23.370331+0000","last_became_active":"2024-04-25T00:14:23.368211+0000","last_became_peered":"2024-04-25T00:14:23.368211+0000","last_unstale":"2024-04-25T00:14:23.370331+0000","last_undegraded":"2024-04-25T00:14:23.370331+0000","last_fullsized":"2024-04-25T00:14:23.370331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:27.842724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.385933+0000","last_change":"2024-04-25T00:14:22.345792+0000","last_active":"2024-04-25T00:14:23.385933+0000","last_peered":"2024-04-25T00:14:23.385933+0000","last_clean":"2024-04-25T00:14:23.385933+0000","last_became_active":"2024-04-25T00:14:22.345288+0000","last_became_peered":"2024-04-25T00:14:22.345288+0000","last_unstale":"2024-04-25T00:14:23.385933+0000","last_undegraded":"2024-04-25T00:14:23.385933+0000","last_fullsized":"2024-04-25T00:14:23.385933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:45.792758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362768+0000","last_change":"2024-04-25T00:14:23.367302+0000","last_active":"2024-04-25T00:14:24.362768+0000","last_peered":"2024-04-25T00:14:24.362768+0000","last_clean":"2024-04-25T00:14:24.362768+0000","last_became_active":"2024-04-25T00:14:23.367026+0000","last_became_peered":"2024-04-25T00:14:23.367026+0000","last_unstale":"2024-04-25T00:14:24.362768+0000","last_undegraded":"2024-04-25T00:14:24.362768+0000","last_fullsized":"2024-04-25T00:14:24.362768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:30.600953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.366101+0000","last_change":"2024-04-25T00:14:22.338007+0000","last_active":"2024-04-25T00:14:24.366101+0000","last_peered":"2024-04-25T00:14:24.366101+0000","last_clean":"2024-04-25T00:14:24.366101+0000","last_became_active":"2024-04-25T00:14:22.337563+0000","last_became_peered":"2024-04-25T00:14:22.337563+0000","last_unstale":"2024-04-25T00:14:24.366101+0000","last_undegraded":"2024-04-25T00:14:24.366101+0000","last_fullsized":"2024-04-25T00:14:24.366101+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:10.436890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.378883+0000","last_change":"2024-04-25T00:14:23.369053+0000","last_active":"2024-04-25T00:14:23.378883+0000","last_peered":"2024-04-25T00:14:23.378883+0000","last_clean":"2024-04-25T00:14:23.378883+0000","last_became_active":"2024-04-25T00:14:23.368533+0000","last_became_peered":"2024-04-25T00:14:23.368533+0000","last_unstale":"2024-04-25T00:14:23.378883+0000","last_undegraded":"2024-04-25T00:14:23.378883+0000","last_fullsized":"2024-04-25T00:14:23.378883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:54.268073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365302+0000","last_change":"2024-04-25T00:14:22.343797+0000","last_active":"2024-04-25T00:14:24.365302+0000","last_peered":"2024-04-25T00:14:24.365302+0000","last_clean":"2024-04-25T00:14:24.365302+0000","last_became_active":"2024-04-25T00:14:22.343077+0000","last_became_peered":"2024-04-25T00:14:22.343077+0000","last_unstale":"2024-04-25T00:14:24.365302+0000","last_undegraded":"2024-04-25T00:14:24.365302+0000","last_fullsized":"2024-04-25T00:14:24.365302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:09.512347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365835+0000","last_change":"2024-04-25T00:14:23.368185+0000","last_active":"2024-04-25T00:14:24.365835+0000","last_peered":"2024-04-25T00:14:24.365835+0000","last_clean":"2024-04-25T00:14:24.365835+0000","last_became_active":"2024-04-25T00:14:23.367850+0000","last_became_peered":"2024-04-25T00:14:23.367850+0000","last_unstale":"2024-04-25T00:14:24.365835+0000","last_undegraded":"2024-04-25T00:14:24.365835+0000","last_fullsized":"2024-04-25T00:14:24.365835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:06.045434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.387449+0000","last_change":"2024-04-25T00:14:22.348575+0000","last_active":"2024-04-25T00:14:23.387449+0000","last_peered":"2024-04-25T00:14:23.387449+0000","last_clean":"2024-04-25T00:14:23.387449+0000","last_became_active":"2024-04-25T00:14:22.346712+0000","last_became_peered":"2024-04-25T00:14:22.346712+0000","last_unstale":"2024-04-25T00:14:23.387449+0000","last_undegraded":"2024-04-25T00:14:23.387449+0000","last_fullsized":"2024-04-25T00:14:23.387449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:44.218808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362150+0000","last_change":"2024-04-25T00:14:23.375834+0000","last_active":"2024-04-25T00:14:24.362150+0000","last_peered":"2024-04-25T00:14:24.362150+0000","last_clean":"2024-04-25T00:14:24.362150+0000","last_became_active":"2024-04-25T00:14:23.375581+0000","last_became_peered":"2024-04-25T00:14:23.375581+0000","last_unstale":"2024-04-25T00:14:24.362150+0000","last_undegraded":"2024-04-25T00:14:24.362150+0000","last_fullsized":"2024-04-25T00:14:24.362150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:35.757421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.358125+0000","last_change":"2024-04-25T00:14:22.349355+0000","last_active":"2024-04-25T00:14:23.358125+0000","last_peered":"2024-04-25T00:14:23.358125+0000","last_clean":"2024-04-25T00:14:23.358125+0000","last_became_active":"2024-04-25T00:14:22.349100+0000","last_became_peered":"2024-04-25T00:14:22.349100+0000","last_unstale":"2024-04-25T00:14:23.358125+0000","last_undegraded":"2024-04-25T00:14:23.358125+0000","last_fullsized":"2024-04-25T00:14:23.358125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:20.810365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.364546+0000","last_change":"2024-04-25T00:14:23.358714+0000","last_active":"2024-04-25T00:14:23.364546+0000","last_peered":"2024-04-25T00:14:23.364546+0000","last_clean":"2024-04-25T00:14:23.364546+0000","last_became_active":"2024-04-25T00:14:23.358427+0000","last_became_peered":"2024-04-25T00:14:23.358427+0000","last_unstale":"2024-04-25T00:14:23.364546+0000","last_undegraded":"2024-04-25T00:14:23.364546+0000","last_fullsized":"2024-04-25T00:14:23.364546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:48.292005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.351787+0000","last_change":"2024-04-25T00:14:22.343802+0000","last_active":"2024-04-25T00:14:23.351787+0000","last_peered":"2024-04-25T00:14:23.351787+0000","last_clean":"2024-04-25T00:14:23.351787+0000","last_became_active":"2024-04-25T00:14:22.343539+0000","last_became_peered":"2024-04-25T00:14:22.343539+0000","last_unstale":"2024-04-25T00:14:23.351787+0000","last_undegraded":"2024-04-25T00:14:23.351787+0000","last_fullsized":"2024-04-25T00:14:23.351787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:38.540125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364819+0000","last_change":"2024-04-25T00:14:23.375118+0000","last_active":"2024-04-25T00:14:24.364819+0000","last_peered":"2024-04-25T00:14:24.364819+0000","last_clean":"2024-04-25T00:14:24.364819+0000","last_became_active":"2024-04-25T00:14:23.374696+0000","last_became_peered":"2024-04-25T00:14:23.374696+0000","last_unstale":"2024-04-25T00:14:24.364819+0000","last_undegraded":"2024-04-25T00:14:24.364819+0000","last_fullsized":"2024-04-25T00:14:24.364819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:11.078353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368613+0000","last_change":"2024-04-25T00:14:22.342276+0000","last_active":"2024-04-25T00:14:24.368613+0000","last_peered":"2024-04-25T00:14:24.368613+0000","last_clean":"2024-04-25T00:14:24.368613+0000","last_became_active":"2024-04-25T00:14:22.341787+0000","last_became_peered":"2024-04-25T00:14:22.341787+0000","last_unstale":"2024-04-25T00:14:24.368613+0000","last_undegraded":"2024-04-25T00:14:24.368613+0000","last_fullsized":"2024-04-25T00:14:24.368613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:38.460331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.371351+0000","last_change":"2024-04-25T00:14:23.369380+0000","last_active":"2024-04-25T00:14:23.371351+0000","last_peered":"2024-04-25T00:14:23.371351+0000","last_clean":"2024-04-25T00:14:23.371351+0000","last_became_active":"2024-04-25T00:14:23.369093+0000","last_became_peered":"2024-04-25T00:14:23.369093+0000","last_unstale":"2024-04-25T00:14:23.371351+0000","last_undegraded":"2024-04-25T00:14:23.371351+0000","last_fullsized":"2024-04-25T00:14:23.371351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:09.799071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.357270+0000","last_change":"2024-04-25T00:14:22.349310+0000","last_active":"2024-04-25T00:14:23.357270+0000","last_peered":"2024-04-25T00:14:23.357270+0000","last_clean":"2024-04-25T00:14:23.357270+0000","last_became_active":"2024-04-25T00:14:22.347510+0000","last_became_peered":"2024-04-25T00:14:22.347510+0000","last_unstale":"2024-04-25T00:14:23.357270+0000","last_undegraded":"2024-04-25T00:14:23.357270+0000","last_fullsized":"2024-04-25T00:14:23.357270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:45.792758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.377858+0000","last_change":"2024-04-25T00:14:23.373096+0000","last_active":"2024-04-25T00:14:23.377858+0000","last_peered":"2024-04-25T00:14:23.377858+0000","last_clean":"2024-04-25T00:14:23.377858+0000","last_became_active":"2024-04-25T00:14:23.372844+0000","last_became_peered":"2024-04-25T00:14:23.372844+0000","last_unstale":"2024-04-25T00:14:23.377858+0000","last_undegraded":"2024-04-25T00:14:23.377858+0000","last_fullsized":"2024-04-25T00:14:23.377858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:24.423768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364984+0000","last_change":"2024-04-25T00:14:22.345472+0000","last_active":"2024-04-25T00:14:24.364984+0000","last_peered":"2024-04-25T00:14:24.364984+0000","last_clean":"2024-04-25T00:14:24.364984+0000","last_became_active":"2024-04-25T00:14:22.344909+0000","last_became_peered":"2024-04-25T00:14:22.344909+0000","last_unstale":"2024-04-25T00:14:24.364984+0000","last_undegraded":"2024-04-25T00:14:24.364984+0000","last_fullsized":"2024-04-25T00:14:24.364984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:14.055977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.364354+0000","last_change":"2024-04-25T00:14:23.360218+0000","last_active":"2024-04-25T00:14:23.364354+0000","last_peered":"2024-04-25T00:14:23.364354+0000","last_clean":"2024-04-25T00:14:23.364354+0000","last_became_active":"2024-04-25T00:14:23.359915+0000","last_became_peered":"2024-04-25T00:14:23.359915+0000","last_unstale":"2024-04-25T00:14:23.364354+0000","last_undegraded":"2024-04-25T00:14:23.364354+0000","last_fullsized":"2024-04-25T00:14:23.364354+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:11.442161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362558+0000","last_change":"2024-04-25T00:14:22.344458+0000","last_active":"2024-04-25T00:14:24.362558+0000","last_peered":"2024-04-25T00:14:24.362558+0000","last_clean":"2024-04-25T00:14:24.362558+0000","last_became_active":"2024-04-25T00:14:22.343093+0000","last_became_peered":"2024-04-25T00:14:22.343093+0000","last_unstale":"2024-04-25T00:14:24.362558+0000","last_undegraded":"2024-04-25T00:14:24.362558+0000","last_fullsized":"2024-04-25T00:14:24.362558+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:38.540125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.369170+0000","last_change":"2024-04-25T00:14:23.364973+0000","last_active":"2024-04-25T00:14:23.369170+0000","last_peered":"2024-04-25T00:14:23.369170+0000","last_clean":"2024-04-25T00:14:23.369170+0000","last_became_active":"2024-04-25T00:14:23.364727+0000","last_became_peered":"2024-04-25T00:14:23.364727+0000","last_unstale":"2024-04-25T00:14:23.369170+0000","last_undegraded":"2024-04-25T00:14:23.369170+0000","last_fullsized":"2024-04-25T00:14:23.369170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:39.069760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.390560+0000","last_change":"2024-04-25T00:14:22.353296+0000","last_active":"2024-04-25T00:14:23.390560+0000","last_peered":"2024-04-25T00:14:23.390560+0000","last_clean":"2024-04-25T00:14:23.390560+0000","last_became_active":"2024-04-25T00:14:22.353015+0000","last_became_peered":"2024-04-25T00:14:22.353015+0000","last_unstale":"2024-04-25T00:14:23.390560+0000","last_undegraded":"2024-04-25T00:14:23.390560+0000","last_fullsized":"2024-04-25T00:14:23.390560+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:26.222098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.364714+0000","last_change":"2024-04-25T00:14:23.364620+0000","last_active":"2024-04-25T00:14:24.364714+0000","last_peered":"2024-04-25T00:14:24.364714+0000","last_clean":"2024-04-25T00:14:24.364714+0000","last_became_active":"2024-04-25T00:14:23.364322+0000","last_became_peered":"2024-04-25T00:14:23.364322+0000","last_unstale":"2024-04-25T00:14:24.364714+0000","last_undegraded":"2024-04-25T00:14:24.364714+0000","last_fullsized":"2024-04-25T00:14:24.364714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:34.577592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362235+0000","last_change":"2024-04-25T00:14:22.349316+0000","last_active":"2024-04-25T00:14:24.362235+0000","last_peered":"2024-04-25T00:14:24.362235+0000","last_clean":"2024-04-25T00:14:24.362235+0000","last_became_active":"2024-04-25T00:14:22.347911+0000","last_became_peered":"2024-04-25T00:14:22.347911+0000","last_unstale":"2024-04-25T00:14:24.362235+0000","last_undegraded":"2024-04-25T00:14:24.362235+0000","last_fullsized":"2024-04-25T00:14:24.362235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:03.623259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.372371+0000","last_change":"2024-04-25T00:14:23.369744+0000","last_active":"2024-04-25T00:14:23.372371+0000","last_peered":"2024-04-25T00:14:23.372371+0000","last_clean":"2024-04-25T00:14:23.372371+0000","last_became_active":"2024-04-25T00:14:23.369484+0000","last_became_peered":"2024-04-25T00:14:23.369484+0000","last_unstale":"2024-04-25T00:14:23.372371+0000","last_undegraded":"2024-04-25T00:14:23.372371+0000","last_fullsized":"2024-04-25T00:14:23.372371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:22.223038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.352206+0000","last_change":"2024-04-25T00:14:22.337009+0000","last_active":"2024-04-25T00:14:23.352206+0000","last_peered":"2024-04-25T00:14:23.352206+0000","last_clean":"2024-04-25T00:14:23.352206+0000","last_became_active":"2024-04-25T00:14:22.336671+0000","last_became_peered":"2024-04-25T00:14:22.336671+0000","last_unstale":"2024-04-25T00:14:23.352206+0000","last_undegraded":"2024-04-25T00:14:23.352206+0000","last_fullsized":"2024-04-25T00:14:23.352206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:17.894895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.371707+0000","last_change":"2024-04-25T00:14:23.367167+0000","last_active":"2024-04-25T00:14:23.371707+0000","last_peered":"2024-04-25T00:14:23.371707+0000","last_clean":"2024-04-25T00:14:23.371707+0000","last_became_active":"2024-04-25T00:14:23.366866+0000","last_became_peered":"2024-04-25T00:14:23.366866+0000","last_unstale":"2024-04-25T00:14:23.371707+0000","last_undegraded":"2024-04-25T00:14:23.371707+0000","last_fullsized":"2024-04-25T00:14:23.371707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:36.474335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"81'8","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.388415+0000","last_change":"2024-04-25T00:14:22.348088+0000","last_active":"2024-04-25T00:14:23.388415+0000","last_peered":"2024-04-25T00:14:23.388415+0000","last_clean":"2024-04-25T00:14:23.388415+0000","last_became_active":"2024-04-25T00:14:22.347792+0000","last_became_peered":"2024-04-25T00:14:22.347792+0000","last_unstale":"2024-04-25T00:14:23.388415+0000","last_undegraded":"2024-04-25T00:14:23.388415+0000","last_fullsized":"2024-04-25T00:14:23.388415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:44.314934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362555+0000","last_change":"2024-04-25T00:14:23.370011+0000","last_active":"2024-04-25T00:14:24.362555+0000","last_peered":"2024-04-25T00:14:24.362555+0000","last_clean":"2024-04-25T00:14:24.362555+0000","last_became_active":"2024-04-25T00:14:23.369735+0000","last_became_peered":"2024-04-25T00:14:23.369735+0000","last_unstale":"2024-04-25T00:14:24.362555+0000","last_undegraded":"2024-04-25T00:14:24.362555+0000","last_fullsized":"2024-04-25T00:14:24.362555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:09.031156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.366338+0000","last_change":"2024-04-25T00:14:22.339608+0000","last_active":"2024-04-25T00:14:24.366338+0000","last_peered":"2024-04-25T00:14:24.366338+0000","last_clean":"2024-04-25T00:14:24.366338+0000","last_became_active":"2024-04-25T00:14:22.339299+0000","last_became_peered":"2024-04-25T00:14:22.339299+0000","last_unstale":"2024-04-25T00:14:24.366338+0000","last_undegraded":"2024-04-25T00:14:24.366338+0000","last_fullsized":"2024-04-25T00:14:24.366338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:38.064488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.366087+0000","last_change":"2024-04-25T00:14:23.357737+0000","last_active":"2024-04-25T00:14:24.366087+0000","last_peered":"2024-04-25T00:14:24.366087+0000","last_clean":"2024-04-25T00:14:24.366087+0000","last_became_active":"2024-04-25T00:14:23.357296+0000","last_became_peered":"2024-04-25T00:14:23.357296+0000","last_unstale":"2024-04-25T00:14:24.366087+0000","last_undegraded":"2024-04-25T00:14:24.366087+0000","last_fullsized":"2024-04-25T00:14:24.366087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:18.985534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.388993+0000","last_change":"2024-04-25T00:14:22.348678+0000","last_active":"2024-04-25T00:14:23.388993+0000","last_peered":"2024-04-25T00:14:23.388993+0000","last_clean":"2024-04-25T00:14:23.388993+0000","last_became_active":"2024-04-25T00:14:22.348431+0000","last_became_peered":"2024-04-25T00:14:22.348431+0000","last_unstale":"2024-04-25T00:14:23.388993+0000","last_undegraded":"2024-04-25T00:14:23.388993+0000","last_fullsized":"2024-04-25T00:14:23.388993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:03.623259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365023+0000","last_change":"2024-04-25T00:14:23.365490+0000","last_active":"2024-04-25T00:14:24.365023+0000","last_peered":"2024-04-25T00:14:24.365023+0000","last_clean":"2024-04-25T00:14:24.365023+0000","last_became_active":"2024-04-25T00:14:23.364805+0000","last_became_peered":"2024-04-25T00:14:23.364805+0000","last_unstale":"2024-04-25T00:14:24.365023+0000","last_undegraded":"2024-04-25T00:14:24.365023+0000","last_fullsized":"2024-04-25T00:14:24.365023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:06.045434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365065+0000","last_change":"2024-04-25T00:14:22.341018+0000","last_active":"2024-04-25T00:14:24.365065+0000","last_peered":"2024-04-25T00:14:24.365065+0000","last_clean":"2024-04-25T00:14:24.365065+0000","last_became_active":"2024-04-25T00:14:22.339802+0000","last_became_peered":"2024-04-25T00:14:22.339802+0000","last_unstale":"2024-04-25T00:14:24.365065+0000","last_undegraded":"2024-04-25T00:14:24.365065+0000","last_fullsized":"2024-04-25T00:14:24.365065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:35.839599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362292+0000","last_change":"2024-04-25T00:14:23.374816+0000","last_active":"2024-04-25T00:14:24.362292+0000","last_peered":"2024-04-25T00:14:24.362292+0000","last_clean":"2024-04-25T00:14:24.362292+0000","last_became_active":"2024-04-25T00:14:23.374564+0000","last_became_peered":"2024-04-25T00:14:23.374564+0000","last_unstale":"2024-04-25T00:14:24.362292+0000","last_undegraded":"2024-04-25T00:14:24.362292+0000","last_fullsized":"2024-04-25T00:14:24.362292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:17:30.903650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368220+0000","last_change":"2024-04-25T00:14:22.345796+0000","last_active":"2024-04-25T00:14:24.368220+0000","last_peered":"2024-04-25T00:14:24.368220+0000","last_clean":"2024-04-25T00:14:24.368220+0000","last_became_active":"2024-04-25T00:14:22.345474+0000","last_became_peered":"2024-04-25T00:14:22.345474+0000","last_unstale":"2024-04-25T00:14:24.368220+0000","last_undegraded":"2024-04-25T00:14:24.368220+0000","last_fullsized":"2024-04-25T00:14:24.368220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:05.040163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368268+0000","last_change":"2024-04-25T00:14:23.364454+0000","last_active":"2024-04-25T00:14:24.368268+0000","last_peered":"2024-04-25T00:14:24.368268+0000","last_clean":"2024-04-25T00:14:24.368268+0000","last_became_active":"2024-04-25T00:14:23.364176+0000","last_became_peered":"2024-04-25T00:14:23.364176+0000","last_unstale":"2024-04-25T00:14:24.368268+0000","last_undegraded":"2024-04-25T00:14:24.368268+0000","last_fullsized":"2024-04-25T00:14:24.368268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:39.545397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.346755+0000","last_change":"2024-04-25T00:14:22.333635+0000","last_active":"2024-04-25T00:14:23.346755+0000","last_peered":"2024-04-25T00:14:23.346755+0000","last_clean":"2024-04-25T00:14:23.346755+0000","last_became_active":"2024-04-25T00:14:22.333349+0000","last_became_peered":"2024-04-25T00:14:22.333349+0000","last_unstale":"2024-04-25T00:14:23.346755+0000","last_undegraded":"2024-04-25T00:14:23.346755+0000","last_fullsized":"2024-04-25T00:14:23.346755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:07.126905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368247+0000","last_change":"2024-04-25T00:14:23.370193+0000","last_active":"2024-04-25T00:14:24.368247+0000","last_peered":"2024-04-25T00:14:24.368247+0000","last_clean":"2024-04-25T00:14:24.368247+0000","last_became_active":"2024-04-25T00:14:23.369877+0000","last_became_peered":"2024-04-25T00:14:23.369877+0000","last_unstale":"2024-04-25T00:14:24.368247+0000","last_undegraded":"2024-04-25T00:14:24.368247+0000","last_fullsized":"2024-04-25T00:14:24.368247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:21.855680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368342+0000","last_change":"2024-04-25T00:14:22.336426+0000","last_active":"2024-04-25T00:14:24.368342+0000","last_peered":"2024-04-25T00:14:24.368342+0000","last_clean":"2024-04-25T00:14:24.368342+0000","last_became_active":"2024-04-25T00:14:22.336161+0000","last_became_peered":"2024-04-25T00:14:22.336161+0000","last_unstale":"2024-04-25T00:14:24.368342+0000","last_undegraded":"2024-04-25T00:14:24.368342+0000","last_fullsized":"2024-04-25T00:14:24.368342+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:39.032147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362385+0000","last_change":"2024-04-25T00:14:23.368361+0000","last_active":"2024-04-25T00:14:24.362385+0000","last_peered":"2024-04-25T00:14:24.362385+0000","last_clean":"2024-04-25T00:14:24.362385+0000","last_became_active":"2024-04-25T00:14:23.368052+0000","last_became_peered":"2024-04-25T00:14:23.368052+0000","last_unstale":"2024-04-25T00:14:24.362385+0000","last_undegraded":"2024-04-25T00:14:24.362385+0000","last_fullsized":"2024-04-25T00:14:24.362385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:38.819101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.346727+0000","last_change":"2024-04-25T00:14:22.341894+0000","last_active":"2024-04-25T00:14:23.346727+0000","last_peered":"2024-04-25T00:14:23.346727+0000","last_clean":"2024-04-25T00:14:23.346727+0000","last_became_active":"2024-04-25T00:14:22.340075+0000","last_became_peered":"2024-04-25T00:14:22.340075+0000","last_unstale":"2024-04-25T00:14:23.346727+0000","last_undegraded":"2024-04-25T00:14:23.346727+0000","last_fullsized":"2024-04-25T00:14:23.346727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:31.291550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365995+0000","last_change":"2024-04-25T00:14:23.358008+0000","last_active":"2024-04-25T00:14:24.365995+0000","last_peered":"2024-04-25T00:14:24.365995+0000","last_clean":"2024-04-25T00:14:24.365995+0000","last_became_active":"2024-04-25T00:14:23.357682+0000","last_became_peered":"2024-04-25T00:14:23.357682+0000","last_unstale":"2024-04-25T00:14:24.365995+0000","last_undegraded":"2024-04-25T00:14:24.365995+0000","last_fullsized":"2024-04-25T00:14:24.365995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:40.037419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.388850+0000","last_change":"2024-04-25T00:14:22.340464+0000","last_active":"2024-04-25T00:14:23.388850+0000","last_peered":"2024-04-25T00:14:23.388850+0000","last_clean":"2024-04-25T00:14:23.388850+0000","last_became_active":"2024-04-25T00:14:22.339360+0000","last_became_peered":"2024-04-25T00:14:22.339360+0000","last_unstale":"2024-04-25T00:14:23.388850+0000","last_undegraded":"2024-04-25T00:14:23.388850+0000","last_fullsized":"2024-04-25T00:14:23.388850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:20.850408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362771+0000","last_change":"2024-04-25T00:14:23.375013+0000","last_active":"2024-04-25T00:14:24.362771+0000","last_peered":"2024-04-25T00:14:24.362771+0000","last_clean":"2024-04-25T00:14:24.362771+0000","last_became_active":"2024-04-25T00:14:23.374687+0000","last_became_peered":"2024-04-25T00:14:23.374687+0000","last_unstale":"2024-04-25T00:14:24.362771+0000","last_undegraded":"2024-04-25T00:14:24.362771+0000","last_fullsized":"2024-04-25T00:14:24.362771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:17.343132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.367641+0000","last_change":"2024-04-25T00:14:22.339303+0000","last_active":"2024-04-25T00:14:24.367641+0000","last_peered":"2024-04-25T00:14:24.367641+0000","last_clean":"2024-04-25T00:14:24.367641+0000","last_became_active":"2024-04-25T00:14:22.339037+0000","last_became_peered":"2024-04-25T00:14:22.339037+0000","last_unstale":"2024-04-25T00:14:24.367641+0000","last_undegraded":"2024-04-25T00:14:24.367641+0000","last_fullsized":"2024-04-25T00:14:24.367641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:55.787224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362579+0000","last_change":"2024-04-25T00:14:23.370316+0000","last_active":"2024-04-25T00:14:24.362579+0000","last_peered":"2024-04-25T00:14:24.362579+0000","last_clean":"2024-04-25T00:14:24.362579+0000","last_became_active":"2024-04-25T00:14:23.370071+0000","last_became_peered":"2024-04-25T00:14:23.370071+0000","last_unstale":"2024-04-25T00:14:24.362579+0000","last_undegraded":"2024-04-25T00:14:24.362579+0000","last_fullsized":"2024-04-25T00:14:24.362579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:45.667262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365091+0000","last_change":"2024-04-25T00:14:22.344977+0000","last_active":"2024-04-25T00:14:24.365091+0000","last_peered":"2024-04-25T00:14:24.365091+0000","last_clean":"2024-04-25T00:14:24.365091+0000","last_became_active":"2024-04-25T00:14:22.344692+0000","last_became_peered":"2024-04-25T00:14:22.344692+0000","last_unstale":"2024-04-25T00:14:24.365091+0000","last_undegraded":"2024-04-25T00:14:24.365091+0000","last_fullsized":"2024-04-25T00:14:24.365091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:41.357213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368555+0000","last_change":"2024-04-25T00:14:23.367032+0000","last_active":"2024-04-25T00:14:24.368555+0000","last_peered":"2024-04-25T00:14:24.368555+0000","last_clean":"2024-04-25T00:14:24.368555+0000","last_became_active":"2024-04-25T00:14:23.366783+0000","last_became_peered":"2024-04-25T00:14:23.366783+0000","last_unstale":"2024-04-25T00:14:24.368555+0000","last_undegraded":"2024-04-25T00:14:24.368555+0000","last_fullsized":"2024-04-25T00:14:24.368555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.798029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.366192+0000","last_change":"2024-04-25T00:14:22.334227+0000","last_active":"2024-04-25T00:14:24.366192+0000","last_peered":"2024-04-25T00:14:24.366192+0000","last_clean":"2024-04-25T00:14:24.366192+0000","last_became_active":"2024-04-25T00:14:22.333833+0000","last_became_peered":"2024-04-25T00:14:22.333833+0000","last_unstale":"2024-04-25T00:14:24.366192+0000","last_undegraded":"2024-04-25T00:14:24.366192+0000","last_fullsized":"2024-04-25T00:14:24.366192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:33.572320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.372274+0000","last_change":"2024-04-25T00:14:23.371020+0000","last_active":"2024-04-25T00:14:23.372274+0000","last_peered":"2024-04-25T00:14:23.372274+0000","last_clean":"2024-04-25T00:14:23.372274+0000","last_became_active":"2024-04-25T00:14:23.370728+0000","last_became_peered":"2024-04-25T00:14:23.370728+0000","last_unstale":"2024-04-25T00:14:23.372274+0000","last_undegraded":"2024-04-25T00:14:23.372274+0000","last_fullsized":"2024-04-25T00:14:23.372274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:51.866574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362499+0000","last_change":"2024-04-25T00:14:22.346044+0000","last_active":"2024-04-25T00:14:24.362499+0000","last_peered":"2024-04-25T00:14:24.362499+0000","last_clean":"2024-04-25T00:14:24.362499+0000","last_became_active":"2024-04-25T00:14:22.345794+0000","last_became_peered":"2024-04-25T00:14:22.345794+0000","last_unstale":"2024-04-25T00:14:24.362499+0000","last_undegraded":"2024-04-25T00:14:24.362499+0000","last_fullsized":"2024-04-25T00:14:24.362499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:20.850408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.387185+0000","last_change":"2024-04-25T00:14:23.381001+0000","last_active":"2024-04-25T00:14:23.387185+0000","last_peered":"2024-04-25T00:14:23.387185+0000","last_clean":"2024-04-25T00:14:23.387185+0000","last_became_active":"2024-04-25T00:14:23.380716+0000","last_became_peered":"2024-04-25T00:14:23.380716+0000","last_unstale":"2024-04-25T00:14:23.387185+0000","last_undegraded":"2024-04-25T00:14:23.387185+0000","last_fullsized":"2024-04-25T00:14:23.387185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:44.213908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.368264+0000","last_change":"2024-04-25T00:14:22.342686+0000","last_active":"2024-04-25T00:14:24.368264+0000","last_peered":"2024-04-25T00:14:24.368264+0000","last_clean":"2024-04-25T00:14:24.368264+0000","last_became_active":"2024-04-25T00:14:22.342353+0000","last_became_peered":"2024-04-25T00:14:22.342353+0000","last_unstale":"2024-04-25T00:14:24.368264+0000","last_undegraded":"2024-04-25T00:14:24.368264+0000","last_fullsized":"2024-04-25T00:14:24.368264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:16:49.259016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365835+0000","last_change":"2024-04-25T00:14:23.356293+0000","last_active":"2024-04-25T00:14:24.365835+0000","last_peered":"2024-04-25T00:14:24.365835+0000","last_clean":"2024-04-25T00:14:24.365835+0000","last_became_active":"2024-04-25T00:14:23.355930+0000","last_became_peered":"2024-04-25T00:14:23.355930+0000","last_unstale":"2024-04-25T00:14:24.365835+0000","last_undegraded":"2024-04-25T00:14:24.365835+0000","last_fullsized":"2024-04-25T00:14:24.365835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:39.465602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.365878+0000","last_change":"2024-04-25T00:14:22.334025+0000","last_active":"2024-04-25T00:14:24.365878+0000","last_peered":"2024-04-25T00:14:24.365878+0000","last_clean":"2024-04-25T00:14:24.365878+0000","last_became_active":"2024-04-25T00:14:22.333627+0000","last_became_peered":"2024-04-25T00:14:22.333627+0000","last_unstale":"2024-04-25T00:14:24.365878+0000","last_undegraded":"2024-04-25T00:14:24.365878+0000","last_fullsized":"2024-04-25T00:14:24.365878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:06.155564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T00:14:24.362402+0000","last_change":"2024-04-25T00:14:23.381188+0000","last_active":"2024-04-25T00:14:24.362402+0000","last_peered":"2024-04-25T00:14:24.362402+0000","last_clean":"2024-04-25T00:14:24.362402+0000","last_became_active":"2024-04-25T00:14:23.380913+0000","last_became_peered":"2024-04-25T00:14:23.380913+0000","last_unstale":"2024-04-25T00:14:24.362402+0000","last_undegraded":"2024-04-25T00:14:24.362402+0000","last_fullsized":"2024-04-25T00:14:24.362402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:53.475676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.351095+0000","last_change":"2024-04-25T00:14:22.352063+0000","last_active":"2024-04-25T00:14:23.351095+0000","last_peered":"2024-04-25T00:14:23.351095+0000","last_clean":"2024-04-25T00:14:23.351095+0000","last_became_active":"2024-04-25T00:14:22.351776+0000","last_became_peered":"2024-04-25T00:14:22.351776+0000","last_unstale":"2024-04-25T00:14:23.351095+0000","last_undegraded":"2024-04-25T00:14:23.351095+0000","last_fullsized":"2024-04-25T00:14:23.351095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:06.247583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.382137+0000","last_change":"2024-04-25T00:14:23.375210+0000","last_active":"2024-04-25T00:14:23.382137+0000","last_peered":"2024-04-25T00:14:23.382137+0000","last_clean":"2024-04-25T00:14:23.382137+0000","last_became_active":"2024-04-25T00:14:23.374934+0000","last_became_peered":"2024-04-25T00:14:23.374934+0000","last_unstale":"2024-04-25T00:14:23.382137+0000","last_undegraded":"2024-04-25T00:14:23.382137+0000","last_fullsized":"2024-04-25T00:14:23.382137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:22.319144+0000","last_clean_scrub_stamp":"2024-04-25T00:14:22.319144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:24.423768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.352190+0000","last_change":"2024-04-25T00:14:22.341760+0000","last_active":"2024-04-25T00:14:23.352190+0000","last_peered":"2024-04-25T00:14:23.352190+0000","last_clean":"2024-04-25T00:14:23.352190+0000","last_became_active":"2024-04-25T00:14:22.340148+0000","last_became_peered":"2024-04-25T00:14:22.340148+0000","last_unstale":"2024-04-25T00:14:23.352190+0000","last_undegraded":"2024-04-25T00:14:23.352190+0000","last_fullsized":"2024-04-25T00:14:23.352190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:21.313872+0000","last_clean_scrub_stamp":"2024-04-25T00:14:21.313872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:17.739303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":281,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T00:14:23.351152+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:14:23.351152+0000","last_peered":"2024-04-25T00:14:23.351152+0000","last_clean":"2024-04-25T00:14:23.351152+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:14:23.351152+0000","last_undegraded":"2024-04-25T00:14:23.351152+0000","last_fullsized":"2024-04-25T00:14:23.351152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1601536,"data_stored":1043808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":455600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":453384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":453544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":453384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":453384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1003520,"data_stored":453454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1617920,"data_stored":1046070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:14:27.813 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:14:27.813 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:14:27.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.995+0000 7f1283600640 1 Processor -- start 2024-04-25T00:14:27.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.995+0000 7f1283600640 1 -- start start 2024-04-25T00:14:27.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.995+0000 7f1283600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f127c160020 0x7f127c166ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.995+0000 7f1283600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f127c0ce000 con 0x7f127c159810 2024-04-25T00:14:27.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.995+0000 7f1283600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f127c06eec0 con 0x7f127c06e950 2024-04-25T00:14:27.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.995+0000 7f1283600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f127c06e210 con 0x7f127c160020 2024-04-25T00:14:27.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.995+0000 7f1280c00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f127c159810 0x7f127c0cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41118/0 (socket says 172.21.15.83:41118) 2024-04-25T00:14:27.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.995+0000 7f1280c00640 1 -- 172.21.15.83:0/2882132168 learned_addr learned my addr 172.21.15.83:0/2882132168 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:27.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.995+0000 7f1281600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f127c160020 0x7f127c166ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.995+0000 7f1281600640 1 -- 172.21.15.83:0/2882132168 >> v1:172.21.15.86:6789/0 conn(0x7f127c06e950 legacy=0x7f127c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:28.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.995+0000 7f1281600640 1 -- 172.21.15.83:0/2882132168 >> v1:172.21.15.83:6789/0 conn(0x7f127c159810 legacy=0x7f127c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.995+0000 7f127b400640 1 -- 172.21.15.83:0/2882132168 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3285799027 0 0) 0x7f127c0ce000 con 0x7f127c159810 2024-04-25T00:14:28.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.995+0000 7f1281600640 1 -- 172.21.15.83:0/2882132168 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127c06dc50 con 0x7f127c160020 2024-04-25T00:14:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.999+0000 7f1281600640 1 --2- 172.21.15.83:0/2882132168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f127c160020 0x7f127c166ca0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1274009d90 tx=0x7f127401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8843278dc0cfa60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.999+0000 7f127b400640 1 -- 172.21.15.83:0/2882132168 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740b2070 con 0x7f127c160020 2024-04-25T00:14:28.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.999+0000 7f127b400640 1 -- 172.21.15.83:0/2882132168 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12740a5870 con 0x7f127c160020 2024-04-25T00:14:28.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.999+0000 7f127b400640 1 -- 172.21.15.83:0/2882132168 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740a5c40 con 0x7f127c160020 2024-04-25T00:14:28.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.999+0000 7f1283600640 1 -- 172.21.15.83:0/2882132168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f127c160020 msgr2=0x7f127c166ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.999+0000 7f1283600640 1 --2- 172.21.15.83:0/2882132168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f127c160020 0x7f127c166ca0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1274009d90 tx=0x7f127401c8a0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.999+0000 7f1283600640 1 -- 172.21.15.83:0/2882132168 shutdown_connections 2024-04-25T00:14:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.999+0000 7f1283600640 1 --2- 172.21.15.83:0/2882132168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f127c160020 0x7f127c166ca0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.999+0000 7f1283600640 1 -- 172.21.15.83:0/2882132168 >> 172.21.15.83:0/2882132168 conn(0x7f127c0b1830 msgr2=0x7f127c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.999+0000 7f1283600640 1 -- 172.21.15.83:0/2882132168 shutdown_connections 2024-04-25T00:14:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.999+0000 7f1283600640 1 -- 172.21.15.83:0/2882132168 wait complete. 2024-04-25T00:14:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.999+0000 7f1283600640 1 Processor -- start 2024-04-25T00:14:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:27.999+0000 7f1283600640 1 -- start start 2024-04-25T00:14:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1283600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f127c06e950 0x7f127c150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1283600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f127c159810 0x7f127c150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1283600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f127c160020 0x7f127c1fa570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1283600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f127c169b80 con 0x7f127c160020 2024-04-25T00:14:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1283600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f127c169a00 con 0x7f127c06e950 2024-04-25T00:14:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1283600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f127c169d00 con 0x7f127c159810 2024-04-25T00:14:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1281600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f127c160020 0x7f127c1fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1281600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f127c160020 0x7f127c1fa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40872/0 (socket says 172.21.15.83:40872) 2024-04-25T00:14:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1281600640 1 -- 172.21.15.83:0/2170241457 learned_addr learned my addr 172.21.15.83:0/2170241457 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1280c00640 1 --2- 172.21.15.83:0/2170241457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f127c06e950 0x7f127c150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f127be00640 1 --2- 172.21.15.83:0/2170241457 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f127c159810 0x7f127c150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1281600640 1 -- 172.21.15.83:0/2170241457 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f127c159810 msgr2=0x7f127c150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1281600640 1 --2- 172.21.15.83:0/2170241457 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f127c159810 0x7f127c150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1281600640 1 -- 172.21.15.83:0/2170241457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f127c06e950 msgr2=0x7f127c150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1281600640 1 --2- 172.21.15.83:0/2170241457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f127c06e950 0x7f127c150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1281600640 1 -- 172.21.15.83:0/2170241457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127c156430 con 0x7f127c160020 2024-04-25T00:14:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1281600640 1 --2- 172.21.15.83:0/2170241457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f127c160020 0x7f127c1fa570 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f12740b0040 tx=0x7f1274019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:28.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1278c00640 1 -- 172.21.15.83:0/2170241457 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740bd020 con 0x7f127c160020 2024-04-25T00:14:28.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1278c00640 1 -- 172.21.15.83:0/2170241457 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12740b8670 con 0x7f127c160020 2024-04-25T00:14:28.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1278c00640 1 -- 172.21.15.83:0/2170241457 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740b2040 con 0x7f127c160020 2024-04-25T00:14:28.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1283600640 1 -- 172.21.15.83:0/2170241457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f127c0b2f00 con 0x7f127c160020 2024-04-25T00:14:28.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1283600640 1 -- 172.21.15.83:0/2170241457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f127c155590 con 0x7f127c160020 2024-04-25T00:14:28.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.003+0000 7f1283600640 1 -- 172.21.15.83:0/2170241457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f127c04f510 con 0x7f127c160020 2024-04-25T00:14:28.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.007+0000 7f1278c00640 1 -- 172.21.15.83:0/2170241457 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f12740c0020 con 0x7f127c160020 2024-04-25T00:14:28.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.007+0000 7f1278c00640 1 --2- 172.21.15.83:0/2170241457 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f125c0678c0 0x7f125c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.007+0000 7f1278c00640 1 -- 172.21.15.83:0/2170241457 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f1274120cf0 con 0x7f127c160020 2024-04-25T00:14:28.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.007+0000 7f1280c00640 1 --2- 172.21.15.83:0/2170241457 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f125c0678c0 0x7f125c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.007+0000 7f1280c00640 1 --2- 172.21.15.83:0/2170241457 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f125c0678c0 0x7f125c069d80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1270000f70 tx=0x7f1270007b80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:28.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.011+0000 7f1278c00640 1 -- 172.21.15.83:0/2170241457 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1274125070 con 0x7f127c160020 2024-04-25T00:14:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.219+0000 7f1283600640 1 -- 172.21.15.83:0/2170241457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f127c072c90 con 0x7f127c160020 2024-04-25T00:14:28.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.223+0000 7f1278c00640 1 -- 172.21.15.83:0/2170241457 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f12740c7030 con 0x7f127c160020 2024-04-25T00:14:28.226 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-25T00:14:28.227 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylmOqooLBAApte+X6YkRmrpe12YdjYekg== 2024-04-25T00:14:28.227 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-25T00:14:28.227 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-25T00:14:28.227 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:14:28.227 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-25T00:14:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.227+0000 7f1283600640 1 -- 172.21.15.83:0/2170241457 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f125c0678c0 msgr2=0x7f125c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.227+0000 7f1283600640 1 --2- 172.21.15.83:0/2170241457 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f125c0678c0 0x7f125c069d80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1270000f70 tx=0x7f1270007b80 comp rx=0 tx=0).stop 2024-04-25T00:14:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.227+0000 7f1283600640 1 -- 172.21.15.83:0/2170241457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f127c160020 msgr2=0x7f127c1fa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.227+0000 7f1283600640 1 --2- 172.21.15.83:0/2170241457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f127c160020 0x7f127c1fa570 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f12740b0040 tx=0x7f1274019640 comp rx=0 tx=0).stop 2024-04-25T00:14:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.227+0000 7f1283600640 1 -- 172.21.15.83:0/2170241457 shutdown_connections 2024-04-25T00:14:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.227+0000 7f1283600640 1 --2- 172.21.15.83:0/2170241457 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f125c0678c0 0x7f125c069d80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.227+0000 7f1283600640 1 --2- 172.21.15.83:0/2170241457 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f127c159810 0x7f127c150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.227+0000 7f1283600640 1 --2- 172.21.15.83:0/2170241457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f127c160020 0x7f127c1fa570 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.227+0000 7f1283600640 1 --2- 172.21.15.83:0/2170241457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f127c06e950 0x7f127c150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.227+0000 7f1283600640 1 -- 172.21.15.83:0/2170241457 >> 172.21.15.83:0/2170241457 conn(0x7f127c0b1830 msgr2=0x7f127c0adb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.227+0000 7f1283600640 1 -- 172.21.15.83:0/2170241457 shutdown_connections 2024-04-25T00:14:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.227+0000 7f1283600640 1 -- 172.21.15.83:0/2170241457 wait complete. 2024-04-25T00:14:28.249 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:28.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59e6000640 1 Processor -- start 2024-04-25T00:14:28.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59e6000640 1 -- start start 2024-04-25T00:14:28.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59e6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f59e015f940 0x7f59e00cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59e6000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59e00aebf0 con 0x7f59e015f560 2024-04-25T00:14:28.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59e6000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59e00aeed0 con 0x7f59e00d03b0 2024-04-25T00:14:28.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59e6000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f59e00b36d0 con 0x7f59e015f940 2024-04-25T00:14:28.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59df400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f59e015f560 0x7f59e00b2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41134/0 (socket says 172.21.15.83:41134) 2024-04-25T00:14:28.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59df400640 1 -- 172.21.15.83:0/3226507157 learned_addr learned my addr 172.21.15.83:0/3226507157 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:28.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59d6a00640 1 --2- 172.21.15.83:0/3226507157 >> v2:172.21.15.86:3300/0 conn(0x7f59e015f940 0x7f59e00cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59dea00640 1 -- 172.21.15.83:0/3226507157 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1490105835 0 0) 0x7f59e00aebf0 con 0x7f59e015f560 2024-04-25T00:14:28.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59dea00640 1 -- 172.21.15.83:0/3226507157 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59c4003530 con 0x7f59e015f560 2024-04-25T00:14:28.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59dea00640 1 -- 172.21.15.83:0/3226507157 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2028219083 0 0) 0x7f59c4003530 con 0x7f59e015f560 2024-04-25T00:14:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59d6a00640 1 -- 172.21.15.83:0/3226507157 >> v1:172.21.15.86:6789/0 conn(0x7f59e00d03b0 legacy=0x7f59e00afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59d6a00640 1 -- 172.21.15.83:0/3226507157 >> v1:172.21.15.83:6789/0 conn(0x7f59e015f560 legacy=0x7f59e00b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.383+0000 7f59d6a00640 1 -- 172.21.15.83:0/3226507157 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e00af1b0 con 0x7f59e015f940 2024-04-25T00:14:28.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.387+0000 7f59d6a00640 1 --2- 172.21.15.83:0/3226507157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e015f940 0x7f59e00cfe70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f59c80097c0 tx=0x7f59c801ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4590223cdd622a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:28.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.387+0000 7f59dea00640 1 -- 172.21.15.83:0/3226507157 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80b5070 con 0x7f59e015f940 2024-04-25T00:14:28.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.387+0000 7f59dea00640 1 -- 172.21.15.83:0/3226507157 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c80a8630 con 0x7f59e015f940 2024-04-25T00:14:28.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.387+0000 7f59dea00640 1 -- 172.21.15.83:0/3226507157 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0001a00 con 0x7f59e015f940 2024-04-25T00:14:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.387+0000 7f59e6000640 1 -- 172.21.15.83:0/3226507157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e015f940 msgr2=0x7f59e00cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.387+0000 7f59e6000640 1 --2- 172.21.15.83:0/3226507157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e015f940 0x7f59e00cfe70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f59c80097c0 tx=0x7f59c801ebb0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.387+0000 7f59e6000640 1 -- 172.21.15.83:0/3226507157 shutdown_connections 2024-04-25T00:14:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.387+0000 7f59e6000640 1 --2- 172.21.15.83:0/3226507157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e015f940 0x7f59e00cfe70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.387+0000 7f59e6000640 1 -- 172.21.15.83:0/3226507157 >> 172.21.15.83:0/3226507157 conn(0x7f59e00d1d70 msgr2=0x7f59e00d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.387+0000 7f59e6000640 1 -- 172.21.15.83:0/3226507157 shutdown_connections 2024-04-25T00:14:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.387+0000 7f59e6000640 1 -- 172.21.15.83:0/3226507157 wait complete. 2024-04-25T00:14:28.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59e6000640 1 Processor -- start 2024-04-25T00:14:28.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59e6000640 1 -- start start 2024-04-25T00:14:28.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59e6000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59e00d03b0 0x7f59e00c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59e6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e015f560 0x7f59e0151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59e6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e015f940 0x7f59e0151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59e6000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59e00af800 con 0x7f59e015f940 2024-04-25T00:14:28.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59e6000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59e00d0800 con 0x7f59e015f560 2024-04-25T00:14:28.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59e6000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f59e00d2a20 con 0x7f59e00d03b0 2024-04-25T00:14:28.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e015f940 0x7f59e0151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e015f940 0x7f59e0151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40882/0 (socket says 172.21.15.83:40882) 2024-04-25T00:14:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59dfe00640 1 -- 172.21.15.83:0/1423244125 learned_addr learned my addr 172.21.15.83:0/1423244125 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59df400640 1 --2- 172.21.15.83:0/1423244125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59e00d03b0 0x7f59e00c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59d6a00640 1 --2- 172.21.15.83:0/1423244125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e015f560 0x7f59e0151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59dfe00640 1 -- 172.21.15.83:0/1423244125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59e00d03b0 msgr2=0x7f59e00c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59dfe00640 1 --2- 172.21.15.83:0/1423244125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59e00d03b0 0x7f59e00c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59dfe00640 1 -- 172.21.15.83:0/1423244125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e015f560 msgr2=0x7f59e0151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59dfe00640 1 --2- 172.21.15.83:0/1423244125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e015f560 0x7f59e0151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59dfe00640 1 -- 172.21.15.83:0/1423244125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e0152440 con 0x7f59e015f940 2024-04-25T00:14:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59dfe00640 1 --2- 172.21.15.83:0/1423244125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e015f940 0x7f59e0151480 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f59d800c430 tx=0x7f59d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59d7e00640 1 -- 172.21.15.83:0/1423244125 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8005220 con 0x7f59e015f940 2024-04-25T00:14:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59d7e00640 1 -- 172.21.15.83:0/1423244125 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d80053c0 con 0x7f59e015f940 2024-04-25T00:14:28.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59d7e00640 1 -- 172.21.15.83:0/1423244125 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8007270 con 0x7f59e015f940 2024-04-25T00:14:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59e6000640 1 -- 172.21.15.83:0/1423244125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e01519c0 con 0x7f59e015f940 2024-04-25T00:14:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.391+0000 7f59e6000640 1 -- 172.21.15.83:0/1423244125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e00c97b0 con 0x7f59e015f940 2024-04-25T00:14:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.395+0000 7f59e6000640 1 -- 172.21.15.83:0/1423244125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e009fdd0 con 0x7f59e015f940 2024-04-25T00:14:28.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.395+0000 7f59d7e00640 1 -- 172.21.15.83:0/1423244125 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f59d8019020 con 0x7f59e015f940 2024-04-25T00:14:28.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.395+0000 7f59d7e00640 1 --2- 172.21.15.83:0/1423244125 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59bc0678f0 0x7f59bc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.395+0000 7f59d7e00640 1 -- 172.21.15.83:0/1423244125 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f59d8084fa0 con 0x7f59e015f940 2024-04-25T00:14:28.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.395+0000 7f59df400640 1 --2- 172.21.15.83:0/1423244125 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59bc0678f0 0x7f59bc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.395+0000 7f59df400640 1 --2- 172.21.15.83:0/1423244125 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59bc0678f0 0x7f59bc069db0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f59d0001570 tx=0x7f59d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:28.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.399+0000 7f59d7e00640 1 -- 172.21.15.83:0/1423244125 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f59e0152440 con 0x7f59e015f940 2024-04-25T00:14:28.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.591+0000 7f59e6000640 1 -- 172.21.15.83:0/1423244125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59e00c7c20 con 0x7f59e015f940 2024-04-25T00:14:28.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.591+0000 7f59d7e00640 1 -- 172.21.15.83:0/1423244125 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5174 (secure 0 0 0) 0x7f59d8055270 con 0x7f59e015f940 2024-04-25T00:14:28.595 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:28.596 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/3856734881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3856734881},{"type":"v1","addr":"172.21.15.86:6834","nonce":3856734881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/12764228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":12764228},{"type":"v1","addr":"172.21.15.86:6835","nonce":12764228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1680607346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1680607346},{"type":"v1","addr":"172.21.15.83:6837","nonce":1680607346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:22.706751+0000","modified":"2024-04-25T00:14:25.734428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7023},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7023":{"gid":7023,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/121840722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":121840722},{"type":"v1","addr":"172.21.15.83:6836","nonce":121840722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7023,"qdb_cluster":[7023]},"id":6}]} 2024-04-25T00:14:28.596 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 53 2024-04-25T00:14:28.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.595+0000 7f59e6000640 1 -- 172.21.15.83:0/1423244125 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59bc0678f0 msgr2=0x7f59bc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.595+0000 7f59e6000640 1 --2- 172.21.15.83:0/1423244125 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59bc0678f0 0x7f59bc069db0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f59d0001570 tx=0x7f59d0009040 comp rx=0 tx=0).stop 2024-04-25T00:14:28.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.595+0000 7f59e6000640 1 -- 172.21.15.83:0/1423244125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e015f940 msgr2=0x7f59e0151480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.595+0000 7f59e6000640 1 --2- 172.21.15.83:0/1423244125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e015f940 0x7f59e0151480 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f59d800c430 tx=0x7f59d800c8f0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.595+0000 7f59e6000640 1 -- 172.21.15.83:0/1423244125 shutdown_connections 2024-04-25T00:14:28.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.595+0000 7f59e6000640 1 --2- 172.21.15.83:0/1423244125 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59bc0678f0 0x7f59bc069db0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.595+0000 7f59e6000640 1 --2- 172.21.15.83:0/1423244125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e015f940 0x7f59e0151480 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.595+0000 7f59e6000640 1 --2- 172.21.15.83:0/1423244125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e015f560 0x7f59e0151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.595+0000 7f59e6000640 1 --2- 172.21.15.83:0/1423244125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59e00d03b0 0x7f59e00c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.595+0000 7f59e6000640 1 -- 172.21.15.83:0/1423244125 >> 172.21.15.83:0/1423244125 conn(0x7f59e00d1d70 msgr2=0x7f59e015aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:28.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.595+0000 7f59e6000640 1 -- 172.21.15.83:0/1423244125 shutdown_connections 2024-04-25T00:14:28.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:28.595+0000 7f59e6000640 1 -- 172.21.15.83:0/1423244125 wait complete. 2024-04-25T00:14:29.019 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:14:22.706751+0000', 'modified': '2024-04-25T00:14:25.734428+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7023}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7023': {'gid': 7023, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6836/121840722', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 121840722}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 121840722}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7023, 'qdb_cluster': [7023]} 2024-04-25T00:14:29.019 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:14:29.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:14:29.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.151+0000 7f2b55400640 1 Processor -- start 2024-04-25T00:14:29.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.151+0000 7f2b55400640 1 -- start start 2024-04-25T00:14:29.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b55400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b5006cfd0 0x7f2b50070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b55400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2b5006d3b0 0x7f2b5006d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b55400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2b500ca630 0x7f2b500caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b55400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2b5006e160 con 0x7f2b5006d3b0 2024-04-25T00:14:29.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b55400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2b5006ea10 con 0x7f2b500ca630 2024-04-25T00:14:29.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b55400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5006ecf0 con 0x7f2b5006cfd0 2024-04-25T00:14:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b47e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2b5006d3b0 0x7f2b5006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b47e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2b5006d3b0 0x7f2b5006d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40892/0 (socket says 172.21.15.83:40892) 2024-04-25T00:14:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b47e00640 1 -- 172.21.15.83:0/1346728303 learned_addr learned my addr 172.21.15.83:0/1346728303 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b4f200640 1 --2- 172.21.15.83:0/1346728303 >> v2:172.21.15.86:3300/0 conn(0x7f2b500ca630 0x7f2b500caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b4e800640 1 --2- 172.21.15.83:0/1346728303 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b5006cfd0 0x7f2b50070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b47e00640 1 -- 172.21.15.83:0/1346728303 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b5006cfd0 msgr2=0x7f2b50070d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b47e00640 1 --2- 172.21.15.83:0/1346728303 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b5006cfd0 0x7f2b50070d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b47e00640 1 -- 172.21.15.83:0/1346728303 >> v2:172.21.15.86:3300/0 conn(0x7f2b500ca630 msgr2=0x7f2b500caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b47e00640 1 --2- 172.21.15.83:0/1346728303 >> v2:172.21.15.86:3300/0 conn(0x7f2b500ca630 0x7f2b500caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.155+0000 7f2b47e00640 1 -- 172.21.15.83:0/1346728303 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5006efd0 con 0x7f2b5006d3b0 2024-04-25T00:14:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b47e00640 1 --2- 172.21.15.83:0/1346728303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b5006d3b0 0x7f2b5006d810 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f2b5015f860 tx=0x7f2b3801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fa070997e2382a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b4de00640 1 -- 172.21.15.83:0/1346728303 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380b0070 con 0x7f2b5006d3b0 2024-04-25T00:14:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b4de00640 1 -- 172.21.15.83:0/1346728303 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b380ab600 con 0x7f2b5006d3b0 2024-04-25T00:14:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b4de00640 1 -- 172.21.15.83:0/1346728303 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380ab9d0 con 0x7f2b5006d3b0 2024-04-25T00:14:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 -- 172.21.15.83:0/1346728303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b5006d3b0 msgr2=0x7f2b5006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 --2- 172.21.15.83:0/1346728303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b5006d3b0 0x7f2b5006d810 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f2b5015f860 tx=0x7f2b3801c520 comp rx=0 tx=0).stop 2024-04-25T00:14:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 -- 172.21.15.83:0/1346728303 shutdown_connections 2024-04-25T00:14:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 --2- 172.21.15.83:0/1346728303 >> v2:172.21.15.86:3300/0 conn(0x7f2b500ca630 0x7f2b500caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 --2- 172.21.15.83:0/1346728303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b5006d3b0 0x7f2b5006d810 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 --2- 172.21.15.83:0/1346728303 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b5006cfd0 0x7f2b50070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 -- 172.21.15.83:0/1346728303 >> 172.21.15.83:0/1346728303 conn(0x7f2b500cfa60 msgr2=0x7f2b500cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 -- 172.21.15.83:0/1346728303 shutdown_connections 2024-04-25T00:14:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 -- 172.21.15.83:0/1346728303 wait complete. 2024-04-25T00:14:29.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 Processor -- start 2024-04-25T00:14:29.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 -- start start 2024-04-25T00:14:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b5006cfd0 0x7f2b500cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b5006d3b0 0x7f2b5013edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b500ca630 0x7f2b500c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b50167b50 con 0x7f2b5006d3b0 2024-04-25T00:14:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b50145970 con 0x7f2b5006cfd0 2024-04-25T00:14:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.159+0000 7f2b55400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2b50167cd0 con 0x7f2b500ca630 2024-04-25T00:14:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b47e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b5006d3b0 0x7f2b5013edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b47e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b5006d3b0 0x7f2b5013edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40896/0 (socket says 172.21.15.83:40896) 2024-04-25T00:14:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b47e00640 1 -- 172.21.15.83:0/3739696993 learned_addr learned my addr 172.21.15.83:0/3739696993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b4e800640 1 --2- 172.21.15.83:0/3739696993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b5006cfd0 0x7f2b500cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b4f200640 1 --2- 172.21.15.83:0/3739696993 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b500ca630 0x7f2b500c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b47e00640 1 -- 172.21.15.83:0/3739696993 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b500ca630 msgr2=0x7f2b500c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b47e00640 1 --2- 172.21.15.83:0/3739696993 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b500ca630 0x7f2b500c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b47e00640 1 -- 172.21.15.83:0/3739696993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b5006cfd0 msgr2=0x7f2b500cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b47e00640 1 --2- 172.21.15.83:0/3739696993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b5006cfd0 0x7f2b500cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b47e00640 1 -- 172.21.15.83:0/3739696993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5013fe60 con 0x7f2b5006d3b0 2024-04-25T00:14:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b47e00640 1 --2- 172.21.15.83:0/3739696993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b5006d3b0 0x7f2b5013edf0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f2b380ae040 tx=0x7f2b380192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b46a00640 1 -- 172.21.15.83:0/3739696993 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380b0040 con 0x7f2b5006d3b0 2024-04-25T00:14:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b46a00640 1 -- 172.21.15.83:0/3739696993 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b380b4420 con 0x7f2b5006d3b0 2024-04-25T00:14:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b46a00640 1 -- 172.21.15.83:0/3739696993 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380b47f0 con 0x7f2b5006d3b0 2024-04-25T00:14:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b55400640 1 -- 172.21.15.83:0/3739696993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b50141a10 con 0x7f2b5006d3b0 2024-04-25T00:14:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b55400640 1 -- 172.21.15.83:0/3739696993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b50141fd0 con 0x7f2b5006d3b0 2024-04-25T00:14:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b46a00640 1 -- 172.21.15.83:0/3739696993 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2b500cb810 con 0x7f2b5006d3b0 2024-04-25T00:14:29.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.163+0000 7f2b46a00640 1 --2- 172.21.15.83:0/3739696993 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2b2c0678c0 0x7f2b2c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b46a00640 1 -- 172.21.15.83:0/3739696993 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f2b3811f6c0 con 0x7f2b5006d3b0 2024-04-25T00:14:29.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b4e800640 1 --2- 172.21.15.83:0/3739696993 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2b2c0678c0 0x7f2b2c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 Processor -- start 2024-04-25T00:14:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 -- start start 2024-04-25T00:14:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b4e800640 1 --2- 172.21.15.83:0/3739696993 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2b2c0678c0 0x7f2b2c069d80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2b400099d0 tx=0x7f2b40008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2ec9ef0 0x556ec2eca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2eca810 0x556ec2e80680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2e80bc0 0x556ec2e80fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ec2cce5e0 con 0x556ec2e80bc0 2024-04-25T00:14:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556ec2c55c50 con 0x556ec2eca810 2024-04-25T00:14:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x556ec2cca5a0 con 0x556ec2ec9ef0 2024-04-25T00:14:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b4f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2e80bc0 0x556ec2e80fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b4e800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2ec9ef0 0x556ec2eca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b4f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2e80bc0 0x556ec2e80fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40902/0 (socket says 172.21.15.83:40902) 2024-04-25T00:14:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b47e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2eca810 0x556ec2e80680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b4f200640 1 -- 172.21.15.83:0/3897343177 learned_addr learned my addr 172.21.15.83:0/3897343177 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b4f200640 1 -- 172.21.15.83:0/3897343177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2ec9ef0 msgr2=0x556ec2eca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b4f200640 1 --2- 172.21.15.83:0/3897343177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2ec9ef0 0x556ec2eca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b4f200640 1 -- 172.21.15.83:0/3897343177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2eca810 msgr2=0x556ec2e80680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b4f200640 1 --2- 172.21.15.83:0/3897343177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2eca810 0x556ec2e80680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b4f200640 1 -- 172.21.15.83:0/3897343177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec2c53d10 con 0x556ec2e80bc0 2024-04-25T00:14:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b4f200640 1 --2- 172.21.15.83:0/3897343177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2e80bc0 0x556ec2e80fa0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f2b480074c0 tx=0x7f2b48009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f177c0ae143a861e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b3f400640 1 -- 172.21.15.83:0/3897343177 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48019020 con 0x556ec2e80bc0 2024-04-25T00:14:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b3f400640 1 -- 172.21.15.83:0/3897343177 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b48015600 con 0x556ec2e80bc0 2024-04-25T00:14:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b3f400640 1 -- 172.21.15.83:0/3897343177 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b480159d0 con 0x556ec2e80bc0 2024-04-25T00:14:29.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 -- 172.21.15.83:0/3897343177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2e80bc0 msgr2=0x556ec2e80fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 --2- 172.21.15.83:0/3897343177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2e80bc0 0x556ec2e80fa0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f2b480074c0 tx=0x7f2b48009750 comp rx=0 tx=0).stop 2024-04-25T00:14:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 -- 172.21.15.83:0/3897343177 shutdown_connections 2024-04-25T00:14:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 --2- 172.21.15.83:0/3897343177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2e80bc0 0x556ec2e80fa0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 --2- 172.21.15.83:0/3897343177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2eca810 0x556ec2e80680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 --2- 172.21.15.83:0/3897343177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2ec9ef0 0x556ec2eca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 -- 172.21.15.83:0/3897343177 >> 172.21.15.83:0/3897343177 conn(0x556ec2e7cb70 msgr2=0x556ec2e7ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 -- 172.21.15.83:0/3897343177 shutdown_connections 2024-04-25T00:14:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.167+0000 7f2b589a9000 1 -- 172.21.15.83:0/3897343177 wait complete. 2024-04-25T00:14:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 1 -- start start 2024-04-25T00:14:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 1 Processor -- start 2024-04-25T00:14:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2e80bc0 0x556ec2eaef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2ec9ef0 0x556ec2eaf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2eca810 0x556ec2eaf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ec2c88520 con 0x556ec2eca810 2024-04-25T00:14:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556ec2c9b340 con 0x556ec2ec9ef0 2024-04-25T00:14:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x556ec2c74810 con 0x556ec2e80bc0 2024-04-25T00:14:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 10 client.0 _ll_register_callbacks cb 0x556ec2e6b5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b4f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2eca810 0x556ec2eaf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b4f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2eca810 0x556ec2eaf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40910/0 (socket says 172.21.15.83:40910) 2024-04-25T00:14:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b4f200640 1 -- 172.21.15.83:0/3543435785 learned_addr learned my addr 172.21.15.83:0/3543435785 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b4e800640 1 --2- 172.21.15.83:0/3543435785 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2e80bc0 0x556ec2eaef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b47e00640 1 --2- 172.21.15.83:0/3543435785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2ec9ef0 0x556ec2eaf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b4f200640 1 -- 172.21.15.83:0/3543435785 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2e80bc0 msgr2=0x556ec2eaef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b4f200640 1 --2- 172.21.15.83:0/3543435785 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2e80bc0 0x556ec2eaef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b4f200640 1 -- 172.21.15.83:0/3543435785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2ec9ef0 msgr2=0x556ec2eaf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b4f200640 1 --2- 172.21.15.83:0/3543435785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2ec9ef0 0x556ec2eaf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b4f200640 1 -- 172.21.15.83:0/3543435785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ec2c667b0 con 0x556ec2eca810 2024-04-25T00:14:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b4f200640 1 --2- 172.21.15.83:0/3543435785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2eca810 0x556ec2eaf9a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2b48018040 tx=0x7f2b48004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abb87b870a96460 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b33e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b33e00640 1 -- 172.21.15.83:0/3543435785 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48024070 con 0x556ec2eca810 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b33e00640 1 -- 172.21.15.83:0/3543435785 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f2b48029c40 con 0x556ec2eca810 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 1 -- 172.21.15.83:0/3543435785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b48015600 con 0x556ec2eca810 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b33e00640 1 -- 172.21.15.83:0/3543435785 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f2b48029e40 con 0x556ec2eca810 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 10 client.7182 fetch_fsmap learned FSMap version 53 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 1 -- 172.21.15.83:0/3543435785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x556ec2c55ea0 con 0x556ec2eca810 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b33e00640 1 -- 172.21.15.83:0/3543435785 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b48015600 con 0x556ec2eca810 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b33e00640 1 -- 172.21.15.83:0/3543435785 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f2b4801e620 con 0x556ec2eca810 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 10 client.7182 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 10 client.7182 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 20 client.7182 populate_metadata read hostname 'smithi083' 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 1 --2- 172.21.15.83:0/3543435785 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x556ec2ed9f90 0x556ec2edc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 4 client.7182 mds_command: new command op to 7023 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:14:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b589a9000 1 -- 172.21.15.83:0/3543435785 --> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ec2c69ab0 con 0x556ec2ed9f90 2024-04-25T00:14:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.171+0000 7f2b4e800640 1 --2- 172.21.15.83:0/3543435785 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x556ec2ed9f90 0x556ec2edc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b4e800640 1 --2- 172.21.15.83:0/3543435785 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x556ec2ed9f90 0x556ec2edc380 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e155298dba4452 server_cookie=8df91f57050ef152 in_seq=0 out_seq=0 2024-04-25T00:14:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b33e00640 10 client.7182 ms_handle_connect on v2:172.21.15.83:6834/121840722 2024-04-25T00:14:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b33e00640 1 -- 172.21.15.83:0/3543435785 <== mds.0 v2:172.21.15.83:6834/121840722 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ec2c69ab0 con 0x556ec2ed9f90 2024-04-25T00:14:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b33e00640 10 client.7182 handle_command_reply: tid=0 2024-04-25T00:14:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b589a9000 1 client.7182 shutdown 2024-04-25T00:14:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b589a9000 1 -- 172.21.15.83:0/3543435785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2eca810 msgr2=0x556ec2eaf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b589a9000 1 --2- 172.21.15.83:0/3543435785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2eca810 0x556ec2eaf9a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2b48018040 tx=0x7f2b48004ac0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b589a9000 1 -- 172.21.15.83:0/3543435785 shutdown_connections 2024-04-25T00:14:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b589a9000 1 --2- 172.21.15.83:0/3543435785 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x556ec2ed9f90 0x556ec2edc380 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b589a9000 1 --2- 172.21.15.83:0/3543435785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2eca810 0x556ec2eaf9a0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b589a9000 1 --2- 172.21.15.83:0/3543435785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2ec9ef0 0x556ec2eaf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b589a9000 1 --2- 172.21.15.83:0/3543435785 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2e80bc0 0x556ec2eaef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b33e00640 0 client.7182 ms_handle_reset on v2:172.21.15.83:6834/121840722 2024-04-25T00:14:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b589a9000 1 -- 172.21.15.83:0/3543435785 >> 172.21.15.83:0/3543435785 conn(0x556ec2e7cb70 msgr2=0x556ec2e7f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b589a9000 1 -- 172.21.15.83:0/3543435785 shutdown_connections 2024-04-25T00:14:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b589a9000 1 -- 172.21.15.83:0/3543435785 wait complete. 2024-04-25T00:14:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.175+0000 7f2b589a9000 20 client.7182 trim_cache size 0 max 16384 2024-04-25T00:14:29.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.191+0000 7f2b589a9000 1 Processor -- start 2024-04-25T00:14:29.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.191+0000 7f2b589a9000 1 -- start start 2024-04-25T00:14:29.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.191+0000 7f2b589a9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2e80bc0 0x556ec2f3e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.191+0000 7f2b589a9000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2ec9ef0 0x556ec2f3e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.191+0000 7f2b589a9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2eca810 0x556ec2f3edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.191+0000 7f2b589a9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b50202dc0 con 0x556ec2eca810 2024-04-25T00:14:29.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.191+0000 7f2b589a9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b50143b70 con 0x556ec2e80bc0 2024-04-25T00:14:29.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.191+0000 7f2b589a9000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2b50202820 con 0x556ec2ec9ef0 2024-04-25T00:14:29.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b4f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2eca810 0x556ec2f3edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b4f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2eca810 0x556ec2f3edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40912/0 (socket says 172.21.15.83:40912) 2024-04-25T00:14:29.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b4f200640 1 -- 172.21.15.83:0/1986997443 learned_addr learned my addr 172.21.15.83:0/1986997443 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:29.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b47e00640 1 --2- 172.21.15.83:0/1986997443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2ec9ef0 0x556ec2f3e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b4e800640 1 --2- 172.21.15.83:0/1986997443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2e80bc0 0x556ec2f3e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b4f200640 1 -- 172.21.15.83:0/1986997443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2ec9ef0 msgr2=0x556ec2f3e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:29.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b4f200640 1 --2- 172.21.15.83:0/1986997443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2ec9ef0 0x556ec2f3e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b4f200640 1 -- 172.21.15.83:0/1986997443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2e80bc0 msgr2=0x556ec2f3e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:29.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b4f200640 1 --2- 172.21.15.83:0/1986997443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2e80bc0 0x556ec2f3e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b4f200640 1 -- 172.21.15.83:0/1986997443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec2c56e80 con 0x556ec2eca810 2024-04-25T00:14:29.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b4f200640 1 --2- 172.21.15.83:0/1986997443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2eca810 0x556ec2f3edc0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009790 tx=0x7f2b48020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194b2ac9469fd658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b3ea00640 1 -- 172.21.15.83:0/1986997443 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48036070 con 0x556ec2eca810 2024-04-25T00:14:29.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b3ea00640 1 -- 172.21.15.83:0/1986997443 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b48015600 con 0x556ec2eca810 2024-04-25T00:14:29.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b3ea00640 1 -- 172.21.15.83:0/1986997443 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48019070 con 0x556ec2eca810 2024-04-25T00:14:29.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 -- 172.21.15.83:0/1986997443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2eca810 msgr2=0x556ec2f3edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 --2- 172.21.15.83:0/1986997443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2eca810 0x556ec2f3edc0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009790 tx=0x7f2b48020070 comp rx=0 tx=0).stop 2024-04-25T00:14:29.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 -- 172.21.15.83:0/1986997443 shutdown_connections 2024-04-25T00:14:29.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 --2- 172.21.15.83:0/1986997443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2ec9ef0 0x556ec2f3e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 --2- 172.21.15.83:0/1986997443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2eca810 0x556ec2f3edc0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 --2- 172.21.15.83:0/1986997443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2e80bc0 0x556ec2f3e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 -- 172.21.15.83:0/1986997443 >> 172.21.15.83:0/1986997443 conn(0x556ec2e7cb70 msgr2=0x556ec2ec87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:29.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 -- 172.21.15.83:0/1986997443 shutdown_connections 2024-04-25T00:14:29.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 -- 172.21.15.83:0/1986997443 wait complete. 2024-04-25T00:14:29.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 -- start start 2024-04-25T00:14:29.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 Processor -- start 2024-04-25T00:14:29.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2e80bc0 0x556ec2f420f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2ec9ef0 0x556ec2edcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2eca810 0x556ec2edd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ec2cda180 con 0x556ec2ec9ef0 2024-04-25T00:14:29.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556ec2c97810 con 0x556ec2e80bc0 2024-04-25T00:14:29.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x556ec2c66dd0 con 0x556ec2eca810 2024-04-25T00:14:29.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b589a9000 10 client.0 _ll_register_callbacks cb 0x7f2b380094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:29.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b47e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2ec9ef0 0x556ec2edcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b47e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2ec9ef0 0x556ec2edcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40916/0 (socket says 172.21.15.83:40916) 2024-04-25T00:14:29.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b47e00640 1 -- 172.21.15.83:0/1829454642 learned_addr learned my addr 172.21.15.83:0/1829454642 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:29.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b4f200640 1 --2- 172.21.15.83:0/1829454642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2eca810 0x556ec2edd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b4e800640 1 --2- 172.21.15.83:0/1829454642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2e80bc0 0x556ec2f420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b47e00640 1 -- 172.21.15.83:0/1829454642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2eca810 msgr2=0x556ec2edd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:29.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b47e00640 1 --2- 172.21.15.83:0/1829454642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2eca810 0x556ec2edd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b47e00640 1 -- 172.21.15.83:0/1829454642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2e80bc0 msgr2=0x556ec2f420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:29.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b47e00640 1 --2- 172.21.15.83:0/1829454642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2e80bc0 0x556ec2f420f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.195+0000 7f2b47e00640 1 -- 172.21.15.83:0/1829454642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ec2c9b760 con 0x556ec2ec9ef0 2024-04-25T00:14:29.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b47e00640 1 --2- 172.21.15.83:0/1829454642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2ec9ef0 0x556ec2edcaf0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2b3801ca90 tx=0x7f2b380ef560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a2f3af6dce9eb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b33e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:14:29.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b33e00640 1 -- 172.21.15.83:0/1829454642 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380ef960 con 0x556ec2ec9ef0 2024-04-25T00:14:29.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b33e00640 1 -- 172.21.15.83:0/1829454642 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f2b380edc60 con 0x556ec2ec9ef0 2024-04-25T00:14:29.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 -- 172.21.15.83:0/1829454642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b48015600 con 0x556ec2ec9ef0 2024-04-25T00:14:29.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b33e00640 1 -- 172.21.15.83:0/1829454642 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f2b380f3d10 con 0x556ec2ec9ef0 2024-04-25T00:14:29.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 10 client.7188 fetch_fsmap learned FSMap version 53 2024-04-25T00:14:29.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 -- 172.21.15.83:0/1829454642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x556ec2d279e0 con 0x556ec2ec9ef0 2024-04-25T00:14:29.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b33e00640 1 -- 172.21.15.83:0/1829454642 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b48015600 con 0x556ec2ec9ef0 2024-04-25T00:14:29.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b33e00640 1 -- 172.21.15.83:0/1829454642 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f2b380f60b0 con 0x556ec2ec9ef0 2024-04-25T00:14:29.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 10 client.7188 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T00:14:29.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 10 client.7188 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:14:29.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 20 client.7188 populate_metadata read hostname 'smithi083' 2024-04-25T00:14:29.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 --2- 172.21.15.83:0/1829454642 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x556ec2edd570 0x556ec2edd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 4 client.7188 mds_command: new command op to 7023 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:14:29.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 -- 172.21.15.83:0/1829454642 --> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] -- command(tid 0: {"prefix": "status"}) -- 0x556ec2cdbb20 con 0x556ec2edd570 2024-04-25T00:14:29.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b4e800640 1 --2- 172.21.15.83:0/1829454642 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x556ec2edd570 0x556ec2edd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b4e800640 1 --2- 172.21.15.83:0/1829454642 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x556ec2edd570 0x556ec2edd950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3fed745500a53a server_cookie=3e34731bd2115d33 in_seq=0 out_seq=0 2024-04-25T00:14:29.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b33e00640 10 client.7188 ms_handle_connect on v2:172.21.15.83:6834/121840722 2024-04-25T00:14:29.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b33e00640 1 -- 172.21.15.83:0/1829454642 <== mds.0 v2:172.21.15.83:6834/121840722 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x556ec2cdbb20 con 0x556ec2edd570 2024-04-25T00:14:29.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b33e00640 10 client.7188 handle_command_reply: tid=0 2024-04-25T00:14:29.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 client.7188 shutdown 2024-04-25T00:14:29.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 -- 172.21.15.83:0/1829454642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2ec9ef0 msgr2=0x556ec2edcaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 --2- 172.21.15.83:0/1829454642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2ec9ef0 0x556ec2edcaf0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2b3801ca90 tx=0x7f2b380ef560 comp rx=0 tx=0).stop 2024-04-25T00:14:29.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 -- 172.21.15.83:0/1829454642 shutdown_connections 2024-04-25T00:14:29.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 --2- 172.21.15.83:0/1829454642 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x556ec2edd570 0x556ec2edd950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 --2- 172.21.15.83:0/1829454642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556ec2eca810 0x556ec2edd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 --2- 172.21.15.83:0/1829454642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ec2ec9ef0 0x556ec2edcaf0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b33e00640 0 client.7188 ms_handle_reset on v2:172.21.15.83:6834/121840722 2024-04-25T00:14:29.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 --2- 172.21.15.83:0/1829454642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ec2e80bc0 0x556ec2f420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 -- 172.21.15.83:0/1829454642 >> 172.21.15.83:0/1829454642 conn(0x556ec2e7cb70 msgr2=0x556ec2ec9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:29.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 -- 172.21.15.83:0/1829454642 shutdown_connections 2024-04-25T00:14:29.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 1 -- 172.21.15.83:0/1829454642 wait complete. 2024-04-25T00:14:29.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b589a9000 20 client.7188 trim_cache size 0 max 16384 2024-04-25T00:14:29.208 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:14:29.209 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "d58ae83a-ad72-4e18-824a-86e779979c35", 2024-04-25T00:14:29.209 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-25T00:14:29.209 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7023, 2024-04-25T00:14:29.209 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T00:14:29.209 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T00:14:29.209 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T00:14:29.209 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.8506928350000003, 2024-04-25T00:14:29.209 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 53, 2024-04-25T00:14:29.209 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 82, 2024-04-25T00:14:29.209 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 82, 2024-04-25T00:14:29.209 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 10.793396087 2024-04-25T00:14:29.209 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:14:29.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b55400640 1 -- 172.21.15.83:0/3739696993 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2b2c0678c0 msgr2=0x7f2b2c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b55400640 1 --2- 172.21.15.83:0/3739696993 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2b2c0678c0 0x7f2b2c069d80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2b400099d0 tx=0x7f2b40008040 comp rx=0 tx=0).stop 2024-04-25T00:14:29.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b55400640 1 -- 172.21.15.83:0/3739696993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b5006d3b0 msgr2=0x7f2b5013edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b55400640 1 --2- 172.21.15.83:0/3739696993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b5006d3b0 0x7f2b5013edf0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f2b380ae040 tx=0x7f2b380192c0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b55400640 1 -- 172.21.15.83:0/3739696993 shutdown_connections 2024-04-25T00:14:29.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b55400640 1 --2- 172.21.15.83:0/3739696993 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2b2c0678c0 0x7f2b2c069d80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b55400640 1 --2- 172.21.15.83:0/3739696993 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b500ca630 0x7f2b500c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b55400640 1 --2- 172.21.15.83:0/3739696993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b5006d3b0 0x7f2b5013edf0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b55400640 1 --2- 172.21.15.83:0/3739696993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b5006cfd0 0x7f2b500cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b55400640 1 -- 172.21.15.83:0/3739696993 >> 172.21.15.83:0/3739696993 conn(0x7f2b500cfa60 msgr2=0x7f2b5015e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:29.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b55400640 1 -- 172.21.15.83:0/3739696993 shutdown_connections 2024-04-25T00:14:29.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.199+0000 7f2b55400640 1 -- 172.21.15.83:0/3739696993 wait complete. 2024-04-25T00:14:29.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:14:29.319 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-04-25T00:14:29.315+0000 7f1a8a200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:14:29.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.355+0000 7f4504800640 1 Processor -- start 2024-04-25T00:14:29.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.355+0000 7f4504800640 1 -- start start 2024-04-25T00:14:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.355+0000 7f4504800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44fc0b4360 0x7f44fc0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.355+0000 7f4504800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f44fc0b2b30 0x7f44fc0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.355+0000 7f4504800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44fc06ef80 con 0x7f44fc1553e0 2024-04-25T00:14:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.355+0000 7f4504800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f44fc15b510 con 0x7f44fc0b2b30 2024-04-25T00:14:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.355+0000 7f4504800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f44fc15b7f0 con 0x7f44fc0b4360 2024-04-25T00:14:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.355+0000 7f4502800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f44fc0b2b30 0x7f44fc0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.355+0000 7f4502800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f44fc0b2b30 0x7f44fc0b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:50008/0 (socket says 172.21.15.83:50008) 2024-04-25T00:14:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.355+0000 7f4502800640 1 -- 172.21.15.83:0/3348322216 learned_addr learned my addr 172.21.15.83:0/3348322216 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.355+0000 7f4501400640 1 --2- 172.21.15.83:0/3348322216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44fc0b4360 0x7f44fc0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.359+0000 7f4501400640 1 -- 172.21.15.83:0/3348322216 >> v2:172.21.15.86:3300/0 conn(0x7f44fc0b2b30 msgr2=0x7f44fc0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.359+0000 7f4501400640 1 --2- 172.21.15.83:0/3348322216 >> v2:172.21.15.86:3300/0 conn(0x7f44fc0b2b30 0x7f44fc0b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.359+0000 7f4501400640 1 -- 172.21.15.83:0/3348322216 >> v1:172.21.15.86:6789/0 conn(0x7f44fc1553e0 legacy=0x7f44fc06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.359+0000 7f4501400640 1 -- 172.21.15.83:0/3348322216 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc15bad0 con 0x7f44fc0b4360 2024-04-25T00:14:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.359+0000 7f4501400640 1 --2- 172.21.15.83:0/3348322216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44fc0b4360 0x7f44fc0b47e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0097c0 tx=0x7f44ec01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccfbdf6024201480 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.359+0000 7f4500a00640 1 -- 172.21.15.83:0/3348322216 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec0b0070 con 0x7f44fc0b4360 2024-04-25T00:14:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.359+0000 7f4500a00640 1 -- 172.21.15.83:0/3348322216 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44ec0ab600 con 0x7f44fc0b4360 2024-04-25T00:14:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.359+0000 7f4500a00640 1 -- 172.21.15.83:0/3348322216 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec0a4e60 con 0x7f44fc0b4360 2024-04-25T00:14:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 -- 172.21.15.83:0/3348322216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44fc0b4360 msgr2=0x7f44fc0b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 --2- 172.21.15.83:0/3348322216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44fc0b4360 0x7f44fc0b47e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0097c0 tx=0x7f44ec01c410 comp rx=0 tx=0).stop 2024-04-25T00:14:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 -- 172.21.15.83:0/3348322216 shutdown_connections 2024-04-25T00:14:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 --2- 172.21.15.83:0/3348322216 >> v2:172.21.15.86:3300/0 conn(0x7f44fc0b2b30 0x7f44fc0b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 --2- 172.21.15.83:0/3348322216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44fc0b4360 0x7f44fc0b47e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 -- 172.21.15.83:0/3348322216 >> 172.21.15.83:0/3348322216 conn(0x7f44fc0b1830 msgr2=0x7f44fc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 -- 172.21.15.83:0/3348322216 shutdown_connections 2024-04-25T00:14:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 -- 172.21.15.83:0/3348322216 wait complete. 2024-04-25T00:14:29.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 Processor -- start 2024-04-25T00:14:29.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 -- start start 2024-04-25T00:14:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0b2b30 0x7f44fc155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44fc0b4360 0x7f44fc15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44fc1553e0 0x7f44fc14b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc06ebd0 con 0x7f44fc1553e0 2024-04-25T00:14:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc06d3b0 con 0x7f44fc0b2b30 2024-04-25T00:14:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4504800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f44fc15b210 con 0x7f44fc0b4360 2024-04-25T00:14:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4501400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44fc0b4360 0x7f44fc15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4502800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44fc1553e0 0x7f44fc14b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4501e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0b2b30 0x7f44fc155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4501400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44fc0b4360 0x7f44fc15aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:60330/0 (socket says 172.21.15.83:60330) 2024-04-25T00:14:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4501400640 1 -- 172.21.15.83:0/547054044 learned_addr learned my addr 172.21.15.83:0/547054044 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4502800640 1 -- 172.21.15.83:0/547054044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44fc0b4360 msgr2=0x7f44fc15aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4502800640 1 --2- 172.21.15.83:0/547054044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44fc0b4360 0x7f44fc15aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4502800640 1 -- 172.21.15.83:0/547054044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0b2b30 msgr2=0x7f44fc155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4502800640 1 --2- 172.21.15.83:0/547054044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0b2b30 0x7f44fc155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.363+0000 7f4502800640 1 -- 172.21.15.83:0/547054044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc15af70 con 0x7f44fc1553e0 2024-04-25T00:14:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.367+0000 7f4502800640 1 --2- 172.21.15.83:0/547054044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44fc1553e0 0x7f44fc14b220 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f44f8010f30 tx=0x7f44f800e550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.367+0000 7f44f6000640 1 -- 172.21.15.83:0/547054044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f800b310 con 0x7f44fc1553e0 2024-04-25T00:14:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.367+0000 7f44f6000640 1 -- 172.21.15.83:0/547054044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f80039a0 con 0x7f44fc1553e0 2024-04-25T00:14:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.367+0000 7f44f6000640 1 -- 172.21.15.83:0/547054044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f8003d70 con 0x7f44fc1553e0 2024-04-25T00:14:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.367+0000 7f4504800640 1 -- 172.21.15.83:0/547054044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44fc159150 con 0x7f44fc1553e0 2024-04-25T00:14:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.367+0000 7f4504800640 1 -- 172.21.15.83:0/547054044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44fc159710 con 0x7f44fc1553e0 2024-04-25T00:14:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.367+0000 7f4504800640 1 -- 172.21.15.83:0/547054044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44fc0737f0 con 0x7f44fc1553e0 2024-04-25T00:14:29.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.367+0000 7f44f6000640 1 -- 172.21.15.83:0/547054044 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f44f8014050 con 0x7f44fc1553e0 2024-04-25T00:14:29.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.367+0000 7f44f6000640 1 --2- 172.21.15.83:0/547054044 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44e00678c0 0x7f44e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.367+0000 7f44f6000640 1 -- 172.21.15.83:0/547054044 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f44f8088390 con 0x7f44fc1553e0 2024-04-25T00:14:29.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.367+0000 7f4501e00640 1 --2- 172.21.15.83:0/547054044 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44e00678c0 0x7f44e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.371+0000 7f4501e00640 1 --2- 172.21.15.83:0/547054044 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44e00678c0 0x7f44e0069d80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f44f0001770 tx=0x7f44f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.371+0000 7f44f6000640 1 -- 172.21.15.83:0/547054044 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f44f8024020 con 0x7f44fc1553e0 2024-04-25T00:14:29.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.563+0000 7f4504800640 1 -- 172.21.15.83:0/547054044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f44fc154b60 con 0x7f44fc1553e0 2024-04-25T00:14:29.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.563+0000 7f44f6000640 1 -- 172.21.15.83:0/547054044 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7f44f8012040 con 0x7f44fc1553e0 2024-04-25T00:14:29.568 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:14:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.567+0000 7f4504800640 1 -- 172.21.15.83:0/547054044 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44e00678c0 msgr2=0x7f44e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.567+0000 7f4504800640 1 --2- 172.21.15.83:0/547054044 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44e00678c0 0x7f44e0069d80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f44f0001770 tx=0x7f44f0009040 comp rx=0 tx=0).stop 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.567+0000 7f4504800640 1 -- 172.21.15.83:0/547054044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44fc1553e0 msgr2=0x7f44fc14b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.567+0000 7f4504800640 1 --2- 172.21.15.83:0/547054044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44fc1553e0 0x7f44fc14b220 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f44f8010f30 tx=0x7f44f800e550 comp rx=0 tx=0).stop 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.567+0000 7f4504800640 1 -- 172.21.15.83:0/547054044 shutdown_connections 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.567+0000 7f4504800640 1 --2- 172.21.15.83:0/547054044 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44e00678c0 0x7f44e0069d80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.567+0000 7f4504800640 1 --2- 172.21.15.83:0/547054044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f44fc0b4360 0x7f44fc15aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.567+0000 7f4504800640 1 --2- 172.21.15.83:0/547054044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44fc1553e0 0x7f44fc14b220 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.567+0000 7f4504800640 1 --2- 172.21.15.83:0/547054044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0b2b30 0x7f44fc155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.567+0000 7f4504800640 1 -- 172.21.15.83:0/547054044 >> 172.21.15.83:0/547054044 conn(0x7f44fc0b1830 msgr2=0x7f44fc14d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.567+0000 7f4504800640 1 -- 172.21.15.83:0/547054044 shutdown_connections 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29.567+0000 7f4504800640 1 -- 172.21.15.83:0/547054044 wait complete. 2024-04-25T00:14:29.589 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:14:29.589 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:14:29.589 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-25T00:14:29.589 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:14:29.589 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:14:29.589 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:14:29.589 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:14:29.589 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:14:29.589 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:14:29.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:29.590 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-25T00:14:29.598 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:14:29.599 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:14:29.599 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:14:29.599 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:29.599 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-25T00:14:29.599 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:29.599 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:29.599 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:29.599 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-25T00:14:29.599 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:29.599 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:29.599 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-25T00:14:29.599 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:29.599 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:29.599 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:14:29.600 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:29.600 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:14:29.600 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42265sec preferred_lft 42265sec 2024-04-25T00:14:29.600 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-25T00:14:29.600 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:29.600 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:14:29.600 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:8d:d4:7d:b9 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:29.600 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:14:29.600 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:29.600 INFO:teuthology.orchestra.run.smithi083.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:14:29.600 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:29.600 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:14:29.600 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:29.600 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:14:29.601 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:29.601 INFO:teuthology.orchestra.run.smithi083.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:14:29.601 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:14:29.601 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:14:29.601 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:29.601 INFO:teuthology.orchestra.run.smithi083.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:14:29.601 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:14:29.601 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:14:29.601 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:29.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:29.602 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:14:29.663 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-25T00:14:29.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:29.665 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-25T00:14:29.724 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-25T00:14:29.724 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:14:29.724 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:14:29.724 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:14:29.724 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:14:29.725 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:14:29.725 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:14:29.725 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:29.725 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:29.725 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:29.725 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:14:29.725 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:14:29.725 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:14:29.725 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:14:29.726 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:14:29.726 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:14:29.726 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:14:29.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:14:29.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:29.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:29.857 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:14:29.866 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:14:29.866 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:14:29.867 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:14:29.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:29.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-25T00:14:29.928 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:14:29.928 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:14:29.980 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:14:29.981 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:14:29.981 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:14:30.036 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T00:14:30.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:30.037 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:14:30.123 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:14:30.124 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:14:30.137 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:14:30.137 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:14:30.137 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:14:30.138 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:14:30.138 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:14:30.141 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:14:30.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:30.141 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:14:30.161 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:14:30.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:14:30.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:30.272 INFO:teuthology.orchestra.run.smithi083.stdout:client6781 2024-04-25T00:14:30.273 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:14:30.273 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:30.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:14:30.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f5927e00640 1 Processor -- start 2024-04-25T00:14:30.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f5927e00640 1 -- start start 2024-04-25T00:14:30.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f5927e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200b47e0 0x7f59200b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f5927e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59200b4c70 con 0x7f59200b4360 2024-04-25T00:14:30.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f5927e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f592006ef80 con 0x7f59201553e0 2024-04-25T00:14:30.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f5927e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f592015b510 con 0x7f59200b47e0 2024-04-25T00:14:30.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f5924a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f59200b4360 0x7f59200b2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41140/0 (socket says 172.21.15.83:41140) 2024-04-25T00:14:30.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f5924a00640 1 -- 172.21.15.83:0/319753540 learned_addr learned my addr 172.21.15.83:0/319753540 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:30.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f5925e00640 1 --2- 172.21.15.83:0/319753540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200b47e0 0x7f59200b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f591fe00640 1 -- 172.21.15.83:0/319753540 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2244014388 0 0) 0x7f59200b4c70 con 0x7f59200b4360 2024-04-25T00:14:30.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f591fe00640 1 -- 172.21.15.83:0/319753540 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5908003530 con 0x7f59200b4360 2024-04-25T00:14:30.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f591fe00640 1 -- 172.21.15.83:0/319753540 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2319182835 0 0) 0x7f5908003530 con 0x7f59200b4360 2024-04-25T00:14:30.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f591fe00640 1 -- 172.21.15.83:0/319753540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200b47e0 msgr2=0x7f59200b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:30.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f591fe00640 1 --2- 172.21.15.83:0/319753540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200b47e0 0x7f59200b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f591fe00640 1 -- 172.21.15.83:0/319753540 >> v1:172.21.15.86:6789/0 conn(0x7f59201553e0 legacy=0x7f592006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f591fe00640 1 -- 172.21.15.83:0/319753540 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592015b7f0 con 0x7f59200b4360 2024-04-25T00:14:30.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.459+0000 7f591fe00640 1 -- 172.21.15.83:0/319753540 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f590c001680 con 0x7f59200b4360 2024-04-25T00:14:30.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f591fe00640 1 -- 172.21.15.83:0/319753540 >> v1:172.21.15.83:6789/0 conn(0x7f59200b4360 legacy=0x7f59200b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f591fe00640 1 --2- 172.21.15.83:0/319753540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59080039f0 0x7f5908003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f591fe00640 1 --2- 172.21.15.83:0/319753540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59080043a0 0x7f5908004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f591fe00640 1 --2- 172.21.15.83:0/319753540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200b47e0 0x7f5908005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f591fe00640 1 -- 172.21.15.83:0/319753540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f592015b510 con 0x7f59080039f0 2024-04-25T00:14:30.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f591fe00640 1 -- 172.21.15.83:0/319753540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f590800bab0 con 0x7f59080043a0 2024-04-25T00:14:30.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f591fe00640 1 -- 172.21.15.83:0/319753540 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f590800bcb0 con 0x7f59200b47e0 2024-04-25T00:14:30.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f5925400640 1 --2- 172.21.15.83:0/319753540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59080039f0 0x7f5908003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f5924a00640 1 --2- 172.21.15.83:0/319753540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59080043a0 0x7f5908004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f5925e00640 1 --2- 172.21.15.83:0/319753540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200b47e0 0x7f5908005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f5925400640 1 -- 172.21.15.83:0/319753540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200b47e0 msgr2=0x7f5908005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:30.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f5925400640 1 --2- 172.21.15.83:0/319753540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200b47e0 0x7f5908005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f5925400640 1 -- 172.21.15.83:0/319753540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59080043a0 msgr2=0x7f5908004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:30.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f5925400640 1 --2- 172.21.15.83:0/319753540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59080043a0 0x7f5908004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f5925400640 1 -- 172.21.15.83:0/319753540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5908003530 con 0x7f59080039f0 2024-04-25T00:14:30.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f5925e00640 1 --2- 172.21.15.83:0/319753540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200b47e0 0x7f5908005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:30.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f5925400640 1 --2- 172.21.15.83:0/319753540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59080039f0 0x7f5908003dd0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5914001f60 tx=0x7f591401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f2b841d47582f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:30.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f591fe00640 1 -- 172.21.15.83:0/319753540 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914001b60 con 0x7f59080039f0 2024-04-25T00:14:30.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f591fe00640 1 -- 172.21.15.83:0/319753540 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59140a8660 con 0x7f59080039f0 2024-04-25T00:14:30.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.463+0000 7f591fe00640 1 -- 172.21.15.83:0/319753540 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140a8a50 con 0x7f59080039f0 2024-04-25T00:14:30.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5927e00640 1 -- 172.21.15.83:0/319753540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59080039f0 msgr2=0x7f5908003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5927e00640 1 --2- 172.21.15.83:0/319753540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59080039f0 0x7f5908003dd0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5914001f60 tx=0x7f591401c8a0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5925e00640 1 -- 172.21.15.83:0/319753540 reap_dead start 2024-04-25T00:14:30.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5927e00640 1 -- 172.21.15.83:0/319753540 shutdown_connections 2024-04-25T00:14:30.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5927e00640 1 -- 172.21.15.83:0/319753540 >> 172.21.15.83:0/319753540 conn(0x7f59200b1830 msgr2=0x7f59200b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:30.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5927e00640 1 -- 172.21.15.83:0/319753540 shutdown_connections 2024-04-25T00:14:30.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5927e00640 1 -- 172.21.15.83:0/319753540 wait complete. 2024-04-25T00:14:30.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5927e00640 1 Processor -- start 2024-04-25T00:14:30.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5927e00640 1 -- start start 2024-04-25T00:14:30.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5927e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59080039f0 0x7f592014e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5927e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f592014f130 0x7f592014e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5927e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5920155cd0 0x7f59200cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5927e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5920155940 con 0x7f59080039f0 2024-04-25T00:14:30.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5927e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59201557c0 con 0x7f5920155cd0 2024-04-25T00:14:30.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5927e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f592006ebd0 con 0x7f592014f130 2024-04-25T00:14:30.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5925400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59080039f0 0x7f592014e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5925400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59080039f0 0x7f592014e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40940/0 (socket says 172.21.15.83:40940) 2024-04-25T00:14:30.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5925400640 1 -- 172.21.15.83:0/3847932623 learned_addr learned my addr 172.21.15.83:0/3847932623 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:30.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5925e00640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5920155cd0 0x7f59200cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5924a00640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f592014f130 0x7f592014e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5925400640 1 -- 172.21.15.83:0/3847932623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f592014f130 msgr2=0x7f592014e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5925400640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f592014f130 0x7f592014e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5925400640 1 -- 172.21.15.83:0/3847932623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5920155cd0 msgr2=0x7f59200cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5925400640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5920155cd0 0x7f59200cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5925400640 1 -- 172.21.15.83:0/3847932623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5920158910 con 0x7f59080039f0 2024-04-25T00:14:30.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5925e00640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5920155cd0 0x7f59200cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:30.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.467+0000 7f5925400640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59080039f0 0x7f592014e310 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f591401c6d0 tx=0x7f5908005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:30.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.471+0000 7f591d600640 1 -- 172.21.15.83:0/3847932623 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140a8810 con 0x7f59080039f0 2024-04-25T00:14:30.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.471+0000 7f591d600640 1 -- 172.21.15.83:0/3847932623 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59140a8460 con 0x7f59080039f0 2024-04-25T00:14:30.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.471+0000 7f591d600640 1 -- 172.21.15.83:0/3847932623 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914006d60 con 0x7f59080039f0 2024-04-25T00:14:30.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.471+0000 7f5927e00640 1 -- 172.21.15.83:0/3847932623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5920158bf0 con 0x7f59080039f0 2024-04-25T00:14:30.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.471+0000 7f5927e00640 1 -- 172.21.15.83:0/3847932623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59200cd480 con 0x7f59080039f0 2024-04-25T00:14:30.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.471+0000 7f5927e00640 1 -- 172.21.15.83:0/3847932623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59200ccfc0 con 0x7f59080039f0 2024-04-25T00:14:30.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.471+0000 7f591d600640 1 -- 172.21.15.83:0/3847932623 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5914003980 con 0x7f59080039f0 2024-04-25T00:14:30.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.471+0000 7f591d600640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59000678f0 0x7f5900069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.471+0000 7f591d600640 1 -- 172.21.15.83:0/3847932623 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f591411fd80 con 0x7f59080039f0 2024-04-25T00:14:30.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.471+0000 7f5924a00640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59000678f0 0x7f5900069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.471+0000 7f5924a00640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59000678f0 0x7f5900069db0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5920155b70 tx=0x7f590c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:30.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.475+0000 7f591d600640 1 -- 172.21.15.83:0/3847932623 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f59200ccfc0 con 0x7f59080039f0 2024-04-25T00:14:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.667+0000 7f5927e00640 1 -- 172.21.15.83:0/3847932623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f59200737f0 con 0x7f59080039f0 2024-04-25T00:14:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.667+0000 7f591d600640 1 -- 172.21.15.83:0/3847932623 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7f59140c2030 con 0x7f59080039f0 2024-04-25T00:14:30.670 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:14:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.667+0000 7f5927e00640 1 -- 172.21.15.83:0/3847932623 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59000678f0 msgr2=0x7f5900069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.667+0000 7f5927e00640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59000678f0 0x7f5900069db0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5920155b70 tx=0x7f590c009040 comp rx=0 tx=0).stop 2024-04-25T00:14:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.667+0000 7f5927e00640 1 -- 172.21.15.83:0/3847932623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59080039f0 msgr2=0x7f592014e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.667+0000 7f5927e00640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59080039f0 0x7f592014e310 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f591401c6d0 tx=0x7f5908005840 comp rx=0 tx=0).stop 2024-04-25T00:14:30.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.667+0000 7f5927e00640 1 -- 172.21.15.83:0/3847932623 shutdown_connections 2024-04-25T00:14:30.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.667+0000 7f5927e00640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59000678f0 0x7f5900069db0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.667+0000 7f5927e00640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f592014f130 0x7f592014e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.667+0000 7f5927e00640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5920155cd0 0x7f59200cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.667+0000 7f5927e00640 1 --2- 172.21.15.83:0/3847932623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59080039f0 0x7f592014e310 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.667+0000 7f5927e00640 1 -- 172.21.15.83:0/3847932623 >> 172.21.15.83:0/3847932623 conn(0x7f59200b1830 msgr2=0x7f592015c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:30.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.667+0000 7f5927e00640 1 -- 172.21.15.83:0/3847932623 shutdown_connections 2024-04-25T00:14:30.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:30.667+0000 7f5927e00640 1 -- 172.21.15.83:0/3847932623 wait complete. 2024-04-25T00:14:30.691 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:14:30.691 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:14:30.691 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T00:14:30.691 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:14:30.691 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:14:30.691 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:14:30.691 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:14:30.691 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:14:30.692 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:14:30.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:30.692 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42258sec preferred_lft 42258sec 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:35:63:ea:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:30.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:30.706 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:14:30.787 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T00:14:31.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:31.275 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T00:14:31.331 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T00:14:31.331 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:14:31.331 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:14:31.332 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:14:31.332 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:14:31.332 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:14:31.332 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:14:31.332 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:31.332 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:31.332 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:31.332 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:14:31.332 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:14:31.332 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:14:31.333 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:14:31.333 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:14:31.333 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:14:31.333 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:14:31.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:14:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:14:31.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:14:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:31.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:31.480 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T00:14:31.488 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:14:31.489 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:14:31.489 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:14:31.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:31.490 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T00:14:31.552 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:14:31.552 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:14:31.605 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:14:31.605 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:14:31.605 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:14:31.656 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T00:14:31.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:31.656 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:14:31.744 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:14:31.744 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:14:31.756 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:14:31.756 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:14:31.756 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:14:31.756 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with new device syntax: 1@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:14:31.757 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:14:31.760 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:14:31.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:31.760 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:14:31.779 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:14:31.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:14:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:14:31.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:14:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:31.877 INFO:teuthology.orchestra.run.smithi086.stdout:client7200 2024-04-25T00:14:31.877 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:14:31.878 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:31.890 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:14:31.890 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:31.899 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:14:31.900 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:31.949 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:32.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.131+0000 7f3e3d800640 1 Processor -- start 2024-04-25T00:14:32.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.131+0000 7f3e3d800640 1 -- start start 2024-04-25T00:14:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e3d800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e380b4360 0x7f3e380b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e3d800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e380b2b30 0x7f3e380b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e3d800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e3806ef80 con 0x7f3e381553e0 2024-04-25T00:14:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e3d800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3e3815b510 con 0x7f3e380b2b30 2024-04-25T00:14:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e3d800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3e3815b7f0 con 0x7f3e380b4360 2024-04-25T00:14:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e36a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3e381553e0 0x7f3e3806e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41150/0 (socket says 172.21.15.83:41150) 2024-04-25T00:14:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e36a00640 1 -- 172.21.15.83:0/767464794 learned_addr learned my addr 172.21.15.83:0/767464794 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e37400640 1 --2- 172.21.15.83:0/767464794 >> v2:172.21.15.83:3300/0 conn(0x7f3e380b2b30 0x7f3e380b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e2e000640 1 --2- 172.21.15.83:0/767464794 >> v2:172.21.15.86:3300/0 conn(0x7f3e380b4360 0x7f3e380b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e36000640 1 -- 172.21.15.83:0/767464794 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902901052 0 0) 0x7f3e3806ef80 con 0x7f3e381553e0 2024-04-25T00:14:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e36000640 1 -- 172.21.15.83:0/767464794 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e1c005740 con 0x7f3e381553e0 2024-04-25T00:14:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e37400640 1 -- 172.21.15.83:0/767464794 >> v2:172.21.15.86:3300/0 conn(0x7f3e380b4360 msgr2=0x7f3e380b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e37400640 1 --2- 172.21.15.83:0/767464794 >> v2:172.21.15.86:3300/0 conn(0x7f3e380b4360 0x7f3e380b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e37400640 1 -- 172.21.15.83:0/767464794 >> v1:172.21.15.83:6789/0 conn(0x7f3e381553e0 legacy=0x7f3e3806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.135+0000 7f3e37400640 1 -- 172.21.15.83:0/767464794 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3815bad0 con 0x7f3e380b2b30 2024-04-25T00:14:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.139+0000 7f3e37400640 1 --2- 172.21.15.83:0/767464794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e380b2b30 0x7f3e380b2fc0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3e380cf5b0 tx=0x7f3e3001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962914eaed96b280 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.139+0000 7f3e36000640 1 -- 172.21.15.83:0/767464794 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300b2070 con 0x7f3e380b2b30 2024-04-25T00:14:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.139+0000 7f3e36000640 1 -- 172.21.15.83:0/767464794 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e300a5760 con 0x7f3e380b2b30 2024-04-25T00:14:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.139+0000 7f3e36000640 1 -- 172.21.15.83:0/767464794 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300a5b30 con 0x7f3e380b2b30 2024-04-25T00:14:32.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.139+0000 7f3e3d800640 1 -- 172.21.15.83:0/767464794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e380b2b30 msgr2=0x7f3e380b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.139+0000 7f3e3d800640 1 --2- 172.21.15.83:0/767464794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e380b2b30 0x7f3e380b2fc0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3e380cf5b0 tx=0x7f3e3001c8a0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.139+0000 7f3e3d800640 1 -- 172.21.15.83:0/767464794 shutdown_connections 2024-04-25T00:14:32.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.139+0000 7f3e3d800640 1 --2- 172.21.15.83:0/767464794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e380b2b30 0x7f3e380b2fc0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.139+0000 7f3e3d800640 1 --2- 172.21.15.83:0/767464794 >> v2:172.21.15.86:3300/0 conn(0x7f3e380b4360 0x7f3e380b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.139+0000 7f3e3d800640 1 -- 172.21.15.83:0/767464794 >> 172.21.15.83:0/767464794 conn(0x7f3e380b1830 msgr2=0x7f3e380b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:32.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.139+0000 7f3e3d800640 1 -- 172.21.15.83:0/767464794 shutdown_connections 2024-04-25T00:14:32.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.139+0000 7f3e3d800640 1 -- 172.21.15.83:0/767464794 wait complete. 2024-04-25T00:14:32.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.139+0000 7f3e3d800640 1 Processor -- start 2024-04-25T00:14:32.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.139+0000 7f3e3d800640 1 -- start start 2024-04-25T00:14:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e3d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e380b2b30 0x7f3e3814f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e3d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e380b4360 0x7f3e3814bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e3d800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e381553e0 0x7f3e3814c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e3d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3806ebd0 con 0x7f3e380b2b30 2024-04-25T00:14:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e3d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3806d3b0 con 0x7f3e380b4360 2024-04-25T00:14:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e3d800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3e38157430 con 0x7f3e381553e0 2024-04-25T00:14:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e36a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e380b2b30 0x7f3e3814f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e36a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e380b2b30 0x7f3e3814f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40960/0 (socket says 172.21.15.83:40960) 2024-04-25T00:14:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e36a00640 1 -- 172.21.15.83:0/96518807 learned_addr learned my addr 172.21.15.83:0/96518807 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e37400640 1 --2- 172.21.15.83:0/96518807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e381553e0 0x7f3e3814c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e2e000640 1 --2- 172.21.15.83:0/96518807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e380b4360 0x7f3e3814bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e36a00640 1 -- 172.21.15.83:0/96518807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e381553e0 msgr2=0x7f3e3814c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e36a00640 1 --2- 172.21.15.83:0/96518807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e381553e0 0x7f3e3814c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e36a00640 1 -- 172.21.15.83:0/96518807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e380b4360 msgr2=0x7f3e3814bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e36a00640 1 --2- 172.21.15.83:0/96518807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e380b4360 0x7f3e3814bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e36a00640 1 -- 172.21.15.83:0/96518807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3806e9b0 con 0x7f3e380b2b30 2024-04-25T00:14:32.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e36a00640 1 --2- 172.21.15.83:0/96518807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e380b2b30 0x7f3e3814f0c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800c3f0 tx=0x7f3e2800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:32.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e2f400640 1 -- 172.21.15.83:0/96518807 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28017070 con 0x7f3e380b2b30 2024-04-25T00:14:32.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e2f400640 1 -- 172.21.15.83:0/96518807 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e28005cf0 con 0x7f3e380b2b30 2024-04-25T00:14:32.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e3d800640 1 -- 172.21.15.83:0/96518807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3814c920 con 0x7f3e380b2b30 2024-04-25T00:14:32.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e2f400640 1 -- 172.21.15.83:0/96518807 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e280072f0 con 0x7f3e380b2b30 2024-04-25T00:14:32.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e3d800640 1 -- 172.21.15.83:0/96518807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e38157960 con 0x7f3e380b2b30 2024-04-25T00:14:32.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e3d800640 1 -- 172.21.15.83:0/96518807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e380737f0 con 0x7f3e380b2b30 2024-04-25T00:14:32.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.143+0000 7f3e2f400640 1 -- 172.21.15.83:0/96518807 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3e2801a070 con 0x7f3e380b2b30 2024-04-25T00:14:32.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.147+0000 7f3e2f400640 1 --2- 172.21.15.83:0/96518807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e140678c0 0x7f3e14069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.147+0000 7f3e2f400640 1 -- 172.21.15.83:0/96518807 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f3e28085260 con 0x7f3e380b2b30 2024-04-25T00:14:32.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.147+0000 7f3e2e000640 1 --2- 172.21.15.83:0/96518807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e140678c0 0x7f3e14069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.147+0000 7f3e2e000640 1 --2- 172.21.15.83:0/96518807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e140678c0 0x7f3e14069d80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3e38154c40 tx=0x7f3e20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:32.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.151+0000 7f3e2f400640 1 -- 172.21.15.83:0/96518807 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3e28055520 con 0x7f3e380b2b30 2024-04-25T00:14:32.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.307+0000 7f3e3d800640 1 -- 172.21.15.83:0/96518807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e3804f5f0 con 0x7f3e380b2b30 2024-04-25T00:14:32.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.307+0000 7f3e2f400640 1 -- 172.21.15.83:0/96518807 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16801 (secure 0 0 0) 0x7f3e28059820 con 0x7f3e380b2b30 2024-04-25T00:14:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:32.311 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:24.352251+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:21.082598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:21.713421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:32.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.307+0000 7f3e3d800640 1 -- 172.21.15.83:0/96518807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e140678c0 msgr2=0x7f3e14069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.307+0000 7f3e3d800640 1 --2- 172.21.15.83:0/96518807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e140678c0 0x7f3e14069d80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3e38154c40 tx=0x7f3e20008040 comp rx=0 tx=0).stop 2024-04-25T00:14:32.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.307+0000 7f3e3d800640 1 -- 172.21.15.83:0/96518807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e380b2b30 msgr2=0x7f3e3814f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.307+0000 7f3e3d800640 1 --2- 172.21.15.83:0/96518807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e380b2b30 0x7f3e3814f0c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800c3f0 tx=0x7f3e2800c8b0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.307+0000 7f3e3d800640 1 -- 172.21.15.83:0/96518807 shutdown_connections 2024-04-25T00:14:32.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.307+0000 7f3e3d800640 1 --2- 172.21.15.83:0/96518807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e140678c0 0x7f3e14069d80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.307+0000 7f3e3d800640 1 --2- 172.21.15.83:0/96518807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e381553e0 0x7f3e3814c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.307+0000 7f3e3d800640 1 --2- 172.21.15.83:0/96518807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e380b4360 0x7f3e3814bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.307+0000 7f3e3d800640 1 --2- 172.21.15.83:0/96518807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e380b2b30 0x7f3e3814f0c0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.307+0000 7f3e3d800640 1 -- 172.21.15.83:0/96518807 >> 172.21.15.83:0/96518807 conn(0x7f3e380b1830 msgr2=0x7f3e38071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:32.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.307+0000 7f3e3d800640 1 -- 172.21.15.83:0/96518807 shutdown_connections 2024-04-25T00:14:32.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.311+0000 7f3e3d800640 1 -- 172.21.15.83:0/96518807 wait complete. 2024-04-25T00:14:32.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:14:21.082598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:21.713421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:14:32.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:14:32.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.467+0000 7ff30c000640 1 Processor -- start 2024-04-25T00:14:32.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff30c000640 1 -- start start 2024-04-25T00:14:32.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff30c000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff30415e0e0 0x7ff3040d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff30c000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff30415e4c0 0x7ff3040ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff30c000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3040b4b10 con 0x7ff3040cea80 2024-04-25T00:14:32.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff30c000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff3040d0f50 con 0x7ff30415e4c0 2024-04-25T00:14:32.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff30c000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff3040cc310 con 0x7ff30415e0e0 2024-04-25T00:14:32.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff30a000640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff3040cea80 0x7ff30406c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41152/0 (socket says 172.21.15.83:41152) 2024-04-25T00:14:32.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff30a000640 1 -- 172.21.15.83:0/3302274166 learned_addr learned my addr 172.21.15.83:0/3302274166 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:32.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff308c00640 1 --2- 172.21.15.83:0/3302274166 >> v2:172.21.15.86:3300/0 conn(0x7ff30415e4c0 0x7ff3040ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff309600640 1 --2- 172.21.15.83:0/3302274166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff30415e0e0 0x7ff3040d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff303e00640 1 -- 172.21.15.83:0/3302274166 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2662188581 0 0) 0x7ff2fc0014a0 con 0x7ff3040cea80 2024-04-25T00:14:32.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff303e00640 1 -- 172.21.15.83:0/3302274166 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2ec003530 con 0x7ff3040cea80 2024-04-25T00:14:32.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff309600640 1 -- 172.21.15.83:0/3302274166 >> v2:172.21.15.86:3300/0 conn(0x7ff30415e4c0 msgr2=0x7ff3040ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:32.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff309600640 1 --2- 172.21.15.83:0/3302274166 >> v2:172.21.15.86:3300/0 conn(0x7ff30415e4c0 0x7ff3040ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff309600640 1 -- 172.21.15.83:0/3302274166 >> v1:172.21.15.83:6789/0 conn(0x7ff3040cea80 legacy=0x7ff30406c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff309600640 1 -- 172.21.15.83:0/3302274166 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3040d1180 con 0x7ff30415e0e0 2024-04-25T00:14:32.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.471+0000 7ff308c00640 1 --2- 172.21.15.83:0/3302274166 >> v2:172.21.15.86:3300/0 conn(0x7ff30415e4c0 0x7ff3040ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.475+0000 7ff309600640 1 --2- 172.21.15.83:0/3302274166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff30415e0e0 0x7ff3040d0a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40097c0 tx=0x7ff2f401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dae1366e8401357 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.475+0000 7ff303e00640 1 -- 172.21.15.83:0/3302274166 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40b2070 con 0x7ff30415e0e0 2024-04-25T00:14:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.475+0000 7ff303e00640 1 -- 172.21.15.83:0/3302274166 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f40ad640 con 0x7ff30415e0e0 2024-04-25T00:14:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.475+0000 7ff303e00640 1 -- 172.21.15.83:0/3302274166 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40ada10 con 0x7ff30415e0e0 2024-04-25T00:14:32.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.475+0000 7ff30c000640 1 -- 172.21.15.83:0/3302274166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff30415e0e0 msgr2=0x7ff3040d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.475+0000 7ff30c000640 1 --2- 172.21.15.83:0/3302274166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff30415e0e0 0x7ff3040d0a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40097c0 tx=0x7ff2f401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.475+0000 7ff30c000640 1 -- 172.21.15.83:0/3302274166 shutdown_connections 2024-04-25T00:14:32.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.475+0000 7ff30c000640 1 --2- 172.21.15.83:0/3302274166 >> v2:172.21.15.86:3300/0 conn(0x7ff30415e4c0 0x7ff3040ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.475+0000 7ff30c000640 1 --2- 172.21.15.83:0/3302274166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff30415e0e0 0x7ff3040d0a10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.475+0000 7ff30c000640 1 -- 172.21.15.83:0/3302274166 >> 172.21.15.83:0/3302274166 conn(0x7ff3040d1cc0 msgr2=0x7ff3040d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:32.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.475+0000 7ff30c000640 1 -- 172.21.15.83:0/3302274166 shutdown_connections 2024-04-25T00:14:32.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.475+0000 7ff30c000640 1 -- 172.21.15.83:0/3302274166 wait complete. 2024-04-25T00:14:32.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.475+0000 7ff30c000640 1 Processor -- start 2024-04-25T00:14:32.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30c000640 1 -- start start 2024-04-25T00:14:32.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30c000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3040cea80 0x7ff304157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30415e0e0 0x7ff304156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30c000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30415e4c0 0x7ff3041faba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30c000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff30416aed0 con 0x7ff30415e4c0 2024-04-25T00:14:32.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3040ce050 con 0x7ff30415e0e0 2024-04-25T00:14:32.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30c000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff30416b050 con 0x7ff3040cea80 2024-04-25T00:14:32.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30415e4c0 0x7ff3041faba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30415e4c0 0x7ff3041faba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40972/0 (socket says 172.21.15.83:40972) 2024-04-25T00:14:32.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30a000640 1 -- 172.21.15.83:0/4101583553 learned_addr learned my addr 172.21.15.83:0/4101583553 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:32.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff309600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3040cea80 0x7ff304157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff308c00640 1 --2- 172.21.15.83:0/4101583553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30415e0e0 0x7ff304156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30a000640 1 -- 172.21.15.83:0/4101583553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3040cea80 msgr2=0x7ff304157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30a000640 1 --2- 172.21.15.83:0/4101583553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3040cea80 0x7ff304157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30a000640 1 -- 172.21.15.83:0/4101583553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30415e0e0 msgr2=0x7ff304156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30a000640 1 --2- 172.21.15.83:0/4101583553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30415e0e0 0x7ff304156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30a000640 1 -- 172.21.15.83:0/4101583553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30406efd0 con 0x7ff30415e4c0 2024-04-25T00:14:32.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30a000640 1 --2- 172.21.15.83:0/4101583553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30415e4c0 0x7ff3041faba0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00cb10 tx=0x7ff2fc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:32.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff301600640 1 -- 172.21.15.83:0/4101583553 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc017070 con 0x7ff30415e4c0 2024-04-25T00:14:32.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff301600640 1 -- 172.21.15.83:0/4101583553 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2fc00ac10 con 0x7ff30415e4c0 2024-04-25T00:14:32.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff301600640 1 -- 172.21.15.83:0/4101583553 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc012400 con 0x7ff30415e4c0 2024-04-25T00:14:32.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30c000640 1 -- 172.21.15.83:0/4101583553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff304155290 con 0x7ff30415e4c0 2024-04-25T00:14:32.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30c000640 1 -- 172.21.15.83:0/4101583553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff304155850 con 0x7ff30415e4c0 2024-04-25T00:14:32.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.479+0000 7ff30c000640 1 -- 172.21.15.83:0/4101583553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff30406f1a0 con 0x7ff30415e4c0 2024-04-25T00:14:32.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.483+0000 7ff301600640 1 -- 172.21.15.83:0/4101583553 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff2fc01a070 con 0x7ff30415e4c0 2024-04-25T00:14:32.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.483+0000 7ff301600640 1 --2- 172.21.15.83:0/4101583553 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff2e80678c0 0x7ff2e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.483+0000 7ff301600640 1 -- 172.21.15.83:0/4101583553 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7ff2fc085b20 con 0x7ff30415e4c0 2024-04-25T00:14:32.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.483+0000 7ff309600640 1 --2- 172.21.15.83:0/4101583553 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff2e80678c0 0x7ff2e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.483+0000 7ff309600640 1 --2- 172.21.15.83:0/4101583553 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff2e80678c0 0x7ff2e8069d80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40af000 tx=0x7ff2f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:32.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.487+0000 7ff301600640 1 -- 172.21.15.83:0/4101583553 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff3040b4b10 con 0x7ff30415e4c0 2024-04-25T00:14:32.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.643+0000 7ff30c000640 1 -- 172.21.15.83:0/4101583553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff3041572f0 con 0x7ff30415e4c0 2024-04-25T00:14:32.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.643+0000 7ff301600640 1 -- 172.21.15.83:0/4101583553 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7ff2fc055e70 con 0x7ff30415e4c0 2024-04-25T00:14:32.649 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:14:32.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.647+0000 7ff30c000640 1 -- 172.21.15.83:0/4101583553 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff2e80678c0 msgr2=0x7ff2e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.647+0000 7ff30c000640 1 --2- 172.21.15.83:0/4101583553 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff2e80678c0 0x7ff2e8069d80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40af000 tx=0x7ff2f4008040 comp rx=0 tx=0).stop 2024-04-25T00:14:32.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.647+0000 7ff30c000640 1 -- 172.21.15.83:0/4101583553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30415e4c0 msgr2=0x7ff3041faba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.647+0000 7ff30c000640 1 --2- 172.21.15.83:0/4101583553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30415e4c0 0x7ff3041faba0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00cb10 tx=0x7ff2fc00a060 comp rx=0 tx=0).stop 2024-04-25T00:14:32.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.647+0000 7ff30c000640 1 -- 172.21.15.83:0/4101583553 shutdown_connections 2024-04-25T00:14:32.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.647+0000 7ff30c000640 1 --2- 172.21.15.83:0/4101583553 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff2e80678c0 0x7ff2e8069d80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.647+0000 7ff30c000640 1 --2- 172.21.15.83:0/4101583553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30415e4c0 0x7ff3041faba0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.647+0000 7ff30c000640 1 --2- 172.21.15.83:0/4101583553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30415e0e0 0x7ff304156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.647+0000 7ff30c000640 1 --2- 172.21.15.83:0/4101583553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3040cea80 0x7ff304157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.647+0000 7ff30c000640 1 -- 172.21.15.83:0/4101583553 >> 172.21.15.83:0/4101583553 conn(0x7ff3040d1cc0 msgr2=0x7ff3040b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:32.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.647+0000 7ff30c000640 1 -- 172.21.15.83:0/4101583553 shutdown_connections 2024-04-25T00:14:32.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.647+0000 7ff30c000640 1 -- 172.21.15.83:0/4101583553 wait complete. 2024-04-25T00:14:32.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:14:32.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.803+0000 7f587ce00640 1 Processor -- start 2024-04-25T00:14:32.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.803+0000 7f587ce00640 1 -- start start 2024-04-25T00:14:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f587ce00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58781551e0 0x7f587815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f587ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f587806e4c0 0x7f587806e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f587ce00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f587815c560 con 0x7f58781555c0 2024-04-25T00:14:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f587ce00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f587806ee70 con 0x7f587806e4c0 2024-04-25T00:14:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f587ce00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5878072460 con 0x7f58781551e0 2024-04-25T00:14:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f586d400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f58781555c0 0x7f5878071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41166/0 (socket says 172.21.15.83:41166) 2024-04-25T00:14:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f586d400640 1 -- 172.21.15.83:0/1458156537 learned_addr learned my addr 172.21.15.83:0/1458156537 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f5875e00640 1 --2- 172.21.15.83:0/1458156537 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58781551e0 0x7f587815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f5876800640 1 --2- 172.21.15.83:0/1458156537 >> v2:172.21.15.86:3300/0 conn(0x7f587806e4c0 0x7f587806e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f5875400640 1 -- 172.21.15.83:0/1458156537 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1050773986 0 0) 0x7f587815c560 con 0x7f58781555c0 2024-04-25T00:14:32.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f5875400640 1 -- 172.21.15.83:0/1458156537 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f585c003530 con 0x7f58781555c0 2024-04-25T00:14:32.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f5875e00640 1 -- 172.21.15.83:0/1458156537 >> v2:172.21.15.86:3300/0 conn(0x7f587806e4c0 msgr2=0x7f587806e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:32.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f5875e00640 1 --2- 172.21.15.83:0/1458156537 >> v2:172.21.15.86:3300/0 conn(0x7f587806e4c0 0x7f587806e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f5875e00640 1 -- 172.21.15.83:0/1458156537 >> v1:172.21.15.83:6789/0 conn(0x7f58781555c0 legacy=0x7f5878071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.807+0000 7f5875e00640 1 -- 172.21.15.83:0/1458156537 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587815c7d0 con 0x7f58781551e0 2024-04-25T00:14:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.811+0000 7f5875e00640 1 --2- 172.21.15.83:0/1458156537 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58781551e0 0x7f587815bea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58680097c0 tx=0x7f586801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61f016492acf78ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.811+0000 7f5875400640 1 -- 172.21.15.83:0/1458156537 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680b0070 con 0x7f58781551e0 2024-04-25T00:14:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.811+0000 7f5875400640 1 -- 172.21.15.83:0/1458156537 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58680ab600 con 0x7f58781551e0 2024-04-25T00:14:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.811+0000 7f5875400640 1 -- 172.21.15.83:0/1458156537 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680ab9d0 con 0x7f58781551e0 2024-04-25T00:14:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.811+0000 7f587ce00640 1 -- 172.21.15.83:0/1458156537 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58781551e0 msgr2=0x7f587815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.811+0000 7f587ce00640 1 --2- 172.21.15.83:0/1458156537 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58781551e0 0x7f587815bea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58680097c0 tx=0x7f586801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.811+0000 7f587ce00640 1 -- 172.21.15.83:0/1458156537 shutdown_connections 2024-04-25T00:14:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.811+0000 7f587ce00640 1 --2- 172.21.15.83:0/1458156537 >> v2:172.21.15.86:3300/0 conn(0x7f587806e4c0 0x7f587806e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.811+0000 7f587ce00640 1 --2- 172.21.15.83:0/1458156537 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58781551e0 0x7f587815bea0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.811+0000 7f587ce00640 1 -- 172.21.15.83:0/1458156537 >> 172.21.15.83:0/1458156537 conn(0x7f58780b1830 msgr2=0x7f58780b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.811+0000 7f587ce00640 1 -- 172.21.15.83:0/1458156537 shutdown_connections 2024-04-25T00:14:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.811+0000 7f587ce00640 1 -- 172.21.15.83:0/1458156537 wait complete. 2024-04-25T00:14:32.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.811+0000 7f587ce00640 1 Processor -- start 2024-04-25T00:14:32.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.811+0000 7f587ce00640 1 -- start start 2024-04-25T00:14:32.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f587ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f587806e4c0 0x7f5878145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f587ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58781551e0 0x7f587814d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f587ce00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58781555c0 0x7f58781495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f587ce00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f587806df50 con 0x7f587806e4c0 2024-04-25T00:14:32.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f587ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f587806ddd0 con 0x7f58781551e0 2024-04-25T00:14:32.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f587ce00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f587806e0d0 con 0x7f58781555c0 2024-04-25T00:14:32.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f5875e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f587806e4c0 0x7f5878145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f5875e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f587806e4c0 0x7f5878145530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40976/0 (socket says 172.21.15.83:40976) 2024-04-25T00:14:32.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f5875e00640 1 -- 172.21.15.83:0/370767958 learned_addr learned my addr 172.21.15.83:0/370767958 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:32.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f5875e00640 1 -- 172.21.15.83:0/370767958 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58781555c0 msgr2=0x7f58781495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f5875e00640 1 --2- 172.21.15.83:0/370767958 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58781555c0 0x7f58781495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f5875e00640 1 -- 172.21.15.83:0/370767958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58781551e0 msgr2=0x7f587814d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f5875e00640 1 --2- 172.21.15.83:0/370767958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58781551e0 0x7f587814d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f5875e00640 1 -- 172.21.15.83:0/370767958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58780af7a0 con 0x7f587806e4c0 2024-04-25T00:14:32.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f5875e00640 1 --2- 172.21.15.83:0/370767958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f587806e4c0 0x7f5878145530 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f586801a0d0 tx=0x7f586801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:32.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f586ea00640 1 -- 172.21.15.83:0/370767958 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680b0050 con 0x7f587806e4c0 2024-04-25T00:14:32.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f586ea00640 1 -- 172.21.15.83:0/370767958 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58680bc070 con 0x7f587806e4c0 2024-04-25T00:14:32.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f587ce00640 1 -- 172.21.15.83:0/370767958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58780afa80 con 0x7f587806e4c0 2024-04-25T00:14:32.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f586ea00640 1 -- 172.21.15.83:0/370767958 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680b5830 con 0x7f587806e4c0 2024-04-25T00:14:32.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f587ce00640 1 -- 172.21.15.83:0/370767958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58780b0040 con 0x7f587806e4c0 2024-04-25T00:14:32.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f587ce00640 1 -- 172.21.15.83:0/370767958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587804f510 con 0x7f587806e4c0 2024-04-25T00:14:32.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.815+0000 7f586ea00640 1 -- 172.21.15.83:0/370767958 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f58680b59d0 con 0x7f587806e4c0 2024-04-25T00:14:32.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.819+0000 7f586ea00640 1 --2- 172.21.15.83:0/370767958 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f58540678c0 0x7f5854069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:32.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.819+0000 7f586ea00640 1 -- 172.21.15.83:0/370767958 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f5868120520 con 0x7f587806e4c0 2024-04-25T00:14:32.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.819+0000 7f586d400640 1 --2- 172.21.15.83:0/370767958 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f58540678c0 0x7f5854069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:32.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.819+0000 7f586d400640 1 --2- 172.21.15.83:0/370767958 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f58540678c0 0x7f5854069d80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5860007fd0 tx=0x7f5860007940 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:32.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.823+0000 7f586ea00640 1 -- 172.21.15.83:0/370767958 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5868016090 con 0x7f587806e4c0 2024-04-25T00:14:32.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.979+0000 7f587ce00640 1 -- 172.21.15.83:0/370767958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f58780ce5d0 con 0x7f587806e4c0 2024-04-25T00:14:32.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.979+0000 7f586ea00640 1 -- 172.21.15.83:0/370767958 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f58680f0e30 con 0x7f587806e4c0 2024-04-25T00:14:32.982 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:14:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.979+0000 7f587ce00640 1 -- 172.21.15.83:0/370767958 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f58540678c0 msgr2=0x7f5854069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.979+0000 7f587ce00640 1 --2- 172.21.15.83:0/370767958 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f58540678c0 0x7f5854069d80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5860007fd0 tx=0x7f5860007940 comp rx=0 tx=0).stop 2024-04-25T00:14:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.979+0000 7f587ce00640 1 -- 172.21.15.83:0/370767958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f587806e4c0 msgr2=0x7f5878145530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.979+0000 7f587ce00640 1 --2- 172.21.15.83:0/370767958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f587806e4c0 0x7f5878145530 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f586801a0d0 tx=0x7f586801ace0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.979+0000 7f587ce00640 1 -- 172.21.15.83:0/370767958 shutdown_connections 2024-04-25T00:14:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.979+0000 7f587ce00640 1 --2- 172.21.15.83:0/370767958 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f58540678c0 0x7f5854069d80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.979+0000 7f587ce00640 1 --2- 172.21.15.83:0/370767958 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58781555c0 0x7f58781495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.979+0000 7f587ce00640 1 --2- 172.21.15.83:0/370767958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58781551e0 0x7f587814d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.979+0000 7f587ce00640 1 --2- 172.21.15.83:0/370767958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f587806e4c0 0x7f5878145530 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.979+0000 7f587ce00640 1 -- 172.21.15.83:0/370767958 >> 172.21.15.83:0/370767958 conn(0x7f58780b1830 msgr2=0x7f58780cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.979+0000 7f587ce00640 1 -- 172.21.15.83:0/370767958 shutdown_connections 2024-04-25T00:14:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:32.979+0000 7f587ce00640 1 -- 172.21.15.83:0/370767958 wait complete. 2024-04-25T00:14:33.003 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:14:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.135+0000 7fa760800640 1 Processor -- start 2024-04-25T00:14:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.135+0000 7fa760800640 1 -- start start 2024-04-25T00:14:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.135+0000 7fa760800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa75815de90 0x7fa758071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa760800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa75815e270 0x7fa758071c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa760800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa75806dcf0 con 0x7fa75806e840 2024-04-25T00:14:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa760800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa7580b4460 con 0x7fa75815de90 2024-04-25T00:14:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa760800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa7580b2ce0 con 0x7fa75815e270 2024-04-25T00:14:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa75e800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa75806e840 0x7fa758161bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41170/0 (socket says 172.21.15.83:41170) 2024-04-25T00:14:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa75de00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa75815de90 0x7fa758071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa75e800640 1 -- 172.21.15.83:0/3645179126 learned_addr learned my addr 172.21.15.83:0/3645179126 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa75d400640 1 --2- 172.21.15.83:0/3645179126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa75815e270 0x7fa758071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa75de00640 1 -- 172.21.15.83:0/3645179126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa75815e270 msgr2=0x7fa758071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa75de00640 1 --2- 172.21.15.83:0/3645179126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa75815e270 0x7fa758071c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa75ca00640 1 -- 172.21.15.83:0/3645179126 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2691503055 0 0) 0x7fa75806dcf0 con 0x7fa75806e840 2024-04-25T00:14:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa75de00640 1 -- 172.21.15.83:0/3645179126 >> v1:172.21.15.83:6789/0 conn(0x7fa75806e840 legacy=0x7fa758161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa75de00640 1 -- 172.21.15.83:0/3645179126 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75806dfd0 con 0x7fa75815de90 2024-04-25T00:14:33.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa75de00640 1 --2- 172.21.15.83:0/3645179126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75815de90 0x7fa758071740 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0097c0 tx=0x7fa74c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a2876fe44db35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:33.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.139+0000 7fa75ca00640 1 -- 172.21.15.83:0/3645179126 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0b0070 con 0x7fa75815de90 2024-04-25T00:14:33.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa75ca00640 1 -- 172.21.15.83:0/3645179126 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74c0ab600 con 0x7fa75815de90 2024-04-25T00:14:33.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa75ca00640 1 -- 172.21.15.83:0/3645179126 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0ab9d0 con 0x7fa75815de90 2024-04-25T00:14:33.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 -- 172.21.15.83:0/3645179126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75815de90 msgr2=0x7fa758071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 --2- 172.21.15.83:0/3645179126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75815de90 0x7fa758071740 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0097c0 tx=0x7fa74c01c410 comp rx=0 tx=0).stop 2024-04-25T00:14:33.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 -- 172.21.15.83:0/3645179126 shutdown_connections 2024-04-25T00:14:33.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 --2- 172.21.15.83:0/3645179126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa75815e270 0x7fa758071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 --2- 172.21.15.83:0/3645179126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75815de90 0x7fa758071740 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 -- 172.21.15.83:0/3645179126 >> 172.21.15.83:0/3645179126 conn(0x7fa7580b1830 msgr2=0x7fa7580b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:33.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 -- 172.21.15.83:0/3645179126 shutdown_connections 2024-04-25T00:14:33.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 -- 172.21.15.83:0/3645179126 wait complete. 2024-04-25T00:14:33.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 Processor -- start 2024-04-25T00:14:33.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 -- start start 2024-04-25T00:14:33.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa75806e840 0x7fa758160f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75815de90 0x7fa7581481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75815e270 0x7fa7581486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa75816ad60 con 0x7fa75815de90 2024-04-25T00:14:33.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580ad860 con 0x7fa75815e270 2024-04-25T00:14:33.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa760800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa75816aee0 con 0x7fa75806e840 2024-04-25T00:14:33.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa75d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75815de90 0x7fa7581481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa75d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75815de90 0x7fa7581481b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40996/0 (socket says 172.21.15.83:40996) 2024-04-25T00:14:33.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa75d400640 1 -- 172.21.15.83:0/2013090548 learned_addr learned my addr 172.21.15.83:0/2013090548 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:33.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa75de00640 1 --2- 172.21.15.83:0/2013090548 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa75806e840 0x7fa758160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa75e800640 1 --2- 172.21.15.83:0/2013090548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75815e270 0x7fa7581486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa75d400640 1 -- 172.21.15.83:0/2013090548 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa75806e840 msgr2=0x7fa758160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa75d400640 1 --2- 172.21.15.83:0/2013090548 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa75806e840 0x7fa758160f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa75d400640 1 -- 172.21.15.83:0/2013090548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75815e270 msgr2=0x7fa7581486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa75d400640 1 --2- 172.21.15.83:0/2013090548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75815e270 0x7fa7581486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.143+0000 7fa75d400640 1 -- 172.21.15.83:0/2013090548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa758147cf0 con 0x7fa75815de90 2024-04-25T00:14:33.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.147+0000 7fa75d400640 1 --2- 172.21.15.83:0/2013090548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75815de90 0x7fa7581481b0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa74800e960 tx=0x7fa74800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:33.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.147+0000 7fa752000640 1 -- 172.21.15.83:0/2013090548 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748017070 con 0x7fa75815de90 2024-04-25T00:14:33.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.147+0000 7fa752000640 1 -- 172.21.15.83:0/2013090548 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74800ca00 con 0x7fa75815de90 2024-04-25T00:14:33.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.147+0000 7fa752000640 1 -- 172.21.15.83:0/2013090548 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74800cdb0 con 0x7fa75815de90 2024-04-25T00:14:33.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.147+0000 7fa760800640 1 -- 172.21.15.83:0/2013090548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa758161450 con 0x7fa75815de90 2024-04-25T00:14:33.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.147+0000 7fa760800640 1 -- 172.21.15.83:0/2013090548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa75815ead0 con 0x7fa75815de90 2024-04-25T00:14:33.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.147+0000 7fa760800640 1 -- 172.21.15.83:0/2013090548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa75804f510 con 0x7fa75815de90 2024-04-25T00:14:33.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.147+0000 7fa752000640 1 -- 172.21.15.83:0/2013090548 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa74801a070 con 0x7fa75815de90 2024-04-25T00:14:33.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.147+0000 7fa752000640 1 --2- 172.21.15.83:0/2013090548 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa73c067910 0x7fa73c069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.147+0000 7fa752000640 1 -- 172.21.15.83:0/2013090548 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fa748085e20 con 0x7fa75815de90 2024-04-25T00:14:33.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.147+0000 7fa75de00640 1 --2- 172.21.15.83:0/2013090548 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa73c067910 0x7fa73c069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.151+0000 7fa75de00640 1 --2- 172.21.15.83:0/2013090548 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa73c067910 0x7fa73c069dd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0ad000 tx=0x7fa74c0a4f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:33.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.151+0000 7fa752000640 1 -- 172.21.15.83:0/2013090548 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa748056160 con 0x7fa75815de90 2024-04-25T00:14:33.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.307+0000 7fa760800640 1 -- 172.21.15.83:0/2013090548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa758156e10 con 0x7fa75815de90 2024-04-25T00:14:33.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.307+0000 7fa752000640 1 -- 172.21.15.83:0/2013090548 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7fa74805a460 con 0x7fa75815de90 2024-04-25T00:14:33.311 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:14:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.311+0000 7fa760800640 1 -- 172.21.15.83:0/2013090548 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa73c067910 msgr2=0x7fa73c069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.311+0000 7fa760800640 1 --2- 172.21.15.83:0/2013090548 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa73c067910 0x7fa73c069dd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0ad000 tx=0x7fa74c0a4f90 comp rx=0 tx=0).stop 2024-04-25T00:14:33.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.311+0000 7fa760800640 1 -- 172.21.15.83:0/2013090548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75815de90 msgr2=0x7fa7581481b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.311+0000 7fa760800640 1 --2- 172.21.15.83:0/2013090548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75815de90 0x7fa7581481b0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa74800e960 tx=0x7fa74800ee20 comp rx=0 tx=0).stop 2024-04-25T00:14:33.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.311+0000 7fa760800640 1 -- 172.21.15.83:0/2013090548 shutdown_connections 2024-04-25T00:14:33.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.311+0000 7fa760800640 1 --2- 172.21.15.83:0/2013090548 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa73c067910 0x7fa73c069dd0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.311+0000 7fa760800640 1 --2- 172.21.15.83:0/2013090548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa75815e270 0x7fa7581486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.311+0000 7fa760800640 1 --2- 172.21.15.83:0/2013090548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75815de90 0x7fa7581481b0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.311+0000 7fa760800640 1 --2- 172.21.15.83:0/2013090548 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa75806e840 0x7fa758160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.311+0000 7fa760800640 1 -- 172.21.15.83:0/2013090548 >> 172.21.15.83:0/2013090548 conn(0x7fa7580b1830 msgr2=0x7fa7580b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:33.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.311+0000 7fa760800640 1 -- 172.21.15.83:0/2013090548 shutdown_connections 2024-04-25T00:14:33.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.311+0000 7fa760800640 1 -- 172.21.15.83:0/2013090548 wait complete. 2024-04-25T00:14:33.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:14:33.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e44c00640 1 Processor -- start 2024-04-25T00:14:33.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e44c00640 1 -- start start 2024-04-25T00:14:33.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e44c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e400cab50 0x7f6e4015f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e44c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e400cb210 con 0x7f6e401446e0 2024-04-25T00:14:33.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e44c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e400cb4f0 con 0x7f6e4006ee20 2024-04-25T00:14:33.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e44c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6e400715b0 con 0x7f6e400cab50 2024-04-25T00:14:33.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3de00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6e401446e0 0x7f6e4006e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41176/0 (socket says 172.21.15.83:41176) 2024-04-25T00:14:33.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3de00640 1 -- 172.21.15.83:0/406075246 learned_addr learned my addr 172.21.15.83:0/406075246 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:33.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 -- 172.21.15.83:0/406075246 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3190965546 0 0) 0x7f6e400cb210 con 0x7f6e401446e0 2024-04-25T00:14:33.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 -- 172.21.15.83:0/406075246 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e24005740 con 0x7f6e401446e0 2024-04-25T00:14:33.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3e800640 1 --2- 172.21.15.83:0/406075246 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e400cab50 0x7f6e4015f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 -- 172.21.15.83:0/406075246 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 370042903 0 0) 0x7f6e24005740 con 0x7f6e401446e0 2024-04-25T00:14:33.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 -- 172.21.15.83:0/406075246 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e400cab50 msgr2=0x7f6e4015f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:33.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 --2- 172.21.15.83:0/406075246 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e400cab50 0x7f6e4015f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 -- 172.21.15.83:0/406075246 >> v1:172.21.15.86:6789/0 conn(0x7f6e4006ee20 legacy=0x7f6e40071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 -- 172.21.15.83:0/406075246 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e400717e0 con 0x7f6e401446e0 2024-04-25T00:14:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 -- 172.21.15.83:0/406075246 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f6e30001680 con 0x7f6e401446e0 2024-04-25T00:14:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 -- 172.21.15.83:0/406075246 >> v1:172.21.15.83:6789/0 conn(0x7f6e401446e0 legacy=0x7f6e4006e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 --2- 172.21.15.83:0/406075246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e24005c00 0x7f6e24006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 --2- 172.21.15.83:0/406075246 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e400cab50 0x7f6e24006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 --2- 172.21.15.83:0/406075246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e24006f60 0x7f6e240073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 -- 172.21.15.83:0/406075246 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2400dd70 con 0x7f6e24006f60 2024-04-25T00:14:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 -- 172.21.15.83:0/406075246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2400def0 con 0x7f6e24005c00 2024-04-25T00:14:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.467+0000 7f6e3d400640 1 -- 172.21.15.83:0/406075246 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6e2400e0c0 con 0x7f6e400cab50 2024-04-25T00:14:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.471+0000 7f6e35400640 1 --2- 172.21.15.83:0/406075246 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e400cab50 0x7f6e24006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.471+0000 7f6e3de00640 1 --2- 172.21.15.83:0/406075246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e24005c00 0x7f6e24006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.471+0000 7f6e3e800640 1 --2- 172.21.15.83:0/406075246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e24006f60 0x7f6e240073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.471+0000 7f6e3de00640 1 -- 172.21.15.83:0/406075246 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e400cab50 msgr2=0x7f6e24006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:33.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.471+0000 7f6e3de00640 1 --2- 172.21.15.83:0/406075246 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e400cab50 0x7f6e24006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.471+0000 7f6e3de00640 1 -- 172.21.15.83:0/406075246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e24006f60 msgr2=0x7f6e240073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:33.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.471+0000 7f6e3de00640 1 --2- 172.21.15.83:0/406075246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e24006f60 0x7f6e240073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.471+0000 7f6e3de00640 1 -- 172.21.15.83:0/406075246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e24005740 con 0x7f6e24005c00 2024-04-25T00:14:33.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.471+0000 7f6e3e800640 1 --2- 172.21.15.83:0/406075246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e24006f60 0x7f6e240073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:33.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.471+0000 7f6e35400640 1 --2- 172.21.15.83:0/406075246 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e400cab50 0x7f6e24006a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:33.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.471+0000 7f6e3de00640 1 --2- 172.21.15.83:0/406075246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e24005c00 0x7f6e24006020 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000c240 tx=0x7f6e3001cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74315cb958b9798d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:33.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.471+0000 7f6e3d400640 1 -- 172.21.15.83:0/406075246 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30007d20 con 0x7f6e24005c00 2024-04-25T00:14:33.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.471+0000 7f6e3d400640 1 -- 172.21.15.83:0/406075246 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e300a9070 con 0x7f6e24005c00 2024-04-25T00:14:33.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.471+0000 7f6e3d400640 1 -- 172.21.15.83:0/406075246 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e300a5de0 con 0x7f6e24005c00 2024-04-25T00:14:33.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e44c00640 1 -- 172.21.15.83:0/406075246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e24005c00 msgr2=0x7f6e24006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e44c00640 1 --2- 172.21.15.83:0/406075246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e24005c00 0x7f6e24006020 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000c240 tx=0x7f6e3001cc80 comp rx=0 tx=0).stop 2024-04-25T00:14:33.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e3e800640 1 -- 172.21.15.83:0/406075246 reap_dead start 2024-04-25T00:14:33.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e44c00640 1 -- 172.21.15.83:0/406075246 shutdown_connections 2024-04-25T00:14:33.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e44c00640 1 -- 172.21.15.83:0/406075246 >> 172.21.15.83:0/406075246 conn(0x7f6e400cfa60 msgr2=0x7f6e400cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:33.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e44c00640 1 -- 172.21.15.83:0/406075246 shutdown_connections 2024-04-25T00:14:33.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e44c00640 1 -- 172.21.15.83:0/406075246 wait complete. 2024-04-25T00:14:33.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e44c00640 1 Processor -- start 2024-04-25T00:14:33.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e44c00640 1 -- start start 2024-04-25T00:14:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e44c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e24005c00 0x7f6e401407f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e44c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e401f49c0 0x7f6e400c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e44c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e401f4e20 0x7f6e400c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e44c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40144ac0 con 0x7f6e401f49c0 2024-04-25T00:14:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e44c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40071b50 con 0x7f6e401f4e20 2024-04-25T00:14:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e44c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6e4015f350 con 0x7f6e24005c00 2024-04-25T00:14:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e35400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e401f49c0 0x7f6e400c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e35400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e401f49c0 0x7f6e400c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41024/0 (socket says 172.21.15.83:41024) 2024-04-25T00:14:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e35400640 1 -- 172.21.15.83:0/668362991 learned_addr learned my addr 172.21.15.83:0/668362991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e3de00640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e24005c00 0x7f6e401407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e3e800640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e401f4e20 0x7f6e400c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e35400640 1 -- 172.21.15.83:0/668362991 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e24005c00 msgr2=0x7f6e401407f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e35400640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e24005c00 0x7f6e401407f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e35400640 1 -- 172.21.15.83:0/668362991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e401f4e20 msgr2=0x7f6e400c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e35400640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e401f4e20 0x7f6e400c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e35400640 1 -- 172.21.15.83:0/668362991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e400cee80 con 0x7f6e401f49c0 2024-04-25T00:14:33.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.475+0000 7f6e3e800640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e401f4e20 0x7f6e400c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:33.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.479+0000 7f6e35400640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e401f49c0 0x7f6e400c8db0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800f890 tx=0x7f6e2800fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:33.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.479+0000 7f6e36a00640 1 -- 172.21.15.83:0/668362991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e280052e0 con 0x7f6e401f49c0 2024-04-25T00:14:33.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.479+0000 7f6e36a00640 1 -- 172.21.15.83:0/668362991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e28018070 con 0x7f6e401f49c0 2024-04-25T00:14:33.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.479+0000 7f6e44c00640 1 -- 172.21.15.83:0/668362991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e400ceba0 con 0x7f6e401f49c0 2024-04-25T00:14:33.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.479+0000 7f6e36a00640 1 -- 172.21.15.83:0/668362991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2800d8f0 con 0x7f6e401f49c0 2024-04-25T00:14:33.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.479+0000 7f6e44c00640 1 -- 172.21.15.83:0/668362991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e401f5730 con 0x7f6e401f49c0 2024-04-25T00:14:33.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.479+0000 7f6e44c00640 1 -- 172.21.15.83:0/668362991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e40068230 con 0x7f6e401f49c0 2024-04-25T00:14:33.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.479+0000 7f6e36a00640 1 -- 172.21.15.83:0/668362991 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6e2800da90 con 0x7f6e401f49c0 2024-04-25T00:14:33.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.479+0000 7f6e36a00640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6e1c0678f0 0x7f6e1c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.479+0000 7f6e36a00640 1 -- 172.21.15.83:0/668362991 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f6e28085e80 con 0x7f6e401f49c0 2024-04-25T00:14:33.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.479+0000 7f6e3de00640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6e1c0678f0 0x7f6e1c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.483+0000 7f6e3de00640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6e1c0678f0 0x7f6e1c069db0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000c240 tx=0x7f6e300b1040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:33.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.483+0000 7f6e36a00640 1 -- 172.21.15.83:0/668362991 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6e2801a310 con 0x7f6e401f49c0 2024-04-25T00:14:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.651+0000 7f6e44c00640 1 -- 172.21.15.83:0/668362991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6e400ce2c0 con 0x7f6e401f49c0 2024-04-25T00:14:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.655+0000 7f6e36a00640 1 -- 172.21.15.83:0/668362991 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v82) ==== 80+0+0 (secure 0 0 0) 0x7f6e28056790 con 0x7f6e401f49c0 2024-04-25T00:14:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:14:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.655+0000 7f6e44c00640 1 -- 172.21.15.83:0/668362991 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6e1c0678f0 msgr2=0x7f6e1c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.655+0000 7f6e44c00640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6e1c0678f0 0x7f6e1c069db0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000c240 tx=0x7f6e300b1040 comp rx=0 tx=0).stop 2024-04-25T00:14:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.655+0000 7f6e44c00640 1 -- 172.21.15.83:0/668362991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e401f49c0 msgr2=0x7f6e400c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.655+0000 7f6e44c00640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e401f49c0 0x7f6e400c8db0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800f890 tx=0x7f6e2800fd50 comp rx=0 tx=0).stop 2024-04-25T00:14:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.655+0000 7f6e44c00640 1 -- 172.21.15.83:0/668362991 shutdown_connections 2024-04-25T00:14:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.655+0000 7f6e44c00640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6e1c0678f0 0x7f6e1c069db0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.655+0000 7f6e44c00640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e401f4e20 0x7f6e400c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.655+0000 7f6e44c00640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e401f49c0 0x7f6e400c8db0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.655+0000 7f6e44c00640 1 --2- 172.21.15.83:0/668362991 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e24005c00 0x7f6e401407f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.655+0000 7f6e44c00640 1 -- 172.21.15.83:0/668362991 >> 172.21.15.83:0/668362991 conn(0x7f6e400cfa60 msgr2=0x7f6e400b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.655+0000 7f6e44c00640 1 -- 172.21.15.83:0/668362991 shutdown_connections 2024-04-25T00:14:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.655+0000 7f6e44c00640 1 -- 172.21.15.83:0/668362991 wait complete. 2024-04-25T00:14:33.678 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:14:33.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:33.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:14:33.705 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:14:33.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:33.705 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:14:33.761 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:14:33.761 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:33.772 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:14:33.772 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:33.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9744800640 1 Processor -- start 2024-04-25T00:14:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9744800640 1 -- start start 2024-04-25T00:14:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9744800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f973c1553e0 0x7f973c06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9744800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f973c06ea00 0x7f973c0b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9744800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c0b48a0 0x7f973c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9744800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f973c06ef80 con 0x7f973c1553e0 2024-04-25T00:14:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9744800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f973c15b510 con 0x7f973c06ea00 2024-04-25T00:14:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9744800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f973c15b7f0 con 0x7f973c0b48a0 2024-04-25T00:14:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9741e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f973c1553e0 0x7f973c06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9741e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f973c1553e0 0x7f973c06e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41032/0 (socket says 172.21.15.83:41032) 2024-04-25T00:14:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9741e00640 1 -- 172.21.15.83:0/1102335603 learned_addr learned my addr 172.21.15.83:0/1102335603 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9741e00640 1 -- 172.21.15.83:0/1102335603 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c0b48a0 msgr2=0x7f973c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9742800640 1 --2- 172.21.15.83:0/1102335603 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c0b48a0 0x7f973c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9741400640 1 --2- 172.21.15.83:0/1102335603 >> v2:172.21.15.86:3300/0 conn(0x7f973c06ea00 0x7f973c0b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9741e00640 1 --2- 172.21.15.83:0/1102335603 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c0b48a0 0x7f973c15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9741e00640 1 -- 172.21.15.83:0/1102335603 >> v2:172.21.15.86:3300/0 conn(0x7f973c06ea00 msgr2=0x7f973c0b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:33.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9741e00640 1 --2- 172.21.15.83:0/1102335603 >> v2:172.21.15.86:3300/0 conn(0x7f973c06ea00 0x7f973c0b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9741e00640 1 -- 172.21.15.83:0/1102335603 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973c15bad0 con 0x7f973c1553e0 2024-04-25T00:14:33.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9741400640 1 --2- 172.21.15.83:0/1102335603 >> v2:172.21.15.86:3300/0 conn(0x7f973c06ea00 0x7f973c0b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:33.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.955+0000 7f9742800640 1 --2- 172.21.15.83:0/1102335603 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c0b48a0 0x7f973c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9741e00640 1 --2- 172.21.15.83:0/1102335603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f973c1553e0 0x7f973c06e4c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f973c0cf5b0 tx=0x7f973001c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d773beee822ca9ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9740a00640 1 -- 172.21.15.83:0/1102335603 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97300b0070 con 0x7f973c1553e0 2024-04-25T00:14:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9740a00640 1 -- 172.21.15.83:0/1102335603 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97300ab600 con 0x7f973c1553e0 2024-04-25T00:14:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9740a00640 1 -- 172.21.15.83:0/1102335603 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97300ab9d0 con 0x7f973c1553e0 2024-04-25T00:14:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9744800640 1 -- 172.21.15.83:0/1102335603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f973c1553e0 msgr2=0x7f973c06e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9744800640 1 --2- 172.21.15.83:0/1102335603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f973c1553e0 0x7f973c06e4c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f973c0cf5b0 tx=0x7f973001c1d0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9744800640 1 -- 172.21.15.83:0/1102335603 shutdown_connections 2024-04-25T00:14:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9744800640 1 --2- 172.21.15.83:0/1102335603 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c0b48a0 0x7f973c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9744800640 1 --2- 172.21.15.83:0/1102335603 >> v2:172.21.15.86:3300/0 conn(0x7f973c06ea00 0x7f973c0b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9744800640 1 --2- 172.21.15.83:0/1102335603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f973c1553e0 0x7f973c06e4c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9744800640 1 -- 172.21.15.83:0/1102335603 >> 172.21.15.83:0/1102335603 conn(0x7f973c0b1830 msgr2=0x7f973c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9744800640 1 -- 172.21.15.83:0/1102335603 shutdown_connections 2024-04-25T00:14:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9744800640 1 -- 172.21.15.83:0/1102335603 wait complete. 2024-04-25T00:14:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9744800640 1 Processor -- start 2024-04-25T00:14:33.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.959+0000 7f9744800640 1 -- start start 2024-04-25T00:14:33.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9744800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973c06ea00 0x7f973c14a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9744800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c0b48a0 0x7f973c1670c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9744800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f973c1553e0 0x7f973c167600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9744800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f973c06d530 con 0x7f973c1553e0 2024-04-25T00:14:33.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9744800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f973c06d3b0 con 0x7f973c06ea00 2024-04-25T00:14:33.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9744800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f973c06d6b0 con 0x7f973c0b48a0 2024-04-25T00:14:33.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9742800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f973c1553e0 0x7f973c167600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9742800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f973c1553e0 0x7f973c167600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41042/0 (socket says 172.21.15.83:41042) 2024-04-25T00:14:33.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9742800640 1 -- 172.21.15.83:0/2014853570 learned_addr learned my addr 172.21.15.83:0/2014853570 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:33.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9741400640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c0b48a0 0x7f973c1670c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9741e00640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973c06ea00 0x7f973c14a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9742800640 1 -- 172.21.15.83:0/2014853570 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c0b48a0 msgr2=0x7f973c1670c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9742800640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c0b48a0 0x7f973c1670c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9742800640 1 -- 172.21.15.83:0/2014853570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973c06ea00 msgr2=0x7f973c14a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9742800640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973c06ea00 0x7f973c14a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9742800640 1 -- 172.21.15.83:0/2014853570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973c0b1f20 con 0x7f973c1553e0 2024-04-25T00:14:33.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9741e00640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973c06ea00 0x7f973c14a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:33.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9742800640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f973c1553e0 0x7f973c167600 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f973800cf20 tx=0x7f973800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9736000640 1 -- 172.21.15.83:0/2014853570 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973801a070 con 0x7f973c1553e0 2024-04-25T00:14:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9736000640 1 -- 172.21.15.83:0/2014853570 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f973800d840 con 0x7f973c1553e0 2024-04-25T00:14:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9736000640 1 -- 172.21.15.83:0/2014853570 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973800dc30 con 0x7f973c1553e0 2024-04-25T00:14:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9744800640 1 -- 172.21.15.83:0/2014853570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973c14d280 con 0x7f973c1553e0 2024-04-25T00:14:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9744800640 1 -- 172.21.15.83:0/2014853570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f973c14d840 con 0x7f973c1553e0 2024-04-25T00:14:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.963+0000 7f9744800640 1 -- 172.21.15.83:0/2014853570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f973c0737f0 con 0x7f973c1553e0 2024-04-25T00:14:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.967+0000 7f9736000640 1 -- 172.21.15.83:0/2014853570 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f973801d070 con 0x7f973c1553e0 2024-04-25T00:14:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.967+0000 7f9736000640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97200678c0 0x7f9720069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.967+0000 7f9736000640 1 -- 172.21.15.83:0/2014853570 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f9738086610 con 0x7f973c1553e0 2024-04-25T00:14:33.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.967+0000 7f9741e00640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97200678c0 0x7f9720069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.967+0000 7f9741e00640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97200678c0 0x7f9720069d80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f97300ad000 tx=0x7f97300a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:33.971+0000 7f9736000640 1 -- 172.21.15.83:0/2014853570 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9738056960 con 0x7f973c1553e0 2024-04-25T00:14:34.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.127+0000 7f9744800640 1 -- 172.21.15.83:0/2014853570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f973c0ccc00 con 0x7f973c1553e0 2024-04-25T00:14:34.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.127+0000 7f9736000640 1 -- 172.21.15.83:0/2014853570 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16801 (secure 0 0 0) 0x7f973805ac60 con 0x7f973c1553e0 2024-04-25T00:14:34.132 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:34.133 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:24.352251+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:21.082598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:21.713421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:34.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.131+0000 7f9744800640 1 -- 172.21.15.83:0/2014853570 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97200678c0 msgr2=0x7f9720069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.131+0000 7f9744800640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97200678c0 0x7f9720069d80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f97300ad000 tx=0x7f97300a6000 comp rx=0 tx=0).stop 2024-04-25T00:14:34.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.131+0000 7f9744800640 1 -- 172.21.15.83:0/2014853570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f973c1553e0 msgr2=0x7f973c167600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.131+0000 7f9744800640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f973c1553e0 0x7f973c167600 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f973800cf20 tx=0x7f973800a7d0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.131+0000 7f9744800640 1 -- 172.21.15.83:0/2014853570 shutdown_connections 2024-04-25T00:14:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.131+0000 7f9744800640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97200678c0 0x7f9720069d80 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.131+0000 7f9744800640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f973c0b48a0 0x7f973c1670c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.131+0000 7f9744800640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f973c1553e0 0x7f973c167600 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.131+0000 7f9744800640 1 --2- 172.21.15.83:0/2014853570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973c06ea00 0x7f973c14a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.131+0000 7f9744800640 1 -- 172.21.15.83:0/2014853570 >> 172.21.15.83:0/2014853570 conn(0x7f973c0b1830 msgr2=0x7f973c15d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.131+0000 7f9744800640 1 -- 172.21.15.83:0/2014853570 shutdown_connections 2024-04-25T00:14:34.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.131+0000 7f9744800640 1 -- 172.21.15.83:0/2014853570 wait complete. 2024-04-25T00:14:34.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:14:21.082598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:21.713421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:14:34.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:14:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.287+0000 7f276c800640 1 Processor -- start 2024-04-25T00:14:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.287+0000 7f276c800640 1 -- start start 2024-04-25T00:14:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.287+0000 7f276c800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f27641551e0 0x7f276415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.287+0000 7f276c800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f27641555c0 0x7f2764071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.287+0000 7f276c800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f276415c560 con 0x7f276406e4c0 2024-04-25T00:14:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.287+0000 7f276c800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2764072460 con 0x7f27641551e0 2024-04-25T00:14:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.287+0000 7f276c800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f276415c7d0 con 0x7f27641555c0 2024-04-25T00:14:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.287+0000 7f2769e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f27641551e0 0x7f276415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.287+0000 7f276a800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f276406e4c0 0x7f276415e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41178/0 (socket says 172.21.15.83:41178) 2024-04-25T00:14:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.287+0000 7f276a800640 1 -- 172.21.15.83:0/231046222 learned_addr learned my addr 172.21.15.83:0/231046222 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.287+0000 7f2769400640 1 --2- 172.21.15.83:0/231046222 >> v2:172.21.15.86:3300/0 conn(0x7f27641555c0 0x7f2764071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.287+0000 7f2769e00640 1 -- 172.21.15.83:0/231046222 >> v2:172.21.15.86:3300/0 conn(0x7f27641555c0 msgr2=0x7f2764071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.287+0000 7f2769e00640 1 --2- 172.21.15.83:0/231046222 >> v2:172.21.15.86:3300/0 conn(0x7f27641555c0 0x7f2764071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.287+0000 7f2769e00640 1 -- 172.21.15.83:0/231046222 >> v1:172.21.15.83:6789/0 conn(0x7f276406e4c0 legacy=0x7f276415e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.291+0000 7f2769e00640 1 -- 172.21.15.83:0/231046222 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276415cab0 con 0x7f27641551e0 2024-04-25T00:14:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.291+0000 7f2768a00640 1 -- 172.21.15.83:0/231046222 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3804246564 0 0) 0x7f276415c560 con 0x7f276406e4c0 2024-04-25T00:14:34.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.291+0000 7f2769e00640 1 --2- 172.21.15.83:0/231046222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27641551e0 0x7f276415bea0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f27580097c0 tx=0x7f275801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb24d19d6fec03c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:34.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.291+0000 7f2768a00640 1 -- 172.21.15.83:0/231046222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27580b2070 con 0x7f27641551e0 2024-04-25T00:14:34.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.291+0000 7f2768a00640 1 -- 172.21.15.83:0/231046222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27580ad680 con 0x7f27641551e0 2024-04-25T00:14:34.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.291+0000 7f2768a00640 1 -- 172.21.15.83:0/231046222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27580ada50 con 0x7f27641551e0 2024-04-25T00:14:34.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.291+0000 7f276c800640 1 -- 172.21.15.83:0/231046222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27641551e0 msgr2=0x7f276415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.291+0000 7f276c800640 1 --2- 172.21.15.83:0/231046222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27641551e0 0x7f276415bea0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f27580097c0 tx=0x7f275801c460 comp rx=0 tx=0).stop 2024-04-25T00:14:34.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.291+0000 7f276c800640 1 -- 172.21.15.83:0/231046222 shutdown_connections 2024-04-25T00:14:34.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.291+0000 7f276c800640 1 --2- 172.21.15.83:0/231046222 >> v2:172.21.15.86:3300/0 conn(0x7f27641555c0 0x7f2764071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.291+0000 7f276c800640 1 --2- 172.21.15.83:0/231046222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27641551e0 0x7f276415bea0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.291+0000 7f276c800640 1 -- 172.21.15.83:0/231046222 >> 172.21.15.83:0/231046222 conn(0x7f27640b1830 msgr2=0x7f27640b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:34.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.291+0000 7f276c800640 1 -- 172.21.15.83:0/231046222 shutdown_connections 2024-04-25T00:14:34.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.291+0000 7f276c800640 1 -- 172.21.15.83:0/231046222 wait complete. 2024-04-25T00:14:34.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f276c800640 1 Processor -- start 2024-04-25T00:14:34.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f276c800640 1 -- start start 2024-04-25T00:14:34.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f276c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f276406e4c0 0x7f27640aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:34.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f276c800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27641551e0 0x7f2764159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:34.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f276c800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27641555c0 0x7f276415a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:34.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f276c800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2764167e10 con 0x7f27641551e0 2024-04-25T00:14:34.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f276c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2764167c90 con 0x7f276406e4c0 2024-04-25T00:14:34.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f276c800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2764167f90 con 0x7f27641555c0 2024-04-25T00:14:34.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f2769400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27641551e0 0x7f2764159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:34.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f2769400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27641551e0 0x7f2764159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44064/0 (socket says 172.21.15.83:44064) 2024-04-25T00:14:34.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f2769400640 1 -- 172.21.15.83:0/2562776904 learned_addr learned my addr 172.21.15.83:0/2562776904 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:34.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f276a800640 1 --2- 172.21.15.83:0/2562776904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27641555c0 0x7f276415a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:34.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f2769e00640 1 --2- 172.21.15.83:0/2562776904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f276406e4c0 0x7f27640aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:34.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f2769400640 1 -- 172.21.15.83:0/2562776904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27641555c0 msgr2=0x7f276415a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f2769400640 1 --2- 172.21.15.83:0/2562776904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27641555c0 0x7f276415a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f2769400640 1 -- 172.21.15.83:0/2562776904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f276406e4c0 msgr2=0x7f27640aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f2769400640 1 --2- 172.21.15.83:0/2562776904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f276406e4c0 0x7f27640aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f2769400640 1 -- 172.21.15.83:0/2562776904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276414fdc0 con 0x7f27641551e0 2024-04-25T00:14:34.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f2769400640 1 --2- 172.21.15.83:0/2562776904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27641551e0 0x7f2764159dd0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f2754006e90 tx=0x7f275400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:34.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f275e000640 1 -- 172.21.15.83:0/2562776904 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275400b6c0 con 0x7f27641551e0 2024-04-25T00:14:34.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f275e000640 1 -- 172.21.15.83:0/2562776904 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2754019070 con 0x7f27641551e0 2024-04-25T00:14:34.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f276c800640 1 -- 172.21.15.83:0/2562776904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27641500a0 con 0x7f27641551e0 2024-04-25T00:14:34.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f275e000640 1 -- 172.21.15.83:0/2562776904 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275400bac0 con 0x7f27641551e0 2024-04-25T00:14:34.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.295+0000 7f276c800640 1 -- 172.21.15.83:0/2562776904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2764150660 con 0x7f27641551e0 2024-04-25T00:14:34.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.299+0000 7f276c800640 1 -- 172.21.15.83:0/2562776904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f276404f510 con 0x7f27641551e0 2024-04-25T00:14:34.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.299+0000 7f275e000640 1 -- 172.21.15.83:0/2562776904 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f275400bc60 con 0x7f27641551e0 2024-04-25T00:14:34.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.299+0000 7f275e000640 1 --2- 172.21.15.83:0/2562776904 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27480678f0 0x7f2748069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:34.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.299+0000 7f275e000640 1 -- 172.21.15.83:0/2562776904 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f2754086280 con 0x7f27641551e0 2024-04-25T00:14:34.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.299+0000 7f2769e00640 1 --2- 172.21.15.83:0/2562776904 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27480678f0 0x7f2748069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:34.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.299+0000 7f2769e00640 1 --2- 172.21.15.83:0/2562776904 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27480678f0 0x7f2748069db0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f27580af000 tx=0x7f27580a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:34.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.303+0000 7f275e000640 1 -- 172.21.15.83:0/2562776904 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2754056c20 con 0x7f27641551e0 2024-04-25T00:14:34.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:14:34.399 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:14:34.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.463+0000 7f276c800640 1 -- 172.21.15.83:0/2562776904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2764067af0 con 0x7f27641551e0 2024-04-25T00:14:34.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.463+0000 7f275e000640 1 -- 172.21.15.83:0/2562776904 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f2754056520 con 0x7f27641551e0 2024-04-25T00:14:34.467 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:14:34.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.463+0000 7f276c800640 1 -- 172.21.15.83:0/2562776904 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27480678f0 msgr2=0x7f2748069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.463+0000 7f276c800640 1 --2- 172.21.15.83:0/2562776904 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27480678f0 0x7f2748069db0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f27580af000 tx=0x7f27580a7000 comp rx=0 tx=0).stop 2024-04-25T00:14:34.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.463+0000 7f276c800640 1 -- 172.21.15.83:0/2562776904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27641551e0 msgr2=0x7f2764159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.463+0000 7f276c800640 1 --2- 172.21.15.83:0/2562776904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27641551e0 0x7f2764159dd0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f2754006e90 tx=0x7f275400dc60 comp rx=0 tx=0).stop 2024-04-25T00:14:34.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.463+0000 7f276c800640 1 -- 172.21.15.83:0/2562776904 shutdown_connections 2024-04-25T00:14:34.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.463+0000 7f276c800640 1 --2- 172.21.15.83:0/2562776904 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27480678f0 0x7f2748069db0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.463+0000 7f276c800640 1 --2- 172.21.15.83:0/2562776904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27641555c0 0x7f276415a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.463+0000 7f276c800640 1 --2- 172.21.15.83:0/2562776904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27641551e0 0x7f2764159dd0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.463+0000 7f276c800640 1 --2- 172.21.15.83:0/2562776904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f276406e4c0 0x7f27640aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.463+0000 7f276c800640 1 -- 172.21.15.83:0/2562776904 >> 172.21.15.83:0/2562776904 conn(0x7f27640b1830 msgr2=0x7f2764154870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:34.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.463+0000 7f276c800640 1 -- 172.21.15.83:0/2562776904 shutdown_connections 2024-04-25T00:14:34.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.463+0000 7f276c800640 1 -- 172.21.15.83:0/2562776904 wait complete. 2024-04-25T00:14:34.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:14:34.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcc03c00640 1 Processor -- start 2024-04-25T00:14:34.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcc03c00640 1 -- start start 2024-04-25T00:14:34.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcc03c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcbfc1551e0 0x7fcbfc15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:34.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcc03c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcbfc06e4c0 0x7fcbfc06e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:34.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcc03c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbfc15c560 con 0x7fcbfc1555c0 2024-04-25T00:14:34.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcc03c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fcbfc072460 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcc03c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcbfc15c7d0 con 0x7fcbfc1551e0 2024-04-25T00:14:34.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcbf3e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcbfc1555c0 0x7fcbfc071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45458/0 (socket says 172.21.15.83:45458) 2024-04-25T00:14:34.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcbf3e00640 1 -- 172.21.15.83:0/2219957587 learned_addr learned my addr 172.21.15.83:0/2219957587 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:34.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcc01c00640 1 --2- 172.21.15.83:0/2219957587 >> v2:172.21.15.83:3300/0 conn(0x7fcbfc06e4c0 0x7fcbfc06e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:34.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcc01200640 1 --2- 172.21.15.83:0/2219957587 >> v2:172.21.15.86:3300/0 conn(0x7fcbfc1551e0 0x7fcbfc15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:34.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcc01c00640 1 -- 172.21.15.83:0/2219957587 >> v2:172.21.15.86:3300/0 conn(0x7fcbfc1551e0 msgr2=0x7fcbfc15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:34.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcc01c00640 1 --2- 172.21.15.83:0/2219957587 >> v2:172.21.15.86:3300/0 conn(0x7fcbfc1551e0 0x7fcbfc15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcc01c00640 1 -- 172.21.15.83:0/2219957587 >> v1:172.21.15.83:6789/0 conn(0x7fcbfc1555c0 legacy=0x7fcbfc071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcc01c00640 1 -- 172.21.15.83:0/2219957587 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc15cab0 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.671+0000 7fcbfbe00640 1 -- 172.21.15.83:0/2219957587 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3693343647 0 0) 0x7fcbfc15c560 con 0x7fcbfc1555c0 2024-04-25T00:14:34.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.675+0000 7fcc01c00640 1 --2- 172.21.15.83:0/2219957587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbfc06e4c0 0x7fcbfc06e950 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009d90 tx=0x7fcbf401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a94cac272c5d94b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:34.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.675+0000 7fcbfbe00640 1 -- 172.21.15.83:0/2219957587 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf40b2070 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.675+0000 7fcbfbe00640 1 -- 172.21.15.83:0/2219957587 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf40a57f0 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.675+0000 7fcbfbe00640 1 -- 172.21.15.83:0/2219957587 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf40a5bc0 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.675+0000 7fcc03c00640 1 -- 172.21.15.83:0/2219957587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbfc06e4c0 msgr2=0x7fcbfc06e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.675+0000 7fcc03c00640 1 --2- 172.21.15.83:0/2219957587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbfc06e4c0 0x7fcbfc06e950 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009d90 tx=0x7fcbf401c930 comp rx=0 tx=0).stop 2024-04-25T00:14:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.675+0000 7fcc03c00640 1 -- 172.21.15.83:0/2219957587 shutdown_connections 2024-04-25T00:14:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.675+0000 7fcc03c00640 1 --2- 172.21.15.83:0/2219957587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbfc06e4c0 0x7fcbfc06e950 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.675+0000 7fcc03c00640 1 --2- 172.21.15.83:0/2219957587 >> v2:172.21.15.86:3300/0 conn(0x7fcbfc1551e0 0x7fcbfc15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.675+0000 7fcc03c00640 1 -- 172.21.15.83:0/2219957587 >> 172.21.15.83:0/2219957587 conn(0x7fcbfc0b1830 msgr2=0x7fcbfc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.675+0000 7fcc03c00640 1 -- 172.21.15.83:0/2219957587 shutdown_connections 2024-04-25T00:14:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.675+0000 7fcc03c00640 1 -- 172.21.15.83:0/2219957587 wait complete. 2024-04-25T00:14:34.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc03c00640 1 Processor -- start 2024-04-25T00:14:34.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc03c00640 1 -- start start 2024-04-25T00:14:34.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc03c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbfc06e4c0 0x7fcbfc1521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:34.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc03c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcbfc1551e0 0x7fcbfc14ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:34.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc03c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbfc1555c0 0x7fcbfc153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:34.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc03c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc15db70 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc03c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc15d9f0 con 0x7fcbfc1555c0 2024-04-25T00:14:34.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc03c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcbfc15dcf0 con 0x7fcbfc1551e0 2024-04-25T00:14:34.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc01200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbfc06e4c0 0x7fcbfc1521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:34.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc01200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbfc06e4c0 0x7fcbfc1521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44092/0 (socket says 172.21.15.83:44092) 2024-04-25T00:14:34.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc01200640 1 -- 172.21.15.83:0/3259080332 learned_addr learned my addr 172.21.15.83:0/3259080332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:34.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcbf3e00640 1 --2- 172.21.15.83:0/3259080332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcbfc1551e0 0x7fcbfc14ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:34.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc01c00640 1 --2- 172.21.15.83:0/3259080332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbfc1555c0 0x7fcbfc153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:34.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc01200640 1 -- 172.21.15.83:0/3259080332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcbfc1551e0 msgr2=0x7fcbfc14ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc01200640 1 --2- 172.21.15.83:0/3259080332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcbfc1551e0 0x7fcbfc14ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc01200640 1 -- 172.21.15.83:0/3259080332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbfc1555c0 msgr2=0x7fcbfc153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc01200640 1 --2- 172.21.15.83:0/3259080332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbfc1555c0 0x7fcbfc153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc01200640 1 -- 172.21.15.83:0/3259080332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc0cf140 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc01200640 1 --2- 172.21.15.83:0/3259080332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbfc06e4c0 0x7fcbfc1521d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00e9b0 tx=0x7fcbec00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcbf9600640 1 -- 172.21.15.83:0/3259080332 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec017070 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcbf9600640 1 -- 172.21.15.83:0/3259080332 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbec00c9a0 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc03c00640 1 -- 172.21.15.83:0/3259080332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbfc152ad0 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcbf9600640 1 -- 172.21.15.83:0/3259080332 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec00cd70 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.679+0000 7fcc03c00640 1 -- 172.21.15.83:0/3259080332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbfc153090 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.683+0000 7fcc03c00640 1 -- 172.21.15.83:0/3259080332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbfc04f510 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.683+0000 7fcbf9600640 1 -- 172.21.15.83:0/3259080332 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcbec01a070 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.683+0000 7fcbf9600640 1 --2- 172.21.15.83:0/3259080332 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcbd80678c0 0x7fcbd8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.683+0000 7fcbf9600640 1 -- 172.21.15.83:0/3259080332 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fcbec085f80 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.683+0000 7fcbf3e00640 1 --2- 172.21.15.83:0/3259080332 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcbd80678c0 0x7fcbd8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:34.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.683+0000 7fcbf3e00640 1 --2- 172.21.15.83:0/3259080332 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcbd80678c0 0x7fcbd8069d80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc1548b0 tx=0x7fcbe4007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:34.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.687+0000 7fcbf9600640 1 -- 172.21.15.83:0/3259080332 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcbec056890 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.843+0000 7fcc03c00640 1 -- 172.21.15.83:0/3259080332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcbfc06ce30 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.843+0000 7fcbf9600640 1 -- 172.21.15.83:0/3259080332 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7fcbec056190 con 0x7fcbfc06e4c0 2024-04-25T00:14:34.847 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:14:34.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.843+0000 7fcc03c00640 1 -- 172.21.15.83:0/3259080332 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcbd80678c0 msgr2=0x7fcbd8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.843+0000 7fcc03c00640 1 --2- 172.21.15.83:0/3259080332 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcbd80678c0 0x7fcbd8069d80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc1548b0 tx=0x7fcbe4007c10 comp rx=0 tx=0).stop 2024-04-25T00:14:34.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.843+0000 7fcc03c00640 1 -- 172.21.15.83:0/3259080332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbfc06e4c0 msgr2=0x7fcbfc1521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.843+0000 7fcc03c00640 1 --2- 172.21.15.83:0/3259080332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbfc06e4c0 0x7fcbfc1521d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00e9b0 tx=0x7fcbec00ee70 comp rx=0 tx=0).stop 2024-04-25T00:14:34.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.843+0000 7fcc03c00640 1 -- 172.21.15.83:0/3259080332 shutdown_connections 2024-04-25T00:14:34.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.843+0000 7fcc03c00640 1 --2- 172.21.15.83:0/3259080332 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcbd80678c0 0x7fcbd8069d80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.843+0000 7fcc03c00640 1 --2- 172.21.15.83:0/3259080332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcbfc1551e0 0x7fcbfc14ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.843+0000 7fcc03c00640 1 --2- 172.21.15.83:0/3259080332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbfc1555c0 0x7fcbfc153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.843+0000 7fcc03c00640 1 --2- 172.21.15.83:0/3259080332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbfc06e4c0 0x7fcbfc1521d0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.843+0000 7fcc03c00640 1 -- 172.21.15.83:0/3259080332 >> 172.21.15.83:0/3259080332 conn(0x7fcbfc0b1830 msgr2=0x7fcbfc15edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:34.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.847+0000 7fcc03c00640 1 -- 172.21.15.83:0/3259080332 shutdown_connections 2024-04-25T00:14:34.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.847+0000 7fcc03c00640 1 -- 172.21.15.83:0/3259080332 wait complete. 2024-04-25T00:14:34.868 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:14:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.999+0000 7f01b3e00640 1 Processor -- start 2024-04-25T00:14:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:34.999+0000 7f01b3e00640 1 -- start start 2024-04-25T00:14:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01b3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f01ac06e950 0x7f01ac15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01b3e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01ac15fe50 0x7f01ac0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01b3e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ac0ce000 con 0x7f01ac159810 2024-04-25T00:14:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01b3e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f01ac06e210 con 0x7f01ac06e950 2024-04-25T00:14:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01b3e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f01ac06da00 con 0x7f01ac15fe50 2024-04-25T00:14:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01b1400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f01ac159810 0x7f01ac0cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45466/0 (socket says 172.21.15.83:45466) 2024-04-25T00:14:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01b1400640 1 -- 172.21.15.83:0/4087558396 learned_addr learned my addr 172.21.15.83:0/4087558396 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01b1e00640 1 --2- 172.21.15.83:0/4087558396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01ac15fe50 0x7f01ac0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01b0a00640 1 --2- 172.21.15.83:0/4087558396 >> v2:172.21.15.86:3300/0 conn(0x7f01ac06e950 0x7f01ac15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01abe00640 1 -- 172.21.15.83:0/4087558396 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3374734480 0 0) 0x7f01ac0ce000 con 0x7f01ac159810 2024-04-25T00:14:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01abe00640 1 -- 172.21.15.83:0/4087558396 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0194003530 con 0x7f01ac159810 2024-04-25T00:14:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01abe00640 1 -- 172.21.15.83:0/4087558396 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3166921349 0 0) 0x7f0194003530 con 0x7f01ac159810 2024-04-25T00:14:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01b0a00640 1 -- 172.21.15.83:0/4087558396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01ac15fe50 msgr2=0x7f01ac0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01b0a00640 1 --2- 172.21.15.83:0/4087558396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01ac15fe50 0x7f01ac0ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01b0a00640 1 -- 172.21.15.83:0/4087558396 >> v1:172.21.15.83:6789/0 conn(0x7f01ac159810 legacy=0x7f01ac0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01b0a00640 1 -- 172.21.15.83:0/4087558396 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac06eec0 con 0x7f01ac06e950 2024-04-25T00:14:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.003+0000 7f01b1e00640 1 --2- 172.21.15.83:0/4087558396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01ac15fe50 0x7f01ac0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.007+0000 7f01b0a00640 1 --2- 172.21.15.83:0/4087558396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ac06e950 0x7f01ac15f910 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f01ac06db80 tx=0x7f019801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2986fb3e13cce8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.007+0000 7f01abe00640 1 -- 172.21.15.83:0/4087558396 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01980b0070 con 0x7f01ac06e950 2024-04-25T00:14:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.007+0000 7f01abe00640 1 -- 172.21.15.83:0/4087558396 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01980ab400 con 0x7f01ac06e950 2024-04-25T00:14:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.007+0000 7f01abe00640 1 -- 172.21.15.83:0/4087558396 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0001680 con 0x7f01ac06e950 2024-04-25T00:14:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.007+0000 7f01b3e00640 1 -- 172.21.15.83:0/4087558396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ac06e950 msgr2=0x7f01ac15f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.007+0000 7f01b3e00640 1 --2- 172.21.15.83:0/4087558396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ac06e950 0x7f01ac15f910 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f01ac06db80 tx=0x7f019801c330 comp rx=0 tx=0).stop 2024-04-25T00:14:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.007+0000 7f01b3e00640 1 -- 172.21.15.83:0/4087558396 shutdown_connections 2024-04-25T00:14:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.007+0000 7f01b3e00640 1 --2- 172.21.15.83:0/4087558396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01ac15fe50 0x7f01ac0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.007+0000 7f01b3e00640 1 --2- 172.21.15.83:0/4087558396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ac06e950 0x7f01ac15f910 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.007+0000 7f01b3e00640 1 -- 172.21.15.83:0/4087558396 >> 172.21.15.83:0/4087558396 conn(0x7f01ac0b1830 msgr2=0x7f01ac0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.007+0000 7f01b3e00640 1 -- 172.21.15.83:0/4087558396 shutdown_connections 2024-04-25T00:14:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.007+0000 7f01b3e00640 1 -- 172.21.15.83:0/4087558396 wait complete. 2024-04-25T00:14:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.007+0000 7f01b3e00640 1 Processor -- start 2024-04-25T00:14:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.007+0000 7f01b3e00640 1 -- start start 2024-04-25T00:14:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ac06e950 0x7f01ac0aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ac159810 0x7f01ac15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b3e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01ac15fe50 0x7f01ac15df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac070f20 con 0x7f01ac06e950 2024-04-25T00:14:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac070da0 con 0x7f01ac159810 2024-04-25T00:14:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b3e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f01ac168cb0 con 0x7f01ac15fe50 2024-04-25T00:14:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ac06e950 0x7f01ac0aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ac06e950 0x7f01ac0aef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44106/0 (socket says 172.21.15.83:44106) 2024-04-25T00:14:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b1400640 1 -- 172.21.15.83:0/1671111497 learned_addr learned my addr 172.21.15.83:0/1671111497 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b1e00640 1 --2- 172.21.15.83:0/1671111497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01ac15fe50 0x7f01ac15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b0a00640 1 --2- 172.21.15.83:0/1671111497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ac159810 0x7f01ac15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b1400640 1 -- 172.21.15.83:0/1671111497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01ac15fe50 msgr2=0x7f01ac15df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b1400640 1 --2- 172.21.15.83:0/1671111497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01ac15fe50 0x7f01ac15df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b1400640 1 -- 172.21.15.83:0/1671111497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ac159810 msgr2=0x7f01ac15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b1400640 1 --2- 172.21.15.83:0/1671111497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ac159810 0x7f01ac15b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b1400640 1 -- 172.21.15.83:0/1671111497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac15a8b0 con 0x7f01ac06e950 2024-04-25T00:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b1400640 1 --2- 172.21.15.83:0/1671111497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ac06e950 0x7f01ac0aef00 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f01a000c750 tx=0x7f01a000cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01a9600640 1 -- 172.21.15.83:0/1671111497 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0005ae0 con 0x7f01ac06e950 2024-04-25T00:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01a9600640 1 -- 172.21.15.83:0/1671111497 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a0005ca0 con 0x7f01ac06e950 2024-04-25T00:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01a9600640 1 -- 172.21.15.83:0/1671111497 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a00072b0 con 0x7f01ac06e950 2024-04-25T00:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b3e00640 1 -- 172.21.15.83:0/1671111497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac15ab90 con 0x7f01ac06e950 2024-04-25T00:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b3e00640 1 -- 172.21.15.83:0/1671111497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac15b150 con 0x7f01ac06e950 2024-04-25T00:14:35.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01b3e00640 1 -- 172.21.15.83:0/1671111497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ac04f510 con 0x7f01ac06e950 2024-04-25T00:14:35.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.011+0000 7f01a9600640 1 -- 172.21.15.83:0/1671111497 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f01a0007450 con 0x7f01ac06e950 2024-04-25T00:14:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.015+0000 7f01a9600640 1 --2- 172.21.15.83:0/1671111497 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f018c0678c0 0x7f018c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.015+0000 7f01a9600640 1 -- 172.21.15.83:0/1671111497 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f01a0085d10 con 0x7f01ac06e950 2024-04-25T00:14:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.015+0000 7f01b0a00640 1 --2- 172.21.15.83:0/1671111497 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f018c0678c0 0x7f018c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.015+0000 7f01b0a00640 1 --2- 172.21.15.83:0/1671111497 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f018c0678c0 0x7f018c069d80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f01980ad000 tx=0x7f01980a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:35.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.019+0000 7f01a9600640 1 -- 172.21.15.83:0/1671111497 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f01a008b020 con 0x7f01ac06e950 2024-04-25T00:14:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.171+0000 7f01b3e00640 1 -- 172.21.15.83:0/1671111497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f01ac0008e0 con 0x7f01ac06e950 2024-04-25T00:14:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.171+0000 7f01a9600640 1 -- 172.21.15.83:0/1671111497 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7f01a0056620 con 0x7f01ac06e950 2024-04-25T00:14:35.176 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:14:35.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.175+0000 7f01b3e00640 1 -- 172.21.15.83:0/1671111497 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f018c0678c0 msgr2=0x7f018c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.175+0000 7f01b3e00640 1 --2- 172.21.15.83:0/1671111497 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f018c0678c0 0x7f018c069d80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f01980ad000 tx=0x7f01980a6000 comp rx=0 tx=0).stop 2024-04-25T00:14:35.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.175+0000 7f01b3e00640 1 -- 172.21.15.83:0/1671111497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ac06e950 msgr2=0x7f01ac0aef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.175+0000 7f01b3e00640 1 --2- 172.21.15.83:0/1671111497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ac06e950 0x7f01ac0aef00 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f01a000c750 tx=0x7f01a000cb50 comp rx=0 tx=0).stop 2024-04-25T00:14:35.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.175+0000 7f01b3e00640 1 -- 172.21.15.83:0/1671111497 shutdown_connections 2024-04-25T00:14:35.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.175+0000 7f01b3e00640 1 --2- 172.21.15.83:0/1671111497 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f018c0678c0 0x7f018c069d80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.175+0000 7f01b3e00640 1 --2- 172.21.15.83:0/1671111497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01ac15fe50 0x7f01ac15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.175+0000 7f01b3e00640 1 --2- 172.21.15.83:0/1671111497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ac159810 0x7f01ac15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.175+0000 7f01b3e00640 1 --2- 172.21.15.83:0/1671111497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ac06e950 0x7f01ac0aef00 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.175+0000 7f01b3e00640 1 -- 172.21.15.83:0/1671111497 >> 172.21.15.83:0/1671111497 conn(0x7f01ac0b1830 msgr2=0x7f01ac071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:35.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.175+0000 7f01b3e00640 1 -- 172.21.15.83:0/1671111497 shutdown_connections 2024-04-25T00:14:35.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.175+0000 7f01b3e00640 1 -- 172.21.15.83:0/1671111497 wait complete. 2024-04-25T00:14:35.197 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:14:35.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.327+0000 7f3560800640 1 Processor -- start 2024-04-25T00:14:35.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.327+0000 7f3560800640 1 -- start start 2024-04-25T00:14:35.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.331+0000 7f3560800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f35580cab50 0x7f355815f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.331+0000 7f3560800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35580cafa0 con 0x7f355806ee20 2024-04-25T00:14:35.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.331+0000 7f3560800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35580cb210 con 0x7f35581446e0 2024-04-25T00:14:35.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.331+0000 7f3560800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f35580715b0 con 0x7f35580cab50 2024-04-25T00:14:35.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.331+0000 7f355d400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f355806ee20 0x7f3558071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45468/0 (socket says 172.21.15.83:45468) 2024-04-25T00:14:35.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.331+0000 7f355d400640 1 -- 172.21.15.83:0/1337764904 learned_addr learned my addr 172.21.15.83:0/1337764904 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:35.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.331+0000 7f355e800640 1 --2- 172.21.15.83:0/1337764904 >> v2:172.21.15.86:3300/0 conn(0x7f35580cab50 0x7f355815f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.331+0000 7f355ca00640 1 -- 172.21.15.83:0/1337764904 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1027023564 0 0) 0x7f35580cafa0 con 0x7f355806ee20 2024-04-25T00:14:35.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.331+0000 7f355ca00640 1 -- 172.21.15.83:0/1337764904 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3540003530 con 0x7f355806ee20 2024-04-25T00:14:35.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.331+0000 7f355e800640 1 -- 172.21.15.83:0/1337764904 >> v1:172.21.15.86:6789/0 conn(0x7f35581446e0 legacy=0x7f355806e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.331+0000 7f355e800640 1 -- 172.21.15.83:0/1337764904 >> v1:172.21.15.83:6789/0 conn(0x7f355806ee20 legacy=0x7f3558071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.331+0000 7f355e800640 1 -- 172.21.15.83:0/1337764904 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35580cb4f0 con 0x7f35580cab50 2024-04-25T00:14:35.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.335+0000 7f355e800640 1 --2- 172.21.15.83:0/1337764904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35580cab50 0x7f355815f350 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3554009d90 tx=0x7f355401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec240e9935e84795 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.335+0000 7f355ca00640 1 -- 172.21.15.83:0/1337764904 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35540b2070 con 0x7f35580cab50 2024-04-25T00:14:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.335+0000 7f355ca00640 1 -- 172.21.15.83:0/1337764904 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35540a5830 con 0x7f35580cab50 2024-04-25T00:14:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.335+0000 7f355ca00640 1 -- 172.21.15.83:0/1337764904 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35540a5be0 con 0x7f35580cab50 2024-04-25T00:14:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.335+0000 7f3560800640 1 -- 172.21.15.83:0/1337764904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35580cab50 msgr2=0x7f355815f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.335+0000 7f3560800640 1 --2- 172.21.15.83:0/1337764904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35580cab50 0x7f355815f350 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3554009d90 tx=0x7f355401c970 comp rx=0 tx=0).stop 2024-04-25T00:14:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.335+0000 7f3560800640 1 -- 172.21.15.83:0/1337764904 shutdown_connections 2024-04-25T00:14:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.335+0000 7f3560800640 1 --2- 172.21.15.83:0/1337764904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35580cab50 0x7f355815f350 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.335+0000 7f3560800640 1 -- 172.21.15.83:0/1337764904 >> 172.21.15.83:0/1337764904 conn(0x7f35580cfa60 msgr2=0x7f35580cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.335+0000 7f3560800640 1 -- 172.21.15.83:0/1337764904 shutdown_connections 2024-04-25T00:14:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.335+0000 7f3560800640 1 -- 172.21.15.83:0/1337764904 wait complete. 2024-04-25T00:14:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.335+0000 7f3560800640 1 Processor -- start 2024-04-25T00:14:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.335+0000 7f3560800640 1 -- start start 2024-04-25T00:14:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f3560800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f355806ee20 0x7f355815c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f3560800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35580cab50 0x7f3558140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f3560800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35581446e0 0x7f3558141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f3560800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3558168cb0 con 0x7f355806ee20 2024-04-25T00:14:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f3560800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3558071b50 con 0x7f35580cab50 2024-04-25T00:14:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f3560800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3558168e30 con 0x7f35581446e0 2024-04-25T00:14:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f355de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f355806ee20 0x7f355815c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f355de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f355806ee20 0x7f355815c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44118/0 (socket says 172.21.15.83:44118) 2024-04-25T00:14:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f355de00640 1 -- 172.21.15.83:0/4287384863 learned_addr learned my addr 172.21.15.83:0/4287384863 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f355d400640 1 --2- 172.21.15.83:0/4287384863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35580cab50 0x7f3558140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f355e800640 1 --2- 172.21.15.83:0/4287384863 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35581446e0 0x7f3558141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f355de00640 1 -- 172.21.15.83:0/4287384863 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35581446e0 msgr2=0x7f3558141e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f355de00640 1 --2- 172.21.15.83:0/4287384863 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35581446e0 0x7f3558141e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f355de00640 1 -- 172.21.15.83:0/4287384863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35580cab50 msgr2=0x7f3558140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f355de00640 1 --2- 172.21.15.83:0/4287384863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35580cab50 0x7f3558140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f355de00640 1 -- 172.21.15.83:0/4287384863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35580b2950 con 0x7f355806ee20 2024-04-25T00:14:35.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f355de00640 1 --2- 172.21.15.83:0/4287384863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f355806ee20 0x7f355815c0b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f354c00c530 tx=0x7f354c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:35.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f3552000640 1 -- 172.21.15.83:0/4287384863 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c016020 con 0x7f355806ee20 2024-04-25T00:14:35.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f3552000640 1 -- 172.21.15.83:0/4287384863 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f354c005220 con 0x7f355806ee20 2024-04-25T00:14:35.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f3552000640 1 -- 172.21.15.83:0/4287384863 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c007030 con 0x7f355806ee20 2024-04-25T00:14:35.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f3560800640 1 -- 172.21.15.83:0/4287384863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35580b2c30 con 0x7f355806ee20 2024-04-25T00:14:35.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f3560800640 1 -- 172.21.15.83:0/4287384863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35580b31f0 con 0x7f355806ee20 2024-04-25T00:14:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f3560800640 1 -- 172.21.15.83:0/4287384863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3558068230 con 0x7f355806ee20 2024-04-25T00:14:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.339+0000 7f3552000640 1 -- 172.21.15.83:0/4287384863 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f354c0071d0 con 0x7f355806ee20 2024-04-25T00:14:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.343+0000 7f3552000640 1 --2- 172.21.15.83:0/4287384863 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f353c0678c0 0x7f353c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.343+0000 7f3552000640 1 -- 172.21.15.83:0/4287384863 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f354c085840 con 0x7f355806ee20 2024-04-25T00:14:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.343+0000 7f355d400640 1 --2- 172.21.15.83:0/4287384863 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f353c0678c0 0x7f353c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.343+0000 7f355d400640 1 --2- 172.21.15.83:0/4287384863 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f353c0678c0 0x7f353c069d80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f355815cc70 tx=0x7f3548009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:35.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.347+0000 7f3552000640 1 -- 172.21.15.83:0/4287384863 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f354c020020 con 0x7f355806ee20 2024-04-25T00:14:35.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.515+0000 7f3560800640 1 -- 172.21.15.83:0/4287384863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f35580ceae0 con 0x7f355806ee20 2024-04-25T00:14:35.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.519+0000 7f3552000640 1 -- 172.21.15.83:0/4287384863 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v82) ==== 80+0+0 (secure 0 0 0) 0x7f354c0074c0 con 0x7f355806ee20 2024-04-25T00:14:35.521 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:14:35.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.519+0000 7f3560800640 1 -- 172.21.15.83:0/4287384863 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f353c0678c0 msgr2=0x7f353c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.519+0000 7f3560800640 1 --2- 172.21.15.83:0/4287384863 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f353c0678c0 0x7f353c069d80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f355815cc70 tx=0x7f3548009040 comp rx=0 tx=0).stop 2024-04-25T00:14:35.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.519+0000 7f3560800640 1 -- 172.21.15.83:0/4287384863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f355806ee20 msgr2=0x7f355815c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.519+0000 7f3560800640 1 --2- 172.21.15.83:0/4287384863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f355806ee20 0x7f355815c0b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f354c00c530 tx=0x7f354c00c9f0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.519+0000 7f3560800640 1 -- 172.21.15.83:0/4287384863 shutdown_connections 2024-04-25T00:14:35.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.519+0000 7f3560800640 1 --2- 172.21.15.83:0/4287384863 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f353c0678c0 0x7f353c069d80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.519+0000 7f3560800640 1 --2- 172.21.15.83:0/4287384863 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35581446e0 0x7f3558141e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.519+0000 7f3560800640 1 --2- 172.21.15.83:0/4287384863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35580cab50 0x7f3558140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.519+0000 7f3560800640 1 --2- 172.21.15.83:0/4287384863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f355806ee20 0x7f355815c0b0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.519+0000 7f3560800640 1 -- 172.21.15.83:0/4287384863 >> 172.21.15.83:0/4287384863 conn(0x7f35580cfa60 msgr2=0x7f35580c8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:35.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.519+0000 7f3560800640 1 -- 172.21.15.83:0/4287384863 shutdown_connections 2024-04-25T00:14:35.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.519+0000 7f3560800640 1 -- 172.21.15.83:0/4287384863 wait complete. 2024-04-25T00:14:35.542 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T00:14:35.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:35.543 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T00:14:35.571 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:14:35.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:35.572 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:14:35.622 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:35.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.751+0000 7f09bba00640 1 Processor -- start 2024-04-25T00:14:35.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.751+0000 7f09bba00640 1 -- start start 2024-04-25T00:14:35.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.755+0000 7f09bba00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09b40cc510 0x7f09b415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.755+0000 7f09bba00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09b415cf90 0x7f09b40ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.755+0000 7f09bba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09b40ccc60 con 0x7f09b41599f0 2024-04-25T00:14:35.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.755+0000 7f09bba00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f09b4159dd0 con 0x7f09b415cf90 2024-04-25T00:14:35.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.755+0000 7f09bba00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f09b406dba0 con 0x7f09b40cc510 2024-04-25T00:14:35.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.755+0000 7f09abe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09b40cc510 0x7f09b415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.755+0000 7f09b9a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09b415cf90 0x7f09b40ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.755+0000 7f09abe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09b40cc510 0x7f09b415ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:49566/0 (socket says 172.21.15.83:49566) 2024-04-25T00:14:35.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.755+0000 7f09abe00640 1 -- 172.21.15.83:0/1789046920 learned_addr learned my addr 172.21.15.83:0/1789046920 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:35.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.755+0000 7f09abe00640 1 -- 172.21.15.83:0/1789046920 >> v2:172.21.15.86:3300/0 conn(0x7f09b415cf90 msgr2=0x7f09b40ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:35.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.755+0000 7f09abe00640 1 --2- 172.21.15.83:0/1789046920 >> v2:172.21.15.86:3300/0 conn(0x7f09b415cf90 0x7f09b40ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.755+0000 7f09abe00640 1 -- 172.21.15.83:0/1789046920 >> v1:172.21.15.86:6789/0 conn(0x7f09b41599f0 legacy=0x7f09b415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.755+0000 7f09abe00640 1 -- 172.21.15.83:0/1789046920 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b406de80 con 0x7f09b40cc510 2024-04-25T00:14:35.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.759+0000 7f09abe00640 1 --2- 172.21.15.83:0/1789046920 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09b40cc510 0x7f09b415ca50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f099c009870 tx=0x7f099c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8a56c03e619ff56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:35.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.759+0000 7f09b3e00640 1 -- 172.21.15.83:0/1789046920 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c0b2070 con 0x7f09b40cc510 2024-04-25T00:14:35.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.759+0000 7f09b3e00640 1 -- 172.21.15.83:0/1789046920 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f099c0ad680 con 0x7f09b40cc510 2024-04-25T00:14:35.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.759+0000 7f09b3e00640 1 -- 172.21.15.83:0/1789046920 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c0ada50 con 0x7f09b40cc510 2024-04-25T00:14:35.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.759+0000 7f09bba00640 1 -- 172.21.15.83:0/1789046920 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09b40cc510 msgr2=0x7f09b415ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.759+0000 7f09bba00640 1 --2- 172.21.15.83:0/1789046920 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09b40cc510 0x7f09b415ca50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f099c009870 tx=0x7f099c01c4d0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.759+0000 7f09bba00640 1 -- 172.21.15.83:0/1789046920 shutdown_connections 2024-04-25T00:14:35.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.759+0000 7f09bba00640 1 --2- 172.21.15.83:0/1789046920 >> v2:172.21.15.86:3300/0 conn(0x7f09b415cf90 0x7f09b40ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.759+0000 7f09bba00640 1 --2- 172.21.15.83:0/1789046920 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09b40cc510 0x7f09b415ca50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.759+0000 7f09bba00640 1 -- 172.21.15.83:0/1789046920 >> 172.21.15.83:0/1789046920 conn(0x7f09b40b1830 msgr2=0x7f09b40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:35.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.759+0000 7f09bba00640 1 -- 172.21.15.83:0/1789046920 shutdown_connections 2024-04-25T00:14:35.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.759+0000 7f09bba00640 1 -- 172.21.15.83:0/1789046920 wait complete. 2024-04-25T00:14:35.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.759+0000 7f09bba00640 1 Processor -- start 2024-04-25T00:14:35.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.759+0000 7f09bba00640 1 -- start start 2024-04-25T00:14:35.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09bba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b40cc510 0x7f09b414e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09bba00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09b41599f0 0x7f09b4158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09bba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b415cf90 0x7f09b4158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09bba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09b40cf5b0 con 0x7f09b40cc510 2024-04-25T00:14:35.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09bba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09b40cf430 con 0x7f09b415cf90 2024-04-25T00:14:35.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09bba00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f09b40cf730 con 0x7f09b41599f0 2024-04-25T00:14:35.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09b9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b40cc510 0x7f09b414e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09b9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b40cc510 0x7f09b414e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44124/0 (socket says 172.21.15.83:44124) 2024-04-25T00:14:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09b9000640 1 -- 172.21.15.83:0/3930232931 learned_addr learned my addr 172.21.15.83:0/3930232931 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09b9a00640 1 --2- 172.21.15.83:0/3930232931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b415cf90 0x7f09b4158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09abe00640 1 --2- 172.21.15.83:0/3930232931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09b41599f0 0x7f09b4158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09b9000640 1 -- 172.21.15.83:0/3930232931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09b41599f0 msgr2=0x7f09b4158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09b9000640 1 --2- 172.21.15.83:0/3930232931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09b41599f0 0x7f09b4158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09b9000640 1 -- 172.21.15.83:0/3930232931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b415cf90 msgr2=0x7f09b4158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09b9000640 1 --2- 172.21.15.83:0/3930232931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b415cf90 0x7f09b4158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09b9000640 1 -- 172.21.15.83:0/3930232931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b4154060 con 0x7f09b40cc510 2024-04-25T00:14:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09b9000640 1 --2- 172.21.15.83:0/3930232931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b40cc510 0x7f09b414e7a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f09a400c420 tx=0x7f09a400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09b1600640 1 -- 172.21.15.83:0/3930232931 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a40052d0 con 0x7f09b40cc510 2024-04-25T00:14:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09b1600640 1 -- 172.21.15.83:0/3930232931 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a4005470 con 0x7f09b40cc510 2024-04-25T00:14:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09bba00640 1 -- 172.21.15.83:0/3930232931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b4154340 con 0x7f09b40cc510 2024-04-25T00:14:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09b1600640 1 -- 172.21.15.83:0/3930232931 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4007260 con 0x7f09b40cc510 2024-04-25T00:14:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09bba00640 1 -- 172.21.15.83:0/3930232931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b4154e20 con 0x7f09b40cc510 2024-04-25T00:14:35.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09bba00640 1 -- 172.21.15.83:0/3930232931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b40737f0 con 0x7f09b40cc510 2024-04-25T00:14:35.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.763+0000 7f09b1600640 1 -- 172.21.15.83:0/3930232931 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f09a4007400 con 0x7f09b40cc510 2024-04-25T00:14:35.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.767+0000 7f09b1600640 1 --2- 172.21.15.83:0/3930232931 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f09900678f0 0x7f0990069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:35.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.767+0000 7f09b1600640 1 -- 172.21.15.83:0/3930232931 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f09a4085250 con 0x7f09b40cc510 2024-04-25T00:14:35.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.767+0000 7f09abe00640 1 --2- 172.21.15.83:0/3930232931 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f09900678f0 0x7f0990069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:35.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.767+0000 7f09abe00640 1 --2- 172.21.15.83:0/3930232931 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f09900678f0 0x7f0990069db0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f099c0af000 tx=0x7f099c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:35.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.771+0000 7f09b1600640 1 -- 172.21.15.83:0/3930232931 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f09b40ccc60 con 0x7f09b40cc510 2024-04-25T00:14:35.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.959+0000 7f09bba00640 1 -- 172.21.15.83:0/3930232931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09b404f5f0 con 0x7f09b40cc510 2024-04-25T00:14:35.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.959+0000 7f09b1600640 1 -- 172.21.15.83:0/3930232931 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5174 (secure 0 0 0) 0x7f09a40555a0 con 0x7f09b40cc510 2024-04-25T00:14:35.962 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:35.962 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/3856734881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3856734881},{"type":"v1","addr":"172.21.15.86:6834","nonce":3856734881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/12764228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":12764228},{"type":"v1","addr":"172.21.15.86:6835","nonce":12764228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1680607346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1680607346},{"type":"v1","addr":"172.21.15.83:6837","nonce":1680607346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:22.706751+0000","modified":"2024-04-25T00:14:25.734428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7023},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7023":{"gid":7023,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/121840722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":121840722},{"type":"v1","addr":"172.21.15.83:6836","nonce":121840722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7023,"qdb_cluster":[7023]},"id":6}]} 2024-04-25T00:14:35.963 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 53 2024-04-25T00:14:35.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.959+0000 7f09bba00640 1 -- 172.21.15.83:0/3930232931 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f09900678f0 msgr2=0x7f0990069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.959+0000 7f09bba00640 1 --2- 172.21.15.83:0/3930232931 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f09900678f0 0x7f0990069db0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f099c0af000 tx=0x7f099c0a7000 comp rx=0 tx=0).stop 2024-04-25T00:14:35.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.959+0000 7f09bba00640 1 -- 172.21.15.83:0/3930232931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b40cc510 msgr2=0x7f09b414e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:35.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.959+0000 7f09bba00640 1 --2- 172.21.15.83:0/3930232931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b40cc510 0x7f09b414e7a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f09a400c420 tx=0x7f09a400c8e0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.959+0000 7f09bba00640 1 -- 172.21.15.83:0/3930232931 shutdown_connections 2024-04-25T00:14:35.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.959+0000 7f09bba00640 1 --2- 172.21.15.83:0/3930232931 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f09900678f0 0x7f0990069db0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.959+0000 7f09bba00640 1 --2- 172.21.15.83:0/3930232931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09b41599f0 0x7f09b4158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.959+0000 7f09bba00640 1 --2- 172.21.15.83:0/3930232931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b415cf90 0x7f09b4158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.959+0000 7f09bba00640 1 --2- 172.21.15.83:0/3930232931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b40cc510 0x7f09b414e7a0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:35.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.959+0000 7f09bba00640 1 -- 172.21.15.83:0/3930232931 >> 172.21.15.83:0/3930232931 conn(0x7f09b40b1830 msgr2=0x7f09b40b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:35.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.959+0000 7f09bba00640 1 -- 172.21.15.83:0/3930232931 shutdown_connections 2024-04-25T00:14:35.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:35.959+0000 7f09bba00640 1 -- 172.21.15.83:0/3930232931 wait complete. 2024-04-25T00:14:35.983 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 session ls 2024-04-25T00:14:36.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.115+0000 7f1460800640 1 Processor -- start 2024-04-25T00:14:36.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.115+0000 7f1460800640 1 -- start start 2024-04-25T00:14:36.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.115+0000 7f1460800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14581551e0 0x7f145815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.119+0000 7f1460800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145806e4c0 0x7f145806e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.119+0000 7f1460800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f145815c560 con 0x7f14581555c0 2024-04-25T00:14:36.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.119+0000 7f1460800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1458072460 con 0x7f14581551e0 2024-04-25T00:14:36.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.119+0000 7f1460800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f145815c7d0 con 0x7f145806e4c0 2024-04-25T00:14:36.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.119+0000 7f145e800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145806e4c0 0x7f145806e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.119+0000 7f145de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14581551e0 0x7f145815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.119+0000 7f145e800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145806e4c0 0x7f145806e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:49580/0 (socket says 172.21.15.83:49580) 2024-04-25T00:14:36.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.119+0000 7f145de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14581551e0 0x7f145815bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:60048/0 (socket says 172.21.15.83:60048) 2024-04-25T00:14:36.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.119+0000 7f145e800640 1 -- 172.21.15.83:0/1708165030 learned_addr learned my addr 172.21.15.83:0/1708165030 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:36.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.119+0000 7f145e800640 1 -- 172.21.15.83:0/1708165030 >> v2:172.21.15.86:3300/0 conn(0x7f14581551e0 msgr2=0x7f145815bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:36.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.119+0000 7f145e800640 1 --2- 172.21.15.83:0/1708165030 >> v2:172.21.15.86:3300/0 conn(0x7f14581551e0 0x7f145815bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.119+0000 7f145e800640 1 -- 172.21.15.83:0/1708165030 >> v1:172.21.15.86:6789/0 conn(0x7f14581555c0 legacy=0x7f1458071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.119+0000 7f145e800640 1 -- 172.21.15.83:0/1708165030 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145815cab0 con 0x7f145806e4c0 2024-04-25T00:14:36.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.123+0000 7f145e800640 1 --2- 172.21.15.83:0/1708165030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145806e4c0 0x7f145806e950 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1454009e80 tx=0x7f145401ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c2653fda010963 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:36.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.123+0000 7f145ca00640 1 -- 172.21.15.83:0/1708165030 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14540a55e0 con 0x7f145806e4c0 2024-04-25T00:14:36.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.123+0000 7f145ca00640 1 -- 172.21.15.83:0/1708165030 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14540a5780 con 0x7f145806e4c0 2024-04-25T00:14:36.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.123+0000 7f145ca00640 1 -- 172.21.15.83:0/1708165030 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14540a5b50 con 0x7f145806e4c0 2024-04-25T00:14:36.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.123+0000 7f1460800640 1 -- 172.21.15.83:0/1708165030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145806e4c0 msgr2=0x7f145806e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.123+0000 7f1460800640 1 --2- 172.21.15.83:0/1708165030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145806e4c0 0x7f145806e950 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1454009e80 tx=0x7f145401ca10 comp rx=0 tx=0).stop 2024-04-25T00:14:36.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.123+0000 7f1460800640 1 -- 172.21.15.83:0/1708165030 shutdown_connections 2024-04-25T00:14:36.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.123+0000 7f1460800640 1 --2- 172.21.15.83:0/1708165030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145806e4c0 0x7f145806e950 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.123+0000 7f1460800640 1 --2- 172.21.15.83:0/1708165030 >> v2:172.21.15.86:3300/0 conn(0x7f14581551e0 0x7f145815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.123+0000 7f1460800640 1 -- 172.21.15.83:0/1708165030 >> 172.21.15.83:0/1708165030 conn(0x7f14580b1830 msgr2=0x7f14580b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:36.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.123+0000 7f1460800640 1 -- 172.21.15.83:0/1708165030 shutdown_connections 2024-04-25T00:14:36.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.123+0000 7f1460800640 1 -- 172.21.15.83:0/1708165030 wait complete. 2024-04-25T00:14:36.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.123+0000 7f1460800640 1 Processor -- start 2024-04-25T00:14:36.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.123+0000 7f1460800640 1 -- start start 2024-04-25T00:14:36.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f1460800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145806e4c0 0x7f1458149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f1460800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14581551e0 0x7f14581530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f1460800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14581555c0 0x7f1458153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f1460800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f145806df90 con 0x7f14581551e0 2024-04-25T00:14:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f1460800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f145806de10 con 0x7f14581555c0 2024-04-25T00:14:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f1460800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f145806e110 con 0x7f145806e4c0 2024-04-25T00:14:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f145d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14581551e0 0x7f14581530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f145d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14581551e0 0x7f14581530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44126/0 (socket says 172.21.15.83:44126) 2024-04-25T00:14:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f145d400640 1 -- 172.21.15.83:0/2838558205 learned_addr learned my addr 172.21.15.83:0/2838558205 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f145de00640 1 --2- 172.21.15.83:0/2838558205 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145806e4c0 0x7f1458149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f145e800640 1 --2- 172.21.15.83:0/2838558205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14581555c0 0x7f1458153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f145d400640 1 -- 172.21.15.83:0/2838558205 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145806e4c0 msgr2=0x7f1458149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f145d400640 1 --2- 172.21.15.83:0/2838558205 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145806e4c0 0x7f1458149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f145d400640 1 -- 172.21.15.83:0/2838558205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14581555c0 msgr2=0x7f1458153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f145d400640 1 --2- 172.21.15.83:0/2838558205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14581555c0 0x7f1458153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f145d400640 1 -- 172.21.15.83:0/2838558205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145814ef30 con 0x7f14581551e0 2024-04-25T00:14:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f145d400640 1 --2- 172.21.15.83:0/2838558205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14581551e0 0x7f14581530c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f144800c340 tx=0x7f144800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f1452000640 1 -- 172.21.15.83:0/2838558205 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14480052d0 con 0x7f14581551e0 2024-04-25T00:14:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f1452000640 1 -- 172.21.15.83:0/2838558205 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1448005470 con 0x7f14581551e0 2024-04-25T00:14:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f1452000640 1 -- 172.21.15.83:0/2838558205 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448007260 con 0x7f14581551e0 2024-04-25T00:14:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f1460800640 1 -- 172.21.15.83:0/2838558205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145814f210 con 0x7f14581551e0 2024-04-25T00:14:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.127+0000 7f1460800640 1 -- 172.21.15.83:0/2838558205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145814f7d0 con 0x7f14581551e0 2024-04-25T00:14:36.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f1452000640 1 -- 172.21.15.83:0/2838558205 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1448007400 con 0x7f14581551e0 2024-04-25T00:14:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f1452000640 1 --2- 172.21.15.83:0/2838558205 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f143c0678c0 0x7f143c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f1452000640 1 -- 172.21.15.83:0/2838558205 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f14480851a0 con 0x7f14581551e0 2024-04-25T00:14:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f145de00640 1 --2- 172.21.15.83:0/2838558205 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f143c0678c0 0x7f143c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f1463d30000 1 Processor -- start 2024-04-25T00:14:36.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f1463d30000 1 -- start start 2024-04-25T00:14:36.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f145de00640 1 --2- 172.21.15.83:0/2838558205 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f143c0678c0 0x7f143c069d80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f144c009b60 tx=0x7f144c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f1463d30000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b081b220 0x5617b081b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f1463d30000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081bb40 0x5617b07ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f1463d30000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b07cee20 0x5617b07cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f1463d30000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5617b061f830 con 0x5617b07cee20 2024-04-25T00:14:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f1463d30000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5617b05bce70 con 0x5617b081bb40 2024-04-25T00:14:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f1463d30000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x5617b061b7f0 con 0x5617b081b220 2024-04-25T00:14:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f145e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b07cee20 0x5617b07cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f145e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b07cee20 0x5617b07cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44142/0 (socket says 172.21.15.83:44142) 2024-04-25T00:14:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f145e800640 1 -- 172.21.15.83:0/941515005 learned_addr learned my addr 172.21.15.83:0/941515005 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f145de00640 1 --2- 172.21.15.83:0/941515005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b081b220 0x5617b081b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f145d400640 1 --2- 172.21.15.83:0/941515005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081bb40 0x5617b07ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f145e800640 1 -- 172.21.15.83:0/941515005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b081b220 msgr2=0x5617b081b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f145e800640 1 --2- 172.21.15.83:0/941515005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b081b220 0x5617b081b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f145e800640 1 -- 172.21.15.83:0/941515005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081bb40 msgr2=0x5617b07ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:36.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f145e800640 1 --2- 172.21.15.83:0/941515005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081bb40 0x5617b07ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f145e800640 1 -- 172.21.15.83:0/941515005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b05da150 con 0x5617b07cee20 2024-04-25T00:14:36.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f145e800640 1 --2- 172.21.15.83:0/941515005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b07cee20 0x5617b07cf200 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f14540af040 tx=0x7f1454004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e72e6907934f4ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:36.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f1446a00640 1 -- 172.21.15.83:0/941515005 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14540a55e0 con 0x5617b07cee20 2024-04-25T00:14:36.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f1446a00640 1 -- 172.21.15.83:0/941515005 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14540a5780 con 0x5617b07cee20 2024-04-25T00:14:36.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.131+0000 7f1446a00640 1 -- 172.21.15.83:0/941515005 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14540a5bb0 con 0x5617b07cee20 2024-04-25T00:14:36.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 -- 172.21.15.83:0/941515005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b07cee20 msgr2=0x5617b07cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 --2- 172.21.15.83:0/941515005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b07cee20 0x5617b07cf200 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f14540af040 tx=0x7f1454004ac0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 -- 172.21.15.83:0/941515005 shutdown_connections 2024-04-25T00:14:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 --2- 172.21.15.83:0/941515005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b07cee20 0x5617b07cf200 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 --2- 172.21.15.83:0/941515005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081bb40 0x5617b07ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 --2- 172.21.15.83:0/941515005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b081b220 0x5617b081b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 -- 172.21.15.83:0/941515005 >> 172.21.15.83:0/941515005 conn(0x5617b07fbdb0 msgr2=0x5617b07fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 -- 172.21.15.83:0/941515005 shutdown_connections 2024-04-25T00:14:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 -- 172.21.15.83:0/941515005 wait complete. 2024-04-25T00:14:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 -- start start 2024-04-25T00:14:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 Processor -- start 2024-04-25T00:14:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 0x5617b0822580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081b220 0x5617b0822ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081bb40 0x5617b07d45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5617b02ea680 con 0x5617b081bb40 2024-04-25T00:14:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5617b05dd0a0 con 0x5617b081b220 2024-04-25T00:14:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x5617b062d8d0 con 0x5617b07cee20 2024-04-25T00:14:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 10 client.0 _ll_register_callbacks cb 0x5617b05cc330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f145d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081b220 0x5617b0822ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f145de00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 0x5617b0822580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f145d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081b220 0x5617b0822ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:60070/0 (socket says 172.21.15.83:60070) 2024-04-25T00:14:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f145e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081bb40 0x5617b07d45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f145d400640 1 -- 172.21.15.83:0/2466234869 learned_addr learned my addr 172.21.15.83:0/2466234869 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f145d400640 1 -- 172.21.15.83:0/2466234869 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 msgr2=0x5617b0822580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f145d400640 1 --2- 172.21.15.83:0/2466234869 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 0x5617b0822580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f145d400640 1 -- 172.21.15.83:0/2466234869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081bb40 msgr2=0x5617b07d45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f145d400640 1 --2- 172.21.15.83:0/2466234869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081bb40 0x5617b07d45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f145d400640 1 -- 172.21.15.83:0/2466234869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b05c1ab0 con 0x5617b081b220 2024-04-25T00:14:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f145e800640 1 --2- 172.21.15.83:0/2466234869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081bb40 0x5617b07d45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f145d400640 1 --2- 172.21.15.83:0/2466234869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081b220 0x5617b0822ac0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f144800a040 tx=0x7f144800cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e397abe066e0bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f143b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T00:14:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f143b400640 1 -- 172.21.15.83:0/2466234869 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144806d180 con 0x5617b081b220 2024-04-25T00:14:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 -- 172.21.15.83:0/2466234869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14540a5780 con 0x5617b081b220 2024-04-25T00:14:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f143b400640 1 -- 172.21.15.83:0/2466234869 <== mon.1 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f144806d320 con 0x5617b081b220 2024-04-25T00:14:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f143b400640 1 -- 172.21.15.83:0/2466234869 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f1448083120 con 0x5617b081b220 2024-04-25T00:14:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 10 client.6862 fetch_fsmap learned FSMap version 53 2024-04-25T00:14:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.135+0000 7f1463d30000 1 -- 172.21.15.83:0/2466234869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x5617b06a1230 con 0x5617b081b220 2024-04-25T00:14:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f143b400640 1 -- 172.21.15.83:0/2466234869 <== mon.1 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14540a5780 con 0x5617b081b220 2024-04-25T00:14:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f143b400640 1 -- 172.21.15.83:0/2466234869 <== mon.1 v2:172.21.15.86:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f1448054950 con 0x5617b081b220 2024-04-25T00:14:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 10 client.6862 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T00:14:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 10 client.6862 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T00:14:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 20 client.6862 populate_metadata read hostname 'smithi083' 2024-04-25T00:14:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 1 --2- 172.21.15.83:0/2466234869 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x5617b07d5370 0x5617b07d5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 4 client.6862 mds_command: new command op to 7023 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:14:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 1 -- 172.21.15.83:0/2466234869 --> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617b05bacd0 con 0x5617b07d5370 2024-04-25T00:14:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f145de00640 1 --2- 172.21.15.83:0/2466234869 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x5617b07d5370 0x5617b07d5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f145de00640 1 --2- 172.21.15.83:0/2466234869 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x5617b07d5370 0x5617b07d5780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5deeeed46b49f2e server_cookie=ef4d6e96cbfd0440 in_seq=0 out_seq=0 2024-04-25T00:14:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f143b400640 10 client.6862 ms_handle_connect on v2:172.21.15.83:6834/121840722 2024-04-25T00:14:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f143b400640 1 -- 172.21.15.83:0/2466234869 <== mds.0 v2:172.21.15.83:6834/121840722 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5617b05bacd0 con 0x5617b07d5370 2024-04-25T00:14:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f143b400640 10 client.6862 handle_command_reply: tid=0 2024-04-25T00:14:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 1 client.6862 shutdown 2024-04-25T00:14:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 1 -- 172.21.15.83:0/2466234869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081b220 msgr2=0x5617b0822ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 1 --2- 172.21.15.83:0/2466234869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081b220 0x5617b0822ac0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f144800a040 tx=0x7f144800cf90 comp rx=0 tx=0).stop 2024-04-25T00:14:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 1 -- 172.21.15.83:0/2466234869 shutdown_connections 2024-04-25T00:14:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 1 --2- 172.21.15.83:0/2466234869 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x5617b07d5370 0x5617b07d5780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 1 --2- 172.21.15.83:0/2466234869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081bb40 0x5617b07d45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 1 --2- 172.21.15.83:0/2466234869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081b220 0x5617b0822ac0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 1 --2- 172.21.15.83:0/2466234869 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 0x5617b0822580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f143b400640 0 client.6862 ms_handle_reset on v2:172.21.15.83:6834/121840722 2024-04-25T00:14:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 1 -- 172.21.15.83:0/2466234869 >> 172.21.15.83:0/2466234869 conn(0x5617b07fbdb0 msgr2=0x5617b07fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 1 -- 172.21.15.83:0/2466234869 shutdown_connections 2024-04-25T00:14:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 1 -- 172.21.15.83:0/2466234869 wait complete. 2024-04-25T00:14:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.139+0000 7f1463d30000 20 client.6862 trim_cache size 0 max 16384 2024-04-25T00:14:36.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.155+0000 7f1463d30000 1 Processor -- start 2024-04-25T00:14:36.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.155+0000 7f1463d30000 1 -- start start 2024-04-25T00:14:36.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 0x5617b0887130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081b220 0x5617b0887670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081bb40 0x5617b0887bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14582039a0 con 0x5617b081bb40 2024-04-25T00:14:36.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1458166fb0 con 0x5617b081b220 2024-04-25T00:14:36.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f145814acd0 con 0x5617b07cee20 2024-04-25T00:14:36.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f145e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081bb40 0x5617b0887bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f145e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081bb40 0x5617b0887bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44160/0 (socket says 172.21.15.83:44160) 2024-04-25T00:14:36.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f145e800640 1 -- 172.21.15.83:0/2621854312 learned_addr learned my addr 172.21.15.83:0/2621854312 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:36.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f145de00640 1 --2- 172.21.15.83:0/2621854312 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 0x5617b0887130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f145d400640 1 --2- 172.21.15.83:0/2621854312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081b220 0x5617b0887670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f145e800640 1 -- 172.21.15.83:0/2621854312 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 msgr2=0x5617b0887130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:36.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f145e800640 1 --2- 172.21.15.83:0/2621854312 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 0x5617b0887130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f145e800640 1 -- 172.21.15.83:0/2621854312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081b220 msgr2=0x5617b0887670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:36.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f145e800640 1 --2- 172.21.15.83:0/2621854312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081b220 0x5617b0887670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f145e800640 1 -- 172.21.15.83:0/2621854312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b0678550 con 0x5617b081bb40 2024-04-25T00:14:36.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f145d400640 1 --2- 172.21.15.83:0/2621854312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081b220 0x5617b0887670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:36.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f145e800640 1 --2- 172.21.15.83:0/2621854312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081bb40 0x5617b0887bb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1454009e50 tx=0x7f1454009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e879e9af15d5da2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:36.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1446000640 1 -- 172.21.15.83:0/2621854312 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1454004040 con 0x5617b081bb40 2024-04-25T00:14:36.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1446000640 1 -- 172.21.15.83:0/2621854312 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14540041e0 con 0x5617b081bb40 2024-04-25T00:14:36.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1446000640 1 -- 172.21.15.83:0/2621854312 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14540045d0 con 0x5617b081bb40 2024-04-25T00:14:36.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 -- 172.21.15.83:0/2621854312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081bb40 msgr2=0x5617b0887bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 --2- 172.21.15.83:0/2621854312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081bb40 0x5617b0887bb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1454009e50 tx=0x7f1454009810 comp rx=0 tx=0).stop 2024-04-25T00:14:36.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 -- 172.21.15.83:0/2621854312 shutdown_connections 2024-04-25T00:14:36.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 --2- 172.21.15.83:0/2621854312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081bb40 0x5617b0887bb0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 --2- 172.21.15.83:0/2621854312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081b220 0x5617b0887670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 --2- 172.21.15.83:0/2621854312 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 0x5617b0887130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 -- 172.21.15.83:0/2621854312 >> 172.21.15.83:0/2621854312 conn(0x5617b07fbdb0 msgr2=0x5617b0832610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:36.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 -- 172.21.15.83:0/2621854312 shutdown_connections 2024-04-25T00:14:36.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 -- 172.21.15.83:0/2621854312 wait complete. 2024-04-25T00:14:36.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 -- start start 2024-04-25T00:14:36.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.159+0000 7f1463d30000 1 Processor -- start 2024-04-25T00:14:36.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 0x5617b088bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081b220 0x5617b088c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081bb40 0x5617b081fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5617b02ea680 con 0x5617b081b220 2024-04-25T00:14:36.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f145814b320 con 0x5617b081bb40 2024-04-25T00:14:36.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x5617b0623e60 con 0x5617b07cee20 2024-04-25T00:14:36.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 10 client.0 _ll_register_callbacks cb 0x7f144c009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:36.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f145d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081b220 0x5617b088c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f145d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081b220 0x5617b088c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44178/0 (socket says 172.21.15.83:44178) 2024-04-25T00:14:36.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f145d400640 1 -- 172.21.15.83:0/2458387503 learned_addr learned my addr 172.21.15.83:0/2458387503 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:36.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f145de00640 1 --2- 172.21.15.83:0/2458387503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 0x5617b088bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f145e800640 1 --2- 172.21.15.83:0/2458387503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081bb40 0x5617b081fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f145d400640 1 -- 172.21.15.83:0/2458387503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 msgr2=0x5617b088bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:36.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f145d400640 1 --2- 172.21.15.83:0/2458387503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 0x5617b088bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f145d400640 1 -- 172.21.15.83:0/2458387503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081bb40 msgr2=0x5617b081fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:36.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f145d400640 1 --2- 172.21.15.83:0/2458387503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081bb40 0x5617b081fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f145d400640 1 -- 172.21.15.83:0/2458387503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b05c61f0 con 0x5617b081b220 2024-04-25T00:14:36.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f145e800640 1 --2- 172.21.15.83:0/2458387503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081bb40 0x5617b081fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:36.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f145d400640 1 --2- 172.21.15.83:0/2458387503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081b220 0x5617b088c0d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f14480537b0 tx=0x7f1448083410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253321e96b22b1ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:36.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f143b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:14:36.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f143b400640 1 -- 172.21.15.83:0/2458387503 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448055040 con 0x5617b081b220 2024-04-25T00:14:36.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f143b400640 1 -- 172.21.15.83:0/2458387503 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f1448051250 con 0x5617b081b220 2024-04-25T00:14:36.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 -- 172.21.15.83:0/2458387503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14540041e0 con 0x5617b081b220 2024-04-25T00:14:36.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f143b400640 1 -- 172.21.15.83:0/2458387503 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f1448079050 con 0x5617b081b220 2024-04-25T00:14:36.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 10 client.7290 fetch_fsmap learned FSMap version 53 2024-04-25T00:14:36.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 -- 172.21.15.83:0/2458387503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x5617b05e5850 con 0x5617b081b220 2024-04-25T00:14:36.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f143b400640 1 -- 172.21.15.83:0/2458387503 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14540041e0 con 0x5617b081b220 2024-04-25T00:14:36.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f143b400640 1 -- 172.21.15.83:0/2458387503 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f144807c080 con 0x5617b081b220 2024-04-25T00:14:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 10 client.7290 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T00:14:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 10 client.7290 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T00:14:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 20 client.7290 populate_metadata read hostname 'smithi083' 2024-04-25T00:14:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 --2- 172.21.15.83:0/2458387503 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x5617b0820540 0x5617b0820920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 4 client.7290 mds_command: new command op to 7023 tid=0 multi_id=0 [{"prefix": "session ls"}] 2024-04-25T00:14:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 -- 172.21.15.83:0/2458387503 --> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] -- command(tid 0: {"prefix": "session ls"}) -- 0x5617b062cd70 con 0x5617b0820540 2024-04-25T00:14:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f145de00640 1 --2- 172.21.15.83:0/2458387503 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x5617b0820540 0x5617b0820920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f145de00640 1 --2- 172.21.15.83:0/2458387503 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x5617b0820540 0x5617b0820920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa3928e394a6c85 server_cookie=d5a4df72a0039a3b in_seq=0 out_seq=0 2024-04-25T00:14:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f143b400640 10 client.7290 ms_handle_connect on v2:172.21.15.83:6834/121840722 2024-04-25T00:14:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f143b400640 1 -- 172.21.15.83:0/2458387503 <== mds.0 v2:172.21.15.83:6834/121840722 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617b062cd70 con 0x5617b0820540 2024-04-25T00:14:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f143b400640 10 client.7290 handle_command_reply: tid=0 2024-04-25T00:14:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 client.7290 shutdown 2024-04-25T00:14:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 -- 172.21.15.83:0/2458387503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081b220 msgr2=0x5617b088c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 --2- 172.21.15.83:0/2458387503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081b220 0x5617b088c0d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f14480537b0 tx=0x7f1448083410 comp rx=0 tx=0).stop 2024-04-25T00:14:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 -- 172.21.15.83:0/2458387503 shutdown_connections 2024-04-25T00:14:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 --2- 172.21.15.83:0/2458387503 >> [v2:172.21.15.83:6834/121840722,v1:172.21.15.83:6836/121840722] conn(0x5617b0820540 0x5617b0820920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 --2- 172.21.15.83:0/2458387503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617b081bb40 0x5617b081fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 --2- 172.21.15.83:0/2458387503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617b081b220 0x5617b088c0d0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 --2- 172.21.15.83:0/2458387503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5617b07cee20 0x5617b088bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f143b400640 0 client.7290 ms_handle_reset on v2:172.21.15.83:6834/121840722 2024-04-25T00:14:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 -- 172.21.15.83:0/2458387503 >> 172.21.15.83:0/2458387503 conn(0x5617b07fbdb0 msgr2=0x5617b07cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 -- 172.21.15.83:0/2458387503 shutdown_connections 2024-04-25T00:14:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 1 -- 172.21.15.83:0/2458387503 wait complete. 2024-04-25T00:14:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1463d30000 20 client.7290 trim_cache size 0 max 16384 2024-04-25T00:14:36.173 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-25T00:14:36.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1460800640 1 -- 172.21.15.83:0/2838558205 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f143c0678c0 msgr2=0x7f143c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1460800640 1 --2- 172.21.15.83:0/2838558205 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f143c0678c0 0x7f143c069d80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f144c009b60 tx=0x7f144c009190 comp rx=0 tx=0).stop 2024-04-25T00:14:36.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1460800640 1 -- 172.21.15.83:0/2838558205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14581551e0 msgr2=0x7f14581530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1460800640 1 --2- 172.21.15.83:0/2838558205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14581551e0 0x7f14581530c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f144800c340 tx=0x7f144800c800 comp rx=0 tx=0).stop 2024-04-25T00:14:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1460800640 1 -- 172.21.15.83:0/2838558205 shutdown_connections 2024-04-25T00:14:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1460800640 1 --2- 172.21.15.83:0/2838558205 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f143c0678c0 0x7f143c069d80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1460800640 1 --2- 172.21.15.83:0/2838558205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14581555c0 0x7f1458153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1460800640 1 --2- 172.21.15.83:0/2838558205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14581551e0 0x7f14581530c0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.163+0000 7f1460800640 1 --2- 172.21.15.83:0/2838558205 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145806e4c0 0x7f1458149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.167+0000 7f1460800640 1 -- 172.21.15.83:0/2838558205 >> 172.21.15.83:0/2838558205 conn(0x7f14580b1830 msgr2=0x7f14580b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.167+0000 7f1460800640 1 -- 172.21.15.83:0/2838558205 shutdown_connections 2024-04-25T00:14:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.167+0000 7f1460800640 1 -- 172.21.15.83:0/2838558205 wait complete. 2024-04-25T00:14:36.188 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-04-25T00:14:36.345 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-25T00:14:36.345 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-04-25T00:14:36.345 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:14:36.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:14:36.444 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:14:36.527 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:14:36.528 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:14:36.528 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:14:36.528 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:36.536 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:14:36.537 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:14:36.537 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:14:36.537 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:36.546 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:14:36.546 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:14:36.546 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:14:36.546 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:36.593 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:14:36.593 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:14:36.593 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:14:36.593 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:36.602 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:14:36.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:36.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bdc400640 1 Processor -- start 2024-04-25T00:14:36.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bdc400640 1 -- start start 2024-04-25T00:14:36.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bdc400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6bd41553e0 0x7f6bd406e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bdc400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6bd406ea00 0x7f6bd40b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bdc400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bd40b3b40 con 0x7f6bd40b48a0 2024-04-25T00:14:36.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bdc400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd406ef80 con 0x7f6bd41553e0 2024-04-25T00:14:36.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bdc400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd415b510 con 0x7f6bd406ea00 2024-04-25T00:14:36.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bd9a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6bd41553e0 0x7f6bd406e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bd9a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6bd41553e0 0x7f6bd406e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44184/0 (socket says 172.21.15.83:44184) 2024-04-25T00:14:36.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bd9a00640 1 -- 172.21.15.83:0/3873709453 learned_addr learned my addr 172.21.15.83:0/3873709453 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:36.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bd9000640 1 --2- 172.21.15.83:0/3873709453 >> v2:172.21.15.86:3300/0 conn(0x7f6bd406ea00 0x7f6bd40b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bd9a00640 1 -- 172.21.15.83:0/3873709453 >> v2:172.21.15.86:3300/0 conn(0x7f6bd406ea00 msgr2=0x7f6bd40b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:36.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bd9a00640 1 --2- 172.21.15.83:0/3873709453 >> v2:172.21.15.86:3300/0 conn(0x7f6bd406ea00 0x7f6bd40b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bd9a00640 1 -- 172.21.15.83:0/3873709453 >> v1:172.21.15.86:6789/0 conn(0x7f6bd40b48a0 legacy=0x7f6bd415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.767+0000 7f6bd9a00640 1 -- 172.21.15.83:0/3873709453 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd415b7f0 con 0x7f6bd41553e0 2024-04-25T00:14:36.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.771+0000 7f6bd9a00640 1 --2- 172.21.15.83:0/3873709453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd41553e0 0x7f6bd406e4c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40cf5b0 tx=0x7f6bc801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383a21e08024695e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:36.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.771+0000 7f6bd3e00640 1 -- 172.21.15.83:0/3873709453 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc80b0070 con 0x7f6bd41553e0 2024-04-25T00:14:36.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.771+0000 7f6bd3e00640 1 -- 172.21.15.83:0/3873709453 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc80ab600 con 0x7f6bd41553e0 2024-04-25T00:14:36.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.771+0000 7f6bd3e00640 1 -- 172.21.15.83:0/3873709453 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc80ab9d0 con 0x7f6bd41553e0 2024-04-25T00:14:36.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.771+0000 7f6bdc400640 1 -- 172.21.15.83:0/3873709453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd41553e0 msgr2=0x7f6bd406e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.771+0000 7f6bdc400640 1 --2- 172.21.15.83:0/3873709453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd41553e0 0x7f6bd406e4c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40cf5b0 tx=0x7f6bc801c490 comp rx=0 tx=0).stop 2024-04-25T00:14:36.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.771+0000 7f6bdc400640 1 -- 172.21.15.83:0/3873709453 shutdown_connections 2024-04-25T00:14:36.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.771+0000 7f6bdc400640 1 --2- 172.21.15.83:0/3873709453 >> v2:172.21.15.86:3300/0 conn(0x7f6bd406ea00 0x7f6bd40b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.771+0000 7f6bdc400640 1 --2- 172.21.15.83:0/3873709453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd41553e0 0x7f6bd406e4c0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.771+0000 7f6bdc400640 1 -- 172.21.15.83:0/3873709453 >> 172.21.15.83:0/3873709453 conn(0x7f6bd40b1830 msgr2=0x7f6bd40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:36.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.771+0000 7f6bdc400640 1 -- 172.21.15.83:0/3873709453 shutdown_connections 2024-04-25T00:14:36.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.771+0000 7f6bdc400640 1 -- 172.21.15.83:0/3873709453 wait complete. 2024-04-25T00:14:36.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bdc400640 1 Processor -- start 2024-04-25T00:14:36.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bdc400640 1 -- start start 2024-04-25T00:14:36.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bdc400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bd406ea00 0x7f6bd4071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bdc400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd40b48a0 0x7f6bd414a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bdc400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd41553e0 0x7f6bd414a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bdc400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd4155940 con 0x7f6bd41553e0 2024-04-25T00:14:36.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bdc400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd41557c0 con 0x7f6bd40b48a0 2024-04-25T00:14:36.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bdc400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd4167d90 con 0x7f6bd406ea00 2024-04-25T00:14:36.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bda400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd41553e0 0x7f6bd414a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bda400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd41553e0 0x7f6bd414a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44196/0 (socket says 172.21.15.83:44196) 2024-04-25T00:14:36.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bda400640 1 -- 172.21.15.83:0/647658818 learned_addr learned my addr 172.21.15.83:0/647658818 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:36.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bd9000640 1 --2- 172.21.15.83:0/647658818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd40b48a0 0x7f6bd414a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bd9a00640 1 --2- 172.21.15.83:0/647658818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bd406ea00 0x7f6bd4071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bda400640 1 -- 172.21.15.83:0/647658818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bd406ea00 msgr2=0x7f6bd4071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bda400640 1 --2- 172.21.15.83:0/647658818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bd406ea00 0x7f6bd4071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bda400640 1 -- 172.21.15.83:0/647658818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd40b48a0 msgr2=0x7f6bd414a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bda400640 1 --2- 172.21.15.83:0/647658818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd40b48a0 0x7f6bd414a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bda400640 1 -- 172.21.15.83:0/647658818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd40cf1f0 con 0x7f6bd41553e0 2024-04-25T00:14:36.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bda400640 1 --2- 172.21.15.83:0/647658818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd41553e0 0x7f6bd414a710 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00c730 tx=0x7f6bcc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:36.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bd1600640 1 -- 172.21.15.83:0/647658818 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc003b80 con 0x7f6bd41553e0 2024-04-25T00:14:36.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bd1600640 1 -- 172.21.15.83:0/647658818 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bcc003d20 con 0x7f6bd41553e0 2024-04-25T00:14:36.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bd1600640 1 -- 172.21.15.83:0/647658818 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc00aa30 con 0x7f6bd41553e0 2024-04-25T00:14:36.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bdc400640 1 -- 172.21.15.83:0/647658818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd415a6b0 con 0x7f6bd41553e0 2024-04-25T00:14:36.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.775+0000 7f6bdc400640 1 -- 172.21.15.83:0/647658818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd415ac70 con 0x7f6bd41553e0 2024-04-25T00:14:36.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.779+0000 7f6bdc400640 1 -- 172.21.15.83:0/647658818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd40737f0 con 0x7f6bd41553e0 2024-04-25T00:14:36.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.779+0000 7f6bd1600640 1 -- 172.21.15.83:0/647658818 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6bcc00abd0 con 0x7f6bd41553e0 2024-04-25T00:14:36.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.779+0000 7f6bd1600640 1 --2- 172.21.15.83:0/647658818 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6bb40678f0 0x7f6bb4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:36.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.779+0000 7f6bd1600640 1 -- 172.21.15.83:0/647658818 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f6bcc085c30 con 0x7f6bd41553e0 2024-04-25T00:14:36.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.779+0000 7f6bd9a00640 1 --2- 172.21.15.83:0/647658818 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6bb40678f0 0x7f6bb4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:36.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.779+0000 7f6bd9a00640 1 --2- 172.21.15.83:0/647658818 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6bb40678f0 0x7f6bb4069db0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80ad000 tx=0x7f6bc8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:36.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.783+0000 7f6bd1600640 1 -- 172.21.15.83:0/647658818 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6bcc055e60 con 0x7f6bd41553e0 2024-04-25T00:14:36.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.971+0000 7f6bdc400640 1 -- 172.21.15.83:0/647658818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bd40cf3c0 con 0x7f6bd41553e0 2024-04-25T00:14:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.971+0000 7f6bd1600640 1 -- 172.21.15.83:0/647658818 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5174 (secure 0 0 0) 0x7f6bcc05a160 con 0x7f6bd41553e0 2024-04-25T00:14:36.976 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:36.976 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/3856734881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3856734881},{"type":"v1","addr":"172.21.15.86:6834","nonce":3856734881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/12764228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":12764228},{"type":"v1","addr":"172.21.15.86:6835","nonce":12764228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1680607346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1680607346},{"type":"v1","addr":"172.21.15.83:6837","nonce":1680607346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:22.706751+0000","modified":"2024-04-25T00:14:25.734428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7023},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7023":{"gid":7023,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/121840722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":121840722},{"type":"v1","addr":"172.21.15.83:6836","nonce":121840722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7023,"qdb_cluster":[7023]},"id":6}]} 2024-04-25T00:14:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 53 2024-04-25T00:14:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.975+0000 7f6bdc400640 1 -- 172.21.15.83:0/647658818 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6bb40678f0 msgr2=0x7f6bb4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.975+0000 7f6bdc400640 1 --2- 172.21.15.83:0/647658818 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6bb40678f0 0x7f6bb4069db0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80ad000 tx=0x7f6bc8008040 comp rx=0 tx=0).stop 2024-04-25T00:14:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.975+0000 7f6bdc400640 1 -- 172.21.15.83:0/647658818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd41553e0 msgr2=0x7f6bd414a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.975+0000 7f6bdc400640 1 --2- 172.21.15.83:0/647658818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd41553e0 0x7f6bd414a710 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00c730 tx=0x7f6bcc00cbf0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.975+0000 7f6bdc400640 1 -- 172.21.15.83:0/647658818 shutdown_connections 2024-04-25T00:14:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.975+0000 7f6bdc400640 1 --2- 172.21.15.83:0/647658818 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6bb40678f0 0x7f6bb4069db0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.975+0000 7f6bdc400640 1 --2- 172.21.15.83:0/647658818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd41553e0 0x7f6bd414a710 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.975+0000 7f6bdc400640 1 --2- 172.21.15.83:0/647658818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bd40b48a0 0x7f6bd414a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.975+0000 7f6bdc400640 1 --2- 172.21.15.83:0/647658818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bd406ea00 0x7f6bd4071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.975+0000 7f6bdc400640 1 -- 172.21.15.83:0/647658818 >> 172.21.15.83:0/647658818 conn(0x7f6bd40b1830 msgr2=0x7f6bd4152e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.975+0000 7f6bdc400640 1 -- 172.21.15.83:0/647658818 shutdown_connections 2024-04-25T00:14:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:36.975+0000 7f6bdc400640 1 -- 172.21.15.83:0/647658818 wait complete. 2024-04-25T00:14:36.997 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.127+0000 7fa6fce00640 1 Processor -- start 2024-04-25T00:14:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.127+0000 7fa6fce00640 1 -- start start 2024-04-25T00:14:37.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6fce00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa6f81599f0 0x7fa6f80cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6fce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa6f815cec0 0x7fa6f80ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6fce00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6f80ccc60 con 0x7fa6f815c340 2024-04-25T00:14:37.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6fce00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f8159dd0 con 0x7fa6f81599f0 2024-04-25T00:14:37.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6fce00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f806dba0 con 0x7fa6f815cec0 2024-04-25T00:14:37.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6f5e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa6f81599f0 0x7fa6f80cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6efe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa6f815c340 0x7fa6f815c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45474/0 (socket says 172.21.15.83:45474) 2024-04-25T00:14:37.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6f5e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa6f81599f0 0x7fa6f80cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44200/0 (socket says 172.21.15.83:44200) 2024-04-25T00:14:37.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6efe00640 1 -- 172.21.15.83:0/894197925 learned_addr learned my addr 172.21.15.83:0/894197925 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6f6800640 1 --2- 172.21.15.83:0/894197925 >> v2:172.21.15.86:3300/0 conn(0x7fa6f815cec0 0x7fa6f80ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6f5e00640 1 -- 172.21.15.83:0/894197925 >> v2:172.21.15.86:3300/0 conn(0x7fa6f815cec0 msgr2=0x7fa6f80ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6f5e00640 1 --2- 172.21.15.83:0/894197925 >> v2:172.21.15.86:3300/0 conn(0x7fa6f815cec0 0x7fa6f80ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6f5e00640 1 -- 172.21.15.83:0/894197925 >> v1:172.21.15.83:6789/0 conn(0x7fa6f815c340 legacy=0x7fa6f815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6f5e00640 1 -- 172.21.15.83:0/894197925 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f806de80 con 0x7fa6f81599f0 2024-04-25T00:14:37.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.131+0000 7fa6f5e00640 1 --2- 172.21.15.83:0/894197925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f81599f0 0x7fa6f80cc510 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80097c0 tx=0x7fa6e801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6825901753991eb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:37.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6f5400640 1 -- 172.21.15.83:0/894197925 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80b0070 con 0x7fa6f81599f0 2024-04-25T00:14:37.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6f5400640 1 -- 172.21.15.83:0/894197925 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e80ab600 con 0x7fa6f81599f0 2024-04-25T00:14:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6f5400640 1 -- 172.21.15.83:0/894197925 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80a4e60 con 0x7fa6f81599f0 2024-04-25T00:14:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 -- 172.21.15.83:0/894197925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f81599f0 msgr2=0x7fa6f80cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 --2- 172.21.15.83:0/894197925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f81599f0 0x7fa6f80cc510 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80097c0 tx=0x7fa6e801c3d0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 -- 172.21.15.83:0/894197925 shutdown_connections 2024-04-25T00:14:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 --2- 172.21.15.83:0/894197925 >> v2:172.21.15.86:3300/0 conn(0x7fa6f815cec0 0x7fa6f80ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 --2- 172.21.15.83:0/894197925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f81599f0 0x7fa6f80cc510 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 -- 172.21.15.83:0/894197925 >> 172.21.15.83:0/894197925 conn(0x7fa6f80b1830 msgr2=0x7fa6f80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 -- 172.21.15.83:0/894197925 shutdown_connections 2024-04-25T00:14:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 -- 172.21.15.83:0/894197925 wait complete. 2024-04-25T00:14:37.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 Processor -- start 2024-04-25T00:14:37.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 -- start start 2024-04-25T00:14:37.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6f81599f0 0x7fa6f8153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f815c340 0x7fa6f8154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6f815cec0 0x7fa6f8155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f80cf800 con 0x7fa6f815c340 2024-04-25T00:14:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f80cf680 con 0x7fa6f81599f0 2024-04-25T00:14:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6fce00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f8168cd0 con 0x7fa6f815cec0 2024-04-25T00:14:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f815c340 0x7fa6f8154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f815c340 0x7fa6f8154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44216/0 (socket says 172.21.15.83:44216) 2024-04-25T00:14:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6efe00640 1 -- 172.21.15.83:0/1437046344 learned_addr learned my addr 172.21.15.83:0/1437046344 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6f5e00640 1 --2- 172.21.15.83:0/1437046344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6f81599f0 0x7fa6f8153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6f6800640 1 --2- 172.21.15.83:0/1437046344 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6f815cec0 0x7fa6f8155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6efe00640 1 -- 172.21.15.83:0/1437046344 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6f815cec0 msgr2=0x7fa6f8155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6efe00640 1 --2- 172.21.15.83:0/1437046344 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6f815cec0 0x7fa6f8155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6efe00640 1 -- 172.21.15.83:0/1437046344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6f81599f0 msgr2=0x7fa6f8153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6efe00640 1 --2- 172.21.15.83:0/1437046344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6f81599f0 0x7fa6f8153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.135+0000 7fa6efe00640 1 -- 172.21.15.83:0/1437046344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f815ada0 con 0x7fa6f815c340 2024-04-25T00:14:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.139+0000 7fa6efe00640 1 --2- 172.21.15.83:0/1437046344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f815c340 0x7fa6f8154af0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000c450 tx=0x7fa6e000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.139+0000 7fa6ee000640 1 -- 172.21.15.83:0/1437046344 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0017070 con 0x7fa6f815c340 2024-04-25T00:14:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.139+0000 7fa6ee000640 1 -- 172.21.15.83:0/1437046344 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e0005cf0 con 0x7fa6f815c340 2024-04-25T00:14:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.139+0000 7fa6ee000640 1 -- 172.21.15.83:0/1437046344 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0007290 con 0x7fa6f815c340 2024-04-25T00:14:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.139+0000 7fa6fce00640 1 -- 172.21.15.83:0/1437046344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f815b080 con 0x7fa6f815c340 2024-04-25T00:14:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.139+0000 7fa6fce00640 1 -- 172.21.15.83:0/1437046344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f815b640 con 0x7fa6f815c340 2024-04-25T00:14:37.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.139+0000 7fa6fce00640 1 -- 172.21.15.83:0/1437046344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f80737f0 con 0x7fa6f815c340 2024-04-25T00:14:37.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.139+0000 7fa6ee000640 1 -- 172.21.15.83:0/1437046344 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa6e001a070 con 0x7fa6f815c340 2024-04-25T00:14:37.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.139+0000 7fa6ee000640 1 --2- 172.21.15.83:0/1437046344 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa6d40678f0 0x7fa6d4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.139+0000 7fa6ee000640 1 -- 172.21.15.83:0/1437046344 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fa6e0085120 con 0x7fa6f815c340 2024-04-25T00:14:37.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.139+0000 7fa6f5e00640 1 --2- 172.21.15.83:0/1437046344 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa6d40678f0 0x7fa6d4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.139+0000 7fa6f5e00640 1 --2- 172.21.15.83:0/1437046344 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa6d40678f0 0x7fa6d4069db0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80ad000 tx=0x7fa6e80a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:37.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.143+0000 7fa6ee000640 1 -- 172.21.15.83:0/1437046344 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa6f80ccc60 con 0x7fa6f815c340 2024-04-25T00:14:37.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.303+0000 7fa6fce00640 1 -- 172.21.15.83:0/1437046344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6f804f5f0 con 0x7fa6f815c340 2024-04-25T00:14:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.303+0000 7fa6ee000640 1 -- 172.21.15.83:0/1437046344 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16801 (secure 0 0 0) 0x7fa6e0055ac0 con 0x7fa6f815c340 2024-04-25T00:14:37.309 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:37.309 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:24.352251+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:21.082598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:21.713421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.307+0000 7fa6fce00640 1 -- 172.21.15.83:0/1437046344 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa6d40678f0 msgr2=0x7fa6d4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.307+0000 7fa6fce00640 1 --2- 172.21.15.83:0/1437046344 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa6d40678f0 0x7fa6d4069db0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80ad000 tx=0x7fa6e80a6000 comp rx=0 tx=0).stop 2024-04-25T00:14:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.307+0000 7fa6fce00640 1 -- 172.21.15.83:0/1437046344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f815c340 msgr2=0x7fa6f8154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.307+0000 7fa6fce00640 1 --2- 172.21.15.83:0/1437046344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f815c340 0x7fa6f8154af0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000c450 tx=0x7fa6e000c910 comp rx=0 tx=0).stop 2024-04-25T00:14:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.307+0000 7fa6fce00640 1 -- 172.21.15.83:0/1437046344 shutdown_connections 2024-04-25T00:14:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.307+0000 7fa6fce00640 1 --2- 172.21.15.83:0/1437046344 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa6d40678f0 0x7fa6d4069db0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.307+0000 7fa6fce00640 1 --2- 172.21.15.83:0/1437046344 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6f815cec0 0x7fa6f8155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.307+0000 7fa6fce00640 1 --2- 172.21.15.83:0/1437046344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f815c340 0x7fa6f8154af0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.307+0000 7fa6fce00640 1 --2- 172.21.15.83:0/1437046344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6f81599f0 0x7fa6f8153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.307+0000 7fa6fce00640 1 -- 172.21.15.83:0/1437046344 >> 172.21.15.83:0/1437046344 conn(0x7fa6f80b1830 msgr2=0x7fa6f80b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.307+0000 7fa6fce00640 1 -- 172.21.15.83:0/1437046344 shutdown_connections 2024-04-25T00:14:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.307+0000 7fa6fce00640 1 -- 172.21.15.83:0/1437046344 wait complete. 2024-04-25T00:14:37.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:14:21.082598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:21.713421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:14:37.331 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:14:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f0651600640 1 Processor -- start 2024-04-25T00:14:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f0651600640 1 -- start start 2024-04-25T00:14:37.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f0651600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f064c159810 0x7f064c0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f0651600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f064c06e950 0x7f064c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f0651600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064c15fe50 0x7f064c15c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f0651600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f064c06e210 con 0x7f064c159810 2024-04-25T00:14:37.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f0651600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f064c06da00 con 0x7f064c06e950 2024-04-25T00:14:37.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f0651600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f064c0ce000 con 0x7f064c15fe50 2024-04-25T00:14:37.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f064aa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f064c159810 0x7f064c0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f064aa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f064c159810 0x7f064c0cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44224/0 (socket says 172.21.15.83:44224) 2024-04-25T00:14:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f064aa00640 1 -- 172.21.15.83:0/2454318628 learned_addr learned my addr 172.21.15.83:0/2454318628 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f064b400640 1 --2- 172.21.15.83:0/2454318628 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064c15fe50 0x7f064c15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f0642000640 1 --2- 172.21.15.83:0/2454318628 >> v2:172.21.15.86:3300/0 conn(0x7f064c06e950 0x7f064c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f064aa00640 1 -- 172.21.15.83:0/2454318628 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064c15fe50 msgr2=0x7f064c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f064aa00640 1 --2- 172.21.15.83:0/2454318628 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064c15fe50 0x7f064c15c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f064aa00640 1 -- 172.21.15.83:0/2454318628 >> v2:172.21.15.86:3300/0 conn(0x7f064c06e950 msgr2=0x7f064c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f064aa00640 1 --2- 172.21.15.83:0/2454318628 >> v2:172.21.15.86:3300/0 conn(0x7f064c06e950 0x7f064c15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f064aa00640 1 -- 172.21.15.83:0/2454318628 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c06eec0 con 0x7f064c159810 2024-04-25T00:14:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.467+0000 7f0642000640 1 --2- 172.21.15.83:0/2454318628 >> v2:172.21.15.86:3300/0 conn(0x7f064c06e950 0x7f064c15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:37.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.471+0000 7f064aa00640 1 --2- 172.21.15.83:0/2454318628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c159810 0x7f064c0cc4a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f064c0709a0 tx=0x7f063c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640a8b3a504c9ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:37.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.471+0000 7f064a000640 1 -- 172.21.15.83:0/2454318628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c0b0070 con 0x7f064c159810 2024-04-25T00:14:37.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.471+0000 7f064a000640 1 -- 172.21.15.83:0/2454318628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f063c0ab600 con 0x7f064c159810 2024-04-25T00:14:37.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.471+0000 7f064a000640 1 -- 172.21.15.83:0/2454318628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c0ab9d0 con 0x7f064c159810 2024-04-25T00:14:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.471+0000 7f0651600640 1 -- 172.21.15.83:0/2454318628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c159810 msgr2=0x7f064c0cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.471+0000 7f0651600640 1 --2- 172.21.15.83:0/2454318628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c159810 0x7f064c0cc4a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f064c0709a0 tx=0x7f063c01c490 comp rx=0 tx=0).stop 2024-04-25T00:14:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.471+0000 7f0651600640 1 -- 172.21.15.83:0/2454318628 shutdown_connections 2024-04-25T00:14:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.471+0000 7f0651600640 1 --2- 172.21.15.83:0/2454318628 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064c15fe50 0x7f064c15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.471+0000 7f0651600640 1 --2- 172.21.15.83:0/2454318628 >> v2:172.21.15.86:3300/0 conn(0x7f064c06e950 0x7f064c15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.471+0000 7f0651600640 1 --2- 172.21.15.83:0/2454318628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c159810 0x7f064c0cc4a0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.471+0000 7f0651600640 1 -- 172.21.15.83:0/2454318628 >> 172.21.15.83:0/2454318628 conn(0x7f064c0b1830 msgr2=0x7f064c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.471+0000 7f0651600640 1 -- 172.21.15.83:0/2454318628 shutdown_connections 2024-04-25T00:14:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.471+0000 7f0651600640 1 -- 172.21.15.83:0/2454318628 wait complete. 2024-04-25T00:14:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f0651600640 1 Processor -- start 2024-04-25T00:14:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f0651600640 1 -- start start 2024-04-25T00:14:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f0651600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c06e950 0x7f064c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f0651600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f064c159810 0x7f064c151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f0651600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064c15fe50 0x7f064c150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f0651600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f064c159d70 con 0x7f064c06e950 2024-04-25T00:14:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f0651600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f064c159bf0 con 0x7f064c159810 2024-04-25T00:14:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f0651600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f064c070da0 con 0x7f064c15fe50 2024-04-25T00:14:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f064aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c06e950 0x7f064c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f064aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c06e950 0x7f064c0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44230/0 (socket says 172.21.15.83:44230) 2024-04-25T00:14:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f064aa00640 1 -- 172.21.15.83:0/1265999051 learned_addr learned my addr 172.21.15.83:0/1265999051 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f0642000640 1 --2- 172.21.15.83:0/1265999051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f064c159810 0x7f064c151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f064b400640 1 --2- 172.21.15.83:0/1265999051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064c15fe50 0x7f064c150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f064aa00640 1 -- 172.21.15.83:0/1265999051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064c15fe50 msgr2=0x7f064c150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f064aa00640 1 --2- 172.21.15.83:0/1265999051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064c15fe50 0x7f064c150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f064aa00640 1 -- 172.21.15.83:0/1265999051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f064c159810 msgr2=0x7f064c151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f064aa00640 1 --2- 172.21.15.83:0/1265999051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f064c159810 0x7f064c151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f064aa00640 1 -- 172.21.15.83:0/1265999051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c151050 con 0x7f064c06e950 2024-04-25T00:14:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f064aa00640 1 --2- 172.21.15.83:0/1265999051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c06e950 0x7f064c0b0bc0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f063c01cd30 tx=0x7f063c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f0643400640 1 -- 172.21.15.83:0/1265999051 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c0bb070 con 0x7f064c06e950 2024-04-25T00:14:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f0643400640 1 -- 172.21.15.83:0/1265999051 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f063c0b6640 con 0x7f064c06e950 2024-04-25T00:14:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f0651600640 1 -- 172.21.15.83:0/1265999051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f064c150050 con 0x7f064c06e950 2024-04-25T00:14:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f0643400640 1 -- 172.21.15.83:0/1265999051 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c0b0040 con 0x7f064c06e950 2024-04-25T00:14:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.475+0000 7f0651600640 1 -- 172.21.15.83:0/1265999051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f064c0b1530 con 0x7f064c06e950 2024-04-25T00:14:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.479+0000 7f0651600640 1 -- 172.21.15.83:0/1265999051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f064c04f510 con 0x7f064c06e950 2024-04-25T00:14:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.479+0000 7f0643400640 1 -- 172.21.15.83:0/1265999051 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f063c0ab480 con 0x7f064c06e950 2024-04-25T00:14:37.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.479+0000 7f0643400640 1 --2- 172.21.15.83:0/1265999051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f06280678f0 0x7f0628069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.479+0000 7f0643400640 1 -- 172.21.15.83:0/1265999051 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f063c11f7d0 con 0x7f064c06e950 2024-04-25T00:14:37.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.479+0000 7f0642000640 1 --2- 172.21.15.83:0/1265999051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f06280678f0 0x7f0628069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.479+0000 7f0642000640 1 --2- 172.21.15.83:0/1265999051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f06280678f0 0x7f0628069db0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f064c0b0b50 tx=0x7f0634008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.483+0000 7f0643400640 1 -- 172.21.15.83:0/1265999051 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f063c016110 con 0x7f064c06e950 2024-04-25T00:14:37.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.643+0000 7f0651600640 1 -- 172.21.15.83:0/1265999051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f064c15b9a0 con 0x7f064c06e950 2024-04-25T00:14:37.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.643+0000 7f0643400640 1 -- 172.21.15.83:0/1265999051 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f063c0efbc0 con 0x7f064c06e950 2024-04-25T00:14:37.646 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:14:37.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.643+0000 7f0651600640 1 -- 172.21.15.83:0/1265999051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f06280678f0 msgr2=0x7f0628069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.643+0000 7f0651600640 1 --2- 172.21.15.83:0/1265999051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f06280678f0 0x7f0628069db0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f064c0b0b50 tx=0x7f0634008040 comp rx=0 tx=0).stop 2024-04-25T00:14:37.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.643+0000 7f0651600640 1 -- 172.21.15.83:0/1265999051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c06e950 msgr2=0x7f064c0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.643+0000 7f0651600640 1 --2- 172.21.15.83:0/1265999051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c06e950 0x7f064c0b0bc0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f063c01cd30 tx=0x7f063c019230 comp rx=0 tx=0).stop 2024-04-25T00:14:37.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.643+0000 7f0651600640 1 -- 172.21.15.83:0/1265999051 shutdown_connections 2024-04-25T00:14:37.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.643+0000 7f0651600640 1 --2- 172.21.15.83:0/1265999051 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f06280678f0 0x7f0628069db0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.643+0000 7f0651600640 1 --2- 172.21.15.83:0/1265999051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064c15fe50 0x7f064c150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.643+0000 7f0651600640 1 --2- 172.21.15.83:0/1265999051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f064c159810 0x7f064c151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.643+0000 7f0651600640 1 --2- 172.21.15.83:0/1265999051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c06e950 0x7f064c0b0bc0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.643+0000 7f0651600640 1 -- 172.21.15.83:0/1265999051 >> 172.21.15.83:0/1265999051 conn(0x7f064c0b1830 msgr2=0x7f064c15db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:37.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.643+0000 7f0651600640 1 -- 172.21.15.83:0/1265999051 shutdown_connections 2024-04-25T00:14:37.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.643+0000 7f0651600640 1 -- 172.21.15.83:0/1265999051 wait complete. 2024-04-25T00:14:37.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:14:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.799+0000 7fec38600640 1 Processor -- start 2024-04-25T00:14:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.799+0000 7fec38600640 1 -- start start 2024-04-25T00:14:37.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec38600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fec3006e950 0x7fec3015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec38600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3015fe50 0x7fec300ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec38600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec300ce000 con 0x7fec30159810 2024-04-25T00:14:37.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec38600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fec3006e210 con 0x7fec3006e950 2024-04-25T00:14:37.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec38600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fec3006da00 con 0x7fec3015fe50 2024-04-25T00:14:37.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec35c00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fec30159810 0x7fec300cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45488/0 (socket says 172.21.15.83:45488) 2024-04-25T00:14:37.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec35c00640 1 -- 172.21.15.83:0/465396601 learned_addr learned my addr 172.21.15.83:0/465396601 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:37.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 -- 172.21.15.83:0/465396601 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 413298432 0 0) 0x7fec300ce000 con 0x7fec30159810 2024-04-25T00:14:37.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 -- 172.21.15.83:0/465396601 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec18003530 con 0x7fec30159810 2024-04-25T00:14:37.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec36600640 1 --2- 172.21.15.83:0/465396601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3015fe50 0x7fec300ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec35200640 1 --2- 172.21.15.83:0/465396601 >> v2:172.21.15.86:3300/0 conn(0x7fec3006e950 0x7fec3015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 -- 172.21.15.83:0/465396601 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3762009136 0 0) 0x7fec18003530 con 0x7fec30159810 2024-04-25T00:14:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 -- 172.21.15.83:0/465396601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3015fe50 msgr2=0x7fec300ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 --2- 172.21.15.83:0/465396601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3015fe50 0x7fec300ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 -- 172.21.15.83:0/465396601 >> v2:172.21.15.86:3300/0 conn(0x7fec3006e950 msgr2=0x7fec3015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 --2- 172.21.15.83:0/465396601 >> v2:172.21.15.86:3300/0 conn(0x7fec3006e950 0x7fec3015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 -- 172.21.15.83:0/465396601 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3006eec0 con 0x7fec30159810 2024-04-25T00:14:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 -- 172.21.15.83:0/465396601 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fec24001680 con 0x7fec30159810 2024-04-25T00:14:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec35200640 1 --2- 172.21.15.83:0/465396601 >> v2:172.21.15.86:3300/0 conn(0x7fec3006e950 0x7fec3015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 -- 172.21.15.83:0/465396601 >> v1:172.21.15.83:6789/0 conn(0x7fec30159810 legacy=0x7fec300cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 --2- 172.21.15.83:0/465396601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec18003a30 0x7fec18003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 --2- 172.21.15.83:0/465396601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3015fe50 0x7fec18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 --2- 172.21.15.83:0/465396601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec18004d50 0x7fec180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 -- 172.21.15.83:0/465396601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec3006e210 con 0x7fec18004d50 2024-04-25T00:14:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 -- 172.21.15.83:0/465396601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec3006da00 con 0x7fec18003a30 2024-04-25T00:14:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec2fe00640 1 -- 172.21.15.83:0/465396601 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fec1800bb00 con 0x7fec3015fe50 2024-04-25T00:14:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec36600640 1 --2- 172.21.15.83:0/465396601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec18004d50 0x7fec180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec35c00640 1 --2- 172.21.15.83:0/465396601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec18003a30 0x7fec18003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec35200640 1 --2- 172.21.15.83:0/465396601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3015fe50 0x7fec18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec36600640 1 -- 172.21.15.83:0/465396601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3015fe50 msgr2=0x7fec18004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec36600640 1 --2- 172.21.15.83:0/465396601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3015fe50 0x7fec18004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec36600640 1 -- 172.21.15.83:0/465396601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec18003a30 msgr2=0x7fec18003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec36600640 1 --2- 172.21.15.83:0/465396601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec18003a30 0x7fec18003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.803+0000 7fec36600640 1 -- 172.21.15.83:0/465396601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec18003530 con 0x7fec18004d50 2024-04-25T00:14:37.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.807+0000 7fec36600640 1 --2- 172.21.15.83:0/465396601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec18004d50 0x7fec180051d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fec28009fa0 tx=0x7fec2801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb1481379239dda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.807+0000 7fec2fe00640 1 -- 172.21.15.83:0/465396601 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec280b0070 con 0x7fec18004d50 2024-04-25T00:14:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.807+0000 7fec2fe00640 1 -- 172.21.15.83:0/465396601 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec280ab600 con 0x7fec18004d50 2024-04-25T00:14:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.807+0000 7fec2fe00640 1 -- 172.21.15.83:0/465396601 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28009bf0 con 0x7fec18004d50 2024-04-25T00:14:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.807+0000 7fec38600640 1 -- 172.21.15.83:0/465396601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec18004d50 msgr2=0x7fec180051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.807+0000 7fec38600640 1 --2- 172.21.15.83:0/465396601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec18004d50 0x7fec180051d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fec28009fa0 tx=0x7fec2801c920 comp rx=0 tx=0).stop 2024-04-25T00:14:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.807+0000 7fec36600640 1 -- 172.21.15.83:0/465396601 reap_dead start 2024-04-25T00:14:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.807+0000 7fec38600640 1 -- 172.21.15.83:0/465396601 shutdown_connections 2024-04-25T00:14:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.807+0000 7fec38600640 1 -- 172.21.15.83:0/465396601 >> 172.21.15.83:0/465396601 conn(0x7fec300b1830 msgr2=0x7fec300b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.807+0000 7fec38600640 1 -- 172.21.15.83:0/465396601 shutdown_connections 2024-04-25T00:14:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.807+0000 7fec38600640 1 -- 172.21.15.83:0/465396601 wait complete. 2024-04-25T00:14:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec38600640 1 Processor -- start 2024-04-25T00:14:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec38600640 1 -- start start 2024-04-25T00:14:37.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec38600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec18004d50 0x7fec30155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec38600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3014a7a0 0x7fec300c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec38600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3014b070 0x7fec300b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec38600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec30070f20 con 0x7fec18004d50 2024-04-25T00:14:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec38600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec30070da0 con 0x7fec3014a7a0 2024-04-25T00:14:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec38600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fec300cecc0 con 0x7fec3014b070 2024-04-25T00:14:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec35c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec18004d50 0x7fec30155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec35c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec18004d50 0x7fec30155170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44246/0 (socket says 172.21.15.83:44246) 2024-04-25T00:14:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec35c00640 1 -- 172.21.15.83:0/672159149 learned_addr learned my addr 172.21.15.83:0/672159149 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec36600640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3014b070 0x7fec300b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec35200640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3014a7a0 0x7fec300c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec35c00640 1 -- 172.21.15.83:0/672159149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3014b070 msgr2=0x7fec300b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec35c00640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3014b070 0x7fec300b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec35c00640 1 -- 172.21.15.83:0/672159149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3014a7a0 msgr2=0x7fec300c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec35c00640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3014a7a0 0x7fec300c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec35c00640 1 -- 172.21.15.83:0/672159149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec300c4fe0 con 0x7fec18004d50 2024-04-25T00:14:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec35200640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3014a7a0 0x7fec300c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec36600640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3014b070 0x7fec300b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec35c00640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec18004d50 0x7fec30155170 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fec24012310 tx=0x7fec240017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec2d600640 1 -- 172.21.15.83:0/672159149 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2401c070 con 0x7fec18004d50 2024-04-25T00:14:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec2d600640 1 -- 172.21.15.83:0/672159149 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec24014750 con 0x7fec18004d50 2024-04-25T00:14:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec2d600640 1 -- 172.21.15.83:0/672159149 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24014b20 con 0x7fec18004d50 2024-04-25T00:14:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec38600640 1 -- 172.21.15.83:0/672159149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec300aef60 con 0x7fec18004d50 2024-04-25T00:14:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec38600640 1 -- 172.21.15.83:0/672159149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3014b4f0 con 0x7fec18004d50 2024-04-25T00:14:37.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.811+0000 7fec38600640 1 -- 172.21.15.83:0/672159149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3004f510 con 0x7fec18004d50 2024-04-25T00:14:37.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.815+0000 7fec2d600640 1 -- 172.21.15.83:0/672159149 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fec24014cc0 con 0x7fec18004d50 2024-04-25T00:14:37.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.815+0000 7fec2d600640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec100678c0 0x7fec10069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.815+0000 7fec2d600640 1 -- 172.21.15.83:0/672159149 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fec240886f0 con 0x7fec18004d50 2024-04-25T00:14:37.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.815+0000 7fec35200640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec100678c0 0x7fec10069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.815+0000 7fec35200640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec100678c0 0x7fec10069d80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fec300cd350 tx=0x7fec1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:37.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.819+0000 7fec2d600640 1 -- 172.21.15.83:0/672159149 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fec300c4fe0 con 0x7fec18004d50 2024-04-25T00:14:37.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.971+0000 7fec38600640 1 -- 172.21.15.83:0/672159149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fec300ccdc0 con 0x7fec18004d50 2024-04-25T00:14:37.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.971+0000 7fec2d600640 1 -- 172.21.15.83:0/672159149 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7fec24059090 con 0x7fec18004d50 2024-04-25T00:14:37.977 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:14:37.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.975+0000 7fec38600640 1 -- 172.21.15.83:0/672159149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec100678c0 msgr2=0x7fec10069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.975+0000 7fec38600640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec100678c0 0x7fec10069d80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fec300cd350 tx=0x7fec1c008040 comp rx=0 tx=0).stop 2024-04-25T00:14:37.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.975+0000 7fec38600640 1 -- 172.21.15.83:0/672159149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec18004d50 msgr2=0x7fec30155170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.975+0000 7fec38600640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec18004d50 0x7fec30155170 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fec24012310 tx=0x7fec240017b0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.975+0000 7fec38600640 1 -- 172.21.15.83:0/672159149 shutdown_connections 2024-04-25T00:14:37.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.975+0000 7fec38600640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec100678c0 0x7fec10069d80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.975+0000 7fec38600640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec3014b070 0x7fec300b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.975+0000 7fec38600640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec3014a7a0 0x7fec300c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.975+0000 7fec38600640 1 --2- 172.21.15.83:0/672159149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec18004d50 0x7fec30155170 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.975+0000 7fec38600640 1 -- 172.21.15.83:0/672159149 >> 172.21.15.83:0/672159149 conn(0x7fec300b1830 msgr2=0x7fec30158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:37.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.975+0000 7fec38600640 1 -- 172.21.15.83:0/672159149 shutdown_connections 2024-04-25T00:14:37.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:37.975+0000 7fec38600640 1 -- 172.21.15.83:0/672159149 wait complete. 2024-04-25T00:14:37.997 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:14:38.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffb05e00640 1 Processor -- start 2024-04-25T00:14:38.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffb05e00640 1 -- start start 2024-04-25T00:14:38.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffb05e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb00159810 0x7ffb000cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffb05e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0006e950 0x7ffb0015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffb05e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb0015fe50 0x7ffb0015c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffb05e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ffb0006e210 con 0x7ffb00159810 2024-04-25T00:14:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffb05e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffb0006da00 con 0x7ffb0015fe50 2024-04-25T00:14:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffb05e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ffb000ce000 con 0x7ffb0006e950 2024-04-25T00:14:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffafea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb00159810 0x7ffb000cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffafea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb00159810 0x7ffb000cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44248/0 (socket says 172.21.15.83:44248) 2024-04-25T00:14:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffafea00640 1 -- 172.21.15.83:0/1398285497 learned_addr learned my addr 172.21.15.83:0/1398285497 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffaf6000640 1 --2- 172.21.15.83:0/1398285497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0006e950 0x7ffb0015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffaff400640 1 --2- 172.21.15.83:0/1398285497 >> v2:172.21.15.86:3300/0 conn(0x7ffb0015fe50 0x7ffb0015c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffafea00640 1 -- 172.21.15.83:0/1398285497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0006e950 msgr2=0x7ffb0015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffafea00640 1 --2- 172.21.15.83:0/1398285497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0006e950 0x7ffb0015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffafea00640 1 -- 172.21.15.83:0/1398285497 >> v2:172.21.15.86:3300/0 conn(0x7ffb0015fe50 msgr2=0x7ffb0015c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffafea00640 1 --2- 172.21.15.83:0/1398285497 >> v2:172.21.15.86:3300/0 conn(0x7ffb0015fe50 0x7ffb0015c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.131+0000 7ffafea00640 1 -- 172.21.15.83:0/1398285497 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0006eec0 con 0x7ffb00159810 2024-04-25T00:14:38.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.135+0000 7ffafea00640 1 --2- 172.21.15.83:0/1398285497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb00159810 0x7ffb000cc4a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ffb000cdb40 tx=0x7ffaf001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c25586bdf0d0dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:38.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.135+0000 7ffafe000640 1 -- 172.21.15.83:0/1398285497 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00b0070 con 0x7ffb00159810 2024-04-25T00:14:38.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.135+0000 7ffafe000640 1 -- 172.21.15.83:0/1398285497 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf00ab600 con 0x7ffb00159810 2024-04-25T00:14:38.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.135+0000 7ffafe000640 1 -- 172.21.15.83:0/1398285497 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00ab9d0 con 0x7ffb00159810 2024-04-25T00:14:38.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.135+0000 7ffb05e00640 1 -- 172.21.15.83:0/1398285497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb00159810 msgr2=0x7ffb000cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.135+0000 7ffb05e00640 1 --2- 172.21.15.83:0/1398285497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb00159810 0x7ffb000cc4a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ffb000cdb40 tx=0x7ffaf001c490 comp rx=0 tx=0).stop 2024-04-25T00:14:38.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.135+0000 7ffb05e00640 1 -- 172.21.15.83:0/1398285497 shutdown_connections 2024-04-25T00:14:38.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.135+0000 7ffb05e00640 1 --2- 172.21.15.83:0/1398285497 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0006e950 0x7ffb0015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.135+0000 7ffb05e00640 1 --2- 172.21.15.83:0/1398285497 >> v2:172.21.15.86:3300/0 conn(0x7ffb0015fe50 0x7ffb0015c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.135+0000 7ffb05e00640 1 --2- 172.21.15.83:0/1398285497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb00159810 0x7ffb000cc4a0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.135+0000 7ffb05e00640 1 -- 172.21.15.83:0/1398285497 >> 172.21.15.83:0/1398285497 conn(0x7ffb000b1830 msgr2=0x7ffb000b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:38.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.135+0000 7ffb05e00640 1 -- 172.21.15.83:0/1398285497 shutdown_connections 2024-04-25T00:14:38.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.135+0000 7ffb05e00640 1 -- 172.21.15.83:0/1398285497 wait complete. 2024-04-25T00:14:38.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffb05e00640 1 Processor -- start 2024-04-25T00:14:38.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffb05e00640 1 -- start start 2024-04-25T00:14:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffb05e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb0006e950 0x7ffb001f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffb05e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb00159810 0x7ffb00150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffb05e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0015fe50 0x7ffb00071100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffb05e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffb00159d70 con 0x7ffb0006e950 2024-04-25T00:14:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffb05e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb00159bf0 con 0x7ffb00159810 2024-04-25T00:14:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffb05e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ffb00070da0 con 0x7ffb0015fe50 2024-04-25T00:14:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffafea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb0006e950 0x7ffb001f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffafea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb0006e950 0x7ffb001f6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44252/0 (socket says 172.21.15.83:44252) 2024-04-25T00:14:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffafea00640 1 -- 172.21.15.83:0/2559993602 learned_addr learned my addr 172.21.15.83:0/2559993602 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffaf6000640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb00159810 0x7ffb00150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffafea00640 1 -- 172.21.15.83:0/2559993602 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0015fe50 msgr2=0x7ffb00071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffaff400640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0015fe50 0x7ffb00071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffafea00640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0015fe50 0x7ffb00071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffafea00640 1 -- 172.21.15.83:0/2559993602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb00159810 msgr2=0x7ffb00150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffafea00640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb00159810 0x7ffb00150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffafea00640 1 -- 172.21.15.83:0/2559993602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb001513d0 con 0x7ffb0006e950 2024-04-25T00:14:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffaff400640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0015fe50 0x7ffb00071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffafea00640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb0006e950 0x7ffb001f6e70 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ffaf001c970 tx=0x7ffaf0019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffaf7e00640 1 -- 172.21.15.83:0/2559993602 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00bb070 con 0x7ffb0006e950 2024-04-25T00:14:38.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffaf7e00640 1 -- 172.21.15.83:0/2559993602 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf00b6640 con 0x7ffb0006e950 2024-04-25T00:14:38.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffaf7e00640 1 -- 172.21.15.83:0/2559993602 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00b0040 con 0x7ffb0006e950 2024-04-25T00:14:38.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffb05e00640 1 -- 172.21.15.83:0/2559993602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb001516b0 con 0x7ffb0006e950 2024-04-25T00:14:38.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.139+0000 7ffb05e00640 1 -- 172.21.15.83:0/2559993602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb00151c70 con 0x7ffb0006e950 2024-04-25T00:14:38.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.143+0000 7ffb05e00640 1 -- 172.21.15.83:0/2559993602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb0004f510 con 0x7ffb0006e950 2024-04-25T00:14:38.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.143+0000 7ffaf7e00640 1 -- 172.21.15.83:0/2559993602 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffaf00ab480 con 0x7ffb0006e950 2024-04-25T00:14:38.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.143+0000 7ffaf7e00640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffadc0678c0 0x7ffadc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.143+0000 7ffaf7e00640 1 -- 172.21.15.83:0/2559993602 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7ffaf011f920 con 0x7ffb0006e950 2024-04-25T00:14:38.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.143+0000 7ffaf6000640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffadc0678c0 0x7ffadc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.143+0000 7ffaf6000640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffadc0678c0 0x7ffadc069d80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ffb000b4de0 tx=0x7ffae8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:38.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.147+0000 7ffaf7e00640 1 -- 172.21.15.83:0/2559993602 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffaf0016110 con 0x7ffb0006e950 2024-04-25T00:14:38.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.303+0000 7ffb05e00640 1 -- 172.21.15.83:0/2559993602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffb0015fc30 con 0x7ffb0006e950 2024-04-25T00:14:38.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.303+0000 7ffaf7e00640 1 -- 172.21.15.83:0/2559993602 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7ffaf00f02c0 con 0x7ffb0006e950 2024-04-25T00:14:38.308 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:14:38.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.303+0000 7ffb05e00640 1 -- 172.21.15.83:0/2559993602 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffadc0678c0 msgr2=0x7ffadc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.303+0000 7ffb05e00640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffadc0678c0 0x7ffadc069d80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ffb000b4de0 tx=0x7ffae8008040 comp rx=0 tx=0).stop 2024-04-25T00:14:38.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.303+0000 7ffb05e00640 1 -- 172.21.15.83:0/2559993602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb0006e950 msgr2=0x7ffb001f6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.303+0000 7ffb05e00640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb0006e950 0x7ffb001f6e70 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ffaf001c970 tx=0x7ffaf0019230 comp rx=0 tx=0).stop 2024-04-25T00:14:38.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.303+0000 7ffb05e00640 1 -- 172.21.15.83:0/2559993602 shutdown_connections 2024-04-25T00:14:38.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.303+0000 7ffb05e00640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffadc0678c0 0x7ffadc069d80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.303+0000 7ffb05e00640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0015fe50 0x7ffb00071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.303+0000 7ffb05e00640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb00159810 0x7ffb00150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.303+0000 7ffb05e00640 1 --2- 172.21.15.83:0/2559993602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb0006e950 0x7ffb001f6e70 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.303+0000 7ffb05e00640 1 -- 172.21.15.83:0/2559993602 >> 172.21.15.83:0/2559993602 conn(0x7ffb000b1830 msgr2=0x7ffb000d04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:38.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.303+0000 7ffb05e00640 1 -- 172.21.15.83:0/2559993602 shutdown_connections 2024-04-25T00:14:38.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.303+0000 7ffb05e00640 1 -- 172.21.15.83:0/2559993602 wait complete. 2024-04-25T00:14:38.327 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:38.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9487200640 1 Processor -- start 2024-04-25T00:14:38.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9487200640 1 -- start start 2024-04-25T00:14:38.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9487200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f948806b1e0 0x7f94880b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9487200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f94880ce090 0x7f948806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9487200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94880b0520 con 0x7f948815f6e0 2024-04-25T00:14:38.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9487200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f948806bb00 con 0x7f94880ce090 2024-04-25T00:14:38.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9487200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f948815e6f0 con 0x7f948806b1e0 2024-04-25T00:14:38.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9485400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f94880ce090 0x7f948806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9486800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f948815f6e0 0x7f9488071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45498/0 (socket says 172.21.15.83:45498) 2024-04-25T00:14:38.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9485400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f94880ce090 0x7f948806b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44268/0 (socket says 172.21.15.83:44268) 2024-04-25T00:14:38.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9486800640 1 -- 172.21.15.83:0/4048281875 learned_addr learned my addr 172.21.15.83:0/4048281875 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:38.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9485400640 1 -- 172.21.15.83:0/4048281875 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f948806b1e0 msgr2=0x7f94880b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:38.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9485e00640 1 --2- 172.21.15.83:0/4048281875 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f948806b1e0 0x7f94880b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9485400640 1 --2- 172.21.15.83:0/4048281875 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f948806b1e0 0x7f94880b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9485400640 1 -- 172.21.15.83:0/4048281875 >> v1:172.21.15.83:6789/0 conn(0x7f948815f6e0 legacy=0x7f9488071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9485400640 1 -- 172.21.15.83:0/4048281875 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948815e9d0 con 0x7f94880ce090 2024-04-25T00:14:38.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.463+0000 7f9485e00640 1 --2- 172.21.15.83:0/4048281875 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f948806b1e0 0x7f94880b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:38.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.467+0000 7f9485400640 1 --2- 172.21.15.83:0/4048281875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94880ce090 0x7f948806b5c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f94740097c0 tx=0x7f947401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267b00b110e970ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:38.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.467+0000 7f9484a00640 1 -- 172.21.15.83:0/4048281875 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94740b0070 con 0x7f94880ce090 2024-04-25T00:14:38.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.467+0000 7f9484a00640 1 -- 172.21.15.83:0/4048281875 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94740ab600 con 0x7f94880ce090 2024-04-25T00:14:38.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.467+0000 7f9484a00640 1 -- 172.21.15.83:0/4048281875 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94740ab9d0 con 0x7f94880ce090 2024-04-25T00:14:38.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.467+0000 7f9487200640 1 -- 172.21.15.83:0/4048281875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94880ce090 msgr2=0x7f948806b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.467+0000 7f9487200640 1 --2- 172.21.15.83:0/4048281875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94880ce090 0x7f948806b5c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f94740097c0 tx=0x7f947401c410 comp rx=0 tx=0).stop 2024-04-25T00:14:38.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.467+0000 7f9487200640 1 -- 172.21.15.83:0/4048281875 shutdown_connections 2024-04-25T00:14:38.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.467+0000 7f9487200640 1 --2- 172.21.15.83:0/4048281875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94880ce090 0x7f948806b5c0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.467+0000 7f9487200640 1 --2- 172.21.15.83:0/4048281875 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f948806b1e0 0x7f94880b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.467+0000 7f9487200640 1 -- 172.21.15.83:0/4048281875 >> 172.21.15.83:0/4048281875 conn(0x7f94880cd500 msgr2=0x7f94880cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:38.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.467+0000 7f9487200640 1 -- 172.21.15.83:0/4048281875 shutdown_connections 2024-04-25T00:14:38.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.467+0000 7f9487200640 1 -- 172.21.15.83:0/4048281875 wait complete. 2024-04-25T00:14:38.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.467+0000 7f9487200640 1 Processor -- start 2024-04-25T00:14:38.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.467+0000 7f9487200640 1 -- start start 2024-04-25T00:14:38.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9487200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f948806b1e0 0x7f9488154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9487200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94880ce090 0x7f9488152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9487200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f948815f6e0 0x7f94880b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9487200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f948815e4a0 con 0x7f948815f6e0 2024-04-25T00:14:38.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9487200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f948815e320 con 0x7f94880ce090 2024-04-25T00:14:38.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9487200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f948815fae0 con 0x7f948806b1e0 2024-04-25T00:14:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9486800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f948815f6e0 0x7f94880b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9486800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f948815f6e0 0x7f94880b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44272/0 (socket says 172.21.15.83:44272) 2024-04-25T00:14:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9486800640 1 -- 172.21.15.83:0/491499441 learned_addr learned my addr 172.21.15.83:0/491499441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9485400640 1 --2- 172.21.15.83:0/491499441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94880ce090 0x7f9488152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9485e00640 1 --2- 172.21.15.83:0/491499441 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f948806b1e0 0x7f9488154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9486800640 1 -- 172.21.15.83:0/491499441 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f948806b1e0 msgr2=0x7f9488154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9486800640 1 --2- 172.21.15.83:0/491499441 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f948806b1e0 0x7f9488154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9486800640 1 -- 172.21.15.83:0/491499441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94880ce090 msgr2=0x7f9488152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9486800640 1 --2- 172.21.15.83:0/491499441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94880ce090 0x7f9488152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9486800640 1 -- 172.21.15.83:0/491499441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9488153350 con 0x7f948815f6e0 2024-04-25T00:14:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9486800640 1 --2- 172.21.15.83:0/491499441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f948815f6e0 0x7f94880b1ae0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f948000ca80 tx=0x7f948000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:38.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f947e000640 1 -- 172.21.15.83:0/491499441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480017070 con 0x7f948815f6e0 2024-04-25T00:14:38.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f947e000640 1 -- 172.21.15.83:0/491499441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f948000ac30 con 0x7f948815f6e0 2024-04-25T00:14:38.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f947e000640 1 -- 172.21.15.83:0/491499441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480012600 con 0x7f948815f6e0 2024-04-25T00:14:38.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9487200640 1 -- 172.21.15.83:0/491499441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9488153630 con 0x7f948815f6e0 2024-04-25T00:14:38.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9487200640 1 -- 172.21.15.83:0/491499441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9488153bf0 con 0x7f948815f6e0 2024-04-25T00:14:38.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f9487200640 1 -- 172.21.15.83:0/491499441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f948804f510 con 0x7f948815f6e0 2024-04-25T00:14:38.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.471+0000 7f947e000640 1 -- 172.21.15.83:0/491499441 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f948001a070 con 0x7f948815f6e0 2024-04-25T00:14:38.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.475+0000 7f947e000640 1 --2- 172.21.15.83:0/491499441 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f94680678f0 0x7f9468069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.475+0000 7f947e000640 1 -- 172.21.15.83:0/491499441 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f9480085a00 con 0x7f948815f6e0 2024-04-25T00:14:38.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.475+0000 7f9485e00640 1 --2- 172.21.15.83:0/491499441 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f94680678f0 0x7f9468069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.475+0000 7f9485e00640 1 --2- 172.21.15.83:0/491499441 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f94680678f0 0x7f9468069db0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9478006fd0 tx=0x7f9478008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:38.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.479+0000 7f947e000640 1 -- 172.21.15.83:0/491499441 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f948000ae20 con 0x7f948815f6e0 2024-04-25T00:14:38.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.663+0000 7f9487200640 1 -- 172.21.15.83:0/491499441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9488072d50 con 0x7f948815f6e0 2024-04-25T00:14:38.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.663+0000 7f947e000640 1 -- 172.21.15.83:0/491499441 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5174 (secure 0 0 0) 0x7f9480055d50 con 0x7f948815f6e0 2024-04-25T00:14:38.669 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:38.669 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/3856734881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3856734881},{"type":"v1","addr":"172.21.15.86:6834","nonce":3856734881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/12764228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":12764228},{"type":"v1","addr":"172.21.15.86:6835","nonce":12764228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1680607346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1680607346},{"type":"v1","addr":"172.21.15.83:6837","nonce":1680607346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:22.706751+0000","modified":"2024-04-25T00:14:25.734428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7023},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7023":{"gid":7023,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/121840722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":121840722},{"type":"v1","addr":"172.21.15.83:6836","nonce":121840722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7023,"qdb_cluster":[7023]},"id":6}]} 2024-04-25T00:14:38.669 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 53 2024-04-25T00:14:38.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.667+0000 7f9487200640 1 -- 172.21.15.83:0/491499441 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f94680678f0 msgr2=0x7f9468069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.667+0000 7f9487200640 1 --2- 172.21.15.83:0/491499441 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f94680678f0 0x7f9468069db0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9478006fd0 tx=0x7f9478008040 comp rx=0 tx=0).stop 2024-04-25T00:14:38.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.667+0000 7f9487200640 1 -- 172.21.15.83:0/491499441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f948815f6e0 msgr2=0x7f94880b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.667+0000 7f9487200640 1 --2- 172.21.15.83:0/491499441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f948815f6e0 0x7f94880b1ae0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f948000ca80 tx=0x7f948000cf40 comp rx=0 tx=0).stop 2024-04-25T00:14:38.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.667+0000 7f9487200640 1 -- 172.21.15.83:0/491499441 shutdown_connections 2024-04-25T00:14:38.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.667+0000 7f9487200640 1 --2- 172.21.15.83:0/491499441 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f94680678f0 0x7f9468069db0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.667+0000 7f9487200640 1 --2- 172.21.15.83:0/491499441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f948815f6e0 0x7f94880b1ae0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.667+0000 7f9487200640 1 --2- 172.21.15.83:0/491499441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94880ce090 0x7f9488152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.667+0000 7f9487200640 1 --2- 172.21.15.83:0/491499441 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f948806b1e0 0x7f9488154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.667+0000 7f9487200640 1 -- 172.21.15.83:0/491499441 >> 172.21.15.83:0/491499441 conn(0x7f94880cd500 msgr2=0x7f948815fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:38.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.667+0000 7f9487200640 1 -- 172.21.15.83:0/491499441 shutdown_connections 2024-04-25T00:14:38.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.667+0000 7f9487200640 1 -- 172.21.15.83:0/491499441 wait complete. 2024-04-25T00:14:38.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:38.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4ff200640 1 Processor -- start 2024-04-25T00:14:38.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4ff200640 1 -- start start 2024-04-25T00:14:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4ff200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4f806ce10 0x7fb4f806fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4ff200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4f80b5400 con 0x7fb4f815bb20 2024-04-25T00:14:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4ff200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4f80b56e0 con 0x7fb4f8072cb0 2024-04-25T00:14:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4ff200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb4f80b51b0 con 0x7fb4f806ce10 2024-04-25T00:14:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fde00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb4f815bb20 0x7fb4f8161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45504/0 (socket says 172.21.15.83:45504) 2024-04-25T00:14:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fde00640 1 -- 172.21.15.83:0/909332472 learned_addr learned my addr 172.21.15.83:0/909332472 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fe800640 1 --2- 172.21.15.83:0/909332472 >> v2:172.21.15.86:3300/0 conn(0x7fb4f806ce10 0x7fb4f806fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 -- 172.21.15.83:0/909332472 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1117638926 0 0) 0x7fb4f80b5400 con 0x7fb4f815bb20 2024-04-25T00:14:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 -- 172.21.15.83:0/909332472 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4e0003530 con 0x7fb4f815bb20 2024-04-25T00:14:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 -- 172.21.15.83:0/909332472 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 595304034 0 0) 0x7fb4e0003530 con 0x7fb4f815bb20 2024-04-25T00:14:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 -- 172.21.15.83:0/909332472 >> v2:172.21.15.86:3300/0 conn(0x7fb4f806ce10 msgr2=0x7fb4f806fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 --2- 172.21.15.83:0/909332472 >> v2:172.21.15.86:3300/0 conn(0x7fb4f806ce10 0x7fb4f806fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 -- 172.21.15.83:0/909332472 >> v1:172.21.15.86:6789/0 conn(0x7fb4f8072cb0 legacy=0x7fb4f815eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 -- 172.21.15.83:0/909332472 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f80b59c0 con 0x7fb4f815bb20 2024-04-25T00:14:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 -- 172.21.15.83:0/909332472 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fb4ec001680 con 0x7fb4f815bb20 2024-04-25T00:14:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 -- 172.21.15.83:0/909332472 >> v1:172.21.15.83:6789/0 conn(0x7fb4f815bb20 legacy=0x7fb4f8161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 --2- 172.21.15.83:0/909332472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e00039c0 0x7fb4e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 --2- 172.21.15.83:0/909332472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e0004810 0x7fb4e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 --2- 172.21.15.83:0/909332472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb4e00051d0 0x7fb4e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 -- 172.21.15.83:0/909332472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f80b51b0 con 0x7fb4e0004810 2024-04-25T00:14:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 -- 172.21.15.83:0/909332472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e000c070 con 0x7fb4e00039c0 2024-04-25T00:14:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fca00640 1 -- 172.21.15.83:0/909332472 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e000c270 con 0x7fb4e00051d0 2024-04-25T00:14:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fd400640 1 --2- 172.21.15.83:0/909332472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e0004810 0x7fb4e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fde00640 1 --2- 172.21.15.83:0/909332472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e00039c0 0x7fb4e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fe800640 1 --2- 172.21.15.83:0/909332472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb4e00051d0 0x7fb4e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fd400640 1 -- 172.21.15.83:0/909332472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb4e00051d0 msgr2=0x7fb4e000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fd400640 1 --2- 172.21.15.83:0/909332472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb4e00051d0 0x7fb4e000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fd400640 1 -- 172.21.15.83:0/909332472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e00039c0 msgr2=0x7fb4e0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fd400640 1 --2- 172.21.15.83:0/909332472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e00039c0 0x7fb4e0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fe800640 1 -- 172.21.15.83:0/909332472 reap_dead start 2024-04-25T00:14:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fd400640 1 -- 172.21.15.83:0/909332472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e0003530 con 0x7fb4e0004810 2024-04-25T00:14:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.823+0000 7fb4fde00640 1 --2- 172.21.15.83:0/909332472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e00039c0 0x7fb4e0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:38.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.827+0000 7fb4fd400640 1 --2- 172.21.15.83:0/909332472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e0004810 0x7fb4e0004c90 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8001750 tx=0x7fb4e801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c052ae09e89fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:38.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.827+0000 7fb4fca00640 1 -- 172.21.15.83:0/909332472 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80013a0 con 0x7fb4e0004810 2024-04-25T00:14:38.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.827+0000 7fb4fca00640 1 -- 172.21.15.83:0/909332472 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e80052f0 con 0x7fb4e0004810 2024-04-25T00:14:38.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.827+0000 7fb4fca00640 1 -- 172.21.15.83:0/909332472 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8003840 con 0x7fb4e0004810 2024-04-25T00:14:38.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.827+0000 7fb4ff200640 1 -- 172.21.15.83:0/909332472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e0004810 msgr2=0x7fb4e0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.827+0000 7fb4ff200640 1 --2- 172.21.15.83:0/909332472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e0004810 0x7fb4e0004c90 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8001750 tx=0x7fb4e801c3c0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.827+0000 7fb4ff200640 1 -- 172.21.15.83:0/909332472 shutdown_connections 2024-04-25T00:14:38.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.827+0000 7fb4ff200640 1 --2- 172.21.15.83:0/909332472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e0004810 0x7fb4e0004c90 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.827+0000 7fb4ff200640 1 -- 172.21.15.83:0/909332472 >> 172.21.15.83:0/909332472 conn(0x7fb4f806c2f0 msgr2=0x7fb4f806c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:38.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.827+0000 7fb4ff200640 1 -- 172.21.15.83:0/909332472 shutdown_connections 2024-04-25T00:14:38.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.827+0000 7fb4ff200640 1 -- 172.21.15.83:0/909332472 wait complete. 2024-04-25T00:14:38.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4ff200640 1 Processor -- start 2024-04-25T00:14:38.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4ff200640 1 -- start start 2024-04-25T00:14:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4ff200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb4e0004810 0x7fb4f806ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4ff200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8153e30 0x7fb4f81f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4ff200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4f8154290 0x7fb4f80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4ff200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f80b3b90 con 0x7fb4f8154290 2024-04-25T00:14:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4ff200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f815e680 con 0x7fb4f8153e30 2024-04-25T00:14:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4ff200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f80b3d10 con 0x7fb4e0004810 2024-04-25T00:14:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4fe800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4f8154290 0x7fb4f80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4fe800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4f8154290 0x7fb4f80b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44284/0 (socket says 172.21.15.83:44284) 2024-04-25T00:14:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4fe800640 1 -- 172.21.15.83:0/3720216299 learned_addr learned my addr 172.21.15.83:0/3720216299 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4fd400640 1 --2- 172.21.15.83:0/3720216299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8153e30 0x7fb4f81f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4fde00640 1 --2- 172.21.15.83:0/3720216299 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb4e0004810 0x7fb4f806ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4fe800640 1 -- 172.21.15.83:0/3720216299 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb4e0004810 msgr2=0x7fb4f806ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4fe800640 1 --2- 172.21.15.83:0/3720216299 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb4e0004810 0x7fb4f806ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4fe800640 1 -- 172.21.15.83:0/3720216299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8153e30 msgr2=0x7fb4f81f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4fe800640 1 --2- 172.21.15.83:0/3720216299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8153e30 0x7fb4f81f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4fe800640 1 -- 172.21.15.83:0/3720216299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f8154730 con 0x7fb4f8154290 2024-04-25T00:14:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4fe800640 1 --2- 172.21.15.83:0/3720216299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4f8154290 0x7fb4f80b1b40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8073610 tx=0x7fb4f400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4f2000640 1 -- 172.21.15.83:0/3720216299 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f400fce0 con 0x7fb4f8154290 2024-04-25T00:14:38.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4f2000640 1 -- 172.21.15.83:0/3720216299 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f400fe80 con 0x7fb4f8154290 2024-04-25T00:14:38.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4f2000640 1 -- 172.21.15.83:0/3720216299 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4015670 con 0x7fb4f8154290 2024-04-25T00:14:38.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4ff200640 1 -- 172.21.15.83:0/3720216299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f8154a10 con 0x7fb4f8154290 2024-04-25T00:14:38.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4ff200640 1 -- 172.21.15.83:0/3720216299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f8154fd0 con 0x7fb4f8154290 2024-04-25T00:14:38.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.831+0000 7fb4ff200640 1 -- 172.21.15.83:0/3720216299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f804f510 con 0x7fb4f8154290 2024-04-25T00:14:38.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.835+0000 7fb4f2000640 1 -- 172.21.15.83:0/3720216299 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb4f4015810 con 0x7fb4f8154290 2024-04-25T00:14:38.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.835+0000 7fb4f2000640 1 --2- 172.21.15.83:0/3720216299 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb4dc0678c0 0x7fb4dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:38.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.835+0000 7fb4f2000640 1 -- 172.21.15.83:0/3720216299 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fb4f4086c50 con 0x7fb4f8154290 2024-04-25T00:14:38.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.835+0000 7fb4fde00640 1 --2- 172.21.15.83:0/3720216299 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb4dc0678c0 0x7fb4dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:38.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.835+0000 7fb4fde00640 1 --2- 172.21.15.83:0/3720216299 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb4dc0678c0 0x7fb4dc069d80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec001ce0 tx=0x7fb4ec007cb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:38.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.839+0000 7fb4f2000640 1 -- 172.21.15.83:0/3720216299 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb4f4019030 con 0x7fb4f8154290 2024-04-25T00:14:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.999+0000 7fb4ff200640 1 -- 172.21.15.83:0/3720216299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4f806ca90 con 0x7fb4f8154290 2024-04-25T00:14:39.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.999+0000 7fb4f2000640 1 -- 172.21.15.83:0/3720216299 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16801 (secure 0 0 0) 0x7fb4f4010040 con 0x7fb4f8154290 2024-04-25T00:14:39.002 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:39.003 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:24.352251+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:21.082598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:21.713421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:39.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.999+0000 7fb4ff200640 1 -- 172.21.15.83:0/3720216299 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb4dc0678c0 msgr2=0x7fb4dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.999+0000 7fb4ff200640 1 --2- 172.21.15.83:0/3720216299 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb4dc0678c0 0x7fb4dc069d80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec001ce0 tx=0x7fb4ec007cb0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.999+0000 7fb4ff200640 1 -- 172.21.15.83:0/3720216299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4f8154290 msgr2=0x7fb4f80b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.999+0000 7fb4ff200640 1 --2- 172.21.15.83:0/3720216299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4f8154290 0x7fb4f80b1b40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8073610 tx=0x7fb4f400f1b0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.999+0000 7fb4ff200640 1 -- 172.21.15.83:0/3720216299 shutdown_connections 2024-04-25T00:14:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.999+0000 7fb4ff200640 1 --2- 172.21.15.83:0/3720216299 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb4dc0678c0 0x7fb4dc069d80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.999+0000 7fb4ff200640 1 --2- 172.21.15.83:0/3720216299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4f8154290 0x7fb4f80b1b40 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.999+0000 7fb4ff200640 1 --2- 172.21.15.83:0/3720216299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8153e30 0x7fb4f81f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.999+0000 7fb4ff200640 1 --2- 172.21.15.83:0/3720216299 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb4e0004810 0x7fb4f806ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.999+0000 7fb4ff200640 1 -- 172.21.15.83:0/3720216299 >> 172.21.15.83:0/3720216299 conn(0x7fb4f806c2f0 msgr2=0x7fb4f815ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.999+0000 7fb4ff200640 1 -- 172.21.15.83:0/3720216299 shutdown_connections 2024-04-25T00:14:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:38.999+0000 7fb4ff200640 1 -- 172.21.15.83:0/3720216299 wait complete. 2024-04-25T00:14:39.025 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:14:39.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.155+0000 7fefed400640 1 Processor -- start 2024-04-25T00:14:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefed400640 1 -- start start 2024-04-25T00:14:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefed400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fefe8145940 0x7fefe8072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefed400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fefe806de00 0x7fefe80d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefed400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fefe80cd2a0 0x7fefe80cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefed400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fefe80d1a40 con 0x7fefe80cd2a0 2024-04-25T00:14:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefed400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fefe806e260 con 0x7fefe8145940 2024-04-25T00:14:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefed400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fefe80d0f00 con 0x7fefe806de00 2024-04-25T00:14:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefe7200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fefe80cd2a0 0x7fefe80cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefe7200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fefe80cd2a0 0x7fefe80cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44298/0 (socket says 172.21.15.83:44298) 2024-04-25T00:14:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefe7200640 1 -- 172.21.15.83:0/750581234 learned_addr learned my addr 172.21.15.83:0/750581234 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefe5e00640 1 --2- 172.21.15.83:0/750581234 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fefe806de00 0x7fefe80d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefe6800640 1 --2- 172.21.15.83:0/750581234 >> v2:172.21.15.86:3300/0 conn(0x7fefe8145940 0x7fefe8072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefe7200640 1 -- 172.21.15.83:0/750581234 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fefe806de00 msgr2=0x7fefe80d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefe7200640 1 --2- 172.21.15.83:0/750581234 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fefe806de00 0x7fefe80d09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefe7200640 1 -- 172.21.15.83:0/750581234 >> v2:172.21.15.86:3300/0 conn(0x7fefe8145940 msgr2=0x7fefe8072ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefe7200640 1 --2- 172.21.15.83:0/750581234 >> v2:172.21.15.86:3300/0 conn(0x7fefe8145940 0x7fefe8072ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefe7200640 1 -- 172.21.15.83:0/750581234 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe80d11a0 con 0x7fefe80cd2a0 2024-04-25T00:14:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefe5e00640 1 --2- 172.21.15.83:0/750581234 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fefe806de00 0x7fefe80d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.159+0000 7fefe6800640 1 --2- 172.21.15.83:0/750581234 >> v2:172.21.15.86:3300/0 conn(0x7fefe8145940 0x7fefe8072ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:39.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefe7200640 1 --2- 172.21.15.83:0/750581234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe80cd2a0 0x7fefe80cd730 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fefe80720c0 tx=0x7fefe001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d2a40f60da165e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefe5400640 1 -- 172.21.15.83:0/750581234 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe00a55e0 con 0x7fefe80cd2a0 2024-04-25T00:14:39.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefe5400640 1 -- 172.21.15.83:0/750581234 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe00a5780 con 0x7fefe80cd2a0 2024-04-25T00:14:39.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefe5400640 1 -- 172.21.15.83:0/750581234 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe00a5b50 con 0x7fefe80cd2a0 2024-04-25T00:14:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefed400640 1 -- 172.21.15.83:0/750581234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe80cd2a0 msgr2=0x7fefe80cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefed400640 1 --2- 172.21.15.83:0/750581234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe80cd2a0 0x7fefe80cd730 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fefe80720c0 tx=0x7fefe001c960 comp rx=0 tx=0).stop 2024-04-25T00:14:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefed400640 1 -- 172.21.15.83:0/750581234 shutdown_connections 2024-04-25T00:14:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefed400640 1 --2- 172.21.15.83:0/750581234 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fefe806de00 0x7fefe80d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefed400640 1 --2- 172.21.15.83:0/750581234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe80cd2a0 0x7fefe80cd730 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefed400640 1 --2- 172.21.15.83:0/750581234 >> v2:172.21.15.86:3300/0 conn(0x7fefe8145940 0x7fefe8072ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefed400640 1 -- 172.21.15.83:0/750581234 >> 172.21.15.83:0/750581234 conn(0x7fefe806c2f0 msgr2=0x7fefe806c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefed400640 1 -- 172.21.15.83:0/750581234 shutdown_connections 2024-04-25T00:14:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefed400640 1 -- 172.21.15.83:0/750581234 wait complete. 2024-04-25T00:14:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefed400640 1 Processor -- start 2024-04-25T00:14:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.163+0000 7fefed400640 1 -- start start 2024-04-25T00:14:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefed400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefe806de00 0x7fefe8138190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefed400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe80cd2a0 0x7fefe8157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefed400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fefe8145940 0x7fefe815cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefed400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fefe80d02a0 con 0x7fefe80cd2a0 2024-04-25T00:14:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefed400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fefe80d0120 con 0x7fefe806de00 2024-04-25T00:14:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefed400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fefe80d0420 con 0x7fefe8145940 2024-04-25T00:14:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefe5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe80cd2a0 0x7fefe8157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefe5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe80cd2a0 0x7fefe8157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44302/0 (socket says 172.21.15.83:44302) 2024-04-25T00:14:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefe5e00640 1 -- 172.21.15.83:0/702756393 learned_addr learned my addr 172.21.15.83:0/702756393 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefe6800640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefe806de00 0x7fefe8138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefe7200640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fefe8145940 0x7fefe815cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefe5e00640 1 -- 172.21.15.83:0/702756393 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fefe8145940 msgr2=0x7fefe815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefe5e00640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fefe8145940 0x7fefe815cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefe5e00640 1 -- 172.21.15.83:0/702756393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefe806de00 msgr2=0x7fefe8138190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefe5e00640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefe806de00 0x7fefe8138190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefe5e00640 1 -- 172.21.15.83:0/702756393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe8158130 con 0x7fefe80cd2a0 2024-04-25T00:14:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefe6800640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefe806de00 0x7fefe8138190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefe5e00640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe80cd2a0 0x7fefe8157bf0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fefd400d830 tx=0x7fefd400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefdea00640 1 -- 172.21.15.83:0/702756393 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4018070 con 0x7fefe80cd2a0 2024-04-25T00:14:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefdea00640 1 -- 172.21.15.83:0/702756393 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd400be10 con 0x7fefe80cd2a0 2024-04-25T00:14:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefdea00640 1 -- 172.21.15.83:0/702756393 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4004250 con 0x7fefe80cd2a0 2024-04-25T00:14:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefed400640 1 -- 172.21.15.83:0/702756393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe813fe10 con 0x7fefe80cd2a0 2024-04-25T00:14:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefed400640 1 -- 172.21.15.83:0/702756393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe81403d0 con 0x7fefe80cd2a0 2024-04-25T00:14:39.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefed400640 1 -- 172.21.15.83:0/702756393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe80736c0 con 0x7fefe80cd2a0 2024-04-25T00:14:39.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.167+0000 7fefdea00640 1 -- 172.21.15.83:0/702756393 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fefd40098b0 con 0x7fefe80cd2a0 2024-04-25T00:14:39.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.171+0000 7fefdea00640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fefc80678c0 0x7fefc8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.171+0000 7fefdea00640 1 -- 172.21.15.83:0/702756393 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fefd4086080 con 0x7fefe80cd2a0 2024-04-25T00:14:39.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.171+0000 7fefe6800640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fefc80678c0 0x7fefc8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.171+0000 7fefe6800640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fefc80678c0 0x7fefc8069d80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fefe80b2780 tx=0x7fefd8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.175+0000 7fefdea00640 1 -- 172.21.15.83:0/702756393 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fefd4056ac0 con 0x7fefe80cd2a0 2024-04-25T00:14:39.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.363+0000 7fefed400640 1 -- 172.21.15.83:0/702756393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefe8143970 con 0x7fefe80cd2a0 2024-04-25T00:14:39.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.363+0000 7fefdea00640 1 -- 172.21.15.83:0/702756393 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5174 (secure 0 0 0) 0x7fefd40563c0 con 0x7fefe80cd2a0 2024-04-25T00:14:39.369 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:39.369 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/3856734881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3856734881},{"type":"v1","addr":"172.21.15.86:6834","nonce":3856734881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/12764228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":12764228},{"type":"v1","addr":"172.21.15.86:6835","nonce":12764228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1680607346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1680607346},{"type":"v1","addr":"172.21.15.83:6837","nonce":1680607346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:22.706751+0000","modified":"2024-04-25T00:14:25.734428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7023},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7023":{"gid":7023,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/121840722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":121840722},{"type":"v1","addr":"172.21.15.83:6836","nonce":121840722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7023,"qdb_cluster":[7023]},"id":6}]} 2024-04-25T00:14:39.369 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 53 2024-04-25T00:14:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.367+0000 7fefed400640 1 -- 172.21.15.83:0/702756393 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fefc80678c0 msgr2=0x7fefc8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.367+0000 7fefed400640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fefc80678c0 0x7fefc8069d80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fefe80b2780 tx=0x7fefd8008040 comp rx=0 tx=0).stop 2024-04-25T00:14:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.367+0000 7fefed400640 1 -- 172.21.15.83:0/702756393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe80cd2a0 msgr2=0x7fefe8157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.367+0000 7fefed400640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe80cd2a0 0x7fefe8157bf0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fefd400d830 tx=0x7fefd400dcf0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.367+0000 7fefed400640 1 -- 172.21.15.83:0/702756393 shutdown_connections 2024-04-25T00:14:39.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.367+0000 7fefed400640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fefc80678c0 0x7fefc8069d80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.367+0000 7fefed400640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fefe8145940 0x7fefe815cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.367+0000 7fefed400640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefe80cd2a0 0x7fefe8157bf0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.367+0000 7fefed400640 1 --2- 172.21.15.83:0/702756393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefe806de00 0x7fefe8138190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.367+0000 7fefed400640 1 -- 172.21.15.83:0/702756393 >> 172.21.15.83:0/702756393 conn(0x7fefe806c2f0 msgr2=0x7fefe813aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:39.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.367+0000 7fefed400640 1 -- 172.21.15.83:0/702756393 shutdown_connections 2024-04-25T00:14:39.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.367+0000 7fefed400640 1 -- 172.21.15.83:0/702756393 wait complete. 2024-04-25T00:14:39.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:39.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98ad800640 1 Processor -- start 2024-04-25T00:14:39.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98ad800640 1 -- start start 2024-04-25T00:14:39.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98ad800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a80b4360 0x7f98a80b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98ad800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98a80b2b30 0x7f98a80b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98ad800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a806ef80 con 0x7f98a81553e0 2024-04-25T00:14:39.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98ad800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f98a815b510 con 0x7f98a80b2b30 2024-04-25T00:14:39.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98ad800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f98a815b7f0 con 0x7f98a80b4360 2024-04-25T00:14:39.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a6a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f98a81553e0 0x7f98a806e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45516/0 (socket says 172.21.15.83:45516) 2024-04-25T00:14:39.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a6a00640 1 -- 172.21.15.83:0/1547192532 learned_addr learned my addr 172.21.15.83:0/1547192532 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:39.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a7400640 1 --2- 172.21.15.83:0/1547192532 >> v2:172.21.15.86:3300/0 conn(0x7f98a80b2b30 0x7f98a80b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a6000640 1 --2- 172.21.15.83:0/1547192532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a80b4360 0x7f98a80b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 -- 172.21.15.83:0/1547192532 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 631949936 0 0) 0x7f98a806ef80 con 0x7f98a81553e0 2024-04-25T00:14:39.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 -- 172.21.15.83:0/1547192532 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f988c003530 con 0x7f98a81553e0 2024-04-25T00:14:39.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 -- 172.21.15.83:0/1547192532 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1534222391 0 0) 0x7f988c003530 con 0x7f98a81553e0 2024-04-25T00:14:39.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 -- 172.21.15.83:0/1547192532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a80b4360 msgr2=0x7f98a80b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 --2- 172.21.15.83:0/1547192532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a80b4360 0x7f98a80b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 -- 172.21.15.83:0/1547192532 >> v2:172.21.15.86:3300/0 conn(0x7f98a80b2b30 msgr2=0x7f98a80b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 --2- 172.21.15.83:0/1547192532 >> v2:172.21.15.86:3300/0 conn(0x7f98a80b2b30 0x7f98a80b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 -- 172.21.15.83:0/1547192532 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a815bad0 con 0x7f98a81553e0 2024-04-25T00:14:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 -- 172.21.15.83:0/1547192532 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f9898001680 con 0x7f98a81553e0 2024-04-25T00:14:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 -- 172.21.15.83:0/1547192532 >> v1:172.21.15.83:6789/0 conn(0x7f98a81553e0 legacy=0x7f98a806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 --2- 172.21.15.83:0/1547192532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f988c0039c0 0x7f988c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 --2- 172.21.15.83:0/1547192532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a80b4360 0x7f988c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 --2- 172.21.15.83:0/1547192532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f988c004d50 0x7f988c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 -- 172.21.15.83:0/1547192532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98a815b510 con 0x7f988c004d50 2024-04-25T00:14:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 -- 172.21.15.83:0/1547192532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98a815b7f0 con 0x7f988c0039c0 2024-04-25T00:14:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a5600640 1 -- 172.21.15.83:0/1547192532 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f988c00b9d0 con 0x7f98a80b4360 2024-04-25T00:14:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a7400640 1 --2- 172.21.15.83:0/1547192532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f988c004d50 0x7f988c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a6a00640 1 --2- 172.21.15.83:0/1547192532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f988c0039c0 0x7f988c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a6000640 1 --2- 172.21.15.83:0/1547192532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a80b4360 0x7f988c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a7400640 1 -- 172.21.15.83:0/1547192532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a80b4360 msgr2=0x7f988c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a7400640 1 --2- 172.21.15.83:0/1547192532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a80b4360 0x7f988c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a7400640 1 -- 172.21.15.83:0/1547192532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f988c0039c0 msgr2=0x7f988c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a7400640 1 --2- 172.21.15.83:0/1547192532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f988c0039c0 0x7f988c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.527+0000 7f98a7400640 1 -- 172.21.15.83:0/1547192532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988c003530 con 0x7f988c004d50 2024-04-25T00:14:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.531+0000 7f98a7400640 1 --2- 172.21.15.83:0/1547192532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f988c004d50 0x7f988c0051d0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009fc0 tx=0x7f98a001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9bae45cd2f9ae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.531+0000 7f98a5600640 1 -- 172.21.15.83:0/1547192532 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a00b2070 con 0x7f988c004d50 2024-04-25T00:14:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.531+0000 7f98a5600640 1 -- 172.21.15.83:0/1547192532 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a00a5bc0 con 0x7f988c004d50 2024-04-25T00:14:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.531+0000 7f98a5600640 1 -- 172.21.15.83:0/1547192532 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0009c10 con 0x7f988c004d50 2024-04-25T00:14:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.531+0000 7f98ad800640 1 -- 172.21.15.83:0/1547192532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f988c004d50 msgr2=0x7f988c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.531+0000 7f98ad800640 1 --2- 172.21.15.83:0/1547192532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f988c004d50 0x7f988c0051d0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009fc0 tx=0x7f98a001ccc0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.531+0000 7f98a7400640 1 -- 172.21.15.83:0/1547192532 reap_dead start 2024-04-25T00:14:39.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.531+0000 7f98ad800640 1 -- 172.21.15.83:0/1547192532 shutdown_connections 2024-04-25T00:14:39.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.531+0000 7f98ad800640 1 -- 172.21.15.83:0/1547192532 >> 172.21.15.83:0/1547192532 conn(0x7f98a80b1830 msgr2=0x7f98a80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:39.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.531+0000 7f98ad800640 1 -- 172.21.15.83:0/1547192532 shutdown_connections 2024-04-25T00:14:39.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.531+0000 7f98ad800640 1 -- 172.21.15.83:0/1547192532 wait complete. 2024-04-25T00:14:39.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98ad800640 1 Processor -- start 2024-04-25T00:14:39.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98ad800640 1 -- start start 2024-04-25T00:14:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98ad800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f988c004d50 0x7f98a8158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98ad800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a8150b60 0x7f98a8150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98ad800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a8155b30 0x7f98a8155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98ad800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98a81557c0 con 0x7f98a8150b60 2024-04-25T00:14:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98ad800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98a8070f30 con 0x7f98a8155b30 2024-04-25T00:14:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98ad800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f98a8155940 con 0x7f988c004d50 2024-04-25T00:14:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98a6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a8150b60 0x7f98a8150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98a6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a8150b60 0x7f98a8150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44312/0 (socket says 172.21.15.83:44312) 2024-04-25T00:14:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98a6000640 1 -- 172.21.15.83:0/956813848 learned_addr learned my addr 172.21.15.83:0/956813848 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98a6a00640 1 --2- 172.21.15.83:0/956813848 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f988c004d50 0x7f98a8158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98a7400640 1 --2- 172.21.15.83:0/956813848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a8155b30 0x7f98a8155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98a6000640 1 -- 172.21.15.83:0/956813848 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f988c004d50 msgr2=0x7f98a8158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98a6000640 1 --2- 172.21.15.83:0/956813848 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f988c004d50 0x7f98a8158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98a6000640 1 -- 172.21.15.83:0/956813848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a8155b30 msgr2=0x7f98a8155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98a6000640 1 --2- 172.21.15.83:0/956813848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a8155b30 0x7f98a8155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98a6000640 1 -- 172.21.15.83:0/956813848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a81568a0 con 0x7f98a8150b60 2024-04-25T00:14:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98a6000640 1 --2- 172.21.15.83:0/956813848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a8150b60 0x7f98a8150fe0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f98940119d0 tx=0x7f9894011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f989ea00640 1 -- 172.21.15.83:0/956813848 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989400faf0 con 0x7f98a8150b60 2024-04-25T00:14:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f989ea00640 1 -- 172.21.15.83:0/956813848 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f989400fc90 con 0x7f98a8150b60 2024-04-25T00:14:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f989ea00640 1 -- 172.21.15.83:0/956813848 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894014440 con 0x7f98a8150b60 2024-04-25T00:14:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98ad800640 1 -- 172.21.15.83:0/956813848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a8156b80 con 0x7f98a8150b60 2024-04-25T00:14:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.535+0000 7f98ad800640 1 -- 172.21.15.83:0/956813848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a8157140 con 0x7f98a8150b60 2024-04-25T00:14:39.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.539+0000 7f98ad800640 1 -- 172.21.15.83:0/956813848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a80737f0 con 0x7f98a8150b60 2024-04-25T00:14:39.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.539+0000 7f989ea00640 1 -- 172.21.15.83:0/956813848 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9894012040 con 0x7f98a8150b60 2024-04-25T00:14:39.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.539+0000 7f989ea00640 1 --2- 172.21.15.83:0/956813848 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98880678c0 0x7f9888069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.539+0000 7f989ea00640 1 -- 172.21.15.83:0/956813848 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f98940864e0 con 0x7f98a8150b60 2024-04-25T00:14:39.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.539+0000 7f98a6a00640 1 --2- 172.21.15.83:0/956813848 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98880678c0 0x7f9888069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.539+0000 7f98a6a00640 1 --2- 172.21.15.83:0/956813848 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98880678c0 0x7f9888069d80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9898000c10 tx=0x7f9898009b10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.543+0000 7f989ea00640 1 -- 172.21.15.83:0/956813848 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9894019030 con 0x7f98a8150b60 2024-04-25T00:14:39.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.731+0000 7f98ad800640 1 -- 172.21.15.83:0/956813848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98a80b4140 con 0x7f98a8150b60 2024-04-25T00:14:39.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.731+0000 7f989ea00640 1 -- 172.21.15.83:0/956813848 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5174 (secure 0 0 0) 0x7f9894056df0 con 0x7f98a8150b60 2024-04-25T00:14:39.737 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:39.737 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/3856734881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3856734881},{"type":"v1","addr":"172.21.15.86:6834","nonce":3856734881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/12764228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":12764228},{"type":"v1","addr":"172.21.15.86:6835","nonce":12764228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1680607346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1680607346},{"type":"v1","addr":"172.21.15.83:6837","nonce":1680607346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:22.706751+0000","modified":"2024-04-25T00:14:25.734428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7023},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7023":{"gid":7023,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/121840722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":121840722},{"type":"v1","addr":"172.21.15.83:6836","nonce":121840722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7023,"qdb_cluster":[7023]},"id":6}]} 2024-04-25T00:14:39.737 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 53 2024-04-25T00:14:39.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.735+0000 7f98ad800640 1 -- 172.21.15.83:0/956813848 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98880678c0 msgr2=0x7f9888069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.735+0000 7f98ad800640 1 --2- 172.21.15.83:0/956813848 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98880678c0 0x7f9888069d80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9898000c10 tx=0x7f9898009b10 comp rx=0 tx=0).stop 2024-04-25T00:14:39.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.735+0000 7f98ad800640 1 -- 172.21.15.83:0/956813848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a8150b60 msgr2=0x7f98a8150fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.735+0000 7f98ad800640 1 --2- 172.21.15.83:0/956813848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a8150b60 0x7f98a8150fe0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f98940119d0 tx=0x7f9894011dd0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.735+0000 7f98ad800640 1 -- 172.21.15.83:0/956813848 shutdown_connections 2024-04-25T00:14:39.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.735+0000 7f98ad800640 1 --2- 172.21.15.83:0/956813848 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f98880678c0 0x7f9888069d80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.735+0000 7f98ad800640 1 --2- 172.21.15.83:0/956813848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a8155b30 0x7f98a8155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.735+0000 7f98ad800640 1 --2- 172.21.15.83:0/956813848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a8150b60 0x7f98a8150fe0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.735+0000 7f98ad800640 1 --2- 172.21.15.83:0/956813848 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f988c004d50 0x7f98a8158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.735+0000 7f98ad800640 1 -- 172.21.15.83:0/956813848 >> 172.21.15.83:0/956813848 conn(0x7f98a80b1830 msgr2=0x7f98a81546c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:39.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.735+0000 7f98ad800640 1 -- 172.21.15.83:0/956813848 shutdown_connections 2024-04-25T00:14:39.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.735+0000 7f98ad800640 1 -- 172.21.15.83:0/956813848 wait complete. 2024-04-25T00:14:39.757 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.887+0000 7efc57600640 1 Processor -- start 2024-04-25T00:14:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.887+0000 7efc57600640 1 -- start start 2024-04-25T00:14:39.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.891+0000 7efc57600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc500b4360 0x7efc500b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.891+0000 7efc57600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efc500b2b30 0x7efc500b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.891+0000 7efc57600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc5006ef80 con 0x7efc501553e0 2024-04-25T00:14:39.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.891+0000 7efc57600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7efc5015b510 con 0x7efc500b2b30 2024-04-25T00:14:39.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.891+0000 7efc57600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efc5015b7f0 con 0x7efc500b4360 2024-04-25T00:14:39.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.891+0000 7efc55600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efc500b2b30 0x7efc500b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.891+0000 7efc55600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efc500b2b30 0x7efc500b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44326/0 (socket says 172.21.15.83:44326) 2024-04-25T00:14:39.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.891+0000 7efc55600640 1 -- 172.21.15.83:0/1370007013 learned_addr learned my addr 172.21.15.83:0/1370007013 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:39.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.891+0000 7efc54c00640 1 --1- 172.21.15.83:0/1370007013 >> v1:172.21.15.83:6789/0 conn(0x7efc501553e0 0x7efc5006e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45524/0 (socket says 172.21.15.83:45524) 2024-04-25T00:14:39.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.891+0000 7efc4fe00640 1 --2- 172.21.15.83:0/1370007013 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc500b4360 0x7efc500b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.891+0000 7efc55600640 1 -- 172.21.15.83:0/1370007013 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc500b4360 msgr2=0x7efc500b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.891+0000 7efc55600640 1 --2- 172.21.15.83:0/1370007013 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc500b4360 0x7efc500b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.891+0000 7efc55600640 1 -- 172.21.15.83:0/1370007013 >> v1:172.21.15.83:6789/0 conn(0x7efc501553e0 legacy=0x7efc5006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.891+0000 7efc55600640 1 -- 172.21.15.83:0/1370007013 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5015bad0 con 0x7efc500b2b30 2024-04-25T00:14:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc55600640 1 --2- 172.21.15.83:0/1370007013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc500b2b30 0x7efc500b2fc0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7efc48009870 tx=0x7efc4801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fab5625949d1d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc4f400640 1 -- 172.21.15.83:0/1370007013 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc480b0070 con 0x7efc500b2b30 2024-04-25T00:14:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc4f400640 1 -- 172.21.15.83:0/1370007013 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc480ab600 con 0x7efc500b2b30 2024-04-25T00:14:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc4f400640 1 -- 172.21.15.83:0/1370007013 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc480ab9d0 con 0x7efc500b2b30 2024-04-25T00:14:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 -- 172.21.15.83:0/1370007013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc500b2b30 msgr2=0x7efc500b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 --2- 172.21.15.83:0/1370007013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc500b2b30 0x7efc500b2fc0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7efc48009870 tx=0x7efc4801c4d0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 -- 172.21.15.83:0/1370007013 shutdown_connections 2024-04-25T00:14:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 --2- 172.21.15.83:0/1370007013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc500b2b30 0x7efc500b2fc0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 --2- 172.21.15.83:0/1370007013 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc500b4360 0x7efc500b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 -- 172.21.15.83:0/1370007013 >> 172.21.15.83:0/1370007013 conn(0x7efc500b1830 msgr2=0x7efc500b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 -- 172.21.15.83:0/1370007013 shutdown_connections 2024-04-25T00:14:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 -- 172.21.15.83:0/1370007013 wait complete. 2024-04-25T00:14:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 Processor -- start 2024-04-25T00:14:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 -- start start 2024-04-25T00:14:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc500b2b30 0x7efc5015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc500b4360 0x7efc5015f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc501553e0 0x7efc5014dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc5006ebd0 con 0x7efc500b2b30 2024-04-25T00:14:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc50167d90 con 0x7efc500b4360 2024-04-25T00:14:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc57600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efc50157e90 con 0x7efc501553e0 2024-04-25T00:14:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc54c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc500b2b30 0x7efc5015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc54c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc500b2b30 0x7efc5015d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44328/0 (socket says 172.21.15.83:44328) 2024-04-25T00:14:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.895+0000 7efc54c00640 1 -- 172.21.15.83:0/3490377585 learned_addr learned my addr 172.21.15.83:0/3490377585 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc55600640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc501553e0 0x7efc5014dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc4fe00640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc500b4360 0x7efc5015f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc54c00640 1 -- 172.21.15.83:0/3490377585 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc501553e0 msgr2=0x7efc5014dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc54c00640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc501553e0 0x7efc5014dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc54c00640 1 -- 172.21.15.83:0/3490377585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc500b4360 msgr2=0x7efc5015f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc54c00640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc500b4360 0x7efc5015f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc54c00640 1 -- 172.21.15.83:0/3490377585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc50158170 con 0x7efc500b2b30 2024-04-25T00:14:39.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc55600640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc501553e0 0x7efc5014dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:39.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc4fe00640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc500b4360 0x7efc5015f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:39.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc54c00640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc500b2b30 0x7efc5015d5e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7efc4000c790 tx=0x7efc4000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc4cc00640 1 -- 172.21.15.83:0/3490377585 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40017070 con 0x7efc500b2b30 2024-04-25T00:14:39.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc4cc00640 1 -- 172.21.15.83:0/3490377585 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc4000a640 con 0x7efc500b2b30 2024-04-25T00:14:39.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc4cc00640 1 -- 172.21.15.83:0/3490377585 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4000a9f0 con 0x7efc500b2b30 2024-04-25T00:14:39.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc57600640 1 -- 172.21.15.83:0/3490377585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc50158450 con 0x7efc500b2b30 2024-04-25T00:14:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc57600640 1 -- 172.21.15.83:0/3490377585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc50158a10 con 0x7efc500b2b30 2024-04-25T00:14:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc57600640 1 -- 172.21.15.83:0/3490377585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc500737f0 con 0x7efc500b2b30 2024-04-25T00:14:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc4cc00640 1 -- 172.21.15.83:0/3490377585 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efc4001a070 con 0x7efc500b2b30 2024-04-25T00:14:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc4cc00640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc300678c0 0x7efc30069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc4cc00640 1 -- 172.21.15.83:0/3490377585 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7efc40085330 con 0x7efc500b2b30 2024-04-25T00:14:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.899+0000 7efc4fe00640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc300678c0 0x7efc30069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.903+0000 7efc4fe00640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc300678c0 0x7efc30069d80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00b9f0 tx=0x7efc3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:39.903+0000 7efc4cc00640 1 -- 172.21.15.83:0/3490377585 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efc40055720 con 0x7efc500b2b30 2024-04-25T00:14:40.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.063+0000 7efc57600640 1 -- 172.21.15.83:0/3490377585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc5004f5f0 con 0x7efc500b2b30 2024-04-25T00:14:40.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.063+0000 7efc4cc00640 1 -- 172.21.15.83:0/3490377585 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16801 (secure 0 0 0) 0x7efc40059a20 con 0x7efc500b2b30 2024-04-25T00:14:40.067 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:40.067 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:24.352251+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:21.082598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:21.713421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:40.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.063+0000 7efc57600640 1 -- 172.21.15.83:0/3490377585 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc300678c0 msgr2=0x7efc30069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.063+0000 7efc57600640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc300678c0 0x7efc30069d80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00b9f0 tx=0x7efc3c008040 comp rx=0 tx=0).stop 2024-04-25T00:14:40.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.063+0000 7efc57600640 1 -- 172.21.15.83:0/3490377585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc500b2b30 msgr2=0x7efc5015d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.063+0000 7efc57600640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc500b2b30 0x7efc5015d5e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7efc4000c790 tx=0x7efc4000cc50 comp rx=0 tx=0).stop 2024-04-25T00:14:40.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.063+0000 7efc57600640 1 -- 172.21.15.83:0/3490377585 shutdown_connections 2024-04-25T00:14:40.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.063+0000 7efc57600640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efc300678c0 0x7efc30069d80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.063+0000 7efc57600640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc501553e0 0x7efc5014dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.063+0000 7efc57600640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc500b4360 0x7efc5015f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.063+0000 7efc57600640 1 --2- 172.21.15.83:0/3490377585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc500b2b30 0x7efc5015d5e0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.063+0000 7efc57600640 1 -- 172.21.15.83:0/3490377585 >> 172.21.15.83:0/3490377585 conn(0x7efc500b1830 msgr2=0x7efc50152ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:40.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.067+0000 7efc57600640 1 -- 172.21.15.83:0/3490377585 shutdown_connections 2024-04-25T00:14:40.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.067+0000 7efc57600640 1 -- 172.21.15.83:0/3490377585 wait complete. 2024-04-25T00:14:40.089 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:14:40.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.219+0000 7f9808a00640 1 Processor -- start 2024-04-25T00:14:40.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.219+0000 7f9808a00640 1 -- start start 2024-04-25T00:14:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.223+0000 7f9808a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f98040cab50 0x7f980415f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.223+0000 7f9808a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98040cafa0 con 0x7f98041446e0 2024-04-25T00:14:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.223+0000 7f9808a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98040cb210 con 0x7f980406ee20 2024-04-25T00:14:40.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.223+0000 7f9808a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f98040715b0 con 0x7f98040cab50 2024-04-25T00:14:40.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.223+0000 7f9801e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f98041446e0 0x7f980406e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45540/0 (socket says 172.21.15.83:45540) 2024-04-25T00:14:40.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.223+0000 7f9801e00640 1 -- 172.21.15.83:0/2578942351 learned_addr learned my addr 172.21.15.83:0/2578942351 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:40.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.223+0000 7f9802800640 1 --2- 172.21.15.83:0/2578942351 >> v2:172.21.15.83:3300/0 conn(0x7f98040cab50 0x7f980415f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.223+0000 7f9802800640 1 -- 172.21.15.83:0/2578942351 >> v1:172.21.15.86:6789/0 conn(0x7f980406ee20 legacy=0x7f9804071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.223+0000 7f9802800640 1 -- 172.21.15.83:0/2578942351 >> v1:172.21.15.83:6789/0 conn(0x7f98041446e0 legacy=0x7f980406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.223+0000 7f9802800640 1 -- 172.21.15.83:0/2578942351 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98040cb4f0 con 0x7f98040cab50 2024-04-25T00:14:40.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.223+0000 7f9801400640 1 -- 172.21.15.83:0/2578942351 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3201673783 0 0) 0x7f98040cafa0 con 0x7f98041446e0 2024-04-25T00:14:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.223+0000 7f9802800640 1 --2- 172.21.15.83:0/2578942351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98040cab50 0x7f980415f350 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0097c0 tx=0x7f97fc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb01a625e69cf17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9801400640 1 -- 172.21.15.83:0/2578942351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0b0070 con 0x7f98040cab50 2024-04-25T00:14:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9801400640 1 -- 172.21.15.83:0/2578942351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97fc0ab600 con 0x7f98040cab50 2024-04-25T00:14:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9801400640 1 -- 172.21.15.83:0/2578942351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0ab9d0 con 0x7f98040cab50 2024-04-25T00:14:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 -- 172.21.15.83:0/2578942351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98040cab50 msgr2=0x7f980415f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 --2- 172.21.15.83:0/2578942351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98040cab50 0x7f980415f350 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0097c0 tx=0x7f97fc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 -- 172.21.15.83:0/2578942351 shutdown_connections 2024-04-25T00:14:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 --2- 172.21.15.83:0/2578942351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98040cab50 0x7f980415f350 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 -- 172.21.15.83:0/2578942351 >> 172.21.15.83:0/2578942351 conn(0x7f98040cfa60 msgr2=0x7f98040cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 -- 172.21.15.83:0/2578942351 shutdown_connections 2024-04-25T00:14:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 -- 172.21.15.83:0/2578942351 wait complete. 2024-04-25T00:14:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 Processor -- start 2024-04-25T00:14:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 -- start start 2024-04-25T00:14:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f980406ee20 0x7f98040c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98040cab50 0x7f980406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98041446e0 0x7f980406db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9804168cb0 con 0x7f98041446e0 2024-04-25T00:14:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9804071b50 con 0x7f98040cab50 2024-04-25T00:14:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9808a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9804168e30 con 0x7f980406ee20 2024-04-25T00:14:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9802800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98041446e0 0x7f980406db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9802800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98041446e0 0x7f980406db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44342/0 (socket says 172.21.15.83:44342) 2024-04-25T00:14:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9802800640 1 -- 172.21.15.83:0/1060251872 learned_addr learned my addr 172.21.15.83:0/1060251872 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f9801e00640 1 --2- 172.21.15.83:0/1060251872 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f980406ee20 0x7f98040c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.227+0000 7f97f9400640 1 --2- 172.21.15.83:0/1060251872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98040cab50 0x7f980406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f9802800640 1 -- 172.21.15.83:0/1060251872 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f980406ee20 msgr2=0x7f98040c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f9802800640 1 --2- 172.21.15.83:0/1060251872 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f980406ee20 0x7f98040c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f9802800640 1 -- 172.21.15.83:0/1060251872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98040cab50 msgr2=0x7f980406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f9802800640 1 --2- 172.21.15.83:0/1060251872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98040cab50 0x7f980406d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f9802800640 1 -- 172.21.15.83:0/1060251872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98040b4630 con 0x7f98041446e0 2024-04-25T00:14:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f9802800640 1 --2- 172.21.15.83:0/1060251872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98041446e0 0x7f980406db40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0190b0 tx=0x7f97fc01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f97faa00640 1 -- 172.21.15.83:0/1060251872 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0b0050 con 0x7f98041446e0 2024-04-25T00:14:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f97faa00640 1 -- 172.21.15.83:0/1060251872 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97fc0bc070 con 0x7f98041446e0 2024-04-25T00:14:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f97faa00640 1 -- 172.21.15.83:0/1060251872 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0b5830 con 0x7f98041446e0 2024-04-25T00:14:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f9808a00640 1 -- 172.21.15.83:0/1060251872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98040b4910 con 0x7f98041446e0 2024-04-25T00:14:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f9808a00640 1 -- 172.21.15.83:0/1060251872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98040b4ed0 con 0x7f98041446e0 2024-04-25T00:14:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f9808a00640 1 -- 172.21.15.83:0/1060251872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98040b4380 con 0x7f98041446e0 2024-04-25T00:14:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f97faa00640 1 -- 172.21.15.83:0/1060251872 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9804072f00 con 0x7f98041446e0 2024-04-25T00:14:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f97faa00640 1 --2- 172.21.15.83:0/1060251872 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97e00678c0 0x7f97e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f97faa00640 1 -- 172.21.15.83:0/1060251872 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f97fc120400 con 0x7f98041446e0 2024-04-25T00:14:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.231+0000 7f9801e00640 1 --2- 172.21.15.83:0/1060251872 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97e00678c0 0x7f97e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.235+0000 7f9801e00640 1 --2- 172.21.15.83:0/1060251872 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97e00678c0 0x7f97e0069d80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f97f4000f70 tx=0x7f97f4007240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:40.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.235+0000 7f97faa00640 1 -- 172.21.15.83:0/1060251872 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f97fc016090 con 0x7f98041446e0 2024-04-25T00:14:40.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.423+0000 7f9808a00640 1 -- 172.21.15.83:0/1060251872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9804068230 con 0x7f98041446e0 2024-04-25T00:14:40.429 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-25T00:14:40.427+0000 7efeeee00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:14:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.427+0000 7f97faa00640 1 -- 172.21.15.83:0/1060251872 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) ==== 161+0+0 (secure 0 0 0) 0x7f97fc0f06c0 con 0x7f98041446e0 2024-04-25T00:14:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:14:40.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.431+0000 7f9808a00640 1 -- 172.21.15.83:0/1060251872 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97e00678c0 msgr2=0x7f97e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.431+0000 7f9808a00640 1 --2- 172.21.15.83:0/1060251872 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97e00678c0 0x7f97e0069d80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f97f4000f70 tx=0x7f97f4007240 comp rx=0 tx=0).stop 2024-04-25T00:14:40.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.431+0000 7f9808a00640 1 -- 172.21.15.83:0/1060251872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98041446e0 msgr2=0x7f980406db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.431+0000 7f9808a00640 1 --2- 172.21.15.83:0/1060251872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98041446e0 0x7f980406db40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0190b0 tx=0x7f97fc01ace0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.431+0000 7f9808a00640 1 -- 172.21.15.83:0/1060251872 shutdown_connections 2024-04-25T00:14:40.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.431+0000 7f9808a00640 1 --2- 172.21.15.83:0/1060251872 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f97e00678c0 0x7f97e0069d80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.431+0000 7f9808a00640 1 --2- 172.21.15.83:0/1060251872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98041446e0 0x7f980406db40 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.431+0000 7f9808a00640 1 --2- 172.21.15.83:0/1060251872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98040cab50 0x7f980406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.431+0000 7f9808a00640 1 --2- 172.21.15.83:0/1060251872 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f980406ee20 0x7f98040c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.431+0000 7f9808a00640 1 -- 172.21.15.83:0/1060251872 >> 172.21.15.83:0/1060251872 conn(0x7f98040cfa60 msgr2=0x7f980415e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:40.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.431+0000 7f9808a00640 1 -- 172.21.15.83:0/1060251872 shutdown_connections 2024-04-25T00:14:40.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.431+0000 7f9808a00640 1 -- 172.21.15.83:0/1060251872 wait complete. 2024-04-25T00:14:40.440 INFO:tasks.ceph.mds.a.smithi083.stderr: -2154> 2024-04-25T00:14:29.315+0000 7f1a8a200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:14:40.454 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:14:40.486 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-25T00:14:40.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.591+0000 7fdaf6a00640 1 Processor -- start 2024-04-25T00:14:40.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.591+0000 7fdaf6a00640 1 -- start start 2024-04-25T00:14:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.591+0000 7fdaf6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdaf015c0a0 0x7fdaf00cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.591+0000 7fdaf6a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdaf00d2690 0x7fdaf00ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.591+0000 7fdaf6a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdaf00ceac0 0x7fdaf00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.591+0000 7fdaf6a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf00cef10 con 0x7fdaf00d2690 2024-04-25T00:14:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.591+0000 7fdaf6a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf00cd1e0 con 0x7fdaf015c0a0 2024-04-25T00:14:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.591+0000 7fdaf6a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf00cd430 con 0x7fdaf00ceac0 2024-04-25T00:14:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.591+0000 7fdaf4c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdaf00d2690 0x7fdaf00ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.591+0000 7fdaf4c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdaf00d2690 0x7fdaf00ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44374/0 (socket says 172.21.15.83:44374) 2024-04-25T00:14:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.591+0000 7fdaf4c00640 1 -- 172.21.15.83:0/1794042356 learned_addr learned my addr 172.21.15.83:0/1794042356 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.591+0000 7fdaf5600640 1 --2- 172.21.15.83:0/1794042356 >> v2:172.21.15.86:3300/0 conn(0x7fdaf015c0a0 0x7fdaf00cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.591+0000 7fdaf6000640 1 --2- 172.21.15.83:0/1794042356 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdaf00ceac0 0x7fdaf00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaf4c00640 1 -- 172.21.15.83:0/1794042356 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdaf00ceac0 msgr2=0x7fdaf00cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaf4c00640 1 --2- 172.21.15.83:0/1794042356 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdaf00ceac0 0x7fdaf00cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaf4c00640 1 -- 172.21.15.83:0/1794042356 >> v2:172.21.15.86:3300/0 conn(0x7fdaf015c0a0 msgr2=0x7fdaf00cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaf4c00640 1 --2- 172.21.15.83:0/1794042356 >> v2:172.21.15.86:3300/0 conn(0x7fdaf015c0a0 0x7fdaf00cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaf4c00640 1 -- 172.21.15.83:0/1794042356 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf00cd710 con 0x7fdaf00d2690 2024-04-25T00:14:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaf5600640 1 --2- 172.21.15.83:0/1794042356 >> v2:172.21.15.86:3300/0 conn(0x7fdaf015c0a0 0x7fdaf00cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaf4c00640 1 --2- 172.21.15.83:0/1794042356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf00d2690 0x7fdaf00ce580 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0073170 tx=0x7fdadc01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67606d9f39f9bf3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaefe00640 1 -- 172.21.15.83:0/1794042356 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0b0070 con 0x7fdaf00d2690 2024-04-25T00:14:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaefe00640 1 -- 172.21.15.83:0/1794042356 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdadc0ab600 con 0x7fdaf00d2690 2024-04-25T00:14:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaefe00640 1 -- 172.21.15.83:0/1794042356 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0ab9d0 con 0x7fdaf00d2690 2024-04-25T00:14:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1794042356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf00d2690 msgr2=0x7fdaf00ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaf6a00640 1 --2- 172.21.15.83:0/1794042356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf00d2690 0x7fdaf00ce580 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0073170 tx=0x7fdadc01c1d0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1794042356 shutdown_connections 2024-04-25T00:14:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaf6a00640 1 --2- 172.21.15.83:0/1794042356 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdaf00ceac0 0x7fdaf00cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaf6a00640 1 --2- 172.21.15.83:0/1794042356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf00d2690 0x7fdaf00ce580 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaf6a00640 1 --2- 172.21.15.83:0/1794042356 >> v2:172.21.15.86:3300/0 conn(0x7fdaf015c0a0 0x7fdaf00cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.595+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1794042356 >> 172.21.15.83:0/1794042356 conn(0x7fdaf00d1b70 msgr2=0x7fdaf00d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1794042356 shutdown_connections 2024-04-25T00:14:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1794042356 wait complete. 2024-04-25T00:14:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf6a00640 1 Processor -- start 2024-04-25T00:14:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf6a00640 1 -- start start 2024-04-25T00:14:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf6a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdaf00ceac0 0x7fdaf00b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf00d2690 0x7fdaf0158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf015c0a0 0x7fdaf015dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf6a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf00aef20 con 0x7fdaf00d2690 2024-04-25T00:14:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf015c690 con 0x7fdaf015c0a0 2024-04-25T00:14:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf6a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf00af0a0 con 0x7fdaf00ceac0 2024-04-25T00:14:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf00d2690 0x7fdaf0158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf00d2690 0x7fdaf0158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44376/0 (socket says 172.21.15.83:44376) 2024-04-25T00:14:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf4c00640 1 -- 172.21.15.83:0/1366805926 learned_addr learned my addr 172.21.15.83:0/1366805926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf6000640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf015c0a0 0x7fdaf015dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf5600640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdaf00ceac0 0x7fdaf00b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf4c00640 1 -- 172.21.15.83:0/1366805926 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdaf00ceac0 msgr2=0x7fdaf00b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf4c00640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdaf00ceac0 0x7fdaf00b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf4c00640 1 -- 172.21.15.83:0/1366805926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf015c0a0 msgr2=0x7fdaf015dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf4c00640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf015c0a0 0x7fdaf015dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf4c00640 1 -- 172.21.15.83:0/1366805926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf0158a20 con 0x7fdaf00d2690 2024-04-25T00:14:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf6000640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf015c0a0 0x7fdaf015dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf4c00640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf00d2690 0x7fdaf0158250 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fdadc01a880 tx=0x7fdadc018f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaed600640 1 -- 172.21.15.83:0/1366805926 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0b9070 con 0x7fdaf00d2690 2024-04-25T00:14:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaed600640 1 -- 172.21.15.83:0/1366805926 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdadc0b4400 con 0x7fdaf00d2690 2024-04-25T00:14:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaed600640 1 -- 172.21.15.83:0/1366805926 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0b0040 con 0x7fdaf00d2690 2024-04-25T00:14:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.599+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1366805926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf0158790 con 0x7fdaf00d2690 2024-04-25T00:14:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.603+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1366805926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf00b6060 con 0x7fdaf00d2690 2024-04-25T00:14:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.603+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1366805926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf00b6230 con 0x7fdaf00d2690 2024-04-25T00:14:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.603+0000 7fdaed600640 1 -- 172.21.15.83:0/1366805926 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdadc0b33a0 con 0x7fdaf00d2690 2024-04-25T00:14:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.603+0000 7fdaed600640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdad00678c0 0x7fdad0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.603+0000 7fdaed600640 1 -- 172.21.15.83:0/1366805926 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7fdadc11f100 con 0x7fdaf00d2690 2024-04-25T00:14:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.603+0000 7fdaf5600640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdad00678c0 0x7fdad0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.603+0000 7fdaf5600640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdad00678c0 0x7fdad0069d80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdae4004430 tx=0x7fdae4003a00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.607+0000 7fdaed600640 1 -- 172.21.15.83:0/1366805926 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdadc016110 con 0x7fdaf00d2690 2024-04-25T00:14:40.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:40.799+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1366805926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdaf004f510 con 0x7fdaf00d2690 2024-04-25T00:14:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.435+0000 7fdaed600640 1 -- 172.21.15.83:0/1366805926 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) ==== 104+0+0 (secure 0 0 0) 0x7fdadc0efa50 con 0x7fdaf00d2690 2024-04-25T00:14:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.435+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1366805926 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdad00678c0 msgr2=0x7fdad0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.435+0000 7fdaf6a00640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdad00678c0 0x7fdad0069d80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdae4004430 tx=0x7fdae4003a00 comp rx=0 tx=0).stop 2024-04-25T00:14:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.439+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1366805926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf00d2690 msgr2=0x7fdaf0158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.439+0000 7fdaf6a00640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf00d2690 0x7fdaf0158250 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fdadc01a880 tx=0x7fdadc018f70 comp rx=0 tx=0).stop 2024-04-25T00:14:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.439+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1366805926 shutdown_connections 2024-04-25T00:14:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.439+0000 7fdaf6a00640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fdad00678c0 0x7fdad0069d80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.439+0000 7fdaf6a00640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf015c0a0 0x7fdaf015dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.439+0000 7fdaf6a00640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaf00d2690 0x7fdaf0158250 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.439+0000 7fdaf6a00640 1 --2- 172.21.15.83:0/1366805926 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdaf00ceac0 0x7fdaf00b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.439+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1366805926 >> 172.21.15.83:0/1366805926 conn(0x7fdaf00d1b70 msgr2=0x7fdaf00b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.439+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1366805926 shutdown_connections 2024-04-25T00:14:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.439+0000 7fdaf6a00640 1 -- 172.21.15.83:0/1366805926 wait complete. 2024-04-25T00:14:41.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:14:41.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.643+0000 7f7c2ce00640 1 Processor -- start 2024-04-25T00:14:41.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.643+0000 7f7c2ce00640 1 -- start start 2024-04-25T00:14:41.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c2ce00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7c281597d0 0x7f7c280cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:41.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c2ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7c280d0010 0x7f7c2815c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c2ce00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c2815c7e0 0x7f7c28070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c2ce00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7c281605c0 con 0x7f7c281597d0 2024-04-25T00:14:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c2ce00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7c2815cff0 con 0x7f7c280d0010 2024-04-25T00:14:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c2ce00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7c2815f800 con 0x7f7c2815c7e0 2024-04-25T00:14:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c27400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7c281597d0 0x7f7c280cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c27400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7c281597d0 0x7f7c280cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44384/0 (socket says 172.21.15.83:44384) 2024-04-25T00:14:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c27400640 1 -- 172.21.15.83:0/4281016712 learned_addr learned my addr 172.21.15.83:0/4281016712 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c27e00640 1 --2- 172.21.15.83:0/4281016712 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c2815c7e0 0x7f7c28070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c1ea00640 1 --2- 172.21.15.83:0/4281016712 >> v2:172.21.15.86:3300/0 conn(0x7f7c280d0010 0x7f7c2815c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c27400640 1 -- 172.21.15.83:0/4281016712 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c2815c7e0 msgr2=0x7f7c28070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c27400640 1 --2- 172.21.15.83:0/4281016712 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c2815c7e0 0x7f7c28070bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c27400640 1 -- 172.21.15.83:0/4281016712 >> v2:172.21.15.86:3300/0 conn(0x7f7c280d0010 msgr2=0x7f7c2815c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c27400640 1 --2- 172.21.15.83:0/4281016712 >> v2:172.21.15.86:3300/0 conn(0x7f7c280d0010 0x7f7c2815c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:41.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.647+0000 7f7c27400640 1 -- 172.21.15.83:0/4281016712 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2815fae0 con 0x7f7c281597d0 2024-04-25T00:14:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c27400640 1 --2- 172.21.15.83:0/4281016712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c281597d0 0x7f7c280cfad0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f7c2815fa10 tx=0x7f7c1801c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817b935faa24c877 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c26a00640 1 -- 172.21.15.83:0/4281016712 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c180ac070 con 0x7f7c281597d0 2024-04-25T00:14:41.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c26a00640 1 -- 172.21.15.83:0/4281016712 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c180073f0 con 0x7f7c281597d0 2024-04-25T00:14:41.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c26a00640 1 -- 172.21.15.83:0/4281016712 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c180077c0 con 0x7f7c281597d0 2024-04-25T00:14:41.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c2ce00640 1 -- 172.21.15.83:0/4281016712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c281597d0 msgr2=0x7f7c280cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c2ce00640 1 --2- 172.21.15.83:0/4281016712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c281597d0 0x7f7c280cfad0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f7c2815fa10 tx=0x7f7c1801c610 comp rx=0 tx=0).stop 2024-04-25T00:14:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c2ce00640 1 -- 172.21.15.83:0/4281016712 shutdown_connections 2024-04-25T00:14:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c2ce00640 1 --2- 172.21.15.83:0/4281016712 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c2815c7e0 0x7f7c28070bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c2ce00640 1 --2- 172.21.15.83:0/4281016712 >> v2:172.21.15.86:3300/0 conn(0x7f7c280d0010 0x7f7c2815c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c2ce00640 1 --2- 172.21.15.83:0/4281016712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c281597d0 0x7f7c280cfad0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c2ce00640 1 -- 172.21.15.83:0/4281016712 >> 172.21.15.83:0/4281016712 conn(0x7f7c2806b3b0 msgr2=0x7f7c2806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c2ce00640 1 -- 172.21.15.83:0/4281016712 shutdown_connections 2024-04-25T00:14:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c2ce00640 1 -- 172.21.15.83:0/4281016712 wait complete. 2024-04-25T00:14:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c2ce00640 1 Processor -- start 2024-04-25T00:14:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.651+0000 7f7c2ce00640 1 -- start start 2024-04-25T00:14:41.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c2ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c280d0010 0x7f7c2814b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:41.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c2ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c281597d0 0x7f7c281482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c2ce00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c2815c7e0 0x7f7c28148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c2ce00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c280b2950 con 0x7f7c280d0010 2024-04-25T00:14:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c2ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2815cdd0 con 0x7f7c281597d0 2024-04-25T00:14:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c2ce00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7c280b2ad0 con 0x7f7c2815c7e0 2024-04-25T00:14:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c27400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c280d0010 0x7f7c2814b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c27400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c280d0010 0x7f7c2814b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44392/0 (socket says 172.21.15.83:44392) 2024-04-25T00:14:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c27400640 1 -- 172.21.15.83:0/1415195883 learned_addr learned my addr 172.21.15.83:0/1415195883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c1ea00640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c281597d0 0x7f7c281482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c27e00640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c2815c7e0 0x7f7c28148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c27400640 1 -- 172.21.15.83:0/1415195883 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c2815c7e0 msgr2=0x7f7c28148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c27400640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c2815c7e0 0x7f7c28148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c27400640 1 -- 172.21.15.83:0/1415195883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c281597d0 msgr2=0x7f7c281482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c27400640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c281597d0 0x7f7c281482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c27400640 1 -- 172.21.15.83:0/1415195883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2815b310 con 0x7f7c280d0010 2024-04-25T00:14:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c1ea00640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c281597d0 0x7f7c281482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c27400640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c280d0010 0x7f7c2814b9e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f7c1801acc0 tx=0x7f7c180193b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c1fe00640 1 -- 172.21.15.83:0/1415195883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c180ac040 con 0x7f7c280d0010 2024-04-25T00:14:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c1fe00640 1 -- 172.21.15.83:0/1415195883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c180bd650 con 0x7f7c280d0010 2024-04-25T00:14:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c1fe00640 1 -- 172.21.15.83:0/1415195883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c18009510 con 0x7f7c280d0010 2024-04-25T00:14:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c2ce00640 1 -- 172.21.15.83:0/1415195883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2815b5f0 con 0x7f7c280d0010 2024-04-25T00:14:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c2ce00640 1 -- 172.21.15.83:0/1415195883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c28152240 con 0x7f7c280d0010 2024-04-25T00:14:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.655+0000 7f7c2ce00640 1 -- 172.21.15.83:0/1415195883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c2804f510 con 0x7f7c280d0010 2024-04-25T00:14:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.659+0000 7f7c1fe00640 1 -- 172.21.15.83:0/1415195883 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7c180af070 con 0x7f7c280d0010 2024-04-25T00:14:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.659+0000 7f7c1fe00640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c040678c0 0x7f7c04069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.659+0000 7f7c1fe00640 1 -- 172.21.15.83:0/1415195883 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f7c181208f0 con 0x7f7c280d0010 2024-04-25T00:14:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.659+0000 7f7c1ea00640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c040678c0 0x7f7c04069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.659+0000 7f7c1ea00640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c040678c0 0x7f7c04069d80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7c10004660 tx=0x7f7c10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:41.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.663+0000 7f7c1fe00640 1 -- 172.21.15.83:0/1415195883 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7c180c4030 con 0x7f7c280d0010 2024-04-25T00:14:41.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:41.819+0000 7f7c2ce00640 1 -- 172.21.15.83:0/1415195883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7c28166840 con 0x7f7c280d0010 2024-04-25T00:14:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.435+0000 7f7c1fe00640 1 -- 172.21.15.83:0/1415195883 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v84) ==== 181+0+0 (secure 0 0 0) 0x7f7c180165e0 con 0x7f7c280d0010 2024-04-25T00:14:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:14:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.439+0000 7f7c2ce00640 1 -- 172.21.15.83:0/1415195883 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c040678c0 msgr2=0x7f7c04069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.439+0000 7f7c2ce00640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c040678c0 0x7f7c04069d80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7c10004660 tx=0x7f7c10008040 comp rx=0 tx=0).stop 2024-04-25T00:14:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.439+0000 7f7c2ce00640 1 -- 172.21.15.83:0/1415195883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c280d0010 msgr2=0x7f7c2814b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.439+0000 7f7c2ce00640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c280d0010 0x7f7c2814b9e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f7c1801acc0 tx=0x7f7c180193b0 comp rx=0 tx=0).stop 2024-04-25T00:14:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.439+0000 7f7c2ce00640 1 -- 172.21.15.83:0/1415195883 shutdown_connections 2024-04-25T00:14:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.439+0000 7f7c2ce00640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7c040678c0 0x7f7c04069d80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.439+0000 7f7c2ce00640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c2815c7e0 0x7f7c28148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.439+0000 7f7c2ce00640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c281597d0 0x7f7c281482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.439+0000 7f7c2ce00640 1 --2- 172.21.15.83:0/1415195883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c280d0010 0x7f7c2814b9e0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.439+0000 7f7c2ce00640 1 -- 172.21.15.83:0/1415195883 >> 172.21.15.83:0/1415195883 conn(0x7f7c2806b3b0 msgr2=0x7f7c280ae160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.439+0000 7f7c2ce00640 1 -- 172.21.15.83:0/1415195883 shutdown_connections 2024-04-25T00:14:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.439+0000 7f7c2ce00640 1 -- 172.21.15.83:0/1415195883 wait complete. 2024-04-25T00:14:42.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:14:42.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.603+0000 7f2e62200640 1 Processor -- start 2024-04-25T00:14:42.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.603+0000 7f2e62200640 1 -- start start 2024-04-25T00:14:42.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.603+0000 7f2e62200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e5c1455d0 0x7f2e5c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:42.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.603+0000 7f2e62200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e5c0ce610 con 0x7f2e5c0d0450 2024-04-25T00:14:42.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.603+0000 7f2e62200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e5c0ce8f0 con 0x7f2e5c0ce080 2024-04-25T00:14:42.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.603+0000 7f2e62200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2e5c0cebd0 con 0x7f2e5c1455d0 2024-04-25T00:14:42.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.603+0000 7f2e5be00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2e5c0d0450 0x7f2e5c0b2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45560/0 (socket says 172.21.15.83:45560) 2024-04-25T00:14:42.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.603+0000 7f2e5be00640 1 -- 172.21.15.83:0/1817964097 learned_addr learned my addr 172.21.15.83:0/1817964097 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:42.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.603+0000 7f2e5b400640 1 --2- 172.21.15.83:0/1817964097 >> v2:172.21.15.86:3300/0 conn(0x7f2e5c1455d0 0x7f2e5c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:42.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.603+0000 7f2e5a000640 1 -- 172.21.15.83:0/1817964097 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2925016582 0 0) 0x7f2e5c0ce610 con 0x7f2e5c0d0450 2024-04-25T00:14:42.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.607+0000 7f2e5a000640 1 -- 172.21.15.83:0/1817964097 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e40003530 con 0x7f2e5c0d0450 2024-04-25T00:14:42.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.607+0000 7f2e5b400640 1 -- 172.21.15.83:0/1817964097 >> v1:172.21.15.86:6789/0 conn(0x7f2e5c0ce080 legacy=0x7f2e5c0cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:42.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.607+0000 7f2e5b400640 1 -- 172.21.15.83:0/1817964097 >> v1:172.21.15.83:6789/0 conn(0x7f2e5c0d0450 legacy=0x7f2e5c0b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:42.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.607+0000 7f2e5b400640 1 -- 172.21.15.83:0/1817964097 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c0ceeb0 con 0x7f2e5c1455d0 2024-04-25T00:14:42.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.607+0000 7f2e5b400640 1 --2- 172.21.15.83:0/1817964097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e5c1455d0 0x7f2e5c06d7e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009d90 tx=0x7f2e4c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c654aa42279c5901 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:42.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.607+0000 7f2e5a000640 1 -- 172.21.15.83:0/1817964097 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0b2070 con 0x7f2e5c1455d0 2024-04-25T00:14:42.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.607+0000 7f2e5a000640 1 -- 172.21.15.83:0/1817964097 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e4c0a5990 con 0x7f2e5c1455d0 2024-04-25T00:14:42.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.607+0000 7f2e5a000640 1 -- 172.21.15.83:0/1817964097 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0a5d60 con 0x7f2e5c1455d0 2024-04-25T00:14:42.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.607+0000 7f2e62200640 1 -- 172.21.15.83:0/1817964097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e5c1455d0 msgr2=0x7f2e5c06d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:42.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.607+0000 7f2e62200640 1 --2- 172.21.15.83:0/1817964097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e5c1455d0 0x7f2e5c06d7e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009d90 tx=0x7f2e4c01c970 comp rx=0 tx=0).stop 2024-04-25T00:14:42.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e62200640 1 -- 172.21.15.83:0/1817964097 shutdown_connections 2024-04-25T00:14:42.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e62200640 1 --2- 172.21.15.83:0/1817964097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e5c1455d0 0x7f2e5c06d7e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:42.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e62200640 1 -- 172.21.15.83:0/1817964097 >> 172.21.15.83:0/1817964097 conn(0x7f2e5c06b3b0 msgr2=0x7f2e5c06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:42.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e62200640 1 -- 172.21.15.83:0/1817964097 shutdown_connections 2024-04-25T00:14:42.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e62200640 1 -- 172.21.15.83:0/1817964097 wait complete. 2024-04-25T00:14:42.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e62200640 1 Processor -- start 2024-04-25T00:14:42.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e62200640 1 -- start start 2024-04-25T00:14:42.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e62200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e5c0ce080 0x7f2e5c072950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:42.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e62200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c0d0450 0x7f2e5c15e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:42.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e62200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c1455d0 0x7f2e5c1534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:42.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e62200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c0b3bd0 con 0x7f2e5c1455d0 2024-04-25T00:14:42.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e62200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c0b3a50 con 0x7f2e5c0ce080 2024-04-25T00:14:42.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e62200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2e5c0b3d50 con 0x7f2e5c0d0450 2024-04-25T00:14:42.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e5be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c1455d0 0x7f2e5c1534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e5be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c1455d0 0x7f2e5c1534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44398/0 (socket says 172.21.15.83:44398) 2024-04-25T00:14:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e5be00640 1 -- 172.21.15.83:0/2117143300 learned_addr learned my addr 172.21.15.83:0/2117143300 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e5b400640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e5c0ce080 0x7f2e5c072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e5aa00640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c0d0450 0x7f2e5c15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e5be00640 1 -- 172.21.15.83:0/2117143300 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c0d0450 msgr2=0x7f2e5c15e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e5be00640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c0d0450 0x7f2e5c15e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e5be00640 1 -- 172.21.15.83:0/2117143300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e5c0ce080 msgr2=0x7f2e5c072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e5be00640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e5c0ce080 0x7f2e5c072950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e5be00640 1 -- 172.21.15.83:0/2117143300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c15e9d0 con 0x7f2e5c1455d0 2024-04-25T00:14:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e5aa00640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c0d0450 0x7f2e5c15e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.611+0000 7f2e5be00640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c1455d0 0x7f2e5c1534e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400c420 tx=0x7f2e5400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.615+0000 7f2e53400640 1 -- 172.21.15.83:0/2117143300 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54017070 con 0x7f2e5c1455d0 2024-04-25T00:14:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.615+0000 7f2e53400640 1 -- 172.21.15.83:0/2117143300 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e54005cf0 con 0x7f2e5c1455d0 2024-04-25T00:14:42.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.615+0000 7f2e53400640 1 -- 172.21.15.83:0/2117143300 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e540072d0 con 0x7f2e5c1455d0 2024-04-25T00:14:42.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.615+0000 7f2e62200640 1 -- 172.21.15.83:0/2117143300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5c0718b0 con 0x7f2e5c1455d0 2024-04-25T00:14:42.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.615+0000 7f2e62200640 1 -- 172.21.15.83:0/2117143300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5c071e70 con 0x7f2e5c1455d0 2024-04-25T00:14:42.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.615+0000 7f2e62200640 1 -- 172.21.15.83:0/2117143300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e5c0736f0 con 0x7f2e5c1455d0 2024-04-25T00:14:42.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.615+0000 7f2e53400640 1 -- 172.21.15.83:0/2117143300 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2e54007470 con 0x7f2e5c1455d0 2024-04-25T00:14:42.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.615+0000 7f2e53400640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e3c0678f0 0x7f2e3c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:42.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.615+0000 7f2e53400640 1 -- 172.21.15.83:0/2117143300 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5749+0+0 (secure 0 0 0) 0x7f2e54084f00 con 0x7f2e5c1455d0 2024-04-25T00:14:42.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.615+0000 7f2e5b400640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e3c0678f0 0x7f2e3c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:42.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.619+0000 7f2e5b400640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e3c0678f0 0x7f2e3c069db0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0af000 tx=0x7f2e4c0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:42.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.619+0000 7f2e53400640 1 -- 172.21.15.83:0/2117143300 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2e54055470 con 0x7f2e5c1455d0 2024-04-25T00:14:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:42.783+0000 7f2e62200640 1 -- 172.21.15.83:0/2117143300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2e5c04f5f0 con 0x7f2e5c1455d0 2024-04-25T00:14:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.443+0000 7f2e53400640 1 -- 172.21.15.83:0/2117143300 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v85) ==== 169+0+0 (secure 0 0 0) 0x7f2e54059770 con 0x7f2e5c1455d0 2024-04-25T00:14:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-25T00:14:43.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.447+0000 7f2e62200640 1 -- 172.21.15.83:0/2117143300 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e3c0678f0 msgr2=0x7f2e3c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.447+0000 7f2e62200640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e3c0678f0 0x7f2e3c069db0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0af000 tx=0x7f2e4c0a7070 comp rx=0 tx=0).stop 2024-04-25T00:14:43.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.447+0000 7f2e62200640 1 -- 172.21.15.83:0/2117143300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c1455d0 msgr2=0x7f2e5c1534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.447+0000 7f2e62200640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c1455d0 0x7f2e5c1534e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400c420 tx=0x7f2e5400c8e0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.447+0000 7f2e62200640 1 -- 172.21.15.83:0/2117143300 shutdown_connections 2024-04-25T00:14:43.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.447+0000 7f2e62200640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2e3c0678f0 0x7f2e3c069db0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.447+0000 7f2e62200640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e5c0d0450 0x7f2e5c15e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.447+0000 7f2e62200640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e5c1455d0 0x7f2e5c1534e0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.447+0000 7f2e62200640 1 --2- 172.21.15.83:0/2117143300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e5c0ce080 0x7f2e5c072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.447+0000 7f2e62200640 1 -- 172.21.15.83:0/2117143300 >> 172.21.15.83:0/2117143300 conn(0x7f2e5c06b3b0 msgr2=0x7f2e5c15b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.447+0000 7f2e62200640 1 -- 172.21.15.83:0/2117143300 shutdown_connections 2024-04-25T00:14:43.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.447+0000 7f2e62200640 1 -- 172.21.15.83:0/2117143300 wait complete. 2024-04-25T00:14:43.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:14:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.615+0000 7fcf3f000640 1 Processor -- start 2024-04-25T00:14:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf3f000640 1 -- start start 2024-04-25T00:14:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf3f000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf380700b0 0x7fcf380cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf3f000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf3815e640 con 0x7fcf380cee70 2024-04-25T00:14:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf3f000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf380d1350 con 0x7fcf3815e260 2024-04-25T00:14:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf3f000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcf380d0d90 con 0x7fcf380700b0 2024-04-25T00:14:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf37400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcf380cee70 0x7fcf380ccc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45564/0 (socket says 172.21.15.83:45564) 2024-04-25T00:14:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf37400640 1 -- 172.21.15.83:0/2991045506 learned_addr learned my addr 172.21.15.83:0/2991045506 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf3d000640 1 --2- 172.21.15.83:0/2991045506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf380700b0 0x7fcf380cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 -- 172.21.15.83:0/2991045506 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 239082034 0 0) 0x7fcf3815e640 con 0x7fcf380cee70 2024-04-25T00:14:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 -- 172.21.15.83:0/2991045506 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf1c003530 con 0x7fcf380cee70 2024-04-25T00:14:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 -- 172.21.15.83:0/2991045506 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 298420625 0 0) 0x7fcf1c003530 con 0x7fcf380cee70 2024-04-25T00:14:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 -- 172.21.15.83:0/2991045506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf380700b0 msgr2=0x7fcf380cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 --2- 172.21.15.83:0/2991045506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf380700b0 0x7fcf380cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 -- 172.21.15.83:0/2991045506 >> v1:172.21.15.86:6789/0 conn(0x7fcf3815e260 legacy=0x7fcf380ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 -- 172.21.15.83:0/2991045506 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf380d1070 con 0x7fcf380cee70 2024-04-25T00:14:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 -- 172.21.15.83:0/2991045506 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fcf240016a0 con 0x7fcf380cee70 2024-04-25T00:14:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 -- 172.21.15.83:0/2991045506 >> v1:172.21.15.83:6789/0 conn(0x7fcf380cee70 legacy=0x7fcf380ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 --2- 172.21.15.83:0/2991045506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf1c003a60 0x7fcf1c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 --2- 172.21.15.83:0/2991045506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1c004810 0x7fcf1c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 --2- 172.21.15.83:0/2991045506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf380700b0 0x7fcf1c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 -- 172.21.15.83:0/2991045506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf380d0d90 con 0x7fcf1c004810 2024-04-25T00:14:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 -- 172.21.15.83:0/2991045506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c00bad0 con 0x7fcf1c003a60 2024-04-25T00:14:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf36a00640 1 -- 172.21.15.83:0/2991045506 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c00bcd0 con 0x7fcf380700b0 2024-04-25T00:14:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf37400640 1 --2- 172.21.15.83:0/2991045506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1c004810 0x7fcf1c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf37e00640 1 --2- 172.21.15.83:0/2991045506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf1c003a60 0x7fcf1c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf3d000640 1 --2- 172.21.15.83:0/2991045506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf380700b0 0x7fcf1c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf37400640 1 -- 172.21.15.83:0/2991045506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf380700b0 msgr2=0x7fcf1c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf37400640 1 --2- 172.21.15.83:0/2991045506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf380700b0 0x7fcf1c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf37400640 1 -- 172.21.15.83:0/2991045506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf1c003a60 msgr2=0x7fcf1c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf37400640 1 --2- 172.21.15.83:0/2991045506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf1c003a60 0x7fcf1c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.619+0000 7fcf37400640 1 -- 172.21.15.83:0/2991045506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c003530 con 0x7fcf1c004810 2024-04-25T00:14:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.623+0000 7fcf37400640 1 --2- 172.21.15.83:0/2991045506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1c004810 0x7fcf1c004c90 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400c290 tx=0x7fcf2401f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88c7cbd7c7475e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.623+0000 7fcf36a00640 1 -- 172.21.15.83:0/2991045506 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24007d60 con 0x7fcf1c004810 2024-04-25T00:14:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.623+0000 7fcf36a00640 1 -- 172.21.15.83:0/2991045506 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf240ac070 con 0x7fcf1c004810 2024-04-25T00:14:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.623+0000 7fcf36a00640 1 -- 172.21.15.83:0/2991045506 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24005b20 con 0x7fcf1c004810 2024-04-25T00:14:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.623+0000 7fcf3f000640 1 -- 172.21.15.83:0/2991045506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1c004810 msgr2=0x7fcf1c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.623+0000 7fcf3f000640 1 --2- 172.21.15.83:0/2991045506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1c004810 0x7fcf1c004c90 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400c290 tx=0x7fcf2401f450 comp rx=0 tx=0).stop 2024-04-25T00:14:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.623+0000 7fcf3d000640 1 -- 172.21.15.83:0/2991045506 reap_dead start 2024-04-25T00:14:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.623+0000 7fcf3f000640 1 -- 172.21.15.83:0/2991045506 shutdown_connections 2024-04-25T00:14:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.623+0000 7fcf3f000640 1 -- 172.21.15.83:0/2991045506 >> 172.21.15.83:0/2991045506 conn(0x7fcf380b3950 msgr2=0x7fcf380b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.623+0000 7fcf3f000640 1 -- 172.21.15.83:0/2991045506 shutdown_connections 2024-04-25T00:14:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.623+0000 7fcf3f000640 1 -- 172.21.15.83:0/2991045506 wait complete. 2024-04-25T00:14:43.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3f000640 1 Processor -- start 2024-04-25T00:14:43.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3f000640 1 -- start start 2024-04-25T00:14:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3f000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf1c004810 0x7fcf38159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf380ca980 0x7fcf380cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3f000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf380b4a40 0x7fcf380b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3f000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf380d0a20 con 0x7fcf380b4a40 2024-04-25T00:14:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf380cc750 con 0x7fcf380ca980 2024-04-25T00:14:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3f000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcf380b4470 con 0x7fcf1c004810 2024-04-25T00:14:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf380b4a40 0x7fcf380b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf380b4a40 0x7fcf380b4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44426/0 (socket says 172.21.15.83:44426) 2024-04-25T00:14:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3d000640 1 -- 172.21.15.83:0/2087137083 learned_addr learned my addr 172.21.15.83:0/2087137083 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf37400640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf380ca980 0x7fcf380cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf37e00640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf1c004810 0x7fcf38159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3d000640 1 -- 172.21.15.83:0/2087137083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf1c004810 msgr2=0x7fcf38159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3d000640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf1c004810 0x7fcf38159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3d000640 1 -- 172.21.15.83:0/2087137083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf380ca980 msgr2=0x7fcf380cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3d000640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf380ca980 0x7fcf380cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3d000640 1 -- 172.21.15.83:0/2087137083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf380c98e0 con 0x7fcf380b4a40 2024-04-25T00:14:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf37e00640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf1c004810 0x7fcf38159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3d000640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf380b4a40 0x7fcf380b4ed0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fcf38071020 tx=0x7fcf3000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf2fe00640 1 -- 172.21.15.83:0/2087137083 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30019070 con 0x7fcf380b4a40 2024-04-25T00:14:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf2fe00640 1 -- 172.21.15.83:0/2087137083 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf300039a0 con 0x7fcf380b4a40 2024-04-25T00:14:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf2fe00640 1 -- 172.21.15.83:0/2087137083 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30003d90 con 0x7fcf380b4a40 2024-04-25T00:14:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3f000640 1 -- 172.21.15.83:0/2087137083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf380c9bc0 con 0x7fcf380b4a40 2024-04-25T00:14:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3f000640 1 -- 172.21.15.83:0/2087137083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf380ca180 con 0x7fcf380b4a40 2024-04-25T00:14:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.627+0000 7fcf3f000640 1 -- 172.21.15.83:0/2087137083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3804f510 con 0x7fcf380b4a40 2024-04-25T00:14:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.631+0000 7fcf2fe00640 1 -- 172.21.15.83:0/2087137083 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcf3001c070 con 0x7fcf380b4a40 2024-04-25T00:14:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.631+0000 7fcf2fe00640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcf180678f0 0x7fcf18069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.631+0000 7fcf2fe00640 1 -- 172.21.15.83:0/2087137083 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5335+0+0 (secure 0 0 0) 0x7fcf300865b0 con 0x7fcf380b4a40 2024-04-25T00:14:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.631+0000 7fcf37e00640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcf180678f0 0x7fcf18069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.631+0000 7fcf37e00640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcf180678f0 0x7fcf18069db0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcf28000c10 tx=0x7fcf28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:43.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.635+0000 7fcf2fe00640 1 -- 172.21.15.83:0/2087137083 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcf30010040 con 0x7fcf380b4a40 2024-04-25T00:14:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.791+0000 7fcf3f000640 1 -- 172.21.15.83:0/2087137083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fcf3815a9a0 con 0x7fcf380b4a40 2024-04-25T00:14:43.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.795+0000 7fcf2fe00640 1 -- 172.21.15.83:0/2087137083 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7fcf30057250 con 0x7fcf380b4a40 2024-04-25T00:14:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.795+0000 7fcf3f000640 1 -- 172.21.15.83:0/2087137083 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcf180678f0 msgr2=0x7fcf18069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.795+0000 7fcf3f000640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcf180678f0 0x7fcf18069db0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcf28000c10 tx=0x7fcf28009040 comp rx=0 tx=0).stop 2024-04-25T00:14:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.795+0000 7fcf3f000640 1 -- 172.21.15.83:0/2087137083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf380b4a40 msgr2=0x7fcf380b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.795+0000 7fcf3f000640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf380b4a40 0x7fcf380b4ed0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fcf38071020 tx=0x7fcf3000f580 comp rx=0 tx=0).stop 2024-04-25T00:14:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.799+0000 7fcf3f000640 1 -- 172.21.15.83:0/2087137083 shutdown_connections 2024-04-25T00:14:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.799+0000 7fcf3f000640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcf180678f0 0x7fcf18069db0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.799+0000 7fcf3f000640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf380b4a40 0x7fcf380b4ed0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.799+0000 7fcf3f000640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf380ca980 0x7fcf380cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.799+0000 7fcf3f000640 1 --2- 172.21.15.83:0/2087137083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf1c004810 0x7fcf38159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.799+0000 7fcf3f000640 1 -- 172.21.15.83:0/2087137083 >> 172.21.15.83:0/2087137083 conn(0x7fcf380b3950 msgr2=0x7fcf380cd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:43.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.799+0000 7fcf3f000640 1 -- 172.21.15.83:0/2087137083 shutdown_connections 2024-04-25T00:14:43.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.799+0000 7fcf3f000640 1 -- 172.21.15.83:0/2087137083 wait complete. 2024-04-25T00:14:43.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop' 2024-04-25T00:14:43.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.951+0000 7f9089600640 1 Processor -- start 2024-04-25T00:14:43.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.951+0000 7f9089600640 1 -- start start 2024-04-25T00:14:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.951+0000 7f9089600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f908415df00 0x7f908406d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.951+0000 7f9089600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f908406e7b0 0x7f90840716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.951+0000 7f9089600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f908406e0c0 con 0x7f9084071bf0 2024-04-25T00:14:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.951+0000 7f9089600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9084072060 con 0x7f908415df00 2024-04-25T00:14:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.951+0000 7f9089600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f908406f150 con 0x7f908406e7b0 2024-04-25T00:14:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.951+0000 7f9082a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f908415df00 0x7f908406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.951+0000 7f9082a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f908415df00 0x7f908406d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44438/0 (socket says 172.21.15.83:44438) 2024-04-25T00:14:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.951+0000 7f9082a00640 1 -- 172.21.15.83:0/682104703 learned_addr learned my addr 172.21.15.83:0/682104703 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.951+0000 7f9082a00640 1 -- 172.21.15.83:0/682104703 >> v2:172.21.15.86:3300/0 conn(0x7f908406e7b0 msgr2=0x7f90840716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.951+0000 7f907a000640 1 --2- 172.21.15.83:0/682104703 >> v2:172.21.15.86:3300/0 conn(0x7f908406e7b0 0x7f90840716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9082a00640 1 --2- 172.21.15.83:0/682104703 >> v2:172.21.15.86:3300/0 conn(0x7f908406e7b0 0x7f90840716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9082a00640 1 -- 172.21.15.83:0/682104703 >> v1:172.21.15.86:6789/0 conn(0x7f9084071bf0 legacy=0x7f9084161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:43.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9082a00640 1 -- 172.21.15.83:0/682104703 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90840ad990 con 0x7f908415df00 2024-04-25T00:14:43.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f907a000640 1 --2- 172.21.15.83:0/682104703 >> v2:172.21.15.86:3300/0 conn(0x7f908406e7b0 0x7f90840716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9082a00640 1 --2- 172.21.15.83:0/682104703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908415df00 0x7f908406d970 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f90840705c0 tx=0x7f907401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7413948392d1f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9082000640 1 -- 172.21.15.83:0/682104703 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90740a5580 con 0x7f908415df00 2024-04-25T00:14:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9082000640 1 -- 172.21.15.83:0/682104703 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90740a5720 con 0x7f908415df00 2024-04-25T00:14:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9082000640 1 -- 172.21.15.83:0/682104703 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90740a5af0 con 0x7f908415df00 2024-04-25T00:14:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9089600640 1 -- 172.21.15.83:0/682104703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908415df00 msgr2=0x7f908406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9089600640 1 --2- 172.21.15.83:0/682104703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908415df00 0x7f908406d970 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f90840705c0 tx=0x7f907401c890 comp rx=0 tx=0).stop 2024-04-25T00:14:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9089600640 1 -- 172.21.15.83:0/682104703 shutdown_connections 2024-04-25T00:14:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9089600640 1 --2- 172.21.15.83:0/682104703 >> v2:172.21.15.86:3300/0 conn(0x7f908406e7b0 0x7f90840716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9089600640 1 --2- 172.21.15.83:0/682104703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908415df00 0x7f908406d970 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9089600640 1 -- 172.21.15.83:0/682104703 >> 172.21.15.83:0/682104703 conn(0x7f90840b1830 msgr2=0x7f90840b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9089600640 1 -- 172.21.15.83:0/682104703 shutdown_connections 2024-04-25T00:14:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.955+0000 7f9089600640 1 -- 172.21.15.83:0/682104703 wait complete. 2024-04-25T00:14:43.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9089600640 1 Processor -- start 2024-04-25T00:14:43.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9089600640 1 -- start start 2024-04-25T00:14:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9089600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908406e7b0 0x7f908414a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9089600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9084071bf0 0x7f908415e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9089600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f908415df00 0x7f908414b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9089600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f90840b3120 con 0x7f908406e7b0 2024-04-25T00:14:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9089600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f908406e3a0 con 0x7f908415df00 2024-04-25T00:14:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9089600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f90840b32a0 con 0x7f9084071bf0 2024-04-25T00:14:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9082a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908406e7b0 0x7f908414a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9082a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908406e7b0 0x7f908414a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44446/0 (socket says 172.21.15.83:44446) 2024-04-25T00:14:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9082a00640 1 -- 172.21.15.83:0/2416900775 learned_addr learned my addr 172.21.15.83:0/2416900775 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9083400640 1 --2- 172.21.15.83:0/2416900775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f908415df00 0x7f908414b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f907a000640 1 --2- 172.21.15.83:0/2416900775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9084071bf0 0x7f908415e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9082a00640 1 -- 172.21.15.83:0/2416900775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9084071bf0 msgr2=0x7f908415e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9082a00640 1 --2- 172.21.15.83:0/2416900775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9084071bf0 0x7f908415e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9082a00640 1 -- 172.21.15.83:0/2416900775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f908415df00 msgr2=0x7f908414b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9082a00640 1 --2- 172.21.15.83:0/2416900775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f908415df00 0x7f908414b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9082a00640 1 -- 172.21.15.83:0/2416900775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9084161400 con 0x7f908406e7b0 2024-04-25T00:14:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9082a00640 1 --2- 172.21.15.83:0/2416900775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908406e7b0 0x7f908414a110 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f90740af040 tx=0x7f9074019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f907b400640 1 -- 172.21.15.83:0/2416900775 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90740bc020 con 0x7f908406e7b0 2024-04-25T00:14:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f907b400640 1 -- 172.21.15.83:0/2416900775 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90740b8690 con 0x7f908406e7b0 2024-04-25T00:14:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f907b400640 1 -- 172.21.15.83:0/2416900775 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90740b8a80 con 0x7f908406e7b0 2024-04-25T00:14:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9089600640 1 -- 172.21.15.83:0/2416900775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908414a950 con 0x7f908406e7b0 2024-04-25T00:14:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.959+0000 7f9089600640 1 -- 172.21.15.83:0/2416900775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908414abf0 con 0x7f908406e7b0 2024-04-25T00:14:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.963+0000 7f9089600640 1 -- 172.21.15.83:0/2416900775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90840737f0 con 0x7f908406e7b0 2024-04-25T00:14:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.963+0000 7f907b400640 1 -- 172.21.15.83:0/2416900775 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f90740bf020 con 0x7f908406e7b0 2024-04-25T00:14:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.963+0000 7f907b400640 1 --2- 172.21.15.83:0/2416900775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f90600678c0 0x7f9060069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.963+0000 7f907b400640 1 -- 172.21.15.83:0/2416900775 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5335+0+0 (secure 0 0 0) 0x7f90741209d0 con 0x7f908406e7b0 2024-04-25T00:14:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.963+0000 7f907a000640 1 --2- 172.21.15.83:0/2416900775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f90600678c0 0x7f9060069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.963+0000 7f907a000640 1 --2- 172.21.15.83:0/2416900775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f90600678c0 0x7f9060069d80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f906c006fd0 tx=0x7f906c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:43.967+0000 7f907b400640 1 -- 172.21.15.83:0/2416900775 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f90740c6030 con 0x7f908406e7b0 2024-04-25T00:14:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.187+0000 7f9089600640 1 -- 172.21.15.83:0/2416900775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]} v 0) -- 0x7f908404f5f0 con 0x7f908406e7b0 2024-04-25T00:14:44.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.443+0000 7f907b400640 1 -- 172.21.15.83:0/2416900775 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]}]=0 v238) ==== 145+0+0 (secure 0 0 0) 0x7f90740165a0 con 0x7f908406e7b0 2024-04-25T00:14:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.447+0000 7f9089600640 1 -- 172.21.15.83:0/2416900775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f90600678c0 msgr2=0x7f9060069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.447+0000 7f9089600640 1 --2- 172.21.15.83:0/2416900775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f90600678c0 0x7f9060069d80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f906c006fd0 tx=0x7f906c008040 comp rx=0 tx=0).stop 2024-04-25T00:14:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.447+0000 7f9089600640 1 -- 172.21.15.83:0/2416900775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908406e7b0 msgr2=0x7f908414a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.447+0000 7f9089600640 1 --2- 172.21.15.83:0/2416900775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908406e7b0 0x7f908414a110 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f90740af040 tx=0x7f9074019630 comp rx=0 tx=0).stop 2024-04-25T00:14:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.447+0000 7f9089600640 1 -- 172.21.15.83:0/2416900775 shutdown_connections 2024-04-25T00:14:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.447+0000 7f9089600640 1 --2- 172.21.15.83:0/2416900775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f90600678c0 0x7f9060069d80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.447+0000 7f9089600640 1 --2- 172.21.15.83:0/2416900775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9084071bf0 0x7f908415e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.447+0000 7f9089600640 1 --2- 172.21.15.83:0/2416900775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f908415df00 0x7f908414b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.447+0000 7f9089600640 1 --2- 172.21.15.83:0/2416900775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908406e7b0 0x7f908414a110 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.447+0000 7f9089600640 1 -- 172.21.15.83:0/2416900775 >> 172.21.15.83:0/2416900775 conn(0x7f90840b1830 msgr2=0x7f9084156b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.447+0000 7f9089600640 1 -- 172.21.15.83:0/2416900775 shutdown_connections 2024-04-25T00:14:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.447+0000 7f9089600640 1 -- 172.21.15.83:0/2416900775 wait complete. 2024-04-25T00:14:44.471 INFO:tasks.cephfs_test_runner:That when a `tell` command is sent using the python CLI, ... ok 2024-04-25T00:14:44.471 INFO:tasks.cephfs_test_runner:Starting test: test_version_splitting (tasks.cephfs.test_sessionmap.TestSessionMap) That when many sessions are updated, they are correctly 2024-04-25T00:14:44.472 INFO:tasks.cephfs_test_runner:test_version_splitting (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-04-25T00:14:44.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f433d600640 1 Processor -- start 2024-04-25T00:14:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f433d600640 1 -- start start 2024-04-25T00:14:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f433d600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f43381599f0 0x7f43380cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f433d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f433815ce90 0x7f43380ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f433d600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43380ccc60 con 0x7f433815c340 2024-04-25T00:14:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f433d600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4338159dd0 con 0x7f433815ce90 2024-04-25T00:14:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f433d600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f433806dba0 con 0x7f43381599f0 2024-04-25T00:14:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f4336a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f43381599f0 0x7f43380cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f4337400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f433815ce90 0x7f43380ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f4336a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f43381599f0 0x7f43380cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:36414/0 (socket says 172.21.15.83:36414) 2024-04-25T00:14:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f4336a00640 1 -- 172.21.15.83:0/3763958218 learned_addr learned my addr 172.21.15.83:0/3763958218 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f4336a00640 1 -- 172.21.15.83:0/3763958218 >> v2:172.21.15.86:3300/0 conn(0x7f433815ce90 msgr2=0x7f43380ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f4336a00640 1 --2- 172.21.15.83:0/3763958218 >> v2:172.21.15.86:3300/0 conn(0x7f433815ce90 0x7f43380ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f4336a00640 1 -- 172.21.15.83:0/3763958218 >> v1:172.21.15.86:6789/0 conn(0x7f433815c340 legacy=0x7f433815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.607+0000 7f4336a00640 1 -- 172.21.15.83:0/3763958218 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433806de80 con 0x7f43381599f0 2024-04-25T00:14:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.611+0000 7f4336a00640 1 --2- 172.21.15.83:0/3763958218 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f43381599f0 0x7f43380cc510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f43280098a0 tx=0x7f432801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57f86f7c0ede0ead server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.611+0000 7f4336000640 1 -- 172.21.15.83:0/3763958218 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43280b1070 con 0x7f43381599f0 2024-04-25T00:14:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.611+0000 7f4336000640 1 -- 172.21.15.83:0/3763958218 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43280ac620 con 0x7f43381599f0 2024-04-25T00:14:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.611+0000 7f4336000640 1 -- 172.21.15.83:0/3763958218 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43280ac9f0 con 0x7f43381599f0 2024-04-25T00:14:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.611+0000 7f433d600640 1 -- 172.21.15.83:0/3763958218 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f43381599f0 msgr2=0x7f43380cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.611+0000 7f433d600640 1 --2- 172.21.15.83:0/3763958218 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f43381599f0 0x7f43380cc510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f43280098a0 tx=0x7f432801c4b0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.611+0000 7f433d600640 1 -- 172.21.15.83:0/3763958218 shutdown_connections 2024-04-25T00:14:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.611+0000 7f433d600640 1 --2- 172.21.15.83:0/3763958218 >> v2:172.21.15.86:3300/0 conn(0x7f433815ce90 0x7f43380ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.611+0000 7f433d600640 1 --2- 172.21.15.83:0/3763958218 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f43381599f0 0x7f43380cc510 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.611+0000 7f433d600640 1 -- 172.21.15.83:0/3763958218 >> 172.21.15.83:0/3763958218 conn(0x7f43380b1830 msgr2=0x7f43380b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:44.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.611+0000 7f433d600640 1 -- 172.21.15.83:0/3763958218 shutdown_connections 2024-04-25T00:14:44.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.611+0000 7f433d600640 1 -- 172.21.15.83:0/3763958218 wait complete. 2024-04-25T00:14:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f433d600640 1 Processor -- start 2024-04-25T00:14:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f433d600640 1 -- start start 2024-04-25T00:14:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f433d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43381599f0 0x7f433814c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f433d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f433815c340 0x7f43380b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f433d600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f433815ce90 0x7f4338166010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f433d600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43380cf7c0 con 0x7f43381599f0 2024-04-25T00:14:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f433d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43380cf640 con 0x7f433815c340 2024-04-25T00:14:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f433d600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4338168cd0 con 0x7f433815ce90 2024-04-25T00:14:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f4336a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43381599f0 0x7f433814c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f4336a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43381599f0 0x7f433814c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38412/0 (socket says 172.21.15.83:38412) 2024-04-25T00:14:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f4336a00640 1 -- 172.21.15.83:0/3712729875 learned_addr learned my addr 172.21.15.83:0/3712729875 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f432e000640 1 --2- 172.21.15.83:0/3712729875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f433815c340 0x7f43380b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f4337400640 1 --2- 172.21.15.83:0/3712729875 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f433815ce90 0x7f4338166010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f4336a00640 1 -- 172.21.15.83:0/3712729875 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f433815ce90 msgr2=0x7f4338166010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f4336a00640 1 --2- 172.21.15.83:0/3712729875 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f433815ce90 0x7f4338166010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f4336a00640 1 -- 172.21.15.83:0/3712729875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f433815c340 msgr2=0x7f43380b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f4336a00640 1 --2- 172.21.15.83:0/3712729875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f433815c340 0x7f43380b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f4336a00640 1 -- 172.21.15.83:0/3712729875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43380b3e90 con 0x7f43381599f0 2024-04-25T00:14:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f4336a00640 1 --2- 172.21.15.83:0/3712729875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43381599f0 0x7f433814c560 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f432801ab60 tx=0x7f432801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f432f400640 1 -- 172.21.15.83:0/3712729875 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43280b1040 con 0x7f43381599f0 2024-04-25T00:14:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f432f400640 1 -- 172.21.15.83:0/3712729875 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43280bd070 con 0x7f43381599f0 2024-04-25T00:14:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f432f400640 1 -- 172.21.15.83:0/3712729875 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43280b7660 con 0x7f43381599f0 2024-04-25T00:14:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f433d600640 1 -- 172.21.15.83:0/3712729875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43380b4170 con 0x7f43381599f0 2024-04-25T00:14:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f433d600640 1 -- 172.21.15.83:0/3712729875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43380b4730 con 0x7f43381599f0 2024-04-25T00:14:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.615+0000 7f433d600640 1 -- 172.21.15.83:0/3712729875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43380737f0 con 0x7f43381599f0 2024-04-25T00:14:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.619+0000 7f432f400640 1 -- 172.21.15.83:0/3712729875 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f43280b7800 con 0x7f43381599f0 2024-04-25T00:14:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.619+0000 7f432f400640 1 --2- 172.21.15.83:0/3712729875 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f43140678f0 0x7f4314069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.619+0000 7f432f400640 1 -- 172.21.15.83:0/3712729875 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5335+0+0 (secure 0 0 0) 0x7f432811fed0 con 0x7f43381599f0 2024-04-25T00:14:44.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.619+0000 7f432e000640 1 --2- 172.21.15.83:0/3712729875 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f43140678f0 0x7f4314069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:44.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.619+0000 7f432e000640 1 --2- 172.21.15.83:0/3712729875 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f43140678f0 0x7f4314069db0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f43380b4e10 tx=0x7f4320009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.623+0000 7f432f400640 1 -- 172.21.15.83:0/3712729875 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4328016130 con 0x7f43381599f0 2024-04-25T00:14:44.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.779+0000 7f433d600640 1 -- 172.21.15.83:0/3712729875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f433804f5f0 con 0x7f43381599f0 2024-04-25T00:14:44.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.779+0000 7f432f400640 1 -- 172.21.15.83:0/3712729875 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+13208 (secure 0 0 0) 0x7f43280f0b70 con 0x7f43381599f0 2024-04-25T00:14:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:44.783 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":85,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:43.442779+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6836/121840722":"2024-04-26T00:14:40.430811+0000","172.21.15.83:6834/121840722":"2024-04-26T00:14:40.430811+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.779+0000 7f433d600640 1 -- 172.21.15.83:0/3712729875 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f43140678f0 msgr2=0x7f4314069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.779+0000 7f433d600640 1 --2- 172.21.15.83:0/3712729875 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f43140678f0 0x7f4314069db0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f43380b4e10 tx=0x7f4320009040 comp rx=0 tx=0).stop 2024-04-25T00:14:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.779+0000 7f433d600640 1 -- 172.21.15.83:0/3712729875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43381599f0 msgr2=0x7f433814c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.779+0000 7f433d600640 1 --2- 172.21.15.83:0/3712729875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43381599f0 0x7f433814c560 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f432801ab60 tx=0x7f432801acf0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.779+0000 7f433d600640 1 -- 172.21.15.83:0/3712729875 shutdown_connections 2024-04-25T00:14:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.779+0000 7f433d600640 1 --2- 172.21.15.83:0/3712729875 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f43140678f0 0x7f4314069db0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.779+0000 7f433d600640 1 --2- 172.21.15.83:0/3712729875 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f433815ce90 0x7f4338166010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.779+0000 7f433d600640 1 --2- 172.21.15.83:0/3712729875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f433815c340 0x7f43380b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.779+0000 7f433d600640 1 --2- 172.21.15.83:0/3712729875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43381599f0 0x7f433814c560 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.779+0000 7f433d600640 1 -- 172.21.15.83:0/3712729875 >> 172.21.15.83:0/3712729875 conn(0x7f43380b1830 msgr2=0x7f43380af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.783+0000 7f433d600640 1 -- 172.21.15.83:0/3712729875 shutdown_connections 2024-04-25T00:14:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.783+0000 7f433d600640 1 -- 172.21.15.83:0/3712729875 wait complete. 2024-04-25T00:14:44.804 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:14:44.804 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:14:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d48e00640 1 Processor -- start 2024-04-25T00:14:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d48e00640 1 -- start start 2024-04-25T00:14:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d48e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d441555c0 0x7f4d4415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d48e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4d4406e4c0 0x7f4d4406e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d48e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d4415c560 con 0x7f4d441551e0 2024-04-25T00:14:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d48e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4d4406ee70 con 0x7f4d4406e4c0 2024-04-25T00:14:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d48e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4d44072460 con 0x7f4d441555c0 2024-04-25T00:14:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4d441551e0 0x7f4d44071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54714/0 (socket says 172.21.15.83:54714) 2024-04-25T00:14:44.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41e00640 1 -- 172.21.15.83:0/3627846211 learned_addr learned my addr 172.21.15.83:0/3627846211 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:44.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d42800640 1 --2- 172.21.15.83:0/3627846211 >> v2:172.21.15.86:3300/0 conn(0x7f4d4406e4c0 0x7f4d4406e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:44.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d39400640 1 --2- 172.21.15.83:0/3627846211 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d441555c0 0x7f4d4415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 -- 172.21.15.83:0/3627846211 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3684233602 0 0) 0x7f4d4415c560 con 0x7f4d441551e0 2024-04-25T00:14:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 -- 172.21.15.83:0/3627846211 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d28003530 con 0x7f4d441551e0 2024-04-25T00:14:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 -- 172.21.15.83:0/3627846211 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 559132949 0 0) 0x7f4d28003530 con 0x7f4d441551e0 2024-04-25T00:14:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 -- 172.21.15.83:0/3627846211 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d441555c0 msgr2=0x7f4d4415bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 --2- 172.21.15.83:0/3627846211 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d441555c0 0x7f4d4415bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 -- 172.21.15.83:0/3627846211 >> v2:172.21.15.86:3300/0 conn(0x7f4d4406e4c0 msgr2=0x7f4d4406e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 --2- 172.21.15.83:0/3627846211 >> v2:172.21.15.86:3300/0 conn(0x7f4d4406e4c0 0x7f4d4406e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 -- 172.21.15.83:0/3627846211 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4415c7d0 con 0x7f4d441551e0 2024-04-25T00:14:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 -- 172.21.15.83:0/3627846211 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f4d34001680 con 0x7f4d441551e0 2024-04-25T00:14:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 -- 172.21.15.83:0/3627846211 >> v1:172.21.15.83:6789/0 conn(0x7f4d441551e0 legacy=0x7f4d44071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 --2- 172.21.15.83:0/3627846211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28003a30 0x7f4d28003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 --2- 172.21.15.83:0/3627846211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d28004810 0x7f4d28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 --2- 172.21.15.83:0/3627846211 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d441555c0 0x7f4d280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 -- 172.21.15.83:0/3627846211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4406ee70 con 0x7f4d28003a30 2024-04-25T00:14:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 -- 172.21.15.83:0/3627846211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d44072460 con 0x7f4d28004810 2024-04-25T00:14:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41400640 1 -- 172.21.15.83:0/3627846211 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2800b9d0 con 0x7f4d441555c0 2024-04-25T00:14:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41e00640 1 --2- 172.21.15.83:0/3627846211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28003a30 0x7f4d28003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d42800640 1 --2- 172.21.15.83:0/3627846211 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d441555c0 0x7f4d280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d39400640 1 --2- 172.21.15.83:0/3627846211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d28004810 0x7f4d28004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41e00640 1 -- 172.21.15.83:0/3627846211 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d441555c0 msgr2=0x7f4d280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41e00640 1 --2- 172.21.15.83:0/3627846211 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d441555c0 0x7f4d280051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41e00640 1 -- 172.21.15.83:0/3627846211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d28004810 msgr2=0x7f4d28004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41e00640 1 --2- 172.21.15.83:0/3627846211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d28004810 0x7f4d28004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.939+0000 7f4d41e00640 1 -- 172.21.15.83:0/3627846211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d28003530 con 0x7f4d28003a30 2024-04-25T00:14:44.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.943+0000 7f4d41e00640 1 --2- 172.21.15.83:0/3627846211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28003a30 0x7f4d28003e50 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400c2f0 tx=0x7f4d3401cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddf1f19ef6d11a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:44.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.943+0000 7f4d41400640 1 -- 172.21.15.83:0/3627846211 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34007d60 con 0x7f4d28003a30 2024-04-25T00:14:44.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.943+0000 7f4d41400640 1 -- 172.21.15.83:0/3627846211 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d340a9020 con 0x7f4d28003a30 2024-04-25T00:14:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.943+0000 7f4d41400640 1 -- 172.21.15.83:0/3627846211 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34005ae0 con 0x7f4d28003a30 2024-04-25T00:14:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.943+0000 7f4d48e00640 1 -- 172.21.15.83:0/3627846211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28003a30 msgr2=0x7f4d28003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.943+0000 7f4d48e00640 1 --2- 172.21.15.83:0/3627846211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28003a30 0x7f4d28003e50 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400c2f0 tx=0x7f4d3401cd40 comp rx=0 tx=0).stop 2024-04-25T00:14:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.943+0000 7f4d42800640 1 -- 172.21.15.83:0/3627846211 reap_dead start 2024-04-25T00:14:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.943+0000 7f4d48e00640 1 -- 172.21.15.83:0/3627846211 shutdown_connections 2024-04-25T00:14:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.943+0000 7f4d48e00640 1 -- 172.21.15.83:0/3627846211 >> 172.21.15.83:0/3627846211 conn(0x7f4d440b1830 msgr2=0x7f4d440b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.943+0000 7f4d48e00640 1 -- 172.21.15.83:0/3627846211 shutdown_connections 2024-04-25T00:14:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.943+0000 7f4d48e00640 1 -- 172.21.15.83:0/3627846211 wait complete. 2024-04-25T00:14:44.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d48e00640 1 Processor -- start 2024-04-25T00:14:44.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d48e00640 1 -- start start 2024-04-25T00:14:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d48e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28003a30 0x7f4d44161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d48e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d4415a010 0x7f4d44143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d48e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d4415a490 0x7f4d441443c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d48e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4415db70 con 0x7f4d28003a30 2024-04-25T00:14:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d48e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4415d9f0 con 0x7f4d4415a490 2024-04-25T00:14:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d48e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4d4415dcf0 con 0x7f4d4415a010 2024-04-25T00:14:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d41e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28003a30 0x7f4d44161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d41e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28003a30 0x7f4d44161300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38426/0 (socket says 172.21.15.83:38426) 2024-04-25T00:14:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d41e00640 1 -- 172.21.15.83:0/3715653396 learned_addr learned my addr 172.21.15.83:0/3715653396 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d41e00640 1 -- 172.21.15.83:0/3715653396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d4415a010 msgr2=0x7f4d44143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d41e00640 1 --2- 172.21.15.83:0/3715653396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d4415a010 0x7f4d44143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d41e00640 1 -- 172.21.15.83:0/3715653396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d4415a490 msgr2=0x7f4d441443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d41e00640 1 --2- 172.21.15.83:0/3715653396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d4415a490 0x7f4d441443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d41e00640 1 -- 172.21.15.83:0/3715653396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d441e8b20 con 0x7f4d28003a30 2024-04-25T00:14:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d41e00640 1 --2- 172.21.15.83:0/3715653396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28003a30 0x7f4d44161300 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f4d340ab040 tx=0x7f4d28005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d3aa00640 1 -- 172.21.15.83:0/3715653396 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d340049f0 con 0x7f4d28003a30 2024-04-25T00:14:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d3aa00640 1 -- 172.21.15.83:0/3715653396 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d34004b90 con 0x7f4d28003a30 2024-04-25T00:14:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d3aa00640 1 -- 172.21.15.83:0/3715653396 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d340070d0 con 0x7f4d28003a30 2024-04-25T00:14:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d48e00640 1 -- 172.21.15.83:0/3715653396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d441e8e00 con 0x7f4d28003a30 2024-04-25T00:14:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.947+0000 7f4d48e00640 1 -- 172.21.15.83:0/3715653396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d441e93c0 con 0x7f4d28003a30 2024-04-25T00:14:44.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.951+0000 7f4d48e00640 1 -- 172.21.15.83:0/3715653396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d4404f510 con 0x7f4d28003a30 2024-04-25T00:14:44.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.951+0000 7f4d3aa00640 1 -- 172.21.15.83:0/3715653396 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4d34007270 con 0x7f4d28003a30 2024-04-25T00:14:44.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.951+0000 7f4d3aa00640 1 --2- 172.21.15.83:0/3715653396 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4d200678c0 0x7f4d20069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:44.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.951+0000 7f4d3aa00640 1 -- 172.21.15.83:0/3715653396 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5335+0+0 (secure 0 0 0) 0x7f4d3411ecd0 con 0x7f4d28003a30 2024-04-25T00:14:44.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.951+0000 7f4d39400640 1 --2- 172.21.15.83:0/3715653396 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4d200678c0 0x7f4d20069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:44.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.951+0000 7f4d39400640 1 --2- 172.21.15.83:0/3715653396 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4d200678c0 0x7f4d20069d80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4d441549c0 tx=0x7f4d2c009340 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:44.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:44.955+0000 7f4d3aa00640 1 -- 172.21.15.83:0/3715653396 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4d340bf030 con 0x7f4d28003a30 2024-04-25T00:14:45.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.111+0000 7f4d48e00640 1 -- 172.21.15.83:0/3715653396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4d440cf3c0 con 0x7f4d28003a30 2024-04-25T00:14:45.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.111+0000 7f4d3aa00640 1 -- 172.21.15.83:0/3715653396 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f4d34016a50 con 0x7f4d28003a30 2024-04-25T00:14:45.115 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:14:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.115+0000 7f4d48e00640 1 -- 172.21.15.83:0/3715653396 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4d200678c0 msgr2=0x7f4d20069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.115+0000 7f4d48e00640 1 --2- 172.21.15.83:0/3715653396 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4d200678c0 0x7f4d20069d80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4d441549c0 tx=0x7f4d2c009340 comp rx=0 tx=0).stop 2024-04-25T00:14:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.115+0000 7f4d48e00640 1 -- 172.21.15.83:0/3715653396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28003a30 msgr2=0x7f4d44161300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.115+0000 7f4d48e00640 1 --2- 172.21.15.83:0/3715653396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28003a30 0x7f4d44161300 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f4d340ab040 tx=0x7f4d28005840 comp rx=0 tx=0).stop 2024-04-25T00:14:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.115+0000 7f4d48e00640 1 -- 172.21.15.83:0/3715653396 shutdown_connections 2024-04-25T00:14:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.115+0000 7f4d48e00640 1 --2- 172.21.15.83:0/3715653396 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4d200678c0 0x7f4d20069d80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.115+0000 7f4d48e00640 1 --2- 172.21.15.83:0/3715653396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d4415a010 0x7f4d44143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.115+0000 7f4d48e00640 1 --2- 172.21.15.83:0/3715653396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d4415a490 0x7f4d441443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.115+0000 7f4d48e00640 1 --2- 172.21.15.83:0/3715653396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28003a30 0x7f4d44161300 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.115+0000 7f4d48e00640 1 -- 172.21.15.83:0/3715653396 >> 172.21.15.83:0/3715653396 conn(0x7f4d440b1830 msgr2=0x7f4d440b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.115+0000 7f4d48e00640 1 -- 172.21.15.83:0/3715653396 shutdown_connections 2024-04-25T00:14:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.115+0000 7f4d48e00640 1 -- 172.21.15.83:0/3715653396 wait complete. 2024-04-25T00:14:45.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting' 2024-04-25T00:14:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.267+0000 7fbc48e00640 1 Processor -- start 2024-04-25T00:14:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.267+0000 7fbc48e00640 1 -- start start 2024-04-25T00:14:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.267+0000 7fbc48e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc441618d0 0x7fbc440ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.267+0000 7fbc48e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbc440d1930 0x7fbc440ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.267+0000 7fbc48e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbc440cf070 0x7fbc4406c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.267+0000 7fbc48e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbc440d11d0 con 0x7fbc440cf070 2024-04-25T00:14:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.267+0000 7fbc48e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbc44161cb0 con 0x7fbc440d1930 2024-04-25T00:14:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.267+0000 7fbc48e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbc440d0970 con 0x7fbc441618d0 2024-04-25T00:14:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc42800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbc440cf070 0x7fbc4406c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc42800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbc440cf070 0x7fbc4406c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38430/0 (socket says 172.21.15.83:38430) 2024-04-25T00:14:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc42800640 1 -- 172.21.15.83:0/3229113615 learned_addr learned my addr 172.21.15.83:0/3229113615 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc41e00640 1 --2- 172.21.15.83:0/3229113615 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc441618d0 0x7fbc440ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc41400640 1 --2- 172.21.15.83:0/3229113615 >> v2:172.21.15.86:3300/0 conn(0x7fbc440d1930 0x7fbc440ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc42800640 1 -- 172.21.15.83:0/3229113615 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc441618d0 msgr2=0x7fbc440ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc42800640 1 --2- 172.21.15.83:0/3229113615 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc441618d0 0x7fbc440ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:45.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc42800640 1 -- 172.21.15.83:0/3229113615 >> v2:172.21.15.86:3300/0 conn(0x7fbc440d1930 msgr2=0x7fbc440ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:45.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc42800640 1 --2- 172.21.15.83:0/3229113615 >> v2:172.21.15.86:3300/0 conn(0x7fbc440d1930 0x7fbc440ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:45.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc42800640 1 -- 172.21.15.83:0/3229113615 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc440d0c10 con 0x7fbc440cf070 2024-04-25T00:14:45.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc41e00640 1 --2- 172.21.15.83:0/3229113615 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc441618d0 0x7fbc440ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:45.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc42800640 1 --2- 172.21.15.83:0/3229113615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc440cf070 0x7fbc4406c2f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fbc440726d0 tx=0x7fbc3c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8071d847e2976a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:45.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc40a00640 1 -- 172.21.15.83:0/3229113615 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c0aa430 con 0x7fbc440cf070 2024-04-25T00:14:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc40a00640 1 -- 172.21.15.83:0/3229113615 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc3c0af070 con 0x7fbc440cf070 2024-04-25T00:14:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.271+0000 7fbc40a00640 1 -- 172.21.15.83:0/3229113615 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c0aa7f0 con 0x7fbc440cf070 2024-04-25T00:14:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 -- 172.21.15.83:0/3229113615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc440cf070 msgr2=0x7fbc4406c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 --2- 172.21.15.83:0/3229113615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc440cf070 0x7fbc4406c2f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fbc440726d0 tx=0x7fbc3c01c4f0 comp rx=0 tx=0).stop 2024-04-25T00:14:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 -- 172.21.15.83:0/3229113615 shutdown_connections 2024-04-25T00:14:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 --2- 172.21.15.83:0/3229113615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc440cf070 0x7fbc4406c2f0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 --2- 172.21.15.83:0/3229113615 >> v2:172.21.15.86:3300/0 conn(0x7fbc440d1930 0x7fbc440ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 --2- 172.21.15.83:0/3229113615 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc441618d0 0x7fbc440ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 -- 172.21.15.83:0/3229113615 >> 172.21.15.83:0/3229113615 conn(0x7fbc440d1d70 msgr2=0x7fbc440d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 -- 172.21.15.83:0/3229113615 shutdown_connections 2024-04-25T00:14:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 -- 172.21.15.83:0/3229113615 wait complete. 2024-04-25T00:14:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 Processor -- start 2024-04-25T00:14:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 -- start start 2024-04-25T00:14:45.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc440cf070 0x7fbc4415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc440d1930 0x7fbc440cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc441618d0 0x7fbc44151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc440ccac0 con 0x7fbc440d1930 2024-04-25T00:14:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc440cc940 con 0x7fbc440cf070 2024-04-25T00:14:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc48e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbc440ccc40 con 0x7fbc441618d0 2024-04-25T00:14:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc41400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc440d1930 0x7fbc440cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc41400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc440d1930 0x7fbc440cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38436/0 (socket says 172.21.15.83:38436) 2024-04-25T00:14:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc41400640 1 -- 172.21.15.83:0/53639136 learned_addr learned my addr 172.21.15.83:0/53639136 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc41e00640 1 --2- 172.21.15.83:0/53639136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc440cf070 0x7fbc4415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc42800640 1 --2- 172.21.15.83:0/53639136 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc441618d0 0x7fbc44151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc41400640 1 -- 172.21.15.83:0/53639136 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc441618d0 msgr2=0x7fbc44151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc41400640 1 --2- 172.21.15.83:0/53639136 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc441618d0 0x7fbc44151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc41400640 1 -- 172.21.15.83:0/53639136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc440cf070 msgr2=0x7fbc4415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc41400640 1 --2- 172.21.15.83:0/53639136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc440cf070 0x7fbc4415d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.275+0000 7fbc41400640 1 -- 172.21.15.83:0/53639136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4415e5b0 con 0x7fbc440d1930 2024-04-25T00:14:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.279+0000 7fbc41400640 1 --2- 172.21.15.83:0/53639136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc440d1930 0x7fbc440cb150 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000d800 tx=0x7fbc3000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.279+0000 7fbc3a000640 1 -- 172.21.15.83:0/53639136 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30018020 con 0x7fbc440d1930 2024-04-25T00:14:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.279+0000 7fbc3a000640 1 -- 172.21.15.83:0/53639136 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc300096a0 con 0x7fbc440d1930 2024-04-25T00:14:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.279+0000 7fbc3a000640 1 -- 172.21.15.83:0/53639136 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30009a50 con 0x7fbc440d1930 2024-04-25T00:14:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.279+0000 7fbc48e00640 1 -- 172.21.15.83:0/53639136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc440ca670 con 0x7fbc440d1930 2024-04-25T00:14:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.279+0000 7fbc48e00640 1 -- 172.21.15.83:0/53639136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc440cac30 con 0x7fbc440d1930 2024-04-25T00:14:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.279+0000 7fbc48e00640 1 -- 172.21.15.83:0/53639136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc4409fdd0 con 0x7fbc440d1930 2024-04-25T00:14:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.279+0000 7fbc3a000640 1 -- 172.21.15.83:0/53639136 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbc3001b020 con 0x7fbc440d1930 2024-04-25T00:14:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.279+0000 7fbc3a000640 1 --2- 172.21.15.83:0/53639136 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc240678c0 0x7fbc24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.279+0000 7fbc3a000640 1 -- 172.21.15.83:0/53639136 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5335+0+0 (secure 0 0 0) 0x7fbc30085f60 con 0x7fbc440d1930 2024-04-25T00:14:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.279+0000 7fbc41e00640 1 --2- 172.21.15.83:0/53639136 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc240678c0 0x7fbc24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.283+0000 7fbc41e00640 1 --2- 172.21.15.83:0/53639136 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc240678c0 0x7fbc24069d80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbc440684f0 tx=0x7fbc340097f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:45.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.283+0000 7fbc3a000640 1 -- 172.21.15.83:0/53639136 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbc30022020 con 0x7fbc440d1930 2024-04-25T00:14:45.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:45.495+0000 7fbc48e00640 1 -- 172.21.15.83:0/53639136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]} v 0) -- 0x7fbc4404f5f0 con 0x7fbc440d1930 2024-04-25T00:14:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.451+0000 7fbc3a000640 1 -- 172.21.15.83:0/53639136 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]}]=0 v240) ==== 148+0+0 (secure 0 0 0) 0x7fbc300565b0 con 0x7fbc440d1930 2024-04-25T00:14:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.455+0000 7fbc48e00640 1 -- 172.21.15.83:0/53639136 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc240678c0 msgr2=0x7fbc24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.455+0000 7fbc48e00640 1 --2- 172.21.15.83:0/53639136 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc240678c0 0x7fbc24069d80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbc440684f0 tx=0x7fbc340097f0 comp rx=0 tx=0).stop 2024-04-25T00:14:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.455+0000 7fbc48e00640 1 -- 172.21.15.83:0/53639136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc440d1930 msgr2=0x7fbc440cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.455+0000 7fbc48e00640 1 --2- 172.21.15.83:0/53639136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc440d1930 0x7fbc440cb150 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000d800 tx=0x7fbc3000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:14:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.455+0000 7fbc48e00640 1 -- 172.21.15.83:0/53639136 shutdown_connections 2024-04-25T00:14:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.455+0000 7fbc48e00640 1 --2- 172.21.15.83:0/53639136 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbc240678c0 0x7fbc24069d80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.455+0000 7fbc48e00640 1 --2- 172.21.15.83:0/53639136 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc441618d0 0x7fbc44151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.455+0000 7fbc48e00640 1 --2- 172.21.15.83:0/53639136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc440d1930 0x7fbc440cb150 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.455+0000 7fbc48e00640 1 --2- 172.21.15.83:0/53639136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc440cf070 0x7fbc4415d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:46.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.455+0000 7fbc48e00640 1 -- 172.21.15.83:0/53639136 >> 172.21.15.83:0/53639136 conn(0x7fbc440d1d70 msgr2=0x7fbc440b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:46.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.455+0000 7fbc48e00640 1 -- 172.21.15.83:0/53639136 shutdown_connections 2024-04-25T00:14:46.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.455+0000 7fbc48e00640 1 -- 172.21.15.83:0/53639136 wait complete. 2024-04-25T00:14:46.483 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:14:46.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.627+0000 7fa02de00640 1 Processor -- start 2024-04-25T00:14:46.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.627+0000 7fa02de00640 1 -- start start 2024-04-25T00:14:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.631+0000 7fa02de00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa0280700b0 0x7fa0280cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.631+0000 7fa02de00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa02815e640 con 0x7fa02815e260 2024-04-25T00:14:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.631+0000 7fa02de00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0280d1350 con 0x7fa0280cee70 2024-04-25T00:14:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.631+0000 7fa02de00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa0280d0d90 con 0x7fa0280700b0 2024-04-25T00:14:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.631+0000 7fa027400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa0280700b0 0x7fa0280cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.631+0000 7fa026a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa02815e260 0x7fa0280ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54726/0 (socket says 172.21.15.83:54726) 2024-04-25T00:14:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.631+0000 7fa026a00640 1 -- 172.21.15.83:0/4191315194 learned_addr learned my addr 172.21.15.83:0/4191315194 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.631+0000 7fa027400640 1 -- 172.21.15.83:0/4191315194 >> v1:172.21.15.86:6789/0 conn(0x7fa0280cee70 legacy=0x7fa0280ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.631+0000 7fa027400640 1 -- 172.21.15.83:0/4191315194 >> v1:172.21.15.83:6789/0 conn(0x7fa02815e260 legacy=0x7fa0280ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.631+0000 7fa027400640 1 -- 172.21.15.83:0/4191315194 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0280d1070 con 0x7fa0280700b0 2024-04-25T00:14:46.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.635+0000 7fa027400640 1 --2- 172.21.15.83:0/4191315194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0280700b0 0x7fa0280cd380 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa020009d90 tx=0x7fa0280ce0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bf3c052300a2ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:46.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.635+0000 7fa025600640 1 -- 172.21.15.83:0/4191315194 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0200b2070 con 0x7fa0280700b0 2024-04-25T00:14:46.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.635+0000 7fa025600640 1 -- 172.21.15.83:0/4191315194 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0200a5920 con 0x7fa0280700b0 2024-04-25T00:14:46.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.635+0000 7fa025600640 1 -- 172.21.15.83:0/4191315194 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0200a5cf0 con 0x7fa0280700b0 2024-04-25T00:14:46.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.635+0000 7fa02de00640 1 -- 172.21.15.83:0/4191315194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0280700b0 msgr2=0x7fa0280cd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:46.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.635+0000 7fa02de00640 1 --2- 172.21.15.83:0/4191315194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0280700b0 0x7fa0280cd380 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa020009d90 tx=0x7fa0280ce0d0 comp rx=0 tx=0).stop 2024-04-25T00:14:46.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.635+0000 7fa02de00640 1 -- 172.21.15.83:0/4191315194 shutdown_connections 2024-04-25T00:14:46.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.635+0000 7fa02de00640 1 --2- 172.21.15.83:0/4191315194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0280700b0 0x7fa0280cd380 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:46.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.635+0000 7fa02de00640 1 -- 172.21.15.83:0/4191315194 >> 172.21.15.83:0/4191315194 conn(0x7fa0280b3950 msgr2=0x7fa0280b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:46.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.635+0000 7fa02de00640 1 -- 172.21.15.83:0/4191315194 shutdown_connections 2024-04-25T00:14:46.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.635+0000 7fa02de00640 1 -- 172.21.15.83:0/4191315194 wait complete. 2024-04-25T00:14:46.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.635+0000 7fa02de00640 1 Processor -- start 2024-04-25T00:14:46.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.635+0000 7fa02de00640 1 -- start start 2024-04-25T00:14:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa02de00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa0280700b0 0x7fa028158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa02de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0280cee70 0x7fa0281500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa02de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa02815e260 0x7fa0280c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa02de00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa02816bf40 con 0x7fa02815e260 2024-04-25T00:14:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa02de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa02816bdc0 con 0x7fa0280cee70 2024-04-25T00:14:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa02de00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa02816c0c0 con 0x7fa0280700b0 2024-04-25T00:14:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa027400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa02815e260 0x7fa0280c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa027400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa02815e260 0x7fa0280c5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38458/0 (socket says 172.21.15.83:38458) 2024-04-25T00:14:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa027400640 1 -- 172.21.15.83:0/366340283 learned_addr learned my addr 172.21.15.83:0/366340283 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa026a00640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa0280700b0 0x7fa028158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa026000640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0280cee70 0x7fa0281500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa027400640 1 -- 172.21.15.83:0/366340283 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa0280700b0 msgr2=0x7fa028158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa027400640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa0280700b0 0x7fa028158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:46.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa027400640 1 -- 172.21.15.83:0/366340283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0280cee70 msgr2=0x7fa0281500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:46.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa027400640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0280cee70 0x7fa0281500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:46.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa027400640 1 -- 172.21.15.83:0/366340283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa028157b60 con 0x7fa02815e260 2024-04-25T00:14:46.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa026000640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0280cee70 0x7fa0281500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:46.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa027400640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa02815e260 0x7fa0280c5800 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa02001a5a0 tx=0x7fa020019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:46.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa01f400640 1 -- 172.21.15.83:0/366340283 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0200b2040 con 0x7fa02815e260 2024-04-25T00:14:46.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa01f400640 1 -- 172.21.15.83:0/366340283 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0200ad440 con 0x7fa02815e260 2024-04-25T00:14:46.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa02de00640 1 -- 172.21.15.83:0/366340283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa028157e40 con 0x7fa02815e260 2024-04-25T00:14:46.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa01f400640 1 -- 172.21.15.83:0/366340283 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0200ad830 con 0x7fa02815e260 2024-04-25T00:14:46.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa02de00640 1 -- 172.21.15.83:0/366340283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa028158400 con 0x7fa02815e260 2024-04-25T00:14:46.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa02de00640 1 -- 172.21.15.83:0/366340283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa02804f510 con 0x7fa02815e260 2024-04-25T00:14:46.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.639+0000 7fa01f400640 1 -- 172.21.15.83:0/366340283 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa0200c1020 con 0x7fa02815e260 2024-04-25T00:14:46.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.643+0000 7fa01f400640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa0080678c0 0x7fa008069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:46.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.643+0000 7fa01f400640 1 -- 172.21.15.83:0/366340283 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5335+0+0 (secure 0 0 0) 0x7fa020120330 con 0x7fa02815e260 2024-04-25T00:14:46.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.643+0000 7fa026a00640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa0080678c0 0x7fa008069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:46.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.643+0000 7fa026a00640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa0080678c0 0x7fa008069d80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa018005890 tx=0x7fa018009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:46.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.647+0000 7fa01f400640 1 -- 172.21.15.83:0/366340283 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa0200c8020 con 0x7fa02815e260 2024-04-25T00:14:46.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.799+0000 7fa02de00640 1 -- 172.21.15.83:0/366340283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa0280d18a0 con 0x7fa02815e260 2024-04-25T00:14:46.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.803+0000 7fa01f400640 1 -- 172.21.15.83:0/366340283 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7fa020016560 con 0x7fa02815e260 2024-04-25T00:14:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.807+0000 7fa02de00640 1 -- 172.21.15.83:0/366340283 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa0080678c0 msgr2=0x7fa008069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.807+0000 7fa02de00640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa0080678c0 0x7fa008069d80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa018005890 tx=0x7fa018009040 comp rx=0 tx=0).stop 2024-04-25T00:14:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.807+0000 7fa02de00640 1 -- 172.21.15.83:0/366340283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa02815e260 msgr2=0x7fa0280c5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.807+0000 7fa02de00640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa02815e260 0x7fa0280c5800 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa02001a5a0 tx=0x7fa020019710 comp rx=0 tx=0).stop 2024-04-25T00:14:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.807+0000 7fa02de00640 1 -- 172.21.15.83:0/366340283 shutdown_connections 2024-04-25T00:14:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.807+0000 7fa02de00640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa0080678c0 0x7fa008069d80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.807+0000 7fa02de00640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa02815e260 0x7fa0280c5800 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:46.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.807+0000 7fa02de00640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0280cee70 0x7fa0281500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:46.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.807+0000 7fa02de00640 1 --2- 172.21.15.83:0/366340283 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa0280700b0 0x7fa028158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:46.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.807+0000 7fa02de00640 1 -- 172.21.15.83:0/366340283 >> 172.21.15.83:0/366340283 conn(0x7fa0280b3950 msgr2=0x7fa0280b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:46.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.807+0000 7fa02de00640 1 -- 172.21.15.83:0/366340283 shutdown_connections 2024-04-25T00:14:46.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:46.807+0000 7fa02de00640 1 -- 172.21.15.83:0/366340283 wait complete. 2024-04-25T00:14:46.832 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:14:46.832 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:14:46.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:14:47.071 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:14:47.072 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:14:47.072 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:47.080 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:14:47.081 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:14:47.081 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:47.090 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:14:47.090 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:14:47.090 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:47.133 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:14:47.133 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:14:47.133 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:47.141 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:14:47.142 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:14:47.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:47.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.307+0000 7f85b9800640 1 Processor -- start 2024-04-25T00:14:47.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.307+0000 7f85b9800640 1 -- start start 2024-04-25T00:14:47.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.307+0000 7f85b9800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f85b40cc510 0x7f85b415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:47.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.307+0000 7f85b9800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85b415cf90 0x7f85b40ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:47.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.307+0000 7f85b9800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b40ccc60 con 0x7f85b41599f0 2024-04-25T00:14:47.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.307+0000 7f85b9800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f85b4159dd0 con 0x7f85b415cf90 2024-04-25T00:14:47.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.307+0000 7f85b9800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f85b406dba0 con 0x7f85b40cc510 2024-04-25T00:14:47.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.307+0000 7f85b3400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85b415cf90 0x7f85b40ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:47.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.307+0000 7f85b3400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85b415cf90 0x7f85b40ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38466/0 (socket says 172.21.15.83:38466) 2024-04-25T00:14:47.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.307+0000 7f85b3400640 1 -- 172.21.15.83:0/2277429965 learned_addr learned my addr 172.21.15.83:0/2277429965 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:47.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.307+0000 7f85b3400640 1 -- 172.21.15.83:0/2277429965 >> v2:172.21.15.86:3300/0 conn(0x7f85b40cc510 msgr2=0x7f85b415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:47.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.307+0000 7f85b3400640 1 --2- 172.21.15.83:0/2277429965 >> v2:172.21.15.86:3300/0 conn(0x7f85b40cc510 0x7f85b415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.307+0000 7f85b3400640 1 -- 172.21.15.83:0/2277429965 >> v1:172.21.15.86:6789/0 conn(0x7f85b41599f0 legacy=0x7f85b415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:47.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.307+0000 7f85b3400640 1 -- 172.21.15.83:0/2277429965 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b406de80 con 0x7f85b415cf90 2024-04-25T00:14:47.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.311+0000 7f85b3400640 1 --2- 172.21.15.83:0/2277429965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b415cf90 0x7f85b40ce6e0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009870 tx=0x7f85ac01c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88c5497a7de0d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:47.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.311+0000 7f85b2000640 1 -- 172.21.15.83:0/2277429965 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0b0070 con 0x7f85b415cf90 2024-04-25T00:14:47.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.311+0000 7f85b2000640 1 -- 172.21.15.83:0/2277429965 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85ac0ab600 con 0x7f85b415cf90 2024-04-25T00:14:47.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.311+0000 7f85b2000640 1 -- 172.21.15.83:0/2277429965 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0ab9d0 con 0x7f85b415cf90 2024-04-25T00:14:47.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.311+0000 7f85b9800640 1 -- 172.21.15.83:0/2277429965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b415cf90 msgr2=0x7f85b40ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:47.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.311+0000 7f85b9800640 1 --2- 172.21.15.83:0/2277429965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b415cf90 0x7f85b40ce6e0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009870 tx=0x7f85ac01c270 comp rx=0 tx=0).stop 2024-04-25T00:14:47.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.311+0000 7f85b9800640 1 -- 172.21.15.83:0/2277429965 shutdown_connections 2024-04-25T00:14:47.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.311+0000 7f85b9800640 1 --2- 172.21.15.83:0/2277429965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b415cf90 0x7f85b40ce6e0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.311+0000 7f85b9800640 1 --2- 172.21.15.83:0/2277429965 >> v2:172.21.15.86:3300/0 conn(0x7f85b40cc510 0x7f85b415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.311+0000 7f85b9800640 1 -- 172.21.15.83:0/2277429965 >> 172.21.15.83:0/2277429965 conn(0x7f85b40b1830 msgr2=0x7f85b40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:47.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.311+0000 7f85b9800640 1 -- 172.21.15.83:0/2277429965 shutdown_connections 2024-04-25T00:14:47.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.311+0000 7f85b9800640 1 -- 172.21.15.83:0/2277429965 wait complete. 2024-04-25T00:14:47.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85b9800640 1 Processor -- start 2024-04-25T00:14:47.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85b9800640 1 -- start start 2024-04-25T00:14:47.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85b9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b40cc510 0x7f85b4152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:47.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85b9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b41599f0 0x7f85b4140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:47.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85b9800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85b415cf90 0x7f85b41508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:47.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85b9800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85b40cf5b0 con 0x7f85b41599f0 2024-04-25T00:14:47.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85b9800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85b40cf430 con 0x7f85b40cc510 2024-04-25T00:14:47.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85b9800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f85b40cf730 con 0x7f85b415cf90 2024-04-25T00:14:47.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85aa000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b41599f0 0x7f85b4140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:47.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85aa000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b41599f0 0x7f85b4140650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38476/0 (socket says 172.21.15.83:38476) 2024-04-25T00:14:47.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85aa000640 1 -- 172.21.15.83:0/314323775 learned_addr learned my addr 172.21.15.83:0/314323775 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:47.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85b2a00640 1 --2- 172.21.15.83:0/314323775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b40cc510 0x7f85b4152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:47.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85b3400640 1 --2- 172.21.15.83:0/314323775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85b415cf90 0x7f85b41508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85aa000640 1 -- 172.21.15.83:0/314323775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85b415cf90 msgr2=0x7f85b41508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85aa000640 1 --2- 172.21.15.83:0/314323775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85b415cf90 0x7f85b41508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85aa000640 1 -- 172.21.15.83:0/314323775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b40cc510 msgr2=0x7f85b4152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85aa000640 1 --2- 172.21.15.83:0/314323775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b40cc510 0x7f85b4152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85aa000640 1 -- 172.21.15.83:0/314323775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b4150f40 con 0x7f85b41599f0 2024-04-25T00:14:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85aa000640 1 --2- 172.21.15.83:0/314323775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b41599f0 0x7f85b4140650 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f859c00b410 tx=0x7f859c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85ab400640 1 -- 172.21.15.83:0/314323775 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c015020 con 0x7f85b41599f0 2024-04-25T00:14:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85ab400640 1 -- 172.21.15.83:0/314323775 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f859c011830 con 0x7f85b41599f0 2024-04-25T00:14:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85b9800640 1 -- 172.21.15.83:0/314323775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b4151240 con 0x7f85b41599f0 2024-04-25T00:14:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85ab400640 1 -- 172.21.15.83:0/314323775 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c011c20 con 0x7f85b41599f0 2024-04-25T00:14:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85b9800640 1 -- 172.21.15.83:0/314323775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b4151800 con 0x7f85b41599f0 2024-04-25T00:14:47.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.315+0000 7f85b9800640 1 -- 172.21.15.83:0/314323775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b40737f0 con 0x7f85b41599f0 2024-04-25T00:14:47.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.319+0000 7f85ab400640 1 -- 172.21.15.83:0/314323775 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f859c018020 con 0x7f85b41599f0 2024-04-25T00:14:47.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.319+0000 7f85ab400640 1 --2- 172.21.15.83:0/314323775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85900678c0 0x7f8590069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:47.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.319+0000 7f85ab400640 1 -- 172.21.15.83:0/314323775 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5335+0+0 (secure 0 0 0) 0x7f859c085900 con 0x7f85b41599f0 2024-04-25T00:14:47.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.319+0000 7f85b2a00640 1 --2- 172.21.15.83:0/314323775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85900678c0 0x7f8590069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:47.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.319+0000 7f85b2a00640 1 --2- 172.21.15.83:0/314323775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85900678c0 0x7f8590069d80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f85a4001f20 tx=0x7f85a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:47.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.323+0000 7f85ab400640 1 -- 172.21.15.83:0/314323775 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f859c01f020 con 0x7f85b41599f0 2024-04-25T00:14:47.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.515+0000 7f85b9800640 1 -- 172.21.15.83:0/314323775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85b404f5f0 con 0x7f85b41599f0 2024-04-25T00:14:47.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.515+0000 7f85ab400640 1 -- 172.21.15.83:0/314323775 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+3863 (secure 0 0 0) 0x7f859c055f50 con 0x7f85b41599f0 2024-04-25T00:14:47.520 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:47.520 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/3856734881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3856734881},{"type":"v1","addr":"172.21.15.86:6834","nonce":3856734881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/12764228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":12764228},{"type":"v1","addr":"172.21.15.86:6835","nonce":12764228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1680607346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1680607346},{"type":"v1","addr":"172.21.15.83:6837","nonce":1680607346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":7377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3635456506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3635456506},{"type":"v1","addr":"172.21.15.83:6836","nonce":3635456506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[]} 2024-04-25T00:14:47.520 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 55 2024-04-25T00:14:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.519+0000 7f85b9800640 1 -- 172.21.15.83:0/314323775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85900678c0 msgr2=0x7f8590069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.519+0000 7f85b9800640 1 --2- 172.21.15.83:0/314323775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85900678c0 0x7f8590069d80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f85a4001f20 tx=0x7f85a4009040 comp rx=0 tx=0).stop 2024-04-25T00:14:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.519+0000 7f85b9800640 1 -- 172.21.15.83:0/314323775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b41599f0 msgr2=0x7f85b4140650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.519+0000 7f85b9800640 1 --2- 172.21.15.83:0/314323775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b41599f0 0x7f85b4140650 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f859c00b410 tx=0x7f859c00b8d0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.519+0000 7f85b9800640 1 -- 172.21.15.83:0/314323775 shutdown_connections 2024-04-25T00:14:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.519+0000 7f85b9800640 1 --2- 172.21.15.83:0/314323775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f85900678c0 0x7f8590069d80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.519+0000 7f85b9800640 1 --2- 172.21.15.83:0/314323775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85b415cf90 0x7f85b41508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.519+0000 7f85b9800640 1 --2- 172.21.15.83:0/314323775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b41599f0 0x7f85b4140650 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.519+0000 7f85b9800640 1 --2- 172.21.15.83:0/314323775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b40cc510 0x7f85b4152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.519+0000 7f85b9800640 1 -- 172.21.15.83:0/314323775 >> 172.21.15.83:0/314323775 conn(0x7f85b40b1830 msgr2=0x7f85b4152f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.519+0000 7f85b9800640 1 -- 172.21.15.83:0/314323775 shutdown_connections 2024-04-25T00:14:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.519+0000 7f85b9800640 1 -- 172.21.15.83:0/314323775 wait complete. 2024-04-25T00:14:47.541 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:14:47.542 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:14:47.542 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:14:47.542 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:14:47.542 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:14:47.542 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:14:47.542 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:14:47.543 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:14:47.543 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:14:47.543 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:14:47.543 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:14:47.543 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:14:47.543 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:14:47.543 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:14:47.544 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:14:47.544 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:14:47.556 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:14:47.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:14:47.557 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:14:47.558 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:14:47.560 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:14:47.561 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:14:47.562 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:14:47.562 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:14:47.603 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:14:47.604 DEBUG:teuthology.parallel:result is None 2024-04-25T00:14:47.605 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:14:47.606 DEBUG:teuthology.parallel:result is None 2024-04-25T00:14:47.611 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:14:47.611 DEBUG:teuthology.parallel:result is None 2024-04-25T00:14:47.613 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:14:47.613 DEBUG:teuthology.parallel:result is None 2024-04-25T00:14:47.613 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:14:47.688 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-25T00:14:47.691 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-25T00:14:47.701 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2024-04-25T00:14:47.702 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2024-04-25T00:14:47.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.755+0000 7fa813400640 1 Processor -- start 2024-04-25T00:14:47.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.755+0000 7fa813400640 1 -- start start 2024-04-25T00:14:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.755+0000 7fa813400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa80c06cfd0 0x7fa80c070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.755+0000 7fa813400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa80c06e160 con 0x7fa80c06d3b0 2024-04-25T00:14:47.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.755+0000 7fa813400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa80c06ea10 con 0x7fa80c0cb640 2024-04-25T00:14:47.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.755+0000 7fa813400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa80c06df20 con 0x7fa80c06cfd0 2024-04-25T00:14:47.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.755+0000 7fa80be00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa80c06d3b0 0x7fa80c06d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54750/0 (socket says 172.21.15.83:54750) 2024-04-25T00:14:47.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.755+0000 7fa80be00640 1 -- 172.21.15.83:0/1943976342 learned_addr learned my addr 172.21.15.83:0/1943976342 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:47.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.755+0000 7fa810a00640 1 --2- 172.21.15.83:0/1943976342 >> v2:172.21.15.86:3300/0 conn(0x7fa80c06cfd0 0x7fa80c070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:47.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.759+0000 7fa80b400640 1 -- 172.21.15.83:0/1943976342 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 779908735 0 0) 0x7fa80c06e160 con 0x7fa80c06d3b0 2024-04-25T00:14:47.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.759+0000 7fa80b400640 1 -- 172.21.15.83:0/1943976342 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7f0003530 con 0x7fa80c06d3b0 2024-04-25T00:14:47.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.759+0000 7fa80b400640 1 -- 172.21.15.83:0/1943976342 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3219061660 0 0) 0x7fa80c06ea10 con 0x7fa80c0cb640 2024-04-25T00:14:47.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.759+0000 7fa810a00640 1 -- 172.21.15.83:0/1943976342 >> v1:172.21.15.86:6789/0 conn(0x7fa80c0cb640 legacy=0x7fa80c165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:47.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.759+0000 7fa810a00640 1 -- 172.21.15.83:0/1943976342 >> v1:172.21.15.83:6789/0 conn(0x7fa80c06d3b0 legacy=0x7fa80c06d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:47.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.759+0000 7fa810a00640 1 -- 172.21.15.83:0/1943976342 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c06ecf0 con 0x7fa80c06cfd0 2024-04-25T00:14:47.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.759+0000 7fa810a00640 1 --2- 172.21.15.83:0/1943976342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c06cfd0 0x7fa80c070d10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0097c0 tx=0x7fa7fc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a0aa961bbc3e1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:47.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.759+0000 7fa80b400640 1 -- 172.21.15.83:0/1943976342 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc0b0070 con 0x7fa80c06cfd0 2024-04-25T00:14:47.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.759+0000 7fa80b400640 1 -- 172.21.15.83:0/1943976342 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7fc0ab600 con 0x7fa80c06cfd0 2024-04-25T00:14:47.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.759+0000 7fa80b400640 1 -- 172.21.15.83:0/1943976342 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc0ab9d0 con 0x7fa80c06cfd0 2024-04-25T00:14:47.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 -- 172.21.15.83:0/1943976342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c06cfd0 msgr2=0x7fa80c070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:47.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 --2- 172.21.15.83:0/1943976342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c06cfd0 0x7fa80c070d10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0097c0 tx=0x7fa7fc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 -- 172.21.15.83:0/1943976342 shutdown_connections 2024-04-25T00:14:47.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 --2- 172.21.15.83:0/1943976342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c06cfd0 0x7fa80c070d10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 -- 172.21.15.83:0/1943976342 >> 172.21.15.83:0/1943976342 conn(0x7fa80c0cfa60 msgr2=0x7fa80c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:47.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 -- 172.21.15.83:0/1943976342 shutdown_connections 2024-04-25T00:14:47.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 -- 172.21.15.83:0/1943976342 wait complete. 2024-04-25T00:14:47.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 Processor -- start 2024-04-25T00:14:47.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 -- start start 2024-04-25T00:14:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa80c06cfd0 0x7fa80c1443e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa80c06d3b0 0x7fa80c138c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c0cb640 0x7fa80c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c169a80 con 0x7fa80c06d3b0 2024-04-25T00:14:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c145970 con 0x7fa80c0cb640 2024-04-25T00:14:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa813400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa80c169c00 con 0x7fa80c06cfd0 2024-04-25T00:14:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa80be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa80c06d3b0 0x7fa80c138c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa80be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa80c06d3b0 0x7fa80c138c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38512/0 (socket says 172.21.15.83:38512) 2024-04-25T00:14:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa80be00640 1 -- 172.21.15.83:0/4085270395 learned_addr learned my addr 172.21.15.83:0/4085270395 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa810a00640 1 --2- 172.21.15.83:0/4085270395 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa80c06cfd0 0x7fa80c1443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa811400640 1 --2- 172.21.15.83:0/4085270395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c0cb640 0x7fa80c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa80be00640 1 -- 172.21.15.83:0/4085270395 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa80c06cfd0 msgr2=0x7fa80c1443e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa80be00640 1 --2- 172.21.15.83:0/4085270395 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa80c06cfd0 0x7fa80c1443e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa80be00640 1 -- 172.21.15.83:0/4085270395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c0cb640 msgr2=0x7fa80c0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:47.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa80be00640 1 --2- 172.21.15.83:0/4085270395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c0cb640 0x7fa80c0b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa80be00640 1 -- 172.21.15.83:0/4085270395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c0b4bc0 con 0x7fa80c06d3b0 2024-04-25T00:14:47.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.763+0000 7fa80be00640 1 --2- 172.21.15.83:0/4085270395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa80c06d3b0 0x7fa80c138c20 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800c3f0 tx=0x7fa7f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:47.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.767+0000 7fa808c00640 1 -- 172.21.15.83:0/4085270395 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8017070 con 0x7fa80c06d3b0 2024-04-25T00:14:47.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.767+0000 7fa808c00640 1 -- 172.21.15.83:0/4085270395 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f8005cf0 con 0x7fa80c06d3b0 2024-04-25T00:14:47.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.767+0000 7fa813400640 1 -- 172.21.15.83:0/4085270395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80c0b4e80 con 0x7fa80c06d3b0 2024-04-25T00:14:47.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.767+0000 7fa808c00640 1 -- 172.21.15.83:0/4085270395 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f80072f0 con 0x7fa80c06d3b0 2024-04-25T00:14:47.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.767+0000 7fa813400640 1 -- 172.21.15.83:0/4085270395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80c0b5440 con 0x7fa80c06d3b0 2024-04-25T00:14:47.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.767+0000 7fa813400640 1 -- 172.21.15.83:0/4085270395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa80c068230 con 0x7fa80c06d3b0 2024-04-25T00:14:47.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.767+0000 7fa808c00640 1 -- 172.21.15.83:0/4085270395 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa7f801a070 con 0x7fa80c06d3b0 2024-04-25T00:14:47.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.767+0000 7fa808c00640 1 --2- 172.21.15.83:0/4085270395 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa7ec0678c0 0x7fa7ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:47.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.767+0000 7fa808c00640 1 -- 172.21.15.83:0/4085270395 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5335+0+0 (secure 0 0 0) 0x7fa7f8084f10 con 0x7fa80c06d3b0 2024-04-25T00:14:47.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.767+0000 7fa810a00640 1 --2- 172.21.15.83:0/4085270395 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa7ec0678c0 0x7fa7ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.767+0000 7fa810a00640 1 --2- 172.21.15.83:0/4085270395 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa7ec0678c0 0x7fa7ec069d80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0ad000 tx=0x7fa7fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.771+0000 7fa808c00640 1 -- 172.21.15.83:0/4085270395 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa7f8055560 con 0x7fa80c06d3b0 2024-04-25T00:14:47.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:47.939+0000 7fa813400640 1 -- 172.21.15.83:0/4085270395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa80c0b13d0 con 0x7fa80c06d3b0 2024-04-25T00:14:48.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.467+0000 7fa808c00640 1 -- 172.21.15.83:0/4085270395 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v87) ==== 97+0+0 (secure 0 0 0) 0x7fa7f8059860 con 0x7fa80c06d3b0 2024-04-25T00:14:48.472 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-25T00:14:48.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.471+0000 7fa813400640 1 -- 172.21.15.83:0/4085270395 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa7ec0678c0 msgr2=0x7fa7ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:48.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.471+0000 7fa813400640 1 --2- 172.21.15.83:0/4085270395 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa7ec0678c0 0x7fa7ec069d80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0ad000 tx=0x7fa7fc008040 comp rx=0 tx=0).stop 2024-04-25T00:14:48.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.471+0000 7fa813400640 1 -- 172.21.15.83:0/4085270395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa80c06d3b0 msgr2=0x7fa80c138c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:48.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.471+0000 7fa813400640 1 --2- 172.21.15.83:0/4085270395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa80c06d3b0 0x7fa80c138c20 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800c3f0 tx=0x7fa7f800c8b0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.471+0000 7fa813400640 1 -- 172.21.15.83:0/4085270395 shutdown_connections 2024-04-25T00:14:48.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.471+0000 7fa813400640 1 --2- 172.21.15.83:0/4085270395 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa7ec0678c0 0x7fa7ec069d80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.471+0000 7fa813400640 1 --2- 172.21.15.83:0/4085270395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c0cb640 0x7fa80c0b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.471+0000 7fa813400640 1 --2- 172.21.15.83:0/4085270395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa80c06d3b0 0x7fa80c138c20 secure :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800c3f0 tx=0x7fa7f800c8b0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.471+0000 7fa813400640 1 --2- 172.21.15.83:0/4085270395 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa80c06cfd0 0x7fa80c1443e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.471+0000 7fa813400640 1 -- 172.21.15.83:0/4085270395 >> 172.21.15.83:0/4085270395 conn(0x7fa80c0cfa60 msgr2=0x7fa80c0af860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:48.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.471+0000 7fa813400640 1 -- 172.21.15.83:0/4085270395 shutdown_connections 2024-04-25T00:14:48.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.471+0000 7fa813400640 1 -- 172.21.15.83:0/4085270395 wait complete. 2024-04-25T00:14:48.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:14:48.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.627+0000 7fb08f600640 1 Processor -- start 2024-04-25T00:14:48.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.627+0000 7fb08f600640 1 -- start start 2024-04-25T00:14:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.627+0000 7fb08f600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb0881599f0 0x7fb0880cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.627+0000 7fb08f600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb08815cec0 0x7fb0880ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.627+0000 7fb08f600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0880ccc60 con 0x7fb08815c340 2024-04-25T00:14:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.627+0000 7fb08f600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb088159dd0 con 0x7fb08815cec0 2024-04-25T00:14:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.627+0000 7fb08f600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb08806dba0 con 0x7fb0881599f0 2024-04-25T00:14:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.631+0000 7fb08d600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb08815cec0 0x7fb0880ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.631+0000 7fb08d600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb08815cec0 0x7fb0880ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38526/0 (socket says 172.21.15.83:38526) 2024-04-25T00:14:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.631+0000 7fb08d600640 1 -- 172.21.15.83:0/3460032922 learned_addr learned my addr 172.21.15.83:0/3460032922 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.631+0000 7fb08cc00640 1 --2- 172.21.15.83:0/3460032922 >> v2:172.21.15.86:3300/0 conn(0x7fb0881599f0 0x7fb0880cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.631+0000 7fb08d600640 1 -- 172.21.15.83:0/3460032922 >> v2:172.21.15.86:3300/0 conn(0x7fb0881599f0 msgr2=0x7fb0880cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.631+0000 7fb08d600640 1 --2- 172.21.15.83:0/3460032922 >> v2:172.21.15.86:3300/0 conn(0x7fb0881599f0 0x7fb0880cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.631+0000 7fb08d600640 1 -- 172.21.15.83:0/3460032922 >> v1:172.21.15.86:6789/0 conn(0x7fb08815c340 legacy=0x7fb08815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.631+0000 7fb08d600640 1 -- 172.21.15.83:0/3460032922 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08806de80 con 0x7fb08815cec0 2024-04-25T00:14:48.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.631+0000 7fb08d600640 1 --2- 172.21.15.83:0/3460032922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb08815cec0 0x7fb0880ce6e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb080009e50 tx=0x7fb08001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ec2388cc73d657 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:48.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.631+0000 7fb087e00640 1 -- 172.21.15.83:0/3460032922 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0800a5620 con 0x7fb08815cec0 2024-04-25T00:14:48.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.631+0000 7fb087e00640 1 -- 172.21.15.83:0/3460032922 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0800a57c0 con 0x7fb08815cec0 2024-04-25T00:14:48.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.631+0000 7fb087e00640 1 -- 172.21.15.83:0/3460032922 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0800a5b90 con 0x7fb08815cec0 2024-04-25T00:14:48.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 -- 172.21.15.83:0/3460032922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb08815cec0 msgr2=0x7fb0880ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:48.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 --2- 172.21.15.83:0/3460032922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb08815cec0 0x7fb0880ce6e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb080009e50 tx=0x7fb08001c960 comp rx=0 tx=0).stop 2024-04-25T00:14:48.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 -- 172.21.15.83:0/3460032922 shutdown_connections 2024-04-25T00:14:48.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 --2- 172.21.15.83:0/3460032922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb08815cec0 0x7fb0880ce6e0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 --2- 172.21.15.83:0/3460032922 >> v2:172.21.15.86:3300/0 conn(0x7fb0881599f0 0x7fb0880cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 -- 172.21.15.83:0/3460032922 >> 172.21.15.83:0/3460032922 conn(0x7fb0880b1830 msgr2=0x7fb0880b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:48.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 -- 172.21.15.83:0/3460032922 shutdown_connections 2024-04-25T00:14:48.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 -- 172.21.15.83:0/3460032922 wait complete. 2024-04-25T00:14:48.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 Processor -- start 2024-04-25T00:14:48.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 -- start start 2024-04-25T00:14:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0881599f0 0x7fb0880b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb08815c340 0x7fb0881529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08815cec0 0x7fb088152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb0880cf800 con 0x7fb08815c340 2024-04-25T00:14:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0880cf680 con 0x7fb08815cec0 2024-04-25T00:14:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08f600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb088168cd0 con 0x7fb0881599f0 2024-04-25T00:14:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb07fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb08815c340 0x7fb0881529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:48.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb07fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb08815c340 0x7fb0881529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38542/0 (socket says 172.21.15.83:38542) 2024-04-25T00:14:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb07fe00640 1 -- 172.21.15.83:0/1596755282 learned_addr learned my addr 172.21.15.83:0/1596755282 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08cc00640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0881599f0 0x7fb0880b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.635+0000 7fb08d600640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08815cec0 0x7fb088152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb07fe00640 1 -- 172.21.15.83:0/1596755282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0881599f0 msgr2=0x7fb0880b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb07fe00640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0881599f0 0x7fb0880b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb07fe00640 1 -- 172.21.15.83:0/1596755282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08815cec0 msgr2=0x7fb088152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb07fe00640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08815cec0 0x7fb088152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb07fe00640 1 -- 172.21.15.83:0/1596755282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb088158bf0 con 0x7fb08815c340 2024-04-25T00:14:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb08d600640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08815cec0 0x7fb088152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb07fe00640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb08815c340 0x7fb0881529c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb07000c500 tx=0x7fb07000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb085600640 1 -- 172.21.15.83:0/1596755282 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070016020 con 0x7fb08815c340 2024-04-25T00:14:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb085600640 1 -- 172.21.15.83:0/1596755282 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb070005220 con 0x7fb08815c340 2024-04-25T00:14:48.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb085600640 1 -- 172.21.15.83:0/1596755282 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070007030 con 0x7fb08815c340 2024-04-25T00:14:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb08f600640 1 -- 172.21.15.83:0/1596755282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0880b3710 con 0x7fb08815c340 2024-04-25T00:14:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb08f600640 1 -- 172.21.15.83:0/1596755282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0880b3cd0 con 0x7fb08815c340 2024-04-25T00:14:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb08f600640 1 -- 172.21.15.83:0/1596755282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0880737f0 con 0x7fb08815c340 2024-04-25T00:14:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb085600640 1 -- 172.21.15.83:0/1596755282 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb0700071f0 con 0x7fb08815c340 2024-04-25T00:14:48.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.639+0000 7fb085600640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb0640678c0 0x7fb064069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:48.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.643+0000 7fb085600640 1 -- 172.21.15.83:0/1596755282 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5249+0+0 (secure 0 0 0) 0x7fb070085420 con 0x7fb08815c340 2024-04-25T00:14:48.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.643+0000 7fb08cc00640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb0640678c0 0x7fb064069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:48.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.643+0000 7fb08cc00640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb0640678c0 0x7fb064069d80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb078009ac0 tx=0x7fb078009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:48.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.643+0000 7fb085600640 1 -- 172.21.15.83:0/1596755282 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb070020020 con 0x7fb08815c340 2024-04-25T00:14:48.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.855+0000 7fb08f600640 1 -- 172.21.15.83:0/1596755282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb08804f5f0 con 0x7fb08815c340 2024-04-25T00:14:48.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.855+0000 7fb085600640 1 -- 172.21.15.83:0/1596755282 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7fb0880ccc60 con 0x7fb08815c340 2024-04-25T00:14:48.862 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:48.862 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:14:48.863 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-25T00:14:48.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.863 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-25T00:14:48.864 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmytlRFRAA3vHLqK3vHSB+8+TZHAuj7Q==", 2024-04-25T00:14:48.864 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.865 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:48.865 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:14:48.866 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:14:48.867 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.867 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.868 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-25T00:14:48.868 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm088MIRAApJvvePXyFoO8JnVErd57zQ==", 2024-04-25T00:14:48.868 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.868 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:48.868 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:14:48.868 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:14:48.868 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.869 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.869 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-25T00:14:48.869 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmCqZXHBAAPy5JGbEU11i4gc3gFtbGIQ==", 2024-04-25T00:14:48.869 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.870 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:48.870 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:14:48.870 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:14:48.870 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.870 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.870 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-25T00:14:48.870 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmiJZGKBAA5CBwH/cA+LOAj+inS/Hx2g==", 2024-04-25T00:14:48.870 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.870 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:48.870 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:14:48.871 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:14:48.874 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.875 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.875 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-25T00:14:48.875 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBhnylmMOiqLRAAdqrCTz4KfghoJ7cZc+b4qg==", 2024-04-25T00:14:48.875 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.875 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:48.875 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:48.875 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.875 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.876 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-25T00:14:48.876 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBknylmlp8OOBAAl+y4vBrCzDyV/ne9qICQ+w==", 2024-04-25T00:14:48.876 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.876 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:48.876 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:48.876 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.876 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.877 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-25T00:14:48.877 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBonylm85MOBhAA3ciY1UT/Qg4Pe72OA0IvnA==", 2024-04-25T00:14:48.877 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.877 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:48.877 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:48.878 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.878 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.878 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-25T00:14:48.878 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBrnylmnAbDEhAAkCQXjbiKINiftX71HBKvQA==", 2024-04-25T00:14:48.878 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.878 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:48.878 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:48.878 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.878 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.879 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-25T00:14:48.879 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBxnylmxhy2KxAAs6qjxZA+Fj5umHU/4FVqGw==", 2024-04-25T00:14:48.879 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.879 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:48.879 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:48.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.879 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.880 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-25T00:14:48.880 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB0nylmGfSlNRAA45tuuh+5ISOMlhaFdCE5rQ==", 2024-04-25T00:14:48.880 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.880 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:48.880 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:48.880 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.881 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.881 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-25T00:14:48.881 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB4nylmouyBARAAz4s4S1BSiDU9YxQ/Vj0d0A==", 2024-04-25T00:14:48.881 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.881 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:48.881 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:48.881 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.881 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.882 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-25T00:14:48.882 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB7nylmtIh0CxAApWJLvC8Zlt298VJ55gPrqw==", 2024-04-25T00:14:48.882 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.882 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-25T00:14:48.882 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-25T00:14:48.882 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.882 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.882 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-25T00:14:48.882 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmOqooLBAApte+X6YkRmrpe12YdjYekg==", 2024-04-25T00:14:48.882 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.882 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:48.882 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-04-25T00:14:48.883 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:14:48.883 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.883 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-25T00:14:48.883 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmDH93NRAASQzJNC3nI0NeU8cv951eBA==", 2024-04-25T00:14:48.883 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.883 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:48.883 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:14:48.883 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:14:48.883 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:14:48.883 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylmHw3+MRAAciXRmexPxmrf7U2jKwpZQQ==", 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBfnylmIsMCABAAu+yzPRupFFqA/yH4GSZ+8w==", 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-25T00:14:48.884 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdnylmZ7SKNRAArVHYbh7fBiOUNRTenIwT1w==", 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.886 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-25T00:14:48.886 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm6TMqERAAiPL81HRFrlz4ku20HemS4w==", 2024-04-25T00:14:48.886 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.886 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:14:48.886 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-25T00:14:48.886 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.886 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:14:48.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:48.886 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-25T00:14:48.886 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBenylm9VRpDRAA5XUtYwjiWBpF1ObJ/Y5g0w==", 2024-04-25T00:14:48.886 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-25T00:14:48.886 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-25T00:14:48.886 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-25T00:14:48.887 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-25T00:14:48.887 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.887 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:48.887 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T00:14:48.887 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:14:48.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.859+0000 7fb08f600640 1 -- 172.21.15.83:0/1596755282 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb0640678c0 msgr2=0x7fb064069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:48.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.859+0000 7fb08f600640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb0640678c0 0x7fb064069d80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb078009ac0 tx=0x7fb078009190 comp rx=0 tx=0).stop 2024-04-25T00:14:48.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.859+0000 7fb08f600640 1 -- 172.21.15.83:0/1596755282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb08815c340 msgr2=0x7fb0881529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:48.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.859+0000 7fb08f600640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb08815c340 0x7fb0881529c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb07000c500 tx=0x7fb07000c9c0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.859+0000 7fb08f600640 1 -- 172.21.15.83:0/1596755282 shutdown_connections 2024-04-25T00:14:48.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.859+0000 7fb08f600640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb0640678c0 0x7fb064069d80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.859+0000 7fb08f600640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08815cec0 0x7fb088152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.859+0000 7fb08f600640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb08815c340 0x7fb0881529c0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.859+0000 7fb08f600640 1 --2- 172.21.15.83:0/1596755282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0881599f0 0x7fb0880b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:48.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.859+0000 7fb08f600640 1 -- 172.21.15.83:0/1596755282 >> 172.21.15.83:0/1596755282 conn(0x7fb0880b1830 msgr2=0x7fb0881664f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:48.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.859+0000 7fb08f600640 1 -- 172.21.15.83:0/1596755282 shutdown_connections 2024-04-25T00:14:48.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:48.859+0000 7fb08f600640 1 -- 172.21.15.83:0/1596755282 wait complete. 2024-04-25T00:14:48.891 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.023+0000 7f7806000640 1 Processor -- start 2024-04-25T00:14:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.023+0000 7f7806000640 1 -- start start 2024-04-25T00:14:49.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.023+0000 7f7806000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f78000cc510 0x7f780015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.023+0000 7f7806000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78000ccc60 con 0x7f780015cf90 2024-04-25T00:14:49.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.023+0000 7f7806000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f780006dba0 con 0x7f78001599f0 2024-04-25T00:14:49.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.023+0000 7f7806000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7800159dd0 con 0x7f78000cc510 2024-04-25T00:14:49.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.023+0000 7f77ffe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f780015cf90 0x7f78000ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54756/0 (socket says 172.21.15.83:54756) 2024-04-25T00:14:49.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.027+0000 7f77ffe00640 1 -- 172.21.15.83:0/3463438672 learned_addr learned my addr 172.21.15.83:0/3463438672 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:49.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.027+0000 7f77fea00640 1 --2- 172.21.15.83:0/3463438672 >> v2:172.21.15.83:3300/0 conn(0x7f78000cc510 0x7f780015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.027+0000 7f77fea00640 1 -- 172.21.15.83:0/3463438672 >> v1:172.21.15.86:6789/0 conn(0x7f78001599f0 legacy=0x7f780015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.027+0000 7f77fea00640 1 -- 172.21.15.83:0/3463438672 >> v1:172.21.15.83:6789/0 conn(0x7f780015cf90 legacy=0x7f78000ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.027+0000 7f77fea00640 1 -- 172.21.15.83:0/3463438672 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780006de80 con 0x7f78000cc510 2024-04-25T00:14:49.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.027+0000 7f77fe000640 1 -- 172.21.15.83:0/3463438672 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1024790718 0 0) 0x7f77f8001120 con 0x7f780015cf90 2024-04-25T00:14:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.027+0000 7f77fea00640 1 --2- 172.21.15.83:0/3463438672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78000cc510 0x7f780015ca50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0097f0 tx=0x7f77ec01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f08a58ee53a7702 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.027+0000 7f77fe000640 1 -- 172.21.15.83:0/3463438672 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0b0070 con 0x7f78000cc510 2024-04-25T00:14:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.027+0000 7f77fe000640 1 -- 172.21.15.83:0/3463438672 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77ec0ab600 con 0x7f78000cc510 2024-04-25T00:14:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.027+0000 7f77fe000640 1 -- 172.21.15.83:0/3463438672 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0ab9d0 con 0x7f78000cc510 2024-04-25T00:14:49.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.027+0000 7f7806000640 1 -- 172.21.15.83:0/3463438672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78000cc510 msgr2=0x7f780015ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.027+0000 7f7806000640 1 --2- 172.21.15.83:0/3463438672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78000cc510 0x7f780015ca50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0097f0 tx=0x7f77ec01c410 comp rx=0 tx=0).stop 2024-04-25T00:14:49.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f7806000640 1 -- 172.21.15.83:0/3463438672 shutdown_connections 2024-04-25T00:14:49.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f7806000640 1 --2- 172.21.15.83:0/3463438672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78000cc510 0x7f780015ca50 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f7806000640 1 -- 172.21.15.83:0/3463438672 >> 172.21.15.83:0/3463438672 conn(0x7f78000b1830 msgr2=0x7f78000b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:49.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f7806000640 1 -- 172.21.15.83:0/3463438672 shutdown_connections 2024-04-25T00:14:49.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f7806000640 1 -- 172.21.15.83:0/3463438672 wait complete. 2024-04-25T00:14:49.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f7806000640 1 Processor -- start 2024-04-25T00:14:49.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f7806000640 1 -- start start 2024-04-25T00:14:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f7806000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78000cc510 0x7f78001f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f7806000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78001599f0 0x7f7800167180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f7806000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780015cf90 0x7f7800150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f7806000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78000cf780 con 0x7f780015cf90 2024-04-25T00:14:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f7806000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78000cf600 con 0x7f78001599f0 2024-04-25T00:14:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f7806000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f78000cf900 con 0x7f78000cc510 2024-04-25T00:14:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77ffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780015cf90 0x7f7800150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77ffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780015cf90 0x7f7800150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38556/0 (socket says 172.21.15.83:38556) 2024-04-25T00:14:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77ffe00640 1 -- 172.21.15.83:0/3515165996 learned_addr learned my addr 172.21.15.83:0/3515165996 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77fea00640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78001599f0 0x7f7800167180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77ff400640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78000cc510 0x7f78001f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77ffe00640 1 -- 172.21.15.83:0/3515165996 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78000cc510 msgr2=0x7f78001f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77ffe00640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78000cc510 0x7f78001f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77ffe00640 1 -- 172.21.15.83:0/3515165996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78001599f0 msgr2=0x7f7800167180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77ffe00640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78001599f0 0x7f7800167180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77ffe00640 1 -- 172.21.15.83:0/3515165996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7800072a70 con 0x7f780015cf90 2024-04-25T00:14:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77fea00640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78001599f0 0x7f7800167180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77ff400640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78000cc510 0x7f78001f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77ffe00640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780015cf90 0x7f7800150330 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f77f800c410 tx=0x7f77f800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:49.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77f7400640 1 -- 172.21.15.83:0/3515165996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f80074e0 con 0x7f780015cf90 2024-04-25T00:14:49.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77f7400640 1 -- 172.21.15.83:0/3515165996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f80047f0 con 0x7f780015cf90 2024-04-25T00:14:49.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.031+0000 7f77f7400640 1 -- 172.21.15.83:0/3515165996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8007680 con 0x7f780015cf90 2024-04-25T00:14:49.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.035+0000 7f7806000640 1 -- 172.21.15.83:0/3515165996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780015d610 con 0x7f780015cf90 2024-04-25T00:14:49.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.035+0000 7f7806000640 1 -- 172.21.15.83:0/3515165996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780015dbd0 con 0x7f780015cf90 2024-04-25T00:14:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.035+0000 7f7806000640 1 -- 172.21.15.83:0/3515165996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78000737f0 con 0x7f780015cf90 2024-04-25T00:14:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.035+0000 7f77f7400640 1 -- 172.21.15.83:0/3515165996 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f77f8011070 con 0x7f780015cf90 2024-04-25T00:14:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.035+0000 7f77f7400640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f77e00678c0 0x7f77e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.035+0000 7f77f7400640 1 -- 172.21.15.83:0/3515165996 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5249+0+0 (secure 0 0 0) 0x7f77f80855e0 con 0x7f780015cf90 2024-04-25T00:14:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.035+0000 7f77ff400640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f77e00678c0 0x7f77e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.035+0000 7f77ff400640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f77e00678c0 0x7f77e0069d80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f77f0007fd0 tx=0x7f77f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:49.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.039+0000 7f77f7400640 1 -- 172.21.15.83:0/3515165996 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f78000ccc60 con 0x7f780015cf90 2024-04-25T00:14:49.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.195+0000 7f7806000640 1 -- 172.21.15.83:0/3515165996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f780004f5f0 con 0x7f780015cf90 2024-04-25T00:14:49.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.195+0000 7f77f7400640 1 -- 172.21.15.83:0/3515165996 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+13081 (secure 0 0 0) 0x7f77f805c070 con 0x7f780015cf90 2024-04-25T00:14:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:49.200 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:48.480144+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:49.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.199+0000 7f7806000640 1 -- 172.21.15.83:0/3515165996 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f77e00678c0 msgr2=0x7f77e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.199+0000 7f7806000640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f77e00678c0 0x7f77e0069d80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f77f0007fd0 tx=0x7f77f0009040 comp rx=0 tx=0).stop 2024-04-25T00:14:49.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.199+0000 7f7806000640 1 -- 172.21.15.83:0/3515165996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780015cf90 msgr2=0x7f7800150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.199+0000 7f7806000640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780015cf90 0x7f7800150330 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f77f800c410 tx=0x7f77f800c8d0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.199+0000 7f7806000640 1 -- 172.21.15.83:0/3515165996 shutdown_connections 2024-04-25T00:14:49.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.199+0000 7f7806000640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f77e00678c0 0x7f77e0069d80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.199+0000 7f7806000640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780015cf90 0x7f7800150330 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.199+0000 7f7806000640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78001599f0 0x7f7800167180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.199+0000 7f7806000640 1 --2- 172.21.15.83:0/3515165996 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78000cc510 0x7f78001f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.199+0000 7f7806000640 1 -- 172.21.15.83:0/3515165996 >> 172.21.15.83:0/3515165996 conn(0x7f78000b1830 msgr2=0x7f780015aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:49.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.199+0000 7f7806000640 1 -- 172.21.15.83:0/3515165996 shutdown_connections 2024-04-25T00:14:49.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.199+0000 7f7806000640 1 -- 172.21.15.83:0/3515165996 wait complete. 2024-04-25T00:14:49.220 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:14:49.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:14:49.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.355+0000 7f5092a00640 1 Processor -- start 2024-04-25T00:14:49.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.355+0000 7f5092a00640 1 -- start start 2024-04-25T00:14:49.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.355+0000 7f5092a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f508c1551e0 0x7f508c15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.355+0000 7f5092a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f508c06e4c0 0x7f508c06e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.355+0000 7f5092a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f508c15c560 con 0x7f508c1555c0 2024-04-25T00:14:49.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.355+0000 7f5092a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f508c06ee70 con 0x7f508c06e4c0 2024-04-25T00:14:49.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.355+0000 7f5092a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f508c072460 con 0x7f508c1551e0 2024-04-25T00:14:49.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.359+0000 7f5083400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f508c1555c0 0x7f508c071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:56966/0 (socket says 172.21.15.83:56966) 2024-04-25T00:14:49.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.359+0000 7f5083400640 1 -- 172.21.15.83:0/2168770325 learned_addr learned my addr 172.21.15.83:0/2168770325 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:49.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.359+0000 7f508be00640 1 --2- 172.21.15.83:0/2168770325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f508c1551e0 0x7f508c15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.359+0000 7f5090a00640 1 --2- 172.21.15.83:0/2168770325 >> v2:172.21.15.86:3300/0 conn(0x7f508c06e4c0 0x7f508c06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.359+0000 7f508b400640 1 -- 172.21.15.83:0/2168770325 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1307175258 0 0) 0x7f508c15c560 con 0x7f508c1555c0 2024-04-25T00:14:49.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.359+0000 7f508b400640 1 -- 172.21.15.83:0/2168770325 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5070003530 con 0x7f508c1555c0 2024-04-25T00:14:49.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.359+0000 7f508be00640 1 -- 172.21.15.83:0/2168770325 >> v2:172.21.15.86:3300/0 conn(0x7f508c06e4c0 msgr2=0x7f508c06e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:49.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.359+0000 7f508be00640 1 --2- 172.21.15.83:0/2168770325 >> v2:172.21.15.86:3300/0 conn(0x7f508c06e4c0 0x7f508c06e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.359+0000 7f508be00640 1 -- 172.21.15.83:0/2168770325 >> v1:172.21.15.86:6789/0 conn(0x7f508c1555c0 legacy=0x7f508c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.359+0000 7f508be00640 1 -- 172.21.15.83:0/2168770325 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c15c7d0 con 0x7f508c1551e0 2024-04-25T00:14:49.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.359+0000 7f508be00640 1 --2- 172.21.15.83:0/2168770325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f508c1551e0 0x7f508c15bea0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f507c009d90 tx=0x7f507c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c9d3e0a3c34b422 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:49.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.359+0000 7f508b400640 1 -- 172.21.15.83:0/2168770325 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c0b2070 con 0x7f508c1551e0 2024-04-25T00:14:49.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.359+0000 7f508b400640 1 -- 172.21.15.83:0/2168770325 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f507c0a5940 con 0x7f508c1551e0 2024-04-25T00:14:49.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.359+0000 7f508b400640 1 -- 172.21.15.83:0/2168770325 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c0a5d10 con 0x7f508c1551e0 2024-04-25T00:14:49.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 -- 172.21.15.83:0/2168770325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f508c1551e0 msgr2=0x7f508c15bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 --2- 172.21.15.83:0/2168770325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f508c1551e0 0x7f508c15bea0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f507c009d90 tx=0x7f507c01c970 comp rx=0 tx=0).stop 2024-04-25T00:14:49.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 -- 172.21.15.83:0/2168770325 shutdown_connections 2024-04-25T00:14:49.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 --2- 172.21.15.83:0/2168770325 >> v2:172.21.15.86:3300/0 conn(0x7f508c06e4c0 0x7f508c06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 --2- 172.21.15.83:0/2168770325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f508c1551e0 0x7f508c15bea0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 -- 172.21.15.83:0/2168770325 >> 172.21.15.83:0/2168770325 conn(0x7f508c0b1830 msgr2=0x7f508c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:49.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 -- 172.21.15.83:0/2168770325 shutdown_connections 2024-04-25T00:14:49.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 -- 172.21.15.83:0/2168770325 wait complete. 2024-04-25T00:14:49.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 Processor -- start 2024-04-25T00:14:49.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 -- start start 2024-04-25T00:14:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f508c06e4c0 0x7f508c15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f508c1551e0 0x7f508c1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f508c1555c0 0x7f508c1404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f508c15db70 con 0x7f508c06e4c0 2024-04-25T00:14:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f508c15d9f0 con 0x7f508c1555c0 2024-04-25T00:14:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5092a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f508c15dcf0 con 0x7f508c1551e0 2024-04-25T00:14:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f508be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f508c06e4c0 0x7f508c15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f508be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f508c06e4c0 0x7f508c15e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38560/0 (socket says 172.21.15.83:38560) 2024-04-25T00:14:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f508be00640 1 -- 172.21.15.83:0/3803843775 learned_addr learned my addr 172.21.15.83:0/3803843775 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5083400640 1 --2- 172.21.15.83:0/3803843775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f508c1551e0 0x7f508c1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f5090a00640 1 --2- 172.21.15.83:0/3803843775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f508c1555c0 0x7f508c1404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f508be00640 1 -- 172.21.15.83:0/3803843775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f508c1551e0 msgr2=0x7f508c1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f508be00640 1 --2- 172.21.15.83:0/3803843775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f508c1551e0 0x7f508c1534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f508be00640 1 -- 172.21.15.83:0/3803843775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f508c1555c0 msgr2=0x7f508c1404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f508be00640 1 --2- 172.21.15.83:0/3803843775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f508c1555c0 0x7f508c1404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.363+0000 7f508be00640 1 -- 172.21.15.83:0/3803843775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c140a00 con 0x7f508c06e4c0 2024-04-25T00:14:49.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.367+0000 7f508be00640 1 --2- 172.21.15.83:0/3803843775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f508c06e4c0 0x7f508c15e990 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f507c0b0040 tx=0x7f507c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:49.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.367+0000 7f5088c00640 1 -- 172.21.15.83:0/3803843775 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c0b2050 con 0x7f508c06e4c0 2024-04-25T00:14:49.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.367+0000 7f5088c00640 1 -- 172.21.15.83:0/3803843775 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f507c0be070 con 0x7f508c06e4c0 2024-04-25T00:14:49.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.367+0000 7f5088c00640 1 -- 172.21.15.83:0/3803843775 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c0ada90 con 0x7f508c06e4c0 2024-04-25T00:14:49.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.367+0000 7f5092a00640 1 -- 172.21.15.83:0/3803843775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508c14f8d0 con 0x7f508c06e4c0 2024-04-25T00:14:49.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.367+0000 7f5092a00640 1 -- 172.21.15.83:0/3803843775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508c14fe90 con 0x7f508c06e4c0 2024-04-25T00:14:49.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.367+0000 7f5092a00640 1 -- 172.21.15.83:0/3803843775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f508c04f510 con 0x7f508c06e4c0 2024-04-25T00:14:49.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.367+0000 7f5088c00640 1 -- 172.21.15.83:0/3803843775 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f507c0a57c0 con 0x7f508c06e4c0 2024-04-25T00:14:49.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.367+0000 7f5088c00640 1 --2- 172.21.15.83:0/3803843775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50680678f0 0x7f5068069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.367+0000 7f5088c00640 1 -- 172.21.15.83:0/3803843775 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5249+0+0 (secure 0 0 0) 0x7f507c120b60 con 0x7f508c06e4c0 2024-04-25T00:14:49.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.367+0000 7f5083400640 1 --2- 172.21.15.83:0/3803843775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50680678f0 0x7f5068069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.371+0000 7f5083400640 1 --2- 172.21.15.83:0/3803843775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50680678f0 0x7f5068069db0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5074007fd0 tx=0x7f5074009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.371+0000 7f5088c00640 1 -- 172.21.15.83:0/3803843775 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f507c016560 con 0x7f508c06e4c0 2024-04-25T00:14:49.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.535+0000 7f5092a00640 1 -- 172.21.15.83:0/3803843775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f508c0cfad0 con 0x7f508c06e4c0 2024-04-25T00:14:49.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.535+0000 7f5088c00640 1 -- 172.21.15.83:0/3803843775 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f507c0f1170 con 0x7f508c06e4c0 2024-04-25T00:14:49.539 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:14:49.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.535+0000 7f5092a00640 1 -- 172.21.15.83:0/3803843775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50680678f0 msgr2=0x7f5068069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.535+0000 7f5092a00640 1 --2- 172.21.15.83:0/3803843775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50680678f0 0x7f5068069db0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5074007fd0 tx=0x7f5074009040 comp rx=0 tx=0).stop 2024-04-25T00:14:49.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.535+0000 7f5092a00640 1 -- 172.21.15.83:0/3803843775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f508c06e4c0 msgr2=0x7f508c15e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.535+0000 7f5092a00640 1 --2- 172.21.15.83:0/3803843775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f508c06e4c0 0x7f508c15e990 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f507c0b0040 tx=0x7f507c019710 comp rx=0 tx=0).stop 2024-04-25T00:14:49.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.535+0000 7f5092a00640 1 -- 172.21.15.83:0/3803843775 shutdown_connections 2024-04-25T00:14:49.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.535+0000 7f5092a00640 1 --2- 172.21.15.83:0/3803843775 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50680678f0 0x7f5068069db0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.535+0000 7f5092a00640 1 --2- 172.21.15.83:0/3803843775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f508c1551e0 0x7f508c1534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.535+0000 7f5092a00640 1 --2- 172.21.15.83:0/3803843775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f508c1555c0 0x7f508c1404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.535+0000 7f5092a00640 1 --2- 172.21.15.83:0/3803843775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f508c06e4c0 0x7f508c15e990 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.535+0000 7f5092a00640 1 -- 172.21.15.83:0/3803843775 >> 172.21.15.83:0/3803843775 conn(0x7f508c0b1830 msgr2=0x7f508c0b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.535+0000 7f5092a00640 1 -- 172.21.15.83:0/3803843775 shutdown_connections 2024-04-25T00:14:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.535+0000 7f5092a00640 1 -- 172.21.15.83:0/3803843775 wait complete. 2024-04-25T00:14:49.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:14:49.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e22e00640 1 Processor -- start 2024-04-25T00:14:49.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e22e00640 1 -- start start 2024-04-25T00:14:49.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e22e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e1c1599f0 0x7f9e1c0cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e22e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9e1c15c340 0x7f9e1c15c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e22e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e1c15ccc0 0x7f9e1c15f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e22e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9e1c159dd0 con 0x7f9e1c15c340 2024-04-25T00:14:49.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e22e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9e1c0ccc60 con 0x7f9e1c15ccc0 2024-04-25T00:14:49.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e22e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9e1c06dba0 con 0x7f9e1c1599f0 2024-04-25T00:14:49.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e1b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9e1c15c340 0x7f9e1c15c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e1b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9e1c15c340 0x7f9e1c15c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38564/0 (socket says 172.21.15.83:38564) 2024-04-25T00:14:49.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e1b400640 1 -- 172.21.15.83:0/372591757 learned_addr learned my addr 172.21.15.83:0/372591757 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:49.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e1be00640 1 --2- 172.21.15.83:0/372591757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e1c1599f0 0x7f9e1c0cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e20e00640 1 --2- 172.21.15.83:0/372591757 >> v2:172.21.15.86:3300/0 conn(0x7f9e1c15ccc0 0x7f9e1c15f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e1b400640 1 -- 172.21.15.83:0/372591757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e1c1599f0 msgr2=0x7f9e1c0cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e1b400640 1 --2- 172.21.15.83:0/372591757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e1c1599f0 0x7f9e1c0cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e1b400640 1 -- 172.21.15.83:0/372591757 >> v2:172.21.15.86:3300/0 conn(0x7f9e1c15ccc0 msgr2=0x7f9e1c15f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e1b400640 1 --2- 172.21.15.83:0/372591757 >> v2:172.21.15.86:3300/0 conn(0x7f9e1c15ccc0 0x7f9e1c15f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e1b400640 1 -- 172.21.15.83:0/372591757 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c06de80 con 0x7f9e1c15c340 2024-04-25T00:14:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.691+0000 7f9e20e00640 1 --2- 172.21.15.83:0/372591757 >> v2:172.21.15.86:3300/0 conn(0x7f9e1c15ccc0 0x7f9e1c15f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:49.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.695+0000 7f9e1b400640 1 --2- 172.21.15.83:0/372591757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1c15c340 0x7f9e1c15c780 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009870 tx=0x7f9e0801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae034ef1fe5d67a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:49.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.695+0000 7f9e1aa00640 1 -- 172.21.15.83:0/372591757 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e080b2070 con 0x7f9e1c15c340 2024-04-25T00:14:49.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.695+0000 7f9e1aa00640 1 -- 172.21.15.83:0/372591757 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e080a9070 con 0x7f9e1c15c340 2024-04-25T00:14:49.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.695+0000 7f9e1aa00640 1 -- 172.21.15.83:0/372591757 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e080ad880 con 0x7f9e1c15c340 2024-04-25T00:14:49.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.695+0000 7f9e22e00640 1 -- 172.21.15.83:0/372591757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1c15c340 msgr2=0x7f9e1c15c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.695+0000 7f9e22e00640 1 --2- 172.21.15.83:0/372591757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1c15c340 0x7f9e1c15c780 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009870 tx=0x7f9e0801c490 comp rx=0 tx=0).stop 2024-04-25T00:14:49.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.695+0000 7f9e22e00640 1 -- 172.21.15.83:0/372591757 shutdown_connections 2024-04-25T00:14:49.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.695+0000 7f9e22e00640 1 --2- 172.21.15.83:0/372591757 >> v2:172.21.15.86:3300/0 conn(0x7f9e1c15ccc0 0x7f9e1c15f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.695+0000 7f9e22e00640 1 --2- 172.21.15.83:0/372591757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1c15c340 0x7f9e1c15c780 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.695+0000 7f9e22e00640 1 --2- 172.21.15.83:0/372591757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e1c1599f0 0x7f9e1c0cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.695+0000 7f9e22e00640 1 -- 172.21.15.83:0/372591757 >> 172.21.15.83:0/372591757 conn(0x7f9e1c0b1830 msgr2=0x7f9e1c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.695+0000 7f9e22e00640 1 -- 172.21.15.83:0/372591757 shutdown_connections 2024-04-25T00:14:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.695+0000 7f9e22e00640 1 -- 172.21.15.83:0/372591757 wait complete. 2024-04-25T00:14:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e22e00640 1 Processor -- start 2024-04-25T00:14:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e22e00640 1 -- start start 2024-04-25T00:14:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e22e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e1c1599f0 0x7f9e1c14d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e22e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1c15c340 0x7f9e1c1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e22e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c15ccc0 0x7f9e1c158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e22e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c0ce860 con 0x7f9e1c15c340 2024-04-25T00:14:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e22e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c0ce6e0 con 0x7f9e1c15ccc0 2024-04-25T00:14:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e22e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9e1c0ce9e0 con 0x7f9e1c1599f0 2024-04-25T00:14:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e1b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1c15c340 0x7f9e1c1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e1b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1c15c340 0x7f9e1c1581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38578/0 (socket says 172.21.15.83:38578) 2024-04-25T00:14:49.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e1b400640 1 -- 172.21.15.83:0/1131013126 learned_addr learned my addr 172.21.15.83:0/1131013126 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:49.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e1be00640 1 --2- 172.21.15.83:0/1131013126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e1c1599f0 0x7f9e1c14d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e20e00640 1 --2- 172.21.15.83:0/1131013126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c15ccc0 0x7f9e1c158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e1b400640 1 -- 172.21.15.83:0/1131013126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e1c1599f0 msgr2=0x7f9e1c14d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e1b400640 1 --2- 172.21.15.83:0/1131013126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e1c1599f0 0x7f9e1c14d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e1b400640 1 -- 172.21.15.83:0/1131013126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c15ccc0 msgr2=0x7f9e1c158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e1b400640 1 --2- 172.21.15.83:0/1131013126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c15ccc0 0x7f9e1c158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e1b400640 1 -- 172.21.15.83:0/1131013126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c1f5030 con 0x7f9e1c15c340 2024-04-25T00:14:49.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e1b400640 1 --2- 172.21.15.83:0/1131013126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1c15c340 0x7f9e1c1581f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f9e080b0040 tx=0x7f9e08019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:49.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e13e00640 1 -- 172.21.15.83:0/1131013126 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e080bb070 con 0x7f9e1c15c340 2024-04-25T00:14:49.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e13e00640 1 -- 172.21.15.83:0/1131013126 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e080addb0 con 0x7f9e1c15c340 2024-04-25T00:14:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e13e00640 1 -- 172.21.15.83:0/1131013126 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e080b2040 con 0x7f9e1c15c340 2024-04-25T00:14:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e22e00640 1 -- 172.21.15.83:0/1131013126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c1f52f0 con 0x7f9e1c15c340 2024-04-25T00:14:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e22e00640 1 -- 172.21.15.83:0/1131013126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c158d60 con 0x7f9e1c15c340 2024-04-25T00:14:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.699+0000 7f9e22e00640 1 -- 172.21.15.83:0/1131013126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1c0737f0 con 0x7f9e1c15c340 2024-04-25T00:14:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.703+0000 7f9e13e00640 1 -- 172.21.15.83:0/1131013126 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9e080be070 con 0x7f9e1c15c340 2024-04-25T00:14:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.703+0000 7f9e13e00640 1 --2- 172.21.15.83:0/1131013126 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9dfc0678c0 0x7f9dfc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.703+0000 7f9e13e00640 1 -- 172.21.15.83:0/1131013126 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5249+0+0 (secure 0 0 0) 0x7f9e0811ecb0 con 0x7f9e1c15c340 2024-04-25T00:14:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.703+0000 7f9e1be00640 1 --2- 172.21.15.83:0/1131013126 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9dfc0678c0 0x7f9dfc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:49.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.703+0000 7f9e1be00640 1 --2- 172.21.15.83:0/1131013126 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9dfc0678c0 0x7f9dfc069d80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c004610 tx=0x7f9e0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:49.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.707+0000 7f9e13e00640 1 -- 172.21.15.83:0/1131013126 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9e08016110 con 0x7f9e1c15c340 2024-04-25T00:14:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.863+0000 7f9e22e00640 1 -- 172.21.15.83:0/1131013126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9e1c04f5f0 con 0x7f9e1c15c340 2024-04-25T00:14:49.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.863+0000 7f9e13e00640 1 -- 172.21.15.83:0/1131013126 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v89) ==== 84+0+65 (secure 0 0 0) 0x7f9e080ef350 con 0x7f9e1c15c340 2024-04-25T00:14:49.869 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:49.869 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-25T00:14:49.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:14:49.869 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-25T00:14:49.869 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-25T00:14:49.869 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:14:49.869 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-25T00:14:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.867+0000 7f9e22e00640 1 -- 172.21.15.83:0/1131013126 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9dfc0678c0 msgr2=0x7f9dfc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.867+0000 7f9e22e00640 1 --2- 172.21.15.83:0/1131013126 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9dfc0678c0 0x7f9dfc069d80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c004610 tx=0x7f9e0c008040 comp rx=0 tx=0).stop 2024-04-25T00:14:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.867+0000 7f9e22e00640 1 -- 172.21.15.83:0/1131013126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1c15c340 msgr2=0x7f9e1c1581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.867+0000 7f9e22e00640 1 --2- 172.21.15.83:0/1131013126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1c15c340 0x7f9e1c1581f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f9e080b0040 tx=0x7f9e08019230 comp rx=0 tx=0).stop 2024-04-25T00:14:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.867+0000 7f9e22e00640 1 -- 172.21.15.83:0/1131013126 shutdown_connections 2024-04-25T00:14:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.867+0000 7f9e22e00640 1 --2- 172.21.15.83:0/1131013126 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9dfc0678c0 0x7f9dfc069d80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.867+0000 7f9e22e00640 1 --2- 172.21.15.83:0/1131013126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c15ccc0 0x7f9e1c158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.867+0000 7f9e22e00640 1 --2- 172.21.15.83:0/1131013126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1c15c340 0x7f9e1c1581f0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.867+0000 7f9e22e00640 1 --2- 172.21.15.83:0/1131013126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e1c1599f0 0x7f9e1c14d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.867+0000 7f9e22e00640 1 -- 172.21.15.83:0/1131013126 >> 172.21.15.83:0/1131013126 conn(0x7f9e1c0b1830 msgr2=0x7f9e1c1527b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.867+0000 7f9e22e00640 1 -- 172.21.15.83:0/1131013126 shutdown_connections 2024-04-25T00:14:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:49.867+0000 7f9e22e00640 1 -- 172.21.15.83:0/1131013126 wait complete. 2024-04-25T00:14:49.890 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:14:49.890 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:14:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f147a200640 1 Processor -- start 2024-04-25T00:14:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f147a200640 1 -- start start 2024-04-25T00:14:50.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f147a200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14741445e0 0x7f14740b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:50.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f147a200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14740cfd40 0x7f14740d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:50.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f147a200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f147415b480 con 0x7f1474072120 2024-04-25T00:14:50.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f147a200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f14740d0710 con 0x7f14740cfd40 2024-04-25T00:14:50.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f147a200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f147415b7d0 con 0x7f14741445e0 2024-04-25T00:14:50.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f1473e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14740cfd40 0x7f14740d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:50.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f1472a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1474072120 0x7f147415bc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:56982/0 (socket says 172.21.15.83:56982) 2024-04-25T00:14:50.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f1473e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14740cfd40 0x7f14740d01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:38746/0 (socket says 172.21.15.83:38746) 2024-04-25T00:14:50.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f1472a00640 1 -- 172.21.15.83:0/868032190 learned_addr learned my addr 172.21.15.83:0/868032190 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:50.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f1473400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14741445e0 0x7f14740b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:50.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f1472000640 1 -- 172.21.15.83:0/868032190 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2482177317 0 0) 0x7f147415b480 con 0x7f1474072120 2024-04-25T00:14:50.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f1472000640 1 -- 172.21.15.83:0/868032190 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1458005740 con 0x7f1474072120 2024-04-25T00:14:50.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f1473400640 1 -- 172.21.15.83:0/868032190 >> v2:172.21.15.86:3300/0 conn(0x7f14740cfd40 msgr2=0x7f14740d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f1473400640 1 --2- 172.21.15.83:0/868032190 >> v2:172.21.15.86:3300/0 conn(0x7f14740cfd40 0x7f14740d01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f1473400640 1 -- 172.21.15.83:0/868032190 >> v1:172.21.15.86:6789/0 conn(0x7f1474072120 legacy=0x7f147415bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.027+0000 7f1473400640 1 -- 172.21.15.83:0/868032190 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147415c370 con 0x7f14741445e0 2024-04-25T00:14:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.031+0000 7f1473400640 1 --2- 172.21.15.83:0/868032190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14741445e0 0x7f14740b4330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f14640097c0 tx=0x7f146401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbc6ac9671c1b4ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.031+0000 7f1472000640 1 -- 172.21.15.83:0/868032190 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14640b0070 con 0x7f14741445e0 2024-04-25T00:14:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.031+0000 7f1472000640 1 -- 172.21.15.83:0/868032190 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14640ab600 con 0x7f14741445e0 2024-04-25T00:14:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.031+0000 7f1472000640 1 -- 172.21.15.83:0/868032190 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14640ab9d0 con 0x7f14741445e0 2024-04-25T00:14:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.031+0000 7f147a200640 1 -- 172.21.15.83:0/868032190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14741445e0 msgr2=0x7f14740b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.031+0000 7f147a200640 1 --2- 172.21.15.83:0/868032190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14741445e0 0x7f14740b4330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f14640097c0 tx=0x7f146401c410 comp rx=0 tx=0).stop 2024-04-25T00:14:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.031+0000 7f147a200640 1 -- 172.21.15.83:0/868032190 shutdown_connections 2024-04-25T00:14:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.031+0000 7f147a200640 1 --2- 172.21.15.83:0/868032190 >> v2:172.21.15.86:3300/0 conn(0x7f14740cfd40 0x7f14740d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.031+0000 7f147a200640 1 --2- 172.21.15.83:0/868032190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14741445e0 0x7f14740b4330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.031+0000 7f147a200640 1 -- 172.21.15.83:0/868032190 >> 172.21.15.83:0/868032190 conn(0x7f147406b3b0 msgr2=0x7f147406b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.031+0000 7f147a200640 1 -- 172.21.15.83:0/868032190 shutdown_connections 2024-04-25T00:14:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.031+0000 7f147a200640 1 -- 172.21.15.83:0/868032190 wait complete. 2024-04-25T00:14:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f147a200640 1 Processor -- start 2024-04-25T00:14:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f147a200640 1 -- start start 2024-04-25T00:14:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f147a200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1474072120 0x7f1474139fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f147a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14740cfd40 0x7f147415e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f147a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14741445e0 0x7f1474142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f147a200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1474072520 con 0x7f14741445e0 2024-04-25T00:14:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f147a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1474072ed0 con 0x7f14740cfd40 2024-04-25T00:14:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f147a200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f14740726a0 con 0x7f1474072120 2024-04-25T00:14:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f1473e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14741445e0 0x7f1474142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f1473e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14741445e0 0x7f1474142200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38586/0 (socket says 172.21.15.83:38586) 2024-04-25T00:14:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f1473e00640 1 -- 172.21.15.83:0/95685606 learned_addr learned my addr 172.21.15.83:0/95685606 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f1472a00640 1 --2- 172.21.15.83:0/95685606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14740cfd40 0x7f147415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f1473400640 1 --2- 172.21.15.83:0/95685606 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1474072120 0x7f1474139fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f1473e00640 1 -- 172.21.15.83:0/95685606 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1474072120 msgr2=0x7f1474139fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f1473e00640 1 --2- 172.21.15.83:0/95685606 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1474072120 0x7f1474139fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f1473e00640 1 -- 172.21.15.83:0/95685606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14740cfd40 msgr2=0x7f147415e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f1473e00640 1 --2- 172.21.15.83:0/95685606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14740cfd40 0x7f147415e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f1473e00640 1 -- 172.21.15.83:0/95685606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147413a5b0 con 0x7f14741445e0 2024-04-25T00:14:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f1473e00640 1 --2- 172.21.15.83:0/95685606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14741445e0 0x7f1474142200 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f146c009dd0 tx=0x7f146c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f146b400640 1 -- 172.21.15.83:0/95685606 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c018020 con 0x7f14741445e0 2024-04-25T00:14:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f146b400640 1 -- 172.21.15.83:0/95685606 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f146c014480 con 0x7f14741445e0 2024-04-25T00:14:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f147a200640 1 -- 172.21.15.83:0/95685606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147413a890 con 0x7f14741445e0 2024-04-25T00:14:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f146b400640 1 -- 172.21.15.83:0/95685606 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c014870 con 0x7f14741445e0 2024-04-25T00:14:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.035+0000 7f147a200640 1 -- 172.21.15.83:0/95685606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f147413b130 con 0x7f14741445e0 2024-04-25T00:14:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.039+0000 7f147a200640 1 -- 172.21.15.83:0/95685606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14740736f0 con 0x7f14741445e0 2024-04-25T00:14:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.039+0000 7f146b400640 1 -- 172.21.15.83:0/95685606 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f146c01b020 con 0x7f14741445e0 2024-04-25T00:14:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.039+0000 7f146b400640 1 --2- 172.21.15.83:0/95685606 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f14540678c0 0x7f1454069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.039+0000 7f146b400640 1 -- 172.21.15.83:0/95685606 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5249+0+0 (secure 0 0 0) 0x7f146c086530 con 0x7f14741445e0 2024-04-25T00:14:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.039+0000 7f1473400640 1 --2- 172.21.15.83:0/95685606 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f14540678c0 0x7f1454069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.039+0000 7f1473400640 1 --2- 172.21.15.83:0/95685606 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f14540678c0 0x7f1454069d80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f14640ad000 tx=0x7f14640a4f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.043+0000 7f146b400640 1 -- 172.21.15.83:0/95685606 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f146c022070 con 0x7f14741445e0 2024-04-25T00:14:50.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.211+0000 7f147a200640 1 -- 172.21.15.83:0/95685606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f147413aa60 con 0x7f14741445e0 2024-04-25T00:14:50.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.483+0000 7f146b400640 1 -- 172.21.15.83:0/95685606 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v90) ==== 138+0+0 (secure 0 0 0) 0x7f146c010040 con 0x7f14741445e0 2024-04-25T00:14:50.487 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-25T00:14:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.483+0000 7f147a200640 1 -- 172.21.15.83:0/95685606 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f14540678c0 msgr2=0x7f1454069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.483+0000 7f147a200640 1 --2- 172.21.15.83:0/95685606 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f14540678c0 0x7f1454069d80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f14640ad000 tx=0x7f14640a4f90 comp rx=0 tx=0).stop 2024-04-25T00:14:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.483+0000 7f147a200640 1 -- 172.21.15.83:0/95685606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14741445e0 msgr2=0x7f1474142200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.483+0000 7f147a200640 1 --2- 172.21.15.83:0/95685606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14741445e0 0x7f1474142200 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f146c009dd0 tx=0x7f146c00c490 comp rx=0 tx=0).stop 2024-04-25T00:14:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.483+0000 7f147a200640 1 -- 172.21.15.83:0/95685606 shutdown_connections 2024-04-25T00:14:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.483+0000 7f147a200640 1 --2- 172.21.15.83:0/95685606 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f14540678c0 0x7f1454069d80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.483+0000 7f147a200640 1 --2- 172.21.15.83:0/95685606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14741445e0 0x7f1474142200 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.483+0000 7f147a200640 1 --2- 172.21.15.83:0/95685606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14740cfd40 0x7f147415e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.483+0000 7f147a200640 1 --2- 172.21.15.83:0/95685606 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1474072120 0x7f1474139fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.483+0000 7f147a200640 1 -- 172.21.15.83:0/95685606 >> 172.21.15.83:0/95685606 conn(0x7f147406b3b0 msgr2=0x7f1474143350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.487+0000 7f147a200640 1 -- 172.21.15.83:0/95685606 shutdown_connections 2024-04-25T00:14:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.487+0000 7f147a200640 1 -- 172.21.15.83:0/95685606 wait complete. 2024-04-25T00:14:50.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:14:50.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f7075a00640 1 Processor -- start 2024-04-25T00:14:50.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f7075a00640 1 -- start start 2024-04-25T00:14:50.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f7075a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f70701555c0 0x7f707015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:50.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f7075a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f707006e4c0 0x7f707006e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:50.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f7075a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f707015c560 con 0x7f70701551e0 2024-04-25T00:14:50.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f7075a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7070072460 con 0x7f70701555c0 2024-04-25T00:14:50.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f7075a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f707015c7d0 con 0x7f707006e4c0 2024-04-25T00:14:50.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f706ea00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f70701551e0 0x7f7070071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54766/0 (socket says 172.21.15.83:54766) 2024-04-25T00:14:50.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f706ea00640 1 -- 172.21.15.83:0/3736595803 learned_addr learned my addr 172.21.15.83:0/3736595803 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:50.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f7066000640 1 --2- 172.21.15.83:0/3736595803 >> v2:172.21.15.83:3300/0 conn(0x7f70701555c0 0x7f707015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:50.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f706f400640 1 --2- 172.21.15.83:0/3736595803 >> v2:172.21.15.86:3300/0 conn(0x7f707006e4c0 0x7f707006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:50.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f706e000640 1 -- 172.21.15.83:0/3736595803 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 411801806 0 0) 0x7f707015c560 con 0x7f70701551e0 2024-04-25T00:14:50.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f706e000640 1 -- 172.21.15.83:0/3736595803 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7054003530 con 0x7f70701551e0 2024-04-25T00:14:50.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f7066000640 1 -- 172.21.15.83:0/3736595803 >> v2:172.21.15.86:3300/0 conn(0x7f707006e4c0 msgr2=0x7f707006e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:50.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f7066000640 1 --2- 172.21.15.83:0/3736595803 >> v2:172.21.15.86:3300/0 conn(0x7f707006e4c0 0x7f707006e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:50.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f7066000640 1 -- 172.21.15.83:0/3736595803 >> v1:172.21.15.83:6789/0 conn(0x7f70701551e0 legacy=0x7f7070071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:50.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.655+0000 7f7066000640 1 -- 172.21.15.83:0/3736595803 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707015cab0 con 0x7f70701555c0 2024-04-25T00:14:50.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.659+0000 7f7066000640 1 --2- 172.21.15.83:0/3736595803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70701555c0 0x7f707015bea0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f70580097c0 tx=0x7f705801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df91fa0fd32bee71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:50.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.659+0000 7f706e000640 1 -- 172.21.15.83:0/3736595803 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70580b0070 con 0x7f70701555c0 2024-04-25T00:14:50.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.659+0000 7f706e000640 1 -- 172.21.15.83:0/3736595803 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70580ab600 con 0x7f70701555c0 2024-04-25T00:14:50.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.659+0000 7f706e000640 1 -- 172.21.15.83:0/3736595803 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70580ab9d0 con 0x7f70701555c0 2024-04-25T00:14:50.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.659+0000 7f7075a00640 1 -- 172.21.15.83:0/3736595803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70701555c0 msgr2=0x7f707015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:50.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.659+0000 7f7075a00640 1 --2- 172.21.15.83:0/3736595803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70701555c0 0x7f707015bea0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f70580097c0 tx=0x7f705801c460 comp rx=0 tx=0).stop 2024-04-25T00:14:50.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.659+0000 7f7075a00640 1 -- 172.21.15.83:0/3736595803 shutdown_connections 2024-04-25T00:14:50.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.659+0000 7f7075a00640 1 --2- 172.21.15.83:0/3736595803 >> v2:172.21.15.86:3300/0 conn(0x7f707006e4c0 0x7f707006e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:50.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.659+0000 7f7075a00640 1 --2- 172.21.15.83:0/3736595803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70701555c0 0x7f707015bea0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:50.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.659+0000 7f7075a00640 1 -- 172.21.15.83:0/3736595803 >> 172.21.15.83:0/3736595803 conn(0x7f70700b1830 msgr2=0x7f70700b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:50.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.659+0000 7f7075a00640 1 -- 172.21.15.83:0/3736595803 shutdown_connections 2024-04-25T00:14:50.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.659+0000 7f7075a00640 1 -- 172.21.15.83:0/3736595803 wait complete. 2024-04-25T00:14:50.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7075a00640 1 Processor -- start 2024-04-25T00:14:50.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7075a00640 1 -- start start 2024-04-25T00:14:50.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7075a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707006e4c0 0x7f70701477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:50.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7075a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70701551e0 0x7f707014ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:50.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7075a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70701555c0 0x7f7070153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:50.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7075a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f707015dbb0 con 0x7f70701551e0 2024-04-25T00:14:50.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7075a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f707015da30 con 0x7f707006e4c0 2024-04-25T00:14:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7075a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f707015dd30 con 0x7f70701555c0 2024-04-25T00:14:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7066000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70701551e0 0x7f707014ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7066000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70701551e0 0x7f707014ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38596/0 (socket says 172.21.15.83:38596) 2024-04-25T00:14:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7066000640 1 -- 172.21.15.83:0/701723270 learned_addr learned my addr 172.21.15.83:0/701723270 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f706ea00640 1 --2- 172.21.15.83:0/701723270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707006e4c0 0x7f70701477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f706f400640 1 --2- 172.21.15.83:0/701723270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70701555c0 0x7f7070153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7066000640 1 -- 172.21.15.83:0/701723270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70701555c0 msgr2=0x7f7070153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7066000640 1 --2- 172.21.15.83:0/701723270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70701555c0 0x7f7070153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7066000640 1 -- 172.21.15.83:0/701723270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707006e4c0 msgr2=0x7f70701477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7066000640 1 --2- 172.21.15.83:0/701723270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707006e4c0 0x7f70701477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7066000640 1 -- 172.21.15.83:0/701723270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70700b2b70 con 0x7f70701551e0 2024-04-25T00:14:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7066000640 1 --2- 172.21.15.83:0/701723270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70701551e0 0x7f707014ab40 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f7058019070 tx=0x7f7058019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:50.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7067400640 1 -- 172.21.15.83:0/701723270 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70580b0040 con 0x7f70701551e0 2024-04-25T00:14:50.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7067400640 1 -- 172.21.15.83:0/701723270 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70580ba070 con 0x7f70701551e0 2024-04-25T00:14:50.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7067400640 1 -- 172.21.15.83:0/701723270 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70580b4830 con 0x7f70701551e0 2024-04-25T00:14:50.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7075a00640 1 -- 172.21.15.83:0/701723270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70700b2e50 con 0x7f70701551e0 2024-04-25T00:14:50.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7075a00640 1 -- 172.21.15.83:0/701723270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70700b3410 con 0x7f70701551e0 2024-04-25T00:14:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.663+0000 7f7075a00640 1 -- 172.21.15.83:0/701723270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f707004f510 con 0x7f70701551e0 2024-04-25T00:14:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.667+0000 7f7067400640 1 -- 172.21.15.83:0/701723270 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f70580ab4c0 con 0x7f70701551e0 2024-04-25T00:14:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.667+0000 7f7067400640 1 --2- 172.21.15.83:0/701723270 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f704c0678c0 0x7f704c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.667+0000 7f7067400640 1 -- 172.21.15.83:0/701723270 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5619+0+0 (secure 0 0 0) 0x7f705811f920 con 0x7f70701551e0 2024-04-25T00:14:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.667+0000 7f706ea00640 1 --2- 172.21.15.83:0/701723270 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f704c0678c0 0x7f704c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.667+0000 7f706ea00640 1 --2- 172.21.15.83:0/701723270 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f704c0678c0 0x7f704c069d80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7060007fd0 tx=0x7f7060009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:50.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.671+0000 7f7067400640 1 -- 172.21.15.83:0/701723270 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7058016050 con 0x7f70701551e0 2024-04-25T00:14:50.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:50.831+0000 7f7075a00640 1 -- 172.21.15.83:0/701723270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f707006ce30 con 0x7f70701551e0 2024-04-25T00:14:51.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.487+0000 7f7067400640 1 -- 172.21.15.83:0/701723270 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v91) ==== 170+0+0 (secure 0 0 0) 0x7f70580efdd0 con 0x7f70701551e0 2024-04-25T00:14:51.493 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-25T00:14:51.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.491+0000 7f7075a00640 1 -- 172.21.15.83:0/701723270 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f704c0678c0 msgr2=0x7f704c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.491+0000 7f7075a00640 1 --2- 172.21.15.83:0/701723270 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f704c0678c0 0x7f704c069d80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7060007fd0 tx=0x7f7060009040 comp rx=0 tx=0).stop 2024-04-25T00:14:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.491+0000 7f7075a00640 1 -- 172.21.15.83:0/701723270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70701551e0 msgr2=0x7f707014ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.491+0000 7f7075a00640 1 --2- 172.21.15.83:0/701723270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70701551e0 0x7f707014ab40 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f7058019070 tx=0x7f7058019200 comp rx=0 tx=0).stop 2024-04-25T00:14:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.491+0000 7f7075a00640 1 -- 172.21.15.83:0/701723270 shutdown_connections 2024-04-25T00:14:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.491+0000 7f7075a00640 1 --2- 172.21.15.83:0/701723270 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f704c0678c0 0x7f704c069d80 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.491+0000 7f7075a00640 1 --2- 172.21.15.83:0/701723270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f70701555c0 0x7f7070153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.491+0000 7f7075a00640 1 --2- 172.21.15.83:0/701723270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70701551e0 0x7f707014ab40 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.491+0000 7f7075a00640 1 --2- 172.21.15.83:0/701723270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707006e4c0 0x7f70701477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.491+0000 7f7075a00640 1 -- 172.21.15.83:0/701723270 >> 172.21.15.83:0/701723270 conn(0x7f70700b1830 msgr2=0x7f70700713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.491+0000 7f7075a00640 1 -- 172.21.15.83:0/701723270 shutdown_connections 2024-04-25T00:14:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.491+0000 7f7075a00640 1 -- 172.21.15.83:0/701723270 wait complete. 2024-04-25T00:14:51.535 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:14:51.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.667+0000 7f56cac00640 1 Processor -- start 2024-04-25T00:14:51.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.667+0000 7f56cac00640 1 -- start start 2024-04-25T00:14:51.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.671+0000 7f56cac00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f56c415de90 0x7f56c4071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:51.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.671+0000 7f56cac00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f56c415e270 0x7f56c4071c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:51.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.671+0000 7f56cac00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56c406dcf0 con 0x7f56c406e840 2024-04-25T00:14:51.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.671+0000 7f56cac00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f56c40b4460 con 0x7f56c415de90 2024-04-25T00:14:51.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.671+0000 7f56cac00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f56c40b2ce0 con 0x7f56c415e270 2024-04-25T00:14:51.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.671+0000 7f56c3e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f56c415de90 0x7f56c4071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:51.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.671+0000 7f56c8c00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f56c406e840 0x7f56c4161bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54772/0 (socket says 172.21.15.83:54772) 2024-04-25T00:14:51.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.671+0000 7f56c8c00640 1 -- 172.21.15.83:0/2602369470 learned_addr learned my addr 172.21.15.83:0/2602369470 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:51.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.671+0000 7f56bb400640 1 --2- 172.21.15.83:0/2602369470 >> v2:172.21.15.86:3300/0 conn(0x7f56c415e270 0x7f56c4071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:51.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.671+0000 7f56c3400640 1 -- 172.21.15.83:0/2602369470 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3156271216 0 0) 0x7f56c406dcf0 con 0x7f56c406e840 2024-04-25T00:14:51.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.671+0000 7f56c3e00640 1 -- 172.21.15.83:0/2602369470 >> v2:172.21.15.86:3300/0 conn(0x7f56c415e270 msgr2=0x7f56c4071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:51.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.671+0000 7f56c3e00640 1 --2- 172.21.15.83:0/2602369470 >> v2:172.21.15.86:3300/0 conn(0x7f56c415e270 0x7f56c4071c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:51.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.671+0000 7f56c3e00640 1 -- 172.21.15.83:0/2602369470 >> v1:172.21.15.83:6789/0 conn(0x7f56c406e840 legacy=0x7f56c4161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:51.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.671+0000 7f56c3e00640 1 -- 172.21.15.83:0/2602369470 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c406dfd0 con 0x7f56c415de90 2024-04-25T00:14:51.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56c3e00640 1 --2- 172.21.15.83:0/2602369470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c415de90 0x7f56c4071740 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f56c40736c0 tx=0x7f56b401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cb9652e2c9bf5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:51.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56c3400640 1 -- 172.21.15.83:0/2602369470 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b40b2070 con 0x7f56c415de90 2024-04-25T00:14:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56c3400640 1 -- 172.21.15.83:0/2602369470 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b40a5850 con 0x7f56c415de90 2024-04-25T00:14:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56c3400640 1 -- 172.21.15.83:0/2602369470 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b40a5c20 con 0x7f56c415de90 2024-04-25T00:14:51.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 -- 172.21.15.83:0/2602369470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c415de90 msgr2=0x7f56c4071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:51.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 --2- 172.21.15.83:0/2602369470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c415de90 0x7f56c4071740 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f56c40736c0 tx=0x7f56b401c8a0 comp rx=0 tx=0).stop 2024-04-25T00:14:51.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 -- 172.21.15.83:0/2602369470 shutdown_connections 2024-04-25T00:14:51.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 --2- 172.21.15.83:0/2602369470 >> v2:172.21.15.86:3300/0 conn(0x7f56c415e270 0x7f56c4071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:51.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 --2- 172.21.15.83:0/2602369470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c415de90 0x7f56c4071740 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:51.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 -- 172.21.15.83:0/2602369470 >> 172.21.15.83:0/2602369470 conn(0x7f56c40b1830 msgr2=0x7f56c40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:51.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 -- 172.21.15.83:0/2602369470 shutdown_connections 2024-04-25T00:14:51.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 -- 172.21.15.83:0/2602369470 wait complete. 2024-04-25T00:14:51.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 Processor -- start 2024-04-25T00:14:51.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 -- start start 2024-04-25T00:14:51.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56c406e840 0x7f56c4152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:51.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c415de90 0x7f56c4162d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:51.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c415e270 0x7f56c4163260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:51.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f56c416ad60 con 0x7f56c415de90 2024-04-25T00:14:51.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40ad860 con 0x7f56c415e270 2024-04-25T00:14:51.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.675+0000 7f56cac00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f56c416aee0 con 0x7f56c406e840 2024-04-25T00:14:51.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56bb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c415de90 0x7f56c4162d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:51.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56bb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c415de90 0x7f56c4162d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38618/0 (socket says 172.21.15.83:38618) 2024-04-25T00:14:51.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56bb400640 1 -- 172.21.15.83:0/571579465 learned_addr learned my addr 172.21.15.83:0/571579465 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:51.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56c3e00640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56c406e840 0x7f56c4152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:51.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56c8c00640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c415e270 0x7f56c4163260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:51.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56bb400640 1 -- 172.21.15.83:0/571579465 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56c406e840 msgr2=0x7f56c4152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:51.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56bb400640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56c406e840 0x7f56c4152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:51.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56bb400640 1 -- 172.21.15.83:0/571579465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c415e270 msgr2=0x7f56c4163260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:51.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56bb400640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c415e270 0x7f56c4163260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:51.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56bb400640 1 -- 172.21.15.83:0/571579465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c415f080 con 0x7f56c415de90 2024-04-25T00:14:51.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56c8c00640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c415e270 0x7f56c4163260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:51.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56bb400640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c415de90 0x7f56c4162d20 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00d800 tx=0x7f56ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:51.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56c0c00640 1 -- 172.21.15.83:0/571579465 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00b6c0 con 0x7f56c415de90 2024-04-25T00:14:51.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56c0c00640 1 -- 172.21.15.83:0/571579465 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ac019070 con 0x7f56c415de90 2024-04-25T00:14:51.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56c0c00640 1 -- 172.21.15.83:0/571579465 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00bae0 con 0x7f56c415de90 2024-04-25T00:14:51.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56cac00640 1 -- 172.21.15.83:0/571579465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c415f360 con 0x7f56c415de90 2024-04-25T00:14:51.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56cac00640 1 -- 172.21.15.83:0/571579465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c415f920 con 0x7f56c415de90 2024-04-25T00:14:51.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56cac00640 1 -- 172.21.15.83:0/571579465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c404f510 con 0x7f56c415de90 2024-04-25T00:14:51.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.679+0000 7f56c0c00640 1 -- 172.21.15.83:0/571579465 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f56ac00bc80 con 0x7f56c415de90 2024-04-25T00:14:51.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.683+0000 7f56c0c00640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f56a00678c0 0x7f56a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:51.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.683+0000 7f56c0c00640 1 -- 172.21.15.83:0/571579465 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6001+0+0 (secure 0 0 0) 0x7f56ac086d50 con 0x7f56c415de90 2024-04-25T00:14:51.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.683+0000 7f56c3e00640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f56a00678c0 0x7f56a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:51.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.683+0000 7f56c3e00640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f56a00678c0 0x7f56a0069d80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f56b40af000 tx=0x7f56b40a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:51.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.687+0000 7f56c0c00640 1 -- 172.21.15.83:0/571579465 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f56ac057740 con 0x7f56c415de90 2024-04-25T00:14:51.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:51.871+0000 7f56cac00640 1 -- 172.21.15.83:0/571579465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f56c4153ce0 con 0x7f56c415de90 2024-04-25T00:14:52.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.511+0000 7f56c0c00640 1 -- 172.21.15.83:0/571579465 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v58) ==== 174+0+0 (secure 0 0 0) 0x7f56ac057040 con 0x7f56c415de90 2024-04-25T00:14:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T00:14:52.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.515+0000 7f56cac00640 1 -- 172.21.15.83:0/571579465 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f56a00678c0 msgr2=0x7f56a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:52.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.515+0000 7f56cac00640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f56a00678c0 0x7f56a0069d80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f56b40af000 tx=0x7f56b40a7070 comp rx=0 tx=0).stop 2024-04-25T00:14:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.515+0000 7f56cac00640 1 -- 172.21.15.83:0/571579465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c415de90 msgr2=0x7f56c4162d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.515+0000 7f56cac00640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c415de90 0x7f56c4162d20 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00d800 tx=0x7f56ac00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:14:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.515+0000 7f56cac00640 1 -- 172.21.15.83:0/571579465 shutdown_connections 2024-04-25T00:14:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.515+0000 7f56cac00640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f56a00678c0 0x7f56a0069d80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.515+0000 7f56cac00640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c415e270 0x7f56c4163260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.515+0000 7f56cac00640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c415de90 0x7f56c4162d20 secure :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00d800 tx=0x7f56ac00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:14:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.515+0000 7f56cac00640 1 --2- 172.21.15.83:0/571579465 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56c406e840 0x7f56c4152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.515+0000 7f56cac00640 1 -- 172.21.15.83:0/571579465 >> 172.21.15.83:0/571579465 conn(0x7f56c40b1830 msgr2=0x7f56c4160e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:52.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.519+0000 7f56cac00640 1 -- 172.21.15.83:0/571579465 shutdown_connections 2024-04-25T00:14:52.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.519+0000 7f56cac00640 1 -- 172.21.15.83:0/571579465 wait complete. 2024-04-25T00:14:52.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:52.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.691+0000 7f7711a00640 1 Processor -- start 2024-04-25T00:14:52.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.691+0000 7f7711a00640 1 -- start start 2024-04-25T00:14:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.695+0000 7f7711a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f770c144470 0x7f770c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.695+0000 7f7711a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f770c0b34f0 0x7f770c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.695+0000 7f7711a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f770c0b3f40 con 0x7f770c0b3970 2024-04-25T00:14:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.695+0000 7f7711a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f770c144850 con 0x7f770c144470 2024-04-25T00:14:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.695+0000 7f7711a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f770c0b4220 con 0x7f770c0b34f0 2024-04-25T00:14:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.695+0000 7f770be00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f770c144470 0x7f770c15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.695+0000 7f770be00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f770c144470 0x7f770c15f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38628/0 (socket says 172.21.15.83:38628) 2024-04-25T00:14:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.695+0000 7f770be00640 1 -- 172.21.15.83:0/679230266 learned_addr learned my addr 172.21.15.83:0/679230266 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.695+0000 7f7703e00640 1 --2- 172.21.15.83:0/679230266 >> v2:172.21.15.86:3300/0 conn(0x7f770c0b34f0 0x7f770c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.695+0000 7f770be00640 1 -- 172.21.15.83:0/679230266 >> v2:172.21.15.86:3300/0 conn(0x7f770c0b34f0 msgr2=0x7f770c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.695+0000 7f770be00640 1 --2- 172.21.15.83:0/679230266 >> v2:172.21.15.86:3300/0 conn(0x7f770c0b34f0 0x7f770c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.695+0000 7f770be00640 1 -- 172.21.15.83:0/679230266 >> v1:172.21.15.86:6789/0 conn(0x7f770c0b3970 legacy=0x7f770c0ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.695+0000 7f770be00640 1 -- 172.21.15.83:0/679230266 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c15e5e0 con 0x7f770c144470 2024-04-25T00:14:52.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.699+0000 7f770be00640 1 --2- 172.21.15.83:0/679230266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c144470 0x7f770c15f0c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f770c0d06c0 tx=0x7f76fc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9511f2d6467659f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:52.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.699+0000 7f770b400640 1 -- 172.21.15.83:0/679230266 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc0b0070 con 0x7f770c144470 2024-04-25T00:14:52.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.699+0000 7f770b400640 1 -- 172.21.15.83:0/679230266 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76fc0ab600 con 0x7f770c144470 2024-04-25T00:14:52.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.699+0000 7f770b400640 1 -- 172.21.15.83:0/679230266 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc0ab9d0 con 0x7f770c144470 2024-04-25T00:14:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.699+0000 7f7711a00640 1 -- 172.21.15.83:0/679230266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c144470 msgr2=0x7f770c15f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.699+0000 7f7711a00640 1 --2- 172.21.15.83:0/679230266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c144470 0x7f770c15f0c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f770c0d06c0 tx=0x7f76fc01c460 comp rx=0 tx=0).stop 2024-04-25T00:14:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.699+0000 7f7711a00640 1 -- 172.21.15.83:0/679230266 shutdown_connections 2024-04-25T00:14:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.699+0000 7f7711a00640 1 --2- 172.21.15.83:0/679230266 >> v2:172.21.15.86:3300/0 conn(0x7f770c0b34f0 0x7f770c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.699+0000 7f7711a00640 1 --2- 172.21.15.83:0/679230266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c144470 0x7f770c15f0c0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.699+0000 7f7711a00640 1 -- 172.21.15.83:0/679230266 >> 172.21.15.83:0/679230266 conn(0x7f770c06b3b0 msgr2=0x7f770c06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.699+0000 7f7711a00640 1 -- 172.21.15.83:0/679230266 shutdown_connections 2024-04-25T00:14:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.699+0000 7f7711a00640 1 -- 172.21.15.83:0/679230266 wait complete. 2024-04-25T00:14:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.699+0000 7f7711a00640 1 Processor -- start 2024-04-25T00:14:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.699+0000 7f7711a00640 1 -- start start 2024-04-25T00:14:52.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f770c0b34f0 0x7f770c159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:52.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c0b3970 0x7f770c163d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c144470 0x7f770c0ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f770c1676e0 con 0x7f770c144470 2024-04-25T00:14:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f770c0d0420 con 0x7f770c0b34f0 2024-04-25T00:14:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f770c167860 con 0x7f770c0b3970 2024-04-25T00:14:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c144470 0x7f770c0ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c144470 0x7f770c0ced00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38640/0 (socket says 172.21.15.83:38640) 2024-04-25T00:14:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711000640 1 -- 172.21.15.83:0/970482636 learned_addr learned my addr 172.21.15.83:0/970482636 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f770be00640 1 --2- 172.21.15.83:0/970482636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f770c0b34f0 0x7f770c159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7703e00640 1 --2- 172.21.15.83:0/970482636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c0b3970 0x7f770c163d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711000640 1 -- 172.21.15.83:0/970482636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c0b3970 msgr2=0x7f770c163d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711000640 1 --2- 172.21.15.83:0/970482636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c0b3970 0x7f770c163d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711000640 1 -- 172.21.15.83:0/970482636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f770c0b34f0 msgr2=0x7f770c159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711000640 1 --2- 172.21.15.83:0/970482636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f770c0b34f0 0x7f770c159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711000640 1 -- 172.21.15.83:0/970482636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c0b31b0 con 0x7f770c144470 2024-04-25T00:14:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711000640 1 --2- 172.21.15.83:0/970482636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c144470 0x7f770c0ced00 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f770400c770 tx=0x7f770400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7708c00640 1 -- 172.21.15.83:0/970482636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704003b80 con 0x7f770c144470 2024-04-25T00:14:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7708c00640 1 -- 172.21.15.83:0/970482636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7704003d20 con 0x7f770c144470 2024-04-25T00:14:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711a00640 1 -- 172.21.15.83:0/970482636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770c15a8e0 con 0x7f770c144470 2024-04-25T00:14:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7708c00640 1 -- 172.21.15.83:0/970482636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770400aa90 con 0x7f770c144470 2024-04-25T00:14:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711a00640 1 -- 172.21.15.83:0/970482636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f770c161e00 con 0x7f770c144470 2024-04-25T00:14:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7711a00640 1 -- 172.21.15.83:0/970482636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770c04f510 con 0x7f770c144470 2024-04-25T00:14:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.703+0000 7f7708c00640 1 -- 172.21.15.83:0/970482636 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f770400ac30 con 0x7f770c144470 2024-04-25T00:14:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.707+0000 7f7708c00640 1 --2- 172.21.15.83:0/970482636 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76e80678f0 0x7f76e8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.707+0000 7f7708c00640 1 -- 172.21.15.83:0/970482636 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f7704085c80 con 0x7f770c144470 2024-04-25T00:14:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.707+0000 7f770be00640 1 --2- 172.21.15.83:0/970482636 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76e80678f0 0x7f76e8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:52.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.707+0000 7f770be00640 1 --2- 172.21.15.83:0/970482636 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76e80678f0 0x7f76e8069db0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0ad000 tx=0x7f76fc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:52.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.711+0000 7f7708c00640 1 -- 172.21.15.83:0/970482636 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7704055f40 con 0x7f770c144470 2024-04-25T00:14:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.867+0000 7f7711a00640 1 -- 172.21.15.83:0/970482636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f770c0cdce0 con 0x7f770c144470 2024-04-25T00:14:52.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.867+0000 7f7708c00640 1 -- 172.21.15.83:0/970482636 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16802 (secure 0 0 0) 0x7f770405a240 con 0x7f770c144470 2024-04-25T00:14:52.871 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:52.871 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":92,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:51.879194+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:50.260735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:50.880394+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:52.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.867+0000 7f7711a00640 1 -- 172.21.15.83:0/970482636 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76e80678f0 msgr2=0x7f76e8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:52.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.867+0000 7f7711a00640 1 --2- 172.21.15.83:0/970482636 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76e80678f0 0x7f76e8069db0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0ad000 tx=0x7f76fc0a6000 comp rx=0 tx=0).stop 2024-04-25T00:14:52.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.867+0000 7f7711a00640 1 -- 172.21.15.83:0/970482636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c144470 msgr2=0x7f770c0ced00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:52.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.867+0000 7f7711a00640 1 --2- 172.21.15.83:0/970482636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c144470 0x7f770c0ced00 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f770400c770 tx=0x7f770400cc30 comp rx=0 tx=0).stop 2024-04-25T00:14:52.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.867+0000 7f7711a00640 1 -- 172.21.15.83:0/970482636 shutdown_connections 2024-04-25T00:14:52.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.867+0000 7f7711a00640 1 --2- 172.21.15.83:0/970482636 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f76e80678f0 0x7f76e8069db0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:52.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.867+0000 7f7711a00640 1 --2- 172.21.15.83:0/970482636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f770c0b3970 0x7f770c163d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:52.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.867+0000 7f7711a00640 1 --2- 172.21.15.83:0/970482636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f770c144470 0x7f770c0ced00 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:52.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.867+0000 7f7711a00640 1 --2- 172.21.15.83:0/970482636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f770c0b34f0 0x7f770c159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:52.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.867+0000 7f7711a00640 1 -- 172.21.15.83:0/970482636 >> 172.21.15.83:0/970482636 conn(0x7f770c06b3b0 msgr2=0x7f770c15d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:52.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.867+0000 7f7711a00640 1 -- 172.21.15.83:0/970482636 shutdown_connections 2024-04-25T00:14:52.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:52.867+0000 7f7711a00640 1 -- 172.21.15.83:0/970482636 wait complete. 2024-04-25T00:14:52.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:53.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.023+0000 7fe8a8600640 1 Processor -- start 2024-04-25T00:14:53.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.023+0000 7fe8a8600640 1 -- start start 2024-04-25T00:14:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.023+0000 7fe8a8600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe8a01599f0 0x7fe8a00cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.023+0000 7fe8a8600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8a00ccc60 con 0x7fe8a015cec0 2024-04-25T00:14:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.023+0000 7fe8a8600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8a006dba0 con 0x7fe8a015c340 2024-04-25T00:14:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.023+0000 7fe8a8600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe8a0159dd0 con 0x7fe8a01599f0 2024-04-25T00:14:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.027+0000 7fe8a6600640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe8a015cec0 0x7fe8a00ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54774/0 (socket says 172.21.15.83:54774) 2024-04-25T00:14:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.027+0000 7fe8a6600640 1 -- 172.21.15.83:0/1367255235 learned_addr learned my addr 172.21.15.83:0/1367255235 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:53.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.027+0000 7fe8a5c00640 1 --2- 172.21.15.83:0/1367255235 >> v2:172.21.15.86:3300/0 conn(0x7fe8a01599f0 0x7fe8a00cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.027+0000 7fe89fe00640 1 -- 172.21.15.83:0/1367255235 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 901192769 0 0) 0x7fe8980014a0 con 0x7fe8a015cec0 2024-04-25T00:14:53.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.027+0000 7fe89fe00640 1 -- 172.21.15.83:0/1367255235 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe888003530 con 0x7fe8a015cec0 2024-04-25T00:14:53.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.027+0000 7fe8a5c00640 1 -- 172.21.15.83:0/1367255235 >> v1:172.21.15.86:6789/0 conn(0x7fe8a015c340 legacy=0x7fe8a015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.027+0000 7fe89fe00640 1 -- 172.21.15.83:0/1367255235 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1502666877 0 0) 0x7fe8a006dba0 con 0x7fe8a015c340 2024-04-25T00:14:53.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.027+0000 7fe8a5c00640 1 -- 172.21.15.83:0/1367255235 >> v1:172.21.15.83:6789/0 conn(0x7fe8a015cec0 legacy=0x7fe8a00ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.027+0000 7fe8a5c00640 1 -- 172.21.15.83:0/1367255235 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a006de80 con 0x7fe8a01599f0 2024-04-25T00:14:53.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.027+0000 7fe8a5c00640 1 --2- 172.21.15.83:0/1367255235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a01599f0 0x7fe8a00cc510 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe8940097f0 tx=0x7fe89401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bfd899534b2589d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:53.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.027+0000 7fe89fe00640 1 -- 172.21.15.83:0/1367255235 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8940b0070 con 0x7fe8a01599f0 2024-04-25T00:14:53.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.027+0000 7fe89fe00640 1 -- 172.21.15.83:0/1367255235 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8940ab600 con 0x7fe8a01599f0 2024-04-25T00:14:53.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe89fe00640 1 -- 172.21.15.83:0/1367255235 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8940ab9d0 con 0x7fe8a01599f0 2024-04-25T00:14:53.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 -- 172.21.15.83:0/1367255235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a01599f0 msgr2=0x7fe8a00cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 --2- 172.21.15.83:0/1367255235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a01599f0 0x7fe8a00cc510 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe8940097f0 tx=0x7fe89401c410 comp rx=0 tx=0).stop 2024-04-25T00:14:53.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 -- 172.21.15.83:0/1367255235 shutdown_connections 2024-04-25T00:14:53.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 --2- 172.21.15.83:0/1367255235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a01599f0 0x7fe8a00cc510 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 -- 172.21.15.83:0/1367255235 >> 172.21.15.83:0/1367255235 conn(0x7fe8a00b1830 msgr2=0x7fe8a00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:53.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 -- 172.21.15.83:0/1367255235 shutdown_connections 2024-04-25T00:14:53.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 -- 172.21.15.83:0/1367255235 wait complete. 2024-04-25T00:14:53.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 Processor -- start 2024-04-25T00:14:53.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 -- start start 2024-04-25T00:14:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a01599f0 0x7fe8a014ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a015c340 0x7fe8a0150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8a015cec0 0x7fe8a014d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0169ce0 con 0x7fe8a01599f0 2024-04-25T00:14:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a00cf850 con 0x7fe8a015c340 2024-04-25T00:14:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a8600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a0169e60 con 0x7fe8a015cec0 2024-04-25T00:14:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a01599f0 0x7fe8a014ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a01599f0 0x7fe8a014ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38650/0 (socket says 172.21.15.83:38650) 2024-04-25T00:14:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a5c00640 1 -- 172.21.15.83:0/3932877807 learned_addr learned my addr 172.21.15.83:0/3932877807 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a6600640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8a015cec0 0x7fe8a014d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a5200640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a015c340 0x7fe8a0150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a5c00640 1 -- 172.21.15.83:0/3932877807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8a015cec0 msgr2=0x7fe8a014d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a5c00640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8a015cec0 0x7fe8a014d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a5c00640 1 -- 172.21.15.83:0/3932877807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a015c340 msgr2=0x7fe8a0150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a5c00640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a015c340 0x7fe8a0150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a5c00640 1 -- 172.21.15.83:0/3932877807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a014e480 con 0x7fe8a01599f0 2024-04-25T00:14:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.031+0000 7fe8a5200640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a015c340 0x7fe8a0150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.035+0000 7fe8a5c00640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a01599f0 0x7fe8a014ffd0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe894019020 tx=0x7fe89401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.035+0000 7fe89d600640 1 -- 172.21.15.83:0/3932877807 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8940b0050 con 0x7fe8a01599f0 2024-04-25T00:14:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.035+0000 7fe89d600640 1 -- 172.21.15.83:0/3932877807 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8940b1040 con 0x7fe8a01599f0 2024-04-25T00:14:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.035+0000 7fe89d600640 1 -- 172.21.15.83:0/3932877807 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894006e20 con 0x7fe8a01599f0 2024-04-25T00:14:53.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.035+0000 7fe8a8600640 1 -- 172.21.15.83:0/3932877807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a014e760 con 0x7fe8a01599f0 2024-04-25T00:14:53.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.035+0000 7fe8a8600640 1 -- 172.21.15.83:0/3932877807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a014ed20 con 0x7fe8a01599f0 2024-04-25T00:14:53.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.035+0000 7fe8a8600640 1 -- 172.21.15.83:0/3932877807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a014ef60 con 0x7fe8a01599f0 2024-04-25T00:14:53.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.035+0000 7fe89d600640 1 -- 172.21.15.83:0/3932877807 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe8940be030 con 0x7fe8a01599f0 2024-04-25T00:14:53.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.035+0000 7fe89d600640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe8800678c0 0x7fe880069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.035+0000 7fe89d600640 1 -- 172.21.15.83:0/3932877807 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7fe89411fcf0 con 0x7fe8a01599f0 2024-04-25T00:14:53.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.039+0000 7fe8a5200640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe8800678c0 0x7fe880069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.039+0000 7fe8a5200640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe8800678c0 0x7fe880069d80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00712d0 tx=0x7fe88c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:53.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.043+0000 7fe89d600640 1 -- 172.21.15.83:0/3932877807 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe8a014ef60 con 0x7fe8a01599f0 2024-04-25T00:14:53.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.199+0000 7fe8a8600640 1 -- 172.21.15.83:0/3932877807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8a014cd20 con 0x7fe8a01599f0 2024-04-25T00:14:53.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.199+0000 7fe89d600640 1 -- 172.21.15.83:0/3932877807 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16802 (secure 0 0 0) 0x7fe8940c5020 con 0x7fe8a01599f0 2024-04-25T00:14:53.203 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:53.203 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":92,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:51.879194+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:50.260735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:50.880394+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.199+0000 7fe8a8600640 1 -- 172.21.15.83:0/3932877807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe8800678c0 msgr2=0x7fe880069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.199+0000 7fe8a8600640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe8800678c0 0x7fe880069d80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00712d0 tx=0x7fe88c009040 comp rx=0 tx=0).stop 2024-04-25T00:14:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.199+0000 7fe8a8600640 1 -- 172.21.15.83:0/3932877807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a01599f0 msgr2=0x7fe8a014ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.199+0000 7fe8a8600640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a01599f0 0x7fe8a014ffd0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe894019020 tx=0x7fe89401ac50 comp rx=0 tx=0).stop 2024-04-25T00:14:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.199+0000 7fe8a8600640 1 -- 172.21.15.83:0/3932877807 shutdown_connections 2024-04-25T00:14:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.199+0000 7fe8a8600640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe8800678c0 0x7fe880069d80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.199+0000 7fe8a8600640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8a015cec0 0x7fe8a014d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.199+0000 7fe8a8600640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a015c340 0x7fe8a0150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.199+0000 7fe8a8600640 1 --2- 172.21.15.83:0/3932877807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a01599f0 0x7fe8a014ffd0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.199+0000 7fe8a8600640 1 -- 172.21.15.83:0/3932877807 >> 172.21.15.83:0/3932877807 conn(0x7fe8a00b1830 msgr2=0x7fe8a0071760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.199+0000 7fe8a8600640 1 -- 172.21.15.83:0/3932877807 shutdown_connections 2024-04-25T00:14:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.199+0000 7fe8a8600640 1 -- 172.21.15.83:0/3932877807 wait complete. 2024-04-25T00:14:53.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:14:53.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4eeb000640 1 Processor -- start 2024-04-25T00:14:53.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4eeb000640 1 -- start start 2024-04-25T00:14:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4eeb000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee415e270 0x7f4ee40b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4eeb000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4ee4071e50 0x7f4ee40b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4eeb000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ee406dcf0 con 0x7f4ee415de90 2024-04-25T00:14:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4eeb000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee40b2ce0 con 0x7f4ee4071e50 2024-04-25T00:14:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4eeb000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee406dfd0 con 0x7f4ee415e270 2024-04-25T00:14:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4ee9000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4ee4071e50 0x7f4ee40b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4ee9000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4ee4071e50 0x7f4ee40b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38652/0 (socket says 172.21.15.83:38652) 2024-04-25T00:14:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4ee9000640 1 -- 172.21.15.83:0/1448522311 learned_addr learned my addr 172.21.15.83:0/1448522311 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4ee3400640 1 --2- 172.21.15.83:0/1448522311 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee415e270 0x7f4ee40b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4ee9000640 1 -- 172.21.15.83:0/1448522311 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee415e270 msgr2=0x7f4ee40b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4ee9000640 1 --2- 172.21.15.83:0/1448522311 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee415e270 0x7f4ee40b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4ee9000640 1 -- 172.21.15.83:0/1448522311 >> v1:172.21.15.86:6789/0 conn(0x7f4ee415de90 legacy=0x7f4ee4071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.359+0000 7f4ee9000640 1 -- 172.21.15.83:0/1448522311 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee406e2b0 con 0x7f4ee4071e50 2024-04-25T00:14:53.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.363+0000 7f4ee9000640 1 --2- 172.21.15.83:0/1448522311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4071e50 0x7f4ee40b3990 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009e50 tx=0x7f4edc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ab6a38bce94b49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:53.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.363+0000 7f4ee2a00640 1 -- 172.21.15.83:0/1448522311 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc0a56f0 con 0x7f4ee4071e50 2024-04-25T00:14:53.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.363+0000 7f4ee2a00640 1 -- 172.21.15.83:0/1448522311 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4edc0a5890 con 0x7f4ee4071e50 2024-04-25T00:14:53.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.363+0000 7f4ee2a00640 1 -- 172.21.15.83:0/1448522311 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc0a5c60 con 0x7f4ee4071e50 2024-04-25T00:14:53.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.363+0000 7f4eeb000640 1 -- 172.21.15.83:0/1448522311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4071e50 msgr2=0x7f4ee40b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.363+0000 7f4eeb000640 1 --2- 172.21.15.83:0/1448522311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4071e50 0x7f4ee40b3990 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009e50 tx=0x7f4edc01c9a0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.363+0000 7f4eeb000640 1 -- 172.21.15.83:0/1448522311 shutdown_connections 2024-04-25T00:14:53.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.363+0000 7f4eeb000640 1 --2- 172.21.15.83:0/1448522311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4071e50 0x7f4ee40b3990 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.363+0000 7f4eeb000640 1 --2- 172.21.15.83:0/1448522311 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee415e270 0x7f4ee40b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.363+0000 7f4eeb000640 1 -- 172.21.15.83:0/1448522311 >> 172.21.15.83:0/1448522311 conn(0x7f4ee40b1830 msgr2=0x7f4ee40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:53.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.363+0000 7f4eeb000640 1 -- 172.21.15.83:0/1448522311 shutdown_connections 2024-04-25T00:14:53.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.363+0000 7f4eeb000640 1 -- 172.21.15.83:0/1448522311 wait complete. 2024-04-25T00:14:53.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4eeb000640 1 Processor -- start 2024-04-25T00:14:53.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4eeb000640 1 -- start start 2024-04-25T00:14:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4eeb000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee4071e50 0x7f4ee414f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4eeb000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee415de90 0x7f4ee415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4eeb000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee415e270 0x7f4ee40cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4eeb000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee40b4050 con 0x7f4ee415e270 2024-04-25T00:14:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4eeb000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee40b3ed0 con 0x7f4ee415de90 2024-04-25T00:14:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4eeb000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee40b41d0 con 0x7f4ee4071e50 2024-04-25T00:14:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4ee9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee415e270 0x7f4ee40cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4ee9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee415e270 0x7f4ee40cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38656/0 (socket says 172.21.15.83:38656) 2024-04-25T00:14:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4ee9000640 1 -- 172.21.15.83:0/2573094677 learned_addr learned my addr 172.21.15.83:0/2573094677 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4ee3e00640 1 --2- 172.21.15.83:0/2573094677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee4071e50 0x7f4ee414f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4ee3400640 1 --2- 172.21.15.83:0/2573094677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee415de90 0x7f4ee415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4ee9000640 1 -- 172.21.15.83:0/2573094677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee4071e50 msgr2=0x7f4ee414f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4ee9000640 1 --2- 172.21.15.83:0/2573094677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee4071e50 0x7f4ee414f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4ee9000640 1 -- 172.21.15.83:0/2573094677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee415de90 msgr2=0x7f4ee415c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4ee9000640 1 --2- 172.21.15.83:0/2573094677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee415de90 0x7f4ee415c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4ee9000640 1 -- 172.21.15.83:0/2573094677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee415cb00 con 0x7f4ee415e270 2024-04-25T00:14:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4ee9000640 1 --2- 172.21.15.83:0/2573094677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee415e270 0x7f4ee40cf5b0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f4edc01a5d0 tx=0x7f4edc019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4edbe00640 1 -- 172.21.15.83:0/2573094677 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc0bc020 con 0x7f4ee415e270 2024-04-25T00:14:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4edbe00640 1 -- 172.21.15.83:0/2573094677 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4edc0b8670 con 0x7f4ee415e270 2024-04-25T00:14:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4edbe00640 1 -- 172.21.15.83:0/2573094677 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc0b8a20 con 0x7f4ee415e270 2024-04-25T00:14:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4eeb000640 1 -- 172.21.15.83:0/2573094677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee41fa9d0 con 0x7f4ee415e270 2024-04-25T00:14:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4eeb000640 1 -- 172.21.15.83:0/2573094677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee41faf90 con 0x7f4ee415e270 2024-04-25T00:14:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.367+0000 7f4eeb000640 1 -- 172.21.15.83:0/2573094677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee404f510 con 0x7f4ee415e270 2024-04-25T00:14:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.371+0000 7f4edbe00640 1 -- 172.21.15.83:0/2573094677 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4edc0bf020 con 0x7f4ee415e270 2024-04-25T00:14:53.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.371+0000 7f4edbe00640 1 --2- 172.21.15.83:0/2573094677 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ec40678c0 0x7f4ec4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.371+0000 7f4edbe00640 1 -- 172.21.15.83:0/2573094677 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f4edc120ca0 con 0x7f4ee415e270 2024-04-25T00:14:53.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.371+0000 7f4ee3e00640 1 --2- 172.21.15.83:0/2573094677 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ec40678c0 0x7f4ec4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.371+0000 7f4ee3e00640 1 --2- 172.21.15.83:0/2573094677 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ec40678c0 0x7f4ec4069d80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4071790 tx=0x7f4ed4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:53.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.375+0000 7f4edbe00640 1 -- 172.21.15.83:0/2573094677 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4edc0c6030 con 0x7f4ee415e270 2024-04-25T00:14:53.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:53.567+0000 7f4eeb000640 1 -- 172.21.15.83:0/2573094677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4ee4161900 con 0x7f4ee415e270 2024-04-25T00:14:54.481 INFO:tasks.ceph.mds.d.smithi086.stderr:2024-04-25T00:14:54.479+0000 7f9732000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:14:54.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.527+0000 7f4edbe00640 1 -- 172.21.15.83:0/2573094677 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) ==== 118+0+0 (secure 0 0 0) 0x7f4edc016620 con 0x7f4ee415e270 2024-04-25T00:14:54.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.527+0000 7f4eeb000640 1 -- 172.21.15.83:0/2573094677 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ec40678c0 msgr2=0x7f4ec4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:54.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.527+0000 7f4eeb000640 1 --2- 172.21.15.83:0/2573094677 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ec40678c0 0x7f4ec4069d80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4071790 tx=0x7f4ed4009040 comp rx=0 tx=0).stop 2024-04-25T00:14:54.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.527+0000 7f4eeb000640 1 -- 172.21.15.83:0/2573094677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee415e270 msgr2=0x7f4ee40cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:54.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.527+0000 7f4eeb000640 1 --2- 172.21.15.83:0/2573094677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee415e270 0x7f4ee40cf5b0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f4edc01a5d0 tx=0x7f4edc019740 comp rx=0 tx=0).stop 2024-04-25T00:14:54.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.527+0000 7f4eeb000640 1 -- 172.21.15.83:0/2573094677 shutdown_connections 2024-04-25T00:14:54.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.527+0000 7f4eeb000640 1 --2- 172.21.15.83:0/2573094677 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4ec40678c0 0x7f4ec4069d80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.527+0000 7f4eeb000640 1 --2- 172.21.15.83:0/2573094677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee415e270 0x7f4ee40cf5b0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.527+0000 7f4eeb000640 1 --2- 172.21.15.83:0/2573094677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee415de90 0x7f4ee415c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.527+0000 7f4eeb000640 1 --2- 172.21.15.83:0/2573094677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee4071e50 0x7f4ee414f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.527+0000 7f4eeb000640 1 -- 172.21.15.83:0/2573094677 >> 172.21.15.83:0/2573094677 conn(0x7f4ee40b1830 msgr2=0x7f4ee41685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:54.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.527+0000 7f4eeb000640 1 -- 172.21.15.83:0/2573094677 shutdown_connections 2024-04-25T00:14:54.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.527+0000 7f4eeb000640 1 -- 172.21.15.83:0/2573094677 wait complete. 2024-04-25T00:14:54.553 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:14:54.554 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:14:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f453f400640 1 Processor -- start 2024-04-25T00:14:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f453f400640 1 -- start start 2024-04-25T00:14:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f453f400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f453806e950 0x7f453815f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f453f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f453815fe50 0x7f45380ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f453f400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45380ce000 con 0x7f4538159810 2024-04-25T00:14:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f453f400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f453806e210 con 0x7f453806e950 2024-04-25T00:14:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f453f400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f45381602a0 con 0x7f453815fe50 2024-04-25T00:14:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f4537e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f453806e950 0x7f453815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f4537e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f453806e950 0x7f453815f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51806/0 (socket says 172.21.15.83:51806) 2024-04-25T00:14:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f4537e00640 1 -- 172.21.15.83:0/3867630080 learned_addr learned my addr 172.21.15.83:0/3867630080 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f453d400640 1 --2- 172.21.15.83:0/3867630080 >> v2:172.21.15.86:3300/0 conn(0x7f453815fe50 0x7f45380ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f4537e00640 1 -- 172.21.15.83:0/3867630080 >> v2:172.21.15.86:3300/0 conn(0x7f453815fe50 msgr2=0x7f45380ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f4537e00640 1 --2- 172.21.15.83:0/3867630080 >> v2:172.21.15.86:3300/0 conn(0x7f453815fe50 0x7f45380ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f4537e00640 1 -- 172.21.15.83:0/3867630080 >> v1:172.21.15.86:6789/0 conn(0x7f4538159810 legacy=0x7f45380cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f4537e00640 1 -- 172.21.15.83:0/3867630080 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453806eec0 con 0x7f453806e950 2024-04-25T00:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.691+0000 7f453d400640 1 --2- 172.21.15.83:0/3867630080 >> v2:172.21.15.86:3300/0 conn(0x7f453815fe50 0x7f45380ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.695+0000 7f4537e00640 1 --2- 172.21.15.83:0/3867630080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f453806e950 0x7f453815f910 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f45380af750 tx=0x7f452401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6b19c7cf2938d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.695+0000 7f4537400640 1 -- 172.21.15.83:0/3867630080 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45240b2070 con 0x7f453806e950 2024-04-25T00:14:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.695+0000 7f4537400640 1 -- 172.21.15.83:0/3867630080 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45240ad680 con 0x7f453806e950 2024-04-25T00:14:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.695+0000 7f4537400640 1 -- 172.21.15.83:0/3867630080 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45240ada50 con 0x7f453806e950 2024-04-25T00:14:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.695+0000 7f453f400640 1 -- 172.21.15.83:0/3867630080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f453806e950 msgr2=0x7f453815f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.695+0000 7f453f400640 1 --2- 172.21.15.83:0/3867630080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f453806e950 0x7f453815f910 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f45380af750 tx=0x7f452401c490 comp rx=0 tx=0).stop 2024-04-25T00:14:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.695+0000 7f453f400640 1 -- 172.21.15.83:0/3867630080 shutdown_connections 2024-04-25T00:14:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.695+0000 7f453f400640 1 --2- 172.21.15.83:0/3867630080 >> v2:172.21.15.86:3300/0 conn(0x7f453815fe50 0x7f45380ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.695+0000 7f453f400640 1 --2- 172.21.15.83:0/3867630080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f453806e950 0x7f453815f910 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.695+0000 7f453f400640 1 -- 172.21.15.83:0/3867630080 >> 172.21.15.83:0/3867630080 conn(0x7f45380b1830 msgr2=0x7f45380b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.695+0000 7f453f400640 1 -- 172.21.15.83:0/3867630080 shutdown_connections 2024-04-25T00:14:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.695+0000 7f453f400640 1 -- 172.21.15.83:0/3867630080 wait complete. 2024-04-25T00:14:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.695+0000 7f453f400640 1 Processor -- start 2024-04-25T00:14:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.695+0000 7f453f400640 1 -- start start 2024-04-25T00:14:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453f400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f453806e950 0x7f4538141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4538159810 0x7f45380b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f453815fe50 0x7f45380b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453f400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4538070f20 con 0x7f453815fe50 2024-04-25T00:14:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453f400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4538070da0 con 0x7f4538159810 2024-04-25T00:14:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453f400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4538168cb0 con 0x7f453806e950 2024-04-25T00:14:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f453815fe50 0x7f45380b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f453815fe50 0x7f45380b3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51810/0 (socket says 172.21.15.83:51810) 2024-04-25T00:14:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453d400640 1 -- 172.21.15.83:0/3569844261 learned_addr learned my addr 172.21.15.83:0/3569844261 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f4537e00640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4538159810 0x7f45380b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453ca00640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f453806e950 0x7f4538141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453d400640 1 -- 172.21.15.83:0/3569844261 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f453806e950 msgr2=0x7f4538141480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453d400640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f453806e950 0x7f4538141480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453d400640 1 -- 172.21.15.83:0/3569844261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4538159810 msgr2=0x7f45380b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453d400640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4538159810 0x7f45380b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453d400640 1 -- 172.21.15.83:0/3569844261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4538155910 con 0x7f453815fe50 2024-04-25T00:14:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f4537e00640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4538159810 0x7f45380b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453ca00640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f453806e950 0x7f4538141480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453d400640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f453815fe50 0x7f45380b3e90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f4530007e70 tx=0x7f453000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f4534c00640 1 -- 172.21.15.83:0/3569844261 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453001a070 con 0x7f453815fe50 2024-04-25T00:14:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f4534c00640 1 -- 172.21.15.83:0/3569844261 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4530003bc0 con 0x7f453815fe50 2024-04-25T00:14:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f4534c00640 1 -- 172.21.15.83:0/3569844261 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530004030 con 0x7f453815fe50 2024-04-25T00:14:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453f400640 1 -- 172.21.15.83:0/3569844261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4538155bf0 con 0x7f453815fe50 2024-04-25T00:14:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453f400640 1 -- 172.21.15.83:0/3569844261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45381561b0 con 0x7f453815fe50 2024-04-25T00:14:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.699+0000 7f453f400640 1 -- 172.21.15.83:0/3569844261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f453804f510 con 0x7f453815fe50 2024-04-25T00:14:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.703+0000 7f4534c00640 1 -- 172.21.15.83:0/3569844261 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f453001d070 con 0x7f453815fe50 2024-04-25T00:14:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.703+0000 7f4534c00640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f45180678c0 0x7f4518069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.703+0000 7f4534c00640 1 -- 172.21.15.83:0/3569844261 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f4530086120 con 0x7f453815fe50 2024-04-25T00:14:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.703+0000 7f453ca00640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f45180678c0 0x7f4518069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:54.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.703+0000 7f453ca00640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f45180678c0 0x7f4518069d80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f452c000f70 tx=0x7f452c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:54.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.707+0000 7f4534c00640 1 -- 172.21.15.83:0/3569844261 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4530010070 con 0x7f453815fe50 2024-04-25T00:14:54.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:54.895+0000 7f453f400640 1 -- 172.21.15.83:0/3569844261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4538072c90 con 0x7f453815fe50 2024-04-25T00:14:55.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.527+0000 7f4534c00640 1 -- 172.21.15.83:0/3569844261 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) ==== 122+0+0 (secure 0 0 0) 0x7f4530056ac0 con 0x7f453815fe50 2024-04-25T00:14:55.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.531+0000 7f453f400640 1 -- 172.21.15.83:0/3569844261 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f45180678c0 msgr2=0x7f4518069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:55.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.531+0000 7f453f400640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f45180678c0 0x7f4518069d80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f452c000f70 tx=0x7f452c009040 comp rx=0 tx=0).stop 2024-04-25T00:14:55.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.531+0000 7f453f400640 1 -- 172.21.15.83:0/3569844261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f453815fe50 msgr2=0x7f45380b3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:55.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.531+0000 7f453f400640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f453815fe50 0x7f45380b3e90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f4530007e70 tx=0x7f453000d4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:55.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.531+0000 7f453f400640 1 -- 172.21.15.83:0/3569844261 shutdown_connections 2024-04-25T00:14:55.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.531+0000 7f453f400640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f45180678c0 0x7f4518069d80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:55.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.531+0000 7f453f400640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f453815fe50 0x7f45380b3e90 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:55.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.531+0000 7f453f400640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4538159810 0x7f45380b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:55.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.531+0000 7f453f400640 1 --2- 172.21.15.83:0/3569844261 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f453806e950 0x7f4538141480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:55.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.531+0000 7f453f400640 1 -- 172.21.15.83:0/3569844261 >> 172.21.15.83:0/3569844261 conn(0x7f45380b1830 msgr2=0x7f45381547e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:55.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.531+0000 7f453f400640 1 -- 172.21.15.83:0/3569844261 shutdown_connections 2024-04-25T00:14:55.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.531+0000 7f453f400640 1 -- 172.21.15.83:0/3569844261 wait complete. 2024-04-25T00:14:55.558 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:14:55.558 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.691+0000 7fd94da00640 1 Processor -- start 2024-04-25T00:14:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.691+0000 7fd94da00640 1 -- start start 2024-04-25T00:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.691+0000 7fd94da00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9481616e0 0x7fd9480d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.691+0000 7fd94da00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd9480d0e00 0x7fd9480cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.691+0000 7fd94da00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9480b2f00 con 0x7fd9480cc710 2024-04-25T00:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.691+0000 7fd94da00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd9480d1240 con 0x7fd9480d0e00 2024-04-25T00:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.691+0000 7fd94da00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd9480b2940 con 0x7fd9481616e0 2024-04-25T00:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.691+0000 7fd93f400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd9480d0e00 0x7fd9480cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.695+0000 7fd93f400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd9480d0e00 0x7fd9480cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51812/0 (socket says 172.21.15.83:51812) 2024-04-25T00:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.695+0000 7fd93f400640 1 -- 172.21.15.83:0/1799675894 learned_addr learned my addr 172.21.15.83:0/1799675894 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.695+0000 7fd947e00640 1 --2- 172.21.15.83:0/1799675894 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9481616e0 0x7fd9480d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.695+0000 7fd93f400640 1 -- 172.21.15.83:0/1799675894 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9481616e0 msgr2=0x7fd9480d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.695+0000 7fd93f400640 1 --2- 172.21.15.83:0/1799675894 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9481616e0 0x7fd9480d1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.695+0000 7fd93f400640 1 -- 172.21.15.83:0/1799675894 >> v1:172.21.15.86:6789/0 conn(0x7fd9480cc710 legacy=0x7fd9480ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.695+0000 7fd93f400640 1 -- 172.21.15.83:0/1799675894 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9480b2c20 con 0x7fd9480d0e00 2024-04-25T00:14:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.695+0000 7fd947e00640 1 --2- 172.21.15.83:0/1799675894 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9481616e0 0x7fd9480d1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.695+0000 7fd93f400640 1 --2- 172.21.15.83:0/1799675894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9480d0e00 0x7fd9480cc1d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fd930009870 tx=0x7fd93001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c2c76d045dd322 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:55.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.695+0000 7fd947400640 1 -- 172.21.15.83:0/1799675894 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9300b0070 con 0x7fd9480d0e00 2024-04-25T00:14:55.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.695+0000 7fd947400640 1 -- 172.21.15.83:0/1799675894 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9300ab600 con 0x7fd9480d0e00 2024-04-25T00:14:55.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.695+0000 7fd947400640 1 -- 172.21.15.83:0/1799675894 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9300ab9d0 con 0x7fd9480d0e00 2024-04-25T00:14:55.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.695+0000 7fd94da00640 1 -- 172.21.15.83:0/1799675894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9480d0e00 msgr2=0x7fd9480cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:55.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.695+0000 7fd94da00640 1 --2- 172.21.15.83:0/1799675894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9480d0e00 0x7fd9480cc1d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fd930009870 tx=0x7fd93001c520 comp rx=0 tx=0).stop 2024-04-25T00:14:55.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94da00640 1 -- 172.21.15.83:0/1799675894 shutdown_connections 2024-04-25T00:14:55.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94da00640 1 --2- 172.21.15.83:0/1799675894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9480d0e00 0x7fd9480cc1d0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:55.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94da00640 1 --2- 172.21.15.83:0/1799675894 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9481616e0 0x7fd9480d1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:55.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94da00640 1 -- 172.21.15.83:0/1799675894 >> 172.21.15.83:0/1799675894 conn(0x7fd9480d1b80 msgr2=0x7fd9480d1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:55.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94da00640 1 -- 172.21.15.83:0/1799675894 shutdown_connections 2024-04-25T00:14:55.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94da00640 1 -- 172.21.15.83:0/1799675894 wait complete. 2024-04-25T00:14:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94da00640 1 Processor -- start 2024-04-25T00:14:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94da00640 1 -- start start 2024-04-25T00:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94da00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9480cc710 0x7fd948154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9480d0e00 0x7fd9480b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9481616e0 0x7fd948156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94da00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd94816b070 con 0x7fd9481616e0 2024-04-25T00:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd94816aef0 con 0x7fd9480d0e00 2024-04-25T00:14:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94da00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd94816b1f0 con 0x7fd9480cc710 2024-04-25T00:14:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9481616e0 0x7fd948156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9481616e0 0x7fd948156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51828/0 (socket says 172.21.15.83:51828) 2024-04-25T00:14:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94d000640 1 -- 172.21.15.83:0/903759736 learned_addr learned my addr 172.21.15.83:0/903759736 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd947e00640 1 --2- 172.21.15.83:0/903759736 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9480cc710 0x7fd948154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd93f400640 1 --2- 172.21.15.83:0/903759736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9480d0e00 0x7fd9480b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94d000640 1 -- 172.21.15.83:0/903759736 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9480cc710 msgr2=0x7fd948154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94d000640 1 --2- 172.21.15.83:0/903759736 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9480cc710 0x7fd948154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94d000640 1 -- 172.21.15.83:0/903759736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9480d0e00 msgr2=0x7fd9480b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94d000640 1 --2- 172.21.15.83:0/903759736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9480d0e00 0x7fd9480b00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94d000640 1 -- 172.21.15.83:0/903759736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9480c8480 con 0x7fd9481616e0 2024-04-25T00:14:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd94d000640 1 --2- 172.21.15.83:0/903759736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9481616e0 0x7fd948156e80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fd94000b9a0 tx=0x7fd94000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.699+0000 7fd944c00640 1 -- 172.21.15.83:0/903759736 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940019070 con 0x7fd9481616e0 2024-04-25T00:14:55.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.703+0000 7fd944c00640 1 -- 172.21.15.83:0/903759736 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd94000cba0 con 0x7fd9481616e0 2024-04-25T00:14:55.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.703+0000 7fd944c00640 1 -- 172.21.15.83:0/903759736 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940014440 con 0x7fd9481616e0 2024-04-25T00:14:55.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.703+0000 7fd94da00640 1 -- 172.21.15.83:0/903759736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9480c73e0 con 0x7fd9481616e0 2024-04-25T00:14:55.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.703+0000 7fd94da00640 1 -- 172.21.15.83:0/903759736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9480c79a0 con 0x7fd9481616e0 2024-04-25T00:14:55.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.703+0000 7fd94da00640 1 -- 172.21.15.83:0/903759736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd94804f510 con 0x7fd9481616e0 2024-04-25T00:14:55.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.703+0000 7fd944c00640 1 -- 172.21.15.83:0/903759736 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd94001c070 con 0x7fd9481616e0 2024-04-25T00:14:55.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.703+0000 7fd944c00640 1 --2- 172.21.15.83:0/903759736 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd9240678c0 0x7fd924069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:55.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.703+0000 7fd944c00640 1 -- 172.21.15.83:0/903759736 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fd940085950 con 0x7fd9481616e0 2024-04-25T00:14:55.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.703+0000 7fd947e00640 1 --2- 172.21.15.83:0/903759736 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd9240678c0 0x7fd924069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:55.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.703+0000 7fd947e00640 1 --2- 172.21.15.83:0/903759736 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd9240678c0 0x7fd924069d80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd938006fd0 tx=0x7fd938008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:55.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.707+0000 7fd944c00640 1 -- 172.21.15.83:0/903759736 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd940055d40 con 0x7fd9481616e0 2024-04-25T00:14:55.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.895+0000 7fd94da00640 1 -- 172.21.15.83:0/903759736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd948158340 con 0x7fd9481616e0 2024-04-25T00:14:55.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.895+0000 7fd944c00640 1 -- 172.21.15.83:0/903759736 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5177 (secure 0 0 0) 0x7fd94005a040 con 0x7fd9481616e0 2024-04-25T00:14:55.898 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:55.898 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/45139956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":45139956},{"type":"v1","addr":"172.21.15.83:6835","nonce":45139956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7455,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1609715757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1609715757},{"type":"v1","addr":"172.21.15.83:6837","nonce":1609715757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/3915910404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3915910404},{"type":"v1","addr":"172.21.15.86:6834","nonce":3915910404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:51.879542+0000","modified":"2024-04-25T00:14:54.899792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7467":{"gid":7467,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/3612679148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":3612679148},{"type":"v1","addr":"172.21.15.86:6835","nonce":3612679148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7467,"qdb_cluster":[7467]},"id":7}]} 2024-04-25T00:14:55.899 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-04-25T00:14:55.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.895+0000 7fd94da00640 1 -- 172.21.15.83:0/903759736 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd9240678c0 msgr2=0x7fd924069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:55.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.895+0000 7fd94da00640 1 --2- 172.21.15.83:0/903759736 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd9240678c0 0x7fd924069d80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd938006fd0 tx=0x7fd938008040 comp rx=0 tx=0).stop 2024-04-25T00:14:55.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.895+0000 7fd94da00640 1 -- 172.21.15.83:0/903759736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9481616e0 msgr2=0x7fd948156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:55.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.895+0000 7fd94da00640 1 --2- 172.21.15.83:0/903759736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9481616e0 0x7fd948156e80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fd94000b9a0 tx=0x7fd94000be60 comp rx=0 tx=0).stop 2024-04-25T00:14:55.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.895+0000 7fd94da00640 1 -- 172.21.15.83:0/903759736 shutdown_connections 2024-04-25T00:14:55.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.895+0000 7fd94da00640 1 --2- 172.21.15.83:0/903759736 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd9240678c0 0x7fd924069d80 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:55.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.895+0000 7fd94da00640 1 --2- 172.21.15.83:0/903759736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9481616e0 0x7fd948156e80 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:55.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.895+0000 7fd94da00640 1 --2- 172.21.15.83:0/903759736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9480d0e00 0x7fd9480b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:55.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.895+0000 7fd94da00640 1 --2- 172.21.15.83:0/903759736 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9480cc710 0x7fd948154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:55.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.895+0000 7fd94da00640 1 -- 172.21.15.83:0/903759736 >> 172.21.15.83:0/903759736 conn(0x7fd9480d1b80 msgr2=0x7fd9480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:55.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.895+0000 7fd94da00640 1 -- 172.21.15.83:0/903759736 shutdown_connections 2024-04-25T00:14:55.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:55.895+0000 7fd94da00640 1 -- 172.21.15.83:0/903759736 wait complete. 2024-04-25T00:14:55.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:56.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f5971800640 1 Processor -- start 2024-04-25T00:14:56.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f5971800640 1 -- start start 2024-04-25T00:14:56.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f5971800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f596c15bf70 0x7f596c0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f5971800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f596c0d0940 0x7f596c165b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f5971800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f596c0cc9b0 con 0x7f596c0cfd80 2024-04-25T00:14:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f5971800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f596c0b5ad0 con 0x7f596c15bf70 2024-04-25T00:14:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f5971800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f596c0ccc90 con 0x7f596c0d0940 2024-04-25T00:14:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f596aa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f596c15bf70 0x7f596c0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f596aa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f596c15bf70 0x7f596c0cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51832/0 (socket says 172.21.15.83:51832) 2024-04-25T00:14:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f596aa00640 1 -- 172.21.15.83:0/1364351113 learned_addr learned my addr 172.21.15.83:0/1364351113 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f596b400640 1 --2- 172.21.15.83:0/1364351113 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f596c0d0940 0x7f596c165b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f596aa00640 1 -- 172.21.15.83:0/1364351113 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f596c0d0940 msgr2=0x7f596c165b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f596aa00640 1 --2- 172.21.15.83:0/1364351113 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f596c0d0940 0x7f596c165b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f596aa00640 1 -- 172.21.15.83:0/1364351113 >> v1:172.21.15.83:6789/0 conn(0x7f596c0cfd80 legacy=0x7f596c0d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f596aa00640 1 -- 172.21.15.83:0/1364351113 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c0ccf70 con 0x7f596c15bf70 2024-04-25T00:14:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f596b400640 1 --2- 172.21.15.83:0/1364351113 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f596c0d0940 0x7f596c165b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.051+0000 7f596a000640 1 -- 172.21.15.83:0/1364351113 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 990586761 0 0) 0x7f596c0cc9b0 con 0x7f596c0cfd80 2024-04-25T00:14:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.055+0000 7f596aa00640 1 --2- 172.21.15.83:0/1364351113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f596c15bf70 0x7f596c0cf840 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f595c009870 tx=0x7f595c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c06e9bd63d3ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.055+0000 7f596a000640 1 -- 172.21.15.83:0/1364351113 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c0a95f0 con 0x7f596c15bf70 2024-04-25T00:14:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.055+0000 7f596a000640 1 -- 172.21.15.83:0/1364351113 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595c0a9790 con 0x7f596c15bf70 2024-04-25T00:14:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.055+0000 7f596a000640 1 -- 172.21.15.83:0/1364351113 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c0a9b60 con 0x7f596c15bf70 2024-04-25T00:14:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.055+0000 7f5971800640 1 -- 172.21.15.83:0/1364351113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f596c15bf70 msgr2=0x7f596c0cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.055+0000 7f5971800640 1 --2- 172.21.15.83:0/1364351113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f596c15bf70 0x7f596c0cf840 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f595c009870 tx=0x7f595c01c530 comp rx=0 tx=0).stop 2024-04-25T00:14:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.055+0000 7f5971800640 1 -- 172.21.15.83:0/1364351113 shutdown_connections 2024-04-25T00:14:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.055+0000 7f5971800640 1 --2- 172.21.15.83:0/1364351113 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f596c0d0940 0x7f596c165b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.055+0000 7f5971800640 1 --2- 172.21.15.83:0/1364351113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f596c15bf70 0x7f596c0cf840 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.055+0000 7f5971800640 1 -- 172.21.15.83:0/1364351113 >> 172.21.15.83:0/1364351113 conn(0x7f596c0d1c70 msgr2=0x7f596c0d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.055+0000 7f5971800640 1 -- 172.21.15.83:0/1364351113 shutdown_connections 2024-04-25T00:14:56.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.055+0000 7f5971800640 1 -- 172.21.15.83:0/1364351113 wait complete. 2024-04-25T00:14:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.055+0000 7f5971800640 1 Processor -- start 2024-04-25T00:14:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.055+0000 7f5971800640 1 -- start start 2024-04-25T00:14:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f5971800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f596c0cfd80 0x7f596c14d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f5971800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c0d0940 0x7f596c1f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f5971800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f596c15bf70 0x7f596c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f5971800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f596c0cda50 con 0x7f596c0cfd80 2024-04-25T00:14:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f5971800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f596c15c350 con 0x7f596c0d0940 2024-04-25T00:14:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f5971800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f596c14d6f0 con 0x7f596c15bf70 2024-04-25T00:14:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f596aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f596c0cfd80 0x7f596c14d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f596aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f596c0cfd80 0x7f596c14d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51836/0 (socket says 172.21.15.83:51836) 2024-04-25T00:14:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f596aa00640 1 -- 172.21.15.83:0/2177244857 learned_addr learned my addr 172.21.15.83:0/2177244857 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f596b400640 1 --2- 172.21.15.83:0/2177244857 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f596c15bf70 0x7f596c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f5963e00640 1 --2- 172.21.15.83:0/2177244857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c0d0940 0x7f596c1f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f596aa00640 1 -- 172.21.15.83:0/2177244857 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f596c15bf70 msgr2=0x7f596c0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f596aa00640 1 --2- 172.21.15.83:0/2177244857 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f596c15bf70 0x7f596c0b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f596aa00640 1 -- 172.21.15.83:0/2177244857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c0d0940 msgr2=0x7f596c1f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f596aa00640 1 --2- 172.21.15.83:0/2177244857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c0d0940 0x7f596c1f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f596aa00640 1 -- 172.21.15.83:0/2177244857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c0b3dc0 con 0x7f596c0cfd80 2024-04-25T00:14:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f596aa00640 1 --2- 172.21.15.83:0/2177244857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f596c0cfd80 0x7f596c14d1b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f595c0ab040 tx=0x7f595c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f5962a00640 1 -- 172.21.15.83:0/2177244857 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c004020 con 0x7f596c0cfd80 2024-04-25T00:14:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f5962a00640 1 -- 172.21.15.83:0/2177244857 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595c0041c0 con 0x7f596c0cfd80 2024-04-25T00:14:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f5962a00640 1 -- 172.21.15.83:0/2177244857 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c003620 con 0x7f596c0cfd80 2024-04-25T00:14:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f5971800640 1 -- 172.21.15.83:0/2177244857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c0b40a0 con 0x7f596c0cfd80 2024-04-25T00:14:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f5971800640 1 -- 172.21.15.83:0/2177244857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c0b4660 con 0x7f596c0cfd80 2024-04-25T00:14:56.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f5971800640 1 -- 172.21.15.83:0/2177244857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596c09fd60 con 0x7f596c0cfd80 2024-04-25T00:14:56.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.059+0000 7f5962a00640 1 -- 172.21.15.83:0/2177244857 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f595c0c2060 con 0x7f596c0cfd80 2024-04-25T00:14:56.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.063+0000 7f5962a00640 1 --2- 172.21.15.83:0/2177244857 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59480678c0 0x7f5948069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.063+0000 7f5962a00640 1 -- 172.21.15.83:0/2177244857 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f595c120490 con 0x7f596c0cfd80 2024-04-25T00:14:56.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.063+0000 7f5963e00640 1 --2- 172.21.15.83:0/2177244857 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59480678c0 0x7f5948069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.063+0000 7f5963e00640 1 --2- 172.21.15.83:0/2177244857 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59480678c0 0x7f5948069d80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f596c156f10 tx=0x7f5954009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:56.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.067+0000 7f5962a00640 1 -- 172.21.15.83:0/2177244857 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f595c0ad020 con 0x7f596c0cfd80 2024-04-25T00:14:56.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.223+0000 7f5971800640 1 -- 172.21.15.83:0/2177244857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f596c04f5f0 con 0x7f596c0cfd80 2024-04-25T00:14:56.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.223+0000 7f5962a00640 1 -- 172.21.15.83:0/2177244857 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16789 (secure 0 0 0) 0x7f595c016150 con 0x7f596c0cfd80 2024-04-25T00:14:56.227 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:56.227 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":93,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:53.503988+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:50.260735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:50.880394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:56.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.223+0000 7f5971800640 1 -- 172.21.15.83:0/2177244857 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59480678c0 msgr2=0x7f5948069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.223+0000 7f5971800640 1 --2- 172.21.15.83:0/2177244857 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59480678c0 0x7f5948069d80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f596c156f10 tx=0x7f5954009040 comp rx=0 tx=0).stop 2024-04-25T00:14:56.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.223+0000 7f5971800640 1 -- 172.21.15.83:0/2177244857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f596c0cfd80 msgr2=0x7f596c14d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.223+0000 7f5971800640 1 --2- 172.21.15.83:0/2177244857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f596c0cfd80 0x7f596c14d1b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f595c0ab040 tx=0x7f595c0046d0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.223+0000 7f5971800640 1 -- 172.21.15.83:0/2177244857 shutdown_connections 2024-04-25T00:14:56.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.223+0000 7f5971800640 1 --2- 172.21.15.83:0/2177244857 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f59480678c0 0x7f5948069d80 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.223+0000 7f5971800640 1 --2- 172.21.15.83:0/2177244857 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f596c15bf70 0x7f596c0b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.223+0000 7f5971800640 1 --2- 172.21.15.83:0/2177244857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c0d0940 0x7f596c1f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.223+0000 7f5971800640 1 --2- 172.21.15.83:0/2177244857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f596c0cfd80 0x7f596c14d1b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.223+0000 7f5971800640 1 -- 172.21.15.83:0/2177244857 >> 172.21.15.83:0/2177244857 conn(0x7f596c0d1c70 msgr2=0x7f596c15a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:56.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.223+0000 7f5971800640 1 -- 172.21.15.83:0/2177244857 shutdown_connections 2024-04-25T00:14:56.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.223+0000 7f5971800640 1 -- 172.21.15.83:0/2177244857 wait complete. 2024-04-25T00:14:56.249 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:14:56.249 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:14:56.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.379+0000 7fb847e00640 1 Processor -- start 2024-04-25T00:14:56.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.379+0000 7fb847e00640 1 -- start start 2024-04-25T00:14:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb847e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb8400cf810 0x7fb840144130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb847e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb84015e5b0 0x7fb84006e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb847e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb84015eb70 con 0x7fb84006eb90 2024-04-25T00:14:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb847e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb840141400 con 0x7fb84015e5b0 2024-04-25T00:14:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb847e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb840144670 con 0x7fb8400cf810 2024-04-25T00:14:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb844a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb84015e5b0 0x7fb84006e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb844a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb84015e5b0 0x7fb84006e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51852/0 (socket says 172.21.15.83:51852) 2024-04-25T00:14:56.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb844a00640 1 -- 172.21.15.83:0/1284179612 learned_addr learned my addr 172.21.15.83:0/1284179612 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:56.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb845e00640 1 --1- 172.21.15.83:0/1284179612 >> v1:172.21.15.83:6789/0 conn(0x7fb84006eb90 0x7fb840162040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51326/0 (socket says 172.21.15.83:51326) 2024-04-25T00:14:56.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb845400640 1 --2- 172.21.15.83:0/1284179612 >> v2:172.21.15.86:3300/0 conn(0x7fb8400cf810 0x7fb840144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb844a00640 1 -- 172.21.15.83:0/1284179612 >> v2:172.21.15.86:3300/0 conn(0x7fb8400cf810 msgr2=0x7fb840144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:56.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb844a00640 1 --2- 172.21.15.83:0/1284179612 >> v2:172.21.15.86:3300/0 conn(0x7fb8400cf810 0x7fb840144130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb844a00640 1 -- 172.21.15.83:0/1284179612 >> v1:172.21.15.83:6789/0 conn(0x7fb84006eb90 legacy=0x7fb840162040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb844a00640 1 -- 172.21.15.83:0/1284179612 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84015ee50 con 0x7fb84015e5b0 2024-04-25T00:14:56.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.383+0000 7fb83fe00640 1 -- 172.21.15.83:0/1284179612 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2103923628 0 0) 0x7fb84015eb70 con 0x7fb84006eb90 2024-04-25T00:14:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb844a00640 1 --2- 172.21.15.83:0/1284179612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb84015e5b0 0x7fb84006e650 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb8400b2c10 tx=0x7fb82c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fce7e4c5c53107c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb83fe00640 1 -- 172.21.15.83:0/1284179612 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c0b0070 con 0x7fb84015e5b0 2024-04-25T00:14:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb83fe00640 1 -- 172.21.15.83:0/1284179612 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb82c0ab600 con 0x7fb84015e5b0 2024-04-25T00:14:56.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb83fe00640 1 -- 172.21.15.83:0/1284179612 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c0ab9d0 con 0x7fb84015e5b0 2024-04-25T00:14:56.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 -- 172.21.15.83:0/1284179612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb84015e5b0 msgr2=0x7fb84006e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 --2- 172.21.15.83:0/1284179612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb84015e5b0 0x7fb84006e650 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb8400b2c10 tx=0x7fb82c01c520 comp rx=0 tx=0).stop 2024-04-25T00:14:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 -- 172.21.15.83:0/1284179612 shutdown_connections 2024-04-25T00:14:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 --2- 172.21.15.83:0/1284179612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb84015e5b0 0x7fb84006e650 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 --2- 172.21.15.83:0/1284179612 >> v2:172.21.15.86:3300/0 conn(0x7fb8400cf810 0x7fb840144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 -- 172.21.15.83:0/1284179612 >> 172.21.15.83:0/1284179612 conn(0x7fb8400b1d50 msgr2=0x7fb8400b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 -- 172.21.15.83:0/1284179612 shutdown_connections 2024-04-25T00:14:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 -- 172.21.15.83:0/1284179612 wait complete. 2024-04-25T00:14:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 Processor -- start 2024-04-25T00:14:56.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 -- start start 2024-04-25T00:14:56.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb84006eb90 0x7fb84013d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8400cf810 0x7fb8400afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb84015e5b0 0x7fb840157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb840162b60 con 0x7fb84006eb90 2024-04-25T00:14:56.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8400cfe00 con 0x7fb8400cf810 2024-04-25T00:14:56.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.387+0000 7fb847e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb8400b4680 con 0x7fb84015e5b0 2024-04-25T00:14:56.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb845400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb84006eb90 0x7fb84013d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb845400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb84006eb90 0x7fb84013d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51858/0 (socket says 172.21.15.83:51858) 2024-04-25T00:14:56.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb845400640 1 -- 172.21.15.83:0/644883473 learned_addr learned my addr 172.21.15.83:0/644883473 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:56.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb844a00640 1 --2- 172.21.15.83:0/644883473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8400cf810 0x7fb8400afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb845e00640 1 --2- 172.21.15.83:0/644883473 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb84015e5b0 0x7fb840157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb845400640 1 -- 172.21.15.83:0/644883473 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb84015e5b0 msgr2=0x7fb840157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb845400640 1 --2- 172.21.15.83:0/644883473 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb84015e5b0 0x7fb840157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb845400640 1 -- 172.21.15.83:0/644883473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8400cf810 msgr2=0x7fb8400afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb845400640 1 --2- 172.21.15.83:0/644883473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8400cf810 0x7fb8400afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb845400640 1 -- 172.21.15.83:0/644883473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb840157950 con 0x7fb84006eb90 2024-04-25T00:14:56.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb845400640 1 --2- 172.21.15.83:0/644883473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb84006eb90 0x7fb84013d1d0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb83400d830 tx=0x7fb83400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:56.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb83d600640 1 -- 172.21.15.83:0/644883473 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83400b6c0 con 0x7fb84006eb90 2024-04-25T00:14:56.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb83d600640 1 -- 172.21.15.83:0/644883473 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb834019070 con 0x7fb84006eb90 2024-04-25T00:14:56.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb847e00640 1 -- 172.21.15.83:0/644883473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8400b05b0 con 0x7fb84006eb90 2024-04-25T00:14:56.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb83d600640 1 -- 172.21.15.83:0/644883473 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83400bae0 con 0x7fb84006eb90 2024-04-25T00:14:56.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb847e00640 1 -- 172.21.15.83:0/644883473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8400b0b70 con 0x7fb84006eb90 2024-04-25T00:14:56.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb847e00640 1 -- 172.21.15.83:0/644883473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb84006e440 con 0x7fb84006eb90 2024-04-25T00:14:56.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb83d600640 1 -- 172.21.15.83:0/644883473 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb83401e460 con 0x7fb84006eb90 2024-04-25T00:14:56.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.391+0000 7fb83d600640 1 --2- 172.21.15.83:0/644883473 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8200678c0 0x7fb820069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.395+0000 7fb83d600640 1 -- 172.21.15.83:0/644883473 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fb8340864e0 con 0x7fb84006eb90 2024-04-25T00:14:56.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.395+0000 7fb844a00640 1 --2- 172.21.15.83:0/644883473 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8200678c0 0x7fb820069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.395+0000 7fb844a00640 1 --2- 172.21.15.83:0/644883473 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8200678c0 0x7fb820069d80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0ad000 tx=0x7fb82c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:56.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.399+0000 7fb83d600640 1 -- 172.21.15.83:0/644883473 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb8340567a0 con 0x7fb84006eb90 2024-04-25T00:14:56.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.555+0000 7fb847e00640 1 -- 172.21.15.83:0/644883473 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8400b29f0 con 0x7fb8200678c0 2024-04-25T00:14:56.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.559+0000 7fb83d600640 1 -- 172.21.15.83:0/644883473 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252084 (secure 0 0 0) 0x7fb8400b29f0 con 0x7fb8200678c0 2024-04-25T00:14:56.567 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:56.569 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:14:56.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.563+0000 7fb847e00640 1 -- 172.21.15.83:0/644883473 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8200678c0 msgr2=0x7fb820069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.563+0000 7fb847e00640 1 --2- 172.21.15.83:0/644883473 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8200678c0 0x7fb820069d80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0ad000 tx=0x7fb82c008040 comp rx=0 tx=0).stop 2024-04-25T00:14:56.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.563+0000 7fb847e00640 1 -- 172.21.15.83:0/644883473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb84006eb90 msgr2=0x7fb84013d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.563+0000 7fb847e00640 1 --2- 172.21.15.83:0/644883473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb84006eb90 0x7fb84013d1d0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb83400d830 tx=0x7fb83400dcf0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.563+0000 7fb847e00640 1 -- 172.21.15.83:0/644883473 shutdown_connections 2024-04-25T00:14:56.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.563+0000 7fb847e00640 1 --2- 172.21.15.83:0/644883473 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb8200678c0 0x7fb820069d80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.563+0000 7fb847e00640 1 --2- 172.21.15.83:0/644883473 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb84015e5b0 0x7fb840157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.563+0000 7fb847e00640 1 --2- 172.21.15.83:0/644883473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8400cf810 0x7fb8400afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.563+0000 7fb847e00640 1 --2- 172.21.15.83:0/644883473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb84006eb90 0x7fb84013d1d0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.563+0000 7fb847e00640 1 -- 172.21.15.83:0/644883473 >> 172.21.15.83:0/644883473 conn(0x7fb8400b1d50 msgr2=0x7fb8401437c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:56.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.563+0000 7fb847e00640 1 -- 172.21.15.83:0/644883473 shutdown_connections 2024-04-25T00:14:56.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.563+0000 7fb847e00640 1 -- 172.21.15.83:0/644883473 wait complete. 2024-04-25T00:14:56.587 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-04-25T00:14:56.487809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259716,"kb_used_data":10248,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715004,"statfs":{"total":773094113280,"available":772828164096,"internally_reserved":0,"allocated":10493952,"data_stored":5406260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.996399"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508445+0000","last_change":"2024-04-25T00:14:52.517972+0000","last_active":"2024-04-25T00:14:53.508445+0000","last_peered":"2024-04-25T00:14:53.508445+0000","last_clean":"2024-04-25T00:14:53.508445+0000","last_became_active":"2024-04-25T00:14:52.517694+0000","last_became_peered":"2024-04-25T00:14:52.517694+0000","last_unstale":"2024-04-25T00:14:53.508445+0000","last_undegraded":"2024-04-25T00:14:53.508445+0000","last_fullsized":"2024-04-25T00:14:53.508445+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:24.016100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509653+0000","last_change":"2024-04-25T00:14:52.521208+0000","last_active":"2024-04-25T00:14:53.509653+0000","last_peered":"2024-04-25T00:14:53.509653+0000","last_clean":"2024-04-25T00:14:53.509653+0000","last_became_active":"2024-04-25T00:14:52.520679+0000","last_became_peered":"2024-04-25T00:14:52.520679+0000","last_unstale":"2024-04-25T00:14:53.509653+0000","last_undegraded":"2024-04-25T00:14:53.509653+0000","last_fullsized":"2024-04-25T00:14:53.509653+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:22.938318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.507928+0000","last_change":"2024-04-25T00:14:52.534061+0000","last_active":"2024-04-25T00:14:53.507928+0000","last_peered":"2024-04-25T00:14:53.507928+0000","last_clean":"2024-04-25T00:14:53.507928+0000","last_became_active":"2024-04-25T00:14:52.532699+0000","last_became_peered":"2024-04-25T00:14:52.532699+0000","last_unstale":"2024-04-25T00:14:53.507928+0000","last_undegraded":"2024-04-25T00:14:53.507928+0000","last_fullsized":"2024-04-25T00:14:53.507928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:45.529300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744560+0000","last_change":"2024-04-25T00:14:52.546549+0000","last_active":"2024-04-25T00:14:53.744560+0000","last_peered":"2024-04-25T00:14:53.744560+0000","last_clean":"2024-04-25T00:14:53.744560+0000","last_became_active":"2024-04-25T00:14:52.545833+0000","last_became_peered":"2024-04-25T00:14:52.545833+0000","last_unstale":"2024-04-25T00:14:53.744560+0000","last_undegraded":"2024-04-25T00:14:53.744560+0000","last_fullsized":"2024-04-25T00:14:53.744560+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:27.677704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745186+0000","last_change":"2024-04-25T00:14:52.530494+0000","last_active":"2024-04-25T00:14:53.745186+0000","last_peered":"2024-04-25T00:14:53.745186+0000","last_clean":"2024-04-25T00:14:53.745186+0000","last_became_active":"2024-04-25T00:14:52.530254+0000","last_became_peered":"2024-04-25T00:14:52.530254+0000","last_unstale":"2024-04-25T00:14:53.745186+0000","last_undegraded":"2024-04-25T00:14:53.745186+0000","last_fullsized":"2024-04-25T00:14:53.745186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:14.836642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509233+0000","last_change":"2024-04-25T00:14:52.522558+0000","last_active":"2024-04-25T00:14:53.509233+0000","last_peered":"2024-04-25T00:14:53.509233+0000","last_clean":"2024-04-25T00:14:53.509233+0000","last_became_active":"2024-04-25T00:14:52.522110+0000","last_became_peered":"2024-04-25T00:14:52.522110+0000","last_unstale":"2024-04-25T00:14:53.509233+0000","last_undegraded":"2024-04-25T00:14:53.509233+0000","last_fullsized":"2024-04-25T00:14:53.509233+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:56.739421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509094+0000","last_change":"2024-04-25T00:14:52.527261+0000","last_active":"2024-04-25T00:14:53.509094+0000","last_peered":"2024-04-25T00:14:53.509094+0000","last_clean":"2024-04-25T00:14:53.509094+0000","last_became_active":"2024-04-25T00:14:52.526945+0000","last_became_peered":"2024-04-25T00:14:52.526945+0000","last_unstale":"2024-04-25T00:14:53.509094+0000","last_undegraded":"2024-04-25T00:14:53.509094+0000","last_fullsized":"2024-04-25T00:14:53.509094+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:07.988481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509107+0000","last_change":"2024-04-25T00:14:52.533974+0000","last_active":"2024-04-25T00:14:53.509107+0000","last_peered":"2024-04-25T00:14:53.509107+0000","last_clean":"2024-04-25T00:14:53.509107+0000","last_became_active":"2024-04-25T00:14:52.533702+0000","last_became_peered":"2024-04-25T00:14:52.533702+0000","last_unstale":"2024-04-25T00:14:53.509107+0000","last_undegraded":"2024-04-25T00:14:53.509107+0000","last_fullsized":"2024-04-25T00:14:53.509107+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:21.700580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509803+0000","last_change":"2024-04-25T00:14:52.531007+0000","last_active":"2024-04-25T00:14:53.509803+0000","last_peered":"2024-04-25T00:14:53.509803+0000","last_clean":"2024-04-25T00:14:53.509803+0000","last_became_active":"2024-04-25T00:14:52.530740+0000","last_became_peered":"2024-04-25T00:14:52.530740+0000","last_unstale":"2024-04-25T00:14:53.509803+0000","last_undegraded":"2024-04-25T00:14:53.509803+0000","last_fullsized":"2024-04-25T00:14:53.509803+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:59.770333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745323+0000","last_change":"2024-04-25T00:14:52.521577+0000","last_active":"2024-04-25T00:14:53.745323+0000","last_peered":"2024-04-25T00:14:53.745323+0000","last_clean":"2024-04-25T00:14:53.745323+0000","last_became_active":"2024-04-25T00:14:52.521295+0000","last_became_peered":"2024-04-25T00:14:52.521295+0000","last_unstale":"2024-04-25T00:14:53.745323+0000","last_undegraded":"2024-04-25T00:14:53.745323+0000","last_fullsized":"2024-04-25T00:14:53.745323+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:41.604306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531497+0000","last_change":"2024-04-25T00:14:52.538705+0000","last_active":"2024-04-25T00:14:53.531497+0000","last_peered":"2024-04-25T00:14:53.531497+0000","last_clean":"2024-04-25T00:14:53.531497+0000","last_became_active":"2024-04-25T00:14:52.538365+0000","last_became_peered":"2024-04-25T00:14:52.538365+0000","last_unstale":"2024-04-25T00:14:53.531497+0000","last_undegraded":"2024-04-25T00:14:53.531497+0000","last_fullsized":"2024-04-25T00:14:53.531497+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:22.938318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745099+0000","last_change":"2024-04-25T00:14:52.535903+0000","last_active":"2024-04-25T00:14:53.745099+0000","last_peered":"2024-04-25T00:14:53.745099+0000","last_clean":"2024-04-25T00:14:53.745099+0000","last_became_active":"2024-04-25T00:14:52.535460+0000","last_became_peered":"2024-04-25T00:14:52.535460+0000","last_unstale":"2024-04-25T00:14:53.745099+0000","last_undegraded":"2024-04-25T00:14:53.745099+0000","last_fullsized":"2024-04-25T00:14:53.745099+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:22.645056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744534+0000","last_change":"2024-04-25T00:14:52.537936+0000","last_active":"2024-04-25T00:14:53.744534+0000","last_peered":"2024-04-25T00:14:53.744534+0000","last_clean":"2024-04-25T00:14:53.744534+0000","last_became_active":"2024-04-25T00:14:52.537653+0000","last_became_peered":"2024-04-25T00:14:52.537653+0000","last_unstale":"2024-04-25T00:14:53.744534+0000","last_undegraded":"2024-04-25T00:14:53.744534+0000","last_fullsized":"2024-04-25T00:14:53.744534+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:44.818488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745653+0000","last_change":"2024-04-25T00:14:52.522787+0000","last_active":"2024-04-25T00:14:53.745653+0000","last_peered":"2024-04-25T00:14:53.745653+0000","last_clean":"2024-04-25T00:14:53.745653+0000","last_became_active":"2024-04-25T00:14:52.522480+0000","last_became_peered":"2024-04-25T00:14:52.522480+0000","last_unstale":"2024-04-25T00:14:53.745653+0000","last_undegraded":"2024-04-25T00:14:53.745653+0000","last_fullsized":"2024-04-25T00:14:53.745653+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:16.493058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745019+0000","last_change":"2024-04-25T00:14:52.540161+0000","last_active":"2024-04-25T00:14:53.745019+0000","last_peered":"2024-04-25T00:14:53.745019+0000","last_clean":"2024-04-25T00:14:53.745019+0000","last_became_active":"2024-04-25T00:14:52.539070+0000","last_became_peered":"2024-04-25T00:14:52.539070+0000","last_unstale":"2024-04-25T00:14:53.745019+0000","last_undegraded":"2024-04-25T00:14:53.745019+0000","last_fullsized":"2024-04-25T00:14:53.745019+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:28.821898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744868+0000","last_change":"2024-04-25T00:14:52.534154+0000","last_active":"2024-04-25T00:14:53.744868+0000","last_peered":"2024-04-25T00:14:53.744868+0000","last_clean":"2024-04-25T00:14:53.744868+0000","last_became_active":"2024-04-25T00:14:52.532048+0000","last_became_peered":"2024-04-25T00:14:52.532048+0000","last_unstale":"2024-04-25T00:14:53.744868+0000","last_undegraded":"2024-04-25T00:14:53.744868+0000","last_fullsized":"2024-04-25T00:14:53.744868+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:45:16.996586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508591+0000","last_change":"2024-04-25T00:14:52.522849+0000","last_active":"2024-04-25T00:14:53.508591+0000","last_peered":"2024-04-25T00:14:53.508591+0000","last_clean":"2024-04-25T00:14:53.508591+0000","last_became_active":"2024-04-25T00:14:52.522275+0000","last_became_peered":"2024-04-25T00:14:52.522275+0000","last_unstale":"2024-04-25T00:14:53.508591+0000","last_undegraded":"2024-04-25T00:14:53.508591+0000","last_fullsized":"2024-04-25T00:14:53.508591+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:00.743577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509818+0000","last_change":"2024-04-25T00:14:52.522322+0000","last_active":"2024-04-25T00:14:53.509818+0000","last_peered":"2024-04-25T00:14:53.509818+0000","last_clean":"2024-04-25T00:14:53.509818+0000","last_became_active":"2024-04-25T00:14:52.522035+0000","last_became_peered":"2024-04-25T00:14:52.522035+0000","last_unstale":"2024-04-25T00:14:53.509818+0000","last_undegraded":"2024-04-25T00:14:53.509818+0000","last_fullsized":"2024-04-25T00:14:53.509818+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:46.512512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508159+0000","last_change":"2024-04-25T00:14:52.518924+0000","last_active":"2024-04-25T00:14:53.508159+0000","last_peered":"2024-04-25T00:14:53.508159+0000","last_clean":"2024-04-25T00:14:53.508159+0000","last_became_active":"2024-04-25T00:14:52.518629+0000","last_became_peered":"2024-04-25T00:14:52.518629+0000","last_unstale":"2024-04-25T00:14:53.508159+0000","last_undegraded":"2024-04-25T00:14:53.508159+0000","last_fullsized":"2024-04-25T00:14:53.508159+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:28.550925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745189+0000","last_change":"2024-04-25T00:14:52.542989+0000","last_active":"2024-04-25T00:14:53.745189+0000","last_peered":"2024-04-25T00:14:53.745189+0000","last_clean":"2024-04-25T00:14:53.745189+0000","last_became_active":"2024-04-25T00:14:52.542650+0000","last_became_peered":"2024-04-25T00:14:52.542650+0000","last_unstale":"2024-04-25T00:14:53.745189+0000","last_undegraded":"2024-04-25T00:14:53.745189+0000","last_fullsized":"2024-04-25T00:14:53.745189+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:38.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509527+0000","last_change":"2024-04-25T00:14:52.537817+0000","last_active":"2024-04-25T00:14:53.509527+0000","last_peered":"2024-04-25T00:14:53.509527+0000","last_clean":"2024-04-25T00:14:53.509527+0000","last_became_active":"2024-04-25T00:14:52.537567+0000","last_became_peered":"2024-04-25T00:14:52.537567+0000","last_unstale":"2024-04-25T00:14:53.509527+0000","last_undegraded":"2024-04-25T00:14:53.509527+0000","last_fullsized":"2024-04-25T00:14:53.509527+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:33.797911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745085+0000","last_change":"2024-04-25T00:14:52.529301+0000","last_active":"2024-04-25T00:14:53.745085+0000","last_peered":"2024-04-25T00:14:53.745085+0000","last_clean":"2024-04-25T00:14:53.745085+0000","last_became_active":"2024-04-25T00:14:52.528229+0000","last_became_peered":"2024-04-25T00:14:52.528229+0000","last_unstale":"2024-04-25T00:14:53.745085+0000","last_undegraded":"2024-04-25T00:14:53.745085+0000","last_fullsized":"2024-04-25T00:14:53.745085+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:56.739421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508224+0000","last_change":"2024-04-25T00:14:52.543215+0000","last_active":"2024-04-25T00:14:53.508224+0000","last_peered":"2024-04-25T00:14:53.508224+0000","last_clean":"2024-04-25T00:14:53.508224+0000","last_became_active":"2024-04-25T00:14:52.540249+0000","last_became_peered":"2024-04-25T00:14:52.540249+0000","last_unstale":"2024-04-25T00:14:53.508224+0000","last_undegraded":"2024-04-25T00:14:53.508224+0000","last_fullsized":"2024-04-25T00:14:53.508224+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:30.972848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744777+0000","last_change":"2024-04-25T00:14:52.515994+0000","last_active":"2024-04-25T00:14:53.744777+0000","last_peered":"2024-04-25T00:14:53.744777+0000","last_clean":"2024-04-25T00:14:53.744777+0000","last_became_active":"2024-04-25T00:14:52.515718+0000","last_became_peered":"2024-04-25T00:14:52.515718+0000","last_unstale":"2024-04-25T00:14:53.744777+0000","last_undegraded":"2024-04-25T00:14:53.744777+0000","last_fullsized":"2024-04-25T00:14:53.744777+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:07.988481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745111+0000","last_change":"2024-04-25T00:14:52.548896+0000","last_active":"2024-04-25T00:14:53.745111+0000","last_peered":"2024-04-25T00:14:53.745111+0000","last_clean":"2024-04-25T00:14:53.745111+0000","last_became_active":"2024-04-25T00:14:52.547845+0000","last_became_peered":"2024-04-25T00:14:52.547845+0000","last_unstale":"2024-04-25T00:14:53.745111+0000","last_undegraded":"2024-04-25T00:14:53.745111+0000","last_fullsized":"2024-04-25T00:14:53.745111+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:56.739421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745097+0000","last_change":"2024-04-25T00:14:52.536499+0000","last_active":"2024-04-25T00:14:53.745097+0000","last_peered":"2024-04-25T00:14:53.745097+0000","last_clean":"2024-04-25T00:14:53.745097+0000","last_became_active":"2024-04-25T00:14:52.536125+0000","last_became_peered":"2024-04-25T00:14:52.536125+0000","last_unstale":"2024-04-25T00:14:53.745097+0000","last_undegraded":"2024-04-25T00:14:53.745097+0000","last_fullsized":"2024-04-25T00:14:53.745097+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:07.988481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531429+0000","last_change":"2024-04-25T00:14:52.544966+0000","last_active":"2024-04-25T00:14:53.531429+0000","last_peered":"2024-04-25T00:14:53.531429+0000","last_clean":"2024-04-25T00:14:53.531429+0000","last_became_active":"2024-04-25T00:14:52.544686+0000","last_became_peered":"2024-04-25T00:14:52.544686+0000","last_unstale":"2024-04-25T00:14:53.531429+0000","last_undegraded":"2024-04-25T00:14:53.531429+0000","last_fullsized":"2024-04-25T00:14:53.531429+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:41.604306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509631+0000","last_change":"2024-04-25T00:14:52.529806+0000","last_active":"2024-04-25T00:14:53.509631+0000","last_peered":"2024-04-25T00:14:53.509631+0000","last_clean":"2024-04-25T00:14:53.509631+0000","last_became_active":"2024-04-25T00:14:52.529524+0000","last_became_peered":"2024-04-25T00:14:52.529524+0000","last_unstale":"2024-04-25T00:14:53.509631+0000","last_undegraded":"2024-04-25T00:14:53.509631+0000","last_fullsized":"2024-04-25T00:14:53.509631+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:41.604306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509627+0000","last_change":"2024-04-25T00:14:52.544251+0000","last_active":"2024-04-25T00:14:53.509627+0000","last_peered":"2024-04-25T00:14:53.509627+0000","last_clean":"2024-04-25T00:14:53.509627+0000","last_became_active":"2024-04-25T00:14:52.543984+0000","last_became_peered":"2024-04-25T00:14:52.543984+0000","last_unstale":"2024-04-25T00:14:53.509627+0000","last_undegraded":"2024-04-25T00:14:53.509627+0000","last_fullsized":"2024-04-25T00:14:53.509627+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:15.967409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744903+0000","last_change":"2024-04-25T00:14:52.543402+0000","last_active":"2024-04-25T00:14:53.744903+0000","last_peered":"2024-04-25T00:14:53.744903+0000","last_clean":"2024-04-25T00:14:53.744903+0000","last_became_active":"2024-04-25T00:14:52.543128+0000","last_became_peered":"2024-04-25T00:14:52.543128+0000","last_unstale":"2024-04-25T00:14:53.744903+0000","last_undegraded":"2024-04-25T00:14:53.744903+0000","last_fullsized":"2024-04-25T00:14:53.744903+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:04.926801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509112+0000","last_change":"2024-04-25T00:14:52.517426+0000","last_active":"2024-04-25T00:14:53.509112+0000","last_peered":"2024-04-25T00:14:53.509112+0000","last_clean":"2024-04-25T00:14:53.509112+0000","last_became_active":"2024-04-25T00:14:52.517104+0000","last_became_peered":"2024-04-25T00:14:52.517104+0000","last_unstale":"2024-04-25T00:14:53.509112+0000","last_undegraded":"2024-04-25T00:14:53.509112+0000","last_fullsized":"2024-04-25T00:14:53.509112+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:22.645056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509164+0000","last_change":"2024-04-25T00:14:52.546048+0000","last_active":"2024-04-25T00:14:53.509164+0000","last_peered":"2024-04-25T00:14:53.509164+0000","last_clean":"2024-04-25T00:14:53.509164+0000","last_became_active":"2024-04-25T00:14:52.545768+0000","last_became_peered":"2024-04-25T00:14:52.545768+0000","last_unstale":"2024-04-25T00:14:53.509164+0000","last_undegraded":"2024-04-25T00:14:53.509164+0000","last_fullsized":"2024-04-25T00:14:53.509164+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:43.680130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508112+0000","last_change":"2024-04-25T00:14:52.515584+0000","last_active":"2024-04-25T00:14:53.508112+0000","last_peered":"2024-04-25T00:14:53.508112+0000","last_clean":"2024-04-25T00:14:53.508112+0000","last_became_active":"2024-04-25T00:14:52.515333+0000","last_became_peered":"2024-04-25T00:14:52.515333+0000","last_unstale":"2024-04-25T00:14:53.508112+0000","last_undegraded":"2024-04-25T00:14:53.508112+0000","last_fullsized":"2024-04-25T00:14:53.508112+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:20:40.553267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"92'4","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744707+0000","last_change":"2024-04-25T00:14:51.512978+0000","last_active":"2024-04-25T00:14:53.744707+0000","last_peered":"2024-04-25T00:14:53.744707+0000","last_clean":"2024-04-25T00:14:53.744707+0000","last_became_active":"2024-04-25T00:14:51.512717+0000","last_became_peered":"2024-04-25T00:14:51.512717+0000","last_unstale":"2024-04-25T00:14:53.744707+0000","last_undegraded":"2024-04-25T00:14:53.744707+0000","last_fullsized":"2024-04-25T00:14:53.744707+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:21.471082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509182+0000","last_change":"2024-04-25T00:14:52.533579+0000","last_active":"2024-04-25T00:14:53.509182+0000","last_peered":"2024-04-25T00:14:53.509182+0000","last_clean":"2024-04-25T00:14:53.509182+0000","last_became_active":"2024-04-25T00:14:52.533208+0000","last_became_peered":"2024-04-25T00:14:52.533208+0000","last_unstale":"2024-04-25T00:14:53.509182+0000","last_undegraded":"2024-04-25T00:14:53.509182+0000","last_fullsized":"2024-04-25T00:14:53.509182+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:36.353389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509621+0000","last_change":"2024-04-25T00:14:51.513807+0000","last_active":"2024-04-25T00:14:53.509621+0000","last_peered":"2024-04-25T00:14:53.509621+0000","last_clean":"2024-04-25T00:14:53.509621+0000","last_became_active":"2024-04-25T00:14:51.513524+0000","last_became_peered":"2024-04-25T00:14:51.513524+0000","last_unstale":"2024-04-25T00:14:53.509621+0000","last_undegraded":"2024-04-25T00:14:53.509621+0000","last_fullsized":"2024-04-25T00:14:53.509621+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:13.698946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744990+0000","last_change":"2024-04-25T00:14:52.528252+0000","last_active":"2024-04-25T00:14:53.744990+0000","last_peered":"2024-04-25T00:14:53.744990+0000","last_clean":"2024-04-25T00:14:53.744990+0000","last_became_active":"2024-04-25T00:14:52.527839+0000","last_became_peered":"2024-04-25T00:14:52.527839+0000","last_unstale":"2024-04-25T00:14:53.744990+0000","last_undegraded":"2024-04-25T00:14:53.744990+0000","last_fullsized":"2024-04-25T00:14:53.744990+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:00.073031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744802+0000","last_change":"2024-04-25T00:14:51.519847+0000","last_active":"2024-04-25T00:14:53.744802+0000","last_peered":"2024-04-25T00:14:53.744802+0000","last_clean":"2024-04-25T00:14:53.744802+0000","last_became_active":"2024-04-25T00:14:51.518123+0000","last_became_peered":"2024-04-25T00:14:51.518123+0000","last_unstale":"2024-04-25T00:14:53.744802+0000","last_undegraded":"2024-04-25T00:14:53.744802+0000","last_fullsized":"2024-04-25T00:14:53.744802+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:13.128042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531196+0000","last_change":"2024-04-25T00:14:52.538781+0000","last_active":"2024-04-25T00:14:53.531196+0000","last_peered":"2024-04-25T00:14:53.531196+0000","last_clean":"2024-04-25T00:14:53.531196+0000","last_became_active":"2024-04-25T00:14:52.538202+0000","last_became_peered":"2024-04-25T00:14:52.538202+0000","last_unstale":"2024-04-25T00:14:53.531196+0000","last_undegraded":"2024-04-25T00:14:53.531196+0000","last_fullsized":"2024-04-25T00:14:53.531196+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:38.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745185+0000","last_change":"2024-04-25T00:14:51.516911+0000","last_active":"2024-04-25T00:14:53.745185+0000","last_peered":"2024-04-25T00:14:53.745185+0000","last_clean":"2024-04-25T00:14:53.745185+0000","last_became_active":"2024-04-25T00:14:51.516675+0000","last_became_peered":"2024-04-25T00:14:51.516675+0000","last_unstale":"2024-04-25T00:14:53.745185+0000","last_undegraded":"2024-04-25T00:14:53.745185+0000","last_fullsized":"2024-04-25T00:14:53.745185+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:13.128042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744692+0000","last_change":"2024-04-25T00:14:52.521917+0000","last_active":"2024-04-25T00:14:53.744692+0000","last_peered":"2024-04-25T00:14:53.744692+0000","last_clean":"2024-04-25T00:14:53.744692+0000","last_became_active":"2024-04-25T00:14:52.521499+0000","last_became_peered":"2024-04-25T00:14:52.521499+0000","last_unstale":"2024-04-25T00:14:53.744692+0000","last_undegraded":"2024-04-25T00:14:53.744692+0000","last_fullsized":"2024-04-25T00:14:53.744692+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:07.461779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745364+0000","last_change":"2024-04-25T00:14:51.510850+0000","last_active":"2024-04-25T00:14:53.745364+0000","last_peered":"2024-04-25T00:14:53.745364+0000","last_clean":"2024-04-25T00:14:53.745364+0000","last_became_active":"2024-04-25T00:14:51.510479+0000","last_became_peered":"2024-04-25T00:14:51.510479+0000","last_unstale":"2024-04-25T00:14:53.745364+0000","last_undegraded":"2024-04-25T00:14:53.745364+0000","last_fullsized":"2024-04-25T00:14:53.745364+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:13.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508456+0000","last_change":"2024-04-25T00:14:52.517849+0000","last_active":"2024-04-25T00:14:53.508456+0000","last_peered":"2024-04-25T00:14:53.508456+0000","last_clean":"2024-04-25T00:14:53.508456+0000","last_became_active":"2024-04-25T00:14:52.517593+0000","last_became_peered":"2024-04-25T00:14:52.517593+0000","last_unstale":"2024-04-25T00:14:53.508456+0000","last_undegraded":"2024-04-25T00:14:53.508456+0000","last_fullsized":"2024-04-25T00:14:53.508456+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:55.977402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531041+0000","last_change":"2024-04-25T00:14:51.516511+0000","last_active":"2024-04-25T00:14:53.531041+0000","last_peered":"2024-04-25T00:14:53.531041+0000","last_clean":"2024-04-25T00:14:53.531041+0000","last_became_active":"2024-04-25T00:14:51.515960+0000","last_became_peered":"2024-04-25T00:14:51.515960+0000","last_unstale":"2024-04-25T00:14:53.531041+0000","last_undegraded":"2024-04-25T00:14:53.531041+0000","last_fullsized":"2024-04-25T00:14:53.531041+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:01.079041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509878+0000","last_change":"2024-04-25T00:14:52.526791+0000","last_active":"2024-04-25T00:14:53.509878+0000","last_peered":"2024-04-25T00:14:53.509878+0000","last_clean":"2024-04-25T00:14:53.509878+0000","last_became_active":"2024-04-25T00:14:52.525048+0000","last_became_peered":"2024-04-25T00:14:52.525048+0000","last_unstale":"2024-04-25T00:14:53.509878+0000","last_undegraded":"2024-04-25T00:14:53.509878+0000","last_fullsized":"2024-04-25T00:14:53.509878+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:22.474854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744653+0000","last_change":"2024-04-25T00:14:51.521606+0000","last_active":"2024-04-25T00:14:53.744653+0000","last_peered":"2024-04-25T00:14:53.744653+0000","last_clean":"2024-04-25T00:14:53.744653+0000","last_became_active":"2024-04-25T00:14:51.520796+0000","last_became_peered":"2024-04-25T00:14:51.520796+0000","last_unstale":"2024-04-25T00:14:53.744653+0000","last_undegraded":"2024-04-25T00:14:53.744653+0000","last_fullsized":"2024-04-25T00:14:53.744653+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:46.406675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744735+0000","last_change":"2024-04-25T00:14:52.515560+0000","last_active":"2024-04-25T00:14:53.744735+0000","last_peered":"2024-04-25T00:14:53.744735+0000","last_clean":"2024-04-25T00:14:53.744735+0000","last_became_active":"2024-04-25T00:14:52.515279+0000","last_became_peered":"2024-04-25T00:14:52.515279+0000","last_unstale":"2024-04-25T00:14:53.744735+0000","last_undegraded":"2024-04-25T00:14:53.744735+0000","last_fullsized":"2024-04-25T00:14:53.744735+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:12.955085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744279+0000","last_change":"2024-04-25T00:14:51.516515+0000","last_active":"2024-04-25T00:14:53.744279+0000","last_peered":"2024-04-25T00:14:53.744279+0000","last_clean":"2024-04-25T00:14:53.744279+0000","last_became_active":"2024-04-25T00:14:51.516270+0000","last_became_peered":"2024-04-25T00:14:51.516270+0000","last_unstale":"2024-04-25T00:14:53.744279+0000","last_undegraded":"2024-04-25T00:14:53.744279+0000","last_fullsized":"2024-04-25T00:14:53.744279+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:58.766561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.507975+0000","last_change":"2024-04-25T00:14:52.514745+0000","last_active":"2024-04-25T00:14:53.507975+0000","last_peered":"2024-04-25T00:14:53.507975+0000","last_clean":"2024-04-25T00:14:53.507975+0000","last_became_active":"2024-04-25T00:14:52.514446+0000","last_became_peered":"2024-04-25T00:14:52.514446+0000","last_unstale":"2024-04-25T00:14:53.507975+0000","last_undegraded":"2024-04-25T00:14:53.507975+0000","last_fullsized":"2024-04-25T00:14:53.507975+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:57.564616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745306+0000","last_change":"2024-04-25T00:14:51.515705+0000","last_active":"2024-04-25T00:14:53.745306+0000","last_peered":"2024-04-25T00:14:53.745306+0000","last_clean":"2024-04-25T00:14:53.745306+0000","last_became_active":"2024-04-25T00:14:51.515448+0000","last_became_peered":"2024-04-25T00:14:51.515448+0000","last_unstale":"2024-04-25T00:14:53.745306+0000","last_undegraded":"2024-04-25T00:14:53.745306+0000","last_fullsized":"2024-04-25T00:14:53.745306+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:12.379516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744931+0000","last_change":"2024-04-25T00:14:52.534318+0000","last_active":"2024-04-25T00:14:53.744931+0000","last_peered":"2024-04-25T00:14:53.744931+0000","last_clean":"2024-04-25T00:14:53.744931+0000","last_became_active":"2024-04-25T00:14:52.532716+0000","last_became_peered":"2024-04-25T00:14:52.532716+0000","last_unstale":"2024-04-25T00:14:53.744931+0000","last_undegraded":"2024-04-25T00:14:53.744931+0000","last_fullsized":"2024-04-25T00:14:53.744931+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:22.645056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508339+0000","last_change":"2024-04-25T00:14:51.523093+0000","last_active":"2024-04-25T00:14:53.508339+0000","last_peered":"2024-04-25T00:14:53.508339+0000","last_clean":"2024-04-25T00:14:53.508339+0000","last_became_active":"2024-04-25T00:14:51.522776+0000","last_became_peered":"2024-04-25T00:14:51.522776+0000","last_unstale":"2024-04-25T00:14:53.508339+0000","last_undegraded":"2024-04-25T00:14:53.508339+0000","last_fullsized":"2024-04-25T00:14:53.508339+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:26.673932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509338+0000","last_change":"2024-04-25T00:14:52.533353+0000","last_active":"2024-04-25T00:14:53.509338+0000","last_peered":"2024-04-25T00:14:53.509338+0000","last_clean":"2024-04-25T00:14:53.509338+0000","last_became_active":"2024-04-25T00:14:52.533086+0000","last_became_peered":"2024-04-25T00:14:52.533086+0000","last_unstale":"2024-04-25T00:14:53.509338+0000","last_undegraded":"2024-04-25T00:14:53.509338+0000","last_fullsized":"2024-04-25T00:14:53.509338+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:36.422235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"92'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745399+0000","last_change":"2024-04-25T00:14:51.521745+0000","last_active":"2024-04-25T00:14:53.745399+0000","last_peered":"2024-04-25T00:14:53.745399+0000","last_clean":"2024-04-25T00:14:53.745399+0000","last_became_active":"2024-04-25T00:14:51.521422+0000","last_became_peered":"2024-04-25T00:14:51.521422+0000","last_unstale":"2024-04-25T00:14:53.745399+0000","last_undegraded":"2024-04-25T00:14:53.745399+0000","last_fullsized":"2024-04-25T00:14:53.745399+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:55.735649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745336+0000","last_change":"2024-04-25T00:14:52.534542+0000","last_active":"2024-04-25T00:14:53.745336+0000","last_peered":"2024-04-25T00:14:53.745336+0000","last_clean":"2024-04-25T00:14:53.745336+0000","last_became_active":"2024-04-25T00:14:52.534247+0000","last_became_peered":"2024-04-25T00:14:52.534247+0000","last_unstale":"2024-04-25T00:14:53.745336+0000","last_undegraded":"2024-04-25T00:14:53.745336+0000","last_fullsized":"2024-04-25T00:14:53.745336+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:22.938318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"92'6","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509744+0000","last_change":"2024-04-25T00:14:51.521244+0000","last_active":"2024-04-25T00:14:53.509744+0000","last_peered":"2024-04-25T00:14:53.509744+0000","last_clean":"2024-04-25T00:14:53.509744+0000","last_became_active":"2024-04-25T00:14:51.520964+0000","last_became_peered":"2024-04-25T00:14:51.520964+0000","last_unstale":"2024-04-25T00:14:53.509744+0000","last_undegraded":"2024-04-25T00:14:53.509744+0000","last_fullsized":"2024-04-25T00:14:53.509744+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:46.406675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531245+0000","last_change":"2024-04-25T00:14:52.532881+0000","last_active":"2024-04-25T00:14:53.531245+0000","last_peered":"2024-04-25T00:14:53.531245+0000","last_clean":"2024-04-25T00:14:53.531245+0000","last_became_active":"2024-04-25T00:14:52.532443+0000","last_became_peered":"2024-04-25T00:14:52.532443+0000","last_unstale":"2024-04-25T00:14:53.531245+0000","last_undegraded":"2024-04-25T00:14:53.531245+0000","last_fullsized":"2024-04-25T00:14:53.531245+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:14.836642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745275+0000","last_change":"2024-04-25T00:14:51.516174+0000","last_active":"2024-04-25T00:14:53.745275+0000","last_peered":"2024-04-25T00:14:53.745275+0000","last_clean":"2024-04-25T00:14:53.745275+0000","last_became_active":"2024-04-25T00:14:51.515922+0000","last_became_peered":"2024-04-25T00:14:51.515922+0000","last_unstale":"2024-04-25T00:14:53.745275+0000","last_undegraded":"2024-04-25T00:14:53.745275+0000","last_fullsized":"2024-04-25T00:14:53.745275+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:41.712373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509603+0000","last_change":"2024-04-25T00:14:52.532293+0000","last_active":"2024-04-25T00:14:53.509603+0000","last_peered":"2024-04-25T00:14:53.509603+0000","last_clean":"2024-04-25T00:14:53.509603+0000","last_became_active":"2024-04-25T00:14:52.531899+0000","last_became_peered":"2024-04-25T00:14:52.531899+0000","last_unstale":"2024-04-25T00:14:53.509603+0000","last_undegraded":"2024-04-25T00:14:53.509603+0000","last_fullsized":"2024-04-25T00:14:53.509603+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:14.836642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744420+0000","last_change":"2024-04-25T00:14:51.513489+0000","last_active":"2024-04-25T00:14:53.744420+0000","last_peered":"2024-04-25T00:14:53.744420+0000","last_clean":"2024-04-25T00:14:53.744420+0000","last_became_active":"2024-04-25T00:14:51.513211+0000","last_became_peered":"2024-04-25T00:14:51.513211+0000","last_unstale":"2024-04-25T00:14:53.744420+0000","last_undegraded":"2024-04-25T00:14:53.744420+0000","last_fullsized":"2024-04-25T00:14:53.744420+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:45.508740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509517+0000","last_change":"2024-04-25T00:14:52.535915+0000","last_active":"2024-04-25T00:14:53.509517+0000","last_peered":"2024-04-25T00:14:53.509517+0000","last_clean":"2024-04-25T00:14:53.509517+0000","last_became_active":"2024-04-25T00:14:52.535690+0000","last_became_peered":"2024-04-25T00:14:52.535690+0000","last_unstale":"2024-04-25T00:14:53.509517+0000","last_undegraded":"2024-04-25T00:14:53.509517+0000","last_fullsized":"2024-04-25T00:14:53.509517+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:51.025060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509172+0000","last_change":"2024-04-25T00:14:51.521951+0000","last_active":"2024-04-25T00:14:53.509172+0000","last_peered":"2024-04-25T00:14:53.509172+0000","last_clean":"2024-04-25T00:14:53.509172+0000","last_became_active":"2024-04-25T00:14:51.521567+0000","last_became_peered":"2024-04-25T00:14:51.521567+0000","last_unstale":"2024-04-25T00:14:53.509172+0000","last_undegraded":"2024-04-25T00:14:53.509172+0000","last_fullsized":"2024-04-25T00:14:53.509172+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:16.457613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508701+0000","last_change":"2024-04-25T00:14:52.523082+0000","last_active":"2024-04-25T00:14:53.508701+0000","last_peered":"2024-04-25T00:14:53.508701+0000","last_clean":"2024-04-25T00:14:53.508701+0000","last_became_active":"2024-04-25T00:14:52.522611+0000","last_became_peered":"2024-04-25T00:14:52.522611+0000","last_unstale":"2024-04-25T00:14:53.508701+0000","last_undegraded":"2024-04-25T00:14:53.508701+0000","last_fullsized":"2024-04-25T00:14:53.508701+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:06.961202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744636+0000","last_change":"2024-04-25T00:14:51.514707+0000","last_active":"2024-04-25T00:14:53.744636+0000","last_peered":"2024-04-25T00:14:53.744636+0000","last_clean":"2024-04-25T00:14:53.744636+0000","last_became_active":"2024-04-25T00:14:51.514398+0000","last_became_peered":"2024-04-25T00:14:51.514398+0000","last_unstale":"2024-04-25T00:14:53.744636+0000","last_undegraded":"2024-04-25T00:14:53.744636+0000","last_fullsized":"2024-04-25T00:14:53.744636+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:41.712373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509258+0000","last_change":"2024-04-25T00:14:52.529814+0000","last_active":"2024-04-25T00:14:53.509258+0000","last_peered":"2024-04-25T00:14:53.509258+0000","last_clean":"2024-04-25T00:14:53.509258+0000","last_became_active":"2024-04-25T00:14:52.529525+0000","last_became_peered":"2024-04-25T00:14:52.529525+0000","last_unstale":"2024-04-25T00:14:53.509258+0000","last_undegraded":"2024-04-25T00:14:53.509258+0000","last_fullsized":"2024-04-25T00:14:53.509258+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:36.226306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509800+0000","last_change":"2024-04-25T00:14:51.519177+0000","last_active":"2024-04-25T00:14:53.509800+0000","last_peered":"2024-04-25T00:14:53.509800+0000","last_clean":"2024-04-25T00:14:53.509800+0000","last_became_active":"2024-04-25T00:14:51.518780+0000","last_became_peered":"2024-04-25T00:14:51.518780+0000","last_unstale":"2024-04-25T00:14:53.509800+0000","last_undegraded":"2024-04-25T00:14:53.509800+0000","last_fullsized":"2024-04-25T00:14:53.509800+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:41.712373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509350+0000","last_change":"2024-04-25T00:14:52.524475+0000","last_active":"2024-04-25T00:14:53.509350+0000","last_peered":"2024-04-25T00:14:53.509350+0000","last_clean":"2024-04-25T00:14:53.509350+0000","last_became_active":"2024-04-25T00:14:52.524170+0000","last_became_peered":"2024-04-25T00:14:52.524170+0000","last_unstale":"2024-04-25T00:14:53.509350+0000","last_undegraded":"2024-04-25T00:14:53.509350+0000","last_fullsized":"2024-04-25T00:14:53.509350+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:38.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745509+0000","last_change":"2024-04-25T00:14:51.518281+0000","last_active":"2024-04-25T00:14:53.745509+0000","last_peered":"2024-04-25T00:14:53.745509+0000","last_clean":"2024-04-25T00:14:53.745509+0000","last_became_active":"2024-04-25T00:14:51.518016+0000","last_became_peered":"2024-04-25T00:14:51.518016+0000","last_unstale":"2024-04-25T00:14:53.745509+0000","last_undegraded":"2024-04-25T00:14:53.745509+0000","last_fullsized":"2024-04-25T00:14:53.745509+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:37.196765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745123+0000","last_change":"2024-04-25T00:14:52.521190+0000","last_active":"2024-04-25T00:14:53.745123+0000","last_peered":"2024-04-25T00:14:53.745123+0000","last_clean":"2024-04-25T00:14:53.745123+0000","last_became_active":"2024-04-25T00:14:52.520323+0000","last_became_peered":"2024-04-25T00:14:52.520323+0000","last_unstale":"2024-04-25T00:14:53.745123+0000","last_undegraded":"2024-04-25T00:14:53.745123+0000","last_fullsized":"2024-04-25T00:14:53.745123+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:38.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531212+0000","last_change":"2024-04-25T00:14:51.517499+0000","last_active":"2024-04-25T00:14:53.531212+0000","last_peered":"2024-04-25T00:14:53.531212+0000","last_clean":"2024-04-25T00:14:53.531212+0000","last_became_active":"2024-04-25T00:14:51.516256+0000","last_became_peered":"2024-04-25T00:14:51.516256+0000","last_unstale":"2024-04-25T00:14:53.531212+0000","last_undegraded":"2024-04-25T00:14:53.531212+0000","last_fullsized":"2024-04-25T00:14:53.531212+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:35.977865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508016+0000","last_change":"2024-04-25T00:14:52.518545+0000","last_active":"2024-04-25T00:14:53.508016+0000","last_peered":"2024-04-25T00:14:53.508016+0000","last_clean":"2024-04-25T00:14:53.508016+0000","last_became_active":"2024-04-25T00:14:52.518349+0000","last_became_peered":"2024-04-25T00:14:52.518349+0000","last_unstale":"2024-04-25T00:14:53.508016+0000","last_undegraded":"2024-04-25T00:14:53.508016+0000","last_fullsized":"2024-04-25T00:14:53.508016+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:05.464806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509657+0000","last_change":"2024-04-25T00:14:51.521137+0000","last_active":"2024-04-25T00:14:53.509657+0000","last_peered":"2024-04-25T00:14:53.509657+0000","last_clean":"2024-04-25T00:14:53.509657+0000","last_became_active":"2024-04-25T00:14:51.520880+0000","last_became_peered":"2024-04-25T00:14:51.520880+0000","last_unstale":"2024-04-25T00:14:53.509657+0000","last_undegraded":"2024-04-25T00:14:53.509657+0000","last_fullsized":"2024-04-25T00:14:53.509657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:02.743200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508309+0000","last_change":"2024-04-25T00:14:52.520152+0000","last_active":"2024-04-25T00:14:53.508309+0000","last_peered":"2024-04-25T00:14:53.508309+0000","last_clean":"2024-04-25T00:14:53.508309+0000","last_became_active":"2024-04-25T00:14:52.519374+0000","last_became_peered":"2024-04-25T00:14:52.519374+0000","last_unstale":"2024-04-25T00:14:53.508309+0000","last_undegraded":"2024-04-25T00:14:53.508309+0000","last_fullsized":"2024-04-25T00:14:53.508309+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:17.886622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509154+0000","last_change":"2024-04-25T00:14:51.519999+0000","last_active":"2024-04-25T00:14:53.509154+0000","last_peered":"2024-04-25T00:14:53.509154+0000","last_clean":"2024-04-25T00:14:53.509154+0000","last_became_active":"2024-04-25T00:14:51.519734+0000","last_became_peered":"2024-04-25T00:14:51.519734+0000","last_unstale":"2024-04-25T00:14:53.509154+0000","last_undegraded":"2024-04-25T00:14:53.509154+0000","last_fullsized":"2024-04-25T00:14:53.509154+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:39.607770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744537+0000","last_change":"2024-04-25T00:14:52.518406+0000","last_active":"2024-04-25T00:14:53.744537+0000","last_peered":"2024-04-25T00:14:53.744537+0000","last_clean":"2024-04-25T00:14:53.744537+0000","last_became_active":"2024-04-25T00:14:52.518151+0000","last_became_peered":"2024-04-25T00:14:52.518151+0000","last_unstale":"2024-04-25T00:14:53.744537+0000","last_undegraded":"2024-04-25T00:14:53.744537+0000","last_fullsized":"2024-04-25T00:14:53.744537+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:34.309725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509226+0000","last_change":"2024-04-25T00:14:51.511936+0000","last_active":"2024-04-25T00:14:53.509226+0000","last_peered":"2024-04-25T00:14:53.509226+0000","last_clean":"2024-04-25T00:14:53.509226+0000","last_became_active":"2024-04-25T00:14:51.511671+0000","last_became_peered":"2024-04-25T00:14:51.511671+0000","last_unstale":"2024-04-25T00:14:53.509226+0000","last_undegraded":"2024-04-25T00:14:53.509226+0000","last_fullsized":"2024-04-25T00:14:53.509226+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:12.379516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509768+0000","last_change":"2024-04-25T00:14:52.529212+0000","last_active":"2024-04-25T00:14:53.509768+0000","last_peered":"2024-04-25T00:14:53.509768+0000","last_clean":"2024-04-25T00:14:53.509768+0000","last_became_active":"2024-04-25T00:14:52.528900+0000","last_became_peered":"2024-04-25T00:14:52.528900+0000","last_unstale":"2024-04-25T00:14:53.509768+0000","last_undegraded":"2024-04-25T00:14:53.509768+0000","last_fullsized":"2024-04-25T00:14:53.509768+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:50.985016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509254+0000","last_change":"2024-04-25T00:14:51.521000+0000","last_active":"2024-04-25T00:14:53.509254+0000","last_peered":"2024-04-25T00:14:53.509254+0000","last_clean":"2024-04-25T00:14:53.509254+0000","last_became_active":"2024-04-25T00:14:51.520726+0000","last_became_peered":"2024-04-25T00:14:51.520726+0000","last_unstale":"2024-04-25T00:14:53.509254+0000","last_undegraded":"2024-04-25T00:14:53.509254+0000","last_fullsized":"2024-04-25T00:14:53.509254+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:46.910183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.507882+0000","last_change":"2024-04-25T00:14:52.520178+0000","last_active":"2024-04-25T00:14:53.507882+0000","last_peered":"2024-04-25T00:14:53.507882+0000","last_clean":"2024-04-25T00:14:53.507882+0000","last_became_active":"2024-04-25T00:14:52.519896+0000","last_became_peered":"2024-04-25T00:14:52.519896+0000","last_unstale":"2024-04-25T00:14:53.507882+0000","last_undegraded":"2024-04-25T00:14:53.507882+0000","last_fullsized":"2024-04-25T00:14:53.507882+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:57.182997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745238+0000","last_change":"2024-04-25T00:14:51.514643+0000","last_active":"2024-04-25T00:14:53.745238+0000","last_peered":"2024-04-25T00:14:53.745238+0000","last_clean":"2024-04-25T00:14:53.745238+0000","last_became_active":"2024-04-25T00:14:51.514472+0000","last_became_peered":"2024-04-25T00:14:51.514472+0000","last_unstale":"2024-04-25T00:14:53.745238+0000","last_undegraded":"2024-04-25T00:14:53.745238+0000","last_fullsized":"2024-04-25T00:14:53.745238+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:26.554045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744690+0000","last_change":"2024-04-25T00:14:52.536465+0000","last_active":"2024-04-25T00:14:53.744690+0000","last_peered":"2024-04-25T00:14:53.744690+0000","last_clean":"2024-04-25T00:14:53.744690+0000","last_became_active":"2024-04-25T00:14:52.536200+0000","last_became_peered":"2024-04-25T00:14:52.536200+0000","last_unstale":"2024-04-25T00:14:53.744690+0000","last_undegraded":"2024-04-25T00:14:53.744690+0000","last_fullsized":"2024-04-25T00:14:53.744690+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:50.842042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"93'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.530015+0000","last_change":"2024-04-25T00:14:51.519777+0000","last_active":"2024-04-25T00:14:53.530015+0000","last_peered":"2024-04-25T00:14:53.530015+0000","last_clean":"2024-04-25T00:14:53.530015+0000","last_became_active":"2024-04-25T00:14:51.519515+0000","last_became_peered":"2024-04-25T00:14:51.519515+0000","last_unstale":"2024-04-25T00:14:53.530015+0000","last_undegraded":"2024-04-25T00:14:53.530015+0000","last_fullsized":"2024-04-25T00:14:53.530015+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:06.458007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744404+0000","last_change":"2024-04-25T00:14:52.517941+0000","last_active":"2024-04-25T00:14:53.744404+0000","last_peered":"2024-04-25T00:14:53.744404+0000","last_clean":"2024-04-25T00:14:53.744404+0000","last_became_active":"2024-04-25T00:14:52.517675+0000","last_became_peered":"2024-04-25T00:14:52.517675+0000","last_unstale":"2024-04-25T00:14:53.744404+0000","last_undegraded":"2024-04-25T00:14:53.744404+0000","last_fullsized":"2024-04-25T00:14:53.744404+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:36.450062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509557+0000","last_change":"2024-04-25T00:14:51.517299+0000","last_active":"2024-04-25T00:14:53.509557+0000","last_peered":"2024-04-25T00:14:53.509557+0000","last_clean":"2024-04-25T00:14:53.509557+0000","last_became_active":"2024-04-25T00:14:51.517004+0000","last_became_peered":"2024-04-25T00:14:51.517004+0000","last_unstale":"2024-04-25T00:14:53.509557+0000","last_undegraded":"2024-04-25T00:14:53.509557+0000","last_fullsized":"2024-04-25T00:14:53.509557+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:52.589376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531194+0000","last_change":"2024-04-25T00:14:52.530122+0000","last_active":"2024-04-25T00:14:53.531194+0000","last_peered":"2024-04-25T00:14:53.531194+0000","last_clean":"2024-04-25T00:14:53.531194+0000","last_became_active":"2024-04-25T00:14:52.529866+0000","last_became_peered":"2024-04-25T00:14:52.529866+0000","last_unstale":"2024-04-25T00:14:53.531194+0000","last_undegraded":"2024-04-25T00:14:53.531194+0000","last_fullsized":"2024-04-25T00:14:53.531194+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:56.739421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745293+0000","last_change":"2024-04-25T00:14:51.513962+0000","last_active":"2024-04-25T00:14:53.745293+0000","last_peered":"2024-04-25T00:14:53.745293+0000","last_clean":"2024-04-25T00:14:53.745293+0000","last_became_active":"2024-04-25T00:14:51.513711+0000","last_became_peered":"2024-04-25T00:14:51.513711+0000","last_unstale":"2024-04-25T00:14:53.745293+0000","last_undegraded":"2024-04-25T00:14:53.745293+0000","last_fullsized":"2024-04-25T00:14:53.745293+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:29.130112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509520+0000","last_change":"2024-04-25T00:14:52.539935+0000","last_active":"2024-04-25T00:14:53.509520+0000","last_peered":"2024-04-25T00:14:53.509520+0000","last_clean":"2024-04-25T00:14:53.509520+0000","last_became_active":"2024-04-25T00:14:52.539617+0000","last_became_peered":"2024-04-25T00:14:52.539617+0000","last_unstale":"2024-04-25T00:14:53.509520+0000","last_undegraded":"2024-04-25T00:14:53.509520+0000","last_fullsized":"2024-04-25T00:14:53.509520+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:08.714777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744783+0000","last_change":"2024-04-25T00:14:51.516586+0000","last_active":"2024-04-25T00:14:53.744783+0000","last_peered":"2024-04-25T00:14:53.744783+0000","last_clean":"2024-04-25T00:14:53.744783+0000","last_became_active":"2024-04-25T00:14:51.516088+0000","last_became_peered":"2024-04-25T00:14:51.516088+0000","last_unstale":"2024-04-25T00:14:53.744783+0000","last_undegraded":"2024-04-25T00:14:53.744783+0000","last_fullsized":"2024-04-25T00:14:53.744783+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:52.589376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509458+0000","last_change":"2024-04-25T00:14:52.535104+0000","last_active":"2024-04-25T00:14:53.509458+0000","last_peered":"2024-04-25T00:14:53.509458+0000","last_clean":"2024-04-25T00:14:53.509458+0000","last_became_active":"2024-04-25T00:14:52.534827+0000","last_became_peered":"2024-04-25T00:14:52.534827+0000","last_unstale":"2024-04-25T00:14:53.509458+0000","last_undegraded":"2024-04-25T00:14:53.509458+0000","last_fullsized":"2024-04-25T00:14:53.509458+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:56.396750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509556+0000","last_change":"2024-04-25T00:14:51.519047+0000","last_active":"2024-04-25T00:14:53.509556+0000","last_peered":"2024-04-25T00:14:53.509556+0000","last_clean":"2024-04-25T00:14:53.509556+0000","last_became_active":"2024-04-25T00:14:51.518767+0000","last_became_peered":"2024-04-25T00:14:51.518767+0000","last_unstale":"2024-04-25T00:14:53.509556+0000","last_undegraded":"2024-04-25T00:14:53.509556+0000","last_fullsized":"2024-04-25T00:14:53.509556+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:13.389688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745561+0000","last_change":"2024-04-25T00:14:52.524928+0000","last_active":"2024-04-25T00:14:53.745561+0000","last_peered":"2024-04-25T00:14:53.745561+0000","last_clean":"2024-04-25T00:14:53.745561+0000","last_became_active":"2024-04-25T00:14:52.524551+0000","last_became_peered":"2024-04-25T00:14:52.524551+0000","last_unstale":"2024-04-25T00:14:53.745561+0000","last_undegraded":"2024-04-25T00:14:53.745561+0000","last_fullsized":"2024-04-25T00:14:53.745561+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:46.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509235+0000","last_change":"2024-04-25T00:14:51.517435+0000","last_active":"2024-04-25T00:14:53.509235+0000","last_peered":"2024-04-25T00:14:53.509235+0000","last_clean":"2024-04-25T00:14:53.509235+0000","last_became_active":"2024-04-25T00:14:51.517082+0000","last_became_peered":"2024-04-25T00:14:51.517082+0000","last_unstale":"2024-04-25T00:14:53.509235+0000","last_undegraded":"2024-04-25T00:14:53.509235+0000","last_fullsized":"2024-04-25T00:14:53.509235+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:00.869615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531586+0000","last_change":"2024-04-25T00:14:52.538711+0000","last_active":"2024-04-25T00:14:53.531586+0000","last_peered":"2024-04-25T00:14:53.531586+0000","last_clean":"2024-04-25T00:14:53.531586+0000","last_became_active":"2024-04-25T00:14:52.538445+0000","last_became_peered":"2024-04-25T00:14:52.538445+0000","last_unstale":"2024-04-25T00:14:53.531586+0000","last_undegraded":"2024-04-25T00:14:53.531586+0000","last_fullsized":"2024-04-25T00:14:53.531586+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:59.770333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509355+0000","last_change":"2024-04-25T00:14:51.516012+0000","last_active":"2024-04-25T00:14:53.509355+0000","last_peered":"2024-04-25T00:14:53.509355+0000","last_clean":"2024-04-25T00:14:53.509355+0000","last_became_active":"2024-04-25T00:14:51.515625+0000","last_became_peered":"2024-04-25T00:14:51.515625+0000","last_unstale":"2024-04-25T00:14:53.509355+0000","last_undegraded":"2024-04-25T00:14:53.509355+0000","last_fullsized":"2024-04-25T00:14:53.509355+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:47.065775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531102+0000","last_change":"2024-04-25T00:14:52.531091+0000","last_active":"2024-04-25T00:14:53.531102+0000","last_peered":"2024-04-25T00:14:53.531102+0000","last_clean":"2024-04-25T00:14:53.531102+0000","last_became_active":"2024-04-25T00:14:52.530720+0000","last_became_peered":"2024-04-25T00:14:52.530720+0000","last_unstale":"2024-04-25T00:14:53.531102+0000","last_undegraded":"2024-04-25T00:14:53.531102+0000","last_fullsized":"2024-04-25T00:14:53.531102+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:22.645056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509647+0000","last_change":"2024-04-25T00:14:51.519517+0000","last_active":"2024-04-25T00:14:53.509647+0000","last_peered":"2024-04-25T00:14:53.509647+0000","last_clean":"2024-04-25T00:14:53.509647+0000","last_became_active":"2024-04-25T00:14:51.519245+0000","last_became_peered":"2024-04-25T00:14:51.519245+0000","last_unstale":"2024-04-25T00:14:53.509647+0000","last_undegraded":"2024-04-25T00:14:53.509647+0000","last_fullsized":"2024-04-25T00:14:53.509647+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:13.485814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":305,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744794+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:14:53.744794+0000","last_peered":"2024-04-25T00:14:53.744794+0000","last_clean":"2024-04-25T00:14:53.744794+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:14:53.744794+0000","last_undegraded":"2024-04-25T00:14:53.744794+0000","last_fullsized":"2024-04-25T00:14:53.744794+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1757184,"data_stored":1118116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1175552,"data_stored":529804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1155072,"data_stored":527588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1163264,"data_stored":527680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1159168,"data_stored":528170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1159168,"data_stored":527622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1159168,"data_stored":527622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1765376,"data_stored":1119658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:14:56.592 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:14:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec8f000640 1 Processor -- start 2024-04-25T00:14:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec8f000640 1 -- start start 2024-04-25T00:14:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec8f000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fec880b4360 0x7fec880b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec8f000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec880b2b30 0x7fec880b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec8f000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec8806ef80 con 0x7fec881553e0 2024-04-25T00:14:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec8f000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fec8815b510 con 0x7fec880b4360 2024-04-25T00:14:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec8f000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fec8815b7f0 con 0x7fec880b2b30 2024-04-25T00:14:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fec881553e0 0x7fec8806e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51330/0 (socket says 172.21.15.83:51330) 2024-04-25T00:14:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87e00640 1 -- 172.21.15.83:0/196294504 learned_addr learned my addr 172.21.15.83:0/196294504 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec7f400640 1 --2- 172.21.15.83:0/196294504 >> v2:172.21.15.86:3300/0 conn(0x7fec880b4360 0x7fec880b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec8d000640 1 --2- 172.21.15.83:0/196294504 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec880b2b30 0x7fec880b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87400640 1 -- 172.21.15.83:0/196294504 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3868400519 0 0) 0x7fec8806ef80 con 0x7fec881553e0 2024-04-25T00:14:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87400640 1 -- 172.21.15.83:0/196294504 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec6c003530 con 0x7fec881553e0 2024-04-25T00:14:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87400640 1 -- 172.21.15.83:0/196294504 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1608816177 0 0) 0x7fec6c003530 con 0x7fec881553e0 2024-04-25T00:14:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87400640 1 -- 172.21.15.83:0/196294504 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec880b2b30 msgr2=0x7fec880b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87400640 1 --2- 172.21.15.83:0/196294504 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec880b2b30 0x7fec880b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87400640 1 -- 172.21.15.83:0/196294504 >> v2:172.21.15.86:3300/0 conn(0x7fec880b4360 msgr2=0x7fec880b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87400640 1 --2- 172.21.15.83:0/196294504 >> v2:172.21.15.86:3300/0 conn(0x7fec880b4360 0x7fec880b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87400640 1 -- 172.21.15.83:0/196294504 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8815bad0 con 0x7fec881553e0 2024-04-25T00:14:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87400640 1 -- 172.21.15.83:0/196294504 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fec78001680 con 0x7fec881553e0 2024-04-25T00:14:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87400640 1 -- 172.21.15.83:0/196294504 >> v1:172.21.15.83:6789/0 conn(0x7fec881553e0 legacy=0x7fec8806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87400640 1 --2- 172.21.15.83:0/196294504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c0039c0 0x7fec6c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87400640 1 --2- 172.21.15.83:0/196294504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec6c004810 0x7fec6c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.763+0000 7fec87400640 1 --2- 172.21.15.83:0/196294504 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec880b2b30 0x7fec6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec87400640 1 -- 172.21.15.83:0/196294504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec8815b510 con 0x7fec6c004810 2024-04-25T00:14:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec87400640 1 -- 172.21.15.83:0/196294504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec8815b7f0 con 0x7fec6c0039c0 2024-04-25T00:14:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec87400640 1 -- 172.21.15.83:0/196294504 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fec6c00dc60 con 0x7fec880b2b30 2024-04-25T00:14:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec7f400640 1 --2- 172.21.15.83:0/196294504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec6c004810 0x7fec6c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec8d000640 1 --2- 172.21.15.83:0/196294504 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec880b2b30 0x7fec6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec87e00640 1 --2- 172.21.15.83:0/196294504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c0039c0 0x7fec6c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec7f400640 1 -- 172.21.15.83:0/196294504 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec880b2b30 msgr2=0x7fec6c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:56.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec7f400640 1 --2- 172.21.15.83:0/196294504 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec880b2b30 0x7fec6c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec7f400640 1 -- 172.21.15.83:0/196294504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c0039c0 msgr2=0x7fec6c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:56.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec7f400640 1 --2- 172.21.15.83:0/196294504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c0039c0 0x7fec6c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec7f400640 1 -- 172.21.15.83:0/196294504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6c003530 con 0x7fec6c004810 2024-04-25T00:14:56.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec7f400640 1 --2- 172.21.15.83:0/196294504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec6c004810 0x7fec6c004c90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fec70009ae0 tx=0x7fec7001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d2c1c632186632 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:56.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec87400640 1 -- 172.21.15.83:0/196294504 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec700096b0 con 0x7fec6c004810 2024-04-25T00:14:56.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec87400640 1 -- 172.21.15.83:0/196294504 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec700a8410 con 0x7fec6c004810 2024-04-25T00:14:56.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.767+0000 7fec87400640 1 -- 172.21.15.83:0/196294504 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec700a8800 con 0x7fec6c004810 2024-04-25T00:14:56.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8f000640 1 -- 172.21.15.83:0/196294504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec6c004810 msgr2=0x7fec6c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8f000640 1 --2- 172.21.15.83:0/196294504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec6c004810 0x7fec6c004c90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fec70009ae0 tx=0x7fec7001cab0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8d000640 1 -- 172.21.15.83:0/196294504 reap_dead start 2024-04-25T00:14:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8f000640 1 -- 172.21.15.83:0/196294504 shutdown_connections 2024-04-25T00:14:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8f000640 1 -- 172.21.15.83:0/196294504 >> 172.21.15.83:0/196294504 conn(0x7fec880b1830 msgr2=0x7fec880b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8f000640 1 -- 172.21.15.83:0/196294504 shutdown_connections 2024-04-25T00:14:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8f000640 1 -- 172.21.15.83:0/196294504 wait complete. 2024-04-25T00:14:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8f000640 1 Processor -- start 2024-04-25T00:14:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8f000640 1 -- start start 2024-04-25T00:14:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c004810 0x7fec88148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8f000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec8815a9a0 0x7fec88148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8f000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec880c9990 0x7fec88151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8f000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec8806ebd0 con 0x7fec880c9990 2024-04-25T00:14:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec880b3750 con 0x7fec6c004810 2024-04-25T00:14:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8f000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fec88149890 con 0x7fec8815a9a0 2024-04-25T00:14:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec880c9990 0x7fec88151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c004810 0x7fec88148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c004810 0x7fec88148860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:48850/0 (socket says 172.21.15.83:48850) 2024-04-25T00:14:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec87e00640 1 -- 172.21.15.83:0/2324661280 learned_addr learned my addr 172.21.15.83:0/2324661280 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec7f400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec8815a9a0 0x7fec88148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8d000640 1 -- 172.21.15.83:0/2324661280 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec8815a9a0 msgr2=0x7fec88148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8d000640 1 --2- 172.21.15.83:0/2324661280 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec8815a9a0 0x7fec88148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8d000640 1 -- 172.21.15.83:0/2324661280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c004810 msgr2=0x7fec88148860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8d000640 1 --2- 172.21.15.83:0/2324661280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c004810 0x7fec88148860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8d000640 1 -- 172.21.15.83:0/2324661280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec88149b70 con 0x7fec880c9990 2024-04-25T00:14:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.771+0000 7fec8d000640 1 --2- 172.21.15.83:0/2324661280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec880c9990 0x7fec88151f40 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fec88068b90 tx=0x7fec800115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.775+0000 7fec84c00640 1 -- 172.21.15.83:0/2324661280 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80011e10 con 0x7fec880c9990 2024-04-25T00:14:56.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.775+0000 7fec84c00640 1 -- 172.21.15.83:0/2324661280 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8001b070 con 0x7fec880c9990 2024-04-25T00:14:56.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.775+0000 7fec84c00640 1 -- 172.21.15.83:0/2324661280 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec800039a0 con 0x7fec880c9990 2024-04-25T00:14:56.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.775+0000 7fec8f000640 1 -- 172.21.15.83:0/2324661280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec880c8eb0 con 0x7fec880c9990 2024-04-25T00:14:56.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.775+0000 7fec8f000640 1 -- 172.21.15.83:0/2324661280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec880c9470 con 0x7fec880c9990 2024-04-25T00:14:56.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.775+0000 7fec8f000640 1 -- 172.21.15.83:0/2324661280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec880737f0 con 0x7fec880c9990 2024-04-25T00:14:56.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.775+0000 7fec84c00640 1 -- 172.21.15.83:0/2324661280 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fec80003b60 con 0x7fec880c9990 2024-04-25T00:14:56.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.775+0000 7fec84c00640 1 --2- 172.21.15.83:0/2324661280 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec640678c0 0x7fec64069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:56.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.775+0000 7fec84c00640 1 -- 172.21.15.83:0/2324661280 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fec80086520 con 0x7fec880c9990 2024-04-25T00:14:56.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.775+0000 7fec87e00640 1 --2- 172.21.15.83:0/2324661280 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec640678c0 0x7fec64069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:56.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.775+0000 7fec87e00640 1 --2- 172.21.15.83:0/2324661280 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec640678c0 0x7fec64069d80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fec78001ec0 tx=0x7fec78007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:56.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.779+0000 7fec84c00640 1 -- 172.21.15.83:0/2324661280 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fec80025020 con 0x7fec880c9990 2024-04-25T00:14:56.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.931+0000 7fec8f000640 1 -- 172.21.15.83:0/2324661280 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fec8804f5f0 con 0x7fec640678c0 2024-04-25T00:14:56.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.939+0000 7fec84c00640 1 -- 172.21.15.83:0/2324661280 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252084 (secure 0 0 0) 0x7fec8804f5f0 con 0x7fec640678c0 2024-04-25T00:14:56.946 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:14:56.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.943+0000 7fec8f000640 1 -- 172.21.15.83:0/2324661280 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec640678c0 msgr2=0x7fec64069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.943+0000 7fec8f000640 1 --2- 172.21.15.83:0/2324661280 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec640678c0 0x7fec64069d80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fec78001ec0 tx=0x7fec78007040 comp rx=0 tx=0).stop 2024-04-25T00:14:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.943+0000 7fec8f000640 1 -- 172.21.15.83:0/2324661280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec880c9990 msgr2=0x7fec88151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.943+0000 7fec8f000640 1 --2- 172.21.15.83:0/2324661280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec880c9990 0x7fec88151f40 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fec88068b90 tx=0x7fec800115a0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.943+0000 7fec8f000640 1 -- 172.21.15.83:0/2324661280 shutdown_connections 2024-04-25T00:14:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.943+0000 7fec8f000640 1 --2- 172.21.15.83:0/2324661280 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fec640678c0 0x7fec64069d80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.943+0000 7fec8f000640 1 --2- 172.21.15.83:0/2324661280 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec8815a9a0 0x7fec88148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.943+0000 7fec8f000640 1 --2- 172.21.15.83:0/2324661280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec880c9990 0x7fec88151f40 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.943+0000 7fec8f000640 1 --2- 172.21.15.83:0/2324661280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c004810 0x7fec88148860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.943+0000 7fec8f000640 1 -- 172.21.15.83:0/2324661280 >> 172.21.15.83:0/2324661280 conn(0x7fec880b1830 msgr2=0x7fec88071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.943+0000 7fec8f000640 1 -- 172.21.15.83:0/2324661280 shutdown_connections 2024-04-25T00:14:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:56.943+0000 7fec8f000640 1 -- 172.21.15.83:0/2324661280 wait complete. 2024-04-25T00:14:56.965 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-04-25T00:14:56.487809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259716,"kb_used_data":10248,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715004,"statfs":{"total":773094113280,"available":772828164096,"internally_reserved":0,"allocated":10493952,"data_stored":5406260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.996399"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508445+0000","last_change":"2024-04-25T00:14:52.517972+0000","last_active":"2024-04-25T00:14:53.508445+0000","last_peered":"2024-04-25T00:14:53.508445+0000","last_clean":"2024-04-25T00:14:53.508445+0000","last_became_active":"2024-04-25T00:14:52.517694+0000","last_became_peered":"2024-04-25T00:14:52.517694+0000","last_unstale":"2024-04-25T00:14:53.508445+0000","last_undegraded":"2024-04-25T00:14:53.508445+0000","last_fullsized":"2024-04-25T00:14:53.508445+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:24.016100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509653+0000","last_change":"2024-04-25T00:14:52.521208+0000","last_active":"2024-04-25T00:14:53.509653+0000","last_peered":"2024-04-25T00:14:53.509653+0000","last_clean":"2024-04-25T00:14:53.509653+0000","last_became_active":"2024-04-25T00:14:52.520679+0000","last_became_peered":"2024-04-25T00:14:52.520679+0000","last_unstale":"2024-04-25T00:14:53.509653+0000","last_undegraded":"2024-04-25T00:14:53.509653+0000","last_fullsized":"2024-04-25T00:14:53.509653+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:22.938318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.507928+0000","last_change":"2024-04-25T00:14:52.534061+0000","last_active":"2024-04-25T00:14:53.507928+0000","last_peered":"2024-04-25T00:14:53.507928+0000","last_clean":"2024-04-25T00:14:53.507928+0000","last_became_active":"2024-04-25T00:14:52.532699+0000","last_became_peered":"2024-04-25T00:14:52.532699+0000","last_unstale":"2024-04-25T00:14:53.507928+0000","last_undegraded":"2024-04-25T00:14:53.507928+0000","last_fullsized":"2024-04-25T00:14:53.507928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:45.529300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744560+0000","last_change":"2024-04-25T00:14:52.546549+0000","last_active":"2024-04-25T00:14:53.744560+0000","last_peered":"2024-04-25T00:14:53.744560+0000","last_clean":"2024-04-25T00:14:53.744560+0000","last_became_active":"2024-04-25T00:14:52.545833+0000","last_became_peered":"2024-04-25T00:14:52.545833+0000","last_unstale":"2024-04-25T00:14:53.744560+0000","last_undegraded":"2024-04-25T00:14:53.744560+0000","last_fullsized":"2024-04-25T00:14:53.744560+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:27.677704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745186+0000","last_change":"2024-04-25T00:14:52.530494+0000","last_active":"2024-04-25T00:14:53.745186+0000","last_peered":"2024-04-25T00:14:53.745186+0000","last_clean":"2024-04-25T00:14:53.745186+0000","last_became_active":"2024-04-25T00:14:52.530254+0000","last_became_peered":"2024-04-25T00:14:52.530254+0000","last_unstale":"2024-04-25T00:14:53.745186+0000","last_undegraded":"2024-04-25T00:14:53.745186+0000","last_fullsized":"2024-04-25T00:14:53.745186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:14.836642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509233+0000","last_change":"2024-04-25T00:14:52.522558+0000","last_active":"2024-04-25T00:14:53.509233+0000","last_peered":"2024-04-25T00:14:53.509233+0000","last_clean":"2024-04-25T00:14:53.509233+0000","last_became_active":"2024-04-25T00:14:52.522110+0000","last_became_peered":"2024-04-25T00:14:52.522110+0000","last_unstale":"2024-04-25T00:14:53.509233+0000","last_undegraded":"2024-04-25T00:14:53.509233+0000","last_fullsized":"2024-04-25T00:14:53.509233+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:56.739421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509094+0000","last_change":"2024-04-25T00:14:52.527261+0000","last_active":"2024-04-25T00:14:53.509094+0000","last_peered":"2024-04-25T00:14:53.509094+0000","last_clean":"2024-04-25T00:14:53.509094+0000","last_became_active":"2024-04-25T00:14:52.526945+0000","last_became_peered":"2024-04-25T00:14:52.526945+0000","last_unstale":"2024-04-25T00:14:53.509094+0000","last_undegraded":"2024-04-25T00:14:53.509094+0000","last_fullsized":"2024-04-25T00:14:53.509094+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:07.988481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509107+0000","last_change":"2024-04-25T00:14:52.533974+0000","last_active":"2024-04-25T00:14:53.509107+0000","last_peered":"2024-04-25T00:14:53.509107+0000","last_clean":"2024-04-25T00:14:53.509107+0000","last_became_active":"2024-04-25T00:14:52.533702+0000","last_became_peered":"2024-04-25T00:14:52.533702+0000","last_unstale":"2024-04-25T00:14:53.509107+0000","last_undegraded":"2024-04-25T00:14:53.509107+0000","last_fullsized":"2024-04-25T00:14:53.509107+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:21.700580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509803+0000","last_change":"2024-04-25T00:14:52.531007+0000","last_active":"2024-04-25T00:14:53.509803+0000","last_peered":"2024-04-25T00:14:53.509803+0000","last_clean":"2024-04-25T00:14:53.509803+0000","last_became_active":"2024-04-25T00:14:52.530740+0000","last_became_peered":"2024-04-25T00:14:52.530740+0000","last_unstale":"2024-04-25T00:14:53.509803+0000","last_undegraded":"2024-04-25T00:14:53.509803+0000","last_fullsized":"2024-04-25T00:14:53.509803+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:59.770333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745323+0000","last_change":"2024-04-25T00:14:52.521577+0000","last_active":"2024-04-25T00:14:53.745323+0000","last_peered":"2024-04-25T00:14:53.745323+0000","last_clean":"2024-04-25T00:14:53.745323+0000","last_became_active":"2024-04-25T00:14:52.521295+0000","last_became_peered":"2024-04-25T00:14:52.521295+0000","last_unstale":"2024-04-25T00:14:53.745323+0000","last_undegraded":"2024-04-25T00:14:53.745323+0000","last_fullsized":"2024-04-25T00:14:53.745323+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:41.604306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531497+0000","last_change":"2024-04-25T00:14:52.538705+0000","last_active":"2024-04-25T00:14:53.531497+0000","last_peered":"2024-04-25T00:14:53.531497+0000","last_clean":"2024-04-25T00:14:53.531497+0000","last_became_active":"2024-04-25T00:14:52.538365+0000","last_became_peered":"2024-04-25T00:14:52.538365+0000","last_unstale":"2024-04-25T00:14:53.531497+0000","last_undegraded":"2024-04-25T00:14:53.531497+0000","last_fullsized":"2024-04-25T00:14:53.531497+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:22.938318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745099+0000","last_change":"2024-04-25T00:14:52.535903+0000","last_active":"2024-04-25T00:14:53.745099+0000","last_peered":"2024-04-25T00:14:53.745099+0000","last_clean":"2024-04-25T00:14:53.745099+0000","last_became_active":"2024-04-25T00:14:52.535460+0000","last_became_peered":"2024-04-25T00:14:52.535460+0000","last_unstale":"2024-04-25T00:14:53.745099+0000","last_undegraded":"2024-04-25T00:14:53.745099+0000","last_fullsized":"2024-04-25T00:14:53.745099+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:22.645056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744534+0000","last_change":"2024-04-25T00:14:52.537936+0000","last_active":"2024-04-25T00:14:53.744534+0000","last_peered":"2024-04-25T00:14:53.744534+0000","last_clean":"2024-04-25T00:14:53.744534+0000","last_became_active":"2024-04-25T00:14:52.537653+0000","last_became_peered":"2024-04-25T00:14:52.537653+0000","last_unstale":"2024-04-25T00:14:53.744534+0000","last_undegraded":"2024-04-25T00:14:53.744534+0000","last_fullsized":"2024-04-25T00:14:53.744534+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:44.818488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745653+0000","last_change":"2024-04-25T00:14:52.522787+0000","last_active":"2024-04-25T00:14:53.745653+0000","last_peered":"2024-04-25T00:14:53.745653+0000","last_clean":"2024-04-25T00:14:53.745653+0000","last_became_active":"2024-04-25T00:14:52.522480+0000","last_became_peered":"2024-04-25T00:14:52.522480+0000","last_unstale":"2024-04-25T00:14:53.745653+0000","last_undegraded":"2024-04-25T00:14:53.745653+0000","last_fullsized":"2024-04-25T00:14:53.745653+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:16.493058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745019+0000","last_change":"2024-04-25T00:14:52.540161+0000","last_active":"2024-04-25T00:14:53.745019+0000","last_peered":"2024-04-25T00:14:53.745019+0000","last_clean":"2024-04-25T00:14:53.745019+0000","last_became_active":"2024-04-25T00:14:52.539070+0000","last_became_peered":"2024-04-25T00:14:52.539070+0000","last_unstale":"2024-04-25T00:14:53.745019+0000","last_undegraded":"2024-04-25T00:14:53.745019+0000","last_fullsized":"2024-04-25T00:14:53.745019+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:28.821898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744868+0000","last_change":"2024-04-25T00:14:52.534154+0000","last_active":"2024-04-25T00:14:53.744868+0000","last_peered":"2024-04-25T00:14:53.744868+0000","last_clean":"2024-04-25T00:14:53.744868+0000","last_became_active":"2024-04-25T00:14:52.532048+0000","last_became_peered":"2024-04-25T00:14:52.532048+0000","last_unstale":"2024-04-25T00:14:53.744868+0000","last_undegraded":"2024-04-25T00:14:53.744868+0000","last_fullsized":"2024-04-25T00:14:53.744868+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:45:16.996586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508591+0000","last_change":"2024-04-25T00:14:52.522849+0000","last_active":"2024-04-25T00:14:53.508591+0000","last_peered":"2024-04-25T00:14:53.508591+0000","last_clean":"2024-04-25T00:14:53.508591+0000","last_became_active":"2024-04-25T00:14:52.522275+0000","last_became_peered":"2024-04-25T00:14:52.522275+0000","last_unstale":"2024-04-25T00:14:53.508591+0000","last_undegraded":"2024-04-25T00:14:53.508591+0000","last_fullsized":"2024-04-25T00:14:53.508591+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:00.743577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509818+0000","last_change":"2024-04-25T00:14:52.522322+0000","last_active":"2024-04-25T00:14:53.509818+0000","last_peered":"2024-04-25T00:14:53.509818+0000","last_clean":"2024-04-25T00:14:53.509818+0000","last_became_active":"2024-04-25T00:14:52.522035+0000","last_became_peered":"2024-04-25T00:14:52.522035+0000","last_unstale":"2024-04-25T00:14:53.509818+0000","last_undegraded":"2024-04-25T00:14:53.509818+0000","last_fullsized":"2024-04-25T00:14:53.509818+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:46.512512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508159+0000","last_change":"2024-04-25T00:14:52.518924+0000","last_active":"2024-04-25T00:14:53.508159+0000","last_peered":"2024-04-25T00:14:53.508159+0000","last_clean":"2024-04-25T00:14:53.508159+0000","last_became_active":"2024-04-25T00:14:52.518629+0000","last_became_peered":"2024-04-25T00:14:52.518629+0000","last_unstale":"2024-04-25T00:14:53.508159+0000","last_undegraded":"2024-04-25T00:14:53.508159+0000","last_fullsized":"2024-04-25T00:14:53.508159+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:28.550925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745189+0000","last_change":"2024-04-25T00:14:52.542989+0000","last_active":"2024-04-25T00:14:53.745189+0000","last_peered":"2024-04-25T00:14:53.745189+0000","last_clean":"2024-04-25T00:14:53.745189+0000","last_became_active":"2024-04-25T00:14:52.542650+0000","last_became_peered":"2024-04-25T00:14:52.542650+0000","last_unstale":"2024-04-25T00:14:53.745189+0000","last_undegraded":"2024-04-25T00:14:53.745189+0000","last_fullsized":"2024-04-25T00:14:53.745189+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:38.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509527+0000","last_change":"2024-04-25T00:14:52.537817+0000","last_active":"2024-04-25T00:14:53.509527+0000","last_peered":"2024-04-25T00:14:53.509527+0000","last_clean":"2024-04-25T00:14:53.509527+0000","last_became_active":"2024-04-25T00:14:52.537567+0000","last_became_peered":"2024-04-25T00:14:52.537567+0000","last_unstale":"2024-04-25T00:14:53.509527+0000","last_undegraded":"2024-04-25T00:14:53.509527+0000","last_fullsized":"2024-04-25T00:14:53.509527+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:33.797911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745085+0000","last_change":"2024-04-25T00:14:52.529301+0000","last_active":"2024-04-25T00:14:53.745085+0000","last_peered":"2024-04-25T00:14:53.745085+0000","last_clean":"2024-04-25T00:14:53.745085+0000","last_became_active":"2024-04-25T00:14:52.528229+0000","last_became_peered":"2024-04-25T00:14:52.528229+0000","last_unstale":"2024-04-25T00:14:53.745085+0000","last_undegraded":"2024-04-25T00:14:53.745085+0000","last_fullsized":"2024-04-25T00:14:53.745085+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:56.739421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508224+0000","last_change":"2024-04-25T00:14:52.543215+0000","last_active":"2024-04-25T00:14:53.508224+0000","last_peered":"2024-04-25T00:14:53.508224+0000","last_clean":"2024-04-25T00:14:53.508224+0000","last_became_active":"2024-04-25T00:14:52.540249+0000","last_became_peered":"2024-04-25T00:14:52.540249+0000","last_unstale":"2024-04-25T00:14:53.508224+0000","last_undegraded":"2024-04-25T00:14:53.508224+0000","last_fullsized":"2024-04-25T00:14:53.508224+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:30.972848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744777+0000","last_change":"2024-04-25T00:14:52.515994+0000","last_active":"2024-04-25T00:14:53.744777+0000","last_peered":"2024-04-25T00:14:53.744777+0000","last_clean":"2024-04-25T00:14:53.744777+0000","last_became_active":"2024-04-25T00:14:52.515718+0000","last_became_peered":"2024-04-25T00:14:52.515718+0000","last_unstale":"2024-04-25T00:14:53.744777+0000","last_undegraded":"2024-04-25T00:14:53.744777+0000","last_fullsized":"2024-04-25T00:14:53.744777+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:07.988481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745111+0000","last_change":"2024-04-25T00:14:52.548896+0000","last_active":"2024-04-25T00:14:53.745111+0000","last_peered":"2024-04-25T00:14:53.745111+0000","last_clean":"2024-04-25T00:14:53.745111+0000","last_became_active":"2024-04-25T00:14:52.547845+0000","last_became_peered":"2024-04-25T00:14:52.547845+0000","last_unstale":"2024-04-25T00:14:53.745111+0000","last_undegraded":"2024-04-25T00:14:53.745111+0000","last_fullsized":"2024-04-25T00:14:53.745111+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:56.739421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745097+0000","last_change":"2024-04-25T00:14:52.536499+0000","last_active":"2024-04-25T00:14:53.745097+0000","last_peered":"2024-04-25T00:14:53.745097+0000","last_clean":"2024-04-25T00:14:53.745097+0000","last_became_active":"2024-04-25T00:14:52.536125+0000","last_became_peered":"2024-04-25T00:14:52.536125+0000","last_unstale":"2024-04-25T00:14:53.745097+0000","last_undegraded":"2024-04-25T00:14:53.745097+0000","last_fullsized":"2024-04-25T00:14:53.745097+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:07.988481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531429+0000","last_change":"2024-04-25T00:14:52.544966+0000","last_active":"2024-04-25T00:14:53.531429+0000","last_peered":"2024-04-25T00:14:53.531429+0000","last_clean":"2024-04-25T00:14:53.531429+0000","last_became_active":"2024-04-25T00:14:52.544686+0000","last_became_peered":"2024-04-25T00:14:52.544686+0000","last_unstale":"2024-04-25T00:14:53.531429+0000","last_undegraded":"2024-04-25T00:14:53.531429+0000","last_fullsized":"2024-04-25T00:14:53.531429+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:41.604306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509631+0000","last_change":"2024-04-25T00:14:52.529806+0000","last_active":"2024-04-25T00:14:53.509631+0000","last_peered":"2024-04-25T00:14:53.509631+0000","last_clean":"2024-04-25T00:14:53.509631+0000","last_became_active":"2024-04-25T00:14:52.529524+0000","last_became_peered":"2024-04-25T00:14:52.529524+0000","last_unstale":"2024-04-25T00:14:53.509631+0000","last_undegraded":"2024-04-25T00:14:53.509631+0000","last_fullsized":"2024-04-25T00:14:53.509631+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:41.604306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509627+0000","last_change":"2024-04-25T00:14:52.544251+0000","last_active":"2024-04-25T00:14:53.509627+0000","last_peered":"2024-04-25T00:14:53.509627+0000","last_clean":"2024-04-25T00:14:53.509627+0000","last_became_active":"2024-04-25T00:14:52.543984+0000","last_became_peered":"2024-04-25T00:14:52.543984+0000","last_unstale":"2024-04-25T00:14:53.509627+0000","last_undegraded":"2024-04-25T00:14:53.509627+0000","last_fullsized":"2024-04-25T00:14:53.509627+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:15.967409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744903+0000","last_change":"2024-04-25T00:14:52.543402+0000","last_active":"2024-04-25T00:14:53.744903+0000","last_peered":"2024-04-25T00:14:53.744903+0000","last_clean":"2024-04-25T00:14:53.744903+0000","last_became_active":"2024-04-25T00:14:52.543128+0000","last_became_peered":"2024-04-25T00:14:52.543128+0000","last_unstale":"2024-04-25T00:14:53.744903+0000","last_undegraded":"2024-04-25T00:14:53.744903+0000","last_fullsized":"2024-04-25T00:14:53.744903+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:04.926801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509112+0000","last_change":"2024-04-25T00:14:52.517426+0000","last_active":"2024-04-25T00:14:53.509112+0000","last_peered":"2024-04-25T00:14:53.509112+0000","last_clean":"2024-04-25T00:14:53.509112+0000","last_became_active":"2024-04-25T00:14:52.517104+0000","last_became_peered":"2024-04-25T00:14:52.517104+0000","last_unstale":"2024-04-25T00:14:53.509112+0000","last_undegraded":"2024-04-25T00:14:53.509112+0000","last_fullsized":"2024-04-25T00:14:53.509112+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:22.645056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509164+0000","last_change":"2024-04-25T00:14:52.546048+0000","last_active":"2024-04-25T00:14:53.509164+0000","last_peered":"2024-04-25T00:14:53.509164+0000","last_clean":"2024-04-25T00:14:53.509164+0000","last_became_active":"2024-04-25T00:14:52.545768+0000","last_became_peered":"2024-04-25T00:14:52.545768+0000","last_unstale":"2024-04-25T00:14:53.509164+0000","last_undegraded":"2024-04-25T00:14:53.509164+0000","last_fullsized":"2024-04-25T00:14:53.509164+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:43.680130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508112+0000","last_change":"2024-04-25T00:14:52.515584+0000","last_active":"2024-04-25T00:14:53.508112+0000","last_peered":"2024-04-25T00:14:53.508112+0000","last_clean":"2024-04-25T00:14:53.508112+0000","last_became_active":"2024-04-25T00:14:52.515333+0000","last_became_peered":"2024-04-25T00:14:52.515333+0000","last_unstale":"2024-04-25T00:14:53.508112+0000","last_undegraded":"2024-04-25T00:14:53.508112+0000","last_fullsized":"2024-04-25T00:14:53.508112+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:20:40.553267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"92'4","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744707+0000","last_change":"2024-04-25T00:14:51.512978+0000","last_active":"2024-04-25T00:14:53.744707+0000","last_peered":"2024-04-25T00:14:53.744707+0000","last_clean":"2024-04-25T00:14:53.744707+0000","last_became_active":"2024-04-25T00:14:51.512717+0000","last_became_peered":"2024-04-25T00:14:51.512717+0000","last_unstale":"2024-04-25T00:14:53.744707+0000","last_undegraded":"2024-04-25T00:14:53.744707+0000","last_fullsized":"2024-04-25T00:14:53.744707+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:21.471082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509182+0000","last_change":"2024-04-25T00:14:52.533579+0000","last_active":"2024-04-25T00:14:53.509182+0000","last_peered":"2024-04-25T00:14:53.509182+0000","last_clean":"2024-04-25T00:14:53.509182+0000","last_became_active":"2024-04-25T00:14:52.533208+0000","last_became_peered":"2024-04-25T00:14:52.533208+0000","last_unstale":"2024-04-25T00:14:53.509182+0000","last_undegraded":"2024-04-25T00:14:53.509182+0000","last_fullsized":"2024-04-25T00:14:53.509182+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:36.353389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509621+0000","last_change":"2024-04-25T00:14:51.513807+0000","last_active":"2024-04-25T00:14:53.509621+0000","last_peered":"2024-04-25T00:14:53.509621+0000","last_clean":"2024-04-25T00:14:53.509621+0000","last_became_active":"2024-04-25T00:14:51.513524+0000","last_became_peered":"2024-04-25T00:14:51.513524+0000","last_unstale":"2024-04-25T00:14:53.509621+0000","last_undegraded":"2024-04-25T00:14:53.509621+0000","last_fullsized":"2024-04-25T00:14:53.509621+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:13.698946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744990+0000","last_change":"2024-04-25T00:14:52.528252+0000","last_active":"2024-04-25T00:14:53.744990+0000","last_peered":"2024-04-25T00:14:53.744990+0000","last_clean":"2024-04-25T00:14:53.744990+0000","last_became_active":"2024-04-25T00:14:52.527839+0000","last_became_peered":"2024-04-25T00:14:52.527839+0000","last_unstale":"2024-04-25T00:14:53.744990+0000","last_undegraded":"2024-04-25T00:14:53.744990+0000","last_fullsized":"2024-04-25T00:14:53.744990+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:00.073031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744802+0000","last_change":"2024-04-25T00:14:51.519847+0000","last_active":"2024-04-25T00:14:53.744802+0000","last_peered":"2024-04-25T00:14:53.744802+0000","last_clean":"2024-04-25T00:14:53.744802+0000","last_became_active":"2024-04-25T00:14:51.518123+0000","last_became_peered":"2024-04-25T00:14:51.518123+0000","last_unstale":"2024-04-25T00:14:53.744802+0000","last_undegraded":"2024-04-25T00:14:53.744802+0000","last_fullsized":"2024-04-25T00:14:53.744802+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:13.128042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531196+0000","last_change":"2024-04-25T00:14:52.538781+0000","last_active":"2024-04-25T00:14:53.531196+0000","last_peered":"2024-04-25T00:14:53.531196+0000","last_clean":"2024-04-25T00:14:53.531196+0000","last_became_active":"2024-04-25T00:14:52.538202+0000","last_became_peered":"2024-04-25T00:14:52.538202+0000","last_unstale":"2024-04-25T00:14:53.531196+0000","last_undegraded":"2024-04-25T00:14:53.531196+0000","last_fullsized":"2024-04-25T00:14:53.531196+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:38.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745185+0000","last_change":"2024-04-25T00:14:51.516911+0000","last_active":"2024-04-25T00:14:53.745185+0000","last_peered":"2024-04-25T00:14:53.745185+0000","last_clean":"2024-04-25T00:14:53.745185+0000","last_became_active":"2024-04-25T00:14:51.516675+0000","last_became_peered":"2024-04-25T00:14:51.516675+0000","last_unstale":"2024-04-25T00:14:53.745185+0000","last_undegraded":"2024-04-25T00:14:53.745185+0000","last_fullsized":"2024-04-25T00:14:53.745185+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:13.128042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744692+0000","last_change":"2024-04-25T00:14:52.521917+0000","last_active":"2024-04-25T00:14:53.744692+0000","last_peered":"2024-04-25T00:14:53.744692+0000","last_clean":"2024-04-25T00:14:53.744692+0000","last_became_active":"2024-04-25T00:14:52.521499+0000","last_became_peered":"2024-04-25T00:14:52.521499+0000","last_unstale":"2024-04-25T00:14:53.744692+0000","last_undegraded":"2024-04-25T00:14:53.744692+0000","last_fullsized":"2024-04-25T00:14:53.744692+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:07.461779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745364+0000","last_change":"2024-04-25T00:14:51.510850+0000","last_active":"2024-04-25T00:14:53.745364+0000","last_peered":"2024-04-25T00:14:53.745364+0000","last_clean":"2024-04-25T00:14:53.745364+0000","last_became_active":"2024-04-25T00:14:51.510479+0000","last_became_peered":"2024-04-25T00:14:51.510479+0000","last_unstale":"2024-04-25T00:14:53.745364+0000","last_undegraded":"2024-04-25T00:14:53.745364+0000","last_fullsized":"2024-04-25T00:14:53.745364+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:13.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508456+0000","last_change":"2024-04-25T00:14:52.517849+0000","last_active":"2024-04-25T00:14:53.508456+0000","last_peered":"2024-04-25T00:14:53.508456+0000","last_clean":"2024-04-25T00:14:53.508456+0000","last_became_active":"2024-04-25T00:14:52.517593+0000","last_became_peered":"2024-04-25T00:14:52.517593+0000","last_unstale":"2024-04-25T00:14:53.508456+0000","last_undegraded":"2024-04-25T00:14:53.508456+0000","last_fullsized":"2024-04-25T00:14:53.508456+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:55.977402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531041+0000","last_change":"2024-04-25T00:14:51.516511+0000","last_active":"2024-04-25T00:14:53.531041+0000","last_peered":"2024-04-25T00:14:53.531041+0000","last_clean":"2024-04-25T00:14:53.531041+0000","last_became_active":"2024-04-25T00:14:51.515960+0000","last_became_peered":"2024-04-25T00:14:51.515960+0000","last_unstale":"2024-04-25T00:14:53.531041+0000","last_undegraded":"2024-04-25T00:14:53.531041+0000","last_fullsized":"2024-04-25T00:14:53.531041+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:01.079041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509878+0000","last_change":"2024-04-25T00:14:52.526791+0000","last_active":"2024-04-25T00:14:53.509878+0000","last_peered":"2024-04-25T00:14:53.509878+0000","last_clean":"2024-04-25T00:14:53.509878+0000","last_became_active":"2024-04-25T00:14:52.525048+0000","last_became_peered":"2024-04-25T00:14:52.525048+0000","last_unstale":"2024-04-25T00:14:53.509878+0000","last_undegraded":"2024-04-25T00:14:53.509878+0000","last_fullsized":"2024-04-25T00:14:53.509878+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:22.474854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744653+0000","last_change":"2024-04-25T00:14:51.521606+0000","last_active":"2024-04-25T00:14:53.744653+0000","last_peered":"2024-04-25T00:14:53.744653+0000","last_clean":"2024-04-25T00:14:53.744653+0000","last_became_active":"2024-04-25T00:14:51.520796+0000","last_became_peered":"2024-04-25T00:14:51.520796+0000","last_unstale":"2024-04-25T00:14:53.744653+0000","last_undegraded":"2024-04-25T00:14:53.744653+0000","last_fullsized":"2024-04-25T00:14:53.744653+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:46.406675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744735+0000","last_change":"2024-04-25T00:14:52.515560+0000","last_active":"2024-04-25T00:14:53.744735+0000","last_peered":"2024-04-25T00:14:53.744735+0000","last_clean":"2024-04-25T00:14:53.744735+0000","last_became_active":"2024-04-25T00:14:52.515279+0000","last_became_peered":"2024-04-25T00:14:52.515279+0000","last_unstale":"2024-04-25T00:14:53.744735+0000","last_undegraded":"2024-04-25T00:14:53.744735+0000","last_fullsized":"2024-04-25T00:14:53.744735+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:12.955085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744279+0000","last_change":"2024-04-25T00:14:51.516515+0000","last_active":"2024-04-25T00:14:53.744279+0000","last_peered":"2024-04-25T00:14:53.744279+0000","last_clean":"2024-04-25T00:14:53.744279+0000","last_became_active":"2024-04-25T00:14:51.516270+0000","last_became_peered":"2024-04-25T00:14:51.516270+0000","last_unstale":"2024-04-25T00:14:53.744279+0000","last_undegraded":"2024-04-25T00:14:53.744279+0000","last_fullsized":"2024-04-25T00:14:53.744279+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:58.766561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.507975+0000","last_change":"2024-04-25T00:14:52.514745+0000","last_active":"2024-04-25T00:14:53.507975+0000","last_peered":"2024-04-25T00:14:53.507975+0000","last_clean":"2024-04-25T00:14:53.507975+0000","last_became_active":"2024-04-25T00:14:52.514446+0000","last_became_peered":"2024-04-25T00:14:52.514446+0000","last_unstale":"2024-04-25T00:14:53.507975+0000","last_undegraded":"2024-04-25T00:14:53.507975+0000","last_fullsized":"2024-04-25T00:14:53.507975+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:57.564616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745306+0000","last_change":"2024-04-25T00:14:51.515705+0000","last_active":"2024-04-25T00:14:53.745306+0000","last_peered":"2024-04-25T00:14:53.745306+0000","last_clean":"2024-04-25T00:14:53.745306+0000","last_became_active":"2024-04-25T00:14:51.515448+0000","last_became_peered":"2024-04-25T00:14:51.515448+0000","last_unstale":"2024-04-25T00:14:53.745306+0000","last_undegraded":"2024-04-25T00:14:53.745306+0000","last_fullsized":"2024-04-25T00:14:53.745306+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:12.379516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744931+0000","last_change":"2024-04-25T00:14:52.534318+0000","last_active":"2024-04-25T00:14:53.744931+0000","last_peered":"2024-04-25T00:14:53.744931+0000","last_clean":"2024-04-25T00:14:53.744931+0000","last_became_active":"2024-04-25T00:14:52.532716+0000","last_became_peered":"2024-04-25T00:14:52.532716+0000","last_unstale":"2024-04-25T00:14:53.744931+0000","last_undegraded":"2024-04-25T00:14:53.744931+0000","last_fullsized":"2024-04-25T00:14:53.744931+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:22.645056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508339+0000","last_change":"2024-04-25T00:14:51.523093+0000","last_active":"2024-04-25T00:14:53.508339+0000","last_peered":"2024-04-25T00:14:53.508339+0000","last_clean":"2024-04-25T00:14:53.508339+0000","last_became_active":"2024-04-25T00:14:51.522776+0000","last_became_peered":"2024-04-25T00:14:51.522776+0000","last_unstale":"2024-04-25T00:14:53.508339+0000","last_undegraded":"2024-04-25T00:14:53.508339+0000","last_fullsized":"2024-04-25T00:14:53.508339+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:26.673932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509338+0000","last_change":"2024-04-25T00:14:52.533353+0000","last_active":"2024-04-25T00:14:53.509338+0000","last_peered":"2024-04-25T00:14:53.509338+0000","last_clean":"2024-04-25T00:14:53.509338+0000","last_became_active":"2024-04-25T00:14:52.533086+0000","last_became_peered":"2024-04-25T00:14:52.533086+0000","last_unstale":"2024-04-25T00:14:53.509338+0000","last_undegraded":"2024-04-25T00:14:53.509338+0000","last_fullsized":"2024-04-25T00:14:53.509338+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:36.422235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"92'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745399+0000","last_change":"2024-04-25T00:14:51.521745+0000","last_active":"2024-04-25T00:14:53.745399+0000","last_peered":"2024-04-25T00:14:53.745399+0000","last_clean":"2024-04-25T00:14:53.745399+0000","last_became_active":"2024-04-25T00:14:51.521422+0000","last_became_peered":"2024-04-25T00:14:51.521422+0000","last_unstale":"2024-04-25T00:14:53.745399+0000","last_undegraded":"2024-04-25T00:14:53.745399+0000","last_fullsized":"2024-04-25T00:14:53.745399+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:55.735649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745336+0000","last_change":"2024-04-25T00:14:52.534542+0000","last_active":"2024-04-25T00:14:53.745336+0000","last_peered":"2024-04-25T00:14:53.745336+0000","last_clean":"2024-04-25T00:14:53.745336+0000","last_became_active":"2024-04-25T00:14:52.534247+0000","last_became_peered":"2024-04-25T00:14:52.534247+0000","last_unstale":"2024-04-25T00:14:53.745336+0000","last_undegraded":"2024-04-25T00:14:53.745336+0000","last_fullsized":"2024-04-25T00:14:53.745336+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:22.938318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"92'6","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509744+0000","last_change":"2024-04-25T00:14:51.521244+0000","last_active":"2024-04-25T00:14:53.509744+0000","last_peered":"2024-04-25T00:14:53.509744+0000","last_clean":"2024-04-25T00:14:53.509744+0000","last_became_active":"2024-04-25T00:14:51.520964+0000","last_became_peered":"2024-04-25T00:14:51.520964+0000","last_unstale":"2024-04-25T00:14:53.509744+0000","last_undegraded":"2024-04-25T00:14:53.509744+0000","last_fullsized":"2024-04-25T00:14:53.509744+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:46.406675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531245+0000","last_change":"2024-04-25T00:14:52.532881+0000","last_active":"2024-04-25T00:14:53.531245+0000","last_peered":"2024-04-25T00:14:53.531245+0000","last_clean":"2024-04-25T00:14:53.531245+0000","last_became_active":"2024-04-25T00:14:52.532443+0000","last_became_peered":"2024-04-25T00:14:52.532443+0000","last_unstale":"2024-04-25T00:14:53.531245+0000","last_undegraded":"2024-04-25T00:14:53.531245+0000","last_fullsized":"2024-04-25T00:14:53.531245+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:14.836642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745275+0000","last_change":"2024-04-25T00:14:51.516174+0000","last_active":"2024-04-25T00:14:53.745275+0000","last_peered":"2024-04-25T00:14:53.745275+0000","last_clean":"2024-04-25T00:14:53.745275+0000","last_became_active":"2024-04-25T00:14:51.515922+0000","last_became_peered":"2024-04-25T00:14:51.515922+0000","last_unstale":"2024-04-25T00:14:53.745275+0000","last_undegraded":"2024-04-25T00:14:53.745275+0000","last_fullsized":"2024-04-25T00:14:53.745275+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:41.712373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509603+0000","last_change":"2024-04-25T00:14:52.532293+0000","last_active":"2024-04-25T00:14:53.509603+0000","last_peered":"2024-04-25T00:14:53.509603+0000","last_clean":"2024-04-25T00:14:53.509603+0000","last_became_active":"2024-04-25T00:14:52.531899+0000","last_became_peered":"2024-04-25T00:14:52.531899+0000","last_unstale":"2024-04-25T00:14:53.509603+0000","last_undegraded":"2024-04-25T00:14:53.509603+0000","last_fullsized":"2024-04-25T00:14:53.509603+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:14.836642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744420+0000","last_change":"2024-04-25T00:14:51.513489+0000","last_active":"2024-04-25T00:14:53.744420+0000","last_peered":"2024-04-25T00:14:53.744420+0000","last_clean":"2024-04-25T00:14:53.744420+0000","last_became_active":"2024-04-25T00:14:51.513211+0000","last_became_peered":"2024-04-25T00:14:51.513211+0000","last_unstale":"2024-04-25T00:14:53.744420+0000","last_undegraded":"2024-04-25T00:14:53.744420+0000","last_fullsized":"2024-04-25T00:14:53.744420+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:45.508740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509517+0000","last_change":"2024-04-25T00:14:52.535915+0000","last_active":"2024-04-25T00:14:53.509517+0000","last_peered":"2024-04-25T00:14:53.509517+0000","last_clean":"2024-04-25T00:14:53.509517+0000","last_became_active":"2024-04-25T00:14:52.535690+0000","last_became_peered":"2024-04-25T00:14:52.535690+0000","last_unstale":"2024-04-25T00:14:53.509517+0000","last_undegraded":"2024-04-25T00:14:53.509517+0000","last_fullsized":"2024-04-25T00:14:53.509517+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:50:51.025060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509172+0000","last_change":"2024-04-25T00:14:51.521951+0000","last_active":"2024-04-25T00:14:53.509172+0000","last_peered":"2024-04-25T00:14:53.509172+0000","last_clean":"2024-04-25T00:14:53.509172+0000","last_became_active":"2024-04-25T00:14:51.521567+0000","last_became_peered":"2024-04-25T00:14:51.521567+0000","last_unstale":"2024-04-25T00:14:53.509172+0000","last_undegraded":"2024-04-25T00:14:53.509172+0000","last_fullsized":"2024-04-25T00:14:53.509172+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:16.457613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508701+0000","last_change":"2024-04-25T00:14:52.523082+0000","last_active":"2024-04-25T00:14:53.508701+0000","last_peered":"2024-04-25T00:14:53.508701+0000","last_clean":"2024-04-25T00:14:53.508701+0000","last_became_active":"2024-04-25T00:14:52.522611+0000","last_became_peered":"2024-04-25T00:14:52.522611+0000","last_unstale":"2024-04-25T00:14:53.508701+0000","last_undegraded":"2024-04-25T00:14:53.508701+0000","last_fullsized":"2024-04-25T00:14:53.508701+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:06.961202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744636+0000","last_change":"2024-04-25T00:14:51.514707+0000","last_active":"2024-04-25T00:14:53.744636+0000","last_peered":"2024-04-25T00:14:53.744636+0000","last_clean":"2024-04-25T00:14:53.744636+0000","last_became_active":"2024-04-25T00:14:51.514398+0000","last_became_peered":"2024-04-25T00:14:51.514398+0000","last_unstale":"2024-04-25T00:14:53.744636+0000","last_undegraded":"2024-04-25T00:14:53.744636+0000","last_fullsized":"2024-04-25T00:14:53.744636+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:41.712373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509258+0000","last_change":"2024-04-25T00:14:52.529814+0000","last_active":"2024-04-25T00:14:53.509258+0000","last_peered":"2024-04-25T00:14:53.509258+0000","last_clean":"2024-04-25T00:14:53.509258+0000","last_became_active":"2024-04-25T00:14:52.529525+0000","last_became_peered":"2024-04-25T00:14:52.529525+0000","last_unstale":"2024-04-25T00:14:53.509258+0000","last_undegraded":"2024-04-25T00:14:53.509258+0000","last_fullsized":"2024-04-25T00:14:53.509258+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:36.226306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509800+0000","last_change":"2024-04-25T00:14:51.519177+0000","last_active":"2024-04-25T00:14:53.509800+0000","last_peered":"2024-04-25T00:14:53.509800+0000","last_clean":"2024-04-25T00:14:53.509800+0000","last_became_active":"2024-04-25T00:14:51.518780+0000","last_became_peered":"2024-04-25T00:14:51.518780+0000","last_unstale":"2024-04-25T00:14:53.509800+0000","last_undegraded":"2024-04-25T00:14:53.509800+0000","last_fullsized":"2024-04-25T00:14:53.509800+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:41.712373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509350+0000","last_change":"2024-04-25T00:14:52.524475+0000","last_active":"2024-04-25T00:14:53.509350+0000","last_peered":"2024-04-25T00:14:53.509350+0000","last_clean":"2024-04-25T00:14:53.509350+0000","last_became_active":"2024-04-25T00:14:52.524170+0000","last_became_peered":"2024-04-25T00:14:52.524170+0000","last_unstale":"2024-04-25T00:14:53.509350+0000","last_undegraded":"2024-04-25T00:14:53.509350+0000","last_fullsized":"2024-04-25T00:14:53.509350+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:38.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745509+0000","last_change":"2024-04-25T00:14:51.518281+0000","last_active":"2024-04-25T00:14:53.745509+0000","last_peered":"2024-04-25T00:14:53.745509+0000","last_clean":"2024-04-25T00:14:53.745509+0000","last_became_active":"2024-04-25T00:14:51.518016+0000","last_became_peered":"2024-04-25T00:14:51.518016+0000","last_unstale":"2024-04-25T00:14:53.745509+0000","last_undegraded":"2024-04-25T00:14:53.745509+0000","last_fullsized":"2024-04-25T00:14:53.745509+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:37.196765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745123+0000","last_change":"2024-04-25T00:14:52.521190+0000","last_active":"2024-04-25T00:14:53.745123+0000","last_peered":"2024-04-25T00:14:53.745123+0000","last_clean":"2024-04-25T00:14:53.745123+0000","last_became_active":"2024-04-25T00:14:52.520323+0000","last_became_peered":"2024-04-25T00:14:52.520323+0000","last_unstale":"2024-04-25T00:14:53.745123+0000","last_undegraded":"2024-04-25T00:14:53.745123+0000","last_fullsized":"2024-04-25T00:14:53.745123+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:38.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531212+0000","last_change":"2024-04-25T00:14:51.517499+0000","last_active":"2024-04-25T00:14:53.531212+0000","last_peered":"2024-04-25T00:14:53.531212+0000","last_clean":"2024-04-25T00:14:53.531212+0000","last_became_active":"2024-04-25T00:14:51.516256+0000","last_became_peered":"2024-04-25T00:14:51.516256+0000","last_unstale":"2024-04-25T00:14:53.531212+0000","last_undegraded":"2024-04-25T00:14:53.531212+0000","last_fullsized":"2024-04-25T00:14:53.531212+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:35.977865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508016+0000","last_change":"2024-04-25T00:14:52.518545+0000","last_active":"2024-04-25T00:14:53.508016+0000","last_peered":"2024-04-25T00:14:53.508016+0000","last_clean":"2024-04-25T00:14:53.508016+0000","last_became_active":"2024-04-25T00:14:52.518349+0000","last_became_peered":"2024-04-25T00:14:52.518349+0000","last_unstale":"2024-04-25T00:14:53.508016+0000","last_undegraded":"2024-04-25T00:14:53.508016+0000","last_fullsized":"2024-04-25T00:14:53.508016+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:05.464806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509657+0000","last_change":"2024-04-25T00:14:51.521137+0000","last_active":"2024-04-25T00:14:53.509657+0000","last_peered":"2024-04-25T00:14:53.509657+0000","last_clean":"2024-04-25T00:14:53.509657+0000","last_became_active":"2024-04-25T00:14:51.520880+0000","last_became_peered":"2024-04-25T00:14:51.520880+0000","last_unstale":"2024-04-25T00:14:53.509657+0000","last_undegraded":"2024-04-25T00:14:53.509657+0000","last_fullsized":"2024-04-25T00:14:53.509657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:02.743200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.508309+0000","last_change":"2024-04-25T00:14:52.520152+0000","last_active":"2024-04-25T00:14:53.508309+0000","last_peered":"2024-04-25T00:14:53.508309+0000","last_clean":"2024-04-25T00:14:53.508309+0000","last_became_active":"2024-04-25T00:14:52.519374+0000","last_became_peered":"2024-04-25T00:14:52.519374+0000","last_unstale":"2024-04-25T00:14:53.508309+0000","last_undegraded":"2024-04-25T00:14:53.508309+0000","last_fullsized":"2024-04-25T00:14:53.508309+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:17.886622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509154+0000","last_change":"2024-04-25T00:14:51.519999+0000","last_active":"2024-04-25T00:14:53.509154+0000","last_peered":"2024-04-25T00:14:53.509154+0000","last_clean":"2024-04-25T00:14:53.509154+0000","last_became_active":"2024-04-25T00:14:51.519734+0000","last_became_peered":"2024-04-25T00:14:51.519734+0000","last_unstale":"2024-04-25T00:14:53.509154+0000","last_undegraded":"2024-04-25T00:14:53.509154+0000","last_fullsized":"2024-04-25T00:14:53.509154+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:39.607770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744537+0000","last_change":"2024-04-25T00:14:52.518406+0000","last_active":"2024-04-25T00:14:53.744537+0000","last_peered":"2024-04-25T00:14:53.744537+0000","last_clean":"2024-04-25T00:14:53.744537+0000","last_became_active":"2024-04-25T00:14:52.518151+0000","last_became_peered":"2024-04-25T00:14:52.518151+0000","last_unstale":"2024-04-25T00:14:53.744537+0000","last_undegraded":"2024-04-25T00:14:53.744537+0000","last_fullsized":"2024-04-25T00:14:53.744537+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:34.309725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509226+0000","last_change":"2024-04-25T00:14:51.511936+0000","last_active":"2024-04-25T00:14:53.509226+0000","last_peered":"2024-04-25T00:14:53.509226+0000","last_clean":"2024-04-25T00:14:53.509226+0000","last_became_active":"2024-04-25T00:14:51.511671+0000","last_became_peered":"2024-04-25T00:14:51.511671+0000","last_unstale":"2024-04-25T00:14:53.509226+0000","last_undegraded":"2024-04-25T00:14:53.509226+0000","last_fullsized":"2024-04-25T00:14:53.509226+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:12.379516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509768+0000","last_change":"2024-04-25T00:14:52.529212+0000","last_active":"2024-04-25T00:14:53.509768+0000","last_peered":"2024-04-25T00:14:53.509768+0000","last_clean":"2024-04-25T00:14:53.509768+0000","last_became_active":"2024-04-25T00:14:52.528900+0000","last_became_peered":"2024-04-25T00:14:52.528900+0000","last_unstale":"2024-04-25T00:14:53.509768+0000","last_undegraded":"2024-04-25T00:14:53.509768+0000","last_fullsized":"2024-04-25T00:14:53.509768+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:50.985016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509254+0000","last_change":"2024-04-25T00:14:51.521000+0000","last_active":"2024-04-25T00:14:53.509254+0000","last_peered":"2024-04-25T00:14:53.509254+0000","last_clean":"2024-04-25T00:14:53.509254+0000","last_became_active":"2024-04-25T00:14:51.520726+0000","last_became_peered":"2024-04-25T00:14:51.520726+0000","last_unstale":"2024-04-25T00:14:53.509254+0000","last_undegraded":"2024-04-25T00:14:53.509254+0000","last_fullsized":"2024-04-25T00:14:53.509254+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:46.910183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.507882+0000","last_change":"2024-04-25T00:14:52.520178+0000","last_active":"2024-04-25T00:14:53.507882+0000","last_peered":"2024-04-25T00:14:53.507882+0000","last_clean":"2024-04-25T00:14:53.507882+0000","last_became_active":"2024-04-25T00:14:52.519896+0000","last_became_peered":"2024-04-25T00:14:52.519896+0000","last_unstale":"2024-04-25T00:14:53.507882+0000","last_undegraded":"2024-04-25T00:14:53.507882+0000","last_fullsized":"2024-04-25T00:14:53.507882+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:57.182997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745238+0000","last_change":"2024-04-25T00:14:51.514643+0000","last_active":"2024-04-25T00:14:53.745238+0000","last_peered":"2024-04-25T00:14:53.745238+0000","last_clean":"2024-04-25T00:14:53.745238+0000","last_became_active":"2024-04-25T00:14:51.514472+0000","last_became_peered":"2024-04-25T00:14:51.514472+0000","last_unstale":"2024-04-25T00:14:53.745238+0000","last_undegraded":"2024-04-25T00:14:53.745238+0000","last_fullsized":"2024-04-25T00:14:53.745238+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:26.554045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744690+0000","last_change":"2024-04-25T00:14:52.536465+0000","last_active":"2024-04-25T00:14:53.744690+0000","last_peered":"2024-04-25T00:14:53.744690+0000","last_clean":"2024-04-25T00:14:53.744690+0000","last_became_active":"2024-04-25T00:14:52.536200+0000","last_became_peered":"2024-04-25T00:14:52.536200+0000","last_unstale":"2024-04-25T00:14:53.744690+0000","last_undegraded":"2024-04-25T00:14:53.744690+0000","last_fullsized":"2024-04-25T00:14:53.744690+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:50.842042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"93'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.530015+0000","last_change":"2024-04-25T00:14:51.519777+0000","last_active":"2024-04-25T00:14:53.530015+0000","last_peered":"2024-04-25T00:14:53.530015+0000","last_clean":"2024-04-25T00:14:53.530015+0000","last_became_active":"2024-04-25T00:14:51.519515+0000","last_became_peered":"2024-04-25T00:14:51.519515+0000","last_unstale":"2024-04-25T00:14:53.530015+0000","last_undegraded":"2024-04-25T00:14:53.530015+0000","last_fullsized":"2024-04-25T00:14:53.530015+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:06.458007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744404+0000","last_change":"2024-04-25T00:14:52.517941+0000","last_active":"2024-04-25T00:14:53.744404+0000","last_peered":"2024-04-25T00:14:53.744404+0000","last_clean":"2024-04-25T00:14:53.744404+0000","last_became_active":"2024-04-25T00:14:52.517675+0000","last_became_peered":"2024-04-25T00:14:52.517675+0000","last_unstale":"2024-04-25T00:14:53.744404+0000","last_undegraded":"2024-04-25T00:14:53.744404+0000","last_fullsized":"2024-04-25T00:14:53.744404+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:36.450062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509557+0000","last_change":"2024-04-25T00:14:51.517299+0000","last_active":"2024-04-25T00:14:53.509557+0000","last_peered":"2024-04-25T00:14:53.509557+0000","last_clean":"2024-04-25T00:14:53.509557+0000","last_became_active":"2024-04-25T00:14:51.517004+0000","last_became_peered":"2024-04-25T00:14:51.517004+0000","last_unstale":"2024-04-25T00:14:53.509557+0000","last_undegraded":"2024-04-25T00:14:53.509557+0000","last_fullsized":"2024-04-25T00:14:53.509557+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:52.589376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531194+0000","last_change":"2024-04-25T00:14:52.530122+0000","last_active":"2024-04-25T00:14:53.531194+0000","last_peered":"2024-04-25T00:14:53.531194+0000","last_clean":"2024-04-25T00:14:53.531194+0000","last_became_active":"2024-04-25T00:14:52.529866+0000","last_became_peered":"2024-04-25T00:14:52.529866+0000","last_unstale":"2024-04-25T00:14:53.531194+0000","last_undegraded":"2024-04-25T00:14:53.531194+0000","last_fullsized":"2024-04-25T00:14:53.531194+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:56.739421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745293+0000","last_change":"2024-04-25T00:14:51.513962+0000","last_active":"2024-04-25T00:14:53.745293+0000","last_peered":"2024-04-25T00:14:53.745293+0000","last_clean":"2024-04-25T00:14:53.745293+0000","last_became_active":"2024-04-25T00:14:51.513711+0000","last_became_peered":"2024-04-25T00:14:51.513711+0000","last_unstale":"2024-04-25T00:14:53.745293+0000","last_undegraded":"2024-04-25T00:14:53.745293+0000","last_fullsized":"2024-04-25T00:14:53.745293+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:29.130112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509520+0000","last_change":"2024-04-25T00:14:52.539935+0000","last_active":"2024-04-25T00:14:53.509520+0000","last_peered":"2024-04-25T00:14:53.509520+0000","last_clean":"2024-04-25T00:14:53.509520+0000","last_became_active":"2024-04-25T00:14:52.539617+0000","last_became_peered":"2024-04-25T00:14:52.539617+0000","last_unstale":"2024-04-25T00:14:53.509520+0000","last_undegraded":"2024-04-25T00:14:53.509520+0000","last_fullsized":"2024-04-25T00:14:53.509520+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:08.714777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744783+0000","last_change":"2024-04-25T00:14:51.516586+0000","last_active":"2024-04-25T00:14:53.744783+0000","last_peered":"2024-04-25T00:14:53.744783+0000","last_clean":"2024-04-25T00:14:53.744783+0000","last_became_active":"2024-04-25T00:14:51.516088+0000","last_became_peered":"2024-04-25T00:14:51.516088+0000","last_unstale":"2024-04-25T00:14:53.744783+0000","last_undegraded":"2024-04-25T00:14:53.744783+0000","last_fullsized":"2024-04-25T00:14:53.744783+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:52.589376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509458+0000","last_change":"2024-04-25T00:14:52.535104+0000","last_active":"2024-04-25T00:14:53.509458+0000","last_peered":"2024-04-25T00:14:53.509458+0000","last_clean":"2024-04-25T00:14:53.509458+0000","last_became_active":"2024-04-25T00:14:52.534827+0000","last_became_peered":"2024-04-25T00:14:52.534827+0000","last_unstale":"2024-04-25T00:14:53.509458+0000","last_undegraded":"2024-04-25T00:14:53.509458+0000","last_fullsized":"2024-04-25T00:14:53.509458+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:56.396750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509556+0000","last_change":"2024-04-25T00:14:51.519047+0000","last_active":"2024-04-25T00:14:53.509556+0000","last_peered":"2024-04-25T00:14:53.509556+0000","last_clean":"2024-04-25T00:14:53.509556+0000","last_became_active":"2024-04-25T00:14:51.518767+0000","last_became_peered":"2024-04-25T00:14:51.518767+0000","last_unstale":"2024-04-25T00:14:53.509556+0000","last_undegraded":"2024-04-25T00:14:53.509556+0000","last_fullsized":"2024-04-25T00:14:53.509556+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:13.389688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.745561+0000","last_change":"2024-04-25T00:14:52.524928+0000","last_active":"2024-04-25T00:14:53.745561+0000","last_peered":"2024-04-25T00:14:53.745561+0000","last_clean":"2024-04-25T00:14:53.745561+0000","last_became_active":"2024-04-25T00:14:52.524551+0000","last_became_peered":"2024-04-25T00:14:52.524551+0000","last_unstale":"2024-04-25T00:14:53.745561+0000","last_undegraded":"2024-04-25T00:14:53.745561+0000","last_fullsized":"2024-04-25T00:14:53.745561+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:46.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509235+0000","last_change":"2024-04-25T00:14:51.517435+0000","last_active":"2024-04-25T00:14:53.509235+0000","last_peered":"2024-04-25T00:14:53.509235+0000","last_clean":"2024-04-25T00:14:53.509235+0000","last_became_active":"2024-04-25T00:14:51.517082+0000","last_became_peered":"2024-04-25T00:14:51.517082+0000","last_unstale":"2024-04-25T00:14:53.509235+0000","last_undegraded":"2024-04-25T00:14:53.509235+0000","last_fullsized":"2024-04-25T00:14:53.509235+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:00.869615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531586+0000","last_change":"2024-04-25T00:14:52.538711+0000","last_active":"2024-04-25T00:14:53.531586+0000","last_peered":"2024-04-25T00:14:53.531586+0000","last_clean":"2024-04-25T00:14:53.531586+0000","last_became_active":"2024-04-25T00:14:52.538445+0000","last_became_peered":"2024-04-25T00:14:52.538445+0000","last_unstale":"2024-04-25T00:14:53.531586+0000","last_undegraded":"2024-04-25T00:14:53.531586+0000","last_fullsized":"2024-04-25T00:14:53.531586+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:59.770333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509355+0000","last_change":"2024-04-25T00:14:51.516012+0000","last_active":"2024-04-25T00:14:53.509355+0000","last_peered":"2024-04-25T00:14:53.509355+0000","last_clean":"2024-04-25T00:14:53.509355+0000","last_became_active":"2024-04-25T00:14:51.515625+0000","last_became_peered":"2024-04-25T00:14:51.515625+0000","last_unstale":"2024-04-25T00:14:53.509355+0000","last_undegraded":"2024-04-25T00:14:53.509355+0000","last_fullsized":"2024-04-25T00:14:53.509355+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:47.065775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.531102+0000","last_change":"2024-04-25T00:14:52.531091+0000","last_active":"2024-04-25T00:14:53.531102+0000","last_peered":"2024-04-25T00:14:53.531102+0000","last_clean":"2024-04-25T00:14:53.531102+0000","last_became_active":"2024-04-25T00:14:52.530720+0000","last_became_peered":"2024-04-25T00:14:52.530720+0000","last_unstale":"2024-04-25T00:14:53.531102+0000","last_undegraded":"2024-04-25T00:14:53.531102+0000","last_fullsized":"2024-04-25T00:14:53.531102+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.488524+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.488524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:22.645056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.509647+0000","last_change":"2024-04-25T00:14:51.519517+0000","last_active":"2024-04-25T00:14:53.509647+0000","last_peered":"2024-04-25T00:14:53.509647+0000","last_clean":"2024-04-25T00:14:53.509647+0000","last_became_active":"2024-04-25T00:14:51.519245+0000","last_became_peered":"2024-04-25T00:14:51.519245+0000","last_unstale":"2024-04-25T00:14:53.509647+0000","last_undegraded":"2024-04-25T00:14:53.509647+0000","last_fullsized":"2024-04-25T00:14:53.509647+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:50.484752+0000","last_clean_scrub_stamp":"2024-04-25T00:14:50.484752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:13.485814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":305,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T00:14:53.744794+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:14:53.744794+0000","last_peered":"2024-04-25T00:14:53.744794+0000","last_clean":"2024-04-25T00:14:53.744794+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:14:53.744794+0000","last_undegraded":"2024-04-25T00:14:53.744794+0000","last_fullsized":"2024-04-25T00:14:53.744794+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1757184,"data_stored":1118116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1175552,"data_stored":529804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1155072,"data_stored":527588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1163264,"data_stored":527680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1159168,"data_stored":528170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1159168,"data_stored":527622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1159168,"data_stored":527622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1765376,"data_stored":1119658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:14:56.970 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:14:56.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:14:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.147+0000 7ffa69800640 1 Processor -- start 2024-04-25T00:14:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.147+0000 7ffa69800640 1 -- start start 2024-04-25T00:14:57.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.147+0000 7ffa69800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffa6415de90 0x7ffa64071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.147+0000 7ffa69800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa64162a10 con 0x7ffa6406e840 2024-04-25T00:14:57.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.147+0000 7ffa69800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa6406dcf0 con 0x7ffa6415e270 2024-04-25T00:14:57.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.147+0000 7ffa69800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ffa640b4460 con 0x7ffa6415de90 2024-04-25T00:14:57.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.147+0000 7ffa62a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffa6415de90 0x7ffa64071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.147+0000 7ffa63400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ffa6406e840 0x7ffa64162300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51340/0 (socket says 172.21.15.83:51340) 2024-04-25T00:14:57.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.147+0000 7ffa63400640 1 -- 172.21.15.83:0/3222278979 learned_addr learned my addr 172.21.15.83:0/3222278979 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:57.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.147+0000 7ffa62a00640 1 -- 172.21.15.83:0/3222278979 >> v1:172.21.15.86:6789/0 conn(0x7ffa6415e270 legacy=0x7ffa64161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.147+0000 7ffa62a00640 1 -- 172.21.15.83:0/3222278979 >> v1:172.21.15.83:6789/0 conn(0x7ffa6406e840 legacy=0x7ffa64162300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.147+0000 7ffa62a00640 1 -- 172.21.15.83:0/3222278979 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6406dfd0 con 0x7ffa6415de90 2024-04-25T00:14:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.151+0000 7ffa62a00640 1 --2- 172.21.15.83:0/3222278979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa6415de90 0x7ffa64071740 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ffa540097c0 tx=0x7ffa5401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74d0627466d7baa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.151+0000 7ffa61600640 1 -- 172.21.15.83:0/3222278979 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa540b0070 con 0x7ffa6415de90 2024-04-25T00:14:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.151+0000 7ffa61600640 1 -- 172.21.15.83:0/3222278979 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa540ab600 con 0x7ffa6415de90 2024-04-25T00:14:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.151+0000 7ffa61600640 1 -- 172.21.15.83:0/3222278979 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa540ab9d0 con 0x7ffa6415de90 2024-04-25T00:14:57.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.151+0000 7ffa69800640 1 -- 172.21.15.83:0/3222278979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa6415de90 msgr2=0x7ffa64071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.151+0000 7ffa69800640 1 --2- 172.21.15.83:0/3222278979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa6415de90 0x7ffa64071740 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ffa540097c0 tx=0x7ffa5401c410 comp rx=0 tx=0).stop 2024-04-25T00:14:57.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.151+0000 7ffa69800640 1 -- 172.21.15.83:0/3222278979 shutdown_connections 2024-04-25T00:14:57.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.151+0000 7ffa69800640 1 --2- 172.21.15.83:0/3222278979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa6415de90 0x7ffa64071740 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.151+0000 7ffa69800640 1 -- 172.21.15.83:0/3222278979 >> 172.21.15.83:0/3222278979 conn(0x7ffa640b1830 msgr2=0x7ffa640b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:57.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.151+0000 7ffa69800640 1 -- 172.21.15.83:0/3222278979 shutdown_connections 2024-04-25T00:14:57.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.151+0000 7ffa69800640 1 -- 172.21.15.83:0/3222278979 wait complete. 2024-04-25T00:14:57.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa69800640 1 Processor -- start 2024-04-25T00:14:57.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa69800640 1 -- start start 2024-04-25T00:14:57.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa69800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6406e840 0x7ffa64150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa69800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa6415de90 0x7ffa6413f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa69800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa6415e270 0x7ffa64162d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa69800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa640ad860 con 0x7ffa6415e270 2024-04-25T00:14:57.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa69800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa64072090 con 0x7ffa6406e840 2024-04-25T00:14:57.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa69800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6416c580 con 0x7ffa6415de90 2024-04-25T00:14:57.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa63400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa6415e270 0x7ffa64162d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa63400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa6415e270 0x7ffa64162d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51888/0 (socket says 172.21.15.83:51888) 2024-04-25T00:14:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa63400640 1 -- 172.21.15.83:0/2329450953 learned_addr learned my addr 172.21.15.83:0/2329450953 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa62a00640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6406e840 0x7ffa64150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa62000640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa6415de90 0x7ffa6413f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa63400640 1 -- 172.21.15.83:0/2329450953 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa6415de90 msgr2=0x7ffa6413f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa63400640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa6415de90 0x7ffa6413f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa63400640 1 -- 172.21.15.83:0/2329450953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6406e840 msgr2=0x7ffa64150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa63400640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6406e840 0x7ffa64150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa63400640 1 -- 172.21.15.83:0/2329450953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6413fa10 con 0x7ffa6415e270 2024-04-25T00:14:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa62a00640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6406e840 0x7ffa64150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa62000640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa6415de90 0x7ffa6413f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa63400640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa6415e270 0x7ffa64162d20 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00cab0 tx=0x7ffa5c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa5aa00640 1 -- 172.21.15.83:0/2329450953 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c017070 con 0x7ffa6415e270 2024-04-25T00:14:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa5aa00640 1 -- 172.21.15.83:0/2329450953 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5c00ac30 con 0x7ffa6415e270 2024-04-25T00:14:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa5aa00640 1 -- 172.21.15.83:0/2329450953 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c012400 con 0x7ffa6415e270 2024-04-25T00:14:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa69800640 1 -- 172.21.15.83:0/2329450953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa64150ea0 con 0x7ffa6415e270 2024-04-25T00:14:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa69800640 1 -- 172.21.15.83:0/2329450953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa64151460 con 0x7ffa6415e270 2024-04-25T00:14:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.155+0000 7ffa69800640 1 -- 172.21.15.83:0/2329450953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa6404f510 con 0x7ffa6415e270 2024-04-25T00:14:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.159+0000 7ffa5aa00640 1 -- 172.21.15.83:0/2329450953 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffa5c01a070 con 0x7ffa6415e270 2024-04-25T00:14:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.159+0000 7ffa5aa00640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffa440678c0 0x7ffa44069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.159+0000 7ffa5aa00640 1 -- 172.21.15.83:0/2329450953 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7ffa5c085af0 con 0x7ffa6415e270 2024-04-25T00:14:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.159+0000 7ffa62a00640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffa440678c0 0x7ffa44069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.159+0000 7ffa62a00640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffa440678c0 0x7ffa44069d80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ffa540ad000 tx=0x7ffa540a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.163+0000 7ffa5aa00640 1 -- 172.21.15.83:0/2329450953 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffa5c055db0 con 0x7ffa6415e270 2024-04-25T00:14:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.375+0000 7ffa69800640 1 -- 172.21.15.83:0/2329450953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ffa64169220 con 0x7ffa6415e270 2024-04-25T00:14:57.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.375+0000 7ffa5aa00640 1 -- 172.21.15.83:0/2329450953 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7ffa5c05a0b0 con 0x7ffa6415e270 2024-04-25T00:14:57.381 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-25T00:14:57.381 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBenylmOqooLBAApte+X6YkRmrpe12YdjYekg== 2024-04-25T00:14:57.381 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-25T00:14:57.381 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-25T00:14:57.381 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:14:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-25T00:14:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.379+0000 7ffa69800640 1 -- 172.21.15.83:0/2329450953 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffa440678c0 msgr2=0x7ffa44069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.379+0000 7ffa69800640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffa440678c0 0x7ffa44069d80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ffa540ad000 tx=0x7ffa540a6000 comp rx=0 tx=0).stop 2024-04-25T00:14:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.379+0000 7ffa69800640 1 -- 172.21.15.83:0/2329450953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa6415e270 msgr2=0x7ffa64162d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.379+0000 7ffa69800640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa6415e270 0x7ffa64162d20 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00cab0 tx=0x7ffa5c00cf70 comp rx=0 tx=0).stop 2024-04-25T00:14:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.379+0000 7ffa69800640 1 -- 172.21.15.83:0/2329450953 shutdown_connections 2024-04-25T00:14:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.379+0000 7ffa69800640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ffa440678c0 0x7ffa44069d80 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.379+0000 7ffa69800640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa6415de90 0x7ffa6413f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.379+0000 7ffa69800640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa6415e270 0x7ffa64162d20 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.379+0000 7ffa69800640 1 --2- 172.21.15.83:0/2329450953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6406e840 0x7ffa64150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.379+0000 7ffa69800640 1 -- 172.21.15.83:0/2329450953 >> 172.21.15.83:0/2329450953 conn(0x7ffa640b1830 msgr2=0x7ffa64152ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.379+0000 7ffa69800640 1 -- 172.21.15.83:0/2329450953 shutdown_connections 2024-04-25T00:14:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.379+0000 7ffa69800640 1 -- 172.21.15.83:0/2329450953 wait complete. 2024-04-25T00:14:57.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:57.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb58600640 1 Processor -- start 2024-04-25T00:14:57.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb58600640 1 -- start start 2024-04-25T00:14:57.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb58600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb501553e0 0x7fcb5006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb58600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb5006ef80 con 0x7fcb500b4a70 2024-04-25T00:14:57.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb58600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb5015b510 con 0x7fcb5006ea00 2024-04-25T00:14:57.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb58600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcb5015b7f0 con 0x7fcb501553e0 2024-04-25T00:14:57.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb56600640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcb500b4a70 0x7fcb500b2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51342/0 (socket says 172.21.15.83:51342) 2024-04-25T00:14:57.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb56600640 1 -- 172.21.15.83:0/2853307187 learned_addr learned my addr 172.21.15.83:0/2853307187 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb55c00640 1 --2- 172.21.15.83:0/2853307187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb501553e0 0x7fcb5006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 -- 172.21.15.83:0/2853307187 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3250040248 0 0) 0x7fcb5006ef80 con 0x7fcb500b4a70 2024-04-25T00:14:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 -- 172.21.15.83:0/2853307187 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb38005740 con 0x7fcb500b4a70 2024-04-25T00:14:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 -- 172.21.15.83:0/2853307187 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4146459208 0 0) 0x7fcb38005740 con 0x7fcb500b4a70 2024-04-25T00:14:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 -- 172.21.15.83:0/2853307187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb501553e0 msgr2=0x7fcb5006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 --2- 172.21.15.83:0/2853307187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb501553e0 0x7fcb5006e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 -- 172.21.15.83:0/2853307187 >> v1:172.21.15.86:6789/0 conn(0x7fcb5006ea00 legacy=0x7fcb500b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 -- 172.21.15.83:0/2853307187 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5015bad0 con 0x7fcb500b4a70 2024-04-25T00:14:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 -- 172.21.15.83:0/2853307187 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fcb48001a00 con 0x7fcb500b4a70 2024-04-25T00:14:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 -- 172.21.15.83:0/2853307187 >> v1:172.21.15.83:6789/0 conn(0x7fcb500b4a70 legacy=0x7fcb500b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 --2- 172.21.15.83:0/2853307187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb38005c00 0x7fcb38006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 --2- 172.21.15.83:0/2853307187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38006a20 0x7fcb38006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 --2- 172.21.15.83:0/2853307187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb501553e0 0x7fcb38007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 -- 172.21.15.83:0/2853307187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5015b7f0 con 0x7fcb38006a20 2024-04-25T00:14:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 -- 172.21.15.83:0/2853307187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3800dd10 con 0x7fcb38005c00 2024-04-25T00:14:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb4fe00640 1 -- 172.21.15.83:0/2853307187 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3800df10 con 0x7fcb501553e0 2024-04-25T00:14:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb55200640 1 --2- 172.21.15.83:0/2853307187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38006a20 0x7fcb38006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb56600640 1 --2- 172.21.15.83:0/2853307187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb501553e0 0x7fcb38007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb55c00640 1 --2- 172.21.15.83:0/2853307187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb38005c00 0x7fcb38006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb55200640 1 -- 172.21.15.83:0/2853307187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb501553e0 msgr2=0x7fcb38007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb55200640 1 --2- 172.21.15.83:0/2853307187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb501553e0 0x7fcb38007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb55200640 1 -- 172.21.15.83:0/2853307187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb38005c00 msgr2=0x7fcb38006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb55200640 1 --2- 172.21.15.83:0/2853307187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb38005c00 0x7fcb38006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.535+0000 7fcb55200640 1 -- 172.21.15.83:0/2853307187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb38005740 con 0x7fcb38006a20 2024-04-25T00:14:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.539+0000 7fcb55200640 1 --2- 172.21.15.83:0/2853307187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38006a20 0x7fcb38006ea0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c001780 tx=0x7fcb3c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb7032a29870e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.539+0000 7fcb4fe00640 1 -- 172.21.15.83:0/2853307187 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c0013a0 con 0x7fcb38006a20 2024-04-25T00:14:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.539+0000 7fcb4fe00640 1 -- 172.21.15.83:0/2853307187 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3c0052f0 con 0x7fcb38006a20 2024-04-25T00:14:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.539+0000 7fcb4fe00640 1 -- 172.21.15.83:0/2853307187 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c003840 con 0x7fcb38006a20 2024-04-25T00:14:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.539+0000 7fcb58600640 1 -- 172.21.15.83:0/2853307187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38006a20 msgr2=0x7fcb38006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.539+0000 7fcb58600640 1 --2- 172.21.15.83:0/2853307187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38006a20 0x7fcb38006ea0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c001780 tx=0x7fcb3c01c420 comp rx=0 tx=0).stop 2024-04-25T00:14:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.539+0000 7fcb56600640 1 -- 172.21.15.83:0/2853307187 reap_dead start 2024-04-25T00:14:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.539+0000 7fcb58600640 1 -- 172.21.15.83:0/2853307187 shutdown_connections 2024-04-25T00:14:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.539+0000 7fcb58600640 1 -- 172.21.15.83:0/2853307187 >> 172.21.15.83:0/2853307187 conn(0x7fcb500b1830 msgr2=0x7fcb500b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb58600640 1 -- 172.21.15.83:0/2853307187 shutdown_connections 2024-04-25T00:14:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb58600640 1 -- 172.21.15.83:0/2853307187 wait complete. 2024-04-25T00:14:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb58600640 1 Processor -- start 2024-04-25T00:14:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb58600640 1 -- start start 2024-04-25T00:14:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb58600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38006a20 0x7fcb50152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb58600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb501f6420 0x7fcb500af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb58600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb500cfa30 0x7fcb500cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb58600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5006d530 con 0x7fcb38006a20 2024-04-25T00:14:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb58600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5006d3b0 con 0x7fcb501f6420 2024-04-25T00:14:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb58600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcb5006d6b0 con 0x7fcb500cfa30 2024-04-25T00:14:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb55c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38006a20 0x7fcb50152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb55c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38006a20 0x7fcb50152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51902/0 (socket says 172.21.15.83:51902) 2024-04-25T00:14:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb55c00640 1 -- 172.21.15.83:0/665067846 learned_addr learned my addr 172.21.15.83:0/665067846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:57.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb56600640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb500cfa30 0x7fcb500cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb55200640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb501f6420 0x7fcb500af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb55c00640 1 -- 172.21.15.83:0/665067846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb500cfa30 msgr2=0x7fcb500cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb55c00640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb500cfa30 0x7fcb500cfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb55c00640 1 -- 172.21.15.83:0/665067846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb501f6420 msgr2=0x7fcb500af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb55c00640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb501f6420 0x7fcb500af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb55c00640 1 -- 172.21.15.83:0/665067846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb500af540 con 0x7fcb38006a20 2024-04-25T00:14:57.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb55200640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb501f6420 0x7fcb500af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:57.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb56600640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb500cfa30 0x7fcb500cfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:57.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb55c00640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38006a20 0x7fcb50152090 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcb44011960 tx=0x7fcb44011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb4d600640 1 -- 172.21.15.83:0/665067846 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400faf0 con 0x7fcb38006a20 2024-04-25T00:14:57.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb4d600640 1 -- 172.21.15.83:0/665067846 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb4400fc90 con 0x7fcb38006a20 2024-04-25T00:14:57.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb4d600640 1 -- 172.21.15.83:0/665067846 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44014440 con 0x7fcb38006a20 2024-04-25T00:14:57.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.543+0000 7fcb58600640 1 -- 172.21.15.83:0/665067846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb501f68b0 con 0x7fcb38006a20 2024-04-25T00:14:57.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.547+0000 7fcb58600640 1 -- 172.21.15.83:0/665067846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb501f6e70 con 0x7fcb38006a20 2024-04-25T00:14:57.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.547+0000 7fcb58600640 1 -- 172.21.15.83:0/665067846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb500737f0 con 0x7fcb38006a20 2024-04-25T00:14:57.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.547+0000 7fcb4d600640 1 -- 172.21.15.83:0/665067846 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcb44012040 con 0x7fcb38006a20 2024-04-25T00:14:57.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.547+0000 7fcb4d600640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb300678c0 0x7fcb30069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.547+0000 7fcb4d600640 1 -- 172.21.15.83:0/665067846 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fcb44086480 con 0x7fcb38006a20 2024-04-25T00:14:57.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.547+0000 7fcb55200640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb300678c0 0x7fcb30069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.547+0000 7fcb55200640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb300678c0 0x7fcb30069d80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c001750 tx=0x7fcb3c0011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.551+0000 7fcb4d600640 1 -- 172.21.15.83:0/665067846 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcb44019030 con 0x7fcb38006a20 2024-04-25T00:14:57.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.739+0000 7fcb58600640 1 -- 172.21.15.83:0/665067846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb50155c80 con 0x7fcb38006a20 2024-04-25T00:14:57.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.739+0000 7fcb4d600640 1 -- 172.21.15.83:0/665067846 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5177 (secure 0 0 0) 0x7fcb4401b300 con 0x7fcb38006a20 2024-04-25T00:14:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:14:57.744 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/45139956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":45139956},{"type":"v1","addr":"172.21.15.83:6835","nonce":45139956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7455,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1609715757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1609715757},{"type":"v1","addr":"172.21.15.83:6837","nonce":1609715757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/3915910404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3915910404},{"type":"v1","addr":"172.21.15.86:6834","nonce":3915910404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:51.879542+0000","modified":"2024-04-25T00:14:54.899792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7467":{"gid":7467,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/3612679148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":3612679148},{"type":"v1","addr":"172.21.15.86:6835","nonce":3612679148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7467,"qdb_cluster":[7467]},"id":7}]} 2024-04-25T00:14:57.745 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-04-25T00:14:57.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.739+0000 7fcb58600640 1 -- 172.21.15.83:0/665067846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb300678c0 msgr2=0x7fcb30069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.739+0000 7fcb58600640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb300678c0 0x7fcb30069d80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c001750 tx=0x7fcb3c0011a0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.739+0000 7fcb58600640 1 -- 172.21.15.83:0/665067846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38006a20 msgr2=0x7fcb50152090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.739+0000 7fcb58600640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38006a20 0x7fcb50152090 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcb44011960 tx=0x7fcb44011e20 comp rx=0 tx=0).stop 2024-04-25T00:14:57.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.739+0000 7fcb58600640 1 -- 172.21.15.83:0/665067846 shutdown_connections 2024-04-25T00:14:57.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.739+0000 7fcb58600640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcb300678c0 0x7fcb30069d80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.739+0000 7fcb58600640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb500cfa30 0x7fcb500cfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.739+0000 7fcb58600640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb501f6420 0x7fcb500af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.739+0000 7fcb58600640 1 --2- 172.21.15.83:0/665067846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb38006a20 0x7fcb50152090 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.739+0000 7fcb58600640 1 -- 172.21.15.83:0/665067846 >> 172.21.15.83:0/665067846 conn(0x7fcb500b1830 msgr2=0x7fcb5015d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:57.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.739+0000 7fcb58600640 1 -- 172.21.15.83:0/665067846 shutdown_connections 2024-04-25T00:14:57.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:57.739+0000 7fcb58600640 1 -- 172.21.15.83:0/665067846 wait complete. 2024-04-25T00:14:58.167 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:14:51.879542+0000', 'modified': '2024-04-25T00:14:54.899792+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7467': {'gid': 7467, 'name': 'd', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6835/3612679148', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6833', 'nonce': 3612679148}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 3612679148}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7467, 'qdb_cluster': [7467]} 2024-04-25T00:14:58.168 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:14:58.168 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T00:14:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.299+0000 7f9c3ce00640 1 Processor -- start 2024-04-25T00:14:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c3ce00640 1 -- start start 2024-04-25T00:14:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c3ce00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c3806cfd0 0x7f9c38070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c3ce00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c3806d3b0 0x7f9c3806d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c3ce00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c3806e160 con 0x7f9c380ca630 2024-04-25T00:14:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c3ce00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9c3806ea10 con 0x7f9c3806cfd0 2024-04-25T00:14:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c3ce00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9c3806ecf0 con 0x7f9c3806d3b0 2024-04-25T00:14:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c35e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c3806cfd0 0x7f9c38070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c36800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9c380ca630 0x7f9c380caac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51358/0 (socket says 172.21.15.83:51358) 2024-04-25T00:14:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c36800640 1 -- 172.21.15.83:0/242569812 learned_addr learned my addr 172.21.15.83:0/242569812 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c35400640 1 --2- 172.21.15.83:0/242569812 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c3806d3b0 0x7f9c3806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c35e00640 1 -- 172.21.15.83:0/242569812 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c3806d3b0 msgr2=0x7f9c3806d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c35e00640 1 --2- 172.21.15.83:0/242569812 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c3806d3b0 0x7f9c3806d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c35e00640 1 -- 172.21.15.83:0/242569812 >> v1:172.21.15.83:6789/0 conn(0x7f9c380ca630 legacy=0x7f9c380caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c35e00640 1 -- 172.21.15.83:0/242569812 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c3806efd0 con 0x7f9c3806cfd0 2024-04-25T00:14:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c35400640 1 --2- 172.21.15.83:0/242569812 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c3806d3b0 0x7f9c3806d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.303+0000 7f9c34a00640 1 -- 172.21.15.83:0/242569812 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3279662699 0 0) 0x7f9c3806e160 con 0x7f9c380ca630 2024-04-25T00:14:58.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.307+0000 7f9c35e00640 1 --2- 172.21.15.83:0/242569812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c3806cfd0 0x7f9c38070d10 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9c280097c0 tx=0x7f9c2801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90daaf1246cf72fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:58.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.307+0000 7f9c34a00640 1 -- 172.21.15.83:0/242569812 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c280b0070 con 0x7f9c3806cfd0 2024-04-25T00:14:58.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.307+0000 7f9c34a00640 1 -- 172.21.15.83:0/242569812 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c280ab600 con 0x7f9c3806cfd0 2024-04-25T00:14:58.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.307+0000 7f9c34a00640 1 -- 172.21.15.83:0/242569812 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c280ab9d0 con 0x7f9c3806cfd0 2024-04-25T00:14:58.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.307+0000 7f9c3ce00640 1 -- 172.21.15.83:0/242569812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c3806cfd0 msgr2=0x7f9c38070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.307+0000 7f9c3ce00640 1 --2- 172.21.15.83:0/242569812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c3806cfd0 0x7f9c38070d10 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9c280097c0 tx=0x7f9c2801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.307+0000 7f9c3ce00640 1 -- 172.21.15.83:0/242569812 shutdown_connections 2024-04-25T00:14:58.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.307+0000 7f9c3ce00640 1 --2- 172.21.15.83:0/242569812 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c3806d3b0 0x7f9c3806d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.307+0000 7f9c3ce00640 1 --2- 172.21.15.83:0/242569812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c3806cfd0 0x7f9c38070d10 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.307+0000 7f9c3ce00640 1 -- 172.21.15.83:0/242569812 >> 172.21.15.83:0/242569812 conn(0x7f9c380cfa60 msgr2=0x7f9c380cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:58.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.307+0000 7f9c3ce00640 1 -- 172.21.15.83:0/242569812 shutdown_connections 2024-04-25T00:14:58.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.307+0000 7f9c3ce00640 1 -- 172.21.15.83:0/242569812 wait complete. 2024-04-25T00:14:58.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.307+0000 7f9c3ce00640 1 Processor -- start 2024-04-25T00:14:58.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.307+0000 7f9c3ce00640 1 -- start start 2024-04-25T00:14:58.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c3ce00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c3806cfd0 0x7f9c38134da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c3ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c3806d3b0 0x7f9c381395d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c3ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c380ca630 0x7f9c3813c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c3ce00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c38168b60 con 0x7f9c380ca630 2024-04-25T00:14:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c3ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c38145970 con 0x7f9c3806d3b0 2024-04-25T00:14:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c3ce00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9c38168ce0 con 0x7f9c3806cfd0 2024-04-25T00:14:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c36800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c380ca630 0x7f9c3813c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c36800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c380ca630 0x7f9c3813c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51916/0 (socket says 172.21.15.83:51916) 2024-04-25T00:14:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c36800640 1 -- 172.21.15.83:0/944544899 learned_addr learned my addr 172.21.15.83:0/944544899 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c36800640 1 -- 172.21.15.83:0/944544899 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c3806cfd0 msgr2=0x7f9c38134da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c36800640 1 --2- 172.21.15.83:0/944544899 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c3806cfd0 0x7f9c38134da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c36800640 1 -- 172.21.15.83:0/944544899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c3806d3b0 msgr2=0x7f9c381395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c36800640 1 --2- 172.21.15.83:0/944544899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c3806d3b0 0x7f9c381395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c36800640 1 -- 172.21.15.83:0/944544899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c381380e0 con 0x7f9c380ca630 2024-04-25T00:14:58.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c36800640 1 --2- 172.21.15.83:0/944544899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c380ca630 0x7f9c3813c120 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c7a0 tx=0x7f9c3000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:58.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c2e000640 1 -- 172.21.15.83:0/944544899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30018070 con 0x7f9c380ca630 2024-04-25T00:14:58.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c2e000640 1 -- 172.21.15.83:0/944544899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c3000a640 con 0x7f9c380ca630 2024-04-25T00:14:58.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c2e000640 1 -- 172.21.15.83:0/944544899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000aab0 con 0x7f9c380ca630 2024-04-25T00:14:58.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c3ce00640 1 -- 172.21.15.83:0/944544899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c380b3c70 con 0x7f9c380ca630 2024-04-25T00:14:58.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c3ce00640 1 -- 172.21.15.83:0/944544899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c3813cc70 con 0x7f9c380ca630 2024-04-25T00:14:58.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.311+0000 7f9c2e000640 1 -- 172.21.15.83:0/944544899 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9c3001b070 con 0x7f9c380ca630 2024-04-25T00:14:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c2e000640 1 --2- 172.21.15.83:0/944544899 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9c180678c0 0x7f9c18069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c2e000640 1 -- 172.21.15.83:0/944544899 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f9c300853f0 con 0x7f9c380ca630 2024-04-25T00:14:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c35e00640 1 --2- 172.21.15.83:0/944544899 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9c180678c0 0x7f9c18069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 Processor -- start 2024-04-25T00:14:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 -- start start 2024-04-25T00:14:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c35e00640 1 --2- 172.21.15.83:0/944544899 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9c180678c0 0x7f9c18069d80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9c280ad000 tx=0x7f9c28004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:58.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 0x55e3434ee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 0x55e3434a4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e3432f25e0 con 0x55e3434a4bc0 2024-04-25T00:14:58.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e343279c50 con 0x55e3434ee810 2024-04-25T00:14:58.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55e3432ee5a0 con 0x55e3434edef0 2024-04-25T00:14:58.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c36800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c36800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51930/0 (socket says 172.21.15.83:51930) 2024-04-25T00:14:58.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c36800640 1 -- 172.21.15.83:0/2516324375 learned_addr learned my addr 172.21.15.83:0/2516324375 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:58.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c35e00640 1 --2- 172.21.15.83:0/2516324375 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 0x55e3434ee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c35400640 1 --2- 172.21.15.83:0/2516324375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 0x55e3434a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c36800640 1 -- 172.21.15.83:0/2516324375 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 msgr2=0x55e3434ee2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:58.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c36800640 1 --2- 172.21.15.83:0/2516324375 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 0x55e3434ee2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c36800640 1 -- 172.21.15.83:0/2516324375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 msgr2=0x55e3434a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:58.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c36800640 1 --2- 172.21.15.83:0/2516324375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 0x55e3434a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c36800640 1 -- 172.21.15.83:0/2516324375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e343277d10 con 0x55e3434a4bc0 2024-04-25T00:14:58.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c36800640 1 --2- 172.21.15.83:0/2516324375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434a4fa0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c770 tx=0x7f9c30055ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ee15d86c042a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:58.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c22a00640 1 -- 172.21.15.83:0/2516324375 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3005e280 con 0x55e3434a4bc0 2024-04-25T00:14:58.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c22a00640 1 -- 172.21.15.83:0/2516324375 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c300539a0 con 0x55e3434a4bc0 2024-04-25T00:14:58.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c22a00640 1 -- 172.21.15.83:0/2516324375 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30051d10 con 0x55e3434a4bc0 2024-04-25T00:14:58.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 -- 172.21.15.83:0/2516324375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 msgr2=0x55e3434a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 --2- 172.21.15.83:0/2516324375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434a4fa0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c770 tx=0x7f9c30055ee0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 -- 172.21.15.83:0/2516324375 shutdown_connections 2024-04-25T00:14:58.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 --2- 172.21.15.83:0/2516324375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434a4fa0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 --2- 172.21.15.83:0/2516324375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 0x55e3434a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 --2- 172.21.15.83:0/2516324375 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 0x55e3434ee2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.315+0000 7f9c402ad000 1 -- 172.21.15.83:0/2516324375 >> 172.21.15.83:0/2516324375 conn(0x55e3434a0b70 msgr2=0x55e3434a2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:58.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 1 -- 172.21.15.83:0/2516324375 shutdown_connections 2024-04-25T00:14:58.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 1 -- 172.21.15.83:0/2516324375 wait complete. 2024-04-25T00:14:58.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 1 -- start start 2024-04-25T00:14:58.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 1 Processor -- start 2024-04-25T00:14:58.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434d2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 0x55e3434d3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 0x55e3434d39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e3432ac520 con 0x55e3434a4bc0 2024-04-25T00:14:58.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e343298810 con 0x55e3434ee810 2024-04-25T00:14:58.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55e3434a09f0 con 0x55e3434edef0 2024-04-25T00:14:58.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 10 client.0 _ll_register_callbacks cb 0x55e34348f5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:58.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c35e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434d2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c35e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434d2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51944/0 (socket says 172.21.15.83:51944) 2024-04-25T00:14:58.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c35e00640 1 -- 172.21.15.83:0/3881430815 learned_addr learned my addr 172.21.15.83:0/3881430815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:58.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c36800640 1 --2- 172.21.15.83:0/3881430815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 0x55e3434d39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c35400640 1 --2- 172.21.15.83:0/3881430815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 0x55e3434d3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c35e00640 1 -- 172.21.15.83:0/3881430815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 msgr2=0x55e3434d3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:58.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c35e00640 1 --2- 172.21.15.83:0/3881430815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 0x55e3434d3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c35e00640 1 -- 172.21.15.83:0/3881430815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 msgr2=0x55e3434d39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c35e00640 1 --2- 172.21.15.83:0/3881430815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 0x55e3434d39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c35e00640 1 -- 172.21.15.83:0/3881430815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e34328a7b0 con 0x55e3434a4bc0 2024-04-25T00:14:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c35400640 1 --2- 172.21.15.83:0/3881430815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 0x55e3434d3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c35e00640 1 --2- 172.21.15.83:0/3881430815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434d2f20 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9c28009710 tx=0x7f9c280046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ffed72867ce6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c17400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:14:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c17400640 1 -- 172.21.15.83:0/3881430815 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c280b0040 con 0x55e3434a4bc0 2024-04-25T00:14:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 1 -- 172.21.15.83:0/3881430815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c300539a0 con 0x55e3434a4bc0 2024-04-25T00:14:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c17400640 1 -- 172.21.15.83:0/3881430815 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f9c280c1400 con 0x55e3434a4bc0 2024-04-25T00:14:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c17400640 1 -- 172.21.15.83:0/3881430815 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f9c280cc050 con 0x55e3434a4bc0 2024-04-25T00:14:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 10 client.7602 fetch_fsmap learned FSMap version 62 2024-04-25T00:14:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 1 -- 172.21.15.83:0/3881430815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55e343279ea0 con 0x55e3434a4bc0 2024-04-25T00:14:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c17400640 1 -- 172.21.15.83:0/3881430815 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c300539a0 con 0x55e3434a4bc0 2024-04-25T00:14:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c17400640 1 -- 172.21.15.83:0/3881430815 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f9c280cf020 con 0x55e3434a4bc0 2024-04-25T00:14:58.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 10 client.7602 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:14:58.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 10 client.7602 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T00:14:58.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 20 client.7602 populate_metadata read hostname 'smithi083' 2024-04-25T00:14:58.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 1 --2- 172.21.15.83:0/3881430815 >> [v2:172.21.15.86:6833/3612679148,v1:172.21.15.86:6835/3612679148] conn(0x55e3434fdf90 0x55e343500380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 4 client.7602 mds_command: new command op to 7467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:14:58.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.319+0000 7f9c402ad000 1 -- 172.21.15.83:0/3881430815 --> [v2:172.21.15.86:6833/3612679148,v1:172.21.15.86:6835/3612679148] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e34328dab0 con 0x55e3434fdf90 2024-04-25T00:14:58.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c35400640 1 --2- 172.21.15.83:0/3881430815 >> [v2:172.21.15.86:6833/3612679148,v1:172.21.15.86:6835/3612679148] conn(0x55e3434fdf90 0x55e343500380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c35400640 1 --2- 172.21.15.83:0/3881430815 >> [v2:172.21.15.86:6833/3612679148,v1:172.21.15.86:6835/3612679148] conn(0x55e3434fdf90 0x55e343500380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b25bc00b01cde852 server_cookie=e3fa5aca02d75860 in_seq=0 out_seq=0 2024-04-25T00:14:58.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c17400640 10 client.7602 ms_handle_connect on v2:172.21.15.86:6833/3612679148 2024-04-25T00:14:58.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c17400640 1 -- 172.21.15.83:0/3881430815 <== mds.0 v2:172.21.15.86:6833/3612679148 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e34328dab0 con 0x55e3434fdf90 2024-04-25T00:14:58.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c17400640 10 client.7602 handle_command_reply: tid=0 2024-04-25T00:14:58.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c402ad000 1 client.7602 shutdown 2024-04-25T00:14:58.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c402ad000 1 -- 172.21.15.83:0/3881430815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 msgr2=0x55e3434d2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c402ad000 1 --2- 172.21.15.83:0/3881430815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434d2f20 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9c28009710 tx=0x7f9c280046d0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c402ad000 1 -- 172.21.15.83:0/3881430815 shutdown_connections 2024-04-25T00:14:58.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c402ad000 1 --2- 172.21.15.83:0/3881430815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 0x55e3434d3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c402ad000 1 --2- 172.21.15.83:0/3881430815 >> [v2:172.21.15.86:6833/3612679148,v1:172.21.15.86:6835/3612679148] conn(0x55e3434fdf90 0x55e343500380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c402ad000 1 --2- 172.21.15.83:0/3881430815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 0x55e3434d39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c402ad000 1 --2- 172.21.15.83:0/3881430815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434d2f20 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c17400640 0 client.7602 ms_handle_reset on v2:172.21.15.86:6833/3612679148 2024-04-25T00:14:58.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c402ad000 1 -- 172.21.15.83:0/3881430815 >> 172.21.15.83:0/3881430815 conn(0x55e3434a0b70 msgr2=0x55e3434a37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:58.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c402ad000 1 -- 172.21.15.83:0/3881430815 shutdown_connections 2024-04-25T00:14:58.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c402ad000 1 -- 172.21.15.83:0/3881430815 wait complete. 2024-04-25T00:14:58.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.323+0000 7f9c402ad000 20 client.7602 trim_cache size 0 max 16384 2024-04-25T00:14:58.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.339+0000 7f9c402ad000 1 Processor -- start 2024-04-25T00:14:58.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.339+0000 7f9c402ad000 1 -- start start 2024-04-25T00:14:58.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 0x55e3434a34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 0x55e3434a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c38203b60 con 0x55e3434a4bc0 2024-04-25T00:14:58.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c38202b40 con 0x55e3434ee810 2024-04-25T00:14:58.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9c381fa970 con 0x55e3434edef0 2024-04-25T00:14:58.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c35e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c35e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434a2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51948/0 (socket says 172.21.15.83:51948) 2024-04-25T00:14:58.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c35e00640 1 -- 172.21.15.83:0/599125276 learned_addr learned my addr 172.21.15.83:0/599125276 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:58.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c36800640 1 --2- 172.21.15.83:0/599125276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 0x55e3434a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c35400640 1 --2- 172.21.15.83:0/599125276 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 0x55e3434a34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c35e00640 1 -- 172.21.15.83:0/599125276 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 msgr2=0x55e3434a34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:58.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c35e00640 1 --2- 172.21.15.83:0/599125276 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 0x55e3434a34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c35e00640 1 -- 172.21.15.83:0/599125276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 msgr2=0x55e3434a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:58.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c35e00640 1 --2- 172.21.15.83:0/599125276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 0x55e3434a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c35e00640 1 -- 172.21.15.83:0/599125276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e34327ae80 con 0x55e3434a4bc0 2024-04-25T00:14:58.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c35400640 1 --2- 172.21.15.83:0/599125276 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 0x55e3434a34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:58.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c35e00640 1 --2- 172.21.15.83:0/599125276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434a2f60 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f9c280096e0 tx=0x7f9c280b6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbfae9d03c4dd50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c22000640 1 -- 172.21.15.83:0/599125276 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c280b0070 con 0x55e3434a4bc0 2024-04-25T00:14:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c22000640 1 -- 172.21.15.83:0/599125276 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c280c1b50 con 0x55e3434a4bc0 2024-04-25T00:14:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c22000640 1 -- 172.21.15.83:0/599125276 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c280ab400 con 0x55e3434a4bc0 2024-04-25T00:14:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 -- 172.21.15.83:0/599125276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 msgr2=0x55e3434a2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 --2- 172.21.15.83:0/599125276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434a2f60 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f9c280096e0 tx=0x7f9c280b6000 comp rx=0 tx=0).stop 2024-04-25T00:14:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 -- 172.21.15.83:0/599125276 shutdown_connections 2024-04-25T00:14:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 --2- 172.21.15.83:0/599125276 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434edef0 0x55e3434a34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 --2- 172.21.15.83:0/599125276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434ee810 0x55e3434a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 --2- 172.21.15.83:0/599125276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434a4bc0 0x55e3434a2f60 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 -- 172.21.15.83:0/599125276 >> 172.21.15.83:0/599125276 conn(0x55e3434a0b70 msgr2=0x55e3434fb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 -- 172.21.15.83:0/599125276 shutdown_connections 2024-04-25T00:14:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 -- 172.21.15.83:0/599125276 wait complete. 2024-04-25T00:14:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 -- start start 2024-04-25T00:14:58.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.343+0000 7f9c402ad000 1 Processor -- start 2024-04-25T00:14:58.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434a4bc0 0x55e343566890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434edef0 0x55e343566dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434ee810 0x55e3435002d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e3433a4e80 con 0x55e3434ee810 2024-04-25T00:14:58.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c382024d0 con 0x55e3434edef0 2024-04-25T00:14:58.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55e3432a6c40 con 0x55e3434a4bc0 2024-04-25T00:14:58.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 10 client.0 _ll_register_callbacks cb 0x7f9c280ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:58.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c36800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434ee810 0x55e3435002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c36800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434ee810 0x55e3435002d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51962/0 (socket says 172.21.15.83:51962) 2024-04-25T00:14:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c36800640 1 -- 172.21.15.83:0/2646450031 learned_addr learned my addr 172.21.15.83:0/2646450031 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c36800640 1 -- 172.21.15.83:0/2646450031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434a4bc0 msgr2=0x55e343566890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c36800640 1 --2- 172.21.15.83:0/2646450031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434a4bc0 0x55e343566890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c36800640 1 -- 172.21.15.83:0/2646450031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434edef0 msgr2=0x55e343566dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c36800640 1 --2- 172.21.15.83:0/2646450031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434edef0 0x55e343566dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c36800640 1 -- 172.21.15.83:0/2646450031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3432bf760 con 0x55e3434ee810 2024-04-25T00:14:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c36800640 1 --2- 172.21.15.83:0/2646450031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434ee810 0x55e3435002d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c770 tx=0x7f9c30058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb9645ee9f5ff6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c17400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:14:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c17400640 1 -- 172.21.15.83:0/2646450031 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30051940 con 0x55e3434ee810 2024-04-25T00:14:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c17400640 1 -- 172.21.15.83:0/2646450031 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f9c30051d10 con 0x55e3434ee810 2024-04-25T00:14:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 -- 172.21.15.83:0/2646450031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c280c1b50 con 0x55e3434ee810 2024-04-25T00:14:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c17400640 1 -- 172.21.15.83:0/2646450031 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f9c3005e280 con 0x55e3434ee810 2024-04-25T00:14:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 10 client.7608 fetch_fsmap learned FSMap version 62 2024-04-25T00:14:58.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 -- 172.21.15.83:0/2646450031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55e34334b9e0 con 0x55e3434ee810 2024-04-25T00:14:58.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c17400640 1 -- 172.21.15.83:0/2646450031 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c280c1b50 con 0x55e3434ee810 2024-04-25T00:14:58.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c17400640 1 -- 172.21.15.83:0/2646450031 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f9c3000ac50 con 0x55e3434ee810 2024-04-25T00:14:58.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 10 client.7608 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:14:58.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 10 client.7608 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T00:14:58.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 20 client.7608 populate_metadata read hostname 'smithi083' 2024-04-25T00:14:58.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 --2- 172.21.15.83:0/2646450031 >> [v2:172.21.15.86:6833/3612679148,v1:172.21.15.86:6835/3612679148] conn(0x55e343500810 0x55e343500bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 4 client.7608 mds_command: new command op to 7467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:14:58.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 -- 172.21.15.83:0/2646450031 --> [v2:172.21.15.86:6833/3612679148,v1:172.21.15.86:6835/3612679148] -- command(tid 0: {"prefix": "status"}) -- 0x55e3432ffb20 con 0x55e343500810 2024-04-25T00:14:58.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c35e00640 1 --2- 172.21.15.83:0/2646450031 >> [v2:172.21.15.86:6833/3612679148,v1:172.21.15.86:6835/3612679148] conn(0x55e343500810 0x55e343500bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c35e00640 1 --2- 172.21.15.83:0/2646450031 >> [v2:172.21.15.86:6833/3612679148,v1:172.21.15.86:6835/3612679148] conn(0x55e343500810 0x55e343500bf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95da8087a0f0e2ec server_cookie=5ad1cbc61becdea3 in_seq=0 out_seq=0 2024-04-25T00:14:58.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c17400640 10 client.7608 ms_handle_connect on v2:172.21.15.86:6833/3612679148 2024-04-25T00:14:58.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c17400640 1 -- 172.21.15.83:0/2646450031 <== mds.0 v2:172.21.15.86:6833/3612679148 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55e3432ffb20 con 0x55e343500810 2024-04-25T00:14:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c17400640 10 client.7608 handle_command_reply: tid=0 2024-04-25T00:14:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 client.7608 shutdown 2024-04-25T00:14:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 -- 172.21.15.83:0/2646450031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434ee810 msgr2=0x55e3435002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 --2- 172.21.15.83:0/2646450031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434ee810 0x55e3435002d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c770 tx=0x7f9c30058040 comp rx=0 tx=0).stop 2024-04-25T00:14:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 -- 172.21.15.83:0/2646450031 shutdown_connections 2024-04-25T00:14:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 --2- 172.21.15.83:0/2646450031 >> [v2:172.21.15.86:6833/3612679148,v1:172.21.15.86:6835/3612679148] conn(0x55e343500810 0x55e343500bf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 --2- 172.21.15.83:0/2646450031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3434ee810 0x55e3435002d0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 --2- 172.21.15.83:0/2646450031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3434edef0 0x55e343566dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 --2- 172.21.15.83:0/2646450031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e3434a4bc0 0x55e343566890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c17400640 0 client.7608 ms_handle_reset on v2:172.21.15.86:6833/3612679148 2024-04-25T00:14:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 -- 172.21.15.83:0/2646450031 >> 172.21.15.83:0/2646450031 conn(0x55e3434a0b70 msgr2=0x55e3434ec7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 -- 172.21.15.83:0/2646450031 shutdown_connections 2024-04-25T00:14:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 1 -- 172.21.15.83:0/2646450031 wait complete. 2024-04-25T00:14:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.347+0000 7f9c402ad000 20 client.7608 trim_cache size 0 max 16384 2024-04-25T00:14:58.358 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:14:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "d58ae83a-ad72-4e18-824a-86e779979c35", 2024-04-25T00:14:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-25T00:14:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7467, 2024-04-25T00:14:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T00:14:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T00:14:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T00:14:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.8358585600000001, 2024-04-25T00:14:58.358 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 62, 2024-04-25T00:14:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 93, 2024-04-25T00:14:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 93, 2024-04-25T00:14:58.359 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 10.64737702 2024-04-25T00:14:58.359 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:14:58.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.351+0000 7f9c3ce00640 1 -- 172.21.15.83:0/944544899 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9c180678c0 msgr2=0x7f9c18069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.351+0000 7f9c3ce00640 1 --2- 172.21.15.83:0/944544899 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9c180678c0 0x7f9c18069d80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9c280ad000 tx=0x7f9c28004740 comp rx=0 tx=0).stop 2024-04-25T00:14:58.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.351+0000 7f9c3ce00640 1 -- 172.21.15.83:0/944544899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c380ca630 msgr2=0x7f9c3813c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.351+0000 7f9c3ce00640 1 --2- 172.21.15.83:0/944544899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c380ca630 0x7f9c3813c120 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c7a0 tx=0x7f9c3000cc60 comp rx=0 tx=0).stop 2024-04-25T00:14:58.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.351+0000 7f9c3ce00640 1 -- 172.21.15.83:0/944544899 shutdown_connections 2024-04-25T00:14:58.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.351+0000 7f9c3ce00640 1 --2- 172.21.15.83:0/944544899 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9c180678c0 0x7f9c18069d80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.351+0000 7f9c3ce00640 1 --2- 172.21.15.83:0/944544899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c380ca630 0x7f9c3813c120 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.351+0000 7f9c3ce00640 1 --2- 172.21.15.83:0/944544899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c3806d3b0 0x7f9c381395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.351+0000 7f9c3ce00640 1 --2- 172.21.15.83:0/944544899 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c3806cfd0 0x7f9c38134da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.351+0000 7f9c3ce00640 1 -- 172.21.15.83:0/944544899 >> 172.21.15.83:0/944544899 conn(0x7f9c380cfa60 msgr2=0x7f9c3813a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:58.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.351+0000 7f9c3ce00640 1 -- 172.21.15.83:0/944544899 shutdown_connections 2024-04-25T00:14:58.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.351+0000 7f9c3ce00640 1 -- 172.21.15.83:0/944544899 wait complete. 2024-04-25T00:14:58.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:14:58.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4292e00640 1 Processor -- start 2024-04-25T00:14:58.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4292e00640 1 -- start start 2024-04-25T00:14:58.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4292e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f428c1618d0 0x7f428c0ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4292e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f428c0d1930 0x7f428c0ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4292e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f428c0cf070 0x7f428c06c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4292e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f428c0d11d0 con 0x7f428c0cf070 2024-04-25T00:14:58.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4292e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f428c161cb0 con 0x7f428c1618d0 2024-04-25T00:14:58.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4292e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f428c0d0970 con 0x7f428c0d1930 2024-04-25T00:14:58.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4290e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f428c0cf070 0x7f428c06c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4290e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f428c0cf070 0x7f428c06c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51970/0 (socket says 172.21.15.83:51970) 2024-04-25T00:14:58.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4290e00640 1 -- 172.21.15.83:0/2267739945 learned_addr learned my addr 172.21.15.83:0/2267739945 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:58.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f428b400640 1 --2- 172.21.15.83:0/2267739945 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f428c0d1930 0x7f428c0ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f428be00640 1 --2- 172.21.15.83:0/2267739945 >> v2:172.21.15.86:3300/0 conn(0x7f428c1618d0 0x7f428c0ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4290e00640 1 -- 172.21.15.83:0/2267739945 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f428c0d1930 msgr2=0x7f428c0ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:58.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4290e00640 1 --2- 172.21.15.83:0/2267739945 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f428c0d1930 0x7f428c0ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4290e00640 1 -- 172.21.15.83:0/2267739945 >> v2:172.21.15.86:3300/0 conn(0x7f428c1618d0 msgr2=0x7f428c0ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:58.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4290e00640 1 --2- 172.21.15.83:0/2267739945 >> v2:172.21.15.86:3300/0 conn(0x7f428c1618d0 0x7f428c0ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.503+0000 7f4290e00640 1 -- 172.21.15.83:0/2267739945 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c0d0c10 con 0x7f428c0cf070 2024-04-25T00:14:58.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.507+0000 7f4290e00640 1 --2- 172.21.15.83:0/2267739945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c0cf070 0x7f428c06c2f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f428c071cf0 tx=0x7f428401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e68af8b5c1781ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:58.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.507+0000 7f428aa00640 1 -- 172.21.15.83:0/2267739945 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42840aa430 con 0x7f428c0cf070 2024-04-25T00:14:58.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.507+0000 7f428aa00640 1 -- 172.21.15.83:0/2267739945 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42840af070 con 0x7f428c0cf070 2024-04-25T00:14:58.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.507+0000 7f428aa00640 1 -- 172.21.15.83:0/2267739945 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42840aa7f0 con 0x7f428c0cf070 2024-04-25T00:14:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.507+0000 7f4292e00640 1 -- 172.21.15.83:0/2267739945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c0cf070 msgr2=0x7f428c06c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.507+0000 7f4292e00640 1 --2- 172.21.15.83:0/2267739945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c0cf070 0x7f428c06c2f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f428c071cf0 tx=0x7f428401c4f0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.507+0000 7f4292e00640 1 -- 172.21.15.83:0/2267739945 shutdown_connections 2024-04-25T00:14:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.507+0000 7f4292e00640 1 --2- 172.21.15.83:0/2267739945 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f428c0d1930 0x7f428c0ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.507+0000 7f4292e00640 1 --2- 172.21.15.83:0/2267739945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c0cf070 0x7f428c06c2f0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.507+0000 7f4292e00640 1 --2- 172.21.15.83:0/2267739945 >> v2:172.21.15.86:3300/0 conn(0x7f428c1618d0 0x7f428c0ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.507+0000 7f4292e00640 1 -- 172.21.15.83:0/2267739945 >> 172.21.15.83:0/2267739945 conn(0x7f428c0d1d70 msgr2=0x7f428c0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.507+0000 7f4292e00640 1 -- 172.21.15.83:0/2267739945 shutdown_connections 2024-04-25T00:14:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.507+0000 7f4292e00640 1 -- 172.21.15.83:0/2267739945 wait complete. 2024-04-25T00:14:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4292e00640 1 Processor -- start 2024-04-25T00:14:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4292e00640 1 -- start start 2024-04-25T00:14:58.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4292e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f428c0cf070 0x7f428c1510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4292e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f428c0d1930 0x7f428c152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4292e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c1618d0 0x7f428c1584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4292e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f428c0ccac0 con 0x7f428c1618d0 2024-04-25T00:14:58.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4292e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f428c0cc940 con 0x7f428c0d1930 2024-04-25T00:14:58.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4292e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f428c0ccc40 con 0x7f428c0cf070 2024-04-25T00:14:58.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4290e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c1618d0 0x7f428c1584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4290e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c1618d0 0x7f428c1584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51972/0 (socket says 172.21.15.83:51972) 2024-04-25T00:14:58.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4290e00640 1 -- 172.21.15.83:0/564015100 learned_addr learned my addr 172.21.15.83:0/564015100 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:58.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4290e00640 1 -- 172.21.15.83:0/564015100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f428c0cf070 msgr2=0x7f428c1510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4290e00640 1 --2- 172.21.15.83:0/564015100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f428c0cf070 0x7f428c1510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4290e00640 1 -- 172.21.15.83:0/564015100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f428c0d1930 msgr2=0x7f428c152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f428b400640 1 --2- 172.21.15.83:0/564015100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f428c0d1930 0x7f428c152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4290e00640 1 --2- 172.21.15.83:0/564015100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f428c0d1930 0x7f428c152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4290e00640 1 -- 172.21.15.83:0/564015100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c0ca310 con 0x7f428c1618d0 2024-04-25T00:14:58.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f428b400640 1 --2- 172.21.15.83:0/564015100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f428c0d1930 0x7f428c152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:58.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4290e00640 1 --2- 172.21.15.83:0/564015100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c1618d0 0x7f428c1584b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f42840ac040 tx=0x7f428401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:58.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4283e00640 1 -- 172.21.15.83:0/564015100 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42840bd3f0 con 0x7f428c1618d0 2024-04-25T00:14:58.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4283e00640 1 -- 172.21.15.83:0/564015100 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42840bd5b0 con 0x7f428c1618d0 2024-04-25T00:14:58.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4283e00640 1 -- 172.21.15.83:0/564015100 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42840bd9a0 con 0x7f428c1618d0 2024-04-25T00:14:58.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4292e00640 1 -- 172.21.15.83:0/564015100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428c0ca5f0 con 0x7f428c1618d0 2024-04-25T00:14:58.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.511+0000 7f4292e00640 1 -- 172.21.15.83:0/564015100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428c0cabb0 con 0x7f428c1618d0 2024-04-25T00:14:58.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.515+0000 7f4292e00640 1 -- 172.21.15.83:0/564015100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428c09fdd0 con 0x7f428c1618d0 2024-04-25T00:14:58.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.515+0000 7f4283e00640 1 -- 172.21.15.83:0/564015100 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f42840b33f0 con 0x7f428c1618d0 2024-04-25T00:14:58.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.515+0000 7f4283e00640 1 --2- 172.21.15.83:0/564015100 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f426c0678c0 0x7f426c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:58.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.515+0000 7f4283e00640 1 -- 172.21.15.83:0/564015100 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f42841200c0 con 0x7f428c1618d0 2024-04-25T00:14:58.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.515+0000 7f428be00640 1 --2- 172.21.15.83:0/564015100 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f426c0678c0 0x7f426c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.515+0000 7f428be00640 1 --2- 172.21.15.83:0/564015100 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f426c0678c0 0x7f426c069d80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f427c009fa0 tx=0x7f427c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:58.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.519+0000 7f4283e00640 1 -- 172.21.15.83:0/564015100 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f42840c3060 con 0x7f428c1618d0 2024-04-25T00:14:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.707+0000 7f4292e00640 1 -- 172.21.15.83:0/564015100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f428c0b0670 con 0x7f428c1618d0 2024-04-25T00:14:58.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.707+0000 7f4283e00640 1 -- 172.21.15.83:0/564015100 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) ==== 53+0+73 (secure 0 0 0) 0x7f4284016110 con 0x7f428c1618d0 2024-04-25T00:14:58.713 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:14:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.711+0000 7f4292e00640 1 -- 172.21.15.83:0/564015100 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f426c0678c0 msgr2=0x7f426c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.711+0000 7f4292e00640 1 --2- 172.21.15.83:0/564015100 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f426c0678c0 0x7f426c069d80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f427c009fa0 tx=0x7f427c00b040 comp rx=0 tx=0).stop 2024-04-25T00:14:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.711+0000 7f4292e00640 1 -- 172.21.15.83:0/564015100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c1618d0 msgr2=0x7f428c1584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.711+0000 7f4292e00640 1 --2- 172.21.15.83:0/564015100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c1618d0 0x7f428c1584b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f42840ac040 tx=0x7f428401ad30 comp rx=0 tx=0).stop 2024-04-25T00:14:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.711+0000 7f4292e00640 1 -- 172.21.15.83:0/564015100 shutdown_connections 2024-04-25T00:14:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.711+0000 7f4292e00640 1 --2- 172.21.15.83:0/564015100 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f426c0678c0 0x7f426c069d80 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.711+0000 7f4292e00640 1 --2- 172.21.15.83:0/564015100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c1618d0 0x7f428c1584b0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.711+0000 7f4292e00640 1 --2- 172.21.15.83:0/564015100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f428c0d1930 0x7f428c152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.711+0000 7f4292e00640 1 --2- 172.21.15.83:0/564015100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f428c0cf070 0x7f428c1510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.711+0000 7f4292e00640 1 -- 172.21.15.83:0/564015100 >> 172.21.15.83:0/564015100 conn(0x7f428c0d1d70 msgr2=0x7f428c06d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.711+0000 7f4292e00640 1 -- 172.21.15.83:0/564015100 shutdown_connections 2024-04-25T00:14:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58.711+0000 7f4292e00640 1 -- 172.21.15.83:0/564015100 wait complete. 2024-04-25T00:14:58.734 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:14:58.734 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:14:58.734 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-25T00:14:58.734 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:14:58.734 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:14:58.734 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:14:58.735 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:14:58.735 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:14:58.735 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:14:58.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:58.735 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-25T00:14:58.744 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:14:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:14:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:14:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-25T00:14:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:58.744 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-25T00:14:58.744 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-25T00:14:58.745 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:58.745 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:14:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:14:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42236sec preferred_lft 42236sec 2024-04-25T00:14:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-25T00:14:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:58.745 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:14:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:8d:d4:7d:b9 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:14:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:58.745 INFO:teuthology.orchestra.run.smithi083.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:14:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:14:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:14:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:58.746 INFO:teuthology.orchestra.run.smithi083.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:14:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:14:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:14:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:58.746 INFO:teuthology.orchestra.run.smithi083.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:14:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:14:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:14:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:58.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:58.747 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:14:58.810 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-25T00:14:58.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:58.812 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-25T00:14:58.867 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-25T00:14:58.867 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:14:58.868 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:14:58.868 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:14:58.868 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:14:58.868 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:14:58.868 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:14:58.868 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:58.868 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:58.868 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:58.868 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:14:58.869 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:14:58.869 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:14:58.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:14:58.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:14:58.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:14:58.869 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:14:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:14:58.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:59.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:59.001 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:14:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:14:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:14:59.011 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:14:59.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:59.011 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-25T00:14:59.072 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:14:59.072 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:14:59.124 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:14:59.125 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:14:59.125 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:14:59.176 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T00:14:59.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:59.177 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:14:59.264 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:14:59.264 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:14:59.279 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:14:59.279 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:14:59.279 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:14:59.279 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:14:59.279 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:14:59.283 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:14:59.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:59.284 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:14:59.305 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:14:59.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:14:59.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:59.416 INFO:teuthology.orchestra.run.smithi083.stdout:client6380 2024-04-25T00:14:59.417 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:14:59.417 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:59.425 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:14:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.602+0000 7f7aed600640 1 Processor -- start 2024-04-25T00:14:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.602+0000 7f7aed600640 1 -- start start 2024-04-25T00:14:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.602+0000 7f7aed600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae81599f0 0x7f7ae80cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.602+0000 7f7aed600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ae80ccc60 con 0x7f7ae815c340 2024-04-25T00:14:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.602+0000 7f7aed600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ae806dba0 con 0x7f7ae815ce90 2024-04-25T00:14:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.602+0000 7f7aed600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae8159dd0 con 0x7f7ae81599f0 2024-04-25T00:14:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.602+0000 7f7ae6000640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7ae815c340 0x7f7ae815c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51368/0 (socket says 172.21.15.83:51368) 2024-04-25T00:14:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.602+0000 7f7ae6000640 1 -- 172.21.15.83:0/2504684324 learned_addr learned my addr 172.21.15.83:0/2504684324 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:14:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.606+0000 7f7ae6a00640 1 --2- 172.21.15.83:0/2504684324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae81599f0 0x7f7ae80cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.606+0000 7f7ae5600640 1 -- 172.21.15.83:0/2504684324 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1666950270 0 0) 0x7f7ad4001120 con 0x7f7ae815c340 2024-04-25T00:14:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.606+0000 7f7ae5600640 1 -- 172.21.15.83:0/2504684324 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7acc003530 con 0x7f7ae815c340 2024-04-25T00:14:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.606+0000 7f7ae5600640 1 -- 172.21.15.83:0/2504684324 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3562884737 0 0) 0x7f7ae806dba0 con 0x7f7ae815ce90 2024-04-25T00:14:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.606+0000 7f7ae5600640 1 -- 172.21.15.83:0/2504684324 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ad4001120 con 0x7f7ae815ce90 2024-04-25T00:14:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.606+0000 7f7ae6a00640 1 -- 172.21.15.83:0/2504684324 >> v1:172.21.15.86:6789/0 conn(0x7f7ae815ce90 legacy=0x7f7ae80ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.606+0000 7f7ae6a00640 1 -- 172.21.15.83:0/2504684324 >> v1:172.21.15.83:6789/0 conn(0x7f7ae815c340 legacy=0x7f7ae815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.606+0000 7f7ae6a00640 1 -- 172.21.15.83:0/2504684324 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae806de80 con 0x7f7ae81599f0 2024-04-25T00:14:59.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.606+0000 7f7ae6a00640 1 --2- 172.21.15.83:0/2504684324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae81599f0 0x7f7ae80cc510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009df0 tx=0x7f7ad801c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ea784d8759d9ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:59.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.606+0000 7f7ae5600640 1 -- 172.21.15.83:0/2504684324 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80b2070 con 0x7f7ae81599f0 2024-04-25T00:14:59.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.606+0000 7f7ae5600640 1 -- 172.21.15.83:0/2504684324 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad80a58d0 con 0x7f7ae81599f0 2024-04-25T00:14:59.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.606+0000 7f7ae5600640 1 -- 172.21.15.83:0/2504684324 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80a5ca0 con 0x7f7ae81599f0 2024-04-25T00:14:59.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 -- 172.21.15.83:0/2504684324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae81599f0 msgr2=0x7f7ae80cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:59.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 --2- 172.21.15.83:0/2504684324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae81599f0 0x7f7ae80cc510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009df0 tx=0x7f7ad801c900 comp rx=0 tx=0).stop 2024-04-25T00:14:59.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 -- 172.21.15.83:0/2504684324 shutdown_connections 2024-04-25T00:14:59.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 --2- 172.21.15.83:0/2504684324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae81599f0 0x7f7ae80cc510 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:59.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 -- 172.21.15.83:0/2504684324 >> 172.21.15.83:0/2504684324 conn(0x7f7ae80b1830 msgr2=0x7f7ae80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:59.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 -- 172.21.15.83:0/2504684324 shutdown_connections 2024-04-25T00:14:59.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 -- 172.21.15.83:0/2504684324 wait complete. 2024-04-25T00:14:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 Processor -- start 2024-04-25T00:14:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 -- start start 2024-04-25T00:14:59.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae81599f0 0x7f7ae8159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:59.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ae815c340 0x7f7ae815f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:59.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae815ce90 0x7f7ae815fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:59.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae8169ce0 con 0x7f7ae815c340 2024-04-25T00:14:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae80cf850 con 0x7f7ae815ce90 2024-04-25T00:14:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7aed600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae8169e60 con 0x7f7ae81599f0 2024-04-25T00:14:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7ae6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ae815c340 0x7f7ae815f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7ae6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ae815c340 0x7f7ae815f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51988/0 (socket says 172.21.15.83:51988) 2024-04-25T00:14:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7ae6000640 1 -- 172.21.15.83:0/1923153251 learned_addr learned my addr 172.21.15.83:0/1923153251 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:14:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7ae6a00640 1 --2- 172.21.15.83:0/1923153251 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae81599f0 0x7f7ae8159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7ae7400640 1 --2- 172.21.15.83:0/1923153251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae815ce90 0x7f7ae815fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7ae6000640 1 -- 172.21.15.83:0/1923153251 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae81599f0 msgr2=0x7f7ae8159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7ae6000640 1 --2- 172.21.15.83:0/1923153251 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae81599f0 0x7f7ae8159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7ae6000640 1 -- 172.21.15.83:0/1923153251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae815ce90 msgr2=0x7f7ae815fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7ae6000640 1 --2- 172.21.15.83:0/1923153251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae815ce90 0x7f7ae815fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7ae6000640 1 -- 172.21.15.83:0/1923153251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae815a540 con 0x7f7ae815c340 2024-04-25T00:14:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7ae6000640 1 --2- 172.21.15.83:0/1923153251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ae815c340 0x7f7ae815f8c0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400c690 tx=0x7f7ad400cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:59.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.610+0000 7f7adea00640 1 -- 172.21.15.83:0/1923153251 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4017070 con 0x7f7ae815c340 2024-04-25T00:14:59.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.614+0000 7f7adea00640 1 -- 172.21.15.83:0/1923153251 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad4005ae0 con 0x7f7ae815c340 2024-04-25T00:14:59.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.614+0000 7f7adea00640 1 -- 172.21.15.83:0/1923153251 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4007070 con 0x7f7ae815c340 2024-04-25T00:14:59.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.614+0000 7f7aed600640 1 -- 172.21.15.83:0/1923153251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae815a820 con 0x7f7ae815c340 2024-04-25T00:14:59.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.614+0000 7f7aed600640 1 -- 172.21.15.83:0/1923153251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae815ade0 con 0x7f7ae815c340 2024-04-25T00:14:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.614+0000 7f7aed600640 1 -- 172.21.15.83:0/1923153251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae804f5f0 con 0x7f7ae815c340 2024-04-25T00:14:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.614+0000 7f7adea00640 1 -- 172.21.15.83:0/1923153251 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7ad401a070 con 0x7f7ae815c340 2024-04-25T00:14:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.614+0000 7f7adea00640 1 --2- 172.21.15.83:0/1923153251 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ac80678c0 0x7f7ac8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.614+0000 7f7adea00640 1 -- 172.21.15.83:0/1923153251 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f7ad4085550 con 0x7f7ae815c340 2024-04-25T00:14:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.614+0000 7f7ae6a00640 1 --2- 172.21.15.83:0/1923153251 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ac80678c0 0x7f7ac8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.618+0000 7f7ae6a00640 1 --2- 172.21.15.83:0/1923153251 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ac80678c0 0x7f7ac8069d80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7ad80af000 tx=0x7f7ad80a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:59.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.618+0000 7f7adea00640 1 -- 172.21.15.83:0/1923153251 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7ae80ccc60 con 0x7f7ae815c340 2024-04-25T00:14:59.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.806+0000 7f7aed600640 1 -- 172.21.15.83:0/1923153251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7ae8072c40 con 0x7f7ae815c340 2024-04-25T00:14:59.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.810+0000 7f7adea00640 1 -- 172.21.15.83:0/1923153251 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) ==== 53+0+73 (secure 0 0 0) 0x7f7ad4055dd0 con 0x7f7ae815c340 2024-04-25T00:14:59.813 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:14:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.810+0000 7f7aed600640 1 -- 172.21.15.83:0/1923153251 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ac80678c0 msgr2=0x7f7ac8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.810+0000 7f7aed600640 1 --2- 172.21.15.83:0/1923153251 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ac80678c0 0x7f7ac8069d80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7ad80af000 tx=0x7f7ad80a7070 comp rx=0 tx=0).stop 2024-04-25T00:14:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.810+0000 7f7aed600640 1 -- 172.21.15.83:0/1923153251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ae815c340 msgr2=0x7f7ae815f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.810+0000 7f7aed600640 1 --2- 172.21.15.83:0/1923153251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ae815c340 0x7f7ae815f8c0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400c690 tx=0x7f7ad400cb50 comp rx=0 tx=0).stop 2024-04-25T00:14:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.810+0000 7f7aed600640 1 -- 172.21.15.83:0/1923153251 shutdown_connections 2024-04-25T00:14:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.810+0000 7f7aed600640 1 --2- 172.21.15.83:0/1923153251 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7ac80678c0 0x7f7ac8069d80 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.810+0000 7f7aed600640 1 --2- 172.21.15.83:0/1923153251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae815ce90 0x7f7ae815fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.810+0000 7f7aed600640 1 --2- 172.21.15.83:0/1923153251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ae815c340 0x7f7ae815f8c0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.810+0000 7f7aed600640 1 --2- 172.21.15.83:0/1923153251 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ae81599f0 0x7f7ae8159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.810+0000 7f7aed600640 1 -- 172.21.15.83:0/1923153251 >> 172.21.15.83:0/1923153251 conn(0x7f7ae80b1830 msgr2=0x7f7ae80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.810+0000 7f7aed600640 1 -- 172.21.15.83:0/1923153251 shutdown_connections 2024-04-25T00:14:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:14:59.810+0000 7f7aed600640 1 -- 172.21.15.83:0/1923153251 wait complete. 2024-04-25T00:14:59.834 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:14:59.835 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:14:59.835 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T00:14:59.835 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:14:59.835 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:14:59.835 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:14:59.835 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:14:59.835 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:14:59.835 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:14:59.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:59.835 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T00:14:59.845 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:14:59.845 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:14:59.845 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:14:59.845 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:59.846 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T00:14:59.846 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:59.846 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:59.846 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:59.846 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T00:14:59.846 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:59.846 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:59.846 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T00:14:59.846 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:14:59.846 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:59.846 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:14:59.846 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:59.846 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:14:59.847 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42229sec preferred_lft 42229sec 2024-04-25T00:14:59.847 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T00:14:59.847 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:59.847 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:14:59.847 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:35:63:ea:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:59.847 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:14:59.847 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:59.847 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:14:59.847 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:59.847 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:14:59.847 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:59.847 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:14:59.847 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:59.847 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:14:59.848 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:14:59.848 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:14:59.848 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:59.848 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:14:59.848 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:14:59.848 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:14:59.848 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:59.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:59.849 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:14:59.912 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T00:14:59.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:59.915 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T00:14:59.969 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T00:14:59.969 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:14:59.969 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:14:59.969 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:14:59.969 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:14:59.969 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:14:59.969 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:14:59.969 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:59.969 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:59.969 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:14:59.969 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:14:59.970 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:14:59.970 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:14:59.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:14:59.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:14:59.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:14:59.970 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:15:00.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:15:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:15:00.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:15:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:15:00.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:00.116 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T00:15:00.125 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:15:00.125 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:15:00.126 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:15:00.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:00.127 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T00:15:00.192 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:15:00.192 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:15:00.245 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:15:00.245 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:15:00.246 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:15:00.296 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T00:15:00.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:00.296 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:15:00.385 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:15:00.385 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:15:00.397 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:15:00.397 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:15:00.397 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:15:00.398 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with new device syntax: 1@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:15:00.398 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:15:00.401 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:15:00.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:00.402 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:15:00.422 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:15:00.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:15:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:15:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:15:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:15:00.526 INFO:teuthology.orchestra.run.smithi086.stdout:client6389 2024-04-25T00:15:00.526 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:15:00.526 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:00.535 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:15:00.536 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:00.545 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:15:00.545 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:00.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:15:00.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4443a00640 1 Processor -- start 2024-04-25T00:15:00.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4443a00640 1 -- start start 2024-04-25T00:15:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4443a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f443c15e2c0 0x7f443c0715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4443a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f443c0b4bc0 0x7f443c071b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4443a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f443c0aee90 con 0x7f443c15dee0 2024-04-25T00:15:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4443a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f443c0af7c0 con 0x7f443c15e2c0 2024-04-25T00:15:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4443a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f443c0b42d0 con 0x7f443c0b4bc0 2024-04-25T00:15:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4433e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f443c15e2c0 0x7f443c0715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4433e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f443c15e2c0 0x7f443c0715c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51992/0 (socket says 172.21.15.83:51992) 2024-04-25T00:15:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4433e00640 1 -- 172.21.15.83:0/1905734510 learned_addr learned my addr 172.21.15.83:0/1905734510 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4441a00640 1 --2- 172.21.15.83:0/1905734510 >> v2:172.21.15.86:3300/0 conn(0x7f443c0b4bc0 0x7f443c071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4433e00640 1 -- 172.21.15.83:0/1905734510 >> v2:172.21.15.86:3300/0 conn(0x7f443c0b4bc0 msgr2=0x7f443c071b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4433e00640 1 --2- 172.21.15.83:0/1905734510 >> v2:172.21.15.86:3300/0 conn(0x7f443c0b4bc0 0x7f443c071b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4433e00640 1 -- 172.21.15.83:0/1905734510 >> v1:172.21.15.86:6789/0 conn(0x7f443c15dee0 legacy=0x7f443c0b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:00.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.782+0000 7f4433e00640 1 -- 172.21.15.83:0/1905734510 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c0af170 con 0x7f443c15e2c0 2024-04-25T00:15:00.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.786+0000 7f4433e00640 1 --2- 172.21.15.83:0/1905734510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f443c15e2c0 0x7f443c0715c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f443c0ad860 tx=0x7f442401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c815ef2d0fc19b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.786+0000 7f443be00640 1 -- 172.21.15.83:0/1905734510 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44240b0070 con 0x7f443c15e2c0 2024-04-25T00:15:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.786+0000 7f443be00640 1 -- 172.21.15.83:0/1905734510 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44240ab600 con 0x7f443c15e2c0 2024-04-25T00:15:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.786+0000 7f443be00640 1 -- 172.21.15.83:0/1905734510 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44240ab9d0 con 0x7f443c15e2c0 2024-04-25T00:15:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.786+0000 7f4443a00640 1 -- 172.21.15.83:0/1905734510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f443c15e2c0 msgr2=0x7f443c0715c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.786+0000 7f4443a00640 1 --2- 172.21.15.83:0/1905734510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f443c15e2c0 0x7f443c0715c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f443c0ad860 tx=0x7f442401c3d0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.786+0000 7f4443a00640 1 -- 172.21.15.83:0/1905734510 shutdown_connections 2024-04-25T00:15:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.786+0000 7f4443a00640 1 --2- 172.21.15.83:0/1905734510 >> v2:172.21.15.86:3300/0 conn(0x7f443c0b4bc0 0x7f443c071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.786+0000 7f4443a00640 1 --2- 172.21.15.83:0/1905734510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f443c15e2c0 0x7f443c0715c0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.786+0000 7f4443a00640 1 -- 172.21.15.83:0/1905734510 >> 172.21.15.83:0/1905734510 conn(0x7f443c06b1e0 msgr2=0x7f443c06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.786+0000 7f4443a00640 1 -- 172.21.15.83:0/1905734510 shutdown_connections 2024-04-25T00:15:00.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.786+0000 7f4443a00640 1 -- 172.21.15.83:0/1905734510 wait complete. 2024-04-25T00:15:00.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4443a00640 1 Processor -- start 2024-04-25T00:15:00.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4443a00640 1 -- start start 2024-04-25T00:15:00.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4443a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f443c0b4bc0 0x7f443c0cf960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:00.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4443a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f443c15dee0 0x7f443c0b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:00.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4443a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f443c0b1b10 0x7f443c151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:00.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4443a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f443c06e1c0 con 0x7f443c0b1b10 2024-04-25T00:15:00.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4443a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f443c06e040 con 0x7f443c15dee0 2024-04-25T00:15:00.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4443a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f443c06e340 con 0x7f443c0b4bc0 2024-04-25T00:15:00.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4441a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f443c0b1b10 0x7f443c151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:00.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4441a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f443c0b1b10 0x7f443c151380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51994/0 (socket says 172.21.15.83:51994) 2024-04-25T00:15:00.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4441a00640 1 -- 172.21.15.83:0/2293940970 learned_addr learned my addr 172.21.15.83:0/2293940970 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:00.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4441000640 1 --2- 172.21.15.83:0/2293940970 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f443c0b4bc0 0x7f443c0cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:00.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4433e00640 1 --2- 172.21.15.83:0/2293940970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f443c15dee0 0x7f443c0b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:00.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4441a00640 1 -- 172.21.15.83:0/2293940970 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f443c0b4bc0 msgr2=0x7f443c0cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:00.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4441a00640 1 --2- 172.21.15.83:0/2293940970 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f443c0b4bc0 0x7f443c0cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4441a00640 1 -- 172.21.15.83:0/2293940970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f443c15dee0 msgr2=0x7f443c0b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:00.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4441a00640 1 --2- 172.21.15.83:0/2293940970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f443c15dee0 0x7f443c0b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4441a00640 1 -- 172.21.15.83:0/2293940970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c153370 con 0x7f443c0b1b10 2024-04-25T00:15:00.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4441a00640 1 --2- 172.21.15.83:0/2293940970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f443c0b1b10 0x7f443c151380 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f4434007e60 tx=0x7f443400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:00.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4439600640 1 -- 172.21.15.83:0/2293940970 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434014620 con 0x7f443c0b1b10 2024-04-25T00:15:00.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4439600640 1 -- 172.21.15.83:0/2293940970 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4434019070 con 0x7f443c0b1b10 2024-04-25T00:15:00.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4439600640 1 -- 172.21.15.83:0/2293940970 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434014a00 con 0x7f443c0b1b10 2024-04-25T00:15:00.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4443a00640 1 -- 172.21.15.83:0/2293940970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443c1535e0 con 0x7f443c0b1b10 2024-04-25T00:15:00.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4443a00640 1 -- 172.21.15.83:0/2293940970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443c153ba0 con 0x7f443c0b1b10 2024-04-25T00:15:00.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.790+0000 7f4443a00640 1 -- 172.21.15.83:0/2293940970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f443c153de0 con 0x7f443c0b1b10 2024-04-25T00:15:00.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.794+0000 7f4439600640 1 -- 172.21.15.83:0/2293940970 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4434014ba0 con 0x7f443c0b1b10 2024-04-25T00:15:00.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.794+0000 7f4439600640 1 --2- 172.21.15.83:0/2293940970 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44180678f0 0x7f4418069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:00.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.794+0000 7f4439600640 1 -- 172.21.15.83:0/2293940970 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f4434086790 con 0x7f443c0b1b10 2024-04-25T00:15:00.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.794+0000 7f4441000640 1 --2- 172.21.15.83:0/2293940970 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44180678f0 0x7f4418069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:00.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.794+0000 7f4441000640 1 --2- 172.21.15.83:0/2293940970 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44180678f0 0x7f4418069db0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f443c15d690 tx=0x7f442c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:00.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.798+0000 7f4439600640 1 -- 172.21.15.83:0/2293940970 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f443c153de0 con 0x7f443c0b1b10 2024-04-25T00:15:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.950+0000 7f4443a00640 1 -- 172.21.15.83:0/2293940970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f443c073000 con 0x7f443c0b1b10 2024-04-25T00:15:00.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.954+0000 7f4439600640 1 -- 172.21.15.83:0/2293940970 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16789 (secure 0 0 0) 0x7f4434011260 con 0x7f443c0b1b10 2024-04-25T00:15:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:00.958 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":93,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:53.503988+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:50.260735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:50.880394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:15:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.954+0000 7f4443a00640 1 -- 172.21.15.83:0/2293940970 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44180678f0 msgr2=0x7f4418069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.954+0000 7f4443a00640 1 --2- 172.21.15.83:0/2293940970 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44180678f0 0x7f4418069db0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f443c15d690 tx=0x7f442c009040 comp rx=0 tx=0).stop 2024-04-25T00:15:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.954+0000 7f4443a00640 1 -- 172.21.15.83:0/2293940970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f443c0b1b10 msgr2=0x7f443c151380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.954+0000 7f4443a00640 1 --2- 172.21.15.83:0/2293940970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f443c0b1b10 0x7f443c151380 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f4434007e60 tx=0x7f443400e4b0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.954+0000 7f4443a00640 1 -- 172.21.15.83:0/2293940970 shutdown_connections 2024-04-25T00:15:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.954+0000 7f4443a00640 1 --2- 172.21.15.83:0/2293940970 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f44180678f0 0x7f4418069db0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.954+0000 7f4443a00640 1 --2- 172.21.15.83:0/2293940970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f443c0b1b10 0x7f443c151380 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.954+0000 7f4443a00640 1 --2- 172.21.15.83:0/2293940970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f443c15dee0 0x7f443c0b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.954+0000 7f4443a00640 1 --2- 172.21.15.83:0/2293940970 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f443c0b4bc0 0x7f443c0cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.954+0000 7f4443a00640 1 -- 172.21.15.83:0/2293940970 >> 172.21.15.83:0/2293940970 conn(0x7f443c06b1e0 msgr2=0x7f443c1608f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:00.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.954+0000 7f4443a00640 1 -- 172.21.15.83:0/2293940970 shutdown_connections 2024-04-25T00:15:00.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:00.954+0000 7f4443a00640 1 -- 172.21.15.83:0/2293940970 wait complete. 2024-04-25T00:15:00.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:14:50.260735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:50.880394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:15:00.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:15:01.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8baa00640 1 Processor -- start 2024-04-25T00:15:01.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8baa00640 1 -- start start 2024-04-25T00:15:01.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8baa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd8b41551e0 0x7fd8b415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8baa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8b406e4c0 0x7fd8b406e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8baa00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b415c560 con 0x7fd8b41555c0 2024-04-25T00:15:01.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8baa00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b4072460 con 0x7fd8b41551e0 2024-04-25T00:15:01.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8baa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b415c7d0 con 0x7fd8b406e4c0 2024-04-25T00:15:01.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8b3e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd8b41551e0 0x7fd8b415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8b3e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd8b41551e0 0x7fd8b415bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52000/0 (socket says 172.21.15.83:52000) 2024-04-25T00:15:01.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8b3e00640 1 -- 172.21.15.83:0/318375262 learned_addr learned my addr 172.21.15.83:0/318375262 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:01.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8b3400640 1 --1- 172.21.15.83:0/318375262 >> v1:172.21.15.83:6789/0 conn(0x7fd8b41555c0 0x7fd8b4071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51382/0 (socket says 172.21.15.83:51382) 2024-04-25T00:15:01.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8b8a00640 1 --2- 172.21.15.83:0/318375262 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8b406e4c0 0x7fd8b406e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8b3e00640 1 -- 172.21.15.83:0/318375262 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8b406e4c0 msgr2=0x7fd8b406e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:01.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8b3e00640 1 --2- 172.21.15.83:0/318375262 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8b406e4c0 0x7fd8b406e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8b3e00640 1 -- 172.21.15.83:0/318375262 >> v1:172.21.15.83:6789/0 conn(0x7fd8b41555c0 legacy=0x7fd8b4071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8b3e00640 1 -- 172.21.15.83:0/318375262 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b415cab0 con 0x7fd8b41551e0 2024-04-25T00:15:01.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.114+0000 7fd8b8a00640 1 --2- 172.21.15.83:0/318375262 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8b406e4c0 0x7fd8b406e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:01.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.118+0000 7fd8b3e00640 1 --2- 172.21.15.83:0/318375262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b41551e0 0x7fd8b415bea0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400bb40 tx=0x7fd8a401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d335562fd888e303 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.118+0000 7fd8b2a00640 1 -- 172.21.15.83:0/318375262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40b4070 con 0x7fd8b41551e0 2024-04-25T00:15:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.118+0000 7fd8b2a00640 1 -- 172.21.15.83:0/318375262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a40a79e0 con 0x7fd8b41551e0 2024-04-25T00:15:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.118+0000 7fd8b2a00640 1 -- 172.21.15.83:0/318375262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40a7db0 con 0x7fd8b41551e0 2024-04-25T00:15:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.118+0000 7fd8baa00640 1 -- 172.21.15.83:0/318375262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b41551e0 msgr2=0x7fd8b415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.118+0000 7fd8baa00640 1 --2- 172.21.15.83:0/318375262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b41551e0 0x7fd8b415bea0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400bb40 tx=0x7fd8a401ec10 comp rx=0 tx=0).stop 2024-04-25T00:15:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.118+0000 7fd8baa00640 1 -- 172.21.15.83:0/318375262 shutdown_connections 2024-04-25T00:15:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.118+0000 7fd8baa00640 1 --2- 172.21.15.83:0/318375262 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8b406e4c0 0x7fd8b406e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.118+0000 7fd8baa00640 1 --2- 172.21.15.83:0/318375262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b41551e0 0x7fd8b415bea0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.118+0000 7fd8baa00640 1 -- 172.21.15.83:0/318375262 >> 172.21.15.83:0/318375262 conn(0x7fd8b40b1830 msgr2=0x7fd8b40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:01.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.118+0000 7fd8baa00640 1 -- 172.21.15.83:0/318375262 shutdown_connections 2024-04-25T00:15:01.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.118+0000 7fd8baa00640 1 -- 172.21.15.83:0/318375262 wait complete. 2024-04-25T00:15:01.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.118+0000 7fd8baa00640 1 Processor -- start 2024-04-25T00:15:01.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.118+0000 7fd8baa00640 1 -- start start 2024-04-25T00:15:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8baa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b406e4c0 0x7fd8b41f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8baa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8b41551e0 0x7fd8b40a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8baa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8b41555c0 0x7fd8b41fa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8baa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b406df90 con 0x7fd8b406e4c0 2024-04-25T00:15:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8baa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b406de10 con 0x7fd8b41551e0 2024-04-25T00:15:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8baa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b406e110 con 0x7fd8b41555c0 2024-04-25T00:15:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8b3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b406e4c0 0x7fd8b41f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8b3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b406e4c0 0x7fd8b41f7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52002/0 (socket says 172.21.15.83:52002) 2024-04-25T00:15:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8b3e00640 1 -- 172.21.15.83:0/1287442555 learned_addr learned my addr 172.21.15.83:0/1287442555 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8b8a00640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8b41555c0 0x7fd8b41fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8b3400640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8b41551e0 0x7fd8b40a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8b3e00640 1 -- 172.21.15.83:0/1287442555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8b41555c0 msgr2=0x7fd8b41fa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8b3e00640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8b41555c0 0x7fd8b41fa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8b3e00640 1 -- 172.21.15.83:0/1287442555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8b41551e0 msgr2=0x7fd8b40a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8b3e00640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8b41551e0 0x7fd8b40a9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8b3e00640 1 -- 172.21.15.83:0/1287442555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b415d9c0 con 0x7fd8b406e4c0 2024-04-25T00:15:01.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8b3400640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8b41551e0 0x7fd8b40a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:01.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8b3e00640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b406e4c0 0x7fd8b41f7080 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400b2c0 tx=0x7fd8a401b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:01.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8abe00640 1 -- 172.21.15.83:0/1287442555 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40bf070 con 0x7fd8b406e4c0 2024-04-25T00:15:01.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8abe00640 1 -- 172.21.15.83:0/1287442555 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a40b6050 con 0x7fd8b406e4c0 2024-04-25T00:15:01.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8abe00640 1 -- 172.21.15.83:0/1287442555 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40b4030 con 0x7fd8b406e4c0 2024-04-25T00:15:01.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8baa00640 1 -- 172.21.15.83:0/1287442555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b415dca0 con 0x7fd8b406e4c0 2024-04-25T00:15:01.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8baa00640 1 -- 172.21.15.83:0/1287442555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b415e260 con 0x7fd8b406e4c0 2024-04-25T00:15:01.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.122+0000 7fd8baa00640 1 -- 172.21.15.83:0/1287442555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b404f510 con 0x7fd8b406e4c0 2024-04-25T00:15:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.126+0000 7fd8abe00640 1 -- 172.21.15.83:0/1287442555 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd8a40afd00 con 0x7fd8b406e4c0 2024-04-25T00:15:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.126+0000 7fd8abe00640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd8940678c0 0x7fd894069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.126+0000 7fd8abe00640 1 -- 172.21.15.83:0/1287442555 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fd8a411fcf0 con 0x7fd8b406e4c0 2024-04-25T00:15:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.126+0000 7fd8b3400640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd8940678c0 0x7fd894069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.126+0000 7fd8b3400640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd8940678c0 0x7fd894069d80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd8b415d4a0 tx=0x7fd8a0007d70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.130+0000 7fd8abe00640 1 -- 172.21.15.83:0/1287442555 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd8a4124070 con 0x7fd8b406e4c0 2024-04-25T00:15:01.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.286+0000 7fd8baa00640 1 -- 172.21.15.83:0/1287442555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd8b40ce6a0 con 0x7fd8b406e4c0 2024-04-25T00:15:01.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.286+0000 7fd8abe00640 1 -- 172.21.15.83:0/1287442555 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fd8a4018800 con 0x7fd8b406e4c0 2024-04-25T00:15:01.290 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:15:01.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.286+0000 7fd8baa00640 1 -- 172.21.15.83:0/1287442555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd8940678c0 msgr2=0x7fd894069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.286+0000 7fd8baa00640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd8940678c0 0x7fd894069d80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd8b415d4a0 tx=0x7fd8a0007d70 comp rx=0 tx=0).stop 2024-04-25T00:15:01.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.286+0000 7fd8baa00640 1 -- 172.21.15.83:0/1287442555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b406e4c0 msgr2=0x7fd8b41f7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.286+0000 7fd8baa00640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b406e4c0 0x7fd8b41f7080 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400b2c0 tx=0x7fd8a401b9b0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.286+0000 7fd8baa00640 1 -- 172.21.15.83:0/1287442555 shutdown_connections 2024-04-25T00:15:01.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.286+0000 7fd8baa00640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd8940678c0 0x7fd894069d80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.286+0000 7fd8baa00640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8b41555c0 0x7fd8b41fa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.286+0000 7fd8baa00640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8b41551e0 0x7fd8b40a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.286+0000 7fd8baa00640 1 --2- 172.21.15.83:0/1287442555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b406e4c0 0x7fd8b41f7080 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.286+0000 7fd8baa00640 1 -- 172.21.15.83:0/1287442555 >> 172.21.15.83:0/1287442555 conn(0x7fd8b40b1830 msgr2=0x7fd8b40cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:01.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.286+0000 7fd8baa00640 1 -- 172.21.15.83:0/1287442555 shutdown_connections 2024-04-25T00:15:01.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.286+0000 7fd8baa00640 1 -- 172.21.15.83:0/1287442555 wait complete. 2024-04-25T00:15:01.312 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:15:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 Processor -- start 2024-04-25T00:15:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 -- start start 2024-04-25T00:15:01.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbbf0159810 0x7fbbf00cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbbf006e950 0x7fbbf015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbf015fe50 0x7fbbf015c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbbf006e210 con 0x7fbbf0159810 2024-04-25T00:15:01.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbbf006da00 con 0x7fbbf006e950 2024-04-25T00:15:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf00ce000 con 0x7fbbf015fe50 2024-04-25T00:15:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbede00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbbf0159810 0x7fbbf00cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbede00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbbf0159810 0x7fbbf00cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52018/0 (socket says 172.21.15.83:52018) 2024-04-25T00:15:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbede00640 1 -- 172.21.15.83:0/3313466286 learned_addr learned my addr 172.21.15.83:0/3313466286 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbee800640 1 --2- 172.21.15.83:0/3313466286 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbf015fe50 0x7fbbf015c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbe5400640 1 --2- 172.21.15.83:0/3313466286 >> v2:172.21.15.86:3300/0 conn(0x7fbbf006e950 0x7fbbf015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbede00640 1 -- 172.21.15.83:0/3313466286 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbf015fe50 msgr2=0x7fbbf015c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbede00640 1 --2- 172.21.15.83:0/3313466286 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbf015fe50 0x7fbbf015c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbede00640 1 -- 172.21.15.83:0/3313466286 >> v2:172.21.15.86:3300/0 conn(0x7fbbf006e950 msgr2=0x7fbbf015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbede00640 1 --2- 172.21.15.83:0/3313466286 >> v2:172.21.15.86:3300/0 conn(0x7fbbf006e950 0x7fbbf015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbede00640 1 -- 172.21.15.83:0/3313466286 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf006eec0 con 0x7fbbf0159810 2024-04-25T00:15:01.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbede00640 1 --2- 172.21.15.83:0/3313466286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf0159810 0x7fbbf00cc4a0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00709a0 tx=0x7fbbe001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a2ea86b71c5b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:01.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbed400640 1 -- 172.21.15.83:0/3313466286 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00a56d0 con 0x7fbbf0159810 2024-04-25T00:15:01.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbed400640 1 -- 172.21.15.83:0/3313466286 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbe00a5870 con 0x7fbbf0159810 2024-04-25T00:15:01.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbed400640 1 -- 172.21.15.83:0/3313466286 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00a5c40 con 0x7fbbf0159810 2024-04-25T00:15:01.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 -- 172.21.15.83:0/3313466286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf0159810 msgr2=0x7fbbf00cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 --2- 172.21.15.83:0/3313466286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf0159810 0x7fbbf00cc4a0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00709a0 tx=0x7fbbe001c960 comp rx=0 tx=0).stop 2024-04-25T00:15:01.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 -- 172.21.15.83:0/3313466286 shutdown_connections 2024-04-25T00:15:01.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 --2- 172.21.15.83:0/3313466286 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbf015fe50 0x7fbbf015c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 --2- 172.21.15.83:0/3313466286 >> v2:172.21.15.86:3300/0 conn(0x7fbbf006e950 0x7fbbf015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 --2- 172.21.15.83:0/3313466286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf0159810 0x7fbbf00cc4a0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 -- 172.21.15.83:0/3313466286 >> 172.21.15.83:0/3313466286 conn(0x7fbbf00b1830 msgr2=0x7fbbf00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:01.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 -- 172.21.15.83:0/3313466286 shutdown_connections 2024-04-25T00:15:01.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 -- 172.21.15.83:0/3313466286 wait complete. 2024-04-25T00:15:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 Processor -- start 2024-04-25T00:15:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.450+0000 7fbbf5200640 1 -- start start 2024-04-25T00:15:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbf5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf006e950 0x7fbbf0153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbf5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf0159810 0x7fbbf00b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbf5200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbf015fe50 0x7fbbf00b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbf5200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0159d70 con 0x7fbbf006e950 2024-04-25T00:15:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbf5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0159bf0 con 0x7fbbf0159810 2024-04-25T00:15:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbf5200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf0070da0 con 0x7fbbf015fe50 2024-04-25T00:15:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbede00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf006e950 0x7fbbf0153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbede00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf006e950 0x7fbbf0153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52032/0 (socket says 172.21.15.83:52032) 2024-04-25T00:15:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbede00640 1 -- 172.21.15.83:0/2778978204 learned_addr learned my addr 172.21.15.83:0/2778978204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbe5400640 1 --2- 172.21.15.83:0/2778978204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf0159810 0x7fbbf00b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbee800640 1 --2- 172.21.15.83:0/2778978204 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbf015fe50 0x7fbbf00b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbede00640 1 -- 172.21.15.83:0/2778978204 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbf015fe50 msgr2=0x7fbbf00b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbede00640 1 --2- 172.21.15.83:0/2778978204 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbf015fe50 0x7fbbf00b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbede00640 1 -- 172.21.15.83:0/2778978204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf0159810 msgr2=0x7fbbf00b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbede00640 1 --2- 172.21.15.83:0/2778978204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf0159810 0x7fbbf00b42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbede00640 1 -- 172.21.15.83:0/2778978204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf00b2f50 con 0x7fbbf006e950 2024-04-25T00:15:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbede00640 1 --2- 172.21.15.83:0/2778978204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf006e950 0x7fbbf0153f30 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00af040 tx=0x7fbbe0019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbe6a00640 1 -- 172.21.15.83:0/2778978204 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00bc020 con 0x7fbbf006e950 2024-04-25T00:15:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbe6a00640 1 -- 172.21.15.83:0/2778978204 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbe00b8690 con 0x7fbbf006e950 2024-04-25T00:15:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbe6a00640 1 -- 172.21.15.83:0/2778978204 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00b8a80 con 0x7fbbf006e950 2024-04-25T00:15:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbf5200640 1 -- 172.21.15.83:0/2778978204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf00b3230 con 0x7fbbf006e950 2024-04-25T00:15:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbf5200640 1 -- 172.21.15.83:0/2778978204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf00b37f0 con 0x7fbbf006e950 2024-04-25T00:15:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.454+0000 7fbbf5200640 1 -- 172.21.15.83:0/2778978204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf004f510 con 0x7fbbf006e950 2024-04-25T00:15:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.458+0000 7fbbe6a00640 1 -- 172.21.15.83:0/2778978204 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbbe00bf020 con 0x7fbbf006e950 2024-04-25T00:15:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.458+0000 7fbbe6a00640 1 --2- 172.21.15.83:0/2778978204 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbbcc0678c0 0x7fbbcc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.458+0000 7fbbe6a00640 1 -- 172.21.15.83:0/2778978204 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fbbe0120e90 con 0x7fbbf006e950 2024-04-25T00:15:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.458+0000 7fbbe5400640 1 --2- 172.21.15.83:0/2778978204 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbbcc0678c0 0x7fbbcc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.458+0000 7fbbe5400640 1 --2- 172.21.15.83:0/2778978204 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbbcc0678c0 0x7fbbcc069d80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0155f80 tx=0x7fbbd8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:01.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.462+0000 7fbbe6a00640 1 -- 172.21.15.83:0/2778978204 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbbe00c6030 con 0x7fbbf006e950 2024-04-25T00:15:01.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.618+0000 7fbbf5200640 1 -- 172.21.15.83:0/2778978204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbbf015c2e0 con 0x7fbbf006e950 2024-04-25T00:15:01.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.618+0000 7fbbe6a00640 1 -- 172.21.15.83:0/2778978204 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fbbe00165e0 con 0x7fbbf006e950 2024-04-25T00:15:01.624 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:15:01.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.622+0000 7fbbf5200640 1 -- 172.21.15.83:0/2778978204 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbbcc0678c0 msgr2=0x7fbbcc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.622+0000 7fbbf5200640 1 --2- 172.21.15.83:0/2778978204 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbbcc0678c0 0x7fbbcc069d80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0155f80 tx=0x7fbbd8008040 comp rx=0 tx=0).stop 2024-04-25T00:15:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.622+0000 7fbbf5200640 1 -- 172.21.15.83:0/2778978204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf006e950 msgr2=0x7fbbf0153f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.622+0000 7fbbf5200640 1 --2- 172.21.15.83:0/2778978204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf006e950 0x7fbbf0153f30 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00af040 tx=0x7fbbe0019700 comp rx=0 tx=0).stop 2024-04-25T00:15:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.622+0000 7fbbf5200640 1 -- 172.21.15.83:0/2778978204 shutdown_connections 2024-04-25T00:15:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.622+0000 7fbbf5200640 1 --2- 172.21.15.83:0/2778978204 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbbcc0678c0 0x7fbbcc069d80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.622+0000 7fbbf5200640 1 --2- 172.21.15.83:0/2778978204 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbf015fe50 0x7fbbf00b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.622+0000 7fbbf5200640 1 --2- 172.21.15.83:0/2778978204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf0159810 0x7fbbf00b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.622+0000 7fbbf5200640 1 --2- 172.21.15.83:0/2778978204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf006e950 0x7fbbf0153f30 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.622+0000 7fbbf5200640 1 -- 172.21.15.83:0/2778978204 >> 172.21.15.83:0/2778978204 conn(0x7fbbf00b1830 msgr2=0x7fbbf0072360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.622+0000 7fbbf5200640 1 -- 172.21.15.83:0/2778978204 shutdown_connections 2024-04-25T00:15:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.622+0000 7fbbf5200640 1 -- 172.21.15.83:0/2778978204 wait complete. 2024-04-25T00:15:01.645 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:15:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 Processor -- start 2024-04-25T00:15:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 -- start start 2024-04-25T00:15:01.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f632015e270 0x7f63200b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6320071e50 0x7f63200b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f632006dcf0 con 0x7f632015de90 2024-04-25T00:15:01.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f63200b2ce0 con 0x7f6320071e50 2024-04-25T00:15:01.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f632006dfd0 con 0x7f632015e270 2024-04-25T00:15:01.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6325200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f632015e270 0x7f63200b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6325200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f632015e270 0x7f63200b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:49792/0 (socket says 172.21.15.83:49792) 2024-04-25T00:15:01.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6326600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6320071e50 0x7f63200b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6325200640 1 -- 172.21.15.83:0/3536184821 learned_addr learned my addr 172.21.15.83:0/3536184821 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:01.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6325200640 1 -- 172.21.15.83:0/3536184821 >> v2:172.21.15.86:3300/0 conn(0x7f6320071e50 msgr2=0x7f63200b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:01.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6325200640 1 --2- 172.21.15.83:0/3536184821 >> v2:172.21.15.86:3300/0 conn(0x7f6320071e50 0x7f63200b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6325200640 1 -- 172.21.15.83:0/3536184821 >> v1:172.21.15.86:6789/0 conn(0x7f632015de90 legacy=0x7f6320071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6325200640 1 -- 172.21.15.83:0/3536184821 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632006e2b0 con 0x7f632015e270 2024-04-25T00:15:01.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6325200640 1 --2- 172.21.15.83:0/3536184821 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f632015e270 0x7f63200b3450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f630c009870 tx=0x7f630c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3559524420eb4dad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:01.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f631fe00640 1 -- 172.21.15.83:0/3536184821 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c0b2070 con 0x7f632015e270 2024-04-25T00:15:01.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f631fe00640 1 -- 172.21.15.83:0/3536184821 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630c0a9070 con 0x7f632015e270 2024-04-25T00:15:01.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f631fe00640 1 -- 172.21.15.83:0/3536184821 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c0ad880 con 0x7f632015e270 2024-04-25T00:15:01.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 -- 172.21.15.83:0/3536184821 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f632015e270 msgr2=0x7f63200b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 --2- 172.21.15.83:0/3536184821 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f632015e270 0x7f63200b3450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f630c009870 tx=0x7f630c01c4d0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 -- 172.21.15.83:0/3536184821 shutdown_connections 2024-04-25T00:15:01.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 --2- 172.21.15.83:0/3536184821 >> v2:172.21.15.86:3300/0 conn(0x7f6320071e50 0x7f63200b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 --2- 172.21.15.83:0/3536184821 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f632015e270 0x7f63200b3450 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 -- 172.21.15.83:0/3536184821 >> 172.21.15.83:0/3536184821 conn(0x7f63200b1830 msgr2=0x7f63200b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:01.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 -- 172.21.15.83:0/3536184821 shutdown_connections 2024-04-25T00:15:01.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.782+0000 7f6328600640 1 -- 172.21.15.83:0/3536184821 wait complete. 2024-04-25T00:15:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6328600640 1 Processor -- start 2024-04-25T00:15:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6328600640 1 -- start start 2024-04-25T00:15:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6328600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6320071e50 0x7f632014eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6328600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f632015de90 0x7f6320160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6328600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f632015e270 0x7f63201611f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6328600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63200b4050 con 0x7f632015de90 2024-04-25T00:15:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6328600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63200b3ed0 con 0x7f6320071e50 2024-04-25T00:15:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6328600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f63200b41d0 con 0x7f632015e270 2024-04-25T00:15:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6325200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f632015de90 0x7f6320160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6325200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f632015de90 0x7f6320160cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52042/0 (socket says 172.21.15.83:52042) 2024-04-25T00:15:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6325200640 1 -- 172.21.15.83:0/153561316 learned_addr learned my addr 172.21.15.83:0/153561316 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6326600640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f632015e270 0x7f63201611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6325c00640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6320071e50 0x7f632014eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6325200640 1 -- 172.21.15.83:0/153561316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f632015e270 msgr2=0x7f63201611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6325200640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f632015e270 0x7f63201611f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6325200640 1 -- 172.21.15.83:0/153561316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6320071e50 msgr2=0x7f632014eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6325200640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6320071e50 0x7f632014eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6325200640 1 -- 172.21.15.83:0/153561316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632014f050 con 0x7f632015de90 2024-04-25T00:15:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6325c00640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6320071e50 0x7f632014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6325200640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f632015de90 0x7f6320160cb0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f630c0b0040 tx=0x7f630c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f631d600640 1 -- 172.21.15.83:0/153561316 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c0bb070 con 0x7f632015de90 2024-04-25T00:15:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f631d600640 1 -- 172.21.15.83:0/153561316 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630c0add00 con 0x7f632015de90 2024-04-25T00:15:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f631d600640 1 -- 172.21.15.83:0/153561316 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c0b2040 con 0x7f632015de90 2024-04-25T00:15:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6328600640 1 -- 172.21.15.83:0/153561316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63201565e0 con 0x7f632015de90 2024-04-25T00:15:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6328600640 1 -- 172.21.15.83:0/153561316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6320156ba0 con 0x7f632015de90 2024-04-25T00:15:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.786+0000 7f6328600640 1 -- 172.21.15.83:0/153561316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f632004f510 con 0x7f632015de90 2024-04-25T00:15:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.790+0000 7f631d600640 1 -- 172.21.15.83:0/153561316 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f630c0ab070 con 0x7f632015de90 2024-04-25T00:15:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.790+0000 7f631d600640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63000678c0 0x7f6300069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.790+0000 7f631d600640 1 -- 172.21.15.83:0/153561316 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f630c11ec30 con 0x7f632015de90 2024-04-25T00:15:01.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.790+0000 7f6325c00640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63000678c0 0x7f6300069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.790+0000 7f6325c00640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63000678c0 0x7f6300069d80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6314001710 tx=0x7f6314009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.794+0000 7f631d600640 1 -- 172.21.15.83:0/153561316 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f630c016150 con 0x7f632015de90 2024-04-25T00:15:01.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.950+0000 7f6328600640 1 -- 172.21.15.83:0/153561316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f63201544a0 con 0x7f632015de90 2024-04-25T00:15:02.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.950+0000 7f631d600640 1 -- 172.21.15.83:0/153561316 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f630c0eef80 con 0x7f632015de90 2024-04-25T00:15:02.017 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:15:02.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.950+0000 7f6328600640 1 -- 172.21.15.83:0/153561316 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63000678c0 msgr2=0x7f6300069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.950+0000 7f6328600640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63000678c0 0x7f6300069d80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6314001710 tx=0x7f6314009040 comp rx=0 tx=0).stop 2024-04-25T00:15:02.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.950+0000 7f6328600640 1 -- 172.21.15.83:0/153561316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f632015de90 msgr2=0x7f6320160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.950+0000 7f6328600640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f632015de90 0x7f6320160cb0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f630c0b0040 tx=0x7f630c019270 comp rx=0 tx=0).stop 2024-04-25T00:15:02.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.950+0000 7f6328600640 1 -- 172.21.15.83:0/153561316 shutdown_connections 2024-04-25T00:15:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.950+0000 7f6328600640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f63000678c0 0x7f6300069d80 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.950+0000 7f6328600640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f632015e270 0x7f63201611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.950+0000 7f6328600640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f632015de90 0x7f6320160cb0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.950+0000 7f6328600640 1 --2- 172.21.15.83:0/153561316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6320071e50 0x7f632014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.950+0000 7f6328600640 1 -- 172.21.15.83:0/153561316 >> 172.21.15.83:0/153561316 conn(0x7f63200b1830 msgr2=0x7f63200b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.950+0000 7f6328600640 1 -- 172.21.15.83:0/153561316 shutdown_connections 2024-04-25T00:15:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:01.950+0000 7f6328600640 1 -- 172.21.15.83:0/153561316 wait complete. 2024-04-25T00:15:02.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:15:02.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.154+0000 7f946b200640 1 Processor -- start 2024-04-25T00:15:02.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.154+0000 7f946b200640 1 -- start start 2024-04-25T00:15:02.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.154+0000 7f946b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f94640cb640 0x7f946406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.154+0000 7f946b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9464070d10 0x7f9464165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.154+0000 7f946b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94640711a0 con 0x7f946406cfd0 2024-04-25T00:15:02.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.154+0000 7f946b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f946406e160 con 0x7f94640cb640 2024-04-25T00:15:02.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.154+0000 7f946b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f946406ea10 con 0x7f9464070d10 2024-04-25T00:15:02.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.154+0000 7f9463e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f946406cfd0 0x7f946406d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:37344/0 (socket says 172.21.15.83:37344) 2024-04-25T00:15:02.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.154+0000 7f9463e00640 1 -- 172.21.15.83:0/2738152186 learned_addr learned my addr 172.21.15.83:0/2738152186 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:02.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.154+0000 7f9463400640 1 --2- 172.21.15.83:0/2738152186 >> v2:172.21.15.86:3300/0 conn(0x7f94640cb640 0x7f946406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.154+0000 7f9469200640 1 --2- 172.21.15.83:0/2738152186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9464070d10 0x7f9464165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.158+0000 7f9462a00640 1 -- 172.21.15.83:0/2738152186 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4135499560 0 0) 0x7f94540014a0 con 0x7f946406cfd0 2024-04-25T00:15:02.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.158+0000 7f9462a00640 1 -- 172.21.15.83:0/2738152186 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9448003530 con 0x7f946406cfd0 2024-04-25T00:15:02.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.158+0000 7f9463400640 1 -- 172.21.15.83:0/2738152186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9464070d10 msgr2=0x7f9464165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:02.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.158+0000 7f9463400640 1 --2- 172.21.15.83:0/2738152186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9464070d10 0x7f9464165930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.158+0000 7f9463400640 1 -- 172.21.15.83:0/2738152186 >> v1:172.21.15.86:6789/0 conn(0x7f946406cfd0 legacy=0x7f946406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.158+0000 7f9463400640 1 -- 172.21.15.83:0/2738152186 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946406ecf0 con 0x7f94640cb640 2024-04-25T00:15:02.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.158+0000 7f9469200640 1 --2- 172.21.15.83:0/2738152186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9464070d10 0x7f9464165930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:02.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.158+0000 7f9463400640 1 --2- 172.21.15.83:0/2738152186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94640cb640 0x7f946406dac0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f94500097c0 tx=0x7f945001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dce9164321d8950 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:02.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.158+0000 7f9462a00640 1 -- 172.21.15.83:0/2738152186 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94500b1070 con 0x7f94640cb640 2024-04-25T00:15:02.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.158+0000 7f9462a00640 1 -- 172.21.15.83:0/2738152186 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94500ac410 con 0x7f94640cb640 2024-04-25T00:15:02.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.158+0000 7f9462a00640 1 -- 172.21.15.83:0/2738152186 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94500ac7e0 con 0x7f94640cb640 2024-04-25T00:15:02.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 -- 172.21.15.83:0/2738152186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94640cb640 msgr2=0x7f946406dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 --2- 172.21.15.83:0/2738152186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94640cb640 0x7f946406dac0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f94500097c0 tx=0x7f945001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 -- 172.21.15.83:0/2738152186 shutdown_connections 2024-04-25T00:15:02.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 --2- 172.21.15.83:0/2738152186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9464070d10 0x7f9464165930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 --2- 172.21.15.83:0/2738152186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94640cb640 0x7f946406dac0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 -- 172.21.15.83:0/2738152186 >> 172.21.15.83:0/2738152186 conn(0x7f94640cfa60 msgr2=0x7f94640cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:02.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 -- 172.21.15.83:0/2738152186 shutdown_connections 2024-04-25T00:15:02.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 -- 172.21.15.83:0/2738152186 wait complete. 2024-04-25T00:15:02.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 Processor -- start 2024-04-25T00:15:02.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 -- start start 2024-04-25T00:15:02.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f946406cfd0 0x7f946413d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464070d10 0x7f9464134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94640cb640 0x7f946415a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9464168a20 con 0x7f946406cfd0 2024-04-25T00:15:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f94641688a0 con 0x7f9464070d10 2024-04-25T00:15:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f946b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9464168ba0 con 0x7f94640cb640 2024-04-25T00:15:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f9463e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f946406cfd0 0x7f946413d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f9463e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f946406cfd0 0x7f946413d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52050/0 (socket says 172.21.15.83:52050) 2024-04-25T00:15:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f9463e00640 1 -- 172.21.15.83:0/880266446 learned_addr learned my addr 172.21.15.83:0/880266446 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f9463400640 1 --2- 172.21.15.83:0/880266446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464070d10 0x7f9464134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f9469200640 1 --2- 172.21.15.83:0/880266446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94640cb640 0x7f946415a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f9463e00640 1 -- 172.21.15.83:0/880266446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94640cb640 msgr2=0x7f946415a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f9463e00640 1 --2- 172.21.15.83:0/880266446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94640cb640 0x7f946415a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f9463e00640 1 -- 172.21.15.83:0/880266446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464070d10 msgr2=0x7f9464134fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f9463e00640 1 --2- 172.21.15.83:0/880266446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464070d10 0x7f9464134fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f9463e00640 1 -- 172.21.15.83:0/880266446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946413d940 con 0x7f946406cfd0 2024-04-25T00:15:02.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.162+0000 7f9463e00640 1 --2- 172.21.15.83:0/880266446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f946406cfd0 0x7f946413d400 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f945400cb10 tx=0x7f945400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.166+0000 7f945be00640 1 -- 172.21.15.83:0/880266446 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454017070 con 0x7f946406cfd0 2024-04-25T00:15:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.166+0000 7f945be00640 1 -- 172.21.15.83:0/880266446 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945400ab50 con 0x7f946406cfd0 2024-04-25T00:15:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.166+0000 7f946b200640 1 -- 172.21.15.83:0/880266446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9464135be0 con 0x7f946406cfd0 2024-04-25T00:15:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.166+0000 7f945be00640 1 -- 172.21.15.83:0/880266446 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454012400 con 0x7f946406cfd0 2024-04-25T00:15:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.166+0000 7f946b200640 1 -- 172.21.15.83:0/880266446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946413c920 con 0x7f946406cfd0 2024-04-25T00:15:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.166+0000 7f946b200640 1 -- 172.21.15.83:0/880266446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9464068230 con 0x7f946406cfd0 2024-04-25T00:15:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.166+0000 7f945be00640 1 -- 172.21.15.83:0/880266446 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f945401a070 con 0x7f946406cfd0 2024-04-25T00:15:02.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.166+0000 7f945be00640 1 --2- 172.21.15.83:0/880266446 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f94440678f0 0x7f9444069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.166+0000 7f945be00640 1 -- 172.21.15.83:0/880266446 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f9454085a90 con 0x7f946406cfd0 2024-04-25T00:15:02.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.166+0000 7f9463400640 1 --2- 172.21.15.83:0/880266446 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f94440678f0 0x7f9444069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.170+0000 7f9463400640 1 --2- 172.21.15.83:0/880266446 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f94440678f0 0x7f9444069db0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f94500ae000 tx=0x7f9450008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:02.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.170+0000 7f945be00640 1 -- 172.21.15.83:0/880266446 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f94640711a0 con 0x7f946406cfd0 2024-04-25T00:15:02.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.338+0000 7f946b200640 1 -- 172.21.15.83:0/880266446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f946413b2c0 con 0x7f946406cfd0 2024-04-25T00:15:02.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.338+0000 7f945be00640 1 -- 172.21.15.83:0/880266446 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v93) ==== 80+0+0 (secure 0 0 0) 0x7f9454055df0 con 0x7f946406cfd0 2024-04-25T00:15:02.339 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:15:02.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.338+0000 7f946b200640 1 -- 172.21.15.83:0/880266446 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f94440678f0 msgr2=0x7f9444069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.338+0000 7f946b200640 1 --2- 172.21.15.83:0/880266446 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f94440678f0 0x7f9444069db0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f94500ae000 tx=0x7f9450008040 comp rx=0 tx=0).stop 2024-04-25T00:15:02.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.338+0000 7f946b200640 1 -- 172.21.15.83:0/880266446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f946406cfd0 msgr2=0x7f946413d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.338+0000 7f946b200640 1 --2- 172.21.15.83:0/880266446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f946406cfd0 0x7f946413d400 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f945400cb10 tx=0x7f945400cf10 comp rx=0 tx=0).stop 2024-04-25T00:15:02.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.338+0000 7f946b200640 1 -- 172.21.15.83:0/880266446 shutdown_connections 2024-04-25T00:15:02.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.338+0000 7f946b200640 1 --2- 172.21.15.83:0/880266446 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f94440678f0 0x7f9444069db0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.338+0000 7f946b200640 1 --2- 172.21.15.83:0/880266446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94640cb640 0x7f946415a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.338+0000 7f946b200640 1 --2- 172.21.15.83:0/880266446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9464070d10 0x7f9464134fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.338+0000 7f946b200640 1 --2- 172.21.15.83:0/880266446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f946406cfd0 0x7f946413d400 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.338+0000 7f946b200640 1 -- 172.21.15.83:0/880266446 >> 172.21.15.83:0/880266446 conn(0x7f94640cfa60 msgr2=0x7f9464144df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:02.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.342+0000 7f946b200640 1 -- 172.21.15.83:0/880266446 shutdown_connections 2024-04-25T00:15:02.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.342+0000 7f946b200640 1 -- 172.21.15.83:0/880266446 wait complete. 2024-04-25T00:15:02.360 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:15:02.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:02.361 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:15:02.386 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:15:02.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:02.386 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:15:02.437 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:15:02.437 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:02.447 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:15:02.448 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:02.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:15:02.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.634+0000 7f50cf600640 1 Processor -- start 2024-04-25T00:15:02.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.634+0000 7f50cf600640 1 -- start start 2024-04-25T00:15:02.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.634+0000 7f50cf600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50c8071dc0 0x7f50c806d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.634+0000 7f50cf600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f50c806e7b0 0x7f50c80ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.634+0000 7f50cf600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50c806e0c0 con 0x7f50c815df00 2024-04-25T00:15:02.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.634+0000 7f50cf600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f50c806f150 con 0x7f50c8071dc0 2024-04-25T00:15:02.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.634+0000 7f50cf600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f50c806e3a0 con 0x7f50c806e7b0 2024-04-25T00:15:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.634+0000 7f50bfe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50c8071dc0 0x7f50c806d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.634+0000 7f50bfe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50c8071dc0 0x7f50c806d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52054/0 (socket says 172.21.15.83:52054) 2024-04-25T00:15:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.634+0000 7f50bfe00640 1 -- 172.21.15.83:0/2144538652 learned_addr learned my addr 172.21.15.83:0/2144538652 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.634+0000 7f50bfe00640 1 -- 172.21.15.83:0/2144538652 >> v2:172.21.15.86:3300/0 conn(0x7f50c806e7b0 msgr2=0x7f50c80ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.634+0000 7f50bfe00640 1 --2- 172.21.15.83:0/2144538652 >> v2:172.21.15.86:3300/0 conn(0x7f50c806e7b0 0x7f50c80ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.634+0000 7f50bfe00640 1 -- 172.21.15.83:0/2144538652 >> v1:172.21.15.86:6789/0 conn(0x7f50c815df00 legacy=0x7f50c80716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.634+0000 7f50bfe00640 1 -- 172.21.15.83:0/2144538652 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c80ad990 con 0x7f50c8071dc0 2024-04-25T00:15:02.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.638+0000 7f50bfe00640 1 --2- 172.21.15.83:0/2144538652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8071dc0 0x7f50c806d970 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f50c80705c0 tx=0x7f50b001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953af1dfccd1ef5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:02.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.638+0000 7f50c7e00640 1 -- 172.21.15.83:0/2144538652 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b00b0070 con 0x7f50c8071dc0 2024-04-25T00:15:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.638+0000 7f50c7e00640 1 -- 172.21.15.83:0/2144538652 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b00ab600 con 0x7f50c8071dc0 2024-04-25T00:15:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.638+0000 7f50c7e00640 1 -- 172.21.15.83:0/2144538652 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b00ab9d0 con 0x7f50c8071dc0 2024-04-25T00:15:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.638+0000 7f50cf600640 1 -- 172.21.15.83:0/2144538652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8071dc0 msgr2=0x7f50c806d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.638+0000 7f50cf600640 1 --2- 172.21.15.83:0/2144538652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8071dc0 0x7f50c806d970 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f50c80705c0 tx=0x7f50b001c3c0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.638+0000 7f50cf600640 1 -- 172.21.15.83:0/2144538652 shutdown_connections 2024-04-25T00:15:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.638+0000 7f50cf600640 1 --2- 172.21.15.83:0/2144538652 >> v2:172.21.15.86:3300/0 conn(0x7f50c806e7b0 0x7f50c80ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.638+0000 7f50cf600640 1 --2- 172.21.15.83:0/2144538652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8071dc0 0x7f50c806d970 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.638+0000 7f50cf600640 1 -- 172.21.15.83:0/2144538652 >> 172.21.15.83:0/2144538652 conn(0x7f50c80b1830 msgr2=0x7f50c80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.638+0000 7f50cf600640 1 -- 172.21.15.83:0/2144538652 shutdown_connections 2024-04-25T00:15:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.638+0000 7f50cf600640 1 -- 172.21.15.83:0/2144538652 wait complete. 2024-04-25T00:15:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50cf600640 1 Processor -- start 2024-04-25T00:15:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50cf600640 1 -- start start 2024-04-25T00:15:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50cf600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50c806e7b0 0x7f50c8159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50cf600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8071dc0 0x7f50c814fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50cf600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50c815df00 0x7f50c8153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50cf600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50c80b32a0 con 0x7f50c8071dc0 2024-04-25T00:15:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50cf600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50c80b3120 con 0x7f50c806e7b0 2024-04-25T00:15:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50cf600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f50c816ae80 con 0x7f50c815df00 2024-04-25T00:15:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50bfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8071dc0 0x7f50c814fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50bfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8071dc0 0x7f50c814fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52060/0 (socket says 172.21.15.83:52060) 2024-04-25T00:15:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50bfe00640 1 -- 172.21.15.83:0/3372688162 learned_addr learned my addr 172.21.15.83:0/3372688162 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50cd600640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50c815df00 0x7f50c8153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50ccc00640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50c806e7b0 0x7f50c8159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50bfe00640 1 -- 172.21.15.83:0/3372688162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50c815df00 msgr2=0x7f50c8153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50bfe00640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50c815df00 0x7f50c8153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50bfe00640 1 -- 172.21.15.83:0/3372688162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50c806e7b0 msgr2=0x7f50c8159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50bfe00640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50c806e7b0 0x7f50c8159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50bfe00640 1 -- 172.21.15.83:0/3372688162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c8159f80 con 0x7f50c8071dc0 2024-04-25T00:15:02.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50cd600640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50c815df00 0x7f50c8153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:02.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50bfe00640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8071dc0 0x7f50c814fa40 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f50b00ae040 tx=0x7f50b001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:02.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50c5600640 1 -- 172.21.15.83:0/3372688162 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b00ba020 con 0x7f50c8071dc0 2024-04-25T00:15:02.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50c5600640 1 -- 172.21.15.83:0/3372688162 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b00b6620 con 0x7f50c8071dc0 2024-04-25T00:15:02.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50cf600640 1 -- 172.21.15.83:0/3372688162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c8161370 con 0x7f50c8071dc0 2024-04-25T00:15:02.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50c5600640 1 -- 172.21.15.83:0/3372688162 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b00b0040 con 0x7f50c8071dc0 2024-04-25T00:15:02.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.642+0000 7f50cf600640 1 -- 172.21.15.83:0/3372688162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c8147910 con 0x7f50c8071dc0 2024-04-25T00:15:02.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.646+0000 7f50cf600640 1 -- 172.21.15.83:0/3372688162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c80737f0 con 0x7f50c8071dc0 2024-04-25T00:15:02.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.646+0000 7f50c5600640 1 -- 172.21.15.83:0/3372688162 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f50b00bd020 con 0x7f50c8071dc0 2024-04-25T00:15:02.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.646+0000 7f50c5600640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50a40678c0 0x7f50a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.646+0000 7f50c5600640 1 -- 172.21.15.83:0/3372688162 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f50b0120100 con 0x7f50c8071dc0 2024-04-25T00:15:02.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.646+0000 7f50ccc00640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50a40678c0 0x7f50a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.646+0000 7f50ccc00640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50a40678c0 0x7f50a4069d80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f50b800b810 tx=0x7f50b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:02.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.650+0000 7f50c5600640 1 -- 172.21.15.83:0/3372688162 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f50b00c4030 con 0x7f50c8071dc0 2024-04-25T00:15:02.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.806+0000 7f50cf600640 1 -- 172.21.15.83:0/3372688162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50c814ff80 con 0x7f50c8071dc0 2024-04-25T00:15:02.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.806+0000 7f50c5600640 1 -- 172.21.15.83:0/3372688162 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16789 (secure 0 0 0) 0x7f50c8161370 con 0x7f50c8071dc0 2024-04-25T00:15:02.809 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:02.809 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":93,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:14:53.503988+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:50.260735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:50.880394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:15:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.810+0000 7f50cf600640 1 -- 172.21.15.83:0/3372688162 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50a40678c0 msgr2=0x7f50a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.810+0000 7f50cf600640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50a40678c0 0x7f50a4069d80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f50b800b810 tx=0x7f50b8008040 comp rx=0 tx=0).stop 2024-04-25T00:15:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.810+0000 7f50cf600640 1 -- 172.21.15.83:0/3372688162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8071dc0 msgr2=0x7f50c814fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.810+0000 7f50cf600640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8071dc0 0x7f50c814fa40 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f50b00ae040 tx=0x7f50b001ac00 comp rx=0 tx=0).stop 2024-04-25T00:15:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.810+0000 7f50cf600640 1 -- 172.21.15.83:0/3372688162 shutdown_connections 2024-04-25T00:15:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.810+0000 7f50cf600640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f50a40678c0 0x7f50a4069d80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.810+0000 7f50cf600640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50c815df00 0x7f50c8153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.810+0000 7f50cf600640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8071dc0 0x7f50c814fa40 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.810+0000 7f50cf600640 1 --2- 172.21.15.83:0/3372688162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50c806e7b0 0x7f50c8159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.810+0000 7f50cf600640 1 -- 172.21.15.83:0/3372688162 >> 172.21.15.83:0/3372688162 conn(0x7f50c80b1830 msgr2=0x7f50c80cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.810+0000 7f50cf600640 1 -- 172.21.15.83:0/3372688162 shutdown_connections 2024-04-25T00:15:02.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.810+0000 7f50cf600640 1 -- 172.21.15.83:0/3372688162 wait complete. 2024-04-25T00:15:02.830 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:14:50.260735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:50.880394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:15:02.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:15:02.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.966+0000 7f6992e00640 1 Processor -- start 2024-04-25T00:15:02.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.966+0000 7f6992e00640 1 -- start start 2024-04-25T00:15:02.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.970+0000 7f6992e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f698c06e950 0x7f698c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.970+0000 7f6992e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f698c15fe50 0x7f698c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.970+0000 7f6992e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698c0ce000 con 0x7f698c159810 2024-04-25T00:15:02.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.970+0000 7f6992e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f698c06e210 con 0x7f698c06e950 2024-04-25T00:15:02.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.970+0000 7f6992e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f698c06da00 con 0x7f698c15fe50 2024-04-25T00:15:02.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.970+0000 7f6983400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f698c06e950 0x7f698c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.970+0000 7f6983400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f698c06e950 0x7f698c15f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52074/0 (socket says 172.21.15.83:52074) 2024-04-25T00:15:02.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.970+0000 7f6983400640 1 -- 172.21.15.83:0/1992000431 learned_addr learned my addr 172.21.15.83:0/1992000431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:02.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.970+0000 7f6990e00640 1 --2- 172.21.15.83:0/1992000431 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f698c15fe50 0x7f698c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.970+0000 7f6983400640 1 -- 172.21.15.83:0/1992000431 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f698c15fe50 msgr2=0x7f698c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:02.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.970+0000 7f6983400640 1 --2- 172.21.15.83:0/1992000431 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f698c15fe50 0x7f698c0ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.970+0000 7f6983400640 1 -- 172.21.15.83:0/1992000431 >> v1:172.21.15.83:6789/0 conn(0x7f698c159810 legacy=0x7f698c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.970+0000 7f6983400640 1 -- 172.21.15.83:0/1992000431 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c06eec0 con 0x7f698c06e950 2024-04-25T00:15:02.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.970+0000 7f698b400640 1 -- 172.21.15.83:0/1992000431 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1826090860 0 0) 0x7f698c0ce000 con 0x7f698c159810 2024-04-25T00:15:02.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6983400640 1 --2- 172.21.15.83:0/1992000431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c06e950 0x7f698c15f910 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f698c06db80 tx=0x7f697401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac394da31b4d97f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:02.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f698b400640 1 -- 172.21.15.83:0/1992000431 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740b1070 con 0x7f698c06e950 2024-04-25T00:15:02.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f698b400640 1 -- 172.21.15.83:0/1992000431 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69740ac600 con 0x7f698c06e950 2024-04-25T00:15:02.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f698b400640 1 -- 172.21.15.83:0/1992000431 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740ac9d0 con 0x7f698c06e950 2024-04-25T00:15:02.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 -- 172.21.15.83:0/1992000431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c06e950 msgr2=0x7f698c15f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 --2- 172.21.15.83:0/1992000431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c06e950 0x7f698c15f910 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f698c06db80 tx=0x7f697401c4b0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 -- 172.21.15.83:0/1992000431 shutdown_connections 2024-04-25T00:15:02.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 --2- 172.21.15.83:0/1992000431 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f698c15fe50 0x7f698c0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 --2- 172.21.15.83:0/1992000431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c06e950 0x7f698c15f910 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 -- 172.21.15.83:0/1992000431 >> 172.21.15.83:0/1992000431 conn(0x7f698c0b1830 msgr2=0x7f698c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 -- 172.21.15.83:0/1992000431 shutdown_connections 2024-04-25T00:15:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 -- 172.21.15.83:0/1992000431 wait complete. 2024-04-25T00:15:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 Processor -- start 2024-04-25T00:15:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 -- start start 2024-04-25T00:15:02.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c06e950 0x7f698c157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c159810 0x7f698c153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f698c15fe50 0x7f698c154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f698c070f20 con 0x7f698c159810 2024-04-25T00:15:02.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f698c070da0 con 0x7f698c06e950 2024-04-25T00:15:02.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6992e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f698c168cb0 con 0x7f698c15fe50 2024-04-25T00:15:02.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6983400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c159810 0x7f698c153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6983400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c159810 0x7f698c153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52086/0 (socket says 172.21.15.83:52086) 2024-04-25T00:15:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6983400640 1 -- 172.21.15.83:0/2038861139 learned_addr learned my addr 172.21.15.83:0/2038861139 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f698be00640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c06e950 0x7f698c157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.974+0000 7f6990e00640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f698c15fe50 0x7f698c154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6983400640 1 -- 172.21.15.83:0/2038861139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f698c15fe50 msgr2=0x7f698c154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6983400640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f698c15fe50 0x7f698c154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6983400640 1 -- 172.21.15.83:0/2038861139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c06e950 msgr2=0x7f698c157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6983400640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c06e950 0x7f698c157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6983400640 1 -- 172.21.15.83:0/2038861139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c15b020 con 0x7f698c159810 2024-04-25T00:15:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f698be00640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c06e950 0x7f698c157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6983400640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c159810 0x7f698c153cd0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f697401ab60 tx=0x7f697401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6988c00640 1 -- 172.21.15.83:0/2038861139 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740bb020 con 0x7f698c159810 2024-04-25T00:15:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6988c00640 1 -- 172.21.15.83:0/2038861139 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69740b7620 con 0x7f698c159810 2024-04-25T00:15:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6988c00640 1 -- 172.21.15.83:0/2038861139 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740b1040 con 0x7f698c159810 2024-04-25T00:15:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6992e00640 1 -- 172.21.15.83:0/2038861139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c15b300 con 0x7f698c159810 2024-04-25T00:15:02.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6992e00640 1 -- 172.21.15.83:0/2038861139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c15b8c0 con 0x7f698c159810 2024-04-25T00:15:02.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6992e00640 1 -- 172.21.15.83:0/2038861139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698c04f510 con 0x7f698c159810 2024-04-25T00:15:02.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6988c00640 1 -- 172.21.15.83:0/2038861139 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f69740be020 con 0x7f698c159810 2024-04-25T00:15:02.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6988c00640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69680678c0 0x7f6968069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f6988c00640 1 -- 172.21.15.83:0/2038861139 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f697411fdf0 con 0x7f698c159810 2024-04-25T00:15:02.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.978+0000 7f698be00640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69680678c0 0x7f6968069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.982+0000 7f698be00640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69680678c0 0x7f6968069d80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f697c000f70 tx=0x7f697c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:02.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:02.982+0000 7f6988c00640 1 -- 172.21.15.83:0/2038861139 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f69740c5070 con 0x7f698c159810 2024-04-25T00:15:03.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.138+0000 7f6992e00640 1 -- 172.21.15.83:0/2038861139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f698c0b24a0 con 0x7f698c159810 2024-04-25T00:15:03.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.138+0000 7f6988c00640 1 -- 172.21.15.83:0/2038861139 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f6974016130 con 0x7f698c159810 2024-04-25T00:15:03.140 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:15:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.142+0000 7f6992e00640 1 -- 172.21.15.83:0/2038861139 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69680678c0 msgr2=0x7f6968069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.142+0000 7f6992e00640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69680678c0 0x7f6968069d80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f697c000f70 tx=0x7f697c009040 comp rx=0 tx=0).stop 2024-04-25T00:15:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.142+0000 7f6992e00640 1 -- 172.21.15.83:0/2038861139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c159810 msgr2=0x7f698c153cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.142+0000 7f6992e00640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c159810 0x7f698c153cd0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f697401ab60 tx=0x7f697401acf0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.142+0000 7f6992e00640 1 -- 172.21.15.83:0/2038861139 shutdown_connections 2024-04-25T00:15:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.142+0000 7f6992e00640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69680678c0 0x7f6968069d80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.142+0000 7f6992e00640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f698c15fe50 0x7f698c154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.142+0000 7f6992e00640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c159810 0x7f698c153cd0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.142+0000 7f6992e00640 1 --2- 172.21.15.83:0/2038861139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c06e950 0x7f698c157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.142+0000 7f6992e00640 1 -- 172.21.15.83:0/2038861139 >> 172.21.15.83:0/2038861139 conn(0x7f698c0b1830 msgr2=0x7f698c0710d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:03.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.142+0000 7f6992e00640 1 -- 172.21.15.83:0/2038861139 shutdown_connections 2024-04-25T00:15:03.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.142+0000 7f6992e00640 1 -- 172.21.15.83:0/2038861139 wait complete. 2024-04-25T00:15:03.161 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:15:03.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.298+0000 7fe95ea00640 1 Processor -- start 2024-04-25T00:15:03.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.298+0000 7fe95ea00640 1 -- start start 2024-04-25T00:15:03.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.298+0000 7fe95ea00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe9581551e0 0x7fe95815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.298+0000 7fe95ea00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9581555c0 0x7fe958071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.298+0000 7fe95ea00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe95815c560 con 0x7fe95806e4c0 2024-04-25T00:15:03.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.298+0000 7fe95ea00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe958072460 con 0x7fe9581551e0 2024-04-25T00:15:03.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.298+0000 7fe95ea00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe95815c7d0 con 0x7fe9581555c0 2024-04-25T00:15:03.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.302+0000 7fe957e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe9581551e0 0x7fe95815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.302+0000 7fe957e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe9581551e0 0x7fe95815bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52092/0 (socket says 172.21.15.83:52092) 2024-04-25T00:15:03.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.302+0000 7fe957e00640 1 -- 172.21.15.83:0/4108131805 learned_addr learned my addr 172.21.15.83:0/4108131805 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:03.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.302+0000 7fe957400640 1 --2- 172.21.15.83:0/4108131805 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9581555c0 0x7fe958071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.302+0000 7fe957e00640 1 -- 172.21.15.83:0/4108131805 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9581555c0 msgr2=0x7fe958071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:03.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.302+0000 7fe957e00640 1 --2- 172.21.15.83:0/4108131805 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9581555c0 0x7fe958071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.302+0000 7fe957e00640 1 -- 172.21.15.83:0/4108131805 >> v1:172.21.15.86:6789/0 conn(0x7fe95806e4c0 legacy=0x7fe95815e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.302+0000 7fe957e00640 1 -- 172.21.15.83:0/4108131805 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95815cab0 con 0x7fe9581551e0 2024-04-25T00:15:03.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.302+0000 7fe957400640 1 --2- 172.21.15.83:0/4108131805 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9581555c0 0x7fe958071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:03.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.302+0000 7fe957e00640 1 --2- 172.21.15.83:0/4108131805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9581551e0 0x7fe95815bea0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe948009e50 tx=0x7fe94801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14456d756316219 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:03.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.302+0000 7fe956a00640 1 -- 172.21.15.83:0/4108131805 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9480a5820 con 0x7fe9581551e0 2024-04-25T00:15:03.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.302+0000 7fe956a00640 1 -- 172.21.15.83:0/4108131805 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9480a59c0 con 0x7fe9581551e0 2024-04-25T00:15:03.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.302+0000 7fe956a00640 1 -- 172.21.15.83:0/4108131805 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9480a5d90 con 0x7fe9581551e0 2024-04-25T00:15:03.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 -- 172.21.15.83:0/4108131805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9581551e0 msgr2=0x7fe95815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 --2- 172.21.15.83:0/4108131805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9581551e0 0x7fe95815bea0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe948009e50 tx=0x7fe94801ca30 comp rx=0 tx=0).stop 2024-04-25T00:15:03.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 -- 172.21.15.83:0/4108131805 shutdown_connections 2024-04-25T00:15:03.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 --2- 172.21.15.83:0/4108131805 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9581555c0 0x7fe958071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 --2- 172.21.15.83:0/4108131805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9581551e0 0x7fe95815bea0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 -- 172.21.15.83:0/4108131805 >> 172.21.15.83:0/4108131805 conn(0x7fe9580b1830 msgr2=0x7fe9580b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:03.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 -- 172.21.15.83:0/4108131805 shutdown_connections 2024-04-25T00:15:03.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 -- 172.21.15.83:0/4108131805 wait complete. 2024-04-25T00:15:03.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 Processor -- start 2024-04-25T00:15:03.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 -- start start 2024-04-25T00:15:03.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95806e4c0 0x7fe958150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9581551e0 0x7fe95814f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9581555c0 0x7fe958144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe958167e10 con 0x7fe9581555c0 2024-04-25T00:15:03.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe958167c90 con 0x7fe95806e4c0 2024-04-25T00:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ea00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe958167f90 con 0x7fe9581551e0 2024-04-25T00:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9581555c0 0x7fe958144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9581555c0 0x7fe958144870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52094/0 (socket says 172.21.15.83:52094) 2024-04-25T00:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ca00640 1 -- 172.21.15.83:0/1819887519 learned_addr learned my addr 172.21.15.83:0/1819887519 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe957400640 1 --2- 172.21.15.83:0/1819887519 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9581551e0 0x7fe95814f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe957e00640 1 --2- 172.21.15.83:0/1819887519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95806e4c0 0x7fe958150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ca00640 1 -- 172.21.15.83:0/1819887519 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9581551e0 msgr2=0x7fe95814f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ca00640 1 --2- 172.21.15.83:0/1819887519 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9581551e0 0x7fe95814f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ca00640 1 -- 172.21.15.83:0/1819887519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95806e4c0 msgr2=0x7fe958150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ca00640 1 --2- 172.21.15.83:0/1819887519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95806e4c0 0x7fe958150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ca00640 1 -- 172.21.15.83:0/1819887519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe958153df0 con 0x7fe9581555c0 2024-04-25T00:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe95ca00640 1 --2- 172.21.15.83:0/1819887519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9581555c0 0x7fe958144870 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c340 tx=0x7fe95000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.306+0000 7fe94fe00640 1 -- 172.21.15.83:0/1819887519 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950005220 con 0x7fe9581555c0 2024-04-25T00:15:03.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.310+0000 7fe94fe00640 1 -- 172.21.15.83:0/1819887519 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9500053c0 con 0x7fe9581555c0 2024-04-25T00:15:03.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.310+0000 7fe94fe00640 1 -- 172.21.15.83:0/1819887519 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950007270 con 0x7fe9581555c0 2024-04-25T00:15:03.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.310+0000 7fe95ea00640 1 -- 172.21.15.83:0/1819887519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9581540d0 con 0x7fe9581555c0 2024-04-25T00:15:03.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.310+0000 7fe95ea00640 1 -- 172.21.15.83:0/1819887519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9581512b0 con 0x7fe9581555c0 2024-04-25T00:15:03.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.310+0000 7fe95ea00640 1 -- 172.21.15.83:0/1819887519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9581545f0 con 0x7fe9581555c0 2024-04-25T00:15:03.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.310+0000 7fe94fe00640 1 -- 172.21.15.83:0/1819887519 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe950019020 con 0x7fe9581555c0 2024-04-25T00:15:03.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.310+0000 7fe94fe00640 1 --2- 172.21.15.83:0/1819887519 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9380678c0 0x7fe938069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.310+0000 7fe94fe00640 1 -- 172.21.15.83:0/1819887519 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fe950085060 con 0x7fe9581555c0 2024-04-25T00:15:03.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.310+0000 7fe957e00640 1 --2- 172.21.15.83:0/1819887519 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9380678c0 0x7fe938069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.314+0000 7fe957e00640 1 --2- 172.21.15.83:0/1819887519 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9380678c0 0x7fe938069d80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe9480ae000 tx=0x7fe9480a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:03.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.314+0000 7fe94fe00640 1 -- 172.21.15.83:0/1819887519 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe950055320 con 0x7fe9581555c0 2024-04-25T00:15:03.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.470+0000 7fe95ea00640 1 -- 172.21.15.83:0/1819887519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe95804f510 con 0x7fe9581555c0 2024-04-25T00:15:03.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.470+0000 7fe94fe00640 1 -- 172.21.15.83:0/1819887519 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fe950059620 con 0x7fe9581555c0 2024-04-25T00:15:03.472 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:15:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.474+0000 7fe95ea00640 1 -- 172.21.15.83:0/1819887519 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9380678c0 msgr2=0x7fe938069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.474+0000 7fe95ea00640 1 --2- 172.21.15.83:0/1819887519 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9380678c0 0x7fe938069d80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe9480ae000 tx=0x7fe9480a7070 comp rx=0 tx=0).stop 2024-04-25T00:15:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.474+0000 7fe95ea00640 1 -- 172.21.15.83:0/1819887519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9581555c0 msgr2=0x7fe958144870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.474+0000 7fe95ea00640 1 --2- 172.21.15.83:0/1819887519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9581555c0 0x7fe958144870 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c340 tx=0x7fe95000c800 comp rx=0 tx=0).stop 2024-04-25T00:15:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.474+0000 7fe95ea00640 1 -- 172.21.15.83:0/1819887519 shutdown_connections 2024-04-25T00:15:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.474+0000 7fe95ea00640 1 --2- 172.21.15.83:0/1819887519 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9380678c0 0x7fe938069d80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.474+0000 7fe95ea00640 1 --2- 172.21.15.83:0/1819887519 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9581551e0 0x7fe95814f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.474+0000 7fe95ea00640 1 --2- 172.21.15.83:0/1819887519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9581555c0 0x7fe958144870 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.474+0000 7fe95ea00640 1 --2- 172.21.15.83:0/1819887519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95806e4c0 0x7fe958150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.474+0000 7fe95ea00640 1 -- 172.21.15.83:0/1819887519 >> 172.21.15.83:0/1819887519 conn(0x7fe9580b1830 msgr2=0x7fe9580b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.474+0000 7fe95ea00640 1 -- 172.21.15.83:0/1819887519 shutdown_connections 2024-04-25T00:15:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.474+0000 7fe95ea00640 1 -- 172.21.15.83:0/1819887519 wait complete. 2024-04-25T00:15:03.494 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:15:03.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f0105800640 1 Processor -- start 2024-04-25T00:15:03.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f0105800640 1 -- start start 2024-04-25T00:15:03.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f0105800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0100160020 0x7f0100166ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f0105800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01000ce000 con 0x7f0100159810 2024-04-25T00:15:03.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f0105800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f010006eec0 con 0x7f010006e950 2024-04-25T00:15:03.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f0105800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f010006e210 con 0x7f0100160020 2024-04-25T00:15:03.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fea00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0100159810 0x7f01000cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51396/0 (socket says 172.21.15.83:51396) 2024-04-25T00:15:03.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fea00640 1 -- 172.21.15.83:0/305759461 learned_addr learned my addr 172.21.15.83:0/305759461 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:03.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 -- 172.21.15.83:0/305759461 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 701410831 0 0) 0x7f01000ce000 con 0x7f0100159810 2024-04-25T00:15:03.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 -- 172.21.15.83:0/305759461 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00e4003530 con 0x7f0100159810 2024-04-25T00:15:03.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00ff400640 1 --2- 172.21.15.83:0/305759461 >> v2:172.21.15.86:3300/0 conn(0x7f0100160020 0x7f0100166ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 -- 172.21.15.83:0/305759461 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3181577945 0 0) 0x7f00e4003530 con 0x7f0100159810 2024-04-25T00:15:03.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 -- 172.21.15.83:0/305759461 >> v2:172.21.15.86:3300/0 conn(0x7f0100160020 msgr2=0x7f0100166ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:03.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 --2- 172.21.15.83:0/305759461 >> v2:172.21.15.86:3300/0 conn(0x7f0100160020 0x7f0100166ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 -- 172.21.15.83:0/305759461 >> v1:172.21.15.86:6789/0 conn(0x7f010006e950 legacy=0x7f010015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 -- 172.21.15.83:0/305759461 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010006dc50 con 0x7f0100159810 2024-04-25T00:15:03.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 -- 172.21.15.83:0/305759461 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f00f0001680 con 0x7f0100159810 2024-04-25T00:15:03.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 -- 172.21.15.83:0/305759461 >> v1:172.21.15.83:6789/0 conn(0x7f0100159810 legacy=0x7f01000cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 --2- 172.21.15.83:0/305759461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00e4003a30 0x7f00e4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 --2- 172.21.15.83:0/305759461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e4004810 0x7f00e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 --2- 172.21.15.83:0/305759461 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f00e40052f0 0x7f00e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 -- 172.21.15.83:0/305759461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f010006e210 con 0x7f00e4004810 2024-04-25T00:15:03.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 -- 172.21.15.83:0/305759461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f00e400c240 con 0x7f00e4003a30 2024-04-25T00:15:03.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fd600640 1 -- 172.21.15.83:0/305759461 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f00e400c440 con 0x7f00e40052f0 2024-04-25T00:15:03.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fe000640 1 --2- 172.21.15.83:0/305759461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e4004810 0x7f00e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fea00640 1 --2- 172.21.15.83:0/305759461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00e4003a30 0x7f00e4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00ff400640 1 --2- 172.21.15.83:0/305759461 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f00e40052f0 0x7f00e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fe000640 1 -- 172.21.15.83:0/305759461 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f00e40052f0 msgr2=0x7f00e400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:03.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fe000640 1 --2- 172.21.15.83:0/305759461 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f00e40052f0 0x7f00e400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fe000640 1 -- 172.21.15.83:0/305759461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00e4003a30 msgr2=0x7f00e4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:03.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fe000640 1 --2- 172.21.15.83:0/305759461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00e4003a30 0x7f00e4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00fe000640 1 -- 172.21.15.83:0/305759461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e4003530 con 0x7f00e4004810 2024-04-25T00:15:03.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.634+0000 7f00ff400640 1 -- 172.21.15.83:0/305759461 reap_dead start 2024-04-25T00:15:03.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.638+0000 7f00fe000640 1 --2- 172.21.15.83:0/305759461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e4004810 0x7f00e4004c90 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f00ec001750 tx=0x7f00ec01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb8edc1231f3003 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:03.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.638+0000 7f00fd600640 1 -- 172.21.15.83:0/305759461 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0013a0 con 0x7f00e4004810 2024-04-25T00:15:03.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.638+0000 7f00fd600640 1 -- 172.21.15.83:0/305759461 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ec0052f0 con 0x7f00e4004810 2024-04-25T00:15:03.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.638+0000 7f00fd600640 1 -- 172.21.15.83:0/305759461 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec003840 con 0x7f00e4004810 2024-04-25T00:15:03.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.638+0000 7f0105800640 1 -- 172.21.15.83:0/305759461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e4004810 msgr2=0x7f00e4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.638+0000 7f0105800640 1 --2- 172.21.15.83:0/305759461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e4004810 0x7f00e4004c90 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f00ec001750 tx=0x7f00ec01c380 comp rx=0 tx=0).stop 2024-04-25T00:15:03.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.638+0000 7f0105800640 1 -- 172.21.15.83:0/305759461 shutdown_connections 2024-04-25T00:15:03.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.638+0000 7f0105800640 1 --2- 172.21.15.83:0/305759461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e4004810 0x7f00e4004c90 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.638+0000 7f0105800640 1 -- 172.21.15.83:0/305759461 >> 172.21.15.83:0/305759461 conn(0x7f01000b1830 msgr2=0x7f01000b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:03.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.638+0000 7f0105800640 1 -- 172.21.15.83:0/305759461 shutdown_connections 2024-04-25T00:15:03.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.638+0000 7f0105800640 1 -- 172.21.15.83:0/305759461 wait complete. 2024-04-25T00:15:03.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f0105800640 1 Processor -- start 2024-04-25T00:15:03.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f0105800640 1 -- start start 2024-04-25T00:15:03.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f0105800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e4004810 0x7f010013ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f0105800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01001e5900 0x7f01001551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f0105800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0100155710 0x7f01001409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f0105800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0100159d70 con 0x7f00e4004810 2024-04-25T00:15:03.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f0105800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0100159bf0 con 0x7f0100155710 2024-04-25T00:15:03.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f0105800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0100070da0 con 0x7f01001e5900 2024-04-25T00:15:03.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00fea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e4004810 0x7f010013ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00fea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e4004810 0x7f010013ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52102/0 (socket says 172.21.15.83:52102) 2024-04-25T00:15:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00fea00640 1 -- 172.21.15.83:0/3289674488 learned_addr learned my addr 172.21.15.83:0/3289674488 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00fe000640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01001e5900 0x7f01001551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00ff400640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0100155710 0x7f01001409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00fea00640 1 -- 172.21.15.83:0/3289674488 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01001e5900 msgr2=0x7f01001551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00fea00640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01001e5900 0x7f01001551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00fea00640 1 -- 172.21.15.83:0/3289674488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0100155710 msgr2=0x7f01001409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00fea00640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0100155710 0x7f01001409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00fea00640 1 -- 172.21.15.83:0/3289674488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010013f950 con 0x7f00e4004810 2024-04-25T00:15:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00fe000640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01001e5900 0x7f01001551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00fea00640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e4004810 0x7f010013ecc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f00f0000fc0 tx=0x7f00f0007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00f6a00640 1 -- 172.21.15.83:0/3289674488 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0007e00 con 0x7f00e4004810 2024-04-25T00:15:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00f6a00640 1 -- 172.21.15.83:0/3289674488 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f0018070 con 0x7f00e4004810 2024-04-25T00:15:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f0105800640 1 -- 172.21.15.83:0/3289674488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010013fc30 con 0x7f00e4004810 2024-04-25T00:15:03.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f00f6a00640 1 -- 172.21.15.83:0/3289674488 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f000fc00 con 0x7f00e4004810 2024-04-25T00:15:03.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f0105800640 1 -- 172.21.15.83:0/3289674488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01001401f0 con 0x7f00e4004810 2024-04-25T00:15:03.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.642+0000 7f0105800640 1 -- 172.21.15.83:0/3289674488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010004f510 con 0x7f00e4004810 2024-04-25T00:15:03.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.646+0000 7f00f6a00640 1 -- 172.21.15.83:0/3289674488 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f00f001d460 con 0x7f00e4004810 2024-04-25T00:15:03.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.646+0000 7f00f6a00640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f00e00678f0 0x7f00e0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.646+0000 7f00f6a00640 1 -- 172.21.15.83:0/3289674488 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f00f0085e40 con 0x7f00e4004810 2024-04-25T00:15:03.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.646+0000 7f00fe000640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f00e00678f0 0x7f00e0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.646+0000 7f00fe000640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f00e00678f0 0x7f00e0069db0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f00e40057f0 tx=0x7f00ec0011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:03.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.650+0000 7f00f6a00640 1 -- 172.21.15.83:0/3289674488 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f00f0056750 con 0x7f00e4004810 2024-04-25T00:15:03.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.806+0000 7f0105800640 1 -- 172.21.15.83:0/3289674488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f010015b8c0 con 0x7f00e4004810 2024-04-25T00:15:03.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.806+0000 7f00f6a00640 1 -- 172.21.15.83:0/3289674488 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f00f0056050 con 0x7f00e4004810 2024-04-25T00:15:03.809 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:15:03.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.810+0000 7f0105800640 1 -- 172.21.15.83:0/3289674488 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f00e00678f0 msgr2=0x7f00e0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.810+0000 7f0105800640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f00e00678f0 0x7f00e0069db0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f00e40057f0 tx=0x7f00ec0011a0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.810+0000 7f0105800640 1 -- 172.21.15.83:0/3289674488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e4004810 msgr2=0x7f010013ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.810+0000 7f0105800640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e4004810 0x7f010013ecc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f00f0000fc0 tx=0x7f00f0007720 comp rx=0 tx=0).stop 2024-04-25T00:15:03.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.810+0000 7f0105800640 1 -- 172.21.15.83:0/3289674488 shutdown_connections 2024-04-25T00:15:03.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.810+0000 7f0105800640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f00e00678f0 0x7f00e0069db0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.810+0000 7f0105800640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01001e5900 0x7f01001551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.810+0000 7f0105800640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0100155710 0x7f01001409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.810+0000 7f0105800640 1 --2- 172.21.15.83:0/3289674488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e4004810 0x7f010013ecc0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.810+0000 7f0105800640 1 -- 172.21.15.83:0/3289674488 >> 172.21.15.83:0/3289674488 conn(0x7f01000b1830 msgr2=0x7f0100071b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:03.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.810+0000 7f0105800640 1 -- 172.21.15.83:0/3289674488 shutdown_connections 2024-04-25T00:15:03.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.810+0000 7f0105800640 1 -- 172.21.15.83:0/3289674488 wait complete. 2024-04-25T00:15:03.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:15:03.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f1813e00640 1 Processor -- start 2024-04-25T00:15:03.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f1813e00640 1 -- start start 2024-04-25T00:15:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f1813e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f180c0cb640 0x7f180c06dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f1813e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f180c0711a0 con 0x7f180c070d10 2024-04-25T00:15:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f1813e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f180c06e160 con 0x7f180c06cfd0 2024-04-25T00:15:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f1813e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f180c06ea10 con 0x7f180c0cb640 2024-04-25T00:15:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f1811e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f180c070d10 0x7f180c165930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51406/0 (socket says 172.21.15.83:51406) 2024-04-25T00:15:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f1811e00640 1 -- 172.21.15.83:0/1576529760 learned_addr learned my addr 172.21.15.83:0/1576529760 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f1810a00640 1 --2- 172.21.15.83:0/1576529760 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f180c0cb640 0x7f180c06dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f180be00640 1 -- 172.21.15.83:0/1576529760 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1458182312 0 0) 0x7f18040014a0 con 0x7f180c070d10 2024-04-25T00:15:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f180be00640 1 -- 172.21.15.83:0/1576529760 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17f4003530 con 0x7f180c070d10 2024-04-25T00:15:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f180be00640 1 -- 172.21.15.83:0/1576529760 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2761736470 0 0) 0x7f180c06e160 con 0x7f180c06cfd0 2024-04-25T00:15:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f180be00640 1 -- 172.21.15.83:0/1576529760 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18040014a0 con 0x7f180c06cfd0 2024-04-25T00:15:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f1810a00640 1 -- 172.21.15.83:0/1576529760 >> v1:172.21.15.86:6789/0 conn(0x7f180c06cfd0 legacy=0x7f180c06d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f1810a00640 1 -- 172.21.15.83:0/1576529760 >> v1:172.21.15.83:6789/0 conn(0x7f180c070d10 legacy=0x7f180c165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.966+0000 7f1810a00640 1 -- 172.21.15.83:0/1576529760 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c06ecf0 con 0x7f180c0cb640 2024-04-25T00:15:03.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.970+0000 7f1810a00640 1 --2- 172.21.15.83:0/1576529760 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f180c0cb640 0x7f180c06dac0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f17f80097f0 tx=0x7f17f801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eaae3584fbe6f61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.970+0000 7f180be00640 1 -- 172.21.15.83:0/1576529760 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f80ab5f0 con 0x7f180c0cb640 2024-04-25T00:15:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.970+0000 7f180be00640 1 -- 172.21.15.83:0/1576529760 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f80b0070 con 0x7f180c0cb640 2024-04-25T00:15:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.970+0000 7f180be00640 1 -- 172.21.15.83:0/1576529760 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f80ab9b0 con 0x7f180c0cb640 2024-04-25T00:15:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.970+0000 7f1813e00640 1 -- 172.21.15.83:0/1576529760 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f180c0cb640 msgr2=0x7f180c06dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.970+0000 7f1813e00640 1 --2- 172.21.15.83:0/1576529760 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f180c0cb640 0x7f180c06dac0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f17f80097f0 tx=0x7f17f801c4e0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.970+0000 7f1813e00640 1 -- 172.21.15.83:0/1576529760 shutdown_connections 2024-04-25T00:15:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.970+0000 7f1813e00640 1 --2- 172.21.15.83:0/1576529760 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f180c0cb640 0x7f180c06dac0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.970+0000 7f1813e00640 1 -- 172.21.15.83:0/1576529760 >> 172.21.15.83:0/1576529760 conn(0x7f180c0cfa60 msgr2=0x7f180c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.970+0000 7f1813e00640 1 -- 172.21.15.83:0/1576529760 shutdown_connections 2024-04-25T00:15:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.970+0000 7f1813e00640 1 -- 172.21.15.83:0/1576529760 wait complete. 2024-04-25T00:15:03.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1813e00640 1 Processor -- start 2024-04-25T00:15:03.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1813e00640 1 -- start start 2024-04-25T00:15:03.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1813e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c06cfd0 0x7f180c15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1813e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f180c070d10 0x7f180c138560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1813e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f180c0cb640 0x7f180c158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1813e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f180c169c00 con 0x7f180c06cfd0 2024-04-25T00:15:03.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1813e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f180c169a80 con 0x7f180c0cb640 2024-04-25T00:15:03.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1813e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f180c169d80 con 0x7f180c070d10 2024-04-25T00:15:03.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1811400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c06cfd0 0x7f180c15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1811400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c06cfd0 0x7f180c15e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52104/0 (socket says 172.21.15.83:52104) 2024-04-25T00:15:03.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1811400640 1 -- 172.21.15.83:0/3075173440 learned_addr learned my addr 172.21.15.83:0/3075173440 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:03.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1810a00640 1 --2- 172.21.15.83:0/3075173440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f180c070d10 0x7f180c138560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1811e00640 1 --2- 172.21.15.83:0/3075173440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f180c0cb640 0x7f180c158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1811400640 1 -- 172.21.15.83:0/3075173440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f180c070d10 msgr2=0x7f180c138560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1811400640 1 --2- 172.21.15.83:0/3075173440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f180c070d10 0x7f180c138560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1811400640 1 -- 172.21.15.83:0/3075173440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f180c0cb640 msgr2=0x7f180c158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:03.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1811400640 1 --2- 172.21.15.83:0/3075173440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f180c0cb640 0x7f180c158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:03.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1811400640 1 -- 172.21.15.83:0/3075173440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c138bf0 con 0x7f180c06cfd0 2024-04-25T00:15:03.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1811400640 1 --2- 172.21.15.83:0/3075173440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c06cfd0 0x7f180c15e3b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f180000c4c0 tx=0x7f180000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:03.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1809600640 1 -- 172.21.15.83:0/3075173440 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18000052d0 con 0x7f180c06cfd0 2024-04-25T00:15:03.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1809600640 1 -- 172.21.15.83:0/3075173440 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1800005470 con 0x7f180c06cfd0 2024-04-25T00:15:03.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1809600640 1 -- 172.21.15.83:0/3075173440 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800012870 con 0x7f180c06cfd0 2024-04-25T00:15:03.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1813e00640 1 -- 172.21.15.83:0/3075173440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180c138ed0 con 0x7f180c06cfd0 2024-04-25T00:15:03.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.974+0000 7f1813e00640 1 -- 172.21.15.83:0/3075173440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180c0ce520 con 0x7f180c06cfd0 2024-04-25T00:15:03.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.978+0000 7f1813e00640 1 -- 172.21.15.83:0/3075173440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180c068230 con 0x7f180c06cfd0 2024-04-25T00:15:03.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.978+0000 7f1809600640 1 -- 172.21.15.83:0/3075173440 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1800012a10 con 0x7f180c06cfd0 2024-04-25T00:15:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.978+0000 7f1809600640 1 --2- 172.21.15.83:0/3075173440 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17ec0678c0 0x7f17ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.978+0000 7f1809600640 1 -- 172.21.15.83:0/3075173440 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f1800085980 con 0x7f180c06cfd0 2024-04-25T00:15:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.978+0000 7f1810a00640 1 --2- 172.21.15.83:0/3075173440 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17ec0678c0 0x7f17ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.978+0000 7f1810a00640 1 --2- 172.21.15.83:0/3075173440 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17ec0678c0 0x7f17ec069d80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f17f80af000 tx=0x7f17f80a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:03.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:03.982+0000 7f1809600640 1 -- 172.21.15.83:0/3075173440 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1800055ce0 con 0x7f180c06cfd0 2024-04-25T00:15:04.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.146+0000 7f1813e00640 1 -- 172.21.15.83:0/3075173440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f180c0d00f0 con 0x7f180c06cfd0 2024-04-25T00:15:04.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.146+0000 7f1809600640 1 -- 172.21.15.83:0/3075173440 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v93) ==== 80+0+0 (secure 0 0 0) 0x7f1800059fe0 con 0x7f180c06cfd0 2024-04-25T00:15:04.149 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-25T00:15:04.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.150+0000 7f1813e00640 1 -- 172.21.15.83:0/3075173440 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17ec0678c0 msgr2=0x7f17ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.150+0000 7f1813e00640 1 --2- 172.21.15.83:0/3075173440 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17ec0678c0 0x7f17ec069d80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f17f80af000 tx=0x7f17f80a7000 comp rx=0 tx=0).stop 2024-04-25T00:15:04.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.150+0000 7f1813e00640 1 -- 172.21.15.83:0/3075173440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c06cfd0 msgr2=0x7f180c15e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.150+0000 7f1813e00640 1 --2- 172.21.15.83:0/3075173440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c06cfd0 0x7f180c15e3b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f180000c4c0 tx=0x7f180000c980 comp rx=0 tx=0).stop 2024-04-25T00:15:04.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.150+0000 7f1813e00640 1 -- 172.21.15.83:0/3075173440 shutdown_connections 2024-04-25T00:15:04.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.150+0000 7f1813e00640 1 --2- 172.21.15.83:0/3075173440 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17ec0678c0 0x7f17ec069d80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.150+0000 7f1813e00640 1 --2- 172.21.15.83:0/3075173440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f180c070d10 0x7f180c138560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.150+0000 7f1813e00640 1 --2- 172.21.15.83:0/3075173440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f180c0cb640 0x7f180c158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.150+0000 7f1813e00640 1 --2- 172.21.15.83:0/3075173440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c06cfd0 0x7f180c15e3b0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.150+0000 7f1813e00640 1 -- 172.21.15.83:0/3075173440 >> 172.21.15.83:0/3075173440 conn(0x7f180c0cfa60 msgr2=0x7f180c13c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:04.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.150+0000 7f1813e00640 1 -- 172.21.15.83:0/3075173440 shutdown_connections 2024-04-25T00:15:04.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.150+0000 7f1813e00640 1 -- 172.21.15.83:0/3075173440 wait complete. 2024-04-25T00:15:04.171 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T00:15:04.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:04.171 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T00:15:04.200 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:15:04.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:04.200 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:15:04.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:15:04.258 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:15:04.300 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-04-25T00:15:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2024-04-25T00:15:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:15:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2024-04-25T00:15:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2024-04-25T00:15:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2024-04-25T00:15:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2024-04-25T00:15:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2024-04-25T00:15:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:15:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:15:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2024-04-25T00:15:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:15:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2024-04-25T00:15:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:15:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:15:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2024-04-25T00:15:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:15:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2024-04-25T00:15:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2024-04-25T00:15:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2024-04-25T00:15:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2024-04-25T00:15:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2024-04-25T00:15:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.83,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]" 2024-04-25T00:15:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on skipped message = False 2024-04-25T00:15:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = d58ae83a-ad72-4e18-824a-86e779979c35 2024-04-25T00:15:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.303 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2024-04-25T00:15:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2024-04-25T00:15:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2024-04-25T00:15:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2024-04-25T00:15:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:15:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2024-04-25T00:15:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2024-04-25T00:15:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2024-04-25T00:15:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2024-04-25T00:15:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2024-04-25T00:15:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2024-04-25T00:15:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2024-04-25T00:15:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2024-04-25T00:15:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:15:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 20 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 20 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2024-04-25T00:15:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 10 2024-04-25T00:15:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:15:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2024-04-25T00:15:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:15:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:15:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2024-04-25T00:15:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-04-25T00:15:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.306 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2024-04-25T00:15:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:15:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2024-04-25T00:15:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-04-25T00:15:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-04-25T00:15:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:15:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:15:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2024-04-25T00:15:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-04-25T00:15:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.307 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2024-04-25T00:15:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:15:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-04-25T00:15:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2024-04-25T00:15:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-04-25T00:15:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2024-04-25T00:15:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2024-04-25T00:15:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:15:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2024-04-25T00:15:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2024-04-25T00:15:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:15:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:15:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:15:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:15:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:15:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:15:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:15:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2024-04-25T00:15:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:15:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: mon down mkfs grace = 300 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: mon op complaint time = 120 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: client mount timeout = 600 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = "" 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-04-25T00:15:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout:[mds] 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds = 20 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds balancer = 20 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug frag = True 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug scatterstat = True 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: mds op complaint time = 180 2024-04-25T00:15:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: mds verify scatter = True 2024-04-25T00:15:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-04-25T00:15:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-04-25T00:15:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-04-25T00:15:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: mds_sessionmap_keys_per_op = 1 2024-04-25T00:15:04.341 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-25T00:15:04.341 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-04-25T00:15:04.341 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:15:04.341 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-04-25T00:15:04.341 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.341 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-04-25T00:15:04.341 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.341 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-04-25T00:15:04.342 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.342 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-04-25T00:15:04.342 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-04-25T00:15:04.342 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.342 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-04-25T00:15:04.342 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-04-25T00:15:04.342 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.342 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:15:04.342 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-04-25T00:15:04.342 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.342 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:15:04.342 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:15:04.342 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.343 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-04-25T00:15:04.343 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.343 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-04-25T00:15:04.343 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-04-25T00:15:04.343 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:15:04.343 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:15:04.343 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-04-25T00:15:04.343 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:15:04.343 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-04-25T00:15:04.343 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:15:04.343 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:15:04.343 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-04-25T00:15:04.343 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.83,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]" 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on skipped message = False 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = d58ae83a-ad72-4e18-824a-86e779979c35 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-04-25T00:15:04.344 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-04-25T00:15:04.345 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.345 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-04-25T00:15:04.345 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-04-25T00:15:04.345 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:15:04.345 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.345 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-04-25T00:15:04.345 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-04-25T00:15:04.345 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.345 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-04-25T00:15:04.345 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-04-25T00:15:04.345 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-04-25T00:15:04.345 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-04-25T00:15:04.345 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.346 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-04-25T00:15:04.346 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-04-25T00:15:04.346 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.346 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:15:04.346 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.346 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-04-25T00:15:04.346 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.346 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-04-25T00:15:04.346 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.346 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:15:04.346 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-04-25T00:15:04.346 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.346 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-04-25T00:15:04.346 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-04-25T00:15:04.347 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-04-25T00:15:04.347 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-04-25T00:15:04.347 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 20 2024-04-25T00:15:04.347 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 20 2024-04-25T00:15:04.347 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:15:04.347 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-04-25T00:15:04.347 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 10 2024-04-25T00:15:04.347 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:15:04.347 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-04-25T00:15:04.347 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:15:04.348 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:15:04.348 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-04-25T00:15:04.348 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-25T00:15:04.348 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.348 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-04-25T00:15:04.348 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:15:04.348 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-04-25T00:15:04.348 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-25T00:15:04.348 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-25T00:15:04.348 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:15:04.348 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:15:04.348 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-04-25T00:15:04.348 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-25T00:15:04.348 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.349 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-04-25T00:15:04.349 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:15:04.349 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-25T00:15:04.349 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-04-25T00:15:04.349 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-25T00:15:04.349 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-04-25T00:15:04.349 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-04-25T00:15:04.349 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:15:04.349 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-04-25T00:15:04.349 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-04-25T00:15:04.349 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:15:04.350 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.350 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:15:04.350 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:15:04.350 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:15:04.350 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.350 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:15:04.350 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:15:04.350 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:15:04.350 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.350 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-04-25T00:15:04.350 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:15:04.350 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.350 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:15:04.351 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-04-25T00:15:04.351 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-04-25T00:15:04.351 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:04.351 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-04-25T00:15:04.351 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-04-25T00:15:04.351 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-04-25T00:15:04.351 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-04-25T00:15:04.351 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:15:04.351 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:15:04.351 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-04-25T00:15:04.351 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-25T00:15:04.351 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:15:04.352 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-04-25T00:15:04.352 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-25T00:15:04.352 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-25T00:15:04.352 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-04-25T00:15:04.352 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-04-25T00:15:04.352 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-04-25T00:15:04.352 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-25T00:15:04.352 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-04-25T00:15:04.352 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-04-25T00:15:04.353 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-04-25T00:15:04.353 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-04-25T00:15:04.353 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-04-25T00:15:04.353 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-04-25T00:15:04.353 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:15:04.353 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-04-25T00:15:04.353 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = True 2024-04-25T00:15:04.353 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-04-25T00:15:04.354 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = True 2024-04-25T00:15:04.354 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-25T00:15:04.354 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-25T00:15:04.354 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-25T00:15:04.354 INFO:teuthology.orchestra.run.smithi086.stdout: mds_sessionmap_keys_per_op = 1 2024-04-25T00:15:04.357 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:15:04.357 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:15:04.357 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:15:04.357 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:15:04.358 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:15:04.358 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:15:04.359 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:15:04.359 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:15:04.371 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:15:04.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-25T00:15:04.372 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:15:04.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-25T00:15:04.373 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:15:04.374 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-25T00:15:04.374 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:15:04.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-25T00:15:04.454 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:15:04.469 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:15:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.562+0000 7f6de3200640 1 Processor -- start 2024-04-25T00:15:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.562+0000 7f4fd1000640 1 Processor -- start 2024-04-25T00:15:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.562+0000 7f6de3200640 1 -- start start 2024-04-25T00:15:04.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.562+0000 7f4fd1000640 1 -- start start 2024-04-25T00:15:04.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f6de3200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de406d410 0x7f6de406e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f6de3200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6de40c9fd0 0x7f6de40cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f6de3200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6de40ce8f0 con 0x7f6de414aff0 2024-04-25T00:15:04.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f6de3200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6de40cf020 con 0x7f6de40c9fd0 2024-04-25T00:15:04.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f6de3200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6de40ce330 con 0x7f6de406d410 2024-04-25T00:15:04.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f6de1e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6de414aff0 0x7f6de406cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46652/0 (socket says 172.21.15.83:46652) 2024-04-25T00:15:04.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f6de1e00640 1 -- 172.21.15.83:0/3697403017 learned_addr learned my addr 172.21.15.83:0/3697403017 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:04.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f6de1400640 1 --2- 172.21.15.83:0/3697403017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de406d410 0x7f6de406e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f6de2800640 1 --2- 172.21.15.83:0/3697403017 >> v2:172.21.15.86:3300/0 conn(0x7f6de40c9fd0 0x7f6de40cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f4fd1000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4fcc06d5d0 0x7f4fcc15f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f4fd1000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fcc15c200 con 0x7f4fcc15be20 2024-04-25T00:15:04.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f6de0a00640 1 -- 172.21.15.83:0/3697403017 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2074883388 0 0) 0x7f6de40ce8f0 con 0x7f6de414aff0 2024-04-25T00:15:04.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f6de1400640 1 -- 172.21.15.83:0/3697403017 >> v2:172.21.15.86:3300/0 conn(0x7f6de40c9fd0 msgr2=0x7f6de40cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:04.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f6de1400640 1 --2- 172.21.15.83:0/3697403017 >> v2:172.21.15.86:3300/0 conn(0x7f6de40c9fd0 0x7f6de40cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f4fd1000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fcc160100 con 0x7f4fcc0cedd0 2024-04-25T00:15:04.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f4fd1000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4fcc15fed0 con 0x7f4fcc06d5d0 2024-04-25T00:15:04.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f4fc9e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4fcc15be20 0x7f4fcc06cec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46654/0 (socket says 172.21.15.83:46654) 2024-04-25T00:15:04.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f4fc9e00640 1 -- 172.21.15.83:0/3453276859 learned_addr learned my addr 172.21.15.83:0/3453276859 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:04.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f6de1400640 1 -- 172.21.15.83:0/3697403017 >> v1:172.21.15.83:6789/0 conn(0x7f6de414aff0 legacy=0x7f6de406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f6de1400640 1 -- 172.21.15.83:0/3697403017 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de40ce610 con 0x7f6de406d410 2024-04-25T00:15:04.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f4fc9400640 1 --2- 172.21.15.83:0/3453276859 >> v2:172.21.15.83:3300/0 conn(0x7f4fcc06d5d0 0x7f4fcc15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f4fc9400640 1 -- 172.21.15.83:0/3453276859 >> v1:172.21.15.86:6789/0 conn(0x7f4fcc0cedd0 legacy=0x7f4fcc0b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f4fc9400640 1 -- 172.21.15.83:0/3453276859 >> v1:172.21.15.83:6789/0 conn(0x7f4fcc15be20 legacy=0x7f4fcc06cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.566+0000 7f4fc9400640 1 -- 172.21.15.83:0/3453276859 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc1603e0 con 0x7f4fcc06d5d0 2024-04-25T00:15:04.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.570+0000 7f08dfe00640 1 Processor -- start 2024-04-25T00:15:04.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.570+0000 7f08dfe00640 1 -- start start 2024-04-25T00:15:04.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.570+0000 7f6de1400640 1 --2- 172.21.15.83:0/3697403017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de406d410 0x7f6de406e750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00097c0 tx=0x7f6dd001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f980556e4ad40f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:04.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.570+0000 7f4fc9400640 1 --2- 172.21.15.83:0/3453276859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fcc06d5d0 0x7f4fcc15f990 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80097c0 tx=0x7f4fb801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7dc3ebd79f34c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.570+0000 7f08dfe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f08e00cec40 0x7f08e00d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.570+0000 7f08dfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f08e00d06a0 0x7f08e006ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.570+0000 7f08dfe00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08e00cc9f0 con 0x7f08e0070a30 2024-04-25T00:15:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.570+0000 7f08dfe00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f08e00d0b50 con 0x7f08e00cec40 2024-04-25T00:15:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.570+0000 7f08dfe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f08e00cccd0 con 0x7f08e00d06a0 2024-04-25T00:15:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.570+0000 7f08dea00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f08e0070a30 0x7f08e00ce530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46660/0 (socket says 172.21.15.83:46660) 2024-04-25T00:15:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.570+0000 7f08dea00640 1 -- 172.21.15.83:0/1854845991 learned_addr learned my addr 172.21.15.83:0/1854845991 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.570+0000 7f08d7e00640 1 --2- 172.21.15.83:0/1854845991 >> v2:172.21.15.83:3300/0 conn(0x7f08e00cec40 0x7f08e00d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.570+0000 7f08df400640 1 --2- 172.21.15.83:0/1854845991 >> v2:172.21.15.86:3300/0 conn(0x7f08e00d06a0 0x7f08e006ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f08d7e00640 1 -- 172.21.15.83:0/1854845991 >> v2:172.21.15.86:3300/0 conn(0x7f08e00d06a0 msgr2=0x7f08e006ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:04.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f08d7e00640 1 --2- 172.21.15.83:0/1854845991 >> v2:172.21.15.86:3300/0 conn(0x7f08e00d06a0 0x7f08e006ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f08d7e00640 1 -- 172.21.15.83:0/1854845991 >> v1:172.21.15.83:6789/0 conn(0x7f08e0070a30 legacy=0x7f08e00ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f08d7e00640 1 -- 172.21.15.83:0/1854845991 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e00ccfb0 con 0x7f08e00cec40 2024-04-25T00:15:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f4fc8a00640 1 -- 172.21.15.83:0/3453276859 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80b0070 con 0x7f4fcc06d5d0 2024-04-25T00:15:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f4fc8a00640 1 -- 172.21.15.83:0/3453276859 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb80ab600 con 0x7f4fcc06d5d0 2024-04-25T00:15:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f4fc8a00640 1 -- 172.21.15.83:0/3453276859 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80ab9d0 con 0x7f4fcc06d5d0 2024-04-25T00:15:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f4fd1000640 1 -- 172.21.15.83:0/3453276859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fcc06d5d0 msgr2=0x7f4fcc15f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f4fd1000640 1 --2- 172.21.15.83:0/3453276859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fcc06d5d0 0x7f4fcc15f990 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80097c0 tx=0x7f4fb801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f6de0a00640 1 -- 172.21.15.83:0/3697403017 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00aa620 con 0x7f6de406d410 2024-04-25T00:15:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f6de0a00640 1 -- 172.21.15.83:0/3697403017 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd00af070 con 0x7f6de406d410 2024-04-25T00:15:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f6de0a00640 1 -- 172.21.15.83:0/3697403017 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00aa9e0 con 0x7f6de406d410 2024-04-25T00:15:04.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f4fd1000640 1 -- 172.21.15.83:0/3453276859 shutdown_connections 2024-04-25T00:15:04.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f4fd1000640 1 --2- 172.21.15.83:0/3453276859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fcc06d5d0 0x7f4fcc15f990 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f4fd1000640 1 -- 172.21.15.83:0/3453276859 >> 172.21.15.83:0/3453276859 conn(0x7f4fcc0b1b10 msgr2=0x7f4fcc0b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:04.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f4fd1000640 1 -- 172.21.15.83:0/3453276859 shutdown_connections 2024-04-25T00:15:04.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f4fd1000640 1 -- 172.21.15.83:0/3453276859 wait complete. 2024-04-25T00:15:04.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f6de3200640 1 -- 172.21.15.83:0/3697403017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de406d410 msgr2=0x7f6de406e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f6de3200640 1 --2- 172.21.15.83:0/3697403017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de406d410 0x7f6de406e750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00097c0 tx=0x7f6dd001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f6de3200640 1 -- 172.21.15.83:0/3697403017 shutdown_connections 2024-04-25T00:15:04.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f6de3200640 1 --2- 172.21.15.83:0/3697403017 >> v2:172.21.15.86:3300/0 conn(0x7f6de40c9fd0 0x7f6de40cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f6de3200640 1 --2- 172.21.15.83:0/3697403017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de406d410 0x7f6de406e750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f6de3200640 1 -- 172.21.15.83:0/3697403017 >> 172.21.15.83:0/3697403017 conn(0x7f6de40b1650 msgr2=0x7f6de40b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:04.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f6de3200640 1 -- 172.21.15.83:0/3697403017 shutdown_connections 2024-04-25T00:15:04.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f6de3200640 1 -- 172.21.15.83:0/3697403017 wait complete. 2024-04-25T00:15:04.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.574+0000 7f4fd1000640 1 Processor -- start 2024-04-25T00:15:04.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fd1000640 1 -- start start 2024-04-25T00:15:04.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f08d7e00640 1 --2- 172.21.15.83:0/1854845991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e00cec40 0x7f08e00d0160 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f08c80097c0 tx=0x7f08c801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ca44829989ee8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:04.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f08de000640 1 -- 172.21.15.83:0/1854845991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c80b5070 con 0x7f08e00cec40 2024-04-25T00:15:04.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f08de000640 1 -- 172.21.15.83:0/1854845991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c80a8950 con 0x7f08e00cec40 2024-04-25T00:15:04.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f08de000640 1 -- 172.21.15.83:0/1854845991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c80a8d20 con 0x7f08e00cec40 2024-04-25T00:15:04.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de3200640 1 Processor -- start 2024-04-25T00:15:04.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fd1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fcc06d5d0 0x7f4fcc15e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fd1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc0cedd0 0x7f4fcc070e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fd1000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4fcc15be20 0x7f4fcc13b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fd1000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc169f50 con 0x7f4fcc06d5d0 2024-04-25T00:15:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fd1000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc0b2e70 con 0x7f4fcc0cedd0 2024-04-25T00:15:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fd1000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4fcc16a0d0 con 0x7f4fcc15be20 2024-04-25T00:15:04.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de3200640 1 -- start start 2024-04-25T00:15:04.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fc9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fcc06d5d0 0x7f4fcc15e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f08dfe00640 1 -- 172.21.15.83:0/1854845991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e00cec40 msgr2=0x7f08e00d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f08dfe00640 1 --2- 172.21.15.83:0/1854845991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e00cec40 0x7f08e00d0160 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f08c80097c0 tx=0x7f08c801ec50 comp rx=0 tx=0).stop 2024-04-25T00:15:04.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fc9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fcc06d5d0 0x7f4fcc15e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45796/0 (socket says 172.21.15.83:45796) 2024-04-25T00:15:04.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fc9e00640 1 -- 172.21.15.83:0/2876055057 learned_addr learned my addr 172.21.15.83:0/2876055057 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:04.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fc9400640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc0cedd0 0x7f4fcc070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fca800640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4fcc15be20 0x7f4fcc13b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f08dfe00640 1 -- 172.21.15.83:0/1854845991 shutdown_connections 2024-04-25T00:15:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f08dfe00640 1 --2- 172.21.15.83:0/1854845991 >> v2:172.21.15.86:3300/0 conn(0x7f08e00d06a0 0x7f08e006ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f08dfe00640 1 --2- 172.21.15.83:0/1854845991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e00cec40 0x7f08e00d0160 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f08dfe00640 1 -- 172.21.15.83:0/1854845991 >> 172.21.15.83:0/1854845991 conn(0x7f08e00d1bd0 msgr2=0x7f08e00d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:04.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fc9400640 1 -- 172.21.15.83:0/2876055057 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4fcc15be20 msgr2=0x7f4fcc13b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fc9400640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4fcc15be20 0x7f4fcc13b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fc9400640 1 -- 172.21.15.83:0/2876055057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fcc06d5d0 msgr2=0x7f4fcc15e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fc9400640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fcc06d5d0 0x7f4fcc15e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fc9400640 1 -- 172.21.15.83:0/2876055057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc15a050 con 0x7f4fcc0cedd0 2024-04-25T00:15:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fc9e00640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fcc06d5d0 0x7f4fcc15e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:04.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f08dfe00640 1 -- 172.21.15.83:0/1854845991 shutdown_connections 2024-04-25T00:15:04.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f08dfe00640 1 -- 172.21.15.83:0/1854845991 wait complete. 2024-04-25T00:15:04.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de406d410 0x7f6de4161cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de3200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de40c9fd0 0x7f6de40c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6de414aff0 0x7f6de4070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6de40cc950 con 0x7f6de414aff0 2024-04-25T00:15:04.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6de40cc7d0 con 0x7f6de406d410 2024-04-25T00:15:04.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de3200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6de416f340 con 0x7f6de40c9fd0 2024-04-25T00:15:04.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fc9400640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc0cedd0 0x7f4fcc070e60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80ab000 tx=0x7f4fb8019240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:04.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de406d410 0x7f6de4161cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6de414aff0 0x7f6de4070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de406d410 0x7f6de4161cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:53730/0 (socket says 172.21.15.83:53730) 2024-04-25T00:15:04.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de1e00640 1 -- 172.21.15.83:0/1360461765 learned_addr learned my addr 172.21.15.83:0/1360461765 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:04.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fc2000640 1 -- 172.21.15.83:0/2876055057 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80b0040 con 0x7f4fcc0cedd0 2024-04-25T00:15:04.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fd1000640 1 -- 172.21.15.83:0/2876055057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fcc0713a0 con 0x7f4fcc0cedd0 2024-04-25T00:15:04.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fc2000640 1 -- 172.21.15.83:0/2876055057 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb80b9070 con 0x7f4fcc0cedd0 2024-04-25T00:15:04.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fc2000640 1 -- 172.21.15.83:0/2876055057 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80aaa10 con 0x7f4fcc0cedd0 2024-04-25T00:15:04.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de1400640 1 --2- 172.21.15.83:0/1360461765 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de40c9fd0 0x7f6de40c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de1e00640 1 -- 172.21.15.83:0/1360461765 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de40c9fd0 msgr2=0x7f6de40c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de1e00640 1 --2- 172.21.15.83:0/1360461765 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de40c9fd0 0x7f6de40c48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de1e00640 1 -- 172.21.15.83:0/1360461765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6de414aff0 msgr2=0x7f6de4070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de1e00640 1 --2- 172.21.15.83:0/1360461765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6de414aff0 0x7f6de4070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de1e00640 1 -- 172.21.15.83:0/1360461765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de40c4fa0 con 0x7f6de406d410 2024-04-25T00:15:04.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f6de1400640 1 --2- 172.21.15.83:0/1360461765 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de40c9fd0 0x7f6de40c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:04.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.578+0000 7f4fd1000640 1 -- 172.21.15.83:0/2876055057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fcc15a5b0 con 0x7f4fcc0cedd0 2024-04-25T00:15:04.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f6de1e00640 1 --2- 172.21.15.83:0/1360461765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de406d410 0x7f6de4161cb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400c480 tx=0x7f6dd400c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:04.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f92ffc00640 1 Processor -- start 2024-04-25T00:15:04.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f92ffc00640 1 -- start start 2024-04-25T00:15:04.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f6dda000640 1 -- 172.21.15.83:0/1360461765 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4018070 con 0x7f6de406d410 2024-04-25T00:15:04.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f6dda000640 1 -- 172.21.15.83:0/1360461765 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd4007500 con 0x7f6de406d410 2024-04-25T00:15:04.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f6dda000640 1 -- 172.21.15.83:0/1360461765 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4004a30 con 0x7f6de406d410 2024-04-25T00:15:04.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f4fd1000640 1 -- 172.21.15.83:0/2876055057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fcc04f510 con 0x7f4fcc0cedd0 2024-04-25T00:15:04.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f6de3200640 1 -- 172.21.15.83:0/1360461765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de4146010 con 0x7f6de406d410 2024-04-25T00:15:04.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f6de3200640 1 -- 172.21.15.83:0/1360461765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de4071ce0 con 0x7f6de406d410 2024-04-25T00:15:04.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f08dfe00640 1 Processor -- start 2024-04-25T00:15:04.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f08dfe00640 1 -- start start 2024-04-25T00:15:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f4fc2000640 1 -- 172.21.15.83:0/2876055057 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4fb80aabb0 con 0x7f4fcc0cedd0 2024-04-25T00:15:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f92ffc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f92f80c76a0 0x7f92f80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f92ffc00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f806b460 con 0x7f92f8067660 2024-04-25T00:15:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f92ffc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f806b740 con 0x7f92f80cede0 2024-04-25T00:15:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f92ffc00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f92f806b210 con 0x7f92f80c76a0 2024-04-25T00:15:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f92fd200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f92f80c76a0 0x7f92f80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f92fd200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f92f80c76a0 0x7f92f80ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45826/0 (socket says 172.21.15.83:45826) 2024-04-25T00:15:04.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f92fd200640 1 -- 172.21.15.83:0/1765154505 learned_addr learned my addr 172.21.15.83:0/1765154505 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:04.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f6de3200640 1 -- 172.21.15.83:0/1360461765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de404f510 con 0x7f6de406d410 2024-04-25T00:15:04.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f4fc2000640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4fac0678c0 0x7f4fac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f4fc2000640 1 -- 172.21.15.83:0/2876055057 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f4fb811fdb0 con 0x7f4fcc0cedd0 2024-04-25T00:15:04.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f92fd200640 1 -- 172.21.15.83:0/1765154505 >> v1:172.21.15.86:6789/0 conn(0x7f92f80cede0 legacy=0x7f92f8066f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f92fd200640 1 -- 172.21.15.83:0/1765154505 >> v1:172.21.15.83:6789/0 conn(0x7f92f8067660 legacy=0x7f92f815e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:04.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f92fd200640 1 -- 172.21.15.83:0/1765154505 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f806ba20 con 0x7f92f80c76a0 2024-04-25T00:15:04.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f08dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e0070a30 0x7f08e00cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f08dfe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08e00cec40 0x7f08e00c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f08dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e00d06a0 0x7f08e0154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f08dfe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08e00cda90 con 0x7f08e0070a30 2024-04-25T00:15:04.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f08dfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08e00cd570 con 0x7f08e00d06a0 2024-04-25T00:15:04.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f08dfe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f08e00c91e0 con 0x7f08e00cec40 2024-04-25T00:15:04.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f4fc9e00640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4fac0678c0 0x7f4fac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f08d7e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08e00cec40 0x7f08e00c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f08d7e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08e00cec40 0x7f08e00c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:53794/0 (socket says 172.21.15.83:53794) 2024-04-25T00:15:04.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f08d7e00640 1 -- 172.21.15.83:0/157294999 learned_addr learned my addr 172.21.15.83:0/157294999 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:04.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f08df400640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e00d06a0 0x7f08e0154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.582+0000 7f08dea00640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e0070a30 0x7f08e00cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f4fc9e00640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4fac0678c0 0x7f4fac069d80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc005890 tx=0x7f4fbc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:04.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f08d7e00640 1 -- 172.21.15.83:0/157294999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e00d06a0 msgr2=0x7f08e0154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f08d7e00640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e00d06a0 0x7f08e0154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f08d7e00640 1 -- 172.21.15.83:0/157294999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e0070a30 msgr2=0x7f08e00cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f08d7e00640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e0070a30 0x7f08e00cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f6dda000640 1 -- 172.21.15.83:0/1360461765 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6dd401b070 con 0x7f6de406d410 2024-04-25T00:15:04.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f08d7e00640 1 -- 172.21.15.83:0/157294999 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e0154ba0 con 0x7f08e00cec40 2024-04-25T00:15:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f08dea00640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e0070a30 0x7f08e00cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f08d7e00640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08e00cec40 0x7f08e00c8ca0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f08c80b3040 tx=0x7f08c801b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f08d6a00640 1 -- 172.21.15.83:0/157294999 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c80bf070 con 0x7f08e00cec40 2024-04-25T00:15:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f08d6a00640 1 -- 172.21.15.83:0/157294999 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c80b0720 con 0x7f08e00cec40 2024-04-25T00:15:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f08d6a00640 1 -- 172.21.15.83:0/157294999 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c80b5040 con 0x7f08e00cec40 2024-04-25T00:15:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f08dfe00640 1 -- 172.21.15.83:0/157294999 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e015b0f0 con 0x7f08e00cec40 2024-04-25T00:15:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f08dfe00640 1 -- 172.21.15.83:0/157294999 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e00cb1b0 con 0x7f08e00cec40 2024-04-25T00:15:04.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f6dda000640 1 --2- 172.21.15.83:0/1360461765 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dc4067990 0x7f6dc4069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f6dda000640 1 -- 172.21.15.83:0/1360461765 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f6dd4085220 con 0x7f6de406d410 2024-04-25T00:15:04.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f6de1400640 1 --2- 172.21.15.83:0/1360461765 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dc4067990 0x7f6dc4069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f08dfe00640 1 -- 172.21.15.83:0/157294999 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e004f510 con 0x7f08e00cec40 2024-04-25T00:15:04.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f6de1400640 1 --2- 172.21.15.83:0/1360461765 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dc4067990 0x7f6dc4069e50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00ae000 tx=0x7f6dd00a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:04.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f92fd200640 1 --2- 172.21.15.83:0/1765154505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f80c76a0 0x7f92f80ce8a0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009870 tx=0x7f92ec01ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0ae3e7e3c803a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:04.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f4fc2000640 1 -- 172.21.15.83:0/2876055057 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4fb8124070 con 0x7f4fcc0cedd0 2024-04-25T00:15:04.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f08d6a00640 1 -- 172.21.15.83:0/157294999 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f08c80b0b10 con 0x7f08e00cec40 2024-04-25T00:15:04.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f92f7400640 1 -- 172.21.15.83:0/1765154505 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec0ab5f0 con 0x7f92f80c76a0 2024-04-25T00:15:04.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f92f7400640 1 -- 172.21.15.83:0/1765154505 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ec0b0070 con 0x7f92f80c76a0 2024-04-25T00:15:04.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.586+0000 7f92f7400640 1 -- 172.21.15.83:0/1765154505 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec0ab9b0 con 0x7f92f80c76a0 2024-04-25T00:15:04.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 -- 172.21.15.83:0/1765154505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f80c76a0 msgr2=0x7f92f80ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 --2- 172.21.15.83:0/1765154505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f80c76a0 0x7f92f80ce8a0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009870 tx=0x7f92ec01ce30 comp rx=0 tx=0).stop 2024-04-25T00:15:04.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f08d6a00640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08bc0678c0 0x7f08bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 -- 172.21.15.83:0/1765154505 shutdown_connections 2024-04-25T00:15:04.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 --2- 172.21.15.83:0/1765154505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f80c76a0 0x7f92f80ce8a0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 -- 172.21.15.83:0/1765154505 >> 172.21.15.83:0/1765154505 conn(0x7f92f80cd750 msgr2=0x7f92f80cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f08d6a00640 1 -- 172.21.15.83:0/157294999 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f08c8122690 con 0x7f08e00cec40 2024-04-25T00:15:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 -- 172.21.15.83:0/1765154505 shutdown_connections 2024-04-25T00:15:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 -- 172.21.15.83:0/1765154505 wait complete. 2024-04-25T00:15:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f08dea00640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08bc0678c0 0x7f08bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f08dea00640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08bc0678c0 0x7f08bc069d80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f08d0007fd0 tx=0x7f08d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 Processor -- start 2024-04-25T00:15:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 -- start start 2024-04-25T00:15:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92f8067660 0x7f92f8150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f80c76a0 0x7f92f806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f80cede0 0x7f92f806f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92f80c6ce0 con 0x7f92f80c76a0 2024-04-25T00:15:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92f80c6b60 con 0x7f92f80cede0 2024-04-25T00:15:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92ffc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f92f816c060 con 0x7f92f8067660 2024-04-25T00:15:04.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92fd200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92f8067660 0x7f92f8150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92fdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f80cede0 0x7f92f806f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92fd200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92f8067660 0x7f92f8150100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:53810/0 (socket says 172.21.15.83:53810) 2024-04-25T00:15:04.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92fd200640 1 -- 172.21.15.83:0/446082659 learned_addr learned my addr 172.21.15.83:0/446082659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:04.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f92f7e00640 1 --2- 172.21.15.83:0/446082659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f80c76a0 0x7f92f806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.590+0000 7f6dda000640 1 -- 172.21.15.83:0/1360461765 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6de40c4fa0 con 0x7f6de406d410 2024-04-25T00:15:04.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f92fdc00640 1 -- 172.21.15.83:0/446082659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92f8067660 msgr2=0x7f92f8150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f92fdc00640 1 --2- 172.21.15.83:0/446082659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92f8067660 0x7f92f8150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f92fdc00640 1 -- 172.21.15.83:0/446082659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f80c76a0 msgr2=0x7f92f806f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f92fdc00640 1 --2- 172.21.15.83:0/446082659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f80c76a0 0x7f92f806f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f92fdc00640 1 -- 172.21.15.83:0/446082659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f8157210 con 0x7f92f80cede0 2024-04-25T00:15:04.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f92fd200640 1 --2- 172.21.15.83:0/446082659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92f8067660 0x7f92f8150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:04.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f92fdc00640 1 --2- 172.21.15.83:0/446082659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f80cede0 0x7f92f806f820 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f92f000be00 tx=0x7f92f000bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:04.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f92f4c00640 1 -- 172.21.15.83:0/446082659 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0003980 con 0x7f92f80cede0 2024-04-25T00:15:04.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f92f4c00640 1 -- 172.21.15.83:0/446082659 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92f0003b20 con 0x7f92f80cede0 2024-04-25T00:15:04.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f92f4c00640 1 -- 172.21.15.83:0/446082659 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0004020 con 0x7f92f80cede0 2024-04-25T00:15:04.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f92ffc00640 1 -- 172.21.15.83:0/446082659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f81574b0 con 0x7f92f80cede0 2024-04-25T00:15:04.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f92ffc00640 1 -- 172.21.15.83:0/446082659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f8157a70 con 0x7f92f80cede0 2024-04-25T00:15:04.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f08d6a00640 1 -- 172.21.15.83:0/157294999 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f08c80187b0 con 0x7f08e00cec40 2024-04-25T00:15:04.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f92ffc00640 1 -- 172.21.15.83:0/446082659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f804f510 con 0x7f92f80cede0 2024-04-25T00:15:04.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.594+0000 7f92f4c00640 1 -- 172.21.15.83:0/446082659 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f92f0006d10 con 0x7f92f80cede0 2024-04-25T00:15:04.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.598+0000 7f92f4c00640 1 --2- 172.21.15.83:0/446082659 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92d80678f0 0x7f92d8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:04.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.598+0000 7f92f4c00640 1 -- 172.21.15.83:0/446082659 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f92f0085c00 con 0x7f92f80cede0 2024-04-25T00:15:04.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.598+0000 7f92fd200640 1 --2- 172.21.15.83:0/446082659 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92d80678f0 0x7f92d8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:04.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.598+0000 7f92fd200640 1 --2- 172.21.15.83:0/446082659 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92d80678f0 0x7f92d8069db0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009870 tx=0x7f92ec004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:04.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.602+0000 7f92f4c00640 1 -- 172.21.15.83:0/446082659 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f92f0055ec0 con 0x7f92f80cede0 2024-04-25T00:15:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.822+0000 7f92ffc00640 1 -- 172.21.15.83:0/446082659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) -- 0x7f92f806c6b0 con 0x7f92f80cede0 2024-04-25T00:15:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.822+0000 7f08dfe00640 1 -- 172.21.15.83:0/157294999 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) -- 0x7f08e0070e40 con 0x7f08e00cec40 2024-04-25T00:15:04.824 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-25T00:15:04.822+0000 7efeeee00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:15:04.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.830+0000 7f4fd1000640 1 -- 172.21.15.83:0/2876055057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) -- 0x7f4fcc06ea90 con 0x7f4fcc0cedd0 2024-04-25T00:15:04.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.830+0000 7f6de3200640 1 -- 172.21.15.83:0/1360461765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) -- 0x7f6de41429f0 con 0x7f6de406d410 2024-04-25T00:15:04.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.834+0000 7f92f4c00640 1 -- 172.21.15.83:0/446082659 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 7467 v63) ==== 95+0+0 (secure 0 0 0) 0x7f92f005a1c0 con 0x7f92f80cede0 2024-04-25T00:15:04.834 INFO:teuthology.orchestra.run.smithi083.stderr:failed mds gid 7467 2024-04-25T00:15:04.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.834+0000 7f92ffc00640 1 -- 172.21.15.83:0/446082659 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92d80678f0 msgr2=0x7f92d8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.834+0000 7f92ffc00640 1 --2- 172.21.15.83:0/446082659 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92d80678f0 0x7f92d8069db0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009870 tx=0x7f92ec004740 comp rx=0 tx=0).stop 2024-04-25T00:15:04.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.834+0000 7f92ffc00640 1 -- 172.21.15.83:0/446082659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f80cede0 msgr2=0x7f92f806f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.834+0000 7f92ffc00640 1 --2- 172.21.15.83:0/446082659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f80cede0 0x7f92f806f820 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f92f000be00 tx=0x7f92f000bef0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.834+0000 7f92ffc00640 1 -- 172.21.15.83:0/446082659 shutdown_connections 2024-04-25T00:15:04.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.834+0000 7f92ffc00640 1 --2- 172.21.15.83:0/446082659 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f92d80678f0 0x7f92d8069db0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.834+0000 7f92ffc00640 1 --2- 172.21.15.83:0/446082659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f80cede0 0x7f92f806f820 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.834+0000 7f92ffc00640 1 --2- 172.21.15.83:0/446082659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f80c76a0 0x7f92f806f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.834+0000 7f92ffc00640 1 --2- 172.21.15.83:0/446082659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92f8067660 0x7f92f8150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.834+0000 7f92ffc00640 1 -- 172.21.15.83:0/446082659 >> 172.21.15.83:0/446082659 conn(0x7f92f80cd750 msgr2=0x7f92f8151430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:04.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.834+0000 7f92ffc00640 1 -- 172.21.15.83:0/446082659 shutdown_connections 2024-04-25T00:15:04.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.834+0000 7f92ffc00640 1 -- 172.21.15.83:0/446082659 wait complete. 2024-04-25T00:15:04.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.842+0000 7f08d6a00640 1 -- 172.21.15.83:0/157294999 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 7464 v64) ==== 95+0+0 (secure 0 0 0) 0x7f08c80f3040 con 0x7f08e00cec40 2024-04-25T00:15:04.843 INFO:teuthology.orchestra.run.smithi083.stderr:failed mds gid 7464 2024-04-25T00:15:04.843 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-25T00:15:04.842+0000 7efeeee00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:15:04.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.846+0000 7f08dfe00640 1 -- 172.21.15.83:0/157294999 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08bc0678c0 msgr2=0x7f08bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.846+0000 7f08dfe00640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08bc0678c0 0x7f08bc069d80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f08d0007fd0 tx=0x7f08d0009040 comp rx=0 tx=0).stop 2024-04-25T00:15:04.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.846+0000 7f08dfe00640 1 -- 172.21.15.83:0/157294999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08e00cec40 msgr2=0x7f08e00c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.846+0000 7f08dfe00640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08e00cec40 0x7f08e00c8ca0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f08c80b3040 tx=0x7f08c801b9f0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.846+0000 7f08dfe00640 1 -- 172.21.15.83:0/157294999 shutdown_connections 2024-04-25T00:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.846+0000 7f08dfe00640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08bc0678c0 0x7f08bc069d80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.846+0000 7f08dfe00640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08e00cec40 0x7f08e00c8ca0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.846+0000 7f08dfe00640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e00d06a0 0x7f08e0154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.846+0000 7f08dfe00640 1 --2- 172.21.15.83:0/157294999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e0070a30 0x7f08e00cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.846+0000 7f08dfe00640 1 -- 172.21.15.83:0/157294999 >> 172.21.15.83:0/157294999 conn(0x7f08e00d1bd0 msgr2=0x7f08e00af590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.846+0000 7f08dfe00640 1 -- 172.21.15.83:0/157294999 shutdown_connections 2024-04-25T00:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.846+0000 7f08dfe00640 1 -- 172.21.15.83:0/157294999 wait complete. 2024-04-25T00:15:04.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.850+0000 7f4fc2000640 1 -- 172.21.15.83:0/2876055057 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 7455 v65) ==== 95+0+0 (secure 0 0 0) 0x7f4fcc15a050 con 0x7f4fcc0cedd0 2024-04-25T00:15:04.853 INFO:teuthology.orchestra.run.smithi083.stderr:failed mds gid 7455 2024-04-25T00:15:04.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.854+0000 7f4fd1000640 1 -- 172.21.15.83:0/2876055057 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4fac0678c0 msgr2=0x7f4fac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.854+0000 7f4fd1000640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4fac0678c0 0x7f4fac069d80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc005890 tx=0x7f4fbc009040 comp rx=0 tx=0).stop 2024-04-25T00:15:04.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.854+0000 7f4fd1000640 1 -- 172.21.15.83:0/2876055057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc0cedd0 msgr2=0x7f4fcc070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:04.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.854+0000 7f4fd1000640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc0cedd0 0x7f4fcc070e60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80ab000 tx=0x7f4fb8019240 comp rx=0 tx=0).stop 2024-04-25T00:15:04.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.854+0000 7f4fd1000640 1 -- 172.21.15.83:0/2876055057 shutdown_connections 2024-04-25T00:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.854+0000 7f4fd1000640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4fac0678c0 0x7f4fac069d80 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.854+0000 7f4fd1000640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4fcc15be20 0x7f4fcc13b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.854+0000 7f4fd1000640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc0cedd0 0x7f4fcc070e60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.854+0000 7f4fd1000640 1 --2- 172.21.15.83:0/2876055057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fcc06d5d0 0x7f4fcc15e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.854+0000 7f4fd1000640 1 -- 172.21.15.83:0/2876055057 >> 172.21.15.83:0/2876055057 conn(0x7f4fcc0b1b10 msgr2=0x7f4fcc0b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.854+0000 7f4fd1000640 1 -- 172.21.15.83:0/2876055057 shutdown_connections 2024-04-25T00:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:04.854+0000 7f4fd1000640 1 -- 172.21.15.83:0/2876055057 wait complete. 2024-04-25T00:15:04.865 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:15:04.865 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:15:04.868 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:15:04.868 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:04.881 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:15:04.881 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:15:04.884 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:15:04.884 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:15:04.885 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:15:04.885 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:04.931 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:15:04.932 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:04.953 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2024-04-25T00:15:04.969 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2024-04-25T00:15:05.014 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-25T00:15:05.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:05.850+0000 7f6dda000640 1 -- 172.21.15.83:0/1360461765 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 7452 v66) ==== 95+0+0 (secure 0 0 0) 0x7f6dd40555f0 con 0x7f6de406d410 2024-04-25T00:15:05.852 INFO:teuthology.orchestra.run.smithi083.stderr:failed mds gid 7452 2024-04-25T00:15:05.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:05.854+0000 7f6de3200640 1 -- 172.21.15.83:0/1360461765 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dc4067990 msgr2=0x7f6dc4069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:05.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:05.854+0000 7f6de3200640 1 --2- 172.21.15.83:0/1360461765 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dc4067990 0x7f6dc4069e50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00ae000 tx=0x7f6dd00a7000 comp rx=0 tx=0).stop 2024-04-25T00:15:05.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:05.854+0000 7f6de3200640 1 -- 172.21.15.83:0/1360461765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de406d410 msgr2=0x7f6de4161cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:05.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:05.854+0000 7f6de3200640 1 --2- 172.21.15.83:0/1360461765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de406d410 0x7f6de4161cb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400c480 tx=0x7f6dd400c940 comp rx=0 tx=0).stop 2024-04-25T00:15:05.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:05.854+0000 7f6de3200640 1 -- 172.21.15.83:0/1360461765 shutdown_connections 2024-04-25T00:15:05.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:05.854+0000 7f6de3200640 1 --2- 172.21.15.83:0/1360461765 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6dc4067990 0x7f6dc4069e50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:05.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:05.854+0000 7f6de3200640 1 --2- 172.21.15.83:0/1360461765 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de40c9fd0 0x7f6de40c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:05.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:05.854+0000 7f6de3200640 1 --2- 172.21.15.83:0/1360461765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6de414aff0 0x7f6de4070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:05.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:05.854+0000 7f6de3200640 1 --2- 172.21.15.83:0/1360461765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de406d410 0x7f6de4161cb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:05.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:05.854+0000 7f6de3200640 1 -- 172.21.15.83:0/1360461765 >> 172.21.15.83:0/1360461765 conn(0x7f6de40b1650 msgr2=0x7f6de40af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:05.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:05.854+0000 7f6de3200640 1 -- 172.21.15.83:0/1360461765 shutdown_connections 2024-04-25T00:15:05.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:05.854+0000 7f6de3200640 1 -- 172.21.15.83:0/1360461765 wait complete. 2024-04-25T00:15:05.882 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:15:05.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:15:05.928 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:15:05.928 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:05.928 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:06.015 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-25T00:15:06.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6dd400640 1 Processor -- start 2024-04-25T00:15:06.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6dd400640 1 -- start start 2024-04-25T00:15:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6dd400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6d815e140 0x7fb6d80d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6dd400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb6d80d21f0 0x7fb6d80d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6dd400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6d80d0f50 con 0x7fb6d80d0760 2024-04-25T00:15:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6dd400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb6d80d2650 con 0x7fb6d80d21f0 2024-04-25T00:15:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6dd400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d80d1230 con 0x7fb6d815e140 2024-04-25T00:15:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6d5e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb6d80d21f0 0x7fb6d80d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6d5e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb6d80d21f0 0x7fb6d80d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45884/0 (socket says 172.21.15.83:45884) 2024-04-25T00:15:06.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6d5e00640 1 -- 172.21.15.83:0/2470135810 learned_addr learned my addr 172.21.15.83:0/2470135810 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:06.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6d6800640 1 --2- 172.21.15.83:0/2470135810 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6d815e140 0x7fb6d80d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6d5e00640 1 -- 172.21.15.83:0/2470135810 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6d815e140 msgr2=0x7fb6d80d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:06.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6d5e00640 1 --2- 172.21.15.83:0/2470135810 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6d815e140 0x7fb6d80d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6d5e00640 1 -- 172.21.15.83:0/2470135810 >> v1:172.21.15.86:6789/0 conn(0x7fb6d80d0760 legacy=0x7fb6d80ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.070+0000 7fb6d5e00640 1 -- 172.21.15.83:0/2470135810 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d80d1510 con 0x7fb6d80d21f0 2024-04-25T00:15:06.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.074+0000 7fb6d5e00640 1 --2- 172.21.15.83:0/2470135810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6d80d21f0 0x7fb6d80d0220 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009870 tx=0x7fb6c401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16526ce6218d8508 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:06.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.074+0000 7fb6d5400640 1 -- 172.21.15.83:0/2470135810 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40ab610 con 0x7fb6d80d21f0 2024-04-25T00:15:06.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.074+0000 7fb6d5400640 1 -- 172.21.15.83:0/2470135810 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c40b0070 con 0x7fb6d80d21f0 2024-04-25T00:15:06.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.074+0000 7fb6d5400640 1 -- 172.21.15.83:0/2470135810 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40ab9d0 con 0x7fb6d80d21f0 2024-04-25T00:15:06.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.074+0000 7fb6dd400640 1 -- 172.21.15.83:0/2470135810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6d80d21f0 msgr2=0x7fb6d80d0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.074+0000 7fb6dd400640 1 --2- 172.21.15.83:0/2470135810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6d80d21f0 0x7fb6d80d0220 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009870 tx=0x7fb6c401c560 comp rx=0 tx=0).stop 2024-04-25T00:15:06.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.074+0000 7fb6dd400640 1 -- 172.21.15.83:0/2470135810 shutdown_connections 2024-04-25T00:15:06.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.074+0000 7fb6dd400640 1 --2- 172.21.15.83:0/2470135810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6d80d21f0 0x7fb6d80d0220 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.074+0000 7fb6dd400640 1 --2- 172.21.15.83:0/2470135810 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6d815e140 0x7fb6d80d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.074+0000 7fb6dd400640 1 -- 172.21.15.83:0/2470135810 >> 172.21.15.83:0/2470135810 conn(0x7fb6d809a9f0 msgr2=0x7fb6d809ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:06.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.074+0000 7fb6dd400640 1 -- 172.21.15.83:0/2470135810 shutdown_connections 2024-04-25T00:15:06.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.074+0000 7fb6dd400640 1 -- 172.21.15.83:0/2470135810 wait complete. 2024-04-25T00:15:06.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.074+0000 7fb6dd400640 1 Processor -- start 2024-04-25T00:15:06.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6dd400640 1 -- start start 2024-04-25T00:15:06.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6dd400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6d80d0760 0x7fb6d806f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6dd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d80d21f0 0x7fb6d81556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6dd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6d815e140 0x7fb6d8155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6dd400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d816ae60 con 0x7fb6d815e140 2024-04-25T00:15:06.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6dd400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d816ace0 con 0x7fb6d80d21f0 2024-04-25T00:15:06.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6dd400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d816afe0 con 0x7fb6d80d0760 2024-04-25T00:15:06.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6d7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6d815e140 0x7fb6d8155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6d7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6d815e140 0x7fb6d8155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45900/0 (socket says 172.21.15.83:45900) 2024-04-25T00:15:06.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6d7200640 1 -- 172.21.15.83:0/2821199547 learned_addr learned my addr 172.21.15.83:0/2821199547 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:06.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6d5e00640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d80d21f0 0x7fb6d81556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6d6800640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6d80d0760 0x7fb6d806f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6d7200640 1 -- 172.21.15.83:0/2821199547 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6d80d0760 msgr2=0x7fb6d806f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6d7200640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6d80d0760 0x7fb6d806f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6d7200640 1 -- 172.21.15.83:0/2821199547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d80d21f0 msgr2=0x7fb6d81556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6d7200640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d80d21f0 0x7fb6d81556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6d7200640 1 -- 172.21.15.83:0/2821199547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d80c8f90 con 0x7fb6d815e140 2024-04-25T00:15:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6d5e00640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d80d21f0 0x7fb6d81556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6d6800640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6d80d0760 0x7fb6d806f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6d7200640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6d815e140 0x7fb6d8155c30 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000c770 tx=0x7fb6d000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6cea00640 1 -- 172.21.15.83:0/2821199547 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0003b80 con 0x7fb6d815e140 2024-04-25T00:15:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6cea00640 1 -- 172.21.15.83:0/2821199547 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d0003d20 con 0x7fb6d815e140 2024-04-25T00:15:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6dd400640 1 -- 172.21.15.83:0/2821199547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d80c7cf0 con 0x7fb6d815e140 2024-04-25T00:15:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6cea00640 1 -- 172.21.15.83:0/2821199547 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d000aa90 con 0x7fb6d815e140 2024-04-25T00:15:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6dd400640 1 -- 172.21.15.83:0/2821199547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d80c9510 con 0x7fb6d815e140 2024-04-25T00:15:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.078+0000 7fb6dd400640 1 -- 172.21.15.83:0/2821199547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d804f510 con 0x7fb6d815e140 2024-04-25T00:15:06.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.082+0000 7fb6cea00640 1 -- 172.21.15.83:0/2821199547 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb6d000ac30 con 0x7fb6d815e140 2024-04-25T00:15:06.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.082+0000 7fb6cea00640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6b80678f0 0x7fb6b8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.082+0000 7fb6cea00640 1 -- 172.21.15.83:0/2821199547 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7fb6d0086890 con 0x7fb6d815e140 2024-04-25T00:15:06.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.082+0000 7fb6d6800640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6b80678f0 0x7fb6b8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.082+0000 7fb6d6800640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6b80678f0 0x7fb6b8069db0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb6d806fd80 tx=0x7fb6c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:06.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.086+0000 7fb6cea00640 1 -- 172.21.15.83:0/2821199547 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb6d0057200 con 0x7fb6d815e140 2024-04-25T00:15:06.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.278+0000 7fb6dd400640 1 -- 172.21.15.83:0/2821199547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6d806b800 con 0x7fb6d815e140 2024-04-25T00:15:06.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.278+0000 7fb6cea00640 1 -- 172.21.15.83:0/2821199547 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+4349 (secure 0 0 0) 0x7fb6d0056b00 con 0x7fb6d815e140 2024-04-25T00:15:06.279 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:06.279 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":67,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1992401755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1992401755},{"type":"v1","addr":"172.21.15.86:6833","nonce":1992401755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3621275047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3621275047},{"type":"v1","addr":"172.21.15.86:6835","nonce":3621275047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:51.879542+0000","modified":"2024-04-25T00:15:05.852294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7719":{"gid":7719,"name":"c","rank":0,"incarnation":67,"state":"up:replay","state_seq":1,"addr":"172.21.15.83:6835/565131724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":565131724},{"type":"v1","addr":"172.21.15.83:6835","nonce":565131724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-25T00:15:06.279 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 67 2024-04-25T00:15:06.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.278+0000 7fb6dd400640 1 -- 172.21.15.83:0/2821199547 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6b80678f0 msgr2=0x7fb6b8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.278+0000 7fb6dd400640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6b80678f0 0x7fb6b8069db0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb6d806fd80 tx=0x7fb6c8008040 comp rx=0 tx=0).stop 2024-04-25T00:15:06.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.278+0000 7fb6dd400640 1 -- 172.21.15.83:0/2821199547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6d815e140 msgr2=0x7fb6d8155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.278+0000 7fb6dd400640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6d815e140 0x7fb6d8155c30 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000c770 tx=0x7fb6d000cc30 comp rx=0 tx=0).stop 2024-04-25T00:15:06.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.278+0000 7fb6dd400640 1 -- 172.21.15.83:0/2821199547 shutdown_connections 2024-04-25T00:15:06.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.278+0000 7fb6dd400640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb6b80678f0 0x7fb6b8069db0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.278+0000 7fb6dd400640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6d815e140 0x7fb6d8155c30 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.278+0000 7fb6dd400640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d80d21f0 0x7fb6d81556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.278+0000 7fb6dd400640 1 --2- 172.21.15.83:0/2821199547 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6d80d0760 0x7fb6d806f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.278+0000 7fb6dd400640 1 -- 172.21.15.83:0/2821199547 >> 172.21.15.83:0/2821199547 conn(0x7fb6d809a9f0 msgr2=0x7fb6d8070890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:06.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.278+0000 7fb6dd400640 1 -- 172.21.15.83:0/2821199547 shutdown_connections 2024-04-25T00:15:06.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:06.278+0000 7fb6dd400640 1 -- 172.21.15.83:0/2821199547 wait complete. 2024-04-25T00:15:06.702 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:14:51.879542+0000', 'modified': '2024-04-25T00:15:05.852294+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 96, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7719}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7719': {'gid': 7719, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.83:6835/565131724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 565131724}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 565131724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T00:15:06.703 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7719:up:replay 2024-04-25T00:15:07.704 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:07.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47c1000640 1 Processor -- start 2024-04-25T00:15:07.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47c1000640 1 -- start start 2024-04-25T00:15:07.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47c1000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47bc06fc10 0x7f47bc1691d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:07.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47c1000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47bc0cf6b0 con 0x7f47bc15e080 2024-04-25T00:15:07.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47c1000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47bc0cf990 con 0x7f47bc15e460 2024-04-25T00:15:07.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47c1000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f47bc0cfc70 con 0x7f47bc06fc10 2024-04-25T00:15:07.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47b9e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f47bc15e080 0x7f47bc0cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46710/0 (socket says 172.21.15.83:46710) 2024-04-25T00:15:07.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47b9e00640 1 -- 172.21.15.83:0/4244236515 learned_addr learned my addr 172.21.15.83:0/4244236515 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47ba800640 1 --2- 172.21.15.83:0/4244236515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47bc06fc10 0x7f47bc1691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47b8a00640 1 -- 172.21.15.83:0/4244236515 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1974733830 0 0) 0x7f47bc0cf6b0 con 0x7f47bc15e080 2024-04-25T00:15:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47b8a00640 1 -- 172.21.15.83:0/4244236515 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47a0003530 con 0x7f47bc15e080 2024-04-25T00:15:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47b8a00640 1 -- 172.21.15.83:0/4244236515 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 453719401 0 0) 0x7f47bc0cf990 con 0x7f47bc15e460 2024-04-25T00:15:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47b8a00640 1 -- 172.21.15.83:0/4244236515 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47bc0cf6b0 con 0x7f47bc15e460 2024-04-25T00:15:07.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47ba800640 1 -- 172.21.15.83:0/4244236515 >> v1:172.21.15.86:6789/0 conn(0x7f47bc15e460 legacy=0x7f47bc066f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:07.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47b8a00640 1 -- 172.21.15.83:0/4244236515 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3783839054 0 0) 0x7f47a0003530 con 0x7f47bc15e080 2024-04-25T00:15:07.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47ba800640 1 -- 172.21.15.83:0/4244236515 >> v1:172.21.15.83:6789/0 conn(0x7f47bc15e080 legacy=0x7f47bc0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:07.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.850+0000 7f47ba800640 1 -- 172.21.15.83:0/4244236515 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc162070 con 0x7f47bc06fc10 2024-04-25T00:15:07.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.854+0000 7f47ba800640 1 --2- 172.21.15.83:0/4244236515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47bc06fc10 0x7f47bc1691d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f47b40097f0 tx=0x7f47b401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2d0628132a83714 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:07.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.854+0000 7f47b8a00640 1 -- 172.21.15.83:0/4244236515 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b40b0070 con 0x7f47bc06fc10 2024-04-25T00:15:07.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.854+0000 7f47b8a00640 1 -- 172.21.15.83:0/4244236515 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b40ab400 con 0x7f47bc06fc10 2024-04-25T00:15:07.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.854+0000 7f47b8a00640 1 -- 172.21.15.83:0/4244236515 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac001a00 con 0x7f47bc06fc10 2024-04-25T00:15:07.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.854+0000 7f47c1000640 1 -- 172.21.15.83:0/4244236515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47bc06fc10 msgr2=0x7f47bc1691d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:07.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.854+0000 7f47c1000640 1 --2- 172.21.15.83:0/4244236515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47bc06fc10 0x7f47bc1691d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f47b40097f0 tx=0x7f47b401c400 comp rx=0 tx=0).stop 2024-04-25T00:15:07.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.854+0000 7f47c1000640 1 -- 172.21.15.83:0/4244236515 shutdown_connections 2024-04-25T00:15:07.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.854+0000 7f47c1000640 1 --2- 172.21.15.83:0/4244236515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47bc06fc10 0x7f47bc1691d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:07.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.854+0000 7f47c1000640 1 -- 172.21.15.83:0/4244236515 >> 172.21.15.83:0/4244236515 conn(0x7f47bc0d0cd0 msgr2=0x7f47bc0d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:07.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.854+0000 7f47c1000640 1 -- 172.21.15.83:0/4244236515 shutdown_connections 2024-04-25T00:15:07.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.854+0000 7f47c1000640 1 -- 172.21.15.83:0/4244236515 wait complete. 2024-04-25T00:15:07.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47c1000640 1 Processor -- start 2024-04-25T00:15:07.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47c1000640 1 -- start start 2024-04-25T00:15:07.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47c1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47bc06fc10 0x7f47bc1603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:07.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47c1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47bc15e080 0x7f47bc0b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:07.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47c1000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47bc15e460 0x7f47bc1515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:07.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47c1000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc16bf30 con 0x7f47bc15e080 2024-04-25T00:15:07.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47c1000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc16c0b0 con 0x7f47bc06fc10 2024-04-25T00:15:07.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47c1000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f47bc16c230 con 0x7f47bc15e460 2024-04-25T00:15:07.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47b9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47bc15e080 0x7f47bc0b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:07.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47b9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47bc15e080 0x7f47bc0b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45906/0 (socket says 172.21.15.83:45906) 2024-04-25T00:15:07.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47b9400640 1 -- 172.21.15.83:0/2885482293 learned_addr learned my addr 172.21.15.83:0/2885482293 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:07.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47b9e00640 1 --2- 172.21.15.83:0/2885482293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47bc06fc10 0x7f47bc1603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:07.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47ba800640 1 --2- 172.21.15.83:0/2885482293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47bc15e460 0x7f47bc1515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:07.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47b9400640 1 -- 172.21.15.83:0/2885482293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47bc15e460 msgr2=0x7f47bc1515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:07.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47b9400640 1 --2- 172.21.15.83:0/2885482293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47bc15e460 0x7f47bc1515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:07.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47b9400640 1 -- 172.21.15.83:0/2885482293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47bc06fc10 msgr2=0x7f47bc1603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:07.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47b9400640 1 --2- 172.21.15.83:0/2885482293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47bc06fc10 0x7f47bc1603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:07.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47b9400640 1 -- 172.21.15.83:0/2885482293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc0b01b0 con 0x7f47bc15e080 2024-04-25T00:15:07.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47b9400640 1 --2- 172.21.15.83:0/2885482293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47bc15e080 0x7f47bc0b50f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f47a800c430 tx=0x7f47a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:07.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47b2000640 1 -- 172.21.15.83:0/2885482293 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8017070 con 0x7f47bc15e080 2024-04-25T00:15:07.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47b2000640 1 -- 172.21.15.83:0/2885482293 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47a80052f0 con 0x7f47bc15e080 2024-04-25T00:15:07.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47c1000640 1 -- 172.21.15.83:0/2885482293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47bc161710 con 0x7f47bc15e080 2024-04-25T00:15:07.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47b2000640 1 -- 172.21.15.83:0/2885482293 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8003840 con 0x7f47bc15e080 2024-04-25T00:15:07.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.858+0000 7f47c1000640 1 -- 172.21.15.83:0/2885482293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47bc150b10 con 0x7f47bc15e080 2024-04-25T00:15:07.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.862+0000 7f47c1000640 1 -- 172.21.15.83:0/2885482293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47bc04f510 con 0x7f47bc15e080 2024-04-25T00:15:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.866+0000 7f47b2000640 1 -- 172.21.15.83:0/2885482293 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f47a801a070 con 0x7f47bc15e080 2024-04-25T00:15:07.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.866+0000 7f47b2000640 1 --2- 172.21.15.83:0/2885482293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f479c0678c0 0x7f479c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:07.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.866+0000 7f47b2000640 1 -- 172.21.15.83:0/2885482293 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f47a8085690 con 0x7f47bc15e080 2024-04-25T00:15:07.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.866+0000 7f47b9e00640 1 --2- 172.21.15.83:0/2885482293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f479c0678c0 0x7f479c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:07.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.866+0000 7f47b9e00640 1 --2- 172.21.15.83:0/2885482293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f479c0678c0 0x7f479c069d80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f47ac001570 tx=0x7f47ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:07.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:07.870+0000 7f47b2000640 1 -- 172.21.15.83:0/2885482293 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f47a8021020 con 0x7f47bc15e080 2024-04-25T00:15:08.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:08.054+0000 7f47c1000640 1 -- 172.21.15.83:0/2885482293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47bc06cad0 con 0x7f47bc15e080 2024-04-25T00:15:08.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:08.054+0000 7f47b2000640 1 -- 172.21.15.83:0/2885482293 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5174 (secure 0 0 0) 0x7f47bc161710 con 0x7f47bc15e080 2024-04-25T00:15:08.057 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:08.057 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1992401755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1992401755},{"type":"v1","addr":"172.21.15.86:6833","nonce":1992401755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3621275047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3621275047},{"type":"v1","addr":"172.21.15.86:6835","nonce":3621275047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2870595352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2870595352},{"type":"v1","addr":"172.21.15.83:6837","nonce":2870595352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:51.879542+0000","modified":"2024-04-25T00:15:06.863107+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7719":{"gid":7719,"name":"c","rank":0,"incarnation":67,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.83:6835/565131724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":565131724},{"type":"v1","addr":"172.21.15.83:6835","nonce":565131724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-25T00:15:08.057 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-25T00:15:08.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:08.058+0000 7f47c1000640 1 -- 172.21.15.83:0/2885482293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f479c0678c0 msgr2=0x7f479c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:08.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:08.058+0000 7f47c1000640 1 --2- 172.21.15.83:0/2885482293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f479c0678c0 0x7f479c069d80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f47ac001570 tx=0x7f47ac009040 comp rx=0 tx=0).stop 2024-04-25T00:15:08.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:08.058+0000 7f47c1000640 1 -- 172.21.15.83:0/2885482293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47bc15e080 msgr2=0x7f47bc0b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:08.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:08.058+0000 7f47c1000640 1 --2- 172.21.15.83:0/2885482293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47bc15e080 0x7f47bc0b50f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f47a800c430 tx=0x7f47a800c8f0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:08.058+0000 7f47c1000640 1 -- 172.21.15.83:0/2885482293 shutdown_connections 2024-04-25T00:15:08.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:08.058+0000 7f47c1000640 1 --2- 172.21.15.83:0/2885482293 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f479c0678c0 0x7f479c069d80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:08.058+0000 7f47c1000640 1 --2- 172.21.15.83:0/2885482293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47bc15e460 0x7f47bc1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:08.058+0000 7f47c1000640 1 --2- 172.21.15.83:0/2885482293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47bc15e080 0x7f47bc0b50f0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:08.058+0000 7f47c1000640 1 --2- 172.21.15.83:0/2885482293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47bc06fc10 0x7f47bc1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:08.058+0000 7f47c1000640 1 -- 172.21.15.83:0/2885482293 >> 172.21.15.83:0/2885482293 conn(0x7f47bc0d0cd0 msgr2=0x7f47bc06b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:08.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:08.058+0000 7f47c1000640 1 -- 172.21.15.83:0/2885482293 shutdown_connections 2024-04-25T00:15:08.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:08.058+0000 7f47c1000640 1 -- 172.21.15.83:0/2885482293 wait complete. 2024-04-25T00:15:08.480 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:14:51.879542+0000', 'modified': '2024-04-25T00:15:06.863107+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 96, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7719}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7719': {'gid': 7719, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.83:6835/565131724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 565131724}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 565131724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T00:15:08.480 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7719:up:rejoin 2024-04-25T00:15:09.482 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:09.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.626+0000 7f24f5800640 1 Processor -- start 2024-04-25T00:15:09.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.626+0000 7f24f5800640 1 -- start start 2024-04-25T00:15:09.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24f5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f24f0155470 0x7f24f00ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:09.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24f5800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24f00ae0a0 con 0x7f24f006ed90 2024-04-25T00:15:09.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24f5800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24f00ae380 con 0x7f24f00adb00 2024-04-25T00:15:09.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24f5800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f24f00ae660 con 0x7f24f0155470 2024-04-25T00:15:09.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ef400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f24f006ed90 0x7f24f0071360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46712/0 (socket says 172.21.15.83:46712) 2024-04-25T00:15:09.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ef400640 1 -- 172.21.15.83:0/795573165 learned_addr learned my addr 172.21.15.83:0/795573165 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:09.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24eea00640 1 --2- 172.21.15.83:0/795573165 >> v2:172.21.15.86:3300/0 conn(0x7f24f0155470 0x7f24f00ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:09.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 -- 172.21.15.83:0/795573165 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4171155700 0 0) 0x7f24f00ae0a0 con 0x7f24f006ed90 2024-04-25T00:15:09.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 -- 172.21.15.83:0/795573165 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24d4003530 con 0x7f24f006ed90 2024-04-25T00:15:09.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 -- 172.21.15.83:0/795573165 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1200440287 0 0) 0x7f24d4003530 con 0x7f24f006ed90 2024-04-25T00:15:09.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 -- 172.21.15.83:0/795573165 >> v2:172.21.15.86:3300/0 conn(0x7f24f0155470 msgr2=0x7f24f00ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:09.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 --2- 172.21.15.83:0/795573165 >> v2:172.21.15.86:3300/0 conn(0x7f24f0155470 0x7f24f00ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 -- 172.21.15.83:0/795573165 >> v1:172.21.15.86:6789/0 conn(0x7f24f00adb00 legacy=0x7f24f006e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:09.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 -- 172.21.15.83:0/795573165 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f00ae940 con 0x7f24f006ed90 2024-04-25T00:15:09.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 -- 172.21.15.83:0/795573165 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f24e8001680 con 0x7f24f006ed90 2024-04-25T00:15:09.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24eea00640 1 --2- 172.21.15.83:0/795573165 >> v2:172.21.15.86:3300/0 conn(0x7f24f0155470 0x7f24f00ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:09.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 -- 172.21.15.83:0/795573165 >> v1:172.21.15.83:6789/0 conn(0x7f24f006ed90 legacy=0x7f24f0071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:09.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 --2- 172.21.15.83:0/795573165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24d40039c0 0x7f24d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:09.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 --2- 172.21.15.83:0/795573165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d4004810 0x7f24d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:09.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 --2- 172.21.15.83:0/795573165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24d40051d0 0x7f24d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:09.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 -- 172.21.15.83:0/795573165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f24f00ae660 con 0x7f24d40039c0 2024-04-25T00:15:09.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 -- 172.21.15.83:0/795573165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24d400c070 con 0x7f24d4004810 2024-04-25T00:15:09.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ee000640 1 -- 172.21.15.83:0/795573165 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f24d400c270 con 0x7f24d40051d0 2024-04-25T00:15:09.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24eea00640 1 --2- 172.21.15.83:0/795573165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24d40039c0 0x7f24d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:09.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24e6000640 1 --2- 172.21.15.83:0/795573165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d4004810 0x7f24d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:09.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ef400640 1 --2- 172.21.15.83:0/795573165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24d40051d0 0x7f24d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:09.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24eea00640 1 -- 172.21.15.83:0/795573165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24d40051d0 msgr2=0x7f24d400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:09.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24eea00640 1 --2- 172.21.15.83:0/795573165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24d40051d0 0x7f24d400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24eea00640 1 -- 172.21.15.83:0/795573165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d4004810 msgr2=0x7f24d4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:09.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24eea00640 1 --2- 172.21.15.83:0/795573165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d4004810 0x7f24d4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24eea00640 1 -- 172.21.15.83:0/795573165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d4003530 con 0x7f24d40039c0 2024-04-25T00:15:09.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.630+0000 7f24ef400640 1 -- 172.21.15.83:0/795573165 reap_dead start 2024-04-25T00:15:09.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.634+0000 7f24eea00640 1 --2- 172.21.15.83:0/795573165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24d40039c0 0x7f24d4003de0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f24e000e350 tx=0x7f24e001ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d520067501efc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:09.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.634+0000 7f24ee000640 1 -- 172.21.15.83:0/795573165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e00b2070 con 0x7f24d40039c0 2024-04-25T00:15:09.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.634+0000 7f24ee000640 1 -- 172.21.15.83:0/795573165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e00a5bf0 con 0x7f24d40039c0 2024-04-25T00:15:09.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.634+0000 7f24ee000640 1 -- 172.21.15.83:0/795573165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0009de0 con 0x7f24d40039c0 2024-04-25T00:15:09.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.634+0000 7f24f5800640 1 -- 172.21.15.83:0/795573165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24d40039c0 msgr2=0x7f24d4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:09.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.634+0000 7f24f5800640 1 --2- 172.21.15.83:0/795573165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24d40039c0 0x7f24d4003de0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f24e000e350 tx=0x7f24e001ce00 comp rx=0 tx=0).stop 2024-04-25T00:15:09.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.634+0000 7f24f5800640 1 -- 172.21.15.83:0/795573165 shutdown_connections 2024-04-25T00:15:09.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.634+0000 7f24f5800640 1 --2- 172.21.15.83:0/795573165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24d40039c0 0x7f24d4003de0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.634+0000 7f24f5800640 1 -- 172.21.15.83:0/795573165 >> 172.21.15.83:0/795573165 conn(0x7f24f00b1860 msgr2=0x7f24f00b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:09.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.634+0000 7f24f5800640 1 -- 172.21.15.83:0/795573165 shutdown_connections 2024-04-25T00:15:09.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.634+0000 7f24f5800640 1 -- 172.21.15.83:0/795573165 wait complete. 2024-04-25T00:15:09.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.634+0000 7f24f5800640 1 Processor -- start 2024-04-25T00:15:09.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.634+0000 7f24f5800640 1 -- start start 2024-04-25T00:15:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24f5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d40039c0 0x7f24f01f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24f5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24f00b3630 0x7f24f00b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24f5800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24f009bc40 0x7f24f009c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24f5800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f24f015f280 con 0x7f24f00b3630 2024-04-25T00:15:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24f5800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24f015f100 con 0x7f24d40039c0 2024-04-25T00:15:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24f5800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f24f015f400 con 0x7f24f009bc40 2024-04-25T00:15:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24e6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24f00b3630 0x7f24f00b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24e6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24f00b3630 0x7f24f00b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45924/0 (socket says 172.21.15.83:45924) 2024-04-25T00:15:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24e6000640 1 -- 172.21.15.83:0/2391210407 learned_addr learned my addr 172.21.15.83:0/2391210407 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24eea00640 1 --2- 172.21.15.83:0/2391210407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d40039c0 0x7f24f01f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24ef400640 1 --2- 172.21.15.83:0/2391210407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24f009bc40 0x7f24f009c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24e6000640 1 -- 172.21.15.83:0/2391210407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24f009bc40 msgr2=0x7f24f009c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24e6000640 1 --2- 172.21.15.83:0/2391210407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24f009bc40 0x7f24f009c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24e6000640 1 -- 172.21.15.83:0/2391210407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d40039c0 msgr2=0x7f24f01f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24e6000640 1 --2- 172.21.15.83:0/2391210407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d40039c0 0x7f24f01f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24e6000640 1 -- 172.21.15.83:0/2391210407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f009c950 con 0x7f24f00b3630 2024-04-25T00:15:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24e6000640 1 --2- 172.21.15.83:0/2391210407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24f00b3630 0x7f24f00b3ab0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f24d8001ec0 tx=0x7f24d800e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24e7400640 1 -- 172.21.15.83:0/2391210407 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8019070 con 0x7f24f00b3630 2024-04-25T00:15:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24e7400640 1 -- 172.21.15.83:0/2391210407 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24d80052e0 con 0x7f24f00b3630 2024-04-25T00:15:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24f5800640 1 -- 172.21.15.83:0/2391210407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f009cc30 con 0x7f24f00b3630 2024-04-25T00:15:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24e7400640 1 -- 172.21.15.83:0/2391210407 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8003820 con 0x7f24f00b3630 2024-04-25T00:15:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24f5800640 1 -- 172.21.15.83:0/2391210407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f009d1f0 con 0x7f24f00b3630 2024-04-25T00:15:09.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24f5800640 1 -- 172.21.15.83:0/2391210407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f004f510 con 0x7f24f00b3630 2024-04-25T00:15:09.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.638+0000 7f24e7400640 1 -- 172.21.15.83:0/2391210407 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f24d801c070 con 0x7f24f00b3630 2024-04-25T00:15:09.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.642+0000 7f24e7400640 1 --2- 172.21.15.83:0/2391210407 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f24cc0678c0 0x7f24cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:09.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.642+0000 7f24e7400640 1 -- 172.21.15.83:0/2391210407 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f24d8085580 con 0x7f24f00b3630 2024-04-25T00:15:09.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.642+0000 7f24eea00640 1 --2- 172.21.15.83:0/2391210407 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f24cc0678c0 0x7f24cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:09.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.642+0000 7f24eea00640 1 --2- 172.21.15.83:0/2391210407 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f24cc0678c0 0x7f24cc069d80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f24e00af000 tx=0x7f24e0004ac0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.646+0000 7f24e7400640 1 -- 172.21.15.83:0/2391210407 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f24d8055790 con 0x7f24f00b3630 2024-04-25T00:15:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.830+0000 7f24f5800640 1 -- 172.21.15.83:0/2391210407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24f006cf00 con 0x7f24f00b3630 2024-04-25T00:15:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.830+0000 7f24e7400640 1 -- 172.21.15.83:0/2391210407 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5181 (secure 0 0 0) 0x7f24d8059a90 con 0x7f24f00b3630 2024-04-25T00:15:09.833 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:09.833 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1992401755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1992401755},{"type":"v1","addr":"172.21.15.86:6833","nonce":1992401755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3621275047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3621275047},{"type":"v1","addr":"172.21.15.86:6835","nonce":3621275047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2870595352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2870595352},{"type":"v1","addr":"172.21.15.83:6837","nonce":2870595352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:51.879542+0000","modified":"2024-04-25T00:15:08.872509+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7719":{"gid":7719,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6835/565131724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":565131724},{"type":"v1","addr":"172.21.15.83:6835","nonce":565131724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7719,"qdb_cluster":[7719]},"id":7}]} 2024-04-25T00:15:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 70 2024-04-25T00:15:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.834+0000 7f24f5800640 1 -- 172.21.15.83:0/2391210407 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f24cc0678c0 msgr2=0x7f24cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.834+0000 7f24f5800640 1 --2- 172.21.15.83:0/2391210407 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f24cc0678c0 0x7f24cc069d80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f24e00af000 tx=0x7f24e0004ac0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.834+0000 7f24f5800640 1 -- 172.21.15.83:0/2391210407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24f00b3630 msgr2=0x7f24f00b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.834+0000 7f24f5800640 1 --2- 172.21.15.83:0/2391210407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24f00b3630 0x7f24f00b3ab0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f24d8001ec0 tx=0x7f24d800e8a0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.834+0000 7f24f5800640 1 -- 172.21.15.83:0/2391210407 shutdown_connections 2024-04-25T00:15:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.834+0000 7f24f5800640 1 --2- 172.21.15.83:0/2391210407 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f24cc0678c0 0x7f24cc069d80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.834+0000 7f24f5800640 1 --2- 172.21.15.83:0/2391210407 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24f009bc40 0x7f24f009c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.834+0000 7f24f5800640 1 --2- 172.21.15.83:0/2391210407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24f00b3630 0x7f24f00b3ab0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.834+0000 7f24f5800640 1 --2- 172.21.15.83:0/2391210407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d40039c0 0x7f24f01f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.834+0000 7f24f5800640 1 -- 172.21.15.83:0/2391210407 >> 172.21.15.83:0/2391210407 conn(0x7f24f00b1860 msgr2=0x7f24f00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.834+0000 7f24f5800640 1 -- 172.21.15.83:0/2391210407 shutdown_connections 2024-04-25T00:15:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:09.834+0000 7f24f5800640 1 -- 172.21.15.83:0/2391210407 wait complete. 2024-04-25T00:15:10.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:14:51.879542+0000', 'modified': '2024-04-25T00:15:08.872509+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 96, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7719}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7719': {'gid': 7719, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.83:6835/565131724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 565131724}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 565131724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7719, 'qdb_cluster': [7719]} 2024-04-25T00:15:10.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:15:10.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:15:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.394+0000 7f7e70800640 1 Processor -- start 2024-04-25T00:15:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.394+0000 7f7e70800640 1 -- start start 2024-04-25T00:15:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.394+0000 7f7e70800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7e6815be20 0x7f7e6815f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.394+0000 7f7e70800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7e680cedd0 0x7f7e6806cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.394+0000 7f7e70800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e680acd20 0x7f7e6806d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.394+0000 7f7e70800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7e6815fed0 con 0x7f7e6815be20 2024-04-25T00:15:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.394+0000 7f7e70800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7e680b2e70 con 0x7f7e680cedd0 2024-04-25T00:15:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.394+0000 7f7e70800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7e6815c200 con 0x7f7e680acd20 2024-04-25T00:15:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.394+0000 7f7e6de00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7e6815be20 0x7f7e6815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.394+0000 7f7e6de00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7e6815be20 0x7f7e6815f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45926/0 (socket says 172.21.15.83:45926) 2024-04-25T00:15:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.394+0000 7f7e6de00640 1 -- 172.21.15.83:0/1626793063 learned_addr learned my addr 172.21.15.83:0/1626793063 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.398+0000 7f7e6d400640 1 --2- 172.21.15.83:0/1626793063 >> v2:172.21.15.86:3300/0 conn(0x7f7e680cedd0 0x7f7e6806cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.398+0000 7f7e6e800640 1 --2- 172.21.15.83:0/1626793063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e680acd20 0x7f7e6806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.398+0000 7f7e6de00640 1 -- 172.21.15.83:0/1626793063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e680acd20 msgr2=0x7f7e6806d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.398+0000 7f7e6de00640 1 --2- 172.21.15.83:0/1626793063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e680acd20 0x7f7e6806d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.398+0000 7f7e6de00640 1 -- 172.21.15.83:0/1626793063 >> v2:172.21.15.86:3300/0 conn(0x7f7e680cedd0 msgr2=0x7f7e6806cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.398+0000 7f7e6de00640 1 --2- 172.21.15.83:0/1626793063 >> v2:172.21.15.86:3300/0 conn(0x7f7e680cedd0 0x7f7e6806cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.398+0000 7f7e6de00640 1 -- 172.21.15.83:0/1626793063 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e68160100 con 0x7f7e6815be20 2024-04-25T00:15:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.398+0000 7f7e6de00640 1 --2- 172.21.15.83:0/1626793063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e6815be20 0x7f7e6815f990 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7e6806e7c0 tx=0x7f7e5c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2295597c6bf8829 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.398+0000 7f7e6ca00640 1 -- 172.21.15.83:0/1626793063 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0b0070 con 0x7f7e6815be20 2024-04-25T00:15:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.398+0000 7f7e6ca00640 1 -- 172.21.15.83:0/1626793063 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e5c0ab600 con 0x7f7e6815be20 2024-04-25T00:15:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.398+0000 7f7e6ca00640 1 -- 172.21.15.83:0/1626793063 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0ab9d0 con 0x7f7e6815be20 2024-04-25T00:15:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.398+0000 7f7e70800640 1 -- 172.21.15.83:0/1626793063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e6815be20 msgr2=0x7f7e6815f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.398+0000 7f7e70800640 1 --2- 172.21.15.83:0/1626793063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e6815be20 0x7f7e6815f990 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7e6806e7c0 tx=0x7f7e5c01c520 comp rx=0 tx=0).stop 2024-04-25T00:15:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 -- 172.21.15.83:0/1626793063 shutdown_connections 2024-04-25T00:15:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 --2- 172.21.15.83:0/1626793063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e680acd20 0x7f7e6806d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 --2- 172.21.15.83:0/1626793063 >> v2:172.21.15.86:3300/0 conn(0x7f7e680cedd0 0x7f7e6806cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 --2- 172.21.15.83:0/1626793063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e6815be20 0x7f7e6815f990 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 -- 172.21.15.83:0/1626793063 >> 172.21.15.83:0/1626793063 conn(0x7f7e680b1b10 msgr2=0x7f7e680b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 -- 172.21.15.83:0/1626793063 shutdown_connections 2024-04-25T00:15:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 -- 172.21.15.83:0/1626793063 wait complete. 2024-04-25T00:15:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 Processor -- start 2024-04-25T00:15:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 -- start start 2024-04-25T00:15:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e680acd20 0x7f7e681517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e680cedd0 0x7f7e68144840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e6815be20 0x7f7e680ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e680b3ea0 con 0x7f7e6815be20 2024-04-25T00:15:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e680b3d20 con 0x7f7e680cedd0 2024-04-25T00:15:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7e680b4020 con 0x7f7e680acd20 2024-04-25T00:15:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e6e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e6815be20 0x7f7e680ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e6e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e6815be20 0x7f7e680ae6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45940/0 (socket says 172.21.15.83:45940) 2024-04-25T00:15:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e6e800640 1 -- 172.21.15.83:0/2579345709 learned_addr learned my addr 172.21.15.83:0/2579345709 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e6d400640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e680cedd0 0x7f7e68144840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e6de00640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e680acd20 0x7f7e681517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e6e800640 1 -- 172.21.15.83:0/2579345709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e680acd20 msgr2=0x7f7e681517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e6e800640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e680acd20 0x7f7e681517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e6e800640 1 -- 172.21.15.83:0/2579345709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e680cedd0 msgr2=0x7f7e68144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e6e800640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e680cedd0 0x7f7e68144840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e6e800640 1 -- 172.21.15.83:0/2579345709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e68156120 con 0x7f7e6815be20 2024-04-25T00:15:10.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e6de00640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e680acd20 0x7f7e681517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e6e800640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e6815be20 0x7f7e680ae6f0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f7e64007e70 tx=0x7f7e6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e62000640 1 -- 172.21.15.83:0/2579345709 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64014620 con 0x7f7e6815be20 2024-04-25T00:15:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e62000640 1 -- 172.21.15.83:0/2579345709 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e64019070 con 0x7f7e6815be20 2024-04-25T00:15:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e62000640 1 -- 172.21.15.83:0/2579345709 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64014a20 con 0x7f7e6815be20 2024-04-25T00:15:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.402+0000 7f7e70800640 1 -- 172.21.15.83:0/2579345709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e68156400 con 0x7f7e6815be20 2024-04-25T00:15:10.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.406+0000 7f7e70800640 1 -- 172.21.15.83:0/2579345709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e680aec30 con 0x7f7e6815be20 2024-04-25T00:15:10.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.406+0000 7f7e62000640 1 -- 172.21.15.83:0/2579345709 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7e64014bc0 con 0x7f7e6815be20 2024-04-25T00:15:10.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.406+0000 7f7e62000640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7e4c0678f0 0x7f7e4c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.406+0000 7f7e62000640 1 -- 172.21.15.83:0/2579345709 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f7e64086710 con 0x7f7e6815be20 2024-04-25T00:15:10.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.406+0000 7f7e6de00640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7e4c0678f0 0x7f7e4c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.406+0000 7f7e73d66000 1 Processor -- start 2024-04-25T00:15:10.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.406+0000 7f7e73d66000 1 -- start start 2024-04-25T00:15:10.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.406+0000 7f7e6de00640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7e4c0678f0 0x7f7e4c069db0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0ad000 tx=0x7f7e5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:10.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6bef0 0x55bd60d6c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 0x55bd60d22680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60d22fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bd60b705e0 con 0x55bd60d22bc0 2024-04-25T00:15:10.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bd60af7c50 con 0x55bd60d6bef0 2024-04-25T00:15:10.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55bd60b6c5a0 con 0x55bd60d6c810 2024-04-25T00:15:10.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e6e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60d22fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e6e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60d22fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45942/0 (socket says 172.21.15.83:45942) 2024-04-25T00:15:10.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e6e800640 1 -- 172.21.15.83:0/2683355617 learned_addr learned my addr 172.21.15.83:0/2683355617 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:10.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e6de00640 1 --2- 172.21.15.83:0/2683355617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6bef0 0x55bd60d6c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e6d400640 1 --2- 172.21.15.83:0/2683355617 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 0x55bd60d22680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e6e800640 1 -- 172.21.15.83:0/2683355617 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 msgr2=0x55bd60d22680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:10.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e6e800640 1 --2- 172.21.15.83:0/2683355617 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 0x55bd60d22680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e6e800640 1 -- 172.21.15.83:0/2683355617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6bef0 msgr2=0x55bd60d6c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:10.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e6e800640 1 --2- 172.21.15.83:0/2683355617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6bef0 0x55bd60d6c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e6e800640 1 -- 172.21.15.83:0/2683355617 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd60af5d10 con 0x55bd60d22bc0 2024-04-25T00:15:10.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e6e800640 1 --2- 172.21.15.83:0/2683355617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60d22fa0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f7e6405efb0 tx=0x7f7e64056900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47aa647168a86d6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:10.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e56a00640 1 -- 172.21.15.83:0/2683355617 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64054c00 con 0x55bd60d22bc0 2024-04-25T00:15:10.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e56a00640 1 -- 172.21.15.83:0/2683355617 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e64052f70 con 0x55bd60d22bc0 2024-04-25T00:15:10.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e56a00640 1 -- 172.21.15.83:0/2683355617 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6405f050 con 0x55bd60d22bc0 2024-04-25T00:15:10.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 -- 172.21.15.83:0/2683355617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 msgr2=0x55bd60d22fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:10.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 --2- 172.21.15.83:0/2683355617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60d22fa0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f7e6405efb0 tx=0x7f7e64056900 comp rx=0 tx=0).stop 2024-04-25T00:15:10.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 -- 172.21.15.83:0/2683355617 shutdown_connections 2024-04-25T00:15:10.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 --2- 172.21.15.83:0/2683355617 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 0x55bd60d22680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 --2- 172.21.15.83:0/2683355617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60d22fa0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 --2- 172.21.15.83:0/2683355617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6bef0 0x55bd60d6c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 -- 172.21.15.83:0/2683355617 >> 172.21.15.83:0/2683355617 conn(0x55bd60d1eb70 msgr2=0x55bd60d20f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:10.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 -- 172.21.15.83:0/2683355617 shutdown_connections 2024-04-25T00:15:10.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 -- 172.21.15.83:0/2683355617 wait complete. 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 -- start start 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 Processor -- start 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60d50f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6bef0 0x55bd60d51460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 0x55bd60d519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bd60b2a520 con 0x55bd60d22bc0 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bd60d2b250 con 0x55bd60d6bef0 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55bd60b39810 con 0x55bd60d6c810 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.410+0000 7f7e73d66000 10 client.0 _ll_register_callbacks cb 0x55bd60d0d5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e6de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60d50f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e6de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60d50f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45954/0 (socket says 172.21.15.83:45954) 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e6de00640 1 -- 172.21.15.83:0/2224635304 learned_addr learned my addr 172.21.15.83:0/2224635304 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e6d400640 1 --2- 172.21.15.83:0/2224635304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6bef0 0x55bd60d51460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e6e800640 1 --2- 172.21.15.83:0/2224635304 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 0x55bd60d519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e6de00640 1 -- 172.21.15.83:0/2224635304 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 msgr2=0x55bd60d519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:10.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e6de00640 1 --2- 172.21.15.83:0/2224635304 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 0x55bd60d519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e6de00640 1 -- 172.21.15.83:0/2224635304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6bef0 msgr2=0x55bd60d51460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:10.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e6de00640 1 --2- 172.21.15.83:0/2224635304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6bef0 0x55bd60d51460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e6de00640 1 -- 172.21.15.83:0/2224635304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd60b087b0 con 0x55bd60d22bc0 2024-04-25T00:15:10.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e6d400640 1 --2- 172.21.15.83:0/2224635304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6bef0 0x55bd60d51460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:10.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e6de00640 1 --2- 172.21.15.83:0/2224635304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60d50f20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0ab880 tx=0x7f7e5c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8874a98724c60e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:10.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e4b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:15:10.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e4b400640 1 -- 172.21.15.83:0/2224635304 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0c6060 con 0x55bd60d22bc0 2024-04-25T00:15:10.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 -- 172.21.15.83:0/2224635304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e64052f70 con 0x55bd60d22bc0 2024-04-25T00:15:10.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e4b400640 1 -- 172.21.15.83:0/2224635304 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f7e5c0b5720 con 0x55bd60d22bc0 2024-04-25T00:15:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e4b400640 1 -- 172.21.15.83:0/2224635304 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f7e5c0cc050 con 0x55bd60d22bc0 2024-04-25T00:15:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 10 client.7758 fetch_fsmap learned FSMap version 70 2024-04-25T00:15:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 -- 172.21.15.83:0/2224635304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x55bd60af7ea0 con 0x55bd60d22bc0 2024-04-25T00:15:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e4b400640 1 -- 172.21.15.83:0/2224635304 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e64052f70 con 0x55bd60d22bc0 2024-04-25T00:15:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e4b400640 1 -- 172.21.15.83:0/2224635304 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 70) ==== 3494+0+0 (secure 0 0 0) 0x7f7e5c0cf020 con 0x55bd60d22bc0 2024-04-25T00:15:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 10 client.7758 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T00:15:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 10 client.7758 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:15:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 20 client.7758 populate_metadata read hostname 'smithi083' 2024-04-25T00:15:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 --2- 172.21.15.83:0/2224635304 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x55bd60d7bf90 0x55bd60d7e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 4 client.7758 mds_command: new command op to 7719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:15:10.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 -- 172.21.15.83:0/2224635304 --> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd60b0bab0 con 0x55bd60d7bf90 2024-04-25T00:15:10.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e6d400640 1 --2- 172.21.15.83:0/2224635304 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x55bd60d7bf90 0x55bd60d7e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e6d400640 1 --2- 172.21.15.83:0/2224635304 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x55bd60d7bf90 0x55bd60d7e380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f69cfe13a1b47b server_cookie=c6ff30709b90eea1 in_seq=0 out_seq=0 2024-04-25T00:15:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e4b400640 10 client.7758 ms_handle_connect on v2:172.21.15.83:6834/565131724 2024-04-25T00:15:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e4b400640 1 -- 172.21.15.83:0/2224635304 <== mds.0 v2:172.21.15.83:6834/565131724 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd60b0bab0 con 0x55bd60d7bf90 2024-04-25T00:15:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e4b400640 10 client.7758 handle_command_reply: tid=0 2024-04-25T00:15:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 client.7758 shutdown 2024-04-25T00:15:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 -- 172.21.15.83:0/2224635304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 msgr2=0x55bd60d50f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 --2- 172.21.15.83:0/2224635304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60d50f20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0ab880 tx=0x7f7e5c01ad60 comp rx=0 tx=0).stop 2024-04-25T00:15:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 -- 172.21.15.83:0/2224635304 shutdown_connections 2024-04-25T00:15:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 --2- 172.21.15.83:0/2224635304 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x55bd60d7bf90 0x55bd60d7e380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 --2- 172.21.15.83:0/2224635304 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 0x55bd60d519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 --2- 172.21.15.83:0/2224635304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6bef0 0x55bd60d51460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e4b400640 0 client.7758 ms_handle_reset on v2:172.21.15.83:6834/565131724 2024-04-25T00:15:10.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 --2- 172.21.15.83:0/2224635304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60d50f20 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 -- 172.21.15.83:0/2224635304 >> 172.21.15.83:0/2224635304 conn(0x55bd60d1eb70 msgr2=0x55bd60d217f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:10.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 -- 172.21.15.83:0/2224635304 shutdown_connections 2024-04-25T00:15:10.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 1 -- 172.21.15.83:0/2224635304 wait complete. 2024-04-25T00:15:10.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.414+0000 7f7e73d66000 20 client.7758 trim_cache size 0 max 16384 2024-04-25T00:15:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 Processor -- start 2024-04-25T00:15:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 -- start start 2024-04-25T00:15:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60dd7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6bef0 0x55bd60dd7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6c810 0x55bd60dd7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e68202d40 con 0x55bd60d22bc0 2024-04-25T00:15:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e68159900 con 0x55bd60d6c810 2024-04-25T00:15:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7e682026f0 con 0x55bd60d6bef0 2024-04-25T00:15:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e6de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60dd7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e6de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60dd7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45968/0 (socket says 172.21.15.83:45968) 2024-04-25T00:15:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e6de00640 1 -- 172.21.15.83:0/191797075 learned_addr learned my addr 172.21.15.83:0/191797075 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e6d400640 1 --2- 172.21.15.83:0/191797075 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6bef0 0x55bd60dd7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e6e800640 1 --2- 172.21.15.83:0/191797075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6c810 0x55bd60dd7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e6de00640 1 -- 172.21.15.83:0/191797075 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6bef0 msgr2=0x55bd60dd7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e6de00640 1 --2- 172.21.15.83:0/191797075 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6bef0 0x55bd60dd7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e6de00640 1 -- 172.21.15.83:0/191797075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6c810 msgr2=0x55bd60dd7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e6de00640 1 --2- 172.21.15.83:0/191797075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6c810 0x55bd60dd7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e6de00640 1 -- 172.21.15.83:0/191797075 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd60af8e80 con 0x55bd60d22bc0 2024-04-25T00:15:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e6d400640 1 --2- 172.21.15.83:0/191797075 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6bef0 0x55bd60dd7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e6e800640 1 --2- 172.21.15.83:0/191797075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6c810 0x55bd60dd7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e6de00640 1 --2- 172.21.15.83:0/191797075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60dd7500 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0ae040 tx=0x7f7e5c0be000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51e898adb395eb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e56000640 1 -- 172.21.15.83:0/191797075 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0b0040 con 0x55bd60d22bc0 2024-04-25T00:15:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e56000640 1 -- 172.21.15.83:0/191797075 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e5c0abd40 con 0x55bd60d22bc0 2024-04-25T00:15:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e56000640 1 -- 172.21.15.83:0/191797075 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0d03f0 con 0x55bd60d22bc0 2024-04-25T00:15:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 -- 172.21.15.83:0/191797075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 msgr2=0x55bd60dd7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 --2- 172.21.15.83:0/191797075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60dd7500 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0ae040 tx=0x7f7e5c0be000 comp rx=0 tx=0).stop 2024-04-25T00:15:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 -- 172.21.15.83:0/191797075 shutdown_connections 2024-04-25T00:15:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 --2- 172.21.15.83:0/191797075 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6bef0 0x55bd60dd7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 --2- 172.21.15.83:0/191797075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d6c810 0x55bd60dd7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 --2- 172.21.15.83:0/191797075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d22bc0 0x55bd60dd7500 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 -- 172.21.15.83:0/191797075 >> 172.21.15.83:0/191797075 conn(0x55bd60d1eb70 msgr2=0x55bd60d794e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 -- 172.21.15.83:0/191797075 shutdown_connections 2024-04-25T00:15:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.434+0000 7f7e73d66000 1 -- 172.21.15.83:0/191797075 wait complete. 2024-04-25T00:15:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 1 -- start start 2024-04-25T00:15:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 1 Processor -- start 2024-04-25T00:15:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d22bc0 0x55bd60ddcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d6bef0 0x55bd60ddf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 0x55bd60de1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bd60b6f4d0 con 0x55bd60d6bef0 2024-04-25T00:15:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bd60c22e80 con 0x55bd60d22bc0 2024-04-25T00:15:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55bd60b08dd0 con 0x55bd60d6c810 2024-04-25T00:15:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 10 client.0 _ll_register_callbacks cb 0x7f7e5c0ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e6d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d6bef0 0x55bd60ddf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e6d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d6bef0 0x55bd60ddf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45980/0 (socket says 172.21.15.83:45980) 2024-04-25T00:15:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e6d400640 1 -- 172.21.15.83:0/3974975179 learned_addr learned my addr 172.21.15.83:0/3974975179 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e6de00640 1 --2- 172.21.15.83:0/3974975179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d22bc0 0x55bd60ddcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e6e800640 1 --2- 172.21.15.83:0/3974975179 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 0x55bd60de1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e6d400640 1 -- 172.21.15.83:0/3974975179 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 msgr2=0x55bd60de1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e6d400640 1 --2- 172.21.15.83:0/3974975179 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 0x55bd60de1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e6d400640 1 -- 172.21.15.83:0/3974975179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d22bc0 msgr2=0x55bd60ddcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e6d400640 1 --2- 172.21.15.83:0/3974975179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d22bc0 0x55bd60ddcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e6d400640 1 -- 172.21.15.83:0/3974975179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd60b3d760 con 0x55bd60d6bef0 2024-04-25T00:15:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e6d400640 1 --2- 172.21.15.83:0/3974975179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d6bef0 0x55bd60ddf2b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f7e58000c10 tx=0x7f7e58004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409c45e124ef9ee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e4b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:15:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e4b400640 1 -- 172.21.15.83:0/3974975179 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58026060 con 0x55bd60d6bef0 2024-04-25T00:15:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e4b400640 1 -- 172.21.15.83:0/3974975179 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f7e58016430 con 0x55bd60d6bef0 2024-04-25T00:15:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 1 -- 172.21.15.83:0/3974975179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e5c0abd40 con 0x55bd60d6bef0 2024-04-25T00:15:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e4b400640 1 -- 172.21.15.83:0/3974975179 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f7e5802c050 con 0x55bd60d6bef0 2024-04-25T00:15:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 10 client.7764 fetch_fsmap learned FSMap version 70 2024-04-25T00:15:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 1 -- 172.21.15.83:0/3974975179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x55bd60bc99e0 con 0x55bd60d6bef0 2024-04-25T00:15:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e4b400640 1 -- 172.21.15.83:0/3974975179 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e5c0abd40 con 0x55bd60d6bef0 2024-04-25T00:15:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e4b400640 1 -- 172.21.15.83:0/3974975179 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 70) ==== 3494+0+0 (secure 0 0 0) 0x7f7e5802f020 con 0x55bd60d6bef0 2024-04-25T00:15:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 10 client.7764 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T00:15:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 10 client.7764 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:15:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 20 client.7764 populate_metadata read hostname 'smithi083' 2024-04-25T00:15:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 1 --2- 172.21.15.83:0/3974975179 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x55bd60de1d40 0x55bd60de4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 4 client.7764 mds_command: new command op to 7719 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:15:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e73d66000 1 -- 172.21.15.83:0/3974975179 --> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] -- command(tid 0: {"prefix": "status"}) -- 0x55bd60b7db20 con 0x55bd60de1d40 2024-04-25T00:15:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e6de00640 1 --2- 172.21.15.83:0/3974975179 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x55bd60de1d40 0x55bd60de4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e6de00640 1 --2- 172.21.15.83:0/3974975179 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x55bd60de1d40 0x55bd60de4130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b07d0bcd81325420 server_cookie=ad3a777cfe0e4003 in_seq=0 out_seq=0 2024-04-25T00:15:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.438+0000 7f7e4b400640 10 client.7764 ms_handle_connect on v2:172.21.15.83:6834/565131724 2024-04-25T00:15:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e4b400640 1 -- 172.21.15.83:0/3974975179 <== mds.0 v2:172.21.15.83:6834/565131724 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55bd60b7db20 con 0x55bd60de1d40 2024-04-25T00:15:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e4b400640 10 client.7764 handle_command_reply: tid=0 2024-04-25T00:15:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e73d66000 1 client.7764 shutdown 2024-04-25T00:15:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e73d66000 1 -- 172.21.15.83:0/3974975179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d6bef0 msgr2=0x55bd60ddf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e73d66000 1 --2- 172.21.15.83:0/3974975179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d6bef0 0x55bd60ddf2b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f7e58000c10 tx=0x7f7e58004410 comp rx=0 tx=0).stop 2024-04-25T00:15:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e73d66000 1 -- 172.21.15.83:0/3974975179 shutdown_connections 2024-04-25T00:15:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e73d66000 1 --2- 172.21.15.83:0/3974975179 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x55bd60de1d40 0x55bd60de4130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e73d66000 1 --2- 172.21.15.83:0/3974975179 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bd60d6c810 0x55bd60de1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e73d66000 1 --2- 172.21.15.83:0/3974975179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd60d6bef0 0x55bd60ddf2b0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e73d66000 1 --2- 172.21.15.83:0/3974975179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd60d22bc0 0x55bd60ddcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e4b400640 0 client.7764 ms_handle_reset on v2:172.21.15.83:6834/565131724 2024-04-25T00:15:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e73d66000 1 -- 172.21.15.83:0/3974975179 >> 172.21.15.83:0/3974975179 conn(0x55bd60d1eb70 msgr2=0x55bd60d6a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e73d66000 1 -- 172.21.15.83:0/3974975179 shutdown_connections 2024-04-25T00:15:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e73d66000 1 -- 172.21.15.83:0/3974975179 wait complete. 2024-04-25T00:15:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e73d66000 20 client.7764 trim_cache size 0 max 16384 2024-04-25T00:15:10.447 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:15:10.447 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "d58ae83a-ad72-4e18-824a-86e779979c35", 2024-04-25T00:15:10.447 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-25T00:15:10.447 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7719, 2024-04-25T00:15:10.447 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T00:15:10.447 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T00:15:10.447 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T00:15:10.447 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 4.5840112380000004, 2024-04-25T00:15:10.447 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 70, 2024-04-25T00:15:10.447 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 97, 2024-04-25T00:15:10.447 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 97, 2024-04-25T00:15:10.447 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 5.4233762710000004 2024-04-25T00:15:10.448 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:15:10.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e70800640 1 -- 172.21.15.83:0/2579345709 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7e4c0678f0 msgr2=0x7f7e4c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:10.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e70800640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7e4c0678f0 0x7f7e4c069db0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0ad000 tx=0x7f7e5c008040 comp rx=0 tx=0).stop 2024-04-25T00:15:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e70800640 1 -- 172.21.15.83:0/2579345709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e6815be20 msgr2=0x7f7e680ae6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e70800640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e6815be20 0x7f7e680ae6f0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f7e64007e70 tx=0x7f7e6400e4b0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e70800640 1 -- 172.21.15.83:0/2579345709 shutdown_connections 2024-04-25T00:15:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e70800640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f7e4c0678f0 0x7f7e4c069db0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e70800640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e6815be20 0x7f7e680ae6f0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e70800640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e680cedd0 0x7f7e68144840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e70800640 1 --2- 172.21.15.83:0/2579345709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e680acd20 0x7f7e681517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e70800640 1 -- 172.21.15.83:0/2579345709 >> 172.21.15.83:0/2579345709 conn(0x7f7e680b1b10 msgr2=0x7f7e6815e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e70800640 1 -- 172.21.15.83:0/2579345709 shutdown_connections 2024-04-25T00:15:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.442+0000 7f7e70800640 1 -- 172.21.15.83:0/2579345709 wait complete. 2024-04-25T00:15:10.461 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:15:10.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.594+0000 7fd5a4600640 1 Processor -- start 2024-04-25T00:15:10.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.594+0000 7fd5a4600640 1 -- start start 2024-04-25T00:15:10.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.594+0000 7fd5a4600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd59c155470 0x7fd59c0ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.594+0000 7fd5a4600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd59c0adb00 0x7fd59c06e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a4600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd59c06ebc0 0x7fd59c15e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a4600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd59c0ae0a0 con 0x7fd59c155470 2024-04-25T00:15:10.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a4600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd59c0ae380 con 0x7fd59c06ebc0 2024-04-25T00:15:10.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a4600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd59c0ae660 con 0x7fd59c0adb00 2024-04-25T00:15:10.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a1c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd59c155470 0x7fd59c0ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a1c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd59c155470 0x7fd59c0ad5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45992/0 (socket says 172.21.15.83:45992) 2024-04-25T00:15:10.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a1c00640 1 -- 172.21.15.83:0/174392426 learned_addr learned my addr 172.21.15.83:0/174392426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:10.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a1200640 1 --2- 172.21.15.83:0/174392426 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd59c0adb00 0x7fd59c06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a2600640 1 --2- 172.21.15.83:0/174392426 >> v2:172.21.15.86:3300/0 conn(0x7fd59c06ebc0 0x7fd59c15e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a1c00640 1 -- 172.21.15.83:0/174392426 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd59c0adb00 msgr2=0x7fd59c06e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:10.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a1c00640 1 --2- 172.21.15.83:0/174392426 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd59c0adb00 0x7fd59c06e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a1c00640 1 -- 172.21.15.83:0/174392426 >> v2:172.21.15.86:3300/0 conn(0x7fd59c06ebc0 msgr2=0x7fd59c15e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:10.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a1c00640 1 --2- 172.21.15.83:0/174392426 >> v2:172.21.15.86:3300/0 conn(0x7fd59c06ebc0 0x7fd59c15e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a1c00640 1 -- 172.21.15.83:0/174392426 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59c0ae940 con 0x7fd59c155470 2024-04-25T00:15:10.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a2600640 1 --2- 172.21.15.83:0/174392426 >> v2:172.21.15.86:3300/0 conn(0x7fd59c06ebc0 0x7fd59c15e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:10.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.598+0000 7fd5a1c00640 1 --2- 172.21.15.83:0/174392426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c155470 0x7fd59c0ad5c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fd59c06c150 tx=0x7fd59001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82dab59bbe5b231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:10.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd59be00640 1 -- 172.21.15.83:0/174392426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900b0070 con 0x7fd59c155470 2024-04-25T00:15:10.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd59be00640 1 -- 172.21.15.83:0/174392426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5900ab600 con 0x7fd59c155470 2024-04-25T00:15:10.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd59be00640 1 -- 172.21.15.83:0/174392426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900ab9d0 con 0x7fd59c155470 2024-04-25T00:15:10.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 -- 172.21.15.83:0/174392426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c155470 msgr2=0x7fd59c0ad5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:10.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 --2- 172.21.15.83:0/174392426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c155470 0x7fd59c0ad5c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fd59c06c150 tx=0x7fd59001c260 comp rx=0 tx=0).stop 2024-04-25T00:15:10.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 -- 172.21.15.83:0/174392426 shutdown_connections 2024-04-25T00:15:10.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 --2- 172.21.15.83:0/174392426 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd59c0adb00 0x7fd59c06e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 --2- 172.21.15.83:0/174392426 >> v2:172.21.15.86:3300/0 conn(0x7fd59c06ebc0 0x7fd59c15e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 --2- 172.21.15.83:0/174392426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c155470 0x7fd59c0ad5c0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 -- 172.21.15.83:0/174392426 >> 172.21.15.83:0/174392426 conn(0x7fd59c0b1860 msgr2=0x7fd59c0b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:10.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 -- 172.21.15.83:0/174392426 shutdown_connections 2024-04-25T00:15:10.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 -- 172.21.15.83:0/174392426 wait complete. 2024-04-25T00:15:10.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 Processor -- start 2024-04-25T00:15:10.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 -- start start 2024-04-25T00:15:10.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c06ebc0 0x7fd59c1424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd59c0adb00 0x7fd59c09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd59c155470 0x7fd59c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c0716f0 con 0x7fd59c06ebc0 2024-04-25T00:15:10.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c071570 con 0x7fd59c155470 2024-04-25T00:15:10.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a4600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd59c071870 con 0x7fd59c0adb00 2024-04-25T00:15:10.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c06ebc0 0x7fd59c1424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c06ebc0 0x7fd59c1424c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46004/0 (socket says 172.21.15.83:46004) 2024-04-25T00:15:10.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a1c00640 1 -- 172.21.15.83:0/3034897561 learned_addr learned my addr 172.21.15.83:0/3034897561 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a1200640 1 --2- 172.21.15.83:0/3034897561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd59c0adb00 0x7fd59c09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.602+0000 7fd5a2600640 1 --2- 172.21.15.83:0/3034897561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd59c155470 0x7fd59c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd5a1c00640 1 -- 172.21.15.83:0/3034897561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd59c0adb00 msgr2=0x7fd59c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd5a1c00640 1 --2- 172.21.15.83:0/3034897561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd59c0adb00 0x7fd59c09a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd5a1c00640 1 -- 172.21.15.83:0/3034897561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd59c155470 msgr2=0x7fd59c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd5a1c00640 1 --2- 172.21.15.83:0/3034897561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd59c155470 0x7fd59c0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd5a1c00640 1 -- 172.21.15.83:0/3034897561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59c09cab0 con 0x7fd59c06ebc0 2024-04-25T00:15:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd5a1c00640 1 --2- 172.21.15.83:0/3034897561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c06ebc0 0x7fd59c1424c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd5900ae040 tx=0x7fd590019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd599600640 1 -- 172.21.15.83:0/3034897561 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900b0040 con 0x7fd59c06ebc0 2024-04-25T00:15:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd599600640 1 -- 172.21.15.83:0/3034897561 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5900b9070 con 0x7fd59c06ebc0 2024-04-25T00:15:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd599600640 1 -- 172.21.15.83:0/3034897561 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900b4660 con 0x7fd59c06ebc0 2024-04-25T00:15:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd5a4600640 1 -- 172.21.15.83:0/3034897561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59c09cd90 con 0x7fd59c06ebc0 2024-04-25T00:15:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd5a4600640 1 -- 172.21.15.83:0/3034897561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd59c09d350 con 0x7fd59c06ebc0 2024-04-25T00:15:10.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd5a4600640 1 -- 172.21.15.83:0/3034897561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd59c04f510 con 0x7fd59c06ebc0 2024-04-25T00:15:10.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd599600640 1 -- 172.21.15.83:0/3034897561 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd5900b4800 con 0x7fd59c06ebc0 2024-04-25T00:15:10.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd599600640 1 --2- 172.21.15.83:0/3034897561 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd57c0678c0 0x7fd57c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:10.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd599600640 1 -- 172.21.15.83:0/3034897561 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7fd59011f9b0 con 0x7fd59c06ebc0 2024-04-25T00:15:10.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.606+0000 7fd5a1200640 1 --2- 172.21.15.83:0/3034897561 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd57c0678c0 0x7fd57c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:10.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.610+0000 7fd5a1200640 1 --2- 172.21.15.83:0/3034897561 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd57c0678c0 0x7fd57c069d80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0b4490 tx=0x7fd588009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:10.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.610+0000 7fd599600640 1 -- 172.21.15.83:0/3034897561 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd590016110 con 0x7fd59c06ebc0 2024-04-25T00:15:10.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.802+0000 7fd5a4600640 1 -- 172.21.15.83:0/3034897561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd59c072cc0 con 0x7fd59c06ebc0 2024-04-25T00:15:10.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.802+0000 7fd599600640 1 -- 172.21.15.83:0/3034897561 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) ==== 53+0+73 (secure 0 0 0) 0x7fd5900efc50 con 0x7fd59c06ebc0 2024-04-25T00:15:10.804 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:15:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.806+0000 7fd5a4600640 1 -- 172.21.15.83:0/3034897561 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd57c0678c0 msgr2=0x7fd57c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.806+0000 7fd5a4600640 1 --2- 172.21.15.83:0/3034897561 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd57c0678c0 0x7fd57c069d80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0b4490 tx=0x7fd588009190 comp rx=0 tx=0).stop 2024-04-25T00:15:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.806+0000 7fd5a4600640 1 -- 172.21.15.83:0/3034897561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c06ebc0 msgr2=0x7fd59c1424c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.806+0000 7fd5a4600640 1 --2- 172.21.15.83:0/3034897561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c06ebc0 0x7fd59c1424c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd5900ae040 tx=0x7fd590019000 comp rx=0 tx=0).stop 2024-04-25T00:15:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.806+0000 7fd5a4600640 1 -- 172.21.15.83:0/3034897561 shutdown_connections 2024-04-25T00:15:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.806+0000 7fd5a4600640 1 --2- 172.21.15.83:0/3034897561 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fd57c0678c0 0x7fd57c069d80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.806+0000 7fd5a4600640 1 --2- 172.21.15.83:0/3034897561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd59c0adb00 0x7fd59c09a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.806+0000 7fd5a4600640 1 --2- 172.21.15.83:0/3034897561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd59c155470 0x7fd59c0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.806+0000 7fd5a4600640 1 --2- 172.21.15.83:0/3034897561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c06ebc0 0x7fd59c1424c0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.806+0000 7fd5a4600640 1 -- 172.21.15.83:0/3034897561 >> 172.21.15.83:0/3034897561 conn(0x7fd59c0b1860 msgr2=0x7fd59c071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:10.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.806+0000 7fd5a4600640 1 -- 172.21.15.83:0/3034897561 shutdown_connections 2024-04-25T00:15:10.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:10.806+0000 7fd5a4600640 1 -- 172.21.15.83:0/3034897561 wait complete. 2024-04-25T00:15:10.824 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:15:10.824 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:15:10.824 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-25T00:15:10.825 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:15:10.825 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:15:10.825 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:15:10.825 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:15:10.825 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:15:10.825 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:15:10.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:10.825 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-25T00:15:10.834 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:15:10.834 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:15:10.834 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:15:10.834 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:10.834 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-25T00:15:10.835 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:10.835 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:15:10.835 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T00:15:10.835 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-25T00:15:10.835 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:15:10.835 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T00:15:10.835 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-25T00:15:10.835 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:15:10.835 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:15:10.835 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:15:10.835 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:15:10.836 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:15:10.836 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42223sec preferred_lft 42223sec 2024-04-25T00:15:10.836 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-25T00:15:10.836 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:10.836 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:15:10.836 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:8d:d4:7d:b9 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:15:10.836 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:15:10.836 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:10.836 INFO:teuthology.orchestra.run.smithi083.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:15:10.836 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:15:10.836 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:15:10.836 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:10.837 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:15:10.837 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:10.837 INFO:teuthology.orchestra.run.smithi083.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:15:10.837 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:15:10.837 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:15:10.837 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:10.837 INFO:teuthology.orchestra.run.smithi083.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:15:10.837 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:15:10.837 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:15:10.837 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:10.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:10.839 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:15:10.860 INFO:tasks.ceph.mds.c.smithi083.stderr:2024-04-25T00:15:10.858+0000 7fd6a8200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:15:10.877 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-25T00:15:10.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:10.880 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-25T00:15:10.932 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-25T00:15:10.933 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:15:10.933 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:15:10.933 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:15:10.933 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:15:10.933 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:15:10.933 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:15:10.933 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:15:10.933 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:15:10.933 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:15:10.933 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:15:10.934 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:15:10.934 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T00:15:10.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:15:10.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:15:10.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:15:10.934 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T00:15:11.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:15:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:15:11.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:11.066 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:15:11.075 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:15:11.075 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:15:11.076 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:15:11.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:11.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-25T00:15:11.136 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:15:11.136 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:15:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:15:11.189 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:15:11.189 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:15:11.240 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T00:15:11.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:11.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:15:11.328 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:15:11.328 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:15:11.341 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:15:11.341 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:15:11.341 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:15:11.341 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:15:11.341 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:15:11.344 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:15:11.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:11.345 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:15:11.364 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:15:11.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:15:11.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:15:11.465 INFO:teuthology.orchestra.run.smithi083.stdout:client7773 2024-04-25T00:15:11.465 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:15:11.466 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:11.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:15:11.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf11800640 1 Processor -- start 2024-04-25T00:15:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf11800640 1 -- start start 2024-04-25T00:15:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf11800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbf0c158690 0x7fbf0c15e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf11800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf0c0cef70 0x7fbf0c066f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf11800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf0c15f060 con 0x7fbf0c067490 2024-04-25T00:15:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf11800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbf0c067900 con 0x7fbf0c0cef70 2024-04-25T00:15:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf11800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbf0c15f340 con 0x7fbf0c158690 2024-04-25T00:15:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf02000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf0c0cef70 0x7fbf0c066f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf02000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf0c0cef70 0x7fbf0c066f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46016/0 (socket says 172.21.15.83:46016) 2024-04-25T00:15:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf02000640 1 -- 172.21.15.83:0/3830980413 learned_addr learned my addr 172.21.15.83:0/3830980413 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf0aa00640 1 --2- 172.21.15.83:0/3830980413 >> v2:172.21.15.86:3300/0 conn(0x7fbf0c158690 0x7fbf0c15e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf02000640 1 -- 172.21.15.83:0/3830980413 >> v2:172.21.15.86:3300/0 conn(0x7fbf0c158690 msgr2=0x7fbf0c15e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf02000640 1 --2- 172.21.15.83:0/3830980413 >> v2:172.21.15.86:3300/0 conn(0x7fbf0c158690 0x7fbf0c15e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf02000640 1 -- 172.21.15.83:0/3830980413 >> v1:172.21.15.86:6789/0 conn(0x7fbf0c067490 legacy=0x7fbf0c06f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.650+0000 7fbf02000640 1 -- 172.21.15.83:0/3830980413 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0c15f620 con 0x7fbf0c0cef70 2024-04-25T00:15:11.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.654+0000 7fbf02000640 1 --2- 172.21.15.83:0/3830980413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf0c0cef70 0x7fbf0c066f50 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c09eeb0 tx=0x7fbef401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15830eac4864db15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:11.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.654+0000 7fbf0a000640 1 -- 172.21.15.83:0/3830980413 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef40b1070 con 0x7fbf0c0cef70 2024-04-25T00:15:11.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.654+0000 7fbf0a000640 1 -- 172.21.15.83:0/3830980413 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef40ac620 con 0x7fbf0c0cef70 2024-04-25T00:15:11.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.654+0000 7fbf0a000640 1 -- 172.21.15.83:0/3830980413 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef40ac9f0 con 0x7fbf0c0cef70 2024-04-25T00:15:11.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.654+0000 7fbf11800640 1 -- 172.21.15.83:0/3830980413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf0c0cef70 msgr2=0x7fbf0c066f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:11.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.654+0000 7fbf11800640 1 --2- 172.21.15.83:0/3830980413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf0c0cef70 0x7fbf0c066f50 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c09eeb0 tx=0x7fbef401c4b0 comp rx=0 tx=0).stop 2024-04-25T00:15:11.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.654+0000 7fbf11800640 1 -- 172.21.15.83:0/3830980413 shutdown_connections 2024-04-25T00:15:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.654+0000 7fbf11800640 1 --2- 172.21.15.83:0/3830980413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf0c0cef70 0x7fbf0c066f50 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.654+0000 7fbf11800640 1 --2- 172.21.15.83:0/3830980413 >> v2:172.21.15.86:3300/0 conn(0x7fbf0c158690 0x7fbf0c15e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.654+0000 7fbf11800640 1 -- 172.21.15.83:0/3830980413 >> 172.21.15.83:0/3830980413 conn(0x7fbf0c0d0cd0 msgr2=0x7fbf0c0d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.654+0000 7fbf11800640 1 -- 172.21.15.83:0/3830980413 shutdown_connections 2024-04-25T00:15:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.654+0000 7fbf11800640 1 -- 172.21.15.83:0/3830980413 wait complete. 2024-04-25T00:15:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf11800640 1 Processor -- start 2024-04-25T00:15:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf11800640 1 -- start start 2024-04-25T00:15:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf11800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf0c067490 0x7fbf0c1486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf11800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf0c0cef70 0x7fbf0c153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf11800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf0c158690 0x7fbf0c152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf11800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c15edb0 con 0x7fbf0c158690 2024-04-25T00:15:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf11800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c15ec30 con 0x7fbf0c067490 2024-04-25T00:15:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf11800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbf0c06bcc0 con 0x7fbf0c0cef70 2024-04-25T00:15:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf0b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf0c158690 0x7fbf0c152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf02000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf0c0cef70 0x7fbf0c153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf0aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf0c067490 0x7fbf0c1486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf0b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf0c158690 0x7fbf0c152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46022/0 (socket says 172.21.15.83:46022) 2024-04-25T00:15:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf0b400640 1 -- 172.21.15.83:0/2625583380 learned_addr learned my addr 172.21.15.83:0/2625583380 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf0b400640 1 -- 172.21.15.83:0/2625583380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf0c0cef70 msgr2=0x7fbf0c153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf0b400640 1 --2- 172.21.15.83:0/2625583380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf0c0cef70 0x7fbf0c153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf0b400640 1 -- 172.21.15.83:0/2625583380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf0c067490 msgr2=0x7fbf0c1486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf0b400640 1 --2- 172.21.15.83:0/2625583380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf0c067490 0x7fbf0c1486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf0b400640 1 -- 172.21.15.83:0/2625583380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0c1546c0 con 0x7fbf0c158690 2024-04-25T00:15:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf0b400640 1 --2- 172.21.15.83:0/2625583380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf0c158690 0x7fbf0c152300 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400c3c0 tx=0x7fbf0400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf03400640 1 -- 172.21.15.83:0/2625583380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04011790 con 0x7fbf0c158690 2024-04-25T00:15:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf03400640 1 -- 172.21.15.83:0/2625583380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf04017070 con 0x7fbf0c158690 2024-04-25T00:15:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf03400640 1 -- 172.21.15.83:0/2625583380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04011b70 con 0x7fbf0c158690 2024-04-25T00:15:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf11800640 1 -- 172.21.15.83:0/2625583380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0c1549a0 con 0x7fbf0c158690 2024-04-25T00:15:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.658+0000 7fbf11800640 1 -- 172.21.15.83:0/2625583380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0c154f60 con 0x7fbf0c158690 2024-04-25T00:15:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.662+0000 7fbf11800640 1 -- 172.21.15.83:0/2625583380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0c04f510 con 0x7fbf0c158690 2024-04-25T00:15:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.662+0000 7fbf03400640 1 -- 172.21.15.83:0/2625583380 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbf04011d10 con 0x7fbf0c158690 2024-04-25T00:15:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.662+0000 7fbf03400640 1 --2- 172.21.15.83:0/2625583380 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbee80678c0 0x7fbee8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.662+0000 7fbf03400640 1 -- 172.21.15.83:0/2625583380 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7fbf04085c90 con 0x7fbf0c158690 2024-04-25T00:15:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.662+0000 7fbf0aa00640 1 --2- 172.21.15.83:0/2625583380 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbee80678c0 0x7fbee8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.662+0000 7fbf0aa00640 1 --2- 172.21.15.83:0/2625583380 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbee80678c0 0x7fbee8069d80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0b6310 tx=0x7fbefc009800 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.666+0000 7fbf03400640 1 -- 172.21.15.83:0/2625583380 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbf04055ea0 con 0x7fbf0c158690 2024-04-25T00:15:11.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.854+0000 7fbf11800640 1 -- 172.21.15.83:0/2625583380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbf0c148c20 con 0x7fbf0c158690 2024-04-25T00:15:11.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.854+0000 7fbf03400640 1 -- 172.21.15.83:0/2625583380 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) ==== 53+0+73 (secure 0 0 0) 0x7fbf0405a1a0 con 0x7fbf0c158690 2024-04-25T00:15:11.855 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:15:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.854+0000 7fbf11800640 1 -- 172.21.15.83:0/2625583380 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbee80678c0 msgr2=0x7fbee8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.854+0000 7fbf11800640 1 --2- 172.21.15.83:0/2625583380 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbee80678c0 0x7fbee8069d80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0b6310 tx=0x7fbefc009800 comp rx=0 tx=0).stop 2024-04-25T00:15:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.854+0000 7fbf11800640 1 -- 172.21.15.83:0/2625583380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf0c158690 msgr2=0x7fbf0c152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.854+0000 7fbf11800640 1 --2- 172.21.15.83:0/2625583380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf0c158690 0x7fbf0c152300 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400c3c0 tx=0x7fbf0400c880 comp rx=0 tx=0).stop 2024-04-25T00:15:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.854+0000 7fbf11800640 1 -- 172.21.15.83:0/2625583380 shutdown_connections 2024-04-25T00:15:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.854+0000 7fbf11800640 1 --2- 172.21.15.83:0/2625583380 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbee80678c0 0x7fbee8069d80 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.854+0000 7fbf11800640 1 --2- 172.21.15.83:0/2625583380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf0c0cef70 0x7fbf0c153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.854+0000 7fbf11800640 1 --2- 172.21.15.83:0/2625583380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf0c158690 0x7fbf0c152300 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.854+0000 7fbf11800640 1 --2- 172.21.15.83:0/2625583380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf0c067490 0x7fbf0c1486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.854+0000 7fbf11800640 1 -- 172.21.15.83:0/2625583380 >> 172.21.15.83:0/2625583380 conn(0x7fbf0c0d0cd0 msgr2=0x7fbf0c06bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.854+0000 7fbf11800640 1 -- 172.21.15.83:0/2625583380 shutdown_connections 2024-04-25T00:15:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:11.854+0000 7fbf11800640 1 -- 172.21.15.83:0/2625583380 wait complete. 2024-04-25T00:15:11.876 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:15:11.876 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:15:11.876 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T00:15:11.876 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:15:11.876 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:15:11.876 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:15:11.877 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:15:11.877 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:15:11.877 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:15:11.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:11.877 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T00:15:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:15:11.887 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:15:11.887 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:15:11.887 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:11.888 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T00:15:11.888 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:15:11.888 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:15:11.888 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T00:15:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:15:11.888 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:15:11.888 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T00:15:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:15:11.888 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:15:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:15:11.888 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:15:11.888 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:15:11.888 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42217sec preferred_lft 42217sec 2024-04-25T00:15:11.889 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T00:15:11.889 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:15:11.889 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:35:63:ea:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:15:11.889 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:15:11.889 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:15:11.889 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:15:11.889 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:15:11.889 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:11.889 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:15:11.889 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:15:11.889 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:15:11.890 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:15:11.890 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:15:11.890 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:15:11.890 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T00:15:11.890 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:15:11.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:11.891 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:15:11.953 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T00:15:11.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:11.955 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T00:15:12.010 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T00:15:12.011 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:15:12.011 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:15:12.011 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:15:12.011 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:15:12.011 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:15:12.011 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:15:12.011 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:15:12.011 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:15:12.011 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:15:12.011 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:15:12.012 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:15:12.012 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T00:15:12.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:15:12.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:15:12.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:15:12.012 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T00:15:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:15:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:15:12.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:15:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:15:12.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:12.155 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T00:15:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:15:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:15:12.165 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:15:12.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:12.165 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T00:15:12.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:15:12.228 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:15:12.280 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:15:12.281 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:15:12.281 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:15:12.331 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T00:15:12.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:12.332 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:15:12.419 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:15:12.420 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:15:12.433 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T00:15:12.433 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T00:15:12.433 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-25T00:15:12.433 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with new device syntax: 1@d58ae83a-ad72-4e18-824a-86e779979c35.cephfs=/ 2024-04-25T00:15:12.433 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-25T00:15:12.436 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:15:12.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:12.437 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:15:12.456 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:15:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:15:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:15:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T00:15:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:15:12.556 INFO:teuthology.orchestra.run.smithi086.stdout:client7782 2024-04-25T00:15:12.556 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:15:12.556 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:12.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:15:12.770 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":7782,"entity":{"name":{"type":"client","num":7782},"addr":{"type":"any","addr":"172.21.15.86:0","nonce":3118175616}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":0.32799247599999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99925169833631211,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7782 172.21.15.86:0/3118175616","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi086","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}},{"id":7773,"entity":{"name":{"type":"client","num":7773},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":4035192431}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":1.4199674369999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.9967337661194533,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7773 192.168.0.1:0/4035192431","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi083","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}}] 2024-04-25T00:15:12.771 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi086", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.86:0", "nonce": 3118175616, "type": "any" }, "name": { "num": 7782, "type": "client" } }, "id": 7782, "inst": "client.7782 172.21.15.86:0/3118175616", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9992516983363121 }, "state": "open", "uptime": 0.327992476 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi083", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4035192431, "type": "any" }, "name": { "num": 7773, "type": "client" } }, "id": 7773, "inst": "client.7773 192.168.0.1:0/4035192431", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9967337661194533 }, "state": "open", "uptime": 1.419967437 } ] 2024-04-25T00:15:12.771 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T00:15:12.936 INFO:teuthology.orchestra.run.smithi083.stdout:{"message":"","return_code":0} 2024-04-25T00:15:12.936 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T00:15:12.936 DEBUG:teuthology.orchestra.run.smithi083:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show session 2024-04-25T00:15:12.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b3b576400 1 -- start start 2024-04-25T00:15:12.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b3b576400 1 Processor -- start 2024-04-25T00:15:12.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b3b576400 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556240621800 0x5562407da580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:12.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b3b576400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556240621400 0x5562407dab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:12.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b3b576400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556240621000 0x5562407db080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b3b576400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55624151c180 con 0x556240621400 2024-04-25T00:15:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b3b576400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55624151c300 con 0x556240621000 2024-04-25T00:15:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b3b576400 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55624151c480 con 0x556240621800 2024-04-25T00:15:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b38c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556240621400 0x5562407dab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b38c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556240621400 0x5562407dab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46050/0 (socket says 172.21.15.83:46050) 2024-04-25T00:15:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b38c00640 1 -- 172.21.15.83:0/1109880883 learned_addr learned my addr 172.21.15.83:0/1109880883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b39600640 1 --2- 172.21.15.83:0/1109880883 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556240621800 0x5562407da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b3a000640 1 --2- 172.21.15.83:0/1109880883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556240621000 0x5562407db080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b38c00640 1 -- 172.21.15.83:0/1109880883 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556240621800 msgr2=0x5562407da580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b38c00640 1 --2- 172.21.15.83:0/1109880883 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556240621800 0x5562407da580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b38c00640 1 -- 172.21.15.83:0/1109880883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556240621000 msgr2=0x5562407db080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b38c00640 1 --2- 172.21.15.83:0/1109880883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556240621000 0x5562407db080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b38c00640 1 -- 172.21.15.83:0/1109880883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562413dc960 con 0x556240621400 2024-04-25T00:15:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b38c00640 1 --2- 172.21.15.83:0/1109880883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556240621400 0x5562407dab00 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x5562414829f0 tx=0x5562406e2960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09b98ab0c1c545d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:12.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b36400640 1 -- 172.21.15.83:0/1109880883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5562406569c0 con 0x556240621400 2024-04-25T00:15:12.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b36400640 1 -- 172.21.15.83:0/1109880883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55624066ae00 con 0x556240621400 2024-04-25T00:15:12.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b36400640 1 -- 172.21.15.83:0/1109880883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5562406564e0 con 0x556240621400 2024-04-25T00:15:12.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b3b576400 1 -- 172.21.15.83:0/1109880883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562413dcd20 con 0x556240621400 2024-04-25T00:15:12.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b3b576400 1 -- 172.21.15.83:0/1109880883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5562413dcf00 con 0x556240621400 2024-04-25T00:15:12.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b3b576400 4 waiting for MDS map... 2024-04-25T00:15:12.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.978+0000 7f6b36400640 1 -- 172.21.15.83:0/1109880883 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x55624153a200 con 0x556240621400 2024-04-25T00:15:12.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b36400640 1 -- 172.21.15.83:0/1109880883 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 70) ==== 3494+0+0 (secure 0 0 0) 0x556240677180 con 0x556240621400 2024-04-25T00:15:12.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 4 Got MDS map 70 2024-04-25T00:15:12.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 10 main: main 2024-04-25T00:15:12.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 4 main: connecting to RADOS... 2024-04-25T00:15:12.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 Processor -- start 2024-04-25T00:15:12.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 -- start start 2024-04-25T00:15:12.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5562413f5c00 0x5562407db600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:12.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55624158c000 0x5562407dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:12.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624158c400 0x5562407dc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:12.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55624151c600 con 0x55624158c400 2024-04-25T00:15:12.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55624151c780 con 0x55624158c000 2024-04-25T00:15:12.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55624151c900 con 0x5562413f5c00 2024-04-25T00:15:12.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5562413f5c00 0x5562407db600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:12.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b38c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624158c400 0x5562407dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:12.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b39600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55624158c000 0x5562407dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:12.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5562413f5c00 0x5562407db600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:54014/0 (socket says 172.21.15.83:54014) 2024-04-25T00:15:12.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3a000640 1 -- 172.21.15.83:0/3476450613 learned_addr learned my addr 172.21.15.83:0/3476450613 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:12.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b38c00640 1 -- 172.21.15.83:0/3476450613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5562413f5c00 msgr2=0x5562407db600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:12.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b38c00640 1 --2- 172.21.15.83:0/3476450613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5562413f5c00 0x5562407db600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b38c00640 1 -- 172.21.15.83:0/3476450613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55624158c000 msgr2=0x5562407dbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:12.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b38c00640 1 --2- 172.21.15.83:0/3476450613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55624158c000 0x5562407dbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b38c00640 1 -- 172.21.15.83:0/3476450613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562413dc1e0 con 0x55624158c400 2024-04-25T00:15:12.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b38c00640 1 --2- 172.21.15.83:0/3476450613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624158c400 0x5562407dc100 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x556241483590 tx=0x5562406e26c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d8f1eb0a828c8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:12.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b33c00640 1 -- 172.21.15.83:0/3476450613 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556240656b60 con 0x55624158c400 2024-04-25T00:15:12.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b33c00640 1 -- 172.21.15.83:0/3476450613 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55624066ac40 con 0x55624158c400 2024-04-25T00:15:12.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b33c00640 1 -- 172.21.15.83:0/3476450613 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556240656340 con 0x55624158c400 2024-04-25T00:15:12.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 -- 172.21.15.83:0/3476450613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624158c400 msgr2=0x5562407dc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:12.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 --2- 172.21.15.83:0/3476450613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624158c400 0x5562407dc100 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x556241483590 tx=0x5562406e26c0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 -- 172.21.15.83:0/3476450613 shutdown_connections 2024-04-25T00:15:12.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 --2- 172.21.15.83:0/3476450613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624158c400 0x5562407dc100 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 --2- 172.21.15.83:0/3476450613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55624158c000 0x5562407dbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 --2- 172.21.15.83:0/3476450613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5562413f5c00 0x5562407db600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 -- 172.21.15.83:0/3476450613 >> 172.21.15.83:0/3476450613 conn(0x556240620c00 msgr2=0x55624064f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:12.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 -- 172.21.15.83:0/3476450613 shutdown_connections 2024-04-25T00:15:12.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.982+0000 7f6b3b576400 1 -- 172.21.15.83:0/3476450613 wait complete. 2024-04-25T00:15:12.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b3b576400 1 Processor -- start 2024-04-25T00:15:12.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b3b576400 1 -- start start 2024-04-25T00:15:12.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b3b576400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413f5c00 0x5562407dd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:12.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b3b576400 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55624158d800 0x5562407dcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:12.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b3b576400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624158d400 0x5562407dc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:12.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b3b576400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55624151dc80 con 0x55624158d400 2024-04-25T00:15:12.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b3b576400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55624151db00 con 0x5562413f5c00 2024-04-25T00:15:12.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b3b576400 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55624151d980 con 0x55624158d800 2024-04-25T00:15:12.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b38c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624158d400 0x5562407dc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:12.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b38c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624158d400 0x5562407dc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46072/0 (socket says 172.21.15.83:46072) 2024-04-25T00:15:12.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b3a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413f5c00 0x5562407dd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:12.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b38c00640 1 -- 172.21.15.83:0/1913942225 learned_addr learned my addr 172.21.15.83:0/1913942225 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:12.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b39600640 1 --2- 172.21.15.83:0/1913942225 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55624158d800 0x5562407dcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:12.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b38c00640 1 -- 172.21.15.83:0/1913942225 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55624158d800 msgr2=0x5562407dcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:12.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b38c00640 1 --2- 172.21.15.83:0/1913942225 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55624158d800 0x5562407dcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b38c00640 1 -- 172.21.15.83:0/1913942225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413f5c00 msgr2=0x5562407dd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:12.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b38c00640 1 --2- 172.21.15.83:0/1913942225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413f5c00 0x5562407dd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b38c00640 1 -- 172.21.15.83:0/1913942225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562413dda40 con 0x55624158d400 2024-04-25T00:15:12.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b38c00640 1 --2- 172.21.15.83:0/1913942225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624158d400 0x5562407dc680 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x5562413eeff0 tx=0x5562406e27e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:12.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b31e00640 1 -- 172.21.15.83:0/1913942225 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556240656b60 con 0x55624158d400 2024-04-25T00:15:12.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b31e00640 1 -- 172.21.15.83:0/1913942225 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55624066aa80 con 0x55624158d400 2024-04-25T00:15:12.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b31e00640 1 -- 172.21.15.83:0/1913942225 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556240656340 con 0x55624158d400 2024-04-25T00:15:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b3b576400 1 -- 172.21.15.83:0/1913942225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5562413ddc20 con 0x55624158d400 2024-04-25T00:15:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b3b576400 1 -- 172.21.15.83:0/1913942225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562413dcf00 con 0x55624158d400 2024-04-25T00:15:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b3b576400 4 main: resolving pool 15 2024-04-25T00:15:12.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.986+0000 7f6b31e00640 1 -- 172.21.15.83:0/1913942225 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x55624158ac00 con 0x55624158d400 2024-04-25T00:15:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b31e00640 1 --2- 172.21.15.83:0/1913942225 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x5562415a9400 0x5562407dd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b31e00640 1 -- 172.21.15.83:0/1913942225 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x556240952600 con 0x55624158d400 2024-04-25T00:15:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 4 main: creating IoCtx.. 2024-04-25T00:15:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 --2- 172.21.15.83:0/1913942225 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x55624158d000 0x5562407dc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1913942225 --> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] -- osd_op(unknown.0.0:1 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55624158cc00 con 0x55624158d000 2024-04-25T00:15:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3a000640 1 --2- 172.21.15.83:0/1913942225 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x5562415a9400 0x5562407dd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b39600640 1 --2- 172.21.15.83:0/1913942225 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x55624158d000 0x5562407dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3a000640 1 --2- 172.21.15.83:0/1913942225 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x5562415a9400 0x5562407dd700 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x556240671a10 tx=0x5562409d4a80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b39600640 1 --2- 172.21.15.83:0/1913942225 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x55624158d000 0x5562407dc100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b39600640 1 -- 172.21.15.83:0/1913942225 <== osd.2 v2:172.21.15.83:6824/1702952090 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv5 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x5562406d6000 con 0x55624158d000 2024-04-25T00:15:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1913942225 --> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] -- osd_op(unknown.0.0:2 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55624158c800 con 0x55624158d000 2024-04-25T00:15:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b39600640 1 -- 172.21.15.83:0/1913942225 <== osd.2 v2:172.21.15.83:6824/1702952090 2 ==== osd_op_reply(2 mds0_sessionmap [omap-get-vals out=475b] v0'0 uv5 ondisk = 0) ==== 159+0+475 (crc 0 0 0) 0x5562406d6000 con 0x55624158d000 2024-04-25T00:15:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1913942225 --> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] -- osd_op(unknown.0.0:3 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55624158c400 con 0x55624158d000 2024-04-25T00:15:13.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b39600640 1 -- 172.21.15.83:0/1913942225 <== osd.2 v2:172.21.15.83:6824/1702952090 3 ==== osd_op_reply(3 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x5562406d6000 con 0x55624158d000 2024-04-25T00:15:13.000 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:15:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "0": { 2024-04-25T00:15:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "data": { 2024-04-25T00:15:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions": [ 2024-04-25T00:15:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:15:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7782, 2024-04-25T00:15:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": { 2024-04-25T00:15:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-25T00:15:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-25T00:15:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 7782 2024-04-25T00:15:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-25T00:15:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "any", 2024-04-25T00:15:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.86:0", 2024-04-25T00:15:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3118175616 2024-04-25T00:15:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:15:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "open", 2024-04-25T00:15:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "num_leases": 0, 2024-04-25T00:15:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "num_caps": 0, 2024-04-25T00:15:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "request_load_avg": 0, 2024-04-25T00:15:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 0, 2024-04-25T00:15:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "requests_in_flight": 0, 2024-04-25T00:15:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "num_completed_requests": 0, 2024-04-25T00:15:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "num_completed_flushes": 0, 2024-04-25T00:15:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "reconnecting": false, 2024-04-25T00:15:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "recall_caps": { 2024-04-25T00:15:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "value": 0, 2024-04-25T00:15:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "halflife": 60 2024-04-25T00:15:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "release_caps": { 2024-04-25T00:15:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "value": 0, 2024-04-25T00:15:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "halflife": 60 2024-04-25T00:15:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "recall_caps_throttle": { 2024-04-25T00:15:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "value": 0, 2024-04-25T00:15:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "halflife": 1.5 2024-04-25T00:15:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "recall_caps_throttle2o": { 2024-04-25T00:15:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "value": 0, 2024-04-25T00:15:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "halflife": 0.5 2024-04-25T00:15:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "session_cache_liveness": { 2024-04-25T00:15:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "value": 0, 2024-04-25T00:15:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "halflife": 300 2024-04-25T00:15:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition": { 2024-04-25T00:15:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "value": 0, 2024-04-25T00:15:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "halflife": 30 2024-04-25T00:15:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "last_trim_completed_requests_tid": 0, 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "last_trim_completed_flushes_tid": 0, 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "delegated_inos": [], 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": "client.7782 172.21.15.86:0/3118175616", 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "completed_requests": [], 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "prealloc_inos": [], 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "client_metadata": { 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "client_features": { 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "feature_bits": "0x000000000037fbff" 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "metric_spec": { 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "metric_flags": { 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "feature_bits": "0x000000000080ffff" 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:15:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "1", 2024-04-25T00:15:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi086", 2024-04-25T00:15:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "kernel_version": "6.9.0-rc3-g806ec36b2112", 2024-04-25T00:15:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-25T00:15:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:15:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T00:15:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7773, 2024-04-25T00:15:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": { 2024-04-25T00:15:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-25T00:15:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-25T00:15:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 7773 2024-04-25T00:15:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-25T00:15:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "any", 2024-04-25T00:15:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:15:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4035192431 2024-04-25T00:15:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:15:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "open", 2024-04-25T00:15:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "num_leases": 0, 2024-04-25T00:15:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "num_caps": 0, 2024-04-25T00:15:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "request_load_avg": 0, 2024-04-25T00:15:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 0, 2024-04-25T00:15:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "requests_in_flight": 0, 2024-04-25T00:15:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "num_completed_requests": 0, 2024-04-25T00:15:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "num_completed_flushes": 0, 2024-04-25T00:15:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "reconnecting": false, 2024-04-25T00:15:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "recall_caps": { 2024-04-25T00:15:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "value": 0, 2024-04-25T00:15:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "halflife": 60 2024-04-25T00:15:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "release_caps": { 2024-04-25T00:15:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "value": 0, 2024-04-25T00:15:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "halflife": 60 2024-04-25T00:15:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "recall_caps_throttle": { 2024-04-25T00:15:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "value": 0, 2024-04-25T00:15:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "halflife": 1.5 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "recall_caps_throttle2o": { 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "value": 0, 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "halflife": 0.5 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "session_cache_liveness": { 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "value": 0, 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "halflife": 300 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition": { 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "value": 0, 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "halflife": 30 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "last_trim_completed_requests_tid": 0, 2024-04-25T00:15:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "last_trim_completed_flushes_tid": 0, 2024-04-25T00:15:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "delegated_inos": [], 2024-04-25T00:15:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": "client.7773 192.168.0.1:0/4035192431", 2024-04-25T00:15:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "completed_requests": [], 2024-04-25T00:15:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "prealloc_inos": [], 2024-04-25T00:15:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "client_metadata": { 2024-04-25T00:15:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "client_features": { 2024-04-25T00:15:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "feature_bits": "0x000000000037fbff" 2024-04-25T00:15:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "metric_spec": { 2024-04-25T00:15:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "metric_flags": { 2024-04-25T00:15:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "feature_bits": "0x000000000080ffff" 2024-04-25T00:15:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:15:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-04-25T00:15:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-25T00:15:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: "kernel_version": "6.9.0-rc3-g806ec36b2112", 2024-04-25T00:15:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-25T00:15:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:15:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:15:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T00:15:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: "result": 0 2024-04-25T00:15:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:15:13.010 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:15:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:13.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1913942225 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x55624158d000 msgr2=0x5562407dc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:13.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 --2- 172.21.15.83:0/1913942225 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x55624158d000 0x5562407dc100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1913942225 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x5562415a9400 msgr2=0x5562407dd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:13.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 --2- 172.21.15.83:0/1913942225 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x5562415a9400 0x5562407dd700 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x556240671a10 tx=0x5562409d4a80 comp rx=0 tx=0).stop 2024-04-25T00:15:13.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1913942225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624158d400 msgr2=0x5562407dc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:13.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 --2- 172.21.15.83:0/1913942225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624158d400 0x5562407dc680 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x5562413eeff0 tx=0x5562406e27e0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b39600640 1 -- 172.21.15.83:0/1913942225 reap_dead start 2024-04-25T00:15:13.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1913942225 shutdown_connections 2024-04-25T00:15:13.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1913942225 >> 172.21.15.83:0/1913942225 conn(0x556240620c00 msgr2=0x55624064f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:13.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1913942225 shutdown_connections 2024-04-25T00:15:13.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1913942225 wait complete. 2024-04-25T00:15:13.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1109880883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556240621400 msgr2=0x5562407dab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:13.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 --2- 172.21.15.83:0/1109880883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556240621400 0x5562407dab00 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x5562414829f0 tx=0x5562406e2960 comp rx=0 tx=0).stop 2024-04-25T00:15:13.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1109880883 shutdown_connections 2024-04-25T00:15:13.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 --2- 172.21.15.83:0/1109880883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556240621000 0x5562407db080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 --2- 172.21.15.83:0/1109880883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556240621400 0x5562407dab00 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 --2- 172.21.15.83:0/1109880883 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556240621800 0x5562407da580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1109880883 >> 172.21.15.83:0/1109880883 conn(0x556240621c00 msgr2=0x55624064f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:13.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1109880883 shutdown_connections 2024-04-25T00:15:13.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:12.990+0000 7f6b3b576400 1 -- 172.21.15.83:0/1109880883 wait complete. 2024-04-25T00:15:13.014 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'session'] in time 0:00:00.077688, result: { "0": { "data": { "sessions": [ { "id": 7782, "entity": { "name": { "type": "client", "num": 7782 }, "addr": { "type": "any", "addr": "172.21.15.86:0", "nonce": 3118175616 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.7782 172.21.15.86:0/3118175616", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "1", "hostname": "smithi086", "kernel_version": "6.9.0-rc3-g806ec36b2112", "root": "/" } }, { "id": 7773, "entity": { "name": { "type": "client", "num": 7773 }, "addr": { "type": "any", "addr": "192.168.0.1:0", "nonce": 4035192431 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.7773 192.168.0.1:0/4035192431", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "0", "hostname": "smithi083", "kernel_version": "6.9.0-rc3-g806ec36b2112", "root": "/" } } ] }, "result": 0 } } 2024-04-25T00:15:13.015 INFO:tasks.cephfs.test_sessionmap:SessionMap: { "0": { "data": { "sessions": [ { "id": 7782, "entity": { "name": { "type": "client", "num": 7782 }, "addr": { "type": "any", "addr": "172.21.15.86:0", "nonce": 3118175616 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.7782 172.21.15.86:0/3118175616", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "1", "hostname": "smithi086", "kernel_version": "6.9.0-rc3-g806ec36b2112", "root": "/" } }, { "id": 7773, "entity": { "name": { "type": "client", "num": 7773 }, "addr": { "type": "any", "addr": "192.168.0.1:0", "nonce": 4035192431 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.7773 192.168.0.1:0/4035192431", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "0", "hostname": "smithi083", "kernel_version": "6.9.0-rc3-g806ec36b2112", "root": "/" } } ] }, "result": 0 } } 2024-04-25T00:15:13.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:13.015 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir bravo) 2024-04-25T00:15:13.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:13.067 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch bravo/file_a) 2024-04-25T00:15:13.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:13.120 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec touch bravo/file_b) 2024-04-25T00:15:13.135 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-04-25T00:15:13.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e1a200640 1 Processor -- start 2024-04-25T00:15:13.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e1a200640 1 -- start start 2024-04-25T00:15:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e1a200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e14155300 0x7f8e140cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e1a200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8e140d0180 0x7f8e140b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e1a200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8e140b3180 0x7f8e140cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e1a200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8e140d05e0 con 0x7f8e140d0180 2024-04-25T00:15:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e1a200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8e141556e0 con 0x7f8e140b3180 2024-04-25T00:15:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e1a200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8e14072260 con 0x7f8e14155300 2024-04-25T00:15:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e12a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8e140d0180 0x7f8e140b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e12a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8e140d0180 0x7f8e140b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46084/0 (socket says 172.21.15.83:46084) 2024-04-25T00:15:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e12a00640 1 -- 172.21.15.83:0/2218344372 learned_addr learned my addr 172.21.15.83:0/2218344372 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e13400640 1 --2- 172.21.15.83:0/2218344372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e14155300 0x7f8e140cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e13e00640 1 --2- 172.21.15.83:0/2218344372 >> v2:172.21.15.86:3300/0 conn(0x7f8e140b3180 0x7f8e140cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e12a00640 1 -- 172.21.15.83:0/2218344372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e14155300 msgr2=0x7f8e140cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e12a00640 1 --2- 172.21.15.83:0/2218344372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e14155300 0x7f8e140cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e12a00640 1 -- 172.21.15.83:0/2218344372 >> v2:172.21.15.86:3300/0 conn(0x7f8e140b3180 msgr2=0x7f8e140cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e12a00640 1 --2- 172.21.15.83:0/2218344372 >> v2:172.21.15.86:3300/0 conn(0x7f8e140b3180 0x7f8e140cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e12a00640 1 -- 172.21.15.83:0/2218344372 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e14072540 con 0x7f8e140d0180 2024-04-25T00:15:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.298+0000 7f8e13e00640 1 --2- 172.21.15.83:0/2218344372 >> v2:172.21.15.86:3300/0 conn(0x7f8e140b3180 0x7f8e140cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.302+0000 7f8e12a00640 1 --2- 172.21.15.83:0/2218344372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e140d0180 0x7f8e140b2c40 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f8e140d0cc0 tx=0x7f8e0001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd15f79b6621e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.302+0000 7f8e12000640 1 -- 172.21.15.83:0/2218344372 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e000a7070 con 0x7f8e140d0180 2024-04-25T00:15:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.302+0000 7f8e12000640 1 -- 172.21.15.83:0/2218344372 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e000b0070 con 0x7f8e140d0180 2024-04-25T00:15:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.302+0000 7f8e12000640 1 -- 172.21.15.83:0/2218344372 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e000ab890 con 0x7f8e140d0180 2024-04-25T00:15:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.302+0000 7f8e1a200640 1 -- 172.21.15.83:0/2218344372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e140d0180 msgr2=0x7f8e140b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.302+0000 7f8e1a200640 1 --2- 172.21.15.83:0/2218344372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e140d0180 0x7f8e140b2c40 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f8e140d0cc0 tx=0x7f8e0001c490 comp rx=0 tx=0).stop 2024-04-25T00:15:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.302+0000 7f8e1a200640 1 -- 172.21.15.83:0/2218344372 shutdown_connections 2024-04-25T00:15:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.302+0000 7f8e1a200640 1 --2- 172.21.15.83:0/2218344372 >> v2:172.21.15.86:3300/0 conn(0x7f8e140b3180 0x7f8e140cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.302+0000 7f8e1a200640 1 --2- 172.21.15.83:0/2218344372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e140d0180 0x7f8e140b2c40 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.302+0000 7f8e1a200640 1 --2- 172.21.15.83:0/2218344372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e14155300 0x7f8e140cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.302+0000 7f8e1a200640 1 -- 172.21.15.83:0/2218344372 >> 172.21.15.83:0/2218344372 conn(0x7f8e140b1b10 msgr2=0x7f8e140b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.302+0000 7f8e1a200640 1 -- 172.21.15.83:0/2218344372 shutdown_connections 2024-04-25T00:15:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.302+0000 7f8e1a200640 1 -- 172.21.15.83:0/2218344372 wait complete. 2024-04-25T00:15:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e1a200640 1 Processor -- start 2024-04-25T00:15:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e1a200640 1 -- start start 2024-04-25T00:15:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e1a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e140b3180 0x7f8e140af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e1a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e140d0180 0x7f8e14140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e1a200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e14155300 0x7f8e1415ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e1a200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1415ea50 con 0x7f8e140d0180 2024-04-25T00:15:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e1a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1415e8d0 con 0x7f8e140b3180 2024-04-25T00:15:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e1a200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8e1406dfd0 con 0x7f8e14155300 2024-04-25T00:15:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e12a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e140d0180 0x7f8e14140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e12a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e140d0180 0x7f8e14140590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46092/0 (socket says 172.21.15.83:46092) 2024-04-25T00:15:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e12a00640 1 -- 172.21.15.83:0/1079475629 learned_addr learned my addr 172.21.15.83:0/1079475629 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e12a00640 1 -- 172.21.15.83:0/1079475629 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e14155300 msgr2=0x7f8e1415ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e12a00640 1 --2- 172.21.15.83:0/1079475629 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e14155300 0x7f8e1415ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e12a00640 1 -- 172.21.15.83:0/1079475629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e140b3180 msgr2=0x7f8e140af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e12a00640 1 --2- 172.21.15.83:0/1079475629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e140b3180 0x7f8e140af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e12a00640 1 -- 172.21.15.83:0/1079475629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e140b0080 con 0x7f8e140d0180 2024-04-25T00:15:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e12a00640 1 --2- 172.21.15.83:0/1079475629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e140d0180 0x7f8e14140590 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8e0001ab40 tx=0x7f8e0001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e0b400640 1 -- 172.21.15.83:0/1079475629 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e000be5f0 con 0x7f8e140d0180 2024-04-25T00:15:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e0b400640 1 -- 172.21.15.83:0/1079475629 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e000be790 con 0x7f8e140d0180 2024-04-25T00:15:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e0b400640 1 -- 172.21.15.83:0/1079475629 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e000beb40 con 0x7f8e140d0180 2024-04-25T00:15:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e1a200640 1 -- 172.21.15.83:0/1079475629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e140b0360 con 0x7f8e140d0180 2024-04-25T00:15:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e1a200640 1 -- 172.21.15.83:0/1079475629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e140b0920 con 0x7f8e140d0180 2024-04-25T00:15:13.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.306+0000 7f8e1a200640 1 -- 172.21.15.83:0/1079475629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e1409b530 con 0x7f8e140d0180 2024-04-25T00:15:13.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.310+0000 7f8e0b400640 1 -- 172.21.15.83:0/1079475629 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8e000b43f0 con 0x7f8e140d0180 2024-04-25T00:15:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.310+0000 7f8e0b400640 1 --2- 172.21.15.83:0/1079475629 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8df40678c0 0x7f8df4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.310+0000 7f8e0b400640 1 -- 172.21.15.83:0/1079475629 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f8e00120880 con 0x7f8e140d0180 2024-04-25T00:15:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.310+0000 7f8e13400640 1 --2- 172.21.15.83:0/1079475629 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8df40678c0 0x7f8df4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.310+0000 7f8e13400640 1 --2- 172.21.15.83:0/1079475629 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8df40678c0 0x7f8df4069d80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8e040099c0 tx=0x7f8e04009480 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.314+0000 7f8e0b400640 1 -- 172.21.15.83:0/1079475629 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8e00016110 con 0x7f8e140d0180 2024-04-25T00:15:13.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.502+0000 7f8e1a200640 1 -- 172.21.15.83:0/1079475629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7f8e140b0f40 con 0x7f8e140d0180 2024-04-25T00:15:13.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.882+0000 7f8e0b400640 1 -- 172.21.15.83:0/1079475629 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v71) ==== 105+0+0 (secure 0 0 0) 0x7f8e000f0b20 con 0x7f8e140d0180 2024-04-25T00:15:13.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.886+0000 7f8e1a200640 1 -- 172.21.15.83:0/1079475629 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8df40678c0 msgr2=0x7f8df4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.886+0000 7f8e1a200640 1 --2- 172.21.15.83:0/1079475629 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8df40678c0 0x7f8df4069d80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8e040099c0 tx=0x7f8e04009480 comp rx=0 tx=0).stop 2024-04-25T00:15:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.886+0000 7f8e1a200640 1 -- 172.21.15.83:0/1079475629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e140d0180 msgr2=0x7f8e14140590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.886+0000 7f8e1a200640 1 --2- 172.21.15.83:0/1079475629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e140d0180 0x7f8e14140590 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8e0001ab40 tx=0x7f8e0001acd0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.886+0000 7f8e1a200640 1 -- 172.21.15.83:0/1079475629 shutdown_connections 2024-04-25T00:15:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.886+0000 7f8e1a200640 1 --2- 172.21.15.83:0/1079475629 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f8df40678c0 0x7f8df4069d80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.886+0000 7f8e1a200640 1 --2- 172.21.15.83:0/1079475629 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e14155300 0x7f8e1415ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.886+0000 7f8e1a200640 1 --2- 172.21.15.83:0/1079475629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e140d0180 0x7f8e14140590 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.886+0000 7f8e1a200640 1 --2- 172.21.15.83:0/1079475629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e140b3180 0x7f8e140af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.886+0000 7f8e1a200640 1 -- 172.21.15.83:0/1079475629 >> 172.21.15.83:0/1079475629 conn(0x7f8e140b1b10 msgr2=0x7f8e1406e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.886+0000 7f8e1a200640 1 -- 172.21.15.83:0/1079475629 shutdown_connections 2024-04-25T00:15:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:13.886+0000 7f8e1a200640 1 -- 172.21.15.83:0/1079475629 wait complete. 2024-04-25T00:15:13.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:14.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f042b800640 1 Processor -- start 2024-04-25T00:15:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f042b800640 1 -- start start 2024-04-25T00:15:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f042b800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f042415ce00 0x7f04240acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f042b800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f042406ecf0 con 0x7f04240aed90 2024-04-25T00:15:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f042b800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f042406efd0 con 0x7f0424156340 2024-04-25T00:15:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f042b800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f042406cef0 con 0x7f042415ce00 2024-04-25T00:15:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f0429800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f04240aed90 0x7f042406d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46724/0 (socket says 172.21.15.83:46724) 2024-04-25T00:15:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f0429800640 1 -- 172.21.15.83:0/4083587724 learned_addr learned my addr 172.21.15.83:0/4083587724 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:14.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f0423e00640 1 --2- 172.21.15.83:0/4083587724 >> v2:172.21.15.86:3300/0 conn(0x7f042415ce00 0x7f04240acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:14.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f0423400640 1 -- 172.21.15.83:0/4083587724 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1904990590 0 0) 0x7f042406ecf0 con 0x7f04240aed90 2024-04-25T00:15:14.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f0423400640 1 -- 172.21.15.83:0/4083587724 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f040c003530 con 0x7f04240aed90 2024-04-25T00:15:14.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f0423400640 1 -- 172.21.15.83:0/4083587724 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1264073167 0 0) 0x7f040c003530 con 0x7f04240aed90 2024-04-25T00:15:14.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f0423e00640 1 -- 172.21.15.83:0/4083587724 >> v1:172.21.15.86:6789/0 conn(0x7f0424156340 legacy=0x7f04240ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:14.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f0423e00640 1 -- 172.21.15.83:0/4083587724 >> v1:172.21.15.83:6789/0 conn(0x7f04240aed90 legacy=0x7f042406d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:14.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.050+0000 7f0423e00640 1 -- 172.21.15.83:0/4083587724 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042415d290 con 0x7f042415ce00 2024-04-25T00:15:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.054+0000 7f0423e00640 1 --2- 172.21.15.83:0/4083587724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042415ce00 0x7f04240acd20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f04100097c0 tx=0x7f041001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82a867367180c92c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.054+0000 7f0423400640 1 -- 172.21.15.83:0/4083587724 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04100b0070 con 0x7f042415ce00 2024-04-25T00:15:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.054+0000 7f0423400640 1 -- 172.21.15.83:0/4083587724 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04100ab400 con 0x7f042415ce00 2024-04-25T00:15:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.054+0000 7f0423400640 1 -- 172.21.15.83:0/4083587724 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c001a00 con 0x7f042415ce00 2024-04-25T00:15:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.054+0000 7f042b800640 1 -- 172.21.15.83:0/4083587724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042415ce00 msgr2=0x7f04240acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.054+0000 7f042b800640 1 --2- 172.21.15.83:0/4083587724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042415ce00 0x7f04240acd20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f04100097c0 tx=0x7f041001c3c0 comp rx=0 tx=0).stop 2024-04-25T00:15:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.054+0000 7f042b800640 1 -- 172.21.15.83:0/4083587724 shutdown_connections 2024-04-25T00:15:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.054+0000 7f042b800640 1 --2- 172.21.15.83:0/4083587724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042415ce00 0x7f04240acd20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.054+0000 7f042b800640 1 -- 172.21.15.83:0/4083587724 >> 172.21.15.83:0/4083587724 conn(0x7f04240b1860 msgr2=0x7f04240b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.054+0000 7f042b800640 1 -- 172.21.15.83:0/4083587724 shutdown_connections 2024-04-25T00:15:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.054+0000 7f042b800640 1 -- 172.21.15.83:0/4083587724 wait complete. 2024-04-25T00:15:14.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f042b800640 1 Processor -- start 2024-04-25T00:15:14.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f042b800640 1 -- start start 2024-04-25T00:15:14.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f042b800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04240aed90 0x7f0424150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:14.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f042b800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0424156340 0x7f0424154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:14.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f042b800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042409a9f0 0x7f04241515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:14.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f042b800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f042406e0b0 con 0x7f0424156340 2024-04-25T00:15:14.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f042b800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f042406df30 con 0x7f042409a9f0 2024-04-25T00:15:14.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f042b800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f042416a530 con 0x7f04240aed90 2024-04-25T00:15:14.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f0423e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0424156340 0x7f0424154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:14.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f0423e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0424156340 0x7f0424154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46098/0 (socket says 172.21.15.83:46098) 2024-04-25T00:15:14.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f0423e00640 1 -- 172.21.15.83:0/3517334381 learned_addr learned my addr 172.21.15.83:0/3517334381 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:14.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f0429800640 1 --2- 172.21.15.83:0/3517334381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042409a9f0 0x7f04241515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:14.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f0428e00640 1 --2- 172.21.15.83:0/3517334381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04240aed90 0x7f0424150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:14.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f0423e00640 1 -- 172.21.15.83:0/3517334381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04240aed90 msgr2=0x7f0424150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:14.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f0423e00640 1 --2- 172.21.15.83:0/3517334381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04240aed90 0x7f0424150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:14.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f0423e00640 1 -- 172.21.15.83:0/3517334381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042409a9f0 msgr2=0x7f04241515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:14.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f0423e00640 1 --2- 172.21.15.83:0/3517334381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042409a9f0 0x7f04241515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:14.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f0423e00640 1 -- 172.21.15.83:0/3517334381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0424151310 con 0x7f0424156340 2024-04-25T00:15:14.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f0423e00640 1 --2- 172.21.15.83:0/3517334381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0424156340 0x7f0424154bf0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f0410019ff0 tx=0x7f0410019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:14.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f0420c00640 1 -- 172.21.15.83:0/3517334381 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04100b8020 con 0x7f0424156340 2024-04-25T00:15:14.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f0420c00640 1 -- 172.21.15.83:0/3517334381 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04100b4700 con 0x7f0424156340 2024-04-25T00:15:14.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f042b800640 1 -- 172.21.15.83:0/3517334381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042409afb0 con 0x7f0424156340 2024-04-25T00:15:14.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.058+0000 7f0420c00640 1 -- 172.21.15.83:0/3517334381 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04100b0040 con 0x7f0424156340 2024-04-25T00:15:14.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.062+0000 7f042b800640 1 -- 172.21.15.83:0/3517334381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f042409b570 con 0x7f0424156340 2024-04-25T00:15:14.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.062+0000 7f042b800640 1 -- 172.21.15.83:0/3517334381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04241574a0 con 0x7f0424156340 2024-04-25T00:15:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.062+0000 7f0420c00640 1 -- 172.21.15.83:0/3517334381 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f04100bb020 con 0x7f0424156340 2024-04-25T00:15:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.062+0000 7f0420c00640 1 --2- 172.21.15.83:0/3517334381 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f04040678c0 0x7f0404069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.062+0000 7f0420c00640 1 -- 172.21.15.83:0/3517334381 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f041011f960 con 0x7f0424156340 2024-04-25T00:15:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.062+0000 7f0428e00640 1 --2- 172.21.15.83:0/3517334381 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f04040678c0 0x7f0404069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.062+0000 7f0428e00640 1 --2- 172.21.15.83:0/3517334381 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f04040678c0 0x7f0404069d80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f04240b2310 tx=0x7f0418009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:14.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.066+0000 7f0420c00640 1 -- 172.21.15.83:0/3517334381 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f04241574a0 con 0x7f0424156340 2024-04-25T00:15:14.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.254+0000 7f042b800640 1 -- 172.21.15.83:0/3517334381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f042404f510 con 0x7f0424156340 2024-04-25T00:15:14.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.254+0000 7f0420c00640 1 -- 172.21.15.83:0/3517334381 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5197 (secure 0 0 0) 0x7f0410015fb0 con 0x7f0424156340 2024-04-25T00:15:14.255 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:14.255 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":72,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1992401755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1992401755},{"type":"v1","addr":"172.21.15.86:6833","nonce":1992401755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3621275047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3621275047},{"type":"v1","addr":"172.21.15.86:6835","nonce":3621275047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:51.879542+0000","modified":"2024-04-25T00:15:13.889009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7719,"mds_1":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7719":{"gid":7719,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6835/565131724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":565131724},{"type":"v1","addr":"172.21.15.83:6835","nonce":565131724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7728":{"gid":7728,"name":"a","rank":1,"incarnation":72,"state":"up:creating","state_seq":1,"addr":"172.21.15.83:6837/2870595352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2870595352},{"type":"v1","addr":"172.21.15.83:6837","nonce":2870595352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7719,"qdb_cluster":[7719]},"id":7}]} 2024-04-25T00:15:14.256 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 72 2024-04-25T00:15:14.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.254+0000 7f042b800640 1 -- 172.21.15.83:0/3517334381 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f04040678c0 msgr2=0x7f0404069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:14.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.254+0000 7f042b800640 1 --2- 172.21.15.83:0/3517334381 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f04040678c0 0x7f0404069d80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f04240b2310 tx=0x7f0418009040 comp rx=0 tx=0).stop 2024-04-25T00:15:14.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.254+0000 7f042b800640 1 -- 172.21.15.83:0/3517334381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0424156340 msgr2=0x7f0424154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:14.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.254+0000 7f042b800640 1 --2- 172.21.15.83:0/3517334381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0424156340 0x7f0424154bf0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f0410019ff0 tx=0x7f0410019160 comp rx=0 tx=0).stop 2024-04-25T00:15:14.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.254+0000 7f042b800640 1 -- 172.21.15.83:0/3517334381 shutdown_connections 2024-04-25T00:15:14.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.254+0000 7f042b800640 1 --2- 172.21.15.83:0/3517334381 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f04040678c0 0x7f0404069d80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:14.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.254+0000 7f042b800640 1 --2- 172.21.15.83:0/3517334381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f042409a9f0 0x7f04241515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:14.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.254+0000 7f042b800640 1 --2- 172.21.15.83:0/3517334381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0424156340 0x7f0424154bf0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:14.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.254+0000 7f042b800640 1 --2- 172.21.15.83:0/3517334381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04240aed90 0x7f0424150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:14.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.254+0000 7f042b800640 1 -- 172.21.15.83:0/3517334381 >> 172.21.15.83:0/3517334381 conn(0x7f04240b1860 msgr2=0x7f042415dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:14.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.258+0000 7f042b800640 1 -- 172.21.15.83:0/3517334381 shutdown_connections 2024-04-25T00:15:14.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:14.258+0000 7f042b800640 1 -- 172.21.15.83:0/3517334381 wait complete. 2024-04-25T00:15:14.679 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:14:51.879542+0000', 'modified': '2024-04-25T00:15:13.889009+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 96, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7719, 'mds_1': 7728}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7719': {'gid': 7719, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.83:6835/565131724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 565131724}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 565131724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7728': {'gid': 7728, 'name': 'a', 'rank': 1, 'incarnation': 72, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.83:6837/2870595352', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 2870595352}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 2870595352}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7719, 'qdb_cluster': [7719]} 2024-04-25T00:15:14.679 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7728:up:creating 2024-04-25T00:15:15.680 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:15.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.822+0000 7f6268c00640 1 Processor -- start 2024-04-25T00:15:15.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.822+0000 7f6268c00640 1 -- start start 2024-04-25T00:15:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6268c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f626406e680 0x7f626406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6268c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6264071360 0x7f62640717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6268c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62640ae0a0 con 0x7f6264155470 2024-04-25T00:15:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6268c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f62640ae380 con 0x7f626406e680 2024-04-25T00:15:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6268c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f62640ae660 con 0x7f6264071360 2024-04-25T00:15:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6261400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f626406e680 0x7f626406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6261400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f626406e680 0x7f626406eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39610/0 (socket says 172.21.15.83:39610) 2024-04-25T00:15:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6261400640 1 -- 172.21.15.83:0/1973456856 learned_addr learned my addr 172.21.15.83:0/1973456856 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6261e00640 1 --1- 172.21.15.83:0/1973456856 >> v1:172.21.15.83:6789/0 conn(0x7f6264155470 0x7f62640ad5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44242/0 (socket says 172.21.15.83:44242) 2024-04-25T00:15:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6262800640 1 --2- 172.21.15.83:0/1973456856 >> v2:172.21.15.86:3300/0 conn(0x7f6264071360 0x7f62640717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6261400640 1 -- 172.21.15.83:0/1973456856 >> v2:172.21.15.86:3300/0 conn(0x7f6264071360 msgr2=0x7f62640717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:15.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6261400640 1 --2- 172.21.15.83:0/1973456856 >> v2:172.21.15.86:3300/0 conn(0x7f6264071360 0x7f62640717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:15.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6261400640 1 -- 172.21.15.83:0/1973456856 >> v1:172.21.15.83:6789/0 conn(0x7f6264155470 legacy=0x7f62640ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:15.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6261400640 1 -- 172.21.15.83:0/1973456856 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62640ae940 con 0x7f626406e680 2024-04-25T00:15:15.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.826+0000 7f6262800640 1 --2- 172.21.15.83:0/1973456856 >> v2:172.21.15.86:3300/0 conn(0x7f6264071360 0x7f62640717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:15.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.830+0000 7f6261400640 1 --2- 172.21.15.83:0/1973456856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626406e680 0x7f626406eb00 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f6264158ed0 tx=0x7f625001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a9613b11b7a0d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:15.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.830+0000 7f6260a00640 1 -- 172.21.15.83:0/1973456856 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62500b0070 con 0x7f626406e680 2024-04-25T00:15:15.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.830+0000 7f6260a00640 1 -- 172.21.15.83:0/1973456856 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62500ab600 con 0x7f626406e680 2024-04-25T00:15:15.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.830+0000 7f6260a00640 1 -- 172.21.15.83:0/1973456856 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62500ab9d0 con 0x7f626406e680 2024-04-25T00:15:15.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.830+0000 7f6268c00640 1 -- 172.21.15.83:0/1973456856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626406e680 msgr2=0x7f626406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:15.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.830+0000 7f6268c00640 1 --2- 172.21.15.83:0/1973456856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626406e680 0x7f626406eb00 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f6264158ed0 tx=0x7f625001c520 comp rx=0 tx=0).stop 2024-04-25T00:15:15.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.830+0000 7f6268c00640 1 -- 172.21.15.83:0/1973456856 shutdown_connections 2024-04-25T00:15:15.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.830+0000 7f6268c00640 1 --2- 172.21.15.83:0/1973456856 >> v2:172.21.15.86:3300/0 conn(0x7f6264071360 0x7f62640717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:15.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.830+0000 7f6268c00640 1 --2- 172.21.15.83:0/1973456856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626406e680 0x7f626406eb00 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:15.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.830+0000 7f6268c00640 1 -- 172.21.15.83:0/1973456856 >> 172.21.15.83:0/1973456856 conn(0x7f62640b1860 msgr2=0x7f62640b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:15.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.830+0000 7f6268c00640 1 -- 172.21.15.83:0/1973456856 shutdown_connections 2024-04-25T00:15:15.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.830+0000 7f6268c00640 1 -- 172.21.15.83:0/1973456856 wait complete. 2024-04-25T00:15:15.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.830+0000 7f6268c00640 1 Processor -- start 2024-04-25T00:15:15.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.830+0000 7f6268c00640 1 -- start start 2024-04-25T00:15:15.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6268c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626406e680 0x7f62641fb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:15.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6268c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6264071360 0x7f6264150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:15.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6268c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6264155470 0x7f626409c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:15.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6268c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6264167e10 con 0x7f6264155470 2024-04-25T00:15:15.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6268c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6264167c90 con 0x7f626406e680 2024-04-25T00:15:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6268c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6264167f90 con 0x7f6264071360 2024-04-25T00:15:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6262800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6264155470 0x7f626409c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6262800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6264155470 0x7f626409c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39626/0 (socket says 172.21.15.83:39626) 2024-04-25T00:15:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6262800640 1 -- 172.21.15.83:0/2682338783 learned_addr learned my addr 172.21.15.83:0/2682338783 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6261400640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6264071360 0x7f6264150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6261e00640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626406e680 0x7f62641fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6262800640 1 -- 172.21.15.83:0/2682338783 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6264071360 msgr2=0x7f6264150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6262800640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6264071360 0x7f6264150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6262800640 1 -- 172.21.15.83:0/2682338783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626406e680 msgr2=0x7f62641fb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6262800640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626406e680 0x7f62641fb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6262800640 1 -- 172.21.15.83:0/2682338783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626409c8e0 con 0x7f6264155470 2024-04-25T00:15:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6261e00640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626406e680 0x7f62641fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:15.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6262800640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6264155470 0x7f626409c3a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f625c00c7f0 tx=0x7f625c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:15.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f625a000640 1 -- 172.21.15.83:0/2682338783 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c015070 con 0x7f6264155470 2024-04-25T00:15:15.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f625a000640 1 -- 172.21.15.83:0/2682338783 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f625c007860 con 0x7f6264155470 2024-04-25T00:15:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f625a000640 1 -- 172.21.15.83:0/2682338783 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c007c50 con 0x7f6264155470 2024-04-25T00:15:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6268c00640 1 -- 172.21.15.83:0/2682338783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626409cb80 con 0x7f6264155470 2024-04-25T00:15:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6268c00640 1 -- 172.21.15.83:0/2682338783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62641fad20 con 0x7f6264155470 2024-04-25T00:15:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f6268c00640 1 -- 172.21.15.83:0/2682338783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626404f510 con 0x7f6264155470 2024-04-25T00:15:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.834+0000 7f625a000640 1 -- 172.21.15.83:0/2682338783 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f625c018070 con 0x7f6264155470 2024-04-25T00:15:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.838+0000 7f625a000640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f62440678c0 0x7f6244069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:15.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.838+0000 7f625a000640 1 -- 172.21.15.83:0/2682338783 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f625c0860a0 con 0x7f6264155470 2024-04-25T00:15:15.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.838+0000 7f6261e00640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f62440678c0 0x7f6244069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:15.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.838+0000 7f6261e00640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f62440678c0 0x7f6244069d80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f62540012f0 tx=0x7f6254009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:15.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:15.842+0000 7f625a000640 1 -- 172.21.15.83:0/2682338783 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f625c056990 con 0x7f6264155470 2024-04-25T00:15:15.860 INFO:tasks.ceph.mds.c.smithi083.stderr:2024-04-25T00:15:15.858+0000 7fd6a8200640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T00:15:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.030+0000 7f6268c00640 1 -- 172.21.15.83:0/2682338783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f626409da20 con 0x7f6264155470 2024-04-25T00:15:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.030+0000 7f625a000640 1 -- 172.21.15.83:0/2682338783 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5200 (secure 0 0 0) 0x7f625c056290 con 0x7f6264155470 2024-04-25T00:15:16.032 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:16.032 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1992401755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1992401755},{"type":"v1","addr":"172.21.15.86:6833","nonce":1992401755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3621275047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3621275047},{"type":"v1","addr":"172.21.15.86:6835","nonce":3621275047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:51.879542+0000","modified":"2024-04-25T00:15:14.891731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7719,"mds_1":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7719":{"gid":7719,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6835/565131724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":565131724},{"type":"v1","addr":"172.21.15.83:6835","nonce":565131724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7728":{"gid":7728,"name":"a","rank":1,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2870595352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2870595352},{"type":"v1","addr":"172.21.15.83:6837","nonce":2870595352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7719,"qdb_cluster":[7719,7728]},"id":7}]} 2024-04-25T00:15:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 73 2024-04-25T00:15:16.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.030+0000 7f6268c00640 1 -- 172.21.15.83:0/2682338783 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f62440678c0 msgr2=0x7f6244069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.030+0000 7f6268c00640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f62440678c0 0x7f6244069d80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f62540012f0 tx=0x7f6254009040 comp rx=0 tx=0).stop 2024-04-25T00:15:16.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.030+0000 7f6268c00640 1 -- 172.21.15.83:0/2682338783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6264155470 msgr2=0x7f626409c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.030+0000 7f6268c00640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6264155470 0x7f626409c3a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f625c00c7f0 tx=0x7f625c00ccb0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.034+0000 7f6268c00640 1 -- 172.21.15.83:0/2682338783 shutdown_connections 2024-04-25T00:15:16.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.034+0000 7f6268c00640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f62440678c0 0x7f6244069d80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.034+0000 7f6268c00640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6264071360 0x7f6264150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.034+0000 7f6268c00640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6264155470 0x7f626409c3a0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.034+0000 7f6268c00640 1 --2- 172.21.15.83:0/2682338783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626406e680 0x7f62641fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.034+0000 7f6268c00640 1 -- 172.21.15.83:0/2682338783 >> 172.21.15.83:0/2682338783 conn(0x7f62640b1860 msgr2=0x7f62640afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:16.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.034+0000 7f6268c00640 1 -- 172.21.15.83:0/2682338783 shutdown_connections 2024-04-25T00:15:16.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.034+0000 7f6268c00640 1 -- 172.21.15.83:0/2682338783 wait complete. 2024-04-25T00:15:16.455 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:14:51.879542+0000', 'modified': '2024-04-25T00:15:14.891731+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 96, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7719, 'mds_1': 7728}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7719': {'gid': 7719, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.83:6835/565131724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 565131724}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 565131724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7728': {'gid': 7728, 'name': 'a', 'rank': 1, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/2870595352', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 2870595352}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 2870595352}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7719, 'qdb_cluster': [7719, 7728]} 2024-04-25T00:15:16.456 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-04-25T00:15:16.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:15:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc61f200640 1 Processor -- start 2024-04-25T00:15:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc61f200640 1 -- start start 2024-04-25T00:15:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc61f200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc61815be20 0x7fc61815f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc61f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6180cedd0 0x7fc61806cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc61f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc61815c200 con 0x7fc6180acd20 2024-04-25T00:15:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc61f200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc61815fed0 con 0x7fc61815be20 2024-04-25T00:15:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc61f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc6180b2e70 con 0x7fc6180cedd0 2024-04-25T00:15:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc617e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc61815be20 0x7fc61815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc617e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc61815be20 0x7fc61815f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39640/0 (socket says 172.21.15.83:39640) 2024-04-25T00:15:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc617e00640 1 -- 172.21.15.83:0/2662519433 learned_addr learned my addr 172.21.15.83:0/2662519433 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc60f400640 1 --2- 172.21.15.83:0/2662519433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6180cedd0 0x7fc61806cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc617e00640 1 -- 172.21.15.83:0/2662519433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6180cedd0 msgr2=0x7fc61806cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc617e00640 1 --2- 172.21.15.83:0/2662519433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6180cedd0 0x7fc61806cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc617e00640 1 -- 172.21.15.83:0/2662519433 >> v1:172.21.15.86:6789/0 conn(0x7fc6180acd20 legacy=0x7fc6180b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc617e00640 1 -- 172.21.15.83:0/2662519433 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc618160100 con 0x7fc61815be20 2024-04-25T00:15:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.598+0000 7fc60f400640 1 --2- 172.21.15.83:0/2662519433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6180cedd0 0x7fc61806cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:16.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.602+0000 7fc617e00640 1 --2- 172.21.15.83:0/2662519433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc61815be20 0x7fc61815f990 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fc608009870 tx=0x7fc60801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aabfe2bf8c9c2cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.602+0000 7fc617400640 1 -- 172.21.15.83:0/2662519433 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080b0070 con 0x7fc61815be20 2024-04-25T00:15:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.602+0000 7fc617400640 1 -- 172.21.15.83:0/2662519433 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6080ab600 con 0x7fc61815be20 2024-04-25T00:15:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.602+0000 7fc617400640 1 -- 172.21.15.83:0/2662519433 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080ab9d0 con 0x7fc61815be20 2024-04-25T00:15:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.602+0000 7fc61f200640 1 -- 172.21.15.83:0/2662519433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc61815be20 msgr2=0x7fc61815f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.602+0000 7fc61f200640 1 --2- 172.21.15.83:0/2662519433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc61815be20 0x7fc61815f990 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fc608009870 tx=0x7fc60801c520 comp rx=0 tx=0).stop 2024-04-25T00:15:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.602+0000 7fc61f200640 1 -- 172.21.15.83:0/2662519433 shutdown_connections 2024-04-25T00:15:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.602+0000 7fc61f200640 1 --2- 172.21.15.83:0/2662519433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6180cedd0 0x7fc61806cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.602+0000 7fc61f200640 1 --2- 172.21.15.83:0/2662519433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc61815be20 0x7fc61815f990 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.602+0000 7fc61f200640 1 -- 172.21.15.83:0/2662519433 >> 172.21.15.83:0/2662519433 conn(0x7fc6180b1b10 msgr2=0x7fc6180b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.602+0000 7fc61f200640 1 -- 172.21.15.83:0/2662519433 shutdown_connections 2024-04-25T00:15:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.602+0000 7fc61f200640 1 -- 172.21.15.83:0/2662519433 wait complete. 2024-04-25T00:15:16.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.602+0000 7fc61f200640 1 Processor -- start 2024-04-25T00:15:16.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.602+0000 7fc61f200640 1 -- start start 2024-04-25T00:15:16.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc61f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6180acd20 0x7fc618141610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc61f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6180cedd0 0x7fc618070e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc61f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61815be20 0x7fc618159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc61f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc618168f40 con 0x7fc6180acd20 2024-04-25T00:15:16.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc61f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc61806d810 con 0x7fc61815be20 2024-04-25T00:15:16.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc61f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc6181690c0 con 0x7fc6180cedd0 2024-04-25T00:15:16.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc617e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6180acd20 0x7fc618141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc617e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6180acd20 0x7fc618141610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39650/0 (socket says 172.21.15.83:39650) 2024-04-25T00:15:16.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc617e00640 1 -- 172.21.15.83:0/32610767 learned_addr learned my addr 172.21.15.83:0/32610767 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:16.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc617e00640 1 -- 172.21.15.83:0/32610767 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6180cedd0 msgr2=0x7fc618070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc60f400640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6180cedd0 0x7fc618070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc617e00640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6180cedd0 0x7fc618070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc61d200640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61815be20 0x7fc618159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc617e00640 1 -- 172.21.15.83:0/32610767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61815be20 msgr2=0x7fc618159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc617e00640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61815be20 0x7fc618159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc617e00640 1 -- 172.21.15.83:0/32610767 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc618141b50 con 0x7fc6180acd20 2024-04-25T00:15:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc60f400640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6180cedd0 0x7fc618070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc61d200640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61815be20 0x7fc618159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc617e00640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6180acd20 0x7fc618141610 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fc6080ae040 tx=0x7fc6080192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc614c00640 1 -- 172.21.15.83:0/32610767 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080b0040 con 0x7fc6180acd20 2024-04-25T00:15:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc614c00640 1 -- 172.21.15.83:0/32610767 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6080b9070 con 0x7fc6180acd20 2024-04-25T00:15:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc61f200640 1 -- 172.21.15.83:0/32610767 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc618071720 con 0x7fc6180acd20 2024-04-25T00:15:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc614c00640 1 -- 172.21.15.83:0/32610767 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080b4660 con 0x7fc6180acd20 2024-04-25T00:15:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.606+0000 7fc61f200640 1 -- 172.21.15.83:0/32610767 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc618071ce0 con 0x7fc6180acd20 2024-04-25T00:15:16.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc614c00640 1 -- 172.21.15.83:0/32610767 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc6080b4800 con 0x7fc6180acd20 2024-04-25T00:15:16.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc614c00640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc5f40678c0 0x7fc5f4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc614c00640 1 -- 172.21.15.83:0/32610767 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7fc60811fd50 con 0x7fc6180acd20 2024-04-25T00:15:16.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc60f400640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc5f40678c0 0x7fc5f4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc62272f000 1 Processor -- start 2024-04-25T00:15:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc62272f000 1 -- start start 2024-04-25T00:15:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc60f400640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc5f40678c0 0x7fc5f4069d80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc61814fb10 tx=0x7fc600008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc62272f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63d452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc62272f000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63d45810 0x563a63cfb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc62272f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63cfbbc0 0x563a63cfbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc62272f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a63b495e0 con 0x563a63d44ef0 2024-04-25T00:15:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc62272f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a63ad0c50 con 0x563a63cfbbc0 2024-04-25T00:15:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc62272f000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x563a63b455a0 con 0x563a63d45810 2024-04-25T00:15:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc617e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63d452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc617e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63d452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39658/0 (socket says 172.21.15.83:39658) 2024-04-25T00:15:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc617e00640 1 -- 172.21.15.83:0/3837796433 learned_addr learned my addr 172.21.15.83:0/3837796433 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc60f400640 1 --2- 172.21.15.83:0/3837796433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63d45810 0x563a63cfb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc61d200640 1 --2- 172.21.15.83:0/3837796433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63cfbbc0 0x563a63cfbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc617e00640 1 -- 172.21.15.83:0/3837796433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63d45810 msgr2=0x563a63cfb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc617e00640 1 --2- 172.21.15.83:0/3837796433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63d45810 0x563a63cfb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc617e00640 1 -- 172.21.15.83:0/3837796433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63cfbbc0 msgr2=0x563a63cfbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc617e00640 1 --2- 172.21.15.83:0/3837796433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63cfbbc0 0x563a63cfbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc617e00640 1 -- 172.21.15.83:0/3837796433 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a63aced10 con 0x563a63d44ef0 2024-04-25T00:15:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc617e00640 1 --2- 172.21.15.83:0/3837796433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63d452d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fc608006db0 tx=0x7fc6080ef8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67262e405ab5d08c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc60cc00640 1 -- 172.21.15.83:0/3837796433 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60811d070 con 0x563a63d44ef0 2024-04-25T00:15:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc60cc00640 1 -- 172.21.15.83:0/3837796433 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6080f8a90 con 0x563a63d44ef0 2024-04-25T00:15:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.610+0000 7fc60cc00640 1 -- 172.21.15.83:0/3837796433 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60811e0b0 con 0x563a63d44ef0 2024-04-25T00:15:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 -- 172.21.15.83:0/3837796433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 msgr2=0x563a63d452d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 --2- 172.21.15.83:0/3837796433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63d452d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fc608006db0 tx=0x7fc6080ef8d0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 -- 172.21.15.83:0/3837796433 shutdown_connections 2024-04-25T00:15:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 --2- 172.21.15.83:0/3837796433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63d45810 0x563a63cfb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 --2- 172.21.15.83:0/3837796433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63cfbbc0 0x563a63cfbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 --2- 172.21.15.83:0/3837796433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63d452d0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 -- 172.21.15.83:0/3837796433 >> 172.21.15.83:0/3837796433 conn(0x563a63cf7b70 msgr2=0x563a63cf9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 -- 172.21.15.83:0/3837796433 shutdown_connections 2024-04-25T00:15:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 -- 172.21.15.83:0/3837796433 wait complete. 2024-04-25T00:15:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 -- start start 2024-04-25T00:15:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 Processor -- start 2024-04-25T00:15:16.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63cfbbc0 0x563a63d29f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63d2a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 0x563a63d2a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a63b03520 con 0x563a63d44ef0 2024-04-25T00:15:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a63b16340 con 0x563a63d45810 2024-04-25T00:15:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x563a63b0d670 con 0x563a63cfbbc0 2024-04-25T00:15:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 10 client.0 _ll_register_callbacks cb 0x563a63ce65c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc60f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63d2a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc60f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63d2a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39664/0 (socket says 172.21.15.83:39664) 2024-04-25T00:15:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc60f400640 1 -- 172.21.15.83:0/4170057670 learned_addr learned my addr 172.21.15.83:0/4170057670 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc617e00640 1 --2- 172.21.15.83:0/4170057670 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63cfbbc0 0x563a63d29f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc61d200640 1 --2- 172.21.15.83:0/4170057670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 0x563a63d2a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc60f400640 1 -- 172.21.15.83:0/4170057670 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63cfbbc0 msgr2=0x563a63d29f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc60f400640 1 --2- 172.21.15.83:0/4170057670 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63cfbbc0 0x563a63d29f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc60f400640 1 -- 172.21.15.83:0/4170057670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 msgr2=0x563a63d2a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc60f400640 1 --2- 172.21.15.83:0/4170057670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 0x563a63d2a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc60f400640 1 -- 172.21.15.83:0/4170057670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a63ae17b0 con 0x563a63d44ef0 2024-04-25T00:15:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc60f400640 1 --2- 172.21.15.83:0/4170057670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63d2a460 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fc600012750 tx=0x7fc600004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c57f124d1d35b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc605600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:15:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc605600640 1 -- 172.21.15.83:0/4170057670 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600023020 con 0x563a63d44ef0 2024-04-25T00:15:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc605600640 1 -- 172.21.15.83:0/4170057670 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7fc60001f630 con 0x563a63d44ef0 2024-04-25T00:15:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 -- 172.21.15.83:0/4170057670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6080f8a90 con 0x563a63d44ef0 2024-04-25T00:15:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc605600640 1 -- 172.21.15.83:0/4170057670 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fc60002d050 con 0x563a63d44ef0 2024-04-25T00:15:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 10 client.7830 fetch_fsmap learned FSMap version 73 2024-04-25T00:15:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc62272f000 1 -- 172.21.15.83:0/4170057670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x563a63ad0ea0 con 0x563a63d44ef0 2024-04-25T00:15:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.614+0000 7fc605600640 1 -- 172.21.15.83:0/4170057670 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6080f8a90 con 0x563a63d44ef0 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc605600640 1 -- 172.21.15.83:0/4170057670 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 73) ==== 3514+0+0 (secure 0 0 0) 0x7fc600030020 con 0x563a63d44ef0 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 10 client.7830 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 10 client.7830 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 20 client.7830 populate_metadata read hostname 'smithi083' 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 1 --2- 172.21.15.83:0/4170057670 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x563a63d54f90 0x563a63d57380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 4 client.7830 mds_command: new command op to 7719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 1 -- 172.21.15.83:0/4170057670 --> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a63ae4ab0 con 0x563a63d54f90 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc617e00640 1 --2- 172.21.15.83:0/4170057670 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x563a63d54f90 0x563a63d57380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc605600640 10 client.7830 ms_handle_connect on v2:172.21.15.83:6834/565131724 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc617e00640 1 --2- 172.21.15.83:0/4170057670 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x563a63d54f90 0x563a63d57380 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb5da524b496cb24 server_cookie=56c9e596e6da0245 in_seq=0 out_seq=0 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc605600640 1 -- 172.21.15.83:0/4170057670 <== mds.0 v2:172.21.15.83:6834/565131724 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a63ae4ab0 con 0x563a63d54f90 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc605600640 10 client.7830 handle_command_reply: tid=0 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 1 client.7830 shutdown 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 1 -- 172.21.15.83:0/4170057670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 msgr2=0x563a63d2a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 1 --2- 172.21.15.83:0/4170057670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63d2a460 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fc600012750 tx=0x7fc600004740 comp rx=0 tx=0).stop 2024-04-25T00:15:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 1 -- 172.21.15.83:0/4170057670 shutdown_connections 2024-04-25T00:15:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 1 --2- 172.21.15.83:0/4170057670 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x563a63d54f90 0x563a63d57380 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 1 --2- 172.21.15.83:0/4170057670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 0x563a63d2a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 1 --2- 172.21.15.83:0/4170057670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63d2a460 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc605600640 0 client.7830 ms_handle_reset on v2:172.21.15.83:6834/565131724 2024-04-25T00:15:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 1 --2- 172.21.15.83:0/4170057670 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63cfbbc0 0x563a63d29f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 1 -- 172.21.15.83:0/4170057670 >> 172.21.15.83:0/4170057670 conn(0x563a63cf7b70 msgr2=0x563a63cfa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 1 -- 172.21.15.83:0/4170057670 shutdown_connections 2024-04-25T00:15:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 1 -- 172.21.15.83:0/4170057670 wait complete. 2024-04-25T00:15:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.618+0000 7fc62272f000 20 client.7830 trim_cache size 0 max 16384 2024-04-25T00:15:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.634+0000 7fc62272f000 1 Processor -- start 2024-04-25T00:15:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.634+0000 7fc62272f000 1 -- start start 2024-04-25T00:15:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63cfbbc0 0x563a63cfaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63cfb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 0x563a63d43f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc618203fb0 con 0x563a63d44ef0 2024-04-25T00:15:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc61809bc70 con 0x563a63d45810 2024-04-25T00:15:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc61809c2c0 con 0x563a63cfbbc0 2024-04-25T00:15:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc60f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63cfb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc60f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63cfb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39670/0 (socket says 172.21.15.83:39670) 2024-04-25T00:15:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc60f400640 1 -- 172.21.15.83:0/3658922697 learned_addr learned my addr 172.21.15.83:0/3658922697 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc617e00640 1 --2- 172.21.15.83:0/3658922697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63cfbbc0 0x563a63cfaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc61d200640 1 --2- 172.21.15.83:0/3658922697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 0x563a63d43f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc60f400640 1 -- 172.21.15.83:0/3658922697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63cfbbc0 msgr2=0x563a63cfaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc60f400640 1 --2- 172.21.15.83:0/3658922697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63cfbbc0 0x563a63cfaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc60f400640 1 -- 172.21.15.83:0/3658922697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 msgr2=0x563a63d43f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc60f400640 1 --2- 172.21.15.83:0/3658922697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 0x563a63d43f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc60f400640 1 -- 172.21.15.83:0/3658922697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a63ad1e80 con 0x563a63d44ef0 2024-04-25T00:15:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc617e00640 1 --2- 172.21.15.83:0/3658922697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63cfbbc0 0x563a63cfaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc60f400640 1 --2- 172.21.15.83:0/3658922697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63cfb4b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fc600008010 tx=0x7fc60001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b613ea50d31c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc607e00640 1 -- 172.21.15.83:0/3658922697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600023050 con 0x563a63d44ef0 2024-04-25T00:15:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc607e00640 1 -- 172.21.15.83:0/3658922697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60001a040 con 0x563a63d44ef0 2024-04-25T00:15:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc607e00640 1 -- 172.21.15.83:0/3658922697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600019070 con 0x563a63d44ef0 2024-04-25T00:15:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 -- 172.21.15.83:0/3658922697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 msgr2=0x563a63cfb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 --2- 172.21.15.83:0/3658922697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63cfb4b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fc600008010 tx=0x7fc60001e000 comp rx=0 tx=0).stop 2024-04-25T00:15:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 -- 172.21.15.83:0/3658922697 shutdown_connections 2024-04-25T00:15:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 --2- 172.21.15.83:0/3658922697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 0x563a63d43f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 --2- 172.21.15.83:0/3658922697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63d44ef0 0x563a63cfb4b0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 --2- 172.21.15.83:0/3658922697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63cfbbc0 0x563a63cfaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 -- 172.21.15.83:0/3658922697 >> 172.21.15.83:0/3658922697 conn(0x563a63cf7b70 msgr2=0x563a63d43800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:16.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 -- 172.21.15.83:0/3658922697 shutdown_connections 2024-04-25T00:15:16.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 -- 172.21.15.83:0/3658922697 wait complete. 2024-04-25T00:15:16.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 -- start start 2024-04-25T00:15:16.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 Processor -- start 2024-04-25T00:15:16.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63cfbbc0 0x563a63dbd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 0x563a63d57af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63d58030 0x563a63d58410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a63b55180 con 0x563a63cfbbc0 2024-04-25T00:15:16.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a63b12810 con 0x563a63d45810 2024-04-25T00:15:16.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x563a63ae1dd0 con 0x563a63d58030 2024-04-25T00:15:16.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.638+0000 7fc62272f000 10 client.0 _ll_register_callbacks cb 0x7fc600008620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:16.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc617e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63cfbbc0 0x563a63dbd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc60f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 0x563a63d57af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc617e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63cfbbc0 0x563a63dbd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39676/0 (socket says 172.21.15.83:39676) 2024-04-25T00:15:16.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc617e00640 1 -- 172.21.15.83:0/614483273 learned_addr learned my addr 172.21.15.83:0/614483273 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:16.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc61d200640 1 --2- 172.21.15.83:0/614483273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63d58030 0x563a63d58410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc617e00640 1 -- 172.21.15.83:0/614483273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63d58030 msgr2=0x563a63d58410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc617e00640 1 --2- 172.21.15.83:0/614483273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63d58030 0x563a63d58410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc617e00640 1 -- 172.21.15.83:0/614483273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 msgr2=0x563a63d57af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc617e00640 1 --2- 172.21.15.83:0/614483273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 0x563a63d57af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc617e00640 1 -- 172.21.15.83:0/614483273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a63b16760 con 0x563a63cfbbc0 2024-04-25T00:15:16.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc617e00640 1 --2- 172.21.15.83:0/614483273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63cfbbc0 0x563a63dbd160 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fc6080ec310 tx=0x7fc608006d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354ebf88b4c73295 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:16.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc605600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:15:16.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc605600640 1 -- 172.21.15.83:0/614483273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60811d040 con 0x563a63cfbbc0 2024-04-25T00:15:16.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc605600640 1 -- 172.21.15.83:0/614483273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7fc60811e020 con 0x563a63cfbbc0 2024-04-25T00:15:16.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 -- 172.21.15.83:0/614483273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc60001a040 con 0x563a63cfbbc0 2024-04-25T00:15:16.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc605600640 1 -- 172.21.15.83:0/614483273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fc608100050 con 0x563a63cfbbc0 2024-04-25T00:15:16.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 10 client.7836 fetch_fsmap learned FSMap version 73 2024-04-25T00:15:16.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 -- 172.21.15.83:0/614483273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x563a63ba29e0 con 0x563a63cfbbc0 2024-04-25T00:15:16.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc605600640 1 -- 172.21.15.83:0/614483273 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60001a040 con 0x563a63cfbbc0 2024-04-25T00:15:16.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc605600640 1 -- 172.21.15.83:0/614483273 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 73) ==== 3514+0+0 (secure 0 0 0) 0x7fc608103020 con 0x563a63cfbbc0 2024-04-25T00:15:16.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 10 client.7836 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T00:15:16.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 10 client.7836 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:15:16.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 20 client.7836 populate_metadata read hostname 'smithi083' 2024-04-25T00:15:16.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 --2- 172.21.15.83:0/614483273 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x563a63d58950 0x563a63d58d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 4 client.7836 mds_command: new command op to 7719 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:15:16.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 -- 172.21.15.83:0/614483273 --> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] -- command(tid 0: {"prefix": "status"}) -- 0x563a63b56b20 con 0x563a63d58950 2024-04-25T00:15:16.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc60f400640 1 --2- 172.21.15.83:0/614483273 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x563a63d58950 0x563a63d58d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc60f400640 1 --2- 172.21.15.83:0/614483273 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x563a63d58950 0x563a63d58d30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5db89a4e8e546e92 server_cookie=aa2cee5d2ef939a1 in_seq=0 out_seq=0 2024-04-25T00:15:16.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc605600640 10 client.7836 ms_handle_connect on v2:172.21.15.83:6834/565131724 2024-04-25T00:15:16.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc605600640 1 -- 172.21.15.83:0/614483273 <== mds.0 v2:172.21.15.83:6834/565131724 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x563a63b56b20 con 0x563a63d58950 2024-04-25T00:15:16.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc605600640 10 client.7836 handle_command_reply: tid=0 2024-04-25T00:15:16.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 client.7836 shutdown 2024-04-25T00:15:16.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 -- 172.21.15.83:0/614483273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63cfbbc0 msgr2=0x563a63dbd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 --2- 172.21.15.83:0/614483273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63cfbbc0 0x563a63dbd160 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fc6080ec310 tx=0x7fc608006d00 comp rx=0 tx=0).stop 2024-04-25T00:15:16.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 -- 172.21.15.83:0/614483273 shutdown_connections 2024-04-25T00:15:16.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 --2- 172.21.15.83:0/614483273 >> [v2:172.21.15.83:6834/565131724,v1:172.21.15.83:6835/565131724] conn(0x563a63d58950 0x563a63d58d30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 --2- 172.21.15.83:0/614483273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x563a63d58030 0x563a63d58410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 --2- 172.21.15.83:0/614483273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a63d45810 0x563a63d57af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc605600640 0 client.7836 ms_handle_reset on v2:172.21.15.83:6834/565131724 2024-04-25T00:15:16.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 --2- 172.21.15.83:0/614483273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a63cfbbc0 0x563a63dbd160 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 -- 172.21.15.83:0/614483273 >> 172.21.15.83:0/614483273 conn(0x563a63cf7b70 msgr2=0x563a63d44090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:16.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 -- 172.21.15.83:0/614483273 shutdown_connections 2024-04-25T00:15:16.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 1 -- 172.21.15.83:0/614483273 wait complete. 2024-04-25T00:15:16.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc62272f000 20 client.7836 trim_cache size 0 max 16384 2024-04-25T00:15:16.652 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:15:16.653 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "d58ae83a-ad72-4e18-824a-86e779979c35", 2024-04-25T00:15:16.653 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-25T00:15:16.653 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7719, 2024-04-25T00:15:16.653 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T00:15:16.653 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T00:15:16.653 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T00:15:16.653 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 10.786338989000001, 2024-04-25T00:15:16.653 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 73, 2024-04-25T00:15:16.653 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 97, 2024-04-25T00:15:16.653 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 97, 2024-04-25T00:15:16.654 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 11.625704751000001 2024-04-25T00:15:16.654 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:15:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc61f200640 1 -- 172.21.15.83:0/32610767 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc5f40678c0 msgr2=0x7fc5f4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc61f200640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc5f40678c0 0x7fc5f4069d80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc61814fb10 tx=0x7fc600008040 comp rx=0 tx=0).stop 2024-04-25T00:15:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc61f200640 1 -- 172.21.15.83:0/32610767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6180acd20 msgr2=0x7fc618141610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc61f200640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6180acd20 0x7fc618141610 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fc6080ae040 tx=0x7fc6080192c0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc61f200640 1 -- 172.21.15.83:0/32610767 shutdown_connections 2024-04-25T00:15:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc61f200640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc5f40678c0 0x7fc5f4069d80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc61f200640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6180cedd0 0x7fc618070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc61f200640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61815be20 0x7fc618159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc61f200640 1 --2- 172.21.15.83:0/32610767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6180acd20 0x7fc618141610 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc61f200640 1 -- 172.21.15.83:0/32610767 >> 172.21.15.83:0/32610767 conn(0x7fc6180b1b10 msgr2=0x7fc6180aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc61f200640 1 -- 172.21.15.83:0/32610767 shutdown_connections 2024-04-25T00:15:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.642+0000 7fc61f200640 1 -- 172.21.15.83:0/32610767 wait complete. 2024-04-25T00:15:16.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:15:16.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.798+0000 7fc57a600640 1 Processor -- start 2024-04-25T00:15:16.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.798+0000 7fc57a600640 1 -- start start 2024-04-25T00:15:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.798+0000 7fc57a600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc574155300 0x7fc5740cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.798+0000 7fc57a600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5741556e0 con 0x7fc5740d0180 2024-04-25T00:15:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.798+0000 7fc57a600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc574072260 con 0x7fc5740b3350 2024-04-25T00:15:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.798+0000 7fc57a600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc5740d05e0 con 0x7fc574155300 2024-04-25T00:15:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc56be00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc5740d0180 0x7fc5740b2c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44246/0 (socket says 172.21.15.83:44246) 2024-04-25T00:15:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc56be00640 1 -- 172.21.15.83:0/4185717622 learned_addr learned my addr 172.21.15.83:0/4185717622 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:16.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc573400640 1 --2- 172.21.15.83:0/4185717622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc574155300 0x7fc5740cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 -- 172.21.15.83:0/4185717622 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 673280976 0 0) 0x7fc5741556e0 con 0x7fc5740d0180 2024-04-25T00:15:16.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 -- 172.21.15.83:0/4185717622 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc558003530 con 0x7fc5740d0180 2024-04-25T00:15:16.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 -- 172.21.15.83:0/4185717622 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2667850538 0 0) 0x7fc558003530 con 0x7fc5740d0180 2024-04-25T00:15:16.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 -- 172.21.15.83:0/4185717622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc574155300 msgr2=0x7fc5740cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 --2- 172.21.15.83:0/4185717622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc574155300 0x7fc5740cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 -- 172.21.15.83:0/4185717622 >> v1:172.21.15.86:6789/0 conn(0x7fc5740b3350 legacy=0x7fc5740cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 -- 172.21.15.83:0/4185717622 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc574072540 con 0x7fc5740d0180 2024-04-25T00:15:16.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 -- 172.21.15.83:0/4185717622 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fc55c001680 con 0x7fc5740d0180 2024-04-25T00:15:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 -- 172.21.15.83:0/4185717622 >> v1:172.21.15.83:6789/0 conn(0x7fc5740d0180 legacy=0x7fc5740b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 --2- 172.21.15.83:0/4185717622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5580039f0 0x7fc558003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 --2- 172.21.15.83:0/4185717622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc558004810 0x7fc558004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 --2- 172.21.15.83:0/4185717622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc574155300 0x7fc558005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 -- 172.21.15.83:0/4185717622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc55800bad0 con 0x7fc5580039f0 2024-04-25T00:15:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 -- 172.21.15.83:0/4185717622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc55800bc80 con 0x7fc558004810 2024-04-25T00:15:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc572a00640 1 -- 172.21.15.83:0/4185717622 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc55800be80 con 0x7fc574155300 2024-04-25T00:15:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc573400640 1 --2- 172.21.15.83:0/4185717622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5580039f0 0x7fc558003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc56be00640 1 --2- 172.21.15.83:0/4185717622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc558004810 0x7fc558004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc573e00640 1 --2- 172.21.15.83:0/4185717622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc574155300 0x7fc558005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc573400640 1 -- 172.21.15.83:0/4185717622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc574155300 msgr2=0x7fc558005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc573400640 1 --2- 172.21.15.83:0/4185717622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc574155300 0x7fc558005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc573400640 1 -- 172.21.15.83:0/4185717622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc558004810 msgr2=0x7fc558004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc573400640 1 --2- 172.21.15.83:0/4185717622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc558004810 0x7fc558004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc573400640 1 -- 172.21.15.83:0/4185717622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc558003530 con 0x7fc5580039f0 2024-04-25T00:15:16.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.802+0000 7fc56be00640 1 --2- 172.21.15.83:0/4185717622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc558004810 0x7fc558004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:16.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc573400640 1 --2- 172.21.15.83:0/4185717622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5580039f0 0x7fc558003e10 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fc564009ac0 tx=0x7fc56401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba4316502be5acb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:16.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc572a00640 1 -- 172.21.15.83:0/4185717622 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5640b2070 con 0x7fc5580039f0 2024-04-25T00:15:16.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc572a00640 1 -- 172.21.15.83:0/4185717622 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5640a5640 con 0x7fc5580039f0 2024-04-25T00:15:16.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc572a00640 1 -- 172.21.15.83:0/4185717622 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564009690 con 0x7fc5580039f0 2024-04-25T00:15:16.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc57a600640 1 -- 172.21.15.83:0/4185717622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5580039f0 msgr2=0x7fc558003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc57a600640 1 --2- 172.21.15.83:0/4185717622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5580039f0 0x7fc558003e10 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fc564009ac0 tx=0x7fc56401ca50 comp rx=0 tx=0).stop 2024-04-25T00:15:16.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc573e00640 1 -- 172.21.15.83:0/4185717622 reap_dead start 2024-04-25T00:15:16.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc57a600640 1 -- 172.21.15.83:0/4185717622 shutdown_connections 2024-04-25T00:15:16.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc57a600640 1 -- 172.21.15.83:0/4185717622 >> 172.21.15.83:0/4185717622 conn(0x7fc5740b1b10 msgr2=0x7fc5740b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:16.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc57a600640 1 -- 172.21.15.83:0/4185717622 shutdown_connections 2024-04-25T00:15:16.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc57a600640 1 -- 172.21.15.83:0/4185717622 wait complete. 2024-04-25T00:15:16.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc57a600640 1 Processor -- start 2024-04-25T00:15:16.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc57a600640 1 -- start start 2024-04-25T00:15:16.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc57a600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5580039f0 0x7fc574148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc57a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574153830 0x7fc57406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc57a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57406e8f0 0x7fc5740aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc57a600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc574166c70 con 0x7fc57406e8f0 2024-04-25T00:15:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc57a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc574166df0 con 0x7fc574153830 2024-04-25T00:15:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.806+0000 7fc57a600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc574165aa0 con 0x7fc5580039f0 2024-04-25T00:15:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc573e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57406e8f0 0x7fc5740aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc573e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57406e8f0 0x7fc5740aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39680/0 (socket says 172.21.15.83:39680) 2024-04-25T00:15:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc573e00640 1 -- 172.21.15.83:0/2934143444 learned_addr learned my addr 172.21.15.83:0/2934143444 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc56be00640 1 --2- 172.21.15.83:0/2934143444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574153830 0x7fc57406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc573400640 1 --2- 172.21.15.83:0/2934143444 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5580039f0 0x7fc574148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc573e00640 1 -- 172.21.15.83:0/2934143444 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5580039f0 msgr2=0x7fc574148550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc573e00640 1 --2- 172.21.15.83:0/2934143444 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5580039f0 0x7fc574148550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc573e00640 1 -- 172.21.15.83:0/2934143444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574153830 msgr2=0x7fc57406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc573e00640 1 --2- 172.21.15.83:0/2934143444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574153830 0x7fc57406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc573e00640 1 -- 172.21.15.83:0/2934143444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57415df90 con 0x7fc57406e8f0 2024-04-25T00:15:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc573e00640 1 --2- 172.21.15.83:0/2934143444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57406e8f0 0x7fc5740aed90 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fc57415f940 tx=0x7fc56c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc56b400640 1 -- 172.21.15.83:0/2934143444 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c003980 con 0x7fc57406e8f0 2024-04-25T00:15:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc56b400640 1 -- 172.21.15.83:0/2934143444 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56c018070 con 0x7fc57406e8f0 2024-04-25T00:15:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc57a600640 1 -- 172.21.15.83:0/2934143444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc574164fc0 con 0x7fc57406e8f0 2024-04-25T00:15:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc56b400640 1 -- 172.21.15.83:0/2934143444 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c003d60 con 0x7fc57406e8f0 2024-04-25T00:15:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc57a600640 1 -- 172.21.15.83:0/2934143444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc574165580 con 0x7fc57406e8f0 2024-04-25T00:15:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.810+0000 7fc56b400640 1 -- 172.21.15.83:0/2934143444 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc56c0050d0 con 0x7fc57406e8f0 2024-04-25T00:15:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc56b400640 1 --2- 172.21.15.83:0/2934143444 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc5500678f0 0x7fc550069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc56b400640 1 -- 172.21.15.83:0/2934143444 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7fc56c085ba0 con 0x7fc57406e8f0 2024-04-25T00:15:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc573400640 1 --2- 172.21.15.83:0/2934143444 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc5500678f0 0x7fc550069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 Processor -- start 2024-04-25T00:15:16.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 -- start start 2024-04-25T00:15:16.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc573400640 1 --2- 172.21.15.83:0/2934143444 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc5500678f0 0x7fc550069db0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc5640af000 tx=0x7fc5640046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:16.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609cef0 0x56293609d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56293609d810 0x562936053680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562936053bc0 0x562936053fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562935ea15e0 con 0x562936053bc0 2024-04-25T00:15:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562935e28c50 con 0x56293609d810 2024-04-25T00:15:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x562935e9d5a0 con 0x56293609cef0 2024-04-25T00:15:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc573e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562936053bc0 0x562936053fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc573e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562936053bc0 0x562936053fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39686/0 (socket says 172.21.15.83:39686) 2024-04-25T00:15:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc573e00640 1 -- 172.21.15.83:0/1290923851 learned_addr learned my addr 172.21.15.83:0/1290923851 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc573400640 1 --2- 172.21.15.83:0/1290923851 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609cef0 0x56293609d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc56be00640 1 --2- 172.21.15.83:0/1290923851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56293609d810 0x562936053680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc573e00640 1 -- 172.21.15.83:0/1290923851 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609cef0 msgr2=0x56293609d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc573e00640 1 --2- 172.21.15.83:0/1290923851 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609cef0 0x56293609d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc573e00640 1 -- 172.21.15.83:0/1290923851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56293609d810 msgr2=0x562936053680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc573e00640 1 --2- 172.21.15.83:0/1290923851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56293609d810 0x562936053680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc573e00640 1 -- 172.21.15.83:0/1290923851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562935e26d10 con 0x562936053bc0 2024-04-25T00:15:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc573400640 1 --2- 172.21.15.83:0/1290923851 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609cef0 0x56293609d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc573e00640 1 --2- 172.21.15.83:0/1290923851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562936053bc0 0x562936053fa0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fc56c055970 tx=0x7fc56c0558b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9bd64fd9b899c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:16.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc563e00640 1 -- 172.21.15.83:0/1290923851 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c05a020 con 0x562936053bc0 2024-04-25T00:15:16.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc563e00640 1 -- 172.21.15.83:0/1290923851 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56c05e8e0 con 0x562936053bc0 2024-04-25T00:15:16.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc563e00640 1 -- 172.21.15.83:0/1290923851 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c053f70 con 0x562936053bc0 2024-04-25T00:15:16.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 -- 172.21.15.83:0/1290923851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562936053bc0 msgr2=0x562936053fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 --2- 172.21.15.83:0/1290923851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562936053bc0 0x562936053fa0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fc56c055970 tx=0x7fc56c0558b0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 -- 172.21.15.83:0/1290923851 shutdown_connections 2024-04-25T00:15:16.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 --2- 172.21.15.83:0/1290923851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562936053bc0 0x562936053fa0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 --2- 172.21.15.83:0/1290923851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56293609d810 0x562936053680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 --2- 172.21.15.83:0/1290923851 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609cef0 0x56293609d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 -- 172.21.15.83:0/1290923851 >> 172.21.15.83:0/1290923851 conn(0x56293604fb70 msgr2=0x562936051f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 -- 172.21.15.83:0/1290923851 shutdown_connections 2024-04-25T00:15:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.814+0000 7fc57da2b000 1 -- 172.21.15.83:0/1290923851 wait complete. 2024-04-25T00:15:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 1 -- start start 2024-04-25T00:15:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 1 Processor -- start 2024-04-25T00:15:16.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562936053bc0 0x562936081f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293609cef0 0x562936082460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609d810 0x5629360829a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562935e5b520 con 0x56293609cef0 2024-04-25T00:15:16.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562935ea08f0 con 0x562936053bc0 2024-04-25T00:15:16.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x56293604f9f0 con 0x56293609d810 2024-04-25T00:15:16.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 10 client.0 _ll_register_callbacks cb 0x56293603e5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:16.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc56be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293609cef0 0x562936082460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc56be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293609cef0 0x562936082460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39702/0 (socket says 172.21.15.83:39702) 2024-04-25T00:15:16.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc56be00640 1 -- 172.21.15.83:0/208933197 learned_addr learned my addr 172.21.15.83:0/208933197 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:16.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc573e00640 1 --2- 172.21.15.83:0/208933197 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609d810 0x5629360829a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc56be00640 1 -- 172.21.15.83:0/208933197 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609d810 msgr2=0x5629360829a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc56be00640 1 --2- 172.21.15.83:0/208933197 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609d810 0x5629360829a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc56be00640 1 -- 172.21.15.83:0/208933197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562936053bc0 msgr2=0x562936081f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc56be00640 1 --2- 172.21.15.83:0/208933197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562936053bc0 0x562936081f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc56be00640 1 -- 172.21.15.83:0/208933197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562935e397b0 con 0x56293609cef0 2024-04-25T00:15:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc56be00640 1 --2- 172.21.15.83:0/208933197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293609cef0 0x562936082460 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0135b0 tx=0x7fc55c013a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4dda99b95f7642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc560c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:15:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc560c00640 1 -- 172.21.15.83:0/208933197 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c004ac0 con 0x56293609cef0 2024-04-25T00:15:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc560c00640 1 -- 172.21.15.83:0/208933197 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7fc55c004c60 con 0x56293609cef0 2024-04-25T00:15:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 1 -- 172.21.15.83:0/208933197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc56c05e8e0 con 0x56293609cef0 2024-04-25T00:15:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc560c00640 1 -- 172.21.15.83:0/208933197 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fc55c023050 con 0x56293609cef0 2024-04-25T00:15:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 10 client.7848 fetch_fsmap learned FSMap version 73 2024-04-25T00:15:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 1 -- 172.21.15.83:0/208933197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x562935e28ea0 con 0x56293609cef0 2024-04-25T00:15:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc560c00640 1 -- 172.21.15.83:0/208933197 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56c05e8e0 con 0x56293609cef0 2024-04-25T00:15:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc560c00640 1 -- 172.21.15.83:0/208933197 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 73) ==== 3514+0+0 (secure 0 0 0) 0x7fc55c026020 con 0x56293609cef0 2024-04-25T00:15:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 10 client.7848 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T00:15:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 10 client.7848 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:15:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 20 client.7848 populate_metadata read hostname 'smithi083' 2024-04-25T00:15:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 1 --2- 172.21.15.83:0/208933197 >> [v2:172.21.15.83:6836/2870595352,v1:172.21.15.83:6837/2870595352] conn(0x5629360acf90 0x5629360af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 4 client.7848 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:15:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc57da2b000 1 -- 172.21.15.83:0/208933197 --> [v2:172.21.15.83:6836/2870595352,v1:172.21.15.83:6837/2870595352] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562935e3cab0 con 0x5629360acf90 2024-04-25T00:15:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.818+0000 7fc573400640 1 --2- 172.21.15.83:0/208933197 >> [v2:172.21.15.83:6836/2870595352,v1:172.21.15.83:6837/2870595352] conn(0x5629360acf90 0x5629360af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc573400640 1 --2- 172.21.15.83:0/208933197 >> [v2:172.21.15.83:6836/2870595352,v1:172.21.15.83:6837/2870595352] conn(0x5629360acf90 0x5629360af380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4dd5ddce70541fa1 server_cookie=a7d0b1bccd498f50 in_seq=0 out_seq=0 2024-04-25T00:15:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc560c00640 10 client.7848 ms_handle_connect on v2:172.21.15.83:6836/2870595352 2024-04-25T00:15:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc560c00640 1 -- 172.21.15.83:0/208933197 <== mds.1 v2:172.21.15.83:6836/2870595352 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562935e3cab0 con 0x5629360acf90 2024-04-25T00:15:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc560c00640 10 client.7848 handle_command_reply: tid=0 2024-04-25T00:15:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc57da2b000 1 client.7848 shutdown 2024-04-25T00:15:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc57da2b000 1 -- 172.21.15.83:0/208933197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293609cef0 msgr2=0x562936082460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc57da2b000 1 --2- 172.21.15.83:0/208933197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293609cef0 0x562936082460 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0135b0 tx=0x7fc55c013a70 comp rx=0 tx=0).stop 2024-04-25T00:15:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc57da2b000 1 -- 172.21.15.83:0/208933197 shutdown_connections 2024-04-25T00:15:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc57da2b000 1 --2- 172.21.15.83:0/208933197 >> [v2:172.21.15.83:6836/2870595352,v1:172.21.15.83:6837/2870595352] conn(0x5629360acf90 0x5629360af380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc57da2b000 1 --2- 172.21.15.83:0/208933197 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609d810 0x5629360829a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc57da2b000 1 --2- 172.21.15.83:0/208933197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293609cef0 0x562936082460 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc560c00640 0 client.7848 ms_handle_reset on v2:172.21.15.83:6836/2870595352 2024-04-25T00:15:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc57da2b000 1 --2- 172.21.15.83:0/208933197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562936053bc0 0x562936081f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc57da2b000 1 -- 172.21.15.83:0/208933197 >> 172.21.15.83:0/208933197 conn(0x56293604fb70 msgr2=0x5629360527f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc57da2b000 1 -- 172.21.15.83:0/208933197 shutdown_connections 2024-04-25T00:15:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc57da2b000 1 -- 172.21.15.83:0/208933197 wait complete. 2024-04-25T00:15:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.822+0000 7fc57da2b000 20 client.7848 trim_cache size 0 max 16384 2024-04-25T00:15:16.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.838+0000 7fc57da2b000 1 Processor -- start 2024-04-25T00:15:16.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.838+0000 7fc57da2b000 1 -- start start 2024-04-25T00:15:16.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.838+0000 7fc57da2b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562936053bc0 0x562936051f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.838+0000 7fc57da2b000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609cef0 0x5629360524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.838+0000 7fc57da2b000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56293609d810 0x562936052eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.838+0000 7fc57da2b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc574203fc0 con 0x562936053bc0 2024-04-25T00:15:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.838+0000 7fc57da2b000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc574204690 con 0x56293609d810 2024-04-25T00:15:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.838+0000 7fc57da2b000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc574202960 con 0x56293609cef0 2024-04-25T00:15:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.838+0000 7fc573400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562936053bc0 0x562936051f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.838+0000 7fc573400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562936053bc0 0x562936051f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39714/0 (socket says 172.21.15.83:39714) 2024-04-25T00:15:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.838+0000 7fc573400640 1 -- 172.21.15.83:0/2941209466 learned_addr learned my addr 172.21.15.83:0/2941209466 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc56be00640 1 --2- 172.21.15.83:0/2941209466 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609cef0 0x5629360524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc573e00640 1 --2- 172.21.15.83:0/2941209466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56293609d810 0x562936052eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc573400640 1 -- 172.21.15.83:0/2941209466 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609cef0 msgr2=0x5629360524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc573400640 1 --2- 172.21.15.83:0/2941209466 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609cef0 0x5629360524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc573400640 1 -- 172.21.15.83:0/2941209466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56293609d810 msgr2=0x562936052eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc573400640 1 --2- 172.21.15.83:0/2941209466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56293609d810 0x562936052eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc573400640 1 -- 172.21.15.83:0/2941209466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562935e29e80 con 0x562936053bc0 2024-04-25T00:15:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc573400640 1 --2- 172.21.15.83:0/2941209466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562936053bc0 0x562936051f60 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fc564009ac0 tx=0x7fc5640b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a57e118b5974950 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc563400640 1 -- 172.21.15.83:0/2941209466 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5640b2040 con 0x562936053bc0 2024-04-25T00:15:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc563400640 1 -- 172.21.15.83:0/2941209466 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5640cb070 con 0x562936053bc0 2024-04-25T00:15:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc563400640 1 -- 172.21.15.83:0/2941209466 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5640c7840 con 0x562936053bc0 2024-04-25T00:15:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 -- 172.21.15.83:0/2941209466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562936053bc0 msgr2=0x562936051f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 --2- 172.21.15.83:0/2941209466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562936053bc0 0x562936051f60 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fc564009ac0 tx=0x7fc5640b9040 comp rx=0 tx=0).stop 2024-04-25T00:15:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 -- 172.21.15.83:0/2941209466 shutdown_connections 2024-04-25T00:15:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 --2- 172.21.15.83:0/2941209466 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609cef0 0x5629360524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 --2- 172.21.15.83:0/2941209466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56293609d810 0x562936052eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 --2- 172.21.15.83:0/2941209466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562936053bc0 0x562936051f60 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 -- 172.21.15.83:0/2941209466 >> 172.21.15.83:0/2941209466 conn(0x56293604fb70 msgr2=0x5629360aa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:16.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 -- 172.21.15.83:0/2941209466 shutdown_connections 2024-04-25T00:15:16.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 -- 172.21.15.83:0/2941209466 wait complete. 2024-04-25T00:15:16.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 -- start start 2024-04-25T00:15:16.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 Processor -- start 2024-04-25T00:15:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562936053bc0 0x5629361159b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293609cef0 0x562936115ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609d810 0x562936111f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562935f53e80 con 0x56293609cef0 2024-04-25T00:15:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc574202fb0 con 0x562936053bc0 2024-04-25T00:15:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x562935e55c40 con 0x56293609d810 2024-04-25T00:15:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc57da2b000 10 client.0 _ll_register_callbacks cb 0x7fc55c005d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc56be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293609cef0 0x562936115ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc573400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562936053bc0 0x5629361159b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc56be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293609cef0 0x562936115ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39716/0 (socket says 172.21.15.83:39716) 2024-04-25T00:15:16.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc56be00640 1 -- 172.21.15.83:0/333222999 learned_addr learned my addr 172.21.15.83:0/333222999 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:16.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.842+0000 7fc573e00640 1 --2- 172.21.15.83:0/333222999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609d810 0x562936111f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc56be00640 1 -- 172.21.15.83:0/333222999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609d810 msgr2=0x562936111f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc56be00640 1 --2- 172.21.15.83:0/333222999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609d810 0x562936111f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc56be00640 1 -- 172.21.15.83:0/333222999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562936053bc0 msgr2=0x5629361159b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:16.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc56be00640 1 --2- 172.21.15.83:0/333222999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562936053bc0 0x5629361159b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc56be00640 1 -- 172.21.15.83:0/333222999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562935e6e760 con 0x56293609cef0 2024-04-25T00:15:16.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc56be00640 1 --2- 172.21.15.83:0/333222999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293609cef0 0x562936115ef0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0135b0 tx=0x7fc55c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ca91db6334908c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:16.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc560c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T00:15:16.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc560c00640 1 -- 172.21.15.83:0/333222999 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c004fa0 con 0x56293609cef0 2024-04-25T00:15:16.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc560c00640 1 -- 172.21.15.83:0/333222999 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7fc55c005140 con 0x56293609cef0 2024-04-25T00:15:16.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 -- 172.21.15.83:0/333222999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5640cb070 con 0x56293609cef0 2024-04-25T00:15:16.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc560c00640 1 -- 172.21.15.83:0/333222999 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fc55c038050 con 0x56293609cef0 2024-04-25T00:15:16.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 10 client.7854 fetch_fsmap learned FSMap version 73 2024-04-25T00:15:16.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 -- 172.21.15.83:0/333222999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x562935efa9e0 con 0x56293609cef0 2024-04-25T00:15:16.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc560c00640 1 -- 172.21.15.83:0/333222999 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5640cb070 con 0x56293609cef0 2024-04-25T00:15:16.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc560c00640 1 -- 172.21.15.83:0/333222999 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 73) ==== 3514+0+0 (secure 0 0 0) 0x7fc55c03b020 con 0x56293609cef0 2024-04-25T00:15:16.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 10 client.7854 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T00:15:16.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 10 client.7854 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:15:16.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 20 client.7854 populate_metadata read hostname 'smithi083' 2024-04-25T00:15:16.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 --2- 172.21.15.83:0/333222999 >> [v2:172.21.15.83:6836/2870595352,v1:172.21.15.83:6837/2870595352] conn(0x5629361124b0 0x562936112890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:16.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 4 client.7854 mds_command: new command op to 7728 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:15:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 -- 172.21.15.83:0/333222999 --> [v2:172.21.15.83:6836/2870595352,v1:172.21.15.83:6837/2870595352] -- command(tid 0: {"prefix": "status"}) -- 0x562935eaeb20 con 0x5629361124b0 2024-04-25T00:15:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc573400640 1 --2- 172.21.15.83:0/333222999 >> [v2:172.21.15.83:6836/2870595352,v1:172.21.15.83:6837/2870595352] conn(0x5629361124b0 0x562936112890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc573400640 1 --2- 172.21.15.83:0/333222999 >> [v2:172.21.15.83:6836/2870595352,v1:172.21.15.83:6837/2870595352] conn(0x5629361124b0 0x562936112890 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43610a7e226cce3 server_cookie=e84b6c23be272d2c in_seq=0 out_seq=0 2024-04-25T00:15:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc560c00640 10 client.7854 ms_handle_connect on v2:172.21.15.83:6836/2870595352 2024-04-25T00:15:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc560c00640 1 -- 172.21.15.83:0/333222999 <== mds.1 v2:172.21.15.83:6836/2870595352 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x562935eaeb20 con 0x5629361124b0 2024-04-25T00:15:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc560c00640 10 client.7854 handle_command_reply: tid=0 2024-04-25T00:15:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 client.7854 shutdown 2024-04-25T00:15:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 -- 172.21.15.83:0/333222999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293609cef0 msgr2=0x562936115ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 --2- 172.21.15.83:0/333222999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293609cef0 0x562936115ef0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0135b0 tx=0x7fc55c016000 comp rx=0 tx=0).stop 2024-04-25T00:15:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 -- 172.21.15.83:0/333222999 shutdown_connections 2024-04-25T00:15:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 --2- 172.21.15.83:0/333222999 >> [v2:172.21.15.83:6836/2870595352,v1:172.21.15.83:6837/2870595352] conn(0x5629361124b0 0x562936112890 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 --2- 172.21.15.83:0/333222999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56293609d810 0x562936111f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 --2- 172.21.15.83:0/333222999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293609cef0 0x562936115ef0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc560c00640 0 client.7854 ms_handle_reset on v2:172.21.15.83:6836/2870595352 2024-04-25T00:15:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 --2- 172.21.15.83:0/333222999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562936053bc0 0x5629361159b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 -- 172.21.15.83:0/333222999 >> 172.21.15.83:0/333222999 conn(0x56293604fb70 msgr2=0x56293609b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 -- 172.21.15.83:0/333222999 shutdown_connections 2024-04-25T00:15:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 1 -- 172.21.15.83:0/333222999 wait complete. 2024-04-25T00:15:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57da2b000 20 client.7854 trim_cache size 0 max 16384 2024-04-25T00:15:16.853 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:15:16.853 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "d58ae83a-ad72-4e18-824a-86e779979c35", 2024-04-25T00:15:16.853 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 1, 2024-04-25T00:15:16.853 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7728, 2024-04-25T00:15:16.854 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T00:15:16.854 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T00:15:16.854 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T00:15:16.854 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 2.957254125, 2024-04-25T00:15:16.854 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 73, 2024-04-25T00:15:16.854 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 97, 2024-04-25T00:15:16.854 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 97, 2024-04-25T00:15:16.854 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 10.829151660000001 2024-04-25T00:15:16.854 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:15:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57a600640 1 -- 172.21.15.83:0/2934143444 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc5500678f0 msgr2=0x7fc550069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57a600640 1 --2- 172.21.15.83:0/2934143444 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc5500678f0 0x7fc550069db0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc5640af000 tx=0x7fc5640046d0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57a600640 1 -- 172.21.15.83:0/2934143444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57406e8f0 msgr2=0x7fc5740aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57a600640 1 --2- 172.21.15.83:0/2934143444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57406e8f0 0x7fc5740aed90 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fc57415f940 tx=0x7fc56c00eb90 comp rx=0 tx=0).stop 2024-04-25T00:15:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57a600640 1 -- 172.21.15.83:0/2934143444 shutdown_connections 2024-04-25T00:15:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57a600640 1 --2- 172.21.15.83:0/2934143444 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc5500678f0 0x7fc550069db0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57a600640 1 --2- 172.21.15.83:0/2934143444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57406e8f0 0x7fc5740aed90 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57a600640 1 --2- 172.21.15.83:0/2934143444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574153830 0x7fc57406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57a600640 1 --2- 172.21.15.83:0/2934143444 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5580039f0 0x7fc574148550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57a600640 1 -- 172.21.15.83:0/2934143444 >> 172.21.15.83:0/2934143444 conn(0x7fc5740b1b10 msgr2=0x7fc5740b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57a600640 1 -- 172.21.15.83:0/2934143444 shutdown_connections 2024-04-25T00:15:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:16.846+0000 7fc57a600640 1 -- 172.21.15.83:0/2934143444 wait complete. 2024-04-25T00:15:16.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T00:15:17.031 INFO:teuthology.orchestra.run.smithi083.stdout:{"message":"","return_code":0} 2024-04-25T00:15:17.031 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T00:15:17.032 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump objecter 2024-04-25T00:15:17.188 INFO:teuthology.orchestra.run.smithi083.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":40,"op_send_bytes":11562,"op_resend":33,"op_reply":40,"op_latency":{"avgcount":40,"sum":0.259994071,"avgtime":0.006499851},"op_inflight":0,"oplen_avg":{"avgcount":40,"sum":62},"op":40,"op_r":0,"op_w":40,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":2,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1}} 2024-04-25T00:15:17.188 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 40, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 40, "avgtime": 0.006499851, "sum": 0.259994071 }, "op_pg": 0, "op_r": 0, "op_reply": 40, "op_resend": 33, "op_rmw": 0, "op_send": 40, "op_send_bytes": 11562, "op_w": 40, "oplen_avg": { "avgcount": 40, "sum": 62 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 2, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-04-25T00:15:17.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json export dir /bravo 1 2024-04-25T00:15:17.345 INFO:teuthology.orchestra.run.smithi083.stdout:{"return_code":0} 2024-04-25T00:15:17.345 DEBUG:tasks.cephfs.filesystem:_json_asok output { "return_code": 0 } 2024-04-25T00:15:17.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump objecter 2024-04-25T00:15:17.503 INFO:teuthology.orchestra.run.smithi083.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":40,"op_send_bytes":11562,"op_resend":33,"op_reply":40,"op_latency":{"avgcount":40,"sum":0.259994071,"avgtime":0.006499851},"op_inflight":0,"oplen_avg":{"avgcount":40,"sum":62},"op":40,"op_r":0,"op_w":40,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":2,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1}} 2024-04-25T00:15:17.503 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 40, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 40, "avgtime": 0.006499851, "sum": 0.259994071 }, "op_pg": 0, "op_r": 0, "op_reply": 40, "op_resend": 33, "op_rmw": 0, "op_send": 40, "op_send_bytes": 11562, "op_w": 40, "oplen_avg": { "avgcount": 40, "sum": 62 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 2, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-04-25T00:15:17.504 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-04-25T00:15:22.505 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump objecter 2024-04-25T00:15:22.674 INFO:teuthology.orchestra.run.smithi083.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":44,"op_send_bytes":15032,"op_resend":33,"op_reply":44,"op_latency":{"avgcount":44,"sum":0.271993919,"avgtime":0.006181679},"op_inflight":0,"oplen_avg":{"avgcount":44,"sum":68},"op":44,"op_r":0,"op_w":44,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":4,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":19,"omap_rd":0,"omap_del":1}} 2024-04-25T00:15:22.674 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 1, "omap_rd": 0, "omap_wr": 19, "op": 44, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 44, "avgtime": 0.006181679, "sum": 0.271993919 }, "op_pg": 0, "op_r": 0, "op_reply": 44, "op_resend": 33, "op_rmw": 0, "op_send": 44, "op_send_bytes": 15032, "op_w": 44, "oplen_avg": { "avgcount": 44, "sum": 68 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-04-25T00:15:22.675 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T00:15:22.675 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:15:22.675 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:22.684 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:15:22.684 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:22.733 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:15:22.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.914+0000 7efc00400640 1 Processor -- start 2024-04-25T00:15:22.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.914+0000 7efc00400640 1 -- start start 2024-04-25T00:15:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.914+0000 7efc00400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efbf80b2b60 0x7efbf80b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.914+0000 7efc00400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efbf8072720 0x7efbf8072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.914+0000 7efc00400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efbf8073040 con 0x7efbf815bd30 2024-04-25T00:15:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.914+0000 7efc00400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7efbf8071610 con 0x7efbf80b2b60 2024-04-25T00:15:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.914+0000 7efc00400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efbf815f400 con 0x7efbf8072720 2024-04-25T00:15:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbfd000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efbf80b2b60 0x7efbf80b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbfd000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efbf80b2b60 0x7efbf80b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39718/0 (socket says 172.21.15.83:39718) 2024-04-25T00:15:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbfd000640 1 -- 172.21.15.83:0/81635481 learned_addr learned my addr 172.21.15.83:0/81635481 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:22.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbfda00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efbf815bd30 0x7efbf8071aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44254/0 (socket says 172.21.15.83:44254) 2024-04-25T00:15:22.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbfe400640 1 --2- 172.21.15.83:0/81635481 >> v2:172.21.15.86:3300/0 conn(0x7efbf8072720 0x7efbf8072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:22.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbfd000640 1 -- 172.21.15.83:0/81635481 >> v2:172.21.15.86:3300/0 conn(0x7efbf8072720 msgr2=0x7efbf8072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:22.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbfd000640 1 --2- 172.21.15.83:0/81635481 >> v2:172.21.15.86:3300/0 conn(0x7efbf8072720 0x7efbf8072b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:22.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbfd000640 1 -- 172.21.15.83:0/81635481 >> v1:172.21.15.83:6789/0 conn(0x7efbf815bd30 legacy=0x7efbf8071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:22.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbfd000640 1 -- 172.21.15.83:0/81635481 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf8070fc0 con 0x7efbf80b2b60 2024-04-25T00:15:22.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbf7e00640 1 -- 172.21.15.83:0/81635481 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1328594480 0 0) 0x7efbf8073040 con 0x7efbf815bd30 2024-04-25T00:15:22.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbfe400640 1 --2- 172.21.15.83:0/81635481 >> v2:172.21.15.86:3300/0 conn(0x7efbf8072720 0x7efbf8072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:22.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbfd000640 1 --2- 172.21.15.83:0/81635481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf80b2b60 0x7efbf80b2ff0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7efbf80b4890 tx=0x7efbe401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993598c836af2c28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:22.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbf7e00640 1 -- 172.21.15.83:0/81635481 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe40ab430 con 0x7efbf80b2b60 2024-04-25T00:15:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbf7e00640 1 -- 172.21.15.83:0/81635481 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbe40b0070 con 0x7efbf80b2b60 2024-04-25T00:15:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.918+0000 7efbf7e00640 1 -- 172.21.15.83:0/81635481 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe40ab7f0 con 0x7efbf80b2b60 2024-04-25T00:15:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 -- 172.21.15.83:0/81635481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf80b2b60 msgr2=0x7efbf80b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 --2- 172.21.15.83:0/81635481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf80b2b60 0x7efbf80b2ff0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7efbf80b4890 tx=0x7efbe401c520 comp rx=0 tx=0).stop 2024-04-25T00:15:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 -- 172.21.15.83:0/81635481 shutdown_connections 2024-04-25T00:15:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 --2- 172.21.15.83:0/81635481 >> v2:172.21.15.86:3300/0 conn(0x7efbf8072720 0x7efbf8072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 --2- 172.21.15.83:0/81635481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf80b2b60 0x7efbf80b2ff0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 -- 172.21.15.83:0/81635481 >> 172.21.15.83:0/81635481 conn(0x7efbf80b1860 msgr2=0x7efbf80b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 -- 172.21.15.83:0/81635481 shutdown_connections 2024-04-25T00:15:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 -- 172.21.15.83:0/81635481 wait complete. 2024-04-25T00:15:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 Processor -- start 2024-04-25T00:15:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 -- start start 2024-04-25T00:15:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efbf8072720 0x7efbf81495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf80b2b60 0x7efbf814e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efbf815bd30 0x7efbf814c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efbf816aca0 con 0x7efbf80b2b60 2024-04-25T00:15:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efbf815f080 con 0x7efbf8072720 2024-04-25T00:15:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efc00400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efbf816ae20 con 0x7efbf815bd30 2024-04-25T00:15:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efbfd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf80b2b60 0x7efbf814e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efbfd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf80b2b60 0x7efbf814e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39726/0 (socket says 172.21.15.83:39726) 2024-04-25T00:15:22.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efbfd000640 1 -- 172.21.15.83:0/3416350727 learned_addr learned my addr 172.21.15.83:0/3416350727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:22.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efbfda00640 1 --2- 172.21.15.83:0/3416350727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efbf8072720 0x7efbf81495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:22.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efbfe400640 1 --2- 172.21.15.83:0/3416350727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efbf815bd30 0x7efbf814c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:22.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efbfd000640 1 -- 172.21.15.83:0/3416350727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efbf815bd30 msgr2=0x7efbf814c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:22.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efbfd000640 1 --2- 172.21.15.83:0/3416350727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efbf815bd30 0x7efbf814c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:22.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efbfd000640 1 -- 172.21.15.83:0/3416350727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efbf8072720 msgr2=0x7efbf81495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:22.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efbfd000640 1 --2- 172.21.15.83:0/3416350727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efbf8072720 0x7efbf81495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:22.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.922+0000 7efbfd000640 1 -- 172.21.15.83:0/3416350727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf8149b00 con 0x7efbf80b2b60 2024-04-25T00:15:22.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.926+0000 7efbfd000640 1 --2- 172.21.15.83:0/3416350727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf80b2b60 0x7efbf814e200 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7efbe4019130 tx=0x7efbe401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.926+0000 7efbf5600640 1 -- 172.21.15.83:0/3416350727 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe40b4800 con 0x7efbf80b2b60 2024-04-25T00:15:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.926+0000 7efbf5600640 1 -- 172.21.15.83:0/3416350727 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbe40b49a0 con 0x7efbf80b2b60 2024-04-25T00:15:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.926+0000 7efbf5600640 1 -- 172.21.15.83:0/3416350727 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe40b4d90 con 0x7efbf80b2b60 2024-04-25T00:15:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.926+0000 7efc00400640 1 -- 172.21.15.83:0/3416350727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf8149de0 con 0x7efbf80b2b60 2024-04-25T00:15:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.926+0000 7efc00400640 1 -- 172.21.15.83:0/3416350727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf814a3a0 con 0x7efbf80b2b60 2024-04-25T00:15:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.926+0000 7efc00400640 1 -- 172.21.15.83:0/3416350727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbf804f510 con 0x7efbf80b2b60 2024-04-25T00:15:22.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.926+0000 7efbf5600640 1 -- 172.21.15.83:0/3416350727 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efbe40b3380 con 0x7efbf80b2b60 2024-04-25T00:15:22.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.930+0000 7efbf5600640 1 --2- 172.21.15.83:0/3416350727 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efbd80678c0 0x7efbd8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:22.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.930+0000 7efbf5600640 1 -- 172.21.15.83:0/3416350727 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7efbe40a8070 con 0x7efbf80b2b60 2024-04-25T00:15:22.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.930+0000 7efbfda00640 1 --2- 172.21.15.83:0/3416350727 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efbd80678c0 0x7efbd8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:22.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.930+0000 7efbfda00640 1 --2- 172.21.15.83:0/3416350727 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efbd80678c0 0x7efbd8069d80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7efbec000f70 tx=0x7efbec007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:22.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:22.934+0000 7efbf5600640 1 -- 172.21.15.83:0/3416350727 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efbe4016110 con 0x7efbf80b2b60 2024-04-25T00:15:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.090+0000 7efc00400640 1 -- 172.21.15.83:0/3416350727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efbf8074b80 con 0x7efbf80b2b60 2024-04-25T00:15:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.094+0000 7efbf5600640 1 -- 172.21.15.83:0/3416350727 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+17048 (secure 0 0 0) 0x7efbe40f0440 con 0x7efbf80b2b60 2024-04-25T00:15:23.094 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:23.094 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":97,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:15:04.852008+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:50.260735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:50.880394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1609715757":"2024-04-26T00:15:04.845080+0000","172.21.15.83:6836/1609715757":"2024-04-26T00:15:04.845080+0000","172.21.15.86:6835/3612679148":"2024-04-26T00:15:04.826082+0000","172.21.15.86:6833/3612679148":"2024-04-26T00:15:04.826082+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:15:23.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.094+0000 7efc00400640 1 -- 172.21.15.83:0/3416350727 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efbd80678c0 msgr2=0x7efbd8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.094+0000 7efc00400640 1 --2- 172.21.15.83:0/3416350727 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efbd80678c0 0x7efbd8069d80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7efbec000f70 tx=0x7efbec007df0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.094+0000 7efc00400640 1 -- 172.21.15.83:0/3416350727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf80b2b60 msgr2=0x7efbf814e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.094+0000 7efc00400640 1 --2- 172.21.15.83:0/3416350727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf80b2b60 0x7efbf814e200 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7efbe4019130 tx=0x7efbe401ad60 comp rx=0 tx=0).stop 2024-04-25T00:15:23.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.094+0000 7efc00400640 1 -- 172.21.15.83:0/3416350727 shutdown_connections 2024-04-25T00:15:23.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.094+0000 7efc00400640 1 --2- 172.21.15.83:0/3416350727 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efbd80678c0 0x7efbd8069d80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.094+0000 7efc00400640 1 --2- 172.21.15.83:0/3416350727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efbf815bd30 0x7efbf814c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.094+0000 7efc00400640 1 --2- 172.21.15.83:0/3416350727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf80b2b60 0x7efbf814e200 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.094+0000 7efc00400640 1 --2- 172.21.15.83:0/3416350727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efbf8072720 0x7efbf81495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.094+0000 7efc00400640 1 -- 172.21.15.83:0/3416350727 >> 172.21.15.83:0/3416350727 conn(0x7efbf80b1860 msgr2=0x7efbf81610f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.094+0000 7efc00400640 1 -- 172.21.15.83:0/3416350727 shutdown_connections 2024-04-25T00:15:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.094+0000 7efc00400640 1 -- 172.21.15.83:0/3416350727 wait complete. 2024-04-25T00:15:23.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:14:50.260735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:50.880394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:15:23.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:15:23.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.254+0000 7f4644000640 1 Processor -- start 2024-04-25T00:15:23.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.254+0000 7f4644000640 1 -- start start 2024-04-25T00:15:23.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.254+0000 7f4644000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f463c06d5d0 0x7f463c15f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.254+0000 7f4644000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f463c15c200 con 0x7f463c0cedd0 2024-04-25T00:15:23.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.254+0000 7f4644000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f463c160100 con 0x7f463c15be20 2024-04-25T00:15:23.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.254+0000 7f4644000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f463c15fed0 con 0x7f463c06d5d0 2024-04-25T00:15:23.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.258+0000 7f4642000640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f463c0cedd0 0x7f463c0b3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44270/0 (socket says 172.21.15.83:44270) 2024-04-25T00:15:23.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.258+0000 7f4642000640 1 -- 172.21.15.83:0/638254827 learned_addr learned my addr 172.21.15.83:0/638254827 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.258+0000 7f4640c00640 1 --2- 172.21.15.83:0/638254827 >> v2:172.21.15.86:3300/0 conn(0x7f463c06d5d0 0x7f463c15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.258+0000 7f463be00640 1 -- 172.21.15.83:0/638254827 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3927146381 0 0) 0x7f463c15c200 con 0x7f463c0cedd0 2024-04-25T00:15:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.258+0000 7f463be00640 1 -- 172.21.15.83:0/638254827 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4624003530 con 0x7f463c0cedd0 2024-04-25T00:15:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.258+0000 7f463be00640 1 -- 172.21.15.83:0/638254827 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1346984990 0 0) 0x7f4624003530 con 0x7f463c0cedd0 2024-04-25T00:15:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.258+0000 7f4640c00640 1 -- 172.21.15.83:0/638254827 >> v1:172.21.15.86:6789/0 conn(0x7f463c15be20 legacy=0x7f463c06cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.258+0000 7f4640c00640 1 -- 172.21.15.83:0/638254827 >> v1:172.21.15.83:6789/0 conn(0x7f463c0cedd0 legacy=0x7f463c0b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.258+0000 7f4640c00640 1 -- 172.21.15.83:0/638254827 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c1603e0 con 0x7f463c06d5d0 2024-04-25T00:15:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.258+0000 7f4640c00640 1 --2- 172.21.15.83:0/638254827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f463c06d5d0 0x7f463c15f990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f46280097c0 tx=0x7f462801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62a10eb4ed164b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f463be00640 1 -- 172.21.15.83:0/638254827 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46280b0070 con 0x7f463c06d5d0 2024-04-25T00:15:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f463be00640 1 -- 172.21.15.83:0/638254827 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46280ab400 con 0x7f463c06d5d0 2024-04-25T00:15:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f463be00640 1 -- 172.21.15.83:0/638254827 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634001a00 con 0x7f463c06d5d0 2024-04-25T00:15:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 -- 172.21.15.83:0/638254827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f463c06d5d0 msgr2=0x7f463c15f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 --2- 172.21.15.83:0/638254827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f463c06d5d0 0x7f463c15f990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f46280097c0 tx=0x7f462801c3c0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 -- 172.21.15.83:0/638254827 shutdown_connections 2024-04-25T00:15:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 --2- 172.21.15.83:0/638254827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f463c06d5d0 0x7f463c15f990 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 -- 172.21.15.83:0/638254827 >> 172.21.15.83:0/638254827 conn(0x7f463c0b1b10 msgr2=0x7f463c0b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 -- 172.21.15.83:0/638254827 shutdown_connections 2024-04-25T00:15:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 -- 172.21.15.83:0/638254827 wait complete. 2024-04-25T00:15:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 Processor -- start 2024-04-25T00:15:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 -- start start 2024-04-25T00:15:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f463c06d5d0 0x7f463c1492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463c0cedd0 0x7f463c160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f463c15be20 0x7f463c161020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f463c169f50 con 0x7f463c0cedd0 2024-04-25T00:15:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f463c0b2e70 con 0x7f463c15be20 2024-04-25T00:15:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4644000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f463c16a0d0 con 0x7f463c06d5d0 2024-04-25T00:15:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4640c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463c0cedd0 0x7f463c160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4640c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463c0cedd0 0x7f463c160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39736/0 (socket says 172.21.15.83:39736) 2024-04-25T00:15:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4640c00640 1 -- 172.21.15.83:0/3573924458 learned_addr learned my addr 172.21.15.83:0/3573924458 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4642000640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f463c15be20 0x7f463c161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4641600640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f463c06d5d0 0x7f463c1492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4640c00640 1 -- 172.21.15.83:0/3573924458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f463c06d5d0 msgr2=0x7f463c1492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4640c00640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f463c06d5d0 0x7f463c1492d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4640c00640 1 -- 172.21.15.83:0/3573924458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f463c15be20 msgr2=0x7f463c161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4640c00640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f463c15be20 0x7f463c161020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4640c00640 1 -- 172.21.15.83:0/3573924458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c09b4a0 con 0x7f463c0cedd0 2024-04-25T00:15:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.262+0000 7f4642000640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f463c15be20 0x7f463c161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.266+0000 7f4640c00640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463c0cedd0 0x7f463c160ae0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f46280097c0 tx=0x7f4628019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.266+0000 7f4639600640 1 -- 172.21.15.83:0/3573924458 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46280ba020 con 0x7f463c0cedd0 2024-04-25T00:15:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.266+0000 7f4639600640 1 -- 172.21.15.83:0/3573924458 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4628004220 con 0x7f463c0cedd0 2024-04-25T00:15:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.266+0000 7f4639600640 1 -- 172.21.15.83:0/3573924458 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46280b0040 con 0x7f463c0cedd0 2024-04-25T00:15:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.266+0000 7f4644000640 1 -- 172.21.15.83:0/3573924458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c09b780 con 0x7f463c0cedd0 2024-04-25T00:15:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.266+0000 7f4644000640 1 -- 172.21.15.83:0/3573924458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c09bd40 con 0x7f463c0cedd0 2024-04-25T00:15:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.266+0000 7f4644000640 1 -- 172.21.15.83:0/3573924458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463c04f510 con 0x7f463c0cedd0 2024-04-25T00:15:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.266+0000 7f4639600640 1 -- 172.21.15.83:0/3573924458 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f46280bd070 con 0x7f463c0cedd0 2024-04-25T00:15:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.266+0000 7f4639600640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f461c0678c0 0x7f461c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.266+0000 7f4639600640 1 -- 172.21.15.83:0/3573924458 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f462811ef20 con 0x7f463c0cedd0 2024-04-25T00:15:23.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.266+0000 7f4641600640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f461c0678c0 0x7f461c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.270+0000 7f4641600640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f461c0678c0 0x7f461c069d80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4630000f70 tx=0x7f4630009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:23.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.270+0000 7f4639600640 1 -- 172.21.15.83:0/3573924458 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4628015fb0 con 0x7f463c0cedd0 2024-04-25T00:15:23.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.430+0000 7f4644000640 1 -- 172.21.15.83:0/3573924458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f463c09d830 con 0x7f463c0cedd0 2024-04-25T00:15:23.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.430+0000 7f4639600640 1 -- 172.21.15.83:0/3573924458 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f46280b22f0 con 0x7f463c0cedd0 2024-04-25T00:15:23.432 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:15:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.434+0000 7f4644000640 1 -- 172.21.15.83:0/3573924458 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f461c0678c0 msgr2=0x7f461c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.434+0000 7f4644000640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f461c0678c0 0x7f461c069d80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4630000f70 tx=0x7f4630009040 comp rx=0 tx=0).stop 2024-04-25T00:15:23.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.434+0000 7f4644000640 1 -- 172.21.15.83:0/3573924458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463c0cedd0 msgr2=0x7f463c160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.434+0000 7f4644000640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463c0cedd0 0x7f463c160ae0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f46280097c0 tx=0x7f4628019160 comp rx=0 tx=0).stop 2024-04-25T00:15:23.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.434+0000 7f4644000640 1 -- 172.21.15.83:0/3573924458 shutdown_connections 2024-04-25T00:15:23.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.434+0000 7f4644000640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f461c0678c0 0x7f461c069d80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.434+0000 7f4644000640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f463c15be20 0x7f463c161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.434+0000 7f4644000640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463c0cedd0 0x7f463c160ae0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.434+0000 7f4644000640 1 --2- 172.21.15.83:0/3573924458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f463c06d5d0 0x7f463c1492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.434+0000 7f4644000640 1 -- 172.21.15.83:0/3573924458 >> 172.21.15.83:0/3573924458 conn(0x7f463c0b1b10 msgr2=0x7f463c15c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:23.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.434+0000 7f4644000640 1 -- 172.21.15.83:0/3573924458 shutdown_connections 2024-04-25T00:15:23.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.434+0000 7f4644000640 1 -- 172.21.15.83:0/3573924458 wait complete. 2024-04-25T00:15:23.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:15:23.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.590+0000 7fcd1b800640 1 Processor -- start 2024-04-25T00:15:23.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd1b800640 1 -- start start 2024-04-25T00:15:23.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd1b800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcd1415be20 0x7fcd1415f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd1b800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd140cedd0 0x7fcd1406cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd1b800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd1415c200 con 0x7fcd140acd20 2024-04-25T00:15:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd1b800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fcd1415fed0 con 0x7fcd140cedd0 2024-04-25T00:15:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd1b800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcd140b2e70 con 0x7fcd1415be20 2024-04-25T00:15:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd13e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd140cedd0 0x7fcd1406cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd19800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcd140acd20 0x7fcd140b3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44286/0 (socket says 172.21.15.83:44286) 2024-04-25T00:15:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd19800640 1 -- 172.21.15.83:0/1402199977 learned_addr learned my addr 172.21.15.83:0/1402199977 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd13e00640 1 -- 172.21.15.83:0/1402199977 >> v2:172.21.15.86:3300/0 conn(0x7fcd1415be20 msgr2=0x7fcd1415f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd18e00640 1 --2- 172.21.15.83:0/1402199977 >> v2:172.21.15.86:3300/0 conn(0x7fcd1415be20 0x7fcd1415f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd13e00640 1 --2- 172.21.15.83:0/1402199977 >> v2:172.21.15.86:3300/0 conn(0x7fcd1415be20 0x7fcd1415f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd13e00640 1 -- 172.21.15.83:0/1402199977 >> v1:172.21.15.83:6789/0 conn(0x7fcd140acd20 legacy=0x7fcd140b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd13e00640 1 -- 172.21.15.83:0/1402199977 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd14160100 con 0x7fcd140cedd0 2024-04-25T00:15:23.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd13400640 1 -- 172.21.15.83:0/1402199977 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1751385861 0 0) 0x7fcd1415c200 con 0x7fcd140acd20 2024-04-25T00:15:23.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.594+0000 7fcd18e00640 1 --2- 172.21.15.83:0/1402199977 >> v2:172.21.15.86:3300/0 conn(0x7fcd1415be20 0x7fcd1415f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:23.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.598+0000 7fcd13e00640 1 --2- 172.21.15.83:0/1402199977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd140cedd0 0x7fcd1406cec0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fcd140731f0 tx=0x7fcd0001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a870e062759043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:23.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.598+0000 7fcd13400640 1 -- 172.21.15.83:0/1402199977 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd000b0070 con 0x7fcd140cedd0 2024-04-25T00:15:23.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.598+0000 7fcd13400640 1 -- 172.21.15.83:0/1402199977 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd000ab600 con 0x7fcd140cedd0 2024-04-25T00:15:23.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.598+0000 7fcd13400640 1 -- 172.21.15.83:0/1402199977 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd000ab9d0 con 0x7fcd140cedd0 2024-04-25T00:15:23.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.598+0000 7fcd1b800640 1 -- 172.21.15.83:0/1402199977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd140cedd0 msgr2=0x7fcd1406cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.598+0000 7fcd1b800640 1 --2- 172.21.15.83:0/1402199977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd140cedd0 0x7fcd1406cec0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fcd140731f0 tx=0x7fcd0001c480 comp rx=0 tx=0).stop 2024-04-25T00:15:23.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.598+0000 7fcd1b800640 1 -- 172.21.15.83:0/1402199977 shutdown_connections 2024-04-25T00:15:23.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.598+0000 7fcd1b800640 1 --2- 172.21.15.83:0/1402199977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd140cedd0 0x7fcd1406cec0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.598+0000 7fcd1b800640 1 --2- 172.21.15.83:0/1402199977 >> v2:172.21.15.86:3300/0 conn(0x7fcd1415be20 0x7fcd1415f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.598+0000 7fcd1b800640 1 -- 172.21.15.83:0/1402199977 >> 172.21.15.83:0/1402199977 conn(0x7fcd140b1b10 msgr2=0x7fcd140b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:23.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.598+0000 7fcd1b800640 1 -- 172.21.15.83:0/1402199977 shutdown_connections 2024-04-25T00:15:23.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.598+0000 7fcd1b800640 1 -- 172.21.15.83:0/1402199977 wait complete. 2024-04-25T00:15:23.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.598+0000 7fcd1b800640 1 Processor -- start 2024-04-25T00:15:23.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.598+0000 7fcd1b800640 1 -- start start 2024-04-25T00:15:23.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd1b800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd140acd20 0x7fcd14144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd1b800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd140cedd0 0x7fcd1414f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd1b800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1415be20 0x7fcd1414f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd1b800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd14168f40 con 0x7fcd140acd20 2024-04-25T00:15:23.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd1b800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1406d810 con 0x7fcd1415be20 2024-04-25T00:15:23.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd1b800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcd141690c0 con 0x7fcd140cedd0 2024-04-25T00:15:23.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd18e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd140acd20 0x7fcd14144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd18e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd140acd20 0x7fcd14144fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39740/0 (socket says 172.21.15.83:39740) 2024-04-25T00:15:23.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd18e00640 1 -- 172.21.15.83:0/147904402 learned_addr learned my addr 172.21.15.83:0/147904402 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:23.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd19800640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1415be20 0x7fcd1414f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd18e00640 1 -- 172.21.15.83:0/147904402 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd140cedd0 msgr2=0x7fcd1414f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd13e00640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd140cedd0 0x7fcd1414f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd18e00640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd140cedd0 0x7fcd1414f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd18e00640 1 -- 172.21.15.83:0/147904402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1415be20 msgr2=0x7fcd1414f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd18e00640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1415be20 0x7fcd1414f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd18e00640 1 -- 172.21.15.83:0/147904402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd14145a90 con 0x7fcd140acd20 2024-04-25T00:15:23.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd19800640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1415be20 0x7fcd1414f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:23.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd13e00640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd140cedd0 0x7fcd1414f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:23.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd18e00640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd140acd20 0x7fcd14144fd0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400ce30 tx=0x7fcd0400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:23.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd10c00640 1 -- 172.21.15.83:0/147904402 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0400f890 con 0x7fcd140acd20 2024-04-25T00:15:23.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd10c00640 1 -- 172.21.15.83:0/147904402 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd0400fa30 con 0x7fcd140acd20 2024-04-25T00:15:23.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd10c00640 1 -- 172.21.15.83:0/147904402 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0400fe00 con 0x7fcd140acd20 2024-04-25T00:15:23.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd1b800640 1 -- 172.21.15.83:0/147904402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1414fd30 con 0x7fcd140acd20 2024-04-25T00:15:23.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd1b800640 1 -- 172.21.15.83:0/147904402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd141502f0 con 0x7fcd140acd20 2024-04-25T00:15:23.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd1b800640 1 -- 172.21.15.83:0/147904402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1404f510 con 0x7fcd140acd20 2024-04-25T00:15:23.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.602+0000 7fcd10c00640 1 -- 172.21.15.83:0/147904402 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcd04018400 con 0x7fcd140acd20 2024-04-25T00:15:23.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.606+0000 7fcd10c00640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccf40678c0 0x7fccf4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.606+0000 7fcd10c00640 1 -- 172.21.15.83:0/147904402 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7fcd04086cf0 con 0x7fcd140acd20 2024-04-25T00:15:23.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.606+0000 7fcd13e00640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccf40678c0 0x7fccf4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.606+0000 7fcd13e00640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccf40678c0 0x7fccf4069d80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcd000ad000 tx=0x7fcd00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:23.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.610+0000 7fcd10c00640 1 -- 172.21.15.83:0/147904402 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcd04014030 con 0x7fcd140acd20 2024-04-25T00:15:23.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.766+0000 7fcd1b800640 1 -- 172.21.15.83:0/147904402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcd1415eae0 con 0x7fcd140acd20 2024-04-25T00:15:23.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.766+0000 7fcd10c00640 1 -- 172.21.15.83:0/147904402 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7fcd0400ae20 con 0x7fcd140acd20 2024-04-25T00:15:23.766 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:15:23.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.766+0000 7fcd1b800640 1 -- 172.21.15.83:0/147904402 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccf40678c0 msgr2=0x7fccf4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.766+0000 7fcd1b800640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccf40678c0 0x7fccf4069d80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcd000ad000 tx=0x7fcd00008040 comp rx=0 tx=0).stop 2024-04-25T00:15:23.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.766+0000 7fcd1b800640 1 -- 172.21.15.83:0/147904402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd140acd20 msgr2=0x7fcd14144fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.766+0000 7fcd1b800640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd140acd20 0x7fcd14144fd0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400ce30 tx=0x7fcd0400a650 comp rx=0 tx=0).stop 2024-04-25T00:15:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.766+0000 7fcd1b800640 1 -- 172.21.15.83:0/147904402 shutdown_connections 2024-04-25T00:15:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.766+0000 7fcd1b800640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fccf40678c0 0x7fccf4069d80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.766+0000 7fcd1b800640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd140cedd0 0x7fcd1414f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.766+0000 7fcd1b800640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1415be20 0x7fcd1414f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.766+0000 7fcd1b800640 1 --2- 172.21.15.83:0/147904402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd140acd20 0x7fcd14144fd0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.766+0000 7fcd1b800640 1 -- 172.21.15.83:0/147904402 >> 172.21.15.83:0/147904402 conn(0x7fcd140b1b10 msgr2=0x7fcd141511d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.766+0000 7fcd1b800640 1 -- 172.21.15.83:0/147904402 shutdown_connections 2024-04-25T00:15:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.766+0000 7fcd1b800640 1 -- 172.21.15.83:0/147904402 wait complete. 2024-04-25T00:15:23.787 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:15:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdcea00640 1 Processor -- start 2024-04-25T00:15:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdcea00640 1 -- start start 2024-04-25T00:15:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdcea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efdc815be20 0x7efdc815f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdcea00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc806d5d0 0x7efdc80b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdcea00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdc815c200 con 0x7efdc80cedd0 2024-04-25T00:15:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdcea00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efdc815fed0 con 0x7efdc815be20 2024-04-25T00:15:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdcea00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efdc80b2e70 con 0x7efdc806d5d0 2024-04-25T00:15:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdbf400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efdc80cedd0 0x7efdc806cec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44294/0 (socket says 172.21.15.83:44294) 2024-04-25T00:15:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdbf400640 1 -- 172.21.15.83:0/16065362 learned_addr learned my addr 172.21.15.83:0/16065362 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdcca00640 1 --2- 172.21.15.83:0/16065362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc806d5d0 0x7efdc80b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdc7e00640 1 --2- 172.21.15.83:0/16065362 >> v2:172.21.15.86:3300/0 conn(0x7efdc815be20 0x7efdc815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdcca00640 1 -- 172.21.15.83:0/16065362 >> v2:172.21.15.86:3300/0 conn(0x7efdc815be20 msgr2=0x7efdc815f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdcca00640 1 --2- 172.21.15.83:0/16065362 >> v2:172.21.15.86:3300/0 conn(0x7efdc815be20 0x7efdc815f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdcca00640 1 -- 172.21.15.83:0/16065362 >> v1:172.21.15.83:6789/0 conn(0x7efdc80cedd0 legacy=0x7efdc806cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdcca00640 1 -- 172.21.15.83:0/16065362 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc8160100 con 0x7efdc806d5d0 2024-04-25T00:15:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.926+0000 7efdc7e00640 1 --2- 172.21.15.83:0/16065362 >> v2:172.21.15.86:3300/0 conn(0x7efdc815be20 0x7efdc815f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:23.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.930+0000 7efdcca00640 1 --2- 172.21.15.83:0/16065362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc806d5d0 0x7efdc80b3300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009d90 tx=0x7efdc001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5736d418c4637a49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:23.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.930+0000 7efdc7400640 1 -- 172.21.15.83:0/16065362 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc00b2070 con 0x7efdc806d5d0 2024-04-25T00:15:23.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.930+0000 7efdc7400640 1 -- 172.21.15.83:0/16065362 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc00a5830 con 0x7efdc806d5d0 2024-04-25T00:15:23.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.930+0000 7efdc7400640 1 -- 172.21.15.83:0/16065362 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc00a5c00 con 0x7efdc806d5d0 2024-04-25T00:15:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.930+0000 7efdcea00640 1 -- 172.21.15.83:0/16065362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc806d5d0 msgr2=0x7efdc80b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.930+0000 7efdcea00640 1 --2- 172.21.15.83:0/16065362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc806d5d0 0x7efdc80b3300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009d90 tx=0x7efdc001c970 comp rx=0 tx=0).stop 2024-04-25T00:15:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.930+0000 7efdcea00640 1 -- 172.21.15.83:0/16065362 shutdown_connections 2024-04-25T00:15:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.930+0000 7efdcea00640 1 --2- 172.21.15.83:0/16065362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc806d5d0 0x7efdc80b3300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.930+0000 7efdcea00640 1 --2- 172.21.15.83:0/16065362 >> v2:172.21.15.86:3300/0 conn(0x7efdc815be20 0x7efdc815f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.930+0000 7efdcea00640 1 -- 172.21.15.83:0/16065362 >> 172.21.15.83:0/16065362 conn(0x7efdc80b1b10 msgr2=0x7efdc80b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.930+0000 7efdcea00640 1 -- 172.21.15.83:0/16065362 shutdown_connections 2024-04-25T00:15:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.930+0000 7efdcea00640 1 -- 172.21.15.83:0/16065362 wait complete. 2024-04-25T00:15:23.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcea00640 1 Processor -- start 2024-04-25T00:15:23.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcea00640 1 -- start start 2024-04-25T00:15:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcea00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc806d5d0 0x7efdc814e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc80cedd0 0x7efdc815dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc815be20 0x7efdc813f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efdc81690c0 con 0x7efdc815be20 2024-04-25T00:15:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efdc8168f40 con 0x7efdc80cedd0 2024-04-25T00:15:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcea00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efdc8169240 con 0x7efdc806d5d0 2024-04-25T00:15:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc815be20 0x7efdc813f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdc7e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc806d5d0 0x7efdc814e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdbf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc80cedd0 0x7efdc815dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc815be20 0x7efdc813f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39742/0 (socket says 172.21.15.83:39742) 2024-04-25T00:15:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcca00640 1 -- 172.21.15.83:0/3261982992 learned_addr learned my addr 172.21.15.83:0/3261982992 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcca00640 1 -- 172.21.15.83:0/3261982992 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc806d5d0 msgr2=0x7efdc814e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcca00640 1 --2- 172.21.15.83:0/3261982992 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc806d5d0 0x7efdc814e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcca00640 1 -- 172.21.15.83:0/3261982992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc80cedd0 msgr2=0x7efdc815dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcca00640 1 --2- 172.21.15.83:0/3261982992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc80cedd0 0x7efdc815dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcca00640 1 -- 172.21.15.83:0/3261982992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc814e750 con 0x7efdc815be20 2024-04-25T00:15:23.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcca00640 1 --2- 172.21.15.83:0/3261982992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc815be20 0x7efdc813f580 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7efdc00b0040 tx=0x7efdc0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:23.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdc4c00640 1 -- 172.21.15.83:0/3261982992 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc00b2050 con 0x7efdc815be20 2024-04-25T00:15:23.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdc4c00640 1 -- 172.21.15.83:0/3261982992 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc00be070 con 0x7efdc815be20 2024-04-25T00:15:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcea00640 1 -- 172.21.15.83:0/3261982992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc809ab90 con 0x7efdc815be20 2024-04-25T00:15:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdc4c00640 1 -- 172.21.15.83:0/3261982992 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc00ad870 con 0x7efdc815be20 2024-04-25T00:15:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.934+0000 7efdcea00640 1 -- 172.21.15.83:0/3261982992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc809b150 con 0x7efdc815be20 2024-04-25T00:15:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.938+0000 7efdcea00640 1 -- 172.21.15.83:0/3261982992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc804f510 con 0x7efdc815be20 2024-04-25T00:15:23.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.938+0000 7efdc4c00640 1 -- 172.21.15.83:0/3261982992 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efdc00ada10 con 0x7efdc815be20 2024-04-25T00:15:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.938+0000 7efdc4c00640 1 --2- 172.21.15.83:0/3261982992 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efda40678c0 0x7efda4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.938+0000 7efdc4c00640 1 -- 172.21.15.83:0/3261982992 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7efdc0121160 con 0x7efdc815be20 2024-04-25T00:15:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.938+0000 7efdc7e00640 1 --2- 172.21.15.83:0/3261982992 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efda40678c0 0x7efda4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.938+0000 7efdc7e00640 1 --2- 172.21.15.83:0/3261982992 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efda40678c0 0x7efda4069d80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009580 tx=0x7efdb8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:23.942+0000 7efdc4c00640 1 -- 172.21.15.83:0/3261982992 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efdc0016560 con 0x7efdc815be20 2024-04-25T00:15:24.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.094+0000 7efdcea00640 1 -- 172.21.15.83:0/3261982992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efdc806ea90 con 0x7efdc815be20 2024-04-25T00:15:24.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.098+0000 7efdc4c00640 1 -- 172.21.15.83:0/3261982992 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7efdc00add00 con 0x7efdc815be20 2024-04-25T00:15:24.098 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:15:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.098+0000 7efdcea00640 1 -- 172.21.15.83:0/3261982992 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efda40678c0 msgr2=0x7efda4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.098+0000 7efdcea00640 1 --2- 172.21.15.83:0/3261982992 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efda40678c0 0x7efda4069d80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009580 tx=0x7efdb8009190 comp rx=0 tx=0).stop 2024-04-25T00:15:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.098+0000 7efdcea00640 1 -- 172.21.15.83:0/3261982992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc815be20 msgr2=0x7efdc813f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.098+0000 7efdcea00640 1 --2- 172.21.15.83:0/3261982992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc815be20 0x7efdc813f580 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7efdc00b0040 tx=0x7efdc0019710 comp rx=0 tx=0).stop 2024-04-25T00:15:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.098+0000 7efdcea00640 1 -- 172.21.15.83:0/3261982992 shutdown_connections 2024-04-25T00:15:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.098+0000 7efdcea00640 1 --2- 172.21.15.83:0/3261982992 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7efda40678c0 0x7efda4069d80 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.098+0000 7efdcea00640 1 --2- 172.21.15.83:0/3261982992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc815be20 0x7efdc813f580 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.098+0000 7efdcea00640 1 --2- 172.21.15.83:0/3261982992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc80cedd0 0x7efdc815dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.098+0000 7efdcea00640 1 --2- 172.21.15.83:0/3261982992 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc806d5d0 0x7efdc814e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.098+0000 7efdcea00640 1 -- 172.21.15.83:0/3261982992 >> 172.21.15.83:0/3261982992 conn(0x7efdc80b1b10 msgr2=0x7efdc81551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:24.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.098+0000 7efdcea00640 1 -- 172.21.15.83:0/3261982992 shutdown_connections 2024-04-25T00:15:24.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.098+0000 7efdcea00640 1 -- 172.21.15.83:0/3261982992 wait complete. 2024-04-25T00:15:24.119 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:15:24.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb729200640 1 Processor -- start 2024-04-25T00:15:24.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb729200640 1 -- start start 2024-04-25T00:15:24.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb729200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7240cd880 0x7fb724144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb729200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7240cc390 0x7fb7240cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb729200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb724144da0 con 0x7fb7241449c0 2024-04-25T00:15:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb729200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb724072600 con 0x7fb7240cc390 2024-04-25T00:15:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb729200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb724072e80 con 0x7fb7240cd880 2024-04-25T00:15:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb723400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb7241449c0 0x7fb72406e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44304/0 (socket says 172.21.15.83:44304) 2024-04-25T00:15:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb723400640 1 -- 172.21.15.83:0/876130528 learned_addr learned my addr 172.21.15.83:0/876130528 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb723e00640 1 --2- 172.21.15.83:0/876130528 >> v2:172.21.15.83:3300/0 conn(0x7fb7240cc390 0x7fb7240cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb71aa00640 1 --2- 172.21.15.83:0/876130528 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7240cd880 0x7fb724144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb722a00640 1 -- 172.21.15.83:0/876130528 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2724203743 0 0) 0x7fb724144da0 con 0x7fb7241449c0 2024-04-25T00:15:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb722a00640 1 -- 172.21.15.83:0/876130528 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb708003530 con 0x7fb7241449c0 2024-04-25T00:15:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb723e00640 1 -- 172.21.15.83:0/876130528 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7240cd880 msgr2=0x7fb724144fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb723e00640 1 --2- 172.21.15.83:0/876130528 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7240cd880 0x7fb724144fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb723e00640 1 -- 172.21.15.83:0/876130528 >> v1:172.21.15.83:6789/0 conn(0x7fb7241449c0 legacy=0x7fb72406e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb723e00640 1 -- 172.21.15.83:0/876130528 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7241455a0 con 0x7fb7240cc390 2024-04-25T00:15:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.254+0000 7fb71aa00640 1 --2- 172.21.15.83:0/876130528 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7240cd880 0x7fb724144fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:24.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.258+0000 7fb723e00640 1 --2- 172.21.15.83:0/876130528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7240cc390 0x7fb7240cc820 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0097c0 tx=0x7fb71c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5457bcd7c7007b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:24.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.258+0000 7fb722a00640 1 -- 172.21.15.83:0/876130528 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0b0070 con 0x7fb7240cc390 2024-04-25T00:15:24.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.258+0000 7fb722a00640 1 -- 172.21.15.83:0/876130528 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71c0ab600 con 0x7fb7240cc390 2024-04-25T00:15:24.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.258+0000 7fb722a00640 1 -- 172.21.15.83:0/876130528 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0ab9d0 con 0x7fb7240cc390 2024-04-25T00:15:24.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.258+0000 7fb729200640 1 -- 172.21.15.83:0/876130528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7240cc390 msgr2=0x7fb7240cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.258+0000 7fb729200640 1 --2- 172.21.15.83:0/876130528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7240cc390 0x7fb7240cc820 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0097c0 tx=0x7fb71c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.258+0000 7fb729200640 1 -- 172.21.15.83:0/876130528 shutdown_connections 2024-04-25T00:15:24.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.258+0000 7fb729200640 1 --2- 172.21.15.83:0/876130528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7240cc390 0x7fb7240cc820 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.258+0000 7fb729200640 1 --2- 172.21.15.83:0/876130528 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7240cd880 0x7fb724144fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.258+0000 7fb729200640 1 -- 172.21.15.83:0/876130528 >> 172.21.15.83:0/876130528 conn(0x7fb7240b1650 msgr2=0x7fb7240b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:24.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.258+0000 7fb729200640 1 -- 172.21.15.83:0/876130528 shutdown_connections 2024-04-25T00:15:24.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.258+0000 7fb729200640 1 -- 172.21.15.83:0/876130528 wait complete. 2024-04-25T00:15:24.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb729200640 1 Processor -- start 2024-04-25T00:15:24.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb729200640 1 -- start start 2024-04-25T00:15:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb729200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7240cc390 0x7fb724142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb729200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7240cd880 0x7fb72415be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb729200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7241449c0 0x7fb72415c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb729200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb724160530 con 0x7fb7241449c0 2024-04-25T00:15:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb729200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241603b0 con 0x7fb7240cd880 2024-04-25T00:15:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb729200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb7241606b0 con 0x7fb7240cc390 2024-04-25T00:15:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb723e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7241449c0 0x7fb72415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb723e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7241449c0 0x7fb72415c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39774/0 (socket says 172.21.15.83:39774) 2024-04-25T00:15:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb723e00640 1 -- 172.21.15.83:0/2825809458 learned_addr learned my addr 172.21.15.83:0/2825809458 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb723400640 1 --2- 172.21.15.83:0/2825809458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7240cc390 0x7fb724142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb71aa00640 1 --2- 172.21.15.83:0/2825809458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7240cd880 0x7fb72415be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb723e00640 1 -- 172.21.15.83:0/2825809458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7240cc390 msgr2=0x7fb724142150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb723e00640 1 --2- 172.21.15.83:0/2825809458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7240cc390 0x7fb724142150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb723e00640 1 -- 172.21.15.83:0/2825809458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7240cd880 msgr2=0x7fb72415be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb723e00640 1 --2- 172.21.15.83:0/2825809458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7240cd880 0x7fb72415be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb723e00640 1 -- 172.21.15.83:0/2825809458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72415b960 con 0x7fb7241449c0 2024-04-25T00:15:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb723e00640 1 --2- 172.21.15.83:0/2825809458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7241449c0 0x7fb72415c370 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0097c0 tx=0x7fb71c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb71be00640 1 -- 172.21.15.83:0/2825809458 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0b0040 con 0x7fb7241449c0 2024-04-25T00:15:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb71be00640 1 -- 172.21.15.83:0/2825809458 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71c0b9070 con 0x7fb7241449c0 2024-04-25T00:15:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb729200640 1 -- 172.21.15.83:0/2825809458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72415c9a0 con 0x7fb7241449c0 2024-04-25T00:15:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb71be00640 1 -- 172.21.15.83:0/2825809458 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0b4650 con 0x7fb7241449c0 2024-04-25T00:15:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb729200640 1 -- 172.21.15.83:0/2825809458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72415cf60 con 0x7fb7241449c0 2024-04-25T00:15:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.262+0000 7fb729200640 1 -- 172.21.15.83:0/2825809458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72406de80 con 0x7fb7241449c0 2024-04-25T00:15:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.266+0000 7fb71be00640 1 -- 172.21.15.83:0/2825809458 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb71c0b47f0 con 0x7fb7241449c0 2024-04-25T00:15:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.266+0000 7fb71be00640 1 --2- 172.21.15.83:0/2825809458 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb7000678c0 0x7fb700069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.266+0000 7fb71be00640 1 -- 172.21.15.83:0/2825809458 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7fb71c11f8f0 con 0x7fb7241449c0 2024-04-25T00:15:24.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.266+0000 7fb723400640 1 --2- 172.21.15.83:0/2825809458 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb7000678c0 0x7fb700069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.266+0000 7fb723400640 1 --2- 172.21.15.83:0/2825809458 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb7000678c0 0x7fb700069d80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb714005c10 tx=0x7fb714009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:24.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.270+0000 7fb71be00640 1 -- 172.21.15.83:0/2825809458 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb71c016090 con 0x7fb7241449c0 2024-04-25T00:15:24.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.434+0000 7fb729200640 1 -- 172.21.15.83:0/2825809458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb7240d03e0 con 0x7fb7241449c0 2024-04-25T00:15:24.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.438+0000 7fb71be00640 1 -- 172.21.15.83:0/2825809458 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v97) ==== 80+0+244 (secure 0 0 0) 0x7fb71c0efc30 con 0x7fb7241449c0 2024-04-25T00:15:24.438 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6837/1609715757 2024-04-26T00:15:04.845080+0000 2024-04-25T00:15:24.438 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6836/1609715757 2024-04-26T00:15:04.845080+0000 2024-04-25T00:15:24.438 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.86:6835/3612679148 2024-04-26T00:15:04.826082+0000 2024-04-25T00:15:24.438 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.86:6833/3612679148 2024-04-26T00:15:04.826082+0000 2024-04-25T00:15:24.438 INFO:teuthology.orchestra.run.smithi083.stderr:listed 4 entries 2024-04-25T00:15:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.438+0000 7fb729200640 1 -- 172.21.15.83:0/2825809458 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb7000678c0 msgr2=0x7fb700069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.438+0000 7fb729200640 1 --2- 172.21.15.83:0/2825809458 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb7000678c0 0x7fb700069d80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb714005c10 tx=0x7fb714009040 comp rx=0 tx=0).stop 2024-04-25T00:15:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.438+0000 7fb729200640 1 -- 172.21.15.83:0/2825809458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7241449c0 msgr2=0x7fb72415c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.438+0000 7fb729200640 1 --2- 172.21.15.83:0/2825809458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7241449c0 0x7fb72415c370 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0097c0 tx=0x7fb71c019240 comp rx=0 tx=0).stop 2024-04-25T00:15:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.438+0000 7fb729200640 1 -- 172.21.15.83:0/2825809458 shutdown_connections 2024-04-25T00:15:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.438+0000 7fb729200640 1 --2- 172.21.15.83:0/2825809458 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb7000678c0 0x7fb700069d80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.438+0000 7fb729200640 1 --2- 172.21.15.83:0/2825809458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7241449c0 0x7fb72415c370 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.438+0000 7fb729200640 1 --2- 172.21.15.83:0/2825809458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7240cd880 0x7fb72415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.438+0000 7fb729200640 1 --2- 172.21.15.83:0/2825809458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7240cc390 0x7fb724142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.438+0000 7fb729200640 1 -- 172.21.15.83:0/2825809458 >> 172.21.15.83:0/2825809458 conn(0x7fb7240b1650 msgr2=0x7fb7240af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.438+0000 7fb729200640 1 -- 172.21.15.83:0/2825809458 shutdown_connections 2024-04-25T00:15:24.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.438+0000 7fb729200640 1 -- 172.21.15.83:0/2825809458 wait complete. 2024-04-25T00:15:24.460 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:15:24.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:24.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:15:24.514 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:15:24.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:24.515 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:15:24.537 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:15:24.537 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:24.547 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:15:24.547 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:24.597 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:15:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.730+0000 7f95e0600640 1 Processor -- start 2024-04-25T00:15:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.730+0000 7f95e0600640 1 -- start start 2024-04-25T00:15:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.734+0000 7f95e0600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f95d80aed90 0x7f95d80acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.734+0000 7f95e0600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d806ecf0 con 0x7f95d815ce00 2024-04-25T00:15:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.734+0000 7f95e0600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d806efd0 con 0x7f95d8156340 2024-04-25T00:15:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.734+0000 7f95e0600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f95d806cef0 con 0x7f95d80aed90 2024-04-25T00:15:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.734+0000 7f95dd200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f95d815ce00 0x7f95d806d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37742/0 (socket says 172.21.15.83:37742) 2024-04-25T00:15:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.734+0000 7f95dd200640 1 -- 172.21.15.83:0/2889688769 learned_addr learned my addr 172.21.15.83:0/2889688769 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:24.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.734+0000 7f95de600640 1 --2- 172.21.15.83:0/2889688769 >> v2:172.21.15.86:3300/0 conn(0x7f95d80aed90 0x7f95d80acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.734+0000 7f95d7e00640 1 -- 172.21.15.83:0/2889688769 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 205768241 0 0) 0x7f95d806ecf0 con 0x7f95d815ce00 2024-04-25T00:15:24.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.734+0000 7f95d7e00640 1 -- 172.21.15.83:0/2889688769 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95c0003530 con 0x7f95d815ce00 2024-04-25T00:15:24.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.734+0000 7f95de600640 1 -- 172.21.15.83:0/2889688769 >> v1:172.21.15.86:6789/0 conn(0x7f95d8156340 legacy=0x7f95d80ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.734+0000 7f95de600640 1 -- 172.21.15.83:0/2889688769 >> v1:172.21.15.83:6789/0 conn(0x7f95d815ce00 legacy=0x7f95d806d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.734+0000 7f95de600640 1 -- 172.21.15.83:0/2889688769 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d815d290 con 0x7f95d80aed90 2024-04-25T00:15:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.734+0000 7f95de600640 1 --2- 172.21.15.83:0/2889688769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d80aed90 0x7f95d80acd20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009d90 tx=0x7f95d001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47450f94f0f1932 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95d7e00640 1 -- 172.21.15.83:0/2889688769 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d00b2070 con 0x7f95d80aed90 2024-04-25T00:15:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95d7e00640 1 -- 172.21.15.83:0/2889688769 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d00a5920 con 0x7f95d80aed90 2024-04-25T00:15:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95d7e00640 1 -- 172.21.15.83:0/2889688769 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d00a5cd0 con 0x7f95d80aed90 2024-04-25T00:15:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 -- 172.21.15.83:0/2889688769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d80aed90 msgr2=0x7f95d80acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 --2- 172.21.15.83:0/2889688769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d80aed90 0x7f95d80acd20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009d90 tx=0x7f95d001c970 comp rx=0 tx=0).stop 2024-04-25T00:15:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 -- 172.21.15.83:0/2889688769 shutdown_connections 2024-04-25T00:15:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 --2- 172.21.15.83:0/2889688769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d80aed90 0x7f95d80acd20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 -- 172.21.15.83:0/2889688769 >> 172.21.15.83:0/2889688769 conn(0x7f95d80b1860 msgr2=0x7f95d80b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 -- 172.21.15.83:0/2889688769 shutdown_connections 2024-04-25T00:15:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 -- 172.21.15.83:0/2889688769 wait complete. 2024-04-25T00:15:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 Processor -- start 2024-04-25T00:15:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 -- start start 2024-04-25T00:15:24.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d80aed90 0x7f95d8145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95d8156340 0x7f95d8151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d8152030 0x7f95d81487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95d806e0b0 con 0x7f95d8152030 2024-04-25T00:15:24.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95d806df30 con 0x7f95d80aed90 2024-04-25T00:15:24.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95e0600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f95d816a530 con 0x7f95d8156340 2024-04-25T00:15:24.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95de600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d8152030 0x7f95d81487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95de600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d8152030 0x7f95d81487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57530/0 (socket says 172.21.15.83:57530) 2024-04-25T00:15:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.738+0000 7f95de600640 1 -- 172.21.15.83:0/689755978 learned_addr learned my addr 172.21.15.83:0/689755978 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95dd200640 1 --2- 172.21.15.83:0/689755978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95d8156340 0x7f95d8151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95ddc00640 1 --2- 172.21.15.83:0/689755978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d80aed90 0x7f95d8145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95de600640 1 -- 172.21.15.83:0/689755978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95d8156340 msgr2=0x7f95d8151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95de600640 1 --2- 172.21.15.83:0/689755978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95d8156340 0x7f95d8151af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95de600640 1 -- 172.21.15.83:0/689755978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d80aed90 msgr2=0x7f95d8145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95de600640 1 --2- 172.21.15.83:0/689755978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d80aed90 0x7f95d8145470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95de600640 1 -- 172.21.15.83:0/689755978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d80b4c90 con 0x7f95d8152030 2024-04-25T00:15:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95de600640 1 --2- 172.21.15.83:0/689755978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d8152030 0x7f95d81487c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009d90 tx=0x7f95d0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95d5600640 1 -- 172.21.15.83:0/689755978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d00bd020 con 0x7f95d8152030 2024-04-25T00:15:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95d5600640 1 -- 172.21.15.83:0/689755978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d00b3040 con 0x7f95d8152030 2024-04-25T00:15:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95e0600640 1 -- 172.21.15.83:0/689755978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d80b3bf0 con 0x7f95d8152030 2024-04-25T00:15:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95d5600640 1 -- 172.21.15.83:0/689755978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d00b2050 con 0x7f95d8152030 2024-04-25T00:15:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95e0600640 1 -- 172.21.15.83:0/689755978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d80b41b0 con 0x7f95d8152030 2024-04-25T00:15:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95e0600640 1 -- 172.21.15.83:0/689755978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d804f510 con 0x7f95d8152030 2024-04-25T00:15:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95d5600640 1 -- 172.21.15.83:0/689755978 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f95d00ad650 con 0x7f95d8152030 2024-04-25T00:15:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95d5600640 1 --2- 172.21.15.83:0/689755978 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95b80678c0 0x7f95b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95d5600640 1 -- 172.21.15.83:0/689755978 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f95d0120b80 con 0x7f95d8152030 2024-04-25T00:15:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.742+0000 7f95ddc00640 1 --2- 172.21.15.83:0/689755978 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95b80678c0 0x7f95b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.746+0000 7f95ddc00640 1 --2- 172.21.15.83:0/689755978 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95b80678c0 0x7f95b8069d80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f95d809d0f0 tx=0x7f95cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.746+0000 7f95d5600640 1 -- 172.21.15.83:0/689755978 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f95d00c7020 con 0x7f95d8152030 2024-04-25T00:15:24.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.902+0000 7f95e0600640 1 -- 172.21.15.83:0/689755978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95d8072330 con 0x7f95d8152030 2024-04-25T00:15:24.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.906+0000 7f95d5600640 1 -- 172.21.15.83:0/689755978 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+17048 (secure 0 0 0) 0x7f95d80b4c90 con 0x7f95d8152030 2024-04-25T00:15:24.906 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:24.906 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":97,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:15:04.852008+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:50.260735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:50.880394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1609715757":"2024-04-26T00:15:04.845080+0000","172.21.15.83:6836/1609715757":"2024-04-26T00:15:04.845080+0000","172.21.15.86:6835/3612679148":"2024-04-26T00:15:04.826082+0000","172.21.15.86:6833/3612679148":"2024-04-26T00:15:04.826082+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:15:24.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.906+0000 7f95e0600640 1 -- 172.21.15.83:0/689755978 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95b80678c0 msgr2=0x7f95b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.906+0000 7f95e0600640 1 --2- 172.21.15.83:0/689755978 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95b80678c0 0x7f95b8069d80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f95d809d0f0 tx=0x7f95cc009040 comp rx=0 tx=0).stop 2024-04-25T00:15:24.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.906+0000 7f95e0600640 1 -- 172.21.15.83:0/689755978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d8152030 msgr2=0x7f95d81487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.906+0000 7f95e0600640 1 --2- 172.21.15.83:0/689755978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d8152030 0x7f95d81487c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009d90 tx=0x7f95d0019710 comp rx=0 tx=0).stop 2024-04-25T00:15:24.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.906+0000 7f95e0600640 1 -- 172.21.15.83:0/689755978 shutdown_connections 2024-04-25T00:15:24.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.906+0000 7f95e0600640 1 --2- 172.21.15.83:0/689755978 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95b80678c0 0x7f95b8069d80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.906+0000 7f95e0600640 1 --2- 172.21.15.83:0/689755978 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95d8156340 0x7f95d8151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.906+0000 7f95e0600640 1 --2- 172.21.15.83:0/689755978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d8152030 0x7f95d81487c0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.906+0000 7f95e0600640 1 --2- 172.21.15.83:0/689755978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d80aed90 0x7f95d8145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.906+0000 7f95e0600640 1 -- 172.21.15.83:0/689755978 >> 172.21.15.83:0/689755978 conn(0x7f95d80b1860 msgr2=0x7f95d806e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:24.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.906+0000 7f95e0600640 1 -- 172.21.15.83:0/689755978 shutdown_connections 2024-04-25T00:15:24.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:24.906+0000 7f95e0600640 1 -- 172.21.15.83:0/689755978 wait complete. 2024-04-25T00:15:24.927 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:14:50.260735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:50.880394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:15:24.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:15:25.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe990600640 1 Processor -- start 2024-04-25T00:15:25.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe990600640 1 -- start start 2024-04-25T00:15:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe990600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9880d0350 0x7fe9880b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe990600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe9880b3180 0x7fe9880cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe990600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9881556e0 con 0x7fe988155300 2024-04-25T00:15:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe990600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe988072260 con 0x7fe9880b3180 2024-04-25T00:15:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe990600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe988072540 con 0x7fe9880d0350 2024-04-25T00:15:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe98dc00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe988155300 0x7fe9880cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37750/0 (socket says 172.21.15.83:37750) 2024-04-25T00:15:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe98dc00640 1 -- 172.21.15.83:0/3526304065 learned_addr learned my addr 172.21.15.83:0/3526304065 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe98e600640 1 --2- 172.21.15.83:0/3526304065 >> v2:172.21.15.83:3300/0 conn(0x7fe9880b3180 0x7fe9880cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe98d200640 1 --2- 172.21.15.83:0/3526304065 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9880d0350 0x7fe9880b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe987e00640 1 -- 172.21.15.83:0/3526304065 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1505929185 0 0) 0x7fe9881556e0 con 0x7fe988155300 2024-04-25T00:15:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe987e00640 1 -- 172.21.15.83:0/3526304065 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe970003530 con 0x7fe988155300 2024-04-25T00:15:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe98e600640 1 -- 172.21.15.83:0/3526304065 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9880d0350 msgr2=0x7fe9880b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe98e600640 1 --2- 172.21.15.83:0/3526304065 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9880d0350 0x7fe9880b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe98e600640 1 -- 172.21.15.83:0/3526304065 >> v1:172.21.15.83:6789/0 conn(0x7fe988155300 legacy=0x7fe9880cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.066+0000 7fe98e600640 1 -- 172.21.15.83:0/3526304065 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe988072820 con 0x7fe9880b3180 2024-04-25T00:15:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.070+0000 7fe98e600640 1 --2- 172.21.15.83:0/3526304065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9880b3180 0x7fe9880cec50 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe9800097c0 tx=0x7fe98001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c4158937b7b59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.070+0000 7fe987e00640 1 -- 172.21.15.83:0/3526304065 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9800ab430 con 0x7fe9880b3180 2024-04-25T00:15:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.070+0000 7fe987e00640 1 -- 172.21.15.83:0/3526304065 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9800b0070 con 0x7fe9880b3180 2024-04-25T00:15:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.070+0000 7fe987e00640 1 -- 172.21.15.83:0/3526304065 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9800ab7d0 con 0x7fe9880b3180 2024-04-25T00:15:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.070+0000 7fe990600640 1 -- 172.21.15.83:0/3526304065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9880b3180 msgr2=0x7fe9880cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.070+0000 7fe990600640 1 --2- 172.21.15.83:0/3526304065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9880b3180 0x7fe9880cec50 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe9800097c0 tx=0x7fe98001c410 comp rx=0 tx=0).stop 2024-04-25T00:15:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.070+0000 7fe990600640 1 -- 172.21.15.83:0/3526304065 shutdown_connections 2024-04-25T00:15:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.070+0000 7fe990600640 1 --2- 172.21.15.83:0/3526304065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9880b3180 0x7fe9880cec50 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.070+0000 7fe990600640 1 --2- 172.21.15.83:0/3526304065 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9880d0350 0x7fe9880b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.070+0000 7fe990600640 1 -- 172.21.15.83:0/3526304065 >> 172.21.15.83:0/3526304065 conn(0x7fe9880b1b10 msgr2=0x7fe9880b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.070+0000 7fe990600640 1 -- 172.21.15.83:0/3526304065 shutdown_connections 2024-04-25T00:15:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.070+0000 7fe990600640 1 -- 172.21.15.83:0/3526304065 wait complete. 2024-04-25T00:15:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe990600640 1 Processor -- start 2024-04-25T00:15:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe990600640 1 -- start start 2024-04-25T00:15:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe990600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9880b3180 0x7fe988147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe990600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9880d0350 0x7fe988151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe990600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe988155300 0x7fe98814e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe990600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe98815b2e0 con 0x7fe9880b3180 2024-04-25T00:15:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe990600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe98815b460 con 0x7fe988155300 2024-04-25T00:15:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe990600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe98815b5e0 con 0x7fe9880d0350 2024-04-25T00:15:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe98dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9880b3180 0x7fe988147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe98dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9880b3180 0x7fe988147d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57548/0 (socket says 172.21.15.83:57548) 2024-04-25T00:15:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe98dc00640 1 -- 172.21.15.83:0/315976064 learned_addr learned my addr 172.21.15.83:0/315976064 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe98d200640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9880d0350 0x7fe988151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe98e600640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe988155300 0x7fe98814e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe98dc00640 1 -- 172.21.15.83:0/315976064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9880d0350 msgr2=0x7fe988151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe98dc00640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9880d0350 0x7fe988151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe98dc00640 1 -- 172.21.15.83:0/315976064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe988155300 msgr2=0x7fe98814e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe98dc00640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe988155300 0x7fe98814e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe98dc00640 1 -- 172.21.15.83:0/315976064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9880b0af0 con 0x7fe9880b3180 2024-04-25T00:15:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe98d200640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9880d0350 0x7fe988151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe98dc00640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9880b3180 0x7fe988147d50 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00caa0 tx=0x7fe97c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe985600640 1 -- 172.21.15.83:0/315976064 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c017070 con 0x7fe9880b3180 2024-04-25T00:15:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe985600640 1 -- 172.21.15.83:0/315976064 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe97c00ac00 con 0x7fe9880b3180 2024-04-25T00:15:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe990600640 1 -- 172.21.15.83:0/315976064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe988148290 con 0x7fe9880b3180 2024-04-25T00:15:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe985600640 1 -- 172.21.15.83:0/315976064 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c012400 con 0x7fe9880b3180 2024-04-25T00:15:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe990600640 1 -- 172.21.15.83:0/315976064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9881508b0 con 0x7fe9880b3180 2024-04-25T00:15:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.074+0000 7fe990600640 1 -- 172.21.15.83:0/315976064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe98804f510 con 0x7fe9880b3180 2024-04-25T00:15:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.078+0000 7fe985600640 1 -- 172.21.15.83:0/315976064 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe97c01a070 con 0x7fe9880b3180 2024-04-25T00:15:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.078+0000 7fe985600640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9680678c0 0x7fe968069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.078+0000 7fe985600640 1 -- 172.21.15.83:0/315976064 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7fe97c085c60 con 0x7fe9880b3180 2024-04-25T00:15:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.078+0000 7fe98d200640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9680678c0 0x7fe968069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.078+0000 7fe98d200640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9680678c0 0x7fe968069d80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe988068550 tx=0x7fe974008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.082+0000 7fe985600640 1 -- 172.21.15.83:0/315976064 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe97c056550 con 0x7fe9880b3180 2024-04-25T00:15:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.238+0000 7fe990600640 1 -- 172.21.15.83:0/315976064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe988071990 con 0x7fe9880b3180 2024-04-25T00:15:25.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.238+0000 7fe985600640 1 -- 172.21.15.83:0/315976064 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7fe97c055e50 con 0x7fe9880b3180 2024-04-25T00:15:25.240 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:15:25.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.242+0000 7fe990600640 1 -- 172.21.15.83:0/315976064 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9680678c0 msgr2=0x7fe968069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.242+0000 7fe990600640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9680678c0 0x7fe968069d80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe988068550 tx=0x7fe974008040 comp rx=0 tx=0).stop 2024-04-25T00:15:25.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.242+0000 7fe990600640 1 -- 172.21.15.83:0/315976064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9880b3180 msgr2=0x7fe988147d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.242+0000 7fe990600640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9880b3180 0x7fe988147d50 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00caa0 tx=0x7fe97c00cf60 comp rx=0 tx=0).stop 2024-04-25T00:15:25.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.242+0000 7fe990600640 1 -- 172.21.15.83:0/315976064 shutdown_connections 2024-04-25T00:15:25.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.242+0000 7fe990600640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe9680678c0 0x7fe968069d80 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.242+0000 7fe990600640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9880d0350 0x7fe988151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.242+0000 7fe990600640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe988155300 0x7fe98814e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.242+0000 7fe990600640 1 --2- 172.21.15.83:0/315976064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9880b3180 0x7fe988147d50 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.242+0000 7fe990600640 1 -- 172.21.15.83:0/315976064 >> 172.21.15.83:0/315976064 conn(0x7fe9880b1b10 msgr2=0x7fe9880b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:25.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.242+0000 7fe990600640 1 -- 172.21.15.83:0/315976064 shutdown_connections 2024-04-25T00:15:25.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.242+0000 7fe990600640 1 -- 172.21.15.83:0/315976064 wait complete. 2024-04-25T00:15:25.261 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:15:25.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f8385400640 1 Processor -- start 2024-04-25T00:15:25.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f8385400640 1 -- start start 2024-04-25T00:15:25.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f8385400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8380147920 0x7f838015ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f8385400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83800b4cd0 0x7f8380161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f8385400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8380162490 con 0x7f83800b2c40 2024-04-25T00:15:25.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f8385400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8380147d00 con 0x7f8380147920 2024-04-25T00:15:25.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f8385400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8380162a50 con 0x7f83800b4cd0 2024-04-25T00:15:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f8375e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83800b4cd0 0x7f8380161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f837e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8380147920 0x7f838015ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f8375e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83800b4cd0 0x7f8380161b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:42112/0 (socket says 172.21.15.83:42112) 2024-04-25T00:15:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f8375e00640 1 -- 172.21.15.83:0/4072646638 learned_addr learned my addr 172.21.15.83:0/4072646638 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f837e800640 1 -- 172.21.15.83:0/4072646638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83800b4cd0 msgr2=0x7f8380161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f837e800640 1 --2- 172.21.15.83:0/4072646638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83800b4cd0 0x7f8380161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f837e800640 1 -- 172.21.15.83:0/4072646638 >> v1:172.21.15.86:6789/0 conn(0x7f83800b2c40 legacy=0x7f83800acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f837e800640 1 -- 172.21.15.83:0/4072646638 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8380162770 con 0x7f8380147920 2024-04-25T00:15:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.398+0000 7f8375e00640 1 --2- 172.21.15.83:0/4072646638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83800b4cd0 0x7f8380161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:25.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.402+0000 7f837e800640 1 --2- 172.21.15.83:0/4072646638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380147920 0x7f838015ef10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f83700097c0 tx=0x7f837001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3d9f2cca4422dff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:25.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.402+0000 7f837de00640 1 -- 172.21.15.83:0/4072646638 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83700b2070 con 0x7f8380147920 2024-04-25T00:15:25.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.402+0000 7f837de00640 1 -- 172.21.15.83:0/4072646638 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83700ad640 con 0x7f8380147920 2024-04-25T00:15:25.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.402+0000 7f837de00640 1 -- 172.21.15.83:0/4072646638 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83700ada10 con 0x7f8380147920 2024-04-25T00:15:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.402+0000 7f8385400640 1 -- 172.21.15.83:0/4072646638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380147920 msgr2=0x7f838015ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.402+0000 7f8385400640 1 --2- 172.21.15.83:0/4072646638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380147920 0x7f838015ef10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f83700097c0 tx=0x7f837001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.402+0000 7f8385400640 1 -- 172.21.15.83:0/4072646638 shutdown_connections 2024-04-25T00:15:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.402+0000 7f8385400640 1 --2- 172.21.15.83:0/4072646638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83800b4cd0 0x7f8380161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.402+0000 7f8385400640 1 --2- 172.21.15.83:0/4072646638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380147920 0x7f838015ef10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.402+0000 7f8385400640 1 -- 172.21.15.83:0/4072646638 >> 172.21.15.83:0/4072646638 conn(0x7f83800b1b10 msgr2=0x7f83800b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.402+0000 7f8385400640 1 -- 172.21.15.83:0/4072646638 shutdown_connections 2024-04-25T00:15:25.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.402+0000 7f8385400640 1 -- 172.21.15.83:0/4072646638 wait complete. 2024-04-25T00:15:25.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8385400640 1 Processor -- start 2024-04-25T00:15:25.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8385400640 1 -- start start 2024-04-25T00:15:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8385400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83800b2c40 0x7f838013f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8385400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800b4cd0 0x7f838009c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8385400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8380147920 0x7f838015fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8385400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83800b3460 con 0x7f83800b2c40 2024-04-25T00:15:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8385400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f838015f450 con 0x7f83800b4cd0 2024-04-25T00:15:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8385400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f83801415b0 con 0x7f8380147920 2024-04-25T00:15:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f837e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83800b2c40 0x7f838013f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f837e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83800b2c40 0x7f838013f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57560/0 (socket says 172.21.15.83:57560) 2024-04-25T00:15:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f837e800640 1 -- 172.21.15.83:0/1628713924 learned_addr learned my addr 172.21.15.83:0/1628713924 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8375e00640 1 --2- 172.21.15.83:0/1628713924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800b4cd0 0x7f838009c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f837f200640 1 --2- 172.21.15.83:0/1628713924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8380147920 0x7f838015fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f837e800640 1 -- 172.21.15.83:0/1628713924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8380147920 msgr2=0x7f838015fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f837e800640 1 --2- 172.21.15.83:0/1628713924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8380147920 0x7f838015fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f837e800640 1 -- 172.21.15.83:0/1628713924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800b4cd0 msgr2=0x7f838009c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f837e800640 1 --2- 172.21.15.83:0/1628713924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800b4cd0 0x7f838009c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f837e800640 1 -- 172.21.15.83:0/1628713924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838013fac0 con 0x7f83800b2c40 2024-04-25T00:15:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f837e800640 1 --2- 172.21.15.83:0/1628713924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83800b2c40 0x7f838013f580 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f837001a0d0 tx=0x7f8370019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8377400640 1 -- 172.21.15.83:0/1628713924 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83700ba020 con 0x7f83800b2c40 2024-04-25T00:15:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8377400640 1 -- 172.21.15.83:0/1628713924 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83700b6410 con 0x7f83800b2c40 2024-04-25T00:15:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8385400640 1 -- 172.21.15.83:0/1628713924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8380140920 con 0x7f83800b2c40 2024-04-25T00:15:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8377400640 1 -- 172.21.15.83:0/1628713924 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83700b2040 con 0x7f83800b2c40 2024-04-25T00:15:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8385400640 1 -- 172.21.15.83:0/1628713924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8380140ee0 con 0x7f83800b2c40 2024-04-25T00:15:25.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.406+0000 7f8385400640 1 -- 172.21.15.83:0/1628713924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f838004f510 con 0x7f83800b2c40 2024-04-25T00:15:25.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.410+0000 7f8377400640 1 -- 172.21.15.83:0/1628713924 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f83700bd020 con 0x7f83800b2c40 2024-04-25T00:15:25.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.410+0000 7f8377400640 1 --2- 172.21.15.83:0/1628713924 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f835c0678c0 0x7f835c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.410+0000 7f8377400640 1 -- 172.21.15.83:0/1628713924 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f837011fcf0 con 0x7f83800b2c40 2024-04-25T00:15:25.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.410+0000 7f8375e00640 1 --2- 172.21.15.83:0/1628713924 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f835c0678c0 0x7f835c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.410+0000 7f8375e00640 1 --2- 172.21.15.83:0/1628713924 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f835c0678c0 0x7f835c069d80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f838006b500 tx=0x7f8368008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.414+0000 7f8377400640 1 -- 172.21.15.83:0/1628713924 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f83700c4030 con 0x7f83800b2c40 2024-04-25T00:15:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.582+0000 7f8385400640 1 -- 172.21.15.83:0/1628713924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8380168cd0 con 0x7f83800b2c40 2024-04-25T00:15:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.582+0000 7f8377400640 1 -- 172.21.15.83:0/1628713924 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f8370016090 con 0x7f83800b2c40 2024-04-25T00:15:25.584 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:15:25.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.586+0000 7f8385400640 1 -- 172.21.15.83:0/1628713924 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f835c0678c0 msgr2=0x7f835c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.586+0000 7f8385400640 1 --2- 172.21.15.83:0/1628713924 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f835c0678c0 0x7f835c069d80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f838006b500 tx=0x7f8368008040 comp rx=0 tx=0).stop 2024-04-25T00:15:25.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.586+0000 7f8385400640 1 -- 172.21.15.83:0/1628713924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83800b2c40 msgr2=0x7f838013f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.586+0000 7f8385400640 1 --2- 172.21.15.83:0/1628713924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83800b2c40 0x7f838013f580 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f837001a0d0 tx=0x7f8370019240 comp rx=0 tx=0).stop 2024-04-25T00:15:25.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.586+0000 7f8385400640 1 -- 172.21.15.83:0/1628713924 shutdown_connections 2024-04-25T00:15:25.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.586+0000 7f8385400640 1 --2- 172.21.15.83:0/1628713924 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f835c0678c0 0x7f835c069d80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.586+0000 7f8385400640 1 --2- 172.21.15.83:0/1628713924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8380147920 0x7f838015fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.586+0000 7f8385400640 1 --2- 172.21.15.83:0/1628713924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800b4cd0 0x7f838009c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.586+0000 7f8385400640 1 --2- 172.21.15.83:0/1628713924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83800b2c40 0x7f838013f580 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.586+0000 7f8385400640 1 -- 172.21.15.83:0/1628713924 >> 172.21.15.83:0/1628713924 conn(0x7f83800b1b10 msgr2=0x7f838006bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:25.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.586+0000 7f8385400640 1 -- 172.21.15.83:0/1628713924 shutdown_connections 2024-04-25T00:15:25.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.586+0000 7f8385400640 1 -- 172.21.15.83:0/1628713924 wait complete. 2024-04-25T00:15:25.604 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:15:25.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.742+0000 7f1f88400640 1 Processor -- start 2024-04-25T00:15:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.742+0000 7f1f88400640 1 -- start start 2024-04-25T00:15:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.742+0000 7f1f88400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f8015be20 0x7f1f8015f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.742+0000 7f1f88400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8015c200 con 0x7f1f8006d5d0 2024-04-25T00:15:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.742+0000 7f1f88400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f80160100 con 0x7f1f800cedd0 2024-04-25T00:15:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.742+0000 7f1f88400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8015fed0 con 0x7f1f8015be20 2024-04-25T00:15:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.742+0000 7f1f86400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1f8006d5d0 0x7f1f800b3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37760/0 (socket says 172.21.15.83:37760) 2024-04-25T00:15:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.742+0000 7f1f86400640 1 -- 172.21.15.83:0/3271858631 learned_addr learned my addr 172.21.15.83:0/3271858631 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.742+0000 7f1f85a00640 1 --2- 172.21.15.83:0/3271858631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f8015be20 0x7f1f8015f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.746+0000 7f1f7fe00640 1 -- 172.21.15.83:0/3271858631 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2356811456 0 0) 0x7f1f8015c200 con 0x7f1f8006d5d0 2024-04-25T00:15:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.746+0000 7f1f7fe00640 1 -- 172.21.15.83:0/3271858631 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f68003530 con 0x7f1f8006d5d0 2024-04-25T00:15:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.746+0000 7f1f7fe00640 1 -- 172.21.15.83:0/3271858631 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1308036902 0 0) 0x7f1f80160100 con 0x7f1f800cedd0 2024-04-25T00:15:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.746+0000 7f1f7fe00640 1 -- 172.21.15.83:0/3271858631 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f8015c200 con 0x7f1f800cedd0 2024-04-25T00:15:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.746+0000 7f1f85a00640 1 -- 172.21.15.83:0/3271858631 >> v1:172.21.15.86:6789/0 conn(0x7f1f800cedd0 legacy=0x7f1f8006cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.746+0000 7f1f85a00640 1 -- 172.21.15.83:0/3271858631 >> v1:172.21.15.83:6789/0 conn(0x7f1f8006d5d0 legacy=0x7f1f800b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.746+0000 7f1f85a00640 1 -- 172.21.15.83:0/3271858631 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f801603e0 con 0x7f1f8015be20 2024-04-25T00:15:25.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.746+0000 7f1f85a00640 1 --2- 172.21.15.83:0/3271858631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f8015be20 0x7f1f8015f990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009dc0 tx=0x7f1f7401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f548df0231328bf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:25.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.746+0000 7f1f7fe00640 1 -- 172.21.15.83:0/3271858631 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f740b2070 con 0x7f1f8015be20 2024-04-25T00:15:25.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.746+0000 7f1f7fe00640 1 -- 172.21.15.83:0/3271858631 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f740a5850 con 0x7f1f8015be20 2024-04-25T00:15:25.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.746+0000 7f1f7fe00640 1 -- 172.21.15.83:0/3271858631 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f740a5c20 con 0x7f1f8015be20 2024-04-25T00:15:25.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.746+0000 7f1f88400640 1 -- 172.21.15.83:0/3271858631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f8015be20 msgr2=0x7f1f8015f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.746+0000 7f1f88400640 1 --2- 172.21.15.83:0/3271858631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f8015be20 0x7f1f8015f990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009dc0 tx=0x7f1f7401c990 comp rx=0 tx=0).stop 2024-04-25T00:15:25.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f88400640 1 -- 172.21.15.83:0/3271858631 shutdown_connections 2024-04-25T00:15:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f88400640 1 --2- 172.21.15.83:0/3271858631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f8015be20 0x7f1f8015f990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f88400640 1 -- 172.21.15.83:0/3271858631 >> 172.21.15.83:0/3271858631 conn(0x7f1f800b1b10 msgr2=0x7f1f800b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f88400640 1 -- 172.21.15.83:0/3271858631 shutdown_connections 2024-04-25T00:15:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f88400640 1 -- 172.21.15.83:0/3271858631 wait complete. 2024-04-25T00:15:25.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f88400640 1 Processor -- start 2024-04-25T00:15:25.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f88400640 1 -- start start 2024-04-25T00:15:25.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f88400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8006d5d0 0x7f1f80155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f88400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f800cedd0 0x7f1f80159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f88400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f8015be20 0x7f1f80156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f88400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f80169f50 con 0x7f1f8006d5d0 2024-04-25T00:15:25.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f88400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f800b2e70 con 0x7f1f800cedd0 2024-04-25T00:15:25.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f88400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8016a0d0 con 0x7f1f8015be20 2024-04-25T00:15:25.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f85a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8006d5d0 0x7f1f80155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f85a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8006d5d0 0x7f1f80155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57566/0 (socket says 172.21.15.83:57566) 2024-04-25T00:15:25.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f85a00640 1 -- 172.21.15.83:0/2816771213 learned_addr learned my addr 172.21.15.83:0/2816771213 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:25.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f86400640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f8015be20 0x7f1f80156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f85000640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f800cedd0 0x7f1f80159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f85a00640 1 -- 172.21.15.83:0/2816771213 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f8015be20 msgr2=0x7f1f80156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f85a00640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f8015be20 0x7f1f80156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f85a00640 1 -- 172.21.15.83:0/2816771213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f800cedd0 msgr2=0x7f1f80159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f85a00640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f800cedd0 0x7f1f80159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f85a00640 1 -- 172.21.15.83:0/2816771213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8014d940 con 0x7f1f8006d5d0 2024-04-25T00:15:25.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f85000640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f800cedd0 0x7f1f80159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:25.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f86400640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f8015be20 0x7f1f80156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:25.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f85a00640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8006d5d0 0x7f1f80155c10 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f1f740b0040 tx=0x7f1f74019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:25.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.750+0000 7f1f7d600640 1 -- 172.21.15.83:0/2816771213 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f740b2040 con 0x7f1f8006d5d0 2024-04-25T00:15:25.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.754+0000 7f1f7d600640 1 -- 172.21.15.83:0/2816771213 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f740bf070 con 0x7f1f8006d5d0 2024-04-25T00:15:25.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.754+0000 7f1f7d600640 1 -- 172.21.15.83:0/2816771213 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74009450 con 0x7f1f8006d5d0 2024-04-25T00:15:25.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.754+0000 7f1f88400640 1 -- 172.21.15.83:0/2816771213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f801589a0 con 0x7f1f8006d5d0 2024-04-25T00:15:25.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.754+0000 7f1f88400640 1 -- 172.21.15.83:0/2816771213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f80158f60 con 0x7f1f8006d5d0 2024-04-25T00:15:25.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.754+0000 7f1f88400640 1 -- 172.21.15.83:0/2816771213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f80159680 con 0x7f1f8006d5d0 2024-04-25T00:15:25.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.754+0000 7f1f7d600640 1 -- 172.21.15.83:0/2816771213 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1f740adab0 con 0x7f1f8006d5d0 2024-04-25T00:15:25.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.754+0000 7f1f7d600640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1f600678c0 0x7f1f60069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.754+0000 7f1f7d600640 1 -- 172.21.15.83:0/2816771213 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f1f74121090 con 0x7f1f8006d5d0 2024-04-25T00:15:25.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.754+0000 7f1f85000640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1f600678c0 0x7f1f60069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.754+0000 7f1f85000640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1f600678c0 0x7f1f60069d80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1f80156b90 tx=0x7f1f6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:25.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.758+0000 7f1f7d600640 1 -- 172.21.15.83:0/2816771213 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1f74016580 con 0x7f1f8006d5d0 2024-04-25T00:15:25.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.914+0000 7f1f88400640 1 -- 172.21.15.83:0/2816771213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1f8004f510 con 0x7f1f8006d5d0 2024-04-25T00:15:25.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.914+0000 7f1f7d600640 1 -- 172.21.15.83:0/2816771213 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f1f740f1aa0 con 0x7f1f8006d5d0 2024-04-25T00:15:25.916 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:15:25.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.918+0000 7f1f88400640 1 -- 172.21.15.83:0/2816771213 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1f600678c0 msgr2=0x7f1f60069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.918+0000 7f1f88400640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1f600678c0 0x7f1f60069d80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1f80156b90 tx=0x7f1f6c009040 comp rx=0 tx=0).stop 2024-04-25T00:15:25.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.918+0000 7f1f88400640 1 -- 172.21.15.83:0/2816771213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8006d5d0 msgr2=0x7f1f80155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.918+0000 7f1f88400640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8006d5d0 0x7f1f80155c10 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f1f740b0040 tx=0x7f1f74019730 comp rx=0 tx=0).stop 2024-04-25T00:15:25.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.918+0000 7f1f88400640 1 -- 172.21.15.83:0/2816771213 shutdown_connections 2024-04-25T00:15:25.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.918+0000 7f1f88400640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f1f600678c0 0x7f1f60069d80 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.918+0000 7f1f88400640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f8015be20 0x7f1f80156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.918+0000 7f1f88400640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f800cedd0 0x7f1f80159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.918+0000 7f1f88400640 1 --2- 172.21.15.83:0/2816771213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8006d5d0 0x7f1f80155c10 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.918+0000 7f1f88400640 1 -- 172.21.15.83:0/2816771213 >> 172.21.15.83:0/2816771213 conn(0x7f1f800b1b10 msgr2=0x7f1f800b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:25.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.918+0000 7f1f88400640 1 -- 172.21.15.83:0/2816771213 shutdown_connections 2024-04-25T00:15:25.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:25.918+0000 7f1f88400640 1 -- 172.21.15.83:0/2816771213 wait complete. 2024-04-25T00:15:25.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:15:26.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69b9600640 1 Processor -- start 2024-04-25T00:15:26.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69b9600640 1 -- start start 2024-04-25T00:15:26.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69b9600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69b4066f50 0x7f69b4067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69b9600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69b40b3b90 0x7f69b40b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69b9600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69b40ae1b0 con 0x7f69b4158610 2024-04-25T00:15:26.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69b9600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f69b40ae450 con 0x7f69b4066f50 2024-04-25T00:15:26.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69b9600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f69b40ae730 con 0x7f69b40b3b90 2024-04-25T00:15:26.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69ab400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69b4066f50 0x7f69b4067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69ab400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69b4066f50 0x7f69b4067400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57572/0 (socket says 172.21.15.83:57572) 2024-04-25T00:15:26.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69ab400640 1 -- 172.21.15.83:0/3755254401 learned_addr learned my addr 172.21.15.83:0/3755254401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:26.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69b8c00640 1 --2- 172.21.15.83:0/3755254401 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69b40b3b90 0x7f69b40b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69ab400640 1 -- 172.21.15.83:0/3755254401 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69b40b3b90 msgr2=0x7f69b40b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:26.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69ab400640 1 --2- 172.21.15.83:0/3755254401 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69b40b3b90 0x7f69b40b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69ab400640 1 -- 172.21.15.83:0/3755254401 >> v1:172.21.15.86:6789/0 conn(0x7f69b4158610 legacy=0x7f69b40b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:26.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69ab400640 1 -- 172.21.15.83:0/3755254401 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b40aea10 con 0x7f69b4066f50 2024-04-25T00:15:26.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.074+0000 7f69b8c00640 1 --2- 172.21.15.83:0/3755254401 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69b40b3b90 0x7f69b40b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:26.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.078+0000 7f69ab400640 1 --2- 172.21.15.83:0/3755254401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b4066f50 0x7f69b4067400 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f699c009870 tx=0x7f699c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9a684b5c043d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:26.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.078+0000 7f69b3400640 1 -- 172.21.15.83:0/3755254401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c0b2070 con 0x7f69b4066f50 2024-04-25T00:15:26.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.078+0000 7f69b3400640 1 -- 172.21.15.83:0/3755254401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f699c0ad640 con 0x7f69b4066f50 2024-04-25T00:15:26.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.078+0000 7f69b3400640 1 -- 172.21.15.83:0/3755254401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c0ada10 con 0x7f69b4066f50 2024-04-25T00:15:26.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.078+0000 7f69b9600640 1 -- 172.21.15.83:0/3755254401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b4066f50 msgr2=0x7f69b4067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:26.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.078+0000 7f69b9600640 1 --2- 172.21.15.83:0/3755254401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b4066f50 0x7f69b4067400 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f699c009870 tx=0x7f699c01c4e0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.078+0000 7f69b9600640 1 -- 172.21.15.83:0/3755254401 shutdown_connections 2024-04-25T00:15:26.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.078+0000 7f69b9600640 1 --2- 172.21.15.83:0/3755254401 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69b40b3b90 0x7f69b40b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.078+0000 7f69b9600640 1 --2- 172.21.15.83:0/3755254401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b4066f50 0x7f69b4067400 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.078+0000 7f69b9600640 1 -- 172.21.15.83:0/3755254401 >> 172.21.15.83:0/3755254401 conn(0x7f69b4072b60 msgr2=0x7f69b406b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:26.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.078+0000 7f69b9600640 1 -- 172.21.15.83:0/3755254401 shutdown_connections 2024-04-25T00:15:26.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.078+0000 7f69b9600640 1 -- 172.21.15.83:0/3755254401 wait complete. 2024-04-25T00:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b9600640 1 Processor -- start 2024-04-25T00:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b9600640 1 -- start start 2024-04-25T00:15:26.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b9600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69b4066f50 0x7f69b41589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b40b3b90 0x7f69b415e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b4158610 0x7f69b4146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b9600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f69b40b57e0 con 0x7f69b40b3b90 2024-04-25T00:15:26.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b9600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69b40b5660 con 0x7f69b4158610 2024-04-25T00:15:26.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b9600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f69b416aba0 con 0x7f69b4066f50 2024-04-25T00:15:26.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69ab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b40b3b90 0x7f69b415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69ab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b40b3b90 0x7f69b415e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57576/0 (socket says 172.21.15.83:57576) 2024-04-25T00:15:26.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69ab400640 1 -- 172.21.15.83:0/601860131 learned_addr learned my addr 172.21.15.83:0/601860131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:26.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b3e00640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69b4066f50 0x7f69b41589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b8c00640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b4158610 0x7f69b4146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69ab400640 1 -- 172.21.15.83:0/601860131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69b4066f50 msgr2=0x7f69b41589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:26.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69ab400640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69b4066f50 0x7f69b41589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69ab400640 1 -- 172.21.15.83:0/601860131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b4158610 msgr2=0x7f69b4146b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:26.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69ab400640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b4158610 0x7f69b4146b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69ab400640 1 -- 172.21.15.83:0/601860131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b415aa20 con 0x7f69b40b3b90 2024-04-25T00:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b3e00640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69b4066f50 0x7f69b41589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69ab400640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b40b3b90 0x7f69b415e910 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f699c01ab90 tx=0x7f699c019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b0c00640 1 -- 172.21.15.83:0/601860131 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c0bd070 con 0x7f69b40b3b90 2024-04-25T00:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b0c00640 1 -- 172.21.15.83:0/601860131 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f699c0b6750 con 0x7f69b40b3b90 2024-04-25T00:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b0c00640 1 -- 172.21.15.83:0/601860131 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c0b2070 con 0x7f69b40b3b90 2024-04-25T00:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b9600640 1 -- 172.21.15.83:0/601860131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b415ad00 con 0x7f69b40b3b90 2024-04-25T00:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b9600640 1 -- 172.21.15.83:0/601860131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b415a6d0 con 0x7f69b40b3b90 2024-04-25T00:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.082+0000 7f69b9600640 1 -- 172.21.15.83:0/601860131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b404f510 con 0x7f69b40b3b90 2024-04-25T00:15:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.086+0000 7f69b0c00640 1 -- 172.21.15.83:0/601860131 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f699c0b6b40 con 0x7f69b40b3b90 2024-04-25T00:15:26.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.086+0000 7f69b0c00640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69900678c0 0x7f6990069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.086+0000 7f69b0c00640 1 -- 172.21.15.83:0/601860131 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f699c11fcf0 con 0x7f69b40b3b90 2024-04-25T00:15:26.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.086+0000 7f69b3e00640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69900678c0 0x7f6990069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.086+0000 7f69b3e00640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69900678c0 0x7f6990069d80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f69b4154600 tx=0x7f69a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:26.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.090+0000 7f69b0c00640 1 -- 172.21.15.83:0/601860131 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f699c0160d0 con 0x7f69b40b3b90 2024-04-25T00:15:26.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.258+0000 7f69b9600640 1 -- 172.21.15.83:0/601860131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f69b406cd50 con 0x7f69b40b3b90 2024-04-25T00:15:26.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.258+0000 7f69b0c00640 1 -- 172.21.15.83:0/601860131 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v97) ==== 80+0+244 (secure 0 0 0) 0x7f699c0eff10 con 0x7f69b40b3b90 2024-04-25T00:15:26.258 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6837/1609715757 2024-04-26T00:15:04.845080+0000 2024-04-25T00:15:26.258 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6836/1609715757 2024-04-26T00:15:04.845080+0000 2024-04-25T00:15:26.258 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.86:6835/3612679148 2024-04-26T00:15:04.826082+0000 2024-04-25T00:15:26.259 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.86:6833/3612679148 2024-04-26T00:15:04.826082+0000 2024-04-25T00:15:26.259 INFO:teuthology.orchestra.run.smithi083.stderr:listed 4 entries 2024-04-25T00:15:26.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.258+0000 7f69b9600640 1 -- 172.21.15.83:0/601860131 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69900678c0 msgr2=0x7f6990069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:26.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.258+0000 7f69b9600640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69900678c0 0x7f6990069d80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f69b4154600 tx=0x7f69a4008040 comp rx=0 tx=0).stop 2024-04-25T00:15:26.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.258+0000 7f69b9600640 1 -- 172.21.15.83:0/601860131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b40b3b90 msgr2=0x7f69b415e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:26.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.258+0000 7f69b9600640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b40b3b90 0x7f69b415e910 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f699c01ab90 tx=0x7f699c019280 comp rx=0 tx=0).stop 2024-04-25T00:15:26.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.258+0000 7f69b9600640 1 -- 172.21.15.83:0/601860131 shutdown_connections 2024-04-25T00:15:26.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.258+0000 7f69b9600640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69900678c0 0x7f6990069d80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.258+0000 7f69b9600640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b4158610 0x7f69b4146b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.258+0000 7f69b9600640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b40b3b90 0x7f69b415e910 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.258+0000 7f69b9600640 1 --2- 172.21.15.83:0/601860131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69b4066f50 0x7f69b41589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.258+0000 7f69b9600640 1 -- 172.21.15.83:0/601860131 >> 172.21.15.83:0/601860131 conn(0x7f69b4072b60 msgr2=0x7f69b4156170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:26.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.258+0000 7f69b9600640 1 -- 172.21.15.83:0/601860131 shutdown_connections 2024-04-25T00:15:26.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.258+0000 7f69b9600640 1 -- 172.21.15.83:0/601860131 wait complete. 2024-04-25T00:15:26.279 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T00:15:26.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:26.279 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T00:15:26.313 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:15:26.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:26.314 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:15:26.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T00:15:26.524 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-25T00:15:26.525 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-04-25T00:15:26.525 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T00:15:26.688 INFO:teuthology.orchestra.run.smithi083.stdout:{"message":"","return_code":0} 2024-04-25T00:15:26.689 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T00:15:26.689 DEBUG:teuthology.orchestra.run.smithi083:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show session 2024-04-25T00:15:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.730+0000 7f56370a8400 1 -- start start 2024-04-25T00:15:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.730+0000 7f56370a8400 1 Processor -- start 2024-04-25T00:15:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.730+0000 7f56370a8400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954a6ef800 0x55954a8a8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.730+0000 7f56370a8400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954a6ef400 0x55954a8a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.730+0000 7f56370a8400 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954a6ef000 0x55954a8a9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.730+0000 7f56370a8400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55954b4be180 con 0x55954a6ef400 2024-04-25T00:15:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.730+0000 7f56370a8400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55954b4be300 con 0x55954a6ef800 2024-04-25T00:15:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.730+0000 7f56370a8400 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55954b4be480 con 0x55954a6ef000 2024-04-25T00:15:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.730+0000 7f5634800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954a6ef400 0x55954a8a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.730+0000 7f5634800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954a6ef400 0x55954a8a8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57592/0 (socket says 172.21.15.83:57592) 2024-04-25T00:15:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.730+0000 7f5634800640 1 -- 172.21.15.83:0/1592585243 learned_addr learned my addr 172.21.15.83:0/1592585243 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5635200640 1 --2- 172.21.15.83:0/1592585243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954a6ef800 0x55954a8a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5635c00640 1 --2- 172.21.15.83:0/1592585243 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954a6ef000 0x55954a8a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5634800640 1 -- 172.21.15.83:0/1592585243 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954a6ef000 msgr2=0x55954a8a9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5634800640 1 --2- 172.21.15.83:0/1592585243 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954a6ef000 0x55954a8a9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5634800640 1 -- 172.21.15.83:0/1592585243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954a6ef800 msgr2=0x55954a8a8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5634800640 1 --2- 172.21.15.83:0/1592585243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954a6ef800 0x55954a8a8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5634800640 1 -- 172.21.15.83:0/1592585243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55954b4aa960 con 0x55954a6ef400 2024-04-25T00:15:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5635c00640 1 --2- 172.21.15.83:0/1592585243 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954a6ef000 0x55954a8a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5635200640 1 --2- 172.21.15.83:0/1592585243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954a6ef800 0x55954a8a8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5634800640 1 --2- 172.21.15.83:0/1592585243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954a6ef400 0x55954a8a8b00 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x55954b5549c0 tx=0x55954a7b0960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4e2b2d22b2df85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5632000640 1 -- 172.21.15.83:0/1592585243 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55954a7249c0 con 0x55954a6ef400 2024-04-25T00:15:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5632000640 1 -- 172.21.15.83:0/1592585243 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55954a738e00 con 0x55954a6ef400 2024-04-25T00:15:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5632000640 1 -- 172.21.15.83:0/1592585243 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55954a7244e0 con 0x55954a6ef400 2024-04-25T00:15:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f56370a8400 1 -- 172.21.15.83:0/1592585243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55954b4aad20 con 0x55954a6ef400 2024-04-25T00:15:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f56370a8400 1 -- 172.21.15.83:0/1592585243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55954b4aaf00 con 0x55954a6ef400 2024-04-25T00:15:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f56370a8400 4 waiting for MDS map... 2024-04-25T00:15:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5632000640 1 -- 172.21.15.83:0/1592585243 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x55954b5fa200 con 0x55954a6ef400 2024-04-25T00:15:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5632000640 1 -- 172.21.15.83:0/1592585243 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 74) ==== 3518+0+0 (secure 0 0 0) 0x55954a745180 con 0x55954a6ef400 2024-04-25T00:15:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f56370a8400 4 Got MDS map 74 2024-04-25T00:15:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f56370a8400 10 main: main 2024-04-25T00:15:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f56370a8400 4 main: connecting to RADOS... 2024-04-25T00:15:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f56370a8400 1 Processor -- start 2024-04-25T00:15:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f56370a8400 1 -- start start 2024-04-25T00:15:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f56370a8400 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954b4c7c00 0x55954a8a9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f56370a8400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954b662000 0x55954a8a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f56370a8400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954b662400 0x55954a8aa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f56370a8400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55954b4be600 con 0x55954b662000 2024-04-25T00:15:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f56370a8400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55954b4be780 con 0x55954b662400 2024-04-25T00:15:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f56370a8400 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55954b4be900 con 0x55954b4c7c00 2024-04-25T00:15:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5635200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954b662000 0x55954a8a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5635200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954b662000 0x55954a8a9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57606/0 (socket says 172.21.15.83:57606) 2024-04-25T00:15:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5635200640 1 -- 172.21.15.83:0/700059175 learned_addr learned my addr 172.21.15.83:0/700059175 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5635c00640 1 --2- 172.21.15.83:0/700059175 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954b4c7c00 0x55954a8a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5634800640 1 --2- 172.21.15.83:0/700059175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954b662400 0x55954a8aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5635200640 1 -- 172.21.15.83:0/700059175 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954b4c7c00 msgr2=0x55954a8a9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5635200640 1 --2- 172.21.15.83:0/700059175 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954b4c7c00 0x55954a8a9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5635200640 1 -- 172.21.15.83:0/700059175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954b662400 msgr2=0x55954a8aa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5635200640 1 --2- 172.21.15.83:0/700059175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954b662400 0x55954a8aa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5635200640 1 -- 172.21.15.83:0/700059175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55954b4aa1e0 con 0x55954b662000 2024-04-25T00:15:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f5635200640 1 --2- 172.21.15.83:0/700059175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954b662000 0x55954a8a9b80 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x55954b4ba210 tx=0x55954b612cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c03ce03eb985de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f562f800640 1 -- 172.21.15.83:0/700059175 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55954a7249c0 con 0x55954b662000 2024-04-25T00:15:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f562f800640 1 -- 172.21.15.83:0/700059175 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55954a738540 con 0x55954b662000 2024-04-25T00:15:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.734+0000 7f562f800640 1 -- 172.21.15.83:0/700059175 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55954a7244e0 con 0x55954b662000 2024-04-25T00:15:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 -- 172.21.15.83:0/700059175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954b662000 msgr2=0x55954a8a9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 --2- 172.21.15.83:0/700059175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954b662000 0x55954a8a9b80 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x55954b4ba210 tx=0x55954b612cc0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 -- 172.21.15.83:0/700059175 shutdown_connections 2024-04-25T00:15:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 --2- 172.21.15.83:0/700059175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954b662400 0x55954a8aa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 --2- 172.21.15.83:0/700059175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954b662000 0x55954a8a9b80 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 --2- 172.21.15.83:0/700059175 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954b4c7c00 0x55954a8a9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 -- 172.21.15.83:0/700059175 >> 172.21.15.83:0/700059175 conn(0x55954a6eec00 msgr2=0x55954a71d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 -- 172.21.15.83:0/700059175 shutdown_connections 2024-04-25T00:15:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 -- 172.21.15.83:0/700059175 wait complete. 2024-04-25T00:15:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 Processor -- start 2024-04-25T00:15:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 -- start start 2024-04-25T00:15:26.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954b4c7c00 0x55954a8a9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954b662400 0x55954a8aa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954b662800 0x55954a8aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55954b4bf980 con 0x55954b662400 2024-04-25T00:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55954b4be300 con 0x55954b662800 2024-04-25T00:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55954b4bec00 con 0x55954b4c7c00 2024-04-25T00:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f5635200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954b662400 0x55954a8aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f5635200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954b662400 0x55954a8aa100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57610/0 (socket says 172.21.15.83:57610) 2024-04-25T00:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f5635200640 1 -- 172.21.15.83:0/3609954916 learned_addr learned my addr 172.21.15.83:0/3609954916 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f5634800640 1 --2- 172.21.15.83:0/3609954916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954b662800 0x55954a8aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f5635c00640 1 --2- 172.21.15.83:0/3609954916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954b4c7c00 0x55954a8a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f5635200640 1 -- 172.21.15.83:0/3609954916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954b4c7c00 msgr2=0x55954a8a9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f5635200640 1 --2- 172.21.15.83:0/3609954916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954b4c7c00 0x55954a8a9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f5635200640 1 -- 172.21.15.83:0/3609954916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954b662800 msgr2=0x55954a8aa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f5635200640 1 --2- 172.21.15.83:0/3609954916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954b662800 0x55954a8aa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f5635200640 1 -- 172.21.15.83:0/3609954916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55954b4aba40 con 0x55954b662400 2024-04-25T00:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f5635200640 1 --2- 172.21.15.83:0/3609954916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954b662400 0x55954a8aa100 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x55954b4bb2c0 tx=0x55954b612c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f562da00640 1 -- 172.21.15.83:0/3609954916 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55954a724000 con 0x55954b662400 2024-04-25T00:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f562da00640 1 -- 172.21.15.83:0/3609954916 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55954a738a80 con 0x55954b662400 2024-04-25T00:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 -- 172.21.15.83:0/3609954916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55954b4abc20 con 0x55954b662400 2024-04-25T00:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f562da00640 1 -- 172.21.15.83:0/3609954916 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55954a7249c0 con 0x55954b662400 2024-04-25T00:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 1 -- 172.21.15.83:0/3609954916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55954b4aa1e0 con 0x55954b662400 2024-04-25T00:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.738+0000 7f56370a8400 4 main: resolving pool 15 2024-04-25T00:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f562da00640 1 -- 172.21.15.83:0/3609954916 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x55954b660c00 con 0x55954b662400 2024-04-25T00:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f562da00640 1 --2- 172.21.15.83:0/3609954916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x55954aa27400 0x55954a8ab700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f562da00640 1 -- 172.21.15.83:0/3609954916 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x55954b66e400 con 0x55954b662400 2024-04-25T00:15:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 4 main: creating IoCtx.. 2024-04-25T00:15:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f5635c00640 1 --2- 172.21.15.83:0/3609954916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x55954aa27400 0x55954a8ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 --2- 172.21.15.83:0/3609954916 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x55954b662c00 0x55954a8aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 -- 172.21.15.83:0/3609954916 --> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] -- osd_op(unknown.0.0:1 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55954b663000 con 0x55954b662c00 2024-04-25T00:15:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f5634800640 1 --2- 172.21.15.83:0/3609954916 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x55954b662c00 0x55954a8aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f5635c00640 1 --2- 172.21.15.83:0/3609954916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x55954aa27400 0x55954a8ab700 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55954a7de6f0 tx=0x55954ab14000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f5634800640 1 --2- 172.21.15.83:0/3609954916 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x55954b662c00 0x55954a8aac00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f5634800640 1 -- 172.21.15.83:0/3609954916 <== osd.2 v2:172.21.15.83:6824/1702952090 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv10 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x55954a7a4000 con 0x55954b662c00 2024-04-25T00:15:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 -- 172.21.15.83:0/3609954916 --> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] -- osd_op(unknown.0.0:2 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55954b663400 con 0x55954b662c00 2024-04-25T00:15:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f5634800640 1 -- 172.21.15.83:0/3609954916 <== osd.2 v2:172.21.15.83:6824/1702952090 2 ==== osd_op_reply(2 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv10 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x55954a7a4000 con 0x55954b662c00 2024-04-25T00:15:26.749 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:15:26.750 INFO:teuthology.orchestra.run.smithi083.stdout: "0": { 2024-04-25T00:15:26.750 INFO:teuthology.orchestra.run.smithi083.stdout: "data": { 2024-04-25T00:15:26.750 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions": [] 2024-04-25T00:15:26.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T00:15:26.750 INFO:teuthology.orchestra.run.smithi083.stdout: "result": 0 2024-04-25T00:15:26.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T00:15:26.750 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:15:26.750 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 -- 172.21.15.83:0/3609954916 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x55954b662c00 msgr2=0x55954a8aac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 --2- 172.21.15.83:0/3609954916 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x55954b662c00 0x55954a8aac00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 -- 172.21.15.83:0/3609954916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x55954aa27400 msgr2=0x55954a8ab700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 --2- 172.21.15.83:0/3609954916 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x55954aa27400 0x55954a8ab700 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55954a7de6f0 tx=0x55954ab14000 comp rx=0 tx=0).stop 2024-04-25T00:15:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 -- 172.21.15.83:0/3609954916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954b662400 msgr2=0x55954a8aa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 --2- 172.21.15.83:0/3609954916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954b662400 0x55954a8aa100 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x55954b4bb2c0 tx=0x55954b612c00 comp rx=0 tx=0).stop 2024-04-25T00:15:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f5635200640 1 -- 172.21.15.83:0/3609954916 reap_dead start 2024-04-25T00:15:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 -- 172.21.15.83:0/3609954916 shutdown_connections 2024-04-25T00:15:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 -- 172.21.15.83:0/3609954916 >> 172.21.15.83:0/3609954916 conn(0x55954a6eec00 msgr2=0x55954a71d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 -- 172.21.15.83:0/3609954916 shutdown_connections 2024-04-25T00:15:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 -- 172.21.15.83:0/3609954916 wait complete. 2024-04-25T00:15:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 -- 172.21.15.83:0/1592585243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954a6ef400 msgr2=0x55954a8a8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 --2- 172.21.15.83:0/1592585243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954a6ef400 0x55954a8a8b00 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x55954b5549c0 tx=0x55954a7b0960 comp rx=0 tx=0).stop 2024-04-25T00:15:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 -- 172.21.15.83:0/1592585243 shutdown_connections 2024-04-25T00:15:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 --2- 172.21.15.83:0/1592585243 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55954a6ef000 0x55954a8a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 --2- 172.21.15.83:0/1592585243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55954a6ef400 0x55954a8a8b00 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 --2- 172.21.15.83:0/1592585243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55954a6ef800 0x55954a8a8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 -- 172.21.15.83:0/1592585243 >> 172.21.15.83:0/1592585243 conn(0x55954a6efc00 msgr2=0x55954a71d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 -- 172.21.15.83:0/1592585243 shutdown_connections 2024-04-25T00:15:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:26.742+0000 7f56370a8400 1 -- 172.21.15.83:0/1592585243 wait complete. 2024-04-25T00:15:26.754 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'session'] in time 0:00:00.064681, result: { "0": { "data": { "sessions": [] }, "result": 0 } } 2024-04-25T00:15:26.754 INFO:tasks.cephfs.test_sessionmap:SessionMap: { "0": { "data": { "sessions": [] }, "result": 0 } } 2024-04-25T00:15:26.754 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:15:26.754 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:15:26.888 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:15:26.956 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:15:26.956 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:15:26.956 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:15:26.956 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:26.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:15:26.965 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:15:26.965 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:15:26.965 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:26.974 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:15:26.974 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:15:26.975 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:15:26.975 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:27.017 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:15:27.017 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:15:27.017 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:15:27.017 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:27.026 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:15:27.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:27.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a89600640 1 Processor -- start 2024-04-25T00:15:27.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a89600640 1 -- start start 2024-04-25T00:15:27.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a89600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3a84156340 0x7f3a840acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a89600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a8415ce00 0x7f3a84070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a89600640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3a840ae680 0x7f3a840aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a89600640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3a8406ecf0 con 0x7f3a840ae680 2024-04-25T00:15:27.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a89600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3a8406efd0 con 0x7f3a84156340 2024-04-25T00:15:27.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a89600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8406cef0 con 0x7f3a8415ce00 2024-04-25T00:15:27.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a83400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3a840ae680 0x7f3a840aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a83400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3a840ae680 0x7f3a840aead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57622/0 (socket says 172.21.15.83:57622) 2024-04-25T00:15:27.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a83400640 1 -- 172.21.15.83:0/2679232351 learned_addr learned my addr 172.21.15.83:0/2679232351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:27.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a82a00640 1 --2- 172.21.15.83:0/2679232351 >> v2:172.21.15.86:3300/0 conn(0x7f3a84156340 0x7f3a840acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a82000640 1 --2- 172.21.15.83:0/2679232351 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a8415ce00 0x7f3a84070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a83400640 1 -- 172.21.15.83:0/2679232351 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a8415ce00 msgr2=0x7f3a84070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:27.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a83400640 1 --2- 172.21.15.83:0/2679232351 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a8415ce00 0x7f3a84070e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a83400640 1 -- 172.21.15.83:0/2679232351 >> v2:172.21.15.86:3300/0 conn(0x7f3a84156340 msgr2=0x7f3a840acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:27.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a83400640 1 --2- 172.21.15.83:0/2679232351 >> v2:172.21.15.86:3300/0 conn(0x7f3a84156340 0x7f3a840acd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.194+0000 7f3a83400640 1 -- 172.21.15.83:0/2679232351 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8415d290 con 0x7f3a840ae680 2024-04-25T00:15:27.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.198+0000 7f3a83400640 1 --2- 172.21.15.83:0/2679232351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a840ae680 0x7f3a840aead0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3a840b24a0 tx=0x7f3a7c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cdb34ac8384a51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:27.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.198+0000 7f3a81600640 1 -- 172.21.15.83:0/2679232351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0b0070 con 0x7f3a840ae680 2024-04-25T00:15:27.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.198+0000 7f3a81600640 1 -- 172.21.15.83:0/2679232351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a7c0ab600 con 0x7f3a840ae680 2024-04-25T00:15:27.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.198+0000 7f3a81600640 1 -- 172.21.15.83:0/2679232351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0ab9d0 con 0x7f3a840ae680 2024-04-25T00:15:27.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.198+0000 7f3a89600640 1 -- 172.21.15.83:0/2679232351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a840ae680 msgr2=0x7f3a840aead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.198+0000 7f3a89600640 1 --2- 172.21.15.83:0/2679232351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a840ae680 0x7f3a840aead0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3a840b24a0 tx=0x7f3a7c01c520 comp rx=0 tx=0).stop 2024-04-25T00:15:27.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.198+0000 7f3a89600640 1 -- 172.21.15.83:0/2679232351 shutdown_connections 2024-04-25T00:15:27.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.198+0000 7f3a89600640 1 --2- 172.21.15.83:0/2679232351 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a8415ce00 0x7f3a84070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.198+0000 7f3a89600640 1 --2- 172.21.15.83:0/2679232351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a840ae680 0x7f3a840aead0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.198+0000 7f3a89600640 1 --2- 172.21.15.83:0/2679232351 >> v2:172.21.15.86:3300/0 conn(0x7f3a84156340 0x7f3a840acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.198+0000 7f3a89600640 1 -- 172.21.15.83:0/2679232351 >> 172.21.15.83:0/2679232351 conn(0x7f3a840b1860 msgr2=0x7f3a840b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:27.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.198+0000 7f3a89600640 1 -- 172.21.15.83:0/2679232351 shutdown_connections 2024-04-25T00:15:27.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.198+0000 7f3a89600640 1 -- 172.21.15.83:0/2679232351 wait complete. 2024-04-25T00:15:27.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a89600640 1 Processor -- start 2024-04-25T00:15:27.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a89600640 1 -- start start 2024-04-25T00:15:27.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a89600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a840ae680 0x7f3a8413d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a89600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a84156340 0x7f3a8415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a89600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a841414d0 0x7f3a84141980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a89600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8406dda0 con 0x7f3a840ae680 2024-04-25T00:15:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a89600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8406dc20 con 0x7f3a841414d0 2024-04-25T00:15:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a89600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8406df20 con 0x7f3a84156340 2024-04-25T00:15:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a82a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a840ae680 0x7f3a8413d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a82a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a840ae680 0x7f3a8413d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57634/0 (socket says 172.21.15.83:57634) 2024-04-25T00:15:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a82a00640 1 -- 172.21.15.83:0/800835459 learned_addr learned my addr 172.21.15.83:0/800835459 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a83400640 1 --2- 172.21.15.83:0/800835459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a841414d0 0x7f3a84141980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a82000640 1 --2- 172.21.15.83:0/800835459 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a84156340 0x7f3a8415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a82a00640 1 -- 172.21.15.83:0/800835459 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a84156340 msgr2=0x7f3a8415ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a82a00640 1 --2- 172.21.15.83:0/800835459 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a84156340 0x7f3a8415ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a82a00640 1 -- 172.21.15.83:0/800835459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a841414d0 msgr2=0x7f3a84141980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a82a00640 1 --2- 172.21.15.83:0/800835459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a841414d0 0x7f3a84141980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a82a00640 1 -- 172.21.15.83:0/800835459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8413dab0 con 0x7f3a840ae680 2024-04-25T00:15:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a82a00640 1 --2- 172.21.15.83:0/800835459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a840ae680 0x7f3a8413d570 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f3a74007ef0 tx=0x7f3a7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a7aa00640 1 -- 172.21.15.83:0/800835459 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74014620 con 0x7f3a840ae680 2024-04-25T00:15:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a7aa00640 1 -- 172.21.15.83:0/800835459 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a74019070 con 0x7f3a840ae680 2024-04-25T00:15:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a89600640 1 -- 172.21.15.83:0/800835459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8415df90 con 0x7f3a840ae680 2024-04-25T00:15:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a7aa00640 1 -- 172.21.15.83:0/800835459 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74014a00 con 0x7f3a840ae680 2024-04-25T00:15:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a89600640 1 -- 172.21.15.83:0/800835459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8415e550 con 0x7f3a840ae680 2024-04-25T00:15:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.202+0000 7f3a89600640 1 -- 172.21.15.83:0/800835459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a8404f510 con 0x7f3a840ae680 2024-04-25T00:15:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.206+0000 7f3a7aa00640 1 -- 172.21.15.83:0/800835459 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3a74014ba0 con 0x7f3a840ae680 2024-04-25T00:15:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.206+0000 7f3a7aa00640 1 --2- 172.21.15.83:0/800835459 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3a640678f0 0x7f3a64069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.206+0000 7f3a7aa00640 1 -- 172.21.15.83:0/800835459 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f3a74086880 con 0x7f3a840ae680 2024-04-25T00:15:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.206+0000 7f3a82000640 1 --2- 172.21.15.83:0/800835459 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3a640678f0 0x7f3a64069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.206+0000 7f3a82000640 1 --2- 172.21.15.83:0/800835459 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3a640678f0 0x7f3a64069db0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3a841f8530 tx=0x7f3a70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:27.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.210+0000 7f3a7aa00640 1 -- 172.21.15.83:0/800835459 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3a8413dab0 con 0x7f3a840ae680 2024-04-25T00:15:27.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.402+0000 7f3a89600640 1 -- 172.21.15.83:0/800835459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a84070f70 con 0x7f3a840ae680 2024-04-25T00:15:27.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.402+0000 7f3a7aa00640 1 -- 172.21.15.83:0/800835459 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5201 (secure 0 0 0) 0x7f3a740570e0 con 0x7f3a840ae680 2024-04-25T00:15:27.402 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:27.402 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1992401755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1992401755},{"type":"v1","addr":"172.21.15.86:6833","nonce":1992401755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3621275047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3621275047},{"type":"v1","addr":"172.21.15.86:6835","nonce":3621275047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:51.879542+0000","modified":"2024-04-25T00:15:20.022793+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7719,"mds_1":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7719":{"gid":7719,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6835/565131724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":565131724},{"type":"v1","addr":"172.21.15.83:6835","nonce":565131724}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7728":{"gid":7728,"name":"a","rank":1,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2870595352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2870595352},{"type":"v1","addr":"172.21.15.83:6837","nonce":2870595352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7719,"qdb_cluster":[7728,7719]},"id":7}]} 2024-04-25T00:15:27.403 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 74 2024-04-25T00:15:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.402+0000 7f3a89600640 1 -- 172.21.15.83:0/800835459 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3a640678f0 msgr2=0x7f3a64069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.402+0000 7f3a89600640 1 --2- 172.21.15.83:0/800835459 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3a640678f0 0x7f3a64069db0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3a841f8530 tx=0x7f3a70008040 comp rx=0 tx=0).stop 2024-04-25T00:15:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.402+0000 7f3a89600640 1 -- 172.21.15.83:0/800835459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a840ae680 msgr2=0x7f3a8413d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.402+0000 7f3a89600640 1 --2- 172.21.15.83:0/800835459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a840ae680 0x7f3a8413d570 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f3a74007ef0 tx=0x7f3a7400e4b0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.402+0000 7f3a89600640 1 -- 172.21.15.83:0/800835459 shutdown_connections 2024-04-25T00:15:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.402+0000 7f3a89600640 1 --2- 172.21.15.83:0/800835459 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3a640678f0 0x7f3a64069db0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.402+0000 7f3a89600640 1 --2- 172.21.15.83:0/800835459 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a84156340 0x7f3a8415ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.402+0000 7f3a89600640 1 --2- 172.21.15.83:0/800835459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a841414d0 0x7f3a84141980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.402+0000 7f3a89600640 1 --2- 172.21.15.83:0/800835459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a840ae680 0x7f3a8413d570 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.402+0000 7f3a89600640 1 -- 172.21.15.83:0/800835459 >> 172.21.15.83:0/800835459 conn(0x7f3a840b1860 msgr2=0x7f3a84149a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.402+0000 7f3a89600640 1 -- 172.21.15.83:0/800835459 shutdown_connections 2024-04-25T00:15:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.402+0000 7f3a89600640 1 -- 172.21.15.83:0/800835459 wait complete. 2024-04-25T00:15:27.424 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:15:27.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.558+0000 7fcfff200640 1 Processor -- start 2024-04-25T00:15:27.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.558+0000 7fcfff200640 1 -- start start 2024-04-25T00:15:27.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcfff200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcff8155470 0x7fcff80ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcfff200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcff80adb00 0x7fcff806e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcfff200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcff80ae0a0 con 0x7fcff806ebc0 2024-04-25T00:15:27.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcfff200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcff80ae380 con 0x7fcff80adb00 2024-04-25T00:15:27.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcfff200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcff80ae660 con 0x7fcff8155470 2024-04-25T00:15:27.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcffd200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcff806ebc0 0x7fcff815e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37776/0 (socket says 172.21.15.83:37776) 2024-04-25T00:15:27.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcffd200640 1 -- 172.21.15.83:0/1526542309 learned_addr learned my addr 172.21.15.83:0/1526542309 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:27.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcfef400640 1 --2- 172.21.15.83:0/1526542309 >> v2:172.21.15.86:3300/0 conn(0x7fcff80adb00 0x7fcff806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7e00640 1 --2- 172.21.15.83:0/1526542309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcff8155470 0x7fcff80ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 -- 172.21.15.83:0/1526542309 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 42279933 0 0) 0x7fcff80ae0a0 con 0x7fcff806ebc0 2024-04-25T00:15:27.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 -- 172.21.15.83:0/1526542309 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfdc003530 con 0x7fcff806ebc0 2024-04-25T00:15:27.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 -- 172.21.15.83:0/1526542309 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3881003515 0 0) 0x7fcfdc003530 con 0x7fcff806ebc0 2024-04-25T00:15:27.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 -- 172.21.15.83:0/1526542309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcff8155470 msgr2=0x7fcff80ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:27.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 --2- 172.21.15.83:0/1526542309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcff8155470 0x7fcff80ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 -- 172.21.15.83:0/1526542309 >> v2:172.21.15.86:3300/0 conn(0x7fcff80adb00 msgr2=0x7fcff806e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:27.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 --2- 172.21.15.83:0/1526542309 >> v2:172.21.15.86:3300/0 conn(0x7fcff80adb00 0x7fcff806e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 -- 172.21.15.83:0/1526542309 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff80ae940 con 0x7fcff806ebc0 2024-04-25T00:15:27.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 -- 172.21.15.83:0/1526542309 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fcff0001680 con 0x7fcff806ebc0 2024-04-25T00:15:27.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcfef400640 1 --2- 172.21.15.83:0/1526542309 >> v2:172.21.15.86:3300/0 conn(0x7fcff80adb00 0x7fcff806e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:27.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 -- 172.21.15.83:0/1526542309 >> v1:172.21.15.83:6789/0 conn(0x7fcff806ebc0 legacy=0x7fcff815e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 --2- 172.21.15.83:0/1526542309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0039c0 0x7fcfdc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 --2- 172.21.15.83:0/1526542309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcff8155470 0x7fcfdc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 --2- 172.21.15.83:0/1526542309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfdc004d50 0x7fcfdc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 -- 172.21.15.83:0/1526542309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcff80ae380 con 0x7fcfdc0039c0 2024-04-25T00:15:27.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 -- 172.21.15.83:0/1526542309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcff80ae660 con 0x7fcfdc004d50 2024-04-25T00:15:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7400640 1 -- 172.21.15.83:0/1526542309 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcfdc00b9d0 con 0x7fcff8155470 2024-04-25T00:15:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7e00640 1 --2- 172.21.15.83:0/1526542309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0039c0 0x7fcfdc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcfef400640 1 --2- 172.21.15.83:0/1526542309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcff8155470 0x7fcfdc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcffd200640 1 --2- 172.21.15.83:0/1526542309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfdc004d50 0x7fcfdc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7e00640 1 -- 172.21.15.83:0/1526542309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcff8155470 msgr2=0x7fcfdc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7e00640 1 --2- 172.21.15.83:0/1526542309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcff8155470 0x7fcfdc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7e00640 1 -- 172.21.15.83:0/1526542309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfdc004d50 msgr2=0x7fcfdc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7e00640 1 --2- 172.21.15.83:0/1526542309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfdc004d50 0x7fcfdc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.562+0000 7fcff7e00640 1 -- 172.21.15.83:0/1526542309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc003530 con 0x7fcfdc0039c0 2024-04-25T00:15:27.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.566+0000 7fcff7e00640 1 --2- 172.21.15.83:0/1526542309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0039c0 0x7fcfdc003de0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009fc0 tx=0x7fcfe801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694a238f5fa26a3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:27.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.566+0000 7fcff7400640 1 -- 172.21.15.83:0/1526542309 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe80b2070 con 0x7fcfdc0039c0 2024-04-25T00:15:27.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.566+0000 7fcff7400640 1 -- 172.21.15.83:0/1526542309 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe80a5b80 con 0x7fcfdc0039c0 2024-04-25T00:15:27.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.566+0000 7fcff7400640 1 -- 172.21.15.83:0/1526542309 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8009c10 con 0x7fcfdc0039c0 2024-04-25T00:15:27.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.566+0000 7fcfff200640 1 -- 172.21.15.83:0/1526542309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0039c0 msgr2=0x7fcfdc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.566+0000 7fcfff200640 1 --2- 172.21.15.83:0/1526542309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0039c0 0x7fcfdc003de0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009fc0 tx=0x7fcfe801cc80 comp rx=0 tx=0).stop 2024-04-25T00:15:27.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.566+0000 7fcffd200640 1 -- 172.21.15.83:0/1526542309 reap_dead start 2024-04-25T00:15:27.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.566+0000 7fcfff200640 1 -- 172.21.15.83:0/1526542309 shutdown_connections 2024-04-25T00:15:27.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.566+0000 7fcfff200640 1 -- 172.21.15.83:0/1526542309 >> 172.21.15.83:0/1526542309 conn(0x7fcff80b1860 msgr2=0x7fcff80b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:27.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.566+0000 7fcfff200640 1 -- 172.21.15.83:0/1526542309 shutdown_connections 2024-04-25T00:15:27.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.566+0000 7fcfff200640 1 -- 172.21.15.83:0/1526542309 wait complete. 2024-04-25T00:15:27.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcfff200640 1 Processor -- start 2024-04-25T00:15:27.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcfff200640 1 -- start start 2024-04-25T00:15:27.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcfff200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0039c0 0x7fcff80b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcfff200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff80b3c60 0x7fcff80ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcfff200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcff81f73a0 0x7fcff81f7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcfff200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8166010 con 0x7fcfdc0039c0 2024-04-25T00:15:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcfff200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8165e90 con 0x7fcff80b3c60 2024-04-25T00:15:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcfff200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcff8166190 con 0x7fcff81f73a0 2024-04-25T00:15:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcff7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0039c0 0x7fcff80b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcff7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0039c0 0x7fcff80b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57652/0 (socket says 172.21.15.83:57652) 2024-04-25T00:15:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcff7e00640 1 -- 172.21.15.83:0/3656441037 learned_addr learned my addr 172.21.15.83:0/3656441037 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcfef400640 1 --2- 172.21.15.83:0/3656441037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff80b3c60 0x7fcff80ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcffd200640 1 --2- 172.21.15.83:0/3656441037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcff81f73a0 0x7fcff81f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcff7e00640 1 -- 172.21.15.83:0/3656441037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcff81f73a0 msgr2=0x7fcff81f7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcff7e00640 1 --2- 172.21.15.83:0/3656441037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcff81f73a0 0x7fcff81f7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcff7e00640 1 -- 172.21.15.83:0/3656441037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff80b3c60 msgr2=0x7fcff80ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcff7e00640 1 --2- 172.21.15.83:0/3656441037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff80b3c60 0x7fcff80ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcff7e00640 1 -- 172.21.15.83:0/3656441037 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff81621b0 con 0x7fcfdc0039c0 2024-04-25T00:15:27.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcff7e00640 1 --2- 172.21.15.83:0/3656441037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0039c0 0x7fcff80b32d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fcfe801c6f0 tx=0x7fcfe8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:27.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcff4c00640 1 -- 172.21.15.83:0/3656441037 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe80ba070 con 0x7fcfdc0039c0 2024-04-25T00:15:27.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcff4c00640 1 -- 172.21.15.83:0/3656441037 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe80ad9c0 con 0x7fcfdc0039c0 2024-04-25T00:15:27.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcff4c00640 1 -- 172.21.15.83:0/3656441037 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe80b2040 con 0x7fcfdc0039c0 2024-04-25T00:15:27.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcfff200640 1 -- 172.21.15.83:0/3656441037 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff80b4e60 con 0x7fcfdc0039c0 2024-04-25T00:15:27.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.570+0000 7fcfff200640 1 -- 172.21.15.83:0/3656441037 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff81f7d90 con 0x7fcfdc0039c0 2024-04-25T00:15:27.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.574+0000 7fcfff200640 1 -- 172.21.15.83:0/3656441037 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff804f510 con 0x7fcfdc0039c0 2024-04-25T00:15:27.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.574+0000 7fcff4c00640 1 -- 172.21.15.83:0/3656441037 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcfe80b6410 con 0x7fcfdc0039c0 2024-04-25T00:15:27.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.574+0000 7fcff4c00640 1 --2- 172.21.15.83:0/3656441037 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcfd40678c0 0x7fcfd4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.574+0000 7fcff4c00640 1 -- 172.21.15.83:0/3656441037 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7fcfe8120400 con 0x7fcfdc0039c0 2024-04-25T00:15:27.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.574+0000 7fcfef400640 1 --2- 172.21.15.83:0/3656441037 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcfd40678c0 0x7fcfd4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.574+0000 7fcfef400640 1 --2- 172.21.15.83:0/3656441037 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcfd40678c0 0x7fcfd4069d80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcff81fc580 tx=0x7fcfe0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:27.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.578+0000 7fcff4c00640 1 -- 172.21.15.83:0/3656441037 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcff81621b0 con 0x7fcfdc0039c0 2024-04-25T00:15:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.734+0000 7fcfff200640 1 -- 172.21.15.83:0/3656441037 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcff8072b50 con 0x7fcfdc0039c0 2024-04-25T00:15:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.734+0000 7fcff4c00640 1 -- 172.21.15.83:0/3656441037 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+17048 (secure 0 0 0) 0x7fcfe8016990 con 0x7fcfdc0039c0 2024-04-25T00:15:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":97,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:15:04.852008+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:50.260735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:50.880394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1609715757":"2024-04-26T00:15:04.845080+0000","172.21.15.83:6836/1609715757":"2024-04-26T00:15:04.845080+0000","172.21.15.86:6835/3612679148":"2024-04-26T00:15:04.826082+0000","172.21.15.86:6833/3612679148":"2024-04-26T00:15:04.826082+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:15:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.738+0000 7fcfff200640 1 -- 172.21.15.83:0/3656441037 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcfd40678c0 msgr2=0x7fcfd4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.738+0000 7fcfff200640 1 --2- 172.21.15.83:0/3656441037 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcfd40678c0 0x7fcfd4069d80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcff81fc580 tx=0x7fcfe0009190 comp rx=0 tx=0).stop 2024-04-25T00:15:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.738+0000 7fcfff200640 1 -- 172.21.15.83:0/3656441037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0039c0 msgr2=0x7fcff80b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.738+0000 7fcfff200640 1 --2- 172.21.15.83:0/3656441037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0039c0 0x7fcff80b32d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fcfe801c6f0 tx=0x7fcfe8009750 comp rx=0 tx=0).stop 2024-04-25T00:15:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.738+0000 7fcfff200640 1 -- 172.21.15.83:0/3656441037 shutdown_connections 2024-04-25T00:15:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.738+0000 7fcfff200640 1 --2- 172.21.15.83:0/3656441037 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fcfd40678c0 0x7fcfd4069d80 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.738+0000 7fcfff200640 1 --2- 172.21.15.83:0/3656441037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcff81f73a0 0x7fcff81f7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.738+0000 7fcfff200640 1 --2- 172.21.15.83:0/3656441037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcff80b3c60 0x7fcff80ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.738+0000 7fcfff200640 1 --2- 172.21.15.83:0/3656441037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0039c0 0x7fcff80b32d0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.738+0000 7fcfff200640 1 -- 172.21.15.83:0/3656441037 >> 172.21.15.83:0/3656441037 conn(0x7fcff80b1860 msgr2=0x7fcff815cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.738+0000 7fcfff200640 1 -- 172.21.15.83:0/3656441037 shutdown_connections 2024-04-25T00:15:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.738+0000 7fcfff200640 1 -- 172.21.15.83:0/3656441037 wait complete. 2024-04-25T00:15:27.757 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:10:54.507055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:14:50.260735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:50.880394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:15:27.758 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:15:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.894+0000 7fc2aaa00640 1 Processor -- start 2024-04-25T00:15:27.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc2aaa00640 1 -- start start 2024-04-25T00:15:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc2aaa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a415be20 0x7fc2a415f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc2aaa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2a40cedd0 0x7fc2a406cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc2aaa00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2a415c200 con 0x7fc2a40acd20 2024-04-25T00:15:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc2aaa00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc2a415fed0 con 0x7fc2a40cedd0 2024-04-25T00:15:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc2aaa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a40b2e70 con 0x7fc2a415be20 2024-04-25T00:15:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc29b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2a40cedd0 0x7fc2a406cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc29b400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2a40cedd0 0x7fc2a406cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57662/0 (socket says 172.21.15.83:57662) 2024-04-25T00:15:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc29b400640 1 -- 172.21.15.83:0/588501360 learned_addr learned my addr 172.21.15.83:0/588501360 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc2a8a00640 1 --1- 172.21.15.83:0/588501360 >> v1:172.21.15.83:6789/0 conn(0x7fc2a40acd20 0x7fc2a40b3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37778/0 (socket says 172.21.15.83:37778) 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc2a3e00640 1 --2- 172.21.15.83:0/588501360 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a415be20 0x7fc2a415f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc29b400640 1 -- 172.21.15.83:0/588501360 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a415be20 msgr2=0x7fc2a415f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc29b400640 1 --2- 172.21.15.83:0/588501360 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a415be20 0x7fc2a415f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc29b400640 1 -- 172.21.15.83:0/588501360 >> v1:172.21.15.83:6789/0 conn(0x7fc2a40acd20 legacy=0x7fc2a40b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.898+0000 7fc29b400640 1 -- 172.21.15.83:0/588501360 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a4160100 con 0x7fc2a40cedd0 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.902+0000 7fc29b400640 1 --2- 172.21.15.83:0/588501360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a40cedd0 0x7fc2a406cec0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009870 tx=0x7fc28c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1573557038446f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:27.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.902+0000 7fc2a3400640 1 -- 172.21.15.83:0/588501360 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0b2070 con 0x7fc2a40cedd0 2024-04-25T00:15:27.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.902+0000 7fc2a3400640 1 -- 172.21.15.83:0/588501360 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc28c0a9070 con 0x7fc2a40cedd0 2024-04-25T00:15:27.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.902+0000 7fc2a3400640 1 -- 172.21.15.83:0/588501360 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0ad880 con 0x7fc2a40cedd0 2024-04-25T00:15:27.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.902+0000 7fc2aaa00640 1 -- 172.21.15.83:0/588501360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a40cedd0 msgr2=0x7fc2a406cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.902+0000 7fc2aaa00640 1 --2- 172.21.15.83:0/588501360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a40cedd0 0x7fc2a406cec0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009870 tx=0x7fc28c01c560 comp rx=0 tx=0).stop 2024-04-25T00:15:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.902+0000 7fc2aaa00640 1 -- 172.21.15.83:0/588501360 shutdown_connections 2024-04-25T00:15:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.902+0000 7fc2aaa00640 1 --2- 172.21.15.83:0/588501360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a40cedd0 0x7fc2a406cec0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.902+0000 7fc2aaa00640 1 --2- 172.21.15.83:0/588501360 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a415be20 0x7fc2a415f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.902+0000 7fc2aaa00640 1 -- 172.21.15.83:0/588501360 >> 172.21.15.83:0/588501360 conn(0x7fc2a40b1b10 msgr2=0x7fc2a40b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.902+0000 7fc2aaa00640 1 -- 172.21.15.83:0/588501360 shutdown_connections 2024-04-25T00:15:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.902+0000 7fc2aaa00640 1 -- 172.21.15.83:0/588501360 wait complete. 2024-04-25T00:15:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.902+0000 7fc2aaa00640 1 Processor -- start 2024-04-25T00:15:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.902+0000 7fc2aaa00640 1 -- start start 2024-04-25T00:15:27.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2aaa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a40acd20 0x7fc2a4145200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2aaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a40cedd0 0x7fc2a4144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2aaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a415be20 0x7fc2a4071d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2aaa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a4168f40 con 0x7fc2a40cedd0 2024-04-25T00:15:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2aaa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a406d810 con 0x7fc2a415be20 2024-04-25T00:15:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2aaa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a41690c0 con 0x7fc2a40acd20 2024-04-25T00:15:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc29b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a40cedd0 0x7fc2a4144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc29b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a40cedd0 0x7fc2a4144c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57674/0 (socket says 172.21.15.83:57674) 2024-04-25T00:15:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc29b400640 1 -- 172.21.15.83:0/2531801668 learned_addr learned my addr 172.21.15.83:0/2531801668 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2a3e00640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a40acd20 0x7fc2a4145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc29b400640 1 -- 172.21.15.83:0/2531801668 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a40acd20 msgr2=0x7fc2a4145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc29b400640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a40acd20 0x7fc2a4145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc29b400640 1 -- 172.21.15.83:0/2531801668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a415be20 msgr2=0x7fc2a4071d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2a8a00640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a415be20 0x7fc2a4071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc29b400640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a415be20 0x7fc2a4071d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc29b400640 1 -- 172.21.15.83:0/2531801668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a4149ac0 con 0x7fc2a40cedd0 2024-04-25T00:15:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2a3e00640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a40acd20 0x7fc2a4145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2a8a00640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a415be20 0x7fc2a4071d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:27.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc29b400640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a40cedd0 0x7fc2a4144c90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc28c01ac10 tx=0x7fc28c019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:27.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2a0c00640 1 -- 172.21.15.83:0/2531801668 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0bb070 con 0x7fc2a40cedd0 2024-04-25T00:15:27.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2a0c00640 1 -- 172.21.15.83:0/2531801668 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc28c0addb0 con 0x7fc2a40cedd0 2024-04-25T00:15:27.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2a0c00640 1 -- 172.21.15.83:0/2531801668 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0b2040 con 0x7fc2a40cedd0 2024-04-25T00:15:27.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2aaa00640 1 -- 172.21.15.83:0/2531801668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a4071230 con 0x7fc2a40cedd0 2024-04-25T00:15:27.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2aaa00640 1 -- 172.21.15.83:0/2531801668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a40717f0 con 0x7fc2a40cedd0 2024-04-25T00:15:27.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2aaa00640 1 -- 172.21.15.83:0/2531801668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a404f510 con 0x7fc2a40cedd0 2024-04-25T00:15:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.906+0000 7fc2a0c00640 1 -- 172.21.15.83:0/2531801668 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc28c0be070 con 0x7fc2a40cedd0 2024-04-25T00:15:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.910+0000 7fc2a0c00640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2800678c0 0x7fc280069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.910+0000 7fc2a0c00640 1 -- 172.21.15.83:0/2531801668 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7fc28c11ef70 con 0x7fc2a40cedd0 2024-04-25T00:15:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.910+0000 7fc2a3e00640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2800678c0 0x7fc280069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.910+0000 7fc2a3e00640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2800678c0 0x7fc280069d80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc2a41516a0 tx=0x7fc29400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:27.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:27.914+0000 7fc2a0c00640 1 -- 172.21.15.83:0/2531801668 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc28c016150 con 0x7fc2a40cedd0 2024-04-25T00:15:28.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.070+0000 7fc2aaa00640 1 -- 172.21.15.83:0/2531801668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc2a4072e50 con 0x7fc2a40cedd0 2024-04-25T00:15:28.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.070+0000 7fc2a0c00640 1 -- 172.21.15.83:0/2531801668 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7fc28c0ef7d0 con 0x7fc2a40cedd0 2024-04-25T00:15:28.072 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T00:15:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.074+0000 7fc2aaa00640 1 -- 172.21.15.83:0/2531801668 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2800678c0 msgr2=0x7fc280069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.074+0000 7fc2aaa00640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2800678c0 0x7fc280069d80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc2a41516a0 tx=0x7fc29400b040 comp rx=0 tx=0).stop 2024-04-25T00:15:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.074+0000 7fc2aaa00640 1 -- 172.21.15.83:0/2531801668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a40cedd0 msgr2=0x7fc2a4144c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.074+0000 7fc2aaa00640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a40cedd0 0x7fc2a4144c90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc28c01ac10 tx=0x7fc28c019300 comp rx=0 tx=0).stop 2024-04-25T00:15:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.074+0000 7fc2aaa00640 1 -- 172.21.15.83:0/2531801668 shutdown_connections 2024-04-25T00:15:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.074+0000 7fc2aaa00640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fc2800678c0 0x7fc280069d80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.074+0000 7fc2aaa00640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a415be20 0x7fc2a4071d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.074+0000 7fc2aaa00640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a40cedd0 0x7fc2a4144c90 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.074+0000 7fc2aaa00640 1 --2- 172.21.15.83:0/2531801668 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a40acd20 0x7fc2a4145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.074+0000 7fc2aaa00640 1 -- 172.21.15.83:0/2531801668 >> 172.21.15.83:0/2531801668 conn(0x7fc2a40b1b10 msgr2=0x7fc2a40ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:28.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.074+0000 7fc2aaa00640 1 -- 172.21.15.83:0/2531801668 shutdown_connections 2024-04-25T00:15:28.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.074+0000 7fc2aaa00640 1 -- 172.21.15.83:0/2531801668 wait complete. 2024-04-25T00:15:28.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:15:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a49000640 1 Processor -- start 2024-04-25T00:15:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a49000640 1 -- start start 2024-04-25T00:15:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a49000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3a440d0350 0x7f3a440b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a49000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a440b3180 0x7f3a440cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a49000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a441556e0 con 0x7f3a44155300 2024-04-25T00:15:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a49000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3a44072260 con 0x7f3a440d0350 2024-04-25T00:15:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a49000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3a44072540 con 0x7f3a440b3180 2024-04-25T00:15:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a41400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3a440d0350 0x7f3a440b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a41400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3a440d0350 0x7f3a440b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57690/0 (socket says 172.21.15.83:57690) 2024-04-25T00:15:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a41400640 1 -- 172.21.15.83:0/8964564 learned_addr learned my addr 172.21.15.83:0/8964564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a42800640 1 --2- 172.21.15.83:0/8964564 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a440b3180 0x7f3a440cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a41400640 1 -- 172.21.15.83:0/8964564 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a440b3180 msgr2=0x7f3a440cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a41400640 1 --2- 172.21.15.83:0/8964564 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a440b3180 0x7f3a440cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a41400640 1 -- 172.21.15.83:0/8964564 >> v1:172.21.15.86:6789/0 conn(0x7f3a44155300 legacy=0x7f3a440cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a41400640 1 -- 172.21.15.83:0/8964564 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a44072820 con 0x7f3a440d0350 2024-04-25T00:15:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.230+0000 7f3a42800640 1 --2- 172.21.15.83:0/8964564 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a440b3180 0x7f3a440cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.234+0000 7f3a41400640 1 --2- 172.21.15.83:0/8964564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a440d0350 0x7f3a440b2c40 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000bb10 tx=0x7f3a3001eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1501e45411611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:28.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.234+0000 7f3a40a00640 1 -- 172.21.15.83:0/8964564 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a300b4070 con 0x7f3a440d0350 2024-04-25T00:15:28.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.234+0000 7f3a40a00640 1 -- 172.21.15.83:0/8964564 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a300a7c90 con 0x7f3a440d0350 2024-04-25T00:15:28.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.234+0000 7f3a40a00640 1 -- 172.21.15.83:0/8964564 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a300af660 con 0x7f3a440d0350 2024-04-25T00:15:28.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.234+0000 7f3a49000640 1 -- 172.21.15.83:0/8964564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a440d0350 msgr2=0x7f3a440b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.234+0000 7f3a49000640 1 --2- 172.21.15.83:0/8964564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a440d0350 0x7f3a440b2c40 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000bb10 tx=0x7f3a3001eba0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.234+0000 7f3a49000640 1 -- 172.21.15.83:0/8964564 shutdown_connections 2024-04-25T00:15:28.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.234+0000 7f3a49000640 1 --2- 172.21.15.83:0/8964564 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a440b3180 0x7f3a440cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.234+0000 7f3a49000640 1 --2- 172.21.15.83:0/8964564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a440d0350 0x7f3a440b2c40 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.234+0000 7f3a49000640 1 -- 172.21.15.83:0/8964564 >> 172.21.15.83:0/8964564 conn(0x7f3a440b1b10 msgr2=0x7f3a440b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:28.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.234+0000 7f3a49000640 1 -- 172.21.15.83:0/8964564 shutdown_connections 2024-04-25T00:15:28.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.234+0000 7f3a49000640 1 -- 172.21.15.83:0/8964564 wait complete. 2024-04-25T00:15:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a49000640 1 Processor -- start 2024-04-25T00:15:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a49000640 1 -- start start 2024-04-25T00:15:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a49000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a440b3180 0x7f3a44158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a49000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a440d0350 0x7f3a441f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a49000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a44155300 0x7f3a44157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a49000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4406cec0 con 0x7f3a440d0350 2024-04-25T00:15:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a49000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4406d040 con 0x7f3a44155300 2024-04-25T00:15:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a49000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3a4406d1c0 con 0x7f3a440b3180 2024-04-25T00:15:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a41400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a440d0350 0x7f3a441f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a41400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a440d0350 0x7f3a441f74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57696/0 (socket says 172.21.15.83:57696) 2024-04-25T00:15:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a41400640 1 -- 172.21.15.83:0/3776022190 learned_addr learned my addr 172.21.15.83:0/3776022190 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a41e00640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a440b3180 0x7f3a44158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a42800640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a44155300 0x7f3a44157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a41400640 1 -- 172.21.15.83:0/3776022190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a440b3180 msgr2=0x7f3a44158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a41400640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a440b3180 0x7f3a44158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a41400640 1 -- 172.21.15.83:0/3776022190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a44155300 msgr2=0x7f3a44157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a41400640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a44155300 0x7f3a44157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a41400640 1 -- 172.21.15.83:0/3776022190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a441f7ec0 con 0x7f3a440d0350 2024-04-25T00:15:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a42800640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a44155300 0x7f3a44157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:28.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a41400640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a440d0350 0x7f3a441f74e0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000b2c0 tx=0x7f3a3001b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:28.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a3a000640 1 -- 172.21.15.83:0/3776022190 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a300b4030 con 0x7f3a440d0350 2024-04-25T00:15:28.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a3a000640 1 -- 172.21.15.83:0/3776022190 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a300b6070 con 0x7f3a440d0350 2024-04-25T00:15:28.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a3a000640 1 -- 172.21.15.83:0/3776022190 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a300ad7b0 con 0x7f3a440d0350 2024-04-25T00:15:28.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a49000640 1 -- 172.21.15.83:0/3776022190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a441f81a0 con 0x7f3a440d0350 2024-04-25T00:15:28.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.238+0000 7f3a49000640 1 -- 172.21.15.83:0/3776022190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a44149f80 con 0x7f3a440d0350 2024-04-25T00:15:28.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.242+0000 7f3a49000640 1 -- 172.21.15.83:0/3776022190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a4404f510 con 0x7f3a440d0350 2024-04-25T00:15:28.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.242+0000 7f3a3a000640 1 -- 172.21.15.83:0/3776022190 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3a300c1060 con 0x7f3a440d0350 2024-04-25T00:15:28.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.242+0000 7f3a3a000640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3a24067910 0x7f3a24069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.242+0000 7f3a3a000640 1 -- 172.21.15.83:0/3776022190 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f3a30120180 con 0x7f3a440d0350 2024-04-25T00:15:28.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.242+0000 7f3a41e00640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3a24067910 0x7f3a24069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.242+0000 7f3a41e00640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3a24067910 0x7f3a24069dd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3a34007fd0 tx=0x7f3a34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:28.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.246+0000 7f3a3a000640 1 -- 172.21.15.83:0/3776022190 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3a30018820 con 0x7f3a440d0350 2024-04-25T00:15:28.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.402+0000 7f3a49000640 1 -- 172.21.15.83:0/3776022190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3a44166a50 con 0x7f3a440d0350 2024-04-25T00:15:28.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.402+0000 7f3a3a000640 1 -- 172.21.15.83:0/3776022190 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f3a300f0c10 con 0x7f3a440d0350 2024-04-25T00:15:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T00:15:28.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.406+0000 7f3a49000640 1 -- 172.21.15.83:0/3776022190 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3a24067910 msgr2=0x7f3a24069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.406+0000 7f3a49000640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3a24067910 0x7f3a24069dd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3a34007fd0 tx=0x7f3a34009040 comp rx=0 tx=0).stop 2024-04-25T00:15:28.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.406+0000 7f3a49000640 1 -- 172.21.15.83:0/3776022190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a440d0350 msgr2=0x7f3a441f74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.406+0000 7f3a49000640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a440d0350 0x7f3a441f74e0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000b2c0 tx=0x7f3a3001b940 comp rx=0 tx=0).stop 2024-04-25T00:15:28.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.406+0000 7f3a49000640 1 -- 172.21.15.83:0/3776022190 shutdown_connections 2024-04-25T00:15:28.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.406+0000 7f3a49000640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3a24067910 0x7f3a24069dd0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.406+0000 7f3a49000640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a44155300 0x7f3a44157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.406+0000 7f3a49000640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a440d0350 0x7f3a441f74e0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.406+0000 7f3a49000640 1 --2- 172.21.15.83:0/3776022190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a440b3180 0x7f3a44158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.406+0000 7f3a49000640 1 -- 172.21.15.83:0/3776022190 >> 172.21.15.83:0/3776022190 conn(0x7f3a440b1b10 msgr2=0x7f3a44148a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:28.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.406+0000 7f3a49000640 1 -- 172.21.15.83:0/3776022190 shutdown_connections 2024-04-25T00:15:28.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.406+0000 7f3a49000640 1 -- 172.21.15.83:0/3776022190 wait complete. 2024-04-25T00:15:28.425 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:15:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.562+0000 7f529e200640 1 Processor -- start 2024-04-25T00:15:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.562+0000 7f529e200640 1 -- start start 2024-04-25T00:15:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.562+0000 7f529e200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529806d5d0 0x7f529815f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.562+0000 7f529e200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52980cedd0 0x7f52980b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.562+0000 7f529e200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529815c200 con 0x7f529815be20 2024-04-25T00:15:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.562+0000 7f529e200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f529815fed0 con 0x7f52980cedd0 2024-04-25T00:15:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.562+0000 7f529e200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f52980b2e70 con 0x7f529806d5d0 2024-04-25T00:15:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.566+0000 7f5297400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f529815be20 0x7f529806cec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37780/0 (socket says 172.21.15.83:37780) 2024-04-25T00:15:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.566+0000 7f5297400640 1 -- 172.21.15.83:0/2177312701 learned_addr learned my addr 172.21.15.83:0/2177312701 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.566+0000 7f5297e00640 1 --2- 172.21.15.83:0/2177312701 >> v2:172.21.15.83:3300/0 conn(0x7f52980cedd0 0x7f52980b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.566+0000 7f5296a00640 1 --2- 172.21.15.83:0/2177312701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529806d5d0 0x7f529815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.566+0000 7f5296000640 1 -- 172.21.15.83:0/2177312701 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 931794145 0 0) 0x7f529815c200 con 0x7f529815be20 2024-04-25T00:15:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.566+0000 7f5296000640 1 -- 172.21.15.83:0/2177312701 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f527c003530 con 0x7f529815be20 2024-04-25T00:15:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.566+0000 7f5297e00640 1 -- 172.21.15.83:0/2177312701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529806d5d0 msgr2=0x7f529815f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.566+0000 7f5297e00640 1 --2- 172.21.15.83:0/2177312701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529806d5d0 0x7f529815f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.566+0000 7f5297e00640 1 -- 172.21.15.83:0/2177312701 >> v1:172.21.15.83:6789/0 conn(0x7f529815be20 legacy=0x7f529806cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.566+0000 7f5297e00640 1 -- 172.21.15.83:0/2177312701 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5298160100 con 0x7f52980cedd0 2024-04-25T00:15:28.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.566+0000 7f5297e00640 1 --2- 172.21.15.83:0/2177312701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52980cedd0 0x7f52980b3300 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5290009d90 tx=0x7f529001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d19bda9f582f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:28.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.566+0000 7f5296000640 1 -- 172.21.15.83:0/2177312701 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52900b2070 con 0x7f52980cedd0 2024-04-25T00:15:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.566+0000 7f5296000640 1 -- 172.21.15.83:0/2177312701 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52900a5830 con 0x7f52980cedd0 2024-04-25T00:15:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.566+0000 7f5296000640 1 -- 172.21.15.83:0/2177312701 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52900a5be0 con 0x7f52980cedd0 2024-04-25T00:15:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 -- 172.21.15.83:0/2177312701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52980cedd0 msgr2=0x7f52980b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 --2- 172.21.15.83:0/2177312701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52980cedd0 0x7f52980b3300 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5290009d90 tx=0x7f529001c970 comp rx=0 tx=0).stop 2024-04-25T00:15:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 -- 172.21.15.83:0/2177312701 shutdown_connections 2024-04-25T00:15:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 --2- 172.21.15.83:0/2177312701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52980cedd0 0x7f52980b3300 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 --2- 172.21.15.83:0/2177312701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529806d5d0 0x7f529815f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 -- 172.21.15.83:0/2177312701 >> 172.21.15.83:0/2177312701 conn(0x7f52980b1b10 msgr2=0x7f52980b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 -- 172.21.15.83:0/2177312701 shutdown_connections 2024-04-25T00:15:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 -- 172.21.15.83:0/2177312701 wait complete. 2024-04-25T00:15:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 Processor -- start 2024-04-25T00:15:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 -- start start 2024-04-25T00:15:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529806d5d0 0x7f529813a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52980cedd0 0x7f5298155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529815be20 0x7f529815e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52981690c0 con 0x7f529806d5d0 2024-04-25T00:15:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5298168f40 con 0x7f52980cedd0 2024-04-25T00:15:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f529e200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5298169240 con 0x7f529815be20 2024-04-25T00:15:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f5297400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529806d5d0 0x7f529813a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f5297400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529806d5d0 0x7f529813a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57718/0 (socket says 172.21.15.83:57718) 2024-04-25T00:15:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f5297400640 1 -- 172.21.15.83:0/3107907929 learned_addr learned my addr 172.21.15.83:0/3107907929 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f5297e00640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529815be20 0x7f529815e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f5296a00640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52980cedd0 0x7f5298155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f5297400640 1 -- 172.21.15.83:0/3107907929 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529815be20 msgr2=0x7f529815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f5297400640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529815be20 0x7f529815e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f5297400640 1 -- 172.21.15.83:0/3107907929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52980cedd0 msgr2=0x7f5298155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f5297400640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52980cedd0 0x7f5298155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f5297400640 1 -- 172.21.15.83:0/3107907929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529813af30 con 0x7f529806d5d0 2024-04-25T00:15:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.570+0000 7f5296a00640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52980cedd0 0x7f5298155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.574+0000 7f5297400640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529806d5d0 0x7f529813a9f0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f528800c3f0 tx=0x7f528800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.574+0000 7f528f400640 1 -- 172.21.15.83:0/3107907929 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288017070 con 0x7f529806d5d0 2024-04-25T00:15:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.574+0000 7f528f400640 1 -- 172.21.15.83:0/3107907929 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5288005cf0 con 0x7f529806d5d0 2024-04-25T00:15:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.574+0000 7f529e200640 1 -- 172.21.15.83:0/3107907929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529814f8f0 con 0x7f529806d5d0 2024-04-25T00:15:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.574+0000 7f528f400640 1 -- 172.21.15.83:0/3107907929 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52880072f0 con 0x7f529806d5d0 2024-04-25T00:15:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.574+0000 7f529e200640 1 -- 172.21.15.83:0/3107907929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f529814feb0 con 0x7f529806d5d0 2024-04-25T00:15:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.574+0000 7f529e200640 1 -- 172.21.15.83:0/3107907929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f529804f510 con 0x7f529806d5d0 2024-04-25T00:15:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.574+0000 7f528f400640 1 -- 172.21.15.83:0/3107907929 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f528801a070 con 0x7f529806d5d0 2024-04-25T00:15:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.574+0000 7f528f400640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f52780678f0 0x7f5278069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.574+0000 7f528f400640 1 -- 172.21.15.83:0/3107907929 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f5288085380 con 0x7f529806d5d0 2024-04-25T00:15:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.574+0000 7f5296a00640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f52780678f0 0x7f5278069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.578+0000 7f5296a00640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f52780678f0 0x7f5278069db0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f52981f8710 tx=0x7f5284008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.578+0000 7f528f400640 1 -- 172.21.15.83:0/3107907929 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5288055be0 con 0x7f529806d5d0 2024-04-25T00:15:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.734+0000 7f529e200640 1 -- 172.21.15.83:0/3107907929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f52980afda0 con 0x7f529806d5d0 2024-04-25T00:15:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.734+0000 7f528f400640 1 -- 172.21.15.83:0/3107907929 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f52880554e0 con 0x7f529806d5d0 2024-04-25T00:15:28.735 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T00:15:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.738+0000 7f529e200640 1 -- 172.21.15.83:0/3107907929 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f52780678f0 msgr2=0x7f5278069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.738+0000 7f529e200640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f52780678f0 0x7f5278069db0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f52981f8710 tx=0x7f5284008040 comp rx=0 tx=0).stop 2024-04-25T00:15:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.738+0000 7f529e200640 1 -- 172.21.15.83:0/3107907929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529806d5d0 msgr2=0x7f529813a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.738+0000 7f529e200640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529806d5d0 0x7f529813a9f0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f528800c3f0 tx=0x7f528800c8b0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.738+0000 7f529e200640 1 -- 172.21.15.83:0/3107907929 shutdown_connections 2024-04-25T00:15:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.738+0000 7f529e200640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f52780678f0 0x7f5278069db0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.738+0000 7f529e200640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529815be20 0x7f529815e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.738+0000 7f529e200640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52980cedd0 0x7f5298155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.738+0000 7f529e200640 1 --2- 172.21.15.83:0/3107907929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529806d5d0 0x7f529813a9f0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.738+0000 7f529e200640 1 -- 172.21.15.83:0/3107907929 >> 172.21.15.83:0/3107907929 conn(0x7f52980b1b10 msgr2=0x7f52980b1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.738+0000 7f529e200640 1 -- 172.21.15.83:0/3107907929 shutdown_connections 2024-04-25T00:15:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.738+0000 7f529e200640 1 -- 172.21.15.83:0/3107907929 wait complete. 2024-04-25T00:15:28.757 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.890+0000 7feb3fa00640 1 Processor -- start 2024-04-25T00:15:28.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.890+0000 7feb3fa00640 1 -- start start 2024-04-25T00:15:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.894+0000 7feb3fa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feb38156340 0x7feb380acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.894+0000 7feb3fa00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb3806ecf0 con 0x7feb3815ce00 2024-04-25T00:15:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.894+0000 7feb3fa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb3806efd0 con 0x7feb380aed90 2024-04-25T00:15:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.894+0000 7feb3fa00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7feb3806cef0 con 0x7feb38156340 2024-04-25T00:15:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.894+0000 7feb3d000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feb38156340 0x7feb380acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.894+0000 7feb37e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7feb3815ce00 0x7feb380ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37792/0 (socket says 172.21.15.83:37792) 2024-04-25T00:15:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.894+0000 7feb37e00640 1 -- 172.21.15.83:0/4233273976 learned_addr learned my addr 172.21.15.83:0/4233273976 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.894+0000 7feb3d000640 1 -- 172.21.15.83:0/4233273976 >> v1:172.21.15.86:6789/0 conn(0x7feb380aed90 legacy=0x7feb3806d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.894+0000 7feb3d000640 1 -- 172.21.15.83:0/4233273976 >> v1:172.21.15.83:6789/0 conn(0x7feb3815ce00 legacy=0x7feb380ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.894+0000 7feb3d000640 1 -- 172.21.15.83:0/4233273976 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3815d290 con 0x7feb38156340 2024-04-25T00:15:28.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3d000640 1 --2- 172.21.15.83:0/4233273976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb38156340 0x7feb380acd20 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0097c0 tx=0x7feb2c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb776cb2cedda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:28.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb37400640 1 -- 172.21.15.83:0/4233273976 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0b0070 con 0x7feb38156340 2024-04-25T00:15:28.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb37400640 1 -- 172.21.15.83:0/4233273976 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb2c004230 con 0x7feb38156340 2024-04-25T00:15:28.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb37400640 1 -- 172.21.15.83:0/4233273976 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c003620 con 0x7feb38156340 2024-04-25T00:15:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 -- 172.21.15.83:0/4233273976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb38156340 msgr2=0x7feb380acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 --2- 172.21.15.83:0/4233273976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb38156340 0x7feb380acd20 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0097c0 tx=0x7feb2c01c460 comp rx=0 tx=0).stop 2024-04-25T00:15:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 -- 172.21.15.83:0/4233273976 shutdown_connections 2024-04-25T00:15:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 --2- 172.21.15.83:0/4233273976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb38156340 0x7feb380acd20 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 -- 172.21.15.83:0/4233273976 >> 172.21.15.83:0/4233273976 conn(0x7feb380b1860 msgr2=0x7feb380b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 -- 172.21.15.83:0/4233273976 shutdown_connections 2024-04-25T00:15:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 -- 172.21.15.83:0/4233273976 wait complete. 2024-04-25T00:15:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 Processor -- start 2024-04-25T00:15:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 -- start start 2024-04-25T00:15:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb380aed90 0x7feb3809ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb38156340 0x7feb381430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb381435f0 0x7feb380b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb3806e0b0 con 0x7feb381435f0 2024-04-25T00:15:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb3806df30 con 0x7feb38156340 2024-04-25T00:15:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3fa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7feb3816a530 con 0x7feb380aed90 2024-04-25T00:15:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb381435f0 0x7feb380b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb381435f0 0x7feb380b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57744/0 (socket says 172.21.15.83:57744) 2024-04-25T00:15:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3da00640 1 -- 172.21.15.83:0/3190396645 learned_addr learned my addr 172.21.15.83:0/3190396645 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb37e00640 1 --2- 172.21.15.83:0/3190396645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb38156340 0x7feb381430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.898+0000 7feb3d000640 1 --2- 172.21.15.83:0/3190396645 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb380aed90 0x7feb3809ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb3da00640 1 -- 172.21.15.83:0/3190396645 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb380aed90 msgr2=0x7feb3809ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb3da00640 1 --2- 172.21.15.83:0/3190396645 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb380aed90 0x7feb3809ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb3da00640 1 -- 172.21.15.83:0/3190396645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb38156340 msgr2=0x7feb381430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb3da00640 1 --2- 172.21.15.83:0/3190396645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb38156340 0x7feb381430b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb3da00640 1 -- 172.21.15.83:0/3190396645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb380b2af0 con 0x7feb381435f0 2024-04-25T00:15:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb3da00640 1 --2- 172.21.15.83:0/3190396645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb381435f0 0x7feb380b0810 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7feb3000b970 tx=0x7feb3000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb34c00640 1 -- 172.21.15.83:0/3190396645 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30019070 con 0x7feb381435f0 2024-04-25T00:15:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb34c00640 1 -- 172.21.15.83:0/3190396645 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb3000cb40 con 0x7feb381435f0 2024-04-25T00:15:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb34c00640 1 -- 172.21.15.83:0/3190396645 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30014440 con 0x7feb381435f0 2024-04-25T00:15:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb3fa00640 1 -- 172.21.15.83:0/3190396645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb380b2d90 con 0x7feb381435f0 2024-04-25T00:15:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb3fa00640 1 -- 172.21.15.83:0/3190396645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb380b3350 con 0x7feb381435f0 2024-04-25T00:15:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb3fa00640 1 -- 172.21.15.83:0/3190396645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb3804f510 con 0x7feb381435f0 2024-04-25T00:15:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb34c00640 1 -- 172.21.15.83:0/3190396645 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feb3001c070 con 0x7feb381435f0 2024-04-25T00:15:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb34c00640 1 --2- 172.21.15.83:0/3190396645 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feb180678c0 0x7feb18069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb34c00640 1 -- 172.21.15.83:0/3190396645 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7feb30085a40 con 0x7feb381435f0 2024-04-25T00:15:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.902+0000 7feb3d000640 1 --2- 172.21.15.83:0/3190396645 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feb180678c0 0x7feb18069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.906+0000 7feb3d000640 1 --2- 172.21.15.83:0/3190396645 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feb180678c0 0x7feb18069d80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0ad000 tx=0x7feb2c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:28.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:28.906+0000 7feb34c00640 1 -- 172.21.15.83:0/3190396645 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7feb30055ce0 con 0x7feb381435f0 2024-04-25T00:15:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.098+0000 7feb3fa00640 1 -- 172.21.15.83:0/3190396645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb38072ba0 con 0x7feb381435f0 2024-04-25T00:15:29.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.098+0000 7feb34c00640 1 -- 172.21.15.83:0/3190396645 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5201 (secure 0 0 0) 0x7feb30059fe0 con 0x7feb381435f0 2024-04-25T00:15:29.099 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:29.099 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1992401755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1992401755},{"type":"v1","addr":"172.21.15.86:6833","nonce":1992401755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3621275047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3621275047},{"type":"v1","addr":"172.21.15.86:6835","nonce":3621275047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:51.879542+0000","modified":"2024-04-25T00:15:20.022793+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7719,"mds_1":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7719":{"gid":7719,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6835/565131724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":565131724},{"type":"v1","addr":"172.21.15.83:6835","nonce":565131724}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7728":{"gid":7728,"name":"a","rank":1,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2870595352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2870595352},{"type":"v1","addr":"172.21.15.83:6837","nonce":2870595352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7719,"qdb_cluster":[7728,7719]},"id":7}]} 2024-04-25T00:15:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 74 2024-04-25T00:15:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.098+0000 7feb3fa00640 1 -- 172.21.15.83:0/3190396645 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feb180678c0 msgr2=0x7feb18069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.098+0000 7feb3fa00640 1 --2- 172.21.15.83:0/3190396645 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feb180678c0 0x7feb18069d80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0ad000 tx=0x7feb2c008040 comp rx=0 tx=0).stop 2024-04-25T00:15:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.098+0000 7feb3fa00640 1 -- 172.21.15.83:0/3190396645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb381435f0 msgr2=0x7feb380b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.098+0000 7feb3fa00640 1 --2- 172.21.15.83:0/3190396645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb381435f0 0x7feb380b0810 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7feb3000b970 tx=0x7feb3000be30 comp rx=0 tx=0).stop 2024-04-25T00:15:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.098+0000 7feb3fa00640 1 -- 172.21.15.83:0/3190396645 shutdown_connections 2024-04-25T00:15:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.098+0000 7feb3fa00640 1 --2- 172.21.15.83:0/3190396645 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7feb180678c0 0x7feb18069d80 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.098+0000 7feb3fa00640 1 --2- 172.21.15.83:0/3190396645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb381435f0 0x7feb380b0810 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.098+0000 7feb3fa00640 1 --2- 172.21.15.83:0/3190396645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb38156340 0x7feb381430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.098+0000 7feb3fa00640 1 --2- 172.21.15.83:0/3190396645 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb380aed90 0x7feb3809ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.098+0000 7feb3fa00640 1 -- 172.21.15.83:0/3190396645 >> 172.21.15.83:0/3190396645 conn(0x7feb380b1860 msgr2=0x7feb3806e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:29.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.102+0000 7feb3fa00640 1 -- 172.21.15.83:0/3190396645 shutdown_connections 2024-04-25T00:15:29.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.102+0000 7feb3fa00640 1 -- 172.21.15.83:0/3190396645 wait complete. 2024-04-25T00:15:29.120 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:15:29.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f995f800640 1 Processor -- start 2024-04-25T00:15:29.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f995f800640 1 -- start start 2024-04-25T00:15:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f995f800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f995815bd30 0x7f9958071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f995f800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9958071fe0 0x7f99580b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f995f800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f995815f400 con 0x7f99580b30a0 2024-04-25T00:15:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f995f800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9958071610 con 0x7f995815bd30 2024-04-25T00:15:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f995f800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9958070fc0 con 0x7f9958071fe0 2024-04-25T00:15:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f995ce00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f995815bd30 0x7f9958071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f995ce00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f995815bd30 0x7f9958071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57758/0 (socket says 172.21.15.83:57758) 2024-04-25T00:15:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f995ce00640 1 -- 172.21.15.83:0/421602678 learned_addr learned my addr 172.21.15.83:0/421602678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f994fe00640 1 --2- 172.21.15.83:0/421602678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9958071fe0 0x7f99580b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f995ce00640 1 -- 172.21.15.83:0/421602678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9958071fe0 msgr2=0x7f99580b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f995ce00640 1 --2- 172.21.15.83:0/421602678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9958071fe0 0x7f99580b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f995ce00640 1 -- 172.21.15.83:0/421602678 >> v1:172.21.15.86:6789/0 conn(0x7f99580b30a0 legacy=0x7f9958072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f995ce00640 1 -- 172.21.15.83:0/421602678 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99580712a0 con 0x7f995815bd30 2024-04-25T00:15:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.254+0000 7f994fe00640 1 --2- 172.21.15.83:0/421602678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9958071fe0 0x7f99580b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:29.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.258+0000 7f995ce00640 1 --2- 172.21.15.83:0/421602678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f995815bd30 0x7f9958071aa0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f9958071790 tx=0x7f994801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4e6d90c6c0b2d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:29.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.258+0000 7f9957e00640 1 -- 172.21.15.83:0/421602678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99480a6040 con 0x7f995815bd30 2024-04-25T00:15:29.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.258+0000 7f9957e00640 1 -- 172.21.15.83:0/421602678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99480af070 con 0x7f995815bd30 2024-04-25T00:15:29.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.258+0000 7f9957e00640 1 -- 172.21.15.83:0/421602678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99480aa860 con 0x7f995815bd30 2024-04-25T00:15:29.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.258+0000 7f995f800640 1 -- 172.21.15.83:0/421602678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f995815bd30 msgr2=0x7f9958071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.258+0000 7f995f800640 1 --2- 172.21.15.83:0/421602678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f995815bd30 0x7f9958071aa0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f9958071790 tx=0x7f994801c500 comp rx=0 tx=0).stop 2024-04-25T00:15:29.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.258+0000 7f995f800640 1 -- 172.21.15.83:0/421602678 shutdown_connections 2024-04-25T00:15:29.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.258+0000 7f995f800640 1 --2- 172.21.15.83:0/421602678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9958071fe0 0x7f99580b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.258+0000 7f995f800640 1 --2- 172.21.15.83:0/421602678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f995815bd30 0x7f9958071aa0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.258+0000 7f995f800640 1 -- 172.21.15.83:0/421602678 >> 172.21.15.83:0/421602678 conn(0x7f99580b1860 msgr2=0x7f99580b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:29.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.258+0000 7f995f800640 1 -- 172.21.15.83:0/421602678 shutdown_connections 2024-04-25T00:15:29.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.258+0000 7f995f800640 1 -- 172.21.15.83:0/421602678 wait complete. 2024-04-25T00:15:29.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995f800640 1 Processor -- start 2024-04-25T00:15:29.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995f800640 1 -- start start 2024-04-25T00:15:29.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995f800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9958071fe0 0x7f995813b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995f800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99580b30a0 0x7f995815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f995815bd30 0x7f995809ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995f800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f995815ee70 con 0x7f9958071fe0 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9958073040 con 0x7f995815bd30 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995f800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f995815eff0 con 0x7f99580b30a0 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9958071fe0 0x7f995813b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9958071fe0 0x7f995813b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57772/0 (socket says 172.21.15.83:57772) 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995ce00640 1 -- 172.21.15.83:0/3849204726 learned_addr learned my addr 172.21.15.83:0/3849204726 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f994fe00640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99580b30a0 0x7f995815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995d800640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f995815bd30 0x7f995809ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995ce00640 1 -- 172.21.15.83:0/3849204726 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99580b30a0 msgr2=0x7f995815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995ce00640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99580b30a0 0x7f995815ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995ce00640 1 -- 172.21.15.83:0/3849204726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f995815bd30 msgr2=0x7f995809ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995ce00640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f995815bd30 0x7f995809ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995ce00640 1 -- 172.21.15.83:0/3849204726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99581498b0 con 0x7f9958071fe0 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f994fe00640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99580b30a0 0x7f995815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995ce00640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9958071fe0 0x7f995813b670 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f994801abb0 tx=0x7f994801ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f9955600640 1 -- 172.21.15.83:0/3849204726 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99480b38c0 con 0x7f9958071fe0 2024-04-25T00:15:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f9955600640 1 -- 172.21.15.83:0/3849204726 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99480a6030 con 0x7f9958071fe0 2024-04-25T00:15:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f9955600640 1 -- 172.21.15.83:0/3849204726 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99480b3c60 con 0x7f9958071fe0 2024-04-25T00:15:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995f800640 1 -- 172.21.15.83:0/3849204726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9958149b90 con 0x7f9958071fe0 2024-04-25T00:15:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.262+0000 7f995f800640 1 -- 172.21.15.83:0/3849204726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f995814a150 con 0x7f9958071fe0 2024-04-25T00:15:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.266+0000 7f995f800640 1 -- 172.21.15.83:0/3849204726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f995804f510 con 0x7f9958071fe0 2024-04-25T00:15:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.266+0000 7f9955600640 1 -- 172.21.15.83:0/3849204726 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f99480b2380 con 0x7f9958071fe0 2024-04-25T00:15:29.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.266+0000 7f9955600640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99340678c0 0x7f9934069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.266+0000 7f9955600640 1 -- 172.21.15.83:0/3849204726 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f994811fad0 con 0x7f9958071fe0 2024-04-25T00:15:29.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.266+0000 7f994fe00640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99340678c0 0x7f9934069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.266+0000 7f994fe00640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99340678c0 0x7f9934069d80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9940006fd0 tx=0x7f9940008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:29.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.270+0000 7f9955600640 1 -- 172.21.15.83:0/3849204726 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f99480c4060 con 0x7f9958071fe0 2024-04-25T00:15:29.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.426+0000 7f995f800640 1 -- 172.21.15.83:0/3849204726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99580d0600 con 0x7f9958071fe0 2024-04-25T00:15:29.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.426+0000 7f9955600640 1 -- 172.21.15.83:0/3849204726 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+17048 (secure 0 0 0) 0x7f99480160f0 con 0x7f9958071fe0 2024-04-25T00:15:29.428 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:29.428 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":97,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:15:04.852008+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:50.260735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:50.880394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1609715757":"2024-04-26T00:15:04.845080+0000","172.21.15.83:6836/1609715757":"2024-04-26T00:15:04.845080+0000","172.21.15.86:6835/3612679148":"2024-04-26T00:15:04.826082+0000","172.21.15.86:6833/3612679148":"2024-04-26T00:15:04.826082+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:15:29.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.430+0000 7f995f800640 1 -- 172.21.15.83:0/3849204726 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99340678c0 msgr2=0x7f9934069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.430+0000 7f995f800640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99340678c0 0x7f9934069d80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9940006fd0 tx=0x7f9940008040 comp rx=0 tx=0).stop 2024-04-25T00:15:29.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.430+0000 7f995f800640 1 -- 172.21.15.83:0/3849204726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9958071fe0 msgr2=0x7f995813b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.430+0000 7f995f800640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9958071fe0 0x7f995813b670 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f994801abb0 tx=0x7f994801ad40 comp rx=0 tx=0).stop 2024-04-25T00:15:29.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.430+0000 7f995f800640 1 -- 172.21.15.83:0/3849204726 shutdown_connections 2024-04-25T00:15:29.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.430+0000 7f995f800640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f99340678c0 0x7f9934069d80 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.430+0000 7f995f800640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99580b30a0 0x7f995815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.430+0000 7f995f800640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f995815bd30 0x7f995809ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.430+0000 7f995f800640 1 --2- 172.21.15.83:0/3849204726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9958071fe0 0x7f995813b670 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.430+0000 7f995f800640 1 -- 172.21.15.83:0/3849204726 >> 172.21.15.83:0/3849204726 conn(0x7f99580b1860 msgr2=0x7f9958160900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:29.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.430+0000 7f995f800640 1 -- 172.21.15.83:0/3849204726 shutdown_connections 2024-04-25T00:15:29.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.430+0000 7f995f800640 1 -- 172.21.15.83:0/3849204726 wait complete. 2024-04-25T00:15:29.450 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:15:29.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:29.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f082a800640 1 Processor -- start 2024-04-25T00:15:29.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f082a800640 1 -- start start 2024-04-25T00:15:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f082a800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f082415bf60 0x7f08240cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f082a800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08240b01e0 0x7f0824066f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f082a800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0824072100 con 0x7f0824067490 2024-04-25T00:15:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f082a800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f08240723e0 con 0x7f082415bf60 2024-04-25T00:15:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f082a800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f08240726c0 con 0x7f08240b01e0 2024-04-25T00:15:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f0823400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f082415bf60 0x7f08240cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f0823400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f082415bf60 0x7f08240cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57776/0 (socket says 172.21.15.83:57776) 2024-04-25T00:15:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f0823400640 1 -- 172.21.15.83:0/2088971167 learned_addr learned my addr 172.21.15.83:0/2088971167 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f081aa00640 1 --2- 172.21.15.83:0/2088971167 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08240b01e0 0x7f0824066f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f0823400640 1 -- 172.21.15.83:0/2088971167 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08240b01e0 msgr2=0x7f0824066f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f0823400640 1 --2- 172.21.15.83:0/2088971167 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08240b01e0 0x7f0824066f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f0823400640 1 -- 172.21.15.83:0/2088971167 >> v1:172.21.15.86:6789/0 conn(0x7f0824067490 legacy=0x7f082415fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.586+0000 7f0823400640 1 -- 172.21.15.83:0/2088971167 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08240729a0 con 0x7f082415bf60 2024-04-25T00:15:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.590+0000 7f0823400640 1 --2- 172.21.15.83:0/2088971167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f082415bf60 0x7f08240cebd0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f0814009e50 tx=0x7f081401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87c64f23ba0b426 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.590+0000 7f0822a00640 1 -- 172.21.15.83:0/2088971167 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08140a57f0 con 0x7f082415bf60 2024-04-25T00:15:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.590+0000 7f0822a00640 1 -- 172.21.15.83:0/2088971167 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08140a5990 con 0x7f082415bf60 2024-04-25T00:15:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.590+0000 7f0822a00640 1 -- 172.21.15.83:0/2088971167 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08140a5d60 con 0x7f082415bf60 2024-04-25T00:15:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.590+0000 7f082a800640 1 -- 172.21.15.83:0/2088971167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f082415bf60 msgr2=0x7f08240cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.590+0000 7f082a800640 1 --2- 172.21.15.83:0/2088971167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f082415bf60 0x7f08240cebd0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f0814009e50 tx=0x7f081401ca30 comp rx=0 tx=0).stop 2024-04-25T00:15:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.590+0000 7f082a800640 1 -- 172.21.15.83:0/2088971167 shutdown_connections 2024-04-25T00:15:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.590+0000 7f082a800640 1 --2- 172.21.15.83:0/2088971167 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08240b01e0 0x7f0824066f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.590+0000 7f082a800640 1 --2- 172.21.15.83:0/2088971167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f082415bf60 0x7f08240cebd0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.590+0000 7f082a800640 1 -- 172.21.15.83:0/2088971167 >> 172.21.15.83:0/2088971167 conn(0x7f08240d0cd0 msgr2=0x7f08240d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.590+0000 7f082a800640 1 -- 172.21.15.83:0/2088971167 shutdown_connections 2024-04-25T00:15:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.590+0000 7f082a800640 1 -- 172.21.15.83:0/2088971167 wait complete. 2024-04-25T00:15:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f082a800640 1 Processor -- start 2024-04-25T00:15:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f082a800640 1 -- start start 2024-04-25T00:15:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f082a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0824067490 0x7f08241fae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f082a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240b01e0 0x7f0824165300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f082a800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f082415bf60 0x7f0824156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f082a800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0824168e50 con 0x7f08240b01e0 2024-04-25T00:15:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f082a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0824168cd0 con 0x7f0824067490 2024-04-25T00:15:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f082a800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0824168fd0 con 0x7f082415bf60 2024-04-25T00:15:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f081aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240b01e0 0x7f0824165300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f081aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240b01e0 0x7f0824165300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57778/0 (socket says 172.21.15.83:57778) 2024-04-25T00:15:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f081aa00640 1 -- 172.21.15.83:0/3071660964 learned_addr learned my addr 172.21.15.83:0/3071660964 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f0823400640 1 --2- 172.21.15.83:0/3071660964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0824067490 0x7f08241fae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f0823e00640 1 --2- 172.21.15.83:0/3071660964 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f082415bf60 0x7f0824156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f081aa00640 1 -- 172.21.15.83:0/3071660964 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f082415bf60 msgr2=0x7f0824156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f081aa00640 1 --2- 172.21.15.83:0/3071660964 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f082415bf60 0x7f0824156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f081aa00640 1 -- 172.21.15.83:0/3071660964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0824067490 msgr2=0x7f08241fae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f081aa00640 1 --2- 172.21.15.83:0/3071660964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0824067490 0x7f08241fae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f081aa00640 1 -- 172.21.15.83:0/3071660964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08241fa6a0 con 0x7f08240b01e0 2024-04-25T00:15:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f081aa00640 1 --2- 172.21.15.83:0/3071660964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240b01e0 0x7f0824165300 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f080c00d5f0 tx=0x7f080c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:29.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f081be00640 1 -- 172.21.15.83:0/3071660964 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c0125f0 con 0x7f08240b01e0 2024-04-25T00:15:29.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f081be00640 1 -- 172.21.15.83:0/3071660964 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f080c019070 con 0x7f08240b01e0 2024-04-25T00:15:29.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f082a800640 1 -- 172.21.15.83:0/3071660964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08241561f0 con 0x7f08240b01e0 2024-04-25T00:15:29.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f081be00640 1 -- 172.21.15.83:0/3071660964 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c0129d0 con 0x7f08240b01e0 2024-04-25T00:15:29.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f082a800640 1 -- 172.21.15.83:0/3071660964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0824156750 con 0x7f08240b01e0 2024-04-25T00:15:29.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.594+0000 7f082a800640 1 -- 172.21.15.83:0/3071660964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f082404f510 con 0x7f08240b01e0 2024-04-25T00:15:29.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.598+0000 7f081be00640 1 -- 172.21.15.83:0/3071660964 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f080c012b70 con 0x7f08240b01e0 2024-04-25T00:15:29.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.598+0000 7f081be00640 1 --2- 172.21.15.83:0/3071660964 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08000678c0 0x7f0800069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.598+0000 7f081be00640 1 -- 172.21.15.83:0/3071660964 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f080c0862d0 con 0x7f08240b01e0 2024-04-25T00:15:29.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.598+0000 7f0823400640 1 --2- 172.21.15.83:0/3071660964 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08000678c0 0x7f0800069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.598+0000 7f0823400640 1 --2- 172.21.15.83:0/3071660964 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08000678c0 0x7f0800069d80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f08140ae000 tx=0x7f08140a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:29.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.602+0000 7f081be00640 1 -- 172.21.15.83:0/3071660964 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f08241561f0 con 0x7f08240b01e0 2024-04-25T00:15:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.790+0000 7f082a800640 1 -- 172.21.15.83:0/3071660964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08240b3ab0 con 0x7f08240b01e0 2024-04-25T00:15:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.790+0000 7f081be00640 1 -- 172.21.15.83:0/3071660964 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5201 (secure 0 0 0) 0x7f080c0564e0 con 0x7f08240b01e0 2024-04-25T00:15:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:29.791 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1992401755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1992401755},{"type":"v1","addr":"172.21.15.86:6833","nonce":1992401755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3621275047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3621275047},{"type":"v1","addr":"172.21.15.86:6835","nonce":3621275047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:51.879542+0000","modified":"2024-04-25T00:15:20.022793+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7719,"mds_1":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7719":{"gid":7719,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6835/565131724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":565131724},{"type":"v1","addr":"172.21.15.83:6835","nonce":565131724}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7728":{"gid":7728,"name":"a","rank":1,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2870595352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2870595352},{"type":"v1","addr":"172.21.15.83:6837","nonce":2870595352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7719,"qdb_cluster":[7728,7719]},"id":7}]} 2024-04-25T00:15:29.791 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 74 2024-04-25T00:15:29.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.790+0000 7f082a800640 1 -- 172.21.15.83:0/3071660964 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08000678c0 msgr2=0x7f0800069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.790+0000 7f082a800640 1 --2- 172.21.15.83:0/3071660964 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08000678c0 0x7f0800069d80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f08140ae000 tx=0x7f08140a7070 comp rx=0 tx=0).stop 2024-04-25T00:15:29.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.790+0000 7f082a800640 1 -- 172.21.15.83:0/3071660964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240b01e0 msgr2=0x7f0824165300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.790+0000 7f082a800640 1 --2- 172.21.15.83:0/3071660964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240b01e0 0x7f0824165300 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f080c00d5f0 tx=0x7f080c00dab0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.790+0000 7f082a800640 1 -- 172.21.15.83:0/3071660964 shutdown_connections 2024-04-25T00:15:29.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.790+0000 7f082a800640 1 --2- 172.21.15.83:0/3071660964 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f08000678c0 0x7f0800069d80 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.790+0000 7f082a800640 1 --2- 172.21.15.83:0/3071660964 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f082415bf60 0x7f0824156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.790+0000 7f082a800640 1 --2- 172.21.15.83:0/3071660964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240b01e0 0x7f0824165300 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.790+0000 7f082a800640 1 --2- 172.21.15.83:0/3071660964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0824067490 0x7f08241fae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.790+0000 7f082a800640 1 -- 172.21.15.83:0/3071660964 >> 172.21.15.83:0/3071660964 conn(0x7f08240d0cd0 msgr2=0x7f08240af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:29.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.790+0000 7f082a800640 1 -- 172.21.15.83:0/3071660964 shutdown_connections 2024-04-25T00:15:29.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.790+0000 7f082a800640 1 -- 172.21.15.83:0/3071660964 wait complete. 2024-04-25T00:15:29.812 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.946+0000 7f2b6aa00640 1 Processor -- start 2024-04-25T00:15:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.946+0000 7f2b6aa00640 1 -- start start 2024-04-25T00:15:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.946+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b640b1450 0x7f2b640b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.946+0000 7f2b6aa00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b640d1dd0 con 0x7f2b640b3e20 2024-04-25T00:15:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.946+0000 7f2b6aa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b640d20b0 con 0x7f2b640b1070 2024-04-25T00:15:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.946+0000 7f2b6aa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2b640d2390 con 0x7f2b640b1450 2024-04-25T00:15:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.950+0000 7f2b68a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2b640b3e20 0x7f2b64158070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37802/0 (socket says 172.21.15.83:37802) 2024-04-25T00:15:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.950+0000 7f2b68a00640 1 -- 172.21.15.83:0/1138405472 learned_addr learned my addr 172.21.15.83:0/1138405472 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.950+0000 7f2b63400640 1 --2- 172.21.15.83:0/1138405472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b640b1450 0x7f2b640b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.950+0000 7f2b63400640 1 -- 172.21.15.83:0/1138405472 >> v1:172.21.15.86:6789/0 conn(0x7f2b640b1070 legacy=0x7f2b6415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.950+0000 7f2b63400640 1 -- 172.21.15.83:0/1138405472 >> v1:172.21.15.83:6789/0 conn(0x7f2b640b3e20 legacy=0x7f2b64158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.950+0000 7f2b63400640 1 -- 172.21.15.83:0/1138405472 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b640d2670 con 0x7f2b640b1450 2024-04-25T00:15:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.950+0000 7f2b63400640 1 --2- 172.21.15.83:0/1138405472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b640b1450 0x7f2b640b38e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2b500097f0 tx=0x7f2b5001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83262a23f09aaad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b62a00640 1 -- 172.21.15.83:0/1138405472 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b500b0070 con 0x7f2b640b1450 2024-04-25T00:15:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b62a00640 1 -- 172.21.15.83:0/1138405472 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b500ab600 con 0x7f2b640b1450 2024-04-25T00:15:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b62a00640 1 -- 172.21.15.83:0/1138405472 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b500ab9d0 con 0x7f2b640b1450 2024-04-25T00:15:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1138405472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b640b1450 msgr2=0x7f2b640b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 --2- 172.21.15.83:0/1138405472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b640b1450 0x7f2b640b38e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2b500097f0 tx=0x7f2b5001c450 comp rx=0 tx=0).stop 2024-04-25T00:15:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1138405472 shutdown_connections 2024-04-25T00:15:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 --2- 172.21.15.83:0/1138405472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b640b1450 0x7f2b640b38e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1138405472 >> 172.21.15.83:0/1138405472 conn(0x7f2b640b71b0 msgr2=0x7f2b640b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1138405472 shutdown_connections 2024-04-25T00:15:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1138405472 wait complete. 2024-04-25T00:15:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 Processor -- start 2024-04-25T00:15:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 -- start start 2024-04-25T00:15:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b640b1070 0x7f2b64153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b640b1450 0x7f2b640ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b640b3e20 0x7f2b640d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6416c900 con 0x7f2b640b1070 2024-04-25T00:15:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b640b4250 con 0x7f2b640b3e20 2024-04-25T00:15:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b6aa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6416ca80 con 0x7f2b640b1450 2024-04-25T00:15:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b63e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b640b1070 0x7f2b64153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b63e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b640b1070 0x7f2b64153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57784/0 (socket says 172.21.15.83:57784) 2024-04-25T00:15:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b63e00640 1 -- 172.21.15.83:0/1987532512 learned_addr learned my addr 172.21.15.83:0/1987532512 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b68a00640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b640b3e20 0x7f2b640d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b63400640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b640b1450 0x7f2b640ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b63e00640 1 -- 172.21.15.83:0/1987532512 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b640b1450 msgr2=0x7f2b640ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b63e00640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b640b1450 0x7f2b640ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b63e00640 1 -- 172.21.15.83:0/1987532512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b640b3e20 msgr2=0x7f2b640d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.954+0000 7f2b63e00640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b640b3e20 0x7f2b640d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.958+0000 7f2b63e00640 1 -- 172.21.15.83:0/1987532512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b64154120 con 0x7f2b640b1070 2024-04-25T00:15:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.958+0000 7f2b68a00640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b640b3e20 0x7f2b640d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.958+0000 7f2b63e00640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b640b1070 0x7f2b64153be0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400c420 tx=0x7f2b5400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.958+0000 7f2b5be00640 1 -- 172.21.15.83:0/1987532512 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b540052d0 con 0x7f2b640b1070 2024-04-25T00:15:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.958+0000 7f2b5be00640 1 -- 172.21.15.83:0/1987532512 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b54005470 con 0x7f2b640b1070 2024-04-25T00:15:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.958+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1987532512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b640ceee0 con 0x7f2b640b1070 2024-04-25T00:15:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.958+0000 7f2b5be00640 1 -- 172.21.15.83:0/1987532512 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54007260 con 0x7f2b640b1070 2024-04-25T00:15:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.958+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1987532512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b640cf4a0 con 0x7f2b640b1070 2024-04-25T00:15:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.958+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1987532512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b6404f510 con 0x7f2b640b1070 2024-04-25T00:15:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.958+0000 7f2b5be00640 1 -- 172.21.15.83:0/1987532512 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2b54007400 con 0x7f2b640b1070 2024-04-25T00:15:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.958+0000 7f2b5be00640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2b440678c0 0x7f2b44069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.958+0000 7f2b5be00640 1 -- 172.21.15.83:0/1987532512 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f2b540852d0 con 0x7f2b640b1070 2024-04-25T00:15:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.958+0000 7f2b63400640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2b440678c0 0x7f2b44069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.962+0000 7f2b63400640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2b440678c0 0x7f2b44069d80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2b500ad000 tx=0x7f2b50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:29.962+0000 7f2b5be00640 1 -- 172.21.15.83:0/1987532512 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2b54055610 con 0x7f2b640b1070 2024-04-25T00:15:30.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.154+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1987532512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b6415eca0 con 0x7f2b640b1070 2024-04-25T00:15:30.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.154+0000 7f2b5be00640 1 -- 172.21.15.83:0/1987532512 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5201 (secure 0 0 0) 0x7f2b54059910 con 0x7f2b640b1070 2024-04-25T00:15:30.154 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:30.154 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1992401755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1992401755},{"type":"v1","addr":"172.21.15.86:6833","nonce":1992401755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3621275047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3621275047},{"type":"v1","addr":"172.21.15.86:6835","nonce":3621275047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:51.879542+0000","modified":"2024-04-25T00:15:20.022793+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7719,"mds_1":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7719":{"gid":7719,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6835/565131724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":565131724},{"type":"v1","addr":"172.21.15.83:6835","nonce":565131724}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7728":{"gid":7728,"name":"a","rank":1,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2870595352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2870595352},{"type":"v1","addr":"172.21.15.83:6837","nonce":2870595352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7719,"qdb_cluster":[7728,7719]},"id":7}]} 2024-04-25T00:15:30.154 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 74 2024-04-25T00:15:30.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.154+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1987532512 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2b440678c0 msgr2=0x7f2b44069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.154+0000 7f2b6aa00640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2b440678c0 0x7f2b44069d80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2b500ad000 tx=0x7f2b50008040 comp rx=0 tx=0).stop 2024-04-25T00:15:30.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.154+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1987532512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b640b1070 msgr2=0x7f2b64153be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.154+0000 7f2b6aa00640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b640b1070 0x7f2b64153be0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400c420 tx=0x7f2b5400c8e0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.154+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1987532512 shutdown_connections 2024-04-25T00:15:30.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.154+0000 7f2b6aa00640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2b440678c0 0x7f2b44069d80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.154+0000 7f2b6aa00640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2b640b1450 0x7f2b640ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.154+0000 7f2b6aa00640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b640b3e20 0x7f2b640d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.154+0000 7f2b6aa00640 1 --2- 172.21.15.83:0/1987532512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b640b1070 0x7f2b64153be0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.154+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1987532512 >> 172.21.15.83:0/1987532512 conn(0x7f2b640b71b0 msgr2=0x7f2b6406ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:30.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.154+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1987532512 shutdown_connections 2024-04-25T00:15:30.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.154+0000 7f2b6aa00640 1 -- 172.21.15.83:0/1987532512 wait complete. 2024-04-25T00:15:30.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:15:30.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95cdc00640 1 Processor -- start 2024-04-25T00:15:30.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95cdc00640 1 -- start start 2024-04-25T00:15:30.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95cdc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f95c8156340 0x7f95c80acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95cdc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c815ce00 0x7f95c8070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95cdc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95c806ecf0 con 0x7f95c80ae680 2024-04-25T00:15:30.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95cdc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f95c806efd0 con 0x7f95c8156340 2024-04-25T00:15:30.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95cdc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f95c806cef0 con 0x7f95c815ce00 2024-04-25T00:15:30.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95c6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f95c8156340 0x7f95c80acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95c6000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c815ce00 0x7f95c8070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95c6000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c815ce00 0x7f95c8070e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:42348/0 (socket says 172.21.15.83:42348) 2024-04-25T00:15:30.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95c6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f95c8156340 0x7f95c80acd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:59972/0 (socket says 172.21.15.83:59972) 2024-04-25T00:15:30.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95c6000640 1 -- 172.21.15.83:0/3541798410 learned_addr learned my addr 172.21.15.83:0/3541798410 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:30.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95c6000640 1 -- 172.21.15.83:0/3541798410 >> v2:172.21.15.86:3300/0 conn(0x7f95c8156340 msgr2=0x7f95c80acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:30.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95c6000640 1 --2- 172.21.15.83:0/3541798410 >> v2:172.21.15.86:3300/0 conn(0x7f95c8156340 0x7f95c80acd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95c6000640 1 -- 172.21.15.83:0/3541798410 >> v1:172.21.15.86:6789/0 conn(0x7f95c80ae680 legacy=0x7f95c80aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.310+0000 7f95c6000640 1 -- 172.21.15.83:0/3541798410 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c815d290 con 0x7f95c815ce00 2024-04-25T00:15:30.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.314+0000 7f95c6000640 1 --2- 172.21.15.83:0/3541798410 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c815ce00 0x7f95c8070e40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009870 tx=0x7f95b401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0485b895bc548cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:30.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.314+0000 7f95c5600640 1 -- 172.21.15.83:0/3541798410 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b40b2070 con 0x7f95c815ce00 2024-04-25T00:15:30.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.314+0000 7f95c5600640 1 -- 172.21.15.83:0/3541798410 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b40a9070 con 0x7f95c815ce00 2024-04-25T00:15:30.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.314+0000 7f95c5600640 1 -- 172.21.15.83:0/3541798410 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b40ad880 con 0x7f95c815ce00 2024-04-25T00:15:30.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.314+0000 7f95cdc00640 1 -- 172.21.15.83:0/3541798410 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c815ce00 msgr2=0x7f95c8070e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.314+0000 7f95cdc00640 1 --2- 172.21.15.83:0/3541798410 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c815ce00 0x7f95c8070e40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009870 tx=0x7f95b401c560 comp rx=0 tx=0).stop 2024-04-25T00:15:30.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.314+0000 7f95cdc00640 1 -- 172.21.15.83:0/3541798410 shutdown_connections 2024-04-25T00:15:30.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.314+0000 7f95cdc00640 1 --2- 172.21.15.83:0/3541798410 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c815ce00 0x7f95c8070e40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.314+0000 7f95cdc00640 1 --2- 172.21.15.83:0/3541798410 >> v2:172.21.15.86:3300/0 conn(0x7f95c8156340 0x7f95c80acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.314+0000 7f95cdc00640 1 -- 172.21.15.83:0/3541798410 >> 172.21.15.83:0/3541798410 conn(0x7f95c80b1860 msgr2=0x7f95c80b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:30.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.314+0000 7f95cdc00640 1 -- 172.21.15.83:0/3541798410 shutdown_connections 2024-04-25T00:15:30.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.314+0000 7f95cdc00640 1 -- 172.21.15.83:0/3541798410 wait complete. 2024-04-25T00:15:30.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.314+0000 7f95cdc00640 1 Processor -- start 2024-04-25T00:15:30.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95cdc00640 1 -- start start 2024-04-25T00:15:30.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95cdc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c80ae680 0x7f95c81503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95cdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95c8156340 0x7f95c81f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95cdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95c8150dd0 0x7f95c8151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95cdc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95c806dff0 con 0x7f95c8150dd0 2024-04-25T00:15:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95cdc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95c806de70 con 0x7f95c8156340 2024-04-25T00:15:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95cdc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f95c8168d10 con 0x7f95c80ae680 2024-04-25T00:15:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95c7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95c8150dd0 0x7f95c8151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95c6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95c8156340 0x7f95c81f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95c6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95c8156340 0x7f95c81f70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:59986/0 (socket says 172.21.15.83:59986) 2024-04-25T00:15:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95c6000640 1 -- 172.21.15.83:0/1011658992 learned_addr learned my addr 172.21.15.83:0/1011658992 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95c7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95c8150dd0 0x7f95c8151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57800/0 (socket says 172.21.15.83:57800) 2024-04-25T00:15:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95c6a00640 1 --2- 172.21.15.83:0/1011658992 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c80ae680 0x7f95c81503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95c7400640 1 -- 172.21.15.83:0/1011658992 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c80ae680 msgr2=0x7f95c81503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95c7400640 1 --2- 172.21.15.83:0/1011658992 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c80ae680 0x7f95c81503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95c7400640 1 -- 172.21.15.83:0/1011658992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95c8156340 msgr2=0x7f95c81f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95c7400640 1 --2- 172.21.15.83:0/1011658992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95c8156340 0x7f95c81f70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95c7400640 1 -- 172.21.15.83:0/1011658992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c81f7600 con 0x7f95c8150dd0 2024-04-25T00:15:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95c6000640 1 --2- 172.21.15.83:0/1011658992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95c8156340 0x7f95c81f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95c7400640 1 --2- 172.21.15.83:0/1011658992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95c8150dd0 0x7f95c8151280 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f95c000c730 tx=0x7f95c000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:30.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95bea00640 1 -- 172.21.15.83:0/1011658992 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0003b80 con 0x7f95c8150dd0 2024-04-25T00:15:30.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95bea00640 1 -- 172.21.15.83:0/1011658992 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c0003d20 con 0x7f95c8150dd0 2024-04-25T00:15:30.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95bea00640 1 -- 172.21.15.83:0/1011658992 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0005310 con 0x7f95c8150dd0 2024-04-25T00:15:30.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95cdc00640 1 -- 172.21.15.83:0/1011658992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c8153dc0 con 0x7f95c8150dd0 2024-04-25T00:15:30.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95cdc00640 1 -- 172.21.15.83:0/1011658992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c8154ae0 con 0x7f95c8150dd0 2024-04-25T00:15:30.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.318+0000 7f95cdc00640 1 -- 172.21.15.83:0/1011658992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c804f510 con 0x7f95c8150dd0 2024-04-25T00:15:30.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.322+0000 7f95bea00640 1 -- 172.21.15.83:0/1011658992 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f95c00056c0 con 0x7f95c8150dd0 2024-04-25T00:15:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.322+0000 7f95bea00640 1 --2- 172.21.15.83:0/1011658992 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95a80678f0 0x7f95a8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.322+0000 7f95bea00640 1 -- 172.21.15.83:0/1011658992 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f95c0085dd0 con 0x7f95c8150dd0 2024-04-25T00:15:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.322+0000 7f95c6a00640 1 --2- 172.21.15.83:0/1011658992 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95a80678f0 0x7f95a8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.322+0000 7f95c6a00640 1 --2- 172.21.15.83:0/1011658992 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95a80678f0 0x7f95a8069db0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009ac0 tx=0x7f95b8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:30.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.326+0000 7f95bea00640 1 -- 172.21.15.83:0/1011658992 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f95c81f7600 con 0x7f95c8150dd0 2024-04-25T00:15:30.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.482+0000 7f95cdc00640 1 -- 172.21.15.83:0/1011658992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95c8072ba0 con 0x7f95c8150dd0 2024-04-25T00:15:30.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.482+0000 7f95bea00640 1 -- 172.21.15.83:0/1011658992 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+17048 (secure 0 0 0) 0x7f95c0055fe0 con 0x7f95c8150dd0 2024-04-25T00:15:30.484 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:30.484 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":97,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:15:04.852008+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:14:50.260735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:50.880394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1609715757":"2024-04-26T00:15:04.845080+0000","172.21.15.83:6836/1609715757":"2024-04-26T00:15:04.845080+0000","172.21.15.86:6835/3612679148":"2024-04-26T00:15:04.826082+0000","172.21.15.86:6833/3612679148":"2024-04-26T00:15:04.826082+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:15:30.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.486+0000 7f95cdc00640 1 -- 172.21.15.83:0/1011658992 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95a80678f0 msgr2=0x7f95a8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.486+0000 7f95cdc00640 1 --2- 172.21.15.83:0/1011658992 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95a80678f0 0x7f95a8069db0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009ac0 tx=0x7f95b8009190 comp rx=0 tx=0).stop 2024-04-25T00:15:30.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.486+0000 7f95cdc00640 1 -- 172.21.15.83:0/1011658992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95c8150dd0 msgr2=0x7f95c8151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.486+0000 7f95cdc00640 1 --2- 172.21.15.83:0/1011658992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95c8150dd0 0x7f95c8151280 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f95c000c730 tx=0x7f95c000cbf0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.486+0000 7f95cdc00640 1 -- 172.21.15.83:0/1011658992 shutdown_connections 2024-04-25T00:15:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.486+0000 7f95cdc00640 1 --2- 172.21.15.83:0/1011658992 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f95a80678f0 0x7f95a8069db0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.486+0000 7f95cdc00640 1 --2- 172.21.15.83:0/1011658992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95c8150dd0 0x7f95c8151280 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.486+0000 7f95cdc00640 1 --2- 172.21.15.83:0/1011658992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95c8156340 0x7f95c81f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.486+0000 7f95cdc00640 1 --2- 172.21.15.83:0/1011658992 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c80ae680 0x7f95c81503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.486+0000 7f95cdc00640 1 -- 172.21.15.83:0/1011658992 >> 172.21.15.83:0/1011658992 conn(0x7f95c80b1860 msgr2=0x7f95c815c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.486+0000 7f95cdc00640 1 -- 172.21.15.83:0/1011658992 shutdown_connections 2024-04-25T00:15:30.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.486+0000 7f95cdc00640 1 -- 172.21.15.83:0/1011658992 wait complete. 2024-04-25T00:15:30.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:15:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f8995c00640 1 Processor -- start 2024-04-25T00:15:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f8995c00640 1 -- start start 2024-04-25T00:15:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f8995c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8990155300 0x7f89900cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f8995c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89900d0180 0x7f89900b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f8995c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89901556e0 con 0x7f89900b3180 2024-04-25T00:15:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f8995c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8990072260 con 0x7f89900d0180 2024-04-25T00:15:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f8995c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8990072540 con 0x7f8990155300 2024-04-25T00:15:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f898f400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f89900b3180 0x7f89900cec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37818/0 (socket says 172.21.15.83:37818) 2024-04-25T00:15:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f898f400640 1 -- 172.21.15.83:0/1986736779 learned_addr learned my addr 172.21.15.83:0/1986736779 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f8986000640 1 --2- 172.21.15.83:0/1986736779 >> v2:172.21.15.83:3300/0 conn(0x7f89900d0180 0x7f89900b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f898ea00640 1 --2- 172.21.15.83:0/1986736779 >> v2:172.21.15.86:3300/0 conn(0x7f8990155300 0x7f89900cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f8986000640 1 -- 172.21.15.83:0/1986736779 >> v2:172.21.15.86:3300/0 conn(0x7f8990155300 msgr2=0x7f89900cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f8986000640 1 --2- 172.21.15.83:0/1986736779 >> v2:172.21.15.86:3300/0 conn(0x7f8990155300 0x7f89900cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f8986000640 1 -- 172.21.15.83:0/1986736779 >> v1:172.21.15.83:6789/0 conn(0x7f89900b3180 legacy=0x7f89900cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f8986000640 1 -- 172.21.15.83:0/1986736779 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8990072820 con 0x7f89900d0180 2024-04-25T00:15:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.646+0000 7f898e000640 1 -- 172.21.15.83:0/1986736779 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1976704981 0 0) 0x7f89901556e0 con 0x7f89900b3180 2024-04-25T00:15:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.650+0000 7f8986000640 1 --2- 172.21.15.83:0/1986736779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89900d0180 0x7f89900b2c40 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f8990070870 tx=0x7f897801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6bd2a02ff51bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.650+0000 7f898e000640 1 -- 172.21.15.83:0/1986736779 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89780ab610 con 0x7f89900d0180 2024-04-25T00:15:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.650+0000 7f898e000640 1 -- 172.21.15.83:0/1986736779 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89780b0070 con 0x7f89900d0180 2024-04-25T00:15:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.650+0000 7f898e000640 1 -- 172.21.15.83:0/1986736779 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89780ab9d0 con 0x7f89900d0180 2024-04-25T00:15:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.650+0000 7f8995c00640 1 -- 172.21.15.83:0/1986736779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89900d0180 msgr2=0x7f89900b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.650+0000 7f8995c00640 1 --2- 172.21.15.83:0/1986736779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89900d0180 0x7f89900b2c40 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f8990070870 tx=0x7f897801c3f0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.650+0000 7f8995c00640 1 -- 172.21.15.83:0/1986736779 shutdown_connections 2024-04-25T00:15:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.650+0000 7f8995c00640 1 --2- 172.21.15.83:0/1986736779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89900d0180 0x7f89900b2c40 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.650+0000 7f8995c00640 1 --2- 172.21.15.83:0/1986736779 >> v2:172.21.15.86:3300/0 conn(0x7f8990155300 0x7f89900cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.650+0000 7f8995c00640 1 -- 172.21.15.83:0/1986736779 >> 172.21.15.83:0/1986736779 conn(0x7f89900b1b10 msgr2=0x7f89900b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.650+0000 7f8995c00640 1 -- 172.21.15.83:0/1986736779 shutdown_connections 2024-04-25T00:15:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.650+0000 7f8995c00640 1 -- 172.21.15.83:0/1986736779 wait complete. 2024-04-25T00:15:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.650+0000 7f8995c00640 1 Processor -- start 2024-04-25T00:15:30.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.650+0000 7f8995c00640 1 -- start start 2024-04-25T00:15:30.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8995c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89900b3180 0x7f899009b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8995c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89900d0180 0x7f899009d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8995c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8990155300 0x7f8990152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8995c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f899015b2e0 con 0x7f89900b3180 2024-04-25T00:15:30.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8995c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f899015b460 con 0x7f89900d0180 2024-04-25T00:15:30.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8995c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f899015b5e0 con 0x7f8990155300 2024-04-25T00:15:30.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f898ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89900b3180 0x7f899009b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f898ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89900b3180 0x7f899009b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57826/0 (socket says 172.21.15.83:57826) 2024-04-25T00:15:30.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f898ea00640 1 -- 172.21.15.83:0/3605491251 learned_addr learned my addr 172.21.15.83:0/3605491251 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:30.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8986000640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89900d0180 0x7f899009d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f898f400640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8990155300 0x7f8990152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f898ea00640 1 -- 172.21.15.83:0/3605491251 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8990155300 msgr2=0x7f8990152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f898ea00640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8990155300 0x7f8990152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f898ea00640 1 -- 172.21.15.83:0/3605491251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89900d0180 msgr2=0x7f899009d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f898ea00640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89900d0180 0x7f899009d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f898ea00640 1 -- 172.21.15.83:0/3605491251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899015cb30 con 0x7f89900b3180 2024-04-25T00:15:30.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8986000640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89900d0180 0x7f899009d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:30.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f898ea00640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89900b3180 0x7f899009b1e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f898000d5f0 tx=0x7f898000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:30.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8987400640 1 -- 172.21.15.83:0/3605491251 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89800125f0 con 0x7f89900b3180 2024-04-25T00:15:30.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8987400640 1 -- 172.21.15.83:0/3605491251 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8980019070 con 0x7f89900b3180 2024-04-25T00:15:30.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8995c00640 1 -- 172.21.15.83:0/3605491251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899015ce10 con 0x7f89900b3180 2024-04-25T00:15:30.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8987400640 1 -- 172.21.15.83:0/3605491251 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89800129d0 con 0x7f89900b3180 2024-04-25T00:15:30.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8995c00640 1 -- 172.21.15.83:0/3605491251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899015d3d0 con 0x7f89900b3180 2024-04-25T00:15:30.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8995c00640 1 -- 172.21.15.83:0/3605491251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899004f510 con 0x7f89900b3180 2024-04-25T00:15:30.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.654+0000 7f8987400640 1 -- 172.21.15.83:0/3605491251 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8980012b70 con 0x7f89900b3180 2024-04-25T00:15:30.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.658+0000 7f8987400640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f896c0678c0 0x7f896c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.658+0000 7f8987400640 1 -- 172.21.15.83:0/3605491251 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6273+0+0 (secure 0 0 0) 0x7f8980086440 con 0x7f89900b3180 2024-04-25T00:15:30.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.658+0000 7f8986000640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f896c0678c0 0x7f896c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.658+0000 7f8986000640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f896c0678c0 0x7f896c069d80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f89780af000 tx=0x7f89780a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:30.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.662+0000 7f8987400640 1 -- 172.21.15.83:0/3605491251 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8980056ca0 con 0x7f89900b3180 2024-04-25T00:15:30.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.858+0000 7f8995c00640 1 -- 172.21.15.83:0/3605491251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f89900b1ef0 con 0x7f89900b3180 2024-04-25T00:15:30.861 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-25T00:15:30.858+0000 7efeeee00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:15:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.862+0000 7f8987400640 1 -- 172.21.15.83:0/3605491251 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) ==== 161+0+0 (secure 0 0 0) 0x7f89800565a0 con 0x7f89900b3180 2024-04-25T00:15:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:15:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.862+0000 7f8995c00640 1 -- 172.21.15.83:0/3605491251 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f896c0678c0 msgr2=0x7f896c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.862+0000 7f8995c00640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f896c0678c0 0x7f896c069d80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f89780af000 tx=0x7f89780a7000 comp rx=0 tx=0).stop 2024-04-25T00:15:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.862+0000 7f8995c00640 1 -- 172.21.15.83:0/3605491251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89900b3180 msgr2=0x7f899009b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.862+0000 7f8995c00640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89900b3180 0x7f899009b1e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f898000d5f0 tx=0x7f898000dab0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.866+0000 7f8995c00640 1 -- 172.21.15.83:0/3605491251 shutdown_connections 2024-04-25T00:15:30.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.866+0000 7f8995c00640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f896c0678c0 0x7f896c069d80 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.866+0000 7f8995c00640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8990155300 0x7f8990152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.866+0000 7f8995c00640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89900d0180 0x7f899009d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.866+0000 7f8995c00640 1 --2- 172.21.15.83:0/3605491251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89900b3180 0x7f899009b1e0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.866+0000 7f8995c00640 1 -- 172.21.15.83:0/3605491251 >> 172.21.15.83:0/3605491251 conn(0x7f89900b1b10 msgr2=0x7f89900b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:30.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.866+0000 7f8995c00640 1 -- 172.21.15.83:0/3605491251 shutdown_connections 2024-04-25T00:15:30.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:30.866+0000 7f8995c00640 1 -- 172.21.15.83:0/3605491251 wait complete. 2024-04-25T00:15:30.880 INFO:tasks.ceph.mds.c.smithi083.stderr: -4720> 2024-04-25T00:15:10.858+0000 7fd6a8200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:15:30.888 INFO:tasks.ceph.mds.c.smithi083.stderr: -2581> 2024-04-25T00:15:15.858+0000 7fd6a8200640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T00:15:30.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:15:30.913 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-25T00:15:30.937 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-25T00:15:31.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.038+0000 7f366f000640 1 Processor -- start 2024-04-25T00:15:31.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.038+0000 7f366f000640 1 -- start start 2024-04-25T00:15:31.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f366f000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f366806d5d0 0x7f366815f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:31.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f366f000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f366815c200 con 0x7f366815be20 2024-04-25T00:15:31.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f366f000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3668160100 con 0x7f36680cedd0 2024-04-25T00:15:31.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f366f000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f366815fed0 con 0x7f366806d5d0 2024-04-25T00:15:31.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3667e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f366815be20 0x7f366806cec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37836/0 (socket says 172.21.15.83:37836) 2024-04-25T00:15:31.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3667e00640 1 -- 172.21.15.83:0/1433843059 learned_addr learned my addr 172.21.15.83:0/1433843059 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:31.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3667400640 1 --2- 172.21.15.83:0/1433843059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f366806d5d0 0x7f366815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:31.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 -- 172.21.15.83:0/1433843059 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 172468233 0 0) 0x7f366815c200 con 0x7f366815be20 2024-04-25T00:15:31.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 -- 172.21.15.83:0/1433843059 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f364c003530 con 0x7f366815be20 2024-04-25T00:15:31.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 -- 172.21.15.83:0/1433843059 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3812164420 0 0) 0x7f364c003530 con 0x7f366815be20 2024-04-25T00:15:31.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 -- 172.21.15.83:0/1433843059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f366806d5d0 msgr2=0x7f366815f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:31.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 --2- 172.21.15.83:0/1433843059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f366806d5d0 0x7f366815f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 -- 172.21.15.83:0/1433843059 >> v1:172.21.15.86:6789/0 conn(0x7f36680cedd0 legacy=0x7f36680b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:31.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 -- 172.21.15.83:0/1433843059 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36681603e0 con 0x7f366815be20 2024-04-25T00:15:31.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 -- 172.21.15.83:0/1433843059 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f3658001a00 con 0x7f366815be20 2024-04-25T00:15:31.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 -- 172.21.15.83:0/1433843059 >> v1:172.21.15.83:6789/0 conn(0x7f366815be20 legacy=0x7f366806cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:31.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 --2- 172.21.15.83:0/1433843059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0039f0 0x7f364c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:31.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 --2- 172.21.15.83:0/1433843059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f366806d5d0 0x7f364c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:31.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 --2- 172.21.15.83:0/1433843059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f364c004d50 0x7f364c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:31.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 -- 172.21.15.83:0/1433843059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f364c00b9d0 con 0x7f364c0039f0 2024-04-25T00:15:31.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 -- 172.21.15.83:0/1433843059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f364c00bb50 con 0x7f364c004d50 2024-04-25T00:15:31.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3666a00640 1 -- 172.21.15.83:0/1433843059 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f364c00bd50 con 0x7f366806d5d0 2024-04-25T00:15:31.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3667e00640 1 --2- 172.21.15.83:0/1433843059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0039f0 0x7f364c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:31.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3667400640 1 --2- 172.21.15.83:0/1433843059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f366806d5d0 0x7f364c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:31.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f366d000640 1 --2- 172.21.15.83:0/1433843059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f364c004d50 0x7f364c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:31.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3667e00640 1 -- 172.21.15.83:0/1433843059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f366806d5d0 msgr2=0x7f364c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:31.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3667e00640 1 --2- 172.21.15.83:0/1433843059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f366806d5d0 0x7f364c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3667e00640 1 -- 172.21.15.83:0/1433843059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f364c004d50 msgr2=0x7f364c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:31.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3667e00640 1 --2- 172.21.15.83:0/1433843059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f364c004d50 0x7f364c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.042+0000 7f3667e00640 1 -- 172.21.15.83:0/1433843059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364c003530 con 0x7f364c0039f0 2024-04-25T00:15:31.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.046+0000 7f3667e00640 1 --2- 172.21.15.83:0/1433843059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0039f0 0x7f364c003dd0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f365800a7c0 tx=0x7f365801d240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e8f292c17966a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:31.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.046+0000 7f3666a00640 1 -- 172.21.15.83:0/1433843059 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365800a410 con 0x7f364c0039f0 2024-04-25T00:15:31.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.046+0000 7f3666a00640 1 -- 172.21.15.83:0/1433843059 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36580ad630 con 0x7f364c0039f0 2024-04-25T00:15:31.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.046+0000 7f3666a00640 1 -- 172.21.15.83:0/1433843059 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36580ada20 con 0x7f364c0039f0 2024-04-25T00:15:31.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.046+0000 7f366f000640 1 -- 172.21.15.83:0/1433843059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0039f0 msgr2=0x7f364c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:31.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.046+0000 7f366f000640 1 --2- 172.21.15.83:0/1433843059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0039f0 0x7f364c003dd0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f365800a7c0 tx=0x7f365801d240 comp rx=0 tx=0).stop 2024-04-25T00:15:31.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.046+0000 7f366d000640 1 -- 172.21.15.83:0/1433843059 reap_dead start 2024-04-25T00:15:31.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.046+0000 7f366f000640 1 -- 172.21.15.83:0/1433843059 shutdown_connections 2024-04-25T00:15:31.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.046+0000 7f366f000640 1 -- 172.21.15.83:0/1433843059 >> 172.21.15.83:0/1433843059 conn(0x7f36680b1b10 msgr2=0x7f36680b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:31.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.046+0000 7f366f000640 1 -- 172.21.15.83:0/1433843059 shutdown_connections 2024-04-25T00:15:31.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.046+0000 7f366f000640 1 -- 172.21.15.83:0/1433843059 wait complete. 2024-04-25T00:15:31.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.046+0000 7f366f000640 1 Processor -- start 2024-04-25T00:15:31.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.046+0000 7f366f000640 1 -- start start 2024-04-25T00:15:31.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f366f000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0039f0 0x7f36680ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:31.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f366f000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3668166c80 0x7f366815e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:31.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f366f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3668167100 0x7f366815ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:31.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f366f000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3668167f40 con 0x7f364c0039f0 2024-04-25T00:15:31.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f366f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36680b2e70 con 0x7f3668167100 2024-04-25T00:15:31.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f366f000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f36681680c0 con 0x7f3668166c80 2024-04-25T00:15:31.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f3667e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0039f0 0x7f36680ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:31.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f3667e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0039f0 0x7f36680ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57886/0 (socket says 172.21.15.83:57886) 2024-04-25T00:15:31.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f3667e00640 1 -- 172.21.15.83:0/2708295779 learned_addr learned my addr 172.21.15.83:0/2708295779 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:31.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f3667e00640 1 -- 172.21.15.83:0/2708295779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3668166c80 msgr2=0x7f366815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:31.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f3667e00640 1 --2- 172.21.15.83:0/2708295779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3668166c80 0x7f366815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f3667e00640 1 -- 172.21.15.83:0/2708295779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3668167100 msgr2=0x7f366815ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:31.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f366d000640 1 --2- 172.21.15.83:0/2708295779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3668167100 0x7f366815ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:31.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f3667e00640 1 --2- 172.21.15.83:0/2708295779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3668167100 0x7f366815ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f3667e00640 1 -- 172.21.15.83:0/2708295779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366809ab90 con 0x7f364c0039f0 2024-04-25T00:15:31.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f3667e00640 1 --2- 172.21.15.83:0/2708295779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0039f0 0x7f36680ae840 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f365801d070 tx=0x7f364c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:31.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f365fe00640 1 -- 172.21.15.83:0/2708295779 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36580ad430 con 0x7f364c0039f0 2024-04-25T00:15:31.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f365fe00640 1 -- 172.21.15.83:0/2708295779 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36580017c0 con 0x7f364c0039f0 2024-04-25T00:15:31.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f366f000640 1 -- 172.21.15.83:0/2708295779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366809ae30 con 0x7f364c0039f0 2024-04-25T00:15:31.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f365fe00640 1 -- 172.21.15.83:0/2708295779 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658001970 con 0x7f364c0039f0 2024-04-25T00:15:31.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f366f000640 1 -- 172.21.15.83:0/2708295779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366815c410 con 0x7f364c0039f0 2024-04-25T00:15:31.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f366f000640 1 -- 172.21.15.83:0/2708295779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366804f510 con 0x7f364c0039f0 2024-04-25T00:15:31.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.050+0000 7f365fe00640 1 -- 172.21.15.83:0/2708295779 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f36580bc030 con 0x7f364c0039f0 2024-04-25T00:15:31.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.054+0000 7f365fe00640 1 --2- 172.21.15.83:0/2708295779 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f36480678f0 0x7f3648069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:31.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.054+0000 7f365fe00640 1 -- 172.21.15.83:0/2708295779 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6445+0+0 (secure 0 0 0) 0x7f36581201b0 con 0x7f364c0039f0 2024-04-25T00:15:31.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.054+0000 7f3667400640 1 --2- 172.21.15.83:0/2708295779 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f36480678f0 0x7f3648069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:31.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.054+0000 7f3667400640 1 --2- 172.21.15.83:0/2708295779 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f36480678f0 0x7f3648069db0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3654009a00 tx=0x7f3654006c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:31.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.058+0000 7f365fe00640 1 -- 172.21.15.83:0/2708295779 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f36580c3020 con 0x7f364c0039f0 2024-04-25T00:15:31.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.246+0000 7f366f000640 1 -- 172.21.15.83:0/2708295779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f366806ea90 con 0x7f364c0039f0 2024-04-25T00:15:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.862+0000 7f365fe00640 1 -- 172.21.15.83:0/2708295779 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) ==== 104+0+0 (secure 0 0 0) 0x7f3658016f50 con 0x7f364c0039f0 2024-04-25T00:15:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.866+0000 7f366f000640 1 -- 172.21.15.83:0/2708295779 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f36480678f0 msgr2=0x7f3648069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.866+0000 7f366f000640 1 --2- 172.21.15.83:0/2708295779 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f36480678f0 0x7f3648069db0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3654009a00 tx=0x7f3654006c60 comp rx=0 tx=0).stop 2024-04-25T00:15:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.866+0000 7f366f000640 1 -- 172.21.15.83:0/2708295779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0039f0 msgr2=0x7f36680ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.866+0000 7f366f000640 1 --2- 172.21.15.83:0/2708295779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0039f0 0x7f36680ae840 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f365801d070 tx=0x7f364c005840 comp rx=0 tx=0).stop 2024-04-25T00:15:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.866+0000 7f366f000640 1 -- 172.21.15.83:0/2708295779 shutdown_connections 2024-04-25T00:15:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.866+0000 7f366f000640 1 --2- 172.21.15.83:0/2708295779 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f36480678f0 0x7f3648069db0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.866+0000 7f366f000640 1 --2- 172.21.15.83:0/2708295779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3668166c80 0x7f366815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.866+0000 7f366f000640 1 --2- 172.21.15.83:0/2708295779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3668167100 0x7f366815ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.866+0000 7f366f000640 1 --2- 172.21.15.83:0/2708295779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0039f0 0x7f36680ae840 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.866+0000 7f366f000640 1 -- 172.21.15.83:0/2708295779 >> 172.21.15.83:0/2708295779 conn(0x7f36680b1b10 msgr2=0x7f3668152270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:31.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.866+0000 7f366f000640 1 -- 172.21.15.83:0/2708295779 shutdown_connections 2024-04-25T00:15:31.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:31.866+0000 7f366f000640 1 -- 172.21.15.83:0/2708295779 wait complete. 2024-04-25T00:15:31.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:15:32.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.074+0000 7fe2aa200640 1 Processor -- start 2024-04-25T00:15:32.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.074+0000 7fe2aa200640 1 -- start start 2024-04-25T00:15:32.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2aa200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a40d0630 0x7fe2a40ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2aa200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a4071d80 con 0x7fe2a40cf2a0 2024-04-25T00:15:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2aa200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a4070fc0 con 0x7fe2a415bf10 2024-04-25T00:15:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2aa200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a40712a0 con 0x7fe2a40d0630 2024-04-25T00:15:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a3e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe2a40cf2a0 0x7fe2a406cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37844/0 (socket says 172.21.15.83:37844) 2024-04-25T00:15:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a3e00640 1 -- 172.21.15.83:0/1749893950 learned_addr learned my addr 172.21.15.83:0/1749893950 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:32.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2a00640 1 --2- 172.21.15.83:0/1749893950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a40d0630 0x7fe2a40ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 -- 172.21.15.83:0/1749893950 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2257742083 0 0) 0x7fe2a4071d80 con 0x7fe2a40cf2a0 2024-04-25T00:15:32.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 -- 172.21.15.83:0/1749893950 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe288003530 con 0x7fe2a40cf2a0 2024-04-25T00:15:32.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 -- 172.21.15.83:0/1749893950 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 152823566 0 0) 0x7fe288003530 con 0x7fe2a40cf2a0 2024-04-25T00:15:32.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 -- 172.21.15.83:0/1749893950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a40d0630 msgr2=0x7fe2a40ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:32.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 --2- 172.21.15.83:0/1749893950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a40d0630 0x7fe2a40ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 -- 172.21.15.83:0/1749893950 >> v1:172.21.15.86:6789/0 conn(0x7fe2a415bf10 legacy=0x7fe2a40cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 -- 172.21.15.83:0/1749893950 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a4071580 con 0x7fe2a40cf2a0 2024-04-25T00:15:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 -- 172.21.15.83:0/1749893950 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7fe29c001680 con 0x7fe2a40cf2a0 2024-04-25T00:15:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 -- 172.21.15.83:0/1749893950 >> v1:172.21.15.83:6789/0 conn(0x7fe2a40cf2a0 legacy=0x7fe2a406cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 --2- 172.21.15.83:0/1749893950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2880039f0 0x7fe288003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 --2- 172.21.15.83:0/1749893950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2880043a0 0x7fe288004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 --2- 172.21.15.83:0/1749893950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a40d0630 0x7fe2880050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 -- 172.21.15.83:0/1749893950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a40712a0 con 0x7fe2880039f0 2024-04-25T00:15:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 -- 172.21.15.83:0/1749893950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe28800b9d0 con 0x7fe2880043a0 2024-04-25T00:15:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2000640 1 -- 172.21.15.83:0/1749893950 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe28800bbd0 con 0x7fe2a40d0630 2024-04-25T00:15:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a3400640 1 --2- 172.21.15.83:0/1749893950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2880039f0 0x7fe288003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2a00640 1 --2- 172.21.15.83:0/1749893950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2880043a0 0x7fe288004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a3e00640 1 --2- 172.21.15.83:0/1749893950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a40d0630 0x7fe2880050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a3400640 1 -- 172.21.15.83:0/1749893950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a40d0630 msgr2=0x7fe2880050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:32.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a3400640 1 --2- 172.21.15.83:0/1749893950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a40d0630 0x7fe2880050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a3400640 1 -- 172.21.15.83:0/1749893950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2880043a0 msgr2=0x7fe288004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:32.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a3400640 1 --2- 172.21.15.83:0/1749893950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2880043a0 0x7fe288004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a3400640 1 -- 172.21.15.83:0/1749893950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe288003530 con 0x7fe2880039f0 2024-04-25T00:15:32.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.078+0000 7fe2a2a00640 1 --2- 172.21.15.83:0/1749893950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2880043a0 0x7fe288004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.082+0000 7fe2a3400640 1 --2- 172.21.15.83:0/1749893950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2880039f0 0x7fe288003dd0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe294001f60 tx=0x7fe29401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2336501afb0077 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.082+0000 7fe2a2000640 1 -- 172.21.15.83:0/1749893950 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294001b60 con 0x7fe2880039f0 2024-04-25T00:15:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.082+0000 7fe2a2000640 1 -- 172.21.15.83:0/1749893950 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2940a9680 con 0x7fe2880039f0 2024-04-25T00:15:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.082+0000 7fe2a2000640 1 -- 172.21.15.83:0/1749893950 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2940a9a70 con 0x7fe2880039f0 2024-04-25T00:15:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.082+0000 7fe2aa200640 1 -- 172.21.15.83:0/1749893950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2880039f0 msgr2=0x7fe288003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.082+0000 7fe2aa200640 1 --2- 172.21.15.83:0/1749893950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2880039f0 0x7fe288003dd0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe294001f60 tx=0x7fe29401c8a0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.082+0000 7fe2a3e00640 1 -- 172.21.15.83:0/1749893950 reap_dead start 2024-04-25T00:15:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.082+0000 7fe2aa200640 1 -- 172.21.15.83:0/1749893950 shutdown_connections 2024-04-25T00:15:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.082+0000 7fe2aa200640 1 -- 172.21.15.83:0/1749893950 >> 172.21.15.83:0/1749893950 conn(0x7fe2a40b1c60 msgr2=0x7fe2a40b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.082+0000 7fe2aa200640 1 -- 172.21.15.83:0/1749893950 shutdown_connections 2024-04-25T00:15:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.082+0000 7fe2aa200640 1 -- 172.21.15.83:0/1749893950 wait complete. 2024-04-25T00:15:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.082+0000 7fe2aa200640 1 Processor -- start 2024-04-25T00:15:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.082+0000 7fe2aa200640 1 -- start start 2024-04-25T00:15:32.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2aa200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a414dc80 0x7fe2a4164ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2aa200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a414e060 0x7fe2a4166460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2aa200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a414e4d0 0x7fe2a41669a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2aa200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a4167dc0 con 0x7fe2a414e4d0 2024-04-25T00:15:32.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2aa200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a4167c40 con 0x7fe2a414e060 2024-04-25T00:15:32.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2aa200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a4167f40 con 0x7fe2a414dc80 2024-04-25T00:15:32.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2a3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a414e4d0 0x7fe2a41669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2a3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a414e4d0 0x7fe2a41669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57902/0 (socket says 172.21.15.83:57902) 2024-04-25T00:15:32.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2a3e00640 1 -- 172.21.15.83:0/2975722394 learned_addr learned my addr 172.21.15.83:0/2975722394 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:32.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2a3400640 1 --2- 172.21.15.83:0/2975722394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a414dc80 0x7fe2a4164ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2a2a00640 1 --2- 172.21.15.83:0/2975722394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a414e060 0x7fe2a4166460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2a3e00640 1 -- 172.21.15.83:0/2975722394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a414dc80 msgr2=0x7fe2a4164ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2a3e00640 1 --2- 172.21.15.83:0/2975722394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a414dc80 0x7fe2a4164ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2a3e00640 1 -- 172.21.15.83:0/2975722394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a414e060 msgr2=0x7fe2a4166460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2a3e00640 1 --2- 172.21.15.83:0/2975722394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a414e060 0x7fe2a4166460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2a3e00640 1 -- 172.21.15.83:0/2975722394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a41653c0 con 0x7fe2a414e4d0 2024-04-25T00:15:32.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2a3e00640 1 --2- 172.21.15.83:0/2975722394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a414e4d0 0x7fe2a41669a0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe29c000c10 tx=0x7fe29c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:32.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe29b400640 1 -- 172.21.15.83:0/2975722394 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c00c690 con 0x7fe2a414e4d0 2024-04-25T00:15:32.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe29b400640 1 -- 172.21.15.83:0/2975722394 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29c018070 con 0x7fe2a414e4d0 2024-04-25T00:15:32.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe29b400640 1 -- 172.21.15.83:0/2975722394 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c00cab0 con 0x7fe2a414e4d0 2024-04-25T00:15:32.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2aa200640 1 -- 172.21.15.83:0/2975722394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a41656a0 con 0x7fe2a414e4d0 2024-04-25T00:15:32.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2aa200640 1 -- 172.21.15.83:0/2975722394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a4165c60 con 0x7fe2a414e4d0 2024-04-25T00:15:32.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.086+0000 7fe2aa200640 1 -- 172.21.15.83:0/2975722394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a404f510 con 0x7fe2a414e4d0 2024-04-25T00:15:32.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.090+0000 7fe29b400640 1 -- 172.21.15.83:0/2975722394 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe29c00cc50 con 0x7fe2a414e4d0 2024-04-25T00:15:32.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.090+0000 7fe29b400640 1 --2- 172.21.15.83:0/2975722394 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2840678c0 0x7fe284069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.090+0000 7fe29b400640 1 -- 172.21.15.83:0/2975722394 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6445+0+0 (secure 0 0 0) 0x7fe29c085a90 con 0x7fe2a414e4d0 2024-04-25T00:15:32.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.090+0000 7fe2a3400640 1 --2- 172.21.15.83:0/2975722394 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2840678c0 0x7fe284069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.090+0000 7fe2a3400640 1 --2- 172.21.15.83:0/2975722394 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2840678c0 0x7fe284069d80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe294001f60 tx=0x7fe2940b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:32.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.094+0000 7fe29b400640 1 -- 172.21.15.83:0/2975722394 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe29c055c30 con 0x7fe2a414e4d0 2024-04-25T00:15:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.250+0000 7fe2aa200640 1 -- 172.21.15.83:0/2975722394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe2a40b01a0 con 0x7fe2a414e4d0 2024-04-25T00:15:32.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.962+0000 7fe29b400640 1 -- 172.21.15.83:0/2975722394 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) ==== 181+0+0 (secure 0 0 0) 0x7fe29c059f30 con 0x7fe2a414e4d0 2024-04-25T00:15:32.962 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:15:32.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.966+0000 7fe2aa200640 1 -- 172.21.15.83:0/2975722394 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2840678c0 msgr2=0x7fe284069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.966+0000 7fe2aa200640 1 --2- 172.21.15.83:0/2975722394 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2840678c0 0x7fe284069d80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe294001f60 tx=0x7fe2940b3040 comp rx=0 tx=0).stop 2024-04-25T00:15:32.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.966+0000 7fe2aa200640 1 -- 172.21.15.83:0/2975722394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a414e4d0 msgr2=0x7fe2a41669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.966+0000 7fe2aa200640 1 --2- 172.21.15.83:0/2975722394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a414e4d0 0x7fe2a41669a0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe29c000c10 tx=0x7fe29c00eb90 comp rx=0 tx=0).stop 2024-04-25T00:15:32.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.966+0000 7fe2aa200640 1 -- 172.21.15.83:0/2975722394 shutdown_connections 2024-04-25T00:15:32.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.966+0000 7fe2aa200640 1 --2- 172.21.15.83:0/2975722394 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fe2840678c0 0x7fe284069d80 secure :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe294001f60 tx=0x7fe2940b3040 comp rx=0 tx=0).stop 2024-04-25T00:15:32.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.966+0000 7fe2aa200640 1 --2- 172.21.15.83:0/2975722394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a414e4d0 0x7fe2a41669a0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.966+0000 7fe2aa200640 1 --2- 172.21.15.83:0/2975722394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a414e060 0x7fe2a4166460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.966+0000 7fe2aa200640 1 --2- 172.21.15.83:0/2975722394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a414dc80 0x7fe2a4164ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.966+0000 7fe2aa200640 1 -- 172.21.15.83:0/2975722394 >> 172.21.15.83:0/2975722394 conn(0x7fe2a40b1c60 msgr2=0x7fe2a40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:32.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.966+0000 7fe2aa200640 1 -- 172.21.15.83:0/2975722394 shutdown_connections 2024-04-25T00:15:32.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:32.966+0000 7fe2aa200640 1 -- 172.21.15.83:0/2975722394 wait complete. 2024-04-25T00:15:32.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:15:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f2841a00640 1 Processor -- start 2024-04-25T00:15:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f2841a00640 1 -- start start 2024-04-25T00:15:33.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f2841a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f283c15bf10 0x7f283c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:33.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f2841a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f283c0cf470 0x7f283c06cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:33.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f2841a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f283c071d80 con 0x7f283c0d0460 2024-04-25T00:15:33.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f2841a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f283c0d08c0 con 0x7f283c15bf10 2024-04-25T00:15:33.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f2841a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f283c070fc0 con 0x7f283c0cf470 2024-04-25T00:15:33.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f283aa00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f283c15bf10 0x7f283c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:33.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f283a000640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f283c0d0460 0x7f283c0ced60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37850/0 (socket says 172.21.15.83:37850) 2024-04-25T00:15:33.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f283a000640 1 -- 172.21.15.83:0/1451471809 learned_addr learned my addr 172.21.15.83:0/1451471809 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:33.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f283b400640 1 --2- 172.21.15.83:0/1451471809 >> v2:172.21.15.86:3300/0 conn(0x7f283c0cf470 0x7f283c06cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:33.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f283aa00640 1 -- 172.21.15.83:0/1451471809 >> v2:172.21.15.86:3300/0 conn(0x7f283c0cf470 msgr2=0x7f283c06cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:33.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f283aa00640 1 --2- 172.21.15.83:0/1451471809 >> v2:172.21.15.86:3300/0 conn(0x7f283c0cf470 0x7f283c06cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:33.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f283aa00640 1 -- 172.21.15.83:0/1451471809 >> v1:172.21.15.83:6789/0 conn(0x7f283c0d0460 legacy=0x7f283c0ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:33.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f283aa00640 1 -- 172.21.15.83:0/1451471809 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c0712a0 con 0x7f283c15bf10 2024-04-25T00:15:33.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.130+0000 7f2839600640 1 -- 172.21.15.83:0/1451471809 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 561745654 0 0) 0x7f283c071d80 con 0x7f283c0d0460 2024-04-25T00:15:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.134+0000 7f283aa00640 1 --2- 172.21.15.83:0/1451471809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c15bf10 0x7f283c0cff20 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f283c0b4f00 tx=0x7f282c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cacbc8dae6c90b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.134+0000 7f2839600640 1 -- 172.21.15.83:0/1451471809 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c0b0070 con 0x7f283c15bf10 2024-04-25T00:15:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.134+0000 7f2839600640 1 -- 172.21.15.83:0/1451471809 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f282c0ab600 con 0x7f283c15bf10 2024-04-25T00:15:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.134+0000 7f2839600640 1 -- 172.21.15.83:0/1451471809 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c0ab9d0 con 0x7f283c15bf10 2024-04-25T00:15:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.134+0000 7f2841a00640 1 -- 172.21.15.83:0/1451471809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c15bf10 msgr2=0x7f283c0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.134+0000 7f2841a00640 1 --2- 172.21.15.83:0/1451471809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c15bf10 0x7f283c0cff20 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f283c0b4f00 tx=0x7f282c01c480 comp rx=0 tx=0).stop 2024-04-25T00:15:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.134+0000 7f2841a00640 1 -- 172.21.15.83:0/1451471809 shutdown_connections 2024-04-25T00:15:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.134+0000 7f2841a00640 1 --2- 172.21.15.83:0/1451471809 >> v2:172.21.15.86:3300/0 conn(0x7f283c0cf470 0x7f283c06cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.134+0000 7f2841a00640 1 --2- 172.21.15.83:0/1451471809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c15bf10 0x7f283c0cff20 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.134+0000 7f2841a00640 1 -- 172.21.15.83:0/1451471809 >> 172.21.15.83:0/1451471809 conn(0x7f283c0b1c60 msgr2=0x7f283c0b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.134+0000 7f2841a00640 1 -- 172.21.15.83:0/1451471809 shutdown_connections 2024-04-25T00:15:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.134+0000 7f2841a00640 1 -- 172.21.15.83:0/1451471809 wait complete. 2024-04-25T00:15:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f2841a00640 1 Processor -- start 2024-04-25T00:15:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f2841a00640 1 -- start start 2024-04-25T00:15:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f2841a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f283c0cf470 0x7f283c14f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f2841a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c0d0460 0x7f283c154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f2841a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f283c15bf10 0x7f283c151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f2841a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f283c06dca0 con 0x7f283c0d0460 2024-04-25T00:15:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f2841a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f283c06db20 con 0x7f283c15bf10 2024-04-25T00:15:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f2841a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f283c168a30 con 0x7f283c0cf470 2024-04-25T00:15:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f283a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c0d0460 0x7f283c154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f283a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c0d0460 0x7f283c154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57920/0 (socket says 172.21.15.83:57920) 2024-04-25T00:15:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f283a000640 1 -- 172.21.15.83:0/206168810 learned_addr learned my addr 172.21.15.83:0/206168810 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f283aa00640 1 --2- 172.21.15.83:0/206168810 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f283c0cf470 0x7f283c14f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f283b400640 1 --2- 172.21.15.83:0/206168810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f283c15bf10 0x7f283c151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f283a000640 1 -- 172.21.15.83:0/206168810 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f283c0cf470 msgr2=0x7f283c14f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f283a000640 1 --2- 172.21.15.83:0/206168810 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f283c0cf470 0x7f283c14f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f283a000640 1 -- 172.21.15.83:0/206168810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f283c15bf10 msgr2=0x7f283c151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f283a000640 1 --2- 172.21.15.83:0/206168810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f283c15bf10 0x7f283c151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f283a000640 1 -- 172.21.15.83:0/206168810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c155870 con 0x7f283c0d0460 2024-04-25T00:15:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f283a000640 1 --2- 172.21.15.83:0/206168810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c0d0460 0x7f283c154f00 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f282800c520 tx=0x7f282800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f2832a00640 1 -- 172.21.15.83:0/206168810 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828018070 con 0x7f283c0d0460 2024-04-25T00:15:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f2832a00640 1 -- 172.21.15.83:0/206168810 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28280074e0 con 0x7f283c0d0460 2024-04-25T00:15:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f2832a00640 1 -- 172.21.15.83:0/206168810 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28280049f0 con 0x7f283c0d0460 2024-04-25T00:15:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f2841a00640 1 -- 172.21.15.83:0/206168810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283c155b50 con 0x7f283c0d0460 2024-04-25T00:15:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f2841a00640 1 -- 172.21.15.83:0/206168810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283c156f20 con 0x7f283c0d0460 2024-04-25T00:15:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.138+0000 7f2841a00640 1 -- 172.21.15.83:0/206168810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f283c157120 con 0x7f283c0d0460 2024-04-25T00:15:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.142+0000 7f2832a00640 1 -- 172.21.15.83:0/206168810 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f282801b070 con 0x7f283c0d0460 2024-04-25T00:15:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.142+0000 7f2832a00640 1 --2- 172.21.15.83:0/206168810 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f281c0678c0 0x7f281c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.142+0000 7f2832a00640 1 -- 172.21.15.83:0/206168810 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6007+0+0 (secure 0 0 0) 0x7f2828085170 con 0x7f283c0d0460 2024-04-25T00:15:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.142+0000 7f283aa00640 1 --2- 172.21.15.83:0/206168810 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f281c0678c0 0x7f281c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.142+0000 7f283aa00640 1 --2- 172.21.15.83:0/206168810 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f281c0678c0 0x7f281c069d80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f282c0ad000 tx=0x7f282c0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:33.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.146+0000 7f2832a00640 1 -- 172.21.15.83:0/206168810 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f283c157120 con 0x7f283c0d0460 2024-04-25T00:15:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.298+0000 7f2841a00640 1 -- 172.21.15.83:0/206168810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f283c0cc360 con 0x7f283c0d0460 2024-04-25T00:15:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.970+0000 7f2832a00640 1 -- 172.21.15.83:0/206168810 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) ==== 169+0+0 (secure 0 0 0) 0x7f2828055520 con 0x7f283c0d0460 2024-04-25T00:15:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-25T00:15:33.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.970+0000 7f2841a00640 1 -- 172.21.15.83:0/206168810 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f281c0678c0 msgr2=0x7f281c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.970+0000 7f2841a00640 1 --2- 172.21.15.83:0/206168810 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f281c0678c0 0x7f281c069d80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f282c0ad000 tx=0x7f282c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:15:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.970+0000 7f2841a00640 1 -- 172.21.15.83:0/206168810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c0d0460 msgr2=0x7f283c154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.970+0000 7f2841a00640 1 --2- 172.21.15.83:0/206168810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c0d0460 0x7f283c154f00 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f282800c520 tx=0x7f282800c9e0 comp rx=0 tx=0).stop 2024-04-25T00:15:33.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.974+0000 7f2841a00640 1 -- 172.21.15.83:0/206168810 shutdown_connections 2024-04-25T00:15:33.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.974+0000 7f2841a00640 1 --2- 172.21.15.83:0/206168810 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f281c0678c0 0x7f281c069d80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:33.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.974+0000 7f2841a00640 1 --2- 172.21.15.83:0/206168810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f283c15bf10 0x7f283c151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:33.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.974+0000 7f2841a00640 1 --2- 172.21.15.83:0/206168810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c0d0460 0x7f283c154f00 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:33.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.974+0000 7f2841a00640 1 --2- 172.21.15.83:0/206168810 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f283c0cf470 0x7f283c14f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:33.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.974+0000 7f2841a00640 1 -- 172.21.15.83:0/206168810 >> 172.21.15.83:0/206168810 conn(0x7f283c0b1c60 msgr2=0x7f283c0b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:33.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.974+0000 7f2841a00640 1 -- 172.21.15.83:0/206168810 shutdown_connections 2024-04-25T00:15:33.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:33.974+0000 7f2841a00640 1 -- 172.21.15.83:0/206168810 wait complete. 2024-04-25T00:15:34.003 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:15:34.006 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:15:34.046 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-04-25T00:15:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2024-04-25T00:15:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:15:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2024-04-25T00:15:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2024-04-25T00:15:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2024-04-25T00:15:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2024-04-25T00:15:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2024-04-25T00:15:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:15:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:15:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2024-04-25T00:15:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:15:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2024-04-25T00:15:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:15:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:15:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2024-04-25T00:15:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:15:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2024-04-25T00:15:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2024-04-25T00:15:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2024-04-25T00:15:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2024-04-25T00:15:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2024-04-25T00:15:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.83,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]" 2024-04-25T00:15:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on skipped message = False 2024-04-25T00:15:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = d58ae83a-ad72-4e18-824a-86e779979c35 2024-04-25T00:15:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.049 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2024-04-25T00:15:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2024-04-25T00:15:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2024-04-25T00:15:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2024-04-25T00:15:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:15:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2024-04-25T00:15:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2024-04-25T00:15:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2024-04-25T00:15:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2024-04-25T00:15:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2024-04-25T00:15:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2024-04-25T00:15:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2024-04-25T00:15:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2024-04-25T00:15:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:15:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2024-04-25T00:15:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2024-04-25T00:15:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:15:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2024-04-25T00:15:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2024-04-25T00:15:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2024-04-25T00:15:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2024-04-25T00:15:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 20 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 20 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 10 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:15:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2024-04-25T00:15:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:15:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.055 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:15:34.055 INFO:teuthology.orchestra.run.smithi083.stdout: mon down mkfs grace = 300 2024-04-25T00:15:34.055 INFO:teuthology.orchestra.run.smithi083.stdout: mon op complaint time = 120 2024-04-25T00:15:34.055 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:34.058 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2024-04-25T00:15:34.058 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2024-04-25T00:15:34.058 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2024-04-25T00:15:34.058 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2024-04-25T00:15:34.058 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:15:34.058 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:15:34.058 INFO:teuthology.orchestra.run.smithi083.stdout: client mount timeout = 600 2024-04-25T00:15:34.058 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-04-25T00:15:34.058 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:15:34.058 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = "" 2024-04-25T00:15:34.058 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-04-25T00:15:34.058 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-04-25T00:15:34.059 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2024-04-25T00:15:34.059 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2024-04-25T00:15:34.059 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2024-04-25T00:15:34.059 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2024-04-25T00:15:34.059 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2024-04-25T00:15:34.059 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2024-04-25T00:15:34.059 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2024-04-25T00:15:34.059 INFO:teuthology.orchestra.run.smithi083.stdout:[mds] 2024-04-25T00:15:34.059 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds = 20 2024-04-25T00:15:34.059 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds balancer = 20 2024-04-25T00:15:34.059 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-25T00:15:34.059 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug frag = True 2024-04-25T00:15:34.059 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug scatterstat = True 2024-04-25T00:15:34.059 INFO:teuthology.orchestra.run.smithi083.stdout: mds op complaint time = 180 2024-04-25T00:15:34.060 INFO:teuthology.orchestra.run.smithi083.stdout: mds verify scatter = True 2024-04-25T00:15:34.060 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-04-25T00:15:34.060 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-04-25T00:15:34.060 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-04-25T00:15:34.060 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-25T00:15:34.060 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-04-25T00:15:34.061 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:15:34.061 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-04-25T00:15:34.061 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.061 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-04-25T00:15:34.061 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.061 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-04-25T00:15:34.061 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.061 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-04-25T00:15:34.061 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-04-25T00:15:34.061 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.061 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-04-25T00:15:34.061 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-04-25T00:15:34.061 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.062 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:15:34.062 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-04-25T00:15:34.062 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.062 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:15:34.062 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:15:34.062 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.062 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-04-25T00:15:34.062 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.062 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-04-25T00:15:34.062 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-04-25T00:15:34.062 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:15:34.062 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:15:34.062 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-04-25T00:15:34.063 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:15:34.063 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-04-25T00:15:34.063 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:15:34.063 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:15:34.063 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-04-25T00:15:34.063 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.063 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:15:34.063 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.063 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-04-25T00:15:34.063 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.063 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-04-25T00:15:34.063 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.063 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-04-25T00:15:34.063 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-04-25T00:15:34.064 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-04-25T00:15:34.064 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.83,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]" 2024-04-25T00:15:34.064 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on skipped message = False 2024-04-25T00:15:34.064 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = d58ae83a-ad72-4e18-824a-86e779979c35 2024-04-25T00:15:34.064 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.064 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-04-25T00:15:34.064 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-04-25T00:15:34.064 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.064 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-04-25T00:15:34.064 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-04-25T00:15:34.064 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:15:34.064 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.064 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-04-25T00:15:34.064 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-04-25T00:15:34.065 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.065 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-04-25T00:15:34.065 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-04-25T00:15:34.065 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-04-25T00:15:34.065 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-04-25T00:15:34.065 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.065 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-04-25T00:15:34.065 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-04-25T00:15:34.065 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.065 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:15:34.065 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.065 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-04-25T00:15:34.065 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.065 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-04-25T00:15:34.066 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.066 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:15:34.066 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-04-25T00:15:34.066 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.066 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-04-25T00:15:34.066 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-04-25T00:15:34.066 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-04-25T00:15:34.066 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-04-25T00:15:34.066 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 20 2024-04-25T00:15:34.066 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 20 2024-04-25T00:15:34.066 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:15:34.066 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-04-25T00:15:34.066 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 10 2024-04-25T00:15:34.066 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-04-25T00:15:34.067 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:15:34.068 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-04-25T00:15:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-04-25T00:15:34.070 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-04-25T00:15:34.070 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-04-25T00:15:34.070 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:15:34.070 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:15:34.070 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-04-25T00:15:34.070 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-25T00:15:34.070 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:15:34.070 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-04-25T00:15:34.070 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-25T00:15:34.070 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-25T00:15:34.071 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-04-25T00:15:34.071 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-04-25T00:15:34.071 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-04-25T00:15:34.071 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-25T00:15:34.071 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-04-25T00:15:34.071 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-04-25T00:15:34.071 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-04-25T00:15:34.071 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-04-25T00:15:34.071 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-04-25T00:15:34.071 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-04-25T00:15:34.071 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T00:15:34.071 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-04-25T00:15:34.072 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = True 2024-04-25T00:15:34.072 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-04-25T00:15:34.072 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = True 2024-04-25T00:15:34.072 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-25T00:15:34.072 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-25T00:15:34.072 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-25T00:15:34.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:15:34.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.250+0000 7f655bc00640 1 Processor -- start 2024-04-25T00:15:34.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.250+0000 7f655bc00640 1 -- start start 2024-04-25T00:15:34.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f655bc00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6554160020 0x7f6554166ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:34.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f655bc00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65540ce000 con 0x7f6554159810 2024-04-25T00:15:34.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f655bc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f655406eec0 con 0x7f655406e950 2024-04-25T00:15:34.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f655bc00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f655406e210 con 0x7f6554160020 2024-04-25T00:15:34.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f6559c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6554160020 0x7f6554166ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:34.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f6559200640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6554159810 0x7f65540cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37854/0 (socket says 172.21.15.83:37854) 2024-04-25T00:15:34.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f6559200640 1 -- 172.21.15.83:0/2640918527 learned_addr learned my addr 172.21.15.83:0/2640918527 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:34.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f6559c00640 1 -- 172.21.15.83:0/2640918527 >> v1:172.21.15.86:6789/0 conn(0x7f655406e950 legacy=0x7f655415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:34.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f6559c00640 1 -- 172.21.15.83:0/2640918527 >> v1:172.21.15.83:6789/0 conn(0x7f6554159810 legacy=0x7f65540cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:34.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f6559c00640 1 -- 172.21.15.83:0/2640918527 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655406dc50 con 0x7f6554160020 2024-04-25T00:15:34.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f6553e00640 1 -- 172.21.15.83:0/2640918527 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4231156407 0 0) 0x7f65540ce000 con 0x7f6554159810 2024-04-25T00:15:34.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f6559c00640 1 --2- 172.21.15.83:0/2640918527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554160020 0x7f6554166ca0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f654c009d90 tx=0x7f654c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc696062a2cbf832 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:34.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f6553e00640 1 -- 172.21.15.83:0/2640918527 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0b2070 con 0x7f6554160020 2024-04-25T00:15:34.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f6553e00640 1 -- 172.21.15.83:0/2640918527 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f654c0a58b0 con 0x7f6554160020 2024-04-25T00:15:34.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.254+0000 7f6553e00640 1 -- 172.21.15.83:0/2640918527 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0a5c80 con 0x7f6554160020 2024-04-25T00:15:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 -- 172.21.15.83:0/2640918527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554160020 msgr2=0x7f6554166ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 --2- 172.21.15.83:0/2640918527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554160020 0x7f6554166ca0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f654c009d90 tx=0x7f654c01c8e0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 -- 172.21.15.83:0/2640918527 shutdown_connections 2024-04-25T00:15:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 --2- 172.21.15.83:0/2640918527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554160020 0x7f6554166ca0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 -- 172.21.15.83:0/2640918527 >> 172.21.15.83:0/2640918527 conn(0x7f65540b1830 msgr2=0x7f65540b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 -- 172.21.15.83:0/2640918527 shutdown_connections 2024-04-25T00:15:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 -- 172.21.15.83:0/2640918527 wait complete. 2024-04-25T00:15:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 Processor -- start 2024-04-25T00:15:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 -- start start 2024-04-25T00:15:34.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655406e950 0x7f65540b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:34.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554159810 0x7f655415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:34.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6554160020 0x7f65541420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:34.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6554169b80 con 0x7f6554159810 2024-04-25T00:15:34.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6554169a00 con 0x7f655406e950 2024-04-25T00:15:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f655bc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6554169d00 con 0x7f6554160020 2024-04-25T00:15:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f654be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554159810 0x7f655415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f654be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554159810 0x7f655415b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57942/0 (socket says 172.21.15.83:57942) 2024-04-25T00:15:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f654be00640 1 -- 172.21.15.83:0/4237966459 learned_addr learned my addr 172.21.15.83:0/4237966459 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f6559200640 1 --2- 172.21.15.83:0/4237966459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655406e950 0x7f65540b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f6559c00640 1 --2- 172.21.15.83:0/4237966459 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6554160020 0x7f65541420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f654be00640 1 -- 172.21.15.83:0/4237966459 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6554160020 msgr2=0x7f65541420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f654be00640 1 --2- 172.21.15.83:0/4237966459 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6554160020 0x7f65541420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f654be00640 1 -- 172.21.15.83:0/4237966459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655406e950 msgr2=0x7f65540b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f654be00640 1 --2- 172.21.15.83:0/4237966459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655406e950 0x7f65540b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.258+0000 7f654be00640 1 -- 172.21.15.83:0/4237966459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65540b3e70 con 0x7f6554159810 2024-04-25T00:15:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.262+0000 7f654be00640 1 --2- 172.21.15.83:0/4237966459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554159810 0x7f655415b0b0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f653c00c500 tx=0x7f653c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.262+0000 7f6551600640 1 -- 172.21.15.83:0/4237966459 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c016020 con 0x7f6554159810 2024-04-25T00:15:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.262+0000 7f6551600640 1 -- 172.21.15.83:0/4237966459 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f653c005220 con 0x7f6554159810 2024-04-25T00:15:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.262+0000 7f655bc00640 1 -- 172.21.15.83:0/4237966459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65540b4150 con 0x7f6554159810 2024-04-25T00:15:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.262+0000 7f6551600640 1 -- 172.21.15.83:0/4237966459 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c007030 con 0x7f6554159810 2024-04-25T00:15:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.262+0000 7f655bc00640 1 -- 172.21.15.83:0/4237966459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65540b4710 con 0x7f6554159810 2024-04-25T00:15:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.262+0000 7f655bc00640 1 -- 172.21.15.83:0/4237966459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f655404f510 con 0x7f6554159810 2024-04-25T00:15:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.262+0000 7f6551600640 1 -- 172.21.15.83:0/4237966459 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f653c0071d0 con 0x7f6554159810 2024-04-25T00:15:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.262+0000 7f6551600640 1 --2- 172.21.15.83:0/4237966459 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f65300678c0 0x7f6530069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.262+0000 7f6551600640 1 -- 172.21.15.83:0/4237966459 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f653c085680 con 0x7f6554159810 2024-04-25T00:15:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.262+0000 7f6559200640 1 --2- 172.21.15.83:0/4237966459 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f65300678c0 0x7f6530069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.262+0000 7f6559200640 1 --2- 172.21.15.83:0/4237966459 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f65300678c0 0x7f6530069d80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6544007fd0 tx=0x7f6544007ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:34.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.266+0000 7f6551600640 1 -- 172.21.15.83:0/4237966459 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f653c020020 con 0x7f6554159810 2024-04-25T00:15:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.422+0000 7f655bc00640 1 -- 172.21.15.83:0/4237966459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f655415fa70 con 0x7f6554159810 2024-04-25T00:15:34.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.426+0000 7f6551600640 1 -- 172.21.15.83:0/4237966459 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f653c0074c0 con 0x7f6554159810 2024-04-25T00:15:34.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.426+0000 7f655bc00640 1 -- 172.21.15.83:0/4237966459 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f65300678c0 msgr2=0x7f6530069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:34.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.426+0000 7f655bc00640 1 --2- 172.21.15.83:0/4237966459 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f65300678c0 0x7f6530069d80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6544007fd0 tx=0x7f6544007ba0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.426+0000 7f655bc00640 1 -- 172.21.15.83:0/4237966459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554159810 msgr2=0x7f655415b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:34.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.426+0000 7f655bc00640 1 --2- 172.21.15.83:0/4237966459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554159810 0x7f655415b0b0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f653c00c500 tx=0x7f653c00c9c0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.426+0000 7f655bc00640 1 -- 172.21.15.83:0/4237966459 shutdown_connections 2024-04-25T00:15:34.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.426+0000 7f655bc00640 1 --2- 172.21.15.83:0/4237966459 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f65300678c0 0x7f6530069d80 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.426+0000 7f655bc00640 1 --2- 172.21.15.83:0/4237966459 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6554160020 0x7f65541420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.426+0000 7f655bc00640 1 --2- 172.21.15.83:0/4237966459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554159810 0x7f655415b0b0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.426+0000 7f655bc00640 1 --2- 172.21.15.83:0/4237966459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655406e950 0x7f65540b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.426+0000 7f655bc00640 1 -- 172.21.15.83:0/4237966459 >> 172.21.15.83:0/4237966459 conn(0x7f65540b1830 msgr2=0x7f65541510b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:34.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.426+0000 7f655bc00640 1 -- 172.21.15.83:0/4237966459 shutdown_connections 2024-04-25T00:15:34.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.426+0000 7f655bc00640 1 -- 172.21.15.83:0/4237966459 wait complete. 2024-04-25T00:15:34.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting' 2024-04-25T00:15:34.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:15:34.535 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:15:34.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3548a00640 1 Processor -- start 2024-04-25T00:15:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3548a00640 1 -- start start 2024-04-25T00:15:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3548a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f354415bf70 0x7f35440cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3548a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35440d0940 0x7f3544165b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3548a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35440cc9b0 con 0x7f35440cfd80 2024-04-25T00:15:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3548a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f35440b5ad0 con 0x7f35440d0940 2024-04-25T00:15:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3548a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f35440ccc90 con 0x7f354415bf70 2024-04-25T00:15:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3542800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35440d0940 0x7f3544165b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:34.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3541400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f35440cfd80 0x7f35440d0230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35786/0 (socket says 172.21.15.83:35786) 2024-04-25T00:15:34.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3541400640 1 -- 172.21.15.83:0/890018086 learned_addr learned my addr 172.21.15.83:0/890018086 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:34.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3541e00640 1 --2- 172.21.15.83:0/890018086 >> v2:172.21.15.86:3300/0 conn(0x7f354415bf70 0x7f35440cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:34.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3542800640 1 -- 172.21.15.83:0/890018086 >> v2:172.21.15.86:3300/0 conn(0x7f354415bf70 msgr2=0x7f35440cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:34.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3542800640 1 --2- 172.21.15.83:0/890018086 >> v2:172.21.15.86:3300/0 conn(0x7f354415bf70 0x7f35440cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3542800640 1 -- 172.21.15.83:0/890018086 >> v1:172.21.15.83:6789/0 conn(0x7f35440cfd80 legacy=0x7f35440d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:34.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3542800640 1 -- 172.21.15.83:0/890018086 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35440ccf70 con 0x7f35440d0940 2024-04-25T00:15:34.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.586+0000 7f3540a00640 1 -- 172.21.15.83:0/890018086 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2711170306 0 0) 0x7f35440cc9b0 con 0x7f35440cfd80 2024-04-25T00:15:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.590+0000 7f3542800640 1 --2- 172.21.15.83:0/890018086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35440d0940 0x7f3544165b10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f3544072520 tx=0x7f353c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac05df9abb2755c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.590+0000 7f3540a00640 1 -- 172.21.15.83:0/890018086 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c0b2070 con 0x7f35440d0940 2024-04-25T00:15:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.590+0000 7f3540a00640 1 -- 172.21.15.83:0/890018086 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c0a59c0 con 0x7f35440d0940 2024-04-25T00:15:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.590+0000 7f3540a00640 1 -- 172.21.15.83:0/890018086 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c0a5d90 con 0x7f35440d0940 2024-04-25T00:15:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.590+0000 7f3548a00640 1 -- 172.21.15.83:0/890018086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35440d0940 msgr2=0x7f3544165b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.590+0000 7f3548a00640 1 --2- 172.21.15.83:0/890018086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35440d0940 0x7f3544165b10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f3544072520 tx=0x7f353c01c900 comp rx=0 tx=0).stop 2024-04-25T00:15:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.590+0000 7f3548a00640 1 -- 172.21.15.83:0/890018086 shutdown_connections 2024-04-25T00:15:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.590+0000 7f3548a00640 1 --2- 172.21.15.83:0/890018086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35440d0940 0x7f3544165b10 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.590+0000 7f3548a00640 1 --2- 172.21.15.83:0/890018086 >> v2:172.21.15.86:3300/0 conn(0x7f354415bf70 0x7f35440cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.590+0000 7f3548a00640 1 -- 172.21.15.83:0/890018086 >> 172.21.15.83:0/890018086 conn(0x7f35440d1c70 msgr2=0x7f35440d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.590+0000 7f3548a00640 1 -- 172.21.15.83:0/890018086 shutdown_connections 2024-04-25T00:15:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.590+0000 7f3548a00640 1 -- 172.21.15.83:0/890018086 wait complete. 2024-04-25T00:15:34.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3548a00640 1 Processor -- start 2024-04-25T00:15:34.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3548a00640 1 -- start start 2024-04-25T00:15:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3548a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35440cfd80 0x7f354406dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3548a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35440d0940 0x7f3544153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3548a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f354415bf70 0x7f35440b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:34.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3548a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f35440cda50 con 0x7f35440d0940 2024-04-25T00:15:34.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3548a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f354415c350 con 0x7f35440cfd80 2024-04-25T00:15:34.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3548a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f354414dd50 con 0x7f354415bf70 2024-04-25T00:15:34.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3541400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35440d0940 0x7f3544153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:34.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3541400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35440d0940 0x7f3544153d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60112/0 (socket says 172.21.15.83:60112) 2024-04-25T00:15:34.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3541400640 1 -- 172.21.15.83:0/113735634 learned_addr learned my addr 172.21.15.83:0/113735634 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:34.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3541e00640 1 --2- 172.21.15.83:0/113735634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35440cfd80 0x7f354406dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:34.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3542800640 1 --2- 172.21.15.83:0/113735634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f354415bf70 0x7f35440b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:34.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3541400640 1 -- 172.21.15.83:0/113735634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f354415bf70 msgr2=0x7f35440b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:34.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3541400640 1 --2- 172.21.15.83:0/113735634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f354415bf70 0x7f35440b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3541400640 1 -- 172.21.15.83:0/113735634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35440cfd80 msgr2=0x7f354406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:34.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3541400640 1 --2- 172.21.15.83:0/113735634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35440cfd80 0x7f354406dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3541400640 1 -- 172.21.15.83:0/113735634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354414d290 con 0x7f35440d0940 2024-04-25T00:15:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3541400640 1 --2- 172.21.15.83:0/113735634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35440d0940 0x7f3544153d00 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f353000c480 tx=0x7f353000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f353a000640 1 -- 172.21.15.83:0/113735634 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530018070 con 0x7f35440d0940 2024-04-25T00:15:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f353a000640 1 -- 172.21.15.83:0/113735634 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3530007500 con 0x7f35440d0940 2024-04-25T00:15:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3548a00640 1 -- 172.21.15.83:0/113735634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354414e4b0 con 0x7f35440d0940 2024-04-25T00:15:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f353a000640 1 -- 172.21.15.83:0/113735634 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530004a30 con 0x7f35440d0940 2024-04-25T00:15:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3548a00640 1 -- 172.21.15.83:0/113735634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354414d550 con 0x7f35440d0940 2024-04-25T00:15:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.594+0000 7f3548a00640 1 -- 172.21.15.83:0/113735634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f354409fd60 con 0x7f35440d0940 2024-04-25T00:15:34.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.598+0000 7f353a000640 1 -- 172.21.15.83:0/113735634 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f353001b070 con 0x7f35440d0940 2024-04-25T00:15:34.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.598+0000 7f353a000640 1 --2- 172.21.15.83:0/113735634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35240678c0 0x7f3524069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:34.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.598+0000 7f353a000640 1 -- 172.21.15.83:0/113735634 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f3530084e70 con 0x7f35440d0940 2024-04-25T00:15:34.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.598+0000 7f3541e00640 1 --2- 172.21.15.83:0/113735634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35240678c0 0x7f3524069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:34.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.598+0000 7f3541e00640 1 --2- 172.21.15.83:0/113735634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35240678c0 0x7f3524069d80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3534009b60 tx=0x7f3534009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:34.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.602+0000 7f353a000640 1 -- 172.21.15.83:0/113735634 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f35300553c0 con 0x7f35440d0940 2024-04-25T00:15:34.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.814+0000 7f3548a00640 1 -- 172.21.15.83:0/113735634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]} v 0) -- 0x7f354404f5f0 con 0x7f35440d0940 2024-04-25T00:15:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.970+0000 7f353a000640 1 -- 172.21.15.83:0/113735634 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]}]=0 v284) ==== 145+0+0 (secure 0 0 0) 0x7f35300596c0 con 0x7f35440d0940 2024-04-25T00:15:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.974+0000 7f3548a00640 1 -- 172.21.15.83:0/113735634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35240678c0 msgr2=0x7f3524069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.974+0000 7f3548a00640 1 --2- 172.21.15.83:0/113735634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35240678c0 0x7f3524069d80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3534009b60 tx=0x7f3534009190 comp rx=0 tx=0).stop 2024-04-25T00:15:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.974+0000 7f3548a00640 1 -- 172.21.15.83:0/113735634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35440d0940 msgr2=0x7f3544153d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.974+0000 7f3548a00640 1 --2- 172.21.15.83:0/113735634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35440d0940 0x7f3544153d00 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f353000c480 tx=0x7f353000c940 comp rx=0 tx=0).stop 2024-04-25T00:15:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.974+0000 7f3548a00640 1 -- 172.21.15.83:0/113735634 shutdown_connections 2024-04-25T00:15:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.974+0000 7f3548a00640 1 --2- 172.21.15.83:0/113735634 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f35240678c0 0x7f3524069d80 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.974+0000 7f3548a00640 1 --2- 172.21.15.83:0/113735634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f354415bf70 0x7f35440b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.974+0000 7f3548a00640 1 --2- 172.21.15.83:0/113735634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35440d0940 0x7f3544153d00 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.974+0000 7f3548a00640 1 --2- 172.21.15.83:0/113735634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35440cfd80 0x7f354406dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.974+0000 7f3548a00640 1 -- 172.21.15.83:0/113735634 >> 172.21.15.83:0/113735634 conn(0x7f35440d1c70 msgr2=0x7f35440b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:34.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.974+0000 7f3548a00640 1 -- 172.21.15.83:0/113735634 shutdown_connections 2024-04-25T00:15:34.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:34.974+0000 7f3548a00640 1 -- 172.21.15.83:0/113735634 wait complete. 2024-04-25T00:15:34.993 INFO:tasks.cephfs_test_runner:That when many sessions are updated, they are correctly ... ok 2024-04-25T00:15:34.993 INFO:tasks.cephfs_test_runner: 2024-04-25T00:15:34.993 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T00:15:34.993 INFO:tasks.cephfs_test_runner:Ran 6 tests in 223.012s 2024-04-25T00:15:34.993 INFO:tasks.cephfs_test_runner: 2024-04-25T00:15:34.994 INFO:tasks.cephfs_test_runner:OK (skipped=2) 2024-04-25T00:15:34.994 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T00:15:35.004 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T00:15:35.024 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T00:15:35.024 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:15:35.024 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:35.045 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:15:35.045 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T00:15:35.045 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:35.101 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8557 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:15:35.101 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:15:35.101 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:35.111 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:15:35.111 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T00:15:35.112 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:15:35.161 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350296k,nr_inodes=4087574,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2539 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:15:35.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:35.161 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:15:35.169 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:15:35.170 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:15:35.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:35.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:15:35.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:35.235 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:15:35.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:35.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-25T00:15:35.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:35.464 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T00:15:35.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:35.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-25T00:15:35.538 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2024-04-25T00:15:35.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:15:35.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:35.541 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T00:15:35.551 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:15:35.551 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:15:35.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:35.551 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:15:35.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:35.619 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:15:35.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:35.682 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-25T00:15:35.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:35.831 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T00:15:35.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:15:35.840 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-25T00:15:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-04-25T00:15:35.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:15:35.908 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T00:15:35.916 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T00:15:35.917 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:15:36.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81c7c00640 1 Processor -- start 2024-04-25T00:15:36.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81c7c00640 1 -- start start 2024-04-25T00:15:36.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81c7c00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f81c01581d0 0x7f81c00b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81c7c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f81c00cf3c0 0x7f81c015dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81c7c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81c015e8c0 con 0x7f81c015e4e0 2024-04-25T00:15:36.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81c7c00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f81c00b4c30 con 0x7f81c01581d0 2024-04-25T00:15:36.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81c7c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f81c00b3440 con 0x7f81c00cf3c0 2024-04-25T00:15:36.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81c5200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f81c01581d0 0x7f81c00b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81c5c00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f81c015e4e0 0x7f81c006d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35800/0 (socket says 172.21.15.83:35800) 2024-04-25T00:15:36.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81c5c00640 1 -- 172.21.15.83:0/3632511319 learned_addr learned my addr 172.21.15.83:0/3632511319 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:36.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81bfe00640 1 --2- 172.21.15.83:0/3632511319 >> v2:172.21.15.86:3300/0 conn(0x7f81c00cf3c0 0x7f81c015dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81c5200640 1 -- 172.21.15.83:0/3632511319 >> v2:172.21.15.86:3300/0 conn(0x7f81c00cf3c0 msgr2=0x7f81c015dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:36.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81c5200640 1 --2- 172.21.15.83:0/3632511319 >> v2:172.21.15.86:3300/0 conn(0x7f81c00cf3c0 0x7f81c015dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81c5200640 1 -- 172.21.15.83:0/3632511319 >> v1:172.21.15.83:6789/0 conn(0x7f81c015e4e0 legacy=0x7f81c006d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81c5200640 1 -- 172.21.15.83:0/3632511319 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c00b3720 con 0x7f81c01581d0 2024-04-25T00:15:36.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81bf400640 1 -- 172.21.15.83:0/3632511319 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2281097295 0 0) 0x7f81c015e8c0 con 0x7f81c015e4e0 2024-04-25T00:15:36.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.054+0000 7f81bfe00640 1 --2- 172.21.15.83:0/3632511319 >> v2:172.21.15.86:3300/0 conn(0x7f81c00cf3c0 0x7f81c015dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:36.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.058+0000 7f81c5200640 1 --2- 172.21.15.83:0/3632511319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81c01581d0 0x7f81c00b3f20 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f81c00cc2a0 tx=0x7f81b001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f3e69fc58920c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:36.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.058+0000 7f81bf400640 1 -- 172.21.15.83:0/3632511319 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b00ab680 con 0x7f81c01581d0 2024-04-25T00:15:36.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.058+0000 7f81bf400640 1 -- 172.21.15.83:0/3632511319 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b00b0070 con 0x7f81c01581d0 2024-04-25T00:15:36.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.058+0000 7f81bf400640 1 -- 172.21.15.83:0/3632511319 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b00aba40 con 0x7f81c01581d0 2024-04-25T00:15:36.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.058+0000 7f81c7c00640 1 -- 172.21.15.83:0/3632511319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81c01581d0 msgr2=0x7f81c00b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.058+0000 7f81c7c00640 1 --2- 172.21.15.83:0/3632511319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81c01581d0 0x7f81c00b3f20 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f81c00cc2a0 tx=0x7f81b001c480 comp rx=0 tx=0).stop 2024-04-25T00:15:36.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.058+0000 7f81c7c00640 1 -- 172.21.15.83:0/3632511319 shutdown_connections 2024-04-25T00:15:36.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.058+0000 7f81c7c00640 1 --2- 172.21.15.83:0/3632511319 >> v2:172.21.15.86:3300/0 conn(0x7f81c00cf3c0 0x7f81c015dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.058+0000 7f81c7c00640 1 --2- 172.21.15.83:0/3632511319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81c01581d0 0x7f81c00b3f20 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.058+0000 7f81c7c00640 1 -- 172.21.15.83:0/3632511319 >> 172.21.15.83:0/3632511319 conn(0x7f81c00b1830 msgr2=0x7f81c00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:36.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.058+0000 7f81c7c00640 1 -- 172.21.15.83:0/3632511319 shutdown_connections 2024-04-25T00:15:36.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.058+0000 7f81c7c00640 1 -- 172.21.15.83:0/3632511319 wait complete. 2024-04-25T00:15:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.058+0000 7f81c7c00640 1 Processor -- start 2024-04-25T00:15:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81c7c00640 1 -- start start 2024-04-25T00:15:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81c7c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81c00cf3c0 0x7f81c01f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81c7c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81c01581d0 0x7f81c0151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81c7c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81c015e4e0 0x7f81c0151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81c7c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81c00b4800 con 0x7f81c01581d0 2024-04-25T00:15:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81c7c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f81c006ded0 con 0x7f81c015e4e0 2024-04-25T00:15:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81c7c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f81c00b4980 con 0x7f81c00cf3c0 2024-04-25T00:15:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81bfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81c01581d0 0x7f81c0151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81bfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81c01581d0 0x7f81c0151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60118/0 (socket says 172.21.15.83:60118) 2024-04-25T00:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81bfe00640 1 -- 172.21.15.83:0/3366866678 learned_addr learned my addr 172.21.15.83:0/3366866678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81c5c00640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81c015e4e0 0x7f81c0151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81c5200640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81c00cf3c0 0x7f81c01f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81bfe00640 1 -- 172.21.15.83:0/3366866678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81c00cf3c0 msgr2=0x7f81c01f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81bfe00640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81c00cf3c0 0x7f81c01f5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81bfe00640 1 -- 172.21.15.83:0/3366866678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81c015e4e0 msgr2=0x7f81c0151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81bfe00640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81c015e4e0 0x7f81c0151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81bfe00640 1 -- 172.21.15.83:0/3366866678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c01f63c0 con 0x7f81c01581d0 2024-04-25T00:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81c5c00640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81c015e4e0 0x7f81c0151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81c5200640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81c00cf3c0 0x7f81c01f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81bfe00640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81c01581d0 0x7f81c0151760 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00d800 tx=0x7f81ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81bcc00640 1 -- 172.21.15.83:0/3366866678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac00b6c0 con 0x7f81c01581d0 2024-04-25T00:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81bcc00640 1 -- 172.21.15.83:0/3366866678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ac019070 con 0x7f81c01581d0 2024-04-25T00:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81bcc00640 1 -- 172.21.15.83:0/3366866678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac00baa0 con 0x7f81c01581d0 2024-04-25T00:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81c7c00640 1 -- 172.21.15.83:0/3366866678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c014f830 con 0x7f81c01581d0 2024-04-25T00:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81c7c00640 1 -- 172.21.15.83:0/3366866678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c014bfd0 con 0x7f81c01581d0 2024-04-25T00:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.062+0000 7f81c7c00640 1 -- 172.21.15.83:0/3366866678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c00737f0 con 0x7f81c01581d0 2024-04-25T00:15:36.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.066+0000 7f81bcc00640 1 -- 172.21.15.83:0/3366866678 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f81ac00bc40 con 0x7f81c01581d0 2024-04-25T00:15:36.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.066+0000 7f81bcc00640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81a00678c0 0x7f81a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.066+0000 7f81bcc00640 1 -- 172.21.15.83:0/3366866678 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f81ac085ff0 con 0x7f81c01581d0 2024-04-25T00:15:36.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.066+0000 7f81c5200640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81a00678c0 0x7f81a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.066+0000 7f81c5200640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81a00678c0 0x7f81a0069d80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f81b00af000 tx=0x7f81b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.070+0000 7f81bcc00640 1 -- 172.21.15.83:0/3366866678 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f81ac056bd0 con 0x7f81c01581d0 2024-04-25T00:15:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.222+0000 7f81c7c00640 1 -- 172.21.15.83:0/3366866678 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81c00cc2f0 con 0x7f81a00678c0 2024-04-25T00:15:36.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.222+0000 7f81bcc00640 1 -- 172.21.15.83:0/3366866678 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13520 (secure 0 0 0) 0x7f81c00cc2f0 con 0x7f81a00678c0 2024-04-25T00:15:36.223 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:36.223 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:15:36.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.222+0000 7f81c7c00640 1 -- 172.21.15.83:0/3366866678 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81a00678c0 msgr2=0x7f81a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.222+0000 7f81c7c00640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81a00678c0 0x7f81a0069d80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f81b00af000 tx=0x7f81b0008040 comp rx=0 tx=0).stop 2024-04-25T00:15:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.222+0000 7f81c7c00640 1 -- 172.21.15.83:0/3366866678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81c01581d0 msgr2=0x7f81c0151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.222+0000 7f81c7c00640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81c01581d0 0x7f81c0151760 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00d800 tx=0x7f81ac00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.222+0000 7f81c7c00640 1 -- 172.21.15.83:0/3366866678 shutdown_connections 2024-04-25T00:15:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.222+0000 7f81c7c00640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81a00678c0 0x7f81a0069d80 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.222+0000 7f81c7c00640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81c015e4e0 0x7f81c0151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.222+0000 7f81c7c00640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81c01581d0 0x7f81c0151760 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.222+0000 7f81c7c00640 1 --2- 172.21.15.83:0/3366866678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81c00cf3c0 0x7f81c01f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.222+0000 7f81c7c00640 1 -- 172.21.15.83:0/3366866678 >> 172.21.15.83:0/3366866678 conn(0x7f81c00b1830 msgr2=0x7f81c015baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.222+0000 7f81c7c00640 1 -- 172.21.15.83:0/3366866678 shutdown_connections 2024-04-25T00:15:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.222+0000 7f81c7c00640 1 -- 172.21.15.83:0/3366866678 wait complete. 2024-04-25T00:15:36.243 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-04-25T00:15:34.496927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260120,"kb_used_data":10712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714600,"statfs":{"total":773094113280,"available":772827750400,"internally_reserved":0,"allocated":10969088,"data_stored":5656414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":79,"apply_latency_ms":79,"commit_latency_ns":79000000,"apply_latency_ns":79000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20691,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-92,"num_read_kb":-87,"num_write":-142,"num_write_kb":-69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001406"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":312,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:15:05.186878+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:15:05.186878+0000","last_peered":"2024-04-25T00:15:05.186878+0000","last_clean":"2024-04-25T00:15:05.186878+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:15:05.186878+0000","last_undegraded":"2024-04-25T00:15:05.186878+0000","last_fullsized":"2024-04-25T00:15:05.186878+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32956,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338884,"statfs":{"total":96636764160,"available":96603017216,"internally_reserved":0,"allocated":1822720,"data_stored":1151151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1257472,"data_stored":576517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1208320,"data_stored":553570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1216512,"data_stored":553662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1212416,"data_stored":554084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1208320,"data_stored":553530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1212416,"data_stored":553620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32964,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338876,"statfs":{"total":96636764160,"available":96603009024,"internally_reserved":0,"allocated":1830912,"data_stored":1160280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:15:36.243 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:15:36.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f820a000640 1 Processor -- start 2024-04-25T00:15:36.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f820a000640 1 -- start start 2024-04-25T00:15:36.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f820a000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f82041553e0 0x7f820406e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f820a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f82040b4a70 0x7f82040b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f820a000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f820406ef80 con 0x7f820406ea00 2024-04-25T00:15:36.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f820a000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f820415b510 con 0x7f82041553e0 2024-04-25T00:15:36.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f820a000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f820415b7f0 con 0x7f82040b4a70 2024-04-25T00:15:36.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f8203400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f82041553e0 0x7f820406e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f8203400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f82041553e0 0x7f820406e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60120/0 (socket says 172.21.15.83:60120) 2024-04-25T00:15:36.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f8202a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f820406ea00 0x7f82040b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35810/0 (socket says 172.21.15.83:35810) 2024-04-25T00:15:36.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f8203400640 1 -- 172.21.15.83:0/813729207 learned_addr learned my addr 172.21.15.83:0/813729207 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:36.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f8203e00640 1 --2- 172.21.15.83:0/813729207 >> v2:172.21.15.86:3300/0 conn(0x7f82040b4a70 0x7f82040b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f8203400640 1 -- 172.21.15.83:0/813729207 >> v2:172.21.15.86:3300/0 conn(0x7f82040b4a70 msgr2=0x7f82040b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:36.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f8203400640 1 --2- 172.21.15.83:0/813729207 >> v2:172.21.15.86:3300/0 conn(0x7f82040b4a70 0x7f82040b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f8203400640 1 -- 172.21.15.83:0/813729207 >> v1:172.21.15.83:6789/0 conn(0x7f820406ea00 legacy=0x7f82040b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.378+0000 7f8203400640 1 -- 172.21.15.83:0/813729207 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820415bad0 con 0x7f82041553e0 2024-04-25T00:15:36.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.382+0000 7f8203400640 1 --2- 172.21.15.83:0/813729207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82041553e0 0x7f820406e4c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f820406eeb0 tx=0x7f81f401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf4c3814acf9b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:36.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.382+0000 7f8202000640 1 -- 172.21.15.83:0/813729207 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f40b1070 con 0x7f82041553e0 2024-04-25T00:15:36.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.382+0000 7f8202000640 1 -- 172.21.15.83:0/813729207 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81f40ac600 con 0x7f82041553e0 2024-04-25T00:15:36.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.382+0000 7f8202000640 1 -- 172.21.15.83:0/813729207 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f40ac9d0 con 0x7f82041553e0 2024-04-25T00:15:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.382+0000 7f820a000640 1 -- 172.21.15.83:0/813729207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82041553e0 msgr2=0x7f820406e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.382+0000 7f820a000640 1 --2- 172.21.15.83:0/813729207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82041553e0 0x7f820406e4c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f820406eeb0 tx=0x7f81f401c490 comp rx=0 tx=0).stop 2024-04-25T00:15:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.382+0000 7f820a000640 1 -- 172.21.15.83:0/813729207 shutdown_connections 2024-04-25T00:15:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.382+0000 7f820a000640 1 --2- 172.21.15.83:0/813729207 >> v2:172.21.15.86:3300/0 conn(0x7f82040b4a70 0x7f82040b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.382+0000 7f820a000640 1 --2- 172.21.15.83:0/813729207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82041553e0 0x7f820406e4c0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.382+0000 7f820a000640 1 -- 172.21.15.83:0/813729207 >> 172.21.15.83:0/813729207 conn(0x7f82040b1830 msgr2=0x7f82040b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.382+0000 7f820a000640 1 -- 172.21.15.83:0/813729207 shutdown_connections 2024-04-25T00:15:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.382+0000 7f820a000640 1 -- 172.21.15.83:0/813729207 wait complete. 2024-04-25T00:15:36.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f820a000640 1 Processor -- start 2024-04-25T00:15:36.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f820a000640 1 -- start start 2024-04-25T00:15:36.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f820a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f820406ea00 0x7f82041f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f820a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82040b4a70 0x7f8204145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f820a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82041553e0 0x7f820414d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f820a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82040b3a00 con 0x7f82040b4a70 2024-04-25T00:15:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f820a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82040b3880 con 0x7f82041553e0 2024-04-25T00:15:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f820a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f82040b3b80 con 0x7f820406ea00 2024-04-25T00:15:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f8202a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82040b4a70 0x7f8204145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f8202a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82040b4a70 0x7f8204145150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60130/0 (socket says 172.21.15.83:60130) 2024-04-25T00:15:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f8202a00640 1 -- 172.21.15.83:0/2412769277 learned_addr learned my addr 172.21.15.83:0/2412769277 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f8203400640 1 --2- 172.21.15.83:0/2412769277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f820406ea00 0x7f82041f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f8203e00640 1 --2- 172.21.15.83:0/2412769277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82041553e0 0x7f820414d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f8202a00640 1 -- 172.21.15.83:0/2412769277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f820406ea00 msgr2=0x7f82041f7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f8202a00640 1 --2- 172.21.15.83:0/2412769277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f820406ea00 0x7f82041f7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f8202a00640 1 -- 172.21.15.83:0/2412769277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82041553e0 msgr2=0x7f820414d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f8202a00640 1 --2- 172.21.15.83:0/2412769277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82041553e0 0x7f820414d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f8202a00640 1 -- 172.21.15.83:0/2412769277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820415e410 con 0x7f82040b4a70 2024-04-25T00:15:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f8202a00640 1 --2- 172.21.15.83:0/2412769277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82040b4a70 0x7f8204145150 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00c370 tx=0x7f81ec00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f81fb400640 1 -- 172.21.15.83:0/2412769277 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec0074e0 con 0x7f82040b4a70 2024-04-25T00:15:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f81fb400640 1 -- 172.21.15.83:0/2412769277 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ec007680 con 0x7f82040b4a70 2024-04-25T00:15:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f81fb400640 1 -- 172.21.15.83:0/2412769277 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec004a30 con 0x7f82040b4a70 2024-04-25T00:15:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f820a000640 1 -- 172.21.15.83:0/2412769277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82041594e0 con 0x7f82040b4a70 2024-04-25T00:15:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f820a000640 1 -- 172.21.15.83:0/2412769277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8204159a40 con 0x7f82040b4a70 2024-04-25T00:15:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.386+0000 7f820a000640 1 -- 172.21.15.83:0/2412769277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82040737f0 con 0x7f82040b4a70 2024-04-25T00:15:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.390+0000 7f81fb400640 1 -- 172.21.15.83:0/2412769277 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f81ec01a020 con 0x7f82040b4a70 2024-04-25T00:15:36.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.390+0000 7f81fb400640 1 --2- 172.21.15.83:0/2412769277 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81e00678c0 0x7f81e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.390+0000 7f81fb400640 1 -- 172.21.15.83:0/2412769277 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f81ec085280 con 0x7f82040b4a70 2024-04-25T00:15:36.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.390+0000 7f8203400640 1 --2- 172.21.15.83:0/2412769277 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81e00678c0 0x7f81e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.390+0000 7f8203400640 1 --2- 172.21.15.83:0/2412769277 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81e00678c0 0x7f81e0069d80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f81f40ae000 tx=0x7f81f40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:36.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.394+0000 7f81fb400640 1 -- 172.21.15.83:0/2412769277 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f82041594e0 con 0x7f82040b4a70 2024-04-25T00:15:36.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.550+0000 7f820a000640 1 -- 172.21.15.83:0/2412769277 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f820404f5f0 con 0x7f81e00678c0 2024-04-25T00:15:36.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.550+0000 7f81fb400640 1 -- 172.21.15.83:0/2412769277 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13488 (secure 0 0 0) 0x7f820404f5f0 con 0x7f81e00678c0 2024-04-25T00:15:36.551 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:36.552 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:15:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.554+0000 7f820a000640 1 -- 172.21.15.83:0/2412769277 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81e00678c0 msgr2=0x7f81e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.554+0000 7f820a000640 1 --2- 172.21.15.83:0/2412769277 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81e00678c0 0x7f81e0069d80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f81f40ae000 tx=0x7f81f40a7000 comp rx=0 tx=0).stop 2024-04-25T00:15:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.554+0000 7f820a000640 1 -- 172.21.15.83:0/2412769277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82040b4a70 msgr2=0x7f8204145150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.554+0000 7f820a000640 1 --2- 172.21.15.83:0/2412769277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82040b4a70 0x7f8204145150 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00c370 tx=0x7f81ec00c830 comp rx=0 tx=0).stop 2024-04-25T00:15:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.554+0000 7f820a000640 1 -- 172.21.15.83:0/2412769277 shutdown_connections 2024-04-25T00:15:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.554+0000 7f820a000640 1 --2- 172.21.15.83:0/2412769277 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f81e00678c0 0x7f81e0069d80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.554+0000 7f820a000640 1 --2- 172.21.15.83:0/2412769277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82041553e0 0x7f820414d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.554+0000 7f820a000640 1 --2- 172.21.15.83:0/2412769277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82040b4a70 0x7f8204145150 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.554+0000 7f820a000640 1 --2- 172.21.15.83:0/2412769277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f820406ea00 0x7f82041f7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.554+0000 7f820a000640 1 -- 172.21.15.83:0/2412769277 >> 172.21.15.83:0/2412769277 conn(0x7f82040b1830 msgr2=0x7f820415cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.554+0000 7f820a000640 1 -- 172.21.15.83:0/2412769277 shutdown_connections 2024-04-25T00:15:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.554+0000 7f820a000640 1 -- 172.21.15.83:0/2412769277 wait complete. 2024-04-25T00:15:36.572 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-04-25T00:15:36.497220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":260272,"kb_used_data":10824,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714448,"statfs":{"total":773094113280,"available":772827594752,"internally_reserved":0,"allocated":11083776,"data_stored":5726029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23919,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-95,"num_read_kb":-89,"num_write":-167,"num_write_kb":-83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001355"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":314,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T00:15:31.345362+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:15:31.345362+0000","last_peered":"2024-04-25T00:15:31.345362+0000","last_clean":"2024-04-25T00:15:31.345362+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:15:31.345362+0000","last_undegraded":"2024-04-25T00:15:31.345362+0000","last_fullsized":"2024-04-25T00:15:31.345362+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32972,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338868,"statfs":{"total":96636764160,"available":96603000832,"internally_reserved":0,"allocated":1835008,"data_stored":1157968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32404,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1220608,"data_stored":560387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":566950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32404,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32372,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1220608,"data_stored":560347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":566950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32980,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1843200,"data_stored":1167097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:15:36.572 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T00:15:36.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:15:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.706+0000 7fa612a00640 1 Processor -- start 2024-04-25T00:15:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.706+0000 7fa612a00640 1 -- start start 2024-04-25T00:15:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa612a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa60c164cb0 0x7fa60c1670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa612a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa60c0ccc60 con 0x7fa60c1599f0 2024-04-25T00:15:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa612a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa60c06dba0 con 0x7fa60c0cc510 2024-04-25T00:15:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa612a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa60c159dd0 con 0x7fa60c164cb0 2024-04-25T00:15:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa60be00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa60c1599f0 0x7fa60c15c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35820/0 (socket says 172.21.15.83:35820) 2024-04-25T00:15:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa60be00640 1 -- 172.21.15.83:0/247854087 learned_addr learned my addr 172.21.15.83:0/247854087 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:36.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa610a00640 1 --2- 172.21.15.83:0/247854087 >> v2:172.21.15.83:3300/0 conn(0x7fa60c164cb0 0x7fa60c1670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa60aa00640 1 -- 172.21.15.83:0/247854087 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1343520625 0 0) 0x7fa5fc0014a0 con 0x7fa60c1599f0 2024-04-25T00:15:36.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa60aa00640 1 -- 172.21.15.83:0/247854087 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5f0003530 con 0x7fa60c1599f0 2024-04-25T00:15:36.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa610a00640 1 -- 172.21.15.83:0/247854087 >> v1:172.21.15.86:6789/0 conn(0x7fa60c0cc510 legacy=0x7fa60c15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa610a00640 1 -- 172.21.15.83:0/247854087 >> v1:172.21.15.83:6789/0 conn(0x7fa60c1599f0 legacy=0x7fa60c15c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa610a00640 1 -- 172.21.15.83:0/247854087 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60c06de80 con 0x7fa60c164cb0 2024-04-25T00:15:36.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa610a00640 1 --2- 172.21.15.83:0/247854087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60c164cb0 0x7fa60c1670a0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fa6040097f0 tx=0x7fa60401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346df19d6a6c291b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:36.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa60aa00640 1 -- 172.21.15.83:0/247854087 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6040b0070 con 0x7fa60c164cb0 2024-04-25T00:15:36.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa60aa00640 1 -- 172.21.15.83:0/247854087 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6040ab600 con 0x7fa60c164cb0 2024-04-25T00:15:36.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.710+0000 7fa60aa00640 1 -- 172.21.15.83:0/247854087 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6040ab9d0 con 0x7fa60c164cb0 2024-04-25T00:15:36.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 -- 172.21.15.83:0/247854087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60c164cb0 msgr2=0x7fa60c1670a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 --2- 172.21.15.83:0/247854087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60c164cb0 0x7fa60c1670a0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fa6040097f0 tx=0x7fa60401c410 comp rx=0 tx=0).stop 2024-04-25T00:15:36.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 -- 172.21.15.83:0/247854087 shutdown_connections 2024-04-25T00:15:36.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 --2- 172.21.15.83:0/247854087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60c164cb0 0x7fa60c1670a0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 -- 172.21.15.83:0/247854087 >> 172.21.15.83:0/247854087 conn(0x7fa60c0b1830 msgr2=0x7fa60c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:36.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 -- 172.21.15.83:0/247854087 shutdown_connections 2024-04-25T00:15:36.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 -- 172.21.15.83:0/247854087 wait complete. 2024-04-25T00:15:36.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 Processor -- start 2024-04-25T00:15:36.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 -- start start 2024-04-25T00:15:36.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa60c0cc510 0x7fa60c144500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa60c1599f0 0x7fa60c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60c164cb0 0x7fa60c0cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c0ce240 con 0x7fa60c164cb0 2024-04-25T00:15:36.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c0ce0c0 con 0x7fa60c0cc510 2024-04-25T00:15:36.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa612a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa60c0cf6f0 con 0x7fa60c1599f0 2024-04-25T00:15:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa610a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60c164cb0 0x7fa60c0cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa610a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60c164cb0 0x7fa60c0cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60148/0 (socket says 172.21.15.83:60148) 2024-04-25T00:15:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa610a00640 1 -- 172.21.15.83:0/3606780841 learned_addr learned my addr 172.21.15.83:0/3606780841 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa60be00640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa60c0cc510 0x7fa60c144500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa60b400640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa60c1599f0 0x7fa60c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa610a00640 1 -- 172.21.15.83:0/3606780841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa60c1599f0 msgr2=0x7fa60c0b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa610a00640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa60c1599f0 0x7fa60c0b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa610a00640 1 -- 172.21.15.83:0/3606780841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa60c0cc510 msgr2=0x7fa60c144500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa610a00640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa60c0cc510 0x7fa60c144500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa610a00640 1 -- 172.21.15.83:0/3606780841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60c0b15d0 con 0x7fa60c164cb0 2024-04-25T00:15:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.714+0000 7fa60be00640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa60c0cc510 0x7fa60c144500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.718+0000 7fa610a00640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60c164cb0 0x7fa60c0cfa80 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fa6040ae040 tx=0x7fa6040191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.718+0000 7fa603e00640 1 -- 172.21.15.83:0/3606780841 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6040b8070 con 0x7fa60c164cb0 2024-04-25T00:15:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.718+0000 7fa603e00640 1 -- 172.21.15.83:0/3606780841 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6040b4410 con 0x7fa60c164cb0 2024-04-25T00:15:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.718+0000 7fa612a00640 1 -- 172.21.15.83:0/3606780841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60c144a40 con 0x7fa60c164cb0 2024-04-25T00:15:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.718+0000 7fa603e00640 1 -- 172.21.15.83:0/3606780841 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6040b0040 con 0x7fa60c164cb0 2024-04-25T00:15:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.718+0000 7fa612a00640 1 -- 172.21.15.83:0/3606780841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60c0b1010 con 0x7fa60c164cb0 2024-04-25T00:15:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.718+0000 7fa612a00640 1 -- 172.21.15.83:0/3606780841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa60c0737f0 con 0x7fa60c164cb0 2024-04-25T00:15:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.718+0000 7fa603e00640 1 -- 172.21.15.83:0/3606780841 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa6040bb070 con 0x7fa60c164cb0 2024-04-25T00:15:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.718+0000 7fa603e00640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5ec0678c0 0x7fa5ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.718+0000 7fa603e00640 1 -- 172.21.15.83:0/3606780841 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7fa60411f7c0 con 0x7fa60c164cb0 2024-04-25T00:15:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.718+0000 7fa60be00640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5ec0678c0 0x7fa5ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.718+0000 7fa60be00640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5ec0678c0 0x7fa5ec069d80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc001640 tx=0x7fa5fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:36.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.722+0000 7fa603e00640 1 -- 172.21.15.83:0/3606780841 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa604016090 con 0x7fa60c164cb0 2024-04-25T00:15:36.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.874+0000 7fa612a00640 1 -- 172.21.15.83:0/3606780841 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa60c072c40 con 0x7fa5ec0678c0 2024-04-25T00:15:36.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.874+0000 7fa603e00640 1 -- 172.21.15.83:0/3606780841 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13488 (secure 0 0 0) 0x7fa60c072c40 con 0x7fa5ec0678c0 2024-04-25T00:15:36.877 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:36.877 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:15:36.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.878+0000 7fa612a00640 1 -- 172.21.15.83:0/3606780841 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5ec0678c0 msgr2=0x7fa5ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.878+0000 7fa612a00640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5ec0678c0 0x7fa5ec069d80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc001640 tx=0x7fa5fc009040 comp rx=0 tx=0).stop 2024-04-25T00:15:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.878+0000 7fa612a00640 1 -- 172.21.15.83:0/3606780841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60c164cb0 msgr2=0x7fa60c0cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.878+0000 7fa612a00640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60c164cb0 0x7fa60c0cfa80 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fa6040ae040 tx=0x7fa6040191b0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.878+0000 7fa612a00640 1 -- 172.21.15.83:0/3606780841 shutdown_connections 2024-04-25T00:15:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.878+0000 7fa612a00640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa5ec0678c0 0x7fa5ec069d80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.878+0000 7fa612a00640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa60c1599f0 0x7fa60c0b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.878+0000 7fa612a00640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60c164cb0 0x7fa60c0cfa80 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.878+0000 7fa612a00640 1 --2- 172.21.15.83:0/3606780841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa60c0cc510 0x7fa60c144500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.878+0000 7fa612a00640 1 -- 172.21.15.83:0/3606780841 >> 172.21.15.83:0/3606780841 conn(0x7fa60c0b1830 msgr2=0x7fa60c0ad860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.878+0000 7fa612a00640 1 -- 172.21.15.83:0/3606780841 shutdown_connections 2024-04-25T00:15:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:36.878+0000 7fa612a00640 1 -- 172.21.15.83:0/3606780841 wait complete. 2024-04-25T00:15:36.897 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-04-25T00:15:36.497220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":260272,"kb_used_data":10824,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714448,"statfs":{"total":773094113280,"available":772827594752,"internally_reserved":0,"allocated":11083776,"data_stored":5726029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23919,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-95,"num_read_kb":-89,"num_write":-167,"num_write_kb":-83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001355"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":314,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T00:15:31.345362+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:15:31.345362+0000","last_peered":"2024-04-25T00:15:31.345362+0000","last_clean":"2024-04-25T00:15:31.345362+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:15:31.345362+0000","last_undegraded":"2024-04-25T00:15:31.345362+0000","last_fullsized":"2024-04-25T00:15:31.345362+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32972,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338868,"statfs":{"total":96636764160,"available":96603000832,"internally_reserved":0,"allocated":1835008,"data_stored":1157968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32404,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1220608,"data_stored":560387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":566950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32404,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32372,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1220608,"data_stored":560347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":566950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32980,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1843200,"data_stored":1167097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:15:36.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:15:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f8540800640 1 Processor -- start 2024-04-25T00:15:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f8540800640 1 -- start start 2024-04-25T00:15:37.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f8540800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85381599f0 0x7f85380cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f8540800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f853815cec0 0x7f85380ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f8540800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85380ccc60 con 0x7f853815c340 2024-04-25T00:15:37.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f8540800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8538159dd0 con 0x7f85381599f0 2024-04-25T00:15:37.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f8540800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f853806dba0 con 0x7f853815cec0 2024-04-25T00:15:37.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f853de00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85381599f0 0x7f85380cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f853de00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85381599f0 0x7f85380cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60158/0 (socket says 172.21.15.83:60158) 2024-04-25T00:15:37.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f853de00640 1 -- 172.21.15.83:0/1287770684 learned_addr learned my addr 172.21.15.83:0/1287770684 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:37.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f853e800640 1 --2- 172.21.15.83:0/1287770684 >> v2:172.21.15.86:3300/0 conn(0x7f853815cec0 0x7f85380ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f853de00640 1 -- 172.21.15.83:0/1287770684 >> v2:172.21.15.86:3300/0 conn(0x7f853815cec0 msgr2=0x7f85380ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:37.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f853de00640 1 --2- 172.21.15.83:0/1287770684 >> v2:172.21.15.86:3300/0 conn(0x7f853815cec0 0x7f85380ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f853de00640 1 -- 172.21.15.83:0/1287770684 >> v1:172.21.15.86:6789/0 conn(0x7f853815c340 legacy=0x7f853815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:37.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.034+0000 7f853de00640 1 -- 172.21.15.83:0/1287770684 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853806de80 con 0x7f85381599f0 2024-04-25T00:15:37.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.038+0000 7f853de00640 1 --2- 172.21.15.83:0/1287770684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85381599f0 0x7f85380cc510 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f852c009870 tx=0x7f852c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db643ffab7bb6116 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:37.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.038+0000 7f853ca00640 1 -- 172.21.15.83:0/1287770684 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0b0070 con 0x7f85381599f0 2024-04-25T00:15:37.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.038+0000 7f853ca00640 1 -- 172.21.15.83:0/1287770684 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f852c0ab600 con 0x7f85381599f0 2024-04-25T00:15:37.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.038+0000 7f853ca00640 1 -- 172.21.15.83:0/1287770684 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0ab9d0 con 0x7f85381599f0 2024-04-25T00:15:37.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.038+0000 7f8540800640 1 -- 172.21.15.83:0/1287770684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85381599f0 msgr2=0x7f85380cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.038+0000 7f8540800640 1 --2- 172.21.15.83:0/1287770684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85381599f0 0x7f85380cc510 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f852c009870 tx=0x7f852c01c490 comp rx=0 tx=0).stop 2024-04-25T00:15:37.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.038+0000 7f8540800640 1 -- 172.21.15.83:0/1287770684 shutdown_connections 2024-04-25T00:15:37.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.038+0000 7f8540800640 1 --2- 172.21.15.83:0/1287770684 >> v2:172.21.15.86:3300/0 conn(0x7f853815cec0 0x7f85380ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.038+0000 7f8540800640 1 --2- 172.21.15.83:0/1287770684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85381599f0 0x7f85380cc510 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.038+0000 7f8540800640 1 -- 172.21.15.83:0/1287770684 >> 172.21.15.83:0/1287770684 conn(0x7f85380b1830 msgr2=0x7f85380b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:37.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.038+0000 7f8540800640 1 -- 172.21.15.83:0/1287770684 shutdown_connections 2024-04-25T00:15:37.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.038+0000 7f8540800640 1 -- 172.21.15.83:0/1287770684 wait complete. 2024-04-25T00:15:37.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.038+0000 7f8540800640 1 Processor -- start 2024-04-25T00:15:37.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.038+0000 7f8540800640 1 -- start start 2024-04-25T00:15:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f8540800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85381599f0 0x7f8538155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f8540800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f853815c340 0x7f853815b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f8540800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f853815cec0 0x7f853815b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f8540800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85380cf800 con 0x7f85381599f0 2024-04-25T00:15:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f8540800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85380cf680 con 0x7f853815c340 2024-04-25T00:15:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f8540800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8538168cd0 con 0x7f853815cec0 2024-04-25T00:15:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f853de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85381599f0 0x7f8538155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f853de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85381599f0 0x7f8538155530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60170/0 (socket says 172.21.15.83:60170) 2024-04-25T00:15:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f853de00640 1 -- 172.21.15.83:0/2187563876 learned_addr learned my addr 172.21.15.83:0/2187563876 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f853e800640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f853815cec0 0x7f853815b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f853de00640 1 -- 172.21.15.83:0/2187563876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f853815cec0 msgr2=0x7f853815b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f853d400640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f853815c340 0x7f853815b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f853de00640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f853815cec0 0x7f853815b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f853de00640 1 -- 172.21.15.83:0/2187563876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f853815c340 msgr2=0x7f853815b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f853de00640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f853815c340 0x7f853815b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f853de00640 1 -- 172.21.15.83:0/2187563876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8538165400 con 0x7f85381599f0 2024-04-25T00:15:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f853d400640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f853815c340 0x7f853815b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f853de00640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85381599f0 0x7f8538155530 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f852c0ae040 tx=0x7f852c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f8532000640 1 -- 172.21.15.83:0/2187563876 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0b0040 con 0x7f85381599f0 2024-04-25T00:15:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f8532000640 1 -- 172.21.15.83:0/2187563876 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f852c0ab400 con 0x7f85381599f0 2024-04-25T00:15:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f8540800640 1 -- 172.21.15.83:0/2187563876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85380b3e70 con 0x7f85381599f0 2024-04-25T00:15:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f8532000640 1 -- 172.21.15.83:0/2187563876 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0b4660 con 0x7f85381599f0 2024-04-25T00:15:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f8540800640 1 -- 172.21.15.83:0/2187563876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85380b4430 con 0x7f85381599f0 2024-04-25T00:15:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f8540800640 1 -- 172.21.15.83:0/2187563876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85380737f0 con 0x7f85381599f0 2024-04-25T00:15:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.042+0000 7f8532000640 1 -- 172.21.15.83:0/2187563876 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f852c0bb020 con 0x7f85381599f0 2024-04-25T00:15:37.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.046+0000 7f8532000640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f851c0678c0 0x7f851c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.046+0000 7f8532000640 1 -- 172.21.15.83:0/2187563876 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f852c11f000 con 0x7f85381599f0 2024-04-25T00:15:37.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.046+0000 7f853d400640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f851c0678c0 0x7f851c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.046+0000 7f853d400640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f851c0678c0 0x7f851c069d80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8538152900 tx=0x7f8528009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:37.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.050+0000 7f8532000640 1 -- 172.21.15.83:0/2187563876 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f852c0c2020 con 0x7f85381599f0 2024-04-25T00:15:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.206+0000 7f8540800640 1 -- 172.21.15.83:0/2187563876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85380b4670 con 0x7f85381599f0 2024-04-25T00:15:37.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.206+0000 7f8532000640 1 -- 172.21.15.83:0/2187563876 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+13599 (secure 0 0 0) 0x7f852c016110 con 0x7f85381599f0 2024-04-25T00:15:37.207 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:37.207 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":100,"fsid":"d58ae83a-ad72-4e18-824a-86e779979c35","created":"2024-04-25T00:10:41.411795+0000","modified":"2024-04-25T00:15:33.964289+0000","last_up_change":"2024-04-25T00:10:53.743936+0000","last_in_change":"2024-04-25T00:10:43.925460+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:10:54.507055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fa38ca70-c391-4e90-a2fd-7700d7fa60e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6801","nonce":397603850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6803","nonce":397603850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6807","nonce":397603850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":397603850},{"type":"v1","addr":"172.21.15.83:6805","nonce":397603850}]},"public_addr":"172.21.15.83:6801/397603850","cluster_addr":"172.21.15.83:6803/397603850","heartbeat_back_addr":"172.21.15.83:6807/397603850","heartbeat_front_addr":"172.21.15.83:6805/397603850","state":["exists","up"]},{"osd":1,"uuid":"a944117d-465b-4cfa-9120-d9d9a3516af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6810","nonce":228430303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6814","nonce":228430303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6823","nonce":228430303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":228430303},{"type":"v1","addr":"172.21.15.83:6818","nonce":228430303}]},"public_addr":"172.21.15.83:6810/228430303","cluster_addr":"172.21.15.83:6814/228430303","heartbeat_back_addr":"172.21.15.83:6823/228430303","heartbeat_front_addr":"172.21.15.83:6818/228430303","state":["exists","up"]},{"osd":2,"uuid":"60f71d92-2018-4399-a106-2bd7d9c5f087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6825","nonce":1702952090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6827","nonce":1702952090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6831","nonce":1702952090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":1702952090},{"type":"v1","addr":"172.21.15.83:6829","nonce":1702952090}]},"public_addr":"172.21.15.83:6825/1702952090","cluster_addr":"172.21.15.83:6827/1702952090","heartbeat_back_addr":"172.21.15.83:6831/1702952090","heartbeat_front_addr":"172.21.15.83:6829/1702952090","state":["exists","up"]},{"osd":3,"uuid":"4f1a4f1a-7c74-479f-8a9f-0c4b7015b6e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6811","nonce":1635741185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6815","nonce":1635741185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6822","nonce":1635741185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":1635741185},{"type":"v1","addr":"172.21.15.83:6819","nonce":1635741185}]},"public_addr":"172.21.15.83:6811/1635741185","cluster_addr":"172.21.15.83:6815/1635741185","heartbeat_back_addr":"172.21.15.83:6822/1635741185","heartbeat_front_addr":"172.21.15.83:6819/1635741185","state":["exists","up"]},{"osd":4,"uuid":"dce8a21c-99a4-403f-83ff-ae3a6477b6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6817","nonce":1560646044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6819","nonce":1560646044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6827","nonce":1560646044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":1560646044},{"type":"v1","addr":"172.21.15.86:6823","nonce":1560646044}]},"public_addr":"172.21.15.86:6817/1560646044","cluster_addr":"172.21.15.86:6819/1560646044","heartbeat_back_addr":"172.21.15.86:6827/1560646044","heartbeat_front_addr":"172.21.15.86:6823/1560646044","state":["exists","up"]},{"osd":5,"uuid":"31352365-f5e7-41ee-9423-d44ec6db1f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6805","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6807","nonce":340393403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6809","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6811","nonce":340393403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6815","nonce":340393403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":340393403},{"type":"v1","addr":"172.21.15.86:6813","nonce":340393403}]},"public_addr":"172.21.15.86:6807/340393403","cluster_addr":"172.21.15.86:6811/340393403","heartbeat_back_addr":"172.21.15.86:6815/340393403","heartbeat_front_addr":"172.21.15.86:6813/340393403","state":["exists","up"]},{"osd":6,"uuid":"16fb48a3-c013-4db3-bdd3-34041bfec53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6801","nonce":3882786818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6803","nonce":3882786818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6810","nonce":3882786818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3882786818},{"type":"v1","addr":"172.21.15.86:6806","nonce":3882786818}]},"public_addr":"172.21.15.86:6801/3882786818","cluster_addr":"172.21.15.86:6803/3882786818","heartbeat_back_addr":"172.21.15.86:6810/3882786818","heartbeat_front_addr":"172.21.15.86:6806/3882786818","state":["exists","up"]},{"osd":7,"uuid":"dd0d5c3d-bcc9-4382-a09e-2b0e9d6e6f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6822","nonce":957843207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6826","nonce":957843207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6831","nonce":957843207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":957843207},{"type":"v1","addr":"172.21.15.86:6829","nonce":957843207}]},"public_addr":"172.21.15.86:6822/957843207","cluster_addr":"172.21.15.86:6826/957843207","heartbeat_back_addr":"172.21.15.86:6831/957843207","heartbeat_front_addr":"172.21.15.86:6829/957843207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.571700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.808025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.736126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.748075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.653102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.624499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.561499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:49.627347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6835/565131724":"2024-04-26T00:15:30.862120+0000","172.21.15.83:6837/2870595352":"2024-04-26T00:15:30.862135+0000","172.21.15.83:6837/1609715757":"2024-04-26T00:15:04.845080+0000","172.21.15.83:6836/1609715757":"2024-04-26T00:15:04.845080+0000","172.21.15.83:6834/565131724":"2024-04-26T00:15:30.862120+0000","172.21.15.86:6835/3612679148":"2024-04-26T00:15:04.826082+0000","172.21.15.83:6836/2870595352":"2024-04-26T00:15:30.862135+0000","172.21.15.86:6833/3612679148":"2024-04-26T00:15:04.826082+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:15:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.206+0000 7f8540800640 1 -- 172.21.15.83:0/2187563876 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f851c0678c0 msgr2=0x7f851c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.206+0000 7f8540800640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f851c0678c0 0x7f851c069d80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8538152900 tx=0x7f8528009040 comp rx=0 tx=0).stop 2024-04-25T00:15:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.206+0000 7f8540800640 1 -- 172.21.15.83:0/2187563876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85381599f0 msgr2=0x7f8538155530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.206+0000 7f8540800640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85381599f0 0x7f8538155530 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f852c0ae040 tx=0x7f852c019230 comp rx=0 tx=0).stop 2024-04-25T00:15:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.206+0000 7f8540800640 1 -- 172.21.15.83:0/2187563876 shutdown_connections 2024-04-25T00:15:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.206+0000 7f8540800640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f851c0678c0 0x7f851c069d80 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.210+0000 7f8540800640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f853815cec0 0x7f853815b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.210+0000 7f8540800640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f853815c340 0x7f853815b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.210+0000 7f8540800640 1 --2- 172.21.15.83:0/2187563876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85381599f0 0x7f8538155530 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.210+0000 7f8540800640 1 -- 172.21.15.83:0/2187563876 >> 172.21.15.83:0/2187563876 conn(0x7f85380b1830 msgr2=0x7f8538155d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.210+0000 7f8540800640 1 -- 172.21.15.83:0/2187563876 shutdown_connections 2024-04-25T00:15:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:37.210+0000 7f8540800640 1 -- 172.21.15.83:0/2187563876 wait complete. 2024-04-25T00:15:38.229 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T00:15:38.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:15:38.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.378+0000 7f1223200640 1 Processor -- start 2024-04-25T00:15:38.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.378+0000 7f1223200640 1 -- start start 2024-04-25T00:15:38.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.378+0000 7f1223200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f121c151480 0x7f121c072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.378+0000 7f1223200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f121c0cf6b0 0x7f121c0b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.378+0000 7f1223200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f121c15e000 0x7f121c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.378+0000 7f1223200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f121c15e970 con 0x7f121c15e000 2024-04-25T00:15:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.378+0000 7f1223200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f121c0b3e00 con 0x7f121c0cf6b0 2024-04-25T00:15:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.378+0000 7f1223200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f121c071470 con 0x7f121c151480 2024-04-25T00:15:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.382+0000 7f1221200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f121c15e000 0x7f121c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.382+0000 7f1221200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f121c15e000 0x7f121c15e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60172/0 (socket says 172.21.15.83:60172) 2024-04-25T00:15:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.382+0000 7f1221200640 1 -- 172.21.15.83:0/3574575569 learned_addr learned my addr 172.21.15.83:0/3574575569 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.382+0000 7f121be00640 1 --2- 172.21.15.83:0/3574575569 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f121c151480 0x7f121c072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.382+0000 7f121b400640 1 --2- 172.21.15.83:0/3574575569 >> v2:172.21.15.86:3300/0 conn(0x7f121c0cf6b0 0x7f121c0b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.382+0000 7f1221200640 1 -- 172.21.15.83:0/3574575569 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f121c151480 msgr2=0x7f121c072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.382+0000 7f1221200640 1 --2- 172.21.15.83:0/3574575569 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f121c151480 0x7f121c072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.382+0000 7f1221200640 1 -- 172.21.15.83:0/3574575569 >> v2:172.21.15.86:3300/0 conn(0x7f121c0cf6b0 msgr2=0x7f121c0b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.382+0000 7f1221200640 1 --2- 172.21.15.83:0/3574575569 >> v2:172.21.15.86:3300/0 conn(0x7f121c0cf6b0 0x7f121c0b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.382+0000 7f1221200640 1 -- 172.21.15.83:0/3574575569 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121c071750 con 0x7f121c15e000 2024-04-25T00:15:38.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.382+0000 7f1221200640 1 --2- 172.21.15.83:0/3574575569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f121c15e000 0x7f121c15e430 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f121c0d0710 tx=0x7f121401ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e682c09d0a5fc2f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:38.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.382+0000 7f121aa00640 1 -- 172.21.15.83:0/3574575569 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12140b4070 con 0x7f121c15e000 2024-04-25T00:15:38.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.382+0000 7f121aa00640 1 -- 172.21.15.83:0/3574575569 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12140a7bd0 con 0x7f121c15e000 2024-04-25T00:15:38.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.382+0000 7f121aa00640 1 -- 172.21.15.83:0/3574575569 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12140af440 con 0x7f121c15e000 2024-04-25T00:15:38.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 -- 172.21.15.83:0/3574575569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f121c15e000 msgr2=0x7f121c15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 --2- 172.21.15.83:0/3574575569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f121c15e000 0x7f121c15e430 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f121c0d0710 tx=0x7f121401ebc0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 -- 172.21.15.83:0/3574575569 shutdown_connections 2024-04-25T00:15:38.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 --2- 172.21.15.83:0/3574575569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f121c15e000 0x7f121c15e430 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 --2- 172.21.15.83:0/3574575569 >> v2:172.21.15.86:3300/0 conn(0x7f121c0cf6b0 0x7f121c0b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 --2- 172.21.15.83:0/3574575569 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f121c151480 0x7f121c072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 -- 172.21.15.83:0/3574575569 >> 172.21.15.83:0/3574575569 conn(0x7f121c06b3b0 msgr2=0x7f121c06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:38.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 -- 172.21.15.83:0/3574575569 shutdown_connections 2024-04-25T00:15:38.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 -- 172.21.15.83:0/3574575569 wait complete. 2024-04-25T00:15:38.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 Processor -- start 2024-04-25T00:15:38.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 -- start start 2024-04-25T00:15:38.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f121c0cf6b0 0x7f121c15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121c151480 0x7f121c14a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f121c15e000 0x7f121c159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f121c072770 con 0x7f121c0cf6b0 2024-04-25T00:15:38.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f121c0b39f0 con 0x7f121c151480 2024-04-25T00:15:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1223200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f121c14fa70 con 0x7f121c15e000 2024-04-25T00:15:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f121be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f121c0cf6b0 0x7f121c15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f121be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f121c0cf6b0 0x7f121c15c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60180/0 (socket says 172.21.15.83:60180) 2024-04-25T00:15:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f121be00640 1 -- 172.21.15.83:0/553885994 learned_addr learned my addr 172.21.15.83:0/553885994 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f1221200640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f121c15e000 0x7f121c159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f121b400640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121c151480 0x7f121c14a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f121be00640 1 -- 172.21.15.83:0/553885994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f121c15e000 msgr2=0x7f121c159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f121be00640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f121c15e000 0x7f121c159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f121be00640 1 -- 172.21.15.83:0/553885994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121c151480 msgr2=0x7f121c14a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f121be00640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121c151480 0x7f121c14a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f121be00640 1 -- 172.21.15.83:0/553885994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121c1497b0 con 0x7f121c0cf6b0 2024-04-25T00:15:38.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f121b400640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f121c151480 0x7f121c14a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:38.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.386+0000 7f121be00640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f121c0cf6b0 0x7f121c15c550 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f120c007ec0 tx=0x7f120c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:38.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f1213e00640 1 -- 172.21.15.83:0/553885994 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c014620 con 0x7f121c0cf6b0 2024-04-25T00:15:38.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f1213e00640 1 -- 172.21.15.83:0/553885994 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f120c019070 con 0x7f121c0cf6b0 2024-04-25T00:15:38.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f1213e00640 1 -- 172.21.15.83:0/553885994 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c014a20 con 0x7f121c0cf6b0 2024-04-25T00:15:38.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f1223200640 1 -- 172.21.15.83:0/553885994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121c149a90 con 0x7f121c0cf6b0 2024-04-25T00:15:38.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f1223200640 1 -- 172.21.15.83:0/553885994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f121c15ca90 con 0x7f121c0cf6b0 2024-04-25T00:15:38.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f1223200640 1 -- 172.21.15.83:0/553885994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f121c0736f0 con 0x7f121c0cf6b0 2024-04-25T00:15:38.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f1213e00640 1 -- 172.21.15.83:0/553885994 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f120c014bc0 con 0x7f121c0cf6b0 2024-04-25T00:15:38.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f1213e00640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f11fc0678c0 0x7f11fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f1213e00640 1 -- 172.21.15.83:0/553885994 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f120c086590 con 0x7f121c0cf6b0 2024-04-25T00:15:38.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f121b400640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f11fc0678c0 0x7f11fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f1213e00640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.83:6800/397603850,v1:172.21.15.83:6801/397603850] conn(0x7f11fc06df40 0x7f11fc070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f1213e00640 1 -- 172.21.15.83:0/553885994 --> [v2:172.21.15.83:6800/397603850,v1:172.21.15.83:6801/397603850] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11fc070a50 con 0x7f11fc06df40 2024-04-25T00:15:38.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f1213e00640 1 -- 172.21.15.83:0/553885994 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f120c08d050 con 0x7f121c0cf6b0 2024-04-25T00:15:38.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f1221200640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.83:6800/397603850,v1:172.21.15.83:6801/397603850] conn(0x7f11fc06df40 0x7f11fc070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f121b400640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f11fc0678c0 0x7f11fc069d80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1208004660 tx=0x7f1208008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:38.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.390+0000 7f1221200640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.83:6800/397603850,v1:172.21.15.83:6801/397603850] conn(0x7f11fc06df40 0x7f11fc070380 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:38.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.394+0000 7f1213e00640 1 -- 172.21.15.83:0/553885994 <== osd.0 v2:172.21.15.83:6800/397603850 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f11fc070a50 con 0x7f11fc06df40 2024-04-25T00:15:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.406+0000 7f1223200640 1 -- 172.21.15.83:0/553885994 --> [v2:172.21.15.83:6800/397603850,v1:172.21.15.83:6801/397603850] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f121c04f5f0 con 0x7f11fc06df40 2024-04-25T00:15:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.406+0000 7f1213e00640 1 -- 172.21.15.83:0/553885994 <== osd.0 v2:172.21.15.83:6800/397603850 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f121c04f5f0 con 0x7f11fc06df40 2024-04-25T00:15:38.407 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:15:38.407 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:15:38.407 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:15:38.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.406+0000 7f1223200640 1 -- 172.21.15.83:0/553885994 >> [v2:172.21.15.83:6800/397603850,v1:172.21.15.83:6801/397603850] conn(0x7f11fc06df40 msgr2=0x7f11fc070380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.406+0000 7f1223200640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.83:6800/397603850,v1:172.21.15.83:6801/397603850] conn(0x7f11fc06df40 0x7f11fc070380 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.406+0000 7f1223200640 1 -- 172.21.15.83:0/553885994 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f11fc0678c0 msgr2=0x7f11fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.406+0000 7f1223200640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f11fc0678c0 0x7f11fc069d80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1208004660 tx=0x7f1208008040 comp rx=0 tx=0).stop 2024-04-25T00:15:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.406+0000 7f1223200640 1 -- 172.21.15.83:0/553885994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f121c0cf6b0 msgr2=0x7f121c15c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.406+0000 7f1223200640 1 --2- 172.21.15.83:0/553885994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f121c0cf6b0 0x7f121c15c550 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f120c007ec0 tx=0x7f120c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.406+0000 7f1221200640 1 -- 172.21.15.83:0/553885994 reap_dead start 2024-04-25T00:15:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.406+0000 7f1223200640 1 -- 172.21.15.83:0/553885994 shutdown_connections 2024-04-25T00:15:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.406+0000 7f1223200640 1 -- 172.21.15.83:0/553885994 >> 172.21.15.83:0/553885994 conn(0x7f121c06b3b0 msgr2=0x7f121c0b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:38.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.406+0000 7f1223200640 1 -- 172.21.15.83:0/553885994 shutdown_connections 2024-04-25T00:15:38.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.406+0000 7f1223200640 1 -- 172.21.15.83:0/553885994 wait complete. 2024-04-25T00:15:38.425 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T00:15:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27d0e00640 1 Processor -- start 2024-04-25T00:15:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27d0e00640 1 -- start start 2024-04-25T00:15:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27d0e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f27cc1446e0 0x7f27cc0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27d0e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc15f350 0x7f27cc06e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27d0e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27cc072ff0 con 0x7f27cc06ec50 2024-04-25T00:15:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27d0e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f27cc0715b0 con 0x7f27cc1446e0 2024-04-25T00:15:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27d0e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f27cc144ac0 con 0x7f27cc15f350 2024-04-25T00:15:38.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27ca800640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f27cc06ec50 0x7f27cc15b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35834/0 (socket says 172.21.15.83:35834) 2024-04-25T00:15:38.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27ca800640 1 -- 172.21.15.83:0/149020772 learned_addr learned my addr 172.21.15.83:0/149020772 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:38.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27c9e00640 1 --2- 172.21.15.83:0/149020772 >> v2:172.21.15.86:3300/0 conn(0x7f27cc1446e0 0x7f27cc0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27c9400640 1 --2- 172.21.15.83:0/149020772 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc15f350 0x7f27cc06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27c8a00640 1 -- 172.21.15.83:0/149020772 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1091857583 0 0) 0x7f27cc072ff0 con 0x7f27cc06ec50 2024-04-25T00:15:38.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27c9400640 1 -- 172.21.15.83:0/149020772 >> v2:172.21.15.86:3300/0 conn(0x7f27cc1446e0 msgr2=0x7f27cc0cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:38.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27c9400640 1 --2- 172.21.15.83:0/149020772 >> v2:172.21.15.86:3300/0 conn(0x7f27cc1446e0 0x7f27cc0cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27c9400640 1 -- 172.21.15.83:0/149020772 >> v1:172.21.15.83:6789/0 conn(0x7f27cc06ec50 legacy=0x7f27cc15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.562+0000 7f27c9400640 1 -- 172.21.15.83:0/149020772 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27cc0cb210 con 0x7f27cc15f350 2024-04-25T00:15:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.566+0000 7f27c9400640 1 --2- 172.21.15.83:0/149020772 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc15f350 0x7f27cc06e710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f27b80097c0 tx=0x7f27b801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d425f70ec06c954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.566+0000 7f27c8a00640 1 -- 172.21.15.83:0/149020772 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b80b0070 con 0x7f27cc15f350 2024-04-25T00:15:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.566+0000 7f27c8a00640 1 -- 172.21.15.83:0/149020772 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b80ab600 con 0x7f27cc15f350 2024-04-25T00:15:38.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.566+0000 7f27c8a00640 1 -- 172.21.15.83:0/149020772 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b80ab9d0 con 0x7f27cc15f350 2024-04-25T00:15:38.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.566+0000 7f27d0e00640 1 -- 172.21.15.83:0/149020772 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc15f350 msgr2=0x7f27cc06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.566+0000 7f27d0e00640 1 --2- 172.21.15.83:0/149020772 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc15f350 0x7f27cc06e710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f27b80097c0 tx=0x7f27b801c4a0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.566+0000 7f27d0e00640 1 -- 172.21.15.83:0/149020772 shutdown_connections 2024-04-25T00:15:38.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.566+0000 7f27d0e00640 1 --2- 172.21.15.83:0/149020772 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc15f350 0x7f27cc06e710 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.566+0000 7f27d0e00640 1 --2- 172.21.15.83:0/149020772 >> v2:172.21.15.86:3300/0 conn(0x7f27cc1446e0 0x7f27cc0cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.566+0000 7f27d0e00640 1 -- 172.21.15.83:0/149020772 >> 172.21.15.83:0/149020772 conn(0x7f27cc0cfa60 msgr2=0x7f27cc0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:38.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.566+0000 7f27d0e00640 1 -- 172.21.15.83:0/149020772 shutdown_connections 2024-04-25T00:15:38.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.566+0000 7f27d0e00640 1 -- 172.21.15.83:0/149020772 wait complete. 2024-04-25T00:15:38.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27d0e00640 1 Processor -- start 2024-04-25T00:15:38.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27d0e00640 1 -- start start 2024-04-25T00:15:38.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27d0e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc06ec50 0x7f27cc147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27d0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27cc1446e0 0x7f27cc0c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27d0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27cc15f350 0x7f27cc13d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27d0e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc167a90 con 0x7f27cc1446e0 2024-04-25T00:15:38.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27d0e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc0cd630 con 0x7f27cc15f350 2024-04-25T00:15:38.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27d0e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f27cc167c10 con 0x7f27cc06ec50 2024-04-25T00:15:38.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27c9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27cc1446e0 0x7f27cc0c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27c9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27cc1446e0 0x7f27cc0c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60192/0 (socket says 172.21.15.83:60192) 2024-04-25T00:15:38.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27c9400640 1 -- 172.21.15.83:0/473656735 learned_addr learned my addr 172.21.15.83:0/473656735 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:38.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27c9e00640 1 --2- 172.21.15.83:0/473656735 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc06ec50 0x7f27cc147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27ca800640 1 --2- 172.21.15.83:0/473656735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27cc15f350 0x7f27cc13d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27c9400640 1 -- 172.21.15.83:0/473656735 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc06ec50 msgr2=0x7f27cc147f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27c9400640 1 --2- 172.21.15.83:0/473656735 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc06ec50 0x7f27cc147f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27c9400640 1 -- 172.21.15.83:0/473656735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27cc15f350 msgr2=0x7f27cc13d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27c9400640 1 --2- 172.21.15.83:0/473656735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27cc15f350 0x7f27cc13d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27c9400640 1 -- 172.21.15.83:0/473656735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27cc15a6e0 con 0x7f27cc1446e0 2024-04-25T00:15:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27c9400640 1 --2- 172.21.15.83:0/473656735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27cc1446e0 0x7f27cc0c49a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f27b80ae040 tx=0x7f27b801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27c2000640 1 -- 172.21.15.83:0/473656735 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b80b0050 con 0x7f27cc1446e0 2024-04-25T00:15:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27c2000640 1 -- 172.21.15.83:0/473656735 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b80bc070 con 0x7f27cc1446e0 2024-04-25T00:15:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27d0e00640 1 -- 172.21.15.83:0/473656735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27cc0b2b20 con 0x7f27cc1446e0 2024-04-25T00:15:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27c2000640 1 -- 172.21.15.83:0/473656735 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b80b5830 con 0x7f27cc1446e0 2024-04-25T00:15:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27d0e00640 1 -- 172.21.15.83:0/473656735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27cc0b30e0 con 0x7f27cc1446e0 2024-04-25T00:15:38.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.570+0000 7f27d0e00640 1 -- 172.21.15.83:0/473656735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27cc068230 con 0x7f27cc1446e0 2024-04-25T00:15:38.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.574+0000 7f27c2000640 1 -- 172.21.15.83:0/473656735 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f27b80b59d0 con 0x7f27cc1446e0 2024-04-25T00:15:38.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.574+0000 7f27c2000640 1 --2- 172.21.15.83:0/473656735 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27ac0678c0 0x7f27ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.574+0000 7f27c2000640 1 -- 172.21.15.83:0/473656735 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f27b8120510 con 0x7f27cc1446e0 2024-04-25T00:15:38.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.574+0000 7f27c9e00640 1 --2- 172.21.15.83:0/473656735 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27ac0678c0 0x7f27ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.574+0000 7f27c9e00640 1 --2- 172.21.15.83:0/473656735 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27ac0678c0 0x7f27ac069d80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009af0 tx=0x7f27bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:38.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.578+0000 7f27c2000640 1 -- 172.21.15.83:0/473656735 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f27b8016090 con 0x7f27cc1446e0 2024-04-25T00:15:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.730+0000 7f27d0e00640 1 -- 172.21.15.83:0/473656735 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f27cc06db30 con 0x7f27ac0678c0 2024-04-25T00:15:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.730+0000 7f27c2000640 1 -- 172.21.15.83:0/473656735 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f27cc06db30 con 0x7f27ac0678c0 2024-04-25T00:15:38.733 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T00:15:38.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.734+0000 7f27d0e00640 1 -- 172.21.15.83:0/473656735 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27ac0678c0 msgr2=0x7f27ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.734+0000 7f27d0e00640 1 --2- 172.21.15.83:0/473656735 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27ac0678c0 0x7f27ac069d80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009af0 tx=0x7f27bc008040 comp rx=0 tx=0).stop 2024-04-25T00:15:38.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.734+0000 7f27d0e00640 1 -- 172.21.15.83:0/473656735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27cc1446e0 msgr2=0x7f27cc0c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.734+0000 7f27d0e00640 1 --2- 172.21.15.83:0/473656735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27cc1446e0 0x7f27cc0c49a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f27b80ae040 tx=0x7f27b801ace0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.734+0000 7f27d0e00640 1 -- 172.21.15.83:0/473656735 shutdown_connections 2024-04-25T00:15:38.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.734+0000 7f27d0e00640 1 --2- 172.21.15.83:0/473656735 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f27ac0678c0 0x7f27ac069d80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.734+0000 7f27d0e00640 1 --2- 172.21.15.83:0/473656735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27cc15f350 0x7f27cc13d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.734+0000 7f27d0e00640 1 --2- 172.21.15.83:0/473656735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27cc1446e0 0x7f27cc0c49a0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.734+0000 7f27d0e00640 1 --2- 172.21.15.83:0/473656735 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc06ec50 0x7f27cc147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.734+0000 7f27d0e00640 1 -- 172.21.15.83:0/473656735 >> 172.21.15.83:0/473656735 conn(0x7f27cc0cfa60 msgr2=0x7f27cc06e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:38.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.734+0000 7f27d0e00640 1 -- 172.21.15.83:0/473656735 shutdown_connections 2024-04-25T00:15:38.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.734+0000 7f27d0e00640 1 -- 172.21.15.83:0/473656735 wait complete. 2024-04-25T00:15:38.753 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T00:15:38.753 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:15:38.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.890+0000 7f2a86200640 1 Processor -- start 2024-04-25T00:15:38.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.890+0000 7f2a86200640 1 -- start start 2024-04-25T00:15:38.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.890+0000 7f2a86200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a80144470 0x7f2a8015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.890+0000 7f2a86200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2a800d01e0 0x7f2a800ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.890+0000 7f2a86200640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a800b3f40 con 0x7f2a800b34f0 2024-04-25T00:15:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a86200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2a80144850 con 0x7f2a800d01e0 2024-04-25T00:15:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a86200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2a800b4220 con 0x7f2a80144470 2024-04-25T00:15:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a85800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2a800d01e0 0x7f2a800ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a7fe00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2a800b34f0 0x7f2a800cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35850/0 (socket says 172.21.15.83:35850) 2024-04-25T00:15:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a7fe00640 1 -- 172.21.15.83:0/150118105 learned_addr learned my addr 172.21.15.83:0/150118105 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a84e00640 1 --2- 172.21.15.83:0/150118105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a80144470 0x7f2a8015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a85800640 1 -- 172.21.15.83:0/150118105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a80144470 msgr2=0x7f2a8015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a85800640 1 --2- 172.21.15.83:0/150118105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a80144470 0x7f2a8015f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a85800640 1 -- 172.21.15.83:0/150118105 >> v1:172.21.15.83:6789/0 conn(0x7f2a800b34f0 legacy=0x7f2a800cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a85800640 1 -- 172.21.15.83:0/150118105 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8015e5e0 con 0x7f2a800d01e0 2024-04-25T00:15:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a7f400640 1 -- 172.21.15.83:0/150118105 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 437657711 0 0) 0x7f2a800b3f40 con 0x7f2a800b34f0 2024-04-25T00:15:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a85800640 1 --2- 172.21.15.83:0/150118105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a800d01e0 0x7f2a800ccbd0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f2a780097c0 tx=0x7f2a7801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1fef06368d8816 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a7f400640 1 -- 172.21.15.83:0/150118105 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a780b2070 con 0x7f2a800d01e0 2024-04-25T00:15:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a7f400640 1 -- 172.21.15.83:0/150118105 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a780ad680 con 0x7f2a800d01e0 2024-04-25T00:15:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.894+0000 7f2a7f400640 1 -- 172.21.15.83:0/150118105 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a780ada50 con 0x7f2a800d01e0 2024-04-25T00:15:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 -- 172.21.15.83:0/150118105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a800d01e0 msgr2=0x7f2a800ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 --2- 172.21.15.83:0/150118105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a800d01e0 0x7f2a800ccbd0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f2a780097c0 tx=0x7f2a7801c470 comp rx=0 tx=0).stop 2024-04-25T00:15:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 -- 172.21.15.83:0/150118105 shutdown_connections 2024-04-25T00:15:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 --2- 172.21.15.83:0/150118105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a800d01e0 0x7f2a800ccbd0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 --2- 172.21.15.83:0/150118105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a80144470 0x7f2a8015f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 -- 172.21.15.83:0/150118105 >> 172.21.15.83:0/150118105 conn(0x7f2a8006b3b0 msgr2=0x7f2a8006b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 -- 172.21.15.83:0/150118105 shutdown_connections 2024-04-25T00:15:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 -- 172.21.15.83:0/150118105 wait complete. 2024-04-25T00:15:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 Processor -- start 2024-04-25T00:15:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 -- start start 2024-04-25T00:15:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a800b34f0 0x7f2a80140280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a800d01e0 0x7f2a8013e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a80144470 0x7f2a80159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a80167860 con 0x7f2a80144470 2024-04-25T00:15:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a801676e0 con 0x7f2a800b34f0 2024-04-25T00:15:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a86200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2a801679e0 con 0x7f2a800d01e0 2024-04-25T00:15:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a85800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a80144470 0x7f2a80159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a85800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a80144470 0x7f2a80159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60216/0 (socket says 172.21.15.83:60216) 2024-04-25T00:15:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a85800640 1 -- 172.21.15.83:0/3306651535 learned_addr learned my addr 172.21.15.83:0/3306651535 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a7fe00640 1 --2- 172.21.15.83:0/3306651535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a800d01e0 0x7f2a8013e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a84e00640 1 --2- 172.21.15.83:0/3306651535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a800b34f0 0x7f2a80140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a85800640 1 -- 172.21.15.83:0/3306651535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a800d01e0 msgr2=0x7f2a8013e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a85800640 1 --2- 172.21.15.83:0/3306651535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a800d01e0 0x7f2a8013e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a85800640 1 -- 172.21.15.83:0/3306651535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a800b34f0 msgr2=0x7f2a80140280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a85800640 1 --2- 172.21.15.83:0/3306651535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a800b34f0 0x7f2a80140280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a85800640 1 -- 172.21.15.83:0/3306651535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8015a7a0 con 0x7f2a80144470 2024-04-25T00:15:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.898+0000 7f2a85800640 1 --2- 172.21.15.83:0/3306651535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a80144470 0x7f2a80159d60 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f2a7801a0a0 tx=0x7f2a7801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a7cc00640 1 -- 172.21.15.83:0/3306651535 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a780b2040 con 0x7f2a80144470 2024-04-25T00:15:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a7cc00640 1 -- 172.21.15.83:0/3306651535 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a780bd070 con 0x7f2a80144470 2024-04-25T00:15:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a7cc00640 1 -- 172.21.15.83:0/3306651535 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a780ade10 con 0x7f2a80144470 2024-04-25T00:15:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a86200640 1 -- 172.21.15.83:0/3306651535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a801413a0 con 0x7f2a80144470 2024-04-25T00:15:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a86200640 1 -- 172.21.15.83:0/3306651535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a801431f0 con 0x7f2a80144470 2024-04-25T00:15:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a86200640 1 -- 172.21.15.83:0/3306651535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2a8015a2a0 con 0x7f2a80144470 2024-04-25T00:15:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a7cc00640 1 -- 172.21.15.83:0/3306651535 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2a780c2460 con 0x7f2a80144470 2024-04-25T00:15:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a7cc00640 1 --2- 172.21.15.83:0/3306651535 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2a600678c0 0x7f2a60069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a7cc00640 1 -- 172.21.15.83:0/3306651535 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f2a78120b80 con 0x7f2a80144470 2024-04-25T00:15:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a84e00640 1 --2- 172.21.15.83:0/3306651535 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2a600678c0 0x7f2a60069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a7cc00640 1 --2- 172.21.15.83:0/3306651535 >> [v2:172.21.15.83:6808/228430303,v1:172.21.15.83:6810/228430303] conn(0x7f2a6006df40 0x7f2a60070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a7cc00640 1 -- 172.21.15.83:0/3306651535 --> [v2:172.21.15.83:6808/228430303,v1:172.21.15.83:6810/228430303] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2a780a9ec0 con 0x7f2a6006df40 2024-04-25T00:15:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a7cc00640 1 -- 172.21.15.83:0/3306651535 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f2a78120f30 con 0x7f2a80144470 2024-04-25T00:15:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a7fe00640 1 --2- 172.21.15.83:0/3306651535 >> [v2:172.21.15.83:6808/228430303,v1:172.21.15.83:6810/228430303] conn(0x7f2a6006df40 0x7f2a60070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.902+0000 7f2a84e00640 1 --2- 172.21.15.83:0/3306651535 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2a600678c0 0x7f2a60069d80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2a70003fd0 tx=0x7f2a700039a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.906+0000 7f2a7fe00640 1 --2- 172.21.15.83:0/3306651535 >> [v2:172.21.15.83:6808/228430303,v1:172.21.15.83:6810/228430303] conn(0x7f2a6006df40 0x7f2a60070380 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.906+0000 7f2a7cc00640 1 -- 172.21.15.83:0/3306651535 <== osd.1 v2:172.21.15.83:6808/228430303 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2a780a9ec0 con 0x7f2a6006df40 2024-04-25T00:15:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.918+0000 7f2a86200640 1 -- 172.21.15.83:0/3306651535 --> [v2:172.21.15.83:6808/228430303,v1:172.21.15.83:6810/228430303] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2a80141850 con 0x7f2a6006df40 2024-04-25T00:15:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.918+0000 7f2a7cc00640 1 -- 172.21.15.83:0/3306651535 <== osd.1 v2:172.21.15.83:6808/228430303 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2a80141850 con 0x7f2a6006df40 2024-04-25T00:15:38.919 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:15:38.919 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:15:38.919 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:15:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.918+0000 7f2a86200640 1 -- 172.21.15.83:0/3306651535 >> [v2:172.21.15.83:6808/228430303,v1:172.21.15.83:6810/228430303] conn(0x7f2a6006df40 msgr2=0x7f2a60070380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.918+0000 7f2a86200640 1 --2- 172.21.15.83:0/3306651535 >> [v2:172.21.15.83:6808/228430303,v1:172.21.15.83:6810/228430303] conn(0x7f2a6006df40 0x7f2a60070380 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.918+0000 7f2a86200640 1 -- 172.21.15.83:0/3306651535 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2a600678c0 msgr2=0x7f2a60069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.918+0000 7f2a86200640 1 --2- 172.21.15.83:0/3306651535 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f2a600678c0 0x7f2a60069d80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2a70003fd0 tx=0x7f2a700039a0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.918+0000 7f2a86200640 1 -- 172.21.15.83:0/3306651535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a80144470 msgr2=0x7f2a80159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.918+0000 7f2a86200640 1 --2- 172.21.15.83:0/3306651535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a80144470 0x7f2a80159d60 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f2a7801a0a0 tx=0x7f2a7801acb0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.918+0000 7f2a85800640 1 -- 172.21.15.83:0/3306651535 reap_dead start 2024-04-25T00:15:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.918+0000 7f2a86200640 1 -- 172.21.15.83:0/3306651535 shutdown_connections 2024-04-25T00:15:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.918+0000 7f2a86200640 1 -- 172.21.15.83:0/3306651535 >> 172.21.15.83:0/3306651535 conn(0x7f2a8006b3b0 msgr2=0x7f2a8006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.918+0000 7f2a86200640 1 -- 172.21.15.83:0/3306651535 shutdown_connections 2024-04-25T00:15:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:38.918+0000 7f2a86200640 1 -- 172.21.15.83:0/3306651535 wait complete. 2024-04-25T00:15:38.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T00:15:39.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.074+0000 7ff5c1600640 1 Processor -- start 2024-04-25T00:15:39.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.074+0000 7ff5c1600640 1 -- start start 2024-04-25T00:15:39.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.074+0000 7ff5c1600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5bc0cab50 0x7ff5bc15f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.074+0000 7ff5c1600640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5bc0cafa0 con 0x7ff5bc06ee20 2024-04-25T00:15:39.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.074+0000 7ff5c1600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5bc0cb210 con 0x7ff5bc1446e0 2024-04-25T00:15:39.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.074+0000 7ff5c1600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff5bc0715b0 con 0x7ff5bc0cab50 2024-04-25T00:15:39.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.074+0000 7ff5b2000640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff5bc06ee20 0x7ff5bc071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35856/0 (socket says 172.21.15.83:35856) 2024-04-25T00:15:39.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.074+0000 7ff5b2000640 1 -- 172.21.15.83:0/2155440642 learned_addr learned my addr 172.21.15.83:0/2155440642 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:39.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5bb400640 1 --2- 172.21.15.83:0/2155440642 >> v2:172.21.15.86:3300/0 conn(0x7ff5bc0cab50 0x7ff5bc15f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 -- 172.21.15.83:0/2155440642 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1876480301 0 0) 0x7ff5bc0cafa0 con 0x7ff5bc06ee20 2024-04-25T00:15:39.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 -- 172.21.15.83:0/2155440642 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5a0005740 con 0x7ff5bc06ee20 2024-04-25T00:15:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 -- 172.21.15.83:0/2155440642 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4246205286 0 0) 0x7ff5a0005740 con 0x7ff5bc06ee20 2024-04-25T00:15:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 -- 172.21.15.83:0/2155440642 >> v2:172.21.15.86:3300/0 conn(0x7ff5bc0cab50 msgr2=0x7ff5bc15f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 --2- 172.21.15.83:0/2155440642 >> v2:172.21.15.86:3300/0 conn(0x7ff5bc0cab50 0x7ff5bc15f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 -- 172.21.15.83:0/2155440642 >> v1:172.21.15.86:6789/0 conn(0x7ff5bc1446e0 legacy=0x7ff5bc06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 -- 172.21.15.83:0/2155440642 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc0cb4f0 con 0x7ff5bc06ee20 2024-04-25T00:15:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 -- 172.21.15.83:0/2155440642 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7ff5a4001680 con 0x7ff5bc06ee20 2024-04-25T00:15:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5bb400640 1 --2- 172.21.15.83:0/2155440642 >> v2:172.21.15.86:3300/0 conn(0x7ff5bc0cab50 0x7ff5bc15f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 -- 172.21.15.83:0/2155440642 >> v1:172.21.15.83:6789/0 conn(0x7ff5bc06ee20 legacy=0x7ff5bc071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 --2- 172.21.15.83:0/2155440642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a0005bd0 0x7ff5a0005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 --2- 172.21.15.83:0/2155440642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5a00065d0 0x7ff5a0006aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 --2- 172.21.15.83:0/2155440642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a00072a0 0x7ff5a000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 -- 172.21.15.83:0/2155440642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a000e1d0 con 0x7ff5a0005bd0 2024-04-25T00:15:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 -- 172.21.15.83:0/2155440642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a000e380 con 0x7ff5a00072a0 2024-04-25T00:15:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5ba000640 1 -- 172.21.15.83:0/2155440642 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a000e580 con 0x7ff5a00065d0 2024-04-25T00:15:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5baa00640 1 --2- 172.21.15.83:0/2155440642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a0005bd0 0x7ff5a0005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5b2000640 1 --2- 172.21.15.83:0/2155440642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5a00065d0 0x7ff5a0006aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5bb400640 1 --2- 172.21.15.83:0/2155440642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a00072a0 0x7ff5a000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5baa00640 1 -- 172.21.15.83:0/2155440642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5a00065d0 msgr2=0x7ff5a0006aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:39.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5baa00640 1 --2- 172.21.15.83:0/2155440642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5a00065d0 0x7ff5a0006aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5baa00640 1 -- 172.21.15.83:0/2155440642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a00072a0 msgr2=0x7ff5a000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:39.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5baa00640 1 --2- 172.21.15.83:0/2155440642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a00072a0 0x7ff5a000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5baa00640 1 -- 172.21.15.83:0/2155440642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a0005740 con 0x7ff5a0005bd0 2024-04-25T00:15:39.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5bb400640 1 --2- 172.21.15.83:0/2155440642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a00072a0 0x7ff5a000dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:39.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.078+0000 7ff5bb400640 1 -- 172.21.15.83:0/2155440642 reap_dead start 2024-04-25T00:15:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5baa00640 1 --2- 172.21.15.83:0/2155440642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a0005bd0 0x7ff5a0005fb0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac001f60 tx=0x7ff5ac01c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b826efe12805ce1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5ba000640 1 -- 172.21.15.83:0/2155440642 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac001b60 con 0x7ff5a0005bd0 2024-04-25T00:15:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5ba000640 1 -- 172.21.15.83:0/2155440642 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ac0a9660 con 0x7ff5a0005bd0 2024-04-25T00:15:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5ba000640 1 -- 172.21.15.83:0/2155440642 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0a9a90 con 0x7ff5a0005bd0 2024-04-25T00:15:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 -- 172.21.15.83:0/2155440642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a0005bd0 msgr2=0x7ff5a0005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 --2- 172.21.15.83:0/2155440642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a0005bd0 0x7ff5a0005fb0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac001f60 tx=0x7ff5ac01c840 comp rx=0 tx=0).stop 2024-04-25T00:15:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 -- 172.21.15.83:0/2155440642 shutdown_connections 2024-04-25T00:15:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 --2- 172.21.15.83:0/2155440642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a0005bd0 0x7ff5a0005fb0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 -- 172.21.15.83:0/2155440642 >> 172.21.15.83:0/2155440642 conn(0x7ff5bc0cfa60 msgr2=0x7ff5bc0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 -- 172.21.15.83:0/2155440642 shutdown_connections 2024-04-25T00:15:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 -- 172.21.15.83:0/2155440642 wait complete. 2024-04-25T00:15:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 Processor -- start 2024-04-25T00:15:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 -- start start 2024-04-25T00:15:39.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a0005bd0 0x7ff5bc15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc1401c0 0x7ff5bc143030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc1e6770 0x7ff5bc1e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc15bd10 con 0x7ff5a0005bd0 2024-04-25T00:15:39.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc15bb90 con 0x7ff5bc1401c0 2024-04-25T00:15:39.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.082+0000 7ff5c1600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff5bc15be90 con 0x7ff5bc1e6770 2024-04-25T00:15:39.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5baa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a0005bd0 0x7ff5bc15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5baa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a0005bd0 0x7ff5bc15c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60240/0 (socket says 172.21.15.83:60240) 2024-04-25T00:15:39.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5baa00640 1 -- 172.21.15.83:0/509055289 learned_addr learned my addr 172.21.15.83:0/509055289 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:39.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5bb400640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc1e6770 0x7ff5bc1e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5b2000640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc1401c0 0x7ff5bc143030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5baa00640 1 -- 172.21.15.83:0/509055289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc1e6770 msgr2=0x7ff5bc1e6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5baa00640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc1e6770 0x7ff5bc1e6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5baa00640 1 -- 172.21.15.83:0/509055289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc1401c0 msgr2=0x7ff5bc143030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5baa00640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc1401c0 0x7ff5bc143030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5baa00640 1 -- 172.21.15.83:0/509055289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc140640 con 0x7ff5a0005bd0 2024-04-25T00:15:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5b2000640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc1401c0 0x7ff5bc143030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5baa00640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a0005bd0 0x7ff5bc15c220 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac01c670 tx=0x7ff5a0007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5b3400640 1 -- 172.21.15.83:0/509055289 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0a9a10 con 0x7ff5a0005bd0 2024-04-25T00:15:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5b3400640 1 -- 172.21.15.83:0/509055289 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ac0a9bb0 con 0x7ff5a0005bd0 2024-04-25T00:15:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5b3400640 1 -- 172.21.15.83:0/509055289 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac007040 con 0x7ff5a0005bd0 2024-04-25T00:15:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5c1600640 1 -- 172.21.15.83:0/509055289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5bc15c920 con 0x7ff5a0005bd0 2024-04-25T00:15:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5c1600640 1 -- 172.21.15.83:0/509055289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5bc15cee0 con 0x7ff5a0005bd0 2024-04-25T00:15:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5c1600640 1 -- 172.21.15.83:0/509055289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5bc068230 con 0x7ff5a0005bd0 2024-04-25T00:15:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.086+0000 7ff5b3400640 1 -- 172.21.15.83:0/509055289 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff5ac0071e0 con 0x7ff5a0005bd0 2024-04-25T00:15:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.090+0000 7ff5b3400640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5980678f0 0x7ff598069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.090+0000 7ff5b3400640 1 -- 172.21.15.83:0/509055289 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7ff5ac11ff70 con 0x7ff5a0005bd0 2024-04-25T00:15:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.090+0000 7ff5b2000640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5980678f0 0x7ff598069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.090+0000 7ff5b2000640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5980678f0 0x7ff598069db0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc15adc0 tx=0x7ff5a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:39.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.094+0000 7ff5b3400640 1 -- 172.21.15.83:0/509055289 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff5ac0ad020 con 0x7ff5a0005bd0 2024-04-25T00:15:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.246+0000 7ff5c1600640 1 -- 172.21.15.83:0/509055289 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7ff5bc15d7c0 con 0x7ff5980678f0 2024-04-25T00:15:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.246+0000 7ff5b3400640 1 -- 172.21.15.83:0/509055289 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff5bc15d7c0 con 0x7ff5980678f0 2024-04-25T00:15:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T00:15:39.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.250+0000 7ff5c1600640 1 -- 172.21.15.83:0/509055289 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5980678f0 msgr2=0x7ff598069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.250+0000 7ff5c1600640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5980678f0 0x7ff598069db0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc15adc0 tx=0x7ff5a4009040 comp rx=0 tx=0).stop 2024-04-25T00:15:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.250+0000 7ff5c1600640 1 -- 172.21.15.83:0/509055289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a0005bd0 msgr2=0x7ff5bc15c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.250+0000 7ff5c1600640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a0005bd0 0x7ff5bc15c220 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac01c670 tx=0x7ff5a0007a50 comp rx=0 tx=0).stop 2024-04-25T00:15:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.250+0000 7ff5c1600640 1 -- 172.21.15.83:0/509055289 shutdown_connections 2024-04-25T00:15:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.250+0000 7ff5c1600640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7ff5980678f0 0x7ff598069db0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.250+0000 7ff5c1600640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc1e6770 0x7ff5bc1e6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.250+0000 7ff5c1600640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc1401c0 0x7ff5bc143030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.250+0000 7ff5c1600640 1 --2- 172.21.15.83:0/509055289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5a0005bd0 0x7ff5bc15c220 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.250+0000 7ff5c1600640 1 -- 172.21.15.83:0/509055289 >> 172.21.15.83:0/509055289 conn(0x7ff5bc0cfa60 msgr2=0x7ff5bc1e7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.250+0000 7ff5c1600640 1 -- 172.21.15.83:0/509055289 shutdown_connections 2024-04-25T00:15:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.250+0000 7ff5c1600640 1 -- 172.21.15.83:0/509055289 wait complete. 2024-04-25T00:15:39.270 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T00:15:39.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:15:39.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c1d800640 1 Processor -- start 2024-04-25T00:15:39.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c1d800640 1 -- start start 2024-04-25T00:15:39.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c1d800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c1815de50 0x7f9c180b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c1d800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c180d0450 0x7f9c180b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c1d800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c180b2f10 con 0x7f9c180b44c0 2024-04-25T00:15:39.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c1d800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9c180b2ce0 con 0x7f9c1815de50 2024-04-25T00:15:39.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c1d800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9c180b4f40 con 0x7f9c180d0450 2024-04-25T00:15:39.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c16a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c1815de50 0x7f9c180b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c16a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c1815de50 0x7f9c180b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60250/0 (socket says 172.21.15.83:60250) 2024-04-25T00:15:39.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c16a00640 1 -- 172.21.15.83:0/863822351 learned_addr learned my addr 172.21.15.83:0/863822351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:39.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c17400640 1 --2- 172.21.15.83:0/863822351 >> v2:172.21.15.86:3300/0 conn(0x7f9c180d0450 0x7f9c180b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c16a00640 1 -- 172.21.15.83:0/863822351 >> v2:172.21.15.86:3300/0 conn(0x7f9c180d0450 msgr2=0x7f9c180b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:39.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c16a00640 1 --2- 172.21.15.83:0/863822351 >> v2:172.21.15.86:3300/0 conn(0x7f9c180d0450 0x7f9c180b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c16a00640 1 -- 172.21.15.83:0/863822351 >> v1:172.21.15.86:6789/0 conn(0x7f9c180b44c0 legacy=0x7f9c180cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:39.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c16a00640 1 -- 172.21.15.83:0/863822351 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c180b31f0 con 0x7f9c1815de50 2024-04-25T00:15:39.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.410+0000 7f9c17400640 1 --2- 172.21.15.83:0/863822351 >> v2:172.21.15.86:3300/0 conn(0x7f9c180d0450 0x7f9c180b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:39.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.414+0000 7f9c16a00640 1 --2- 172.21.15.83:0/863822351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c1815de50 0x7f9c180b3cd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9c1806e700 tx=0x7f9c0801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aefc4e69b129410 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:39.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.414+0000 7f9c15600640 1 -- 172.21.15.83:0/863822351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c080b1070 con 0x7f9c1815de50 2024-04-25T00:15:39.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.414+0000 7f9c15600640 1 -- 172.21.15.83:0/863822351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c080ac630 con 0x7f9c1815de50 2024-04-25T00:15:39.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.414+0000 7f9c15600640 1 -- 172.21.15.83:0/863822351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c080a8050 con 0x7f9c1815de50 2024-04-25T00:15:39.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.414+0000 7f9c1d800640 1 -- 172.21.15.83:0/863822351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c1815de50 msgr2=0x7f9c180b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.414+0000 7f9c1d800640 1 --2- 172.21.15.83:0/863822351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c1815de50 0x7f9c180b3cd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9c1806e700 tx=0x7f9c0801c4e0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.414+0000 7f9c1d800640 1 -- 172.21.15.83:0/863822351 shutdown_connections 2024-04-25T00:15:39.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.414+0000 7f9c1d800640 1 --2- 172.21.15.83:0/863822351 >> v2:172.21.15.86:3300/0 conn(0x7f9c180d0450 0x7f9c180b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.414+0000 7f9c1d800640 1 --2- 172.21.15.83:0/863822351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c1815de50 0x7f9c180b3cd0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.414+0000 7f9c1d800640 1 -- 172.21.15.83:0/863822351 >> 172.21.15.83:0/863822351 conn(0x7f9c1806b3b0 msgr2=0x7f9c1806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.414+0000 7f9c1d800640 1 -- 172.21.15.83:0/863822351 shutdown_connections 2024-04-25T00:15:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.414+0000 7f9c1d800640 1 -- 172.21.15.83:0/863822351 wait complete. 2024-04-25T00:15:39.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.414+0000 7f9c1d800640 1 Processor -- start 2024-04-25T00:15:39.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.414+0000 7f9c1d800640 1 -- start start 2024-04-25T00:15:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c1d800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c180b44c0 0x7f9c18152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c1d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c180d0450 0x7f9c1814dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c1d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1815de50 0x7f9c1814e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c1d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1806dd00 con 0x7f9c180d0450 2024-04-25T00:15:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c1d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1806db80 con 0x7f9c1815de50 2024-04-25T00:15:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c1d800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9c18071f40 con 0x7f9c180b44c0 2024-04-25T00:15:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c16000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c180d0450 0x7f9c1814dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c16000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c180d0450 0x7f9c1814dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60258/0 (socket says 172.21.15.83:60258) 2024-04-25T00:15:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c16000640 1 -- 172.21.15.83:0/99166899 learned_addr learned my addr 172.21.15.83:0/99166899 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c16a00640 1 --2- 172.21.15.83:0/99166899 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c180b44c0 0x7f9c18152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c17400640 1 --2- 172.21.15.83:0/99166899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1815de50 0x7f9c1814e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c16000640 1 -- 172.21.15.83:0/99166899 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c180b44c0 msgr2=0x7f9c18152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c16000640 1 --2- 172.21.15.83:0/99166899 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9c180b44c0 0x7f9c18152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c16000640 1 -- 172.21.15.83:0/99166899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1815de50 msgr2=0x7f9c1814e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c16000640 1 --2- 172.21.15.83:0/99166899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1815de50 0x7f9c1814e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c16000640 1 -- 172.21.15.83:0/99166899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1814ee30 con 0x7f9c180d0450 2024-04-25T00:15:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c16000640 1 --2- 172.21.15.83:0/99166899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c180d0450 0x7f9c1814dce0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400c480 tx=0x7f9c0400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c0ea00640 1 -- 172.21.15.83:0/99166899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04012830 con 0x7f9c180d0450 2024-04-25T00:15:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c0ea00640 1 -- 172.21.15.83:0/99166899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c04017070 con 0x7f9c180d0450 2024-04-25T00:15:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c0ea00640 1 -- 172.21.15.83:0/99166899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04012c30 con 0x7f9c180d0450 2024-04-25T00:15:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c1d800640 1 -- 172.21.15.83:0/99166899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1814f110 con 0x7f9c180d0450 2024-04-25T00:15:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c1d800640 1 -- 172.21.15.83:0/99166899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c18158d80 con 0x7f9c180d0450 2024-04-25T00:15:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.418+0000 7f9c1d800640 1 -- 172.21.15.83:0/99166899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c180736f0 con 0x7f9c180d0450 2024-04-25T00:15:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.422+0000 7f9c0ea00640 1 -- 172.21.15.83:0/99166899 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9c0401c460 con 0x7f9c180d0450 2024-04-25T00:15:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.422+0000 7f9c0ea00640 1 --2- 172.21.15.83:0/99166899 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9bf80678c0 0x7f9bf8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.422+0000 7f9c0ea00640 1 -- 172.21.15.83:0/99166899 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f9c04086380 con 0x7f9c180d0450 2024-04-25T00:15:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.422+0000 7f9c16a00640 1 --2- 172.21.15.83:0/99166899 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9bf80678c0 0x7f9bf8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.422+0000 7f9c0ea00640 1 --2- 172.21.15.83:0/99166899 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x7f9bf806df40 0x7f9bf8070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.422+0000 7f9c0ea00640 1 -- 172.21.15.83:0/99166899 --> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9bf8070a50 con 0x7f9bf806df40 2024-04-25T00:15:39.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.422+0000 7f9c0ea00640 1 -- 172.21.15.83:0/99166899 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f9c04086730 con 0x7f9c180d0450 2024-04-25T00:15:39.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.422+0000 7f9c17400640 1 --2- 172.21.15.83:0/99166899 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x7f9bf806df40 0x7f9bf8070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.422+0000 7f9c16a00640 1 --2- 172.21.15.83:0/99166899 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9bf80678c0 0x7f9bf8069d80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9c080ae000 tx=0x7f9c08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:39.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.422+0000 7f9c17400640 1 --2- 172.21.15.83:0/99166899 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x7f9bf806df40 0x7f9bf8070380 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:39.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.422+0000 7f9c0ea00640 1 -- 172.21.15.83:0/99166899 <== osd.2 v2:172.21.15.83:6824/1702952090 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9bf8070a50 con 0x7f9bf806df40 2024-04-25T00:15:39.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.434+0000 7f9c1d800640 1 -- 172.21.15.83:0/99166899 --> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9c1804f5f0 con 0x7f9bf806df40 2024-04-25T00:15:39.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.434+0000 7f9c0ea00640 1 -- 172.21.15.83:0/99166899 <== osd.2 v2:172.21.15.83:6824/1702952090 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9c1804f5f0 con 0x7f9bf806df40 2024-04-25T00:15:39.437 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:15:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:15:39.437 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:15:39.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.438+0000 7f9c1d800640 1 -- 172.21.15.83:0/99166899 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x7f9bf806df40 msgr2=0x7f9bf8070380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.438+0000 7f9c1d800640 1 --2- 172.21.15.83:0/99166899 >> [v2:172.21.15.83:6824/1702952090,v1:172.21.15.83:6825/1702952090] conn(0x7f9bf806df40 0x7f9bf8070380 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.438+0000 7f9c1d800640 1 -- 172.21.15.83:0/99166899 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9bf80678c0 msgr2=0x7f9bf8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.438+0000 7f9c1d800640 1 --2- 172.21.15.83:0/99166899 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9bf80678c0 0x7f9bf8069d80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9c080ae000 tx=0x7f9c08009040 comp rx=0 tx=0).stop 2024-04-25T00:15:39.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.438+0000 7f9c1d800640 1 -- 172.21.15.83:0/99166899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c180d0450 msgr2=0x7f9c1814dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.438+0000 7f9c1d800640 1 --2- 172.21.15.83:0/99166899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c180d0450 0x7f9c1814dce0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400c480 tx=0x7f9c0400c940 comp rx=0 tx=0).stop 2024-04-25T00:15:39.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.438+0000 7f9c17400640 1 -- 172.21.15.83:0/99166899 reap_dead start 2024-04-25T00:15:39.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.438+0000 7f9c1d800640 1 -- 172.21.15.83:0/99166899 shutdown_connections 2024-04-25T00:15:39.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.438+0000 7f9c1d800640 1 -- 172.21.15.83:0/99166899 >> 172.21.15.83:0/99166899 conn(0x7f9c1806b3b0 msgr2=0x7f9c18072300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:39.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.438+0000 7f9c1d800640 1 -- 172.21.15.83:0/99166899 shutdown_connections 2024-04-25T00:15:39.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.438+0000 7f9c1d800640 1 -- 172.21.15.83:0/99166899 wait complete. 2024-04-25T00:15:39.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T00:15:39.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d72800640 1 Processor -- start 2024-04-25T00:15:39.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d72800640 1 -- start start 2024-04-25T00:15:39.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d72800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c15e160 0x7f6d6c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d72800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6d6c15f420 0x7f6d6c0c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d72800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6d6c0ceff0 0x7f6d6c161c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d72800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f6d6c0cb8b0 con 0x7f6d6c15f420 2024-04-25T00:15:39.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d72800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6d6c0ca490 con 0x7f6d6c0ceff0 2024-04-25T00:15:39.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d72800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6d6c15e940 con 0x7f6d6c15e160 2024-04-25T00:15:39.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d63e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6d6c15f420 0x7f6d6c0c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d63e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6d6c15f420 0x7f6d6c0c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60272/0 (socket says 172.21.15.83:60272) 2024-04-25T00:15:39.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d63e00640 1 -- 172.21.15.83:0/3395441300 learned_addr learned my addr 172.21.15.83:0/3395441300 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:39.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d6b400640 1 --2- 172.21.15.83:0/3395441300 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c15e160 0x7f6d6c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d6be00640 1 --2- 172.21.15.83:0/3395441300 >> v2:172.21.15.86:3300/0 conn(0x7f6d6c0ceff0 0x7f6d6c161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d63e00640 1 -- 172.21.15.83:0/3395441300 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c15e160 msgr2=0x7f6d6c0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:39.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d63e00640 1 --2- 172.21.15.83:0/3395441300 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c15e160 0x7f6d6c0c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d63e00640 1 -- 172.21.15.83:0/3395441300 >> v2:172.21.15.86:3300/0 conn(0x7f6d6c0ceff0 msgr2=0x7f6d6c161c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:39.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d63e00640 1 --2- 172.21.15.83:0/3395441300 >> v2:172.21.15.86:3300/0 conn(0x7f6d6c0ceff0 0x7f6d6c161c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.594+0000 7f6d63e00640 1 -- 172.21.15.83:0/3395441300 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6c15ec20 con 0x7f6d6c15f420 2024-04-25T00:15:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.598+0000 7f6d63e00640 1 --2- 172.21.15.83:0/3395441300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c15f420 0x7f6d6c0c92f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0d09c0 tx=0x7f6d5401c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782c24e953a7eb82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.598+0000 7f6d6aa00640 1 -- 172.21.15.83:0/3395441300 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d540b0070 con 0x7f6d6c15f420 2024-04-25T00:15:39.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.598+0000 7f6d6aa00640 1 -- 172.21.15.83:0/3395441300 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d540ab600 con 0x7f6d6c15f420 2024-04-25T00:15:39.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.598+0000 7f6d6aa00640 1 -- 172.21.15.83:0/3395441300 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d540ab9d0 con 0x7f6d6c15f420 2024-04-25T00:15:39.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.598+0000 7f6d72800640 1 -- 172.21.15.83:0/3395441300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c15f420 msgr2=0x7f6d6c0c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.598+0000 7f6d72800640 1 --2- 172.21.15.83:0/3395441300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c15f420 0x7f6d6c0c92f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0d09c0 tx=0x7f6d5401c110 comp rx=0 tx=0).stop 2024-04-25T00:15:39.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.598+0000 7f6d72800640 1 -- 172.21.15.83:0/3395441300 shutdown_connections 2024-04-25T00:15:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.598+0000 7f6d72800640 1 --2- 172.21.15.83:0/3395441300 >> v2:172.21.15.86:3300/0 conn(0x7f6d6c0ceff0 0x7f6d6c161c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.598+0000 7f6d72800640 1 --2- 172.21.15.83:0/3395441300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c15f420 0x7f6d6c0c92f0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.598+0000 7f6d72800640 1 --2- 172.21.15.83:0/3395441300 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c15e160 0x7f6d6c0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.598+0000 7f6d72800640 1 -- 172.21.15.83:0/3395441300 >> 172.21.15.83:0/3395441300 conn(0x7f6d6c0cfa60 msgr2=0x7f6d6c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.598+0000 7f6d72800640 1 -- 172.21.15.83:0/3395441300 shutdown_connections 2024-04-25T00:15:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.598+0000 7f6d72800640 1 -- 172.21.15.83:0/3395441300 wait complete. 2024-04-25T00:15:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.598+0000 7f6d72800640 1 Processor -- start 2024-04-25T00:15:39.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d72800640 1 -- start start 2024-04-25T00:15:39.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d72800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c0ceff0 0x7f6d6c15b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d72800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d6c15e160 0x7f6d6c15d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d72800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c15f420 0x7f6d6c0af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d72800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c0c9da0 con 0x7f6d6c0ceff0 2024-04-25T00:15:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d72800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c0c9c20 con 0x7f6d6c15e160 2024-04-25T00:15:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d72800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6d6c0cd630 con 0x7f6d6c15f420 2024-04-25T00:15:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d6b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c0ceff0 0x7f6d6c15b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d6b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c0ceff0 0x7f6d6c15b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60278/0 (socket says 172.21.15.83:60278) 2024-04-25T00:15:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d6b400640 1 -- 172.21.15.83:0/1002475085 learned_addr learned my addr 172.21.15.83:0/1002475085 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d63e00640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d6c15e160 0x7f6d6c15d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d6be00640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c15f420 0x7f6d6c0af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d6b400640 1 -- 172.21.15.83:0/1002475085 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c15f420 msgr2=0x7f6d6c0af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d6b400640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c15f420 0x7f6d6c0af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d6b400640 1 -- 172.21.15.83:0/1002475085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d6c15e160 msgr2=0x7f6d6c15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d6b400640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d6c15e160 0x7f6d6c15d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d6b400640 1 -- 172.21.15.83:0/1002475085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6c15bf90 con 0x7f6d6c0ceff0 2024-04-25T00:15:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d63e00640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d6c15e160 0x7f6d6c15d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d6b400640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c0ceff0 0x7f6d6c15b760 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00d800 tx=0x7f6d5c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d63400640 1 -- 172.21.15.83:0/1002475085 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00b6c0 con 0x7f6d6c0ceff0 2024-04-25T00:15:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d63400640 1 -- 172.21.15.83:0/1002475085 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d5c019070 con 0x7f6d6c0ceff0 2024-04-25T00:15:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d72800640 1 -- 172.21.15.83:0/1002475085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6c15c270 con 0x7f6d6c0ceff0 2024-04-25T00:15:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d63400640 1 -- 172.21.15.83:0/1002475085 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00ba60 con 0x7f6d6c0ceff0 2024-04-25T00:15:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d72800640 1 -- 172.21.15.83:0/1002475085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6c15c830 con 0x7f6d6c0ceff0 2024-04-25T00:15:39.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.602+0000 7f6d72800640 1 -- 172.21.15.83:0/1002475085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d6c068230 con 0x7f6d6c0ceff0 2024-04-25T00:15:39.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.606+0000 7f6d63400640 1 -- 172.21.15.83:0/1002475085 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6d5c00bc00 con 0x7f6d6c0ceff0 2024-04-25T00:15:39.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.606+0000 7f6d63400640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6d480678f0 0x7f6d48069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.606+0000 7f6d63400640 1 -- 172.21.15.83:0/1002475085 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f6d5c086160 con 0x7f6d6c0ceff0 2024-04-25T00:15:39.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.606+0000 7f6d63e00640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6d480678f0 0x7f6d48069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.606+0000 7f6d63e00640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6d480678f0 0x7f6d48069db0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6d540ad000 tx=0x7f6d54008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.610+0000 7f6d63400640 1 -- 172.21.15.83:0/1002475085 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6d5c056c70 con 0x7f6d6c0ceff0 2024-04-25T00:15:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.762+0000 7f6d72800640 1 -- 172.21.15.83:0/1002475085 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f6d6c06c9f0 con 0x7f6d480678f0 2024-04-25T00:15:39.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.762+0000 7f6d63400640 1 -- 172.21.15.83:0/1002475085 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6d6c06c9f0 con 0x7f6d480678f0 2024-04-25T00:15:39.765 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T00:15:39.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.766+0000 7f6d72800640 1 -- 172.21.15.83:0/1002475085 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6d480678f0 msgr2=0x7f6d48069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.766+0000 7f6d72800640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6d480678f0 0x7f6d48069db0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6d540ad000 tx=0x7f6d54008040 comp rx=0 tx=0).stop 2024-04-25T00:15:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.766+0000 7f6d72800640 1 -- 172.21.15.83:0/1002475085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c0ceff0 msgr2=0x7f6d6c15b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.766+0000 7f6d72800640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c0ceff0 0x7f6d6c15b760 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00d800 tx=0x7f6d5c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.766+0000 7f6d72800640 1 -- 172.21.15.83:0/1002475085 shutdown_connections 2024-04-25T00:15:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.766+0000 7f6d72800640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f6d480678f0 0x7f6d48069db0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.766+0000 7f6d72800640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c15f420 0x7f6d6c0af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.766+0000 7f6d72800640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d6c15e160 0x7f6d6c15d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.766+0000 7f6d72800640 1 --2- 172.21.15.83:0/1002475085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c0ceff0 0x7f6d6c15b760 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.766+0000 7f6d72800640 1 -- 172.21.15.83:0/1002475085 >> 172.21.15.83:0/1002475085 conn(0x7f6d6c0cfa60 msgr2=0x7f6d6c14e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.766+0000 7f6d72800640 1 -- 172.21.15.83:0/1002475085 shutdown_connections 2024-04-25T00:15:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.766+0000 7f6d72800640 1 -- 172.21.15.83:0/1002475085 wait complete. 2024-04-25T00:15:39.786 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T00:15:39.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:15:39.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.922+0000 7fa90d400640 1 Processor -- start 2024-04-25T00:15:39.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.922+0000 7fa90d400640 1 -- start start 2024-04-25T00:15:39.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.926+0000 7fa90d400640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa9081455d0 0x7fa90806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.926+0000 7fa90d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa9080d0450 0x7fa9080b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.926+0000 7fa90d400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9080ce610 con 0x7fa9080ce080 2024-04-25T00:15:39.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.926+0000 7fa90d400640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa9080ce8f0 con 0x7fa9081455d0 2024-04-25T00:15:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.926+0000 7fa90d400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa9080cebd0 con 0x7fa9080d0450 2024-04-25T00:15:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.926+0000 7fa906800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa9081455d0 0x7fa90806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.926+0000 7fa906800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa9081455d0 0x7fa90806d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60286/0 (socket says 172.21.15.83:60286) 2024-04-25T00:15:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.926+0000 7fa906800640 1 -- 172.21.15.83:0/2614693362 learned_addr learned my addr 172.21.15.83:0/2614693362 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.926+0000 7fa907200640 1 --2- 172.21.15.83:0/2614693362 >> v2:172.21.15.86:3300/0 conn(0x7fa9080d0450 0x7fa9080b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.926+0000 7fa906800640 1 -- 172.21.15.83:0/2614693362 >> v2:172.21.15.86:3300/0 conn(0x7fa9080d0450 msgr2=0x7fa9080b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.926+0000 7fa906800640 1 --2- 172.21.15.83:0/2614693362 >> v2:172.21.15.86:3300/0 conn(0x7fa9080d0450 0x7fa9080b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.926+0000 7fa906800640 1 -- 172.21.15.83:0/2614693362 >> v1:172.21.15.86:6789/0 conn(0x7fa9080ce080 legacy=0x7fa9080cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.926+0000 7fa906800640 1 -- 172.21.15.83:0/2614693362 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9080ceeb0 con 0x7fa9081455d0 2024-04-25T00:15:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.926+0000 7fa907200640 1 --2- 172.21.15.83:0/2614693362 >> v2:172.21.15.86:3300/0 conn(0x7fa9080d0450 0x7fa9080b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:39.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa906800640 1 --2- 172.21.15.83:0/2614693362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9081455d0 0x7fa90806d7e0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fa9081609d0 tx=0x7fa8f801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4dcd92deb3078c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:39.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa905400640 1 -- 172.21.15.83:0/2614693362 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80a5820 con 0x7fa9081455d0 2024-04-25T00:15:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa905400640 1 -- 172.21.15.83:0/2614693362 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f80a59c0 con 0x7fa9081455d0 2024-04-25T00:15:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa905400640 1 -- 172.21.15.83:0/2614693362 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80a5d90 con 0x7fa9081455d0 2024-04-25T00:15:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 -- 172.21.15.83:0/2614693362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9081455d0 msgr2=0x7fa90806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 --2- 172.21.15.83:0/2614693362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9081455d0 0x7fa90806d7e0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fa9081609d0 tx=0x7fa8f801c9f0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 -- 172.21.15.83:0/2614693362 shutdown_connections 2024-04-25T00:15:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 --2- 172.21.15.83:0/2614693362 >> v2:172.21.15.86:3300/0 conn(0x7fa9080d0450 0x7fa9080b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 --2- 172.21.15.83:0/2614693362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9081455d0 0x7fa90806d7e0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 -- 172.21.15.83:0/2614693362 >> 172.21.15.83:0/2614693362 conn(0x7fa90806b3b0 msgr2=0x7fa90806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:39.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 -- 172.21.15.83:0/2614693362 shutdown_connections 2024-04-25T00:15:39.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 -- 172.21.15.83:0/2614693362 wait complete. 2024-04-25T00:15:39.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 Processor -- start 2024-04-25T00:15:39.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 -- start start 2024-04-25T00:15:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9080ce080 0x7fa9081516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9080d0450 0x7fa90814de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9081455d0 0x7fa90814e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9080b3a00 con 0x7fa9080d0450 2024-04-25T00:15:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9080b3880 con 0x7fa9081455d0 2024-04-25T00:15:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.930+0000 7fa90d400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa9080b3b80 con 0x7fa9080ce080 2024-04-25T00:15:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa905e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9080d0450 0x7fa90814de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa905e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9080d0450 0x7fa90814de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60288/0 (socket says 172.21.15.83:60288) 2024-04-25T00:15:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa905e00640 1 -- 172.21.15.83:0/1983343113 learned_addr learned my addr 172.21.15.83:0/1983343113 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa906800640 1 --2- 172.21.15.83:0/1983343113 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9080ce080 0x7fa9081516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa907200640 1 --2- 172.21.15.83:0/1983343113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9081455d0 0x7fa90814e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa905e00640 1 -- 172.21.15.83:0/1983343113 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9080ce080 msgr2=0x7fa9081516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa905e00640 1 --2- 172.21.15.83:0/1983343113 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9080ce080 0x7fa9081516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa905e00640 1 -- 172.21.15.83:0/1983343113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9081455d0 msgr2=0x7fa90814e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa905e00640 1 --2- 172.21.15.83:0/1983343113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9081455d0 0x7fa90814e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa905e00640 1 -- 172.21.15.83:0/1983343113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa908151c20 con 0x7fa9080d0450 2024-04-25T00:15:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa905e00640 1 --2- 172.21.15.83:0/1983343113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9080d0450 0x7fa90814de80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400c420 tx=0x7fa8f400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa8fea00640 1 -- 172.21.15.83:0/1983343113 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f40052d0 con 0x7fa9080d0450 2024-04-25T00:15:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa8fea00640 1 -- 172.21.15.83:0/1983343113 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f4005470 con 0x7fa9080d0450 2024-04-25T00:15:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa90d400640 1 -- 172.21.15.83:0/1983343113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90814e9d0 con 0x7fa9080d0450 2024-04-25T00:15:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa8fea00640 1 -- 172.21.15.83:0/1983343113 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4007260 con 0x7fa9080d0450 2024-04-25T00:15:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa90d400640 1 -- 172.21.15.83:0/1983343113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90814ef90 con 0x7fa9080d0450 2024-04-25T00:15:39.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa90d400640 1 -- 172.21.15.83:0/1983343113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9080736f0 con 0x7fa9080d0450 2024-04-25T00:15:39.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa8fea00640 1 -- 172.21.15.83:0/1983343113 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa8f4007400 con 0x7fa9080d0450 2024-04-25T00:15:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.934+0000 7fa8fea00640 1 --2- 172.21.15.83:0/1983343113 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8e80678c0 0x7fa8e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.938+0000 7fa8fea00640 1 -- 172.21.15.83:0/1983343113 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7fa8f4085bf0 con 0x7fa9080d0450 2024-04-25T00:15:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.938+0000 7fa906800640 1 --2- 172.21.15.83:0/1983343113 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8e80678c0 0x7fa8e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.938+0000 7fa8fea00640 1 --2- 172.21.15.83:0/1983343113 >> [v2:172.21.15.83:6809/1635741185,v1:172.21.15.83:6811/1635741185] conn(0x7fa8e806df40 0x7fa8e8070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.938+0000 7fa8fea00640 1 -- 172.21.15.83:0/1983343113 --> [v2:172.21.15.83:6809/1635741185,v1:172.21.15.83:6811/1635741185] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8e8070a50 con 0x7fa8e806df40 2024-04-25T00:15:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.938+0000 7fa8fea00640 1 -- 172.21.15.83:0/1983343113 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fa8f4085fa0 con 0x7fa9080d0450 2024-04-25T00:15:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.938+0000 7fa907200640 1 --2- 172.21.15.83:0/1983343113 >> [v2:172.21.15.83:6809/1635741185,v1:172.21.15.83:6811/1635741185] conn(0x7fa8e806df40 0x7fa8e8070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.938+0000 7fa906800640 1 --2- 172.21.15.83:0/1983343113 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8e80678c0 0x7fa8e8069d80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80ae000 tx=0x7fa8f80a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.938+0000 7fa907200640 1 --2- 172.21.15.83:0/1983343113 >> [v2:172.21.15.83:6809/1635741185,v1:172.21.15.83:6811/1635741185] conn(0x7fa8e806df40 0x7fa8e8070380 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.938+0000 7fa8fea00640 1 -- 172.21.15.83:0/1983343113 <== osd.3 v2:172.21.15.83:6809/1635741185 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa8e8070a50 con 0x7fa8e806df40 2024-04-25T00:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.950+0000 7fa90d400640 1 -- 172.21.15.83:0/1983343113 --> [v2:172.21.15.83:6809/1635741185,v1:172.21.15.83:6811/1635741185] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa90804f5f0 con 0x7fa8e806df40 2024-04-25T00:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.950+0000 7fa8fea00640 1 -- 172.21.15.83:0/1983343113 <== osd.3 v2:172.21.15.83:6809/1635741185 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa90804f5f0 con 0x7fa8e806df40 2024-04-25T00:15:39.952 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:15:39.952 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:15:39.952 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:15:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.950+0000 7fa90d400640 1 -- 172.21.15.83:0/1983343113 >> [v2:172.21.15.83:6809/1635741185,v1:172.21.15.83:6811/1635741185] conn(0x7fa8e806df40 msgr2=0x7fa8e8070380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.950+0000 7fa90d400640 1 --2- 172.21.15.83:0/1983343113 >> [v2:172.21.15.83:6809/1635741185,v1:172.21.15.83:6811/1635741185] conn(0x7fa8e806df40 0x7fa8e8070380 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.950+0000 7fa90d400640 1 -- 172.21.15.83:0/1983343113 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8e80678c0 msgr2=0x7fa8e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.950+0000 7fa90d400640 1 --2- 172.21.15.83:0/1983343113 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fa8e80678c0 0x7fa8e8069d80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80ae000 tx=0x7fa8f80a7070 comp rx=0 tx=0).stop 2024-04-25T00:15:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.950+0000 7fa90d400640 1 -- 172.21.15.83:0/1983343113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9080d0450 msgr2=0x7fa90814de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.950+0000 7fa90d400640 1 --2- 172.21.15.83:0/1983343113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9080d0450 0x7fa90814de80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400c420 tx=0x7fa8f400c8e0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.950+0000 7fa907200640 1 -- 172.21.15.83:0/1983343113 reap_dead start 2024-04-25T00:15:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.950+0000 7fa90d400640 1 -- 172.21.15.83:0/1983343113 shutdown_connections 2024-04-25T00:15:39.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.950+0000 7fa90d400640 1 -- 172.21.15.83:0/1983343113 >> 172.21.15.83:0/1983343113 conn(0x7fa90806b3b0 msgr2=0x7fa908144780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:39.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.950+0000 7fa90d400640 1 -- 172.21.15.83:0/1983343113 shutdown_connections 2024-04-25T00:15:39.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:39.950+0000 7fa90d400640 1 -- 172.21.15.83:0/1983343113 wait complete. 2024-04-25T00:15:39.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T00:15:40.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.106+0000 7f9ea6400640 1 Processor -- start 2024-04-25T00:15:40.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.106+0000 7f9ea6400640 1 -- start start 2024-04-25T00:15:40.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.106+0000 7f9ea6400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ea00cab50 0x7f9ea015f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.106+0000 7f9ea6400640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea00cafa0 con 0x7f9ea01446e0 2024-04-25T00:15:40.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.106+0000 7f9ea6400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea00cb210 con 0x7f9ea006ee20 2024-04-25T00:15:40.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.106+0000 7f9ea6400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9ea00715b0 con 0x7f9ea00cab50 2024-04-25T00:15:40.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.106+0000 7f9e9f400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9ea01446e0 0x7f9ea006e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35868/0 (socket says 172.21.15.83:35868) 2024-04-25T00:15:40.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.106+0000 7f9e9f400640 1 -- 172.21.15.83:0/1812204958 learned_addr learned my addr 172.21.15.83:0/1812204958 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:40.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.106+0000 7f9e9fe00640 1 --2- 172.21.15.83:0/1812204958 >> v2:172.21.15.86:3300/0 conn(0x7f9ea00cab50 0x7f9ea015f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.106+0000 7f9e9ea00640 1 -- 172.21.15.83:0/1812204958 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 577296942 0 0) 0x7f9ea00cafa0 con 0x7f9ea01446e0 2024-04-25T00:15:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.110+0000 7f9e9ea00640 1 -- 172.21.15.83:0/1812204958 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e84005740 con 0x7f9ea01446e0 2024-04-25T00:15:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.110+0000 7f9e9ea00640 1 -- 172.21.15.83:0/1812204958 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3547085215 0 0) 0x7f9ea00cb210 con 0x7f9ea006ee20 2024-04-25T00:15:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.110+0000 7f9e9ea00640 1 -- 172.21.15.83:0/1812204958 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ea00cafa0 con 0x7f9ea006ee20 2024-04-25T00:15:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.110+0000 7f9e9ea00640 1 -- 172.21.15.83:0/1812204958 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2007925122 0 0) 0x7f9e84005740 con 0x7f9ea01446e0 2024-04-25T00:15:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.110+0000 7f9e9fe00640 1 -- 172.21.15.83:0/1812204958 >> v1:172.21.15.86:6789/0 conn(0x7f9ea006ee20 legacy=0x7f9ea0071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.110+0000 7f9e9fe00640 1 -- 172.21.15.83:0/1812204958 >> v1:172.21.15.83:6789/0 conn(0x7f9ea01446e0 legacy=0x7f9ea006e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.110+0000 7f9e9fe00640 1 -- 172.21.15.83:0/1812204958 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea00cb4f0 con 0x7f9ea00cab50 2024-04-25T00:15:40.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.110+0000 7f9e9fe00640 1 --2- 172.21.15.83:0/1812204958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea00cab50 0x7f9ea015f350 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009d90 tx=0x7f9e9801c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85b04f7bdda580df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.110+0000 7f9e9ea00640 1 -- 172.21.15.83:0/1812204958 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90001680 con 0x7f9ea00cab50 2024-04-25T00:15:40.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.110+0000 7f9e9ea00640 1 -- 172.21.15.83:0/1812204958 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e980a64e0 con 0x7f9ea00cab50 2024-04-25T00:15:40.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.110+0000 7f9e9ea00640 1 -- 172.21.15.83:0/1812204958 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e980a68b0 con 0x7f9ea00cab50 2024-04-25T00:15:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 -- 172.21.15.83:0/1812204958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea00cab50 msgr2=0x7f9ea015f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 --2- 172.21.15.83:0/1812204958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea00cab50 0x7f9ea015f350 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009d90 tx=0x7f9e9801c890 comp rx=0 tx=0).stop 2024-04-25T00:15:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 -- 172.21.15.83:0/1812204958 shutdown_connections 2024-04-25T00:15:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 --2- 172.21.15.83:0/1812204958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea00cab50 0x7f9ea015f350 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 -- 172.21.15.83:0/1812204958 >> 172.21.15.83:0/1812204958 conn(0x7f9ea00cfa60 msgr2=0x7f9ea00cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 -- 172.21.15.83:0/1812204958 shutdown_connections 2024-04-25T00:15:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 -- 172.21.15.83:0/1812204958 wait complete. 2024-04-25T00:15:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 Processor -- start 2024-04-25T00:15:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 -- start start 2024-04-25T00:15:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ea006ee20 0x7f9ea00c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ea00cab50 0x7f9ea01f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea01446e0 0x7f9ea0159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea006d9d0 con 0x7f9ea00cab50 2024-04-25T00:15:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea006d850 con 0x7f9ea01446e0 2024-04-25T00:15:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9ea6400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea006db50 con 0x7f9ea006ee20 2024-04-25T00:15:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9e96a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ea00cab50 0x7f9ea01f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9e96a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ea00cab50 0x7f9ea01f29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60290/0 (socket says 172.21.15.83:60290) 2024-04-25T00:15:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9e96a00640 1 -- 172.21.15.83:0/4045406946 learned_addr learned my addr 172.21.15.83:0/4045406946 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9e9fe00640 1 --2- 172.21.15.83:0/4045406946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea01446e0 0x7f9ea0159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9e9f400640 1 --2- 172.21.15.83:0/4045406946 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ea006ee20 0x7f9ea00c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9e96a00640 1 -- 172.21.15.83:0/4045406946 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ea006ee20 msgr2=0x7f9ea00c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9e96a00640 1 --2- 172.21.15.83:0/4045406946 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ea006ee20 0x7f9ea00c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9e96a00640 1 -- 172.21.15.83:0/4045406946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea01446e0 msgr2=0x7f9ea0159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9e96a00640 1 --2- 172.21.15.83:0/4045406946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea01446e0 0x7f9ea0159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.114+0000 7f9e96a00640 1 -- 172.21.15.83:0/4045406946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea015a2f0 con 0x7f9ea00cab50 2024-04-25T00:15:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.118+0000 7f9e96a00640 1 --2- 172.21.15.83:0/4045406946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ea00cab50 0x7f9ea01f29d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800c500 tx=0x7f9e8800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.118+0000 7f9e97e00640 1 -- 172.21.15.83:0/4045406946 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e880052d0 con 0x7f9ea00cab50 2024-04-25T00:15:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.118+0000 7f9e97e00640 1 -- 172.21.15.83:0/4045406946 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e88005490 con 0x7f9ea00cab50 2024-04-25T00:15:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.118+0000 7f9e97e00640 1 -- 172.21.15.83:0/4045406946 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88012850 con 0x7f9ea00cab50 2024-04-25T00:15:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.118+0000 7f9ea6400640 1 -- 172.21.15.83:0/4045406946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea00b3580 con 0x7f9ea00cab50 2024-04-25T00:15:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.118+0000 7f9ea6400640 1 -- 172.21.15.83:0/4045406946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea00b3b00 con 0x7f9ea00cab50 2024-04-25T00:15:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.118+0000 7f9ea6400640 1 -- 172.21.15.83:0/4045406946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea0068230 con 0x7f9ea00cab50 2024-04-25T00:15:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.118+0000 7f9e97e00640 1 -- 172.21.15.83:0/4045406946 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9e880129f0 con 0x7f9ea00cab50 2024-04-25T00:15:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.118+0000 7f9e97e00640 1 --2- 172.21.15.83:0/4045406946 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9e7c0678c0 0x7f9e7c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.118+0000 7f9e97e00640 1 -- 172.21.15.83:0/4045406946 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f9e88085850 con 0x7f9ea00cab50 2024-04-25T00:15:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.118+0000 7f9e9f400640 1 --2- 172.21.15.83:0/4045406946 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9e7c0678c0 0x7f9e7c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.118+0000 7f9e9f400640 1 --2- 172.21.15.83:0/4045406946 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9e7c0678c0 0x7f9e7c069d80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9e900011f0 tx=0x7f9e90001700 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.122+0000 7f9e97e00640 1 -- 172.21.15.83:0/4045406946 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9e88055d20 con 0x7f9ea00cab50 2024-04-25T00:15:40.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.278+0000 7f9ea6400640 1 -- 172.21.15.83:0/4045406946 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f9ea0072230 con 0x7f9e7c0678c0 2024-04-25T00:15:40.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.278+0000 7f9e97e00640 1 -- 172.21.15.83:0/4045406946 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9ea0072230 con 0x7f9e7c0678c0 2024-04-25T00:15:40.279 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T00:15:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.278+0000 7f9ea6400640 1 -- 172.21.15.83:0/4045406946 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9e7c0678c0 msgr2=0x7f9e7c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.278+0000 7f9ea6400640 1 --2- 172.21.15.83:0/4045406946 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9e7c0678c0 0x7f9e7c069d80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9e900011f0 tx=0x7f9e90001700 comp rx=0 tx=0).stop 2024-04-25T00:15:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.278+0000 7f9ea6400640 1 -- 172.21.15.83:0/4045406946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ea00cab50 msgr2=0x7f9ea01f29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.278+0000 7f9ea6400640 1 --2- 172.21.15.83:0/4045406946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ea00cab50 0x7f9ea01f29d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800c500 tx=0x7f9e8800c9c0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.282+0000 7f9ea6400640 1 -- 172.21.15.83:0/4045406946 shutdown_connections 2024-04-25T00:15:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.282+0000 7f9ea6400640 1 --2- 172.21.15.83:0/4045406946 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f9e7c0678c0 0x7f9e7c069d80 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.282+0000 7f9ea6400640 1 --2- 172.21.15.83:0/4045406946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea01446e0 0x7f9ea0159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.282+0000 7f9ea6400640 1 --2- 172.21.15.83:0/4045406946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ea00cab50 0x7f9ea01f29d0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.282+0000 7f9ea6400640 1 --2- 172.21.15.83:0/4045406946 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ea006ee20 0x7f9ea00c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.282+0000 7f9ea6400640 1 -- 172.21.15.83:0/4045406946 >> 172.21.15.83:0/4045406946 conn(0x7f9ea00cfa60 msgr2=0x7f9ea015b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.282+0000 7f9ea6400640 1 -- 172.21.15.83:0/4045406946 shutdown_connections 2024-04-25T00:15:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.282+0000 7f9ea6400640 1 -- 172.21.15.83:0/4045406946 wait complete. 2024-04-25T00:15:40.300 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T00:15:40.300 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:15:40.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.438+0000 7f0a80e00640 1 Processor -- start 2024-04-25T00:15:40.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.438+0000 7f0a80e00640 1 -- start start 2024-04-25T00:15:40.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.438+0000 7f0a80e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0a7c15de50 0x7f0a7c0b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.438+0000 7f0a80e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a7c0d0450 0x7f0a7c0b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.438+0000 7f0a80e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a7c0b2f10 con 0x7f0a7c0b44c0 2024-04-25T00:15:40.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.438+0000 7f0a80e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0a7c0b2ce0 con 0x7f0a7c15de50 2024-04-25T00:15:40.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.438+0000 7f0a80e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7c0b4f40 con 0x7f0a7c0d0450 2024-04-25T00:15:40.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.438+0000 7f0a7a800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a7c0d0450 0x7f0a7c0b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.442+0000 7f0a79e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0a7c15de50 0x7f0a7c0b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.442+0000 7f0a7a800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a7c0d0450 0x7f0a7c0b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:54042/0 (socket says 172.21.15.83:54042) 2024-04-25T00:15:40.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.442+0000 7f0a7a800640 1 -- 172.21.15.83:0/2836832822 learned_addr learned my addr 172.21.15.83:0/2836832822 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:40.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.442+0000 7f0a7a800640 1 -- 172.21.15.83:0/2836832822 >> v2:172.21.15.86:3300/0 conn(0x7f0a7c15de50 msgr2=0x7f0a7c0b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:40.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.442+0000 7f0a79400640 1 -- 172.21.15.83:0/2836832822 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1852286924 0 0) 0x7f0a7c0b2f10 con 0x7f0a7c0b44c0 2024-04-25T00:15:40.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.442+0000 7f0a7a800640 1 --2- 172.21.15.83:0/2836832822 >> v2:172.21.15.86:3300/0 conn(0x7f0a7c15de50 0x7f0a7c0b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.442+0000 7f0a7a800640 1 -- 172.21.15.83:0/2836832822 >> v1:172.21.15.86:6789/0 conn(0x7f0a7c0b44c0 legacy=0x7f0a7c0cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.442+0000 7f0a7a800640 1 -- 172.21.15.83:0/2836832822 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c0b31f0 con 0x7f0a7c0d0450 2024-04-25T00:15:40.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.442+0000 7f0a79e00640 1 --2- 172.21.15.83:0/2836832822 >> v2:172.21.15.86:3300/0 conn(0x7f0a7c15de50 0x7f0a7c0b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.442+0000 7f0a7a800640 1 --2- 172.21.15.83:0/2836832822 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a7c0d0450 0x7f0a7c0b4930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009870 tx=0x7f0a7401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5960986bc3ecb57f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.442+0000 7f0a79400640 1 -- 172.21.15.83:0/2836832822 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a740ab650 con 0x7f0a7c0d0450 2024-04-25T00:15:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.442+0000 7f0a79400640 1 -- 172.21.15.83:0/2836832822 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a740b0070 con 0x7f0a7c0d0450 2024-04-25T00:15:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.442+0000 7f0a79400640 1 -- 172.21.15.83:0/2836832822 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a740aba10 con 0x7f0a7c0d0450 2024-04-25T00:15:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 -- 172.21.15.83:0/2836832822 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a7c0d0450 msgr2=0x7f0a7c0b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 --2- 172.21.15.83:0/2836832822 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a7c0d0450 0x7f0a7c0b4930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009870 tx=0x7f0a7401c560 comp rx=0 tx=0).stop 2024-04-25T00:15:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 -- 172.21.15.83:0/2836832822 shutdown_connections 2024-04-25T00:15:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 --2- 172.21.15.83:0/2836832822 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a7c0d0450 0x7f0a7c0b4930 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 --2- 172.21.15.83:0/2836832822 >> v2:172.21.15.86:3300/0 conn(0x7f0a7c15de50 0x7f0a7c0b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 -- 172.21.15.83:0/2836832822 >> 172.21.15.83:0/2836832822 conn(0x7f0a7c06b3b0 msgr2=0x7f0a7c06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 -- 172.21.15.83:0/2836832822 shutdown_connections 2024-04-25T00:15:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 -- 172.21.15.83:0/2836832822 wait complete. 2024-04-25T00:15:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 Processor -- start 2024-04-25T00:15:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 -- start start 2024-04-25T00:15:40.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7c0b44c0 0x7f0a7c15b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c0d0450 0x7f0a7c150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a7c15de50 0x7f0a7c15fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c06dd00 con 0x7f0a7c0d0450 2024-04-25T00:15:40.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c06db80 con 0x7f0a7c0b44c0 2024-04-25T00:15:40.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a80e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7c071f40 con 0x7f0a7c15de50 2024-04-25T00:15:40.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a71400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c0d0450 0x7f0a7c150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a71400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c0d0450 0x7f0a7c150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60302/0 (socket says 172.21.15.83:60302) 2024-04-25T00:15:40.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a71400640 1 -- 172.21.15.83:0/2462466915 learned_addr learned my addr 172.21.15.83:0/2462466915 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:40.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a79e00640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7c0b44c0 0x7f0a7c15b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a7a800640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a7c15de50 0x7f0a7c15fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a71400640 1 -- 172.21.15.83:0/2462466915 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a7c15de50 msgr2=0x7f0a7c15fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a71400640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a7c15de50 0x7f0a7c15fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a71400640 1 -- 172.21.15.83:0/2462466915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7c0b44c0 msgr2=0x7f0a7c15b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a71400640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7c0b44c0 0x7f0a7c15b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a71400640 1 -- 172.21.15.83:0/2462466915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c151660 con 0x7f0a7c0d0450 2024-04-25T00:15:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a7a800640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a7c15de50 0x7f0a7c15fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a79e00640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7c0b44c0 0x7f0a7c15b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.446+0000 7f0a71400640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c0d0450 0x7f0a7c150950 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400c4d0 tx=0x7f0a6400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a72a00640 1 -- 172.21.15.83:0/2462466915 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64017070 con 0x7f0a7c0d0450 2024-04-25T00:15:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a72a00640 1 -- 172.21.15.83:0/2462466915 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a64005480 con 0x7f0a7c0d0450 2024-04-25T00:15:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a80e00640 1 -- 172.21.15.83:0/2462466915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7c151940 con 0x7f0a7c0d0450 2024-04-25T00:15:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a72a00640 1 -- 172.21.15.83:0/2462466915 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64003840 con 0x7f0a7c0d0450 2024-04-25T00:15:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a80e00640 1 -- 172.21.15.83:0/2462466915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7c151f00 con 0x7f0a7c0d0450 2024-04-25T00:15:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a80e00640 1 -- 172.21.15.83:0/2462466915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a7c0736f0 con 0x7f0a7c0d0450 2024-04-25T00:15:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a72a00640 1 -- 172.21.15.83:0/2462466915 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0a6401a070 con 0x7f0a7c0d0450 2024-04-25T00:15:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a72a00640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0a580678f0 0x7f0a58069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a72a00640 1 -- 172.21.15.83:0/2462466915 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f0a64084fe0 con 0x7f0a7c0d0450 2024-04-25T00:15:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a79e00640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0a580678f0 0x7f0a58069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a72a00640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.86:6816/1560646044,v1:172.21.15.86:6817/1560646044] conn(0x7f0a5806df70 0x7f0a580703b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a72a00640 1 -- 172.21.15.83:0/2462466915 --> [v2:172.21.15.86:6816/1560646044,v1:172.21.15.86:6817/1560646044] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a58070a80 con 0x7f0a5806df70 2024-04-25T00:15:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a72a00640 1 -- 172.21.15.83:0/2462466915 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f0a640875c0 con 0x7f0a7c0d0450 2024-04-25T00:15:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a7a800640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.86:6816/1560646044,v1:172.21.15.86:6817/1560646044] conn(0x7f0a5806df70 0x7f0a580703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.450+0000 7f0a79e00640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0a580678f0 0x7f0a58069db0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009fd0 tx=0x7f0a6c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.454+0000 7f0a7a800640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.86:6816/1560646044,v1:172.21.15.86:6817/1560646044] conn(0x7f0a5806df70 0x7f0a580703b0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.454+0000 7f0a72a00640 1 -- 172.21.15.83:0/2462466915 <== osd.4 v2:172.21.15.86:6816/1560646044 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a58070a80 con 0x7f0a5806df70 2024-04-25T00:15:40.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.466+0000 7f0a80e00640 1 -- 172.21.15.83:0/2462466915 --> [v2:172.21.15.86:6816/1560646044,v1:172.21.15.86:6817/1560646044] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0a7c04f5f0 con 0x7f0a5806df70 2024-04-25T00:15:40.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.466+0000 7f0a72a00640 1 -- 172.21.15.83:0/2462466915 <== osd.4 v2:172.21.15.86:6816/1560646044 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0a7c04f5f0 con 0x7f0a5806df70 2024-04-25T00:15:40.468 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:15:40.468 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:15:40.468 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:15:40.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.466+0000 7f0a80e00640 1 -- 172.21.15.83:0/2462466915 >> [v2:172.21.15.86:6816/1560646044,v1:172.21.15.86:6817/1560646044] conn(0x7f0a5806df70 msgr2=0x7f0a580703b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.466+0000 7f0a80e00640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.86:6816/1560646044,v1:172.21.15.86:6817/1560646044] conn(0x7f0a5806df70 0x7f0a580703b0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.466+0000 7f0a80e00640 1 -- 172.21.15.83:0/2462466915 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0a580678f0 msgr2=0x7f0a58069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.466+0000 7f0a80e00640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f0a580678f0 0x7f0a58069db0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009fd0 tx=0x7f0a6c00b040 comp rx=0 tx=0).stop 2024-04-25T00:15:40.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.466+0000 7f0a80e00640 1 -- 172.21.15.83:0/2462466915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c0d0450 msgr2=0x7f0a7c150950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.466+0000 7f0a80e00640 1 --2- 172.21.15.83:0/2462466915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c0d0450 0x7f0a7c150950 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400c4d0 tx=0x7f0a6400c990 comp rx=0 tx=0).stop 2024-04-25T00:15:40.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.466+0000 7f0a7a800640 1 -- 172.21.15.83:0/2462466915 reap_dead start 2024-04-25T00:15:40.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.466+0000 7f0a80e00640 1 -- 172.21.15.83:0/2462466915 shutdown_connections 2024-04-25T00:15:40.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.466+0000 7f0a80e00640 1 -- 172.21.15.83:0/2462466915 >> 172.21.15.83:0/2462466915 conn(0x7f0a7c06b3b0 msgr2=0x7f0a7c15d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:40.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.466+0000 7f0a80e00640 1 -- 172.21.15.83:0/2462466915 shutdown_connections 2024-04-25T00:15:40.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.466+0000 7f0a80e00640 1 -- 172.21.15.83:0/2462466915 wait complete. 2024-04-25T00:15:40.486 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T00:15:40.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.622+0000 7f147e000640 1 Processor -- start 2024-04-25T00:15:40.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.622+0000 7f147e000640 1 -- start start 2024-04-25T00:15:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f147e000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f147806ee20 0x7f14780cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f147e000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f147815f350 0x7f1478071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f147e000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14780cb210 con 0x7f14781446e0 2024-04-25T00:15:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f147e000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f14780715b0 con 0x7f147815f350 2024-04-25T00:15:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f147e000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1478144ac0 con 0x7f147806ee20 2024-04-25T00:15:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f1477400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f14781446e0 0x7f147806e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35882/0 (socket says 172.21.15.83:35882) 2024-04-25T00:15:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f1477400640 1 -- 172.21.15.83:0/2468479648 learned_addr learned my addr 172.21.15.83:0/2468479648 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f1477e00640 1 --2- 172.21.15.83:0/2468479648 >> v2:172.21.15.83:3300/0 conn(0x7f147815f350 0x7f1478071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f146ea00640 1 --2- 172.21.15.83:0/2468479648 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f147806ee20 0x7f14780cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f1476a00640 1 -- 172.21.15.83:0/2468479648 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3003174081 0 0) 0x7f14780cb210 con 0x7f14781446e0 2024-04-25T00:15:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f1476a00640 1 -- 172.21.15.83:0/2468479648 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f145c003530 con 0x7f14781446e0 2024-04-25T00:15:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f1477e00640 1 -- 172.21.15.83:0/2468479648 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f147806ee20 msgr2=0x7f14780cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f1477e00640 1 --2- 172.21.15.83:0/2468479648 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f147806ee20 0x7f14780cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f1477e00640 1 -- 172.21.15.83:0/2468479648 >> v1:172.21.15.83:6789/0 conn(0x7f14781446e0 legacy=0x7f147806e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f1477e00640 1 -- 172.21.15.83:0/2468479648 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14780cb4f0 con 0x7f147815f350 2024-04-25T00:15:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f1477e00640 1 --2- 172.21.15.83:0/2468479648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147815f350 0x7f1478071fe0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f14700097c0 tx=0x7f147001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f3dd978ae459a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.626+0000 7f1476a00640 1 -- 172.21.15.83:0/2468479648 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14700b0070 con 0x7f147815f350 2024-04-25T00:15:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f1476a00640 1 -- 172.21.15.83:0/2468479648 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14700ab600 con 0x7f147815f350 2024-04-25T00:15:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f1476a00640 1 -- 172.21.15.83:0/2468479648 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14700ab9d0 con 0x7f147815f350 2024-04-25T00:15:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 -- 172.21.15.83:0/2468479648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147815f350 msgr2=0x7f1478071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 --2- 172.21.15.83:0/2468479648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147815f350 0x7f1478071fe0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f14700097c0 tx=0x7f147001c4a0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 -- 172.21.15.83:0/2468479648 shutdown_connections 2024-04-25T00:15:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 --2- 172.21.15.83:0/2468479648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147815f350 0x7f1478071fe0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 --2- 172.21.15.83:0/2468479648 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f147806ee20 0x7f14780cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 -- 172.21.15.83:0/2468479648 >> 172.21.15.83:0/2468479648 conn(0x7f14780cfa60 msgr2=0x7f14780cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:40.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 -- 172.21.15.83:0/2468479648 shutdown_connections 2024-04-25T00:15:40.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 -- 172.21.15.83:0/2468479648 wait complete. 2024-04-25T00:15:40.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 Processor -- start 2024-04-25T00:15:40.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 -- start start 2024-04-25T00:15:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147806ee20 0x7f147815e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14781446e0 0x7f14780c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f147815f350 0x7f1478143030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1478072eb0 con 0x7f147806ee20 2024-04-25T00:15:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1478072d30 con 0x7f147815f350 2024-04-25T00:15:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f147e000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1478073030 con 0x7f14781446e0 2024-04-25T00:15:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f1477400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147806ee20 0x7f147815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f1477400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147806ee20 0x7f147815e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60322/0 (socket says 172.21.15.83:60322) 2024-04-25T00:15:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.630+0000 7f1477400640 1 -- 172.21.15.83:0/912342055 learned_addr learned my addr 172.21.15.83:0/912342055 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f146ea00640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14781446e0 0x7f14780c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f1477e00640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f147815f350 0x7f1478143030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f1477400640 1 -- 172.21.15.83:0/912342055 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14781446e0 msgr2=0x7f14780c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f1477400640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14781446e0 0x7f14780c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f1477400640 1 -- 172.21.15.83:0/912342055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f147815f350 msgr2=0x7f1478143030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f1477400640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f147815f350 0x7f1478143030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f1477400640 1 -- 172.21.15.83:0/912342055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14780c95d0 con 0x7f147806ee20 2024-04-25T00:15:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f1477e00640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f147815f350 0x7f1478143030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f1477400640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147806ee20 0x7f147815e700 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f146800cb70 tx=0x7f146800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f146fe00640 1 -- 172.21.15.83:0/912342055 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468017070 con 0x7f147806ee20 2024-04-25T00:15:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f146fe00640 1 -- 172.21.15.83:0/912342055 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f146800ad40 con 0x7f147806ee20 2024-04-25T00:15:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f147e000640 1 -- 172.21.15.83:0/912342055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14780c4a40 con 0x7f147806ee20 2024-04-25T00:15:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f146fe00640 1 -- 172.21.15.83:0/912342055 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468012620 con 0x7f147806ee20 2024-04-25T00:15:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f147e000640 1 -- 172.21.15.83:0/912342055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14780c98b0 con 0x7f147806ee20 2024-04-25T00:15:40.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f147e000640 1 -- 172.21.15.83:0/912342055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1478068230 con 0x7f147806ee20 2024-04-25T00:15:40.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f146fe00640 1 -- 172.21.15.83:0/912342055 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f146801a070 con 0x7f147806ee20 2024-04-25T00:15:40.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f146fe00640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f14540678c0 0x7f1454069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f146fe00640 1 -- 172.21.15.83:0/912342055 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f14680859f0 con 0x7f147806ee20 2024-04-25T00:15:40.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.634+0000 7f146ea00640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f14540678c0 0x7f1454069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.638+0000 7f146ea00640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f14540678c0 0x7f1454069d80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1478164a50 tx=0x7f1460008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.638+0000 7f146fe00640 1 -- 172.21.15.83:0/912342055 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1468056630 con 0x7f147806ee20 2024-04-25T00:15:40.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.794+0000 7f147e000640 1 -- 172.21.15.83:0/912342055 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f14780b06a0 con 0x7f14540678c0 2024-04-25T00:15:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.794+0000 7f146fe00640 1 -- 172.21.15.83:0/912342055 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f14780b06a0 con 0x7f14540678c0 2024-04-25T00:15:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T00:15:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.794+0000 7f147e000640 1 -- 172.21.15.83:0/912342055 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f14540678c0 msgr2=0x7f1454069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.794+0000 7f147e000640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f14540678c0 0x7f1454069d80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1478164a50 tx=0x7f1460008040 comp rx=0 tx=0).stop 2024-04-25T00:15:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.794+0000 7f147e000640 1 -- 172.21.15.83:0/912342055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147806ee20 msgr2=0x7f147815e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.794+0000 7f147e000640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147806ee20 0x7f147815e700 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f146800cb70 tx=0x7f146800a430 comp rx=0 tx=0).stop 2024-04-25T00:15:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.798+0000 7f147e000640 1 -- 172.21.15.83:0/912342055 shutdown_connections 2024-04-25T00:15:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.798+0000 7f147e000640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f14540678c0 0x7f1454069d80 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.798+0000 7f147e000640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14781446e0 0x7f14780c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.798+0000 7f147e000640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f147815f350 0x7f1478143030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.798+0000 7f147e000640 1 --2- 172.21.15.83:0/912342055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147806ee20 0x7f147815e700 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.798+0000 7f147e000640 1 -- 172.21.15.83:0/912342055 >> 172.21.15.83:0/912342055 conn(0x7f14780cfa60 msgr2=0x7f14780b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.798+0000 7f147e000640 1 -- 172.21.15.83:0/912342055 shutdown_connections 2024-04-25T00:15:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.798+0000 7f147e000640 1 -- 172.21.15.83:0/912342055 wait complete. 2024-04-25T00:15:40.816 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T00:15:40.816 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:15:40.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.954+0000 7f1728400640 1 Processor -- start 2024-04-25T00:15:40.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.954+0000 7f1728400640 1 -- start start 2024-04-25T00:15:40.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.954+0000 7f1728400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f17200ce080 0x7f17200cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.954+0000 7f1728400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17200d0280 0x7f17200b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.958+0000 7f1728400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17200ce610 con 0x7f17201455d0 2024-04-25T00:15:40.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.958+0000 7f1728400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f17200ce8f0 con 0x7f17200ce080 2024-04-25T00:15:40.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.958+0000 7f1728400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f17200cebd0 con 0x7f17200d0280 2024-04-25T00:15:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.958+0000 7f1726400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17200d0280 0x7f17200b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.958+0000 7f1725000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f17200ce080 0x7f17200cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.958+0000 7f1726400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17200d0280 0x7f17200b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.83:54076/0 (socket says 172.21.15.83:54076) 2024-04-25T00:15:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.958+0000 7f1725a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f17201455d0 0x7f172006d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:36228/0 (socket says 172.21.15.83:36228) 2024-04-25T00:15:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.958+0000 7f1726400640 1 -- 172.21.15.83:0/2664409914 learned_addr learned my addr 172.21.15.83:0/2664409914 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.958+0000 7f1726400640 1 -- 172.21.15.83:0/2664409914 >> v2:172.21.15.86:3300/0 conn(0x7f17200ce080 msgr2=0x7f17200cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.958+0000 7f1726400640 1 --2- 172.21.15.83:0/2664409914 >> v2:172.21.15.86:3300/0 conn(0x7f17200ce080 0x7f17200cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.958+0000 7f1726400640 1 -- 172.21.15.83:0/2664409914 >> v1:172.21.15.86:6789/0 conn(0x7f17201455d0 legacy=0x7f172006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.958+0000 7f1726400640 1 -- 172.21.15.83:0/2664409914 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17200ceeb0 con 0x7f17200d0280 2024-04-25T00:15:40.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.958+0000 7f1726400640 1 --2- 172.21.15.83:0/2664409914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17200d0280 0x7f17200b2b30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1718009870 tx=0x7f171801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34580b612f46e7ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f171fe00640 1 -- 172.21.15.83:0/2664409914 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17180a7070 con 0x7f17200d0280 2024-04-25T00:15:40.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f171fe00640 1 -- 172.21.15.83:0/2664409914 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17180b0070 con 0x7f17200d0280 2024-04-25T00:15:40.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f171fe00640 1 -- 172.21.15.83:0/2664409914 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17180ab890 con 0x7f17200d0280 2024-04-25T00:15:40.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 -- 172.21.15.83:0/2664409914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17200d0280 msgr2=0x7f17200b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 --2- 172.21.15.83:0/2664409914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17200d0280 0x7f17200b2b30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1718009870 tx=0x7f171801c560 comp rx=0 tx=0).stop 2024-04-25T00:15:40.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 -- 172.21.15.83:0/2664409914 shutdown_connections 2024-04-25T00:15:40.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 --2- 172.21.15.83:0/2664409914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17200d0280 0x7f17200b2b30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 --2- 172.21.15.83:0/2664409914 >> v2:172.21.15.86:3300/0 conn(0x7f17200ce080 0x7f17200cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 -- 172.21.15.83:0/2664409914 >> 172.21.15.83:0/2664409914 conn(0x7f172006b3b0 msgr2=0x7f172006b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:40.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 -- 172.21.15.83:0/2664409914 shutdown_connections 2024-04-25T00:15:40.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 -- 172.21.15.83:0/2664409914 wait complete. 2024-04-25T00:15:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 Processor -- start 2024-04-25T00:15:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 -- start start 2024-04-25T00:15:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17200ce080 0x7f172013b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17200d0280 0x7f172013e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17201455d0 0x7f1720155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17200b3c50 con 0x7f17201455d0 2024-04-25T00:15:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17200b3ad0 con 0x7f17200ce080 2024-04-25T00:15:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.962+0000 7f1728400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f17200b3dd0 con 0x7f17200d0280 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f1726400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17201455d0 0x7f1720155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f1726400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17201455d0 0x7f1720155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60328/0 (socket says 172.21.15.83:60328) 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f1726400640 1 -- 172.21.15.83:0/4233887601 learned_addr learned my addr 172.21.15.83:0/4233887601 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f1725a00640 1 --2- 172.21.15.83:0/4233887601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17200ce080 0x7f172013b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f1725000640 1 --2- 172.21.15.83:0/4233887601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17200d0280 0x7f172013e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f1726400640 1 -- 172.21.15.83:0/4233887601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17200d0280 msgr2=0x7f172013e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f1726400640 1 --2- 172.21.15.83:0/4233887601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17200d0280 0x7f172013e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f1726400640 1 -- 172.21.15.83:0/4233887601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17200ce080 msgr2=0x7f172013b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f1726400640 1 --2- 172.21.15.83:0/4233887601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17200ce080 0x7f172013b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f1726400640 1 -- 172.21.15.83:0/4233887601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172013d400 con 0x7f17201455d0 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f1726400640 1 --2- 172.21.15.83:0/4233887601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17201455d0 0x7f1720155200 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f171801a190 tx=0x7f171801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f171d600640 1 -- 172.21.15.83:0/4233887601 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17180be5f0 con 0x7f17201455d0 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f171d600640 1 -- 172.21.15.83:0/4233887601 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17180be790 con 0x7f17201455d0 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f171d600640 1 -- 172.21.15.83:0/4233887601 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17180beb40 con 0x7f17201455d0 2024-04-25T00:15:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f1728400640 1 -- 172.21.15.83:0/4233887601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172013d6e0 con 0x7f17201455d0 2024-04-25T00:15:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f1728400640 1 -- 172.21.15.83:0/4233887601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172013dca0 con 0x7f17201455d0 2024-04-25T00:15:40.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f1728400640 1 -- 172.21.15.83:0/4233887601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f17201fb580 con 0x7f17201455d0 2024-04-25T00:15:40.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f171d600640 1 -- 172.21.15.83:0/4233887601 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f17180b43f0 con 0x7f17201455d0 2024-04-25T00:15:40.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f171d600640 1 --2- 172.21.15.83:0/4233887601 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17000678c0 0x7f1700069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.966+0000 7f171d600640 1 -- 172.21.15.83:0/4233887601 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f171811fdf0 con 0x7f17201455d0 2024-04-25T00:15:40.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.970+0000 7f1725a00640 1 --2- 172.21.15.83:0/4233887601 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17000678c0 0x7f1700069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.970+0000 7f171d600640 1 --2- 172.21.15.83:0/4233887601 >> [v2:172.21.15.86:6805/340393403,v1:172.21.15.86:6807/340393403] conn(0x7f170006df40 0x7f1700070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.970+0000 7f171d600640 1 -- 172.21.15.83:0/4233887601 --> [v2:172.21.15.86:6805/340393403,v1:172.21.15.86:6807/340393403] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17180a8e50 con 0x7f170006df40 2024-04-25T00:15:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.970+0000 7f171d600640 1 -- 172.21.15.83:0/4233887601 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f1718122410 con 0x7f17201455d0 2024-04-25T00:15:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.970+0000 7f1725000640 1 --2- 172.21.15.83:0/4233887601 >> [v2:172.21.15.86:6805/340393403,v1:172.21.15.86:6807/340393403] conn(0x7f170006df40 0x7f1700070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.970+0000 7f1725a00640 1 --2- 172.21.15.83:0/4233887601 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17000678c0 0x7f1700069d80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1714007fd0 tx=0x7f1714005b80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.970+0000 7f1725000640 1 --2- 172.21.15.83:0/4233887601 >> [v2:172.21.15.86:6805/340393403,v1:172.21.15.86:6807/340393403] conn(0x7f170006df40 0x7f1700070380 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.970+0000 7f171d600640 1 -- 172.21.15.83:0/4233887601 <== osd.5 v2:172.21.15.86:6805/340393403 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17180a8e50 con 0x7f170006df40 2024-04-25T00:15:40.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.982+0000 7f1728400640 1 -- 172.21.15.83:0/4233887601 --> [v2:172.21.15.86:6805/340393403,v1:172.21.15.86:6807/340393403] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f172013e950 con 0x7f170006df40 2024-04-25T00:15:40.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.982+0000 7f171d600640 1 -- 172.21.15.83:0/4233887601 <== osd.5 v2:172.21.15.86:6805/340393403 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f170c009820 con 0x7f170006df40 2024-04-25T00:15:40.985 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:15:40.985 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:15:40.985 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:15:40.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.982+0000 7f1728400640 1 -- 172.21.15.83:0/4233887601 >> [v2:172.21.15.86:6805/340393403,v1:172.21.15.86:6807/340393403] conn(0x7f170006df40 msgr2=0x7f1700070380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.982+0000 7f1728400640 1 --2- 172.21.15.83:0/4233887601 >> [v2:172.21.15.86:6805/340393403,v1:172.21.15.86:6807/340393403] conn(0x7f170006df40 0x7f1700070380 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.986+0000 7f1728400640 1 -- 172.21.15.83:0/4233887601 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17000678c0 msgr2=0x7f1700069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.986+0000 7f1728400640 1 --2- 172.21.15.83:0/4233887601 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f17000678c0 0x7f1700069d80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1714007fd0 tx=0x7f1714005b80 comp rx=0 tx=0).stop 2024-04-25T00:15:40.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.986+0000 7f1728400640 1 -- 172.21.15.83:0/4233887601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17201455d0 msgr2=0x7f1720155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.986+0000 7f1728400640 1 --2- 172.21.15.83:0/4233887601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17201455d0 0x7f1720155200 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f171801a190 tx=0x7f171801ada0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.986+0000 7f1726400640 1 -- 172.21.15.83:0/4233887601 reap_dead start 2024-04-25T00:15:40.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.986+0000 7f1728400640 1 -- 172.21.15.83:0/4233887601 shutdown_connections 2024-04-25T00:15:40.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.986+0000 7f1728400640 1 -- 172.21.15.83:0/4233887601 >> 172.21.15.83:0/4233887601 conn(0x7f172006b3b0 msgr2=0x7f172013fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:40.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.986+0000 7f1728400640 1 -- 172.21.15.83:0/4233887601 shutdown_connections 2024-04-25T00:15:40.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:40.986+0000 7f1728400640 1 -- 172.21.15.83:0/4233887601 wait complete. 2024-04-25T00:15:41.004 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T00:15:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.138+0000 7f3e99000640 1 Processor -- start 2024-04-25T00:15:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.138+0000 7f3e99000640 1 -- start start 2024-04-25T00:15:41.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.142+0000 7f3e99000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e9406cfd0 0x7f3e94070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.142+0000 7f3e99000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e940cb640 0x7f3e94165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.142+0000 7f3e99000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e9406e160 con 0x7f3e9406d3b0 2024-04-25T00:15:41.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.142+0000 7f3e99000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3e9406df20 con 0x7f3e9406cfd0 2024-04-25T00:15:41.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.142+0000 7f3e99000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3e9406ea10 con 0x7f3e940cb640 2024-04-25T00:15:41.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.142+0000 7f3e91e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e9406cfd0 0x7f3e94070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.142+0000 7f3e92800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e940cb640 0x7f3e94165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.142+0000 7f3e89400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3e9406d3b0 0x7f3e9406d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.83:36232/0 (socket says 172.21.15.83:36232) 2024-04-25T00:15:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.142+0000 7f3e91e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e9406cfd0 0x7f3e94070d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.83:57272/0 (socket says 172.21.15.83:57272) 2024-04-25T00:15:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.142+0000 7f3e89400640 1 -- 172.21.15.83:0/2426088485 learned_addr learned my addr 172.21.15.83:0/2426088485 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.142+0000 7f3e92800640 1 -- 172.21.15.83:0/2426088485 >> v2:172.21.15.86:3300/0 conn(0x7f3e9406cfd0 msgr2=0x7f3e94070d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.142+0000 7f3e92800640 1 --2- 172.21.15.83:0/2426088485 >> v2:172.21.15.86:3300/0 conn(0x7f3e9406cfd0 0x7f3e94070d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.142+0000 7f3e92800640 1 -- 172.21.15.83:0/2426088485 >> v1:172.21.15.86:6789/0 conn(0x7f3e9406d3b0 legacy=0x7f3e9406d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.142+0000 7f3e92800640 1 -- 172.21.15.83:0/2426088485 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9406ecf0 con 0x7f3e940cb640 2024-04-25T00:15:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.146+0000 7f3e92800640 1 --2- 172.21.15.83:0/2426088485 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e940cb640 0x7f3e94165930 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009d90 tx=0x7f3e8c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e1440986eea0c45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.146+0000 7f3e91400640 1 -- 172.21.15.83:0/2426088485 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0b2070 con 0x7f3e940cb640 2024-04-25T00:15:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.146+0000 7f3e91400640 1 -- 172.21.15.83:0/2426088485 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e8c0a5830 con 0x7f3e940cb640 2024-04-25T00:15:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.146+0000 7f3e91400640 1 -- 172.21.15.83:0/2426088485 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0a5c00 con 0x7f3e940cb640 2024-04-25T00:15:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.146+0000 7f3e99000640 1 -- 172.21.15.83:0/2426088485 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e940cb640 msgr2=0x7f3e94165930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.146+0000 7f3e99000640 1 --2- 172.21.15.83:0/2426088485 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e940cb640 0x7f3e94165930 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009d90 tx=0x7f3e8c01c970 comp rx=0 tx=0).stop 2024-04-25T00:15:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.146+0000 7f3e99000640 1 -- 172.21.15.83:0/2426088485 shutdown_connections 2024-04-25T00:15:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.146+0000 7f3e99000640 1 --2- 172.21.15.83:0/2426088485 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e940cb640 0x7f3e94165930 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.146+0000 7f3e99000640 1 --2- 172.21.15.83:0/2426088485 >> v2:172.21.15.86:3300/0 conn(0x7f3e9406cfd0 0x7f3e94070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.146+0000 7f3e99000640 1 -- 172.21.15.83:0/2426088485 >> 172.21.15.83:0/2426088485 conn(0x7f3e940cfa60 msgr2=0x7f3e940cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.146+0000 7f3e99000640 1 -- 172.21.15.83:0/2426088485 shutdown_connections 2024-04-25T00:15:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.146+0000 7f3e99000640 1 -- 172.21.15.83:0/2426088485 wait complete. 2024-04-25T00:15:41.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.146+0000 7f3e99000640 1 Processor -- start 2024-04-25T00:15:41.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.146+0000 7f3e99000640 1 -- start start 2024-04-25T00:15:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e99000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e9406cfd0 0x7f3e940cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e99000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9406d3b0 0x7f3e940b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e99000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e940cb640 0x7f3e9415b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e99000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e941688a0 con 0x7f3e9406cfd0 2024-04-25T00:15:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e99000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e94145970 con 0x7f3e9406d3b0 2024-04-25T00:15:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e99000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3e94168a20 con 0x7f3e940cb640 2024-04-25T00:15:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e91e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e9406cfd0 0x7f3e940cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e91e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e9406cfd0 0x7f3e940cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60344/0 (socket says 172.21.15.83:60344) 2024-04-25T00:15:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e91e00640 1 -- 172.21.15.83:0/103090391 learned_addr learned my addr 172.21.15.83:0/103090391 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e92800640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e940cb640 0x7f3e9415b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e89400640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9406d3b0 0x7f3e940b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e91e00640 1 -- 172.21.15.83:0/103090391 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e940cb640 msgr2=0x7f3e9415b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e91e00640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e940cb640 0x7f3e9415b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e91e00640 1 -- 172.21.15.83:0/103090391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9406d3b0 msgr2=0x7f3e940b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e91e00640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9406d3b0 0x7f3e940b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e91e00640 1 -- 172.21.15.83:0/103090391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e940b45f0 con 0x7f3e9406cfd0 2024-04-25T00:15:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e92800640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e940cb640 0x7f3e9415b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e89400640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9406d3b0 0x7f3e940b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e91e00640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e9406cfd0 0x7f3e940cf2b0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400e960 tx=0x7f3e8400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e8aa00640 1 -- 172.21.15.83:0/103090391 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e84017070 con 0x7f3e9406cfd0 2024-04-25T00:15:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e8aa00640 1 -- 172.21.15.83:0/103090391 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e8400ca00 con 0x7f3e9406cfd0 2024-04-25T00:15:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e8aa00640 1 -- 172.21.15.83:0/103090391 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8400cdb0 con 0x7f3e9406cfd0 2024-04-25T00:15:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e99000640 1 -- 172.21.15.83:0/103090391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e940b48d0 con 0x7f3e9406cfd0 2024-04-25T00:15:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e99000640 1 -- 172.21.15.83:0/103090391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e940b4e90 con 0x7f3e9406cfd0 2024-04-25T00:15:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e99000640 1 -- 172.21.15.83:0/103090391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e94068230 con 0x7f3e9406cfd0 2024-04-25T00:15:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.150+0000 7f3e8aa00640 1 -- 172.21.15.83:0/103090391 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3e8401a070 con 0x7f3e9406cfd0 2024-04-25T00:15:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.154+0000 7f3e8aa00640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e700678f0 0x7f3e70069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.154+0000 7f3e8aa00640 1 -- 172.21.15.83:0/103090391 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f3e84086110 con 0x7f3e9406cfd0 2024-04-25T00:15:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.154+0000 7f3e89400640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e700678f0 0x7f3e70069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.154+0000 7f3e89400640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e700678f0 0x7f3e70069db0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3e9413af40 tx=0x7f3e7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.158+0000 7f3e8aa00640 1 -- 172.21.15.83:0/103090391 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3e84056b90 con 0x7f3e9406cfd0 2024-04-25T00:15:41.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.314+0000 7f3e99000640 1 -- 172.21.15.83:0/103090391 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f3e940b2ea0 con 0x7f3e700678f0 2024-04-25T00:15:41.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.314+0000 7f3e8aa00640 1 -- 172.21.15.83:0/103090391 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3e940b2ea0 con 0x7f3e700678f0 2024-04-25T00:15:41.314 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T00:15:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.314+0000 7f3e99000640 1 -- 172.21.15.83:0/103090391 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e700678f0 msgr2=0x7f3e70069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.314+0000 7f3e99000640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e700678f0 0x7f3e70069db0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3e9413af40 tx=0x7f3e7c009040 comp rx=0 tx=0).stop 2024-04-25T00:15:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.314+0000 7f3e99000640 1 -- 172.21.15.83:0/103090391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e9406cfd0 msgr2=0x7f3e940cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.314+0000 7f3e99000640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e9406cfd0 0x7f3e940cf2b0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400e960 tx=0x7f3e8400ee20 comp rx=0 tx=0).stop 2024-04-25T00:15:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.314+0000 7f3e99000640 1 -- 172.21.15.83:0/103090391 shutdown_connections 2024-04-25T00:15:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.314+0000 7f3e99000640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f3e700678f0 0x7f3e70069db0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.314+0000 7f3e99000640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e940cb640 0x7f3e9415b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.314+0000 7f3e99000640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9406d3b0 0x7f3e940b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.314+0000 7f3e99000640 1 --2- 172.21.15.83:0/103090391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e9406cfd0 0x7f3e940cf2b0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.314+0000 7f3e99000640 1 -- 172.21.15.83:0/103090391 >> 172.21.15.83:0/103090391 conn(0x7f3e940cfa60 msgr2=0x7f3e940725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.314+0000 7f3e99000640 1 -- 172.21.15.83:0/103090391 shutdown_connections 2024-04-25T00:15:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.314+0000 7f3e99000640 1 -- 172.21.15.83:0/103090391 wait complete. 2024-04-25T00:15:41.334 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T00:15:41.334 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:15:41.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb4200640 1 Processor -- start 2024-04-25T00:15:41.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb4200640 1 -- start start 2024-04-25T00:15:41.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb4200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfac0ce080 0x7fbfac0cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb4200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbfac0d0280 0x7fbfac0b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb4200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfac0ce610 con 0x7fbfac1455d0 2024-04-25T00:15:41.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb4200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbfac0d06f0 con 0x7fbfac0d0280 2024-04-25T00:15:41.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb4200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbfac0ce8f0 con 0x7fbfac0ce080 2024-04-25T00:15:41.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb2200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbfac0d0280 0x7fbfac0b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb2200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbfac0d0280 0x7fbfac0b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60346/0 (socket says 172.21.15.83:60346) 2024-04-25T00:15:41.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb2200640 1 -- 172.21.15.83:0/327879213 learned_addr learned my addr 172.21.15.83:0/327879213 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:41.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb0e00640 1 --2- 172.21.15.83:0/327879213 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfac0ce080 0x7fbfac0cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb2200640 1 -- 172.21.15.83:0/327879213 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfac0ce080 msgr2=0x7fbfac0cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:41.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb2200640 1 --2- 172.21.15.83:0/327879213 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfac0ce080 0x7fbfac0cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb2200640 1 -- 172.21.15.83:0/327879213 >> v1:172.21.15.86:6789/0 conn(0x7fbfac1455d0 legacy=0x7fbfac06d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.474+0000 7fbfb2200640 1 -- 172.21.15.83:0/327879213 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfac0cebd0 con 0x7fbfac0d0280 2024-04-25T00:15:41.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.478+0000 7fbfb2200640 1 --2- 172.21.15.83:0/327879213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac0d0280 0x7fbfac0b2b30 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009870 tx=0x7fbfa401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cecf3993d26669b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:41.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.478+0000 7fbfabe00640 1 -- 172.21.15.83:0/327879213 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40a9620 con 0x7fbfac0d0280 2024-04-25T00:15:41.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.478+0000 7fbfabe00640 1 -- 172.21.15.83:0/327879213 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa40a97c0 con 0x7fbfac0d0280 2024-04-25T00:15:41.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.478+0000 7fbfabe00640 1 -- 172.21.15.83:0/327879213 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40a9b90 con 0x7fbfac0d0280 2024-04-25T00:15:41.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.478+0000 7fbfb4200640 1 -- 172.21.15.83:0/327879213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac0d0280 msgr2=0x7fbfac0b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.478+0000 7fbfb4200640 1 --2- 172.21.15.83:0/327879213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac0d0280 0x7fbfac0b2b30 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009870 tx=0x7fbfa401c560 comp rx=0 tx=0).stop 2024-04-25T00:15:41.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.478+0000 7fbfb4200640 1 -- 172.21.15.83:0/327879213 shutdown_connections 2024-04-25T00:15:41.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.478+0000 7fbfb4200640 1 --2- 172.21.15.83:0/327879213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac0d0280 0x7fbfac0b2b30 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.478+0000 7fbfb4200640 1 --2- 172.21.15.83:0/327879213 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfac0ce080 0x7fbfac0cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.478+0000 7fbfb4200640 1 -- 172.21.15.83:0/327879213 >> 172.21.15.83:0/327879213 conn(0x7fbfac06b3b0 msgr2=0x7fbfac06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:41.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.478+0000 7fbfb4200640 1 -- 172.21.15.83:0/327879213 shutdown_connections 2024-04-25T00:15:41.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.478+0000 7fbfb4200640 1 -- 172.21.15.83:0/327879213 wait complete. 2024-04-25T00:15:41.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb4200640 1 Processor -- start 2024-04-25T00:15:41.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb4200640 1 -- start start 2024-04-25T00:15:41.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb4200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfac0ce080 0x7fbfac165770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb4200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac0d0280 0x7fbfac13a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb4200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfac1455d0 0x7fbfac138990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb4200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac0b3a00 con 0x7fbfac0d0280 2024-04-25T00:15:41.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb4200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac0b3880 con 0x7fbfac0ce080 2024-04-25T00:15:41.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb4200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbfac0b3b80 con 0x7fbfac1455d0 2024-04-25T00:15:41.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac0d0280 0x7fbfac13a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac0d0280 0x7fbfac13a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60362/0 (socket says 172.21.15.83:60362) 2024-04-25T00:15:41.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb0e00640 1 -- 172.21.15.83:0/3868572440 learned_addr learned my addr 172.21.15.83:0/3868572440 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:41.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb2200640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfac1455d0 0x7fbfac138990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb1800640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfac0ce080 0x7fbfac165770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb0e00640 1 -- 172.21.15.83:0/3868572440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfac1455d0 msgr2=0x7fbfac138990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb0e00640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfac1455d0 0x7fbfac138990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb0e00640 1 -- 172.21.15.83:0/3868572440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfac0ce080 msgr2=0x7fbfac165770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb0e00640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfac0ce080 0x7fbfac165770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb0e00640 1 -- 172.21.15.83:0/3868572440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfac166030 con 0x7fbfac0d0280 2024-04-25T00:15:41.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb2200640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfac1455d0 0x7fbfac138990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:41.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb0e00640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac0d0280 0x7fbfac13a570 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800d800 tx=0x7fbf9800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:41.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfa9600640 1 -- 172.21.15.83:0/3868572440 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800b6c0 con 0x7fbfac0d0280 2024-04-25T00:15:41.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfa9600640 1 -- 172.21.15.83:0/3868572440 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf98019070 con 0x7fbfac0d0280 2024-04-25T00:15:41.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfa9600640 1 -- 172.21.15.83:0/3868572440 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800ba60 con 0x7fbfac0d0280 2024-04-25T00:15:41.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb4200640 1 -- 172.21.15.83:0/3868572440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfac166310 con 0x7fbfac0d0280 2024-04-25T00:15:41.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb4200640 1 -- 172.21.15.83:0/3868572440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfac1668d0 con 0x7fbfac0d0280 2024-04-25T00:15:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.482+0000 7fbfb4200640 1 -- 172.21.15.83:0/3868572440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbfac0736f0 con 0x7fbfac0d0280 2024-04-25T00:15:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.486+0000 7fbfa9600640 1 -- 172.21.15.83:0/3868572440 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbf9800bc00 con 0x7fbfac0d0280 2024-04-25T00:15:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.486+0000 7fbfa9600640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbf8c0678c0 0x7fbf8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.486+0000 7fbfa9600640 1 -- 172.21.15.83:0/3868572440 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7fbf980860f0 con 0x7fbfac0d0280 2024-04-25T00:15:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.486+0000 7fbfb1800640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbf8c0678c0 0x7fbf8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.486+0000 7fbfa9600640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.86:6800/3882786818,v1:172.21.15.86:6801/3882786818] conn(0x7fbf8c06df40 0x7fbf8c070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.486+0000 7fbfa9600640 1 -- 172.21.15.83:0/3868572440 --> [v2:172.21.15.86:6800/3882786818,v1:172.21.15.86:6801/3882786818] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf8c070a50 con 0x7fbf8c06df40 2024-04-25T00:15:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.486+0000 7fbfa9600640 1 -- 172.21.15.83:0/3868572440 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fbf9808d050 con 0x7fbfac0d0280 2024-04-25T00:15:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.486+0000 7fbfb2200640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.86:6800/3882786818,v1:172.21.15.86:6801/3882786818] conn(0x7fbf8c06df40 0x7fbf8c070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.486+0000 7fbfb1800640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbf8c0678c0 0x7fbf8c069d80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0001cb0 tx=0x7fbfa0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.486+0000 7fbfb2200640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.86:6800/3882786818,v1:172.21.15.86:6801/3882786818] conn(0x7fbf8c06df40 0x7fbf8c070380 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:41.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.486+0000 7fbfa9600640 1 -- 172.21.15.83:0/3868572440 <== osd.6 v2:172.21.15.86:6800/3882786818 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf8c070a50 con 0x7fbf8c06df40 2024-04-25T00:15:41.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.502+0000 7fbfb4200640 1 -- 172.21.15.83:0/3868572440 --> [v2:172.21.15.86:6800/3882786818,v1:172.21.15.86:6801/3882786818] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbfac04f5f0 con 0x7fbf8c06df40 2024-04-25T00:15:41.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.502+0000 7fbfa9600640 1 -- 172.21.15.83:0/3868572440 <== osd.6 v2:172.21.15.86:6800/3882786818 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbfac04f5f0 con 0x7fbf8c06df40 2024-04-25T00:15:41.503 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:15:41.503 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:15:41.503 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:15:41.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.502+0000 7fbfb4200640 1 -- 172.21.15.83:0/3868572440 >> [v2:172.21.15.86:6800/3882786818,v1:172.21.15.86:6801/3882786818] conn(0x7fbf8c06df40 msgr2=0x7fbf8c070380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.502+0000 7fbfb4200640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.86:6800/3882786818,v1:172.21.15.86:6801/3882786818] conn(0x7fbf8c06df40 0x7fbf8c070380 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.502+0000 7fbfb4200640 1 -- 172.21.15.83:0/3868572440 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbf8c0678c0 msgr2=0x7fbf8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.502+0000 7fbfb4200640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fbf8c0678c0 0x7fbf8c069d80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0001cb0 tx=0x7fbfa0009040 comp rx=0 tx=0).stop 2024-04-25T00:15:41.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.502+0000 7fbfb4200640 1 -- 172.21.15.83:0/3868572440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac0d0280 msgr2=0x7fbfac13a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.502+0000 7fbfb4200640 1 --2- 172.21.15.83:0/3868572440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac0d0280 0x7fbfac13a570 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800d800 tx=0x7fbf9800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.502+0000 7fbfb2200640 1 -- 172.21.15.83:0/3868572440 reap_dead start 2024-04-25T00:15:41.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.502+0000 7fbfb4200640 1 -- 172.21.15.83:0/3868572440 shutdown_connections 2024-04-25T00:15:41.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.502+0000 7fbfb4200640 1 -- 172.21.15.83:0/3868572440 >> 172.21.15.83:0/3868572440 conn(0x7fbfac06b3b0 msgr2=0x7fbfac0b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:41.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.502+0000 7fbfb4200640 1 -- 172.21.15.83:0/3868572440 shutdown_connections 2024-04-25T00:15:41.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.502+0000 7fbfb4200640 1 -- 172.21.15.83:0/3868572440 wait complete. 2024-04-25T00:15:41.521 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T00:15:41.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb552a00640 1 Processor -- start 2024-04-25T00:15:41.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb552a00640 1 -- start start 2024-04-25T00:15:41.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb552a00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb54c1446e0 0x7fb54c0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb552a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb54c06ee20 0x7fb54c071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb552a00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb54c0cb210 con 0x7fb54c15f350 2024-04-25T00:15:41.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb552a00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb54c0715b0 con 0x7fb54c1446e0 2024-04-25T00:15:41.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb552a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb54c144ac0 con 0x7fb54c06ee20 2024-04-25T00:15:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb543400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb54c15f350 0x7fb54c06e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35886/0 (socket says 172.21.15.83:35886) 2024-04-25T00:15:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb543400640 1 -- 172.21.15.83:0/2311503995 learned_addr learned my addr 172.21.15.83:0/2311503995 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb54be00640 1 --2- 172.21.15.83:0/2311503995 >> v2:172.21.15.83:3300/0 conn(0x7fb54c1446e0 0x7fb54c0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb550a00640 1 --2- 172.21.15.83:0/2311503995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb54c06ee20 0x7fb54c071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb54be00640 1 -- 172.21.15.83:0/2311503995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb54c06ee20 msgr2=0x7fb54c071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb54be00640 1 --2- 172.21.15.83:0/2311503995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb54c06ee20 0x7fb54c071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb54be00640 1 -- 172.21.15.83:0/2311503995 >> v1:172.21.15.83:6789/0 conn(0x7fb54c15f350 legacy=0x7fb54c06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb54be00640 1 -- 172.21.15.83:0/2311503995 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c0cb4f0 con 0x7fb54c1446e0 2024-04-25T00:15:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb54b400640 1 -- 172.21.15.83:0/2311503995 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2055381390 0 0) 0x7fb54c0cb210 con 0x7fb54c15f350 2024-04-25T00:15:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.658+0000 7fb550a00640 1 --2- 172.21.15.83:0/2311503995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb54c06ee20 0x7fb54c071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.662+0000 7fb54be00640 1 --2- 172.21.15.83:0/2311503995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c1446e0 0x7fb54c0cab50 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0097c0 tx=0x7fb53c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fc69eded2d2e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.662+0000 7fb54b400640 1 -- 172.21.15.83:0/2311503995 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0b0070 con 0x7fb54c1446e0 2024-04-25T00:15:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.662+0000 7fb54b400640 1 -- 172.21.15.83:0/2311503995 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb53c0ab600 con 0x7fb54c1446e0 2024-04-25T00:15:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.662+0000 7fb54b400640 1 -- 172.21.15.83:0/2311503995 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0ab9d0 con 0x7fb54c1446e0 2024-04-25T00:15:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.662+0000 7fb552a00640 1 -- 172.21.15.83:0/2311503995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c1446e0 msgr2=0x7fb54c0cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.662+0000 7fb552a00640 1 --2- 172.21.15.83:0/2311503995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c1446e0 0x7fb54c0cab50 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0097c0 tx=0x7fb53c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.662+0000 7fb552a00640 1 -- 172.21.15.83:0/2311503995 shutdown_connections 2024-04-25T00:15:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.662+0000 7fb552a00640 1 --2- 172.21.15.83:0/2311503995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb54c06ee20 0x7fb54c071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.662+0000 7fb552a00640 1 --2- 172.21.15.83:0/2311503995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c1446e0 0x7fb54c0cab50 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.662+0000 7fb552a00640 1 -- 172.21.15.83:0/2311503995 >> 172.21.15.83:0/2311503995 conn(0x7fb54c0cfa60 msgr2=0x7fb54c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.662+0000 7fb552a00640 1 -- 172.21.15.83:0/2311503995 shutdown_connections 2024-04-25T00:15:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.662+0000 7fb552a00640 1 -- 172.21.15.83:0/2311503995 wait complete. 2024-04-25T00:15:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.662+0000 7fb552a00640 1 Processor -- start 2024-04-25T00:15:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.662+0000 7fb552a00640 1 -- start start 2024-04-25T00:15:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb552a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb54c06ee20 0x7fb54c15b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb552a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb54c1446e0 0x7fb54c163e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb552a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c15f350 0x7fb54c164380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb552a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c072eb0 con 0x7fb54c15f350 2024-04-25T00:15:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb552a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c072d30 con 0x7fb54c1446e0 2024-04-25T00:15:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb552a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb54c073030 con 0x7fb54c06ee20 2024-04-25T00:15:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb550a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c15f350 0x7fb54c164380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb550a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c15f350 0x7fb54c164380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60376/0 (socket says 172.21.15.83:60376) 2024-04-25T00:15:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb550a00640 1 -- 172.21.15.83:0/684203256 learned_addr learned my addr 172.21.15.83:0/684203256 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb54be00640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb54c06ee20 0x7fb54c15b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb543400640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb54c1446e0 0x7fb54c163e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb550a00640 1 -- 172.21.15.83:0/684203256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb54c06ee20 msgr2=0x7fb54c15b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb550a00640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb54c06ee20 0x7fb54c15b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb550a00640 1 -- 172.21.15.83:0/684203256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb54c1446e0 msgr2=0x7fb54c163e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb550a00640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb54c1446e0 0x7fb54c163e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb550a00640 1 -- 172.21.15.83:0/684203256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c0b3380 con 0x7fb54c15f350 2024-04-25T00:15:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb543400640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb54c1446e0 0x7fb54c163e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb550a00640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c15f350 0x7fb54c164380 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fb54400cef0 tx=0x7fb54400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb548c00640 1 -- 172.21.15.83:0/684203256 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54401a070 con 0x7fb54c15f350 2024-04-25T00:15:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb548c00640 1 -- 172.21.15.83:0/684203256 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb54400d860 con 0x7fb54c15f350 2024-04-25T00:15:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb552a00640 1 -- 172.21.15.83:0/684203256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54c0b3660 con 0x7fb54c15f350 2024-04-25T00:15:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb548c00640 1 -- 172.21.15.83:0/684203256 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54400dc50 con 0x7fb54c15f350 2024-04-25T00:15:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb552a00640 1 -- 172.21.15.83:0/684203256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54c13d000 con 0x7fb54c15f350 2024-04-25T00:15:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb552a00640 1 -- 172.21.15.83:0/684203256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb54c068230 con 0x7fb54c15f350 2024-04-25T00:15:41.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.666+0000 7fb548c00640 1 -- 172.21.15.83:0/684203256 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb54401d070 con 0x7fb54c15f350 2024-04-25T00:15:41.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.670+0000 7fb548c00640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb5280678c0 0x7fb528069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.670+0000 7fb548c00640 1 -- 172.21.15.83:0/684203256 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7fb544086470 con 0x7fb54c15f350 2024-04-25T00:15:41.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.670+0000 7fb54be00640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb5280678c0 0x7fb528069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.670+0000 7fb54be00640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb5280678c0 0x7fb528069d80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0ad000 tx=0x7fb53c0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.674+0000 7fb548c00640 1 -- 172.21.15.83:0/684203256 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb5440569f0 con 0x7fb54c15f350 2024-04-25T00:15:41.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.830+0000 7fb552a00640 1 -- 172.21.15.83:0/684203256 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fb54c06e820 con 0x7fb5280678c0 2024-04-25T00:15:41.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.830+0000 7fb548c00640 1 -- 172.21.15.83:0/684203256 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb54c06e820 con 0x7fb5280678c0 2024-04-25T00:15:41.831 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T00:15:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.834+0000 7fb552a00640 1 -- 172.21.15.83:0/684203256 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb5280678c0 msgr2=0x7fb528069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.834+0000 7fb552a00640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb5280678c0 0x7fb528069d80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0ad000 tx=0x7fb53c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:15:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.834+0000 7fb552a00640 1 -- 172.21.15.83:0/684203256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c15f350 msgr2=0x7fb54c164380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.834+0000 7fb552a00640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c15f350 0x7fb54c164380 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fb54400cef0 tx=0x7fb54400a710 comp rx=0 tx=0).stop 2024-04-25T00:15:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.834+0000 7fb552a00640 1 -- 172.21.15.83:0/684203256 shutdown_connections 2024-04-25T00:15:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.834+0000 7fb552a00640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb5280678c0 0x7fb528069d80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.834+0000 7fb552a00640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c15f350 0x7fb54c164380 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.834+0000 7fb552a00640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb54c1446e0 0x7fb54c163e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.834+0000 7fb552a00640 1 --2- 172.21.15.83:0/684203256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb54c06ee20 0x7fb54c15b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.834+0000 7fb552a00640 1 -- 172.21.15.83:0/684203256 >> 172.21.15.83:0/684203256 conn(0x7fb54c0cfa60 msgr2=0x7fb54c06d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.834+0000 7fb552a00640 1 -- 172.21.15.83:0/684203256 shutdown_connections 2024-04-25T00:15:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.834+0000 7fb552a00640 1 -- 172.21.15.83:0/684203256 wait complete. 2024-04-25T00:15:41.852 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T00:15:41.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:15:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.990+0000 7f294a000640 1 Processor -- start 2024-04-25T00:15:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.990+0000 7f294a000640 1 -- start start 2024-04-25T00:15:41.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f294a000640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f29440cec20 0x7f29440cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f294a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f29440b6da0 0x7f29440b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f294a000640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29440d0fe0 con 0x7f294415e0b0 2024-04-25T00:15:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f294a000640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f29440d2970 con 0x7f29440cec20 2024-04-25T00:15:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f294a000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f29440ce150 con 0x7f29440b6da0 2024-04-25T00:15:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f2943400640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f294415e0b0 0x7f29440ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35900/0 (socket says 172.21.15.83:35900) 2024-04-25T00:15:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f2943400640 1 -- 172.21.15.83:0/749008216 learned_addr learned my addr 172.21.15.83:0/749008216 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f2942a00640 1 --2- 172.21.15.83:0/749008216 >> v2:172.21.15.83:3300/0 conn(0x7f29440cec20 0x7f29440cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f2943e00640 1 --2- 172.21.15.83:0/749008216 >> v2:172.21.15.86:3300/0 conn(0x7f29440b6da0 0x7f29440b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f2942000640 1 -- 172.21.15.83:0/749008216 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4032677910 0 0) 0x7f29440d0fe0 con 0x7f294415e0b0 2024-04-25T00:15:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f2942000640 1 -- 172.21.15.83:0/749008216 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2928003530 con 0x7f294415e0b0 2024-04-25T00:15:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f2942a00640 1 -- 172.21.15.83:0/749008216 >> v2:172.21.15.86:3300/0 conn(0x7f29440b6da0 msgr2=0x7f29440b71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f2942a00640 1 --2- 172.21.15.83:0/749008216 >> v2:172.21.15.86:3300/0 conn(0x7f29440b6da0 0x7f29440b71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f2942a00640 1 -- 172.21.15.83:0/749008216 >> v1:172.21.15.83:6789/0 conn(0x7f294415e0b0 legacy=0x7f29440ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.994+0000 7f2942a00640 1 -- 172.21.15.83:0/749008216 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29440cc380 con 0x7f29440cec20 2024-04-25T00:15:41.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.998+0000 7f2942a00640 1 --2- 172.21.15.83:0/749008216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29440cec20 0x7f29440cffc0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f2944161c90 tx=0x7f293001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c0074e353fa442 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.998+0000 7f2942000640 1 -- 172.21.15.83:0/749008216 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29300ab650 con 0x7f29440cec20 2024-04-25T00:15:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.998+0000 7f2942000640 1 -- 172.21.15.83:0/749008216 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29300b0070 con 0x7f29440cec20 2024-04-25T00:15:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.998+0000 7f2942000640 1 -- 172.21.15.83:0/749008216 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29300ab9f0 con 0x7f29440cec20 2024-04-25T00:15:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.998+0000 7f294a000640 1 -- 172.21.15.83:0/749008216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29440cec20 msgr2=0x7f29440cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.998+0000 7f294a000640 1 --2- 172.21.15.83:0/749008216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29440cec20 0x7f29440cffc0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f2944161c90 tx=0x7f293001c450 comp rx=0 tx=0).stop 2024-04-25T00:15:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.998+0000 7f294a000640 1 -- 172.21.15.83:0/749008216 shutdown_connections 2024-04-25T00:15:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.998+0000 7f294a000640 1 --2- 172.21.15.83:0/749008216 >> v2:172.21.15.86:3300/0 conn(0x7f29440b6da0 0x7f29440b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.998+0000 7f294a000640 1 --2- 172.21.15.83:0/749008216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29440cec20 0x7f29440cffc0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.998+0000 7f294a000640 1 -- 172.21.15.83:0/749008216 >> 172.21.15.83:0/749008216 conn(0x7f29440d1c90 msgr2=0x7f29440d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.998+0000 7f294a000640 1 -- 172.21.15.83:0/749008216 shutdown_connections 2024-04-25T00:15:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.998+0000 7f294a000640 1 -- 172.21.15.83:0/749008216 wait complete. 2024-04-25T00:15:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.998+0000 7f294a000640 1 Processor -- start 2024-04-25T00:15:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:41.998+0000 7f294a000640 1 -- start start 2024-04-25T00:15:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f294a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29440b6da0 0x7f2944157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f294a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29440cec20 0x7f2944160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f294a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294415e0b0 0x7f2944146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f294a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29440b4ae0 con 0x7f29440b6da0 2024-04-25T00:15:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f294a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29440ccc20 con 0x7f294415e0b0 2024-04-25T00:15:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f294a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f29440d12c0 con 0x7f29440cec20 2024-04-25T00:15:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f2943400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29440b6da0 0x7f2944157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f2943400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29440b6da0 0x7f2944157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60392/0 (socket says 172.21.15.83:60392) 2024-04-25T00:15:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f2943400640 1 -- 172.21.15.83:0/1811993254 learned_addr learned my addr 172.21.15.83:0/1811993254 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f2942a00640 1 --2- 172.21.15.83:0/1811993254 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29440cec20 0x7f2944160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f2943e00640 1 --2- 172.21.15.83:0/1811993254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294415e0b0 0x7f2944146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f2943400640 1 -- 172.21.15.83:0/1811993254 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29440cec20 msgr2=0x7f2944160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f2943400640 1 --2- 172.21.15.83:0/1811993254 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29440cec20 0x7f2944160cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f2943400640 1 -- 172.21.15.83:0/1811993254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294415e0b0 msgr2=0x7f2944146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f2943400640 1 --2- 172.21.15.83:0/1811993254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294415e0b0 0x7f2944146530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f2943400640 1 -- 172.21.15.83:0/1811993254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2944143e20 con 0x7f29440b6da0 2024-04-25T00:15:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f2943400640 1 --2- 172.21.15.83:0/1811993254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29440b6da0 0x7f2944157f30 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f293400c460 tx=0x7f293400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f293b400640 1 -- 172.21.15.83:0/1811993254 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934017070 con 0x7f29440b6da0 2024-04-25T00:15:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f293b400640 1 -- 172.21.15.83:0/1811993254 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2934005d10 con 0x7f29440b6da0 2024-04-25T00:15:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f294a000640 1 -- 172.21.15.83:0/1811993254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2944144100 con 0x7f29440b6da0 2024-04-25T00:15:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f293b400640 1 -- 172.21.15.83:0/1811993254 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29340072b0 con 0x7f29440b6da0 2024-04-25T00:15:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f294a000640 1 -- 172.21.15.83:0/1811993254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29441446c0 con 0x7f29440b6da0 2024-04-25T00:15:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f294a000640 1 -- 172.21.15.83:0/1811993254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f294409fd60 con 0x7f29440b6da0 2024-04-25T00:15:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.002+0000 7f293b400640 1 -- 172.21.15.83:0/1811993254 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f293401a070 con 0x7f29440b6da0 2024-04-25T00:15:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.006+0000 7f293b400640 1 --2- 172.21.15.83:0/1811993254 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29240678c0 0x7f2924069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.006+0000 7f293b400640 1 -- 172.21.15.83:0/1811993254 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f2934084f90 con 0x7f29440b6da0 2024-04-25T00:15:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.006+0000 7f2942a00640 1 --2- 172.21.15.83:0/1811993254 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29240678c0 0x7f2924069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.006+0000 7f293b400640 1 --2- 172.21.15.83:0/1811993254 >> [v2:172.21.15.86:6820/957843207,v1:172.21.15.86:6822/957843207] conn(0x7f292406df40 0x7f2924070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.006+0000 7f293b400640 1 -- 172.21.15.83:0/1811993254 --> [v2:172.21.15.86:6820/957843207,v1:172.21.15.86:6822/957843207] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2934013e50 con 0x7f292406df40 2024-04-25T00:15:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.006+0000 7f293b400640 1 -- 172.21.15.83:0/1811993254 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f2934087550 con 0x7f29440b6da0 2024-04-25T00:15:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.006+0000 7f2943e00640 1 --2- 172.21.15.83:0/1811993254 >> [v2:172.21.15.86:6820/957843207,v1:172.21.15.86:6822/957843207] conn(0x7f292406df40 0x7f2924070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.006+0000 7f2942a00640 1 --2- 172.21.15.83:0/1811993254 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29240678c0 0x7f2924069d80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f29300af000 tx=0x7f29300a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.006+0000 7f2943e00640 1 --2- 172.21.15.83:0/1811993254 >> [v2:172.21.15.86:6820/957843207,v1:172.21.15.86:6822/957843207] conn(0x7f292406df40 0x7f2924070380 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.006+0000 7f293b400640 1 -- 172.21.15.83:0/1811993254 <== osd.7 v2:172.21.15.86:6820/957843207 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2934013e50 con 0x7f292406df40 2024-04-25T00:15:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.018+0000 7f294a000640 1 -- 172.21.15.83:0/1811993254 --> [v2:172.21.15.86:6820/957843207,v1:172.21.15.86:6822/957843207] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f294404f5f0 con 0x7f292406df40 2024-04-25T00:15:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.018+0000 7f293b400640 1 -- 172.21.15.83:0/1811993254 <== osd.7 v2:172.21.15.86:6820/957843207 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f294404f5f0 con 0x7f292406df40 2024-04-25T00:15:42.021 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T00:15:42.021 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:15:42.021 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T00:15:42.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.022+0000 7f294a000640 1 -- 172.21.15.83:0/1811993254 >> [v2:172.21.15.86:6820/957843207,v1:172.21.15.86:6822/957843207] conn(0x7f292406df40 msgr2=0x7f2924070380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.022+0000 7f294a000640 1 --2- 172.21.15.83:0/1811993254 >> [v2:172.21.15.86:6820/957843207,v1:172.21.15.86:6822/957843207] conn(0x7f292406df40 0x7f2924070380 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.022+0000 7f294a000640 1 -- 172.21.15.83:0/1811993254 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29240678c0 msgr2=0x7f2924069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.022+0000 7f294a000640 1 --2- 172.21.15.83:0/1811993254 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f29240678c0 0x7f2924069d80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f29300af000 tx=0x7f29300a7000 comp rx=0 tx=0).stop 2024-04-25T00:15:42.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.022+0000 7f294a000640 1 -- 172.21.15.83:0/1811993254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29440b6da0 msgr2=0x7f2944157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.022+0000 7f294a000640 1 --2- 172.21.15.83:0/1811993254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29440b6da0 0x7f2944157f30 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f293400c460 tx=0x7f293400c920 comp rx=0 tx=0).stop 2024-04-25T00:15:42.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.022+0000 7f2943e00640 1 -- 172.21.15.83:0/1811993254 reap_dead start 2024-04-25T00:15:42.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.022+0000 7f294a000640 1 -- 172.21.15.83:0/1811993254 shutdown_connections 2024-04-25T00:15:42.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.022+0000 7f294a000640 1 -- 172.21.15.83:0/1811993254 >> 172.21.15.83:0/1811993254 conn(0x7f29440d1c90 msgr2=0x7f29440cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:42.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.022+0000 7f294a000640 1 -- 172.21.15.83:0/1811993254 shutdown_connections 2024-04-25T00:15:42.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.022+0000 7f294a000640 1 -- 172.21.15.83:0/1811993254 wait complete. 2024-04-25T00:15:42.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T00:15:42.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f80ae800640 1 Processor -- start 2024-04-25T00:15:42.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f80ae800640 1 -- start start 2024-04-25T00:15:42.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f80ae800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a815e240 0x7f80a80ca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f80ae800640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f80a815e620 0x7f80a806d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f80ae800640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a80ca0b0 con 0x7f80a806db40 2024-04-25T00:15:42.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f80ae800640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f80a80718b0 con 0x7f80a815e620 2024-04-25T00:15:42.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f80ae800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f80a8071e70 con 0x7f80a815e240 2024-04-25T00:15:42.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f809fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f80a815e620 0x7f80a806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f80a7e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f80a806db40 0x7f80a806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35914/0 (socket says 172.21.15.83:35914) 2024-04-25T00:15:42.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f80a7e00640 1 -- 172.21.15.83:0/2887730990 learned_addr learned my addr 172.21.15.83:0/2887730990 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:42.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f80a7400640 1 --2- 172.21.15.83:0/2887730990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a815e240 0x7f80a80ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f809fe00640 1 -- 172.21.15.83:0/2887730990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a815e240 msgr2=0x7f80a80ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:42.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f809fe00640 1 --2- 172.21.15.83:0/2887730990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a815e240 0x7f80a80ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f809fe00640 1 -- 172.21.15.83:0/2887730990 >> v1:172.21.15.83:6789/0 conn(0x7f80a806db40 legacy=0x7f80a806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f809fe00640 1 -- 172.21.15.83:0/2887730990 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a80ca390 con 0x7f80a815e620 2024-04-25T00:15:42.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.174+0000 7f80a6a00640 1 -- 172.21.15.83:0/2887730990 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1577231340 0 0) 0x7f80a80ca0b0 con 0x7f80a806db40 2024-04-25T00:15:42.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.178+0000 7f809fe00640 1 --2- 172.21.15.83:0/2887730990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80a815e620 0x7f80a806d600 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f80900097c0 tx=0x7f809001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c08d5dbbffedb02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:42.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.178+0000 7f80a6a00640 1 -- 172.21.15.83:0/2887730990 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80900b5070 con 0x7f80a815e620 2024-04-25T00:15:42.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.178+0000 7f80a6a00640 1 -- 172.21.15.83:0/2887730990 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80900a8990 con 0x7f80a815e620 2024-04-25T00:15:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.178+0000 7f80a6a00640 1 -- 172.21.15.83:0/2887730990 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80900a8d60 con 0x7f80a815e620 2024-04-25T00:15:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.178+0000 7f80ae800640 1 -- 172.21.15.83:0/2887730990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80a815e620 msgr2=0x7f80a806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.178+0000 7f80ae800640 1 --2- 172.21.15.83:0/2887730990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80a815e620 0x7f80a806d600 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f80900097c0 tx=0x7f809001ec90 comp rx=0 tx=0).stop 2024-04-25T00:15:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.178+0000 7f80ae800640 1 -- 172.21.15.83:0/2887730990 shutdown_connections 2024-04-25T00:15:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.178+0000 7f80ae800640 1 --2- 172.21.15.83:0/2887730990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80a815e620 0x7f80a806d600 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.178+0000 7f80ae800640 1 --2- 172.21.15.83:0/2887730990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a815e240 0x7f80a80ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.178+0000 7f80ae800640 1 -- 172.21.15.83:0/2887730990 >> 172.21.15.83:0/2887730990 conn(0x7f80a80cfa60 msgr2=0x7f80a80cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.178+0000 7f80ae800640 1 -- 172.21.15.83:0/2887730990 shutdown_connections 2024-04-25T00:15:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.178+0000 7f80ae800640 1 -- 172.21.15.83:0/2887730990 wait complete. 2024-04-25T00:15:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80ae800640 1 Processor -- start 2024-04-25T00:15:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80ae800640 1 -- start start 2024-04-25T00:15:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80ae800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a806db40 0x7f80a8159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80ae800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a815e240 0x7f80a81f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80ae800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80a815e620 0x7f80a80c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80ae800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80a815ecd0 con 0x7f80a815e620 2024-04-25T00:15:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80ae800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80a8071b00 con 0x7f80a815e240 2024-04-25T00:15:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80ae800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f80a815ee50 con 0x7f80a806db40 2024-04-25T00:15:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80a815e620 0x7f80a80c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80a815e620 0x7f80a80c9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60412/0 (socket says 172.21.15.83:60412) 2024-04-25T00:15:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80a7e00640 1 -- 172.21.15.83:0/3865199406 learned_addr learned my addr 172.21.15.83:0/3865199406 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80a7400640 1 --2- 172.21.15.83:0/3865199406 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a806db40 0x7f80a8159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f809fe00640 1 --2- 172.21.15.83:0/3865199406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a815e240 0x7f80a81f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80a7e00640 1 -- 172.21.15.83:0/3865199406 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a806db40 msgr2=0x7f80a8159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80a7e00640 1 --2- 172.21.15.83:0/3865199406 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a806db40 0x7f80a8159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80a7e00640 1 -- 172.21.15.83:0/3865199406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a815e240 msgr2=0x7f80a81f3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80a7e00640 1 --2- 172.21.15.83:0/3865199406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a815e240 0x7f80a81f3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80a7e00640 1 -- 172.21.15.83:0/3865199406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a81f44b0 con 0x7f80a815e620 2024-04-25T00:15:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80a7e00640 1 --2- 172.21.15.83:0/3865199406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80a815e620 0x7f80a80c9010 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f80a000c7a0 tx=0x7f80a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f809f400640 1 -- 172.21.15.83:0/3865199406 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0018070 con 0x7f80a815e620 2024-04-25T00:15:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f809f400640 1 -- 172.21.15.83:0/3865199406 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a000a640 con 0x7f80a815e620 2024-04-25T00:15:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f809f400640 1 -- 172.21.15.83:0/3865199406 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a000aab0 con 0x7f80a815e620 2024-04-25T00:15:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80ae800640 1 -- 172.21.15.83:0/3865199406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a81f74f0 con 0x7f80a815e620 2024-04-25T00:15:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80ae800640 1 -- 172.21.15.83:0/3865199406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a81f4cf0 con 0x7f80a815e620 2024-04-25T00:15:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.182+0000 7f80ae800640 1 -- 172.21.15.83:0/3865199406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a8068230 con 0x7f80a815e620 2024-04-25T00:15:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.186+0000 7f809f400640 1 -- 172.21.15.83:0/3865199406 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f80a000ac50 con 0x7f80a815e620 2024-04-25T00:15:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.186+0000 7f809f400640 1 --2- 172.21.15.83:0/3865199406 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80840678c0 0x7f8084069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.186+0000 7f809f400640 1 -- 172.21.15.83:0/3865199406 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f80a0085290 con 0x7f80a815e620 2024-04-25T00:15:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.186+0000 7f80a7400640 1 --2- 172.21.15.83:0/3865199406 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80840678c0 0x7f8084069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.186+0000 7f80a7400640 1 --2- 172.21.15.83:0/3865199406 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80840678c0 0x7f8084069d80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8098009b60 tx=0x7f8098009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.190+0000 7f809f400640 1 -- 172.21.15.83:0/3865199406 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f80a81f44b0 con 0x7f80a815e620 2024-04-25T00:15:42.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.342+0000 7f80ae800640 1 -- 172.21.15.83:0/3865199406 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f80a80cd770 con 0x7f80840678c0 2024-04-25T00:15:42.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.346+0000 7f809f400640 1 -- 172.21.15.83:0/3865199406 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f80a80cd770 con 0x7f80840678c0 2024-04-25T00:15:42.345 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T00:15:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.346+0000 7f80ae800640 1 -- 172.21.15.83:0/3865199406 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80840678c0 msgr2=0x7f8084069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.346+0000 7f80ae800640 1 --2- 172.21.15.83:0/3865199406 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80840678c0 0x7f8084069d80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8098009b60 tx=0x7f8098009190 comp rx=0 tx=0).stop 2024-04-25T00:15:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.346+0000 7f80ae800640 1 -- 172.21.15.83:0/3865199406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80a815e620 msgr2=0x7f80a80c9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.346+0000 7f80ae800640 1 --2- 172.21.15.83:0/3865199406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80a815e620 0x7f80a80c9010 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f80a000c7a0 tx=0x7f80a000cc60 comp rx=0 tx=0).stop 2024-04-25T00:15:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.346+0000 7f80ae800640 1 -- 172.21.15.83:0/3865199406 shutdown_connections 2024-04-25T00:15:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.346+0000 7f80ae800640 1 --2- 172.21.15.83:0/3865199406 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f80840678c0 0x7f8084069d80 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.346+0000 7f80ae800640 1 --2- 172.21.15.83:0/3865199406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80a815e620 0x7f80a80c9010 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.346+0000 7f80ae800640 1 --2- 172.21.15.83:0/3865199406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a815e240 0x7f80a81f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.346+0000 7f80ae800640 1 --2- 172.21.15.83:0/3865199406 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a806db40 0x7f80a8159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.346+0000 7f80ae800640 1 -- 172.21.15.83:0/3865199406 >> 172.21.15.83:0/3865199406 conn(0x7f80a80cfa60 msgr2=0x7f80a80b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.346+0000 7f80ae800640 1 -- 172.21.15.83:0/3865199406 shutdown_connections 2024-04-25T00:15:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.346+0000 7f80ae800640 1 -- 172.21.15.83:0/3865199406 wait complete. 2024-04-25T00:15:42.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:15:42.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb809e00640 1 Processor -- start 2024-04-25T00:15:42.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb809e00640 1 -- start start 2024-04-25T00:15:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb809e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8040d0df0 0x7fb8040d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb809e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8040d28f0 0x7fb8040c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb809e00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8040d2d80 con 0x7fb80415e140 2024-04-25T00:15:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb809e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb8040d4c20 con 0x7fb8040d28f0 2024-04-25T00:15:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb809e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb8040c7920 con 0x7fb8040d0df0 2024-04-25T00:15:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb808a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb80415e140 0x7fb8040d1bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35928/0 (socket says 172.21.15.83:35928) 2024-04-25T00:15:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb808a00640 1 -- 172.21.15.83:0/3582623110 learned_addr learned my addr 172.21.15.83:0/3582623110 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:15:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb809400640 1 --2- 172.21.15.83:0/3582623110 >> v2:172.21.15.83:3300/0 conn(0x7fb8040d28f0 0x7fb8040c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb7fbe00640 1 --2- 172.21.15.83:0/3582623110 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8040d0df0 0x7fb8040d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb803e00640 1 -- 172.21.15.83:0/3582623110 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2639635642 0 0) 0x7fb8040d2d80 con 0x7fb80415e140 2024-04-25T00:15:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb803e00640 1 -- 172.21.15.83:0/3582623110 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7e8003530 con 0x7fb80415e140 2024-04-25T00:15:42.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb809400640 1 -- 172.21.15.83:0/3582623110 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8040d0df0 msgr2=0x7fb8040d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:42.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb809400640 1 --2- 172.21.15.83:0/3582623110 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8040d0df0 0x7fb8040d23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb809400640 1 -- 172.21.15.83:0/3582623110 >> v1:172.21.15.83:6789/0 conn(0x7fb80415e140 legacy=0x7fb8040d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.502+0000 7fb809400640 1 -- 172.21.15.83:0/3582623110 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8040d12d0 con 0x7fb8040d28f0 2024-04-25T00:15:42.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.506+0000 7fb809400640 1 --2- 172.21.15.83:0/3582623110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8040d28f0 0x7fb8040c8370 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009d90 tx=0x7fb7fc01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60247fd977faaa96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.506+0000 7fb803e00640 1 -- 172.21.15.83:0/3582623110 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0b2070 con 0x7fb8040d28f0 2024-04-25T00:15:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.506+0000 7fb803e00640 1 -- 172.21.15.83:0/3582623110 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7fc0a5970 con 0x7fb8040d28f0 2024-04-25T00:15:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.506+0000 7fb803e00640 1 -- 172.21.15.83:0/3582623110 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0a5d40 con 0x7fb8040d28f0 2024-04-25T00:15:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.506+0000 7fb809e00640 1 -- 172.21.15.83:0/3582623110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8040d28f0 msgr2=0x7fb8040c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.506+0000 7fb809e00640 1 --2- 172.21.15.83:0/3582623110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8040d28f0 0x7fb8040c8370 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009d90 tx=0x7fb7fc01c8b0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.506+0000 7fb809e00640 1 -- 172.21.15.83:0/3582623110 shutdown_connections 2024-04-25T00:15:42.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.506+0000 7fb809e00640 1 --2- 172.21.15.83:0/3582623110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8040d28f0 0x7fb8040c8370 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.506+0000 7fb809e00640 1 --2- 172.21.15.83:0/3582623110 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8040d0df0 0x7fb8040d23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.506+0000 7fb809e00640 1 -- 172.21.15.83:0/3582623110 >> 172.21.15.83:0/3582623110 conn(0x7fb8040d3dd0 msgr2=0x7fb8040d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:42.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.506+0000 7fb809e00640 1 -- 172.21.15.83:0/3582623110 shutdown_connections 2024-04-25T00:15:42.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.506+0000 7fb809e00640 1 -- 172.21.15.83:0/3582623110 wait complete. 2024-04-25T00:15:42.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809e00640 1 Processor -- start 2024-04-25T00:15:42.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809e00640 1 -- start start 2024-04-25T00:15:42.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8040d0df0 0x7fb804150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8040d28f0 0x7fb804153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb80415e140 0x7fb804159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb8040c90c0 con 0x7fb80415e140 2024-04-25T00:15:42.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb80415e520 con 0x7fb8040d0df0 2024-04-25T00:15:42.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb8040c9240 con 0x7fb8040d28f0 2024-04-25T00:15:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb80415e140 0x7fb804159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb80415e140 0x7fb804159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60436/0 (socket says 172.21.15.83:60436) 2024-04-25T00:15:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809400640 1 -- 172.21.15.83:0/3419812070 learned_addr learned my addr 172.21.15.83:0/3419812070 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:15:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb808a00640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8040d0df0 0x7fb804150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb7fbe00640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8040d28f0 0x7fb804153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809400640 1 -- 172.21.15.83:0/3419812070 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8040d28f0 msgr2=0x7fb804153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809400640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8040d28f0 0x7fb804153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809400640 1 -- 172.21.15.83:0/3419812070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8040d0df0 msgr2=0x7fb804150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809400640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8040d0df0 0x7fb804150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809400640 1 -- 172.21.15.83:0/3419812070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb804152020 con 0x7fb80415e140 2024-04-25T00:15:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb808a00640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8040d0df0 0x7fb804150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809400640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb80415e140 0x7fb804159330 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc01af60 tx=0x7fb7fc019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:42.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb801600640 1 -- 172.21.15.83:0/3419812070 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0bd020 con 0x7fb80415e140 2024-04-25T00:15:42.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb801600640 1 -- 172.21.15.83:0/3419812070 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7fc0b8670 con 0x7fb80415e140 2024-04-25T00:15:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb801600640 1 -- 172.21.15.83:0/3419812070 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0b2040 con 0x7fb80415e140 2024-04-25T00:15:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809e00640 1 -- 172.21.15.83:0/3419812070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8041522c0 con 0x7fb80415e140 2024-04-25T00:15:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809e00640 1 -- 172.21.15.83:0/3419812070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb804152880 con 0x7fb80415e140 2024-04-25T00:15:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.510+0000 7fb809e00640 1 -- 172.21.15.83:0/3419812070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb80404f510 con 0x7fb80415e140 2024-04-25T00:15:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.514+0000 7fb801600640 1 -- 172.21.15.83:0/3419812070 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb7fc0c0020 con 0x7fb80415e140 2024-04-25T00:15:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.514+0000 7fb801600640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb7e00678c0 0x7fb7e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.514+0000 7fb801600640 1 -- 172.21.15.83:0/3419812070 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7fb7fc120c50 con 0x7fb80415e140 2024-04-25T00:15:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.514+0000 7fb808a00640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb7e00678c0 0x7fb7e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.514+0000 7fb808a00640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb7e00678c0 0x7fb7e0069d80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4005890 tx=0x7fb7f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:42.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.518+0000 7fb801600640 1 -- 172.21.15.83:0/3419812070 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb7fc0c7030 con 0x7fb80415e140 2024-04-25T00:15:42.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.670+0000 7fb809e00640 1 -- 172.21.15.83:0/3419812070 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8040d2050 con 0x7fb7e00678c0 2024-04-25T00:15:42.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.674+0000 7fb801600640 1 -- 172.21.15.83:0/3419812070 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13445 (secure 0 0 0) 0x7fb8040d2050 con 0x7fb7e00678c0 2024-04-25T00:15:42.673 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:15:42.674 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:15:42.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.674+0000 7fb809e00640 1 -- 172.21.15.83:0/3419812070 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb7e00678c0 msgr2=0x7fb7e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.674+0000 7fb809e00640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb7e00678c0 0x7fb7e0069d80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4005890 tx=0x7fb7f4009040 comp rx=0 tx=0).stop 2024-04-25T00:15:42.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.674+0000 7fb809e00640 1 -- 172.21.15.83:0/3419812070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb80415e140 msgr2=0x7fb804159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.674+0000 7fb809e00640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb80415e140 0x7fb804159330 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc01af60 tx=0x7fb7fc019650 comp rx=0 tx=0).stop 2024-04-25T00:15:42.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.674+0000 7fb809e00640 1 -- 172.21.15.83:0/3419812070 shutdown_connections 2024-04-25T00:15:42.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.674+0000 7fb809e00640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7fb7e00678c0 0x7fb7e0069d80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.674+0000 7fb809e00640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8040d28f0 0x7fb804153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.674+0000 7fb809e00640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb80415e140 0x7fb804159330 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.674+0000 7fb809e00640 1 --2- 172.21.15.83:0/3419812070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8040d0df0 0x7fb804150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.674+0000 7fb809e00640 1 -- 172.21.15.83:0/3419812070 >> 172.21.15.83:0/3419812070 conn(0x7fb8040d3dd0 msgr2=0x7fb80406ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:42.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.674+0000 7fb809e00640 1 -- 172.21.15.83:0/3419812070 shutdown_connections 2024-04-25T00:15:42.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:15:42.674+0000 7fb809e00640 1 -- 172.21.15.83:0/3419812070 wait complete. 2024-04-25T00:15:42.694 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-04-25T00:15:42.498168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260380,"kb_used_data":10920,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714340,"statfs":{"total":773094113280,"available":772827484160,"internally_reserved":0,"allocated":11182080,"data_stored":5766056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.533625"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":318,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T00:15:34.792485+0000","last_change":"2024-04-25T00:10:56.546335+0000","last_active":"2024-04-25T00:15:34.792485+0000","last_peered":"2024-04-25T00:15:34.792485+0000","last_clean":"2024-04-25T00:15:34.792485+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:15:34.792485+0000","last_undegraded":"2024-04-25T00:15:34.792485+0000","last_fullsized":"2024-04-25T00:15:34.792485+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:10:54.757720+0000","last_clean_scrub_stamp":"2024-04-25T00:10:54.757720+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:47.626597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1843200,"data_stored":1163533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32404,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32404,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32404,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574909,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1843200,"data_stored":1163533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:15:42.697 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T00:10:54.757720+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=10, tm_sec=54, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=15, tm_sec=37, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:15:42.698 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T00:16:02.699 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:16:02.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.842+0000 7f5f73c00640 1 Processor -- start 2024-04-25T00:16:02.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.842+0000 7f5f73c00640 1 -- start start 2024-04-25T00:16:02.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.842+0000 7f5f73c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6c1553e0 0x7f5f6c06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.842+0000 7f5f73c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5f6c0b4a70 0x7f5f6c0b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.842+0000 7f5f73c00640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f6c06ef80 con 0x7f5f6c06ea00 2024-04-25T00:16:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.842+0000 7f5f73c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5f6c15b510 con 0x7f5f6c0b4a70 2024-04-25T00:16:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.842+0000 7f5f73c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6c15b7f0 con 0x7f5f6c1553e0 2024-04-25T00:16:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f63e00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5f6c06ea00 0x7f5f6c0b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57060/0 (socket says 172.21.15.83:57060) 2024-04-25T00:16:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f63e00640 1 -- 172.21.15.83:0/3318580322 learned_addr learned my addr 172.21.15.83:0/3318580322 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-25T00:16:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f71200640 1 --2- 172.21.15.83:0/3318580322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6c1553e0 0x7f5f6c06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f71c00640 1 --2- 172.21.15.83:0/3318580322 >> v2:172.21.15.86:3300/0 conn(0x7f5f6c0b4a70 0x7f5f6c0b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 -- 172.21.15.83:0/3318580322 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2511606561 0 0) 0x7f5f6c06ef80 con 0x7f5f6c06ea00 2024-04-25T00:16:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 -- 172.21.15.83:0/3318580322 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f50003530 con 0x7f5f6c06ea00 2024-04-25T00:16:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 -- 172.21.15.83:0/3318580322 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2694455702 0 0) 0x7f5f50003530 con 0x7f5f6c06ea00 2024-04-25T00:16:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 -- 172.21.15.83:0/3318580322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6c1553e0 msgr2=0x7f5f6c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 --2- 172.21.15.83:0/3318580322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6c1553e0 0x7f5f6c06e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 -- 172.21.15.83:0/3318580322 >> v2:172.21.15.86:3300/0 conn(0x7f5f6c0b4a70 msgr2=0x7f5f6c0b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 --2- 172.21.15.83:0/3318580322 >> v2:172.21.15.86:3300/0 conn(0x7f5f6c0b4a70 0x7f5f6c0b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 -- 172.21.15.83:0/3318580322 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c15bad0 con 0x7f5f6c06ea00 2024-04-25T00:16:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 -- 172.21.15.83:0/3318580322 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 224205348 0 0) 0x7f5f54001680 con 0x7f5f6c06ea00 2024-04-25T00:16:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f71c00640 1 --2- 172.21.15.83:0/3318580322 >> v2:172.21.15.86:3300/0 conn(0x7f5f6c0b4a70 0x7f5f6c0b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 -- 172.21.15.83:0/3318580322 >> v1:172.21.15.83:6789/0 conn(0x7f5f6c06ea00 legacy=0x7f5f6c0b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 --2- 172.21.15.83:0/3318580322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6c1553e0 0x7f5f500039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 --2- 172.21.15.83:0/3318580322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50003f40 0x7f5f50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 --2- 172.21.15.83:0/3318580322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f50004d50 0x7f5f500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 -- 172.21.15.83:0/3318580322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c15b510 con 0x7f5f50004d50 2024-04-25T00:16:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 -- 172.21.15.83:0/3318580322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c15b7f0 con 0x7f5f50003f40 2024-04-25T00:16:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f6be00640 1 -- 172.21.15.83:0/3318580322 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5f5000ba20 con 0x7f5f6c1553e0 2024-04-25T00:16:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f71c00640 1 --2- 172.21.15.83:0/3318580322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f50004d50 0x7f5f500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f71200640 1 --2- 172.21.15.83:0/3318580322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6c1553e0 0x7f5f500039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f63e00640 1 --2- 172.21.15.83:0/3318580322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50003f40 0x7f5f50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f71c00640 1 -- 172.21.15.83:0/3318580322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6c1553e0 msgr2=0x7f5f500039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f71c00640 1 --2- 172.21.15.83:0/3318580322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6c1553e0 0x7f5f500039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f71c00640 1 -- 172.21.15.83:0/3318580322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50003f40 msgr2=0x7f5f50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f71c00640 1 --2- 172.21.15.83:0/3318580322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50003f40 0x7f5f50004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.846+0000 7f5f71c00640 1 -- 172.21.15.83:0/3318580322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f50003530 con 0x7f5f50004d50 2024-04-25T00:16:02.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.850+0000 7f5f71c00640 1 --2- 172.21.15.83:0/3318580322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f50004d50 0x7f5f500051d0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009da0 tx=0x7f5f6401ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d491ec5e21f5da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:02.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.850+0000 7f5f6be00640 1 -- 172.21.15.83:0/3318580322 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64009de0 con 0x7f5f50004d50 2024-04-25T00:16:02.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.850+0000 7f5f6be00640 1 -- 172.21.15.83:0/3318580322 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f640b2070 con 0x7f5f50004d50 2024-04-25T00:16:02.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.850+0000 7f5f6be00640 1 -- 172.21.15.83:0/3318580322 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f640ad850 con 0x7f5f50004d50 2024-04-25T00:16:02.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.850+0000 7f5f73c00640 1 -- 172.21.15.83:0/3318580322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f50004d50 msgr2=0x7f5f500051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:02.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.850+0000 7f5f73c00640 1 --2- 172.21.15.83:0/3318580322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f50004d50 0x7f5f500051d0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009da0 tx=0x7f5f6401ed70 comp rx=0 tx=0).stop 2024-04-25T00:16:02.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.850+0000 7f5f71c00640 1 -- 172.21.15.83:0/3318580322 reap_dead start 2024-04-25T00:16:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.850+0000 7f5f73c00640 1 -- 172.21.15.83:0/3318580322 shutdown_connections 2024-04-25T00:16:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.850+0000 7f5f73c00640 1 -- 172.21.15.83:0/3318580322 >> 172.21.15.83:0/3318580322 conn(0x7f5f6c0b1830 msgr2=0x7f5f6c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.850+0000 7f5f73c00640 1 -- 172.21.15.83:0/3318580322 shutdown_connections 2024-04-25T00:16:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.850+0000 7f5f73c00640 1 -- 172.21.15.83:0/3318580322 wait complete. 2024-04-25T00:16:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.850+0000 7f5f73c00640 1 Processor -- start 2024-04-25T00:16:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.850+0000 7f5f73c00640 1 -- start start 2024-04-25T00:16:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f73c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6c14d460 0x7f5f6c14e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f73c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f6c1f8eb0 0x7f5f6c0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f73c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c0ce690 0x7f5f6c1e6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f73c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c0b3a10 con 0x7f5f6c0ce690 2024-04-25T00:16:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f73c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c0b3890 con 0x7f5f6c1f8eb0 2024-04-25T00:16:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f73c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6c0b3b90 con 0x7f5f6c14d460 2024-04-25T00:16:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f71c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c0ce690 0x7f5f6c1e6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f71c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c0ce690 0x7f5f6c1e6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36860/0 (socket says 172.21.15.83:36860) 2024-04-25T00:16:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f71c00640 1 -- 172.21.15.83:0/3775386163 learned_addr learned my addr 172.21.15.83:0/3775386163 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:16:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f63e00640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f6c1f8eb0 0x7f5f6c0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f71200640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6c14d460 0x7f5f6c14e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f71c00640 1 -- 172.21.15.83:0/3775386163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6c14d460 msgr2=0x7f5f6c14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f71c00640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6c14d460 0x7f5f6c14e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f71c00640 1 -- 172.21.15.83:0/3775386163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f6c1f8eb0 msgr2=0x7f5f6c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f71c00640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f6c1f8eb0 0x7f5f6c0ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f71c00640 1 -- 172.21.15.83:0/3775386163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c14ed40 con 0x7f5f6c0ce690 2024-04-25T00:16:02.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f71200640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6c14d460 0x7f5f6c14e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:02.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f71c00640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c0ce690 0x7f5f6c1e6bd0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f5f64099ee0 tx=0x7f5f64009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:02.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f69600640 1 -- 172.21.15.83:0/3775386163 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f640bc030 con 0x7f5f6c0ce690 2024-04-25T00:16:02.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f69600640 1 -- 172.21.15.83:0/3775386163 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f640039a0 con 0x7f5f6c0ce690 2024-04-25T00:16:02.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f69600640 1 -- 172.21.15.83:0/3775386163 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64009de0 con 0x7f5f6c0ce690 2024-04-25T00:16:02.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f73c00640 1 -- 172.21.15.83:0/3775386163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6c14f010 con 0x7f5f6c0ce690 2024-04-25T00:16:02.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f73c00640 1 -- 172.21.15.83:0/3775386163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f6c14f5d0 con 0x7f5f6c0ce690 2024-04-25T00:16:02.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f73c00640 1 -- 172.21.15.83:0/3775386163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f6c0737f0 con 0x7f5f6c0ce690 2024-04-25T00:16:02.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.854+0000 7f5f69600640 1 -- 172.21.15.83:0/3775386163 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5f640ad990 con 0x7f5f6c0ce690 2024-04-25T00:16:02.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.858+0000 7f5f69600640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f480678f0 0x7f5f48069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:02.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.858+0000 7f5f69600640 1 -- 172.21.15.83:0/3775386163 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f5f6411fed0 con 0x7f5f6c0ce690 2024-04-25T00:16:02.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.858+0000 7f5f71200640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f480678f0 0x7f5f48069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:02.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.858+0000 7f5f71200640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f480678f0 0x7f5f48069db0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009be0 tx=0x7f5f5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:02.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:02.862+0000 7f5f69600640 1 -- 172.21.15.83:0/3775386163 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5f64010070 con 0x7f5f6c0ce690 2024-04-25T00:16:03.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.014+0000 7f5f73c00640 1 -- 172.21.15.83:0/3775386163 --> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f6c0ceb10 con 0x7f5f480678f0 2024-04-25T00:16:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.014+0000 7f5f69600640 1 -- 172.21.15.83:0/3775386163 <== mgr.4107 v2:172.21.15.83:6832/565012784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7f5f5c014bb0 con 0x7f5f480678f0 2024-04-25T00:16:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:16:03.017 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T00:16:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.018+0000 7f5f73c00640 1 -- 172.21.15.83:0/3775386163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f480678f0 msgr2=0x7f5f48069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.018+0000 7f5f73c00640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f480678f0 0x7f5f48069db0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009be0 tx=0x7f5f5c008040 comp rx=0 tx=0).stop 2024-04-25T00:16:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.018+0000 7f5f73c00640 1 -- 172.21.15.83:0/3775386163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c0ce690 msgr2=0x7f5f6c1e6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.018+0000 7f5f73c00640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c0ce690 0x7f5f6c1e6bd0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f5f64099ee0 tx=0x7f5f64009750 comp rx=0 tx=0).stop 2024-04-25T00:16:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.018+0000 7f5f73c00640 1 -- 172.21.15.83:0/3775386163 shutdown_connections 2024-04-25T00:16:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.018+0000 7f5f73c00640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f5f480678f0 0x7f5f48069db0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.018+0000 7f5f73c00640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c0ce690 0x7f5f6c1e6bd0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.018+0000 7f5f73c00640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f6c1f8eb0 0x7f5f6c0ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.018+0000 7f5f73c00640 1 --2- 172.21.15.83:0/3775386163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f6c14d460 0x7f5f6c14e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.018+0000 7f5f73c00640 1 -- 172.21.15.83:0/3775386163 >> 172.21.15.83:0/3775386163 conn(0x7f5f6c0b1830 msgr2=0x7f5f6c06e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:03.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.018+0000 7f5f73c00640 1 -- 172.21.15.83:0/3775386163 shutdown_connections 2024-04-25T00:16:03.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.018+0000 7f5f73c00640 1 -- 172.21.15.83:0/3775386163 wait complete. 2024-04-25T00:16:03.038 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-04-25T00:16:02.503157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260364,"kb_used_data":10920,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714356,"statfs":{"total":773094113280,"available":772827500544,"internally_reserved":0,"allocated":11182080,"data_stored":5766056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003200"},"pg_stats":[{"pgid":"1.0","version":"100'119","reported_seq":329,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T00:15:42.570189+0000","last_change":"2024-04-25T00:15:42.570221+0000","last_active":"2024-04-25T00:15:42.570189+0000","last_peered":"2024-04-25T00:15:42.570189+0000","last_clean":"2024-04-25T00:15:42.570189+0000","last_became_active":"2024-04-25T00:10:56.545353+0000","last_became_peered":"2024-04-25T00:10:56.545353+0000","last_unstale":"2024-04-25T00:15:42.570189+0000","last_undegraded":"2024-04-25T00:15:42.570189+0000","last_fullsized":"2024-04-25T00:15:42.570189+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'119","last_scrub_stamp":"2024-04-25T00:15:42.570095+0000","last_deep_scrub":"100'119","last_deep_scrub_stamp":"2024-04-25T00:15:42.570095+0000","last_clean_scrub_stamp":"2024-04-25T00:15:42.570095+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:41.244212+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32976,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338864,"statfs":{"total":96636764160,"available":96602996736,"internally_reserved":0,"allocated":1843200,"data_stored":1163533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32404,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32404,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32404,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1249280,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32976,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338864,"statfs":{"total":96636764160,"available":96602996736,"internally_reserved":0,"allocated":1843200,"data_stored":1163533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:16:03.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T00:16:03.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.170+0000 7f6999200640 1 Processor -- start 2024-04-25T00:16:03.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.170+0000 7f6999200640 1 -- start start 2024-04-25T00:16:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.174+0000 7f6999200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69940cea30 0x7f69940b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.174+0000 7f6999200640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69940b2850 0x7f69940cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.174+0000 7f6999200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69940b2ff0 con 0x7f699415e180 2024-04-25T00:16:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.174+0000 7f6999200640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f69940b11e0 con 0x7f69940b2850 2024-04-25T00:16:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.174+0000 7f6999200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f69940b2da0 con 0x7f69940cea30 2024-04-25T00:16:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.174+0000 7f6993e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69940b2850 0x7f69940cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.174+0000 7f6993e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69940b2850 0x7f69940cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36868/0 (socket says 172.21.15.83:36868) 2024-04-25T00:16:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.174+0000 7f6993e00640 1 -- 172.21.15.83:0/1845711080 learned_addr learned my addr 172.21.15.83:0/1845711080 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:16:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.174+0000 7f6992a00640 1 --2- 172.21.15.83:0/1845711080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69940cea30 0x7f69940b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.174+0000 7f6993e00640 1 -- 172.21.15.83:0/1845711080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69940cea30 msgr2=0x7f69940b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:03.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.174+0000 7f6993e00640 1 --2- 172.21.15.83:0/1845711080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69940cea30 0x7f69940b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.174+0000 7f6993e00640 1 -- 172.21.15.83:0/1845711080 >> v1:172.21.15.86:6789/0 conn(0x7f699415e180 legacy=0x7f69940ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.174+0000 7f6993e00640 1 -- 172.21.15.83:0/1845711080 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69940b32d0 con 0x7f69940b2850 2024-04-25T00:16:03.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6993e00640 1 --2- 172.21.15.83:0/1845711080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b2850 0x7f69940cfdc0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f698c009870 tx=0x7f698c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8d12890744cab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:03.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6992000640 1 -- 172.21.15.83:0/1845711080 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0b0070 con 0x7f69940b2850 2024-04-25T00:16:03.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6992000640 1 -- 172.21.15.83:0/1845711080 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f698c0ab600 con 0x7f69940b2850 2024-04-25T00:16:03.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6992000640 1 -- 172.21.15.83:0/1845711080 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0ab9d0 con 0x7f69940b2850 2024-04-25T00:16:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 -- 172.21.15.83:0/1845711080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b2850 msgr2=0x7f69940cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 --2- 172.21.15.83:0/1845711080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b2850 0x7f69940cfdc0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f698c009870 tx=0x7f698c01c4d0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 -- 172.21.15.83:0/1845711080 shutdown_connections 2024-04-25T00:16:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 --2- 172.21.15.83:0/1845711080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940b2850 0x7f69940cfdc0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 --2- 172.21.15.83:0/1845711080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69940cea30 0x7f69940b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 -- 172.21.15.83:0/1845711080 >> 172.21.15.83:0/1845711080 conn(0x7f69940b35b0 msgr2=0x7f69940b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 -- 172.21.15.83:0/1845711080 shutdown_connections 2024-04-25T00:16:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 -- 172.21.15.83:0/1845711080 wait complete. 2024-04-25T00:16:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 Processor -- start 2024-04-25T00:16:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 -- start start 2024-04-25T00:16:03.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69940b2850 0x7f69940cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:03.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940cea30 0x7f69941565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:03.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f699415e180 0x7f69940ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:03.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f699415e6e0 con 0x7f69940cea30 2024-04-25T00:16:03.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f699415e560 con 0x7f69940b2850 2024-04-25T00:16:03.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.178+0000 7f6999200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f69940d0b10 con 0x7f699415e180 2024-04-25T00:16:03.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f6992a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940cea30 0x7f69941565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:03.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f6992a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940cea30 0x7f69941565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36880/0 (socket says 172.21.15.83:36880) 2024-04-25T00:16:03.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f6992a00640 1 -- 172.21.15.83:0/3637600944 learned_addr learned my addr 172.21.15.83:0/3637600944 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:16:03.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f6993400640 1 --2- 172.21.15.83:0/3637600944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69940b2850 0x7f69940cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:03.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f6993e00640 1 --2- 172.21.15.83:0/3637600944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f699415e180 0x7f69940ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:03.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f6992a00640 1 -- 172.21.15.83:0/3637600944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f699415e180 msgr2=0x7f69940ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f6992a00640 1 --2- 172.21.15.83:0/3637600944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f699415e180 0x7f69940ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f6992a00640 1 -- 172.21.15.83:0/3637600944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69940b2850 msgr2=0x7f69940cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f6992a00640 1 --2- 172.21.15.83:0/3637600944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69940b2850 0x7f69940cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f6992a00640 1 -- 172.21.15.83:0/3637600944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6994158620 con 0x7f69940cea30 2024-04-25T00:16:03.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f6992a00640 1 --2- 172.21.15.83:0/3637600944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940cea30 0x7f69941565c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f698000d830 tx=0x7f698000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:03.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f698b400640 1 -- 172.21.15.83:0/3637600944 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698000b6c0 con 0x7f69940cea30 2024-04-25T00:16:03.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f698b400640 1 -- 172.21.15.83:0/3637600944 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6980019070 con 0x7f69940cea30 2024-04-25T00:16:03.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f6999200640 1 -- 172.21.15.83:0/3637600944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69941578d0 con 0x7f69940cea30 2024-04-25T00:16:03.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f698b400640 1 -- 172.21.15.83:0/3637600944 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698000bae0 con 0x7f69940cea30 2024-04-25T00:16:03.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f6999200640 1 -- 172.21.15.83:0/3637600944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6994157e50 con 0x7f69940cea30 2024-04-25T00:16:03.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f6999200640 1 -- 172.21.15.83:0/3637600944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f699404f510 con 0x7f69940cea30 2024-04-25T00:16:03.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f698b400640 1 -- 172.21.15.83:0/3637600944 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f698001e460 con 0x7f69940cea30 2024-04-25T00:16:03.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f698b400640 1 --2- 172.21.15.83:0/3637600944 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69740678c0 0x7f6974069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:03.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.182+0000 7f698b400640 1 -- 172.21.15.83:0/3637600944 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f6980086390 con 0x7f69940cea30 2024-04-25T00:16:03.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.186+0000 7f6993400640 1 --2- 172.21.15.83:0/3637600944 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69740678c0 0x7f6974069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:03.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.186+0000 7f6993400640 1 --2- 172.21.15.83:0/3637600944 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69740678c0 0x7f6974069d80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6994158ac0 tx=0x7f6984009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:03.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.186+0000 7f698b400640 1 -- 172.21.15.83:0/3637600944 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6994158620 con 0x7f69940cea30 2024-04-25T00:16:03.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.346+0000 7f6999200640 1 -- 172.21.15.83:0/3637600944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f69940703e0 con 0x7f69940cea30 2024-04-25T00:16:03.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.350+0000 7f698b400640 1 -- 172.21.15.83:0/3637600944 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v26)=0 v26) ==== 123+0+0 (secure 0 0 0) 0x7f6980056f30 con 0x7f69940cea30 2024-04-25T00:16:03.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.350+0000 7f698b400640 1 -- 172.21.15.83:0/3637600944 <== mon.0 v2:172.21.15.83:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6980056830 con 0x7f69940cea30 2024-04-25T00:16:03.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.354+0000 7f6999200640 1 -- 172.21.15.83:0/3637600944 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69740678c0 msgr2=0x7f6974069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.354+0000 7f6999200640 1 --2- 172.21.15.83:0/3637600944 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69740678c0 0x7f6974069d80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6994158ac0 tx=0x7f6984009040 comp rx=0 tx=0).stop 2024-04-25T00:16:03.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.354+0000 7f6999200640 1 -- 172.21.15.83:0/3637600944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940cea30 msgr2=0x7f69941565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.354+0000 7f6999200640 1 --2- 172.21.15.83:0/3637600944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940cea30 0x7f69941565c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f698000d830 tx=0x7f698000dcf0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.354+0000 7f6999200640 1 -- 172.21.15.83:0/3637600944 shutdown_connections 2024-04-25T00:16:03.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.354+0000 7f6999200640 1 --2- 172.21.15.83:0/3637600944 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f69740678c0 0x7f6974069d80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.354+0000 7f6999200640 1 --2- 172.21.15.83:0/3637600944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f699415e180 0x7f69940ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.354+0000 7f6999200640 1 --2- 172.21.15.83:0/3637600944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69940cea30 0x7f69941565c0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.354+0000 7f6999200640 1 --2- 172.21.15.83:0/3637600944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69940b2850 0x7f69940cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.354+0000 7f6999200640 1 -- 172.21.15.83:0/3637600944 >> 172.21.15.83:0/3637600944 conn(0x7f69940b35b0 msgr2=0x7f69940d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:03.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.354+0000 7f6999200640 1 -- 172.21.15.83:0/3637600944 shutdown_connections 2024-04-25T00:16:03.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.354+0000 7f6999200640 1 -- 172.21.15.83:0/3637600944 wait complete. 2024-04-25T00:16:03.373 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:16:03.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:16:03.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b76e00640 1 Processor -- start 2024-04-25T00:16:03.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b76e00640 1 -- start start 2024-04-25T00:16:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b76e00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4b70162300 0x7f4b70166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b76e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4b700d0550 0x7f4b700d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b76e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b700715c0 0x7f4b70071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b76e00640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4b700d0ef0 con 0x7f4b70162300 2024-04-25T00:16:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b76e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4b700b3b40 con 0x7f4b700d0550 2024-04-25T00:16:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b76e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4b7006ded0 con 0x7f4b700715c0 2024-04-25T00:16:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b6fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4b70162300 0x7f4b70166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b6fe00640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4b70162300 0x7f4b70166920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36888/0 (socket says 172.21.15.83:36888) 2024-04-25T00:16:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b6fe00640 1 -- 172.21.15.83:0/2728278161 learned_addr learned my addr 172.21.15.83:0/2728278161 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:16:03.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b74e00640 1 --2- 172.21.15.83:0/2728278161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b700715c0 0x7f4b70071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:03.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b6f400640 1 --2- 172.21.15.83:0/2728278161 >> v2:172.21.15.86:3300/0 conn(0x7f4b700d0550 0x7f4b700d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:03.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b6fe00640 1 -- 172.21.15.83:0/2728278161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b700715c0 msgr2=0x7f4b70071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:03.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b6fe00640 1 --2- 172.21.15.83:0/2728278161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b700715c0 0x7f4b70071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b6fe00640 1 -- 172.21.15.83:0/2728278161 >> v2:172.21.15.86:3300/0 conn(0x7f4b700d0550 msgr2=0x7f4b700d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:03.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b6fe00640 1 --2- 172.21.15.83:0/2728278161 >> v2:172.21.15.86:3300/0 conn(0x7f4b700d0550 0x7f4b700d09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b6fe00640 1 -- 172.21.15.83:0/2728278161 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b700afbc0 con 0x7f4b70162300 2024-04-25T00:16:03.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b74e00640 1 --2- 172.21.15.83:0/2728278161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b700715c0 0x7f4b70071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:03.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.510+0000 7f4b6f400640 1 --2- 172.21.15.83:0/2728278161 >> v2:172.21.15.86:3300/0 conn(0x7f4b700d0550 0x7f4b700d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:03.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.514+0000 7f4b6fe00640 1 --2- 172.21.15.83:0/2728278161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b70162300 0x7f4b70166920 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f4b70165e70 tx=0x7f4b6001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b306869e96c8ca24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.514+0000 7f4b6ea00640 1 -- 172.21.15.83:0/2728278161 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b600a6040 con 0x7f4b70162300 2024-04-25T00:16:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.514+0000 7f4b6ea00640 1 -- 172.21.15.83:0/2728278161 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4b600af070 con 0x7f4b70162300 2024-04-25T00:16:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.514+0000 7f4b6ea00640 1 -- 172.21.15.83:0/2728278161 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b600aa880 con 0x7f4b70162300 2024-04-25T00:16:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.514+0000 7f4b76e00640 1 -- 172.21.15.83:0/2728278161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b70162300 msgr2=0x7f4b70166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.514+0000 7f4b76e00640 1 --2- 172.21.15.83:0/2728278161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b70162300 0x7f4b70166920 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f4b70165e70 tx=0x7f4b6001c520 comp rx=0 tx=0).stop 2024-04-25T00:16:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.514+0000 7f4b76e00640 1 -- 172.21.15.83:0/2728278161 shutdown_connections 2024-04-25T00:16:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.514+0000 7f4b76e00640 1 --2- 172.21.15.83:0/2728278161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b700715c0 0x7f4b70071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.514+0000 7f4b76e00640 1 --2- 172.21.15.83:0/2728278161 >> v2:172.21.15.86:3300/0 conn(0x7f4b700d0550 0x7f4b700d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.514+0000 7f4b76e00640 1 --2- 172.21.15.83:0/2728278161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b70162300 0x7f4b70166920 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.514+0000 7f4b76e00640 1 -- 172.21.15.83:0/2728278161 >> 172.21.15.83:0/2728278161 conn(0x7f4b7006b5e0 msgr2=0x7f4b7006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:03.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.514+0000 7f4b76e00640 1 -- 172.21.15.83:0/2728278161 shutdown_connections 2024-04-25T00:16:03.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.514+0000 7f4b76e00640 1 -- 172.21.15.83:0/2728278161 wait complete. 2024-04-25T00:16:03.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.514+0000 7f4b76e00640 1 Processor -- start 2024-04-25T00:16:03.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b76e00640 1 -- start start 2024-04-25T00:16:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b76e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b700715c0 0x7f4b700ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b76e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b700d0550 0x7f4b700cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b76e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b70162300 0x7f4b7015d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b76e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701626e0 con 0x7f4b700d0550 2024-04-25T00:16:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b76e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b700b35d0 con 0x7f4b70162300 2024-04-25T00:16:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b76e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4b7015d810 con 0x7f4b700715c0 2024-04-25T00:16:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b6f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b700d0550 0x7f4b700cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b6f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b700d0550 0x7f4b700cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36900/0 (socket says 172.21.15.83:36900) 2024-04-25T00:16:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b6f400640 1 -- 172.21.15.83:0/3464453280 learned_addr learned my addr 172.21.15.83:0/3464453280 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T00:16:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b74e00640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b70162300 0x7f4b7015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b6fe00640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b700715c0 0x7f4b700ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b6f400640 1 -- 172.21.15.83:0/3464453280 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b700715c0 msgr2=0x7f4b700ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b6f400640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b700715c0 0x7f4b700ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b6f400640 1 -- 172.21.15.83:0/3464453280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b70162300 msgr2=0x7f4b7015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b6f400640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b70162300 0x7f4b7015d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b6f400640 1 -- 172.21.15.83:0/3464453280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b700cef60 con 0x7f4b700d0550 2024-04-25T00:16:03.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b6fe00640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b700715c0 0x7f4b700ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:03.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b6f400640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b700d0550 0x7f4b700cc790 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00e840 tx=0x7f4b5c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:03.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b67e00640 1 -- 172.21.15.83:0/3464453280 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c017070 con 0x7f4b700d0550 2024-04-25T00:16:03.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b67e00640 1 -- 172.21.15.83:0/3464453280 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4b5c00c8d0 con 0x7f4b700d0550 2024-04-25T00:16:03.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b67e00640 1 -- 172.21.15.83:0/3464453280 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00f040 con 0x7f4b700d0550 2024-04-25T00:16:03.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b76e00640 1 -- 172.21.15.83:0/3464453280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b700cf240 con 0x7f4b700d0550 2024-04-25T00:16:03.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b76e00640 1 -- 172.21.15.83:0/3464453280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b700cf800 con 0x7f4b700d0550 2024-04-25T00:16:03.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.518+0000 7f4b76e00640 1 -- 172.21.15.83:0/3464453280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b70072f70 con 0x7f4b700d0550 2024-04-25T00:16:03.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.522+0000 7f4b67e00640 1 -- 172.21.15.83:0/3464453280 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4b5c01a070 con 0x7f4b700d0550 2024-04-25T00:16:03.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.522+0000 7f4b67e00640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b500678c0 0x7f4b50069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:03.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.522+0000 7f4b67e00640 1 -- 172.21.15.83:0/3464453280 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5593+0+0 (secure 0 0 0) 0x7f4b5c085910 con 0x7f4b700d0550 2024-04-25T00:16:03.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.522+0000 7f4b6fe00640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b500678c0 0x7f4b50069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:03.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.522+0000 7f4b6fe00640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b500678c0 0x7f4b50069d80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4b600ae000 tx=0x7f4b60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:03.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.526+0000 7f4b67e00640 1 -- 172.21.15.83:0/3464453280 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4b5c056390 con 0x7f4b700d0550 2024-04-25T00:16:03.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.710+0000 7f4b76e00640 1 -- 172.21.15.83:0/3464453280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b7004f5f0 con 0x7f4b700d0550 2024-04-25T00:16:03.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.714+0000 7f4b67e00640 1 -- 172.21.15.83:0/3464453280 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+3866 (secure 0 0 0) 0x7f4b5c055c90 con 0x7f4b700d0550 2024-04-25T00:16:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T00:16:03.714 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1992401755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1992401755},{"type":"v1","addr":"172.21.15.86:6833","nonce":1992401755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3621275047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3621275047},{"type":"v1","addr":"172.21.15.86:6835","nonce":3621275047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":8016,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/866062794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":866062794},{"type":"v1","addr":"172.21.15.83:6835","nonce":866062794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":8022,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3183818330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3183818330},{"type":"v1","addr":"172.21.15.83:6837","nonce":3183818330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[]} 2024-04-25T00:16:03.714 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 76 2024-04-25T00:16:03.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.714+0000 7f4b76e00640 1 -- 172.21.15.83:0/3464453280 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b500678c0 msgr2=0x7f4b50069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.714+0000 7f4b76e00640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b500678c0 0x7f4b50069d80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4b600ae000 tx=0x7f4b60008040 comp rx=0 tx=0).stop 2024-04-25T00:16:03.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.714+0000 7f4b76e00640 1 -- 172.21.15.83:0/3464453280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b700d0550 msgr2=0x7f4b700cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.714+0000 7f4b76e00640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b700d0550 0x7f4b700cc790 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00e840 tx=0x7f4b5c00ed00 comp rx=0 tx=0).stop 2024-04-25T00:16:03.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.714+0000 7f4b76e00640 1 -- 172.21.15.83:0/3464453280 shutdown_connections 2024-04-25T00:16:03.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.714+0000 7f4b76e00640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.83:6832/565012784,v1:172.21.15.83:6833/565012784] conn(0x7f4b500678c0 0x7f4b50069d80 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.714+0000 7f4b76e00640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b70162300 0x7f4b7015d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.714+0000 7f4b76e00640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b700d0550 0x7f4b700cc790 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.714+0000 7f4b76e00640 1 --2- 172.21.15.83:0/3464453280 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b700715c0 0x7f4b700ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.714+0000 7f4b76e00640 1 -- 172.21.15.83:0/3464453280 >> 172.21.15.83:0/3464453280 conn(0x7f4b7006b5e0 msgr2=0x7f4b700ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:03.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.714+0000 7f4b76e00640 1 -- 172.21.15.83:0/3464453280 shutdown_connections 2024-04-25T00:16:03.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T00:16:03.714+0000 7f4b76e00640 1 -- 172.21.15.83:0/3464453280 wait complete. 2024-04-25T00:16:03.735 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T00:16:03.735 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T00:16:03.735 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:16:03.735 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:03.747 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:16:03.748 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:16:03.748 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:03.800 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:16:03.800 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:16:03.800 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:03.813 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:16:03.814 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:16:03.814 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:03.873 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:16:03.938 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T00:16:03.939 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T00:16:03.939 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:03.999 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T00:16:04.000 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T00:16:04.000 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:04.600 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:16:04.604 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:16:08.549 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-04-25T00:16:10.102 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T00:16:10.102 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T00:16:10.102 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:10.161 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T00:16:10.162 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T00:16:10.162 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:10.226 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T00:16:10.226 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T00:16:10.226 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:10.326 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T00:16:10.326 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T00:16:10.326 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:10.403 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T00:16:10.404 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T00:16:10.404 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:10.483 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T00:16:10.483 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T00:16:10.483 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:10.555 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T00:16:10.555 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T00:16:10.555 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T00:16:10.555 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:10.616 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T00:16:10.616 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T00:16:10.616 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:10.667 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T00:16:10.667 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T00:16:10.667 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T00:16:10.667 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:10.687 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T00:16:10.688 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T00:16:10.688 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:10.703 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T00:16:10.703 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T00:16:10.703 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:16:10.759 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T00:16:10.760 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T00:16:10.760 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'client session with non-allowable root' | head -n 1 2024-04-25T00:16:10.794 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:16:10.794 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T00:16:11.101 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:16:11.101 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T00:16:11.331 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:16:11.331 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T00:16:11.506 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:16:11.506 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T00:16:11.732 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:16:11.732 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T00:16:12.014 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:16:12.015 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T00:16:12.208 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:16:12.208 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T00:16:12.383 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:16:12.384 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T00:16:12.541 INFO:tasks.ceph:Archiving mon data... 2024-04-25T00:16:12.542 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231/data/mon.a.tgz 2024-04-25T00:16:12.542 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-04-25T00:16:12.552 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.zFRwluLj5S 2024-04-25T00:16:12.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.zFRwluLj5S 2024-04-25T00:16:12.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.zFRwluLj5S 2024-04-25T00:16:12.792 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.zFRwluLj5S is 499KB 2024-04-25T00:16:12.858 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.zFRwluLj5S 2024-04-25T00:16:12.867 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231/data/mon.b.tgz 2024-04-25T00:16:12.867 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-04-25T00:16:12.875 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.WSK9GARfGh 2024-04-25T00:16:12.875 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.WSK9GARfGh 2024-04-25T00:16:13.065 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.WSK9GARfGh 2024-04-25T00:16:13.144 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.WSK9GARfGh is 644KB 2024-04-25T00:16:13.214 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.WSK9GARfGh 2024-04-25T00:16:13.222 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231/data/mon.c.tgz 2024-04-25T00:16:13.223 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-04-25T00:16:13.275 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.8x5DqJx8gD 2024-04-25T00:16:13.275 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.8x5DqJx8gD 2024-04-25T00:16:13.467 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.8x5DqJx8gD 2024-04-25T00:16:13.543 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.8x5DqJx8gD is 644KB 2024-04-25T00:16:13.550 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-04-25T00:16:13.610 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.8x5DqJx8gD 2024-04-25T00:16:13.618 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T00:16:13.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T00:16:13.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T00:16:13.692 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T00:16:13.727 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T00:16:13.727 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T00:16:13.729 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231/remote/smithi083/crash 2024-04-25T00:16:13.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T00:16:13.756 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231/remote/smithi086/crash 2024-04-25T00:16:13.757 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T00:16:13.785 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T00:16:13.785 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:16:13.805 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:16:13.874 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T00:16:13.895 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T00:16:13.914 INFO:tasks.ceph:Compressing logs... 2024-04-25T00:16:13.914 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:16:13.939 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:16:13.968 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38862.log 2024-04-25T00:16:13.969 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43287.log 2024-04-25T00:16:13.969 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38377.log 2024-04-25T00:16:13.970 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43287.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38862.log.gz 2024-04-25T00:16:13.970 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45222.log 2024-04-25T00:16:13.971 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43287.log.gz 2024-04-25T00:16:13.971 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39645.log 2024-04-25T00:16:13.972 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38377.log.gz 2024-04-25T00:16:13.972 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38650.log 2024-04-25T00:16:13.972 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45222.log.gz 2024-04-25T00:16:13.973 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35681.log 2024-04-25T00:16:13.973 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39645.log.gz 2024-04-25T00:16:13.974 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48123.log 2024-04-25T00:16:13.974 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38650.log.gz 2024-04-25T00:16:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34962.log 2024-04-25T00:16:13.975 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35681.log.gz 2024-04-25T00:16:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35395.log 2024-04-25T00:16:13.976 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48123.log.gz 2024-04-25T00:16:13.976 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44604.log 2024-04-25T00:16:13.977 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34962.log.gz 2024-04-25T00:16:13.977 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37333.log 2024-04-25T00:16:13.977 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35395.log.gz 2024-04-25T00:16:13.978 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41723.log 2024-04-25T00:16:13.978 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44604.log.gz 2024-04-25T00:16:13.979 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48284.log 2024-04-25T00:16:13.979 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37333.log.gz 2024-04-25T00:16:13.980 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44996.log 2024-04-25T00:16:13.980 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41723.log.gz 2024-04-25T00:16:13.981 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33983.log 2024-04-25T00:16:13.981 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48284.log.gz 2024-04-25T00:16:13.981 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47868.log 2024-04-25T00:16:13.982 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44996.log.gz 2024-04-25T00:16:13.982 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42923.log 2024-04-25T00:16:13.983 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33983.log.gz 2024-04-25T00:16:13.983 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47868.log: gzip -5 --verbose 87.6% -- -- replaced with /var/log/ceph/ceph-client.admin.47868.log.gz /var/log/ceph/ceph-client.admin.38097.log 2024-04-25T00:16:13.983 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-25T00:16:13.984 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42975.log 2024-04-25T00:16:13.984 INFO:teuthology.orchestra.run.smithi083.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42923.log.gz 2024-04-25T00:16:13.985 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.38097.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34610.log 2024-04-25T00:16:13.985 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38097.log.gz 2024-04-25T00:16:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39216.log 2024-04-25T00:16:13.986 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42975.log.gz 2024-04-25T00:16:13.987 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45192.log 2024-04-25T00:16:13.987 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34610.log.gz 2024-04-25T00:16:13.987 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43105.log 2024-04-25T00:16:13.988 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33871.log 2024-04-25T00:16:13.988 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39216.log.gz 2024-04-25T00:16:13.989 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33538.log 2024-04-25T00:16:13.989 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45192.log.gz 2024-04-25T00:16:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49104.log 2024-04-25T00:16:13.991 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43105.log.gz 2024-04-25T00:16:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39801.log 2024-04-25T00:16:13.991 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33538.log.gz 2024-04-25T00:16:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40831.log 2024-04-25T00:16:13.992 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49104.log.gz 2024-04-25T00:16:13.992 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34227.log 2024-04-25T00:16:13.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.33871.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33871.log.gz 2024-04-25T00:16:13.992 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33477.log 2024-04-25T00:16:13.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.34227.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T00:16:13.992 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34227.log.gz 2024-04-25T00:16:13.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.3.33477.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.33477.log.gz 2024-04-25T00:16:13.993 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T00:16:13.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33733.log 2024-04-25T00:16:13.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27945.log 2024-04-25T00:16:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.33733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33733.log.gz 2024-04-25T00:16:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27849.log 2024-04-25T00:16:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.27945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33612.log 2024-04-25T00:16:13.998 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27945.log.gz 2024-04-25T00:16:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.27849.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27849.log.gz 2024-04-25T00:16:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T00:16:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.33612.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33612.log.gz 2024-04-25T00:16:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27897.log 2024-04-25T00:16:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34332.log 2024-04-25T00:16:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.27897.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.27897.log.gz 2024-04-25T00:16:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33997.log 2024-04-25T00:16:14.001 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37705.log 2024-04-25T00:16:14.001 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39801.log.gz 2024-04-25T00:16:14.002 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40831.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T00:16:14.002 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40831.log.gz 2024-04-25T00:16:14.002 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37518.log 2024-04-25T00:16:14.002 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37705.log.gz 2024-04-25T00:16:14.002 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43748.log 2024-04-25T00:16:14.002 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43888.log 2024-04-25T00:16:14.002 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37518.log.gz 2024-04-25T00:16:14.002 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42345.log 2024-04-25T00:16:14.002 INFO:teuthology.orchestra.run.smithi083.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43748.log.gz 2024-04-25T00:16:14.002 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42532.log 2024-04-25T00:16:14.002 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43888.log.gz 2024-04-25T00:16:14.002 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.42345.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42345.log.gz 2024-04-25T00:16:14.002 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35943.log 2024-04-25T00:16:14.003 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37941.log 2024-04-25T00:16:14.003 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42532.log.gz 2024-04-25T00:16:14.003 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46283.log 2024-04-25T00:16:14.003 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35943.log.gz 2024-04-25T00:16:14.003 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38680.log 2024-04-25T00:16:14.003 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37941.log.gz 2024-04-25T00:16:14.003 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46283.log: gzip -5 --verbose 89.0% -- -- replaced with /var/log/ceph/ceph-client.admin.46283.log.gz /var/log/ceph/ceph-client.admin.46704.log 2024-04-25T00:16:14.003 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-25T00:16:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27825.log 2024-04-25T00:16:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.34332.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34332.log.gz 2024-04-25T00:16:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.33997.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33997.log.gz 2024-04-25T00:16:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T00:16:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.27825.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.27825.log.gz 2024-04-25T00:16:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T00:16:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27921.log 2024-04-25T00:16:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27993.log 2024-04-25T00:16:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.27921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27969.log 2024-04-25T00:16:14.005 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27921.log.gz 2024-04-25T00:16:14.005 INFO:teuthology.orchestra.run.smithi086.stderr: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T00:16:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.27993.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T00:16:14.007 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27993.log.gz 2024-04-25T00:16:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.27969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34106.log 2024-04-25T00:16:14.007 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27969.log.gz 2024-04-25T00:16:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33375.log 2024-04-25T00:16:14.010 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41333.log 2024-04-25T00:16:14.010 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38680.log.gz 2024-04-25T00:16:14.010 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39268.log 2024-04-25T00:16:14.010 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46704.log.gz 2024-04-25T00:16:14.010 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49130.log 2024-04-25T00:16:14.010 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41333.log.gz 2024-04-25T00:16:14.010 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45978.log 2024-04-25T00:16:14.010 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39268.log.gz 2024-04-25T00:16:14.010 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48948.log 2024-04-25T00:16:14.010 INFO:teuthology.orchestra.run.smithi083.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49130.log.gz 2024-04-25T00:16:14.010 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47416.log 2024-04-25T00:16:14.010 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45978.log.gz 2024-04-25T00:16:14.011 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41776.log 2024-04-25T00:16:14.011 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48948.log.gz 2024-04-25T00:16:14.011 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42118.log 2024-04-25T00:16:14.011 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47416.log.gz 2024-04-25T00:16:14.011 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37757.log 2024-04-25T00:16:14.011 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41776.log.gz 2024-04-25T00:16:14.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.34106.log: 93.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T00:16:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T00:16:14.014 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34106.log.gz 2024-04-25T00:16:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.33375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33375.log.gz 2024-04-25T00:16:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T00:16:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T00:16:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T00:16:14.015 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40423.log 2024-04-25T00:16:14.015 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42118.log.gz 2024-04-25T00:16:14.015 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46730.log 2024-04-25T00:16:14.015 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37757.log.gz 2024-04-25T00:16:14.015 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40397.log 2024-04-25T00:16:14.015 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40423.log.gz 2024-04-25T00:16:14.015 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46730.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T00:16:14.015 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46730.log.gz 2024-04-25T00:16:14.016 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36773.log 2024-04-25T00:16:14.016 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40397.log.gz 2024-04-25T00:16:14.016 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38455.log 2024-04-25T00:16:14.016 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-mon.b.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T00:16:14.016 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T00:16:14.017 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38272.log 2024-04-25T00:16:14.018 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36773.log.gz 2024-04-25T00:16:14.018 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44455.log 2024-04-25T00:16:14.018 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38455.log.gz 2024-04-25T00:16:14.018 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38624.log 2024-04-25T00:16:14.018 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38272.log.gz 2024-04-25T00:16:14.019 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27873.log 2024-04-25T00:16:14.019 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.27873.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27873.log.gz 2024-04-25T00:16:14.020 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40658.log 2024-04-25T00:16:14.020 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44455.log.gz 2024-04-25T00:16:14.020 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49182.log 2024-04-25T00:16:14.020 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38624.log.gz 2024-04-25T00:16:14.021 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40080.log 2024-04-25T00:16:14.021 INFO:teuthology.orchestra.run.smithi083.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40658.log.gz 2024-04-25T00:16:14.022 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39924.log 2024-04-25T00:16:14.022 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49182.log.gz 2024-04-25T00:16:14.023 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38019.log 2024-04-25T00:16:14.023 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40080.log.gz 2024-04-25T00:16:14.024 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40940.log 2024-04-25T00:16:14.024 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39924.log.gz 2024-04-25T00:16:14.025 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43079.log 2024-04-25T00:16:14.025 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38019.log.gz 2024-04-25T00:16:14.026 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36747.log 2024-04-25T00:16:14.026 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40940.log.gz 2024-04-25T00:16:14.027 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35915.log 2024-04-25T00:16:14.027 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43079.log.gz 2024-04-25T00:16:14.028 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46648.log 2024-04-25T00:16:14.028 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36747.log.gz 2024-04-25T00:16:14.029 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44273.log 2024-04-25T00:16:14.029 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35915.log.gz 2024-04-25T00:16:14.030 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45140.log 2024-04-25T00:16:14.031 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46648.log.gz 2024-04-25T00:16:14.031 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34975.log 2024-04-25T00:16:14.031 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44273.log.gz 2024-04-25T00:16:14.032 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35179.log 2024-04-25T00:16:14.032 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45140.log.gz 2024-04-25T00:16:14.033 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38246.log 2024-04-25T00:16:14.033 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34975.log.gz 2024-04-25T00:16:14.034 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46913.log 2024-04-25T00:16:14.034 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35179.log.gz 2024-04-25T00:16:14.035 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38246.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38246.log.gzgzip 2024-04-25T00:16:14.035 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35185.log 2024-04-25T00:16:14.036 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39950.log 2024-04-25T00:16:14.036 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46913.log.gz 2024-04-25T00:16:14.037 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35185.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.35185.log.gz 2024-04-25T00:16:14.037 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39056.log 2024-04-25T00:16:14.037 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40343.log 2024-04-25T00:16:14.038 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39950.log.gz 2024-04-25T00:16:14.039 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45049.log 2024-04-25T00:16:14.039 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39056.log.gz 2024-04-25T00:16:14.040 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47420.log 2024-04-25T00:16:14.040 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40343.log.gz 2024-04-25T00:16:14.041 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45049.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.34061.log -- replaced with /var/log/ceph/ceph-client.admin.45049.log.gz 2024-04-25T00:16:14.041 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-25T00:16:14.042 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46082.log 2024-04-25T00:16:14.042 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47420.log.gz 2024-04-25T00:16:14.043 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49391.log 2024-04-25T00:16:14.043 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34061.log.gz 2024-04-25T00:16:14.043 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44076.log 2024-04-25T00:16:14.043 INFO:teuthology.orchestra.run.smithi083.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46082.log.gz 2024-04-25T00:16:14.044 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49391.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49391.log.gz 2024-04-25T00:16:14.044 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48728.log 2024-04-25T00:16:14.045 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48560.log 2024-04-25T00:16:14.045 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44076.log.gz 2024-04-25T00:16:14.046 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47270.log 2024-04-25T00:16:14.046 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48728.log.gz 2024-04-25T00:16:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42362.log 2024-04-25T00:16:14.049 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48560.log.gz 2024-04-25T00:16:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39872.log 2024-04-25T00:16:14.051 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47270.log.gz 2024-04-25T00:16:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45900.log 2024-04-25T00:16:14.052 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42362.log.gz 2024-04-25T00:16:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41044.log 2024-04-25T00:16:14.052 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39872.log.gz 2024-04-25T00:16:14.052 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33877.log 2024-04-25T00:16:14.052 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45900.log.gz 2024-04-25T00:16:14.053 INFO:teuthology.orchestra.run.smithi086.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T00:16:14.054 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38507.log 2024-04-25T00:16:14.054 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41044.log.gz 2024-04-25T00:16:14.054 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45248.log 2024-04-25T00:16:14.054 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33877.log.gz 2024-04-25T00:16:14.055 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40475.log 2024-04-25T00:16:14.055 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38507.log.gz 2024-04-25T00:16:14.055 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42196.log 2024-04-25T00:16:14.055 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45248.log.gz 2024-04-25T00:16:14.056 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37731.log 2024-04-25T00:16:14.056 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40475.log.gz 2024-04-25T00:16:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38149.log 2024-04-25T00:16:14.057 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42196.log.gz 2024-04-25T00:16:14.058 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33715.log 2024-04-25T00:16:14.058 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37731.log.gz 2024-04-25T00:16:14.060 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38403.log 2024-04-25T00:16:14.060 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38149.log.gz 2024-04-25T00:16:14.060 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34479.log 2024-04-25T00:16:14.060 INFO:teuthology.orchestra.run.smithi083.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T00:16:14.060 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33715.log.gz 2024-04-25T00:16:14.061 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34981.log 2024-04-25T00:16:14.061 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38403.log.gz 2024-04-25T00:16:14.062 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46186.log 2024-04-25T00:16:14.062 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34479.log.gz 2024-04-25T00:16:14.063 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45546.log 2024-04-25T00:16:14.063 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34981.log.gz 2024-04-25T00:16:14.063 INFO:teuthology.orchestra.run.smithi086.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T00:16:14.064 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45378.log 2024-04-25T00:16:14.064 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46186.log.gz 2024-04-25T00:16:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35182.log 2024-04-25T00:16:14.065 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45546.log.gz 2024-04-25T00:16:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40028.log 2024-04-25T00:16:14.065 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45378.log.gz 2024-04-25T00:16:14.066 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45735.log 2024-04-25T00:16:14.066 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35182.log.gz 2024-04-25T00:16:14.067 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49078.log 2024-04-25T00:16:14.067 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40028.log.gz 2024-04-25T00:16:14.068 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37544.log 2024-04-25T00:16:14.068 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45735.log.gz 2024-04-25T00:16:14.069 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.36331.log 2024-04-25T00:16:14.069 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49078.log.gz 2024-04-25T00:16:14.070 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35172.log 2024-04-25T00:16:14.070 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37544.log.gz 2024-04-25T00:16:14.071 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.2.36331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.36331.log.gz 2024-04-25T00:16:14.071 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41385.log 2024-04-25T00:16:14.071 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35172.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T00:16:14.072 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35172.log.gz 2024-04-25T00:16:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34532.log 2024-04-25T00:16:14.072 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41385.log.gz 2024-04-25T00:16:14.073 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37223.log 2024-04-25T00:16:14.074 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34113.log 2024-04-25T00:16:14.074 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34532.log.gz 2024-04-25T00:16:14.075 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42949.log 2024-04-25T00:16:14.075 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37223.log.gz 2024-04-25T00:16:14.076 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33470.log 2024-04-25T00:16:14.076 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.34113.log.gz 2024-04-25T00:16:14.077 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40992.log 2024-04-25T00:16:14.077 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42949.log.gz 2024-04-25T00:16:14.078 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39394.log 2024-04-25T00:16:14.078 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33470.log.gz 2024-04-25T00:16:14.079 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36851.log 2024-04-25T00:16:14.079 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40992.log.gz 2024-04-25T00:16:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34636.log 2024-04-25T00:16:14.080 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39394.log.gz 2024-04-25T00:16:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41697.log 2024-04-25T00:16:14.081 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36851.log.gz 2024-04-25T00:16:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37601.log 2024-04-25T00:16:14.081 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34636.log.gz 2024-04-25T00:16:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43509.log 2024-04-25T00:16:14.082 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41697.log.gz 2024-04-25T00:16:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36565.log 2024-04-25T00:16:14.083 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37601.log.gz 2024-04-25T00:16:14.084 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43509.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43509.log.gz 2024-04-25T00:16:14.084 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33823.log 2024-04-25T00:16:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36565.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36565.log.gz 2024-04-25T00:16:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45761.log 2024-04-25T00:16:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33823.log: gzip -5 --verbose -- 87.6% /var/log/ceph/ceph-client.admin.35173.log -- replaced with /var/log/ceph/ceph-client.admin.33823.log.gz 2024-04-25T00:16:14.086 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-25T00:16:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41962.log 2024-04-25T00:16:14.087 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45761.log.gz 2024-04-25T00:16:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48315.log 2024-04-25T00:16:14.088 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35173.log.gz 2024-04-25T00:16:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34969.log 2024-04-25T00:16:14.089 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41962.log.gz 2024-04-25T00:16:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46782.log 2024-04-25T00:16:14.090 INFO:teuthology.orchestra.run.smithi083.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.48315.log.gz 2024-04-25T00:16:14.091 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34969.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34969.log.gz 2024-04-25T00:16:14.091 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39749.log 2024-04-25T00:16:14.092 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38836.log 2024-04-25T00:16:14.092 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46782.log.gz 2024-04-25T00:16:14.093 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39749.log: gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.39749.log.gz --verbose 2024-04-25T00:16:14.093 INFO:teuthology.orchestra.run.smithi083.stderr: -- /var/log/ceph/ceph-client.admin.42473.log 2024-04-25T00:16:14.094 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46108.log 2024-04-25T00:16:14.094 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38836.log.gz 2024-04-25T00:16:14.095 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35374.log 2024-04-25T00:16:14.095 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42473.log.gz 2024-04-25T00:16:14.096 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46108.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T00:16:14.096 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46108.log.gz 2024-04-25T00:16:14.097 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41600.log 2024-04-25T00:16:14.097 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35374.log.gz 2024-04-25T00:16:14.097 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46991.log 2024-04-25T00:16:14.099 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45650.log 2024-04-25T00:16:14.099 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41600.log.gz 2024-04-25T00:16:14.100 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48180.log 2024-04-25T00:16:14.100 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46991.log.gz 2024-04-25T00:16:14.100 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35396.log 2024-04-25T00:16:14.101 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45650.log.gz 2024-04-25T00:16:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48974.log 2024-04-25T00:16:14.102 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48180.log.gz 2024-04-25T00:16:14.103 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46834.log 2024-04-25T00:16:14.103 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35396.log.gz 2024-04-25T00:16:14.104 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49338.log 2024-04-25T00:16:14.104 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48974.log.gz 2024-04-25T00:16:14.105 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46834.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46834.log.gz 2024-04-25T00:16:14.105 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37197.log 2024-04-25T00:16:14.106 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49338.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49338.log.gz 2024-04-25T00:16:14.106 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44182.log 2024-04-25T00:16:14.107 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5/var/log/ceph/ceph-client.admin.37197.log: --verbose -- /var/log/ceph/ceph-client.admin.44377.log 2024-04-25T00:16:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44182.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37197.log.gz 2024-04-25T00:16:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41541.log 2024-04-25T00:16:14.109 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44182.log.gz 2024-04-25T00:16:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46860.log 2024-04-25T00:16:14.110 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44377.log.gz 2024-04-25T00:16:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39541.log 2024-04-25T00:16:14.111 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41541.log.gz 2024-04-25T00:16:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39500.log 2024-04-25T00:16:14.112 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46860.log.gz 2024-04-25T00:16:14.113 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46595.log 2024-04-25T00:16:14.113 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39541.log.gz 2024-04-25T00:16:14.113 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35733.log 2024-04-25T00:16:14.114 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39500.log: /var/log/ceph/ceph-client.admin.46595.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39500.log.gz 2024-04-25T00:16:14.115 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46595.log.gz 2024-04-25T00:16:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44481.log 2024-04-25T00:16:14.116 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35629.log 2024-04-25T00:16:14.116 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35733.log: /var/log/ceph/ceph-client.admin.44481.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35733.log.gz 2024-04-25T00:16:14.116 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43862.log 2024-04-25T00:16:14.117 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44481.log.gz 2024-04-25T00:16:14.118 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34973.log 2024-04-25T00:16:14.118 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35629.log.gz 2024-04-25T00:16:14.118 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48456.log 2024-04-25T00:16:14.119 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43862.log.gz 2024-04-25T00:16:14.120 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45023.log 2024-04-25T00:16:14.120 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34973.log.gz 2024-04-25T00:16:14.121 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41411.log 2024-04-25T00:16:14.121 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48456.log.gz 2024-04-25T00:16:14.121 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40755.log 2024-04-25T00:16:14.122 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45023.log.gz 2024-04-25T00:16:14.123 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35397.log 2024-04-25T00:16:14.123 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41411.log.gz 2024-04-25T00:16:14.123 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.40755.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40755.log.gz 2024-04-25T00:16:14.124 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40371.log 2024-04-25T00:16:14.125 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41073.log 2024-04-25T00:16:14.125 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35397.log.gz 2024-04-25T00:16:14.126 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41177.log 2024-04-25T00:16:14.126 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40371.log.gz 2024-04-25T00:16:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48404.log 2024-04-25T00:16:14.127 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41073.log.gz 2024-04-25T00:16:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37466.log 2024-04-25T00:16:14.128 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41177.log.gz 2024-04-25T00:16:14.129 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36383.log 2024-04-25T00:16:14.129 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48404.log.gz 2024-04-25T00:16:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37466.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37466.log.gz 2024-04-25T00:16:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36245.log 2024-04-25T00:16:14.131 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41125.log 2024-04-25T00:16:14.132 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36383.log: /var/log/ceph/ceph-client.admin.36245.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36383.log.gz 2024-04-25T00:16:14.132 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41018.log 2024-04-25T00:16:14.132 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36245.log.gz 2024-04-25T00:16:14.133 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43942.log 2024-04-25T00:16:14.133 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41125.log.gz 2024-04-25T00:16:14.134 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45681.log 2024-04-25T00:16:14.135 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41018.log.gz 2024-04-25T00:16:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37863.log 2024-04-25T00:16:14.136 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43942.log.gz 2024-04-25T00:16:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38298.log 2024-04-25T00:16:14.137 INFO:teuthology.orchestra.run.smithi083.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.45681.log.gz 2024-04-25T00:16:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45274.log 2024-04-25T00:16:14.138 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38298.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37863.log.gz 2024-04-25T00:16:14.138 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38298.log.gz 2024-04-25T00:16:14.139 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41802.log 2024-04-25T00:16:14.139 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48844.log 2024-04-25T00:16:14.140 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45274.log.gz 2024-04-25T00:16:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41802.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41802.log.gz 2024-04-25T00:16:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39827.log 2024-04-25T00:16:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43339.log 2024-04-25T00:16:14.142 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48844.log.gz 2024-04-25T00:16:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37307.log 2024-04-25T00:16:14.143 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39827.log.gz 2024-04-25T00:16:14.144 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41489.log 2024-04-25T00:16:14.144 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43339.log.gz 2024-04-25T00:16:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35785.log 2024-04-25T00:16:14.145 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37307.log.gz 2024-04-25T00:16:14.146 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47088.log 2024-04-25T00:16:14.146 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41489.log.gz 2024-04-25T00:16:14.147 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44050.log 2024-04-25T00:16:14.148 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35785.log.gz 2024-04-25T00:16:14.148 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.47088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47088.log.gz 2024-04-25T00:16:14.148 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47240.log 2024-04-25T00:16:14.149 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42170.log 2024-04-25T00:16:14.149 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44050.log.gz 2024-04-25T00:16:14.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35184.log 2024-04-25T00:16:14.151 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47240.log.gz 2024-04-25T00:16:14.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44299.log 2024-04-25T00:16:14.152 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7%/var/log/ceph/ceph-client.admin.35184.log: -- replaced with /var/log/ceph/ceph-client.admin.42170.log.gz 2024-04-25T00:16:14.152 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48378.log 2024-04-25T00:16:14.153 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35184.log.gz 2024-04-25T00:16:14.154 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44299.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44299.log.gz 2024-04-25T00:16:14.154 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47799.log 2024-04-25T00:16:14.154 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35759.log 2024-04-25T00:16:14.155 INFO:teuthology.orchestra.run.smithi086.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T00:16:14.156 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47799.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48378.log.gz 2024-04-25T00:16:14.156 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41567.log 2024-04-25T00:16:14.156 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47799.log.gz 2024-04-25T00:16:14.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44403.log 2024-04-25T00:16:14.157 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35759.log.gz 2024-04-25T00:16:14.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40553.log 2024-04-25T00:16:14.158 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41567.log.gz 2024-04-25T00:16:14.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36955.log 2024-04-25T00:16:14.159 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44403.log.gz 2024-04-25T00:16:14.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45105.log 2024-04-25T00:16:14.160 INFO:teuthology.orchestra.run.smithi083.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40553.log.gz 2024-04-25T00:16:14.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39030.log 2024-04-25T00:16:14.161 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36955.log.gz 2024-04-25T00:16:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38784.log 2024-04-25T00:16:14.162 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45105.log.gz 2024-04-25T00:16:14.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43994.log 88.6% 2024-04-25T00:16:14.163 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39030.log.gz 2024-04-25T00:16:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34291.log 2024-04-25T00:16:14.164 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38784.log.gz 2024-04-25T00:16:14.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43157.log 2024-04-25T00:16:14.165 INFO:teuthology.orchestra.run.smithi083.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43994.log.gz 2024-04-25T00:16:14.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43968.log 2024-04-25T00:16:14.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38045.log 2024-04-25T00:16:14.167 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43157.log.gz 2024-04-25T00:16:14.168 INFO:teuthology.orchestra.run.smithi083.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.34291.log.gz 2024-04-25T00:16:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43313.log 2024-04-25T00:16:14.168 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43968.log.gz 2024-04-25T00:16:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43448.log 2024-04-25T00:16:14.170 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38045.log.gz 2024-04-25T00:16:14.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36461.log 2024-04-25T00:16:14.171 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43313.log.gz 2024-04-25T00:16:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42395.log 2024-04-25T00:16:14.172 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43448.log.gz 2024-04-25T00:16:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42501.log 2024-04-25T00:16:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36461.log: /var/log/ceph/ceph-client.admin.42395.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36461.log.gz 2024-04-25T00:16:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38732.log 2024-04-25T00:16:14.174 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42395.log.gz 2024-04-25T00:16:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34978.log 2024-04-25T00:16:14.175 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42501.log.gz 2024-04-25T00:16:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39775.log 2024-04-25T00:16:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34978.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38732.log.gz 2024-04-25T00:16:14.177 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34978.log.gz 2024-04-25T00:16:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37085.log 2024-04-25T00:16:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39420.log 2024-04-25T00:16:14.178 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39775.log.gz 2024-04-25T00:16:14.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48258.log 2024-04-25T00:16:14.179 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37085.log.gz 2024-04-25T00:16:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45926.log 2024-04-25T00:16:14.180 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39420.log.gz 2024-04-25T00:16:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39190.log 2024-04-25T00:16:14.181 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48258.log.gz 2024-04-25T00:16:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48045.log 2024-04-25T00:16:14.182 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45926.log.gz 2024-04-25T00:16:14.183 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43392.log 2024-04-25T00:16:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39190.log: /var/log/ceph/ceph-client.admin.48045.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39190.log.gz 2024-04-25T00:16:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40145.log 2024-04-25T00:16:14.184 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48045.log.gz 2024-04-25T00:16:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47654.log 2024-04-25T00:16:14.186 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43392.log.gz 2024-04-25T00:16:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37033.log 2024-04-25T00:16:14.187 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40145.log.gz/var/log/ceph/ceph-client.admin.47654.log: 2024-04-25T00:16:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43535.log 2024-04-25T00:16:14.188 INFO:teuthology.orchestra.run.smithi083.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.47654.log.gz 2024-04-25T00:16:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39567.log 2024-04-25T00:16:14.189 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37033.log.gz 2024-04-25T00:16:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47768.log 2024-04-25T00:16:14.190 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43535.log.gz 2024-04-25T00:16:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42584.log 2024-04-25T00:16:14.191 INFO:teuthology.orchestra.run.smithi083.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.39567.log.gz 2024-04-25T00:16:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40909.log 2024-04-25T00:16:14.192 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47768.log.gz 2024-04-25T00:16:14.193 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47897.log 2024-04-25T00:16:14.193 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42584.log.gz 2024-04-25T00:16:14.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45326.log 2024-04-25T00:16:14.194 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40909.log.gz 2024-04-25T00:16:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47897.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47897.log.gz 2024-04-25T00:16:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36721.log 2024-04-25T00:16:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45326.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T00:16:14.196 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45326.log.gz 2024-04-25T00:16:14.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36357.log 2024-04-25T00:16:14.197 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36721.log.gz 2024-04-25T00:16:14.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34719.log 2024-04-25T00:16:14.199 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34967.log 2024-04-25T00:16:14.199 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36357.log.gz 2024-04-25T00:16:14.200 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34719.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34719.log.gz 2024-04-25T00:16:14.200 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44734.log 2024-04-25T00:16:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39593.log 2024-04-25T00:16:14.202 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34967.log.gz 2024-04-25T00:16:14.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37492.log 2024-04-25T00:16:14.203 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44734.log.gz 2024-04-25T00:16:14.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39593.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39593.log.gz 2024-04-25T00:16:14.204 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35377.log 2024-04-25T00:16:14.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43916.log 2024-04-25T00:16:14.205 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37492.log.gz 2024-04-25T00:16:14.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35377.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35377.log.gz 2024-04-25T00:16:14.206 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39324.log 2024-04-25T00:16:14.206 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5/var/log/ceph/ceph-client.admin.43916.log: --verbose -- /var/log/ceph/ceph-client.admin.45787.log 2024-04-25T00:16:14.208 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43916.log.gz 2024-04-25T00:16:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37679.log 2024-04-25T00:16:14.208 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39324.log.gz 2024-04-25T00:16:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42741.log 2024-04-25T00:16:14.209 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45787.log.gz 2024-04-25T00:16:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44507.log 2024-04-25T00:16:14.210 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37679.log.gz 2024-04-25T00:16:14.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43565.log 2024-04-25T00:16:14.213 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42741.log.gz 2024-04-25T00:16:14.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36087.log 2024-04-25T00:16:14.214 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44507.log.gz 2024-04-25T00:16:14.214 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.43565.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40606.log 2024-04-25T00:16:14.214 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43565.log.gz 2024-04-25T00:16:14.215 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34163.log 2024-04-25T00:16:14.215 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36087.log.gz 2024-04-25T00:16:14.215 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45404.log 2024-04-25T00:16:14.215 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40606.log.gz 2024-04-25T00:16:14.216 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44351.log 2024-04-25T00:16:14.217 INFO:teuthology.orchestra.run.smithi083.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34163.log.gz 2024-04-25T00:16:14.217 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42092.log 2024-04-25T00:16:14.218 INFO:teuthology.orchestra.run.smithi083.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.45404.log.gz 2024-04-25T00:16:14.218 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48534.log 2024-04-25T00:16:14.218 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44351.log.gz 2024-04-25T00:16:14.219 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44559.log 2024-04-25T00:16:14.220 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42092.log.gz 2024-04-25T00:16:14.221 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35889.log 2024-04-25T00:16:14.221 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48534.log.gz 2024-04-25T00:16:14.222 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42421.log 2024-04-25T00:16:14.222 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44559.log.gz 2024-04-25T00:16:14.223 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41988.log 2024-04-25T00:16:14.223 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35889.log.gz 2024-04-25T00:16:14.223 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45501.log 2024-04-25T00:16:14.224 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42421.log.gz 2024-04-25T00:16:14.225 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43845.log 2024-04-25T00:16:14.225 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41988.log.gz 2024-04-25T00:16:14.226 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.45501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40501.log 2024-04-25T00:16:14.226 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45501.log.gz 2024-04-25T00:16:14.227 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.43845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43845.log.gz 2024-04-25T00:16:14.227 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44102.log 2024-04-25T00:16:14.228 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41281.log 2024-04-25T00:16:14.229 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40501.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40501.log.gz 2024-04-25T00:16:14.229 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44102.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44102.log.gz 2024-04-25T00:16:14.229 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34453.log 2024-04-25T00:16:14.230 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38563.log 2024-04-25T00:16:14.231 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41281.log: /var/log/ceph/ceph-client.admin.34453.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41281.log.gz 2024-04-25T00:16:14.231 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34453.log.gz 2024-04-25T00:16:14.231 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39898.log 2024-04-25T00:16:14.233 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41515.log 2024-04-25T00:16:14.233 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38563.log.gz 2024-04-25T00:16:14.233 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48149.log 2024-04-25T00:16:14.233 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39898.log.gz 2024-04-25T00:16:14.234 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47923.log 2024-04-25T00:16:14.234 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41515.log.gz 2024-04-25T00:16:14.235 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37440.log 2024-04-25T00:16:14.235 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48149.log.gz 2024-04-25T00:16:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41910.log 2024-04-25T00:16:14.237 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37440.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.47923.log.gz 2024-04-25T00:16:14.238 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37440.log.gz 2024-04-25T00:16:14.238 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47598.log 2024-04-25T00:16:14.238 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44020.log 2024-04-25T00:16:14.239 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41910.log.gz 2024-04-25T00:16:14.240 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35386.log 2024-04-25T00:16:14.240 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47598.log.gz 2024-04-25T00:16:14.240 INFO:teuthology.orchestra.run.smithi086.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T00:16:14.241 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34506.log 2024-04-25T00:16:14.241 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44020.log.gz 2024-04-25T00:16:14.242 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40210.log 2024-04-25T00:16:14.242 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35386.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35386.log.gz 2024-04-25T00:16:14.243 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39294.log 2024-04-25T00:16:14.243 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34506.log.gz 2024-04-25T00:16:14.243 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40210.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T00:16:14.244 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40210.log.gz 2024-04-25T00:16:14.245 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27707.log 2024-04-25T00:16:14.245 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39294.log.gz 2024-04-25T00:16:14.246 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45077.log 2024-04-25T00:16:14.247 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.27707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48656.log 2024-04-25T00:16:14.248 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.27707.log.gz 2024-04-25T00:16:14.248 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38976.log 2024-04-25T00:16:14.249 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45077.log.gz 2024-04-25T00:16:14.250 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48656.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48656.log.gz 2024-04-25T00:16:14.250 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46939.log 2024-04-25T00:16:14.250 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38976.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38976.log.gz 2024-04-25T00:16:14.251 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42793.log 2024-04-25T00:16:14.253 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33636.log 2024-04-25T00:16:14.253 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46939.log: 87.8%/var/log/ceph/ceph-client.admin.42793.log: -- replaced with /var/log/ceph/ceph-client.admin.46939.log.gz 2024-04-25T00:16:14.253 INFO:teuthology.orchestra.run.smithi083.stderr:gzip 89.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.41437.log 2024-04-25T00:16:14.254 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42793.log.gz 2024-04-25T00:16:14.255 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33636.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.33636.log.gz -5 2024-04-25T00:16:14.255 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35376.log 2024-04-25T00:16:14.256 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36591.log 2024-04-25T00:16:14.256 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41437.log.gz 2024-04-25T00:16:14.257 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38758.log 2024-04-25T00:16:14.257 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35376.log.gz 2024-04-25T00:16:14.258 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37653.log 2024-04-25T00:16:14.258 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36591.log.gz 2024-04-25T00:16:14.259 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35863.log 2024-04-25T00:16:14.259 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38758.log.gz 2024-04-25T00:16:14.261 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44212.log 2024-04-25T00:16:14.261 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37653.log.gz 2024-04-25T00:16:14.261 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42715.log 2024-04-25T00:16:14.261 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35863.log.gz 2024-04-25T00:16:14.262 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43591.log 2024-04-25T00:16:14.263 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42715.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44212.log.gz 2024-04-25T00:16:14.264 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36435.log 2024-04-25T00:16:14.264 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42715.log.gz 2024-04-25T00:16:14.265 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37145.log 2024-04-25T00:16:14.265 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43591.log.gz 2024-04-25T00:16:14.266 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37785.log 2024-04-25T00:16:14.266 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36435.log.gz 2024-04-25T00:16:14.268 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35655.log 2024-04-25T00:16:14.268 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37145.log.gz 2024-04-25T00:16:14.269 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37785.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22898.log 2024-04-25T00:16:14.269 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37785.log.gz 2024-04-25T00:16:14.270 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33741.log 2024-04-25T00:16:14.270 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35655.log.gz 2024-04-25T00:16:14.271 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.tmp-client.admin.22898.log: gzip 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22898.log.gz 2024-04-25T00:16:14.271 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37171.log 2024-04-25T00:16:14.272 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48870.log 2024-04-25T00:16:14.272 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33741.log.gz 2024-04-25T00:16:14.273 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39976.log 2024-04-25T00:16:14.274 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37171.log.gz 2024-04-25T00:16:14.275 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48870.log: gzip 88.3% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.48870.log.gz -- 2024-04-25T00:16:14.275 INFO:teuthology.orchestra.run.smithi083.stderr: /var/log/ceph/ceph-client.admin.40113.log 2024-04-25T00:16:14.276 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47105.log 2024-04-25T00:16:14.276 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39976.log.gz 2024-04-25T00:16:14.277 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34375.log 2024-04-25T00:16:14.277 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40113.log.gz 2024-04-25T00:16:14.278 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47105.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47105.log.gz 2024-04-25T00:16:14.279 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34375.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34375.log.gz 2024-04-25T00:16:14.280 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41229.log 2024-04-25T00:16:14.280 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44812.log 2024-04-25T00:16:14.282 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34087.log 2024-04-25T00:16:14.282 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41229.log.gz 2024-04-25T00:16:14.283 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44812.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44812.log.gz 2024-04-25T00:16:14.285 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34087.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34087.log.gz 2024-04-25T00:16:14.285 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44877.log 2024-04-25T00:16:14.285 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38706.log 2024-04-25T00:16:14.287 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44760.log 2024-04-25T00:16:14.287 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44877.log.gz 2024-04-25T00:16:14.288 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38706.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38706.log.gz 2024-04-25T00:16:14.290 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44760.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44760.log.gz 2024-04-25T00:16:14.290 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37837.log 2024-04-25T00:16:14.291 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33903.log 2024-04-25T00:16:14.292 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41307.log 2024-04-25T00:16:14.292 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37837.log.gz 2024-04-25T00:16:14.293 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33903.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33903.log.gz 2024-04-25T00:16:14.294 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41307.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41307.log.gz 2024-04-25T00:16:14.295 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33931.log 2024-04-25T00:16:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47185.log 2024-04-25T00:16:14.297 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33931.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33931.log.gz 2024-04-25T00:16:14.298 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36643.log 2024-04-25T00:16:14.299 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46478.log 2024-04-25T00:16:14.299 INFO:teuthology.orchestra.run.smithi083.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47185.log.gz 2024-04-25T00:16:14.300 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42447.log 2024-04-25T00:16:14.300 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36643.log.gz 2024-04-25T00:16:14.301 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34927.log 2024-04-25T00:16:14.301 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46478.log.gz 2024-04-25T00:16:14.303 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46004.log 2024-04-25T00:16:14.303 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42447.log.gz 2024-04-25T00:16:14.304 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34927.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34927.log.gz 2024-04-25T00:16:14.304 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46231.log 2024-04-25T00:16:14.305 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49026.log 2024-04-25T00:16:14.306 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46004.log.gz 2024-04-25T00:16:14.306 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44128.log 2024-04-25T00:16:14.306 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46231.log.gz 2024-04-25T00:16:14.307 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49026.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49026.log.gz 2024-04-25T00:16:14.308 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34427.log 2024-04-25T00:16:14.309 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44128.log.gz 2024-04-25T00:16:14.309 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38959.log 2024-04-25T00:16:14.310 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44630.log 2024-04-25T00:16:14.311 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34427.log.gz 2024-04-25T00:16:14.311 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.38959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40966.log 2024-04-25T00:16:14.311 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38959.log.gz 2024-04-25T00:16:14.313 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35577.log 2024-04-25T00:16:14.313 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44630.log.gz 2024-04-25T00:16:14.314 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40966.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40966.log.gz 2024-04-25T00:16:14.314 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33957.log 2024-04-25T00:16:14.315 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37570.log 2024-04-25T00:16:14.316 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35577.log.gz 2024-04-25T00:16:14.317 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33957.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33957.log.gz 2024-04-25T00:16:14.317 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49286.log 2024-04-25T00:16:14.318 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37570.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37570.log.gz 2024-04-25T00:16:14.318 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39082.log 2024-04-25T00:16:14.320 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33851.log 2024-04-25T00:16:14.320 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49286.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49286.log.gz/var/log/ceph/ceph-client.admin.39082.log: 2024-04-25T00:16:14.320 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39082.log.gz 2024-04-25T00:16:14.321 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45300.log 2024-04-25T00:16:14.321 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33851.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33851.log.gz 2024-04-25T00:16:14.322 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44533.log 2024-04-25T00:16:14.323 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34823.log 2024-04-25T00:16:14.324 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45300.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45300.log.gz/var/log/ceph/ceph-client.admin.44533.log: 2024-04-25T00:16:14.324 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44533.log.gz 2024-04-25T00:16:14.325 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.34823.log: /var/log/ceph/ceph-client.admin.34349.log 2024-04-25T00:16:14.325 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34823.log.gz 2024-04-25T00:16:14.326 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T00:16:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36669.log 2024-04-25T00:16:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34349.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34349.log.gz/var/log/ceph/ceph.log: 2024-04-25T00:16:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36669.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36669.log.gz 2024-04-25T00:16:14.330 INFO:teuthology.orchestra.run.smithi083.stderr: 86.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T00:16:14.330 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42610.log 2024-04-25T00:16:14.331 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43420.log 2024-04-25T00:16:14.332 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42610.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42610.log.gz 2024-04-25T00:16:14.332 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48586.log 2024-04-25T00:16:14.333 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43420.log: /var/log/ceph/ceph-client.admin.40449.log 2024-04-25T00:16:14.333 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43420.log.gz 2024-04-25T00:16:14.335 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48586.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48586.log.gz 2024-04-25T00:16:14.335 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44429.log 2024-04-25T00:16:14.335 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40449.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40449.log.gz 2024-04-25T00:16:14.335 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36929.log 2024-04-25T00:16:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46413.log 2024-04-25T00:16:14.338 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44429.log: 87.8%/var/log/ceph/ceph-client.admin.36929.log: -- replaced with /var/log/ceph/ceph-client.admin.44429.log.gz 2024-04-25T00:16:14.338 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36929.log.gz 2024-04-25T00:16:14.339 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43131.log 2024-04-25T00:16:14.339 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46413.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46413.log.gz 2024-04-25T00:16:14.340 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34771.log 2024-04-25T00:16:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39242.log 2024-04-25T00:16:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43131.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43131.log.gz 2024-04-25T00:16:14.342 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34771.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34771.log.gz 2024-04-25T00:16:14.343 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47419.log 2024-04-25T00:16:14.343 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39242.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39242.log.gz 2024-04-25T00:16:14.344 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40772.log 2024-04-25T00:16:14.345 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39448.log 2024-04-25T00:16:14.345 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47419.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47419.log.gz/var/log/ceph/ceph-client.admin.40772.log: 2024-04-25T00:16:14.346 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40772.log.gz 2024-04-25T00:16:14.346 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37915.log 2024-04-25T00:16:14.347 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39448.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39448.log.gz 2024-04-25T00:16:14.347 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38351.log 2024-04-25T00:16:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41858.log 2024-04-25T00:16:14.349 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37915.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37915.log.gz/var/log/ceph/ceph-client.admin.38351.log: 2024-04-25T00:16:14.350 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38351.log.gz 2024-04-25T00:16:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39108.log 2024-04-25T00:16:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41858.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41858.log.gz 2024-04-25T00:16:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42897.log 2024-04-25T00:16:14.352 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44656.log 2024-04-25T00:16:14.353 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39108.log: 89.1%/var/log/ceph/ceph-client.admin.42897.log: -- replaced with /var/log/ceph/ceph-client.admin.39108.log.gz 2024-04-25T00:16:14.353 INFO:teuthology.orchestra.run.smithi083.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42897.log.gz 2024-04-25T00:16:14.354 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40708.log 2024-04-25T00:16:14.354 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44656.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44656.log.gz 2024-04-25T00:16:14.355 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38429.log 2024-04-25T00:16:14.356 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38325.log 2024-04-25T00:16:14.357 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40708.log: /var/log/ceph/ceph-client.admin.38429.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40708.log.gz 2024-04-25T00:16:14.357 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38429.log.gz 2024-04-25T00:16:14.358 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41255.log 2024-04-25T00:16:14.359 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38325.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38325.log.gz 2024-04-25T00:16:14.359 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42558.log 2024-04-25T00:16:14.360 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34875.log 2024-04-25T00:16:14.361 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41255.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41255.log.gz/var/log/ceph/ceph-client.admin.42558.log: 2024-04-25T00:16:14.361 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42558.log.gz 2024-04-25T00:16:14.362 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37811.log 2024-04-25T00:16:14.362 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34875.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34875.log.gz 2024-04-25T00:16:14.363 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39134.log 2024-04-25T00:16:14.364 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45952.log 2024-04-25T00:16:14.365 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37811.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37811.log.gz/var/log/ceph/ceph-client.admin.39134.log: 2024-04-25T00:16:14.365 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39134.log.gz 2024-04-25T00:16:14.366 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36487.log 2024-04-25T00:16:14.366 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45952.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45952.log.gz 2024-04-25T00:16:14.367 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27926.log 2024-04-25T00:16:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44708.log 2024-04-25T00:16:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36487.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36487.log.gz 2024-04-25T00:16:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.27926.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27926.log.gz 2024-04-25T00:16:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38123.log 2024-04-25T00:16:14.370 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44708.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44708.log.gz 2024-04-25T00:16:14.371 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41749.log 2024-04-25T00:16:14.371 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37385.log 2024-04-25T00:16:14.373 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38123.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38123.log.gz/var/log/ceph/ceph-client.admin.41749.log: 2024-04-25T00:16:14.373 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41749.log.gz 2024-04-25T00:16:14.373 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36021.log 2024-04-25T00:16:14.374 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37385.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37385.log.gz 2024-04-25T00:16:14.374 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40236.log 2024-04-25T00:16:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46676.log 2024-04-25T00:16:14.377 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36021.log: /var/log/ceph/ceph-client.admin.40236.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.36021.log.gz 2024-04-25T00:16:14.377 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40236.log.gz 2024-04-25T00:16:14.377 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33767.log 2024-04-25T00:16:14.378 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46676.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46676.log.gz 2024-04-25T00:16:14.378 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48206.log 2024-04-25T00:16:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42636.log 2024-04-25T00:16:14.380 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33767.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33767.log.gz/var/log/ceph/ceph-client.admin.48206.log: 2024-04-25T00:16:14.381 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48206.log.gz 2024-04-25T00:16:14.381 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46134.log 2024-04-25T00:16:14.382 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42636.log.gz 2024-04-25T00:16:14.382 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45454.log 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46134.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.46134.log.gz 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48508.log 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42689.log 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45454.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45454.log.gz/var/log/ceph/ceph-client.admin.48508.log: 2024-04-25T00:16:14.386 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48508.log.gz 2024-04-25T00:16:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33689.log 2024-04-25T00:16:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42689.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42689.log.gz 2024-04-25T00:16:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47833.log 2024-04-25T00:16:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36617.log 2024-04-25T00:16:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33689.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33689.log.gz/var/log/ceph/ceph-client.admin.47833.log: 2024-04-25T00:16:14.390 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47833.log.gz 2024-04-25T00:16:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38071.log 2024-04-25T00:16:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36617.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36617.log.gz 2024-04-25T00:16:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42040.log 2024-04-25T00:16:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46886.log 2024-04-25T00:16:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38071.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38071.log.gz 2024-04-25T00:16:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42040.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42040.log.gz 2024-04-25T00:16:14.394 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37993.log 2024-04-25T00:16:14.395 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46886.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46886.log.gz 2024-04-25T00:16:14.396 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36409.log 2024-04-25T00:16:14.396 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41463.log 2024-04-25T00:16:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37993.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37993.log.gz 2024-04-25T00:16:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36409.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36409.log.gz 2024-04-25T00:16:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49156.log 2024-04-25T00:16:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41463.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41463.log.gz 2024-04-25T00:16:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40054.log 2024-04-25T00:16:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43670.log 2024-04-25T00:16:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49156.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49156.log.gz 2024-04-25T00:16:14.402 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40054.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40054.log.gz 2024-04-25T00:16:14.402 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42767.log 2024-04-25T00:16:14.402 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43670.log.gz 2024-04-25T00:16:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40289.log 2024-04-25T00:16:14.404 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41632.log 2024-04-25T00:16:14.405 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42767.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42767.log.gz 2024-04-25T00:16:14.405 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40289.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40289.log.gz 2024-04-25T00:16:14.406 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34667.log 2024-04-25T00:16:14.407 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41632.log.gz 2024-04-25T00:16:14.407 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37281.log 2024-04-25T00:16:14.408 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43798.log 2024-04-25T00:16:14.409 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34667.log: 89.1%/var/log/ceph/ceph-client.admin.37281.log: -- replaced with /var/log/ceph/ceph-client.admin.34667.log.gz 2024-04-25T00:16:14.409 INFO:teuthology.orchestra.run.smithi083.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37281.log.gz 2024-04-25T00:16:14.410 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39164.log 2024-04-25T00:16:14.411 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43798.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43798.log.gz 2024-04-25T00:16:14.411 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43222.log 2024-04-25T00:16:14.413 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37119.log 2024-04-25T00:16:14.413 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39164.log: 88.7%/var/log/ceph/ceph-client.admin.43222.log: -- replaced with /var/log/ceph/ceph-client.admin.39164.log.gz 2024-04-25T00:16:14.413 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43222.log.gz 2024-04-25T00:16:14.414 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48763.log 2024-04-25T00:16:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37119.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37119.log.gz 2024-04-25T00:16:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43190.log 2024-04-25T00:16:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46361.log 2024-04-25T00:16:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48763.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48763.log.gz 2024-04-25T00:16:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43190.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43190.log.gz 2024-04-25T00:16:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42066.log 2024-04-25T00:16:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46361.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46361.log.gz 2024-04-25T00:16:14.420 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45166.log 2024-04-25T00:16:14.420 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36695.log 2024-04-25T00:16:14.420 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42066.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42066.log.gz 2024-04-25T00:16:14.421 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45166.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45166.log.gz 2024-04-25T00:16:14.421 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39474.log 2024-04-25T00:16:14.422 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36695.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36695.log.gz 2024-04-25T00:16:14.422 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34558.log 2024-04-25T00:16:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48430.log 2024-04-25T00:16:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39474.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39474.log.gz 2024-04-25T00:16:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34558.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34558.log.gz 2024-04-25T00:16:14.425 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45813.log 2024-04-25T00:16:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48430.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48430.log.gz 2024-04-25T00:16:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34401.log 2024-04-25T00:16:14.427 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35811.log 2024-04-25T00:16:14.428 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45813.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45813.log.gz 2024-04-25T00:16:14.428 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34401.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34401.log.gz 2024-04-25T00:16:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47041.log 2024-04-25T00:16:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35811.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35811.log.gz 2024-04-25T00:16:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40527.log 2024-04-25T00:16:14.431 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43617.log 2024-04-25T00:16:14.432 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47041.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47041.log.gz/var/log/ceph/ceph-client.admin.40527.log: 2024-04-25T00:16:14.432 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40527.log.gz 2024-04-25T00:16:14.433 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34797.log 2024-04-25T00:16:14.434 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43617.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43617.log.gz 2024-04-25T00:16:14.434 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46569.log 2024-04-25T00:16:14.435 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47421.log 2024-04-25T00:16:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34797.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34797.log.gz/var/log/ceph/ceph-client.admin.46569.log: 2024-04-25T00:16:14.436 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46569.log.gz 2024-04-25T00:16:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48922.log 2024-04-25T00:16:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47421.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47421.log.gz 2024-04-25T00:16:14.438 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45518.log 2024-04-25T00:16:14.439 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39371.log 2024-04-25T00:16:14.439 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48922.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48922.log.gz/var/log/ceph/ceph-client.admin.45518.log: 2024-04-25T00:16:14.441 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45518.log.gz 2024-04-25T00:16:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T00:16:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.39371.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.39371.log.gz 2024-04-25T00:16:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46257.log 2024-04-25T00:16:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T00:16:14.445 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46257.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46257.log.gz 2024-04-25T00:16:14.445 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mds.c.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T00:16:14.446 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40883.log 2024-04-25T00:16:14.447 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46808.log 2024-04-25T00:16:14.449 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40883.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40883.log.gz 2024-04-25T00:16:14.449 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46808.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46808.log.gz 2024-04-25T00:16:14.450 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43722.log 2024-04-25T00:16:14.452 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47704.log 2024-04-25T00:16:14.453 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43722.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43722.log.gz 2024-04-25T00:16:14.453 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47704.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47704.log.gz 2024-04-25T00:16:14.454 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33795.log 2024-04-25T00:16:14.455 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41884.log 2024-04-25T00:16:14.457 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33795.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33795.log.gz 2024-04-25T00:16:14.458 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41884.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41884.log.gz 2024-04-25T00:16:14.458 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47751.log 2024-04-25T00:16:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37250.log 2024-04-25T00:16:14.461 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.47751.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47751.log.gz 2024-04-25T00:16:14.462 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37250.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37250.log.gz 2024-04-25T00:16:14.462 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T00:16:14.464 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38535.log 2024-04-25T00:16:14.467 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.38535.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38535.log.gz 2024-04-25T00:16:14.467 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47552.log 2024-04-25T00:16:14.470 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47552.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47552.log.gz 2024-04-25T00:16:14.470 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36825.log 2024-04-25T00:16:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36825.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36825.log.gz 2024-04-25T00:16:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49234.log 2024-04-25T00:16:14.476 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36903.log 2024-04-25T00:16:14.476 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49234.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49234.log.gz 2024-04-25T00:16:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36903.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36903.log.gz 2024-04-25T00:16:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45572.log 2024-04-25T00:16:14.482 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45572.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45572.log.gz 2024-04-25T00:16:14.482 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37967.log 2024-04-25T00:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37967.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37967.log.gz 2024-04-25T00:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45598.log 2024-04-25T00:16:14.488 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46335.log 2024-04-25T00:16:14.488 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45598.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45598.log.gz 2024-04-25T00:16:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46335.log: 91.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T00:16:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42819.log 2024-04-25T00:16:14.491 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46335.log.gz 2024-04-25T00:16:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48352.log 2024-04-25T00:16:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42819.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42819.log.gz 2024-04-25T00:16:14.494 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.48352.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43644.log 2024-04-25T00:16:14.494 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48352.log.gz 2024-04-25T00:16:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35995.log 2024-04-25T00:16:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43644.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43644.log.gz 2024-04-25T00:16:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49415.log 2024-04-25T00:16:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35995.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35995.log.gz 2024-04-25T00:16:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48097.log 2024-04-25T00:16:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49415.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49415.log.gz 2024-04-25T00:16:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46965.log 2024-04-25T00:16:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48097.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48097.log.gz 2024-04-25T00:16:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40262.log 2024-04-25T00:16:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46965.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46965.log.gz 2024-04-25T00:16:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41099.log 2024-04-25T00:16:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40262.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40262.log.gz 2024-04-25T00:16:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47214.log 2024-04-25T00:16:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41099.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41099.log.gz 2024-04-25T00:16:14.507 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39671.log 2024-04-25T00:16:14.508 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47214.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47214.log.gz 2024-04-25T00:16:14.508 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44944.log 2024-04-25T00:16:14.509 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39671.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39671.log.gz 2024-04-25T00:16:14.510 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40002.log 2024-04-25T00:16:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44944.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44944.log.gz 2024-04-25T00:16:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36539.log 2024-04-25T00:16:14.513 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40002.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40002.log.gz 2024-04-25T00:16:14.513 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48482.log 2024-04-25T00:16:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36539.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36539.log.gz 2024-04-25T00:16:14.515 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42222.log 2024-04-25T00:16:14.516 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48482.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48482.log.gz 2024-04-25T00:16:14.516 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42014.log 2024-04-25T00:16:14.517 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42222.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42222.log.gz 2024-04-25T00:16:14.518 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38181.log 2024-04-25T00:16:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42014.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42014.log.gz 2024-04-25T00:16:14.520 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42845.log 2024-04-25T00:16:14.521 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38181.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38181.log.gz 2024-04-25T00:16:14.521 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36799.log 2024-04-25T00:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42845.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42845.log.gz 2024-04-25T00:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44325.log 2024-04-25T00:16:14.524 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36799.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36799.log.gz 2024-04-25T00:16:14.524 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34584.log 2024-04-25T00:16:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44325.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44325.log.gz 2024-04-25T00:16:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48612.log 2024-04-25T00:16:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34584.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34584.log.gz 2024-04-25T00:16:14.528 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37059.log 2024-04-25T00:16:14.529 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48612.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48612.log.gz 2024-04-25T00:16:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39002.log 2024-04-25T00:16:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37059.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37059.log.gz 2024-04-25T00:16:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34980.log 2024-04-25T00:16:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39002.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39002.log.gz 2024-04-25T00:16:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40857.log 2024-04-25T00:16:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34980.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34980.log.gz 2024-04-25T00:16:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35969.log 2024-04-25T00:16:14.536 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40857.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40857.log.gz 2024-04-25T00:16:14.536 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47628.log 2024-04-25T00:16:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35969.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35969.log.gz 2024-04-25T00:16:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49208.log 2024-04-25T00:16:14.539 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47628.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47628.log.gz 2024-04-25T00:16:14.540 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46756.log 2024-04-25T00:16:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49208.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49208.log.gz 2024-04-25T00:16:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44970.log 2024-04-25T00:16:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46756.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46756.log.gz 2024-04-25T00:16:14.543 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34693.log 2024-04-25T00:16:14.544 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44970.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44970.log.gz 2024-04-25T00:16:14.545 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36877.log 2024-04-25T00:16:14.546 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34693.log: 92.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T00:16:14.546 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34693.log.gz 2024-04-25T00:16:14.546 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33571.log 2024-04-25T00:16:14.547 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36877.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36877.log.gz 2024-04-25T00:16:14.547 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48071.log 2024-04-25T00:16:14.548 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43001.log 2024-04-25T00:16:14.549 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33571.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33571.log.gz 2024-04-25T00:16:14.549 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48071.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48071.log.gz 2024-04-25T00:16:14.550 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48789.log 2024-04-25T00:16:14.550 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43001.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43001.log.gz 2024-04-25T00:16:14.551 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36203.log 2024-04-25T00:16:14.552 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45707.log 2024-04-25T00:16:14.553 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48789.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48789.log.gz 2024-04-25T00:16:14.553 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.36203.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.36203.log.gz 2024-04-25T00:16:14.553 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46309.log 2024-04-25T00:16:14.554 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45707.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45707.log.gz 2024-04-25T00:16:14.554 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47374.log 2024-04-25T00:16:14.556 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49441.log 2024-04-25T00:16:14.556 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46309.log: /var/log/ceph/ceph-client.admin.47374.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46309.log.gz 2024-04-25T00:16:14.557 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47374.log.gz 2024-04-25T00:16:14.558 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34901.log 2024-04-25T00:16:14.558 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49441.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49441.log.gz 2024-04-25T00:16:14.558 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46446.log 2024-04-25T00:16:14.559 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39723.log 2024-04-25T00:16:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34901.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34901.log.gz 2024-04-25T00:16:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46446.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46446.log.gz 2024-04-25T00:16:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47322.log 2024-04-25T00:16:14.562 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39723.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39723.log.gz 2024-04-25T00:16:14.562 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46056.log 2024-04-25T00:16:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44786.log 2024-04-25T00:16:14.564 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47322.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47322.log.gz 2024-04-25T00:16:14.564 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46056.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46056.log.gz 2024-04-25T00:16:14.565 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47296.log 2024-04-25T00:16:14.566 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44786.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44786.log.gz 2024-04-25T00:16:14.566 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49260.log 2024-04-25T00:16:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34009.log 2024-04-25T00:16:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47296.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47296.log.gz 2024-04-25T00:16:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49260.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49260.log.gz 2024-04-25T00:16:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35393.log 2024-04-25T00:16:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34009.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34009.log.gz 2024-04-25T00:16:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43696.log 2024-04-25T00:16:14.571 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46160.log 2024-04-25T00:16:14.572 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35393.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35393.log.gz 2024-04-25T00:16:14.572 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43696.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43696.log.gz 2024-04-25T00:16:14.573 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36981.log 2024-04-25T00:16:14.574 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46160.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46160.log.gz 2024-04-25T00:16:14.574 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40805.log 2024-04-25T00:16:14.575 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44156.log 2024-04-25T00:16:14.576 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36981.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36981.log.gz 2024-04-25T00:16:14.576 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40805.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40805.log.gz 2024-04-25T00:16:14.577 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43483.log 2024-04-25T00:16:14.577 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44156.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44156.log.gz 2024-04-25T00:16:14.578 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48896.log 2024-04-25T00:16:14.579 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48232.log 2024-04-25T00:16:14.579 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43483.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43483.log.gz 2024-04-25T00:16:14.580 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48896.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48896.log.gz 2024-04-25T00:16:14.581 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41203.log 2024-04-25T00:16:14.581 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48232.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48232.log.gz 2024-04-25T00:16:14.582 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42871.log 2024-04-25T00:16:14.583 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38598.log 2024-04-25T00:16:14.583 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41203.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41203.log.gz 2024-04-25T00:16:14.584 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42871.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42871.log.gz 2024-04-25T00:16:14.584 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38810.log 2024-04-25T00:16:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38598.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38598.log.gz 2024-04-25T00:16:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45839.log 2024-04-25T00:16:14.587 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44682.log 2024-04-25T00:16:14.587 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38810.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38810.log.gz 2024-04-25T00:16:14.588 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45839.log: /var/log/ceph/ceph-client.admin.44682.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45839.log.gz 2024-04-25T00:16:14.589 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42298.log 2024-04-25T00:16:14.589 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44682.log.gz 2024-04-25T00:16:14.590 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37359.log 2024-04-25T00:16:14.591 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42298.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.42298.log.gz -5 2024-04-25T00:16:14.591 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37627.log 2024-04-25T00:16:14.593 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35837.log 2024-04-25T00:16:14.593 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37359.log.gz 2024-04-25T00:16:14.593 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37627.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37627.log.gz 2024-04-25T00:16:14.594 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46543.log 2024-04-25T00:16:14.594 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35837.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35837.log.gz 2024-04-25T00:16:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42248.log 2024-04-25T00:16:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49052.log 2024-04-25T00:16:14.597 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46543.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46543.log.gz 2024-04-25T00:16:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42248.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.42248.log.gz 2024-04-25T00:16:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35707.log 2024-04-25T00:16:14.599 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49052.log.gz 2024-04-25T00:16:14.599 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47973.log 2024-04-25T00:16:14.601 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49364.log 2024-04-25T00:16:14.601 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35707.log.gz 2024-04-25T00:16:14.602 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46030.log 2024-04-25T00:16:14.603 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49364.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47973.log.gz 2024-04-25T00:16:14.603 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49364.log.gz 2024-04-25T00:16:14.604 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38912.log 2024-04-25T00:16:14.604 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46030.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46030.log.gz 2024-04-25T00:16:14.605 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40315.log 2024-04-25T00:16:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43027.log 2024-04-25T00:16:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38912.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38912.log.gz 2024-04-25T00:16:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40315.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40315.log.gz 2024-04-25T00:16:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41830.log 2024-04-25T00:16:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43027.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43027.log.gz 2024-04-25T00:16:14.609 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43053.log 2024-04-25T00:16:14.610 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42144.log 2024-04-25T00:16:14.610 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41830.log: 88.2%/var/log/ceph/ceph-client.admin.43053.log: -- replaced with /var/log/ceph/ceph-client.admin.41830.log.gz 2024-04-25T00:16:14.611 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43053.log.gz 2024-04-25T00:16:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41151.log 2024-04-25T00:16:14.612 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42144.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42144.log.gz 2024-04-25T00:16:14.613 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45624.log 2024-04-25T00:16:14.613 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45352.log 2024-04-25T00:16:14.614 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41151.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41151.log.gz 2024-04-25T00:16:14.615 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45624.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45624.log.gz 2024-04-25T00:16:14.615 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35603.log 2024-04-25T00:16:14.616 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45352.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45352.log.gz 2024-04-25T00:16:14.616 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37007.log 2024-04-25T00:16:14.617 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40580.log 2024-04-25T00:16:14.618 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35603.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35603.log.gz/var/log/ceph/ceph-client.admin.37007.log: 2024-04-25T00:16:14.623 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37007.log.gz 2024-04-25T00:16:14.623 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34035.log 2024-04-25T00:16:14.623 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40580.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40580.log.gz 2024-04-25T00:16:14.623 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35189.log 2024-04-25T00:16:14.623 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33662.log 2024-04-25T00:16:14.623 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34035.log: 89.0%/var/log/ceph/ceph-client.admin.35189.log: -- replaced with /var/log/ceph/ceph-client.admin.34035.log.gz 2024-04-25T00:16:14.623 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35189.log.gz 2024-04-25T00:16:14.623 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41936.log 2024-04-25T00:16:14.624 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33662.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33662.log.gz 2024-04-25T00:16:14.624 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38481.log 2024-04-25T00:16:14.625 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48702.log 2024-04-25T00:16:14.626 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41936.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41936.log.gz/var/log/ceph/ceph-client.admin.38481.log: 2024-04-25T00:16:14.627 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38481.log.gz 2024-04-25T00:16:14.627 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47159.log 2024-04-25T00:16:14.628 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48702.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48702.log.gz 2024-04-25T00:16:14.628 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44845.log 2024-04-25T00:16:14.629 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39619.log 2024-04-25T00:16:14.630 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47159.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47159.log.gz 2024-04-25T00:16:14.630 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44845.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44845.log.gz 2024-04-25T00:16:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35170.log 2024-04-25T00:16:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39619.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39619.log.gz 2024-04-25T00:16:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39697.log 2024-04-25T00:16:14.633 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42663.log 2024-04-25T00:16:14.634 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35170.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35170.log.gz 2024-04-25T00:16:14.634 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39697.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39697.log.gz 2024-04-25T00:16:14.635 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43366.log 2024-04-25T00:16:14.635 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42663.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42663.log.gz 2024-04-25T00:16:14.636 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37889.log 2024-04-25T00:16:14.637 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47348.log 2024-04-25T00:16:14.637 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43366.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43366.log.gz/var/log/ceph/ceph-client.admin.37889.log: 2024-04-25T00:16:14.638 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37889.log.gz 2024-04-25T00:16:14.638 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27823.log 2024-04-25T00:16:14.639 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47348.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47348.log.gz 2024-04-25T00:16:14.640 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47133.log 2024-04-25T00:16:14.641 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49312.log 2024-04-25T00:16:14.641 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.27823.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27823.log.gz/var/log/ceph/ceph-client.admin.47133.log: 2024-04-25T00:16:14.642 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47133.log.gz 2024-04-25T00:16:14.642 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34849.log 2024-04-25T00:16:14.643 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49312.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49312.log.gz 2024-04-25T00:16:14.643 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49000.log 2024-04-25T00:16:14.645 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46622.log 2024-04-25T00:16:14.645 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34849.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34849.log.gz 2024-04-25T00:16:14.646 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49000.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49000.log.gz 2024-04-25T00:16:14.646 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37411.log 2024-04-25T00:16:14.647 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46622.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46622.log.gz 2024-04-25T00:16:14.648 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40632.log 2024-04-25T00:16:14.648 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41359.log 2024-04-25T00:16:14.649 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37411.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37411.log.gz/var/log/ceph/ceph-client.admin.40632.log: 2024-04-25T00:16:14.650 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40632.log.gz 2024-04-25T00:16:14.650 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46387.log 2024-04-25T00:16:14.651 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41359.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41359.log.gz 2024-04-25T00:16:14.652 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34745.log 2024-04-25T00:16:14.652 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36513.log 2024-04-25T00:16:14.653 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46387.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46387.log.gz 2024-04-25T00:16:14.653 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34745.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34745.log.gz 2024-04-25T00:16:14.654 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36513.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36513.log.gz 2024-04-25T00:16:14.658 INFO:teuthology.orchestra.run.smithi083.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T00:16:14.898 INFO:teuthology.orchestra.run.smithi086.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T00:16:14.963 INFO:teuthology.orchestra.run.smithi086.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T00:16:15.012 INFO:teuthology.orchestra.run.smithi086.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T00:16:15.030 INFO:teuthology.orchestra.run.smithi086.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T00:16:15.032 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-25T00:16:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:real 0m1.071s 2024-04-25T00:16:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:user 0m4.124s 2024-04-25T00:16:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m0.347s 2024-04-25T00:16:15.113 INFO:teuthology.orchestra.run.smithi083.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T00:16:15.151 INFO:teuthology.orchestra.run.smithi083.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T00:16:15.270 INFO:teuthology.orchestra.run.smithi083.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T00:16:15.293 INFO:teuthology.orchestra.run.smithi083.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T00:16:15.296 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-25T00:16:15.296 INFO:teuthology.orchestra.run.smithi083.stderr:real 0m1.353s 2024-04-25T00:16:15.296 INFO:teuthology.orchestra.run.smithi083.stderr:user 0m5.348s 2024-04-25T00:16:15.296 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m1.264s 2024-04-25T00:16:15.297 INFO:tasks.ceph:Archiving logs... 2024-04-25T00:16:15.297 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231/remote/smithi083/log 2024-04-25T00:16:15.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:16:16.344 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231/remote/smithi086/log 2024-04-25T00:16:16.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:16:16.869 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T00:16:16.880 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T00:16:16.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:16:16.883 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:16:16.954 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T00:16:16.964 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T00:16:16.965 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:16:16.968 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:16:16.981 INFO:teuthology.orchestra.run.smithi083.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:16:16.981 INFO:teuthology.orchestra.run.smithi083.stdout:============================================================================== 2024-04-25T00:16:16.981 INFO:teuthology.orchestra.run.smithi083.stdout:*hv01.front.sepi 44.4.53.2 2 u 28 64 377 0.068 -1.925 0.822 2024-04-25T00:16:16.981 INFO:teuthology.orchestra.run.smithi083.stdout:+hv02.front.sepi 63.231.80.2 3 u 33 64 377 0.108 -0.776 0.998 2024-04-25T00:16:16.981 INFO:teuthology.orchestra.run.smithi083.stdout:+hv03.front.sepi 74.6.168.72 3 u 29 64 377 0.076 -1.050 0.842 2024-04-25T00:16:16.981 INFO:teuthology.orchestra.run.smithi083.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:16:16.983 INFO:teuthology.orchestra.run.smithi086.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:16:16.983 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================== 2024-04-25T00:16:16.983 INFO:teuthology.orchestra.run.smithi086.stdout:*hv01.front.sepi 44.4.53.2 2 u 32 64 377 0.070 -2.564 0.892 2024-04-25T00:16:16.983 INFO:teuthology.orchestra.run.smithi086.stdout:+hv02.front.sepi 63.231.80.2 3 u 34 64 377 0.081 -0.231 1.237 2024-04-25T00:16:16.983 INFO:teuthology.orchestra.run.smithi086.stdout:+hv03.front.sepi 74.6.168.72 3 u 29 64 377 0.076 -0.246 1.246 2024-04-25T00:16:16.983 INFO:teuthology.orchestra.run.smithi086.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:16:16.984 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T00:16:16.994 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T00:16:16.995 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T00:16:17.005 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T00:16:17.015 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T00:16:17.024 INFO:teuthology.task.internal:Duration was 1027.762716 seconds 2024-04-25T00:16:17.024 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T00:16:17.034 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T00:16:17.035 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:16:17.038 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:16:17.129 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T00:16:17.130 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T00:16:17.130 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:16:17.151 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T00:16:17.151 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:16:17.172 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T00:16:17.172 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:16:17.195 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:16:17.257 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T00:16:17.257 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:16:17.279 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:16:17.381 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T00:16:17.392 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T00:16:17.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:16:17.412 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:16:17.432 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T00:16:17.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:16:17.458 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:16:17.472 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-04-25T00:16:17.496 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-04-25T00:16:17.514 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:16:17.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:16:17.543 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:16:17.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:16:17.562 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T00:16:17.573 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T00:16:17.574 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231/remote/smithi083 2024-04-25T00:16:17.574 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:16:17.610 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672231/remote/smithi086 2024-04-25T00:16:17.610 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:16:17.636 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T00:16:17.636 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:16:17.654 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:16:17.683 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T00:16:17.698 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T00:16:17.698 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T00:16:17.714 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T00:16:17.714 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:16:17.717 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi083.stdout: 658122 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:16 /home/ubuntu/cephtest 2024-04-25T00:16:17.735 INFO:teuthology.orchestra.run.smithi086.stdout: 658121 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:16 /home/ubuntu/cephtest 2024-04-25T00:16:17.737 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T00:16:17.748 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T00:16:17.773 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/sessionmap} duration: 1027.7627158164978 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T00:16:17.773 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:16:17.855 INFO:teuthology.run:pass